Discussion:
Uploading a file to IGI from TDI HTTPClient using IGI REST API
(too old to reply)
JulianL
2019-03-26 06:14:32 UTC
Permalink
Hi,

I am trying to upload CSV files to IGI using TDI with the HTTPClient connector. I can successfully upload a file using Postman and I can also exercise some API operations that prove that I am at least authenticating correctly and setting appropriate headers but the upload operation returns '400 Bad Request' error ('missing_required_parameter'). In the IGI System log I see

com.ibm.identity.customfiles.CustomFileServlet E Missing required parameter or invalid value for type

Versions are:

IGI 5.2.4
TDI 7.1.1 fixpack 7


The IGI API documentation is minimal but provides the following example for the required request:

Request Example
POST https://{appliance_hostname}:9443/v1/customfiles
POST_DATA:
{
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="uploadedFiles[]"; filename="feedFile.txt"

Content-Type:

----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="name"

feedFile.txt

----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="relativePath"

data/SampleFeedFiles

----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="type"

file

----WebKitFormBoundaryE19zNvXGzXaLvS5C
}


Postman offers code samples based on a successful request. The raw HTTP one looks like:

POST /v1/customfiles HTTP/1.1
Host: <host IP>:9443
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
Accept: application/json
Authorization: Basic <auth string>
cache-control: no-cache
Postman-Token: fe9ff01d-b814-4acc-bf6b-4add35e0b488

Content-Disposition: form-data; name="uploadedFiles[]"

E:\Temp\HRS.csv

Content-Disposition: form-data; name="name"

HRS.csv

Content-Disposition: form-data; name="type"

file

Content-Disposition: form-data; name="relativePath"

connectors/test
------WebKitFormBoundary7MA4YWxkTrZu0gW--

In TDI I am using HTTPClient Connector in Call-Reply mode. Regardless of whether I read a pre-prepared body from a file or create one in a 'before call-reply' script I get the same error. I have experimented with using a single boundary as in the Postman HTTP sample and a boundary between each Content-Disposition section as in the IGI Doc but both give the same result.

I am wondering whether the TDI HTTPClient is has the capability to attach files for upload in this way. I cannot find any examples of using TDI HTTPClient and multipart formdata. Any help or advice would be much appreciated.

Regards,
Julian.
Eddie Hartman
2019-03-26 20:45:39 UTC
Permalink
Post by JulianL
Hi,
I am trying to upload CSV files to IGI using TDI with the HTTPClient connector. I can successfully upload a file using Postman and I can also exercise some API operations that prove that I am at least authenticating correctly and setting appropriate headers but the upload operation returns '400 Bad Request' error ('missing_required_parameter'). In the IGI System log I see
com.ibm.identity.customfiles.CustomFileServlet E Missing required parameter or invalid value for type
IGI 5.2.4
TDI 7.1.1 fixpack 7
Request Example
POST https://{appliance_hostname}:9443/v1/customfiles
{
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="uploadedFiles[]"; filename="feedFile.txt"
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="name"
feedFile.txt
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="relativePath"
data/SampleFeedFiles
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="type"
file
----WebKitFormBoundaryE19zNvXGzXaLvS5C
}
POST /v1/customfiles HTTP/1.1
Host: <host IP>:9443
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
Accept: application/json
Authorization: Basic <auth string>
cache-control: no-cache
Postman-Token: fe9ff01d-b814-4acc-bf6b-4add35e0b488
Content-Disposition: form-data; name="uploadedFiles[]"
E:\Temp\HRS.csv
Content-Disposition: form-data; name="name"
HRS.csv
Content-Disposition: form-data; name="type"
file
Content-Disposition: form-data; name="relativePath"
connectors/test
------WebKitFormBoundary7MA4YWxkTrZu0gW--
In TDI I am using HTTPClient Connector in Call-Reply mode. Regardless of whether I read a pre-prepared body from a file or create one in a 'before call-reply' script I get the same error. I have experimented with using a single boundary as in the Postman HTTP sample and a boundary between each Content-Disposition section as in the IGI Doc but both give the same result.
I am wondering whether the TDI HTTPClient is has the capability to attach files for upload in this way. I cannot find any examples of using TDI HTTPClient and multipart formdata. Any help or advice would be much appreciated.
Regards,
Julian.
The HTTP Client connector can absolutely pass all kinds of content in the body. If you set http.body to a java.io.File then that file is loaded for you. You'll need to set the appropriate http.content-type yourself.

/Eddie
JulianL
2019-03-28 05:10:46 UTC
Permalink
Post by Eddie Hartman
Post by JulianL
Hi,
I am trying to upload CSV files to IGI using TDI with the HTTPClient connector. I can successfully upload a file using Postman and I can also exercise some API operations that prove that I am at least authenticating correctly and setting appropriate headers but the upload operation returns '400 Bad Request' error ('missing_required_parameter'). In the IGI System log I see
com.ibm.identity.customfiles.CustomFileServlet E Missing required parameter or invalid value for type
IGI 5.2.4
TDI 7.1.1 fixpack 7
Request Example
POST https://{appliance_hostname}:9443/v1/customfiles
{
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="uploadedFiles[]"; filename="feedFile.txt"
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="name"
feedFile.txt
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="relativePath"
data/SampleFeedFiles
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="type"
file
----WebKitFormBoundaryE19zNvXGzXaLvS5C
}
POST /v1/customfiles HTTP/1.1
Host: <host IP>:9443
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
Accept: application/json
Authorization: Basic <auth string>
cache-control: no-cache
Postman-Token: fe9ff01d-b814-4acc-bf6b-4add35e0b488
Content-Disposition: form-data; name="uploadedFiles[]"
E:\Temp\HRS.csv
Content-Disposition: form-data; name="name"
HRS.csv
Content-Disposition: form-data; name="type"
file
Content-Disposition: form-data; name="relativePath"
connectors/test
------WebKitFormBoundary7MA4YWxkTrZu0gW--
In TDI I am using HTTPClient Connector in Call-Reply mode. Regardless of whether I read a pre-prepared body from a file or create one in a 'before call-reply' script I get the same error. I have experimented with using a single boundary as in the Postman HTTP sample and a boundary between each Content-Disposition section as in the IGI Doc but both give the same result.
I am wondering whether the TDI HTTPClient is has the capability to attach files for upload in this way. I cannot find any examples of using TDI HTTPClient and multipart formdata. Any help or advice would be much appreciated.
Regards,
Julian.
The HTTP Client connector can absolutely pass all kinds of content in the body. If you set http.body to a java.io.File then that file is loaded for you. You'll need to set the appropriate http.content-type yourself.
/Eddie
The problem I had was not knowing where/how to include the actual file content in the body as this was not made clear by the IGI documentation nor could I find any examples of doing something similar. I finally worked out how to do it by pointing a Postman request that had worked with IGI against a mock web server running in TDI and looking at the body it was sending:

Body: ------WebKitFormBoundaryijsPDfeG25PMFyxW
Content-Disposition: form-data; name="uploadedFiles[]"; filename="IGI_in.csv"
Content-Type: application/vnd.ms-excel

<Content of File>

------WebKitFormBoundaryijsPDfeG25PMFyxW
Content-Disposition: form-data; name="name"

HRS.csv
------WebKitFormBoundaryijsPDfeG25PMFyxW
Content-Disposition: form-data; name="type"

file
------WebKitFormBoundaryijsPDfeG25PMFyxW
Content-Disposition: form-data; name="relativePath"

connectors/test
------WebKitFormBoundaryijsPDfeG25PMFyxW--

Regards,
Julian.
Eddie Hartman
2019-03-28 20:10:55 UTC
Permalink
Post by JulianL
Post by Eddie Hartman
Post by JulianL
Hi,
I am trying to upload CSV files to IGI using TDI with the HTTPClient connector. I can successfully upload a file using Postman and I can also exercise some API operations that prove that I am at least authenticating correctly and setting appropriate headers but the upload operation returns '400 Bad Request' error ('missing_required_parameter'). In the IGI System log I see
com.ibm.identity.customfiles.CustomFileServlet E Missing required parameter or invalid value for type
IGI 5.2.4
TDI 7.1.1 fixpack 7
Request Example
POST https://{appliance_hostname}:9443/v1/customfiles
{
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="uploadedFiles[]"; filename="feedFile.txt"
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="name"
feedFile.txt
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="relativePath"
data/SampleFeedFiles
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="type"
file
----WebKitFormBoundaryE19zNvXGzXaLvS5C
}
POST /v1/customfiles HTTP/1.1
Host: <host IP>:9443
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
Accept: application/json
Authorization: Basic <auth string>
cache-control: no-cache
Postman-Token: fe9ff01d-b814-4acc-bf6b-4add35e0b488
Content-Disposition: form-data; name="uploadedFiles[]"
E:\Temp\HRS.csv
Content-Disposition: form-data; name="name"
HRS.csv
Content-Disposition: form-data; name="type"
file
Content-Disposition: form-data; name="relativePath"
connectors/test
------WebKitFormBoundary7MA4YWxkTrZu0gW--
In TDI I am using HTTPClient Connector in Call-Reply mode. Regardless of whether I read a pre-prepared body from a file or create one in a 'before call-reply' script I get the same error. I have experimented with using a single boundary as in the Postman HTTP sample and a boundary between each Content-Disposition section as in the IGI Doc but both give the same result.
I am wondering whether the TDI HTTPClient is has the capability to attach files for upload in this way. I cannot find any examples of using TDI HTTPClient and multipart formdata. Any help or advice would be much appreciated.
Regards,
Julian.
The HTTP Client connector can absolutely pass all kinds of content in the body. If you set http.body to a java.io.File then that file is loaded for you. You'll need to set the appropriate http.content-type yourself.
/Eddie
Body: ------WebKitFormBoundaryijsPDfeG25PMFyxW
Content-Disposition: form-data; name="uploadedFiles[]"; filename="IGI_in.csv"
Content-Type: application/vnd.ms-excel
<Content of File>
------WebKitFormBoundaryijsPDfeG25PMFyxW
Content-Disposition: form-data; name="name"
HRS.csv
------WebKitFormBoundaryijsPDfeG25PMFyxW
Content-Disposition: form-data; name="type"
file
------WebKitFormBoundaryijsPDfeG25PMFyxW
Content-Disposition: form-data; name="relativePath"
connectors/test
------WebKitFormBoundaryijsPDfeG25PMFyxW--
Regards,
Julian.
Nice work, Julian. This is exactly how you approach a challenge like this: find something that does work and then reverse engineer it into your AL :) I use the HTTP Server Connector all the time to figure out what some rest client is doing so that my code can 'talk the talk' correctly :)

/Eddie

Loading...