Unable to create a file using HttpFS Gateway

[sasabrkic]

Hello!

I would appreciate some help with HttpFS Gateway. I would like to use it for a very simple use case where I need to create, read and delete files using REST API.

I was able to read and delete a file using the API, but I am struggling with creation. I followed both the HttpFS Gateway and Hadoop documentation, but no luck. I tried some quick fixes, from changing the Docker image version to trying to create volumes and buckets in advance to trying to PUT to the datanode directly (as seen is the Hadoop documentation). I also tried both v.1.4 and v.2.0.

So, this is what I do. I start containers as per instructions and issue the request:

curl -i -X PUT "http://localhost:14000/webhdfs/v1/exports/text/export.txt?op=CREATE&user.name=hadoop"

The response looks good:

HTTP/1.1 307 Temporary Redirect
Date: Tue, 03 Jun 2025 04:33:34 GMT
Cache-Control: no-cache
Expires: Tue, 03 Jun 2025 04:33:34 GMT
Pragma: no-cache
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Set-Cookie: hadoop.auth="u=hadoop&p=hadoop&t=simple-dt&e=1748961214314&s=VHll3A+dIzq6ylpXF7Cmeuk2iuw/7ZCN8ip6DeJ2O3c="; Path=/; HttpOnly
Location: http://localhost:14000/webhdfs/v1/exports/text/export.txt?op=CREATE&data=true&user.name=hadoop
Content-Length: 0

Now I try to upload the actual file using the Location URL:

curl -i -X PUT -T ~/export.txt "http://localhost:14000/webhdfs/v1/exports/text/export.txt?op=CREATE&data=true&user.name=hadoop"

And then I get Bad Request:

HTTP/1.1 400 Bad Request
Pragma: no-cache
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Set-Cookie: hadoop.auth="u=hadoop&p=hadoop&t=simple-dt&e=1748961384237&s=lMHNa6v8Jlqc7s5Ufl2vrnEWoKAwmKydSHZUhM6Jhw0="; Path=/; HttpOnly
Connection: close

As I said above, I tried different variations, like pre-creating volume and bucket, using different variants of the path, sending/not sending the hadoop.auth cookie.

I see nothing in the logs that could help me troubleshoot. I am just doing a quick evaluation, and I unfortunately haven't had time to go through detailed configuration. I was hoping that the examples in the documentation would work, but maybe some step is missing.

Any help would be greatly appreciated. Thanks!

[jojochuang]

@dombizita @adoroszlai ?

[adoroszlai]

@sasabrkic please try adding -H"Content-Type: application/octet-stream" to the "upload the actual file" curl command

bash-5.1$ curl -i -X PUT -T ~/export.txt "http://httpfs:14000/webhdfs/v1/exports/text/export.txt?op=CREATE&data=true&user.name=hadoop"
HTTP/1.1 400 Bad Request
Pragma: no-cache
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Set-Cookie: hadoop.auth="u=hadoop&p=hadoop&t=simple-dt&e=1748970950712&s=sYBi468BlYTnfaZxqCMAiHbz+mJlMMGfKxXl90i34zo="; Path=/; HttpOnly
Connection: close

bash-5.1$ curl -i -X PUT -T ~/export.txt "http://httpfs:14000/webhdfs/v1/exports/text/export.txt?op=CREATE&data=true&user.name=hadoop" -H"Content-Type: application/octet-stream"
HTTP/1.1 100 Continue

HTTP/1.1 201 Created
Date: Tue, 03 Jun 2025 07:16:02 GMT
Cache-Control: no-cache
Expires: Tue, 03 Jun 2025 07:16:02 GMT
Pragma: no-cache
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Set-Cookie: hadoop.auth="u=hadoop&p=hadoop&t=simple-dt&e=1748970962918&s=IzhlonGJSDISkx9x3zZG5tVQAa0DBry76O61oJFvAp4="; Path=/; HttpOnly
Location: http://httpfs:14000/webhdfs/v1/exports/text/export.txt
Content-Type: application/json
Content-Length: 69

{"Location":"http://httpfs:14000/webhdfs/v1/exports/text/export.txt"}bash-5.1$ 

[peterxcli]

@sasabrkic I'm able to create the file with:

curl -v -X PUT -H "Content-Type: application/octet-stream" -T ~/export.txt "http://localhost:14000/webhdfs/v
1/exports/text/export.txt?op=CREATE&data=true&user.name=hadoop"
HTTP/1.1 201 Created
Date: Tue, 03 Jun 2025 07:09:19 GMT
Cache-Control: no-cache
Expires: Tue, 03 Jun 2025 07:09:19 GMT
Pragma: no-cache
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Set-Cookie: hadoop.auth="u=hadoop&p=hadoop&t=simple-dt&e=1748970559244&s=BpWb4ypcr2zQphg1BNnlvYDC9ul+SitU6UfLocAd4ao="; Path=/; HttpOnly
Location: http://localhost:14000/webhdfs/v1/exports/text/export.txt
Content-Type: application/json
Content-Length: 72

{"Location":"http://localhost:14000/webhdfs/v1/exports/text/export.txt

I only add the header -H "Content-Type: application/octet-stream" and it works.

Looks like it's existing logic:

ozone/hadoop-ozone/httpfsgateway/src/main/java/org/apache/ozone/fs/http/server/CheckUploadContentTypeFilter.java

Lines 81 to 100 in 0e8c8d4

	if (method.equals("PUT") || method.equals("POST")) {
	String op = httpReq.getParameter(HttpFSConstants.OP_PARAM);
	if (op != null && UPLOAD_OPERATIONS.contains(
	StringUtils.toUpperCase(op))) {
	if ("true".equalsIgnoreCase(httpReq
	.getParameter(HttpFSParametersProvider.DataParam.NAME))) {
	String contentType = httpReq.getContentType();
	contentTypeOK =
	HttpFSConstants.UPLOAD_CONTENT_TYPE.equalsIgnoreCase(contentType);
	}
	}
	}
	if (contentTypeOK) {
	chain.doFilter(httpReq, httpRes);
	} else {
	httpRes.sendError(HttpServletResponse.SC_BAD_REQUEST,
	"Data upload requests must have content-type set to '" +
	HttpFSConstants.UPLOAD_CONTENT_TYPE + "'");
	}

[peterxcli]

@adoroszlai

ozone/hadoop-ozone/httpfsgateway/src/main/java/org/apache/ozone/fs/http/server/CheckUploadContentTypeFilter.java

Lines 81 to 100 in 0e8c8d4

	if (method.equals("PUT") || method.equals("POST")) {
	String op = httpReq.getParameter(HttpFSConstants.OP_PARAM);
	if (op != null && UPLOAD_OPERATIONS.contains(
	StringUtils.toUpperCase(op))) {
	if ("true".equalsIgnoreCase(httpReq
	.getParameter(HttpFSParametersProvider.DataParam.NAME))) {
	String contentType = httpReq.getContentType();
	contentTypeOK =
	HttpFSConstants.UPLOAD_CONTENT_TYPE.equalsIgnoreCase(contentType);
	}
	}
	}
	if (contentTypeOK) {
	chain.doFilter(httpReq, httpRes);
	} else {
	httpRes.sendError(HttpServletResponse.SC_BAD_REQUEST,
	"Data upload requests must have content-type set to '" +
	HttpFSConstants.UPLOAD_CONTENT_TYPE + "'");
	}

The sendError doesn't set the msg passed in to response content. Is it intended? Or we have to use

response.setStatus(HttpServletResponse.SC_BAD_REQUEST);
response.setContentType("application/json");
response.getWriter().write("xxxx");

instead to tell user what exception is.

curl -v -X PUT -T export.txt "http://localhost:14000
/webhdfs/v1/exports/text/export.txt?op=CREATE&data=true&user.name=hadoop"

Note: Unnecessary use of -X or --request, PUT is already inferred.
* Host localhost:14000 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:14000...
* Connected to localhost (::1) port 14000
> PUT /webhdfs/v1/exports/text/export.txt?op=CREATE&data=true&user.name=hadoop HTTP/1.1
> Host: localhost:14000
> User-Agent: curl/8.5.0
> Accept: */*
> Content-Length: 82
> 
* We are completely uploaded and fine
< HTTP/1.1 400 Bad Request
< Pragma: no-cache
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Set-Cookie: hadoop.auth="u=hadoop&p=hadoop&t=simple-dt&e=1748972679087&s=7R3l7H77La1MUB2e1G6Oa/BYo2cB+lOHOwAzwBkwL3w="; Path=/; HttpOnly
< Content-Length: 0
< 
* Connection #0 to host localhost left intact

(content length is 0)

[adoroszlai]

I don't think it's intentional, created HDDS-13168.

[sasabrkic]

That solves the issue, thanks for the swift reply!

I guess that the above should be mentioned somewhere in the documentation 😄

Keep up the good work, everybody!

[adoroszlai]

Thanks for trying it. Created HDDS-13167 to fix the omission.

[jojochuang]

Hi, I happen to be writing a Python client user doc, please help proof read it: I included the workarounds in the tutorial:

#8556