Skip to content
This repository has been archived by the owner on Jun 10, 2024. It is now read-only.

add empty-valued header #961

Open
curme opened this issue Nov 10, 2020 · 1 comment
Open

add empty-valued header #961

curme opened this issue Nov 10, 2020 · 1 comment

Comments

@curme
Copy link

curme commented Nov 10, 2020

Expected behavior

I want to add a header with empty value, for example, my request requires a header 'Authorization: ' which is a 'Authorization' header valued as ''.

url = 'xxx.xxxx.com'
headers = {'Content-Type': 'application/json', 'Authorization': ''}
self.crawl(url, callback=self.xxx_handle, method='POST', headers=headers)

Actual behavior

However, actually, the header 'Authorization' seems to be kicked out because it's empty.

Maybe, it might cause by lib 'libcurl', which I found linux command 'curl' has the same behavior:
curl --request GET 'https://github.com/binux/pyspider' --header 'Accept: */*' --header 'Authorization: ' -v

In the curl printed log, we could find that 'Authorization' header was removed:
> GET /binux/pyspider HTTP/1.1
> Host: github.com
> User-Agent: curl/7.61.0
> Accept: */*
>

I want to add empty-valued header into request. Any suggestions?
Thanks a lot!

@curme
Copy link
Author

curme commented Nov 11, 2020

I found that curl has already supported sending empty header by replacing the colon with a semicolon. Exactly in the case I provided above, we could do in this way:
curl --request GET 'https://github.com/binux/pyspider' --header 'Accept: */*' --header 'Authorization;' -v

log:
> GET /binux/pyspider HTTP/1.1
> Host: github.com
> User-Agent: curl/7.61.0
> Accept: */*
> Authorization:
>

Which means 'libcurl' enables users to send empty header in request (BTW I suppose pyspider also rely on 'libcurl').
But I still haven found how to do in pyspider. Need your help

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant