Add request headers to wreq::redirect::HistoryEntry

[dlight]

Is your feature request related to a problem? Please describe.

Request headers are sometimes stripped between redirects, following some security policy. However, when inspecting the redirect history in wreq 6.0.0-rc.28, only response headers are provided.

Describe the solution you'd like

I would like request headers to be provided too. Many times, they will be identical across many hops. Because of this, the request headers could just be Option<HeaderMap>, where a None means it didn't change from the last hop (and we assert that the first hop is a Some()). But, if the request headers changed, it would be interesting to know.

Describe alternatives you've considered

Actually, there is a more involved API to capture redirects as the hops happen, by using a custom rederect policy. But it appears to me that if I do this, wreq will not automatically remove headers due to the security policy (I may be wrong about that). And also, the functions to remove those headers are private. so I would need to copy this kind of stuff into my codebase:

fn remove_sensitive_headers(headers: &mut HeaderMap, next: &Uri, previous: &[Uri]) {
    if let Some(previous) = previous.last() {
        let cross_host = next.host() != previous.host()
            || next.port() != previous.port()
            || next.scheme() != previous.scheme();
        if cross_host {
            /// Avoid dynamic allocation of `HeaderName` by using `from_static`.
            /// https://github.com/hyperium/http/blob/e9de46c9269f0a476b34a02a401212e20f639df2/src/header/map.rs#L3794
            const COOKIE2: HeaderName = HeaderName::from_static("cookie2");

            headers.remove(AUTHORIZATION);
            headers.remove(COOKIE);
            headers.remove(COOKIE2);
            headers.remove(PROXY_AUTHORIZATION);
            headers.remove(WWW_AUTHENTICATE);
        }
    }
}

That's.. not reasonable. So maybe exposing this function as pub and maybe a couple of others would aid creating custom policies. And perhaps a better approach is to offer an API that by default does the right thing and remove the headers (giving an option for the program to not remove them)

The reason for following the spec is that if you do a redirect without removing those headers, you may be sending signals that may flag you as a bot.

Indeed, people with custom redirect policies almost 100% of time want to always apply remove_sensitive_headers and most of other custom code in FollowRedirectPolicy - they just want to choose whether the redirect happens in each case (that is, they just need an API that sends a closure that receives all the info, like previous uri, next uri, request headers, etc, and returns a bool or something)

[0x676e67]

Obviously, a solution is indeed required. Most users want to add callback logic when sending requests. My proposal is to register an on-request callback for policies, and custom callbacks will override the logic of remove_sensitive_headers.