add zstd and snappy compression for query api

[afhassan]

What this PR does:

• Add Snappy and Zstd compression for querier -> QFE responses.
• Use a new internal http header X-Uncompressed-Length to enforce QFE response size limit before decompressing response

Which issue(s) this PR fixes:
Fixes #

Checklist

• Tests updated
• Documentation added
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

[erlan-z]

Added small nit, but overall looks good to me.
Thanks.

[afhassan]

@yeya24 @SungJin1212 could someone take a look when you have some time? Thanks!

[SungJin1212]

@afhassan
Thanks for the contribution. The code looks nice. Can you add an e2e test?
I think it would be good to add tests to TestQuerierResponseCompression.

[afhassan]

@SungJin1212 Thanks for the review, I added unit tests and an integration test for compression. The change done to QFE response size limit is already covered by this test https://github.com/afhassan/cortex/blob/master/integration/query_frontend_test.go#L977

[yeya24]

This change means that we need a two phase deployment for querier to return this header for Query Frontend to properly apply the response size limit. Otherwise, the value is 0.

I think we should find a way (maybe a flag) to properly maintain the previous behavior during rollout.

[afhassan]

I didn't quite get it, do you mean during deployment if Query Frontend is updated first then the limit wouldn't apply because the header is not being returned from Querier?

The old behaviour only works for gzip which is why I changed it. I can reverse it to use the old behaviour when gzip is used and the new one for other compression types.

[yeya24]

Yes. If we use gzip then this limiter won't work until Querier is updated, which is a breaking behavior.

We just need a way to fallback to the old behavior if it is gzip. Other types are fine since they are new

[afhassan]

If header is missing we now fallback to applying limit using size of the response after decoding it. I think this is a cleaner way to still apply the limit and avoid a breaking change if the header is not there.

Applying the limit before decoding response was an optimization in the old behaviour, but I don't think it is considered breaking if we are still applying the limit.