Handle HTTP chunking across multi-byte boundaries #23
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zyc9012
reviewed
Mar 9, 2024
When a response contains multi-byte UTF-8 characters that cross chunk boundaries, appending the two halves to the intermediate string results in two Unicode Replacement Characters being inserted instead of the two halves being joined. Set the response encoding to UTF-8 to ensure it buffers partial characters between chunks. Resolves #22 Co-authored-by: zyc9012 <[email protected]>
Freaky
force-pushed
the
fix-22-multibyte-char-corruption
branch
from
bec58a8 to
7610f9f
Compare
|
CI failures look to be a result of https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/ Instructing it to use OpenSSL's doesn't appear to help, hmm. |
Freaky
force-pushed
the
fix-22-multibyte-char-corruption
branch
3 times, most recently
from
34d1d1c to
7d2266f
Compare
zyc9012
force-pushed
the
fix-22-multibyte-char-corruption
branch
6 times, most recently
from
22ff7db to
d0b1858
Compare
Node hardcodes its default TLS certificates, meaning older versions do not support the root certificate currently used by LetsEncrypt. Use --use-openssl-ca to enable the use of the system's OpenSSL certificates. Since node v7 doesn't support NODE_OPTIONS, conditionally modify the test script with sed.
zyc9012
force-pushed
the
fix-22-multibyte-char-corruption
branch
from
d0b1858 to
d9385fc
Compare
|
Thank you @Freaky. I fixed the CI and squashed my commit with yours. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When a response contains multi-byte UTF-8 characters that cross chunk boundaries, appending the two halves to the intermediate string results in two Unicode Replacement Characters being inserted instead of the two halves being joined.
Use StringDecoder to buffer the partial characters across chunks.
Resolves #22