Add pooling to buffers used for message encoding #849
Assignees
Milestone
Comments
masih
added a commit
that referenced
this issue
Jan 27, 2025
Use `DataDog/zstd` library for: * a more performant implementation * consistent dependency with the one used in Lotus See Benchamark results below. Before: ``` BenchmarkCborEncoding BenchmarkCborEncoding-12 467352 2376 ns/op 14680 B/op 10 allocs/op BenchmarkCborDecoding BenchmarkCborDecoding-12 104410 11347 ns/op 14944 B/op 27 allocs/op BenchmarkZstdEncoding BenchmarkZstdEncoding-12 286735 3897 ns/op 46748 B/op 12 allocs/op BenchmarkZstdDecoding BenchmarkZstdDecoding-12 110794 10783 ns/op 28512 B/op 28 allocs/op ``` After: ``` BenchmarkCborEncoding BenchmarkCborEncoding-12 439345 2436 ns/op 14680 B/op 10 allocs/op BenchmarkCborDecoding BenchmarkCborDecoding-12 105511 11395 ns/op 14944 B/op 27 allocs/op BenchmarkZstdEncoding BenchmarkZstdEncoding-12 247182 4790 ns/op 28248 B/op 11 allocs/op BenchmarkZstdDecoding BenchmarkZstdDecoding-12 113475 10694 ns/op 20617 B/op 28 allocs/op ``` Fixes: #850 #849
masih
added a commit
that referenced
this issue
Jan 27, 2025
Use `DataDog/zstd` library for: * a more performant implementation * consistent dependency with the one used in Lotus See Benchamark results below. Before: ``` BenchmarkCborEncoding BenchmarkCborEncoding-12 467352 2376 ns/op 14680 B/op 10 allocs/op BenchmarkCborDecoding BenchmarkCborDecoding-12 104410 11347 ns/op 14944 B/op 27 allocs/op BenchmarkZstdEncoding BenchmarkZstdEncoding-12 286735 3897 ns/op 46748 B/op 12 allocs/op BenchmarkZstdDecoding BenchmarkZstdDecoding-12 110794 10783 ns/op 28512 B/op 28 allocs/op ``` After: ``` BenchmarkCborEncoding BenchmarkCborEncoding-12 439345 2436 ns/op 14680 B/op 10 allocs/op BenchmarkCborDecoding BenchmarkCborDecoding-12 105511 11395 ns/op 14944 B/op 27 allocs/op BenchmarkZstdEncoding BenchmarkZstdEncoding-12 247182 4790 ns/op 28248 B/op 11 allocs/op BenchmarkZstdDecoding BenchmarkZstdDecoding-12 113475 10694 ns/op 20617 B/op 27 allocs/op ``` Fixes: #850 #849
masih
added a commit
that referenced
this issue
Jan 28, 2025
Pool the decode buffers and set strict max cap allocation for zstd decompressor. See Benchamark results below. Before: ``` BenchmarkCborEncoding BenchmarkCborEncoding-12 467352 2376 ns/op 14680 B/op 10 allocs/op BenchmarkCborDecoding BenchmarkCborDecoding-12 104410 11347 ns/op 14944 B/op 27 allocs/op BenchmarkZstdEncoding BenchmarkZstdEncoding-12 286735 3897 ns/op 46748 B/op 12 allocs/op BenchmarkZstdDecoding BenchmarkZstdDecoding-12 110794 10783 ns/op 28512 B/op 28 allocs/op ``` After: ``` BenchmarkCborEncoding BenchmarkCborEncoding-12 436754 2383 ns/op 14680 B/op 10 allocs/op BenchmarkCborDecoding BenchmarkCborDecoding-12 106809 11280 ns/op 14944 B/op 27 allocs/op BenchmarkZstdEncoding BenchmarkZstdEncoding-12 294043 3918 ns/op 46746 B/op 12 allocs/op BenchmarkZstdDecoding BenchmarkZstdDecoding-12 114854 10747 ns/op 18314 B/op 27 allocs/op ``` Fixes: #850 #849
masih
added a commit
that referenced
this issue
Jan 28, 2025
Pool the decode buffers and set strict max cap allocation for zstd decompressor. See Benchamark results below. Before: ``` BenchmarkCborEncoding BenchmarkCborEncoding-12 467352 2376 ns/op 14680 B/op 10 allocs/op BenchmarkCborDecoding BenchmarkCborDecoding-12 104410 11347 ns/op 14944 B/op 27 allocs/op BenchmarkZstdEncoding BenchmarkZstdEncoding-12 286735 3897 ns/op 46748 B/op 12 allocs/op BenchmarkZstdDecoding BenchmarkZstdDecoding-12 110794 10783 ns/op 28512 B/op 28 allocs/op ``` After: ``` BenchmarkCborEncoding BenchmarkCborEncoding-12 436754 2383 ns/op 14680 B/op 10 allocs/op BenchmarkCborDecoding BenchmarkCborDecoding-12 106809 11280 ns/op 14944 B/op 27 allocs/op BenchmarkZstdEncoding BenchmarkZstdEncoding-12 294043 3918 ns/op 46746 B/op 12 allocs/op BenchmarkZstdDecoding BenchmarkZstdDecoding-12 114854 10747 ns/op 18314 B/op 27 allocs/op ``` Fixes: #849
github-merge-queue bot
pushed a commit
that referenced
this issue
Jan 28, 2025
Pool the decode buffers and set strict max cap allocation for zstd decompressor. See Benchamark results below. Before: ``` BenchmarkCborEncoding BenchmarkCborEncoding-12 467352 2376 ns/op 14680 B/op 10 allocs/op BenchmarkCborDecoding BenchmarkCborDecoding-12 104410 11347 ns/op 14944 B/op 27 allocs/op BenchmarkZstdEncoding BenchmarkZstdEncoding-12 286735 3897 ns/op 46748 B/op 12 allocs/op BenchmarkZstdDecoding BenchmarkZstdDecoding-12 110794 10783 ns/op 28512 B/op 28 allocs/op ``` After: ``` BenchmarkCborEncoding BenchmarkCborEncoding-12 436754 2383 ns/op 14680 B/op 10 allocs/op BenchmarkCborDecoding BenchmarkCborDecoding-12 106809 11280 ns/op 14944 B/op 27 allocs/op BenchmarkZstdEncoding BenchmarkZstdEncoding-12 294043 3918 ns/op 46746 B/op 12 allocs/op BenchmarkZstdDecoding BenchmarkZstdDecoding-12 114854 10747 ns/op 18314 B/op 27 allocs/op ``` Fixes: #849
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
See:
The text was updated successfully, but these errors were encountered: