-
Notifications
You must be signed in to change notification settings - Fork 11
Update on LZMA Compression Memory Usage
lrzip-next
Version 0.7.50 will mark a departure from the way Dictionary Sizes can be specified. lrzip-next
will still automatically determine lzma
dictionary sizes. But if the --dictsize=
option is used, instead of 12-30, the options will range from 0-40. The table below will show the new mappings.
Dictionary Code |
2s Exponent | 2*2^n Dict Size |
lrzip-next Level Default Dict Size |
Per Thread Memory Overhead |
Dictionary Code |
3*2^n Dict Size |
Per Thread Memory Overhead |
---|---|---|---|---|---|---|---|
0 | 11 | 4,096 | 6,354,944 | 1 | 6,144 | 6,378,496 | |
2 | 12 | 8,192 | 6,402,048 | 3 | 12,288 | 6,449,152 | |
4 | 13 | 16,384 | 6,496,256 | 5 | 24,576 | 6,590,464 | |
6 | 14 | 32,768 | 6,684,672 | 7 | 49,152 | 6,873,088 | |
8 | 15 | 65,536 | 1 | 7,061,504 | 9 | 98,304 | 7,438,336 |
10 | 16 | 131,072 | 7,815,168 | 11 | 196,608 | 8,568,832 | |
12 | 17 | 262,144 | 2 | 9,322,496 | 13 | 393,216 | 10,829,824 |
14 | 18 | 524,288 | 12,337,152 | 15 | 786,432 | 15,351,808 | |
16 | 19 | 1,048,576 | 3 | 18,366,464 | 17 | 1,572,864 | 24,395,776 |
18 | 20 | 2,097,152 | 30,425,088 | 19 | 3,145,728 | 42,483,712 | |
20 | 21 | 4,194,304 | 4 | 54,542,336 | 21 | 6,291,456 | 78,659,584 |
22 | 22 | 8,388,608 | 102,776,832 | 23 | 12,582,912 | 151,011,328 | |
24 | 23 | 16,777,216 | 5 | 199,245,824 | 25 | 25,165,824 | 295,714,816 |
26 | 24 | 33,554,432 | 6,7 | 392,183,808 | 27 | 50,331,648 | 585,121,792 |
28 | 25 | 67,108,864 | 8 | 778,059,776 | 29 | 100,663,296 | 1,163,935,744 |
30 | 26 | 134,217,728 | 9 | 1,549,811,712 | 31 | 201,326,592 | 2,321,563,648 |
32 | 27 | 268,435,456 | 3,093,315,584 | 33 | 402,653,184 | 4,636,819,456 | |
34 | 28 | 536,870,912 | 6,180,323,328 | 35 | 805,306,368 | 9,267,331,072 | |
36 | 29 | 1,073,741,824 | 12,354,338,816 | 37 | 1,610,612,736 | 18,528,354,304 | |
38 | 30 | 2,147,483,648 | 24,702,369,792 | 39 | 3,221,225,472 | 37,050,400,768 | |
40 | 31 | 4,294,967,295 | 49,398,431,733 |
Memory Overhead per thread is computed as follows:
Dictionary Size * 11.5 + 6MB + 16KB
Dictionary Size will be automatically reduced if sufficient memory is not available
Home
About
No More 7/9 Compression Level Limits
Threshold Limits
Using tar
with lrzip-next
Piping with lrzip-next
How to Maximize Compression
HOWTO Speed up Compiling
Feature added: Debugging
What we pass to LZMA
What we pass to ZPAQ
What we pass to BZIP3
What we pass to ZSTD
Computing Memory Overhead
Increasing Block Sizes to Compress
LZMA2 Dictionary Sizes NEW
Setting Cost Factor NEW