Skip to content

Update on LZMA Compression Memory Usage

Peter Hyman edited this page Apr 23, 2021 · 1 revision

LZMA2 Style Dictionary Sizes Now in Effect

lrzip-next Version 0.7.50 will mark a departure from the way Dictionary Sizes can be specified. lrzip-next will still automatically determine lzma dictionary sizes. But if the --dictsize= option is used, instead of 12-30, the options will range from 0-40. The table below will show the new mappings.

Dictionary
Code
2s Exponent 2*2^n
Dict Size
lrzip-next Level
Default Dict Size
Per Thread
Memory Overhead
Dictionary
Code
3*2^n
Dict Size
Per Thread
Memory Overhead
0 11 4,096 6,354,944 1 6,144 6,378,496
2 12 8,192 6,402,048 3 12,288 6,449,152
4 13 16,384 6,496,256 5 24,576 6,590,464
6 14 32,768 6,684,672 7 49,152 6,873,088
8 15 65,536 1 7,061,504 9 98,304 7,438,336
10 16 131,072 7,815,168 11 196,608 8,568,832
12 17 262,144 2 9,322,496 13 393,216 10,829,824
14 18 524,288 12,337,152 15 786,432 15,351,808
16 19 1,048,576 3 18,366,464 17 1,572,864 24,395,776
18 20 2,097,152 30,425,088 19 3,145,728 42,483,712
20 21 4,194,304 4 54,542,336 21 6,291,456 78,659,584
22 22 8,388,608 102,776,832 23 12,582,912 151,011,328
24 23 16,777,216 5 199,245,824 25 25,165,824 295,714,816
26 24 33,554,432 6,7 392,183,808 27 50,331,648 585,121,792
28 25 67,108,864 8 778,059,776 29 100,663,296 1,163,935,744
30 26 134,217,728 9 1,549,811,712 31 201,326,592 2,321,563,648
32 27 268,435,456 3,093,315,584 33 402,653,184 4,636,819,456
34 28 536,870,912 6,180,323,328 35 805,306,368 9,267,331,072
36 29 1,073,741,824 12,354,338,816 37 1,610,612,736 18,528,354,304
38 30 2,147,483,648 24,702,369,792 39 3,221,225,472 37,050,400,768
40 31 4,294,967,295 49,398,431,733

Memory Overhead per thread is computed as follows:
Dictionary Size * 11.5 + 6MB + 16KB
Dictionary Size will be automatically reduced if sufficient memory is not available