You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit b51031a
Browse filesBrowse the repository at this point in the historyBrowse files
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
28
-
tempor incididunt ut labore et dolore magna aliqua.
9
+
NDArray (:ref:`API <NDArray>`)
10
+
A compressed, chunked multidimensional data array. Supports NumPy-like
11
+
slicing and broadcasting, and out-of-core computation. Backed by an SChunk.
12
+
Use for array workloads, especially when too large to fit in memory
13
+
uncompressed.
29
14
30
15
CTable (:ref:`API <CTable>`)
31
16
A columnar table for structured data. Columns are stored, compressed, and
32
17
queried independently, with SUMMARY indexes available by default. Use with
33
18
structured data that benefits from compression, speed, and persistence.
34
19
35
-
filters (:class:`API <blosc2.Filter>`)
36
-
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
37
-
tempor incididunt ut labore et dolore magna aliqua.
38
-
39
-
frame
40
-
A serialized format for storing chunks along with a header and trailer for
41
-
metadata. Frames may be contiguous (CFrame) or sparse (SFrame).
42
-
43
-
FULL
44
-
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
45
-
tempor incididunt ut labore et dolore magna aliqua.
46
-
47
20
LazyArray (:ref:`API <LazyArray>`)
48
21
API to store an expression or function and delay computation until the
49
22
value is explicitly requested. Executes and stores results chunk-by-chunk.
@@ -52,28 +25,62 @@ Glossary
52
25
Object that stores an expression consisting of at least one NDArray
53
26
object. Follows the LazyArray API for storage and deferred computation.
54
27
55
-
NDArray (:ref:`API <NDArray>`)
56
-
A compressed, chunked multidimensional data array. Supports NumPy-like
57
-
slicing and broadcasting, and out-of-core computation. Backed by an SChunk.
58
-
Use for array workloads, especially when too large to fit in memory
59
-
uncompressed.
28
+
Compression
29
+
-----------
60
30
61
-
OPSI
62
-
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
63
-
tempor incididunt ut labore et dolore magna aliqua.
31
+
.. glossary::
64
32
65
-
PARTIAL
66
-
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
67
-
tempor incididunt ut labore et dolore magna aliqua.
33
+
codec (:class:`API <blosc2.Codec>`)
34
+
A compressor and matching decompressor that implements a particular
35
+
compression method. Codec choice affects compression/decompression speed and size of the compressed data.
36
+
37
+
clevel
38
+
An integer from 0 (no compression) to 9 (the highest compression level) that controls compression effort. Higher levels generally take longer to compress and may produce smaller results, depending on the codec and data.
39
+
40
+
filters (:class:`API <blosc2.Filter>`)
41
+
Transformations applied to data before the codec compresses it in order to expose patterns that can improve compression. Some filters are reversible (such as SHUFFLE), while others deliberately discard precision (such as TRUNC_PREC).
42
+
43
+
Low-level data structures
44
+
-------------------------
45
+
46
+
.. glossary::
68
47
69
48
SChunk (:ref:`API <SChunk>`)
70
49
The foundational container for managing a sequence of individual,
71
50
compressed chunks. NDArrays and CTable columns are built on top of SChunk.
72
51
Use when you want to directly manipulate raw compresesd data and metadata.
73
52
53
+
frame
54
+
A serialized format for storing chunks along with a header and trailer for
55
+
metadata. Frames may be contiguous (CFrame) or sparse (SFrame).
56
+
57
+
chunk
58
+
The unit of storage and compression, stored within a SChunk. Chunks are
59
+
sized to fit disk/network I/O, typically 1-64MB.
60
+
61
+
block
62
+
The unit of decompression, stored within a chunk. Blocks are sized to
63
+
fit CPU caches, typically 32-512KB.
64
+
74
65
subblock
75
66
An indexing segment within a block. One eighth the length of a block.
76
67
68
+
Indexes
69
+
-------
70
+
71
+
.. glossary::
72
+
77
73
SUMMARY
78
-
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
79
-
tempor incididunt ut labore et dolore magna aliqua.
74
+
Lightweight index that stores per-segment minimum and maximum values to skip segments that cannot match a query.
75
+
76
+
BUCKET
77
+
Stores values sorted within each chunk and groups their positions into buckets to locate possible query matches.
78
+
79
+
PARTIAL
80
+
Stores values sorted separately within each chunk, along with their exact positions, to find matches.
81
+
82
+
FULL
83
+
Stores values sorted together across all chunks, along with their exact positions, to find matches and speed up sorting.
84
+
85
+
OPSI
86
+
Uses repeated ordering cycles to improve filtering and provide exact matching positions for checking conditions on other columns, but is not intended to converge to a globally sorted index.
0 commit comments