Skip to content

Commit b51031a

Browse files
kunalsingh1096lanceu0128
authored andcommitted
docs: divided glossary into readable section and wrote compression and index sections
1 parent aa2cec6 commit b51031a

1 file changed

Lines changed: 55 additions & 48 deletions

File tree

‎doc/glossary/index.rst‎

Lines changed: 55 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -1,49 +1,22 @@
11
Glossary
22
========
33

4-
.. glossary::
5-
6-
block
7-
The unit of decompression, stored within a chunk. Blocks are sized to
8-
fit CPU caches, typically 32-512KB.
4+
Data containers
5+
---------------
96

10-
BUCKET
11-
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
12-
tempor incididunt ut labore et dolore magna aliqua.
13-
14-
chunk
15-
The unit of storage and compression, stored within a SChunk. Chunks are
16-
sized to fit disk/network I/O, typically 1-64MB.
17-
18-
clevel
19-
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
20-
tempor incididunt ut labore et dolore magna aliqua.
21-
22-
codec (:class:`API <blosc2.Codec>`)
23-
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
24-
tempor incididunt ut labore et dolore magna aliqua.
7+
.. glossary::
258

26-
compression parameters (:class:`API <blosc2.CParams>`)
27-
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
28-
tempor incididunt ut labore et dolore magna aliqua.
9+
NDArray (:ref:`API <NDArray>`)
10+
A compressed, chunked multidimensional data array. Supports NumPy-like
11+
slicing and broadcasting, and out-of-core computation. Backed by an SChunk.
12+
Use for array workloads, especially when too large to fit in memory
13+
uncompressed.
2914

3015
CTable (:ref:`API <CTable>`)
3116
A columnar table for structured data. Columns are stored, compressed, and
3217
queried independently, with SUMMARY indexes available by default. Use with
3318
structured data that benefits from compression, speed, and persistence.
3419

35-
filters (:class:`API <blosc2.Filter>`)
36-
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
37-
tempor incididunt ut labore et dolore magna aliqua.
38-
39-
frame
40-
A serialized format for storing chunks along with a header and trailer for
41-
metadata. Frames may be contiguous (CFrame) or sparse (SFrame).
42-
43-
FULL
44-
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
45-
tempor incididunt ut labore et dolore magna aliqua.
46-
4720
LazyArray (:ref:`API <LazyArray>`)
4821
API to store an expression or function and delay computation until the
4922
value is explicitly requested. Executes and stores results chunk-by-chunk.
@@ -52,28 +25,62 @@ Glossary
5225
Object that stores an expression consisting of at least one NDArray
5326
object. Follows the LazyArray API for storage and deferred computation.
5427

55-
NDArray (:ref:`API <NDArray>`)
56-
A compressed, chunked multidimensional data array. Supports NumPy-like
57-
slicing and broadcasting, and out-of-core computation. Backed by an SChunk.
58-
Use for array workloads, especially when too large to fit in memory
59-
uncompressed.
28+
Compression
29+
-----------
6030

61-
OPSI
62-
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
63-
tempor incididunt ut labore et dolore magna aliqua.
31+
.. glossary::
6432

65-
PARTIAL
66-
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
67-
tempor incididunt ut labore et dolore magna aliqua.
33+
codec (:class:`API <blosc2.Codec>`)
34+
A compressor and matching decompressor that implements a particular
35+
compression method. Codec choice affects compression/decompression speed and size of the compressed data.
36+
37+
clevel
38+
An integer from 0 (no compression) to 9 (the highest compression level) that controls compression effort. Higher levels generally take longer to compress and may produce smaller results, depending on the codec and data.
39+
40+
filters (:class:`API <blosc2.Filter>`)
41+
Transformations applied to data before the codec compresses it in order to expose patterns that can improve compression. Some filters are reversible (such as SHUFFLE), while others deliberately discard precision (such as TRUNC_PREC).
42+
43+
Low-level data structures
44+
-------------------------
45+
46+
.. glossary::
6847

6948
SChunk (:ref:`API <SChunk>`)
7049
The foundational container for managing a sequence of individual,
7150
compressed chunks. NDArrays and CTable columns are built on top of SChunk.
7251
Use when you want to directly manipulate raw compresesd data and metadata.
7352

53+
frame
54+
A serialized format for storing chunks along with a header and trailer for
55+
metadata. Frames may be contiguous (CFrame) or sparse (SFrame).
56+
57+
chunk
58+
The unit of storage and compression, stored within a SChunk. Chunks are
59+
sized to fit disk/network I/O, typically 1-64MB.
60+
61+
block
62+
The unit of decompression, stored within a chunk. Blocks are sized to
63+
fit CPU caches, typically 32-512KB.
64+
7465
subblock
7566
An indexing segment within a block. One eighth the length of a block.
7667

68+
Indexes
69+
-------
70+
71+
.. glossary::
72+
7773
SUMMARY
78-
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
79-
tempor incididunt ut labore et dolore magna aliqua.
74+
Lightweight index that stores per-segment minimum and maximum values to skip segments that cannot match a query.
75+
76+
BUCKET
77+
Stores values sorted within each chunk and groups their positions into buckets to locate possible query matches.
78+
79+
PARTIAL
80+
Stores values sorted separately within each chunk, along with their exact positions, to find matches.
81+
82+
FULL
83+
Stores values sorted together across all chunks, along with their exact positions, to find matches and speed up sorting.
84+
85+
OPSI
86+
Uses repeated ordering cycles to improve filtering and provide exact matching positions for checking conditions on other columns, but is not intended to converge to a globally sorted index.

0 commit comments

Comments
 (0)