Skip to content

Commit 7dd0087

Browse files
dfa1claude
andcommitted
docs(adr-0004): explain why resource caps are open-time, not per-scan
Add a "Why open-time, not per-scan" subsection: structural bombs (giant mmap, million-entry segment table, depth/child-count explosion) detonate during open() before a ScanOptions exists, so scan-time caps run after the damage. Enforce each cap at the earliest point the resource is consumed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 8c28e91 commit 7dd0087

1 file changed

Lines changed: 34 additions & 0 deletions

File tree

docs/adr/0004-resource-caps-read-options.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,40 @@ currently no limits on:
2626
The fix is a `ResourceLimits` value that is enforced at open/parse time,
2727
before any byte is decoded.
2828

29+
### Why open-time, not per-scan
30+
31+
The natural instinct is to hang these caps off `ScanOptions`, next to the
32+
existing row `limit`. That is too late. The structural attacks **detonate during
33+
`open()` — before a `ScanOptions` even exists.** `open(path)` already:
34+
35+
1. memory-maps the **entire file** (`channel.map(READ_ONLY, 0, size, arena)`) — a
36+
100 GB file exhausts virtual address space here;
37+
2. parses the postscript → footer → layout-tree flatbuffers;
38+
3. reads the **segment table** (a crafted file can declare millions of entries);
39+
4. walks the **layout tree** (depth / child-count bomb).
40+
41+
By the time a caller builds `ScanOptions` and calls `scan()`, the file is already
42+
mapped and the layout tree already parsed — the OOM / address-space exhaustion /
43+
depth-bomb has already happened. A scan-time check runs after the damage.
44+
45+
The governing rule: **enforce each cap at the earliest point the resource is
46+
consumed.** For the structural caps that is `open()`/parse, not scan.
47+
48+
There is also a scope mismatch. Caps like `maxFileSizeBytes` and
49+
`maxSegmentCount` are properties of the **file + reader session**, not of an
50+
individual scan: one `open()` feeds many `scan()` calls. Placing them on
51+
`ScanOptions` would force the caller to re-pass the same limit on every scan and
52+
*still* could not guard `open()`.
53+
54+
| Resource | Consumed / detonates at | Configured via |
55+
|----------|-------------------------|----------------|
56+
| file mmap, segment table, layout depth / children / node count | `open()` / parse | `ReadOptions` |
57+
| per-chunk decode allocation (`rows × byteWidth`) | decode (during `scan()`) | `ReadOptions` (`maxRowsPerChunk`, a layout-declared count fixed at open) |
58+
| output row count | `scan()` | `ScanOptions.limit` (already exists) |
59+
60+
So `ScanOptions` keeps the one genuinely per-scan knob (output `limit`); every
61+
structural cap moves to a new open-time `ReadOptions`.
62+
2963
### Where limits live — the decision
3064

3165
Three candidates:

0 commit comments

Comments
 (0)