Follow up items for the week of July 11, 2026

[espg]

🤖 from Claude

Sweep of closed PRs from the last 10 days (July 1 – July 11, 2026). Issue #166 ("Follow up items for the week of July 4, 2026") is still open; per this repo's CLAUDE.md §6 on side-effecting actions, closing/consolidating it is left for @espg rather than done by this routine — its still-open items are re-verified against current main and folded in below rather than duplicated.

Re-verification of #166's 10 items:

• (3) the shard_order write-path knob is now resolved. Contrary to PR update benchmarks: forward sharded-vs-inner tdigest/HEALPix matrix #134's own checklist ("Phase 8 — Not started"), it actually shipped inside that same PR (commit 7886243, "phase 8 of issue update benchmarks #133", landed before update benchmarks: forward sharded-vs-inner tdigest/HEALPix matrix #134 merged). Confirmed live in src/zagg/config.py::get_shard_order, src/zagg/grids/healpix.py, src/zagg/processing/write.py, and tests/test_processing.py::TestShardOrderObjectSplit; issue ShardingCodec doesn't consolidate ragged (tdigest) writes — K serial CSR groups dominate shard write time #142 already references it as done.
• (4) the local-backend ragged-write fan-out cap and (7) the CSR-write "option 2" consolidation are in flight, not yet resolved: open draft PR Sharded vlen-bytes layout for ragged t-digest output (issue 209) #211 (issue t-digest ragged output bypasses the sharding codec (~1,792 objects/shard) — needs a sharding-codec-like path #209, opened today) deletes _write_ragged_fanout/src/zagg/csr.py entirely in favor of a sharded vlen-bytes layout, which would obsolete both concerns once merged. Still open on main today, so both remain live risks until Sharded vlen-bytes layout for ragged t-digest output (issue 209) #211 lands.
• (1), (2), (5), (6), (8), (9), (10) — all still outstanding, unchanged since Follow up items for the week of July 4, 2026 #166; re-confirmed against current main.

New items this week

(1) Lambda worker IAM role — delete permission spans Lambda's own distribution artifacts — PR #176

The worker execution role's s3:DeleteObject/PutObject grant covers the whole output bucket, which also holds versions.json and layer zips. Review thread (discussion_r3532665667, unresolved) proposed scoping to a prefix; never mitigated. Confirmed current: deployment/aws/execution_role.yaml's WriteOutputObjects statement still includes s3:DeleteObject with no narrower resource split. No tracking issue found.

(2) No hard cap on Lambda read fan-out budget — PR #183

granule_workers × read_workers × chunk-fetch width can exceed h5coro's S3Driver 100-connection budget. PR #183's body explicitly left this as documentation-only and asked "Add a validate_config warning if you want one?" — never answered. src/zagg/config.py::validate_config still has no such check, and PR #187 already bumped the granule_workers default to 4, increasing exposure.

(3) Direct process_shard callers get an unclamped granule_workers=4 default — PR #187

Callers that bypass the dispatcher aren't clamped to min(K, len(granule_urls)). Raised as "Questions for review" item 3 in the PR; unanswered, no clamp added since.

(4) espg's "belt-and-suspenders" malloc_trim question never answered — PR #140

espg asked in review (issuecomment-4851160003) whether to call malloc_trim at worker start too (not just teardown), given warm-container reuse risk. No reply, and no later PR revisits it. Note: PR #140 fell inside the window #166 already swept, but this specific question wasn't surfaced then — genuinely missed previously, still open now.

(5) Located-field streaming support explicitly rejected, no follow-up filed — PR #165

validate_streaming rejects located fields; the PR body says it's "a small follow-up... say the word and it becomes its own issue/PR." No such issue exists yet. It fails loudly today (not a silent gap), so risk is contained, but the gap is real.

(6) Dead codec-specific render path retained in plot_series.py — PR #196

make_codec_figure/_codec_layout/make_figure flagged in review as "prune in a follow-up. Low priority." (issuecomment-4911533452). Not addressed since.

(7) Small stale/undecided items in PR #182

template.yaml's RecycleMaxInvocations description text left stale (an offer to fix it wasn't taken up); publish_mirror.sh retirement ("in a follow-up" per docs) is still undecided.

(8) runner.py growth under issue #161 — worth re-flagging priority

Not a new gap (already tracked under open issue #161, labels plan+blocked), but the trend is worsening: src/zagg/runner.py is now ~114.6 KB (vs. ~1,360 lines when last measured in PR #70), and deployment/aws/lambda_handler.py (1,163 lines) and src/zagg/config.py (1,747 lines) now also exceed CLAUDE.md §4's ~1000-line-without-discussion convention. Flagging for re-prioritization, not as a new tracking target.

Priority / severity table

Priority	Item	Severity	Source
1	Lambda worker IAM role can delete its own distribution artifacts (versions.json, layer zips) — unscoped s3:DeleteObject	High	#176
2	Zero-photon granule read cost (carried forward from #166 item 1) — wasted full segment I/O by default	High	#166 / PR #96
3	No hard cap on Lambda read fan-out budget — can exceed h5coro's 100-connection budget, risk increased by #187's default bump	Medium-High	#183
4	Sync Lambda-dispatch fallback missing tcp_keepalive (carried forward from #166 item 5) — same idle-kill failure mode #151 was opened for	Medium	#166 / PR #153
5	Unclamped granule_workers=4 default for direct process_shard callers	Medium	#187
6	malloc_trim worker-start question never answered	Medium	#140
7	runner.py / lambda_handler.py / config.py size growth — already tracked (#161), re-flagging priority	Medium	#161
8	Run-level retry budget missing (carried forward from #166 item 6) — no circuit breaker for systemic failures	Low-Medium	#166
9	Local-backend ragged-write fan-out cap / CSR-write consolidation (carried forward from #166 items 4, 7) — in flight via draft PR #211	Low-Medium	#166 / PR #211
10	Located-field streaming rejected, no follow-up filed	Low-Medium	#165
11	publish_mirror.sh --creds ignores region/endpoint (carried forward from #166 item 2)	Low	#166
12	Upstream h5coro hyperslice hardening one-liner (carried forward from #166 item 8)	Low	#166
13	ipyleaflet shardmap viewer browser verification deferred (carried forward from #166 item 9)	Low	#166
14	Benchmark runner abort/progress-streaming nits (carried forward from #166 item 10)	Low	#166
15	Dead codec-specific render path in plot_series.py	Low	#196
16	Stale RecycleMaxInvocations description + undecided publish_mirror.sh retirement	Low	#182

[espg]

🤖 from Claude

Recommendation: sort the 16 items into four dispatch buckets so the self-contained src-only fixes can move immediately while the infra/undecided ones wait on a call from you. Nothing below is code — it's a triage proposal plus the one-pass decision each bucket needs. Anchoring each item to its source PR rather than a list index (the body carries two different numberings).

Bucket A — src-only, low-risk, bundleable as one small-fix PR (needs your green-light)

Self-contained, test-coverable, touch no infra:

• PR Inline selection-read heuristic + granule_workers default 4 #187 item — clamp granule_workers to min(K, len(granule_urls)) for direct process_shard callers (config.py/runner.py). The dispatcher already clamps; this closes the bypass the PR flagged.
• PR Rework benchmark matrix: sharded tdigest, inline-vs-sidecar, K=4, 4 GB #196 item — prune the dead make_codec_figure/_codec_layout/make_figure path in plot_series.py (already flagged "prune in a follow-up, low priority" in review).
• PR Granule-level read concurrency (K in flight, ordered fold) #183 item, conditional — a validate_config warning when granule_workers × read_workers × chunk-width exceeds h5coro's ~100-connection S3Driver budget. PR Granule-level read concurrency (K in flight, ordered fold) #183 explicitly asked "add one if you want?" and it was never answered — yes/no? If yes it joins this bundle.

Tradeoff: bundling keeps review cheap (one PR, a Closes per converted issue) but couples three unrelated fixes; if you'd rather they land independently, say so and I'll file them separately.

Bucket B — needs a decision from you before any code (one-pass)

• PR Reclaim worker RSS on warm Lambda containers (gc + malloc_trim) (#139) #140 item — malloc_trim at worker start (not just teardown). This is your own review question, never answered. (a) add the belt-and-suspenders trim at worker entry given warm-container reuse, or (b) leave teardown-only?
• PR High-resolution morton location channel for ragged/CSR fields #165 item — located-field streaming is cleanly rejected today (fails loud, not a silent gap). File a tracking issue for the follow-up, or leave as-is? Opening an issue is side-effecting, so I need your word per §6.

Bucket C — infra/deploy, cannot touch without you naming the file (§1)

• PR fix main: pin hierarchical seam in worker-double tests + template pins (issue #175) #176 item — the unscoped s3:DeleteObject on the worker role (deployment/aws/execution_role.yaml). Scoping it to the store prefix (excluding versions.json + layer zips) is the right fix but edits an infra template. This is the highest-severity item on the list — a worker can delete its own distribution artifacts. Authorize a dedicated implement issue naming execution_role.yaml?
• PR Small fixes: recycle generation counting + stand_up.sh artifact defaults #182 items — stale RecycleMaxInvocations description in template.yaml + the undecided publish_mirror.sh retirement. Both infra; fold into the same infra issue, or drop?

Bucket D — already tracked / in flight, no new action (dedup)

• CSR-write consolidation → superseded by draft PR Sharded vlen-bytes layout for ragged t-digest output (issue 209) #211 (issue t-digest ragged output bypasses the sharding codec (~1,792 objects/shard) — needs a sharding-codec-like path #209); resolves on merge.
• runner.py / lambda_handler.py / config.py >1000-line growth → tracked under Review zagg codebase #161 (plan+blocked); worth re-prioritizing there, not a new target.
• The remaining Follow up items for the week of July 4, 2026 #166 carry-forwards (zero-photon read cost, sync-dispatch tcp_keepalive, run-level retry budget, publish_mirror.sh --creds region, the h5coro hyperslice one-liner, viewer browser-verification, benchmark-runner nits) are unchanged. The zero-photon read cost is the biggest of these (your severity table's row 2) and really wants its own plan pass, not a small-fix.

Suggested sequencing (pick one)

1. Fast path — green-light Bucket A now; I open the small-fix PR next run, Buckets B/C answered async.
2. Severity-first — stand up the infra issue for the PR fix main: pin hierarchical seam in worker-double tests + template pins (issue #175) #176 DeleteObject gap first (highest severity), Bucket A second.
3. Batch — answer B + C inline and I convert the whole set into 2–3 labeled issues in one pass so tracking is clean.

Concrete asks: (i) green-light Bucket A — and is the PR #183 warning in or out? (ii) PR #140 trim-at-start: yes or no? (iii) PR #165 located-streaming: file or leave? (iv) PR #176 + #182 infra: authorize a dedicated implement issue naming execution_role.yaml/template.yaml, or hold?

Applying waiting pending your call.

---

Generated by Claude Code