test(agent-core-v2): give the sessionIndex list baseline CI headroom - #2615
Conversation
|
commit: |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7377f277ef
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // tripwire keeps 3x headroom. get/count stay at 50ms: they measure ~1ms | ||
| // even at 50k rows. The complexity check is the real guard: 50x the rows | ||
| // must not cost ~50x the time. | ||
| expect(at50k.list).toBeLessThan(300); |
There was a problem hiding this comment.
When the 1k measurement remains fast but the later 50k measurement is delayed by shared-runner load, the unchanged final assertion is still much tighter than this new 300 ms limit—for example, an at1k.list median of 2 ms caps at50k.list at 70 ms. Thus the cited 100–140 ms CI medians can still fail whenever the load spike affects only the later phase, so this change does not reliably provide the intended headroom; the relative check also needs a noise-tolerant bound or a non-wall-clock complexity signal.
Useful? React with 👍 / 👎.
…unner load spikes
…ot wall clock The baseline asserted wall-clock medians (list < 300ms, get/count < 50ms) plus a relative time check — all still load-sensitive on shared CI runners. Replace every time assertion with behavioral complexity assertions recorded at the IQueryStore and fs seams: list must be served by bounded pageByColumn fetches only, get is a single point lookup, count never touches the session collection, no warm read enumerates session directories, and the per-op work snapshot must be identical at 1k, 10k, and 50k sessions. Medians are still logged for phase-to-phase comparison; the retry now absorbs a background reconcile tick landing inside a counting window.
2 participants
Related Issue
No linked issue — the problem is explained below.
Problem
The
baseline: warm listRecent(limit=20) at 1k vs 10k vs 50k sessionstest asserts a wall-clock median below 100ms on shared CI runners. Under runner load the 50k-row list median exceeds the budget and fails unrelated PRs: 100.17ms on a main run (run 30910016839) and 139.55ms on PR #2614. The section comment already states the design intent — numbers are logged for phase-to-phase comparison and only loose complexity budgets should gate CI.What changed
listtripwire from 100ms to 300ms, ~3x headroom over the worst observed CI median (139.55ms); the same median is ~2ms on an uncontended machine, so a genuine regression still trips the assertion.get/countbudgets at 50ms untouched: they measure ~0.8ms even at 50k rows, so they already carry ~60x headroom.at50k.list < at1k.list * 10 + 50) as the primary regression guard, matching the section design.Verified locally: the full
sessionIndex.test.tsfile passes (29/29).Checklist
gen-changesetsskill, or this PR needs no changeset.gen-docsskill, or this PR needs no doc update.