Skip to content

fix(v1): discover the context window under the model card's specs - #2634

Open
parkerpettit wants to merge 1 commit into
mainfrom
fix/compaction-discover-specs-context-window
Open

parkerpettit wants to merge 1 commit into
mainfrom
fix/compaction-discover-specs-context-window

Conversation

@parkerpettit

@parkerpettit parkerpettit commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Prime Inference reports a model's context window as specs.context_window on its /v1/models cards rather than at the top level, so discover_threshold returned None there and Bash auto-compaction (compaction = {} with no summarize_at_tokens) never triggered: no proactive compaction, and an overflow was re-raised instead of compacted. Read the nested field too.

Checked against Prime Inference with the bundled Bash program behind a request-logging proxy. Before: discover_threshold("openai/gpt-4o-mini") was None, and a tool-calling run in auto mode made no checkpoint request while the same run with an explicit --summarize-at-tokens compacted. After: discovery returns 111616 (128000 − 16384) for gpt-4o-mini and 43616 for a 60k-window model, and in auto mode the program issues the checkpoint-summary request once a tool turn crosses the threshold.

Note

Fix discover_threshold to read context-window fields from model card specs metadata

Updates discover_threshold in compaction.py to also inspect a dictionary-valued specs extension when resolving configured context-window fields. Top-level metadata remains preferred, and positive integer validation still applies before deriving the threshold.

Macroscope summarized 7dfbfae.

Prime Inference reports a model's context window as specs.context_window on
its /v1/models cards, not at the top level, so discover_threshold returned
None there and auto-compaction never triggered. Read the nested field too.
@parkerpettit
parkerpettit marked this pull request as ready for review September 19, 2026 17:36
@macroscopeapp

macroscopeapp Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 7dfbfae

Macroscope's review found this PR approvable — This one-file bug fix extends existing context-window lookup to nested model-card specs metadata while preserving current top-level provider behavior. Its runtime impact is limited to allowing the existing compactor to discover thresholds for affected provider responses, with no schema, security, or deployment changes.

You can add or adjust custom eligibility rules. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant