fix(v1): discover the context window under the model card's specs - #2634
Open
parkerpettit wants to merge 1 commit into
Open
parkerpettit wants to merge 1 commit into
parkerpettit wants to merge 1 commit into
Conversation
Prime Inference reports a model's context window as specs.context_window on its /v1/models cards, not at the top level, so discover_threshold returned None there and auto-compaction never triggered. Read the nested field too.
parkerpettit
marked this pull request as ready for review
September 19, 2026 17:36
Contributor
ApprovabilityVerdict: Approved at Macroscope's review found this PR approvable — This one-file bug fix extends existing context-window lookup to nested model-card You can add or adjust custom eligibility rules. Learn more. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Prime Inference reports a model's context window as
specs.context_windowon its/v1/modelscards rather than at the top level, sodiscover_thresholdreturnedNonethere and Bash auto-compaction (compaction = {}with nosummarize_at_tokens) never triggered: no proactive compaction, and an overflow was re-raised instead of compacted. Read the nested field too.Checked against Prime Inference with the bundled Bash program behind a request-logging proxy. Before:
discover_threshold("openai/gpt-4o-mini")wasNone, and a tool-calling run in auto mode made no checkpoint request while the same run with an explicit--summarize-at-tokenscompacted. After: discovery returns 111616 (128000 − 16384) for gpt-4o-mini and 43616 for a 60k-window model, and in auto mode the program issues the checkpoint-summary request once a tool turn crosses the threshold.Note
Fix
discover_thresholdto read context-window fields from model cardspecsmetadataUpdates
discover_thresholdin compaction.py to also inspect a dictionary-valuedspecsextension when resolving configured context-window fields. Top-level metadata remains preferred, and positive integer validation still applies before deriving the threshold.Macroscope summarized 7dfbfae.