Skip to content

fix(proxy): recover stale responses bridge sessions - #4

Merged
hongzexin merged 13 commits into
mainfrom
fix/responses-bridge-stale-anchor-recovery
Aug 22, 2026
Merged

hongzexin merged 13 commits into
mainfrom
fix/responses-bridge-stale-anchor-recovery

Conversation

@hongzexin

Copy link
Copy Markdown

Summary

Fix the production Responses bridge failure loop where an hour-old WebSocket closes, a proxy-injected previous response anchor is rejected on the replacement socket, and client retries exhaust themselves inside the durable cooldown. The change recovers only across proved pre-dispatch boundaries and never silently drops conversation context.

Type of change

  • fix: bug fix
  • Breaking change

Linked issue: repository issues are disabled; production incident request IDs 36864672-0305-4b82-a4e1-ef43054cb83a and 0008e69f-3547-4242-8ca5-32babfc3bedf.

OpenSpec

  • This PR updates an OpenSpec change
  • This PR touches a codex-faithful request path and preserves upstream-equivalent behavior

Change directory: openspec/changes/recover-repeated-clean-close/

Changes

  • Prove an upstream socket is closed before invoking its send primitive, then reconnect and dispatch exactly once. Post-send failures remain ambiguous and non-replayable.
  • Return one atomic retry-circuit snapshot with accurate failure-class copy and a matching Retry-After value; successful completion clears the circuit.
  • Quarantine an explicitly rejected proxy-injected previous response anchor. A verified complete resend may recover once unanchored on the same account; otherwise the current request fails closed and the next complete client resend takes a fresh unanchored bridge. Delta-only and client-owned anchors are never discarded.

Simplicity

  • Works with zero config
  • No new setting, setup step, README section, dashboard navigation, or default change

Test plan

  • OpenSpec 1.10.0 validate --specs: 49 passed, 0 failed
  • proxy architecture check, Ruff lint/format, git diff check: pass
  • ty check: pass
  • focused affected tests: 34 passed
  • related unit and integration files: 680 passed
  • non-Helm unit suite: 5758 passed, 3 skipped
  • HTTP/WebSocket bridge integration suites: 235 passed
  • four Helm rendering files were excluded locally because the external chart dependency fetch timed out; cloud CI remains authoritative for them

Screenshots / output

Before: the desktop reported Invalid previous_response_id and then HTTP 503 cooling down after repeated upstream timeouts.
After regression: the first unproved stale-anchor request returns previous_response_anchor_unrecoverable without a duplicate upstream dispatch; a subsequent complete resend opens one fresh bridge without the rejected anchor and completes.

Checklist

  • Conventional Commit title
  • Production incident linked above; GitHub issues are disabled
  • Added externally failing path regression coverage
  • Ran relevant local CI subsets
  • OpenSpec validation passes
  • Simplicity gates reviewed
  • CHANGELOG not edited by hand

Prevent already-closed WebSocket send races, expose accurate retry cooldowns, and quarantine rejected proxy-injected response anchors until a complete client resend can recover without context loss.

Source sidechat: 019fda58-2804-71b0-aacb-8f150d8b03a7

Recovered from: 2160aea
@hongzexin

Copy link
Copy Markdown
Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 09f28240f9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread app/modules/proxy/_service/http_bridge/request_submit.py
Source sidechat: 019fda58-2804-71b0-aacb-8f150d8b03a7

Recovered from: 09f2824
@hongzexin

Copy link
Copy Markdown
Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 820e0be117

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread app/modules/proxy/_service/http_bridge/streaming.py
Comment thread app/modules/proxy/_service/http_bridge/streaming.py Outdated
Source sidechat: 019fda58-2804-71b0-aacb-8f150d8b03a7

Recovered from: 820e0be
@hongzexin

Copy link
Copy Markdown
Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bad747d854

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread app/modules/proxy/_service/http_bridge/request_submit.py
Source sidechat: 019fda58-2804-71b0-aacb-8f150d8b03a7

Recovered from: bad747d
@hongzexin

Copy link
Copy Markdown
Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9cd8de6efd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread openspec/changes/recover-repeated-clean-close/specs/responses-api-compat/spec.md Outdated
Source sidechat: 019fda58-2804-71b0-aacb-8f150d8b03a7

Recovered from: 9cd8de6
@hongzexin

Copy link
Copy Markdown
Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4572486843

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread app/modules/proxy/_service/http_bridge/request_submit.py Outdated
Comment thread app/modules/proxy/_service/http_bridge/request_submit.py
Keep uploaded-file recovery pinned to the owning account and restore durable recovery aliases when replacement setup fails before the send primitive is reached.

Source sidechat: 019fda58-2804-71b0-aacb-8f150d8b03a7

Recovered from: 4572486
@hongzexin

Copy link
Copy Markdown
Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 11b6a83610

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread app/modules/proxy/_service/http_bridge/request_submit.py Outdated
Keep the clean-close integration contract on the instance registered by the application lifespan so durable retirement timing cannot manufacture an unrelated foreign-owner mismatch.

Source sidechat: 019fda58-2804-71b0-aacb-8f150d8b03a7

Recovered from: 11b6a83
@hongzexin

Copy link
Copy Markdown
Author

CI exposed a deterministic test-harness identity race rather than a product assertion failure: the clean-close reconnect test registered the app lifespan with the runtime instance, then patched request handling to synthetic instance-a. Fixed in 0d88bb3 by binding that test to the registered runtime owner while preserving the 200→200 assertion. Evidence: target repeated 20/20; full bridge integration 236/236; bridge unit 527/527; ty, architecture, Ruff, and diff-check all pass.

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: 0d88bb3e16

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Source sidechat: 019fda58-2804-71b0-aacb-8f150d8b03a7

Recovered from: 0d88bb3
@hongzexin

Copy link
Copy Markdown
Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e86918fda7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread app/modules/proxy/_service/http_bridge/request_submit.py Outdated
Source sidechat: 019fda58-2804-71b0-aacb-8f150d8b03a7

Recovered from: e86918f
@hongzexin

Copy link
Copy Markdown
Author

@codex review

Final review requested for exact head d81ba12 after closing the account-neutral replay finding.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d81ba12013

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread app/modules/proxy/_service/http_bridge/request_submit.py Outdated
Source sidechat: 019fda58-2804-71b0-aacb-8f150d8b03a7

Recovered from: d81ba12
@hongzexin

Copy link
Copy Markdown
Author

@codex review

Final review requested for exact head ff6f3a2 after closing the hard-affinity header isolation finding.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ff6f3a2a1d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread app/modules/proxy/_service/http_bridge/request_submit.py Outdated
Source sidechat: 019fda58-2804-71b0-aacb-8f150d8b03a7

Recovered from: ff6f3a2
@hongzexin

Copy link
Copy Markdown
Author

@codex review

Final review requested for exact head cdb2f1e after pinning all physical-socket recovery to the current owner.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cdb2f1e614

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread app/modules/proxy/_service/http_bridge/request_submit.py
Source sidechat: 019fda58-2804-71b0-aacb-8f150d8b03a7

Recovered from: cdb2f1e
@hongzexin

Copy link
Copy Markdown
Author

@codex review

Final review requested for exact head 3de58f3. This follow-up only aligns the complete unit-suite archive-id oracle with the owner-pinned recovery contract; the focused test passes, and the prior cloud run passed 5,853/5,854 unit tests with only this stale assertion failing.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3de58f3df8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread app/modules/proxy/_service/http_bridge/request_submit.py
Source sidechat: 019fda58-2804-71b0-aacb-8f150d8b03a7

Recovered from: 3de58f3
@hongzexin

Copy link
Copy Markdown
Author

@codex review

Final review requested for exact head f62569e after enforcing the same-account bound inside the reconnect selector for soft and hard keys and adding a selector-drift regression.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: f62569e639

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@hongzexin
hongzexin requested a review from Deng-summer August 22, 2026 11:44
@hongzexin

Copy link
Copy Markdown
Author

Production reliability hotfix is fully green (27 successful checks, 0 failures), exact-head Codex review is clean, and all review threads are resolved. Requesting the required independent write-access approval for merge; deployment remains held until this repository rule is satisfied.

@hongzexin

Copy link
Copy Markdown
Author

@maoqiano Could you please provide the required independent approval for this production reliability hotfix when available? Exact head f62569e has 27/27 applicable checks green, final Codex review clean, and 0 unresolved threads. The deployment is held solely by the organization review rule.

@hongzexin
hongzexin enabled auto-merge August 22, 2026 11:52
@hongzexin
hongzexin merged commit 126cdd5 into main Aug 22, 2026
33 checks passed
@hongzexin
hongzexin deleted the fix/responses-bridge-stale-anchor-recovery branch August 22, 2026 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant