Skip to content

fix(proxy): retry silent bridge creates sooner - #1555

Closed
Komzpa wants to merge 24 commits into
mainfrom
fix/live-http-bridge-missing-created-retry-v2-20260730
Closed

fix(proxy): retry silent bridge creates sooner#1555
Komzpa wants to merge 24 commits into
mainfrom
fix/live-http-bridge-missing-created-retry-v2-20260730

Conversation

@Komzpa

@Komzpa Komzpa commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • recover Responses HTTP bridge turns when the selected upstream accepts a create but emits no response.created, or closes before any user-visible output
  • rebind retry ownership, retire stale gate holders and anchors, and keep settlement/account-health updates aligned with the owner that actually failed or settled the attempt
  • penalize repeated owner-side missing_response_created_timeout failures so route selection can escape an eventless upstream account instead of reselecting it
  • cap the eventless pre-created wait at 15 seconds while honoring a lower configured stuck-gate threshold
  • persist the security lineage needed by durable bridge recovery and tolerate the compatible schema when it already exists
  • reconcile the branch with current main and merge the bridge and capability-lineage Alembic branches into one migration head

Context

The old bridge path could leave an accepted request attached to the same silent owner for minutes because no response.created event arrived to establish a durable anchor. The recovery path now treats that pre-visible state explicitly: it bounds the wait, clears terminal or stale ownership, records owner-side missing-created failures in account health, retries on a valid owner, and keeps accounting aligned with the owner that actually settled the attempt.

The branch's bridge compatibility migrations and the capability-lineage migration now on main both extend the earlier pending-tool-call chain. The final no-op merge revision names both revisions as parents, preserving both schema histories while restoring the repository's single-head Alembic invariant.

Validation

  • uv run --frozen pytest -q tests/unit/test_proxy_http_bridge.py::test_http_bridge_eventless_precreated_deadline_uses_current_send_and_client_safe_cap tests/integration/test_http_responses_bridge.py::test_v1_responses_http_bridge_retries_when_upstream_never_acknowledges_response_create tests/integration/test_http_responses_bridge.py::test_v1_responses_http_bridge_retries_once_when_upstream_closes_before_response_created
  • uv run pytest -q tests/unit/test_proxy_http_bridge.py
  • uv run pytest -q tests/unit/test_proxy_http_bridge.py -k 'eventless_timeout or missing_response_created or force_retires'
  • uv run --frozen pytest -q tests/unit/test_db_migrate.py
  • uv run --frozen ruff check app/modules/proxy/_service/http_bridge/helpers.py tests/unit/test_proxy_http_bridge.py tests/integration/test_http_responses_bridge.py app/db/alembic/versions/20260802_000000_merge_bridge_and_capability_lineage_heads.py tests/unit/test_db_migrate.py
  • uv run ruff check app/modules/proxy/_service/http_bridge/upstream_events.py tests/unit/test_proxy_http_bridge.py
  • uv run --frozen ruff format --check app/modules/proxy/_service/http_bridge/helpers.py tests/unit/test_proxy_http_bridge.py tests/integration/test_http_responses_bridge.py app/db/alembic/versions/20260802_000000_merge_bridge_and_capability_lineage_heads.py tests/unit/test_db_migrate.py
  • uv run --frozen ty check
  • openspec validate recover-codex-desktop-idle-bridge --strict
  • migration policy returned no violations and git diff --check passed
  • live-stack preflight for codex-lb:live-stack-89edc22 returned migration_policy=ok and schema_drift=none

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@github-actions github-actions Bot added the db migration PR changes Alembic database migrations; maintainer must coordinate merge order label Jul 31, 2026
Comment thread app/modules/proxy/security_lineage.py Fixed
@Komzpa Komzpa added the needs rebase Needs rebase or conflict repair against current main label Jul 31, 2026
@Komzpa
Komzpa force-pushed the fix/live-http-bridge-missing-created-retry-v2-20260730 branch from d0d264d to a448b9c Compare July 31, 2026 13:08
@Komzpa Komzpa removed the needs rebase Needs rebase or conflict repair against current main label Jul 31, 2026
@Komzpa
Komzpa force-pushed the fix/live-http-bridge-missing-created-retry-v2-20260730 branch from 6b8464c to 1b12951 Compare July 31, 2026 14:58
@Komzpa

Komzpa commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@Komzpa

Komzpa commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@Komzpa

Komzpa commented Aug 2, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@Komzpa

Komzpa commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@Soju06

Soju06 commented Aug 4, 2026

Copy link
Copy Markdown
Owner

Main moved under this one. #1394 (merged 08-04, a66f793) landed the bounded version of this recovery: 60s eventless cap (helpers.py:201 on main), one fresh-hard replay via recover-fresh-hard-bridge-timeouts, and a durable retry circuit (20260730_add_http_bridge_recovery_attempts). The spec on main now reads: the proxy MUST NOT transparently replay the timed-out request or move it to another account, and neither path may write an account-health failure solely because response.created was missing.

This PR's remaining delta reverses each of those decisions by editing the spec in place: cross-account owner rebind in request_submit.py::_retry_http_bridge_precreated_request, penalize_account=True in upstream_events.py, a 15s cap vs the spec's 60s, and _WEBSOCKET_TRANSPARENT_CLOSE_MAX_REPLAYS = 20 (support.py) raising the pre-created replay budget from 1 to 20. The failure mode the current spec guards against is concrete: after a sent response.create with no response.created, upstream acceptance is unknown — rebinding and resubmitting on another account can double-execute the turn. 20 transparent replays against a flapping upstream is also exactly the amplification pattern behind the #1354 lease exhaustion. If you want route selection to escape a repeatedly-eventless account, propose that as a fresh openspec change against post-#1394 main so it can be weighed against the just-merged bounded design.

Separately, the migration set can't land in any form: 20260722_add_security_lineage_persistence creates columns that don't exist in this repo's models or code (requires_security_work_authorized on usage_history/sticky_sessions/http_bridge_sessions, quota_planner_settings.auto_redeem_expiring_reset_credits / reset_credit_redeem_lead_minutes — zero references under app/), and the migrate.py _LEGACY_EXTRA_COLUMNS/_LEGACY_NULLABLE_COLUMNS additions exist only to suppress the drift those columns cause. That's reconciling a downstream deployment's schema into upstream. The backfill also runs pbkdf2 at 120k iterations per row inside the migration. And 20260802_merge_bridge_and_capability_lineage_heads is already stale: main's head is 20260803_merge_http_bridge_recovery_and_capability_lineage_heads, so after rebase this branch reintroduces dual heads.

Closing in favor of #1394; the _stream_once _TransientStreamError change for continuation streams touches the shared non-bridge path and, if it fixes something real, deserves its own PR with its own tests.

@Komzpa Komzpa added the needs rebase Needs rebase or conflict repair against current main label Aug 4, 2026
@Soju06

Soju06 commented Aug 4, 2026

Copy link
Copy Markdown
Owner

Closing per maintainer decision: superseded by #1394, and as filed this reverses merged spec MUST NOTs (duplicate-execution hazard) and carries fork-only schema migrations — full breakdown above. Any route-selection-escape behavior change needs a fresh owner-approved openspec change without the schema baggage.

@Soju06 Soju06 closed this Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

db migration PR changes Alembic database migrations; maintainer must coordinate merge order needs rebase Needs rebase or conflict repair against current main

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants