Skip to content

fix(e2e): make the debugger assertions fail when the debugger is not connected - #704

Draft
latekvo wants to merge 1 commit into
mainfrom
fix/e2e-debugger-assertions
Draft

fix(e2e): make the debugger assertions fail when the debugger is not connected#704
latekvo wants to merge 1 commit into
mainfrom
fix/e2e-debugger-assertions

Conversation

@latekvo

@latekvo latekvo commented Aug 3, 2026

Copy link
Copy Markdown
Member

Found while reviewing #610 and set aside as out of scope for that review: the weaker of the two assertions is a live defect on main today, the other becomes one when #610 lands.

The defect

assert_ok (lib/common.sh:122-130) records PASS on RT_RC == 0 and never reads RT_JSON. Both tools on lines 84 and 86 answer HTTP 200 while reporting that the debugger is not connected, so the two assertions that exist to check the debugger chain pass over a dead session.

Line 84 is broken on main today. Half-close the CDP socket server-side (the socket lands in CLOSING with no close event, so no terminated cascade) and the cached service still resolves:

ground truth: {"connected":false}

=== shipped assertions (lines 84/86) ===
  ✓ debugger-status status
  ✓ debugger-log-registry logs
=== patched assertions ===
  ✗ debugger-status [status] expected .connected to be true, got 'false'
  ✓ debugger-log-registry logs

Line 86 breaks once #610 lands, which turns the unreachable-runtime 500 into a 200 not_connected payload. Same two lines against a build of that branch with nothing listening on the Metro port:

ground truth: {"status":"not_connected","connected":false,"reason":"metro_not_running"}

=== shipped ===   ✓ debugger-status status   ✓ debugger-log-registry logs
=== patched ===   ✗ debugger-status [status] expected .connected to be true, got 'false'
                  ✗ debugger-log-registry [logs] expected (.status // "connected")=connected got 'not_connected'

Why these two jq paths

Measured against live tool-servers built from both main and #610, in both the connected and the not-connected state:

main connected main dead socket #610 connected #610 not-connected
debugger-status .connected=true .connected=false .connected=true .connected=false
debugger-log-registry no .connected - no .connected, .status="connected" .status="not_connected"

.connected is absent on log-registry's success path in both builds, so assert_true '.connected' would fail a healthy run - that is why the second line uses (.status // "connected"), which is "connected" on main (field absent) and on #610's success, and "not_connected" on its failure. Both assertions therefore work on main and keep working after #610.

No new helper: assert_true / assert_field already exist and the harness already gates launch-app '.launched', restart-app '.restarted', gesture-tap '.tapped' the same way. debugger-status's .connected was the one flag of that shape left bare.

Verification

Repro is a real tool-server per build plus the mock Metro/CDP target, driving the assertion lines lifted verbatim out of the patched phase file. The unpatched assert_ok passes in both states - that is the mutation control. Timing note for anyone re-running it: the half-closed socket self-heals after ~30s when the ws close timeout fires, so steps 5 and 6 have to run back to back.

Deliberately not in this PR

The same sweep found ~20 sibling assert_ok sites whose tool reports failure inside a 200 payload - await-ui-element {success:false}, await-screen-idle {settled:false}, run-sequence {completed:0} with the error in steps[], stop-metro (whose entire body is inside a try/catch returning {stopped:false}, so assert_ok on it can never fail), chromium-cookies set {set:false}, react-profiler-status {session_status:"no_react_runtime"}, and the string-returning tools that answer "No network traffic captured..." at 200. Those verdicts are read off each tool's result contract, not driven live - confirming them needs the Android and Chromium tiers - so they are left for #595, which is already rewriting this harness with the same intent.

One adjacent bug worth recording while it is fresh: 50-rn-bluesky.sh:96 extracts RID with jq '(.requests // .entries // [])[0].requestId' from view-network-logs, which returns a formatted string, not an object. RID is therefore always empty, line 98 is dead and line 100's skip is always taken.

Merges cleanly with #595 (test-merged: Automatic merge went well; that PR rewrites the local D= line above but touches neither assertion).

…connected

`assert_ok` records PASS on the exit code alone, and both of these tools
answer 200 while reporting that the debugger is not connected — so the two
assertions that exist to check the debugger chain pass over a dead session.

`debugger-status` reaches that state on main today: half-close the CDP
socket server-side and the cached service still resolves, so the tool
returns 200 with `connected:false` and `argent run` exits 0.

    ground truth: {"connected":false}
    shipped:  ✓ debugger-status status      ✓ debugger-log-registry logs
    patched:  ✗ debugger-status [status] expected .connected to be true, got 'false'

`debugger-log-registry` has no `connected` field on its success path in
either build, so it is gated on its own `status` discriminant instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant