Skip to content

fix(e2e): lock the shared GitHub session, and stop a lane silently booting legacy - #152

Open
gustavolira wants to merge 7 commits into
mainfrom
fix/nfs-lane-trust
Open

fix(e2e): lock the shared GitHub session, and stop a lane silently booting legacy#152
gustavolira wants to merge 7 commits into
mainfrom
fix/nfs-lane-trust

Conversation

@gustavolira

@gustavolira gustavolira commented Aug 21, 2026

Copy link
Copy Markdown
Member

Three defects found while triaging the overlay NFS migration.

Closes RHIDP-16457, RHIDP-16459. Partially addresses RHIDP-16458.

Re-ordered after checking which of these are live on main today rather than insurance. The session-file race is happening now; the NFS guard is a post-condition that nothing trips yet but that matters more as workspaces lose their legacy lane.

The GitHub session file has no lock, and github races it today — RHIDP-16459

const sessionFileName = `authState_${userid}.json`;   // relative

Relative resolves against process.cwd(), which the worker fixture sets to the workspace's e2e-tests directory — the same value for every project in it.

On main right now, the github workspace has two projects, github-actions and github-issues, and both specs call loginAsGithubUser. Workers default to 50% of cores, so they run concurrently against one unlocked file. Two outcomes: a reader lands mid-write and dies on truncated JSON, or both decide there is no session and both log in — with the same TOTP secret, inside the same 30-second window.

That second one is not speculation; this file already carries the retry for it:

"The two-factor code you entered has already been used"
"too many codes have been submitted"

Still one file per user, deliberately not per project. Scoping per project was the first attempt and would have made it worse: every lane logging in concurrently is exactly the TOTP collision. What was missing is a lock spanning read-or-login-and-write — two lanes that both find no session go on to log in anyway — plus an atomic temp-file write (removed even on failure) and an unreadable session falling through to a full login instead of throwing.

RHDH cookies from another lane were never the hazard: each lane's RHDH lives on its own namespace hostname.

A lane can boot legacy and pass — RHIDP-16457

_applySecrets merges defaults → auth → new-frontend-system → tests/config/rhdh-secrets.yaml. The workspace file is last, so a workspace setting APP_CONFIG_app_packageName or ENABLE_STANDARD_MODULE_FEDERATION overrides the NFS layer and the lane boots the legacy shell — passing, because the legacy suite re-runs and nothing errors.

No workspace does that today, so the throw is a post-condition rather than a live catch. What makes it worth having now is the shape the migration has taken: app-defaults, keycloak and, once #3325 and #3343 land, argocd and roadie have no non-app-next project left. When a legacy lane sat beside the NFS one, a silently-broken NFS lane still had a sibling proving the plugin works. With the lane replaced rather than added, a silent fallback means that workspace's entire coverage is fake and there is no control.

The same reason is why deploy() now logs which mechanism chose the shell:

[nfs] argocd-app-next: new frontend system ON, from the -app-next project name
[nfs] github: new frontend system ON, from configure({ useNewFrontendSystem: true })

Logged from deploy(), not configure() — the worker fixture calls configure() with no arguments for every project before any spec runs, so for github and homepage the first line would have read off, from nothing, and the first line is the one people grep.

The guard also rewrites an unquoted YAML true to "true": coercing only in the comparison green-lit a payload the API server rejects with cannot unmarshal bool into Go struct field.

SidebarTabs — RHIDP-16458, partially

The union hardcoded "Self-service", the legacy shell's scaffolder title; under app-next the page is "Create". Both are members now. Nothing references "Create" yet — this is enabling work for RHIDP-16462, where three workarounds are in flight. Deliberately only widening the type: picking one of them here would be a fourth.

Verification

yarn check   typecheck + eslint (0 errors) + prettier clean
yarn test    141 passed — five consecutive runs, no flake
docs build   clean

Mutation-tested; each turns exactly the matching tests red: guard never reporting a dropped marker (4), intent conflict ignoring the explicit-false guard (1), lock removed (1), temp-file cleanup removed (1), boolean not rewritten in place (1).

Left open

The guard covers the secrets layer only. config/new-frontend-system/dynamic-plugins.yaml also injects plugin-app-auth and plugin-app-integrations, and a workspace can disable either from its own dynamic-plugins.yaml, which merges last — degrading the lane the same silent way. Extending the check across a second merge with different semantics is more than this PR should carry.

Version

Takes 2.1.11. #151 merged and claimed 2.1.10, which is published — latest is 2.1.10.

🤖 Generated with Claude Code

…f pass

Three defects found while triaging the overlay migration, all with the same
shape: the harness cannot tell a working NFS lane from one that quietly fell
back to the legacy shell.

RHIDP-16457 — a workspace could disable the NFS it asked for. The NFS secret
layer merges before the workspace's own tests/config/rhdh-secrets.yaml, so a
workspace setting APP_CONFIG_app_packageName or ENABLE_STANDARD_MODULE_FEDERATION
for its own reasons overrode it. Nothing then failed: the legacy suite re-ran
and passed, and the only thing the lane existed to prove was never exercised.
deploy() now checks the merged secret still carries both markers and throws
naming the key, the value found, and the file responsible. The merge order is
untouched — a workspace has to be able to override defaults, which is exactly
why the outcome needs checking rather than assuming.

Also from RHIDP-16457, and from RHIDP-16461: configure() now logs which of the
three mechanisms decided the shell, because none of them is visible from a
single file and the answer currently requires reading a project name, an
environment variable and a configure() call together. A namespace ending in
-app-next that was explicitly configured with useNewFrontendSystem: false warns,
since it deploys legacy under a name that reads as NFS everywhere. Only that
direction is checked: enabling NFS without the suffix is legitimate and is what
github and homepage do.

RHIDP-16459 — the GitHub session file was a bare relative authState_<user>.json,
resolved against process.cwd(), which the worker fixture sets to the same
workspace directory for every project. A workspace's lanes shared one file per
user, unlocked: one lane could inject another's storage state, and a reader
landing mid-write died on truncated JSON as a flake that looked nothing like the
plugin under test. Every lane added is another writer. The path is now absolute
and keyed by project as well as user, the write goes through a temp file and a
rename, and an unusable session falls through to a full login rather than
throwing.

RHIDP-16458 — SidebarTabs hardcoded "Self-service", which is the legacy shell's
scaffolder title; under app-next the page is "Create", so a spec running there
could not type-check without a cast. Both are members now. Deliberately only
that: which label a lane should use is RHIDP-16462, three workarounds are in
flight for it, and picking one here would be a fourth.

All four new behaviours are mutation-tested — reverting each turns the matching
tests red and nothing else. 136 tests pass; typecheck, eslint and prettier are
clean; the docs build validates its own links.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review found two of the three changes wrong in ways the tests did not reach.

Keying the GitHub session file per project removed cross-lane sharing, which is
what the ticket asked for — and would have made every lane of a workspace log in
to the same GitHub account concurrently. logintoGithub derives its 2FA code from
one shared TOTP secret, so lanes starting inside the same 30-second window submit
the identical code and GitHub rejects the second. This file already carries retry
handling for exactly that error, which is how it is known to happen; bulk-import
would have gone from one login to three simultaneous ones.

So the file is one per user again, and what was actually missing is added
instead: a lock spanning read-or-login-and-write, not just the write, because two
lanes that both decide there is no session go on to log in anyway. RHDH cookies
from another lane were never the hazard — each lane's RHDH lives on its own
namespace hostname.

The [nfs] line was logged from configure(), but the worker fixture calls
configure() with no arguments for every project before any spec runs. For github
and homepage — the two lanes the change was written for, which opt in through
configure({ useNewFrontendSystem: true }) — the first line said "off, from
nothing", and the first line is the one a reader greps. Reported from deploy()
now, once, after configuration is final.

Two smaller ones. The guard accepted an unquoted YAML `true` by coercing in the
comparison only, so it green-lit a payload the API server rejects with "cannot
unmarshal bool into Go struct field ... of type string" — an opaque k8s failure
in place of the guard's own message. It now rewrites the value as well. And the
temp file survived a failed storageState write, accumulating untracked
authState*.tmp files in a directory nothing gitignores them from.

New tests for the lock, both write outcomes, and the boolean rewrite; the
serialisation test waits for the first holder rather than racing two callers
from the same tick, which was flaky. Mutation-tested: removing the lock, the
temp-file cleanup, or the in-place coercion each turns exactly one test red.
141 pass, five consecutive runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@gustavolira

Copy link
Copy Markdown
Member Author

Review pass found two of the three changes wrong in ways the tests did not reach. Both fixed.

The session fix would have caused the failure it prevents

Keying the file per project removed cross-lane sharing, which is what the ticket asked for — and would have made every lane of a workspace log in to the same GitHub account concurrently. logintoGithub derives its 2FA code from one shared TOTP secret, so lanes starting inside the same 30-second window submit the identical code and GitHub rejects the second.

That this happens is not speculation — this file already carries the retry for it:

"The two-factor code you entered has already been used"
"too many codes have been submitted"

bulk-import would have gone from one login to three simultaneous ones, with a single retry to clear it.

So the file is one per user again, and what was actually missing is added instead: a lock spanning read-or-login-and-write, not just the write — two lanes that both decide there is no session go on to log in anyway. RHDH cookies from another lane were never the hazard: each lane's RHDH lives on its own namespace hostname, so they are never sent anywhere they matter.

The [nfs] line said the wrong thing for the lanes it was written for

It logged from configure(). The worker fixture calls configure() with no arguments for every project before any spec runs, so for github and homepage — which opt in via configure({ useNewFrontendSystem: true }) in beforeAll — the output was:

[nfs] github: new frontend system off, from nothing — the default is the legacy shell
[nfs] github: new frontend system ON, from configure({ useNewFrontendSystem: true })

The first line is the one a reader greps. Reported from deploy() now — once, after configuration is final.

Two smaller ones

The guard accepted an unquoted YAML true by coercing in the comparison only, so it green-lit a payload the API server rejects with cannot unmarshal bool into Go struct field ... of type string — an opaque k8s failure in place of the guard's own message. It rewrites the value now, so the tolerance is real.

The temp file survived a failed storageState write, accumulating untracked authState*.tmp in a directory nothing gitignores them from.

Verification

yarn check   typecheck + eslint (0 errors) + prettier clean
yarn test    141 passed — five consecutive runs, no flake
docs build   clean
Mutation Tests red
lock removed 1
temp-file cleanup removed 1
boolean not rewritten in place 1

The serialisation test now waits for the first holder rather than racing two callers from the same tick — the original raced the scheduler and flaked once in five runs.

Left open, deliberately

The guard covers the secrets layer only. config/new-frontend-system/dynamic-plugins.yaml also injects plugin-app-auth and plugin-app-integrations, and a workspace can disable either from its own dynamic-plugins.yaml, which merges last — degrading the lane in the same silent way. Worth a follow-up; extending the marker check across a second merge with different semantics is more than this PR should carry.

gustavolira and others added 2 commits August 24, 2026 09:53
#149 was reduced to a documentation change, so it no longer bumps the version and
2.1.10 is free. The note about the two PRs colliding goes with it — there is only
one version-bumping PR open now.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
#151 merged today and claimed 2.1.10, so the renumber in the previous commit
collided with main and the Version Bump Check would have refused it. Same number
as before, different reason: this is no longer about #149, which now bumps
nothing at all.

Worth noting separately: main says 2.1.10 but the publish workflow for it failed,
so npm latest is still 2.1.9. If that is not resolved before this merges, 2.1.11
publishes over a 2.1.10 that never existed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@gustavolira gustavolira changed the title fix(nfs): make a lane that is not actually running NFS fail instead of pass fix(e2e): lock the shared GitHub session, and stop a lane silently booting legacy Aug 24, 2026
@gustavolira gustavolira reopened this Aug 24, 2026
The PR was DIRTY, which is why GitHub had stopped running checks on it — a pull
request that cannot be merged gets no merge commit to test, so the last two
pushes produced zero check runs and looked like CI hanging.

Both conflicts were version bookkeeping from #151: package.json keeps 2.1.11,
and its 2.1.10 section moves below ours in the changelog rather than competing
for "Current". #151 also touched deployment.ts, which this branch changes, but
git merged those cleanly and the suite is green.

141 tests pass, yarn check clean, docs build clean.
It does not any more, and the guide said it in as many words: "your
rhdh-secrets.yaml still wins on conflicts". True for every key except the two
NFS markers, where overriding now throws instead of quietly deploying the legacy
shell — so the sentence this PR contradicts is corrected in the same change
rather than left to be discovered from a stack trace.

Also documented, because a new hard failure with no docs costs someone an hour:
the `[nfs]` line and what to grep for when a lane behaves like the wrong shell,
why it comes from deploy() rather than configure(), the exact throw and the fact
that it is a post-condition on the caller's own config, and the -app-next
namespace warning together with why the reverse is not checked.

The dynamic-plugins layer is named as the unguarded one. Disabling app-auth or
app-integrations from a workspace's own dynamic-plugins.yaml degrades the lane
the same silent way, and nothing catches it. No workspace does that today — 14
ship their own dynamic-plugins.yaml and none names either package — so a second
guard would be insurance against something nobody does, which is the argument
that just closed rhdh-plugin-export-overlays#3375. Documented instead, and it is
the first place to look when an NFS lane behaves like the legacy shell.

deploy()'s step list gains both the report and the throw.

141 tests pass, yarn check clean, docs build clean.
#153 landed and claimed 2.1.11, which this branch was holding. Third version
collision on this PR — the Version Bump Check only compares against main, so it
cannot see a claim another open PR is already sitting on, and the branch loses
the race every time something else merges first.

Nothing but bookkeeping: the changelog keeps our section on top with #153's
2.1.11 below it, and package.json moves to 2.1.12.

141 tests pass, yarn check clean, docs build clean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant