diff --git a/CLAUDE.md b/CLAUDE.md index cb8f9c1..dbbc8fb 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -133,7 +133,7 @@ Worker fixture creates `RHDHDeployment(projectName)` from the Playwright project ### runOnce — Cross-Worker Deduplication -`deploy()` uses `runOnce()` internally to execute exactly once per test run, even when Playwright restarts workers after test failures. Uses file-based flags with `proper-lockfile` in `/tmp/playwright-once-{ppid}/`. +`deploy()` uses `runOnce()` internally to execute exactly once, even when Playwright restarts workers after test failures. Uses file-based flags with `proper-lockfile` in `/tmp/playwright-once-{ppid}/`. That directory is keyed on the runner PID alone, so a key is shared by every project in the run — `deploy()` is unaffected only because its key carries the namespace (`deploy-${namespace}`), and callers whose setup belongs to one project must do the same. ### Teardown Reporter diff --git a/docs/CLAUDE.md b/docs/CLAUDE.md index d1401f7..433e5a8 100644 --- a/docs/CLAUDE.md +++ b/docs/CLAUDE.md @@ -135,7 +135,7 @@ Namespace cleanup uses a custom Playwright Reporter, not `afterAll` hooks or fix 3-level cascade: package defaults -> auth-specific -> user overrides. Arrays use "replace" strategy by default. Plugin arrays use `byKey: "package"` with normalized keys (strips `-dynamic` suffix). ### deploy() Has Built-in Protection -`rhdh.deploy()` uses `runOnce()` internally — it executes exactly once per test run, even across worker restarts. No wrapping needed unless there's other expensive setup (use `test.runOnce("key", fn)` for that). +`rhdh.deploy()` uses `runOnce()` internally — its key carries the namespace (`deploy-${namespace}`), so it is already once per project, even across worker restarts. No wrapping needed unless there's other expensive setup: use `test.runOnce(`key-${rhdh.deploymentConfig.namespace}`, fn)` for that, because the flag directory is shared by every project in the run and a literal key would leave a second project with no setup. ### E2E_NIGHTLY_MODE Accepts Both Values The code checks `=== "true" || === "1"`. Document both. diff --git a/docs/api/playwright/test-fixtures.md b/docs/api/playwright/test-fixtures.md index 605d2bf..840c745 100644 --- a/docs/api/playwright/test-fixtures.md +++ b/docs/api/playwright/test-fixtures.md @@ -94,13 +94,22 @@ Executes `fn` exactly once per test run, even across worker restarts. Returns `t | Parameter | Type | Description | |-----------|------|-------------| -| `key` | `string` | Unique identifier for this operation | +| `key` | `string` | Unique identifier for this operation, across every spec file **and every project** in the run | | `fn` | `() => Promise \| void` | Function to execute once | +::: warning One key, two projects +The flag file is keyed on the key string alone, in a directory shared by every project in +the run. When one spec runs in two projects — which is what adding an `-app-next` lane +does — the first project's setup satisfies the second, and the second silently skips its +own. End the key with `${rhdh.deploymentConfig.namespace}` whenever the setup belongs to +one project, the way `deploy()` does internally. A literal key is correct only when the +setup really is shared by every project. +::: + ```typescript // Wrap pre-deploy setup that shouldn't repeat test.beforeAll(async ({ rhdh }) => { - await test.runOnce("full-setup", async () => { + await test.runOnce(`full-setup-${rhdh.deploymentConfig.namespace}`, async () => { await $`bash deploy-external-service.sh`; await rhdh.configure({ auth: "keycloak" }); await rhdh.deploy(); // safe to nest, has its own internal protection diff --git a/docs/guide/core-concepts/playwright-fixtures.md b/docs/guide/core-concepts/playwright-fixtures.md index accda7a..9b97779 100644 --- a/docs/guide/core-concepts/playwright-fixtures.md +++ b/docs/guide/core-concepts/playwright-fixtures.md @@ -179,7 +179,7 @@ While `rhdh.deploy()` has built-in protection, you may have **other expensive op ```typescript test.beforeAll(async ({ rhdh }) => { - await test.runOnce("tech-radar-setup", async () => { + await test.runOnce(`tech-radar-setup-${rhdh.deploymentConfig.namespace}`, async () => { await rhdh.configure({ auth: "keycloak" }); await $`bash ${setupScript} ${namespace}`; // expensive external service process.env.DATA_URL = await rhdh.k8sClient.getRouteLocation(namespace, "my-service"); @@ -218,7 +218,7 @@ test.beforeAll(async ({ rhdh }) => { ```typescript test.beforeAll(async ({ rhdh }) => { - await test.runOnce("tech-radar-full-setup", async () => { + await test.runOnce(`tech-radar-full-setup-${rhdh.deploymentConfig.namespace}`, async () => { await rhdh.configure({ auth: "keycloak" }); await $`bash deploy-external-service.sh ${rhdh.deploymentConfig.namespace}`; process.env.DATA_URL = await rhdh.k8sClient.getRouteLocation( @@ -251,24 +251,70 @@ test.describe("Feature B", () => { ### Key: Unique Identifier -The `key` must be globally unique across **all spec files and projects** in the same Playwright run. If two `runOnce` calls in different files use the same key, only the first one will execute. Use a prefix that includes the workspace or project name: +The `key` must be globally unique across **all spec files and projects** in the same Playwright run. If two `runOnce` calls use the same key, only the first one executes. + +Across spec files, a workspace prefix is enough: ```typescript // In tech-radar.spec.ts -await test.runOnce("tech-radar-deploy", async () => { ... }); await test.runOnce("tech-radar-data-provider", async () => { ... }); // In catalog.spec.ts -await test.runOnce("catalog-deploy", async () => { ... }); await test.runOnce("catalog-seed-data", async () => { ... }); ``` +Across **projects** it is not, and this is the half that is easy to miss. The flag +directory is keyed on the Playwright runner's PID alone: + +```ts +const flagDir = path.join(os.tmpdir(), `playwright-once-${process.ppid}`); +const flagFile = path.join(flagDir, `${key}.done`); +``` + +Nothing in it comes from the project. So when one spec runs in two projects — which is +what adding an `-app-next` lane does — the first project's setup satisfies the second, +and the second silently skips its own. For anything that deploys, that means no +deployment at all, then a failure much later on a missing element with nothing pointing +at the cause. + +**Put the namespace in the key whenever the setup belongs to one project.** This is what +`deploy()` does internally (`deploy-${namespace}`), and it is why `deploy()` was never +affected: + +```typescript +test.beforeAll(async ({ rhdh }) => { + await test.runOnce( + `tech-radar-setup-${rhdh.deploymentConfig.namespace}`, + async () => { + await rhdh.configure({ auth: "keycloak" }); + await $`bash deploy-provider.sh ${rhdh.deploymentConfig.namespace}`; + await rhdh.deploy(); + }, + ); +}); +``` + +A **literal** key is the right choice when the setup really is shared — installing an +operator into a fixed namespace that every project then uses. Both intents are real, and +the key is where you say which one you mean: + +```typescript +// Once per project: its own namespace, its own deployment. +await test.runOnce(`my-plugin-setup-${rhdh.deploymentConfig.namespace}`, ...); + +// Once per run: one operator, in a namespace that is not the project's. +await test.runOnce("my-plugin-install-operator", ...); +``` + ### Nesting -`test.runOnce` can be safely nested. Since `rhdh.deploy()` uses `runOnce` internally, wrapping it in an outer `test.runOnce` is harmless — the outer call skips everything on worker restart, and the inner one never runs: +`test.runOnce` can be safely nested. Since `rhdh.deploy()` uses `runOnce` internally, wrapping it in an outer `test.runOnce` is harmless — the outer call skips everything on worker restart, and the inner one never runs. + +Nesting does **not** rescue an unscoped outer key, though: a key shared across projects +skips before `deploy()` is ever reached, so its internal protection never gets a say. ```typescript -await test.runOnce("full-setup", async () => { +await test.runOnce(`full-setup-${rhdh.deploymentConfig.namespace}`, async () => { await $`bash setup.sh`; // protected by outer runOnce await rhdh.deploy(); // has its own internal runOnce (harmless) }); diff --git a/docs/guide/deployment/rhdh-deployment.md b/docs/guide/deployment/rhdh-deployment.md index d77514b..c824ebf 100644 --- a/docs/guide/deployment/rhdh-deployment.md +++ b/docs/guide/deployment/rhdh-deployment.md @@ -303,7 +303,7 @@ import { $ } from "@red-hat-developer-hub/e2e-test-utils/utils"; test.beforeAll(async ({ rhdh }) => { // Wrap in test.runOnce because the setup script is also expensive - await test.runOnce("my-plugin-setup", async () => { + await test.runOnce(`my-plugin-setup-${rhdh.deploymentConfig.namespace}`, async () => { const namespace = rhdh.deploymentConfig.namespace; // Configure RHDH diff --git a/docs/overlay/examples/tech-radar.md b/docs/overlay/examples/tech-radar.md index 121be5d..a18b8b8 100644 --- a/docs/overlay/examples/tech-radar.md +++ b/docs/overlay/examples/tech-radar.md @@ -222,7 +222,7 @@ test.describe("Test tech-radar plugin", () => { // Wrap in runOnce — the external service deployment is expensive // and should not re-run when Playwright restarts the worker after a test failure test.beforeAll(async ({ rhdh }) => { - await test.runOnce("tech-radar-setup", async () => { + await test.runOnce(`tech-radar-setup-${rhdh.deploymentConfig.namespace}`, async () => { const project = rhdh.deploymentConfig.namespace; // Configure RHDH with Keycloak authentication diff --git a/docs/overlay/test-structure/spec-files.md b/docs/overlay/test-structure/spec-files.md index 47df08d..18445a3 100644 --- a/docs/overlay/test-structure/spec-files.md +++ b/docs/overlay/test-structure/spec-files.md @@ -156,7 +156,7 @@ const setupScript = path.join( ); test.beforeAll(async ({ rhdh }) => { - await test.runOnce("tech-radar-setup", async () => { + await test.runOnce(`tech-radar-setup-${rhdh.deploymentConfig.namespace}`, async () => { const project = rhdh.deploymentConfig.namespace; // 1. Configure RHDH first @@ -296,7 +296,7 @@ const setupScript = path.join( test.describe("Test tech-radar plugin", () => { test.beforeAll(async ({ rhdh }) => { - await test.runOnce("tech-radar-setup", async () => { + await test.runOnce(`tech-radar-setup-${rhdh.deploymentConfig.namespace}`, async () => { const project = rhdh.deploymentConfig.namespace; await rhdh.configure({ auth: "keycloak" }); await $`bash ${setupScript} ${project}`; diff --git a/docs/overlay/tutorials/custom-deployment.md b/docs/overlay/tutorials/custom-deployment.md index cc431e4..72bb869 100644 --- a/docs/overlay/tutorials/custom-deployment.md +++ b/docs/overlay/tutorials/custom-deployment.md @@ -22,7 +22,7 @@ Deploy pre-requisites **after** `rhdh.configure()` but **before** `rhdh.deploy() ```typescript test.beforeAll(async ({ rhdh }) => { - await test.runOnce("my-plugin-setup", async () => { + await test.runOnce(`my-plugin-setup-${rhdh.deploymentConfig.namespace}`, async () => { const project = rhdh.deploymentConfig.namespace; // 1. Configure RHDH first @@ -41,7 +41,7 @@ test.beforeAll(async ({ rhdh }) => { ``` ::: tip When is `test.runOnce` needed? -`rhdh.deploy()` already skips automatically on worker restarts. But the pre-deploy steps (deploying external services, running scripts) don't have this protection. `test.runOnce` ensures the **entire setup** runs only once. The `key` must be **globally unique** across all spec files and projects in the same Playwright run — prefix it with your workspace name (e.g., `"tech-radar-setup"`). See [`test.runOnce`](/guide/core-concepts/playwright-fixtures#test-runonce-—-run-any-expensive-operation-once) for details. +`rhdh.deploy()` already skips automatically on worker restarts. But the pre-deploy steps (deploying external services, running scripts) don't have this protection. `test.runOnce` ensures the **entire setup** runs only once. The `key` must be **globally unique** across all spec files and projects in the same Playwright run: prefix it with your workspace name, and — because the setup below deploys into this project's namespace — end it with the namespace, exactly as `deploy()` does internally. Without that, a second project matching the same spec finds the flag already set and deploys nothing. See [`test.runOnce`](/guide/core-concepts/playwright-fixtures#test-runonce-—-run-any-expensive-operation-once) for details. ::: ## Examples @@ -54,7 +54,7 @@ You can deploy pre-requisites directly in TypeScript using the Kubernetes client import { test } from "@red-hat-developer-hub/e2e-test-utils/test"; test.beforeAll(async ({ rhdh }) => { - await test.runOnce("my-plugin-k8s-setup", async () => { + await test.runOnce(`my-plugin-k8s-setup-${rhdh.deploymentConfig.namespace}`, async () => { const project = rhdh.deploymentConfig.namespace; const k8s = rhdh.k8sClient; @@ -88,7 +88,7 @@ import { test } from "@red-hat-developer-hub/e2e-test-utils/test"; import { $ } from "@red-hat-developer-hub/e2e-test-utils/utils"; test.beforeAll(async ({ rhdh }) => { - await test.runOnce("my-plugin-oc-setup", async () => { + await test.runOnce(`my-plugin-oc-setup-${rhdh.deploymentConfig.namespace}`, async () => { const project = rhdh.deploymentConfig.namespace; await rhdh.configure({ auth: "keycloak" }); @@ -119,7 +119,7 @@ import path from "path"; const setupScript = path.join(import.meta.dirname, "deploy-service.sh"); test.beforeAll(async ({ rhdh }) => { - await test.runOnce("my-plugin-script-setup", async () => { + await test.runOnce(`my-plugin-script-setup-${rhdh.deploymentConfig.namespace}`, async () => { const project = rhdh.deploymentConfig.namespace; await rhdh.configure({ auth: "keycloak" }); @@ -135,7 +135,7 @@ The tech-radar plugin requires an external data provider: ```typescript test.beforeAll(async ({ rhdh }) => { - await test.runOnce("tech-radar-setup", async () => { + await test.runOnce(`tech-radar-setup-${rhdh.deploymentConfig.namespace}`, async () => { const project = rhdh.deploymentConfig.namespace; await rhdh.configure({ auth: "keycloak" });