chore: add sharding for playwright tests#7187
Conversation
📝 WalkthroughWalkthroughThis pull request introduces Nx-based Playwright test sharding with a custom plugin that generates shard-specific targets, adds environment-variable-based port key management to prevent test interference, and updates related E2E configuration and dependencies to support distributed test execution. Changes
Sequence DiagramsequenceDiagram
participant Nx as Nx Plugin
participant PM as Package.json
participant PT as Playwright Config
participant Setup as Global Setup
participant DS as Dummy Server
participant PW as Playwright
participant TD as Global Teardown
Nx->>PM: Read playwrightShards config
Nx->>Nx: Generate shard targets<br/>(test:e2e--shard-1-of-6, etc.)
loop For each shard
Note over Nx,PW: Shard Execution
PT->>PT: Determine e2ePortKey<br/>from E2E_PORT_KEY env var
PT->>PT: Delete shard-specific<br/>port-*.txt files
PT->>PT: Resolve PORT via<br/>getTestServerPort(e2ePortKey)
PT->>PT: Start webServer<br/>on resolved PORT
Setup->>DS: Start dummy server<br/>with e2ePortKey
PW->>PW: Run playwright test<br/>--shard={i}/{shardCount}<br/>--project=chromium
TD->>DS: Stop dummy server<br/>using e2ePortKey
end
Nx->>Nx: Parent test:e2e target<br/>waits for all shards
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
View your CI Pipeline Execution ↗ for commit ace374f
☁️ Nx Cloud last updated this comment at |
🚀 Changeset Version PreviewNo changeset entries found. Merging this PR will not cause a version bump for any packages. |
![nx-cloud[bot]](https://avatars.githubusercontent.com/in/80458?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
There was a problem hiding this comment.
Nx Cloud is proposing a fix for your failed CI:
We reordered the imports in hydrateStart.ts to fix the three import/order ESLint errors that were blocking CI. The #tanstack-* package imports (Node.js subpath imports) are not resolvable by the ESLint import plugin and are therefore placed in a trailing group, so all regular relative and type imports must precede them.
Warning
❌ We could not verify this fix.
diff --git a/packages/start-client-core/src/client/hydrateStart.ts b/packages/start-client-core/src/client/hydrateStart.ts
index 3b0ca44cee..d77c98c60e 100644
--- a/packages/start-client-core/src/client/hydrateStart.ts
+++ b/packages/start-client-core/src/client/hydrateStart.ts
@@ -1,14 +1,14 @@
import { hydrate } from '@tanstack/router-core/ssr/client'
+import { ServerFunctionSerializationAdapter } from './ServerFunctionSerializationAdapter'
+import type { AnyStartInstanceOptions } from '../createStart'
+import type { AnyRouter, AnySerializationAdapter } from '@tanstack/router-core'
import { startInstance } from '#tanstack-start-entry'
import {
hasPluginAdapters,
pluginSerializationAdapters,
} from '#tanstack-start-plugin-adapters'
import { getRouter } from '#tanstack-router-entry'
-import { ServerFunctionSerializationAdapter } from './ServerFunctionSerializationAdapter'
-import type { AnyStartInstanceOptions } from '../createStart'
-import type { AnyRouter, AnySerializationAdapter } from '@tanstack/router-core'
export async function hydrateStart(): Promise<AnyRouter> {
const router = await getRouter()
Or Apply changes locally with:
npx nx-cloud apply-locally 0pRD-zd9Y
Apply fix locally with your editor ↗ View interactive diff ↗
🎓 Learn more about Self-Healing CI on nx.dev
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
scripts/nx/playwright-plugin.ts (1)
51-57: Fix the tag typo before anything starts depending on it.
playwright:sharededlooks likeplaywright:sharded. If this tag gets used for filtering or lint constraints later, the misspelling will be easy to miss and hard to unwind.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@scripts/nx/playwright-plugin.ts` around lines 51 - 57, The tags array currently contains a misspelled tag 'playwright:shareded'; update that string to the intended tag 'playwright:sharded' inside the metadata -> targetGroups.playwright object so any filtering or linting later uses the correct tag value.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@nx.json`:
- Line 86: Remove the explicit "analytics": true setting from nx.json; locate
the "analytics" property (the "analytics" key at the root of nx.json) and delete
that entry so telemetry is not forcibly enabled for all developers/CI, leaving
Nx to use its default behavior or be configured separately by the team.
In `@scripts/nx/playwright-plugin.ts`:
- Around line 40-43: Sanitize the value used for E2E_PORT_KEY before it's used
to build marker filenames (e.g., the `port-${E2E_PORT_KEY}.txt` output) so
scoped package names and path-like roots don't create nested filesystem paths;
replace or normalize characters such as "/" and "@" (and any platform path
separators) with a safe delimiter (e.g., "-") or use a basename-style
extraction. Update the code paths where E2E_PORT_KEY is derived (the call site
around `buildShardedTargets(root, projectConfiguration.name ?? root,
projectConfiguration.nx.metadata.playwrightShards, ...)` and the other
occurrences around lines 76-88 that generate the marker file names) to use the
sanitized key so marker files are created as single flat filenames.
- Around line 26-45: Validate that
projectConfiguration.nx?.metadata?.playwrightShards is a positive integer before
calling buildShardedTargets: read it into a local const (e.g. const shards =
projectConfiguration.nx?.metadata?.playwrightShards), check typeof shards ===
'number' && Number.isInteger(shards) && shards > 0, and if the check fails throw
a descriptive Error (including configFilePath and the property name) or return
the non-sharded fallback; then pass the validated shards variable into
buildShardedTargets instead of using the raw property.
---
Nitpick comments:
In `@scripts/nx/playwright-plugin.ts`:
- Around line 51-57: The tags array currently contains a misspelled tag
'playwright:shareded'; update that string to the intended tag
'playwright:sharded' inside the metadata -> targetGroups.playwright object so
any filtering or linting later uses the correct tag value.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: c75440b2-4bd2-4922-adee-5b7e288a5c62
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (9)
.nx/workflows/dynamic-changesets.yamle2e/react-start/rsc/package.jsone2e/react-start/rsc/playwright.config.tse2e/react-start/rsc/tests/setup/global.setup.tse2e/react-start/rsc/tests/setup/global.teardown.tsnx.jsonpackage.jsonscripts/nx/playwright-plugin.mdscripts/nx/playwright-plugin.ts
| "plugin": "./scripts/nx/playwright-plugin.ts" | ||
| } | ||
| ], | ||
| "analytics": true |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Official Nx docs: what does setting "analytics": trueinnx.json do? Does it enable telemetry for all users and CI runs by default, and what opt-in or environment-gated alternatives does Nx recommend?
💡 Result:
Setting "analytics": true in nx.json enables Nx to collect and send anonymous usage telemetry data (command usage, environment info, etc.) to help improve the tool. It is not enabled by default for all users and CI runs. When "analytics" is not set in nx.json: - In interactive terminals, Nx prompts the user on first run to opt in/out, storing true/false in nx.json. - In CI environments, telemetry does not run (no prompt). With "analytics": true explicitly set: - Telemetry runs in interactive sessions and CI. With "analytics": false: - Telemetry is disabled everywhere. Nx recommends the prompt-based opt-in for interactive use and explicitly setting "analytics": true only if desired in CI (no environment variables or other alternatives mentioned in docs).
Citations:
- 1: https://nx.dev/docs/reference/telemetry
- 2: https://nx.dev/docs/reference/nx-json
- 3: https://github.com/nrwl/nx/blob/95621cd3/packages/nx/src/analytics/analytics.ts
Remove "analytics": true from nx.json—this is unrelated to Playwright sharding.
Explicitly setting this enables telemetry for all developers and CI runs, overriding Nx's default behavior (which disables telemetry in CI and prompts users interactively otherwise). This is a repo-wide privacy/process change that should be decided separately from this feature and requires explicit team agreement.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@nx.json` at line 86, Remove the explicit "analytics": true setting from
nx.json; locate the "analytics" property (the "analytics" key at the root of
nx.json) and delete that entry so telemetry is not forcibly enabled for all
developers/CI, leaving Nx to use its default behavior or be configured
separately by the team.
| const projectConfiguration = readJsonFile<{ | ||
| name?: string | ||
| nx?: { metadata?: { playwrightShards?: number } } | ||
| }>(configFilePath) | ||
| const root = dirname(configFilePath) | ||
|
|
||
| if (!projectConfiguration.nx?.metadata?.playwrightShards) { | ||
| return { | ||
| projects: { | ||
| [root]: {}, | ||
| }, | ||
| } | ||
| } | ||
|
|
||
| const { targets, targetGroupEntries } = buildShardedTargets( | ||
| root, | ||
| projectConfiguration.name ?? root, | ||
| projectConfiguration.nx.metadata.playwrightShards, | ||
| ['default', '^production'], | ||
| ) |
There was a problem hiding this comment.
Validate nx.metadata.playwrightShards before using it.
readJsonFile<...>() only types the JSON for TypeScript; it does not guarantee the runtime value is a positive integer. A string, decimal, or negative number here will quietly generate a broken target graph instead of failing fast.
Suggested fix
async function createNodesInternal(
configFilePath: string,
context: CreateNodesContextV2,
) {
const projectConfiguration = readJsonFile<{
name?: string
nx?: { metadata?: { playwrightShards?: number } }
}>(configFilePath)
const root = dirname(configFilePath)
+ const shardCount = projectConfiguration.nx?.metadata?.playwrightShards
- if (!projectConfiguration.nx?.metadata?.playwrightShards) {
+ if (shardCount == null) {
return {
projects: {
[root]: {},
},
}
}
+
+ if (!Number.isInteger(shardCount) || shardCount < 1) {
+ throw new Error(
+ `${configFilePath}: nx.metadata.playwrightShards must be a positive integer`,
+ )
+ }
const { targets, targetGroupEntries } = buildShardedTargets(
root,
projectConfiguration.name ?? root,
- projectConfiguration.nx.metadata.playwrightShards,
+ shardCount,
['default', '^production'],
)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@scripts/nx/playwright-plugin.ts` around lines 26 - 45, Validate that
projectConfiguration.nx?.metadata?.playwrightShards is a positive integer before
calling buildShardedTargets: read it into a local const (e.g. const shards =
projectConfiguration.nx?.metadata?.playwrightShards), check typeof shards ===
'number' && Number.isInteger(shards) && shards > 0, and if the check fails throw
a descriptive Error (including configFilePath and the property name) or return
the non-sharded fallback; then pass the validated shards variable into
buildShardedTargets instead of using the raw property.
| const { targets, targetGroupEntries } = buildShardedTargets( | ||
| root, | ||
| projectConfiguration.name ?? root, | ||
| projectConfiguration.nx.metadata.playwrightShards, |
There was a problem hiding this comment.
Make E2E_PORT_KEY filesystem-safe before deriving marker-file names.
packageJson.name can be scoped, and the fallback root definitely contains path separators. That value eventually becomes port-${E2E_PORT_KEY}.txt, so names like @scope/pkg or e2e/react-start/rsc resolve to nested paths instead of a single port marker file.
Suggested fix
function buildShardedTargets(
projectRoot: string,
packageName: string,
shardCount: number,
testInputs: TargetConfiguration['inputs'],
): {
targets: Record<string, TargetConfiguration>
targetGroupEntries: Array<string>
} {
const targets: Record<string, TargetConfiguration> = {}
const targetGroup: Array<string> = []
+ const safePackageName = packageName.replace(/[\\/]/g, '-')
// Create individual shard targets
for (let shardIndex = 1; shardIndex <= shardCount; shardIndex++) {
const shardTargetName = `${CI_TARGET_NAME}--shard-${shardIndex}-of-${shardCount}`
- const e2ePortKey = `${packageName}-shard-${shardIndex}-of-${shardCount}`
+ const e2ePortKey = `${safePackageName}-shard-${shardIndex}-of-${shardCount}`Also applies to: 76-88
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@scripts/nx/playwright-plugin.ts` around lines 40 - 43, Sanitize the value
used for E2E_PORT_KEY before it's used to build marker filenames (e.g., the
`port-${E2E_PORT_KEY}.txt` output) so scoped package names and path-like roots
don't create nested filesystem paths; replace or normalize characters such as
"/" and "@" (and any platform path separators) with a safe delimiter (e.g., "-")
or use a basename-style extraction. Update the code paths where E2E_PORT_KEY is
derived (the call site around `buildShardedTargets(root,
projectConfiguration.name ?? root,
projectConfiguration.nx.metadata.playwrightShards, ...)` and the other
occurrences around lines 76-88 that generate the marker file names) to use the
sanitized key so marker files are created as single flat filenames.
1 participant
Summary by CodeRabbit
Release Notes
New Features
Chores