Skip to content

perf(ci): keep JS-only products out of the backend merge-queue lanes - #76481

Merged
trunk-io[bot] merged 3 commits into
trunk-lanes-toplevel-dirsfrom
trunk-lanes-js-workspace
Aug 3, 2026
Merged

perf(ci): keep JS-only products out of the backend merge-queue lanes#76481
trunk-io[bot] merged 3 commits into
trunk-lanes-toplevel-dirsfrom
trunk-lanes-js-workspace

Conversation

@gantoine

@gantoine gantoine commented Aug 3, 2026

Copy link
Copy Markdown
Member

Problem

products/desktop is an app imported from another repository. It vendors its own pnpm workspace, so it has an apps/ + packages/ layout instead of the backend/ + frontend/ split the lane rules assume, and the product classifier keys off extension and directory:

const isBackend  = segments[2] === 'backend' || file.endsWith('.py')
const isFrontend = segments[2] === 'frontend' || /\.tsx?$/.test(file)
// neither -> assume both -> allPyProducts() for a non-isolated product

A package.json under packages/, a snapshots.yml under apps/, an icon, a font, the two pnpm files at the product root, postcss.config.mjs, the 32 vendored .py files under tools/: every one of them claimed all 82 Python lanes. That is 889 of desktop's 4,806 tracked files, 18% of the tree, each serializing against every backend PR in the queue.

Measured against the open PRs touching products/desktop when this was written, 3 of 14 were widened purely by this, with a TypeScript-only footprint otherwise:

PR Trigger file Lanes before
#76472 products/desktop/packages/agent/package.json 83
#76471 products/desktop/apps/code/snapshots.yml 83
#76416 products/desktop/packages/harness/package.json 83

Replaying the last 500 merged PRs finds two more: #76339 (a 188-file desktop resync) and #76340, both at 83 lanes for a change no Python suite runs on.

Changes

Two rules, each keyed off a declaration the repo already enforces.

1. A file inside a subtree the product's own pnpm-workspace.yaml declares as a package is frontend-only, along with the two files pnpm itself owns at the workspace root (pnpm-workspace.yaml, pnpm-lock.yaml). The declaration is the signal rather than an extension allowlist, so a path narrows only where the product itself says a JS package lives. A pnpm-workspace.yaml makes its directory a workspace root rather than a member of the repo-root one (the root file excludes products/desktop explicitly), so the lockfile beside it resolves that workspace's packages and nothing else. The repo-root pnpm-lock.yaml stays a tripwire in its own right.

2. A product with no backend surface claims its own lane instead of all of them. Two enforced declarations have to hold together:

  • pytest.ini ignores the subtree (--ignore=products/desktop), so no backend test collects a file under it. ci-backend.yml carries the same exclusion in its path filter, but a filter tuned to over-run is not a safe source for lane assignment, while an --ignore states the suite does not cover the path at all.
  • The product is absent from tach.toml, the enforced Python module graph, so no declared module may import it.

products/desktop is the only product satisfying both today. It also has no manifest.tsx, no backend/, and no entry in frontend/src/products.json, and nothing in this repo's Python imports it. Its .py files are a vendored copy of the upstream repository's own tooling.

What deliberately keeps the old widening:

  • .py inside a declared workspace package of a product that is not backend-detached. The workspace says a directory holds a JS package, not that Python cannot be checked into it.
  • products/<p>/package.json and turbo.json, which decide isolation and contract surface, so CONTRACT_DECLARATIONS has to keep seeing them.
  • Anything outside the declared globs for a product that still has a backend surface.
  • Every fallback. No pnpm-workspace.yaml, an unparseable one, an unreadable pytest.ini, or an unavailable tach graph all leave the product exactly where it was.

Note

Narrowing is the dangerous direction here: a backend lane that stops being claimed lets Trunk run two conflicting PRs in parallel. Both rules are gated on declarations that CI enforces, and pytest.ini, tach.toml, and the repo-root pnpm-lock.yaml are all tripwires, so a PR that changes what these rules read cannot itself run beside anything.

How did you test this code?

node --test .github/scripts/trunk-impacted-targets.test.js — 45 pass, 9 new.

The new cases each guard a distinct regression rather than restating the implementation: the workspace narrowing itself; the two root pnpm files, plus the boundary that a product without a declaration keeps widening (which a basename-only rule would lose); .py inside a declared package still claiming backend lanes for a product that has one (the unsafe direction); files outside the declared globs still widening; the detached product keeping its own lane while a non-detached product with the same file shapes still widens; and parser cases for both readers, since pnpm-workspace.yaml carries a catalog: block right after packages: that a looser reader would swallow, and pytest.ini spells its ignores inside one long addopts line where a line-anchored reader finds nothing.

Whole-tree differential against master. Both versions run over every tracked file under products/, target sets diffed:

files compared: 20569
unchanged     : 19680
NARROWED      :   889   (all products/desktop)
WIDENED       :     0

No product other than desktop changes by a single file, and nothing widens anywhere. Every desktop file that claimed py:core under master now claims only fe:product:desktop / py:product:desktop; the count of desktop files still reaching a backend lane is zero. #76407, which genuinely touches posthog/api/comments.py alongside desktop files, still claims all 86 lanes.

Against the last 500 merged PRs. Replaying each PR's file list through both versions, two move: #76339 from 83 lanes to 1, and #76340 from 83 to 2. No PR gains a lane.

👉 Stay up-to-date with PostHog coding conventions for a smoother review.

Automatic notifications

  • Publish to changelog?
  • Alert Sales and Marketing teams?

Docs update

Not applicable. The rationale lives in the script's own header comments, which this PR extends.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

This started as asking Claude (Claude Code) whether the existing fallback was too aggressive. It measured the open desktop PRs first, and the answer moved from "probably" to a specific number, which is what shaped the first rule. The second came from replaying the last 500 merged PRs and asking why desktop was still widening at all, which surfaced that the product has no backend surface in this repository whatsoever.

Decisions worth a reviewer's eye:

  • I considered simply adding .json/.yml/asset extensions to isFrontend. Rejected: a product could keep a Python-consumed YAML at its root outside backend/, and that fix would silently narrow it. Both rules here key off a declaration instead, so neither can reach a product that has not made the statement.
  • Detachment is deliberately two conditions rather than one. A pytest --ignore alone would narrow a product whose Python another product still imports; requiring absence from tach.toml closes that.
  • products/desktop/package.json and turbo.json still widen, because those feed CONTRACT_DECLARATIONS and untangling the two is a bigger change than this should be. They are the only desktop paths left that reach beyond the product's own lanes, and rule 2 means they no longer reach the backend ones.

Earlier revisions of this description quoted 87 products and 88 lanes. That came from a local tree where six stale directories under products/ (build artifacts left by deleted products) were read as products. The numbers here are from a clean checkout: 81 products, so 82 backend lanes.

/writing-tests and /writing-code-comments invoked before the test and comment edits.

@gantoine gantoine self-assigned this Aug 3, 2026
@trunk-io

trunk-io Bot commented Aug 3, 2026

Copy link
Copy Markdown

Merging to master in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@gantoine
gantoine marked this pull request as ready for review August 3, 2026 02:03
Copilot AI review requested due to automatic review settings August 3, 2026 02:03
@greptile-apps

greptile-apps Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "perf(ci): narrow merge-queue lanes for p..." | Re-trigger Greptile

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes Trunk merge-queue target selection for products that vendor a nested pnpm workspace (e.g. products/desktop), so changes within declared JS package subtrees no longer unnecessarily claim backend (Python) lanes.

Changes:

  • Add support for reading pnpm-workspace.yaml packages: globs per product and compiling them into a workspace matcher.
  • Update product file classification so “unclassified” files inside declared workspace packages narrow to the product’s frontend lane (while .py still claims backend lanes).
  • Extend the unit test suite with new cases covering narrowing behavior, negated globs, and parsing boundaries around catalog:.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
.github/scripts/trunk-impacted-targets.js Loads per-product pnpm workspace declarations and uses them to avoid widening backend lanes for files inside declared JS package subtrees.
.github/scripts/trunk-impacted-targets.test.js Adds focused tests to prevent regressions in workspace parsing and lane narrowing behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@gantoine gantoine changed the title perf(ci): narrow merge-queue lanes for products with nested JS workspaces perf(ci): keep JS-only products out of the backend merge-queue lanes Aug 3, 2026
@gantoine
gantoine force-pushed the trunk-lanes-js-workspace branch from bd2986f to c9a2289 Compare August 3, 2026 12:44
@gantoine
gantoine changed the base branch from master to trunk-lanes-toplevel-dirs August 3, 2026 12:46
@gantoine
gantoine requested review from a team and Copilot August 3, 2026 13:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (3)

.github/scripts/trunk-impacted-targets.js:360

  • compileWorkspaceMatcher can fail to apply common negated globs like !packages/legacy/**. Because globToRegExp anchors the pattern, packages/legacy/** won't match the ancestor dir packages/legacy (no trailing slash), so excluded package roots can still be treated as “in workspace” and incorrectly narrow targets.
    // The globs name package directories, so a file is inside the workspace
    // when one of its ancestor directories matches. Testing the file path
    // itself would miss everything below the package root.
    return (relativePath) => {
        const segments = relativePath.split('/')
        for (let depth = 1; depth < segments.length; depth++) {
            const dir = segments.slice(0, depth).join('/')
            if (include.some((re) => re.test(dir)) && !exclude.some((re) => re.test(dir))) {
                return true

.github/scripts/trunk-impacted-targets.js:427

  • parsePytestIgnores scans the entire pytest.ini text, including comment lines and non-addopts settings. A commented-out --ignore=... (or another config field containing that substring) could accidentally mark a product as backend-detached, which is the risky narrowing direction. Consider restricting parsing to the addopts value and stripping ini comments first.
// Reads the --ignore paths out of pytest's addopts. Nothing matching yields an
// empty list, which leaves every product on the old widening.
function parsePytestIgnores(text) {
    return [...text.matchAll(/--ignore[= ](\S+)/g)].map((match) => match[1].replace(/\/+$/, ''))
}

.github/scripts/trunk-impacted-targets.test.js:510

  • The negated-glob test only covers !packages/legacy but not the more common pnpm form !packages/legacy/** (which currently won’t exclude the package root unless the matcher accounts for trailing slashes). Adding an assertion for the /** form will lock in the intended behavior and prevent accidental narrowing.
test('a negated workspace glob excludes its subtree from the narrowing', () => {
    const matcher = compileWorkspaceMatcher(['packages/*', '!packages/legacy'])
    assert.equal(matcher('packages/agent/package.json'), true)
    assert.equal(matcher('packages/legacy/package.json'), false)
})

@gantoine
gantoine force-pushed the trunk-lanes-js-workspace branch from c9a2289 to eafdd59 Compare August 3, 2026 13:19
@gantoine
gantoine force-pushed the trunk-lanes-js-workspace branch from eafdd59 to cda04c1 Compare August 3, 2026 13:21
gantoine and others added 3 commits August 3, 2026 09:26
…aces

products/desktop vendors its own pnpm workspace, so its manifests, configs,
and assets are neither .py nor .tsx nor under backend/. They fell into the
"could be either domain" case and claimed every backend lane, serializing
TypeScript-only PRs against all of Python.

Files inside a subtree the product's own pnpm-workspace.yaml declares as a
package now claim only the product's frontend lane. Everything else, including
the product root manifests and any .py under a package, keeps widening.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…anes

products/desktop/pnpm-workspace.yaml and products/desktop/pnpm-lock.yaml sit
at the product root, outside every package glob the declaration lists, so the
narrowing did not reach them and each still claimed all 82 backend lanes. A
dependency bump in the vendored workspace serialized against every Python PR
in the queue.

A pnpm-workspace.yaml makes its directory a workspace root rather than a
member of the repo-root one, so the lockfile beside it resolves only that
workspace's packages. Neither file is importable from Python.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
products/desktop is an app imported from another repository. pytest.ini ignores
the subtree, ci-backend.yml excludes it from its path filter, tach.toml never
declares it, and no Python here imports it. Its vendored .py files under tools/
and every config at its root still read as backend to the layout rules and
claimed all 82 backend lanes, so either one serialized against every Python PR
in the queue.

A product that pytest ignores and tach does not declare now claims its own two
lanes instead of all of them. Both declarations are already tripwires, so a PR
that detaches a product cannot itself run beside anything.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@gantoine
gantoine force-pushed the trunk-lanes-js-workspace branch from cda04c1 to 0ff22f3 Compare August 3, 2026 13:26
Comment thread .github/scripts/trunk-impacted-targets.js
@trunk-io
trunk-io Bot merged commit 89c739d into master Aug 3, 2026
205 checks passed
@trunk-io
trunk-io Bot deleted the trunk-lanes-js-workspace branch August 3, 2026 16:10
@trunk-io

trunk-io Bot commented Aug 3, 2026

Copy link
Copy Markdown

This pull request was merged into master as part of stacked PR 76483.

@deployment-status-posthog

deployment-status-posthog Bot commented Aug 3, 2026

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-08-03 16:39 UTC Run
prod-us ✅ Deployed 2026-08-03 16:54 UTC Run
prod-eu ✅ Deployed 2026-08-03 16:57 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants