Skip to content

fix(systemPrompts): skip prompts with unresolved placeholders instead of emitting invalid JS#4

Open
dividedby wants to merge 3 commits into
skrabe:mainfrom
dividedby:fix/skip-unresolved-placeholder-prompts
Open

fix(systemPrompts): skip prompts with unresolved placeholders instead of emitting invalid JS#4
dividedby wants to merge 3 commits into
skrabe:mainfrom
dividedby:fix/skip-unresolved-placeholder-prompts

Conversation

@dividedby

@dividedby dividedby commented Jun 7, 2026

Copy link
Copy Markdown

Problem

Applying customizations against Claude Code 2.1.168 produces a binary that crashes on launch:

ReferenceError: PROMPT_VAR_0 is not defined
      at <anonymous> (/$bunfs/root/src/entrypoints/cli.js:463:41)

The prompt-data identifierMap human-name vocabulary changed between the extractor's 2.1.167 and 2.1.168 outputs (positional PROMPT_VAR_N → semantic names like GLOB_TOOL_NAME, OPTIONAL_TAIL_NOTE), but existing customization markdown still references the old PROMPT_VAR_N names. applyIdentifierMapping (src/systemPromptSync.ts:1438) substitutes by human-name, finds nothing to replace, and leaves the placeholder verbatim. Spliced into a backtick template literal in cli.js, the unresolved ${PLACEHOLDER} is evaluated as an undefined identifier → ReferenceError. There is no post-substitution validation, and the patch is still reported as ✓ applied.

It's not limited to PROMPT_VAR_N or to launch — a renamed semantic placeholder (OPTIONAL_TAIL_NOTE) leaks in a lazy-loaded prompt, so the binary boots but crashes when that path runs:

ReferenceError: OPTIONAL_TAIL_NOTE is not defined
      at <anonymous> (/$bunfs/root/src/entrypoints/cli.js:5156:99)   # e.g. `claude -p ...`

Evidence (vocabulary changed exactly at 2.1.168)

agent-prompt-explore identifierMap tool-description-askuserquestion
bundled data/prompts/prompts-2.1.167.json {0:PROMPT_VAR_0 … 5:PROMPT_VAR_5} {0:PROMPT_VAR_0, 1:PROMPT_VAR_1}
extracted 2.1.168 prompt data {0:GLOB_TOOL_NAME … 5:USE_EMBEDDED_TOOLS_FN} {0:ENTER_PLAN_MODE_TOOL_NAME, 1:EXIT_PLAN_MODE_TOOL_NAME}
customization markdown ${PROMPT_VAR_0..5} ${PROMPT_VAR_0..1}

Fix

Validate before splicing, in src/patches/systemPrompts.ts: once delimiter is known, when delimiter === ''(live template literal), flag anyALL_CAPS_WITH_UNDERSCOREtoken (tweakcc's human-name grammar) that survives **unchanged in both the markdown source and the interpolated output**, andcontinue— skipping the prompt (CC's original blob retained) and recording it as not-applied rather than✓`. This mirrors the existing "incomplete escaping → skip" path.

The two conditions make it precise:

  • markdown ∩ output excludes real minified vars (${HL7}), which are substituted in and never present in the markdown source.
  • backtick scope excludes inert documentation literals (${VERSION} inside a plain '...'/"..." string), which are never evaluated.

Testing

  • pnpm build + pnpm test green (269 passed / 5 skipped), no new failures.
  • Against a CC 2.1.168 native install with markdown authored on the pre-rename vocabulary: before, claude --versionReferenceError: PROMPT_VAR_0; after, the unresolvable prompts are skipped with a clear warning, all resolvable prompts still apply, and both claude --version and claude -p succeed.

Notes / possible follow-up

This heuristic intentionally misses single-word placeholders (REPO, PKG, VERSION-style names without an underscore). A stricter, fully general alternative the maintainer may prefer: validate leaked ${TOKEN} against the union of all identifierMap values across data/prompts/*.json — authoritative (no grammar guessing), catches single-word names too, and never false-positives on real minified vars. Happy to switch to that approach if preferred.

Separately, the reporting is arguably misleading for any prompt whose output retains unresolved placeholders; this PR makes such prompts report as skipped.

Related

Content-side counterpart: skrabe/lobotomized-claude-code#4 realigns the two launch-blocking overrides (explore, askuserquestion) to the 2.1.168 vocabulary so they apply again. This PR (the apply-side guard) stops the crash regardless; that PR restores the customizations.

🤖 Generated with Claude Code

… of emitting invalid JS

When the prompt-data identifierMap vocabulary changes between CC versions
(e.g. PROMPT_VAR_N -> *_TOOL_NAME at 2.1.168) but customization markdown
still references the old human-names, applyIdentifierMapping substitutes
nothing and leaves the ${PLACEHOLDER} verbatim. Spliced into a backtick
template literal in cli.js it becomes an undefined identifier and crashes
the patched binary (ReferenceError at launch, or on a lazy code path).

Guard before splicing: in a backtick context, if an ALL_CAPS_WITH_UNDERSCORE
token survives unchanged in both the markdown source and the interpolated
output, skip the prompt (keep CC's original blob) and report it skipped
rather than applied -- mirroring the existing incomplete-escaping skip path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…solved

The unresolved-placeholder guard skipped any prompt whose interpolated
output contained an ALL_CAPS_WITH_UNDERSCORE `${NAME}` token that also
appeared in the markdown. Its markdown check used
`prompt.content.includes('${'+name+'}')`, which matched even when the
token was backslash-escaped (`\${NAME}`) — intentional literal text, not
an interpolation slot. This false-positived the cowork plugin prompts
(`skill-cowork-plugin-authoring`, `data-cowork-plugin-component-schemas`)
whose `\${CLAUDE_PLUGIN_ROOT}` / `\${VAR_NAME}` env-var docs have an empty
identifierMap, needlessly skipping them on CC 2.1.168.

Add a negative lookbehind to both the output scan and the markdown check
so only genuinely unescaped `${NAME}` placeholders trip the guard. Add
regression tests for the leak (skip) and escaped (apply) cases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
dividedby added a commit to dividedby/lobotomized-claude-code that referenced this pull request Jun 7, 2026
Follow-up to the explore/askuserquestion realign, covering the prompts
that were still auto-skipped by tweakcc-fixed on 2.1.168:

- agent-prompt-memory-synthesis: OPTIONAL_TAIL_NOTE -> EMPTY_STRING
  (same trailing slot, renamed in 2.1.168 prompt data).
- tool-description-workflow: 5 clean slot renames
  (WORKFLOW_INVOCATION_QUALIFIER -> WORKFLOW_TOOL_NAME,
  WORKFLOW_RESEND_NOTE -> WORKFLOW_SCRIPT_PATH_NOTE,
  WORKFLOW_ISOLATION_TYPE -> WORKFLOW_AGENT_ISOLATION_OPTION,
  WORKFLOW_WORKTREE_NOTE -> WORKFLOW_AGENT_ISOLATION_NOTE,
  WORKFLOW_GROUP_GLYPH -> WORKFLOW_GROUP_PREFIX).
- tool-description-croncreate: structural realign of the slot-bearing
  tail to the 2.1.168 shape (CRON_DURABILITY_SECTION, the
  IS_MONITOR_TOOL_ENABLED_FN()? ternary with CRON_CREATE_TOOL_NAME /
  MONITOR_TOOL_NAME, and CRON_DURABLE_RUNTIME_NOTE), dropping the stale
  CRON_DURABLE_FLAG and the bogus CANCEL_TIMEFRAME_DAYS()? guard.
  Condensed prose preserved.

All three bumped to ccVersion 2.1.168 and validated: every unescaped
${NAME} resolves against the 2.1.168 identifierMap. Verified end-to-end
with `--apply` + `claude -p` (boots, no skips).

The two remaining cowork plugin prompts needed no content change; they
were unblocked by the tweakcc-fixed guard fix (skrabe/tweakcc-fixed#4).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@skrabe

skrabe commented Jun 8, 2026

Copy link
Copy Markdown
Owner

Thanks for chasing this down, it's a real gap. Apply shouldn't be able to write a non-booting binary and still report applied.

One change before I merge: the detector regex requires an underscore, so it skips single-word placeholders like ${VERSION} or ${REPO}. Can you swap it for the identifierMap-union check you mentioned, validating each surviving ${TOKEN} against the union of all identifierMap values across data/prompts/*.json? That picks up the single-word names and drops the grammar heuristic. Just the detector, nothing else needs touching.

…Map union

Per skrabe's pre-merge request on PR 4: replace the placeholder detector's
ALL_CAPS_WITH_UNDERSCORE grammar heuristic with validation against the union
of all identifierMap values across the leaf's bundled data/prompts/*.json.

A surviving ${TOKEN} in a live backtick template literal is treated as an
unresolved (leaked) human-name -- skip the prompt, keep CC's original blob,
report not-applied -- iff TOKEN is a member of that union. The skip/retain/
report machinery is untouched; only the detector changed.

Why the union beats the grammar:
- catches single-word names like ${VERSION}/${REPO} the underscore heuristic
  missed (no grammar guessing),
- never false-positives on real minified vars (e.g. ${HL7}) or genuine runtime
  bindings, which are never human-names and so never appear in the union.

A leaked name is stale precisely because it is absent from the CURRENT
version's identifierMap, so the detector validates against the cross-version
union -- not just the version being patched. loadIdentifierMapUnion is cached
per apply and returns empty when the bundled data/ dir is absent (npm builds
strip it via .npmignore), degrading to the pre-guard never-skip behavior.

Prepared by the tweakcc-maint control plane for
dividedby/tweakcc-maint#45

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dividedby

Copy link
Copy Markdown
Author

@skrabe — done, this implements your pre-merge ask from #4 (comment). Pushed as bc60baa on this branch.

The detector now validates a surviving unescaped ${TOKEN} (in a live backtick template literal) against the union of every identifierMap value across the bundled data/prompts/*.json, and skips the prompt iff TOKEN is a member of that union and it appears unescaped in the override markdown. The ALL_CAPS_WITH_UNDERSCORE grammar regex is gone. Only the detector changed — the skip/retain/report machinery is untouched.

  • Catches single-word names like ${VERSION}/${REPO} the underscore heuristic missed.
  • Never false-positives on real minified vars (e.g. ${HL7}) — they're never human-names, so never in the union.
  • Backslash-escaped \${NAME} and inert string-literal ${…} are still ignored.

Implementation: a cached loadIdentifierMapUnion() in systemPromptSync.ts (mirrors the existing loadShadowSet pattern, reuses findRepoPromptsDir()); returns an empty set when data/ is absent (npm builds strip it via .npmignore), degrading to the pre-guard never-skip behavior. The union spans all bundled versions on purpose: a leaked name is stale precisely because it's absent from the current version's map, so a per-version check would miss it.

pnpm build + full pnpm test (275 passed, including new red→green coverage for the single-word / minified-var / escaped cases) + lint all green locally. Boot-verify against stock CC 2.1.168 is yours to run when convenient. Thanks for the review — let me know if you'd like anything adjusted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants