Python `re` dialect parity — FROZEN pending a user-needs signal (ADR-0035)

[okalldal]

Tracking issue for the dialect-parity freeze. Governance: ADR-0035 — Accepted (architect ratified 2026-06-30, merged in #555).

lark-rs lexes terminals with the Rust regex engine; Python Lark uses CPython re. Where the dialects assign different meaning to the same terminal syntax, we have a stream of needs-decision "support vs. categorized-refusal" forks. We are parking the whole cluster rather than resolving the forks one at a time, because:

1. No user signal — lark-rs has no users yet (ADR-0025); none of these divergences has a reported grammar that needs it.
2. Already inert — the divergences are committed XFAILs: known, pinned, non-regressing. Parking is the XFAIL bank's intended use, not a new mechanism.
3. Piecemeal fixes pre-commit an architecture — each ad-hoc "normalise construct X toward Python" is an implicit vote for maximise-compat over regex-crate semantics / selectable modes. We don't want to decide that by accident.

The shipped posture is unchanged: the \</\> normalization and every committed XFAIL stay as-is. This freezes new parity work and defers the dialect-mode decision. It reverses nothing.

Frozen set (carry the frozen label; out of the /next-task, /xfail-burndown, bug-hackathon queues)

• [medium] Python-re anchor dialect: \b/\B and \Z (RC6, N10) — support vs categorized refusal #275 — anchors \b/\B/\Z
• raw_value_len doesn't strip the global-prefix (?i)body flag form (only scoped (?i:body)) #288 — (?i) global-prefix flag normalization in raw_value_len
• [high] Char-class Python-re dialect: POSIX classes + set-operations (H5) #332 — POSIX classes + class set-operations
• [medium · needs-decision] \w/\W Unicode word-class membership diverges from Python re (H5-4) #363 — \w/\W Unicode word-class membership
• [low · needs-decision] Turkish dotless-i U+0131 not folded under /i; Python matches it (H5-7) #365 — Turkish dotless-ı under /i
• [low-medium · needs-decision] Conditional regex (?(id)yes|no) build-rejected & mis-categorized; Python builds and matches (H7-3) #415 — conditional regex (?(id)yes|no)
• [low] Full \N{NAME} named-character escape support (H5-5 follow-up to #364) #461 — \N{NAME} named-character escape
• [follow-up #399] Terminal value-length tiebreak still measures normalized length for COMPOSITE regex bodies (raw==pattern off the single-literal path) #437 — value-length tiebreak on normalized COMPOSITE regex bodies (raw_value_len). Added to the frozen set 2026-06-30 (architect confirm): same Pattern::raw_value_len() normalized-vs-verbatim root cause as the already-frozen raw_value_len doesn't strip the global-prefix (?i)body flag form (only scoped (?i:body)) #288, fires only inside the equal-priority + equal-max_width tie window (the incidental source-length sort lark-rs declares a non-goal — "a discipline for our grammars, not a gap to chase", 0/140 compliance divergences), and has no constructible repro / no user signal. Freezing keeps the whole raw_value_len-parity question parked as one unit with raw_value_len doesn't strip the global-prefix (?i)body flag form (only scoped (?i:body)) #288.

Explicitly NOT frozen

• Leak / categorization hygiene — turning an uncategorized engine error (e.g. [medium] Python-re anchor dialect: \b/\B and \Z (RC6, N10) — support vs categorized refusal #275's \b leak) or a mis-categorized refusal (e.g. [medium] Python-re anchor dialect: \b/\B and \Z (RC6, N10) — support vs categorized refusal #275's \Z) into a clean categorized GrammarError, with no accept/reject change, stays good-autonomous. The freeze is on matching Python's result, not on error quality.
• [high · needs-decision] DFA backend eagerly determinizes a counted-repeat terminal to 2^N states (H4-12) #349 — DFA 2^N determinization pathology (a resource bug, not dialect). Keeps its priority.
• [medium-high · needs-decision] PyO3 Token violates the str eq/hash invariant (set/dict membership breaks) + surface gaps (H7-4) #416 — PyO3 Token eq/hash (a binding bug, not dialect). Keeps its priority.
• [refactor] lookaround/lower.rs: unify the 3 delimited-token idiom recognizers + split into submodules #478 — lookaround/lower.rs / terminal.rs dedup may proceed on its standalone structural merit, with no behaviour change and no dialect-mode machinery.

Routing rule while frozen

New dialect divergences found by the differential fuzzer / bug-hackathon are catalogued as XFAIL and parked here, not triaged into fix-work.

Un-park trigger (ADR-0035)

The freeze ends when either:

1. a real user grammar is broken by a dialect divergence (a concrete consumer demand — the ADR-0026 bar), or
2. the architect decides the dialect-mode architecture (maximise-compat vs. regex-crate semantics vs. selectable modes) independent of a specific report.

At that point, supersede ADR-0035 with the chosen policy and burn the cluster down under it. If modes are adopted, the mode API is escalate-tier (ADR-0025 / §6).