A bare delimiter pair is the empty reference in every implementation - #289
Conversation
Adding .gitkeep for PR creation (default mode). This file will be removed when the task is complete. Issue: #288
A run of delimiters that encloses nothing was read as literal text, so
`(a "" b)` decoded to the two-character text `""`, two empties in a row
merged into a space, and `("" ("" 1))` failed to parse.
An even run of delimiters now takes the n-quote reading only when its body
is substantive - at least one visible character and balanced parentheses.
Otherwise the run is a delimiter pair enclosing nothing: the empty
reference. Odd runs and every existing n-quote meaning are unchanged.
The formatter writes the empty reference as `""` instead of nothing, so a
document holding one reads back as itself.
Refs #288
Mirrors the Rust rule: an even run of delimiters takes the n-quote reading only when its body is substantive, and is otherwise a delimiter pair enclosing nothing. The formatter writes the empty reference as `""`, and the parser keeps an empty id instead of dropping it to null. Refs #288
Ports the rule already landed in Rust and JavaScript: a run of an even number of delimiters that does not enclose a substantive body is the empty reference, and every longer n-quote run keeps the meaning it already had. The formatters write the empty reference as a bare delimiter pair so that a document round trips instead of losing the reference. Adds the shared conformance suite to both implementations. Drops the two unterminated-odd-run cases from the Rust and JavaScript suites: Rust and JavaScript read `(a " b)` as three references while Python and Go read it as two, a pre-existing divergence unrelated to the empty reference.
Replaces the split N=1 / N=2 / N>=3 quoting rules with the universal procedural parser the other implementations use: a run of an even number of delimiters that does not enclose a substantive body is the empty reference, and every longer n-quote run keeps the meaning it already had. EscapeReference writes the empty reference as a bare delimiter pair so that a document round trips instead of losing the reference. Adds the shared conformance suite.
Replaces the three separate quote scanners with one universal procedural parser, matching the other implementations: a run of an even number of delimiters that does not enclose a substantive body is the empty reference, and every longer n-quote run keeps the meaning it already had. escapeReference writes the empty reference as a bare delimiter pair so that a document round trips instead of losing the reference. Adds the shared conformance suite and updates the two multi-quote tests that asserted the old reading of a bare pair as literal text.
A run of an even number of delimiters that does not enclose a substantive
body is now the empty reference, so (a "" b) holds an empty reference,
two in a row stay separate and ("" ("" 1)) parses.
The formatter writes the empty reference as "" and a whitespace-only
reference as it is, so neither disappears from the document.
Part of #288
Describe references as delimited or simple: an n-quoted reference opens with a run of N delimiters and closes with a run of exactly N, and an even run that does not enclose a substantive body is the empty reference. The three delimiters ", ' and ` are documented as equivalent. Bump the package versions so the release workflows publish the fix. Part of #288
Show the document lino-objects-codec writes for an object with empty
keys, (o: ("" (o: ("" 1)))), parsing and round tripping.
Part of #288
konard
changed the title
Working session summaryAll CI runs on HEAD Final state
What landed — the rule from the issue ("a bare delimiter pair is the empty reference, a longer run stays an n-quote delimiter") is implemented in all seven in-repo implementations (Rust, JS, Python, Go, C#, Java, PHP): for a maximal run of N delimiters with N even, the n-quote reading is taken only when the body is substantive (has a non-whitespace character and balanced parens); otherwise the run is consumed as the empty reference. Odd runs are unchanged. Formatters now write the empty reference as Two things the PR body flags for the reviewer: even-run n-quote strings with a whitespace-only or paren-unbalanced body change meaning (still writable as This summary was automatically extracted from the AI working session output. |
🤖 Solution Draft LogThis log file contains the complete execution trace of the AI solution draft process. 💰 Cost: $20.067079📊 Context and tokens usage:Claude Opus 5: (5 sub-sessions)
Total: (10.8K new + 403.5K cache writes + 21.6M cache reads) input tokens, 206.2K output tokens, $20.067079 cost 🤖 Models used:
📎 Log file uploaded as Gist (6835KB)Now working session is ended, feel free to review and add any feedback on the solution draft. |
🎉 Auto-mergedThis pull request has been automatically merged by hive-mind.
Auto-merged by hive-mind with --auto-merge flag |
1 participant
A bare delimiter pair is the empty reference
Fixes #288.
Measured against 0.14.0, an empty reference was read three different ways
depending on context. Every implementation now applies the rule the issue
proposes — give the shortest delimiter priority: a bare delimiter pair is the
empty reference, and a longer run is an n-quote delimiter as it is today.
Reproduction
Before:
(a "" b)held the two-character reference"",(a "" "" b)mergedinto a single reference holding a space, and
("" ("" 1))was a parse error.After: they hold one empty reference, two empty references, and a nested pair of
empty references.
The rule
A maximal run of N identical delimiters (
",'or`— all three behaveidentically) is read as an n-quoted reference when it opens one; otherwise, if N
is even, the whole run is one empty reference. For an even run the n-quoted
reading is only taken when the body is substantive: it holds at least one
non-whitespace character and its parentheses are balanced. Odd runs are
completely unchanged.
(a " " b)a, a reference holding one space,b(a "" b)a, the empty reference,b(a "" "" b)a, two empty references,b(a ""x"" b)a,xwritten with a 2-quote run,b(a """" b)a, one empty reference,b(x "" " "")x, a reference holding"("" ("" 1))("" 1)("" ('' 1))The formatters were fixed to match: they used to drop a reference that held
nothing or only whitespace, which silently deleted it from the document. The
empty reference is now written as
""andRef(" ")as' ', so both readback as themselves.
Landed in every supported language implementation
rust/links-notation/src/parser.rsrust/links-notation/src/lib.rstests/empty_reference_tests.rsjs/src/grammar.pegjs(+ regeneratedparser-generated.js)js/src/Link.jsjs/tests/EmptyReference.test.jspython/links_notation/parser.pypython/links_notation/link.pypython/tests/test_empty_reference.pygo/parser.gogo/lino.gogo/empty_reference_test.gocsharp/.../Parser.pegcsharp/.../Link.cscsharp/.../EmptyReferenceTests.csjava/.../Parser.javajava/.../Link.javajava/.../EmptyReferenceTest.javaphp/src/Parser.phpphp/src/Link.phpphp/tests/EmptyReferenceTest.phpEach suite asserts the same table, rendered the same way (every reference in
angle brackets, so an empty one is visible as
<>), plus a round-trip check.experiments/issue-288/holds one runnable ground-truth script per language;all seven print an identical rendering for the 24 cases from the issue.
The readers in
lino-objects-codecare a separate repository and are nottouched here.
Tests
cargo fmt --check,cargo clippy --all-targets -- -D warnings,cargo test— all greenbun run lint,bun test— 204 passblack --check,isort --check-only,flake8,pytest— 193 passed, 1 skippedgofmt -l,go vet ./...,go test ./...— greendotnet test— 196 passedmvn test(133 tests),mvn spotless:check— greenphpunit— 183 tests, 497 assertions;phpcs— greenThree pre-existing tests encoded the old behaviour and were updated to the new
rule:
MultiQuoteParserTest.testEmptyQuotes/testEmptyDoubleQuotesin Java(which asserted that
''and""are literal two-character strings) andLinkTest::testLinkEscapeReferenceKeepsZeroin PHP (which asserted that awhitespace-only reference is written as nothing).
Documented trade-off
An even run whose body is whitespace-only or contains unbalanced parentheses now
reads as empty references rather than as an n-quote delimiter, so
""a ( b""changes meaning. Such content stays expressible with a different delimiter or an
odd-length run:
'a ( b'or"""a ( b""".Also in this PR
docs/grammar/:GRAMMAR.md,links-notation.ebnf,grammar.linoandsyntax-diagrams.mdnow specify references asdelimited_reference(
n_quoted_referenceorempty_reference) orsimple_reference. N-quotingand the backtick delimiter were previously undocumented.
CHANGELOG.mdentries under## [Unreleased].0.14.0 → 0.15.0, Java0.2.0 → 0.3.0, PHP0.1.0 → 0.2.0, grammar spec0.14.0 → 0.15.0.js/examples/empty_reference.js: the documentlino-objects-codecwrites for{"": {"": 1}}, parsing and round-tripping.Out of scope, reported for the record
For an unterminated odd delimiter run the implementations already disagree,
and this PR does not change that: Rust, JavaScript and C# read
(a " b)asa,",b, while Python, Go, Java and PHP read it asa," b. The sharedconformance table only covers rows that agree everywhere.