Repository navigation
DDIR: write the example programs as corgi lambdas - #935
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every scalar slot in the `.ddp` programs (AoC, examples, test fixtures, server and LDBC programs) is a corgi lambda, so none uses DDIR's term language. Each computes what it did before, checked against the originals on the corgi backend. And and or of conditions are written with corgi's `all` and `any`. The `if_literal` fixture is gone, and `scalar_ops` and `sum_ops` lose the columns that tested only term-language features (a literal-condition `if` of mixed shapes, tuple `len`, `istag` on a non-sum). `binders`, `case_ops` and `tour` take sums apart in lambdas, which the vec backend cannot run, so the corgi gate checks them against recorded output (`<prog>.expected`, from the originals on vec). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
frankmcsherry
force-pushed
the
ddir-lambda-examples
branch
from
October 14, 2026 21:57
dce4c5f to
faf4217
Compare
The term-syntax originals wrapped nearly every single-field key and value in a 1-tuple, as DDIR's projections always built tuples, and the lambda rewrite kept those shapes so it could be checked row for row. Nothing in corgi or the engine needs the wrappers, so each program's own shapes are now bare values. Lambdas bind a value whole when they do not take it apart. Inputs keep their declared shapes, `count` still emits `(n)`, and the server and LDBC programs keep the shapes of their exports and imports. The AoC and fixture programs' exports lose their 1-tuples too, so the three recorded `.expected` files and the `registered` test's reading of `cells` follow. Each program computes what it did before, up to 1-tuple wrappers, checked against the previous commit on the corgi backend. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It pinned how a bare `$n` spliced a row's fields, which lambdas do not do; its programs now pass their values through unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`count` wrapped its count in a 1-tuple, `(n)`, to keep the convention that a value is a tuple so DDIR terms could read it as `$1[0]`. Lambdas read a bare count directly, so both backends and shape inference now give an `Int`. The programs that read counts take a bare `n`, and the server and LDBC programs re-wrap a count where it reaches one of their exports, so those keep their shapes. Each program computes what it did before up to 1-tuple wrappers, checked against the previous commit's programs on a server built before this change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It needed a projection that fell back to a row-wise path, which no longer exists; with the empty-batch guard in apply_ops removed it still passed, in its term and lambda forms alike. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
frankmcsherry
added a commit
to frankmcsherry/WIP
that referenced
this pull request
Oct 15, 2026
…lane, and a note for stray parentheses (#100) * corgi: append takes any number of lists `append` took a pair of lists. It now takes a tuple of one or more, as `zip` already does: `(xs, ys, zs) append` is each row's xs, then ys, then zs. The pair form is unchanged, and `(xs) append` is xs. The point is lists of computed values. `enlist` moves no data (it puts bounds of one element per row over a column), so `(a enlist, b enlist, c enlist) append` is one pass that gathers n columns into a list of n per row. Written with the pair, a left-nested chain of n appends copies each row's growing prefix at every step, about n²/2 copies per row. Lists of numbers keep the pair's direct loop for two lists. For more, a loop copies each part's run per row; when every part is one width per row (as enlisted columns are), it reads and writes no bounds per row and the result is one width per row. Lists held at different storages meet at the wider, as the pair did. Measured on 1M rows (best of 15, release): - (a, b) append, lists of 0-6 and 4 numbers: 7.3 ms before and after. - the same of pairs: 24.5 ms before and after. - three enlisted columns: ((c enlist, c enlist) append, c enlist) append takes 9.9 ms; (c enlist, c enlist, c enlist) append takes 2.8 ms. - three lists: ((a, b) append, a) append takes 15.7 ms; (a, b, a) append takes 12.3 ms. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * corgi: list literals of constants, and ascriptions `: S` that check shapes Two additions to the ML surface, asked for by DDIR, whose example programs are now written in corgi lambdas (TimelyDataflow/differential-dataflow#935). A list literal of constants, `[1, 2, 3]` or `[(0, "GET"), (1, "PUT")]`, is one constant: the parser builds the list's value and emits one `Op::Lit`, which fills it to the scope's length, exactly as a string is. `"ab"` and `[97, 98]` build the same graph. Elements are numbers, strings, `()`, and tuples and lists of these, all of one shape. Nothing else reaches that node today but a list of bytes, so this is not a second spelling: the enlist/append chain DDIR wrote builds each constant as a column and appends them row by row. A list of computed values is not a literal; it is an append of enlisted columns, and a bracket holding anything but constants is a parse error that says so. An ascription `: S` says a value has the shape S: as a stage (`xs : List(int)`), and on a pattern (`(acc : (int, int), x : int) -> ...`). `_` in a shape is a hole, and an enum names its sum, a variant declared without a payload being a hole. It is a check, made when the program is typed; it converts and steers nothing. The one place it gives a shape rather than checking one is the empty list, which has none of its own: `[] : List(int)`. Inside a literal, an empty list may instead take its shape from the elements beside it, `[[1], []]`. The core gains no op. `Pattern`, which checked the nesting of products a tuple pattern takes apart, becomes a shape with holes (`Int`, `Float`, `List`, `Sum` and `Ref` join `Tuple` and `_`), so `Op::Pattern` checks a whole shape. A pattern's own nesting and its ascriptions are one check, as a tuple pattern was. The check reads the value's structure, not its rows. A failed one reads, e.g., "in the pattern List(float), the pattern float wants a Float, not Int"; graph nodes carry no source positions, so it has no line and column, as op errors have none. DDIR's cases, before and after: - day16: 24 lines of `let l = (l, (0, 45, 0, 1, 0) enlist) append in` become one literal, `[(0, 46, 0, 1, 0), (0, 45, 0, 1, 0), ...]`. - day18: `(1099511627776, (0, 40) range) scan ((a, _) -> (a, 2) div)` becomes the 40 powers of two written out, `[549755813888, ..., 2, 1]`. - day03: `(-1, 2) range map (dr -> (dr, (-1, 2) range) cap_list) flatten` becomes `[(-1, -1), (-1, 0), (-1, 1), (0, -1), (0, 0), (0, 1), (1, -1), (1, 0), (1, 1)]`. - unnest: `((a enlist, b enlist) append, (a, b) add enlist) append` becomes `(a enlist, b enlist, (a, b) add enlist) append`, with the previous commit. Every .col program parses to the same graph as before, apart from two whose text changed here: their comments described the fold's accumulator shape in prose (naming U64, now retired), and their lambdas now say it with ascriptions. A new corpus program shows the literals. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * corgi: inject fills a lane the enum leaves open with its payload's shape `inject` built a sum only from an enum that declared every lane's payload shape. The lanes it does not fill still need shapes: they are built as empty columns. The lane it fills now need not have one: left open, it takes the payload's shape, as `branch`'s lanes take theirs from the input. Declared, it is checked as before. So one declaration serves every payload, which is what DDIR's Option and Result want: enum Option = None () | Some in (3 inject Some, "dog" inject Some) builds a {() | Int} and a {() | List<Int>}, and both match by the names `None`/`Some`. Before, DDIR wrote an unconditional Some(x) as `(x, 1) branch Option map_variant None (_ -> ())`. What it does not do: `() inject None` still needs Some's shape, since that lane is built empty and nothing in the program gives its shape. A Result generic in both lanes is the same case. `Op::Inject` carries `Vec<Option<Shape>>`: `None` only for the lane the payload fills. A host that builds the op directly (DDIR's term-language lowering does) wraps its shapes in `Some`. `unwrap(inject x) = x` now cancels only when every lane is declared alike, or there is one lane. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * corgi: a shape error on a one-field tuple of what was wanted says parentheses built it Parentheses build a tuple wherever they appear, so `(e)` written for `e` is a one-field tuple, and the error it causes names the right shapes but is easy to misread. DDIR hit it in every batch of its rewrite, most recently as `(([] : List(int)), xs) append`: Append operand: expected a list, got (List<Int>) When the operand is a one-field tuple holding exactly what was wanted, the error now ends with (a one-field tuple: parentheses build a tuple, so `(e)` is not `e`) This covers the value accessors every op reads its operand through (a pair, a list, a sum, a leaf, an Int, a mask, a Float), merges whose operands must share a shape (`min`, `select`, the comparisons), ascriptions and tuple patterns, an inject's declared lane, and a host word's input. A one-field tuple of something else, as in `(x) fold_add` on an Int, is no such slip and gets no note. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…pen lane Corgi (frankmcsherry/WIP#100) builds `Op::Inject` from optional lane shapes, so DDIR's term lowering passes its declared lanes as `Some`. `Option` and `Result` leave their payload lanes open, so a lambda builds `Some(x)` as `x inject Some`. A pattern that does not fit its input now says it "wants" a product of so many fields. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
day16's 24-row table, day18's powers of two and day03's neighbour offsets are list literals. ast, unnest and tour build their computed lists with one n-ary `append` of enlisted values, in place of nested pairs. Each computes what it did before, checked against the previous commit on the corgi backend. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Its list is one append, nothing falls back to rows, and corgi orders integers by signed value, so its inputs need no contract (it agrees across backends on negative inputs). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
It also carries a fix that missed #934: a DDIR
--comment inside a lambda now reaches corgi as a#comment, which corgi reads (before, corgi rejected it).Every scalar slot in DDIR's
.ddpprograms is now a corgi lambda: the AoC programs,examples/programs, thetests/programsfixtures, and the server and LDBC programs.No program here uses DDIR's term language (
$n,^n,tuple,list,if,fold,case, infix operators, constructors).And and or of conditions are written with corgi's
(c1, c2, ..) allandany.Each program computes what it did before, checked against the original on the corgi backend:
inspectoutput, not only the checked answer;Three fixtures lose what tested only the term language:
if_literalis gone (a literal-conditionifwhose branches differ in shape),scalar_opsdrops tuplelen, andsum_opsdropsistagon a non-sum.binders,case_opsandtourtake sums apart in lambdas, which the vec backend cannot run (it types a lambda's input from the row alone).The corgi gate checks them against recorded output instead:
<prog>.expectedbeside the program, one row per line, from the original programs on vec.The gate uses a recorded file where one exists and the vec backend otherwise.
A second commit makes each program's shapes plain.
The originals wrapped nearly every single-field key and value in a 1-tuple, because DDIR's projections always built tuples, and the first commit kept those shapes so each rewrite could be checked row for row.
Keys and values are now bare where only the program sees them, and lambdas bind a value whole when they do not take it apart.
Inputs keep their declared shapes,
countstill emits(n), and the server and LDBC programs keep the shapes of their exports and imports.Each program computes what it did before up to 1-tuple wrappers, by the same checks, and the three
.expectedfiles are regenerated after confirming they differ only in wrappers.Two more commits: the
spread_valuesfixture is gone (it pinned how a bare$nspliced fields, which lambdas do not do), andcountemits a bareIntrather than(n), in both backends and shape inference.The programs that read counts take a bare
n; the server and LDBC programs re-wrap a count where it reaches an export, adding one operator tocount_gen.These were checked against the previous commit's programs run on a server built before the change.
The
empty_batchfixture is gone: it needed a projection that fell back to a row-wise path, which no longer exists, and it passed with its guard removed.The last two commits bump corgi to frankmcsherry/WIP#100 and use it: constant lists are literals (day16's table, day18's powers of two, day03's offsets), computed lists are one n-ary
append, and a lambda buildsSome(x)asx inject Some.Not converted here: inline programs in Rust tests (mostly
tests/explain.rs), and the SNB program generatorserver/bench/ldbc/snb/rel.py.cargo test -p interactive -p ddir-serverpasses.🤖 Generated with Claude Code