feat!: relation chains through hasMany, semi-join filters, strict errors - #37
Merged
Merged
Conversation
Four paths turned an unresolvable relation reference into a column ref,
producing SQL that errors at runtime instead of being skipped:
- `resolveRelationPath` accepted a terminal segment that names a relation,
so `{ 'user.reports': ... }` became `"user"."reports"`.
- `buildPropertyExpression` fell through to an equality check when an
operator-only object produced no condition, comparing the column against
the raw object (`"user"."reports" = '{"$some":...}'`).
- A `$`-prefixed key reaching the column path was qualified as a column,
so `{ user: { $some: ... } }` became `"user"."$some"`.
- Inside a hasMany EXISTS subquery the relation handlers resolved paths
against the *outer* service's relations and FROM clause, so
`{ reports: { $some: { 'manager.name': 'x' } } }` emitted a ref to a
table the subquery never joined.
All four now drop the condition, matching the documented behaviour for
unresolvable relation paths.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A relation path may now mix belongsTo and hasMany hops in any order and to
any depth. Each belongsTo hop is a LEFT JOIN, each hasMany hop opens a
correlated EXISTS subquery, and the remainder of the path is resolved
inside that subquery against the related service's own relations:
{ assignment: { categories: { $some: { typeId: { $in: ids } } } } }
{ 'assignment.categories.type.name': 'urgent' }
Both previously produced no condition (or, before the preceding fix,
invalid SQL). Dot notation implies $some at every hasMany hop; $none and
$every stay nested-notation only.
Path resolution is now driven by an explicit scope (alias + relations of
the service owning it) instead of always reading this.options, so a
sub-filter resolves against the right service. Aliases are namespaced by
the scope they are built in, so a nested or self-referencing hasMany no
longer shadows the row source it correlates to, and the EXISTS correlation
is always qualified rather than relying on the column being declared in
`properties`.
resolveRelationPath/applyJoinsForWhere/flattenRelationQuery take a scope;
handleHasMany and handleBelongsTo are replaced by a single scope-aware
handleRelation. JSON traversal stays limited to the service's own row
source, since column types are read from its `properties`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit: |
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
feathers-kysely | 2d45b5e | Commit Preview URL Branch Preview URL |
Sep 02 2026, 08:52 AM |
A relation filter the adapter cannot resolve was dropped from the WHERE
clause. That widens the result set, which is how an authorization filter
built by a hook turns into a data leak — the same reasoning already
applied to `$or: []`. It was also inconsistent: an unknown column raises a
database error and an unknown operator a BadRequest, while an unknown
relation path returned every row.
Now rejected:
- a collection operator ($some/$none/$every) on anything but a hasMany
relation
- a path that starts at a declared relation but breaks further along
('user.bogus.name', a missing or non-Kysely target service, or a chain
that cannot be walked because `app.setup()` never ran)
- a dot path inside a hasMany sub-filter that is neither a relation of the
related service nor one of its own columns
Still quiet, because neither can be proven wrong: a dot path whose first
segment is not a declared relation (indistinguishable from a qualified
column ref or JSON access — an unknown column there still surfaces as a
database error), and an empty condition object, which is a no-op like
`$not: {}`.
BREAKING CHANGE: queries that previously returned rows with a silently
dropped relation filter now fail with a BadRequest. Chains longer than one
hop require `app.setup()` to have run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A belongsTo hop in a filter was a LEFT JOIN on the outer builder plus a
null-protect predicate. That broke in three places:
- `$not` with a relation path emitted a ref to a table that was never
joined ("no such column: user.name"): the JOIN pass descended into
`$and`/`$or` but not `$not`. Even with the JOIN registered, the
null-protect would have ended up inside the negation.
- `patch` and `remove` failed the same way on postgres and sqlite:
UPDATE/DELETE take no portable JOIN, so that branch never ran the JOIN
pass at all. (The mysql branch resolves ids with a `find` first and was
unaffected.)
- A relation declared `asArray: false` on a non-unique column multiplied
parent rows: two users came back as four, with `total` reporting four.
All three come from the same choice, so all three are fixed by the same
change: a relation filter now compiles to a correlated EXISTS at every hop
and every depth. Further belongsTo hops in one chain become INNER JOINs
inside that subquery; a hasMany hop behind a belongsTo prefix nests inside
it. A semi-join cannot duplicate rows, needs no null-protect, and stays
correct under negation and in UPDATE/DELETE.
JOINs remain for `$sort`, which needs the related value rather than its
existence. `applyJoinsForWhere` is gone; `applyJoins` now only normalizes
the query shape and joins what `$sort` requires.
Behaviour is unchanged for every query that already worked — the semantics
of `$none` behind a belongsTo (a missing parent excludes the row) are the
same, now because the outer EXISTS is false rather than because of the
null-protect.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A chain is walked through `app.service(name)`, which can fail three ways: no app was ever set up, the target service is not registered, or it is not a KyselyService. Each leaves the hop's relations unknown, so the chain is unresolvable and must be rejected rather than silently dropped. Also asserts that a single hop still works in all three cases — it only needs the querying service's own relation definition. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two guards against regressions in query building, sharing one relation graph and one table of query cases (`test/relation-graph.ts`): three belongsTo hops, two hasMany levels and a to-many behind a to-one, covering both notations, the collection operators, `$not`, `$or` and both sort paths. `test/sql-shape.test.ts` snapshots the compiled SQL and parameters for every case. Kysely compiles without touching a database, so it runs in the normal suite with no schema and no data, pinned to sqlite regardless of `DB` — the subject is our own output, not per-dialect syntax. A change to query building shows up as a readable diff instead of a timing wobble. `bench/relations.bench.ts` executes the same cases against a seeded database (~72k rows from a fixed PRNG seed, `BENCH_SCALE` to scale): pnpm bench --outputJson=bench/baseline.json # reference commit pnpm bench --compare=bench/baseline.json # after the change Every column a case filters or sorts on is indexed — without that, a chained filter degrades to a sequential scan inside a correlated subquery and the timing measures table size instead of the SQL. Two traps are documented in `bench/README.md` because both surface as `NaN` rather than as an error: vitest skips suite hooks in benchmark mode (so setup happens at module scope), and the default 500ms budget can complete zero samples when every iteration is a round-trip. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Including the quoting trap: an empty unquoted value makes loadEnvFile take the next line as the value, so the following variable is silently lost and the run connects to whatever the fallback in test/dialect.ts names. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Indexing every filtered column made the benchmark measure the best case and assume a schema few users have. An un-indexed table is exactly where the shape of the SQL decides the plan, so a regression a covering index would hide is still a regression in practice. Secondary indexes are now behind `BENCH_INDEXES=1`; primary keys remain, so a FK referencing one still gets an index lookup on the target side. The default row counts drop to ~11k, sized so that every case stays measurable un-indexed: 19 of the 20 land between 0.15ms and 1.4ms with rme under 1%, which reads more stable than the indexed run did, and the ordering now tracks hop depth instead of filter selectivity. `$sort by hasMany column` is the exception at ~220ms — a correlated aggregate subquery per row is quadratic without an index. It stays in the table because it is worth watching; `bench/README.md` notes that it completes only a dozen iterations and that scaling up grows it quadratically while the rest grow linearly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
sqlite cannot bind a JS boolean and postgres will not take 0/1 for a boolean column, so the benchmark seed failed outright on `DB=sqlite`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`asArray: false` is the caller's intent, not a guarantee — nothing stops a relation from pointing at a non-unique column. Sorting by such a hop joined it plainly, so three users came back as five rows with `total` reporting five. Filters stopped duplicating when they became semi-joins; sorting still did, because sorting needs the related value and not just its existence. A to-one hop is now joined only when the adapter can prove it matches at most one row: `keyThere` is the target service's `id`, which is how belongsTo is declared in practice and needs no promise from the caller. Anything else — and every to-many hop — takes the ordering value from a `GROUP BY` derived table keyed on the join column, which yields exactly one row per key by construction. The aggregate follows the sort direction, MIN ascending and MAX descending, the rule hasMany sorting already used. That makes it the mechanism for both sort paths, replacing the correlated aggregate the hasMany sort evaluated once per candidate row. On the benchmark's un-indexed data, `$sort` by a hasMany column goes from 221.8ms to 1.3ms (~165x); on postgres its buffer hits drop from 49,513 to 46. Every other case stays inside the measurement noise. `$sort` paths that start at a declared relation and do not resolve now raise a BadRequest instead of emitting SQL that errors: a broken chain, a path not ending on a column, or a hasMany reached through another relation (still unsupported). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Wall-clock timings in the benchmark drift a few percent between runs — the same code twice varies by up to 8% — which drowns out most of what is worth catching. Blocks touched and rows returned do not: they are a property of the plan and the data, so a 2% change in them is a real change. `test/query-costs.test.ts` compiles every case, runs it through `EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON)` and reports blocks, rows and the plan spine, optionally diffed against a recorded baseline. Postgres only (`BUFFERS` is postgres syntax) and skipped unless `BENCH_COSTS` is set, so `pnpm test` is unaffected. The `rows` column earned itself immediately: `nested hasMany two levels` was matching zero rows and therefore measuring nothing, because it filtered on one specific title in randomly seeded data. It now filters on a range. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two sort keys aggregating the same relation
(`{ 'todos.text': 1, 'todos.assigneeId': -1 }`) joined two derived tables
under one alias and the query failed as ambiguous. A regression from moving
sorting off correlated subqueries: those carried no outer alias, so two
keys on one relation used to work.
The alias is now keyed on the sort key's position among the keys that need
a derived table, not on the relation.
Also covers three capabilities the rewrite added but nothing exercised: a
chain whose first or middle hop is not provably unique (aggregated as one
derived table with the remaining hops joined inside it), sorting by a
belongsTo column of a hasMany relation, and a hasMany sort `filter` with
operators or relation paths — it goes through the normal query builder now,
where it used to be equality-only.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…skill - `test/dialect.ts` reads `POSTGRES_HOST` instead of hardcoding localhost, so the postgres run can point somewhere else. - The README keeps the `.env` example but drops the parser explanation; the quoted empty password in the example carries the point. - `.claude/skills/other-orms` records the prior-art candidates to consult before naming an operator or settling a filter semantic, and the constraints that outrank them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relation paths may mix belongsTo and hasMany hops in any order and to any depth, every relation filter compiles to a correlated
EXISTS(a semi-join), and sorting goes through an aggregate rather than a join that can duplicate rows.A belongsTo inside
$some/$none/$everyresolves in the related service's own scope, so'assignment.assignmentCategories.type.name'works too.Breaking changes
Unresolvable filters and sorts are rejected
A filter the adapter could not resolve used to be dropped from the WHERE clause. That widens the result set, which is how an authorization filter built by a hook turns into a data leak — the same reasoning already applied to
$or: []. It was also inconsistent: an unknown column raises a database error and an unknown operator aBadRequest, while an unknown relation path returned every row. Now rejected:app.setup()never ran)$sort, plus a hasMany reached through another relation (unsupported)Still quiet, because neither can be proven wrong: a dot path whose first segment is not a declared relation (indistinguishable from a qualified column ref or JSON access), and an empty condition object, which is a no-op like
$not: {}.Relation filters no longer join
A belongsTo hop used to be a LEFT JOIN plus a null-protect predicate. That broke in three places, all from the same choice:
$notwith a relation path emitted a ref to a table that was never joined — the JOIN pass descended into$and/$orbut not$not. Even with the JOIN registered, the null-protect would have ended up inside the negation.patchandremovefailed the same way on postgres and sqlite: UPDATE/DELETE take no portable JOIN, so that branch never ran the JOIN pass. (The mysql branch resolves ids with afindfirst and was unaffected.)asArray: falseon a non-unique column multiplied parent rows — two rows came back as four, withtotalreporting four.A semi-join cannot duplicate rows, needs no null-protect, and stays correct under negation and in UPDATE/DELETE.
$nonesemantics behind a belongsTo are unchanged (a missing parent excludes the row), now because the outerEXISTSis false.Sorting goes through an aggregate
Filters stopped duplicating; sorting still did, because sorting needs the related value and not just its existence.
asArray: falseis intent, not a guarantee, so a to-one hop is now joined only when the adapter can prove it matches at most one row —keyThereis the target service'sid, which is how belongsTo is declared in practice and needs no promise from the caller. Anything else, and every to-many hop, takes the ordering value from aGROUP BYderived table keyed on the join column: exactly one row per key by construction.That makes it the mechanism for both sort paths, replacing the correlated aggregate the hasMany sort evaluated once per candidate row.
Performance
Measured with the benchmark below, same harness, only
src/adapter.tsswapped. Noise floor established by running the same code twice: median 3.8%, max 8.4%.Filters (JOIN → semi-join): 16 of 19 comparable cases inside the noise floor. Two moved:
$orover two relation legs$orgets faster because theEXISTSform lets postgres decorrelate both legs into hashed subplans built once (540 → 56 buffer hits); JOIN + OR forces a nested loop re-evaluating the correlated subplan per row. The slower case readsassignmentstwice — once for the sort's join, once for the filter's subquery — where one join used to serve both.Sorting (correlated aggregate → derived table):
$sortby hasMany columnBuffer hits on postgres: 49,513 → 46. Every other case stayed inside the noise floor.
Regression guards
test/sql-shape.test.tssnapshots the compiled SQL and parameters for 20 relation cases. Kysely compiles without a database, so it runs in the normal suite with no schema and no data, pinned to sqlite regardless ofDB— the subject is our own output, not per-dialect syntax.bench/relations.bench.tsexecutes the same cases against a seeded database (~11k rows, fixed PRNG seed, secondary indexes off by default because that is where the SQL shape decides the plan):test/query-costs.test.tsreports blocks touched, rows returned and the plan spine per case fromEXPLAIN (ANALYZE, BUFFERS). Deterministic where timings are not, and the row count doubles as a correctness signal. Postgres only, opt-in viaBENCH_COSTS.Details and traps in
bench/README.md.Deliberate choices
$someat every hasMany hop.$none/$everystay nested-notation only — not expressible in a path.$some/$none/$every: column types come from the queried service's ownproperties, so a dot path in a sub-filter is a relation path, and now raises aBadRequestinstead of emitting a ref to the outer table.Testing
795 tests pass against postgres, 672 against sqlite, typecheck clean, lint warnings at baseline. New tests cover mixed chains in both notations,
$nonewith an orphan parent, belongsTo inside$some, nested hasMany alias namespacing, three chained hasMany hops,$notover relation paths,patch/removeby relation filters including a self-referencing one, non-duplication for both filters and sorts on a non-unique to-one, the aggregate direction for a non-unique sort, and every rejection case.🤖 Generated with Claude Code