Skip to content

v1 getDocuments: tracker for wire-only / not_yet_implemented surfaces from #3654 #3655

@QuantumExplorer

Description

@QuantumExplorer

Status

  • (2) SUM and (3) AVG: ✅ landed in feat(drive): document sum + average proof primitives, with SDK fan-out scaffolding and reproducible benchmarks #3661 (drive sum/avg dispatchers wired end-to-end on no-proof and proof paths; SDK DocumentSum / DocumentAverage + DocumentSplitSums / DocumentSplitAverages FromProof impls; ABCI routing through SelectFunction::Sum / ::Avg; wasm-sdk bindings getDocumentsSum / getDocumentsAverage + their WithProofInfo variants). The polymorphic AggregateValue response variant didn't end up needed — SUM and AVG ride dedicated data.sums / data.averages response slots alongside data.counts.
  • Remaining items (1, 4–14) still wire-only.

Background

PR #3654 lands the v1 getDocuments SQL-shaped wire surface (SELECT / WHERE / GROUP BY / HAVING / ORDER BY / LIMIT / OFFSET), but ships several pieces as wire-only: the proto carries them so callers can encode the shape and the future server capability can land without another wire-format change, but the server rejects them at routing time with QuerySyntaxError::Unsupported("… is not yet implemented").

This tracker collects every not_yet_implemented introduced by that PR so each can be picked up as a focused follow-up.

Inventory of not_yet_implemented rejections (rs-drive-abci v1 handler)

# Reject site Notes
1 HAVING clause is not yet implemented Non-empty having rejected wholesale. Decoded Vec<HavingClause> already lands in having_clauses_from_proto (currently #[allow(dead_code)]); execution is "flip the gate + thread into the dispatchers."
2 SELECT SUM is not yet implementedDone in #3661. Drive drive_document_sum_query dispatcher routes (mode × range × group_by × prove) to dedicated executors (point_lookup, total, range_no_proof, range_proof, range_distinct_proof, range_aggregate_carrier_proof, per_in_value). Response shape: data.sums.aggregate_sum (single) / data.sums.entries (per-group), proven via grovedb's verify_aggregate_sum_query.
3 SELECT AVG is not yet implementedDone in #3661. AVG composes count + sum walks under the hood (no separate grovedb primitive); the dispatcher resolves each AVG shape to its matching SUM mode and the client folds sum / count. Response shape: data.averages.aggregate_average / data.averages.entries. Single-aggregate fold uses checked_add on both u64 (count) and i64 (sum) axes; per-group flat-map fold likewise.
4 SELECT MIN is not yet implemented Per-group/global MIN executor needed (semantically distinct from HavingRanking::Min which is cross-group). Waits on a grovedb min/max primitive — the order-by-then-LIMIT-1 emulation has the wrong proof shape for the cryptographic verifier.
5 SELECT MAX is not yet implemented Symmetric to MIN. Same grovedb-primitive dependency.
6 SELECT COUNT(field) is not yet implemented Counting non-null values of a specific field; today only COUNT(*) (empty field) is evaluated. Needs a non-null counter walk that doesn't exist in grovedb.
7 GROUP BY with SELECT DOCUMENTS is not yet implemented Documents-fetch path doesn't accept grouping.
8 GROUP BY on field '<x>' which is not constrained by an In or range where clause is not yet implemented Single-field GROUP BY on a field not present in WHERE.
9 two-field GROUP BY outside the (In, range) compound shape is not yet implemented Only the canonical compound (in_field, range_field) ordering is wired; other orderings need a new merk walk.
10 GROUP BY with more than two fields is not yet implemented group_by.len() > 2. Probably permanent in current architecture; flag for confirmation.
11 OFFSET pagination is not yet implemented Wire-reserved optional uint32 offset = 12;. Cursor pagination via start_after / start_at is the supported path today.
12 multi-projection SELECT is not yet implemented Wire: repeated Select selects. Today selects.len() > 1 rejected. Response shape needs a parallel repeated AggregateValue values field when this lands.
13 start_after / start_at with SELECT COUNT is not yet implemented Paginating count results — workaround today is narrowing the range clause.
14 ORDER BY on aggregate keys is not yet implemented Wire: OrderClause.target.aggregate. Drive's OrderClause only carries a plain field: String; extension needs a new OrderTarget enum at the drive level. Overlaps HavingRanking::Top/Bottom for top-N selection but more general.

Dependencies between items

  • (2) (3) (4) (5) all need a shared numeric-aggregate executor in drive + a polymorphic AggregateValue response variant. (2) and (3) landed in feat(drive): document sum + average proof primitives, with SDK fan-out scaffolding and reproducible benchmarks #3661 via dedicated data.sums / data.averages response slots — the polymorphic AggregateValue variant turned out unnecessary for the SUM/AVG-only case. (4) MIN and (5) MAX still wait on a grovedb min/max primitive (separate from the SUM/AVG plumbing).
  • (12) depends on (4) / (5) for the MIN/MAX projections to be combinable; SUM and AVG could already participate in a multi-projection response if the response shape were extended. Practical implementation likely waits for MIN/MAX so the full surface lands at once.
  • (14) depends on extending drive's OrderClause shape; once done, HavingRanking::Top/Bottom could potentially be consolidated through this path. The SUM/AVG ranking infrastructure (per-group entries with (in_key, key, sum) / (in_key, key, count, sum) triples) is now in place; ordering on those metrics is the remaining piece.
  • (1) is independent — the wire decoder is already in conversions.rs::having_clauses_from_proto; only the gate-flip + dispatcher threading is needed. Now that SUM/AVG executors exist, HAVING aggregates over them (HAVING SUM(x) > 100) have something to evaluate against.
  • (11) is independent — drive already accepts offset on SizedQuery; routing-layer wire-through is the main work.

Suggested first slices

  1. (11) OFFSET — smallest contained piece. Probably one PR.
  2. (1) HAVING — decoder already in tree; executor design is the main work. With SUM/AVG landed, HAVING aggregates over them have concrete evaluation targets. One PR.
  3. (4) (5) MIN / MAX — waits on a grovedb primitive (or a documented decision that the order-by + LIMIT 1 emulation isn't usable for proofs). Out-of-band design work before code.
  4. (14) ORDER BY aggregate — could land independently of MIN/MAX now that SUM/AVG per-group entries exist; reuses the ranking shape already in DocumentSplitSums / DocumentSplitAverages. One PR.
  5. (12) Multi-projection SELECT — depends on (4)/(5) to be useful. One PR once those land.
  6. (7) (8) (9) (10) (13) — GROUP BY / pagination-on-count gaps. Individual PRs as needed.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions