feat(msq): execute system tables with Dart

[FrankChen021]

Description

System tables currently use Calcite's local Bindable path, which prevents Dart from applying its distributed query stages and native operators. This change introduces the system-table datasource and MSQ input needed to execute sys.server_properties through Dart as an initial implementation.

Execution path and query planning

Show execution details

SQL request (`engine = msq-dart`)
  -> Calcite plans `sys.server_properties` as `SystemTableDataSource`
  -> Dart controller discovers source nodes and creates input slices
  -> a worker reads its node-local provider when possible
     or issues an authenticated native Scan query to the source node's `/druid/v2` endpoint
  -> rows are exposed as a lazy `RowBasedSegment`
  -> ordinary MSQ worker stages apply filters, aggregation, windows, frames,
     shuffle, and final result processing

• Adds a generic SystemTableDataSource and a planner bridge for native-capable system-table definitions.
• Adds system-table MSQ input specs, slices, source discovery, serde, and Dart worker readers.
• Registers sys.server_properties as the only Dart-capable table in this change.
• Uses mix shuffle for unordered scans to avoid an unnecessary global sort. Queries with ORDER BY retain the standard global-sort path.
• Preserves authorization and query-timeout semantics, and cancels outstanding component requests when an input slice closes.
• Keeps unsupported engines and unsupported system tables on their existing fallback paths.
• Installs an embedded Dart worker on the Broker. It participates only when needed by a system-table query, uses the same worker-stage processing model as other Dart workers, and keeps sys.server_properties queryable when no Historical is available.

Benchmark

ServerPropertiesEngineBenchmark provides a reproducible standalone benchmark of this Dart path. For the cross-engine comparison below, the same harness was run from a combined checkout containing both this change and the native system-table implementation from apache/druid#20183. The native leg used engine=native with useNativeQueryForSystemTables=true; the Dart leg used engine=msq-dart. Both ran in the same embedded cluster over the same 100,001 generated properties, with 3 warmup iterations followed by 10 measured iterations and alternating engine order.

Workload	PR 20183 native median	Dart median	Result
COUNT(*) and COUNT(DISTINCT property) over 100,001 matching rows	193.62 ms	139.79 ms	Dart 27.8% faster
SELECT * FROM sys.server_properties returning 100,047 rows	235.98 ms	222.38 ms	Dart 5.8% faster

The master Bindable system-table path is not the native baseline: it cannot plan the COUNT(DISTINCT property) workload. The full scan includes 46 ordinary embedded-server properties in addition to the generated benchmark properties. These figures are local embedded-cluster latency measurements rather than production throughput results.

Release note

Dart can now execute SQL queries over sys.server_properties, including distributed filtering, aggregation, ordering, and window processing. Other system tables continue to use the existing Bindable execution path.

---

Key changed/added classes in this PR

• SystemTableDataSource
• SystemTableDataSourcePlanner
• DartSystemTableInputSpecSlicer
• DartSystemTableInputSliceReader
• DartEmbeddedWorkerModule

---

This PR has:

• been self-reviewed.
  • using the concurrency checklist.
• added documentation for new or modified features or behaviors.
• a release note entry in the PR description.
• added Javadocs and intent comments for non-trivial code.
• added unit tests covering the new code paths.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.