[TASK-243] Add subscribe_partition_buckets method and partitions supp…#273
Conversation
|
@luoyuxia @leekeiabstraction PTAL 🙏 |
There was a problem hiding this comment.
Pull request overview
This PR adds first-class support for subscribing to multiple (partition_id, bucket_id) pairs in one call (subscribe_partition_buckets) across the Rust core API and the C++/Python bindings, plus supporting examples and an integration test. It also fills a gap in the C++ bindings by exposing CreatePartition so partitioned-table workflows can be demonstrated end-to-end.
Changes:
- Add
subscribe_partition_bucketsto Rust scanners (record + batch) and expose it via C++/Python bindings. - Add
CreatePartitionto C++ bindings and wire it through the Rust CXX bridge. - Extend C++/Python examples and add an integration test validating batch partition subscription behavior.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| crates/fluss/tests/integration/log_table.rs | Adds integration coverage for subscribe_partition_buckets on partitioned tables |
| crates/fluss/src/client/table/scanner.rs | Implements subscribe_partition_buckets and updates partitioned-table error messaging |
| bindings/python/src/table.rs | Exposes subscribe_partition_buckets on Python LogScanner and updates to_* docs/errors |
| bindings/python/fluss/init.pyi | Adds typing stub for subscribe_partition_buckets |
| bindings/python/example/example.py | Demonstrates subscribe_partition_buckets usage for partitioned table scans |
| bindings/cpp/src/table.cpp | Adds C++ wrapper call LogScanner::SubscribePartitionBuckets |
| bindings/cpp/src/lib.rs | Extends CXX bridge with partition subscriptions + create_partition plumbing |
| bindings/cpp/src/admin.cpp | Implements Admin::CreatePartition in the C++ binding |
| bindings/cpp/include/fluss.hpp | Adds PartitionBucketSubscription struct and declares new APIs |
| bindings/cpp/examples/example.cpp | Adds a partitioned-table example using CreatePartition and batch subscriptions |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@fresh-borzoni hi, could you please rebase main branch? |
afda850 to
301c13c
Compare
|
@luoyuxia Rebased |
luoyuxia
left a comment
There was a problem hiding this comment.
@fresh-borzoni Thanks for the pr. LGTM overall, Left minor comments.
301c13c to
209bb64
Compare
209bb64 to
f04eae3
Compare
|
@luoyuxia TY for the review. |
luoyuxia
left a comment
There was a problem hiding this comment.
@fresh-borzoni Thanks for the pr. LGTM!
Summary
closes #243
subscribe_partition_bucketsmethod across Rust core, C++ and Python bindings to subscribe to multiple partition+bucket combinations at onceCreatePartitionto C++ bindings (was missing, already existed in Python)CreatePartition,SubscribePartitionandSubscribePartitionBucketsusagesubscribe_partition_bucketsexample to Python examplesubscribe_partition_buckets