feat(drive-abci,dashmate): seed Orchard shielded pool at SDK_TEST_DATA devnet genesis#3732
Draft
shumkov wants to merge 17 commits into
Draft
feat(drive-abci,dashmate): seed Orchard shielded pool at SDK_TEST_DATA devnet genesis#3732shumkov wants to merge 17 commits into
shumkov wants to merge 17 commits into
Conversation
…A devnet genesis
Pre-populates the shielded pool with 500_000 Orchard notes (8 owned by two
deterministic test wallets) when a local devnet binary is built with
`--cfg=create_sdk_test_data`. Closes the gap for benchmarking wallet sync at
scale without paying per-note Halo 2 proof time at chain bring-up.
Seeder (drive-abci):
- `create_data_for_shielded_pool` runs inside `create_sdk_test_data` and
emits 500k `ShieldedPoolOperationType::InsertNote` ops through the
production `commitment_tree_insert_op` path. GroveDB's
`preprocess_commitment_tree_ops` batches them into a single
Sinsemilla-frontier load / `append_with_mem_buffer` loop / Merk
propagation.
- Two-tier note generator: filler (random valid Pallas-base `cmx` + 216
bytes of opaque ciphertext) + owned (real `Note::from_parts` encrypted
to a fixed ZIP-32-derived address via `OrchardNoteEncryption<DashMemo>`).
- Genesis-time anchor recording at `block_height=1` matches production's
end-of-block-1 anchor — single recorded anchor suffices for spends via
the wallet's one-checkpoint-at-post-sync-tree-size invariant.
- Determinism: single `StdRng::seed_from_u64(0xDEAD_BEEF)` threaded
through every loop; `seed_a`/`seed_b` test wallets derived via
`SpendingKey::from_zip32_seed(seed, coin_type=1, account=0)`.
dashmate config plumbing:
- New `buildArgs: Record<string,string>` field on `dockerBuild` schema —
generic per-image build-arg map. Dashmate becomes the single source of
truth for `SDK_TEST_DATA` and `CARGO_BUILD_PROFILE`; shell-env
passthrough is dropped.
- `scripts/setup_local_network.sh` writes
`buildArgs.SDK_TEST_DATA="true"` + `CARGO_BUILD_PROFILE="release"` to
each `local_N` after `dashmate setup local`. Release profile is
mandatory — debug-Sinsemilla pushes InitChain past tenderdash's timeout.
(Marked TODO/temporary in the script — removable once Option B
precomputed-snapshot lands, or N drops low enough for debug seeding.)
- `generateEnvsFactory` flattens both `platform.drive.abci.docker.build.buildArgs`
and `platform.dapi.rsDapi.docker.build.buildArgs` into the
docker-compose env so `${KEY}` substitution in the compose `build.args`
blocks picks them up.
`dashmate config set` bug fix:
- The old `config.get(path)` pre-check rejected legal sets of new keys
inside `additionalProperties: <schema>` maps (e.g. `…buildArgs.X`).
Replaced with `Config.isSchemaPathAllowed(path)` which walks the JSON
schema descending through `properties`, `additionalProperties` value
schemas, and `$ref` references. 15 unit tests pin the walker.
Tests:
- 23 in-process tests in `rs-drive-abci`: wallet derivation, note
generator (filler + owned + ρ uniqueness + ciphertext layout +
determinism + per-wallet decrypt + cross-wallet privacy + aggregate
balance), Drive-level integration (count + anchor + cross-platform
byte-identical determinism).
- 15 dashmate unit tests for the schema walker.
- One `#[ignore]`-d functional test in `rs-platform-wallet` that drives
the full `PlatformWalletManager → bind_shielded → coordinator.sync →
shielded_balances` flow against a live SDK_TEST_DATA devnet.
Cost (release profile, 500_000 notes, Apple Silicon Docker Desktop):
~3h 41m wall-clock for the seeder. CPU work is ~95s; the rest is
GroveDB writes through the macOS Docker VM. See
`docs/shielded-seeder-performance.md` for the breakdown and the
Option-B follow-up.
Refs #3714.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…t InitChain
Cuts SDK_TEST_DATA shielded-pool seeding from ~3h41m (Docker on macOS at
N=500k) or ~65 min (native release at N=500k) to ~134 ms at InitChain by
moving the seed work to a one-shot bake during docker image build and
loading the result via `IngestExternalFile` + parent-Merk leaf patch.
End-to-end proven on local devnet: bake-inside-Dockerfile produces a
snapshot file with `combined_root=b682f38442c8...8144e3c3` byte-for-byte
identical to a local macOS bake; InitChain applies it in 134 ms; wallet-A
sync against the snapshot-loaded chain recovers the expected 400 000
balance (4 owned × 100 000) — proving proof verification works against
snapshot-loaded state.
Components
----------
- `packages/rs-drive-abci/src/shielded_snapshot/mod.rs` (NEW):
- `dump_shielded_subtree(grove, w)` — writes one SST per CF + header +
blake3 checksum.
- `apply_shielded_snapshot(grove, r, txn)` — validates header,
`ingest_subtree_sst`, cross-validates `combined_root` against the
reconstructed CommitmentTree, patches parent Merk leaf via
`replace_commitment_tree_subtree_root`. No fallback — any mismatch
is FATAL so the operator notices.
- `drive-abci snapshot-bake --out <path>` subcommand:
- Self-contained: opens fresh tempdir, runs `create_genesis_state`
(which under `cfg(create_sdk_test_data)` seeds the shielded pool),
then dumps the resulting subtree. Uses a NoopCoreRPC stub because
genesis doesn't talk to Core.
- `DRIVE_SHIELDED_SNAPSHOT` env var read in `create_data_for_shielded_pool`:
takes the snapshot fast-path when set, runs the runtime seeder
otherwise. Failure during apply is fatal (no silent fallback).
- Dockerfile: new `bake-shielded-snapshot` stage runs `drive-abci
snapshot-bake` against an in-container tempdir when `SDK_TEST_DATA=true`,
embeds the snapshot at `/opt/dashmate/snapshots/shielded-pool.snap` in
the runtime image, sets `ENV DRIVE_SHIELDED_SNAPSHOT=...`.
- `docs/genesis-snapshot-design.md` — design doc covering format,
cross-validation, threat model, compatibility policy.
Side-effect changes
-------------------
- `ShieldedSeedConfig::sdk_test_data().total_notes`: 500_000 → 5_000 for
fast iteration while the snapshot path is fresh. Bump back when needed.
- `scripts/setup_local_network.sh`: `CARGO_BUILD_PROFILE` set to `dev`
for fast docker rebuilds during snapshot dev. Flip to `release` when
going back to large N for stress testing.
- grovedb dep rev bumped to `04f2d4243872b65fbec33650e15d85571df385e1`
(branch `feat/snapshot-apply-public-api` — DON'T MERGE; adds three
public methods: `ingest_subtree_sst`, `replace_commitment_tree_subtree_root`,
`raw_storage`).
- New deps in `rs-drive-abci`: `rocksdb` (SstFileWriter), `blake3`
(snapshot checksum). Plus `grovedb-path` + `grovedb-storage` as dev-deps
for the data-location test.
Tests
-----
- `dump_only_default_and_aux_cfs_under_shielded_subtree_prefix` — pins
the CF layout the dump enumerates (default CF only, contrary to design's
original §3 guess of default + aux).
- `snapshot_dump_apply_preserves_anchor` — in-process roundtrip; A seeds,
dumps, B applies, asserts anchors match byte-for-byte.
- `bench_native_seed_full` — bake-feasibility benchmark; measures
N=500k seed in release locally.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
shumkov
changed the title
…ake at N=1M Builds on the Phase 1 snapshot bake + apply design with the crypto-level optimization from grovedb PR #751: skip the Sinsemilla frontier append for filler notes (which no wallet owns and nobody needs spend proofs for), keeping the full Sinsemilla path only for the 8 owned notes. Drops bake time at N=1M from a projected 8+ hours (Phase 1, hit during the prior overnight run) to ~12 minutes in-process on native macOS release. Roundtrip test passes: anchor matches byte-for-byte between the bake-side seed and the snapshot apply on a fresh DB. Changes ------- - Cherry-picked grovedb PR #751 onto our existing `feat/snapshot-apply-public-api` branch — exposes `append_raw_without_frontier` / `append_many_without_frontier` on `CommitmentTree` under the `test-seeding-ct` Cargo feature, plus an MMR-root cache that makes append O(1) instead of O(N). grovedb rev bumped: 04f2d424 → 60d121900. - `rs-drive-abci/Cargo.toml`: enable `test-seeding-ct` feature on the grovedb-commitment-tree dep. - `OwnedLayout::compute`: moved owned positions to the **tail** `[N-owned_count, N)` instead of striped. Forced by the frontier-less constraint (PR #751 hard-rejects a non-empty frontier in the no-frontier append path) — must bulk-seed ALL filler first while frontier is empty, then append owned through the regular path. - `seed_shielded_pool_with_config`: replaced the `apply_drive_operations(InsertNote × N)` per-note path with: 1. Open CommitmentTree directly via grovedb's `raw_storage().get_transactional_storage_context(...)`. 2. Bulk-seed filler via `append_many_without_frontier(iter)` — blake3 only, no Pallas math. 3. Per-note `append_raw` + `save` + `commit_mmr` for the 8 owned (mirrors grovedb's existing commitment_tree_insert pattern). 4. Commit subtree's StorageBatch through the transaction. 5. Patch parent Merk leaf via `replace_commitment_tree_subtree_root` using the combined_root from the final `append_raw`'s result (compute_current_state_root hit "Inconsistent store" after intermediate commit_mmr flushes; reading the result-returned roots avoids that). 6. Post-bake assert `count == cfg.total_notes` — catches silent truncation from a panic mid-bake. - Progress logging: emit `seed phase A progress` every 30s with appended/total/pct/elapsed/rate/ETA from inside the bulk iterator. Previously the seeder was silent between start and end, making N=1M bakes invisible. - `docs/genesis-snapshot-design.md` §15 expanded with the F1 constraint, the bake/apply asymmetry, anchor non-spendability caveat, F6 rejection-sampling-skip caveat, and updated test plan. §8 scoped to Phase 1; §10 test #6 (equivalence with runtime seeder) marked retired under Phase 2. Consequences ------------ - Anchor recorded at height 1 reflects only the 8 owned cmx at frontier-positions 0..7 (Sinsemilla frontier has its own counter, independent of BulkAppendTree's total_count). Wallets attempting to construct spend proofs would fail. Devnet-only; gated by cfg(create_sdk_test_data) + SDK_TEST_DATA=true at image build. - combined_root produced by a Phase-2 bake ≠ combined_root a runtime-seeded chain would produce. Cross-validation between bake and apply still works (both read the same stored frontier). - Wallet sync still recovers the 8 owned notes correctly because sync uses chunk proofs authenticated by bulk_state_root (transitively authenticated by the grovedb root), not by the Sinsemilla anchor. Performance ----------- N=1M in-process roundtrip on native macOS release: - seed (filler bulk + owned full): ~10 min - dump (267 MB SST): ~30 s - apply on fresh DB: ~1 min - total: 12 min wall-clock Anchors match byte-for-byte: d1f7ed699e0b0a7741ca91dbe7513abf7c5a53d418206ba7bde3b0a3dd974631 Phase 1 at N=1M (didn't complete after 8+ hours in docker on macOS) projected to ~2-3 hours native release. Phase 2 is roughly 40× faster end-to-end at this N. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…API change bind_shielded now takes &[u8] instead of [u8; 32] after v3.1-dev merge (part of IdentityManager refactor in PR #3651). One-character fix. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The original Regtest-only check made sense for local dashmate setups but blocked the intended use case from issue #3714: stress-testing wallet sync on real Dev networks with N=1M pre-seeded shielded notes via the snapshot image. Real Devnet chains run with Network::Devnet, not Network::Regtest. Mainnet + Testnet still rejected — they must never carry SDK test fixtures (random identities + seeded shielded pool + the junk Sinsemilla anchor that isn't a valid spend anchor). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds the iOS-side measurement surface for shielded sync against a pre-populated note pool (paloma devnet / local snapshot image): - ShieldedService publishes lastSyncDuration + a 1Hz currentSyncElapsed ticker driven by the false→true edge of shieldedSyncIsSyncing. CoreContentView renders both in the ZK Shielded Sync Status section, with monospaced digits so the live ticker doesn't reflow. - New FFI platform_wallet_manager_bind_shielded_with_raw_seed accepts a raw 32-byte ZIP-32 seed, bypassing the MnemonicResolver. Required to bind the chain-side test wallet A (`[0x73; 32]`) that the snapshot bake seeds — no BIP-39 mnemonic can produce that seed. Swift wrapper bindShieldedRawSeed mirrors the existing bindShielded shape. - New orange "Bind Test Wallet A (Shielded)" debug button under the same Sync Status section hardcodes the seed and starts the sync loop. All raw-seed / button code is tagged `TODO(shielded-snapshot-devnet-test)` and is meant to be removed once SwiftExampleApp has a real test-wallet import flow (tracked: #3714). Spec: packages/swift-sdk/SwiftExampleApp/docs/shielded-sync-timing-spec.md Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…DAPI + quorum URLs Devnet had no end-to-end Platform path on the iOS app — `SDK.init` ignored the `platformDAPIAddresses` UserDefaults on non-regtest networks, and even when addresses were provided, the trusted context provider panicked on `Devnet` without an explicit quorum-list URL (no built-in default exists; `quorums.devnet.<name>.networks.dash.org` is templated on a devnet name we don't carry across FFI). Result: switching to Devnet always showed Disconnected. This wires it end-to-end: - DashSDKConfig gains a `quorum_url: *const c_char` field. When set, `dash_sdk_create_trusted` constructs the TrustedHttpContextProvider via `new_with_url(...)` regardless of network — overrides the hardcoded network default so devnet (and non-default mainnet/testnet shards) work. - SDK.init now reads `platformDAPIAddresses` and a new `platformQuorumURL` UserDefaults key and forwards both to the FFI on devnet + regtest unconditionally, and on mainnet/testnet when `useDockerSetup` is set. Helper `withOptionalCStrings` keeps the two-string lifetime contract clean. - OptionsView gets a dedicated devnet branch with three text fields (SPV Peers, DAPI URL, Quorum URL); no toggle since all three are always custom on devnet. The picker's onChange force-enables `useLocalhostCore` and seeds the SPV peers default for devnet so the Sync tab's `startSpv()` path picks up peers without a hidden toggle. - WalletManagerStore.activate compared cached managers by network only, returning the cached one even when AppState handed in a freshly-built SDK. That kept the manager pointing at the old SDK clone (with stale DAPI / quorum endpoints) so proof verification failed forever with "no available addresses to use". Now we cache the configured SDK handle and rebuild the manager when it doesn't match — the FFI `configure` is single-shot (precondition !isConfigured) so we can't swap in place. - SDKLogger.log mirrors to NSLog in addition to stdout so `xcrun simctl spawn booted log stream` and Console.app capture the timing log lines even when no Xcode debugger is attached. Token FFI test fixtures get the new `quorum_url: ptr::null()` field to keep compiling. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…stics
Adds the diagnostic surface for the iOS balance-of-zero investigation
(P0.2) and the followup punch list that came out of the live test
session.
- `tests/shielded_sync_paloma.rs` — new integration test that binds
wallet A (`SEED_A = [0x73; 32]`) against a remote SDK_TEST_DATA
devnet via TrustedHttpContextProvider, fans the gRPC traffic across
all 13 paloma masternodes' Platform gateways (`AddressList` picks
randomly per request), runs one sync pass, asserts balance =
EXPECTED_BALANCE_A = 400_000. Verified PASS against paloma in
1299s: total_scanned=1_000_000, new_notes_total=4, balances={0: 400000}
— confirms decryption + persistence work end-to-end at the Rust
layer, isolates iOS balance=0 to iOS-specific persister callback /
display path.
- `PlatformWalletPersistenceHandler.swift` — temporary NSLog
instrumentation on `persistShieldedNotes` and
`persistShieldedSyncedIndices` so the next iOS sync run surfaces
whether the callbacks fire (via `simctl spawn booted log stream`).
Both tagged `TODO(shielded-snapshot-devnet-test)` for removal once
P0.2 closes. Uses the safer `NSLog("%@", message)` pattern after
the multi-arg variadic form SIGBUSed (verified, May 2026).
- `docs/shielded-sync-devnet-followups.md` — punch list captured from
the live test session (P0.1 cold-sync duration preservation, P0.2
balance investigation, P1.1 auto-discover DAPI from /masternodes,
P1.2 per-chunk progress, P1.3 node-count indicator). Status table
+ prioritized order of attack.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…-out
Two iOS UX improvements that came out of the live paloma devnet test
session — both Swift-only.
**P0.1 — Preserve longest-pass duration**
The existing "Last sync duration" row gets clobbered every time a sync
completes. After a 20-minute cold sync of paloma's 1M-note pool, a
subsequent 3-second steady-state pass would overwrite the headline
number, leaving no way to recover what the cold sync actually took.
ShieldedService now publishes `longestSyncDuration` alongside
`lastSyncDuration`. Set to `max(prev, elapsed)` at every completion;
reset on bind / reset / clearLocalState (the four sites the existing
`lastSyncDuration = nil` pattern already covers).
CoreContentView renders a "Longest pass: N.NN s" row beneath "Last
sync duration", but only when meaningfully longer than the most recent
pass (`longest > last + 0.05s`) so steady-state runs don't render two
identical lines.
**P1.1 — Auto-populate DAPI list from `/masternodes`**
Previously, devnet usage required pasting all DAPI URLs by hand. With
only one URL in the field the SDK funnels every gRPC request through
one masternode (`AddressList::pick_address` picks randomly per request);
on a 1M-note sync that becomes the bottleneck.
`SDK.discoverDAPIAddresses(quorumBase:)` synchronously hits
`{quorumURL}/masternodes`, filters `status == "ENABLED"`, and builds
a comma-separated `https://<ip>:<platformHTTPPort>,…` list. 5-second
URLSession timeout via DispatchSemaphore (init can't be async without
a deeper refactor; the call runs off the main thread inside
`AppState.switchNetwork`'s Task).
SDK init triggers discovery when:
- the network uses overrides (devnet, regtest, or `useDockerSetup`),
- a quorum URL is set,
- and `platformDAPIAddresses` is empty (manual entry wins).
Resolved list is written back to `platformDAPIAddresses` UserDefaults
so OptionsView displays the actual fanned-out URLs. Subsequent SDK
builds skip the network round-trip; clearing the field re-triggers
discovery on the next switch.
OptionsView shows a small "N nodes" label above the DAPI URL field —
"auto (from /masternodes)" when empty, "13 nodes" when populated.
TextField switches to vertical-axis with `lineLimit(1...4)` so the
13-URL list is readable.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Surface live cumulative-notes-scanned during a long shielded sync pass so the iOS UI can render a ticking counter / `ProgressView` instead of staring at a spinner for the 20+ min cold sync of a 1M-note pool. Previously `sync_shielded_notes` was a single async call that returned only at completion — between start and end there was no signal anything was happening. Touches every layer: - **rs-sdk** (`platform/shielded/notes_sync/types.rs`, `sync_shielded_notes.rs`): new `ProgressCallback` type (`Arc<dyn Fn(u64, u64) + Send + Sync>`). Added as an optional field on `ShieldedSyncConfig`; fired once per chunk inside the sliding-window fetch loop with `(cumulative_scanned, latest_block_height)`. Default `None` preserves prior behavior. - **rs-platform-wallet** (`events.rs`, `wallet/shielded/coordinator.rs`, `wallet/shielded/sync.rs`, `manager/mod.rs`): new trait method `PlatformEventHandler::on_shielded_sync_progress`. Coordinator holds an optional progress handler in a `Mutex<Option<…>>` (ArcSwap can't store `dyn Fn` — needs `T: Sized`); installed by the manager in `configure_shielded` with a closure that forwards into the event manager. `sync_notes_across` reads the handler, wraps it in a `ShieldedSyncConfig.on_chunk_completed`, and passes to the SDK. Network-scoped event (no wallet_id) — a single coordinator pass covers every bound IVK. - **rs-platform-wallet-ffi** (`event_handler.rs`): new `on_shielded_sync_progress_fn` slot on `EventHandlerCallbacks` (C-ABI-stable regardless of `shielded` feature). FFIEventHandler dispatches when shielded compiled in. - **swift-sdk** (`PlatformWalletManagerAddressSync.swift`, `PlatformWalletManagerShieldedSync.swift`, `PlatformWalletManager.swift`): wire new `shieldedSyncProgressCallback` C trampoline. PlatformWalletManager publishes `currentShieldedSyncScanned` and `currentShieldedSyncBlockHeight` (cleared on every completion). - **SwiftExampleApp** (`ShieldedService.swift`, `CoreContentView.swift`): ShieldedService republishes via `combineLatest` of the two manager publishers into `currentSyncScanned` / `currentSyncBlockHeight`. CoreContentView's "Syncing… elapsed" row gets a "Scanned this pass: N notes" sub-row + a linear `ProgressView` (indeterminate — chain commitment count isn't separately queried; absolute number is more useful than a fake bar). Both reset across all four bind/clear paths. - **WalletManagerStore.swift**: drive-by fix for the stale-SDK rebuild path — `activeManager` is non-optional so `= nil` failed to compile; let the post-rebuild `activeManager = manager` line overwrite cleanly. Release-mode paloma test (1M-note cold sync) baseline established at 1022 s wall clock; per-chunk callback fires ~once per 2048 notes inside the SDK fetch phase (~6 min in release). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
**P0.2 — wallet A balance=0 after sync, root-caused and fixed**
Rebinding shielded keys on a wallet that already has a persisted
watermark left the next sync starting at near-tree-tip, so the new
IVK never visited the positions where its owned notes lived.
`bind_shielded` calls `unregister_wallet` → `register_wallet` →
`restore_for_wallet`, and that last step rehydrates the watermark
from disk. For the mnemonic-bind path that's correct (same IVK
across restarts). For the raw-seed test path (different IVK) it
silently breaks scanning.
End-to-end validation against paloma (release-Rust integration test
+ debug iOS sim):
Rust test: total_scanned=1_000_000, decrypted_for_driver=4,
balances={0: 400_000}, 1022 s
iOS app: Scanned 1,000,000, New 4, Spent 0,
balance=0.000004 DASH (= 400_000 credits), 539.80 s
Fix: `NetworkShieldedCoordinator::force_rescan_subwallets(wallet_id,
accounts)` zeros the in-memory watermark for the bound subwallets.
Called from `platform_wallet_manager_bind_shielded_with_raw_seed`
right after `wallet.bind_shielded()` returns.
**Test-only scaffolding — strong DELETE-BEFORE-MERGE banner**
Every site that exists only to support the devnet sync-timing test
now carries an explicit `⚠️ TEST-ONLY CODE — DELETE BEFORE MERGE ⚠️ `
header with the full 5-site cleanup checklist:
- `platform_wallet_manager_bind_shielded_with_raw_seed` FFI entry
- `NetworkShieldedCoordinator::force_rescan_subwallets` helper
- Swift `PlatformWalletManager.bindShieldedRawSeed` wrapper
- `ShieldedService.bindWithRawSeed`
- "Bind Test Wallet A (Shielded)" button HStack in CoreContentView
- ATS exception in Info.plist (auto-cleaned with the rest)
Tag: TODO(shielded-snapshot-devnet-test). Tracked: #3714.
**Devnet UX simplification**
Reduced the devnet user-input surface to a single field — the
quorum list service URL — by deriving everything else from
`{quorum}/masternodes`. New shared helper
`SDK.discoverActiveMasternodes` returns each ENABLED masternode's
SPV peer (`ip:CoreP2PPort` from the `address` field) and DAPI URL
(`https://ip:platformHTTPPort`). Both are fetched fresh on every
SDK init / SPV start — self-healing on node churn.
OptionsView devnet branch: drops the manual SPV Peers TextField
and DAPI URL TextField; only the Quorum URL field remains.
SDK.init: always auto-discovers DAPI from /masternodes on devnet,
no longer writes back to UserDefaults.
`CoreContentView.spvPeerOverride`: devnet branch fetches the same
endpoint and extracts each masternode's `address` field verbatim
(paloma reports `ip:20001`, not the canonical 29999 — using
masternode-reported port is correct).
**Bundled Info.plist + ATS exception**
iOS's App Transport Security blocks plain-HTTP requests by default,
which prevented the SDK init from reaching the HTTP-only paloma
quorum-list-server. Switched the SwiftExampleApp project from
auto-generated to a real `Info.plist` (at project root so it stays
out of `PBXFileSystemSynchronizedRootGroup`'s auto-resource
inclusion) with `NSAllowsArbitraryLoads = true`. Plist carries the
same bundle/version/scene-manifest keys Xcode would have synthesized
plus the ATS dict. Test-only, must be removed before any production
release.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…t_data # Conflicts: # Cargo.lock
…t_data # Conflicts: # packages/rs-sdk-ffi/src/types.rs
shumkov
changed the title
Deletes the entire raw-seed bind scaffolding now that P0.2 is resolved and the iOS devnet sync-timing test has been validated end-to-end (0.000004 DASH on paloma matching the chain-side bake spec). Removed sites (the 5 banner-flagged deletions plus the P0.2 diagnostic NSLogs): - `platform_wallet_manager_bind_shielded_with_raw_seed` (rs-platform-wallet-ffi) - `NetworkShieldedCoordinator::force_rescan_subwallets` (rs-platform-wallet) - `PlatformWalletManager.bindShieldedRawSeed` (swift-sdk) - `ShieldedService.bindWithRawSeed` (SwiftExampleApp) - "Bind Test Wallet A (Shielded)" orange button HStack (CoreContentView) - Diagnostic NSLogs on `persistShieldedNotes` and `persistShieldedSyncedIndices` (PlatformWalletPersistenceHandler) What's kept (broader devnet path, not wallet-A-specific): - `SDK.discoverActiveMasternodes` + auto-discovery in `init` - Info.plist + ATS exception (needed for plain-HTTP /masternodes) - Devnet OptionsView simplification (single Quorum URL input) - All P0.1 / P1.1 / P1.2 timing + progress UI - `rs-platform-wallet/tests/shielded_sync_paloma.rs` — the canonical Rust integration test that proves the fix; keeps SEED_A by design Verified: Rust workspace + iOS framework + SwiftExampleApp all build clean. The non-wallet-A devnet flow (mnemonic bind on devnet against the auto-discovered DAPI nodes) is unaffected. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds two `#[ignore]`-gated benchmarks that drove the diagnostic work behind the multi-chunk query change (#3756). `tests/shielded_chunk_timing_bench.rs` — issues N sequential single- chunk `ShieldedEncryptedNotes` fetches against a live SDK_TEST_DATA devnet (paloma by default) and reports per-chunk wall-clock distribution plus the per-chunk net/verify split (from `tracing::info!` inside `fetch_with_metadata_and_proof`). The flat per-chunk cost across count=64 vs count=2048 (1.57s vs 1.52s avg) is what proved the bottleneck is fixed per-request overhead, not bandwidth — directly justifying batching multiple chunks into one proof. `tests/shielded_decrypt_bench.rs` — measures trial-decrypt throughput over generated `ShieldedEncryptedNote` fixtures, single-threaded vs rayon-parallel. Showed decrypt is <1% of cold-sync wall-clock (~1.3s for 1M notes single-threaded on M-series), ruling out decrypt as the target for the multi-chunk PR. Run: cargo test -p platform-wallet --release --features shielded \ --test shielded_chunk_timing_bench -- --ignored --nocapture cargo test -p platform-wallet --release --features shielded \ --test shielded_decrypt_bench -- --ignored --nocapture New dev-deps to keep the benches self-contained: - drive-proof-verifier (for the wire types) - rayon (for parallel decrypt comparison) `rs-sdk-trusted-context-provider` was already a dev-dep for the existing paloma sync test. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Imports re-sorted (std first, then external, then internal — alphabetized within each group) and over-wrapped function signatures collapsed back to one line per `rustfmt`'s defaults. Pure formatting; no behaviour change. Surfaced when running cargo fmt while staging the chunk-timing / decrypt benchmark commit — a handful of files in rs-drive-abci, rs-platform-wallet, and rs-platform-wallet-ffi had drifted out of fmt-clean state. Folding the cleanup into its own commit keeps the bench commit minimal and gives the next person clean diffs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the gap from #3714 for benchmarking wallet sync at scale. When a local devnet's drive-abci is built with
--cfg=create_sdk_test_data,InitChainnow pre-populates the Orchard shielded pool with 500_000 notes (8 owned across two deterministic ZIP-32 test wallets) before block 1 is proposed. No per-note Halo 2 proving at bring-up; deterministic root hash for byte-identical re-runs.Adopts Option A from the design audit: seeder runs through the production
commitment_tree_insert_oppath, batched into a singleapply_drive_operationscall so GroveDB'spreprocess_commitment_tree_opscollapses 500k inserts into one Sinsemilla-frontier load +append_with_mem_bufferloop + one Merk propagation.What's in scope
Drive-abci seeder
create_data_for_shielded_poolinsidecreate_sdk_test_data(regtest-only runtime check kept).cmx, opaque 216-byte ciphertext) + owned (realNote::from_partsencrypted to a fixed wallet viaOrchardNoteEncryption<DashMemo>).block_height = 1(matches production's first end-of-block anchor).StdRngthreaded through every randomness call (seed_a = [0x73;32],seed_b = [0x74;32], derived viaSpendingKey::from_zip32_seed(seed, coin_type=1, account=0)).orchard+zip32deps onrs-drive-abci(gated for use only inside the cfg).dashmate generic
buildArgsconfigdockerBuildschema:buildArgs: Record<string,string>. Replaces ad-hoc shell-env passthrough as the single source of truth for image build args.scripts/setup_local_network.shwritesbuildArgs.SDK_TEST_DATA = "true"+buildArgs.CARGO_BUILD_PROFILE = "release"to eachlocal_Nafterdashmate setup local. Marked TODO/temporary — release profile is mandatory while the seeder is slow.generateEnvsFactoryforwards both drive-abci and rs-dapi build-args into compose's env so${KEY}substitution inbuild.args:resolves.dashmate config setbug fixconfig.get(path)rejected legal sets of new keys underadditionalProperties: <schema>maps. Replaced withConfig.isSchemaPathAllowed(path)— a schema walker that descendsproperties,additionalPropertiesvalue schemas, and$ref. 15 unit tests pin it.What's NOT in scope (follow-ups)
docs/shielded-seeder-performance.mdfor the breakdown and removal-conditions of the TODO.Drive::validate_anchor_existsvia constructed but unsubmitted spend) — design doc §9 acceptance, deemed tautological given current sync test (we drop it here, can revisit).SEED_A/SEED_B— currently duplicated between drive-abci side and the platform-wallet functional test with// keep in synccomment.Test plan
cargo test -p drive-abci --lib 'create_genesis_state::test::shielded'withRUSTFLAGS='--cfg create_sdk_test_data --cfg tokio_unstable'): 23/23 passing — wallet derivation, generator (filler + owned + ρ uniqueness + ciphertext + determinism + per-wallet decrypt + cross-wallet privacy + aggregate balance), Drive integration (count + anchor + cross-platform byte-identical determinism).yarn mocha test/unit/config/Config.spec.js): 15/15 passing.PlatformWalletManager → bind_shielded → coordinator.sync → shielded_balancesrecovers400_000per wallet against an SDK_TEST_DATA-seeded chain (earlier run at N=500). Functional test ships inpackages/rs-platform-wallet/tests/shielded_sync.rsas#[ignore]-d; run withcargo test -p platform-wallet --test shielded_sync --features shielded -- --ignored --nocaptureafteryarn startbrings a seeded devnet up.🤖 Generated with Claude Code