Skip to content

Replace queue fanout with party-0 coordinator - #2285

Draft
philsippl wants to merge 1 commit into
mainfrom
codex/party-0-coordinator
Draft

Replace queue fanout with party-0 coordinator#2285
philsippl wants to merge 1 commit into
mainfrom
codex/party-0-coordinator

Conversation

@philsippl

Copy link
Copy Markdown
Contributor

Summary

  • replace per-party SQS request fanout and request batch synchronization with a fixed party-0 coordinator
  • add a durable PostgreSQL request ledger plus submission and result-polling API
  • distribute ordered batches over a dedicated connection built from the existing MPC TCP/mTLS stack
  • aggregate per-party parse/decryption rejection so a request rejected by any party is terminally rejected and discarded by all parties
  • adapt both GPU and Hawk/HNSW execution paths
  • retain compatibility SQS ingestion and SNS publication on party 0 only
  • remove legacy compressed SNS batch envelopes and the ingested_requests path

Why

SQS/SNS previously provided both transport and FIFO coordination. Making ordering an explicit coordinator responsibility separates coordination from execution and provides the API needed to deprecate the AWS transport later.

Rollout

The SQS/SNS resources can remain initially, but request fanout should route only to party 0 and consumers should expect one SNS result. The existing modifications and HNSW graph recovery mechanisms remain in place; coordinator-led startup reconciliation and a unified effect ledger are follow-up work.

Validation

  • cargo fmt --all --check
  • cargo check -p iris-mpc -p iris-mpc-store -p iris-mpc-bins --all-targets
  • cargo clippy -p iris-mpc -p iris-mpc-store -p iris-mpc-bins --all-targets -- -D warnings
  • cargo test -p iris-mpc -p iris-mpc-common

The PostgreSQL-dependent store tests compiled but were not executed because this workspace did not have a running PostgreSQL instance.

Before marking ready

  • update the three-party Hawk test config with coordinator outbound ports
  • update LocalStack/client compatibility tests for party-0-only input and one SNS result
  • add a three-party malformed-share test proving global terminal rejection and continued progress
  • run the external Hawk and GPU integration suites

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant