Skip to content

refactor(gossip): make gossip into an actor - #833

Open
erhant wants to merge 10 commits into
devfrom
erhant/gossip-as-actor
Open

refactor(gossip): make gossip into an actor#833
erhant wants to merge 10 commits into
devfrom
erhant/gossip-as-actor

Conversation

@erhant

@erhant erhant commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

🎯 Purpose

Turns the sequencer's p2p gossip module into a kameo actor of its own, resolving the // TODO: Should be a separate actor seam in the service's run() (suggested by @Arjentix alongside the actor-encapsulation work).
It now lives as sequencer_gossip_actor under lez/sequencer/actors/, following the same crate layout and actor feature convention as the executor, storage, and RPC server actors (#798).

⚙️ Approach

The former GossipNetwork drive task is the actor now: GossipActor owns the libp2p swarm and drives it from an Actor::next() override (mirroring RpcServerActor::next), selecting over the mailbox, swarm events, and the bootstrap-retry tick. The mailbox replaces the hand-rolled plumbing wholesale:

  • Move the whole sequencer_core::gossip module to a new sequencer_gossip_actor crate at lez/sequencer/actors/gossip; rewrite network.rs as actor.rs.
  • Replace the spawned drive task + CancellationToken + publish mpsc channel + watch channel with the actor mailbox: PublishTransaction (tell), GetConnectedPeers (ask), and RetryBootstrap (tell, sent by the service scheduler every 30s, same shape as ProduceBlock). The rpc_server publishes through a type-erased Recipient<PublishTransaction> with tell(..).try_send(), keeping the old non-blocking, drop-on-overflow publish semantics (bounded mailbox, same 1024 depth).
  • Implement on_stop: the swarm is Option-held and dropped there, so the QUIC sockets are freed the moment the actor reports stopped rather than when the struct is dropped — a shutdown observer can rebind immediately.
  • Strong types per review: peer keys carried as Ed25519PublicKey (map + GetConnectedPeers reply, sorted deterministically via to_bytes), and the pending-publish queue is a new common::bounded_vec_deque::BoundedVecDeque whose eviction is an invariant of the type.
  • Preserve the degrade-to-L1 invariant: the gossip ActorHandle is deliberately excluded from SequencerHandle::failed()/is_healthy() (documented on the field). The old driver watchdog becomes spawn_gossip_outage_watchdog: it warns every 5 minutes only on an abnormal actor stop (panic), stays silent on graceful stop/kill, and its guard aborts the warner on node shutdown.
  • Keep startup fail-fast: GossipActor::new binds and waits for the listen address before the actor is spawned; bootstrap addrs are captured pre-spawn so SequencerHandle::gossip_bootstrap_addrs() stays sync for the multi-node fixtures. Graceful gossip shutdown added to SequencerHandle::shutdown().
  • Adopt the refactor(sequencer, storage): move rocksdb communication to StorageActor #798 actor feature convention: without it the crate is protocol-only (protocol messages + accreditation; validation and seen_cache are private actor implementation details); actor pulls in kameo/libp2p/tokio/borsh and the implementation, with tests declared inside the actor module like the other actor crates. Only the service enables it — rpc_server consumes the crate featureless, exactly as it consumes the executor. The mdns feature moves here from sequencer_core (service forwards it; it implies actor).
  • Delete the dead gossip/message.rs wire-format file — it was never declared as a module (uncompiled), and sequencer_stake_core::SlashApproval is the real slash-approval type.

GossipConfig stays in sequencer_core::config (it is embedded in SequencerConfig), which is the one reason core keeps its libp2p dependency. The ingest path is unchanged: gossiped transactions still enter through the executor's admission door via the IngestSubmit hook wired in the service.

🧪 How to Test

# check
cargo check -p sequencer_gossip_actor

# test
RISC0_DEV_MODE=1 cargo test -p sequencer_gossip_actor --all-features

# run a sequencer
just run-sequencer

🔗 Dependencies

None

🔜 Future Work

  • A mockable publish path if the rpc_server's publish path ever needs testing without a swarm — it holds a type-erased Recipient<PublishTransaction>, so a test can hand it a recipient of any lightweight actor.
  • Kameo link/supervision instead of the warn-only outage watchdog — needs defined restart semantics (rebind + publisher re-wiring); on_stop already frees the sockets so a supervised restart is possible when we get there.
  • Actor supervision/respawn instead of warn-and-restart on a gossip panic; needs re-bind handling and publisher re-wiring, deferred until an actual gossip crash is observed in the wild.
  • A gossip-health surface (metric or getter): is_healthy() intentionally stays true when gossip is down, so the 5-minute log is currently the only operator signal.

📋 PR Completion Checklist

  • Complete PR description
  • Implement the core functionality
  • Add/update tests
  • Add/update documentation and inline comments

@erhant erhant self-assigned this Sep 4, 2026
@erhant
erhant force-pushed the erhant/gossip-as-actor branch from ee237c6 to 0ce4cae Compare September 7, 2026 09:00
@erhant
erhant requested a lite review from Copilot September 7, 2026 11:43

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The connected-peers API currently returns nondeterministically ordered results due to HashSet iteration, which should be stabilized before merging.

Pull request overview

This PR refactors the sequencer’s libp2p gossip implementation into a dedicated kameo actor crate (sequencer_gossip_actor), wiring it into the service and RPC server while preserving the “gossip outage degrades to L1-only” invariant.

Changes:

  • Moves gossip implementation out of sequencer_core::gossip into a new sequencer_gossip_actor crate with an actor feature gate.
  • Replaces the previous drive-task + channels plumbing with a GossipActor mailbox API (PublishTransaction tell, GetConnectedPeers ask) and adds an outage watchdog.
  • Updates service/RPC wiring and features (mdns) to depend on the new actor crate.
File summaries
File Description
lez/sequencer/service/src/lib.rs Spawns GossipActor, stores bootstrap addrs, shuts gossip down explicitly, and keeps gossip out of health aggregation.
lez/sequencer/service/Cargo.toml Adds sequencer_gossip_actor dependency and forwards mdns feature to it.
lez/sequencer/core/src/lib.rs Removes gossip module export from sequencer_core.
lez/sequencer/core/src/gossip/mod.rs Deletes the old core gossip module facade.
lez/sequencer/core/src/gossip/message.rs Deletes unused/uncompiled wire-format file.
lez/sequencer/core/Cargo.toml Removes the mdns feature from sequencer_core.
lez/sequencer/actors/rpc_server/src/actor/service.rs Switches gossip publisher import to sequencer_gossip_actor.
lez/sequencer/actors/rpc_server/src/actor.rs Switches gossip publisher import to sequencer_gossip_actor.
lez/sequencer/actors/rpc_server/Cargo.toml Adds sequencer_gossip_actor dependency (with actor feature).
lez/sequencer/actors/gossip/src/validation.rs Updates references to use sequencer_core config/helpers after move.
lez/sequencer/actors/gossip/src/tests.rs Updates integration-style tests to run against the spawned GossipActor.
lez/sequencer/actors/gossip/src/seen_cache.rs Introduces bounded FIFO dedup cache for seen transaction hashes.
lez/sequencer/actors/gossip/src/lib.rs New crate root with actor-gated exports and module layout.
lez/sequencer/actors/gossip/src/actor.rs Implements GossipActor, mailbox messages, publisher, and outage watchdog.
lez/sequencer/actors/gossip/src/accreditation/mod.rs Updates config import path after move.
lez/sequencer/actors/gossip/Cargo.toml Adds new sequencer_gossip_actor crate and feature definitions.
Cargo.toml Adds the gossip actor crate to the workspace members and dependencies.
Cargo.lock Adds lock entries for the new sequencer_gossip_actor crate and dependents.
Review details

Suppressed comments (1)

lez/sequencer/actors/gossip/src/actor.rs:301

  • connected_pubkeys() iterates a HashSet, so the returned peer list has nondeterministic ordering. This can lead to unstable RPC responses (and flaky tests/metrics if any consumer compares lists). Sort the collected pubkeys before returning to keep the output stable.
  • Files reviewed: 16/18 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.


/// One sequencer's signature over an offence, for peers to collect.
#[derive(Debug, Clone, PartialEq, Eq, BorshSerialize, BorshDeserialize)]
pub struct SlashApprovalMessage {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@schouhy I'm guessing these were going to be used for slashing k/n right? i removed these so that it is more direct for your slashing PR to add later, if that is ok

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reason for removal: they were dead code at the moment

@Arjentix
Arjentix self-requested a review September 7, 2026 18:38

@Arjentix Arjentix left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glad to see it's being moved to a separate actor, nice work!

Comment thread lez/sequencer/service/src/lib.rs Outdated
Comment thread lez/sequencer/service/src/lib.rs Outdated
Comment thread lez/sequencer/service/src/lib.rs Outdated
Comment thread lez/sequencer/service/src/lib.rs Outdated
Comment thread lez/sequencer/service/src/lib.rs
Comment thread lez/sequencer/actors/gossip/src/actor.rs Outdated
Comment thread lez/sequencer/actors/gossip/src/actor.rs Outdated
Comment thread lez/sequencer/actors/gossip/src/actor.rs Outdated
Comment thread lez/sequencer/actors/gossip/src/actor.rs Outdated
Comment thread lez/sequencer/actors/gossip/src/actor.rs
@erhant
erhant marked this pull request as ready for review September 8, 2026 20:29

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The change is a significant runtime/architecture refactor of the sequencer’s p2p gossip subsystem (task → actor + new crate boundaries), which warrants final human validation beyond static review.

Review details
  • Files reviewed: 20/21 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Comment thread lez/common/src/bounded_vec_deque.rs Outdated
Comment thread lez/sequencer/actors/rpc_server/src/actor/service.rs Outdated
Comment on lines +29 to +31
logos-blockchain-core.workspace = true
logos-blockchain-key-management-system-service.workspace = true
logos-blockchain-zone-sdk.workspace = true

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't it be gated by actor feature as well?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is used by accreditation, #833 (comment)

Depending on the resolution of that comment we may remove accreditation module all together

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

accreditation has no consumers anywhere in the tree, so it is the only thing keeping a featureless build from being protocol-only; removing it drops these three deps rather than gating them.

#[cfg(feature = "actor")]
pub use libp2p::Multiaddr;

pub mod accreditation;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is not just actor implementation detail? Where it's used then?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was to be kept for slashing & gathering signatures for it, but looking at https://github.com/logos-blockchain/logos-execution-zone/pull/818/changes#r3968326121 I can see that this is not needed anymore

let me check in with @schouhy on this to be sure

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

asked in #818 (review)

This was referenced Sep 9, 2026

@moudyellaz moudyellaz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks! Left some comments.

Comment thread lez/sequencer/actors/gossip/src/actor/tests.rs Outdated
Comment thread lez/sequencer/actors/gossip/src/actor.rs
Comment thread lez/sequencer/actors/rpc_server/src/actor/service.rs
@erhant
erhant requested a review from moudyellaz September 10, 2026 08:54

@moudyellaz moudyellaz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks for addressing my comments!

@Arjentix Arjentix left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, waiting for resolving older comments related to accreditation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants