Skip to content

Add sr25519 support#3

Draft
sdbondi wants to merge 99 commits intotari-project:masterfrom
sdbondi:add-sr25519-support
Draft

Add sr25519 support#3
sdbondi wants to merge 99 commits intotari-project:masterfrom
sdbondi:add-sr25519-support

Conversation

@sdbondi
Copy link
Member

@sdbondi sdbondi commented May 3, 2024

This can be kept open - just putting this branch in a PR for visibility

Description

Uses tari_crypto to quickly add Schnorr-Ristretto support to libp2p

Notes & open questions

Change checklist

  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • A changelog entry has been made in the appropriate crates

* upstream/master: (68 commits)
  deps: bump snow from 0.9.4 to 0.9.5
  feat(swarm): allow behaviours to share addresses of peers
  chore(webrtc): release version 0.7.1-alpha
  chore: update config.yml for new issues
  deps: bump env_logger from 0.10.1 to 0.10.2
  deps: bump web-sys from 0.3.66 to 0.3.67
  deps: bump regex from 1.10.2 to 1.10.3
  deps: bump proc-macro2 from 1.0.76 to 1.0.78
  fix(upnp): ignore oneshot sender error when receiver is dropped
  deps: bump tj-actions/glob from 18 to 19
  deps: bump h2 from 0.3.20 to 0.3.24
  ci: update `cargo semver-checks` to `v0.27`
  deps: bump Swatinem/rust-cache from 2.7.2 to 2.7.3
  chore: fix clippy lints
  feat: introduce `libp2p-stream`
  deps: bump igd-next from 0.14.2 to 0.14.3
  deps: bump getrandom from 0.2.11 to 0.2.12
  deps: bump clap from 4.4.13 to 4.4.16
  deps: bump smallvec from 1.11.2 to 1.12.0
  deps: bump wasm-bindgen from 0.2.89 to 0.2.90
  ...
* master: (142 commits)
  chore(swarm-derive): lower unreleased version to 0.32.2
  fix(relay): use `web_time` `Instant` and `SystemTime` versions for wasm support.
  deps: bump data-encoding from 2.5.0 to 2.6.0
  deps: bump parking_lot from 0.12.1 to 0.12.2
  deps: promote `futures` to workspace dependency
  docs: enhance  code documentation
  chore: fix typo in ipfs-kad example README.md
  deps: bump serde from 1.0.197 to 1.0.198
  docs: fix typos in some comments
  fix: update Cargo.lock
  deps: bump reqwest from 0.12.3 to 0.12.4
  deps: bump rmp-serde from 1.1.2 to 1.2.0
  deps: bump syn from 2.0.59 to 2.0.60
  deps: bump proc-macro2 from 1.0.80 to 1.0.81
  deps: bump serde_json from 1.0.115 to 1.0.116
  deps: bump thiserror from 1.0.58 to 1.0.59
  deps: bump anyhow from 1.0.81 to 1.0.82
  chore: fix typos in doc comments
  feat(kad): derive Copy for kbucket::key::Key
  deps: bump either from 1.9.0 to 1.11.0
  ...
@sdbondi sdbondi closed this May 3, 2024
@sdbondi sdbondi reopened this May 3, 2024
@sdbondi sdbondi marked this pull request as draft May 3, 2024 07:03
github-merge-queue bot pushed a commit to tari-project/tari-ootle that referenced this pull request May 6, 2024
Description
---
Update to use latest commit in our libp2p fork

Motivation and Context
---
Updates to use latest libp2p including yamux upgrade
libp2p/rust-yamux@460baf2
tari-project/rust-libp2p#3

How Has This Been Tested?
---
Manually and existing tests

What process can a PR reviewer use to test or verify this change?
---
Should work as before

Breaking Changes
---

- [x] None
- [ ] Requires data directory to be deleted
- [ ] Other - Please specify
* tag 'v0.54.1': (135 commits)
  chore(libp2p): release 0.54.1
  chore(metrics): release 0.15.0
  feat(kad): New provider record update strategy
  chore: Fix identity changelog link
  Create funding.json (libp2p#5553)
  chore: fix some comments
  fix(quic): release libp2p-quic 0.11.1
  chore(swarm): add PR number to 0.45.1 release
  fix(swarm): release libp2p-swarm-derive 0.35.0
  chore: prepare libp2p 0.54 release
  feat(websocket,websocket-websys): Add support for `/tls/ws`
  chore(metrics): Add `/tls/ws` in protocol stack tests
  feat(autonat): Implement AutoNATv2
  chore: add Gui as a maintainer
  feat(kad): configurable bucket size
  feat(websocket): Allow wss connections on IP addresses
  docs: As implied by libp2p#5517 make sure everything still builds on docs.rs
  chore: remove RUSTFLAG from docs workflow
  deps: bump EmbarkStudios/cargo-deny-action from 1 to 2
  refactor(*): Transport redesign
  ...
@sdbondi sdbondi force-pushed the add-sr25519-support branch from 4ed2650 to 2d5af69 Compare October 10, 2024 13:32
* master: (36 commits)
  chore: refactor ping tests (libp2p#5655)
  feat: refactor distributed-key-value-store example (libp2p#5652)
  chore(ci): address clippy beta lints (libp2p#5649)
  feat(gossipsub): apply `max_transmit_size` to the published message (libp2p#5642)
  feat(kad): add `Behavior::find_closest_local_peers()` (libp2p#5645)
  fix(swarm-test): set proper version (libp2p#5648)
  deps(ci): update cargo-semver-checks (libp2p#5647)
  chore: fix typo in comment (libp2p#5643)
  feat: make runtime features optional in swarm-test (libp2p#5551)
  deps: bump Swatinem/rust-cache from 2.7.3 to 2.7.5 (libp2p#5633)
  chore: update igd-next to 0.15.1 (libp2p#5625)
  fix(server): removing dependency on libp2p-lookup (libp2p#5610)
  refactor(examples): use tokio instead of async-std in relay-server (libp2p#5600)
  deps: update metrics example dependencies (libp2p#5617)
  chore: update interop test run condition (libp2p#5611)
  chore(autonat-v2): fix dial_back_to_non_libp2p test (libp2p#5621)
  fix(swarm): don't report `NewExternalAddrCandidate` if already confirmed (libp2p#5582)
  chore(ci): address beta clippy lints (libp2p#5606)
  fix(ci): address cargo-deny advisories (libp2p#5596)
  chore(ci): only run interop tests on commits to master (libp2p#5604)
  ...
* master: (61 commits)
  feat(request-response): allow custom sizes for `json` and `cbor` codec
  fix(gossipsub): make sure we have fanout peers when publish
  feat: broadcasting `idontwant` for published messages
  chore(kad): remove default constructor for ProtocolConfig
  chore(allow-block-list): revert version bump
  chore(ci): add Zlib to deny.toml
  chore(identify): revert version bump
  chore(server): revert version bump
  chore(autonat): revert version bump
  chore(kad): revert version bump
  feat(gossipsub): Allow setting a size threshold for IDONTWANT messages
  chore(deps): update Cargo.lock
  feat(gossipsub): implement gossipsub 1.2 beta
  chore: introduce `libp2p-test-utils`
  deps(quic): update quinn to 0.11.6
  chore(mdns): revert version bump
  chore(deps): remove unused deps
  fix: multiple typos of different importance
  feat(mdns): emit `ToSwarm::NewExternalAddrOfPeer` on discovery
  chore: add SQD Network to notable users list
  ...
@sdbondi sdbondi force-pushed the add-sr25519-support branch from a5ec42a to 8f1ceee Compare January 8, 2025 04:59
sdbondi added 2 commits April 3, 2025 15:19
* master: (86 commits)
  deps: update webrtc to 0.12.0
  fix(gossip): correct clippy warning
  chore(deps): bump prometheus-client
  chore: replace deprecated clap attributes
  chore(deps): bump webpki
  chore(ci): allow triggering manually
  chore(examples): fix ipfs-kad example usage
  chore(ci): cargo deny, ignore RUSTSEC-2024-0436
  chore(peer-store): use workspace edition
  chore: update igd-next to 0.16.1
  chore: remove unreachable_patterns lint according to note
  feat(rz): emit `NewExternalAddrOfPeer` when discovering peers
  deps: bump Swatinem/rust-cache from 2.7.7 to 2.7.8
  fix(misc): deprecated #[clap] attributes with #[arg] and #[command]
  chore(deps): bump github.com/quic-go/quic-go
  chore(deps): bump golang.org/x/net
  chore(ci): update golang version for wasm tests
  fix: deprecated #[clap] attributes with #[arg] and #[command]
  fix(gossipsub): race condition for messages already on the network
  docs(request-response): update docs about peer address management
  ...
@sdbondi sdbondi force-pushed the add-sr25519-support branch from b92c8b4 to d322272 Compare April 8, 2025 08:32
sdbondi and others added 5 commits June 17, 2025 16:04
* master: (44 commits)
  chore(websocket): migrate test from async-std to tokio
  chore(request-response): migrate tests from async-std to tokio
  feat: remove async_std support in QUIC crate
  chore(libp2p-mplex): use tokio instead of async_std
  chore(deps): bump github.com/quic-go/quic-go from 0.50.0 to 0.50.1 in /wasm-tests/webtransport-tests/echo-server
  chore: deprecate async-io support in mDNS crate
  chore(swarm): use tokio instead of async_std
  feat(swarm): add Option<PeerId> to IncomingConnectionError
  chore: swarm executor should match runtime in tests
  chore(libp2p-uds): use tokio instead of async_std in tests
  chore(libp2p-websocket): use tokio instead of async-std
  chore(rw-stream-sink): use tokio instead of async-std in tests
  chore(ci): update cargo semver-checks
  feat(gossipsub): feature gate metrics related code
  chore: address clippy beta lints
  feat(swarm): remove deprecated `with_bandwidth_logging`
  feat(gossipsub): send IDONTWANT before publishing a new message
  feat(kad): configurable outbound_substreams timeout
  fix(dns): Report all transport errors in a dial attempt
  fix(rendezvous): log instead of panic when sending response to client
  ...
Amendment to libp2p#5857.
libp2p#5857 changed the tests in `libp2p-request-response` to use `tokio` as executor.
We forgot to also change the test-swarm's executor to use `tokio` instead of `async_std` (see libp2p#6024), so this PR fixes it.

Pull-Request: libp2p#6063.
Remove support for `async_std` in `swarm-test`.

Pull-Request: libp2p#6064.
AgeManning and others added 17 commits October 17, 2025 21:26
I noticed two issues during testing.

1.  We allow our mesh to grow greater than `mesh_n_high`, intentionally
2.  There is a potential underflow in the heartbeat that can cause a panic

For the first 1. This looks like its intentional but I can't recall why we would have added it. I think its counter-intuitive to allow our mesh to grow larger than the specified parameter. I suspect we added it to prevent our mesh from being filled with inbound peers and potentially being eclipsed. I suspect the best approach here is to remove inbound peers in the mesh maintenance rather than exceeding the mesh_n_high configuration.

For 2. There is an underflow which this PR prevents. It can be triggered for low mesh_n_high values, i.e 0. This shouldn't be a concern for regular users, but we shouldn't have code that can panic based on user configuration.

Pull-Request: libp2p#6183.
Adds links to related tech mentioned in examples/README.
This is useful to people new to the field, like me.

Pull-Request: libp2p#6016.
Split off from libp2p#6183, to quote:
>I noticed two issues during testing.
>
>We allow our mesh to grow greater than mesh_n_high, intentionally
>This looks like its intentional but I can't recall why we would have added it. I think its counter-intuitive to allow our mesh to grow larger than the specified parameter. I suspect we added it to prevent our mesh from being filled with inbound peers and potentially being eclipsed. I suspect the best approach here is to remove inbound peers in the mesh maintenance rather than exceeding the mesh_n_high configuration.

Pull-Request: libp2p#6184.
## Description

Changelog was incorrect as we didn't release version `0.5.1`
cc @RolandSherwin
will address the intra doc links issue on a subsequent PR, the remaining [issue](https://github.com/libp2p/rust-libp2p/actions/runs/19040315461/job/54375441096?pr=6195#step:6:760) seems like a bug, opened rust-lang/rust-clippy#16023.

Pull-Request: libp2p#6195.
The rustwasm organization was archived this year. Projects have moved to new URLs to reflect that. This PR just updates those URLs.

See https://blog.rust-lang.org/inside-rust/2025/07/21/sunsetting-the-rustwasm-github-org/

Pull-Request: libp2p#6199.
From libp2p#2217.
Renaming UDS Config symbol from `UdsConfig` in the doc comment and from `TokioUdsConfig`, for better referencing when re-exported.
Left deprecated alias for backward compatibility.

Pull-Request: libp2p#6190.
This removes a `debug_assert` from `poll_flush` because it can be valid for the user to recheck the flush state. In that case the operation should be noop.

A real life code that triggers this assertition is [this](https://github.com/libp2p/rust-asynchronous-codec/blob/c818a83906891caf8aadcae8f899727c2c8393a8/src/framed_write.rs#L263-L266).

Fixes libp2p#5618

Pull-Request: libp2p#6193.
Adds a fast path in `remove_data_messages` to avoid unnecessary work when the caller provides an empty `message_ids` slice.

Pull-Request: libp2p#6208.
This change improves peer removal during `heartbeat` by switching from a two-pass remove logic to an in-place `retain` with very small size intermediate variable `removed_peers_count`.

Pull-Request: libp2p#6209.
it's not required for the `peerid` feature

Pull-Request: libp2p#6226.
Replaces the lossy cast with a safer/checked version when initializing sequence numbers from the unix timestamp in nanoseconds.

Pull-Request: libp2p#6211.
@sdbondi sdbondi force-pushed the add-sr25519-support branch from 366042e to debc012 Compare December 17, 2025 11:01
sdbondi and others added 6 commits December 17, 2025 15:01
Improve gossipsub behavior tests structure.
Split the already big enough file into multiple test files according to their `Behaviour` tests  category.
Document the shared test primitives and rename structures, `InjectNodes` to `BehaviourTestBuilder` and remove the `inject_nodes` functions to use `Default`.

This will help with the new tests for partial messages on sigp#577 which will be eventually moved here

Pull-Request: libp2p#6232.
To help with the upcoming [partial messages](sigp#577)

Pull-Request: libp2p#6234.
Fixes libp2p#6204

PAYLOAD_TYPE: "/libp2p/routing-state-record" → [0x03, 0x01]
DOMAIN_SEP: "libp2p-routing-state" → "libp2p-peer-record"
The payload type [0x03, 0x01] is the multicodec identifier for libp2p-peer-record, as defined in:
[Multicodec table](https://github.com/multiformats/multicodec/blob/master/table.csv)
This matches the implementations in:

[Go libp2p](https://github.com/libp2p/go-libp2p/blob/master/core/peer/record.go#L21-L24)
[JS libp2p](https://github.com/libp2p/js-libp2p/blob/main/packages/peer-record/src/peer-record/consts.ts#L7)

PR Recreation Notice
This PR is a recreation of [libp2p#6205], which was automatically closed after I mistakenly deleted my fork.
It restores the exact same code and commits as before. No additional changes were made.

Follow-up plan (based on discussion in the original PR)

During the previous review, the suggested direction was:

Introduce this change behind a new Cargo feature, which will be false by default.

Deprecate the current implementation, allowing users to switch gradually.

Give the ecosystem time to migrate, before potentially making the new behavior the default in the future.

Why I’m reopening now

This PR is being reopened and then I added the new optional `peer-record-interop` feature flag.

Testing
Added cross-implementation interoperability tests:

core/tests/peer_record_interop.rs - Tests that verify Rust can decode and verify signed peer records from Go and JS
core/tests/fixtures/ - Test fixtures with signed peer records generated by Go and JS implementations
Before the fix, these tests failed with:

BadPayload(UnexpectedPayloadType { expected: [47, 108, ...], got: [3, 1] })

After the fix, all tests pass.
Breaking Change
⚠️ BREAKING CHANGE: Existing signed peer records created with Rust libp2p will not be verifiable with this change, as the domain and payload type have changed. However, this is necessary to fix interoperability with Go/JS libp2p implementations.

Notes & open questions

None - this is a straightforward alignment with the standard used by other libp2p implementations.

Change checklist
[x ] I have performed a self-review of my own code
I have made corresponding changes to the documentation
I have added tests that prove my fix is effective or that my feature works
[x ] A changelog entry has been made in the appropriate crates

Pull-Request: libp2p#6230.
* master: (77 commits)
  feat(tcp): expose socket2 `set_linger`
  fix: use standard peer-record domain and payload type
  feat(gossipsub): log when sending and receiving gossipsub messages
  chore(gossipsub): improve test structure
  chore(ci): fix cargo deny to address RUSTSEC-2025-0134
  fix(gossipsub): avoid direct casting from u128 to u64
  fix(identity): make quick-protobuf dep optional
  refactor(gossipsub): in-place negative-score peer removal
  deps: bump Swatinem/rust-cache from 2.8.1 to 2.8.2
  refactor(gossipsub): do early return in for an empty input
  fix(webtransport-websys): Allow `poll_flush` after `poll_close`
  chore(uds): Renamed UDS Confing to follow re-export convention
  chore: updated rustwasm links to new org home
  chore: fix clippy lints
  chore(ci): fix intra doc links job
  chore: prepare libp2p-upnp v0.6.0 (libp2p#6188)
  fix(gossisub): prevent mesh exceeding mesh_n_high
  chore(README.md): Add Links to mentioned tech
  chore(deps): bump quic-go/quic-go from 0.50.1 to 0.54.1 in /wasm-tests
  chore(dpes): bump `prometheus-client`
  ...
@sdbondi sdbondi force-pushed the add-sr25519-support branch 4 times, most recently from 3601ee3 to 95b732e Compare January 12, 2026 09:22
@sdbondi sdbondi force-pushed the add-sr25519-support branch from 95b732e to 6929685 Compare January 12, 2026 13:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.