Introduce Flow utilities and OffersMessageFlow implementation #3639

shaavan · 2025-03-03T22:18:58Z

This PR presents an alternative approach to #3412.

It introduces the Flow trait and its implementation, OffersMessageFlow, as a mid-level API that abstracts most Offers-related logic out of ChannelManager, while allowing ChannelManager to be parameterized over Flow.

This improves modularity and separation of concerns, making the Offers-related code more maintainable. Additionally, it provides a set of functional utilities for users who do not use ChannelManager in their workflow.

ldk-reviews-bot · 2025-03-03T22:19:01Z

👋 Hi! I see this is a draft PR.
I'll wait to assign reviewers until you mark it as ready for review.
Just convert it out of draft status when you're ready for review!

shaavan · 2025-03-03T22:26:28Z

Right now, this PR is a proof of concept for the approach—it introduces a basic set of utilities to move a significant chunk of Offers-related code out of ChannelManager.

What's in the current version:

Added initial utilities for creating Offers, Refunds, Blinded Paths, and handling message enqueue/dequeue directly within Flow.
Started using these utilities inside Offer payers and builders.

About the CI failures:

The CI is failing because of some documentation issues in this version.
All functional tests for the std case are passing locally.

If this approach looks good, I'll expand it to cover Offer handlers next!

shaavan · 2025-03-03T22:27:13Z

@TheBlueMatt, @jkczyz
A gentle ping! :)

jkczyz

Let's iterate a bit more on this. We should have the MessageContext creation and thus the BlindedPath creation be the domain of OffersMessageFlow, IMO. This means passing a closure to the utility functions to create the blinded paths. Likewise with BlindedPaymentPaths.

jkczyz · 2025-03-04T15:52:23Z

lightning/src/offers/flow.rs

+	message_router: MR,
+
+	our_network_pubkey: PublicKey,
+	highest_seen_timestamp: AtomicUsize,


We'd need to update this any time a block is processed. So either we need an Arc to share with ChannelManager or have Flow require chain::Listen such that ChannelManager (or whoever owns the Flow) can call block_connected.

That makes sense! I’ve implemented the chain::Listen trait for OffersMessageFlow in pr3639.02. This allows block_connected to be called as needed.

Let me know if this approach aligns with what you had in mind! Thanks a lot for the pointer!

jkczyz · 2025-03-04T15:53:55Z

lightning/src/offers/flow.rs

+	#[cfg(not(any(test, feature = "_test_utils")))]
+	pending_offers_messages: Mutex<Vec<(OffersMessage, MessageSendInstructions)>>,
+	#[cfg(any(test, feature = "_test_utils"))]
+	pub(crate) pending_offers_messages: Mutex<Vec<(OffersMessage, MessageSendInstructions)>>,
+
+	pending_async_payments_messages: Mutex<Vec<(AsyncPaymentsMessage, MessageSendInstructions)>>,
+
+	#[cfg(feature = "dnssec")]
+	pending_dns_onion_messages: Mutex<Vec<(DNSResolverMessage, MessageSendInstructions)>>,


I think the idea here was to have the owner such as ChannelManager own the message queues since it would implement the message handlers.

I was deciding between two designs: having ChannelManager hold the queues or having OffersMessageFlow hold them. I chose the latter because:

The long-term goal is to fully separate offers-related code from ChannelManager, and moving the queues out will be a major step in that direction.

Keeping the queues within OffersMessageFlow helps maintain cleaner utility function signatures, as users don’t need to explicitly pass the queues as parameters.

Let me know your thoughts on this approach!

jkczyz · 2025-03-04T15:58:50Z

lightning/src/offers/flow.rs

+	fn create_offer_builder(
+		&self, nonce: Nonce,
+	) -> Result<OfferBuilder<DerivedMetadata, secp256k1::All>, Bolt12SemanticError> {
+		let node_id = self.get_our_node_id();
+		let expanded_key = &self.inbound_payment_key;
+		let secp_ctx = &self.secp_ctx;
+
+		let builder = OfferBuilder::deriving_signing_pubkey(node_id, expanded_key, nonce, secp_ctx)
+			.chain_hash(self.chain_hash);
+
+		Ok(builder)
+	}


I'd question what value utilities like these are really providing to the the user.

The idea I was exploring was to create a mid-level API that remains modular enough to be used in multiple contexts.

For example, self.flow.create_offer_builder simplifies both create_offer_builder and create_async_receive_offer_builder, making the API more reusable and reducing duplication.

Would love to hear your thoughts on whether this with indented design approach.

jkczyz · 2025-03-04T16:04:31Z

lightning/src/offers/flow.rs

+	MR::Target: MessageRouter,
+{
+	fn create_offer_builder(
+		&self, nonce: Nonce,


Seems like the Nonce could be generated from the EntropySource. Though I imagine we need the Nonce for the MessageContext. Don't we want OffersMessageFlow to determine the correct MessageContext to give to the BlindedPaths? Otherwise, the onus is on the user to pass the correct context. So this implies the blinded paths should be created within this method.

Makes sense! My initial reasoning was to keep offer building and blinded path creation separate, as this allowed us to use the same self.flow.create_offer_builder for both create_offer_builder (where we create & append the path later) and create_async_receive_offer_builder (where we already have a vector of paths to append).

That said, I think we can take a middle ground by passing an optional closure as input—if present, it would handle the creation and appending of paths. This keeps things flexible while reducing the onus on the user to manually pass the correct context.

Let me know what you think of this approach. Thanks!

jkczyz · 2025-03-04T16:08:35Z

lightning/src/offers/flow.rs

+	fn create_invoice_builder<'a>(
+		&'a self, refund: &'a Refund, payment_paths: Vec<BlindedPaymentPath>,
+		payment_hash: PaymentHash,
+	) -> Result<InvoiceBuilder<'a, DerivedSigningPubkey>, Bolt12SemanticError> {


Note we need this not only for refunds but also for handling invoice requests. I'm wondering though if we'd rather want a method that does more similar to request_refund_payment and pay_for_offer?

Looking through the section of code where we create an invoice from an invoice request, it seems significantly different from invoice creation from a refund. Given our current trait design, introducing a separate function feels like a cleaner solution.

On the other hand, we could design the Flow trait to be more similar to request_refund_payment and pay_for_offer, but wouldn’t that make it too high-level of an API?

@TheBlueMatt, would love to hear your thoughts on this!

jkczyz · 2025-03-04T16:09:17Z

lightning/src/offers/flow.rs

+	}
+
+	fn create_blinded_paths(
+		&self, peers: Vec<MessageForwardNode>, context: MessageContext,


Again, it seems OffersMessageFlow should be the one determining which context to use.

jkczyz · 2025-03-04T16:26:05Z

lightning/src/offers/flow.rs


 #[cfg(feature = "dnssec")]
 use crate::onion_message::dns_resolution::DNSResolverMessage;

+pub trait Flow {


After going through the PR, I'm not sure how useful the Flow trait is. Either the caller of the utilities can do any processing between calls or we can parameterize OffersMessageFlow with traits (e.g., currency conversion).

I think I agree with your point. We could make OffersMessageFlow a direct parameter of ChannelManager without needing the Flow trait as an intermediary. This way, we still remove Bolt12 responsibilities from ChannelManager while giving users the flexibility to implement their own handlers using OffersMessageFlow utilities.

@TheBlueMatt, I’d love to hear your thoughts on this approach!

- Docs test failing

TheBlueMatt

Were we going to move most of the OffersMessageHandler logic into flow.rs?

shaavan · 2025-03-06T13:11:11Z

Updated from pr3639.01 to pr3639.02 (diff):
Addressed @jkczyz comments

Changes:

Implement chain::Listen of OffersMessageFlow, to allow keep the highest_seen_timestamp synced.

shaavan · 2025-03-06T13:28:31Z

@TheBlueMatt

Were we going to move most of the OffersMessageHandler logic into flow.rs?

Yes, that’s the goal! For this version, I’ve primarily abstracted out the Bolt12 payers/builders from ChannelManager. I wanted to first get a concept ACK on this approach before proceeding with abstracting the handlers out of ChannelManager.

shaavan added 2 commits March 4, 2025 17:55

Introduce OffersMessageFlow

fde130e

Introduce Flow trait

ba8de7c

shaavan force-pushed the flow branch from 65ace16 to eb764be Compare March 4, 2025 12:30

jkczyz reviewed Mar 4, 2025

View reviewed changes

shaavan added 5 commits March 5, 2025 18:41

f: Constraint Flow to only work for trait implementing chain::Listen

1a6821e

f: Implement chain::Listen for OffersMessageFlow

498afb8

Implement Flow for OffersMessageFlow

de24b92

Introduce Flow parameter in ChannelManager

9658d90

- Docs test failing

Introduce flow usage in channelmanager

286ccd7

TheBlueMatt reviewed Mar 5, 2025

View reviewed changes

shaavan force-pushed the flow branch from eb764be to 286ccd7 Compare March 6, 2025 13:11

jkczyz mentioned this pull request Mar 7, 2025

Multiple InvoiceReceived events for single pay_for_offer call #3653

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce Flow utilities and OffersMessageFlow implementation #3639

Introduce Flow utilities and OffersMessageFlow implementation #3639

shaavan commented Mar 3, 2025

ldk-reviews-bot commented Mar 3, 2025

shaavan commented Mar 3, 2025

shaavan commented Mar 3, 2025

jkczyz left a comment

jkczyz Mar 4, 2025

shaavan Mar 6, 2025

jkczyz Mar 4, 2025

shaavan Mar 6, 2025

jkczyz Mar 4, 2025

shaavan Mar 6, 2025

jkczyz Mar 4, 2025 •

edited

Loading

shaavan Mar 6, 2025

jkczyz Mar 4, 2025 •

edited

Loading

shaavan Mar 6, 2025

jkczyz Mar 4, 2025

jkczyz Mar 4, 2025

shaavan Mar 6, 2025

TheBlueMatt left a comment

shaavan commented Mar 6, 2025

shaavan commented Mar 6, 2025

Introduce Flow utilities and OffersMessageFlow implementation #3639

Are you sure you want to change the base?

Introduce Flow utilities and OffersMessageFlow implementation #3639

Conversation

shaavan commented Mar 3, 2025

ldk-reviews-bot commented Mar 3, 2025

shaavan commented Mar 3, 2025

What's in the current version:

About the CI failures:

shaavan commented Mar 3, 2025

jkczyz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkczyz Mar 4, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkczyz Mar 4, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TheBlueMatt left a comment

Choose a reason for hiding this comment

shaavan commented Mar 6, 2025

shaavan commented Mar 6, 2025

jkczyz Mar 4, 2025 •

edited

Loading

jkczyz Mar 4, 2025 •

edited

Loading