[graph-work-side-branch]: temp side branch for graph work #9692
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We want `context.TODO()` to be high signal in the code-base. It should signal clearly that work is required to thread parent context through to the call-site. So to keep the signal-to-noise ratio high, we remove any context.TODO() calls from tests since these will never need to be replace by a parent context.
We want `context.TODO()` to be high signal in the code-base. It should signal clearly that work is required to thread parent context through to the call-site. So to keep the signal-to-noise ratio high, we remove any context.TODO() calls from tests since these will never need to be replace by a parent context. After this commit, there is only a single context.TODO() left in the code-base.
In preparation for starting to thread a single parent context through LND, we update the main `server.Start` method to take a context so that it can later pass it to any subsytem's Start method it calls. We also pass the context to `newServer` since it makes some calls that will eventually reach the DB (for example the graph db).
Pass the parent LND context to the gossiper, let it derive a child context that gets cancelled on Stop. Pass the context through to any methods that will eventually thread it through to any graph DB calls. One `context.TODO()` is added here - this will be removed in the next commit. NOTE: for any internal methods that the context gets passed to, if those methods already listen on the gossiper's `quit` channel, then then don't need to also listen on the passed context's Done() channel because the quit channel is closed at the same time that the context is cancelled.
And remove a context.TODO() that was added in the previous commit.
macaroons+kvdb: improve signal-to-noise ratio of `context.TODO`
discovery: thread context through in preparation for passing to graph DB
The `GossiperSyncer` makes various calls to the `ChannelGraphTimeSeries` interface which threads through to the graph DB. So in preparation for threading context through to all the methods on that interface, we update the GossipSyncer accordingly by passing contexts through. Two `context.TODO()`s are added in this commit. They will be removed in the upcoming commits.
Here, we remove one context.TODO() by threading a context through to the SyncManager.
With this, we move a context.TODO() out of the gossiper and into the brontide package - this will be removed in a future PR which focuses on threading contexts through that code.
Since the ChannelGraphBootstrapper implementation makes a call to the graph DB.
|
Important Review skippedAuto reviews are limited to specific labels. 🏷️ Labels to auto review (1)
Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
Pull reviewers statsStats of the last 30 days for lnd:
|
For any method that takes a context that has a select that listens on the systems quit channel, we should also listen on the ctx since we should not need to worry about if this context is derived internally or externally.
The `GraphSource` interface in the `autopilot` package is directly implemented by the `graphdb.KVStore` and so we will eventually thread contexts through to this interface. So in this commit, we start updating the autopilot system to thread contexts through in preparation for passing the context through to any calls made to the GraphSource. Two context.TODOs are added here which will be addressed in follow up commits.
Remove one context.TODO and add one more.
Continue threading context through the autopilot system and remove the remaining context.TODOs.
This commit cleans up the graph test code by removing unused kvdb type parameters from the `createTextVertex` and `createLightningNode` helper methods. We also pass in the testing parameter now so that we dont need to check the error each time we call `createTestVertex`.
Remove the kvdb.Backend parameter from the `createChannelEdge` helper. This is all in preparation for having the unit tests run against any DB backend.
Remove unused kvdb.Backend param from `randEdgePolicy` and `newEdgePolicy` test helpers.
Replace all tests calls to the private `forEachNode` method on the `KVStore` with the exported ForEachNode method. This is in preparation for having the tests run against an abstract DB backend.
discovery: thread contexts through syncer, sync manager and reliable sender
graph/db: clean up test code in preparation for DB abtraction
autopilot: thread contexts through in preparation for GraphSource methods taking a context
Later on we will create an interface for the persisted graph data. We want this interface to be as small and as neat as possible. In preparation for this, we remove this unused `Wipe` method.
In preparation for creating a clean interface for the graph store, we want to hide anything that is DB specific from the exposed methods on the interface. Currently the `ForEachNodeChannel` and the `FetchOtherNode` methods of the `KVStore` expose a `kvdb.RTx` parameter which is bbolt specific. There is only one call-site of `ForEachNodeChannel` actually makes use of the passed `kvdb.RTx` parameter, and that is in the `establishPersistentConnections` method of the `server` which then passes the tx parameter to `FetchOtherNode`. So to clean-up the interface such that the `kvdb.RTx` is no longer exposed: we instead create one new method called `ForEachSourceNodeChannel` which can be used to replace the above mentioned call-site. So as of this commit, all the remaining call-site of `ForEachNodeChannel` pass in a nil param for `kvdb.RTx` - meaning we can remove the parameter in a future commit.
Unexport the KVStore `FetchOtherNode` and `ForEachNodeChannelTx` methods so that fewer exposed methods are leaking implementation details.
Replace all calls to bbolt specific methods on the KVStore to instead use exported methods on the KVStore that are more db-agnostic.
Since we have not removed all call-sites that make use of this parameter, we can remove it. This helps hide DB-specific details from the interface we will introduce for the graph store.
graph/db: remove various `kvdb` parameters from exported methods
discovery+autopilot: revert passing contexts to `Start` methods
Later on we will store the Alias as a Text field in our sql impls of the graph db. For Postgres, this field then MUST be a valid UTF-8 string. This is also the case in general for the alias according to [bolt 7](https://github.com/lightning/bolts/blob/e1fa25cf00446f3a6a6abbbc9a617cae5b75e39f/07-routing-gossip.md?plain=1#L313)
- Let it do a proper comparison of the full structs passed in. - Pass in a testing parameter so we can remove the returned error. - Make sure the callers are passing in the expected and result parameters in the correct order. - Fix a bug: the compareNodes was not comparing the Features field of the LightningNode structs. Now that it does, one test needed to be updated to properly set the expected Features fields.
Remove the kvdb.Backend return type of the `makeTestGraph` helper. This is in preparation for the helper being used to create a test graph backed by a DB other than bbolt.
The channeldb no longer depends on the graph. So remove the use of MakeTestGraph from tests.
graph+channeldb: test clean-up in preparation for different graph DB backend
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR shows all the work merged into the
elle-graphside branch. This will be a mixture of the remaining work from the Graph Abstraction project along with some of the incoming native SQL graph migration work (mainly clean-up/refactor prep work).We can merge this into master once v0.19 has dropped. I'll leave it in draft until then.
This PR has the following PRs merged in:
context.TODO#9681kvdbparameters from exported methods #9695Startmethods #9704