Skip to content

fix(network): batch concurrent tx status lookups in AcceptTxInDBsFilter#1896

Open
hstarorg wants to merge 3 commits into
LFDT-Panurus:mainfrom
Built-by-Sign:perf/batch-tx-status-lookup
Open

fix(network): batch concurrent tx status lookups in AcceptTxInDBsFilter#1896
hstarorg wants to merge 3 commits into
LFDT-Panurus:mainfrom
Built-by-Sign:perf/batch-tx-status-lookup

Conversation

@hstarorg

Copy link
Copy Markdown
Contributor

AcceptTxInDBsFilter.Accept issued up to two serial DB round trips per call, and FSC can have several such calls in flight concurrently while committing a block. Add GetStatuses(ctx, txIDs) to ttxdb/auditdb StoreService, and a statusBatcher that coalesces concurrent Accept lookups within a 300us window into a single batched query per store, instead of one query pair per transaction.

Fixes #1872

hstarorg added 2 commits July 11, 2026 23:59
AcceptTxInDBsFilter.Accept issued up to two serial DB round trips per
call, and FSC can have several such calls in flight concurrently while
committing a block. Add GetStatuses(ctx, txIDs) to ttxdb/auditdb
StoreService, and a statusBatcher that coalesces concurrent Accept
lookups within a 300us window into a single batched query per store,
instead of one query pair per transaction.

Fixes LFDT-Panurus#1872

Signed-off-by: hstarorg <jayhu@sign.global>
require.NoError/ErrorIs must run on the test's main goroutine (go-require)
and error assertions should use require, not assert (require-error).
Fixes the 4 golangci-lint testifylint findings in filter_test.go and
status_batcher_test.go from the PR LFDT-Panurus#1896 CI run.

Signed-off-by: hstarorg <jayhu@sign.global>
@adecaro

adecaro commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Hi @hstarorg , thanks for submitting this 🙏
Please, next time, get first the issue assigned and then start with the implementation. Many thanks 🙏

@adecaro adecaro self-requested a review July 13, 2026 04:24
@adecaro adecaro self-assigned this Jul 13, 2026
@adecaro adecaro added this to the Q3/26 milestone Jul 13, 2026
@adecaro

adecaro commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Hi @hstarorg , what's the performance improvement that you have seen with this patch?

@hstarorg

Copy link
Copy Markdown
Contributor Author

Heads-up: this filter currently isn't reachable from FSC's committer

Before merging — I want to flag something I found while re-reviewing, since it changes what this PR actually delivers at runtime.

AcceptTxInDBsFilter (the filter this PR optimizes) is not currently registered with FSC's committer anywhere in this repo. The two calls that used to do that —

transactionFilter, err := n.filterProvider.New(tmsID)
...
committer := n.ch.Committer()
if err := committer.AddTransactionFilter(transactionFilter); err != nil {

— were removed from token/services/network/fabric/network.go's connect() in cd7e7a4 (#1200, "remove RWSetProcessor", Aug 12 2025). The filterProvider field/parameter itself was left in place and is still threaded through NewNetwork (both fabric and fabricx variants) and the dig container, but nothing reads it anymore — it's just passed along and stored, never consumed. A repo-wide search (excluding the vendored fabric-smart-client module) turns up zero calls to AddTransactionFilter.

Practical effect: AcceptTxInDBsFilter.Accept is dead code on main today. This PR's batching is correct and well-tested (verified independently — build/vet/gofmt/golangci-lint clean, -race clean, CI green), but it currently has zero runtime effect, because FSC never calls Accept in the first place. Issue #1872's premise ("FSC calls it once per unknown transaction while committing a block") no longer holds on main.

This predates this PR and isn't something it needs to fix by itself, but it seems worth deciding explicitly before merge:

  • merge as prep work and open a follow-up to restore the filterProvider.New + AddTransactionFilter registration (removed in cd7e7a4) — that's the only way this optimization actually takes effect, or
  • fold the re-registration into this PR, or
  • hold off if there's a reason the filter registration was intentionally dropped that I'm missing (worth confirming with whoever wrote cd7e7a4/network service/fabric: remove RWSetProcessor #1199 #1200).

Happy to send the follow-up PR restoring the registration if that's the preferred path.

@hstarorg

Copy link
Copy Markdown
Contributor Author

Hi @hstarorg , what's the performance improvement that you have seen with this patch?
What it does: once that registration exists, concurrent Accept calls
(currently up to 2 serial DB queries each) get merged into one batched
query per store within a 300us window — O(1) per window instead of O(N).

Note: the current caller path has that registration removed
(AddTransactionFilter was dropped in cd7e7a4/#1200), so this filter
isn't invoked today.

@hstarorg

Copy link
Copy Markdown
Contributor Author

Hi @hstarorg , thanks for submitting this 🙏 Please, next time, get first the issue assigned and then start with the implementation. Many thanks 🙏

Understood, will do 🙏

@adecaro

adecaro commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Hi @hstarorg , super interesting. So, the reliance on the FSC's fabric's commit pipeline was removed because that path was too slow. We should not restore therefore it and I would suggest to remove the unused code.

Nevertheless, it might be that your patch might still be helpful in token/services/ttx/finality.go, where in addition to a finality listener, the code queries the db for transaction status. What do you think?

@adecaro

adecaro commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Hi @hstarorg , any follow up on this? Thanks 🙏

@hstarorg

Copy link
Copy Markdown
Contributor Author

Hi @hstarorg , any follow up on this? Thanks 🙏

Thanks for the pointer — that makes sense. I'll rework this PR in that direction: drop the unused AcceptTxInDBsFilter / filterProvider plumbing, and wire the status batching into token/services/ttx/finality.go instead, where the concurrent GetStatus calls actually happen.

Will update the PR shortly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf: block-commit filter issues two serial, non-batched DB round trips per unknown transaction

2 participants