Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Provide ability to paginate transactions similar to invoices and payments #4719

Open
mrfelton opened this issue Oct 25, 2020 · 9 comments · May be fixed by #8998
Open

Provide ability to paginate transactions similar to invoices and payments #4719

mrfelton opened this issue Oct 25, 2020 · 9 comments · May be fixed by #8998
Assignees
@Abdulkbk
Labels
enhancement Improvements to existing features / behaviour feature request Requests for new features rpc Related to the RPC interface wallet The wallet (lnwallet) which LND uses
Milestone
v0.19.0

Comments

@mrfelton
Copy link
Contributor

mrfelton commented Oct 25, 2020 •
edited
Loading

Background

Payments and Invoices have standard controls for pagination that enable you to specify how many items to receive and what index to start from. Transactions have a different set of pagination controls that enable you to specify a start height and end height.

This makes it challenging to build a UI that has decent pagination of all core activity types (transactions, invoices and payments) since the mechanism to fetch the data varies depending on what data type is being fetched.

For Payments and Invoices I can easily fetch a certain 100 items. But for Transactions there is no way for me to fetch a certain block of 100 items - I don't know wether blocks X through Y contain any transactions at all, so finding the last 100 transactions can only be done through a process of trial and error.

The best I can do with the current api is to choose a particular start and end height to fetch transactions between and then count how many were fetched. If there were less than 100 then try fetching some more. Or, if there were more than 100, truncate the result. With an approach like this, I might need to make thousands of calls to GetTransactions just to find the last 100 transactions since I'd effectively have to keep calling GetTransactions with a lower and lower start and end height all the way back until the start of time.

Your environment

  • version of lnd: 0.11
  • which operating system (uname -a on *Nix): all
  • version of btcd, bitcoind, or other backend: any
  • any other relevant environment details

Steps to reproduce

Try building a paginator that pages through transactions in batches of 100.

Expected behaviour

I should be able to specifically ask for 100 transactions form a given offset.

Actual behaviour

I have to do a process of trial and error in order to fetch only the last 100 transactions, or transactions 100-200 etc.
@Roasbeef Roasbeef added enhancement Improvements to existing features / behaviour feature request Requests for new features rpc Related to the RPC interface wallet The wallet (lnwallet) which LND uses labels Oct 26, 2020
@mrfelton mrfelton mentioned this issue Jan 14, 2021
Closed
@mikuhl-dev
Copy link

Need this, very hard to display transactions alongside invoices and payments, as transactions do not support pagination, and transactions use block height for filter instead of dates.

@alexbosworth
Copy link
Contributor

This is important also to avoid timeout issues, although the workaround of block heights could help? Maybe there is also a documentation aspect to this on how to use block heights most efficiently when normal paging isn't available @saubyk

@saubyk saubyk added this to the 0.19.0 milestone Jun 4, 2024
@Abdulkbk
Copy link

Abdulkbk commented Jul 1, 2024

I will be working on addressing this issue

@Abdulkbk
Copy link

I have researched this issue, including how pagination controls were implemented in the payments here and invoices here RPCs. From what I have observed for listchaintxns RPC, transactions are retrieved from a node's wallet using an external package btcwallet.

I have a question: would it be necessary to open a pull request in btcwallet to modify the GetTransactions function to accommodate the new controls that will be added to the listchaintxns RPC? Or is there a better way to achieve this?

cc: @mrfelton @alexbosworth @Roasbeef

@Roasbeef
Copy link
Member

Hi @Abdulkbk you're correct that in order to make this efficient for transactions, we'd need to modify GetTransaction in btcwallet. Right now it's based solely on start and end height, but we also want it to be aware of the number of transaction to return, and also an offset. You should be able to use a database cursor to handle that.

@Abdulkbk
Copy link

Hi @Abdulkbk you're correct that in order to make this efficient for transactions, we'd need to modify GetTransaction in btcwallet. Right now it's based solely on start and end height, but we also want it to be aware of the number of transaction to return, and also an offset. You should be able to use a database cursor to handle that.

Do you mean I should be able to use a database cursor to handle this without necessarily making any changes to GetTransactions in btcwallet? I want to make sure I understand this before proceeding. Thank you.

@Roasbeef
Copy link
Member

So you can use the database interface used in btcwallet to obtain a database cursor.

@Abdulkbk Abdulkbk linked a pull request Aug 11, 2024 that will close this issue
Open
8 tasks
@Abdulkbk
Copy link

Abdulkbk commented Aug 13, 2024 •
edited
Loading

I added a PR #8998 addressing this, in case anyone wants to review :)

@saubyk
Copy link
Collaborator

saubyk commented Aug 13, 2024

I added a PR #8998 addressing this, in case anyone wants to review :)

Hi @Abdulkbk will take it up for review once the dev work starts up for 0.19. It may take a 2-3 weeks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Abdulkbk Abdulkbk

Labels
enhancement Improvements to existing features / behaviour feature request Requests for new features rpc Related to the RPC interface wallet The wallet (lnwallet) which LND uses
Projects
lnd v0.19
Status: In Progress
Milestone
v0.19.0
Development

Successfully merging a pull request may close this issue.

pagination: add pagination to wallet transactions Abdulkbk/lnd
6 participants
@mrfelton @alexbosworth @Roasbeef @mikuhl-dev @saubyk @Abdulkbk