[Feature/REST] Pagination by index for address|scripthash txs/chain endpoint

[junderw]

Mempool (which relies on this endpoint for its address pages) currently can only implement "next page" on the address page since there is no way to skip a certain number of transactions.

ie. If an address has 100 transactions, right now it shows the first 25 and you have to click "show more" and it fetches the next 25 with the last_seen_txid of the last tx on the current page.

With this patch, they can provide links numbered 1, 2, 3, 4, and clicking on 3 will pass "50" instead of the hex txid, which will skip the first 50 transactions and display starting from the 51st transaction.

If there are other endpoints you'd like to be index-paginated where they used to be hex id paginated, we can do something similar as well.

Just submitting my initial impl to get feedback.

[shesek]

Hi @junderw, thanks for writing the PR!

There were three reasons I implemented pagination using the 'last seen' strategy rather than index numbers:

1. Consistency - the set of transactions for a given address may change as the user is browsing it (its sorted by newest first, so new transactions are prepended to the beginning). With index-based paging, clicking 'Next' while a new block arrives could result in inconsistent results, where some transactions are missing while others are duplicated in the UI.

   (Note this guarantee may sometimes break due to reorgs even with 'last seen', but at least that only happens in the exceptional case and not every time there's a new block.)

2. Cacheability - pages that are identified by a start index cannot be cached because they're expected to change. OTOH, a page that is identified by the 'last seen' txid is static and may be cached.

3. Performance - it's theoretically possible to use the last seen txid to seek directly to the position of the db row following it, instead of starting at the first row and skipping the first N. However, this is currently not implemented that way (and would require including the last seen block height in addition to the txid).

Pagination is not available for mempool transactions at all for similar reasons, since the entire tx set may change at any time and there's no reasonably consistent way to paginate it. You can only get the "first" MAX_MEMPOOL_TXS mempool transactions (where "first" is undefined/arbitrary), with paging only being made available for confirmed transactions.

Can you tell me more about your need for index-based paging? Is that so its possible to link directly to the Nth page instead of only the next/previous page?

[junderw]

Is that so its possible to link directly to the Nth page instead of only the next/previous page?

This is precisely the feature we're looking for.

As a user with an address that has 100 pages of transactions, and I want to look for one of my first transactions, the current method requires skipping through 100 clicks and loads of the "next" button etc.

Whereas clicking a "last" or "99" button will just pass along 98 * 25 (2450) to the endpoint to get the transactions.

People who hit "next" or "2" when on "1" we can use the last seen feature as is. Usually people jumping 10-20 pages don't care if a transaction bumped everything down 1 while they reload, since clicking on "page 87" is not a precise window to begin with.