Skip to content

feat: introdue reth era export #15909

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 21 commits into
base: main
Choose a base branch
from

Conversation

lean-apple
Copy link
Contributor

@lean-apple lean-apple commented Apr 24, 2025

Closes #15495.

Only start introducing the functional logic (not the interface with the cli) :

For a certain number of block set up in the config, it fetches the data (header, block body, receipts) to reconstruct the block history and write them directly into era files.

Still to do :

  • Add writing logic,
  • Add tests with import/export rountrip --> started but not finished

@lean-apple lean-apple marked this pull request as ready for review April 28, 2025 06:22
Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome, this looks pretty good!

I think we should considering merging both import and export into one crate and rename era-import @RomanHodulak

and I believe we're missing tx.commit calls in import @RomanHodulak

we could also handle those externally but maybe a better idea to commit file after file?

@RomanHodulak
Copy link
Collaborator

RomanHodulak commented Apr 28, 2025

@mattsse @lean-apple

I think merging and renaming the crates fits.

Commit per ERA file seems best for this use-case, i.e. the file remains imported even if the process is interupted

Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool, only thing missing really is a rebase and moving a few things around

@lean-apple lean-apple requested a review from onbjerg as a code owner May 7, 2025 04:46
Copy link

codspeed-hq bot commented May 8, 2025

CodSpeed Performance Report

Merging #15909 will degrade performances by 10.84%

Comparing lean-apple:reth-era-export (8c9ef87) with main (da95e57)

Summary

❌ 1 regressions
✅ 76 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
hash builder[init size 10000 | update size 100 | num updates 1] 9.1 ms 10.2 ms -10.84%

@github-actions github-actions bot added the S-stale This issue/PR is stale and will close with no further activity label May 30, 2025
@yorickdowne
Copy link
Contributor

@mattsse I see some era1 in the code, I didn’t see era. I might just be missing it, or this is intended to be era1. To be cautious and mention it here:

  • Era1 files are pre-merge
  • Era files are post-merge
  • They have a different format
  • With full 4444 coming, era export (post merge) will be a thing. This will use two file repositories for chains with pre-merge history, era1 and era. And one repository for chains that were merged at genesis, just era.

@lean-apple
Copy link
Contributor Author

@mattsse I see some era1 in the code, I didn’t see era. I might just be missing it, or this is intended to be era1. To be cautious and mention it here:

  • Era1 files are pre-merge
  • Era files are post-merge
  • They have a different format
  • With full 4444 coming, era export (post merge) will be a thing. This will use two file repositories for chains with pre-merge history, era1 and era. And one repository for chains that were merged at genesis, just era.

Hello @yorickdowne :)

Will let @mattsse complete but from what I understood :

This pr is addressing this feature #15495 that is part of this more global tracking issue #13186 only covering pre-merge history.

Until now only .era1 (and more globally .e2s file) is covered, as you can see https://github.com/paradigmxyz/reth/blob/main/crates/era/src/era1_file.rs, but I guess in a second time, another tracking-issue will cover post merge (@mattsse ?).

I guess also we use era as an shorthand when referring to era1 file features, which might be causing some confusion in the naming.

@github-actions github-actions bot removed the S-stale This issue/PR is stale and will close with no further activity label Jun 3, 2025
@lean-apple
Copy link
Contributor Author

Sorry wont have time to continue.

@lean-apple lean-apple closed this Jun 3, 2025
@github-project-automation github-project-automation bot moved this from In Progress to Done in Reth Tracker Jun 3, 2025
@mattsse mattsse reopened this Jun 3, 2025
@github-project-automation github-project-automation bot moved this from Done to In Progress in Reth Tracker Jun 3, 2025
@mattsse mattsse added the M-prevent-stale Prevents old inactive issues/PRs from being closed due to inactivity label Jun 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
M-prevent-stale Prevents old inactive issues/PRs from being closed due to inactivity
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

Add support for exporting history as era store
4 participants