Skip to content

Conversation

jcnelson
Copy link
Member

@jcnelson jcnelson commented Aug 11, 2025

Description

Implements #6350 by re-implementing WritableMarfStore as an enum of a persistent disk-backed variant (the one we currently have), and a new ephemeral variant in which writes are committed to a RAM-backed Sqlite store, and in which reads will hit both the in-RAM sqlite store and an internal ReadOnlyMarfStore handle. This feature will be used to allow the node to replay blocks without the possibility of committing them to disk.

Applicable issues

Additional info (benefits, drawbacks, caveats)

A caveat: blocks produced with the ephemeral variant will not have the same state root hash as they would have had they been produced with the persistent variant. This is a consequence of the fact that the new block's MARF key/value pairs and side-storage live in a separate MARF, whose trie is not linked to the disk-backed parent trie. This is acceptable because the purpose of this PR is to allow the node to replay already-processed blocks in order to extract the transaction receipts. The node already knows the "true" state root hash.

This PR is a draft until it has enough test coverage.

Checklist

  • Test coverage for new or modified code paths
  • Changelog is updated
  • Required documentation changes (e.g., docs/rpc/openapi.yaml and rpc-endpoints.md for v2 endpoints, event-dispatcher.md for new events)
  • New clarity functions have corresponding PR in clarity-benchmarking repo
  • New integration test(s) added to bitcoin-tests.yml

@jcnelson jcnelson requested a review from rdeioris August 11, 2025 14:48
@aldur aldur linked an issue Aug 11, 2025 that may be closed by this pull request
@jcnelson jcnelson marked this pull request as ready for review August 21, 2025 01:40
@jcnelson jcnelson requested review from a team as code owners August 21, 2025 01:40
@jcnelson jcnelson requested review from kantai and obycode August 21, 2025 01:41
jcnelson and others added 4 commits August 21, 2025 04:10
…ich combine to form a WritableMarfStore trait. This factoring consolidates code between PersistentWritableMarfStore and EphemeralMarfStore and separates transaction lifecycle concerns from the act of writing data.
Copy link
Contributor

@kantai kantai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM, just a comment about DRY'ing up the getter functions.

Copy link
Contributor

@obycode obycode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a few nits.

@rdeioris
Copy link
Contributor

Looks very good to me and works flawlessly with my block simulator branch. I will push that branch (based on this) in a couple of days (while the other comments are addressed)

@jcnelson jcnelson requested review from obycode and kantai September 16, 2025 01:43
kantai
kantai previously approved these changes Sep 16, 2025
obycode
obycode previously approved these changes Sep 16, 2025
Copy link
Contributor

@obycode obycode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

rdeioris
rdeioris previously approved these changes Sep 17, 2025
@jcnelson jcnelson dismissed stale reviews from rdeioris, obycode, and kantai via 19c4b0e September 17, 2025 17:05
@jcnelson
Copy link
Member Author

ping @kantai @rdeioris @obycode now that merge conflicts have been fixed

Copy link
Contributor

@obycode obycode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jcnelson jcnelson added this pull request to the merge queue Sep 17, 2025
Merged via the queue into stacks-network:develop with commit 22af7d9 Sep 17, 2025
300 of 301 checks passed
@jcnelson jcnelson deleted the feat/shadow-marf branch September 17, 2025 18:50
Copy link

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 25, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Chainstate] Add support for an ephemeral MARF
5 participants