fullhistory: scope the txhash tuning knobs to the txhash CF (#838)#849
Draft
chowbao wants to merge 2 commits into
Draft
fullhistory: scope the txhash tuning knobs to the txhash CF (#838)#849chowbao wants to merge 2 commits into
chowbao wants to merge 2 commits into
Conversation
Grow CFOptions with the memtable, compaction, and bloom knobs that were DB-wide fields on Tuning, and route the bloom filter through the per-CF override. Tuning now carries only the genuinely DB-wide resources shared across every CF of one store: block cache, WAL cap, background jobs, and open-file cap. This lets a facade scope a workload's tuning to the CF that has that workload instead of leaking it onto every CF in the shared per-chunk DB.
Split txhash's config into the DB-wide Tuning it always set (block cache, WAL cap, background jobs, open files) and a new CFOptions carrying its per-CF workload knobs (bloom, write buffers, compaction triggers). The hotchunk opener merges the per-CF options from every facade so those knobs land on the txhash CF alone. Restores each store's pre-unification standalone tuning: the ledgers CF runs with no bloom filter (it is never probed for missing keys) and default memtables/compaction; the events CFs keep their ZSTD and block-size overrides; the txhash CF keeps its full calibration. The DB-wide resources stay shared, sized to what the three standalone instances used together (only txhash set them).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #838.
The shared hot-chunk config applied txhash's
Tuning— 12-bit blooms, 64 MB write buffers, the 999/999/999 no-compaction triggers — to the ledgers and events CFs too; the per-CF overrides reclaimed only compression and block size. The ledgers CF paid for a bloom filter nothing consults (it is never probed for missing keys), and every CF's memtables were sized for txhash's write pattern.Commit 1 grows
rocksdb.CFOptionswith the memtable/compaction/bloom knobs that were DB-wideTuningfields (each still zero-means-default), and moves the bloom construction to the per-CF override.Tuningnow carries only the genuinely DB-wide resources: block cache, WAL cap, background jobs, open-file cap. The one-shared-NewBloomFilter-moved-into-the-first-BBTO hazard doesn't regress: the filter is still constructed fresh per CF, now only for CFs that opt in.Commit 2 scopes the values. Per the issue's sharpened scope this is a mechanical port of each store's pre-unification standalone tuning (verified against
c445557d^, the parent of the multi-CF collapse):DisableAutoCompactions, 64 MB target files,MaxBytesForLevelBase256 MBTuningat all)DB-wide consolidation: the standalone txhash instance was the only one that set the shared resources (512 MB block cache, 1 GB WAL cap, 8 background jobs, 10k open files; ledger/events rode on defaults), so keeping txhash's values on the shared DB is exactly the three instances' combined footprint — no re-sizing needed.
Tests:
TestConfig_PerCFOptionRoutingpins the routing at the config-construction seam (txhash CF keeps bloom/buffers/no-compaction; ledgers CF gets none of them; events keeps ZSTD/block size, no bloom),TestConfig_DBWideTuningStaysSharedpins the shared values, and the existing rocksdb round-trip test now exercises the knobs throughPerCFOptions. Fullfullhistory-shortsuite and-raceonrocksdb/hotchunkpass.