feat(fct_block_head): gloas payload hash from execution_payload events - #298
Closed
samcm wants to merge 4 commits into
Closed
feat(fct_block_head): gloas payload hash from execution_payload events#298samcm wants to merge 4 commits into
samcm wants to merge 4 commits into
Conversation
…payload events On gloas (ePBS) networks the beacon block no longer embeds the execution payload, so fct_block_head.execution_payload_block_hash is empty and every snooper observation was dropped for lack of slot context - the engine timings tables stayed empty on glamsterdam devnets even with raw data flowing. Map each revealed payload's block hash to its beacon block root via the beacon_api_eth_v1_events_execution_payload SSE events and coalesce that into block_context. The events table only exists on networks running gloas-era xatu, so it is referenced through EXTERNAL_CLUSTER behind a GLOAS_PAYLOAD_EVENTS_DATABASE env gate instead of a declared dependency: deployments opt in by setting the env to the raw database holding the table, and everywhere else the template renders the exact same SQL as before. Verified against live data: glamsterdam-devnet-7 correlates 99.7% of 23k engine events over 2h (42 nodes); on mainnet the gated-off rendering returns checksum-identical results to the current model. Claude-Session: https://claude.ai/code/session_01J2ShnSwGwH4iZuXXwbLgvP
…ndency Correlation now works on every network with no opt-in: the execution_payload events external is a declared dependency, OR-grouped with beacon_api_eth_v2_beacon_block so networks where the table is empty or absent schedule unaffected, and the coalesce prefers the block's own embedded payload hash so mixed pre/post-gloas histories work in one query. Requires the gloas event-table migrations to exist on a network's raw database before this model rolls there; the glamsterdam devnets already have them. Claude-Session: https://claude.ai/code/session_01J2ShnSwGwH4iZuXXwbLgvP
samcm
added a commit
that referenced
this pull request
Jul 17, 2026
On gloas the beacon block no longer embeds the execution payload, so fct_block_head carries no execution_payload_block_hash and every engine snooper observation was dropped for lack of slot context. Map each revealed payload's block hash to its beacon block root via the execution_payload SSE events and coalesce it into block_context; the coalesce prefers the block's own embedded hash so pre-gloas history is untouched. The events dependency is OR-grouped with the beacon block table so networks without payload events schedule unaffected. Verified against live devnet-7 data: 99.7% of 23k engine events over 2h correlate to slots (42 nodes); on mainnet data the rendering returns checksum-identical results to the previous model. Same change is PR #298 against master, blocked there until the gloas raw tables exist beyond the devnets. Claude-Session: https://claude.ai/code/session_01J2ShnSwGwH4iZuXXwbLgvP
CBT's statement splitter cuts on semicolons without comment awareness. Claude-Session: https://claude.ai/code/session_01J2ShnSwGwH4iZuXXwbLgvP
The engine fct models correlate against fct_block_head independently of int_engine_new_payload, so the int-level patch left them empty on gloas. Coalesce the payload hash into fct_block_head from the execution_payload SSE events and revert the int model - every consumer of the column is repaired at once. Claude-Session: https://claude.ai/code/session_01J2ShnSwGwH4iZuXXwbLgvP
samcm
changed the title
Member
Author
|
Closing — this lives on release/glamsterdam-devnet-{6,7} (65c3415), which is what the devnet CBT deployments run, and it'll reach master when the release branch merges back as part of gloas prep. No point maintaining a separate master copy until then. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On gloas (ePBS) networks the beacon block no longer embeds the execution payload, so
fct_block_head.execution_payload_block_hashis empty — and every engine timing model correlates snooper events against that column, so theint_engine_*/fct_engine_*tables stayed empty on the glamsterdam devnets even with raw snooper data flowing.Fix at the source:
fct_block_headcoalesces the payload hash from thebeacon_api_eth_v1_events_execution_payloadSSE events (new external model, copied verbatim fromrelease/glamsterdam-devnet-7) when the block table carries none. Every consumer of the column is repaired at once — an earlier revision of this PR patchedint_engine_new_payloadinstead, but the fct engine models correlate independently, so the column had to be fixed where it's produced.beacon_api_eth_v2_beacon_block, so networks where the events table is empty or absent schedule exactly as before (CBT skips empty/erroring OR-group members).''arm keeps the FixedString(66) column non-NULL.;inside comments (found the hard way in prod).Tested against live ClickHouse via panda: on devnet-7, 299/299 blocks in the last hour resolve a payload hash through the enrichment; the identical rendered INSERT parses through ClickHouse's full analysis. This change is already live on
release/glamsterdam-devnet-{6,7}(65c3415) feeding the devnet deployments.Deploy sequencing for this master PR: the events table ships with gloas-era xatu (
007_gloas_epbs_support, on xatu release branches only today). The rendered SQL references the table unconditionally, so don't roll a master build containing this to networks whose raw DB lacks the table (mainnet/sepolia/hoodidefault) until that xatu migration lands there.Known follow-up (out of scope):
int_engine_get_blobscorrelates viabeacon_api_eth_v1_beacon_blob, which is empty under PeerDAS — needs a versioned-hash mapping from another source.https://claude.ai/code/session_01J2ShnSwGwH4iZuXXwbLgvP