Skip to content

fix(workflows): regenerate the results index once per run - #80

Open
spencer-tb wants to merge 5 commits into
masterfrom
index-once-per-run
Open

fix(workflows): regenerate the results index once per run#80
spencer-tb wants to merge 5 commits into
masterfrom
index-once-per-run

Conversation

@spencer-tb

@spencer-tb spencer-tb commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Test runs no longer randomly disappear. The results index is rebuilt once per workflow run instead of by ~21 racing matrix jobs, so a finished run can't be overwritten out of the listing.

Description

Every matrix job currently regenerates listing.jsonl at job end (fetch all suite JSONs from S3 → hiveview --listing → upload). With ~21 jobs per run the uploads race: last writer wins, and if its S3 snapshot predates another job's upload, that run disappears from the UI until the next regeneration.

This disables index generation on the matrix jobs and adds a single needs: test, if: always() index job per workflow, sharing a concurrency group with a generate-index dispatch for the same S3 path. Applied to the glamsterdam-devnet-8 and frames-devnet-0 workflows (same bug), and to generic.yaml — which never had the race (index generation is disabled there) but only got reindexed by the 3-hourly cron, so daily runs took up to 3h to appear.

Since the per-run index job only fires after the slowest matrix job (besu runs 20–31h), the generate-index cron now matrixes over all five results paths instead of just generic, capping index staleness at 3h for fast clients' uploads. The listing limit is raised to 5000 everywhere while at it — generic sits at ~2500 entries and would soon start silently dropping the oldest runs at the current 3000 cap.

@spencer-tb
spencer-tb marked this pull request as draft August 13, 2026 17:55
@spencer-tb
spencer-tb marked this pull request as ready for review August 18, 2026 16:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants