Repository navigation
Store forecasts by execution and fuse latest completed inputs - #1383
Draft
damonbayer wants to merge 22 commits into
Draft
damonbayer wants to merge 22 commits into
damonbayer wants to merge 22 commits into
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1383 +/- ##
==========================================
+ Coverage 82.32% 85.91% +3.58%
==========================================
Files 41 41
Lines 1703 1732 +29
==========================================
+ Hits 1402 1488 +86
+ Misses 301 244 -57
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Forecast reruns can overwrite earlier fits, and fusion inputs may be produced by separate executions. Store each execution separately and resolve completed inputs across executions so reruns preserve earlier results while downstream work can use the latest available fits.
Execution directories use
YYYY-MM-DD_forecasts/<run_timestamp>_<execution_id>/<location>/<disease>/<model>/. Dagster workers share a directory derived from the persisted start time inAmerica/New_Yorkand the run ID; standalone invocations use a UUID. Each execution savesrun_config.yamlandexecution.json. Fit metadata records training-window settings, and an empty_COMPLETEmarker is created only after all outputs are published successfully.find_fits(base, keys=..., keep=...)with"latest"(default),"earliest", and"all". Selection follows execution-directory names. Supplying keys limits lookups to those fit paths; single-fit selection stops once every requested key is found. Incomplete fits are ignored.FusionConfig.output_basedir; they do not use model resources or inherit upstream location/disease dimensions. Each discovers every location/disease pair with both required completed inputs, selects the latest fit for each input, checks forecast windows and coverage, and records input paths in its metadata. Inputs can come from different executions. Pairs with missing inputs are skipped, and fusion retains overlapping historical trajectories.output_basedir; location overrides still apply to training settings. Fusion and postprocessing discover inputs under their configured output root.00_summary/<disease>/, with one shared00_summary/sources.tsvcontaininglocation,disease,model, andpathcolumns. The numeric prefix places collated outputs before location folders when browsing by name. A summary can include fits from multiple executions; previously written summaries remain unchanged. Postprocessing does not execute fusion.Validation:
00_summary/sources.tsvlayout.all/earliest/latestselection, fusion input provenance, summary table row counts, and collated PDF page counts. The second summary retained US Fable and CA PyRenew from execution 1 while selecting CA Fable and fusion from execution 2.Closes #1315.
Closes #1300