Skip to content

refactor recipes#273

Merged
lazarusA merged 44 commits into
mainfrom
la/refactor_recipes
Jul 2, 2026
Merged

refactor recipes#273
lazarusA merged 44 commits into
mainfrom
la/refactor_recipes

Conversation

@lazarusA

@lazarusA lazarusA commented Jul 2, 2026

Copy link
Copy Markdown
Member

supersedes #264

Modular "Lego-like" Training Dashboards

This PR introduces a major refactor to the training dashboard system, decoupling the previously monolithic train_dashboard figure into modular, "Lego-like" components. This enables fine-grained control over which plots are rendered, how they are arranged, and which components are saved as animations.

Key Changes

  • Modular Builders: The giant train_dashboard and update_step_dashboard! functions in EasyHybridMakie.jl have been replaced with build_dashboards and update_step_dashboards!. These new builder functions can assemble a subset of plots or split them into independent interactive windows.
  • TrainConfig Extensions: Added three new configuration fields to control dashboard behavior (all defaulting to behavior that mimics the original dashboard):
    • dashboard_components: Controls which plots are rendered (e.g. [:loss, :prediction, :timeseries, :monitor]).
    • split_dashboard: A boolean controlling whether the dashboard components are drawn on a single Figure grid layout or spawned as independent Figure windows.
    • save_animations: Controls which dashboard windows to record as .mp4 video streams (e.g. [:loss], or [:all]).
  • Flexible Stream Management: The internal record_or_run tracking inside src/training/dashboard.jl has been upgraded to manage a dictionary of active Makie VideoStream objects (allowing multiple streams if split_dashboard = true).
  • Makie Recipe Fixes: Relaxed overly strict ::Makie.GridPosition types in LossPlot and MonitorPlot legend dispatches to properly accept GridSubposition elements for the new hierarchical layouts.

Backward Compatibility

The old default behavior is unchanged. If no new configuration fields are provided in TrainConfig, EasyHybrid behaves exactly as it did before—rendering all four plot types natively into a single interactive 2x2 grid window and recording a single .mp4 animation if save_training is enabled.

Usage Guide

You can now use TrainConfig to customize the visualization of your training loop:

Example 1: Minimal Dashboard

You can pass the new configuration options directly as keyword arguments to train(). If you are only interested in tracking the training loss and seeing the real-time predictions, you can isolate those components:

train(model, df, ...;
    dashboard_components = [:loss, :prediction],
    split_dashboard = false # (Default) Renders side-by-side in one window
)

Alternatively, you can construct a TrainConfig object explicitly:

train_cfg = TrainConfig(
    dashboard_components = [:loss, :prediction],
    split_dashboard = false
)

train(model, df, ...; train_cfg)

Example 2: Split Dashboard with Specific Animations

If you have a multi-monitor setup and prefer each graph to be in its own large detached window, you can enable split_dashboard. You can also choose to only record animations for specific plots (like the loss curve) to save rendering time and storage:

train_cfg = TrainConfig(
    dashboard_components = [:loss, :prediction, :timeseries, :monitor],
    split_dashboard = true, # Each component spawns in its own standalone window
    save_animations = [:loss] # Only the loss figure is saved as an mp4
)

train(model, df, ...; train_cfg)

Example 3: Save All Split Animations

To create separate videos for every single plotted metric:

train_cfg = TrainConfig(
    dashboard_components = [:loss, :prediction, :timeseries, :monitor],
    split_dashboard = true,
    save_animations = [:all] # Records multiple .mp4 files, one for each component
)

train(model, df, ...; train_cfg)

lazarusA and others added 30 commits March 24, 2026 17:11
* git fetch-depth 0

* changelog fetch
Co-authored-by: CompatHelper Julia <compathelper_noreply@julialang.org>
* gpu, cpu devices

* pfuff, of course symbol or variable indexing will not work on gpu, that needs to be done in the outer loop, refactoring genericHybrid is needed for that

* GPU support

* All tests passing, rewrote mask/nan checks.

* gdev only on batch level, some fixes to dimdata

* dimdata works with gpu

* cleaned

* Dimensional data

* formatting

* cleaned commented code

* MultiNN Support

* Is this necessary?

* tests

* computers are magic, I guess

* to_tuple not needed anymore

* lstm gpu changes -> does break other things?

* GPUArraysCore add

* fallback for functions and callables

* toDataFrame for LSTM

* runic

* local docu

* gpu tutorial

* ne Lux

* working on gpu cluster?

* gpu example - here gpu slower

* new tune and tutorial gpu

* lets try macOS for docs build

* quotes

* we need names, not sure tune works like this

* tune api adapted

* some fixes

* smaller subset bigger NN

* runic + docu tune

* fair timing?

* save_training

* fix cond

* nothing

* tweaks

* benchmarks

* separate lines and runic

* bigger NN

* get rid of gc?

* don't mention gpu for now

---------

Co-authored-by: qfl3x <qfl3x@protonmail.com>
Co-authored-by: Bernhard Ahrens <bahrens@bgc-jena.mpg.de>
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

📚 Documentation preview 🚀

Preview URL: https://EarthyScience.github.io/EasyHybrid.jl/previews/PR273/

Note

The preview will be available once the documentation build completes successfully, and will reflect the last successful build for this PR.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the training dashboard and plotting system in EasyHybrid.jl by introducing modular Makie recipes (LossPlot, MonitorPlot, PredictionPlot, and TimeSeriesPlot), updating the TrainDashboard structure, and adding configuration options for dashboard customization. Feedback on these changes identifies several critical issues: renaming io to streams in src/training/train_optimization.jl leads to an UndefVarError on an unmodified line, and several calls to update_dashboard! and update! on EarlyStopping will throw MethodErrors due to signature mismatches. Additionally, the use of newer Makie.update! features requires bumping the Makie compatibility bound to 0.24 in Project.toml, and setting the initial epoch snapshot's epoch to 0.9 instead of 0 introduces type instability.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/training/train_optimization.jl
Comment thread src/training/train_optimization.jl Outdated
Comment thread src/training/train_optimization.jl Outdated
Comment thread ext/recipes/LossPlot.jl
Comment thread src/training/initialization.jl Outdated
@lazarusA lazarusA merged commit ee5bcca into main Jul 2, 2026
13 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants