Fix optimizer, dataloader label mask, config isolation, and replay analysis bugs - #1
Open
cto-new[bot] wants to merge 2 commits into
Open
Fix optimizer, dataloader label mask, config isolation, and replay analysis bugs#1cto-new[bot] wants to merge 2 commits into
cto-new[bot] wants to merge 2 commits into
Conversation
…ay analysis bugs This change addresses several latent bugs and minor flaws identified during audit: - Fixes optimizer 'betas' initialization from separate arguments to tuple format (required by AdamW and Muon) in model_setup.py. - Corrects padding behavior for prompt encoding in dataset collators to prevent label mask misalignment in data_loader.py. - Fixes imports and type hints for Optional in evaluation.py, avoiding runtime issues. - Ensures config objects are deep-copied during optimizer comparison to avoid cross-contamination in main.py. - Fixes replay_analysis.py visualization functions to reference sample records correctly for plotting distributions. - Adds .gitignore for project artifacts and outputs. These changes improve reliability and reproducibility of training and evaluation.
- Replaces config loading and parameter handling with explicit, in-file initialization - Moves core data logic and pipeline into a single implementation file - Removes all YAML/config file dependencies for easier maintenance - Lays the groundwork for a more concise, demo-friendly training pipeline
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refactors the catastrophic forgetting project for clarity and maintainability by consolidating configuration and implementation into a single, demo-friendly Python file.
Details
Improves maintainability, demo-readiness, and testing clarity while preserving intended training and evaluation behaviors.