Skip to content

Implement CosineAnnealingWarmRestarts LR Scheduler#61

Merged
PatrickRMiles merged 14 commits intoLBANN:mainfrom
michaelmckinsey1:newsched
May 6, 2026
Merged

Implement CosineAnnealingWarmRestarts LR Scheduler#61
PatrickRMiles merged 14 commits intoLBANN:mainfrom
michaelmckinsey1:newsched

Conversation

@michaelmckinsey1
Copy link
Copy Markdown
Collaborator

@michaelmckinsey1 michaelmckinsey1 commented Apr 24, 2026 •
edited
Loading

  • Depends on Improve AMP stability #60
  • Since we sometimes get dice score stuck in local minima, this scheduler will periodically reset LR to jump out of minima

@michaelmckinsey1 michaelmckinsey1 self-assigned this Apr 24, 2026
@michaelmckinsey1 michaelmckinsey1 mentioned this pull request Apr 24, 2026
Closed
2 tasks
@michaelmckinsey1 michaelmckinsey1 mentioned this pull request Apr 30, 2026
Merged
3 tasks
@michaelmckinsey1 michaelmckinsey1 marked this pull request as ready for review April 30, 2026 22:53
@PatrickRMiles PatrickRMiles merged commit 9606bee into LBANN:main May 6, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@PatrickRMiles PatrickRMiles PatrickRMiles approved these changes

Assignees

@michaelmckinsey1 michaelmckinsey1

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@michaelmckinsey1 @PatrickRMiles