Code

Improve consistency of zero_grad#6554

Code

Draft

tohtana wants to merge 10 commits intomastermicrosoft/DeepSpeed:masterfrom tohtana/consistent_zero_gradmicrosoft/DeepSpeed:tohtana/consistent_zero_gradCopy head branch name to clipboard

+90-53

Commits

Commits on Sep 18, 2024

• fix gradient accumulation for z2+offload

  tohtana

  committed
  1848983

  Copy full SHA for 1848983

• Merge branch 'master' into tohtana/fix_grad_acc_z2_offload

  tjruwase

  authored
  3a37b03

  Copy full SHA for 3a37b03

• improve consistency of zero_grad

  tohtana

  committed
  84ca923

  Copy full SHA for 84ca923

• Merge branch 'tohtana/fix_grad_acc_z2_offload' into tohtana/consistent_zero_grad

  tohtana

  committed
  1b095d4

  Copy full SHA for 1b095d4

• fix zero_grad for z1/2 optimizer

  tohtana

  committed
  fea4811

  Copy full SHA for fea4811

• remove type hint for python 3.7

  tohtana

  committed
  b170196

  Copy full SHA for b170196

• remove type hint

  tohtana

  committed
  86da91d

  Copy full SHA for 86da91d

• remove type hint

  tohtana

  committed
  4a59a2b

  Copy full SHA for 4a59a2b

• fix for non-zero optimizer

  tohtana

  committed
  958dfc1

  Copy full SHA for 958dfc1

Commits on Sep 27, 2024

• Merge branch 'master' into tohtana/consistent_zero_grad

  tohtana

  authored
  6e05d2c

  Copy full SHA for 6e05d2c