Skip to content

DeepSpeed v0.3.16
Compare
Choose a tag to compare
@jeffra jeffra released this 30 Apr 19:40
· 2108 commits to master since this release
v0.3.16
a12aaed

v0.3.16 Release notes

  • Full precision (fp32) support for ZeRO Stage2 and Stage3 (#1004)
  • 1-bit LAMB optimizer (#970)
  • Refactor param_dict to config (#1008)
  • [ZeRO Infinity] Allow Init to take a dict for the deepspeed config (#983)
  • Bug fix for zero-1 w.r.t. reduce-scatter (#907)
  • Use amp autocast in ZeRO3 linear (#990)
  • Add find_unused_parameters option to DeepSpeedEngine (#945)
  • Relax dataset type check in deepspeed io (#1012)

Special thanks to our contributors: @stas00, @SeanNaren, @sdtblck, @wbuchwalter, @ghosthamlet, @zhujiangang,

Assets 2
Loading