Skip to content

Z3: optimizations for grad norm calculation and gradient clipping#5504

Merged
loadams merged 25 commits intodeepspeedai:masterfrom nelyahu:zero3_scaled_globalAug 15, 2024
+6-5

Commits

Commits on May 9, 2024

Commits on May 20, 2024

Commits on May 23, 2024

Commits on May 27, 2024

Commits on May 30, 2024

Commits on Jun 7, 2024

Commits on Jun 17, 2024

Commits on Jun 26, 2024

Commits on Jun 27, 2024

Commits on Jul 9, 2024

Commits on Jul 18, 2024

Commits on Jul 23, 2024

Commits on Jul 24, 2024

Commits on Aug 4, 2024

Commits on Aug 6, 2024

Commits on Aug 9, 2024

Commits on Aug 14, 2024