From 47cc9180c607c2f19982f098ada9201f8b9a3bd5 Mon Sep 17 00:00:00 2001 From: Thomas Berends Date: Wed, 11 Sep 2024 20:59:46 +0200 Subject: [PATCH 1/2] Update 01_pytorch_workflow.ipynb --- 01_pytorch_workflow.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/01_pytorch_workflow.ipynb b/01_pytorch_workflow.ipynb index a38bc67a..78d95b22 100644 --- a/01_pytorch_workflow.ipynb +++ b/01_pytorch_workflow.ipynb @@ -880,7 +880,7 @@ ">\n", "> And on the ordering of things, the above is a good default order but you may see slightly different orders. Some rules of thumb: \n", "> * Calculate the loss (`loss = ...`) *before* performing backpropagation on it (`loss.backward()`).\n", - "> * Zero gradients (`optimizer.zero_grad()`) *before* stepping them (`optimizer.step()`).\n", + "> * Zero gradients (`optimizer.zero_grad()`) *before* computing the gradients of the loss with respect to every model parameter (`loss.backward()`).\n", "> * Step the optimizer (`optimizer.step()`) *after* performing backpropagation on the loss (`loss.backward()`).\n", "\n", "For resources to help understand what's happening behind the scenes with backpropagation and gradient descent, see the extra-curriculum section.\n" From 81a7b921a27dd2cd46acba9168294fa3ea2ad962 Mon Sep 17 00:00:00 2001 From: Thomas Berends Date: Wed, 11 Sep 2024 21:02:44 +0200 Subject: [PATCH 2/2] Update 01_pytorch_workflow.ipynb --- docs/01_pytorch_workflow.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/01_pytorch_workflow.ipynb b/docs/01_pytorch_workflow.ipynb index b9176f5e..a1112909 100644 --- a/docs/01_pytorch_workflow.ipynb +++ b/docs/01_pytorch_workflow.ipynb @@ -881,7 +881,7 @@ ">\n", "> And on the ordering of things, the above is a good default order but you may see slightly different orders. Some rules of thumb: \n", "> * Calculate the loss (`loss = ...`) *before* performing backpropagation on it (`loss.backward()`).\n", - "> * Zero gradients (`optimizer.zero_grad()`) *before* stepping them (`optimizer.step()`).\n", + "> * Zero gradients (`optimizer.zero_grad()`) *before* computing the gradients of the loss with respect to every model parameter (`loss.backward()`).\n", "> * Step the optimizer (`optimizer.step()`) *after* performing backpropagation on the loss (`loss.backward()`).\n", "\n", "For resources to help understand what's happening behind the scenes with backpropagation and gradient descent, see the extra-curriculum section.\n"