daviddoji/Fastbook_Julia
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Ch16 Fixes (#322)

Browse Source 
* Fix one typo.

* Fix second typo.

* Fix additional typos.
This commit is contained in:
Jakub Duchniewicz 2020-11-29 15:20:39 +01:00 committed by GitHub
parent 665c238467
commit f4c1ec8d82
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 11 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
16_accel_sgd.ipynb
View File

@ -443,7 +443,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"It's working! So that's how we create SGD from scratch in fastai. Now let's see see what \"momentum\"."
"It's working! So that's how we create SGD from scratch in fastai. Now let's see what \"momentum\" is."
]
},
{
@ -1009,7 +1009,7 @@
"finally: self('after_batch')\n",
"```\n",
"\n",
"The calls of the form `self('...')` are where the callbacks are called. As you see, this happens after every step. The callback will receive the entire state of training, and can also modify it. For instance, the input data and target labels are in `self.xb` and `self.yb`, respectively; a callback can modify these to \\the data the training loop sees. It can also modify `self.loss`, or even the gradients.\n",
"The calls of the form `self('...')` are where the callbacks are called. As you see, this happens after every step. The callback will receive the entire state of training, and can also modify it. For instance, the input data and target labels are in `self.xb` and `self.yb`, respectively; a callback can modify these to alter the data the training loop sees. It can also modify `self.loss`, or even the gradients.\n",
"\n",
"Let's see how this work in practice by writing a callback."
]
@ -1049,7 +1049,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's take a look at an example. Do you recall how in <<chapter_nlp_dive>> we needed to ensure that our special `reset` method was called at the start of training and validation for each epoch? We used the `ModelResetter` callback provided by fastai to do this for us. But how does it owrk? Here's the full source code for that class:"
"Let's take a look at an example. Do you recall how in <<chapter_nlp_dive>> we needed to ensure that our special `reset` method was called at the start of training and validation for each epoch? We used the `ModelResetter` callback provided by fastai to do this for us. But how does it work? Here's the full source code for that class:"
]
},
{
@ -1183,11 +1183,11 @@
"source": [
"The line `raise CancelFitException` tells the training loop to interrupt training at this point. The training loop catches this exception and does not run any further training or validation. The callback control flow exceptions available are:\n",
"\n",
"- `CancelFitException`:: Skip the rest of this batch and go to `after_batch`.\n",
"- `CancelEpochException`:: Skip the rest of the training part of the epoch and go to `after_train`.\n",
"- `CancelTrainException`:: Skip the rest of the validation part of the epoch and go to `after_validate`.\n",
"- `CancelValidException`:: Skip the rest of this epoch and go to `after_epoch`.\n",
"- `CancelBatchException`:: Interrupt training and go to `after_fit`."
"- `CancelBatchException`:: Skip the rest of this batch and go to `after_batch`.\n",
"- `CancelTrainException`:: Skip the rest of the training part of the epoch and go to `after_train`.\n",
"- `CancelValidException`:: Skip the rest of the validation part of the epoch and go to `after_validate`.\n",
"- `CancelEpochException`:: Skip the rest of this epoch and go to `after_epoch`.\n",
"- `CancelFitException`:: Interrupt training and go to `after_fit`."
]
},
{
@ -1197,8 +1197,8 @@
"You can detect if one of those exceptions has occurred and add code that executes right after with the following events:\n",
"\n",
"- `after_cancel_batch`:: Reached immediately after a `CancelBatchException` before proceeding to `after_batch`\n",
"- `after_cancel_train`:: Reached immediately after a `CancelTrainException` before proceeding to `after_epoch`\n",
"- `after_cancel_valid`:: Reached immediately after a `CancelValidException` before proceeding to `after_epoch`\n",
"- `after_cancel_train`:: Reached immediately after a `CancelTrainException` before proceeding to `after_train`\n",
"- `after_cancel_valid`:: Reached immediately after a `CancelValidException` before proceeding to `after_valid`\n",
"- `after_cancel_epoch`:: Reached immediately after a `CancelEpochException` before proceeding to `after_epoch`\n",
"- `after_cancel_fit`:: Reached immediately after a `CancelFitException` before proceeding to `after_fit`"
]
@ -1221,7 +1221,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"In this chapter we took a close look at the training loop, explorig differnet variants of SGD and why they can be more powerful. At the time of writing developping new optimizers is a very active area of research, so by the time you read this chapter there may be an addendum on the book's website that presents new variants. Be sure to check out how our general optimizer framework can help you implement new optimizers very quickly.\n",
"In this chapter we took a close look at the training loop, exploring different variants of SGD and why they can be more powerful. At the time of writing developing new optimizers is a very active area of research, so by the time you read this chapter there may be an addendum on the book's website that presents new variants. Be sure to check out how our general optimizer framework can help you implement new optimizers very quickly.\n",
"\n",
"We also examined the powerful callback system that allows you to customize every bit of the training loop by enabling you to inspect and modify any parameter you like between each step."
]