This commit is contained in:
Sylvain Gugger
2020-05-18 18:18:45 -07:00
parent d8d39c560a
commit 4b1345a068
20 changed files with 276 additions and 287 deletions

View File

@@ -983,7 +983,7 @@
"source": [
"#hide\n",
"# !pip install voila\n",
"# !jupyter serverextension enable voila --sys-prefix"
"# !jupyter serverextension enable voila sys-prefix"
]
},
{

View File

@@ -266,7 +266,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Section 1: That's a Wrap!"
"## Deep Learning in Practice: That's a Wrap!"
]
},
{

View File

@@ -1719,7 +1719,7 @@
"\n",
"1. Take a look at all the differences between the `Embedding` version of `DotProductBias` and the `create_params` version, and try to understand why each of those changes is required. If you're not sure, try reverting each change to see what happens. (NB: even the type of brackets used in `forward` has changed!)\n",
"1. Find three other areas where collaborative filtering is being used, and find out what the pros and cons of this approach are in those areas.\n",
"1. Complete this notebook using the full MovieLens dataset, and compare your results to online benchmarks. See if you can improve your accuracy. Look on the book's website and the fast.ai forum for ideas. Note that there are more columns in the full dataset--see if you can use those too (the next chapter might give you ideas).\n",
"1. Complete this notebook using the full MovieLens dataset, and compare your results to online benchmarks. See if you can improve your accuracy. Look on the book's website and the fast.ai forum for ideas. Note that there are more columns in the full datasetsee if you can use those too (the next chapter might give you ideas).\n",
"1. Create a model for MovieLens that works with cross-entropy loss, and compare it to the model in this chapter."
]
},

View File

@@ -6859,7 +6859,7 @@
"outputs": [],
"source": [
"#hide\n",
"# pip install --pre -f https://sklearn-nightly.scdn8.secure.raxcdn.com scikit-learn --U"
"# pip install pre -f https://sklearn-nightly.scdn8.secure.raxcdn.com scikit-learn U"
]
},
{

View File

@@ -851,7 +851,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Becoming a Deep Learning Practitioner"
"## Understanding fastai's Applications: Wrap Up"
]
},
{

View File

@@ -23,7 +23,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Let's Start with SGD"
"## Establishing a Baseline"
]
},
{
@@ -687,7 +687,7 @@
"1. What does `zero_grad` do in an optimizer?\n",
"1. What does `step` do in an optimizer? How is it implemented in the general optimizer?\n",
"1. Rewrite `sgd_cb` to use the `+=` operator, instead of `add_`.\n",
"1. What is momentum? Write out the equation.\n",
"1. What is \"momentum\"? Write out the equation.\n",
"1. What's a physical analogy for momentum? How does it apply in our model training settings?\n",
"1. What does a bigger value for momentum do to the gradients?\n",
"1. What are the default values of momentum for 1cycle training?\n",
@@ -695,18 +695,18 @@
"1. What do the squared values of the gradients indicate?\n",
"1. How does Adam differ from momentum and RMSProp?\n",
"1. Write out the equation for Adam.\n",
"1. Calculate the value of `unbias_avg` and `w.avg` for a few batches of dummy values.\n",
"1. What's the impact of having a high eps in Adam?\n",
"1. Calculate the values of `unbias_avg` and `w.avg` for a few batches of dummy values.\n",
"1. What's the impact of having a high `eps` in Adam?\n",
"1. Read through the optimizer notebook in fastai's repo, and execute it.\n",
"1. In what situations do dynamic learning rate methods like Adam change the behaviour of weight decay?\n",
"1. In what situations do dynamic learning rate methods like Adam change the behavior of weight decay?\n",
"1. What are the four steps of a training loop?\n",
"1. Why is the use of callbacks better than writing a new training loop for each tweak you want to add?\n",
"1. What are the necessary points in the design of the fastai's callback system that make it as flexible as copying and pasting bits of code?\n",
"1. Why is using callbacks better than writing a new training loop for each tweak you want to add?\n",
"1. What aspects of the design of fastai's callback system make it as flexible as copying and pasting bits of code?\n",
"1. How can you get the list of events available to you when writing a callback?\n",
"1. Write the `ModelResetter` callback (without peeking).\n",
"1. How can you access the necessary attributes of the training loop inside a callback? When can you use or not use the shortcut that goes with it?\n",
"1. How can you access the necessary attributes of the training loop inside a callback? When can you use or not use the shortcuts that go with them?\n",
"1. How can a callback influence the control flow of the training loop.\n",
"1. Write the `TerminateOnNaN` callback (without peeking if possible).\n",
"1. Write the `TerminateOnNaN` callback (without peeking, if possible).\n",
"1. How do you make sure your callback runs after or before another callback?"
]
},
@@ -721,8 +721,8 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"1. Look up the \"rectified Adam\" paper and implement it using the general optimizer framework, and try it out. Search for other recent optimizers that work well in practice, and pick one to implement.\n",
"1. Look at the mixed precision callback with the documentation. Try to understand what each event and line of code does.\n",
"1. Look up the \"Rectified Adam\" paper, implement it using the general optimizer framework, and try it out. Search for other recent optimizers that work well in practice, and pick one to implement.\n",
"1. Look at the mixed-precision callback with the documentation. Try to understand what each event and line of code does.\n",
"1. Implement your own version of ther learning rate finder from scratch. Compare it with fastai's version.\n",
"1. Look at the source code of the callbacks that ship with fastai. See if you can find one that's similar to what you're looking to do, to get some inspiration."
]
@@ -738,11 +738,11 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Congratulations, you have made it to the end of the \"foundations of deep learning\" section. You now understand how all of fastai's applications and most important architectures are built, and the recommended ways to train them, and have all the information you need to build these from scratch. Whilst you probably won't need to create your own training loop, or batchnorm layer, for instance, knowing what is going on behind the scenes is very helpful for debugging, profiling, and deploying your solutions.\n",
"Congratulations, you have made it to the end of the \"foundations of deep learning\" section of the book! You now understand how all of fastai's applications and most important architectures are built, and the recommended ways to train themand you have all the information you need to build these from scratch. While you probably won't need to create your own training loop, or batchnorm layer, for instance, knowing what is going on behind the scenes is very helpful for debugging, profiling, and deploying your solutions.\n",
"\n",
"Since you understand all of the foundations of fastai's applications now, be sure to spend some time digging through fastai's source notebooks, and running and experimenting with parts of them, since you can and see exactly how everything in fastai is developed.\n",
"Since you understand the foundations of fastai's applications now, be sure to spend some time digging through the source notebooks and running and experimenting with parts of them. This will give you a better idea of how everything in fastai is developed.\n",
"\n",
"In the next section, we will be looking even further under the covers, to see how the actual forward and backward passes of a neural network are done, and we will see what tools are at our disposal to get better performance. We will then finish up with a project that brings together everything we have learned throughout the book, which we will use to build a method for interpreting convolutional neural networks."
"In the next section, we will be looking even further under the covers: we'll explore how the actual forward and backward passes of a neural network are done, and we will see what tools are at our disposal to get better performance. We will then continue with a project that brings together all the material in the book, which we will use to build a tool for interpreting convolutional neural networks. Last but not least, we'll finish by building fastai's `Learner` class from scratch."
]
},
{