Update

10_nlp.ipynb

@@ -889,7 +889,9 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {},
+   "metadata": {
+    "hide_input": false
+   },
    "source": [
     "<img alt=\"TK: add title\" width=\"800\" caption=\"TK: add title\" id=\"TK: add it\" src=\"images/att_00071.png\">"
    ]
@@ -2192,6 +2194,22 @@
     "Many people assume or hope that algorithms will come to our defence here. The hope is that we will develop classification algorithms which can automatically recognise auto generated content. The problem, however, is that this will always be an arms race, in which better classification (or discriminator) algorithms can be used to create better generation algorithms."
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Conclusion"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We have looked at the last application covered out of the box by the fastai library: text. We have seen two types of models: language models that can generate texts, and a classifier that determines if a review is positive or negative. To build a state-of-the art classifier, we used a pretrained language model, fine-tuned it to the corpus of our task, then used its body with a new head to do the classification.\n",
+    "\n",
+    "Before we end this section, we will know look at how the fastai library can help you assemble your data on your specific problems."
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},

11_midlevel_data.ipynb

@@ -1204,7 +1204,57 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "TK conclusion and questionnaire"
+    "## Conclusion"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "fastai presents a layered API: it takes one line of code to grab the data when it's in one of the usual settings. It's then easy for beginners to focus on training a model without spending to much time assembling the data. Then the data block API gives you more flexibility by mixing and matching some building blocks. Undeneath it, the mid-level API gives you entire flexibility to apply any transformations on your items. In your real-wrold problems, this is probably what you will need to use, and we hope it makes the step of data-munging as easy as poassible."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Questionnaire"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "1. Why do we say that fastai has a layered API? What does it mean?\n",
+    "1. Why does a `Transform` have a decode method? What does it do?\n",
+    "1. Why does a `Transform` have a setup method? What does it do?\n",
+    "1. How does a `Transform` work when called on a tuple?\n",
+    "1. Which methods do you need to implement when writing your own `Transform`?\n",
+    "1. Write a `Normalize` transform that fully normalizes items (substract the mean and divide by the standard deviation of the dataset), and that can decode that behavior. Try not to peak!\n",
+    "1. Write a `Transform` that does the numericalization of tokenized texts (it should set its vocab automatically from the dataset seen and have a decode method). Look at the source code of fastai if you need help.\n",
+    "1. What is a `Pipeline`?\n",
+    "1. What is a `TfmdLists`? \n",
+    "1. What is a `Datasets`? How is it different from `TfmdLists`?\n",
+    "1. Why are `TfmdLists` and `Datasets` named with an s?\n",
+    "1. How can you build a `DataLoaders` from a `TfmdLists` or a `Datasets`?\n",
+    "1. How do you pass `item_tfms` and `batch_tfms` when building a `DataLoaders` from a `TfmdLists` or a `Datasets`?\n",
+    "1. What do you need to do when you want to have your custom items work with methods like `show_batch` or `show_results`?\n",
+    "1. Why can we easily apply fastai data augmentation transforms to the `SiamesePair` we built?"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Further research"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "1. Use the mid-level API to grab the data on the pets dataset. On the adult dataset (used in chapter 1).\n",
+    "1. Look at the siamese tutorial in the fastai documentation to learn how to customize the behavior of `show_batch` and `show_results` for new type of items. Implement it on your own project."
    ]
   },
   {

12_nlp_dive.ipynb

@@ -1927,7 +1927,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "TK add takeaway form those citations before moving on."
+    "This explains the idea behind why dropout helps to generalize: first, it helps the neurons to cooperate better together, then it makes the activations more noisy, thus making the model more robust."
    ]
   },
   {

13_convolutions.ipynb

@@ -1505,14 +1505,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "This operation of applying a kernel over a grid in this way is called *convolution*. In the paper [A guide to convolution arithmetic for deep learning](https://arxiv.org/abs/1603.07285) there are many great diagrams showing how image kernels can be applied. Here's an example from the paper showing (at bottom) a light blue 4x4 image, with a dark blue 3x3 kernel being applied, creating a 2x2 green output activation map at the top. (We'll be using quite a few images from this paper in this book--when you see images in this style, you'll know they're from this great paper.)"
+    "This operation of applying a kernel over a grid in this way is called *convolution*. In the paper [A guide to convolution arithmetic for deep learning](https://arxiv.org/abs/1603.07285) there are many great diagrams showing how image kernels can be applied. Here's an example from the paper showing (at bottom) a light blue 4x4 image, with a dark blue 3x3 kernel being applied, creating a 2x2 green output activation map at the top. "
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "<img alt=\"Result of applying a 3x3 kernel to a 4x4 image\" width=\"782\" caption=\"Result of applying a 3x3 kernel to a 4x4 image\" id=\"three_ex_four_conv\" src=\"images/att_00028.png\">"
+    "<img alt=\"Result of applying a 3x3 kernel to a 4x4 image\" width=\"782\" caption=\"Result of applying a 3x3 kernel to a 4x4 image (curtesy of Vincent Dumoulin and Francesco Visin)\" id=\"three_ex_four_conv\" src=\"images/att_00028.png\">"
    ]
   },
   {
@@ -1771,7 +1771,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "<img alt=\"4x4 kernel with 5x5 input and 2 pixels of padding\" width=\"783\" caption=\"4x4 kernel with 5x5 input and 2 pixels of padding\" id=\"four_by_five_conv\" src=\"images/att_00029.png\">"
+    "<img alt=\"4x4 kernel with 5x5 input and 2 pixels of padding\" width=\"783\" caption=\"4x4 kernel with 5x5 input and 2 pixels of padding (curtesy of Vincent Dumoulin and Francesco Visin)\" id=\"four_by_five_conv\" src=\"images/att_00029.png\">"
    ]
   },
   {
@@ -1787,7 +1787,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "<img alt=\"3x3 kernel with 5x5 input, stride 2 convolution, and 1 pixel of padding\" width=\"774\" caption=\"3x3 kernel with 5x5 input, stride 2 convolution, and 1 pixel of padding\" id=\"three_by_five_conv\" src=\"images/att_00030.png\">"
+    "<img alt=\"3x3 kernel with 5x5 input, stride 2 convolution, and 1 pixel of padding\" width=\"774\" caption=\"3x3 kernel with 5x5 input, stride 2 convolution, and 1 pixel of padding (curtesy of Vincent Dumoulin and Francesco Visin)\" id=\"three_by_five_conv\" src=\"images/att_00030.png\">"
    ]
   },
   {
@@ -3317,7 +3317,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "TK Add an explanation of the picture"
+    "It shows why log(f) is mre colorful than f when f follows a normal distribution because taking a log changes the gaussian in a quadratic."
    ]
   },
   {
@@ -3661,14 +3661,9 @@
     "\n",
     "These constraints allow us to use far less parameters in our model, without sacrificing the ability to represent complex visual features. That means we can train deeper models faster, with less over-fitting. Although the universal approximation theorem shows that it should be *possible* to represent anything in a fully connected network in one hidden layer, we've seen now that in *practice* we can train much better models by being thoughtful about network architecture.\n",
     "\n",
-    "Convolutions are by far the most common pattern of connectivity we see in neural nets (along with regular linear layers, which we refer to as *fully connected*), but it's likely that many more will be discovered."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "TK add some takeways from the last section to the conclusion"
+    "Convolutions are by far the most common pattern of connectivity we see in neural nets (along with regular linear layers, which we refer to as *fully connected*), but it's likely that many more will be discovered.\n",
+    "\n",
+    "Then we have seen how to interpret the activations of layers in the network to see if training is going well or not, and how Batchnorm helps regularizing the training and makes it smoother. In the next chapter, we will use both of those layers to build the most popular architecture in computer vision: residual networks."
    ]
   },
   {

14_resnet.ipynb

@@ -675,7 +675,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "<img alt=\"Impact of ResNet on loss landscape\" width=\"600\" caption=\"Impact of ResNet on loss landscape\" id=\"resnet_surface\" src=\"images/att_00044.png\">"
+    "<img alt=\"Impact of ResNet on loss landscape\" width=\"600\" caption=\"Impact of ResNet on loss landscape (curtesy of Hao Li et al.)\" id=\"resnet_surface\" src=\"images/att_00044.png\">"
    ]
   },
   {
@@ -1196,7 +1196,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "TK add conclusion"
+    "## Conclusion"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You have know seen how the models we were using in computer vision since the first chapter are built, using skip connections to allow deeper models to be trained. Even if there has been a lot of research in better architectures, they all use one version or another of this trick, to make a direct path from the input to the end of the network. When using transfer learning, resnet is the pretrained model. In the next chapter, we will look at the final details of how the models we actually used were built from it."
    ]
   },
   {

16_accel_sgd.ipynb

@@ -1213,7 +1213,16 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "TK Write a conclusion"
+    "## Conclusion"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We have dug into the training loop by looking at all the variants of SGD and why they can be more powerful. At the time of writing, research has been very active in developping new optimizers, so by the time you read this chapter, there may be an addendum to this chapter on the book website that presents new variants. Be sure to check how our general optimizer framework can help you implement new optimizers very fast.\n",
+    "\n",
+    "For every tweak of the training loop, we use a powerful `Callback` system that allows you to customize every bit of that loop by being able to inspect and modify any parameter between each step of the training loop."
    ]
   },
   {

17_foundations.ipynb

@@ -2338,21 +2338,23 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "In the next chapter, we will start from such a model and see how we build a training loop from scratch and refactor it to what we've been using in previous chapters."
+    "In the last chapter, we will start from such a model and see how we build a training loop from scratch and refactor it to what we've been using in previous chapters."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Things to remember"
+    "## Conclusion"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "TK tweak this and make it a real conclusion\n",
+    "We have looked at the foundations of deep learning, beginning with matrix multiplication and implement the forward and backward passes of a neural net from scratch. We thenr efactored it to show how PyTorch works beneath the hood.\n",
+    "\n",
+    "Here are a few things to remember:\n",
     "\n",
     "- A neural net is basically a bunch of matrix multiplications with non-linearities in-between.\n",
     "- Python is slow so to write fast code we have to vectorize it and take advantage of element-wise arithmetic or broadcasting.\n",

18_CAM.ipynb

@@ -593,7 +593,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "TK Write conclusion"
+    "## Conclusion"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Model interpretation is an are of active research and we just scrapped the sruface of what is possible in this brief chapter. Class activation maps give an insight of why a model predicted a certain result by showing the area of the images that were most responsible for a given prediction. It can help analyze false positives and help you figure out what kind of data is missing in your training to fix them."
    ]
   },
   {

19_learner.ipynb

@@ -1793,7 +1793,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "TK conclusion"
+    "## Conclusion"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We have looked at how the key concepts of the fastai library are implemented by re-implementing them in this chapter. Since it's mostly full of code, you should definitely try to experiment with it by looking at the corresponding notebook on the book website. As a next step, be sure to check the intermediate and advanced tutorials in the fastai documentation, to learn how to customize every bit of the library now that you know the way it's built."
    ]
   },
   {

clean/10_nlp.ipynb

@@ -1489,6 +1489,13 @@
     "## Disinformation and language models"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Conclusion"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},

clean/11_midlevel_data.ipynb

@@ -797,6 +797,56 @@
     "    after_batch=[IntToFloatTensor, Normalize.from_stats(*imagenet_stats)])"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Conclusion"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Questionnaire"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "1. Why do we say that fastai has a layered API? What does it mean?\n",
+    "1. Why does a `Transform` have a decode method? What does it do?\n",
+    "1. Why does a `Transform` have a setup method? What does it do?\n",
+    "1. How does a `Transform` work when called on a tuple?\n",
+    "1. Which methods do you need to implement when writing your own `Transform`?\n",
+    "1. Write a `Normalize` transform that fully normalizes items (substract the mean and divide by the standard deviation of the dataset), and that can decode that behavior. Try not to peak!\n",
+    "1. Write a `Transform` that does the numericalization of tokenized texts (it should set its vocab automatically from the dataset seen and have a decode method). Look at the source code of fastai if you need help.\n",
+    "1. What is a `Pipeline`?\n",
+    "1. What is a `TfmdLists`? \n",
+    "1. What is a `Datasets`? How is it different from `TfmdLists`?\n",
+    "1. Why are `TfmdLists` and `Datasets` named with an s?\n",
+    "1. How can you build a `DataLoaders` from a `TfmdLists` or a `Datasets`?\n",
+    "1. How do you pass `item_tfms` and `batch_tfms` when building a `DataLoaders` from a `TfmdLists` or a `Datasets`?\n",
+    "1. What do you need to do when you want to have your custom items work with methods like `show_batch` or `show_results`?\n",
+    "1. Why can we easily apply fastai data augmentation transforms to the `SiamesePair` we built?"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Further research"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "1. Use the mid-level API to grab the data on the pets dataset. On the adult dataset (used in chapter 1).\n",
+    "1. Look at the siamese tutorial in the fastai documentation to learn how to customize the behavior of `show_batch` and `show_results` for new type of items. Implement it on your own project."
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -804,6 +854,17 @@
     "## Becoming a deep learning practitioner"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Congratulations — you've completed all of the chapters in this book which cover the key practical parts of training and using deep learning! You know how to use all of fastai's built in applications, and how to customise them using the data blocks API and loss functions. You even know how to create a neural network from scratch, and train it! (And hopefully you now know some of the questions to ask to help make sure your creations help improve society too.)\n",
+    "\n",
+    "The knowledge you already have is enough to create full working prototypes of many types of neural network application. More importantly, it will help you understand the capabilities and limitations of deep learning models, and how to design a system which best handles these capabilities and limitations.\n",
+    "\n",
+    "In the rest of this book we will be pulling apart these applications, piece by piece, to understand all of the foundations they are built on. This is important knowledge for a deep learning practitioner, because it is the knowledge which allows you to inspect and debug models that you build, and to create new applications which are customised for your particular projects."
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,

clean/14_resnet.ipynb

@@ -811,6 +811,13 @@
     "learn.fit_one_cycle(20, 3e-3)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Conclusion"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},

clean/16_accel_sgd.ipynb

@@ -663,6 +663,13 @@
     "            raise CancelFitException"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Conclusion"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},

clean/17_foundations.ipynb

@@ -1513,7 +1513,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Things to remember"
+    "## Conclusion"
    ]
   },
   {

clean/18_CAM.ipynb

@@ -413,6 +413,13 @@
     "              interpolation='bilinear', cmap='magma');"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Conclusion"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},

clean/19_learner.ipynb

@@ -1270,6 +1270,13 @@
     "plt.plot(onecyc.lrs);"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Conclusion"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},