daviddoji/Fastbook_Julia
1
0 Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Removed notebooks

Browse Source
This commit is contained in:
daviddoji committed 2023-04-28 20:59:28 +02:00
1 parent 492582ce43
commit 97438cf74a
22 files changed
-49274
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

01_intro.ipynb
-2924
View File
File diff suppressed because it is too large. Load diff
02_production.ipynb
-2014
View File
File diff suppressed because it is too large. Load diff
03_ethics.ipynb
-1061
View File
File diff suppressed because it is too large. Load diff
04_mnist_basics.ipynb
-5863
View File
File diff suppressed because it is too large. Load diff
05_pet_breeds.ipynb
-2733
View File
File diff suppressed because it is too large. Load diff
06_multicat.ipynb
-2247
View File
File diff suppressed because it is too large. Load diff
07_sizing_and_tta.ipynb
-1026
View File
File diff suppressed because it is too large. Load diff
08_collab.ipynb
-2353
View File
File diff suppressed because it is too large. Load diff
09_tabular.ipynb
-10008
View File
File diff suppressed because it is too large. Load diff
10_nlp.ipynb
-2286
View File
File diff suppressed because it is too large. Load diff
11_midlevel_data.ipynb
-1310
View File
File diff suppressed because it is too large. Load diff
12_nlp_dive.ipynb
-2365
View File
File diff suppressed because it is too large. Load diff
13_convolutions.ipynb
-3696
View File
File diff suppressed because it is too large. Load diff
14_resnet.ipynb
-1286
View File
File diff suppressed because it is too large. Load diff
15_arch_details.ipynb
-816
View File
@@ -1,816 +0,0 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"! [ -e /content ] && pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"from fastbook import *"
]
},
{
"cell_type": "raw",
"metadata": {},
"source": [
"[[chapter_arch_details]]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Application Architectures Deep Dive"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We are now in the exciting position that we can fully understand the architectures that we have been using for our state-of-the-art models for computer vision, natural language processing, and tabular analysis. In this chapter, we're going to fill in all the missing details on how fastai's application models work and show you how to build the models they use.\n",
"\n",
"We will also go back to the custom data preprocessing pipeline we saw in <<chapter_midlevel_data>> for Siamese networks and show you how you can use the components in the fastai library to build custom pretrained models for new tasks.\n",
"\n",
"We'll start with computer vision."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Computer Vision"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For computer vision application we use the functions `vision_learner` and `unet_learner` to build our models, depending on the task. In this section we'll explore how to build the `Learner` objects we used in Parts 1 and 2 of this book."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### vision_learner"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's take a look at what happens when we use the `vision_learner` function. We begin by passing this function an architecture to use for the *body* of the network. Most of the time we use a ResNet, which you already know how to create, so we don't need to delve into that any further. Pretrained weights are downloaded as required and loaded into the ResNet.\n",
"\n",
"Then, for transfer learning, the network needs to be *cut*. This refers to slicing off the final layer, which is only responsible for ImageNet-specific categorization. In fact, we do not slice off only this layer, but everything from the adaptive average pooling layer onwards. The reason for this will become clear in just a moment. Since different architectures might use different types of pooling layers, or even completely different kinds of *heads*, we don't just search for the adaptive pooling layer to decide where to cut the pretrained model. Instead, we have a dictionary of information that is used for each model to determine where its body ends, and its head starts. We call this `model_meta`—here it is for resnet-50:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'cut': -2,\n",
" 'split': <function fastai.vision.learner._resnet_split(m)>,\n",
" 'stats': ([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])}"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model_meta[resnet50]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> jargon: Body and Head: The \"head\" of a neural net is the part that is specialized for a particular task. For a CNN, it's generally the part after the adaptive average pooling layer. The \"body\" is everything else, and includes the \"stem\" (which we learned about in <<chapter_resnet>>)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If we take all of the layers prior to the cut point of `-2`, we get the part of the model that fastai will keep for transfer learning. Now, we put on our new head. This is created using the function `create_head`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Sequential(\n",
" (0): AdaptiveConcatPool2d(\n",
" (ap): AdaptiveAvgPool2d(output_size=1)\n",
" (mp): AdaptiveMaxPool2d(output_size=1)\n",
" )\n",
" (1): full: False\n",
" (2): BatchNorm1d(20, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
" (3): Dropout(p=0.25, inplace=False)\n",
" (4): Linear(in_features=20, out_features=512, bias=False)\n",
" (5): ReLU(inplace=True)\n",
" (6): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
" (7): Dropout(p=0.5, inplace=False)\n",
" (8): Linear(in_features=512, out_features=2, bias=False)\n",
")"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#hide_output\n",
"create_head(20,2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```\n",
"Sequential(\n",
" (0): AdaptiveConcatPool2d(\n",
" (ap): AdaptiveAvgPool2d(output_size=1)\n",
" (mp): AdaptiveMaxPool2d(output_size=1)\n",
" )\n",
" (1): Flatten()\n",
" (2): BatchNorm1d(20, eps=1e-05, momentum=0.1, affine=True)\n",
" (3): Dropout(p=0.25, inplace=False)\n",
" (4): Linear(in_features=20, out_features=512, bias=False)\n",
" (5): ReLU(inplace=True)\n",
" (6): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True)\n",
" (7): Dropout(p=0.5, inplace=False)\n",
" (8): Linear(in_features=512, out_features=2, bias=False)\n",
")\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With this function you can choose how many additional linear layers are added to the end, how much dropout to use after each one, and what kind of pooling to use. By default, fastai will apply both average pooling, and max pooling, and will concatenate the two together (this is the `AdaptiveConcatPool2d` layer). This is not a particularly common approach, but it was developed independently at fastai and other research labs in recent years, and tends to provide some small improvement over using just average pooling.\n",
"\n",
"fastai is a bit different from most libraries in that by default it adds two linear layers, rather than one, in the CNN head. The reason for this is that transfer learning can still be useful even, as we have seen, when transferring the pretrained model to very different domains. However, just using a single linear layer is unlikely to be enough in these cases; we have found that using two linear layers can allow transfer learning to be used more quickly and easily, in more situations."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> note: One Last Batchnorm?: One parameter to `create_head` that is worth looking at is `bn_final`. Setting this to `true` will cause a batchnorm layer to be added as your final layer. This can be useful in helping your model scale appropriately for your output activations. We haven't seen this approach published anywhere as yet, but we have found that it works well in practice wherever we have used it."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's now take a look at what `unet_learner` did in the segmentation problem we showed in <<chapter_intro>>."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### unet_learner"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"One of the most interesting architectures in deep learning is the one that we used for segmentation in <<chapter_intro>>. Segmentation is a challenging task, because the output required is really an image, or a pixel grid, containing the predicted label for every pixel. There are other tasks that share a similar basic design, such as increasing the resolution of an image (*super-resolution*), adding color to a black-and-white image (*colorization*), or converting a photo into a synthetic painting (*style transfer*)—these tasks are covered by an [online](https://book.fast.ai/) chapter of this book, so be sure to check it out after you've read this chapter. In each case, we are starting with an image and converting it to some other image of the same dimensions or aspect ratio, but with the pixels altered in some way. We refer to these as *generative vision models*.\n",
"\n",
"The way we do this is to start with the exact same approach to developing a CNN head as we saw in the previous problem. We start with a ResNet, for instance, and cut off the adaptive pooling layer and everything after that. Then we replace those layers with our custom head, which does the generative task.\n",
"\n",
"There was a lot of handwaving in that last sentence! How on earth do we create a CNN head that generates an image? If we start with, say, a 224-pixel input image, then at the end of the ResNet body we will have a 7×7 grid of convolutional activations. How can we convert that into a 224-pixel segmentation mask?\n",
"\n",
"Naturally, we do this with a neural network! So we need some kind of layer that can increase the grid size in a CNN. One very simple approach to this is to replace every pixel in the 7×7 grid with four pixels in a 2×2 square. Each of those four pixels will have the same value—this is known as *nearest neighbor interpolation*. PyTorch provides a layer that does this for us, so one option is to create a head that contains stride-1 convolutional layers (along with batchnorm and ReLU layers as usual) interspersed with 2×2 nearest neighbor interpolation layers. In fact, you can try this now! See if you can create a custom head designed like this, and try it on the CamVid segmentation task. You should find that you get some reasonable results, although they won't be as good as our <<chapter_intro>> results.\n",
"\n",
"Another approach is to replace the nearest neighbor and convolution combination with a *transposed convolution*, otherwise known as a *stride half convolution*. This is identical to a regular convolution, but first zero padding is inserted between all the pixels in the input. This is easiest to see with a picture—<<transp_conv>> shows a diagram from the excellent [convolutional arithmetic paper](https://arxiv.org/abs/1603.07285) we discussed in <<chapter_convolutions>>, showing a 3×3 transposed convolution applied to a 3×3 image."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img alt=\"A transposed convolution\" width=\"815\" caption=\"A transposed convolution (courtesy of Vincent Dumoulin and Francesco Visin)\" id=\"transp_conv\" src=\"images/att_00051.png\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As you see, the result of this is to increase the size of the input. You can try this out now by using fastai's `ConvLayer` class; pass the parameter `transpose=True` to create a transposed convolution, instead of a regular one, in your custom head.\n",
"\n",
"Neither of these approaches, however, works really well. The problem is that our 7×7 grid simply doesn't have enough information to create a 224×224-pixel output. It's asking an awful lot of the activations of each of those grid cells to have enough information to fully regenerate every pixel in the output. The solution to this problem is to use *skip connections*, like in a ResNet, but skipping from the activations in the body of the ResNet all the way over to the activations of the transposed convolution on the opposite side of the architecture. This approach, illustrated in <<unet>>, was developed by Olaf Ronneberger, Philipp Fischer, and Thomas Brox in the 2015 paper [\"U-Net: Convolutional Networks for Biomedical Image Segmentation\"](https://arxiv.org/abs/1505.04597). Although the paper focused on medical applications, the U-Net has revolutionized all kinds of generative vision models."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img alt=\"The U-Net architecture\" width=\"630\" caption=\"The U-Net architecture (courtesy of Olaf Ronneberger, Philipp Fischer, and Thomas Brox)\" id=\"unet\" src=\"images/att_00052.png\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This picture shows the CNN body on the left (in this case, it's a regular CNN, not a ResNet, and they're using 2×2 max pooling instead of stride-2 convolutions, since this paper was written before ResNets came along) and the transposed convolutional (\"up-conv\") layers on the right. Then extra skip connections are shown as gray arrows crossing from left to right (these are sometimes called *cross connections*). You can see why it's called a \"U-Net!\"\n",
"\n",
"With this architecture, the input to the transposed convolutions is not just the lower-resolution grid in the preceding layer, but also the higher-resolution grid in the ResNet head. This allows the U-Net to use all of the information of the original image, as it is needed. One challenge with U-Nets is that the exact architecture depends on the image size. fastai has a unique `DynamicUnet` class that autogenerates an architecture of the right size based on the data provided.\n",
"\n",
"Let's focus now on an example where we leverage the fastai library to write a custom model."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### A Siamese Network"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"from fastai.vision.all import *\n",
"path = untar_data(URLs.PETS)\n",
"files = get_image_files(path/\"images\")\n",
"\n",
"class SiameseImage(fastuple):\n",
" def show(self, ctx=None, **kwargs): \n",
" img1,img2,same_breed = self\n",
" if not isinstance(img1, Tensor):\n",
" if img2.size != img1.size: img2 = img2.resize(img1.size)\n",
" t1,t2 = tensor(img1),tensor(img2)\n",
" t1,t2 = t1.permute(2,0,1),t2.permute(2,0,1)\n",
" else: t1,t2 = img1,img2\n",
" line = t1.new_zeros(t1.shape[0], t1.shape[1], 10)\n",
" return show_image(torch.cat([t1,line,t2], dim=2), \n",
" title=same_breed, ctx=ctx)\n",
" \n",
"def label_func(fname):\n",
" return re.match(r'^(.*)_\\d+.jpg$', fname.name).groups()[0]\n",
"\n",
"class SiameseTransform(Transform):\n",
" def __init__(self, files, label_func, splits):\n",
" self.labels = files.map(label_func).unique()\n",
" self.lbl2files = {l: L(f for f in files if label_func(f) == l) for l in self.labels}\n",
" self.label_func = label_func\n",
" self.valid = {f: self._draw(f) for f in files[splits[1]]}\n",
" \n",
" def encodes(self, f):\n",
" f2,t = self.valid.get(f, self._draw(f))\n",
" img1,img2 = PILImage.create(f),PILImage.create(f2)\n",
" return SiameseImage(img1, img2, t)\n",
" \n",
" def _draw(self, f):\n",
" same = random.random() < 0.5\n",
" cls = self.label_func(f)\n",
" if not same: cls = random.choice(L(l for l in self.labels if l != cls)) \n",
" return random.choice(self.lbl2files[cls]),same\n",
" \n",
"splits = RandomSplitter()(files)\n",
"tfm = SiameseTransform(files, label_func, splits)\n",
"tls = TfmdLists(files, tfm, splits=splits)\n",
"dls = tls.dataloaders(after_item=[Resize(224), ToTensor], \n",
" after_batch=[IntToFloatTensor, Normalize.from_stats(*imagenet_stats)])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's go back to the input pipeline we set up in <<chapter_midlevel_data>> for a Siamese network. If you remember, it consisted of pair of images with the label being `True` or `False`, depending on if they were in the same class or not.\n",
"\n",
"Using what we just saw, let's build a custom model for this task and train it. How? We will use a pretrained architecture and pass our two images through it. Then we can concatenate the results and send them to a custom head that will return two predictions. In terms of modules, this looks like this:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"class SiameseModel(Module):\n",
" def __init__(self, encoder, head):\n",
" self.encoder,self.head = encoder,head\n",
" \n",
" def forward(self, x1, x2):\n",
" ftrs = torch.cat([self.encoder(x1), self.encoder(x2)], dim=1)\n",
" return self.head(ftrs)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To create our encoder, we just need to take a pretrained model and cut it, as we explained before. The function `create_body` does that for us; we just have to pass it the place where we want to cut. As we saw earlier, per the dictionary of metadata for pretrained models, the cut value for a resnet is `-2`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"encoder = create_body(resnet34, cut=-2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Then we can create our head. A look at the encoder tells us the last layer has 512 features, so this head will need to receive `512*2`. Why 2? We have to multiply by 2 because we have two images. So we create the head as follows:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"head = create_head(512*2, 2, ps=0.5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With our encoder and head, we can now build our model:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"model = SiameseModel(encoder, head)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Before using `Learner`, we have two more things to define. First, we must define the loss function we want to use. It's regular cross-entropy, but since our targets are Booleans, we need to convert them to integers or PyTorch will throw an error:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def loss_func(out, targ):\n",
" return nn.CrossEntropyLoss()(out, targ.long())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"More importantly, to take full advantage of transfer learning, we have to define a custom *splitter*. A splitter is a function that tells the fastai library how to split the model into parameter groups. These are used behind the scenes to train only the head of a model when we do transfer learning. \n",
"\n",
"Here we want two parameter groups: one for the encoder and one for the head. We can thus define the following splitter (`params` is just a function that returns all parameters of a given module):"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def siamese_splitter(model):\n",
" return [params(model.encoder), params(model.head)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Then we can define our `Learner` by passing the data, model, loss function, splitter, and any metric we want. Since we are not using a convenience function from fastai for transfer learning (like `vision_learner`), we have to call `learn.freeze` manually. This will make sure only the last parameter group (in this case, the head) is trained:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"learn = Learner(dls, model, loss_func=loss_func, \n",
" splitter=siamese_splitter, metrics=accuracy)\n",
"learn.freeze()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Then we can directly train our model with the usual methods:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>accuracy</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>0.367015</td>\n",
" <td>0.281242</td>\n",
" <td>0.885656</td>\n",
" <td>00:26</td>\n",
" </tr>\n",
" <tr>\n",
" <td>1</td>\n",
" <td>0.307688</td>\n",
" <td>0.214721</td>\n",
" <td>0.915426</td>\n",
" <td>00:26</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>0.275221</td>\n",
" <td>0.170615</td>\n",
" <td>0.936401</td>\n",
" <td>00:26</td>\n",
" </tr>\n",
" <tr>\n",
" <td>3</td>\n",
" <td>0.223771</td>\n",
" <td>0.159633</td>\n",
" <td>0.943843</td>\n",
" <td>00:26</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"learn.fit_one_cycle(4, 3e-3)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Before unfreezing and fine-tuning the whole model a bit more with discriminative learning rates (that is: a lower learning rate for the body and a higher one for the head):"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>accuracy</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>0.212744</td>\n",
" <td>0.159033</td>\n",
" <td>0.944520</td>\n",
" <td>00:35</td>\n",
" </tr>\n",
" <tr>\n",
" <td>1</td>\n",
" <td>0.201893</td>\n",
" <td>0.159615</td>\n",
" <td>0.942490</td>\n",
" <td>00:35</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>0.204606</td>\n",
" <td>0.152338</td>\n",
" <td>0.945196</td>\n",
" <td>00:36</td>\n",
" </tr>\n",
" <tr>\n",
" <td>3</td>\n",
" <td>0.213203</td>\n",
" <td>0.148346</td>\n",
" <td>0.947903</td>\n",
" <td>00:36</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"learn.unfreeze()\n",
"learn.fit_one_cycle(4, slice(1e-6,1e-4))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"94.8\\% is very good when we remember a classifier trained the same way (with no data augmentation) had an error rate of 7%."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that we've seen how to create complete state-of-the-art computer vision models, let's move on to NLP."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Natural Language Processing"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Converting an AWD-LSTM language model into a transfer learning classifier, as we did in <<chapter_nlp>>, follows a very similar process to what we did with `vision_learner` in the first section of this chapter. We do not need a \"meta\" dictionary in this case, because we do not have such a variety of architectures to support in the body. All we need to do is select the stacked RNN for the encoder in the language model, which is a single PyTorch module. This encoder will provide an activation for every word of the input, because a language model needs to output a prediction for every next word.\n",
"\n",
"To create a classifier from this we use an approach described in the [ULMFiT paper](https://arxiv.org/abs/1801.06146) as \"BPTT for Text Classification (BPT3C)\":"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> : We divide the document into fixed-length batches of size *b*. At the beginning of each batch, the model is initialized with the final state of the previous batch; we keep track of the hidden states for mean and max-pooling; gradients are back-propagated to the batches whose hidden states contributed to the final prediction. In practice, we use variable length backpropagation sequences."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In other words, the classifier contains a `for` loop, which loops over each batch of a sequence. The state is maintained across batches, and the activations of each batch are stored. At the end, we use the same average and max concatenated pooling trick that we use for computer vision models—but this time, we do not pool over CNN grid cells, but over RNN sequences.\n",
"\n",
"For this `for` loop we need to gather our data in batches, but each text needs to be treated separately, as they each have their own labels. However, it's very likely that those texts won't all be of the same length, which means we won't be able to put them all in the same array, like we did with the language model.\n",
"\n",
"That's where padding is going to help: when grabbing a bunch of texts, we determine the one with the greatest length, then we fill the ones that are shorter with a special token called `xxpad`. To avoid extreme cases where we have a text with 2,000 tokens in the same batch as a text with 10 tokens (so a lot of padding, and a lot of wasted computation), we alter the randomness by making sure texts of comparable size are put together. The texts will still be in a somewhat random order for the training set (for the validation set we can simply sort them by order of length), but not completely so.\n",
"\n",
"This is done automatically behind the scenes by the fastai library when creating our `DataLoaders`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Tabular"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally, let's take a look at `fastai.tabular` models. (We don't need to look at collaborative filtering separately, since we've already seen that these models are just tabular models, or use the dot product approach, which we've implemented earlier from scratch.)\n",
"\n",
"Here is the `forward` method for `TabularModel`:\n",
"\n",
"```python\n",
"if self.n_emb != 0:\n",
" x = [e(x_cat[:,i]) for i,e in enumerate(self.embeds)]\n",
" x = torch.cat(x, 1)\n",
" x = self.emb_drop(x)\n",
"if self.n_cont != 0:\n",
" x_cont = self.bn_cont(x_cont)\n",
" x = torch.cat([x, x_cont], 1) if self.n_emb != 0 else x_cont\n",
"return self.layers(x)\n",
"```\n",
"\n",
"We won't show `__init__` here, since it's not that interesting, but we will look at each line of code in `forward` in turn. The first line:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```python\n",
"if self.n_emb != 0:\n",
"```\n",
"\n",
"is just testing whether there are any embeddings to deal with—we can skip this section if we only have continuous variables. `self.embeds` contains the embedding matrices, so this gets the activations of each:\n",
" \n",
"```python\n",
" x = [e(x_cat[:,i]) for i,e in enumerate(self.embeds)]\n",
"```\n",
"\n",
"and concatenates them into a single tensor:\n",
"\n",
"```python\n",
" x = torch.cat(x, 1)\n",
"```\n",
"\n",
"Then dropout is applied. You can pass `embd_p` to `__init__` to change this value:\n",
"\n",
"```python\n",
" x = self.emb_drop(x)\n",
"```\n",
"\n",
"Now we test whether there are any continuous variables to deal with:\n",
"\n",
"```python\n",
"if self.n_cont != 0:\n",
"```\n",
"\n",
"They are passed through a batchnorm layer:\n",
"\n",
"```python\n",
" x_cont = self.bn_cont(x_cont)\n",
"```\n",
"\n",
"and concatenated with the embedding activations, if there were any:\n",
"\n",
"```python\n",
" x = torch.cat([x, x_cont], 1) if self.n_emb != 0 else x_cont\n",
"```\n",
"\n",
"Finally, this is passed through the linear layers (each of which includes batchnorm, if `use_bn` is `True`, and dropout, if `ps` is set to some value or list of values):\n",
"\n",
"```python\n",
"return self.layers(x)\n",
"\n",
"```\n",
"\n",
"Congratulations! Now you know every single piece of the architectures used in the fastai library!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Wrapping Up Architectures"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As you can see, the details of deep learning architectures need not scare you now. You can look inside the code of fastai and PyTorch and see just what is going on. More importantly, try to understand *why* it's going on. Take a look at the papers that are being referenced in the code, and try to see how the code matches up to the algorithms that are described.\n",
"\n",
"Now that we have investigated all of the pieces of a model and the data that is passed into it, we can consider what this means for practical deep learning. If you have unlimited data, unlimited memory, and unlimited time, then the advice is easy: train a huge model on all of your data for a really long time. But the reason that deep learning is not straightforward is because your data, memory, and time are typically limited. If you are running out of memory or time, then the solution is to train a smaller model. If you are not able to train for long enough to overfit, then you are not taking advantage of the capacity of your model.\n",
"\n",
"So, step one is to get to the point where you can overfit. Then the question is how to reduce that overfitting. <<reduce_overfit>> shows how we recommend prioritizing the steps from there."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img alt=\"Steps to reducing overfitting\" width=\"400\" caption=\"Steps to reducing overfitting\" id=\"reduce_overfit\" src=\"images/att_00047.png\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Many practitioners, when faced with an overfitting model, start at exactly the wrong end of this diagram. Their starting point is to use a smaller model, or more regularization. Using a smaller model should be absolutely the last step you take, unless training your model is taking up too much time or memory. Reducing the size of your model reduces the ability of your model to learn subtle relationships in your data.\n",
"\n",
"Instead, your first step should be to seek to *create more data*. That could involve adding more labels to data that you already have, finding additional tasks that your model could be asked to solve (or, to think of it another way, identifying different kinds of labels that you could model), or creating additional synthetic data by using more or different data augmentation techniques. Thanks to the development of Mixup and similar approaches, effective data augmentation is now available for nearly all kinds of data.\n",
"\n",
"Once you've got as much data as you think you can reasonably get hold of, and are using it as effectively as possible by taking advantage of all the labels that you can find and doing all the augmentation that makes sense, if you are still overfitting you should think about using more generalizable architectures. For instance, adding batch normalization may improve generalization.\n",
"\n",
"If you are still overfitting after doing the best you can at using your data and tuning your architecture, then you can take a look at regularization. Generally speaking, adding dropout to the last layer or two will do a good job of regularizing your model. However, as we learned from the story of the development of AWD-LSTM, it is often the case that adding dropout of different types throughout your model can help even more. Generally speaking, a larger model with more regularization is more flexible, and can therefore be more accurate than a smaller model with less regularization.\n",
"\n",
"Only after considering all of these options would we recommend that you try using a smaller version of your architecture."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Questionnaire"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. What is the \"head\" of a neural net?\n",
"1. What is the \"body\" of a neural net?\n",
"1. What is \"cutting\" a neural net? Why do we need to do this for transfer learning?\n",
"1. What is `model_meta`? Try printing it to see what's inside.\n",
"1. Read the source code for `create_head` and make sure you understand what each line does.\n",
"1. Look at the output of `create_head` and make sure you understand why each layer is there, and how the `create_head` source created it.\n",
"1. Figure out how to change the dropout, layer size, and number of layers created by `vision_learner`, and see if you can find values that result in better accuracy from the pet recognizer.\n",
"1. What does `AdaptiveConcatPool2d` do?\n",
"1. What is \"nearest neighbor interpolation\"? How can it be used to upsample convolutional activations?\n",
"1. What is a \"transposed convolution\"? What is another name for it?\n",
"1. Create a conv layer with `transpose=True` and apply it to an image. Check the output shape.\n",
"1. Draw the U-Net architecture.\n",
"1. What is \"BPTT for Text Classification\" (BPT3C)?\n",
"1. How do we handle different length sequences in BPT3C?\n",
"1. Try to run each line of `TabularModel.forward` separately, one line per cell, in a notebook, and look at the input and output shapes at each step.\n",
"1. How is `self.layers` defined in `TabularModel`?\n",
"1. What are the five steps for preventing over-fitting?\n",
"1. Why don't we reduce architecture complexity before trying other approaches to preventing overfitting?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Further Research"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. Write your own custom head and try training the pet recognizer with it. See if you can get a better result than fastai's default.\n",
"1. Try switching between `AdaptiveConcatPool2d` and `AdaptiveAvgPool2d` in a CNN head and see what difference it makes.\n",
"1. Write your own custom splitter to create a separate parameter group for every ResNet block, and a separate group for the stem. Try training with it, and see if it improves the pet recognizer.\n",
"1. Read the online chapter about generative image models, and create your own colorizer, super-resolution model, or style transfer model.\n",
"1. Create a custom head using nearest neighbor interpolation and use it to do segmentation on CamVid."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"jupytext": {
"split_at_heading": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
16_accel_sgd.ipynb
-1324
View File
File diff suppressed because it is too large. Load diff
17_foundations.ipynb
-2465
View File
File diff suppressed because it is too large. Load diff
18_CAM.ipynb
-678
View File
@@ -1,678 +0,0 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"! [ -e /content ] && pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": false
},
"outputs": [],
"source": [
"#hide\n",
"from fastbook import *"
]
},
{
"cell_type": "raw",
"metadata": {},
"source": [
"[[chapter_cam]]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# CNN Interpretation with CAM"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that we know how to build up pretty much anything from scratch, let's use that knowledge to create entirely new (and very useful!) functionality: the *class activation map*. It gives us some insight into why a CNN made the predictions it did.\n",
"\n",
"In the process, we'll learn about one handy feature of PyTorch we haven't seen before, the *hook*, and we'll apply many of the concepts introduced in the rest of the book. If you want to really test out your understanding of the material in this book, after you've finished this chapter, try putting it aside and recreating the ideas here yourself from scratch (no peeking!)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## CAM and Hooks"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The class activation map (CAM) was introduced by Bolei Zhou et al. in [\"Learning Deep Features for Discriminative Localization\"](https://arxiv.org/abs/1512.04150). It uses the output of the last convolutional layer (just before the average pooling layer) together with the predictions to give us a heatmap visualization of why the model made its decision. This is a useful tool for interpretation.\n",
"\n",
"More precisely, at each position of our final convolutional layer, we have as many filters as in the last linear layer. We can therefore compute the dot product of those activations with the final weights to get, for each location on our feature map, the score of the feature that was used to make a decision.\n",
"\n",
"We're going to need a way to get access to the activations inside the model while it's training. In PyTorch this can be done with a *hook*. Hooks are PyTorch's equivalent of fastai's callbacks. However, rather than allowing you to inject code into the training loop like a fastai `Learner` callback, hooks allow you to inject code into the forward and backward calculations themselves. We can attach a hook to any layer of the model, and it will be executed when we compute the outputs (forward hook) or during backpropagation (backward hook). A forward hook is a function that takes three things—a module, its input, and its output—and it can perform any behavior you want. (fastai also provides a handy `HookCallback` that we won't cover here, but take a look at the fastai docs; it makes working with hooks a little easier.)\n",
"\n",
"To illustrate, we'll use the same cats and dogs model we trained in <<chapter_intro>>:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>error_rate</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>0.145994</td>\n",
" <td>0.019272</td>\n",
" <td>0.006089</td>\n",
" <td>00:14</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>error_rate</th>\n",
" <th>time</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>0</td>\n",
" <td>0.053405</td>\n",
" <td>0.052540</td>\n",
" <td>0.010825</td>\n",
" <td>00:19</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"path = untar_data(URLs.PETS)/'images'\n",
"def is_cat(x): return x[0].isupper()\n",
"dls = ImageDataLoaders.from_name_func(\n",
" path, get_image_files(path), valid_pct=0.2, seed=21,\n",
" label_func=is_cat, item_tfms=Resize(224))\n",
"learn = vision_learner(dls, resnet34, metrics=error_rate)\n",
"learn.fine_tune(1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To start, we'll grab a cat picture and a batch of data:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"img = PILImage.create(image_cat())\n",
"x, = first(dls.test_dl([img]))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For CAM we want to store the activations of the last convolutional layer. We put our hook function in a class so it has a state that we can access later, and just store a copy of the output:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"class Hook():\n",
" def hook_func(self, m, i, o): self.stored = o.detach().clone()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can then instantiate a `Hook` and attach it to the layer we want, which is the last layer of the CNN body:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"hook_output = Hook()\n",
"hook = learn.model[0].register_forward_hook(hook_output.hook_func)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we can grab a batch and feed it through our model:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"with torch.no_grad(): output = learn.model.eval()(x)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And we can access our stored activations:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"act = hook_output.stored[0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's also double-check our predictions:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([[0.0010, 0.9990]], device='cuda:0')"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"F.softmax(output, dim=-1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We know `0` (for `False`) is \"dog,\" because the classes are automatically sorted in fastai, bu we can still double-check by looking at `dls.vocab`: "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(#2) [False,True]"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dls.vocab"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"So, our model is very confident this was a picture of a cat."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To do the dot product of our weight matrix (2 by number of activations) with the activations (batch size by activations by rows by cols), we use a custom `einsum`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([1, 3, 224, 224])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([2, 7, 7])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"cam_map = torch.einsum('ck,kij->cij', learn.model[1][-1].weight, act)\n",
"cam_map.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For each image in our batch, and for each class, we get a 7×7 feature map that tells us where the activations were higher and where they were lower. This will let us see which areas of the pictures influenced the model's decision.\n",
"\n",
"For instance, we can find out which areas made the model decide this animal was a cat (note that we need to `decode` the input `x` since it's been normalized by the `DataLoader`, and we need to cast to `TensorImage` since at the time this book is written PyTorch does not maintain types when indexing—this may be fixed by the time you are reading this):"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOy9zZIkSY7n9wNUzdwjMqu6e2ZFuCO8UGQfhI+z78EjH4HPwlfhlSK7nJnlTFVlhLupAjwAULPIqs7s6T6wD+UlWZnh4W4favj4A/gDKu7O76/fX7+//v5e+v/3Bfz++v31++u3X78r5++v319/p6/flfP31++vv9PX78r5++v319/p63fl/P31++vv9NW/9cv/7X/9r05lc0XiL0BUaNpwd+K3jrtjHn/X51SUpiA44tAEVEBx4rhOU8Chaf5O4vNNQFUQm4gd+DxgDnwOMEN84maoG00cBATLP46Iowq9OdLy+Pk3DVQd1EAsL5Y4eROkdVyVYcJ/+p/+if/0n/+J+6c/IK2DNPp2Y0xnmOHa0H2j31/Q2wat4Tg2J/N4MMeB+wAbzPHExwFuiNa6GRKnRUQQcXDj+Xznl5//HZsTmJgZ4rGO7o6ZYXOiKphNbE7cHIYhBgxgODbAp+JTsKG4daY1zDvTG2PCmMK0xjSYpkyE4cox4TDHXHDtTIfpwnCYCNNgGAxzDjPMAdGQFZdYHwdDQBQnZMgAc0JeUn6M+LenLKGSMmUpU/WZWJ/pxvADc8MkvquqyFVOPU/kgghILDpuHn/caa3R6przd+aOEs+jaYtjui8xEYnrUULu3PN8oogoLuuTIBLPFcfNsGmYGe6xNv/3//Wn/+P//H//9//6H1ZOkRA0ESAX0ktAQqcun43L8cvPIqAi4HFDef/rgeCeNyYX/Zf1XRGJx5GCyJj4nKGAbuBWjzMEcp1DziuROL5IvetgjksaEZFQlFxED9VGpLPfNj59/oH7/RV3QYjFjwfueWbD3Bhz0IYiYV1wd1QVb8ocglkZpBTQaaDgZjSNcyK2jNYychKnI4VZSvBEUhhTKHAQS8FY+nGuSd1fyY3Juv9acJc4hyHxXERS+SWfd/69nn8er66h1l7j+4qmbZeUJcFwxGqVUwykZOCUqFJIzevydeJ83pTQS10VpyOBvJk4TsqZLtlgKbHX8a4iEzefxlJQ0ZQjj99dpT8Xw/Nm3A1czk+IIghNa6HO6yt9+HOvbyonueCyLJavk66bWrd7/UcpZl54Cdy5cmmJPIReU0nye6VMUFbOYqGXxXE0F93SS9Z9Qz4EqYd2XpRDmG09jUyu6vq95/WKCK+vn3l5+UTTzkARaYjosuYqAqpLIabN8FgS1x22M02C5b2KQn6/rLe74JLXI7V+shQnlvEifNTvNIXh+gCuUiZLMX0dLz3KMk6Xc13XAEDaeh5AXKd7OSPKkKq2eNZlVJahq+9KrluAFV9X6pfHE7K0zO0yTufzkZKhfD7qZQjrNyWn9ezT4Od14+nYU1i8blgc7PJzviX5RshDGke3kEcC/fjlmZRwxfV/fCwiLZ+MnEbtqjC/8fq+cq4fLIBFWn8R4bcIDPWIZRmW9I61GEuhL/BjCdvH47k7c86wVn5aQTi98PX2SrkpeCi1YB+vDs+rSE/kKfxu+R0Ntfr0+ol9vy+4FrBIQSw/JkhvyNbx3jANIbGUcL8YJFEpyculVdznby87cQlNlWFzvVsGrDSo4G0oeb1f93dCLUsl9fP2Ob9weW9ZOA2UgK5jqAiKM+yqULq8pjphOLH1sEOx1lFO5FKn+hp6ua9Dez0bILH8ev4f5Loe8+V2RE7PqhJQ16zeK0cgsW75WWpNvJzE6VnB0i6mg7AZ/04dWNezjEQaMUDq/3mddr3Uv8VzSi7asoHLsl/Vi3NVPN16vqOei35+4lceUrXgw+WcFNwN4ZNpYBMsPVZ+MBY7DIFcjq/lccTXuZdAShoHP/3M9eVp5QXhfn+ht84Yhu5KPUZVxXzi4qgKrSmkck5yvRRsRJSkrSHemTYCztYx5qmcX3tNFY3vjSPWqeCnAx5Q2syWcspXyhbHUaQpGMzrb91PYbtCjguMc9rSZlFBpUVYrjMUEUE8hXwJ5YeTr+dZv/dlWEuWZN2vuye8LURjlHoulJFRp9SzvJ7uw9+SBkVp2hJel3Kf3hQ1VBpSho6rp497K0O3LMUHwyd1g8sbx3qccr4wQqLOiLN/7Yh+6/Ud5bxApqUUaX0uiyMFOK7WW8KyrHWQ8B5LofJ7koHqFTpougdzx2zCnJF5MFsPRbXu77RWp6Yl5MMKxZ5BfnrdSBylZc3Y3UVwFVwVVaX3jgPTbHmruA/F5wxrnMdVjffL0Ahg84CMDaFhUyPUMz8ha5mjdfn5kPMaHCI+1vxQQkGXSJZYet+LfsU58zrFYoFTLVYippIxV5UOYxmKKShioUSa66IIrUJu4nFwPd5X7qA85xLsdZF1AKg4fN1aGdpEMzhYIraVPpLLNcvlh7oLSQMrDW0dlRZefV7QVyIEbS1jU0PtItMXb7GcdNrHOLYjmlD1Cm0zPo3HdDX/V3Rj/Bbq/Pr1TeVUjxjP8oZ0CUxkI5flKEMgkTQpC6JNAqdHAJoWsrzT6Xsdiwe91uNMPk2zzM5GgqGsZsGUBRsvFtyQWHDOOKalVW5NaE3inHOEh9K24r64b2Hfbpg7z+cTb53hk2ZC23ooJyH8SAAXs8mwiamgrdFEllKJJqxpgjZl+uBxHOy9Y/N8aH4RBCQ+i4Ab15VKwYh7CEPnp7UuI+iCG8xhzJney2P9ZynTRXZEE8a6Ih74a+EjjwSWeXrW9A5gmRvLeyi4mIpPQSQP+GgrkFN8WijdRXDreZsbIgH7zSqvAFS21a20hIDgFvF8XrFKQ7Uj0lZyq5I0lp64Ukkzk1Pn+i8wfaI5kYztfRlNIdZRNdCfp5Ei5ZZcp9IdLy+c6DKQwN+gnJkEj2BfssyhIby0E+YK4X0SKVHJC0uBKatouSiZSPuQVCprqNRTTS/plRA6rc0Jqr2c9CXlXl7MQRTRWkBHJUx9ebnyiHNMUEW2jvaOtE7vG611RFsIXVO0t6WYqro8rDaN7ycMnzbx3mLNMrY0D1GUrBfZ8Er6XiD9aWVXNjITKwXJr09UlICWC2qVdldWtfICJcSR4DsTWYovhVQUxazizHhmpejqihb0k3W2Ol0KHhl3npL9MVUTgjmtIHndTCrd+ncgpg9JoXSnBW6vKIN8HngihjJltVwlG1aKYQuFqCqtjFqtecnkUjajkoRVjjFnGXbVqGosGTXLzxih05X3OIEueg0Lf/v1HeWsm9eLsFzD+3R3acU0H4j5Wc/pW88Fy/omqwIEtciWFSw/SxyeD8jmiHhzZUjrdOciihDJnGWBSQhadanz7/K8ILQWyjY94a+DaKPvN/bbndY3RBVDaCI0aSuWUFVGKiNWRiys6JjG8TxQTVHLmAnCQzVtTJX10Jd2Us4moW0KTAHTqnFWUkQ8DIIsoyXnURJdlFxIGkrVzHJaKKatdE0onXwwAKV1rGe9cgUSCis4ahFmlMIFqrhERLXkS4G/QqL1U1p3T8RW4cpleZahcieV9ISUpxKXhxrLa4IHmrhclbstD3rmQuQMH2q9CeE5s9qJFDSShKKa5Ze4piiROZFCyxyMs651xaPf0c7vZGtL01OVLvg+XL18tcinbXBON1/WWokMGRePWV4zpOiUJDNjzoM5B2IzSifrIRRZ4YRdkbQILx+Kcrn36/kkhEhwdFO09YTVhqVHub28cnt5pW8boi1jyxTIPIaoMI8jDIaGouvW6a1H3fN5xAPShF0ZvwuRhe294zPSNOIFzE8DInmuSJjlIiW0WopeyCIFHwlPGkIlK3FmZCkFIfzjMotZqI/1P5VIqNjeXbD1B1aAnoordRCvZ5OkM796r+uz00iGuyzBhazlUjdykSY5/5a8ZxeJRJieH7v8lWgqM9nu61paZp3PxJrG+i7HE89h5U/yFTmDlOsrcim9sIqHzyRQZcYlCQ5iH+/tWiL6c6/vKOflAvKtKoRHcKzLGpz/P+NAUOY03BVpJzwzm8uzhXsvi31m5oS5mC/NTsU6vaCkP7+kzdURKWvo6/P1UOvZr8ydOtIskjStoX1nv7/y8ukzL6+v9O0G2rFpq7BQHihKIR4Qdg6YA92U3jbMGlPA5mBaxLhCwZyJ4my98xxjWdwFET0VjDBoXZUpcrKvMjtLrmXdUylexB3xl6tAIyIECYqUeRgxz7pjYWtzOfMzSB4gYjm3ZPQYmCqTQBPmvsITvBSvrRjUvKCsn/FcGh3Sw1TZKWqe/mF9Jb+3vCW+nruVd0tZk1+5Il+S6ykvC9ILq8Z9AcFUXFrrKZk3KNi6YmuvcEOYZh88+4ovrzqUziPDbyrs+15O6Lt1zutrCYiX9/j4+4svjMBZiLqQVBq9IF4wfDy/LyrB8FmWs7DPpTBIqXsqptQCFmSrB3B6IK2sZS1XnTvrkcMMGwZdafvO/dNnXn/4kfunT+z3l4g/dcMYqyi9jEcmBTRhjs3BHA3doiDfmjJHJNNUGwuuzQniND0N24Laft4nXK+/FtgXGaMUuXxuxT/rCAreUvgsno274iqRhb2I5ekyy5MpK2RJt+yXR1I4pOqrhaCqDFMZYbHzuecHWMmrZVadD3pWz1kkyR2XY6wPJHOJShTFt65VedJYOwImF6NeZY866Xl98enKekiyvVIR3Zl+VgvK85mlQ6hkWD3KcjT5XLXpiTLsiib+/Os7pZS8cwl/GBY04VlasI9mo+SlBIbTOjhnNnTFqACnp/OEtlLvu60YU6+Y/Vx/CtrmUl2v4vxdKm7xGVUBbQGrVOn7nfvrJz79+AdefvhM23bQxjSnJ+1qjhFp98x46tZDByINzBgDV6Ie6gGbT+OSsSGZzsh4MDzYxc4uQZaIV9PLine truncated
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"x_dec = TensorImage(dls.train.decode((x,))[0][0])\n",
"_,ax = plt.subplots()\n",
"x_dec.show(ctx=ax)\n",
"ax.imshow(cam_map[1].detach().cpu(), alpha=0.6, extent=(0,224,224,0),\n",
" interpolation='bilinear', cmap='magma');"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The areas in bright yellow correspond to high activations and the areas in purple to low activations. In this case, we can see the head and the front paw were the two main areas that made the model decide it was a picture of a cat.\n",
"\n",
"Once you're done with your hook, you should remove it as otherwise it might leak some memory:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"hook.remove()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"That's why it's usually a good idea to have the `Hook` class be a *context manager*, registering the hook when you enter it and removing it when you exit. A context manager is a Python construct that calls `__enter__` when the object is created in a `with` clause, and `__exit__` at the end of the `with` clause. For instance, this is how Python handles the `with open(...) as f:` construct that you'll often see for opening files without requiring an explicit `close(f)` at the end. If we define `Hook` as follows:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"class Hook():\n",
" def __init__(self, m):\n",
" self.hook = m.register_forward_hook(self.hook_func) \n",
" def hook_func(self, m, i, o): self.stored = o.detach().clone()\n",
" def __enter__(self, *args): return self\n",
" def __exit__(self, *args): self.hook.remove()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"we can safely use it this way:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"with Hook(learn.model[0]) as hook:\n",
" with torch.no_grad(): output = learn.model.eval()(x.cuda())\n",
" act = hook.stored"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"fastai provides this `Hook` class for you, as well as some other handy classes to make working with hooks easier."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This method is useful, but only works for the last layer. *Gradient CAM* is a variant that addresses this problem."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Gradient CAM"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The method we just saw only lets us compute a heatmap with the last activations, since once we have our features, we have to multiply them by the last weight matrix. This won't work for inner layers in the network. A variant introduced in the paper [\"Grad-CAM: Why Did You Say That? Visual Explanations from Deep Networks via Gradient-based Localization\"](https://arxiv.org/abs/1611.07450) in 2016 uses the gradients of the final activation for the desired class. If you remember a little bit about the backward pass, the gradients of the output of the last layer with respect to the input of that layer are equal to the layer weights, since it is a linear layer.\n",
"\n",
"With deeper layers, we still want the gradients, but they won't just be equal to the weights anymore. We have to calculate them. The gradients of every layer are calculated for us by PyTorch during the backward pass, but they're not stored (except for tensors where `requires_grad` is `True`). We can, however, register a hook on the backward pass, which PyTorch will give the gradients to as a parameter, so we can store them there. For this we will use a `HookBwd` class that works like `Hook`, but intercepts and stores gradients instead of activations:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"class HookBwd():\n",
" def __init__(self, m):\n",
" self.hook = m.register_backward_hook(self.hook_func) \n",
" def hook_func(self, m, gi, go): self.stored = go[0].detach().clone()\n",
" def __enter__(self, *args): return self\n",
" def __exit__(self, *args): self.hook.remove()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Then for the class index `1` (for `True`, which is \"cat\") we intercept the features of the last convolutional layer as before, and compute the gradients of the output activations of our class. We can't just call `output.backward()`, because gradients only make sense with respect to a scalar (which is normally our loss) and `output` is a rank-2 tensor. But if we pick a single image (we'll use `0`) and a single class (we'll use `1`), then we *can* calculate the gradients of any weight or activation we like, with respect to that single value, using `output[0,cls].backward()`. Our hook intercepts the gradients that we'll use as weights:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"cls = 1\n",
"with HookBwd(learn.model[0]) as hookg:\n",
" with Hook(learn.model[0]) as hook:\n",
" output = learn.model.eval()(x.cuda())\n",
" act = hook.stored\n",
" output[0,cls].backward()\n",
" grad = hookg.stored"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The weights for our Grad-CAM are given by the average of our gradients across the feature map. Then it's exactly the same as before:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"w = grad[0].mean(dim=[1,2], keepdim=True)\n",
"cam_map = (w * act[0]).sum(0)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOy9S5YkyY4ldgERVXP3yMxXr8hqfmbNVXBHPeYGeiFcGhfAc0g2q6pfZkS4mYoAHFwAIuaZEdH9esAapGbGz93NTFUEnwvgAiLujj+vP68/r397l/7/fQN/Xn9ef15/fP2pnH9ef17/Rq8/lfPP68/r3+j1p3L+ef15/Ru9/lTOP68/r3+jV//eN/+3//U/MJkrgNRXBaoCbQp3hwOAAw6Hm8Ei+ysiAAQCwNxhW1LYHJj1s1Lv4xC4IP7ND7U5YXNgjgtjXJhzwmxi2sScBvcZrzWYGRyGCYO4QEUgUIgKFAoV/sK6M4jxntzzawpFg0MxBvBP//Q/4b//p/8Rr6+/1Pe6nrBhGIOf3fqB4zjRjgMQgcNh0zDmHXMOmA24D8x5weYAYDSLApgb768JuGQOiOPxuOO3z3+DOZ/R3Pg9ABZrPW1CVGDg390cDudzi9S+iKQNVrgIoAoR/t3c+fDgz8I99g5cSeP7aWuI3V5X7pU73CcAQdOOhsZ1dwCuaFCINKgoWtwb1xsQQX1NhXfRFFBVqHNnFahfIoDGjsMN7g4VxNop+KgCgcfn8/4k1wSgnLjBzdE670ti39wcZhQKVUHTtj2whzdzqDgEFmsQz6IKyTXODRaFCCBwwAw2DTM+Hw7Y//nv/vf/+H/8x//wR/r3XeWENIiEdoIPyVvjDaXiQkJuRMoVh24CLpQ3hCIiFTEWTtYLygDEIpagm1Eh58QcA+YWQmt1T/hGSYhvJRCNG9qeQbbPlfjPIXAXqHacLwc+/fQTXl7e4FBICF0+vMPhYWhkDngIMZSCrqpwV5ihFIf3ANg0oNFwtX0t4ZB471j2UJpQZO40RBQaEqtOhU1LU4ZRUjnXmnoIcn5NSuVk7a97fE8hml9b+4ElEmVEAS0FEMk9pxFIo6hQ5C5zpZfRl/oVe2R8fg2ZkiV5cH7a2jvJvd/uLeUr98odXnvNz7FtfZfzydUImdiUGmG84CXBT/LEv1vIULxL6E/TeJ+1+LvH+8PrB8qZXiaXxMJK1q0BHouTglWvBSAOGohYJMln8xJWykFarnqblGH+nNn6MxXS1qKj7o+CqdgffluBXNtdMLafsfzs0JS31094ff0JrXVMU4g2iEtZXhWBayo+LbLBgbmUIp+L9yylODCn0sJojz0do9etL0EXOGxtS+476PmWXZK1MbJ+ZheIVO7cH4ldS7G1gDgqAFSf4p56jvoMPnuikFx/CrVCXAOxhHcqb7rf4tqL9W/fDESugS/ZgNea7zuc8lRyWPatBA+I+xUViH10OOFltxVBoDhJlGAG+ISLocXX8rnL8Tgop9jXXWuVdvX/3vVD5SQ8Cou1eSdZa/zhSoAqpQjLuW2vF8B9LT42GFbv5I45Zy1aWYXNci/DsT1qyd+6S/ftnl1KjtPoWn2Glm1/e/uE87yBKIfCBRGIzPBAgtYaRDtECbYsoT7WmqWwhtYCv/v3x3UnpKLntfpigRHklnhA+W1d8llDg3Wz1mU6HfBCRAijuYQoN0ggsVQUzjIEqfjlKRUfJWKhES0Flu1ZJZyKbM+1i+tHJ1CQe3MAT44gF0UAcd5vhjPAQi51z3C4rndJZ5h7L/V1B9yI8HK93aBiUJWAzPtN4MkJwBUQhznhbSrnQhzfvr6rnJJQMxS0QFDFDfExdSNAwr20OO4LkCS0oldQqKQUye9ldBO+hLVmhLO1fOltfakoY5dN2T3fPRSldnuDLLvipoCK4PX1Fb11jGloTQuuqCoklEJE0HsL5RRMoy8UAeYYcADaCIndB+bgvWhTmM2C9bsfoNNSNG2Y84p/AxaYgOu71gX19LvlIiSUps9C/mylctXia8sTikp82QNCK+DLe+da7wK2qTtWuIC1v2WoYs0V8WxS8WRKS8Wy0MhNOGNQiRfucor1fGlIEk53bSDCsGXgwxMbvBSMyvvhfXYFXUKZD8+1KkuZdyLLsG13mE7KPBXzg0P5g+v7yum2GdcIhuOD4b+PGZ7uc7tXj4ctF49tfVPJN2EJVWIiyYxJoMEE0FPCyQF4LK7otrTLkpeVcm4u7UHAJEHEQwIDoCbhVWNj+0GFmwZptNoqsrykOUQjBlJ+pohFgkMxbTD3o42eyhQmFJCE3rvwlheJBdKmwCWhwJEwCotjcJhP7HA314UGSpmgkEzceSG7VEiVZ8e9YtWE34b0vppJjtg3Ln2ikvBKqJuv/Zf9vvPLC4whY7LNIacoxKNa3MXKTziWkXmKB+sDItbVRDWCtqEwuMNVADOINmSsyBhsOZD9fpH3HPuv6hBpgOyIUssBAZl/yZveFNw99m3Ls/zB9V3lVMwP2SsNqOWwOZfCJYQCQkniHlXgk/FBoviyHLURwg0wi+89b8CMDBdDoYAoFWtKKCdf4eFJMzPslgsaOgcmaboq5jTMMSGNygOXeCFjVsJZx+PxgKMxMwrgOE7GmgCVVCPus4FpwISjNWU8AgdaJmYAaQI1fvb9euA4DrjPZ8iN5ZVUWwiFQyQVzMJ+CGwa432PrQ6jIKqAKMwFPmx7XxohNwfEuEmJYj8Yz/VV3rxNvk9mux2OWd4tfloi4eOykJKEEjuXWMP6+EylAyQwgTnjdpFIfDmNcWVy0yO5I0Mu3quVZ0iP2VoHoDBEzO6gF468CW2CrnAGzx6tZDj+kigxdUDFMc2fdKKqDgVGpOB57a4lKjCisD/UPF7fjzmR3oL/ypR1WtEMW9LSJnTJh7UIwNMKWtz4yuds2bRv3sJKe+/4LCF2vs4STrrD4p48PUBY/swaq1IFs4wwx4RDoa2jyQGRjqMfaK2HZaUXoyXWgLatFFSbRknD6eFt4jiYtVWRlV1GJDJcYGNtZMYsaa0zHssv1N5+EBzNjXlak3hZGqgwhnusBfGl/Jr3oKWI+5tKeT+JkCJu+MmgLA9KZMO8rMvz9wGnwTCDm0ErNmyBGJaBoJL6grsV/hQQL6/MXAANmETiKT81kzOIZI45P5v3bmha6cNAP/HcvqSLkNc2RAGYScF91cZ3y/JgyrQ7bEs6bYCChuZ7bhM/VE4pC8D3U2TWiSFIfmhCQT6bhWc1d2jrgEpkJvk9dQlj51Gz22KMzCC6wWzA5mBsVmWTNBYBrcOzJDQp/S2nkDHIBnZTgBpLIz49k79AU/TjhvP2gtaZ6LFJj8GNpOaIKr1pbIgi4aRgmOF6XFW/RAktlbm5MtaxjINyy6g8sr1XlhL2mD4lT1Ydaj0rBJHpqLWQ2kNZaxfvvQGyUKbNCO4ZnMwvhDdkzkGgeVugMy7vIdvbxMszSZiPsr15/S4B+TJrmt4ylaJ+xg3uup4vnyWWhGhmwjFrzfbYcYfiGaMu3BBqmZWGWqEwECWHGhCXnlMUUHNEzv7J0O35jufyzLevHyhnPG1mtbbNzFWIxOfyALLsKYU2rVIkyac9QyFIwJ4ozIZJdjfMcbGIP+cSjhSzgKpuaX0R1tp3tBawD9unLaVo2tC0Y4oD02AiQGu4vb7h9vqKfpxQaQHrI45DgAGRIEjQIPTzgLaG42jwy3BdA2JO5S/jQWFoqui9w21GiSCVEuuXRH4462MuyFJWCpt4lQPXC5FeN5QR65urzBT7lom0ikmXUElo0Cp3xX6qbrHpSv4tRVJmSytJ5wEtPdZwPY/6jsT+KP6S5+cK0cpkJCG+bj+dCkxFWJlsr+dfCAJww9O/S2F2zXIw8VOLvAwNAmlYxVGy3YfAxSp/IBVkf7zXb1/fV87MRiUydUIB8VACTXAZIMGlFDWCTgqvLgYGfGVggVycvPEFB9zJBErFlA0aVDYWCtUZ9UOlKAg9aIXCtdZeVpFWld4TEvUobdB24nx5w+unn/Dy9gn9PCHeYTajkLKEW5oSrhowRSCDrKnWFN0aDYsNzBFkAThhobE+erSOxxwl2JXfizVUEahSkVW2Uo9lMiHhWq51lA0kfoUiZKEdSuNissPCRD+x3qULKTgrluNHG6qkBJSHyPVV6axpelrOFeosxdR61mQEyS6qEvukz3A236YUy5/F+veOKEocnln8ZE6ll/Qy5Jv+L8MnzMbSIwYnact1pLJO+72spT4U8nQEe8hrPdzLP3/z+oHn1O2uA4ZG1kZ0ecP9MzxXKhgsw4zmXXmzHsrptrNdMjRdgse7t3rU7XHLU0h89zmlv3vXEDUP2FbBOX9qToPBoNrQjhO315/w9tPPuL19wvnyCm0dig6bhGzMvRiguakSigcakjGg7YCIozXBGHx/kYZF35pUFF0xZhqlBSXDQ0OY+a31CG9hthYE4a0iO1kwIr2YAM/0tT8QDPeq4y33tD6zDCYI/9X4c5mcWitKYyK2p3Rzs/KJZH0Nm1LtciobM+j37hSV+Kpt/2OYqEpjtwgNsTypuLuvTe8Dqb2VxQuEg8nJEqDw2GaOTGAlLRG+jGn+OxGQeFJZ/fmZ/+D6YZ0zV1AibrIQIvWtHJzxwLbQtDwh0HUvvJvIicTLlkejMoYlcoO4VTIq33r/s5R1RyP1t11B4zOMC9OixEAZU7TjBS9vn/D201/w9vYz+nEC2iobBwjmGMHVBaDMyELy+8AcFwCPGNPCCxqzYG0lHQB6bcOH5491JYiw8JzxDA4ihzR0kfLn8wVzqTFL6bpio5Re8e3fSKj3bO72WnAuIT1nKmd9MfaTsC3rxZClNKyJBupKqJdeMJVl30SkJwyElIqT71k7mwYs4SnqzxLFkt2QAFn6VMhs/8lNPvOi587nWhUChmAS8r3f9+7BMkTI2inXTn37qbR/T2Hi76/vKmeLine truncated
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"_,ax = plt.subplots()\n",
"x_dec.show(ctx=ax)\n",
"ax.imshow(cam_map.detach().cpu(), alpha=0.6, extent=(0,224,224,0),\n",
" interpolation='bilinear', cmap='magma');"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The novelty with Grad-CAM is that we can use it on any layer. For example, here we use it on the output of the second-to-last ResNet group:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"with HookBwd(learn.model[0][-2]) as hookg:\n",
" with Hook(learn.model[0][-2]) as hook:\n",
" output = learn.model.eval()(x.cuda())\n",
" act = hook.stored\n",
" output[0,cls].backward()\n",
" grad = hookg.stored"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"w = grad[0].mean(dim=[1,2], keepdim=True)\n",
"cam_map = (w * act[0]).sum(0)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And we can now view the activation map for this layer:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOy9X5JkSW7u9wPgJyKzqrqHQxrNaJKume4KtAFpPdqLlqC1aCt6vWY0icOZ6e6qzDjugB4A+IlsDYea4oP40NGWnZWZ8eccdwfw4cPncIkIfnv89vjt8Z/vof9/X8Bvj98evz3+8uM34/zt8dvjP+njN+P87fHb4z/p4zfj/O3x2+M/6eM34/zt8dvjP+lj/LU//s//0/8SyeYKItfvVRVVxbhhcucWLxxxY7hxoByq3FW5mXEfwiHBkOCuwqFwaDBwVgSB4AEBBIoDERAIIcJajs/JnCdzLtZcrOUsd+Z0wpNt9gjcI79H/gxCICDUd8n3luCmyk0jvyS4GxwCLybcFG4KGs5//0//yH/3T//Ijz984rgHdsDtODjfg/e3hS8FORC7g9xwlLXgMeExJ49zstxZvnjMxVwTIhClrssRFUwFUUE1R+L98c7Xrz+x3InIL2oOVginB4/phCqnB+dypjvTcxxXgAdMh4WyQvAQEMVUEVVEpN4yMASRQAHNoUKiZ0U4hiHkPPH8Pahx914ciAgRgkewHESUQEHAQ3DAoe4NQoSIYAERgRP1HsF0JyKuD6xZjajP9CD6ekXJjxGCwEJyfUS9X42fe64fj8BU+8U5tu64572IgNTfY48FIFErK+ckIqdGRPNeRci4l++b4xyEO77y/XNO4b/9n7//3/+PP/5v/+vfbJxgiFyjEuSNxjam5wGrSUVQEVRBtS6aWgSy3wgkBzj2L6X+L9fzJJeDu+NrseZM44zAV0A49LXsklAOxIcLE9m/zed+fH4OYF7vfpkY95vy+fMPvLx8IkLr93X/4QR9Dw5rgighA0cJ0omZWk14jl3Ppi8Hy+uwdh71fsjz6EqPPLjnIJMTrqp4Xbv0fyIQ9b2ut8d/z8G1bnrE84ft1CgjVa4ru6bu+l4LXkFcr7GWvH8h10Eu2PpQT+cpcb2v1BwqabSy5zNQueatBr6MI58f8qurC9JYg22YET2uz0tC0F7IUu+9Hx2Q+kuJnpPYq6ieen1mzp9DPF2zKEI6X3ptCzVH/NXHXzVO2au1LiD8aaDYnqjmBGrA2hG1Yef116KW/Ld7L+zY3qVHL/a7Us/zGuD6/jTo/uzC41poSkaO56mDnLAgvW1Eu478VA/Z1ykifPr0hdfXz6gNgkjPWB6QyIWDyRUN3HEmy5W19iDkpXkQIQiajsnrd5K/j4hapFEv64WRf6ecgUQbXU5ur9c9E78uW8tltOK9mNmLRJ5/4HmeMso+L6BAcvFRRlTXgBiqNa49/pGIKNeDdszJMShDkbhcZhpmGp1EbCO9HEdctxbPjiW2w91WXuu1Zzb9Yd6X1jpTgVAtW4v9Mur+5FqI27B7DeILcEx/5eRxQK8xfJoWkUYeUqhwf8q/+fjrkVOeU1Iva4zrWv7iI8O9hO6bDi5Dbk8lIhkg2rJrEX94pwh8rYIBO0RSI79fGs+X9Bdvuie2F6C0l9h/TTgcuPQCFj59+sztdic8kGG1GAteShS8HwTGcuNcbYROoDtCt/cV1jW0Ihlxn2+6x6buS1WfIFZHTPY4JKR0/NdvUwtVRFDRMm74aGm9wnsScnyk4Oe+ZgFFEMu1vzyelkA9lzLicALvIFzv2vBZypF+xEqyrykdj7SBdSBo5/Yhaj29wQfP295Y6roSXVBz23PRRl3YZ0flNjR5+qICSP6pIKk7gmdULwTQnxt157EdTP2/htt7+q7l+G8+/h1Ye4XeKGDfAGlD1fIFFFSBjAouTrgQWhijffQTDFMVQrT95ZOhXRHXKwdorO7RXpG8nhrQa+3lxFSMywVRcNI/gDJ5moQLCl8LRnh9eWWMwVqLMUZFDjArP985mhpilveycm2YGOdcCddUMTNWOHGmgaop7msbzLN5NGQ1M+acT1GioGOjBi8E4r+OoD13iorifgWVvdDkchzXIsk51nIMVOSWylM/RJgap3Zke5FzLTx5upbgKarJ01r4cMWg7SD9aQ204UTFnSeU9eHF+77Z925qGbkj88yGfE6g7ogavv92rZt93R1gnpziXj89dpGpQzuERl7PqKTfxzuVkw+T9Rcffx3W4mUIDT2i5y8vSEBrJvbYxK++dyQSyXxJe8LbvHVDjo+5YC/AIoBWfs8oWq+NawCpPI9fLRZvT8YTfCSQSAiikvnAUMUITC/Ca4wBIUx3boCHV46VuZg7GUHrc8UUgcwv3Dgb/qphAuIL32MnZD7ie7j2ZAtIXcOODtLRtiAWQcR6ytt/nXsq9Fde7V5bTiAOYu3na/ak+AJJcioBWDmLJ9KkDW3P1FPUufL7a7FSTjckKqJfUHF7iufHU4AMnvP1a03tf27E9eRwyzBFBmqXcYasdO6dC6kiakg7+w8ergb+GWwUpMn3zzkSfgVtRZ9e+uxE4ild9afX/NuPv2qcWpPvRbz0oiWSoEFHegDpqCNPHksx0zRErYWrNbnyFE1DrghZudvzpC/3NMoAdt7SYyhX1KjoElE8WkeW+ltEmq9pLUACXxNXA7E9gJAL83bccQ/ezweHwfSFOdzMUJUkYjSZOSdYvjin81ggMgoR5L1HXZeYoKastXh/TG63kZC0Pvo5T0lkYdBMYRmk49vRObXY9wLXJwitydYuZ0aypJcDdIrYfMpBa0F3yGvvX/929wtl1GuW+44Gz4ijo0Q7mwaPIUHyasmi+hOWcdgRzN3L6NoZ13p8ev92ROkMfOeOfR+qAxHD0UQNITiaDrYgZ76X/yo//DXWlLqWVY623ptgRaBaCDAuMi0qZdBah7QDjXZkXtzHX3/8O5GzI2bOfQW+7Y21vLkiaYwMRiiqssN7U7YZBSFU9mDsPG9Drmt8Li8T6Wnae7aD/hWUau8aUsZAJEPYUboQgNSAmihW7+dr4QhjGGMMzJQxDnQMRA3EUVPUyiDJaJiQPb1vFFvpvljxyCFSkJCCn7kURdMh+fR93xuibyhYBlApfzOBzyx0T4OWMwqvvzYpBoUU9htchidSiyqdbTLsbZwbg9RcdDTlQ7Tc19DX31PVC/RXyDOnM6q8sup7R259mv8cX/d15YDleLejoqHhhb6kHVUPPBdKaK+0IvPFnR6Ra2GvKJXyS7WWGtKWQTdJB+nsMv2wXAvpwfL6CiJ7eAUPfxqx+r9+hPR/6fFXjdPkBSRqKHICxQQJYYljMRhyZ3DD5OAmg4ExlISNEZgahwlDYCgYMLTery0wYO1w2AMShM+Mbr6I7UE7p6hoKGxqvvF8ExlWHnZDxcjPVxFMhMMGh0rzaxktVDjud24vLww7nsoJipYXzvKWFqLwgqaKKQwjDW+9o2IJfcJxWRzq2IAhgnqgLALHkIS9BZeSBYQpwmjauaNMkU0CGOkY2+D6RjtKKlH33LBXdh1QtRDOnmtQudhSoZwxu/KSZI1Ifn7B/+XBRZIWBH3Ko59Q+f7Sp7nLvxWTXU6o/8tRLaOsW9wBMngKGg2d0+l0vhexkLKLiMDqPdvveQRW99Kr4AoMwXJBIjnkdF4N7TNJUFGGjkSTwIqVV67pkAVhFSHU6dSeh5o3/Ssm+leNc+j9w88qXesTVALzwYg7Qw6OMEyVIYJpYJ6xzQSGSn4JGF6+zuv6hBBHwje5Awmj5jrxNYm1coqjc4q8OaVKIpJ5nj7FlmbGekKhnlOYnxBsGMcYKPn5adHGy8sr95dXxnGgplTIz4H0RMEiwlozmTsVZICacWB4BI/HCTJBByrB0InEaiTPwFhr0qyvSYEMBSuofNdgGohnMT9rIQmLDDCCFWA7MuaXb2Sa2HWVEa9yXBAfDI+aj3TAOdQm7QLaaea/0/GlEwyJEgBEzZkgSjlInuBtOYrKZT3SEbh4lbtKiNJ5aDmbzfrKE10nV1Tr623k0WOQ6K6ddT6/OY6sN2pB5kq96r9GKGmYafSLKx1jo7VceyaGoTRLOyTJJS8izUtg4pGp4POj39E+VER+ZX//5l+AQ16u1JCEaF1rCwkO7hwyGIxNJAzNGqNpUfDh4JWvUpDJJzWFIIJJ5SPSuQPAAl/JaBb83OgGii1XQgtG0eofaHp7EyikPzbaY4LgWTivRaNq2HFwf/nE6+cvfPr8mXG7Y1oESgiEIe5EF9Y9WGshMTPiDeV2ZOnCV7B8IjExVdQWZ0zwQCyVUu+PBR6YBZleejKjorgEmHM3gSVIkz+xyqODbYgPUXnmzrMQNJRTBXVYIrmQ6AhYziAyDkhIvSrnTmopbnfXuZMoEvrk9KqUEJkqiNlOWXZWQkbvznK8iDmXzsla4QWiV3TxaIPYeIoWWbg7Le7Rzf7mpfbPUaivllneU4sBkB3ld+za5JJg5oSmoxW9ymKbOW6V1RP0jvpsiWt9VlHtIo8oJ1Tjc3yvcd54+VXekLmTVB3ppjcGl2RvSBISpjCQlISF70nWnvCIND4qz9H2sheEzqJ8RtOdl+0xlG2oTkbxnAyp2lVDhiejJQOPEkUIFQGxVkbQ48anT5/4/MMPvHz+zO3lFbWReWZbvUMsWFI134JRaOZIGieqwjDndgve3ycCjJ2TnHgk2WGisGYaugbWi7L+HSRum5qfF5JyRw1Po4xGCrnyDhGiDNMTrBUZokxPeZ5vYNXO64LzndNaoRntaFfesudB67k7NrftNvR7WsjLA5enEsuv4GkVh3rGdsQ0TbjZNINHFN7qqJgcR16nXNCWy0AFmvHaawb4MAIfau88EVsKgT2x47olf6vgWBt5EmU7fCexlZh6gwDI0tqFJjxRXQT2/yKgrse/Y5wFawuIe3hFsryLine truncated
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"_,ax = plt.subplots()\n",
"x_dec.show(ctx=ax)\n",
"ax.imshow(cam_map.detach().cpu(), alpha=0.6, extent=(0,224,224,0),\n",
" interpolation='bilinear', cmap='magma');"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Conclusion"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Model interpretation is an area of active research, and we just scraped the surface of what is possible in this brief chapter. Class activation maps give us insight into why a model predicted a certain result by showing the areas of the images that were most responsible for a given prediction. This can help us analyze false positives and figure out what kind of data is missing in our training to avoid them."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Questionnaire"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. What is a \"hook\" in PyTorch?\n",
"1. Which layer does CAM use the outputs of?\n",
"1. Why does CAM require a hook?\n",
"1. Look at the source code of the `ActivationStats` class and see how it uses hooks.\n",
"1. Write a hook that stores the activations of a given layer in a model (without peeking, if possible).\n",
"1. Why do we call `eval` before getting the activations? Why do we use `no_grad`?\n",
"1. Use `torch.einsum` to compute the \"dog\" or \"cat\" score of each of the locations in the last activation of the body of the model.\n",
"1. How do you check which order the categories are in (i.e., the correspondence of index->category)?\n",
"1. Why are we using `decode` when displaying the input image?\n",
"1. What is a \"context manager\"? What special methods need to be defined to create one?\n",
"1. Why can't we use plain CAM for the inner layers of a network?\n",
"1. Why do we need to register a hook on the backward pass in order to do Grad-CAM?\n",
"1. Why can't we call `output.backward()` when `output` is a rank-2 tensor of output activations per image per class?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Further Research"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. Try removing `keepdim` and see what happens. Look up this parameter in the PyTorch docs. Why do we need it in this notebook?\n",
"1. Create a notebook like this one, but for NLP, and use it to find which words in a movie review are most significant in assessing the sentiment of a particular movie review."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"jupytext": {
"split_at_heading": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
19_learner.ipynb
-1929
View File
File diff suppressed because it is too large. Load diff
20_conclusion.ipynb
-88
View File
@@ -1,88 +0,0 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"! [ -e /content ] && pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "raw",
"metadata": {},
"source": [
"[[chapter_conclusion]]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Concluding Thoughts"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Congratulations! You've made it! If you have worked through all of the notebooks to this point, then you have joined the small, but growing group of people that are able to harness the power of deep learning to solve real problems. You may not feel that way yet—in fact you probably don't. We have seen again and again that students that complete the fast.ai courses dramatically underestimate how effective they are as deep learning practitioners. We've also seen that these people are often underestimated by others with a classic academic background. So if you are to rise above your own expectations and the expectations of others, what you do next, after closing this book, is even more important than what you've done to get to this point.\n",
"\n",
"The most important thing is to keep the momentum going. In fact, as you know from your study of optimizers, momentum is something that can build upon itself! So think about what you can do now to maintain and accelerate your deep learning journey. <<do_next>> can give you a few ideas."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img alt=\"What to do next\" width=\"550\" caption=\"What to do next\" id=\"do_next\" src=\"images/att_00053.png\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We've talked a lot in this book about the value of writing, whether it be code or prose. But perhaps you haven't quite written as much as you had hoped so far. That's okay! Now is a great chance to turn that around. You have a lot to say, at this point. Perhaps you have tried some experiments on a dataset that other people don't seem to have looked at in quite the same way. Tell the world about it! Or perhaps thinking about trying out some ideas that occurred to you while you were reading—now is a great time to turn those ideas into code.\n",
"\n",
"If you'd like to share your ideas, one fairly low-key place to do so is the [fast.ai forums](https://forums.fast.ai/). You will find that the community there is very supportive and helpful, so please do drop by and let us know what you've been up to. Or see if you can answer a few questions for those folks who are earlier in their journey than you.\n",
"\n",
"And if you do have some successes, big or small, in your deep learning journey, be sure to let us know! It's especially helpful if you post about them on the forums, because learning about the successes of other students can be extremely motivating.\n",
"\n",
"Perhaps the most important approach for many people to stay connected with their learning journey is to build a community around it. For instance, you could try to set up a small deep learning meetup in your local neighborhood, or a study group, or even offer to do a talk at a local meetup about what you've learned so far or some particular aspect that interested you. It's okay that you are not the world's leading expert just yet—the important thing to remember is that you now know about plenty of stuff that other people don't, so they are very likely to appreciate your perspective.\n",
"\n",
"Another community event which many people find useful is a regular book club or paper reading club. You might find that there are some in your neighbourhood already, and if not you could try to get one started yourself. Even if there is just one other person doing it with you, it will help give you the support and encouragement to get going.\n",
"\n",
"If you are not in a geography where it's easy to get together with like-minded folks in person, drop by the forums, because there are always people starting up virtual study groups. These generally involve a bunch of folks getting together over video chat once a week or so to discuss some deep learning topic.\n",
"\n",
"Hopefully, by this point, you have a few little projects that you've put together and experiments that you've run. Our recommendation for the next step is to pick one of these and make it as good as you can. Really polish it up into the best piece of work that you can—something you are really proud of. This will force you to go much deeper into a topic, which will really test your understanding and give you the opportunity to see what you can do when you really put your mind to it.\n",
"\n",
"Also, you may want to take a look at the fast.ai free online course that covers the same material as this book. Sometimes, seeing the same material in two different ways can really help to crystallize the ideas. In fact, human learning researchers have found that one of the best ways to learn material is to see the same thing from different angles, described in different ways.\n",
"\n",
"Your final mission, should you choose to accept it, is to take this book and give it to somebody that you know—and get somebody else started on their own deep learning journey!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"jupytext": {
"split_at_heading": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
app_blog.ipynb
-253
View File
@@ -1,253 +0,0 @@
{
"cells": [
{
"cell_type": "raw",
"metadata": {},
"source": [
"[[appendix_blog]]\n",
"[appendix]\n",
"[role=\"Creating a blog\"]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Creating a Blog"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Unfortunately, when it comes to blogging, it seems like you have to make a difficult decision: either use a platform that makes it easy but subjects you and your readers to advertisements, paywalls, and fees, or spend hours setting up your own hosting service and weeks learning about all kinds of intricate details. Perhaps the biggest benefit to the \"do-it-yourself\" approach is that you really own your own posts, rather than being at the whim of a service provider and their decisions about how to monetize your content in the future.\n",
"\n",
"It turns out, however, that you can have the best of both worlds! "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Blogging with GitHub Pages"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A great solution is to host your blog on a platform called [GitHub Pages](https://pages.github.com/), which is free, has no ads or pay wall, and makes your data available in a standard way such that you can at any time move your blog to another host. But all the approaches I’ve seen to using GitHub Pages have required knowledge of the command line and arcane tools that only software developers are likely to be familiar with. For instance, GitHub's [own documentation](https://help.github.com/en/github/working-with-github-pages/creating-a-github-pages-site-with-jekyll) on setting up a blog includes a long list of instructions that involve installing the Ruby programming language, using the `git` command-line tool, copying over version numbers, and more—17 steps in total!\n",
"\n",
"To cut down the hassle, we’ve created an easy approach that allows you to use an *entirely browser-based interface* for all your blogging needs. You will be up and running with your new blog within about five minutes. It doesn’t cost anything, and you can easily add your own custom domain to it if you wish to. In this section, we'll explain how to do it, using a template we've created called *fast\\_template*. (NB: be sure to check the [book's website](https://book.fast.ai) for the latest blog recommendations, since new tools are always coming out)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Creating the Repository"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You’ll need an account on GitHub, so head over there now and create an account if you don’t have one already. Make sure that you are logged in. Normally, GitHub is used by software developers for writing code, and they use a sophisticated command-line tool to work with it—but we're going to show you an approach that doesn't use the command line at all!\n",
"\n",
"To get started, point your browser to [https://github.com/fastai/fast_template/generate](https://github.com/fastai/fast_template/generate) (you need to be logged in to GitHub for the link to work). This will allow you to create a place to store your blog, called a *repository*. You will a screen like the one in <<github_repo>>. Note that you have to enter your repository name using the *exact* format shown here—that is, your GitHub username followed by `.github.io`.\n",
"\n",
"<img width=\"440\" src=\"images/fast_template/image1.png\" id=\"github_repo\" caption=\"Creating your repository\" alt=\"Screenshot of the GitHub page for creating a new repository\">\n",
"\n",
"Once you’ve entered that, and any description you like, click \"Create repository from template.\" You have the choice to make the repository \"private,\" but since you are creating a blog that you want other people to read, having the underlying files publicly available hopefully won't be a problem for you.\n",
"\n",
"Now, let's set up your home page!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Setting Up Your Home Page"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"When readers arrive at your blog the first thing that they will see is the content of a file called *index.md*. This is a [markdown](https://guides.github.com/features/mastering-markdown/) file. Markdown is a powerful yet simple way of creating formatted text, such as bullet points, italics, hyperlinks, and so forth. It is very widely used, including for all the formatting in Jupyter notebooks, nearly every part of the GitHub site, and many other places all over the internet. To create markdown text, you can just type in plain English, then add some special characters to add special behavior. For instance, if you type a `*` character before and after a word or phrase, that will put it in *italics*. Let’s try it now.\n",
"\n",
"To open the file, click its filename in GitHub. To edit it, click on the pencil icon at the far right hand side of the screen as shown in <<fastpage_edit>>.\n",
"\n",
"<img width=\"800\" src=\"images/fast_template/image3.png\" alt=\"Screenshot showing where to click to edit the file\" id=\"fastpage_edit\" caption=\"Click Edit this file\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can add to, edit, or replace the texts that you see. Click \"Preview changes\" (<<fastpage_preview>>) to see what your markdown text will look like in your blog. Lines that you have added or changed will appear with a green bar on the lefthand side.\n",
"\n",
"<img width=\"350\" src=\"images/fast_template/image4.png\" alt=\"Screenshot showing where to click to preview changes\" id=\"fastpage_preview\" caption=\"Preview changes to catch any mistake\">\n",
"\n",
"To save your changes, scroll to the bottom of the page and click \"Commit changes,\" as shown in <<fastpage_commit>>. On GitHub, to \"commit\" something means to save it to the GitHub server.\n",
"\n",
"<img width=\"600\" src=\"images/fast_template/image5.png\" alt=\"Screenshot showing where to click to commit the changes\" id=\"fastpage_commit\" caption=\"Commit your changes to save them\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, you should configure your blog’s settings. To do so, click on the file called *\\_config.yml*, then click the edit button like you did for the index file. Change the title, description, and GitHub username values (see <<github_config>>. You need to leave the names before the colons in place, and type your new values in after the colon (and a space) on each line. You can also add to your email address and Twitter username if you wish, but note that these will appear on your public blog if you fill them in here.\n",
"\n",
"<img width=\"800\" src=\"images/fast_template/image6.png\" id=\"github_config\" caption=\"Fill in the config file\" alt=\"Screenshot showing the config file and how to fill it in\">\n",
"\n",
"After you’re done, commit your changes just like you did with the index file, then wait a minute or so while GitHub processes your new blog. Point your web browser to *&lt;username> .github.io* (replacing *&lt;username>* with your GitHub username). You should see your blog, which will look something like <<github_blog>>.\n",
"\n",
"<img width=\"540\" src=\"images/fast_template/image7.png\" id=\"github_blog\" caption=\"Your blog is online!\" alt=\"Screenshot showing the website username.github.io\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Creating Posts"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now you’re ready to create your first post. All your posts will go in the *\\_posts* folder. Click on that now, and then click the \"Create file\" button. You need to be careful to name your file using the format *&lt;year>-&lt;month>-&lt;day>-&lt;name>.md*, as shown in <<fastpage_name>>, where *&lt;year>* is a four-digit number, and *&lt;month>* and *&lt;day>* are two-digit numbers. *&lt;name>* can be anything you want that will help you remember what this post was about. The *.md* extension is for markdown documents.\n",
"\n",
"<img width=\"440\" src=\"images/fast_template/image8.png\" alt=\"Screenshot showing the right syntax to create a new blog post\" id=\"fastpage_name\" caption=\"Naming your posts\">\n",
"\n",
"You can then type the contents of your first post. The only rule is that the first line of your post must be a markdown heading. This is created by putting `# ` at the start of a line, as shown in <<fastpage_title>> (that creates a level-1 heading, which you should just use once at the start of your document; you can create level-2 headings using `## `, level 3 with `###`, and so forth).\n",
"\n",
"<img width=\"300\" src=\"images/fast_template/image9.png\" alt=\"Screenshot showing the start of a blog post\" id=\"fastpage_title\" caption=\"Markdown syntax for a title\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As before, you can click the \"Preview\" button to see how your markdown formatting will look (<<fastpage_preview1>>).\n",
"\n",
"<img width=\"400\" src=\"images/fast_template/image10.png\" alt=\"Screenshot showing the same blog post interpreted in HTML\" id=\"fastpage_preview1\" caption=\"What the previous markdown syntax will look like on your blog\">\n",
"\n",
"And you will need to click the \"Commit new file\" button to save it to GitHub, as shown in <<fastpage_commit1>>.\n",
"\n",
"<img width=\"700\" src=\"images/fast_template/image11.png\" alt=\"Screenshot showing where to click to commit the new file\" id=\"fastpage_commit1\" caption=\"Commit your changes to save them\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Have a look at your blog home page again, and you will see that this post has now appeared--<<fastpage_live>> shows the result with the sample pose we just added. (Remember that you will need to wait a minute or so for GitHub to process the request before the file shows up.)\n",
"\n",
"<img width=\"500\" src=\"images/fast_template/image12.png\" alt=\"Screenshot showing the first post on the blog website\" id=\"fastpage_live\" caption=\"Your first post is live!\">\n",
"\n",
"You may have noticed that we provided a sample blog post, which you can go ahead and delete now. Go to your *\\_posts* folder, as before, and click on *2020-01-14-welcome.md*. Then click the trash icon on the far right, as shown in <<fastpage_delete>>.\n",
"\n",
"<img width=\"400\" src=\"images/fast_template/image13.png\" alt=\"Screenshot showing how to delete the mock post\" id=\"fastpage_delete\" caption=\"Delete the sample blog post\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In GitHub, nothing actually changes until you commit—including when you delete a file! So, after you click the trash icon, scroll down to the bottom of the page and commit your changes.\n",
"\n",
"You can include images in your posts by adding a line of markdown like\n",
"the following:\n",
"\n",
" ![Image description](images/filename.jpg)\n",
"\n",
"For this to work, you will need to put the image inside your *images* folder. To do this, click the *images* folder, them click \"Upload files\" button (<<fastpage_upload>>).\n",
"\n",
"<img width=\"400\" src=\"images/fast_template/image14.png\" alt=\"Screenshot showing how to upload new files\" id=\"fastpage_upload\" caption=\"Upload a file from your computer\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now let's see how to do all of this directly from your computer."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Synchronizing GitHub and Your Computer"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There are lots of reasons you might want to copy your blog content from GitHub to your computer--you might want to be able to read or edit your posts offline, or maybe you’d like a backup in case something happens to your GitHub repository.\n",
"\n",
"GitHub does more than just let you copy your repository to your computer; it lets you *synchronize* it with your computer. That means you can make changes on GitHub, and they’ll copy over to your computer, and you can make changes on your computer, and they’ll copy over to GitHub. You can even let other people access and modify your blog, and their changes and your changes will be automatically combined together the next time you sync.\n",
"\n",
"To make this work, you have to install an application called [GitHub Desktop](https://desktop.github.com/) on your computer. It runs on Mac, Windows, and Linux. Follow the directions to install it, and when you run it it’ll ask you to log in to GitHub and select the repository to sync. Click \"Clone a repository from the Internet,\" as shown in <<fastpage_clone>>.\n",
"\n",
"<img src=\"images/gitblog/image1.png\" width=\"400\" alt=\"A screenshot showing how to clone your repository\" id=\"fastpage_clone\" caption=\"Clone your repository on GitHub Desktop\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Once GitHub has finished syncing your repo, you’ll be able to click \"View the files of your repository in Explorer\" (or Finder), as shown in <<fastpage_explorer>> and you’ll see the local copy of your blog! Try editing one of the files on your computer. Then return to GitHub Desktop, and you’ll see the \"Sync\" button is waiting for you to press it. When you click it your changes will be copied over to GitHub, where you’ll see them reflected on the website.\n",
"\n",
"<img src=\"images/gitblog/image2.png\" width=\"600\" alt=\"A screenshot showing the cloned repository\" id=\"fastpage_explorer\" caption=\"Viewing your files locally\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you haven't used `git` before, GitHub Desktop is a great way to get started. As you'll discover, it's a fundamental tool used by most data scientists. Another tool that we hope you now love is Jupyter Notebooks--and there's a way to write your blog directly with that too!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Jupyter for Blogging"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can also write blog posts using Jupyter notebooks. Your markdown cells, code cells, and all the outputs will appear in your exported blog post. The best way to do this may have changed by the time you are reading this book, so be sure to check out the [book's website](https://book.fast.ai) for the latest information. As we write this, the easiest way to create a blog from notebooks is to use [`fastpages`](http://fastpages.fast.ai/), which is a more advanced version of `fast_template`. \n",
"\n",
"To blog with a notebook, just pop it in the *\\_notebooks* folder in your blog repo, and it will appear in your list of blog posts. When you write your notebook, write whatever you want your audience to see. Since most writing platforms make it hard to include code and outputs, many of us are in the habit of including fewer real examples than we should. This is a great way to instead get into the habit of including lots of examples as you write.\n",
"\n",
"Often, you'll want to hide boilerplate such as import statements. You can add `#hide` to the top of any cell to make it not show up in output. Jupyter displays the result of the last line of a cell, so there's no need to include `print()`. (Including extra code that isn't needed means there's more cognitive overhead for the reader; so don't include code that you don't really need!)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"jupytext": {
"split_at_heading": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
app_jupyter.ipynb
-549
View File
@@ -1,549 +0,0 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"! [ -e /content ] && pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from fastbook import *"
]
},
{
"cell_type": "raw",
"metadata": {},
"source": [
"[appendix]\n",
"[role=\"Jupyter notebook 101\"]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Appendix: Jupyter Notebook 101"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can read this tutorial in the book, but we strongly suggest reading it in a (yes, you guessed it) Jupyter Notebook. This way, you will be able to actually *try* the different commands we will introduce here. If you followed one of our tutorials in the previous section, you should have been left in the course folder. Just click on `nbs` then `dl1` and you should find the tutorial named `00_notebook_tutorial`. Click on it to open a new tab and you'll be ready to go.\n",
"\n",
"If you are on your personal machine, clone the course repository and navigate inside before following the same steps.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Introduction"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's build up from the basics: what is a Jupyter Notebook? Well, we wrote this book using Jupyter Notebooks. A notebook is a document made of cells. You can write in some of them (markdown cells) or you can perform calculations in Python (code cells) and run them like this:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"2"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"1+1"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Cool, huh? This combination of prose and code makes Jupyter Notebook ideal for experimentation: we can see the rationale for each experiment, the code, and the results in one comprehensive document. \n",
"\n",
"Other renowned institutions in academia and industry use Jupyter Notebook, including Google, Microsoft, IBM, Bloomberg, Berkeley and NASA among others. Even Nobel-winning economists [use Jupyter Notebooks](https://paulromer.net/jupyter-mathematica-and-the-future-of-the-research-paper/) for their experiments and some suggest that Jupyter Notebooks will be the [new format for research papers](https://www.theatlantic.com/science/archive/2018/04/the-scientific-paper-is-obsolete/556676/).\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Writing"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A type of cell in which you can write text is called a _Markdown cell_. [_Markdown_](https://en.wikipedia.org/wiki/Markdown) is a very popular markup language. To specify that a cell is Markdown you need to click in the drop-down menu in the toolbar and select Markdown.\n",
"\n",
"Click on the the '+' button on the left and select Markdown from the toolbar. Now you can type your first Markdown cell. Write 'My first markdown cell' and press run.\n",
"\n",
"![add](images/chapter1_add.png)\n",
"\n",
"You should see something like this:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"My first markdown cell"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now try making your first _Code_ cell: follow the same steps as before but don't change the cell type (when you add a cell its default type is _Code_). Type something like 3/2. You should see '1.5' as output."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1.5"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"3/2"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Modes"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you made a mistake in your *Markdown* cell and you have already run it, you will notice that you cannot edit it just by clicking on it. This is because you are in **Command Mode**. Jupyter Notebooks have two distinct modes:\n",
"\n",
"- Edit Mode:: Allows you to edit a cell's content.\n",
"\n",
"- Command Mode:: Allows you to edit the notebook as a whole and use keyboard shortcuts but not edit a cell's content. \n",
"\n",
"You can toggle between these two by either pressing <kbd>ESC</kbd> and <kbd>Enter</kbd> or clicking outside a cell or inside it (you need to double click if it's a Markdown cell). You can always tell which mode you're on: the current cell will have a green border in **Edit Mode** and a blue border in **Command Mode**. Try it!\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Other Important Considerations"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Your notebook is autosaved every 120 seconds. If you want to manually save it you can just press the save button on the upper left corner or press <kbd>s</kbd> in **Command Mode**.\n",
"\n",
"![Save](images/chapter1_save.png)\n",
"\n",
"To know if your *kernel* (the Python engine executing your instructions behind the scenes) is computing or not, you can check the dot in your upper right corner. If the dot is full, it means that the kernel is working. If not, it is idle. You can place the mouse on it and the state of the kernel will be displayed.\n",
"\n",
"![Busy](images/chapter1_busy.png)\n",
"\n",
"There are a couple of shortcuts you must know about which we use **all** the time (always in **Command Mode**). These are:\n",
"\n",
" - Shift+Enter:: Run the code or markdown on a cell\n",
" \n",
" - Up Arrow+Down Arrow:: Toggle across cells\n",
" \n",
" - b:: Create new cell\n",
" \n",
" - 0+0:: Reset Kernel\n",
"\n",
"You can find more shortcuts by typing <kbd>h</kbd> (for help).\n",
"\n",
"You may need to use a terminal in a Jupyter Notebook environment (for example to git pull on a repository). That is very easy to do: just press 'New' in your Home directory and 'Terminal'. Don't know how to use the Terminal? We made a tutorial for that as well. You can find it [here](https://course19.fast.ai/terminal_tutorial.html).\n",
"\n",
"![Terminal](images/chapter1_terminal.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"That's it. This is all you need to know to use Jupyter Notebooks. That said, we have more tips and tricks below, so don't jump to the next section just yet.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Markdown Formatting\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Italics, Bold, Strikethrough, Inline, Blockquotes and Links"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The five most important concepts to format your code appropriately when using Markdown are:\n",
" \n",
"- *Italics*:: Surround your text with \\_ or \\*.\n",
"\n",
"- **Bold**:: Surround your text with \\__ or \\**.\n",
"\n",
"- `inline`:: Surround your text with \\`.\n",
"\n",
"- blockquote:: Place \\> before your text.\n",
"\n",
"- [Links](http://course-v3.fast.ai/):: Surround the text you want to link with \\[\\] and place the link adjacent to the text, surrounded with ().\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Headings"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Notice that including a hashtag before the text in a markdown cell makes the text a heading. The number of hashtags you include will determine the priority of the header (# is level one, ## is level two, ### is level three and #### is level four). We will add three new cells with the + button on the left to see how every level of heading looks.\n",
"\n",
"In the notebook, double click on some headings and find out what level they are!\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Lists"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There are three types of lists in markdown.\n",
"\n",
"Ordered list:\n",
"\n",
"1. Step 1\n",
" 2. Step 1B\n",
"3. Step 3\n",
"\n",
"Unordered list\n",
"\n",
"* learning rate\n",
"* cycle length\n",
"* weight decay\n",
"\n",
"Task list\n",
"\n",
"- [x] Learn Jupyter Notebooks\n",
" - [x] Writing\n",
" - [x] Modes\n",
" - [x] Other Considerations\n",
"- [ ] Change the world\n",
"\n",
"In the notebook, double click on them to see how they are built! \n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Code Capabilities"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Code** cells are different than **Markdown** cells in that they have an output cell. This means that we can _keep_ the results of our code within the notebook and share them. Let's say we want to show a graph that explains the result of an experiment. We can just run the necessary cells and save the notebook. The output will be there when we open it again! Try it out by running the next four cells."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import necessary libraries\n",
"from fastai.vision.all import * \n",
"import matplotlib.pyplot as plt"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from PIL import Image"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(1, 2, 4, 8)"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"a = 1\n",
"b = a + 1\n",
"c = b + a + 1\n",
"d = c + b + a + 1\n",
"a, b, c ,d"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAWsAAAD7CAYAAACsV7WPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAhAElEQVR4nO3deXyU1d3+8c83C3tYM+yEsCTsewiKCtSlKNpaq1VxYbPF6qNdbK19ulqrqK12UR+1tLK51rZanwrq41KJuEEAAdkCAcIOkwAhZM/M+f2R6I8iywRmcs9MrvfrlZdkcptch5Nc3LnPfWbMOYeIiES3BK8DiIjIqamsRURigMpaRCQGqKxFRGKAylpEJAYkReoTp6amuvT09Eh9ehGRuLR8+fJC55zv2McjVtbp6enk5uZG6tOLiMQlMys43uO6DCIiEgNU1iIiMUBlLSISA1TWIiIxQGUtIhIDQiprM0s3s0VmdtDM9prZY2YWsTtJRETkP4V6Zv04sB/oAgwHxgO3RiiTiIgcI9Sy7gW86JyrcM7tBV4HBkUulohI7Ck6Usk9/1pHRXUg7J871LL+I3CtmbUws27AJdQW9n8ws5lmlmtmuX6/P5w5RUSiWmllDTPmLeO5pQXk+4+E/fOHWtaLqT2TPgzsBHKBfx57kHNutnMuyzmX5fN9YbekiEhcqg4EufXZFazZVcxjk0cyqGubsH+NU5a1mSUAbwAvAS2BVKAd8GDY04iIxBjnHHf9YzWL8/zMumIIFw7sFJGvE8qZdXugB/CYc67SOVcEzAUmRSSRiEgM+c0bG3lpxS7uuCiTa7PTIvZ1TlnWzrlCYCtwi5klmVlbYCqwKmKpRERiwNz3t/LEu/lcPyaN28/vG9GvFeo1668DFwN+YDNQA3w/UqFERKLdq6t3c8+r6/jywE7cc/lgzCyiXy+kjS3OuU+ACRFNIiISIz7IL+SOv64iq2c7Hpk8gsSEyBY1aLu5iEi9rNt9mJsXLCc9tQV/mTKaZsmJDfJ1VdYiIiHacaCMaXOX0qpZEvOmZ9OmRXKDfW2VtYhICA6UVjF17lIqqgPMn5FN17bNG/Tr68mYREROoayqdnfiroPlPPPNMWR2SmnwDDqzFhE5iZpAkNufW8nqnYd4ZPIIRqe39ySHzqxFRE7AOcdPX/6Utzfs596vDWbioM6eZdGZtYjICfz+zTz+mruD75zflxvO6ulpFpW1iMhxPP1RAY+8s5lrsnrw/YsyvY6jshYROdbrn+7hF698ygX9O3LfFZHfnRgKlbWIyFGWbj3Ad174hOE92vLYdSNJSoyOmoyOFCIiUWDj3hK+OX8Z3ds1Z87U0TRv0jC7E0OhshYRAXYfKmfqnKU0S05kwYxs2rVs4nWk/6CyFpFG71BZFVPmLKW0sob5M7Lp3q6F15G+QPdZi0ijVlEd4Jvzc9leVMb8GdkM6NLa60jHpbIWkUarJhDk9udXsnz7QR6bPJKz+3TwOtIJ6TKIiDRKzjl+/spa3ly3j19eNpBLh3bxOtJJqaxFpFF65O3NPL90O7dO6MO0c3p5HeeUVNYi0ug8v3Q7v38rjytHdufOif28jhOSU5a1mR055i1gZo82RDgRkXB7c90+fvryGib08/HAlUOiYndiKE65wOica/XZn82sJbAP+FskQ4mIRMLyggPc9twKhnRrw+PXjyQ5SnYnhqK+Sa8C9gPvRSCLiEjEbN5fwox5uXRt25w500bTokls3QxX37KeCixwzrnjfdDMZppZrpnl+v3+M08nIhIGe4srmDpnGcmJCcyfnk2HVk29jlRvIZe1maUB44H5JzrGOTfbOZflnMvy+XzhyCcickaKy6uZNncpxeXVzJs+mrQO0bc7MRT1ObOeAixxzm2NVBgRkXCqqA4wc0Eu+f4jPHnDKAZ3a+N1pNNW37I+4Vm1iEg0CQQdd7z4CR9vPcBD3xjGuRmpXkc6IyGVtZmNBbqhu0BEJAY45/jVv9ayaM1efnbpAC4f3s3rSGcs1DPrqcBLzrmSSIYREQmHx9/NZ8GHBcwc15tvntfb6zhhEdK9K865myMdREQkHP6Wu4PfvrGRrw3vyo8v7u91nLCJnTvCRURO4d8b9vPjl9ZwXkYqv7lqGAkJsbE7MRQqaxGJCyu3H+TWZ1cwoEsKT9wwiiZJ8VVv8TUaEWmUtviPMGPeMnwpTZk7LZtWTWNrd2IoVNYiEtP2H65gypylJJixYEY2vpTY250Yivj750dEGo3DFdVMnbuMA6VVvDDzLNJTW3odKWJ0Zi0iMamyJsC3n17Opn0lPHHDKIZ2b+t1pIjSmbWIxJxg0PGDF1fxQX4Rv7t6GOMz4/+5iHRmLSIxxTnHrxeu49XVe/jxJf35+sjuXkdqECprEYkps3O2MPf9bUw/J52bx8XH7sRQqKxFJGa8vHIn97+2gUuHduHnlw6MmZfkCgeVtYjEhJw8P3f+bTVn9+7A766Or92JoVBZi0jUW73zEN9+ZjkZnVL405RRNE1K9DpSg1NZi0hU21ZYyvS5y2jXognzp4+mdbNkryN5QmUtIlHLX1LJ1LlLCTrHgpuy6di6mdeRPKP7rEUkKh2prGHGvGXsP1zJc98aQx9fK68jeUplLSJRp6omyC3PLGfdnsP8ecooRqS18zqS53QZRESiSjDo+NHfV/HepkLu//oQzu/fyetIUUFlLSJR5cHXN/DPT3Zz58R+XJ3Vw+s4UUNlLSJR4y/vbeFPOVuYcnZPbp3Qx+s4USXksjaza81svZmVmlm+mZ0XyWAi0rj876rd3LtwPZcM7swvvzKoUe1ODEVIC4xmdhHwIHANsBToEslQItK4vL+5kB+8+AnZvdrz+2uGk9jIdieGItS7QX4F3OOc+6ju/V0RyiMijcynu4q5+enl9E5txZ+nZNEsufHtTgzFKS+DmFkikAX4zGyzme00s8fMrPlxjp1pZrlmluv3+yORV0TiyI4DZUybu4zWzZKYPyObNs0b5+7EUIRyzboTkAxcBZwHDAdGAD879kDn3GznXJZzLsvni/8nAxeR01d0pJIpc5ZSHQgyf0Y2nds03t2JoQilrMvr/vuoc26Pc64Q+B0wKXKxRCSeldbtTtx9qJynpmaR0SnF60hR75Rl7Zw7COwEXOTjiEi8qw4E+a/nVrBmVzGPTh5BVnp7ryPFhFBv3ZsL3G5mHc2sHfA94NWIpRKRuOSc479fWsO7G/3c+7UhfHlQZ68jxYxQ7wb5NZAK5AEVwIvAfZEKJSLx6aH/28jfl+/kexdmcN2YNK/jxJSQyto5Vw3cWvcmIlJv8z/Yxv/8O5/J2Wl894IMr+PEHG03F5GIW7RmD3f/ay0XDezEry/X7sTTobIWkYj6ML+I773wCSPT2vHo5BEkJap2Tof+1kQkYtbvOczMBbmkdWjBU1O1O/FMqKxFJCJ2Hixj2tyltGxauzuxbYsmXkeKaXqlGBEJu4OlVUyds5SyqgB///ZYurX9wrNTSD2prEUkrMqrAtw0fxk7Dpbz9Ixs+nXW7sRw0GUQEQmbmkCQ259fwcodh3jk2uGM6d3B60hxQ2UtImHhnONn//yUt9bv557LB3PxYD3tfTiprEUkLH7/1iZeWLaD277UlxvP6ul1nLijshaRM/bMRwU88vYmrs7qzg++nOl1nLikshaRM/L6p3v5xSufcn7/jsy6Yoh2J0aIylpETtuybQf4zgsrGdq9LY9dp92JkaS/WRE5LXn7Srhp3jK6t23OnGmjadFEdwJHkspaROptT3E5U+cspWlyIvNnZNO+pXYnRprKWkTqpbismqlzlnKkoob507Pp0b6F15EaBf3eIiIhq6gO8K0FuWwrLGPejNEM7Nra60iNhspaREISCDq++8JKlhUc4NHJIxjbJ9XrSI2KLoOIyCk55/jFK5/yxtp9/OKygVw2tKvXkRqdkMrazN41swozO1L3tjHSwUQkejz2zmae/Xg73x7fh+nn9PI6TqNUnzPr25xzrere+kUskYhElReWbufhN/P4+shu3HWxfvS9ossgInJCb63bx09eXsO4TB8PXjlUuxM9VJ+yvt/MCs3sfTObcLwDzGymmeWaWa7f7w9LQBHxxvKCg9z2/AoGd2vDE9ePJFm7Ez0V6t/+XUBvoBswG/iXmfU59iDn3GznXJZzLsvn84Uxpog0pM37j3DT/GV0bt2MOdNG07KpbhzzWkhl7Zz72DlX4pyrdM7NB94HJkU2moh4Yd/hCqbOWUpSgjF/RjaprZp6HUk4/WvWDtDFK5E4U1xeuzvxUFkVc6dl07NDS68jSZ1TlrWZtTWziWbWzMySzOx6YBzwRuTjiUhDqagOMHNBLpv3H+HJG0cxpHsbryPJUUK5EJUM3Av0BwLABuBrzjnday0SJwJBxx0vfsLHWw/wh2uGc16G1pyizSnL2jnnB0Y3QBYR8YBzjnv+tZZFa/by00kD+NqIbl5HkuPQvTgijdwTi/OZ/2EB3zy3F98a19vrOHICKmuRRuzvy3fym9c38tVhXfnJpAFex5GTUFmLNFL/3rifu/6xmnP7pvLQN4aRkKAbvKKZylqkEfpkxyFufWYF/Tun8MQNI2mSpCqIdpohkUZma2EpM+YtIzWlCXOnjyalWbLXkSQEKmuRRmR/SQVT5nwMwIIZY+iY0szjRBIqlbVII1FSUc30ucsoLKli7rTR9ErV7sRYorIWaQSqaoJ8+5nlbNxbwhM3jGRYj7ZeR5J60lNpicS5YNDxw7+t4v3NRTz8jWFM6NfR60hyGnRmLRLnZi1az/+u2s2PLu7HlaO6ex1HTpPKWiSO/TlnC39ZspVpY9O5ZfwXnoJeYojKWiRO/XPlLu5btJ5Lh3Th55cN1EtyxTiVtUgcysnz88O/reKs3u15+OphJGp3YsxTWYvEmTU7i7nlmeX07diK2VOyaJac6HUkCQOVtUgcKSgqZfq8pbRt0YT5M7Jprd2JcUOLine truncated
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plt.plot([a,b,c,d])\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can also print images while experimenting."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAMUAAAD6CAIAAACedhTfAAEAAElEQVR4nGT9S49t2ZYmCI1vjDHnWvthdl5+jvt1v9f9viMjyciMjEzIBhSFUqKEEFIBKvED6NKAH0GjJCQEDRrVQAiJBk2aQJZQIpAKVVYpk6yMisrIjLw3btyXu5+nme2915pzPGjMtc1PJFvmJnPTsf1Ya84xx/jG930DRP/ebo4f/+j4H/4v/2f/g//Rn/wn/+j/8B//X//3/5P/4L8tOLstTCJysy76m9/e/eIXX3/z+sPbu5cAgEQmURIFCzHlNNeHD+8vy4koBBxpIjJNE1Xt7u4eEQAERJEUYW2dRAsomrXzpZ0ubg2R4nB3ihSCqjIQERGxrmtkQmXazWWqUsr+eNgdD0GeRJlpGZ6eDFZh0QB1i+aRxCyqWpNBRNlIVYtOzJKBCCJiIhJWAESUmRHhYeN12xQRkZkARKTqVEoRkfQws7RkZlUF0Hu31ms9ERGYKHI8VXok+TzPtjZ3RwYRuTszl1LWvqu11lqJYOHhxCpSp/NpCUqAIaJaIZyZmRmns4gwwqyZr5RelVSZQRGW5u5OmZmJZABR0XsP8yq6m2cmLMtyuVy0FhZJ5RTuFD3DM4IyliIiRQRAuhGisKig90bpnJTpTGAhBTMz61O/PiJCid5874sf/i/+5//T/+Z/6+//J//4//6f/pP/z89+9rPekoQpa6S0lq/fPPzmN2++/ubuw/tLShJFJo31lOkZ6RRahIiYGQATQMLMRARAx08iAIgyzIlIRcJ86S27Revhbqu5tYoSEddXSGEe91VVPSKBiDCzBHrv2jsUycjxSsEEgBgAQ1QDiiQmMBEyOTOZgW2NJhFRsggzKxFREhERkpBjYRGRqo43QJEUGWHek6IwMyc5xfgLykQSEa3LwgwREREAAk5BZLbWwiwzVRVICCtLrVXKfvzjiKRwZh7/WyeNBBGIx0tRRLg7E4GJRSpXWGRsL2zuGZ7u2+onGt+hWohItIqWUpgw9oZngDmZE1Di8dEpr9uLiCgAYFwvYL/fUySFRQSFE1EiAtS60dhAAhbSv/v3f/wHP//y53/w8je//bP/2z/6v+z19LO/8bfPy+tdFXKsne7eX37727vf/O50/wDw07FImIhAmUSECM/0ZVncHYAA4/OP7e7dHvc3AZmR7unBzOa9rz3XlT3HXQknh4+bFyNaRIxdvtvtyMwyIsLNBKS9a+/JikQCRAgwjWdKeASxMjhBkcgEKJNYhAHk9SHMzKTKZkbg66+dKIDMjCIaIKcMSsrIME+iiGR29/SgZCMSQrpRWLpnchKSMF4oIjLh0T0cgDBYlDKZBaJIEJGZuYe5UXKCAJmmKRJEFAQics8RESsRM6tKJgjF3JGRTH3tSZ4REcE5tjYRqPcOIgECNDYkMViFAgQQOBgEygT5iM1OIAJBiEEgAEnX6BGJoKQcmw1EdGmXLYgwE5H+u//uT774/JN/+a/+3w/v/+q4lx9/9ZN1seUU7/vSLuvpwd6+W19/u3x4SJHn+/2x9SW3k45obNhERF4eTuPsExFmxjU4McTJPXpmEACAwcSARyRJkAcyHJFVVQBrzsxECYAIlAARgwlCFEQUiQxCkAWF0xomIgwGSWBsLGbSoORxVRIU47eqzBQJAoFonCbbzgwgiXwcd0TbF5C8vQsijMg1LjzMOm1rHRxuRO4e7nOpmUlJ0e0xyAVIpVLaeDmPiAhz7xkcBQCzxONHJooIZUZmJISICE6e6RQOlQScEuTjhGIQMZwyMq4Lg3m8dWBpTcCqykEdKUSZYGhyEFESOEACIXimRFq0pCAkIASAgnI7E+gavomYcA1iLEEEwjjftc5vWOnN2/ew8w++/6L38z//L345Cd1/uDuf2+UU68prqz2mUibxOfM8rhGQDDhRhIV1dy9FpqKqLCKiGPEfrL33TnD3sdoELETLw6mKSq1rs3W17I0JyuJjtSuQWxRm2k6oESRGNH4MMMEKFimFoQIEwMwgKYkEx/izLZpvF4R5O4vHByFEpLEgMykTtH0xiAgUhkzJYEpiIkqhGAc3ERHA47nGzyKVY1x9z8hMEWGRwqxTbdbdPYm6We89M0VsJ7OqMrNq2QInMYA0d0qi7VWZkikFkKJAulvva7eG7Ko8wjARkimTQJwYRwUyEAxipHIyJ41InmHbBQSAJElIUnggDCTkWypAwDheRAQkRESCj6/mrHX7vO6ZqUJvf/WXv9pXef7k5je//uX9uzfLw713K1zbSh6qZafTDUXpxv3UZlm3xEJAzBEWZq01VZ5qnabC4yRlAae7h0c+fiUzJTiZwARAiXqYt2W1ZWXKwoJpGvuUiJySkwLEgPVu4cQYkTYz3b27BU+iBaWKFpAIMSAMvVwu7hG+RSYA4emW+/3MMg4DiojMIEqiFJEkD0cmAcSM8SrwcVOxJVcUwPgHCQEAedyqxJmZPTKCIpmIWVhVS1FVErZw83D37tHMEyQAW4tIYq7C45EBzyByCnJKIgdAAQFDiJkyyTO6W2tLphdCQCG8peEjSoODASAdQdQ9hFMECU6P8GDRcKdMJAQSGUhQkDIBwclCNHZNRnq4cMltYZOP14nITJ5gmZbZRz7O9LCcXrcT1oe3y+VC3QXaLnnYT55MMnE5Ms/wEeMQ1kelFlBmYqLMiAhhFQEzp1tPjwgiMm++ZkRE2rg/YenZPJIyvVtbephxEhPI08wTTUSEOTPTg3Pko+zungFiMEdGWGRvWNh8GteJUpiVk4k4kruBQhmy3+/3+72qmllvDZIAMt3dM52IWIiZRDNiFC9xTQ2JCOOQTcocASlGWHMBGAyM7DWusY8sAkRFdYQuCIM5k8LDPczMKRMkWqFSa42HtHD0PoITM28nmHCmwz0zCTLKnAQikxBAAuPM69SRGaqaJJEBJI3EKwECj3qC0ilHqm/ZDTmrJGgksgJmRiGOhGgZiYoKmCUzG1EGzIxYEglIApnj/EgLD8qRkyFZv/7tbz777LN2aXfv7/fTAcJ9tU9efm9dDOEeWFsLWBJzqbXWvNs+DDMpwxPMzCNOE6VH727WlDkze+/KhYiUi4gwU2ZGT/dG5n1Z27KmU9VpgljrYXbfrXgyM5K2SpBZRILIR7AmpJNlJAzo1nuSgD3JhUadIJypsoPIYXd8/vz5ixcvdvPc2nK5XL55/buIMO8RHk7gBFh1ZJSUSZk5KgEiwsheiNJjVLORjkwa0SOYmYMJcV19gIKYGVpGUeIZ1j3IktGbNw8AgEjVWus0z9bbYzI7IInHnyk40jOS4ODt+LU0xpb/MpMRWXj2UNXxHkY8BTEokbzfH8fZWutUd/sq2qQlLcTCiaBg8Fg3Cnc4sVNyMrZgmTkwlXFVmFWksAjAIw72cLrWXpmpTw7Pl/vmlpV36zkoUWS3LnG59HGFiIxFVJO1MVsqn89nM5tLjVojLLwzc4RZ61wREW1ZHloTkVIKKFfrAOZ5zkRfLm4m4OV89qW11tvlEmuXpKplN0+dOhEh0rtZOEU6IBEALLwvEecLC3gqmtHNsOwP+yfzvO8tzFxY2toePjx88fmXNzc3L559cnNzM01FpRye3r56qfvDzbevv379+pvWLMKTugeIish2RXhLiDgziVggPlCjiIiQrYROJINJCBHhPiA2p8ydVlUtcwWk9d67mblndB9LIzNATCzklsvS4D7u3CjLx70Z+SILFUgE3N2jj5AP0fVyZsFUlbEL7xFZawXQrPfeiVB0EhEijojWDGAFIuh8WRubiMz7w3pZAIHIKLgRqVpFCrj23psb0XhX7MkejVkIQhCiUYeYU2aAK19r2CQipVaSiCLJJSMo4LSlL0BCkjnAnbfSFxyuIGwFXFAmkKo8svCIGJDaeHZmzkQG8orgZW63rTcPM3cPH7hfmhkiU7YLChFJIiHZcDORCAlv4QHKoEhS8Fx3l9OC1MP+tnle7tfd7uYPfv7DqR72u908HYFincPdvFXN3e746qUKl6+/+c3pfKdSi3LEAFGCiCh5pNuj2hqfIzO29D9HgMkimjTy0O1OaxkHhwbIzJKi9957b90sEiKAVNaEEIOZE0IBho7idORz49QbGB5lIHN8UURuVzWVmTKjW/qoXoSIVDUzKa6YQZCIqOr4fWZGUIQbAtlHvj+Ocx7LBKzMAIwcSgoGDdBu3IrCzATJDQzDhoVwRsaI6iN7U1sHHkPkgfAICsoeHUhmYk7RUZ86RWQ6YRaFqIowMjIdQBHdArWPw5oFrCwCBouOepAEIC76eD5mbtcOLDywgIgQCCEGNCojm2YRISIpSinp1sNpHA1FlYSC2RjGEtjP06cvv//Dr35yPjdVFZFwcnMiiuLhON4cD/snRafL5bIsC1GPCI9eilzXEwECJBEDxFIIoQNrjsh0GRWzinVvbtE7EYlCpKhqOnW3Zj0yM2GRnhGRzAwGc2FmEqUBCidYr+BN8uOSQiaNIEm09RPSKSwjIkNVCdnaYtaKACiRG9oOcO9unhGkiqKTUialUWRmUI6lf80ME5GUKeDCQkQiQmBRZtFMeGZmMmvhK3wAABsuk9vu87HaIzMplUIfq2mAmB0ZlKGFmCGSgiBEpkWMLDtEwARmmFmmC7iUEhGITCJmrqVkqqqKKCVEJGhAtsyclESRUgtFgpW4Q5hZFCzMFC0oBTzw3VFWZea6rlIUIuN/xxYvpbTT+uzp87nu7NKmevPZ97/65MX3BHMVTmK3jDAiiDBQRNQ6GDxNh+fPX54vD+/ef9P6WioDct1jMTKngX1xISIkQBseQ57OFIAMBH9U9biWnBFhPdw9edvWpRTmzBEFRk0fNKDzbbsw4/rfFpmQmTkWLpIYCYrxRUaq4kneLdwnLazce3jvIjJpYVZungmBikg2G1dr3F8I57VDkOZBnradw0QZFKONAQaNzIlJmJnVfcuT8hFAFyJicEZEEIKLine truncated
"text/plain": [
"<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=197x250 at 0x7F3E279ABE50>"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Image.open(image_cat())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Running the App Locally"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You may be running Jupyter Notebook from an interactive coding environment like Gradient, Sagemaker or Salamander. You can also run a Jupyter Notebook server from your local computer. What's more, if you have installed Anaconda you don't even need to install Jupyter (if not, just `pip install jupyter`).\n",
"\n",
"You just need to run `jupyter notebook` in your terminal. Remember to run it from a folder that contains all the folders/files you will want to access. You will be able to open, view, and edit files located within the directory in which you run this command but not files in parent directories.\n",
"\n",
"If a browser tab does not open automatically once you run the command, you should CTRL+CLICK the link starting with 'http://localhost:' and this will open a new tab in your default browser.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Creating a Notebook"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that you have your own Jupyter Notebook server running, you will probably want to write your own notebook. Click on 'New' in the upper left corner and 'Python 3' in the drop-down list (we are going to use a [Python kernel](https://github.com/ipython/ipython) for all our experiments).\n",
"\n",
"![new_notebook](images/chapter1_new_notebook.png)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Shortcuts and Tricks"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here is a list of useful tricks when in a Jupyter Notebook. Make sure you learn them early and use them as often as you can!\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Command Mode Shortcuts"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There are a couple of useful keyboard shortcuts in `Command Mode` that you can leverage to make Jupyter Notebook faster to use. Remember that you can switch back and forth between `Command Mode` and `Edit Mode` with <kbd>Esc</kbd> and <kbd>Enter</kbd>.\n",
"\n",
"- m:: Convert cell to Markdown\n",
"- y:: Convert cell to Code\n",
"- d+d:: Delete cell\n",
"- o:: Toggle between hide or show output\n",
"- Shift+Arrow up/Arrow down:: Select multiple cells. Once you have selected them you can operate on them like a batch (run, copy, paste etc).\n",
"- Shift+M:: Merge selected cells\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Cell Tricks"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There are also some tricks that you can code into a cell:\n",
"\n",
"- `?function-name`:: Shows the definition and docstring for that function\n",
"- `??function-name`:: Shows the source code for that function\n",
"- `doc(function-name)`:: Shows the definition, docstring **and links to the documentation** of the function\n",
"(only works with fastai library imported)\n",
"- Shift+Tab (press once):: See which parameters to pass to a function \n",
"- Shift+Tab (press three times):: Get additional information on the method"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Line Magics"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Line magics are functions that you can run on cells. They should be at the beginning of a line and take as an argument the rest of the line from where they are called. You call them by placing a '%' sign before the command. The most useful ones are:\n",
"\n",
"- `%matplotlib inline`:: Ensures that all matplotlib plots will be plotted in the output cell within the notebook and will be kept in the notebook when saved.\n",
"\n",
"This command is always called together at the beginning of every notebook of the fast.ai course.\n",
"\n",
"``` python\n",
"%matplotlib inline\n",
"```\n",
"\n",
"- `%timeit`:: Runs a line ten thousand times and displays the average time it took to run."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"56.1 µs ± 592 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)\n"
]
}
],
"source": [
"%timeit [i+1 for i in range(1000)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`%debug`: Inspects a function which is showing an error using the [Python debugger](https://docs.python.org/3/library/pdb.html). If you type this in a cell just after an error, you will be directed to a console where you can inspect the values of all the variables.\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}