diff --git a/01_intro.ipynb b/01_intro.ipynb index 7511d20..e6c927b 100644 --- a/01_intro.ipynb +++ b/01_intro.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "code", - "execution_count": 1, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -242,14 +242,14 @@ "\n", "All this means that you have the best of both worlds: the people who know more about the software than anybody, because they wrote it, an expert on maths, and an expert on coding and machine learning, but also people who understand what it feels like to be a relative outsider in maths, and a relative outsider in coding and machine learning.\n", "\n", - "Anybody who has watched sports knows that if you have a two-person commentary team then you also need a third person to do \"special comments\". Our special commentator is Alexis Gallagher. Alexis has a very diverse background: he has been a zoology researcher, screenplay writer, an improv performer, a McKinsey consultant (like Jeremy!), a Swift coder, and a CTO." + "Anybody who has watched sports knows that if you have a two-person commentary team then you also need a third person to do \"special comments\". Our special commentator is Alexis Gallagher. Alexis has a very diverse background: he has been a researcher in mathematical biology, a screenplay writer, an improv performer, a McKinsey consultant (like Jeremy!), a Swift coder, and a CTO." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "> A: I've decided it's time for me to learn about this AI stuff! After all, I've tried pretty much everything else… But I don't really have a background in machine learning, or in Python. Still… how hard can it be? I'm going to be learning throughout this book, just like you are. Look out for my sidebars for learning tips that I found helpful on my journey, and hopefully you will find helpful too." + "> A: I've decided it's time for me to learn about this AI stuff! After all, I've tried pretty much everything else… But I don't really have a background in building machine learning models. Still… how hard can it be? I'm going to be learning throughout this book, just like you are. Look out for my sidebars for learning tips that I found helpful on my journey, and hopefully you will find helpful too." ] }, { @@ -2062,9 +2062,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "scrolled": true - }, + "metadata": {}, "outputs": [ { "data": { @@ -2301,7 +2299,7 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -2916,18 +2914,6 @@ "display_name": "Python 3", "language": "python", "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.7.4" } }, "nbformat": 4, diff --git a/02_production.ipynb b/02_production.ipynb index 1dbb41f..2ef9a23 100644 --- a/02_production.ipynb +++ b/02_production.ipynb @@ -808,6 +808,8 @@ "\n", "Instead, what we normally do in practice is to randomly select part of the image, and crop to just that part. On each epoch (which is one complete pass through all of our images in the dataset) we randomly select a different part of each image. This means that our model can learn to focus on, and recognize, different features in our images. It also reflects how images work in the real world; different photos of the same thing may be framed in slightly different ways.\n", "\n", + "In fact, an entirely untrained neural network knows nothing whatsoever about how images behave. It doesn't even recognise that when an object is rotated by one degree, then it still is a picture of the same thing! So actually training the neural network with examples of images that are in slightly different places, and slightly different sizes, helps it to understand the basic concept of what a *object* is, and how it can be represented in an image.\n", + "\n", "Here is another copy of the previous examples, but this time we are replacing `Resize` with `RandomResizedCrop`, which is the transform that provides the behaviour described above. The most important parameter to pass in is the `min_scale` parameter, which determines how much of the image to select at minimum each time." ] }, @@ -832,24 +834,14 @@ "source": [ "bears = bears.new(item_tfms=RandomResizedCrop(128, min_scale=0.3))\n", "dls = bears.dataloaders(path)\n", - "dls.train.get_idxs = lambda: Inf.ones\n", - "dls.train.show_batch(max_n=4, nrows=1)" + "dls.train.show_batch(max_n=4, nrows=1, unique=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "> note: The `get_idxs` assignment in this code is a little bit magic, and you absolutely don't have to understand it at this point. So feel free to ignore the entirety of this paragraph! This is just if you're curious… Showing different randomly varied versions of the same image is not something we normally have to do in deep learning, so it's not something that fastai provides directly. Therefore to draw the picture of data augmentation on the same image, we had to take advantage of fastai's sophisticated customisation features. DataLoader has a method called `get_idxs`, which is called to decide which items should be selected next. Normally when we are training, this returns a random permutation of all of the indexes in the dataset. But pretty much everything in fastai can be changed, including how the `get_idxs` method is defined, which means we can change how we sample data. So in this case, we are replacing it with a version which always returns the number one. That way, our DataLoader shows the same image again and again! This is a great example of the flexibility that fastai provides. " - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In fact, an entirely untrained neural network knows nothing whatsoever about how images behave. It doesn't even recognise that when an object is rotated by one degree, then it still is a picture of the same thing! So actually training the neural network with examples of images that are in slightly different places, and slightly different sizes, helps it to understand the basic concept of what a *object* is, and how it can be represented in an image.\n", - "\n", - "This is a specific example of a more general technique, called *data augmentation*." + "We use `unique=True` to have the same image repeated with different versions of this `RandomResizedCrop` transform. This is a specific example of a more general technique, called *data augmentation*." ] }, { @@ -887,8 +879,7 @@ "source": [ "bears = bears.new(item_tfms=Resize(128), batch_tfms=aug_transforms(mult=2))\n", "dls = bears.dataloaders(path)\n", - "dls.train.get_idxs = lambda: Inf.ones\n", - "dls.train.show_batch(max_n=8, nrows=2)" + "dls.train.show_batch(max_n=8, nrows=2, unique=True)" ] }, { @@ -1772,7 +1763,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "> A: I've had a chance to see up close how the mobile ML landscape is changing in my work. We offer an iPhone app that depends on computer vision and for years we ran our own computer vision models in the cloud. This was the only way to do it then since those models needed significant memory and compute resources and took minutes to process. This approach required building not only the models (fun!) but infrastructure to ensure a certain number of \"compute worker machines\" was absolutely always running (scary), that more machines would automatically come online if traffic increased, that there was stable storage for large inputs and outputs, that the iOS app could know and tell the user how their job was doing, etc... Nowadays, Apple provides APIs for converting models to run efficiently on device and most iOS devices have dedicated ML hardware, so we run our new models on device. So, in a few years that strategy has gone from impossible to possible but it's still not easy. In our case it's worth it, for a faster user experience and to worry less about servers. What works for you will depend, realistically, on the user experience you're trying to create and what you personally find it easy to do. If you really know how to run servers, do it. If you really know how to build native mobile apps, do that. There are many roads up the hill.\n", + "> A: I've had a chance to see up close how the mobile ML landscape is changing in my work. We offer an iPhone app that depends on computer vision and for years we ran our own computer vision models in the cloud. This was the only way to do it then since those models needed significant memory and compute resources and took minutes to process. This approach required building not only the models (fun!) but infrastructure to ensure a certain number of \"compute worker machines\" was absolutely always running (scary), that more machines would automatically come online if traffic increased, that there was stable storage for large inputs and outputs, that the iOS app could know and tell the user how their job was doing, etc... Nowadays, Apple provides APIs for converting models to run efficiently on device and most iOS devices have dedicated ML hardware, and we run our newer models on device. So, in a few years that strategy has gone from impossible to possible but it's still not easy. In our case it's worth it, for a faster user experience and to worry less about servers. What works for you will depend, realistically, on the user experience you're trying to create and what you personally find it easy to do. If you really know how to run servers, do it. If you really know how to build native mobile apps, do that. There are many roads up the hill.\n", "\n", "Overall, we'd recommend using a simple CPU-based server approach where possible, for as long as you can get away with it. If you're lucky enough to have a very successful application, then you'll be able to justify the investment in more complex deployment approaches at that time.\n", "\n", @@ -1966,18 +1957,6 @@ "display_name": "Python 3", "language": "python", "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.7.4" } }, "nbformat": 4, diff --git a/04_mnist_basics.ipynb b/04_mnist_basics.ipynb index 37d19a0..dd35a15 100644 --- a/04_mnist_basics.ipynb +++ b/04_mnist_basics.ipynb @@ -1520,7 +1520,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "> Important: it's really important for you to commit to memory and practice these bits of tensor jargon: _rank_ is the number of axes or dimensions in a tensor; _shape_ is the size of each axis of a tensor." + "It is really important for you to commit to memory and practice these bits of tensor jargon: _rank_ is the number of axes or dimensions in a tensor; _shape_ is the size of each axis of a tensor.\n", + "\n", + "> A: Watch out because the term \"dimension\" is sometimes used in two ways. Consider that we live in \"three dimensonal space\" where a physical position can be described by a 3-vector `v`. But according to PyTorch, the attribute `v.ndim` (which sure looks like the \"number of dimensions\" of `v`) equals one not three! Why? Because v is a vector, which is a tensor of rank one, meaning that it has only one _axis_ (even if that axis has a length of three). In other words, sometimes dimension is used for the size of an axis (\"space is 3-dimensional\"); other times, it is used for the rank, or the number of axes (\"a matrix has two dimensions\"). When confused, I find it helpful to translate all statements into the terms of rank, axis, and length, which are unambiguous terms." ] }, { @@ -2664,7 +2666,7 @@ "source": [ "Notice the special method `requires_grad_`? That's the magical incantation we use to tell PyTorch that we want to calculate gradients with respect to that variable at that value. It is essentially tagging the variable, so PyTorch will remember to keep track of how to compute gradients of the other, direct calculations on it which you will ask for.\n", "\n", - "> A: This API might throw you if you're coming from math or physics. In those contexts the \"gradient\" of a function is just another function (i.e., its derivative), so you might expect gradient-related APIs to give you a new function. But in deep learning, \"gradients\" usually means the _value_ of a function's derivative at a particular argument value. PyTorch API also puts the focus on that argument, not the function you're actually computing gradients of. It may feel backwards at first but it's just a different perspective.\n", + "> a: This API might throw you if you're coming from math or physics. In those contexts the \"gradient\" of a function is just another function (i.e., its derivative), so you might expect gradient-related APIs to give you a new function. But in deep learning, \"gradients\" usually means the _value_ of a function's derivative at a particular argument value. PyTorch API also puts the focus on that argument, not the function you're actually computing the gradients of. It may feel backwards at first but it's just a different perspective.\n", "\n", "Now we calculate our function with that value. Notice how PyTorch prints not just the value calculated, but also a note that it has a gradient function it'll be using to calculate our gradient when needed:" ] diff --git a/05_pet_breeds.ipynb b/05_pet_breeds.ipynb index 061a9d0..24baf9c 100644 --- a/05_pet_breeds.ipynb +++ b/05_pet_breeds.ipynb @@ -167,7 +167,7 @@ "\n", "We do not have the space to give you a complete regular expression tutorial here, particularly because there are so many excellent ones online. And we know that many of you will already be familiar with this wonderful tool. If you're not, that is totally fine — this is a great opportunity for you to rectify that! We find that regular expressions are one of the most useful tools in our programming toolkit, and many of our students tell us that it is one of the things they are most excited to learn about. So head over to Google and search for *regular expressions tutorial* now, and then come back here after you've had a good look around. The book website also provides a list of our favorites.\n", "\n", - "> a: Not only are regular expresssions dead handy, they also have interesting roots. They are \"regular\" because they were originally examples of a \"regular\" language, the lowest rung within the \"Chomsky hierarchy\", a grammar classification due to the same linguist Noam Chomskey who wrote _Syntactic Structures_, the pioneering work searching for the formal grammar underlying human language. This is one of the charms of computing: it may be that the hammer you reach for every day in fact came from a space ship.\n", + "> a: Not only are regular expressions dead handy, they also have interesting roots. They are \"regular\" because they were originally examples of a \"regular\" language, the lowest rung within the \"Chomsky hierarchy\", a grammar classification due to the same linguist Noam Chomskey who wrote _Syntactic Structures_, the pioneering work searching for the formal grammar underlying human language. This is one of the charms of computing: it may be that the hammer you reach for every day in fact came from a space ship.\n", "\n", "When you are writing a regular expression, the best way to start is just to try it against one example at first. Let's use the `findall` method to try a regular expression against the filename of the `fname` object:" ] @@ -340,7 +340,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We can never just assume that our code is working perfectly. Writing a `DataBlock` is just like writing a blueprint. You will get an error message if you have a syntax error somewhere in your code but you have no garanty that your template is going to work on your source of data as you intend. The first thing to do before we trying to train a model is to use the `show_batch` method and have a look at your data:" + "We can never just assume that our code is working perfectly. Writing a `DataBlock` is just like writing a blueprint. You will get an error message if you have a syntax error somewhere in your code but you have no guaranty that your template is going to work on your source of data as you intend. The first thing to do before we trying to train a model is to use the `show_batch` method and have a look at your data:" ] }, { diff --git a/08_collab.ipynb b/08_collab.ipynb index 5927831..471c2d0 100644 --- a/08_collab.ipynb +++ b/08_collab.ipynb @@ -1583,7 +1583,7 @@ } ], "source": [ - "movie_bias = learn.model.movie_bias.weight.squeeze()\n", + "movie_bias = learn.model.movie_bias.squeeze()\n", "idxs = movie_bias.argsort()[:5]\n", "[dls.classes['title'][i] for i in idxs]" ] @@ -1657,7 +1657,7 @@ "g = ratings.groupby('title')['rating'].count()\n", "top_movies = g.sort_values(ascending=False).index.values[:1000]\n", "top_idxs = tensor([learn.dls.classes['title'].o2i[m] for m in top_movies])\n", - "movie_w = learn.model.movie_factors.weight[top_idxs].cpu().detach()\n", + "movie_w = learn.model.movie_factors[top_idxs].cpu().detach()\n", "movie_pca = movie_w.pca(3)\n", "fac0,fac1,fac2 = movie_pca.t()\n", "idxs = np.random.choice(len(top_movies), 50, replace=False)\n", @@ -2296,6 +2296,18 @@ "display_name": "Python 3", "language": "python", "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.4" } }, "nbformat": 4, diff --git a/clean/01_intro.ipynb b/clean/01_intro.ipynb index b3f94c3..6493daa 100644 --- a/clean/01_intro.ipynb +++ b/clean/01_intro.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "code", - "execution_count": 1, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -966,9 +966,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "scrolled": true - }, + "metadata": {}, "outputs": [ { "data": { @@ -1135,7 +1133,7 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1569,18 +1567,6 @@ "display_name": "Python 3", "language": "python", "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.7.4" } }, "nbformat": 4, diff --git a/clean/02_production.ipynb b/clean/02_production.ipynb index 6857d64..6c950e9 100644 --- a/clean/02_production.ipynb +++ b/clean/02_production.ipynb @@ -411,8 +411,7 @@ "source": [ "bears = bears.new(item_tfms=RandomResizedCrop(128, min_scale=0.3))\n", "dls = bears.dataloaders(path)\n", - "dls.train.get_idxs = lambda: Inf.ones\n", - "dls.train.show_batch(max_n=4, nrows=1)" + "dls.train.show_batch(max_n=4, nrows=1, unique=True)" ] }, { @@ -443,8 +442,7 @@ "source": [ "bears = bears.new(item_tfms=Resize(128), batch_tfms=aug_transforms(mult=2))\n", "dls = bears.dataloaders(path)\n", - "dls.train.get_idxs = lambda: Inf.ones\n", - "dls.train.show_batch(max_n=8, nrows=2)" + "dls.train.show_batch(max_n=8, nrows=2, unique=True)" ] }, { @@ -1082,18 +1080,6 @@ "display_name": "Python 3", "language": "python", "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.7.4" } }, "nbformat": 4, diff --git a/clean/08_collab.ipynb b/clean/08_collab.ipynb index 65be45a..7ec2fb5 100644 --- a/clean/08_collab.ipynb +++ b/clean/08_collab.ipynb @@ -1182,7 +1182,7 @@ } ], "source": [ - "movie_bias = learn.model.movie_bias.weight.squeeze()\n", + "movie_bias = learn.model.movie_bias.squeeze()\n", "idxs = movie_bias.argsort()[:5]\n", "[dls.classes['title'][i] for i in idxs]" ] @@ -1236,7 +1236,7 @@ "g = ratings.groupby('title')['rating'].count()\n", "top_movies = g.sort_values(ascending=False).index.values[:1000]\n", "top_idxs = tensor([learn.dls.classes['title'].o2i[m] for m in top_movies])\n", - "movie_w = learn.model.movie_factors.weight[top_idxs].cpu().detach()\n", + "movie_w = learn.model.movie_factors[top_idxs].cpu().detach()\n", "movie_pca = movie_w.pca(3)\n", "fac0,fac1,fac2 = movie_pca.t()\n", "idxs = np.random.choice(len(top_movies), 50, replace=False)\n", @@ -1703,6 +1703,18 @@ "display_name": "Python 3", "language": "python", "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.4" } }, "nbformat": 4,