daviddoji/Fastbook_Julia
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update

Browse Source
This commit is contained in:
Sylvain Gugger
2020-05-14 17:27:39 -07:00
parent db9c7e81b1
commit a482265f72
4 changed files with 424 additions and 473 deletions
Download Patch File Download Diff File Expand all files Collapse all files

54
clean/08_collab.ipynb
View File

@@ -523,26 +523,6 @@
"one_hot_3 = one_hot(3, n_users).float()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([944, 5])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"user_factors.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -1670,7 +1650,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Sidebar: Kwargs and Delegates"
"### Sidebar: kwargs and Delegates"
]
},
{
@@ -1702,33 +1682,33 @@
"1. How does it solve it?\n",
"1. Why might a collaborative filtering predictive model fail to be a very useful recommendation system?\n",
"1. What does a crosstab representation of collaborative filtering data look like?\n",
"1. Write the code to create a crosstab representation of the MovieLens data (you might need to do some web searching!)\n",
"1. Write the code to create a crosstab representation of the MovieLens data (you might need to do some web searching!).\n",
"1. What is a latent factor? Why is it \"latent\"?\n",
"1. What is a dot product? Calculate a dot product manually using pure python with lists.\n",
"1. What is a dot product? Calculate a dot product manually using pure Python with lists.\n",
"1. What does `pandas.DataFrame.merge` do?\n",
"1. What is an embedding matrix?\n",
"1. What is the relationship between an embedding and a matrix of one-hot encoded vectors?\n",
"1. Why do we need `Embedding` if we could use one-hot encoded vectors for the same thing?\n",
"1. What does an embedding contain before we start training (assuming we're not using a prertained model)?\n",
"1. What is the relationship between an embedding and a matrix of one-hot-encoded vectors?\n",
"1. Why do we need `Embedding` if we could use one-hot-encoded vectors for the same thing?\n",
"1. What does an embedding contain before we start training (assuming we're not using a pretained model)?\n",
"1. Create a class (without peeking, if possible!) and use it.\n",
"1. What does `x[:,0]` return?\n",
"1. Rewrite the `DotProduct` class (without peeking, if possible!) and train a model with it\n",
"1. Rewrite the `DotProduct` class (without peeking, if possible!) and train a model with it.\n",
"1. What is a good loss function to use for MovieLens? Why? \n",
"1. What would happen if we used `CrossEntropy` loss with MovieLens? How would we need to change the model?\n",
"1. What would happen if we used cross-entropy loss with MovieLens? How would we need to change the model?\n",
"1. What is the use of bias in a dot product model?\n",
"1. What is another name for weight decay?\n",
"1. Write the equation for weight decay (without peeking!)\n",
"1. Write the equation for weight decay (without peeking!).\n",
"1. Write the equation for the gradient of weight decay. Why does it help reduce weights?\n",
"1. Why does reducing weights lead to better generalization?\n",
"1. What does `argsort` do in PyTorch?\n",
"1. Does sorting the movie biases give the same result as averaging overall movie ratings by movie? Why / why not?\n",
"1. Does sorting the movie biases give the same result as averaging overall movie ratings by movie? Why/why not?\n",
"1. How do you print the names and details of the layers in a model?\n",
"1. What is the \"bootstrapping problem\" in collaborative filtering?\n",
"1. How could you deal with the bootstrapping problem for new users? For new movies?\n",
"1. How can feedback loops impact collaborative filtering systems?\n",
"1. When using a neural network in collaborative filtering, why can we have different number of factors for movie and user?\n",
"1. Why is there a `nn.Sequential` in the `CollabNN` model?\n",
"1. What kind of model should be use if we want to add metadata about users and items, or information such as date and time, to a collaborative filter model?"
"1. When using a neural network in collaborative filtering, why can we have different numbers of factors for movies and users?\n",
"1. Why is there an `nn.Sequential` in the `CollabNN` model?\n",
"1. What kind of model should we use if we want to add metadata about users and items, or information such as date and time, to a collaborative filtering model?"
]
},
{
@@ -1737,10 +1717,10 @@
"source": [
"### Further Research\n",
"\n",
"1. Take a look at all the differences between the `Embedding` version of `DotProductBias` and the `create_params` version, and try to understand why each of those changes is required. If you're not sure, try reverting each change, to see what happens. (NB: even the type of brackets used in `forward` has changed!)\n",
"1. Find three other areas where collaborative filtering is being used, and find out what pros and cons of this approach in those areas.\n",
"1. Complete this notebook using the full MovieLens dataset, and compare your results to online benchmarks. See if you can improve your accuracy. Look on the book website and forum for ideas. Note that there are more columns in the full dataset--see if you can use those too (the next chapter might give you ideas)\n",
"1. Create a model for MovieLens with works with CrossEntropy loss, and compare it to the model in this chapter."
"1. Take a look at all the differences between the `Embedding` version of `DotProductBias` and the `create_params` version, and try to understand why each of those changes is required. If you're not sure, try reverting each change to see what happens. (NB: even the type of brackets used in `forward` has changed!)\n",
"1. Find three other areas where collaborative filtering is being used, and find out what the pros and cons of this approach are in those areas.\n",
"1. Complete this notebook using the full MovieLens dataset, and compare your results to online benchmarks. See if you can improve your accuracy. Look on the book's website and the fast.ai forum for ideas. Note that there are more columns in the full dataset--see if you can use those too (the next chapter might give you ideas).\n",
"1. Create a model for MovieLens that works with cross-entropy loss, and compare it to the model in this chapter."
]
},
{

32
clean/09_tabular.ipynb
View File

@@ -954,6 +954,7 @@
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"to = (path/'to.pkl').load()"
]
},
@@ -7779,7 +7780,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Finding out of Domain Data"
"### Finding Out-of-Domain Data"
]
},
{
@@ -8311,7 +8312,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Ensembling"
"## Ensembling"
]
},
{
@@ -8378,12 +8379,12 @@
"source": [
"1. What is a continuous variable?\n",
"1. What is a categorical variable?\n",
"1. Provide 2 of the words that are used for the possible values of a categorical variable.\n",
"1. Provide two of the words that are used for the possible values of a categorical variable.\n",
"1. What is a \"dense layer\"?\n",
"1. How do entity embeddings reduce memory usage and speed up neural networks?\n",
"1. What kind of datasets are entity embeddings especially useful for?\n",
"1. What kinds of datasets are entity embeddings especially useful for?\n",
"1. What are the two main families of machine learning algorithms?\n",
"1. Why do some categorical columns need a special ordering in their classes? How do you do this in pandas?\n",
"1. Why do some categorical columns need a special ordering in their classes? How do you do this in Pandas?\n",
"1. Summarize what a decision tree algorithm does.\n",
"1. Why is a date different from a regular categorical or continuous variable, and how can you preprocess it to allow it to be used in a model?\n",
"1. Should you pick a random validation set in the bulldozer competition? If no, what kind of validation set should you pick?\n",
@@ -8394,19 +8395,20 @@
"1. What is bagging?\n",
"1. What is the difference between `max_samples` and `max_features` when creating a random forest?\n",
"1. If you increase `n_estimators` to a very high value, can that lead to overfitting? Why or why not?\n",
"1. What is *out of bag error*?\n",
"1. In the section \"Creating a Random Forest\", just after <<max_features>>, why did `preds.mean(0)` give the same result as our random forest?\n",
"1. What is \"out-of-bag-error\"?\n",
"1. Make a list of reasons why a model's validation set error might be worse than the OOB error. How could you test your hypotheses?\n",
"1. How can you answer each of these things with a random forest? How do they work?:\n",
" - How confident are we in our projections using a particular row of data?\n",
"1. Explain why random forests are well suited to answering each of the following question:\n",
" - How confident are we in our predictions using a particular row of data?\n",
" - For predicting with a particular row of data, what were the most important factors, and how did they influence that prediction?\n",
" - Which columns are the strongest predictors?\n",
" - How do predictions vary, as we vary these columns?\n",
" - How do predictions vary as we vary these columns?\n",
"1. What's the purpose of removing unimportant variables?\n",
"1. What's a good type of plot for showing tree interpreter results?\n",
"1. What is the *extrapolation problem*?\n",
"1. How can you tell if your test or validation set is distributed in a different way to your training set?\n",
"1. Why do we make `saleElapsed` a continuous variable, even although it has less than 9000 distinct values?\n",
"1. What is boosting?\n",
"1. What is the \"extrapolation problem\"?\n",
"1. How can you tell if your test or validation set is distributed in a different way than your training set?\n",
"1. Why do we make `saleElapsed` a continuous variable, even although it has less than 9,000 distinct values?\n",
"1. What is \"boosting\"?\n",
"1. How could we use embeddings with a random forest? Would we expect this to help?\n",
"1. Why might we not always use a neural net for tabular modeling?"
]
@@ -8422,8 +8424,8 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"1. Pick a competition on Kaggle with tabular data (current or past) and try to adapt the techniques seen in this chapter to get the best possible results. Compare yourself to the private leaderboard.\n",
"1. Implement the decision tree algorithm in this chapter from scratch yourself, and try it on this dataset.\n",
"1. Pick a competition on Kaggle with tabular data (current or past) and try to adapt the techniques seen in this chapter to get the best possible results. Compare your results to the private leaderboard.\n",
"1. Implement the decision tree algorithm in this chapter from scratch yourself, and try it on the datase you used in the first exercise.\n",
"1. Use the embeddings from the neural net in this chapter in a random forest, and see if you can improve on the random forest results we saw.\n",
"1. Explain what each line of the source of `TabularModel` does (with the exception of the `BatchNorm1d` and `Dropout` layers)."
]