daviddoji/Fastbook_Julia
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Last updates

Browse Source
This commit is contained in:
Sylvain Gugger
2020-05-20 08:19:05 -07:00
parent cd1aa1f758
commit 3acc79d59a
5 changed files with 16 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
clean/12_nlp_dive.ipynb
View File

@@ -626,7 +626,7 @@
],
"source": [
"learn = Learner(dls, LMModel3(len(vocab), 64), loss_func=F.cross_entropy,\n",
" metrics=accuracy, cbs=ModelReseter)\n",
" metrics=accuracy, cbs=ModelResetter)\n",
"learn.fit_one_cycle(10, 3e-3)"
]
},
@@ -845,7 +845,7 @@
],
"source": [
"learn = Learner(dls, LMModel4(len(vocab), 64), loss_func=loss_func,\n",
" metrics=accuracy, cbs=ModelReseter)\n",
" metrics=accuracy, cbs=ModelResetter)\n",
"learn.fit_one_cycle(15, 3e-3)"
]
},
@@ -1022,7 +1022,7 @@
"source": [
"learn = Learner(dls, LMModel5(len(vocab), 64, 2), \n",
" loss_func=CrossEntropyLossFlat(), \n",
" metrics=accuracy, cbs=ModelReseter)\n",
" metrics=accuracy, cbs=ModelResetter)\n",
"learn.fit_one_cycle(15, 3e-3)"
]
},
@@ -1303,7 +1303,7 @@
"source": [
"learn = Learner(dls, LMModel6(len(vocab), 64, 2), \n",
" loss_func=CrossEntropyLossFlat(), \n",
" metrics=accuracy, cbs=ModelReseter)\n",
" metrics=accuracy, cbs=ModelResetter)\n",
"learn.fit_one_cycle(15, 1e-2)"
]
},
@@ -1382,7 +1382,7 @@
"source": [
"learn = Learner(dls, LMModel7(len(vocab), 64, 2, 0.5),\n",
" loss_func=CrossEntropyLossFlat(), metrics=accuracy,\n",
" cbs=[ModelReseter, RNNRegularizer(alpha=2, beta=1)])"
" cbs=[ModelResetter, RNNRegularizer(alpha=2, beta=1)])"
]
},
{
@@ -1565,7 +1565,7 @@
"1. Why can maintaining the hidden state in an RNN lead to memory and performance problems? How do we fix this problem?\n",
"1. What is \"BPTT\"?\n",
"1. Write code to print out the first few batches of the validation set, including converting the token IDs back into English strings, as we showed for batches of IMDb data in <<chapter_nlp>>.\n",
"1. What does the `ModelReseter` callback do? Why do we need it?\n",
"1. What does the `ModelResetter` callback do? Why do we need it?\n",
"1. What are the downsides of predicting just one output word for each three input words?\n",
"1. Why do we need a custom loss function for `LMModel4`?\n",
"1. Why is the training of `LMModel4` unstable?\n",

2
clean/16_accel_sgd.ipynb
View File

@@ -614,7 +614,7 @@
"metadata": {},
"outputs": [],
"source": [
"class ModelReseter(Callback):\n",
"class ModelResetter(Callback):\n",
" def begin_train(self): self.model.reset()\n",
" def begin_validate(self): self.model.reset()"
]