Fixing tanh formula and typo in 01_intro (#276)

* fixing error in tanh formula #273

* fixes typo described in #241
This commit is contained in:
Tanner Gilbert 2020-10-19 18:55:57 +02:00 committed by GitHub
parent 34c82464f9
commit 0e40e9fbe6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 4 additions and 4 deletions

View File

@ -2282,7 +2282,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"A brief one-line explanation is provided by `doc`. The \"Show in docs\" link take you to the full documentation, where you'll find all the details and lots of examples. Also, most of fastai's methods are just a handful of lines, so you can click the \"source\" link to see exactly what's going on behind the scenes.\n",
"A brief one-line explanation is provided by `doc`. The \"Show in docs\" link takes you to the full documentation, where you'll find all the details and lots of examples. Also, most of fastai's methods are just a handful of lines, so you can click the \"source\" link to see exactly what's going on behind the scenes.\n",
"\n",
"Let's move on to something much less sexy, but perhaps significantly more widely commercially useful: building models from plain *tabular* data."
]

View File

@ -1582,7 +1582,7 @@
"source": [
"In this picture, our input $x_{t}$ enters on the left with the previous hidden state ($h_{t-1}$) and cell state ($c_{t-1}$). The four orange boxes represent four layers (our neural nets) with the activation being either sigmoid ($\\sigma$) or tanh. tanh is just a sigmoid function rescaled to the range -1 to 1. Its mathematical expression can be written like this:\n",
"\n",
"$$\\tanh(x) = \\frac{e^{x} + e^{-x}}{e^{x}-e^{-x}} = 2 \\sigma(2x) - 1$$\n",
"$$\\tanh(x) = \\frac{e^{x} - e^{-x}}{e^{x}+e^{-x}} = 2 \\sigma(2x) - 1$$\n",
"\n",
"where $\\sigma$ is the sigmoid function. The green circles are elementwise operations. What goes out on the right is the new hidden state ($h_{t}$) and new cell state ($c_{t}$), ready for our next input. The new hidden state is also used as output, which is why the arrow splits to go up.\n",
"\n",