Another attempt to remove output - script did not point to ipython
This commit is contained in:
Roger Labbe committed
1 parent
9ae5aea761
commit
71b1b12212
1 file changed
+170
-200
+170
-200
@@ -1,201 +1,171 @@
|
||||
{
|
||||
"metadata": {
|
||||
"name": ""
|
||||
},
|
||||
"nbformat": 3,
|
||||
"nbformat_minor": 0,
|
||||
"worksheets": [
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"#Gaussian Probabilities\n",
|
||||
"\n",
|
||||
"#### Introduction\n",
|
||||
"\n",
|
||||
"The histogram filter uses a vector or multidimensional array of probabilities to represent our belief of the system state. This is very powerful, but it has limitations. Let us list a few. First, it is multimodal, and we may not want a multimodal result. In other words, we would not want our GPS telling us that we are 50% likely to be here, 40% likely to be there, and so on. There are plenty of techniques to go from a multimodal histogram to a unimodal result, but they all involve significant processing time. This becomes worse as we extend the problem into multiple dimensions. In general for $d$ dimensions the computational run time will be $O(n^d)$. So these filters become intractable in higher dimensions. \n",
|
||||
"\n",
|
||||
"Second, the results are hard to analyze. Recall the GPS in your car. It probably has a display that tells you how much error is in the filter - it tells you are at a specific location on the earth, with for example 9 meters of error. How would you derive that information from a histogram filter? There is no clear cut way. Most, if not all of the bins in the histogram will be non-zero, and there will usually be several clusters of higher probabilities. Being multimodal, there is no clear way to derive a single error estimate. Certainly one could come up with a heuristic, but often we want mathematically precise answers, not heuristics.\n",
|
||||
"\n",
|
||||
"Third, the histogram is discrete, but we live in a continuous world. The histogram requires that you model the output of your filter as a set of discrete points. In our dog in the hallway example, we used 10 positions, which is obviously far too few positions for anything but a toy problem. For example, for a 100 meter hallway you would need 10,000 positions to model the hallway to 1cm accuracy. So each sense and update operation would entail performing calculations for 10,000 different probabilities. It gets exponentially worse as we add dimensions. If our dog was roaming in a $100x100 m^2$ courtyard, we would need 100,000,000 bins ($10,000^2$) to get 1cm accuracy.\n",
|
||||
"\n",
|
||||
"Finally, the histogram does not represent what happens in the pysical world very well. For example, in the last chapter we had this as a probability distribution: [ 0.2245871 0.06288015 0.06109133 0.0581008 0.09334062 0.2245871\n",
|
||||
" 0.06288015 0.06109133 0.0581008 0.09334062]. The largest probabilities are in position 0 and position 5. This does not fit our physical intuition at all. A dog cannot be in two places at once.\n",
|
||||
" \n",
|
||||
" \n",
|
||||
"Consider how a bimetallic thermometer works. These thermometers use a strip of bimetallic material bent into a loose coil shape, with a pointer attached to the spring. The two metals expand at different rates, so the strip will either coil tighter or uncoil as the temperature changes, and the pointer indicates the current temperature. It is in some ways a crude system, yet it works well. What kinds of errors might we expect from this system? It will not respond to rapid temperature changes, so it will always lag the current temperature a small amount. There may be frictions in the system causing the pointer to not register the correct value. And so on. Finally, when you read the dial you will rarely be exactly perpindicular to the face of the dial. Viewing the pointer against the dial from an angle will result in a small reading error - 34 might look like 35 from the left of the dial, for example. However, in total all of these effects will generally be low, and we would expect the total errors to always be small. We would probably agree that an error of 2 degrees is reasonable, but an error of 100 degrees would occur extremely rarely, if ever. Furthermore, we would expect the errors to cluster around the correct value. So in most systems we would expect the errors to equally likely to be larger or smaller than the correct value. If the correct temperature was 35 degrees, and we were told the thermometer was accurate within 2 degrees, we wouldn't be suprised to read either 33 degrees or 37 degrees. \n",
|
||||
"\n",
|
||||
"So we desire a unimodal, continuous way to represent probabilities that models how the real world works, and that is very computationally efficient to calculate. As you might guess from the chapter name, gaussian distributions provide all of these features.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"#### Probability Distributions\n",
|
||||
"\n",
|
||||
"Before we go into the math, lets look at a graph of the gaussian distribution. Don't bother reading the code yet; it is not important at this stage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"collapsed": false,
|
||||
"input": [
|
||||
"import math\n",
|
||||
"\n",
|
||||
"def gaussian (x, mu, sigma):\n",
|
||||
" ''' compute the gaussian with the specified mean(mu) and sigma'''\n",
|
||||
" return math.exp (-0.5 * (x-mu)**2 / sigma) / math.sqrt(2.*math.pi*sigma)\n"
|
||||
],
|
||||
"language": "python",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"prompt_number": 1
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"collapsed": false,
|
||||
"input": [
|
||||
"%matplotlib inline\n",
|
||||
"\n",
|
||||
"xs = arange(0,10,0.1)\n",
|
||||
"ys = [gaussian (x, 5, 3) for x in xs]\n",
|
||||
"plot (xs, ys)\n",
|
||||
"show()"
|
||||
],
|
||||
"language": "python",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"metadata": {},
|
||||
"output_type": "display_data",
|
||||
"png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAEACAYAAABS29YJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlYlWX+x/H3UWjKJVNDFNBQQIFUtFBSszBFR0tLs4my\ndBIdszGzsfmZNU3aYlKTbTiNY9poGVlZaaXUWFHmEpmWJZotUIhpTrnkksDh+f1xF2bqERDOfZbP\n67q8CjgPfDzgl++5n3txOY7jICIiAa2O7QAiIlL7VOxFRIKAir2ISBBQsRcRCQIq9iIiQUDFXkQk\nCJyw2Ofk5BAfH09cXByZmZlHfXzBggUkJSXRsWNHevTowYYNGyo+Fh0dTceOHencuTNdu3at2eQi\nIlJpLk/z7N1uN+3atWP58uVERkbSpUsXsrOzSUhIqHjM6tWrSUxMpFGjRuTk5DBlyhTWrFkDQOvW\nrfnwww9p0qRJ7f9NRETkuDx29nl5ecTGxhIdHU1oaCjp6eksXrz4iMd069aNRo0aAZCSksLWrVuP\n+LjWbImI2Oex2BcXF9OyZcuKt6OioiguLj7u4+fMmcOAAQMq3na5XPTp04fk5GRmz55dA3FFRKQ6\nQjx90OVyVfoTvf3228ydO5eVK1dWvG/lypW0aNGCnTt3kpaWRnx8PD179qx+WhERqRaPxT4yMpKi\noqKKt4uKioiKijrqcRs2bGD06NHk5OTQuHHjive3aNECgLCwMAYPHkxeXt5RxT42NpYvv/zypP4S\nIiLBJiYmhi+++KLyFzgelJaWOm3atHEKCgqcQ4cOOUlJSU5+fv4Rj/n666+dmJgYZ/Xq1Ue8f//+\n/c7evXsdx3Gcffv2Od27d3def/31o77GCSIElTvvvNN2BJ+h5+IwPReH6bk4rKq102NnHxISQlZW\nFv369cPtdpORkUFCQgKzZs0CYMyYMdx1113s2rWLsWPHAhAaGkpeXh7bt29nyJAhAJSVlTFs2DD6\n9u1brd9gIiJycjwWe4D+/fvTv3//I943ZsyYiv9/4okneOKJJ466rk2bNnz00Uc1EFFERE6WVtD6\nkNTUVNsRfIaei8P0XBym56L6PC6q8koAl0tz8UVEqqiqtVOdvYhIEFCxFxEJAir2IiJBQMVeRCQI\nqNiLiAQBFXsRkSCgYi8iEgRU7EVEgoCKvYhIEFCxFxEJAifcCE0kmPz0E3z7LWzbBt99B40bQ0SE\n+dOgge10ItWnYi9BzXHgk09gyRLz5+OPoXlziIyEsDDYvdsU/uJiiIqCQYPg0kuhWzcI0b8e8SPa\nCE2CUnk5vPQSTJ0K+/aZAj5oEJx/PoSGHv14x4H16w//UtixA269FUaPhlNP9X5+karWThV7CTqv\nvAJ33GE687vugv79oQrHLQOwbh1MmWJ+Adx2myn66vTFm1TsRY5j1y74859Ngb7/frjkkqoX+d/6\n4AOYNAkOHoSnn4aYmJrJKnIi2uJY5BjefhuSkuDMM01XPnDgyRd6gC5dYPlySE+H886DuXPNkI+I\nr1FnLwHNcSAzEx57DObMgd//vva+1qefwjXXQIcO5mudckrtfS0RDeOI/MzthvHj4b33YNkyM32y\nth08CFddZW76vvginH567X9NCU4axhHBFN0rroBNm+Ddd71T6AFOOw0WLYK2beGCC8y0TRFfoGIv\nAefgQbj4Yvjd70xH36iRd79+3bowcyb84Q9mKmdxsXe/vsixaLKYBJTSUrjySmjWzMyOqVvXTg6X\ny0zJrFsX0tLMq4szz7STRQQ0Zi8BpLwcrr3WrHp96SXfuUF6663w5pvmj8bwpaboBq0EJceBG2+E\nDRsgJwfq1bOd6DDHgbFjYfNmk00rbqUm6AatBKWsLDNU8sorvlXowQzpzJxphnGuv17z8MUOdfbi\n995+20x3XL0aWre2neb49u+H7t0hI8NMCRU5GRrGkaBSWGhWri5YAL17205zYgUFZsfM7Gzo1ct2\nGvFnGsaRoHHgAAwebPam8YdCD+aVx4IF5pVIYaHtNBJM1NmL3xo92sypf+qpmtnnxptmzICFC83q\n3mNtqSxyIursJSgsWmTG6h9/3P8KPcDNN0PTpmY/fRFvUGcvfmfrVjj3XHOISEqK7TTVt2MHdOoE\nzz4LF15oO434G3X2EtDcbrNwavx4/y70AOHhZnfMa681e+2L1CZ19uJXMjNh6VJ46y17WyHUtJtu\ngu3bzRi+SGVp6qUErM2bzcZiH34IZ51lO03N+ekn6NwZ7rkHLr/cdhrxFyr2EpDKy6FnT7j6anO0\nYKBZudLskvnpp9C4se004g9qfMw+JyeH+Ph44uLiyMzMPOrjCxYsICkpiY4dO9KjRw82bNhQ6WtF\nKuuf/4Q6dcweM4GoRw/T1U+caDuJBCzHg7KyMicmJsYpKChwSkpKnKSkJCc/P/+Ix6xatcrZvXu3\n4ziOs2zZMiclJaXS1/78qsJTBBGnoMBxzjzTcTZvtp2kdv34o+OcdZbjvP667STiD6paOz129nl5\necTGxhIdHU1oaCjp6eksXrz4iMd069aNRj+fDpGSksLWrVsrfa3IiTgOjBljOt527WynqV0NGsCs\nWebvu2+f7TQSaDwW++LiYlq2bFnxdlRUFMUejt2ZM2cOAwYMqNa1Isfy3HNmpkqwDG/062duQt99\nt+0kEmg8nlTlqsLSxLfffpu5c+eycuXKKl87ZcqUiv9PTU0lNTW10tdK4Nq3D265xWwaFkxbCjzw\nAHToANddB/HxttOIr8jNzSU3N7fa13ss9pGRkRQVFVW8XVRURFRU1FGP27BhA6NHjyYnJ4fGP08l\nqOy1cGSxF/nF3XebnSHPP992Eu9q3hxuv90cxvLGG/65HYTUvN82wlOruteGpwH90tJSp02bNk5B\nQYFz6NChY95k/frrr52YmBhn9erVVb62OjcZJDhs2mRuyn77re0kdpSWOk779o7z/PO2k4ivqmrt\n9NjZh4SEkJWVRb9+/XC73WRkZJCQkMCsWbMAGDNmDHfddRe7du1i7M9z4kJDQ8nLyzvutSIn8ssR\ng7ffbrrcYBQSYk63uuYa6N8f6te3nUj8nRZVic958UW4805Yv94UvWA2bBhER8O999pOIr5GK2jF\nrx06BImJZgpinz6209hXXAwdO5pffK1a2U4jvkS7XopfmzkTEhJU6H8RGWm2h7jtNttJxN+psxef\n8f33Zqrhu++agi/Gvn3Qti0sXgxduthOI75CwzjityZMgJISsw+OHGnOHJg3D955R1MxxVCxF7+0\nZQt07w75+dCsme00vsfthnPOMTeuhwyxnUZ8gYq9+KUhQ6BrV7j1VttJfNd//ws33GB+IQbTimI5\nNt2gFb/z/vvwwQfmxCY5vrQ0aN3aDOmIVJU6e7HKcaB3b3MoyahRttP4vrVr4dJL4fPPoV4922nE\nJnX24leWLzdzyf/4R9tJ/ENysrm38dhjtpOIv1FnL9Y4jplKOGkSXHGF7TT+47PPzOZwW7boCMNg\nps5e/MaiRabg65DtqmnXzgzlPPCA7STiT9TZixVlZdC+PTz6KPTtazuN/ykqgk6dzAHlLVrYTiM2\nqLMXv7BgAYSHmxkmUnUtW8KIETB9uu0k4i/U2YvXlZaabRHmzoULL7Sdxn/t2GG2ldiwAY5zLpAE\nMHX24vPmzzfzxVXoT054uJmuet99tpOIP1BnL15VUmI29VqwAHr0sJ3G/+3caV4laQvk4KPOXnza\nk0+a4qRCXzPCwmDMGB1uIiemzl685tAhiIuD55+HlBTbaQLH99+bV0sffmhOtZLgoM5efNacOWa6\npQp9zWra1GyQds89tpOIL1NnL15RUgKxsfDCC2Z3S6lZP/xgXjWpuw8e6uzFJ82bZ86WVaGvHU2a\nmLH7zEzbScRXqbOXWldaasaUn35aN2Zr086dZisFzbsPDursxecsWGDm1avQ166wMMjIgPvvt51E\nfJE6e6lVZWVmlefs2ZCaajtN4PtlVe3GjdozJ9Cpsxef8uyz0Ly5Vst6S3g4DB+uHTHlaOrspdaU\nl5uplg8/rJ0tvWnbNvO8f/aZGdqRwKTOXnzGSy9Bgwba2dLbIiLgyivNL1mRX6izl1rhOOYIvTvu\ngMsus50m+BQUmFPAvvwSGjWynUZqgzp78QlvvGG2Rxg0yHaS4NS6NQwYADNn2k4ivkKdvdSKCy4w\ni3yGDbOdJHjl50OvXqbLr1fPdhqpaersxboVK6C42Iwbiz2JiWZtw+zZtpOIL1BnLzVuwAAYPBhG\nj7adRNauNd+LL7+EU06xnUZqkjp7sWr9erNcf/hw20kEzE3yxERzOpgEN3X2UqOuvNJsYfyXv9hO\nIr/IzYU//Qk2bYK6dW2nkZqizl6s+fxzeOstDd/4mgsvNLtivvSS7SRi0wmLfU5ODvHx8cTFxZF5\njP1TN2/eTLdu3Tj11FN58MEHj/hYdHQ0HTt2pHPnznTV3rYB74EHzCEaDRvaTiK/5nLB5MnmYHK9\niA5eHodx3G437dq1Y/ny5URGRtKlSxeys7NJSEioeMzOnTv5+uuvefnll2ncuDETJ06s+Fjr1q35\n8MMPadKkyfEDaBgnIPyyRH/LFjjzTNtp5LfKy6FDB7OqViuaA0ONDuPk5eURGxtLdHQ0oaGhpKen\ns3jx4iMeExYWRnJyMqGhocf8HCrkweGhh8xNWRV631SnDkyaBNOn204itngs9sXFxbRs2bLi7aio\nKIqLiyv9yV0uF3369CE5OZnZmuwbsHbtgrlzdVPW1111FXzxBeTl2U4iNoR4+qDL5TqpT75y5Upa\ntGjBzp07SUtLIz4+np49e57U5xTfM3MmDBwIrVrZTiKehIbCLbeYsXvdrA0+Hot9ZGQkRULine truncated
|
||||
"text": [
|
||||
"<matplotlib.figure.Figure at 0x5da42e8>"
|
||||
]
|
||||
}
|
||||
],
|
||||
"prompt_number": 2
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Probably this is immediately recognizable to you as a 'bell curve'. This curve is ubiquitious because under real world conditions most observations are distributed in such a manner. We will not prove the math here, but the **central limit theorem** proves that under certain conditions the arithmetic mean of independent observations will be distributed in this manner, even if the observations themselves do not have this distribution. In nonmathematical terms, this means that if you take a bunch of measurements from a sensor and use them in a filter, they are very likely to create this distribution.\n",
|
||||
"\n",
|
||||
"Before we go further, a moment for terminology. This is variously called a normal distribution, a Gaussian distribution, or a bell curve. However, other distributions also have a bell shaped curve, so that name is somewhat ambiguous, and we will not use it again."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"#### Gaussian Distributions\n",
|
||||
"\n",
|
||||
"So let us explore how gaussians work. A gaussian is a continuous probability distribution that is completely described with two parameters, the mean ($\\mu$) and the variance ($\\sigma^2$). It is defined as:\n",
|
||||
"$$ \n",
|
||||
"f(x, \\mu, \\sigma) = \\frac{1}{\\sigma\\sqrt{2\\pi}} e^{-0.5*{(x-\\mu)^2}/\\sigma^2 }\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You will not need to understand how this equation comes about, or remember it for this book, but it is useful to look at it to see how it works. Specifically, notice the term for $e$. When $x==\\mu$, the term reduces to $e^0=1$. Any other value of x will result in a smaller value for the exponent term due to the negative sign, so the curve will always be highest at $x==\\mu$.\n",
|
||||
"\n",
|
||||
"Now we will plot a gaussian centered around 23 ($\\mu=23$), with a variance of 1 ($\\sigma^2=1$). "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"collapsed": false,
|
||||
"input": [
|
||||
"xs = arange(16,30,0.1)\n",
|
||||
"ys = [gaussian (x,23,1) for x in xs]\n",
|
||||
"plt.plot (xs,ys, 'r')\n",
|
||||
"plt.axvline(23); plt.axvline(24) \n",
|
||||
"plt.show()"
|
||||
],
|
||||
"language": "python",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"metadata": {},
|
||||
"output_type": "display_data",
|
||||
"png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAEACAYAAABS29YJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X10U2WeB/BvoFEE5NUqJamGNqUNtk0LfeFlqmFoqTBO\nhcqc6VnfZuxoh12GYZzxuLNnZwTXszPVcTloXa3MyOroVmbFobpARIZGWRSC0OHFwqFAKyEI8k4p\nL23TZ/+4ptK3NGmTPPcm3885HJvkPvf+WnO/ffrkPs/VCSEEiIgoog2SXQAREYUew56IKAow7ImI\nogDDnogoCjDsiYiiAMOeiCgK9Bn2drsdKSkpSEpKQnl5ea/b7dixAzExMVizZk3AbYmIKLR0vq6z\n93g8SE5OxqZNm2AwGJCdnY2qqipYLJZu2xUUFGDo0KH48Y9/jPvvv9/vtkREFHo+e/ZOpxNmsxkm\nkwl6vR4lJSWorq7utt1LL72EBQsWIDY2NuC2REQUej7D3u12Iz4+vuOx0WiE2+3utk11dTUWLlwI\nANDpdH63JSKi8PAZ9t7g9mXJkiX4/e9/D51OByEEvKNC/rQlIqLwiPH1osFggMvl6njscrlgNBo7\nbbNz506UlJQAAE6fPo0NGzZAr9f71RYAzGYzDh8+PKBvgogo2iQmJuLQoUP+NxA+tLa2ioSEBNHQ\n0CCuXbsmrFarqKur63X7H/3oR2LNmjUBte2jBNV7+umnZZcwIKy/i1/8QoipU4WoqRGivV0AQogF\nC4Sw2YRoawvusYQQwNNB32e48L0jV6DZ6XMYJyYmBhUVFSgsLMSkSZPwwx/+EBaLBZWVlaisrPT5\nS6S3tkSqtWED8O67wLp1gM0GeIci33kHEALg5cOkYT6HcQBgzpw5mDNnTqfnysrKetx21apVfbYl\nUqUTJ4BHHwVWrwbGjOn82uDBwFtvAVOmADNnAtOmyamRaAA4g3aAbDab7BIGhPV/Y9Ei4Cc/Ae66\nq+fXjUagshJ46CGgrS04xwQA2IK4r/Die0dbfE6qCksB31zFQyRNfT0wfTrw5ZfA0KGdXtLplBGc\nDt/5DrBkCbBgQVAO3W3/RH4KNDvZsydasQJ4/PFuQd+jX/wC+I//CH1NREHGnj1Ft7NngcRE4Isv\ngPHju73creft8QBJScB//zcwdeqAD8+ePfUXe/ZEgVi5Evj+93sM+h4NHgz8/OfA8uWhrYsoyNiz\np+jV2gpMmAB88AGQmdnjJj32vJuaAJMJ2LULuOOOAZXAnj31F3v2RP6qqVF69L0Efa9uvhn44Q+B\nqqrQ1EUUAgx7il5r1gA/+EH/2i5YoLQn0ggO41B08niAuDhg2zYgIaHXzXodZmlrU/4qcDqVIZ1+\n4jAO9ReHcYj8sWWLMlHKR9D7FBMD3Hcf8N57wa2LKEQY9hSd3n134BOjFixQ9kOkARzGoejT3q70\n6h0OYOJEn5v6HGZpbQXGjQN271b21w8cxqH+4jAOUV8++wwYO7bPoO+TXq9co//XvwanLqIQYthT\n9Hn/fWD+/ODsa/58gPdWJg1g2FP02bQJmD07OPuaORPYvh24ciU4+yMKEYY9RZfTp5VVLnNygrO/\nESOAtDTg00+Dsz+iEGHYU3SpqQHy8oAbbgjePvPzlb8WiFSMYU/R5W9/U8I5mGbNUvZLpGIMe4ou\nmzYFP+ynTgUOHADOnQvufomCqM+wt9vtSElJQVJSEsp7uOFydXU1rFYrMjMzMWXKFGzevLnjNZPJ\nhPT0dGRmZiInWGOkRP3V0KCsWJmaGtz93ngjMGOGMkREpFI+J1V5PB4kJydj06ZNMBgMyM7ORlVV\nFSwWS8c2zc3NGDZsGABg7969mD9/Pg4dOgQAmDBhAnbu3IkxXW/gfH0BnFRF4fLHPwKbNys3HvGT\n35Oe/vAH4MgR4D//M6CSOKmK+iuok6qcTifMZjNMJhP0ej1KSkpQ3eWaYm/QA8ClS5dwyy23dHqd\nQU6qEYrxeq/8fI7bk6r5DHu32434+PiOx0ajEW63u9t2a9euhcViwZw5c/Diiy92PK/T6ZCfn4+s\nrCysXLkyiGUTBUgIZZhl5szQ7D89HThzBujh/CBSA59hr9Pp/NrJvHnzsH//fnzwwQd46KGHOp7f\nunUramtrsWHDBrz88svYsmXLwKol6q8jR5RbCg5gOWKfBg0Cpk3j9fakWjG+XjQYDHC5XB2PXS4X\njD4WfMrLy0NbWxvOnDmDsWPHIi4uDgAQGxuL+fPnw+l0Ii8vr1u7pUuXdnxts9lgs9kC/DaI+vDp\np8qHqH52YPplxgzlOP29IQqRDw6HAw6Ho/87ED60traKhIQE0dDQIK5duyasVquoq6vrtM2hQ4dE\ne3u7EEKInTt3ioSEBCGEEM3NzeLixYtCCCEuXbokpk+fLj788MNux+ijBKLg+OlPhVi+POBmAb09\nP/5YiJyc0O2f6DqBZqfPnn1MTAwqKipQWFgIj8eD0tJSWCwWVFZWAgDKysqwZs0avPnmm9Dr9Rg+\nfDjeeecdAMCJEydQXFwMAGhra8MDDzyA2cFaj4QoUJ9+Cvz4x6E9RlYWsG+fsk7OTTeF9lhEAeJ6\n9hT5LlwADAZl0pNeH1DTgC+NzM0Fnn8euOuu0Oyf6Btcz56oq+3bgSlTAg76fpk+nR/Skiox7Cny\nffqpEsLhwLAnlWLYU+TzXokTDt4rcjg2QyrDsKfI5vEowzhTp4bneOPHA8OHAwcPhud4RH5i2FNk\n27cPiIsDuizjEVIcyiEVYthTZHM6lStkwik3VzkukYow7Cmyff45kJ0d3mNmZyvHJVIRhj1Fts8/\nVyY7hVNGBvDFF8C1a+E9LpEPDHuKXFevAvv3A1ZreI87dChgNgN794b3uEQ+MOwpcu3ZA0ycKGfp\ngqwsDuWQqjDsKXLJGK/34rg9qQzDniKXjPF6r6wsYMcOOccm6gHDniKXzLBPTwfq64HLl+Ucn6gL\nhj1FpuZm4PBhIC1NzvFvvBGwWIDdu+Ucn6gLhj1Fpr//HbjzTuCGG+TVkJ3NoRxSDYY9RSaZQzhe\nvCKHVIRhT5Hp88+VNexlYtiTijDsKTLV1gKTJ8utYdIkoKFBuU0hkWQMe4o8V64AR44oY/Yy3XAD\nkJysrLxJJFmfYW+325GSkoKkpCSUl5d3e726uhpWqxWZmZmYMmUKNm/e7HdbopDYu1eZOSvzw1mv\njAzlw2IiyXzecNzj8SA5ORmbNm2CwWBAdnY2qqqqYLFYOrZpbm7GsGHDAAB79+7F/PnzcejQIb/a\nArzhOIXAa68Bn30GrFo14F0N+Ibgy5crl4BWVIRm/xS1gnrDcafTCbPZDJPJBL1ej5KSElRXV3fa\nxhv0AHDp0iXc8s1NIvxpSxQStbVAZqbsKhTs2ZNK+Ax7t9uN+Pj4jsdGoxFut7vbdmvXroXFYsGc\nOXPw4osvBtSWKOj+/nclZNXAalUWZGtvl10JRbkYXy/qdDq/djJv3jzMmzcPW7ZswUMPPYQDBw4E\nVMTSpUs7vrbZbLDZbAG1J+rg8Shj9moJ+zFjgFGjlKtyEhNlV0Ma5nA44HA4+t3eZ9gbDAa4XK6O\nxy6XC0ajsdft8/Ly0NbWhrNnz8JoNPrd9vqwJxqQgweBceOAESNkV/Itq1VZNoFhTwPQtSO8bNmy\ngNr7HMbJyspCfX09Ghsb0dLSgtWrV6OoqKjTNocPH+74kGDXrl0AgLFjx/rVlijo1DSE48Vxe1IB\nnz37mJgYVFRUoLCwEB6PB6WlpbBYLKisrAQAlJWVYc2aNXjzzTeh1+sxfPhwvPPOOz7bEoWUmj6c\n9bJagT//WXYVFOV8XnoZlgJ46SUFU0EBsGQJ8L3vBWV3Qbk08tAhYNYs4MsvQ7N/ikpBvfSSSFOE\nUIZL1NazT0gAzp1T/hFJwrCnyHH8uPLfuDi5dXQ1aJCyrj7XtieJGPYUOfbuVcbH/bxkOKwyMhj2\nJBXDniLHnj3K7QDVyGrlFTkkFcOeIoeaw549e5KMYU+RQ81hn5oKHDgAtLbKroSiFMOeIkNLC1Bf\nr9zkW42GDgXuuEMJfCIJGPYUGQ4cAEwm4KabZFfSO47bk0QMe4oMe/eqdwjHi+P2JBHDniKDmsfr\nvbhGDknEsKfIoIWw965+yfURSAKGPUWGPXuUWapqNm6cMpvWO9OXKIwY9qR9p08Dly4pV7uomU73\nbe+eKMwY9qR9e/cqvXo1LpPQFcftSRKGPWmfFq7E8WLPniRh2JP2aeHDWa/0dKVeojBj2JP2aSns\nk5OVm5hcvSq7EooyDHvSNo8H+OILZe0ZLbjhBiApCairk10JRRmGPWnb4cPArbcCI0bIrsR/aWkc\nyqGw6zPs7XY7UlJSkJSUhPLy8m6vv/3227BarUhPT8eMGTOw57o3sclkQnp6OjIzM5GTkxPcyokA\nbQ3heHHcniSI8fWix+PBokWLsGnTJhgMBmRnZ6OoqAiW61YWTEhIwCeffIKRI0fCbrfj8ccfx7Zt\n2wAoN8R1OBwYM2ZMaL8Lil5auhLHKz0d+Ogj2VVQlPHZs3c6nTCbzTCZTNDr9SgpKUF1dXWnbaZN\nm4aRI0cCAHJzc3Hs2LFOrwdy93OigLFnT+QXn2HvdrsRHx/f8dhoNMLtdve6/Z/+9CfMnTu347FO\np0N+fj6ysrKwcuXKIJRL1IUWwz4uDmhrA06elF0JRRGfwzi6AGYk1tTU4PXXX8fWrVs7ntu6dSvi\n4uJw6tQpFBQUICUlBXl5ed3aLl26tONrm80Gm83m93EpijU1AV99BZjNsisJjE53Xe++QHY1pBEO\nhwMOh6Pf7X2GvcFggMvl6njscrlgNBq7bbdnzx489thjsNvtGD16dMfzcXFxAIDY2FjMnzLine truncated
|
||||
"text": [
|
||||
"<matplotlib.figure.Figure at 0x84d8400>"
|
||||
]
|
||||
}
|
||||
],
|
||||
"prompt_number": 3
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"As we expected the curve is centered around 23. The width of the curve is defined by the variance. If the variance is large than the curve will be wide, and if the variance is small the curve will be narrow.\n",
|
||||
"\n",
|
||||
"So what does this curve *mean*? Assume for a moment that we have a themometer that reads 23$\\,^{\\circ}$. We have asserted (without proof) that the distribution will be *normal*. If that is true, then this chart can be interpreted as a continuous curve depicting our belief that the temperature is any given temperature. In this curve, we assign a probability of the temperature being exactly 23$\\,^{\\circ}$ is 40%. Looking to the right, we assign the probability that the temperature is 24$\\,^{\\circ}$ is about 25%. We find 20$\\,^{\\circ}$ and 26$\\,^{\\circ}$ quite unlikely, and temperatures beyond that range extremely rare. \n",
|
||||
"\n",
|
||||
"Since this is a probability distribution it is required that the area under the curve always equals one. This should be intuitively clear - the area under the curve represents all possible occurances, which must sum to one.\n",
|
||||
"\n",
|
||||
"This leads to an important insight. If the variance is small the curve will be narrow. To keep the area == 1, the curve must also be tall. On the other hand if the variance is large the curve will be wide, and thus it will also have to be short to make the area == 1.\n",
|
||||
"\n",
|
||||
"Let us look at that:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"collapsed": false,
|
||||
"input": [
|
||||
"plt.plot (xs,[gaussian(x, 23, .2) for x in xs],'b')\n",
|
||||
"plt.plot (xs,[gaussian(x, 23, 1) for x in xs],'g')\n",
|
||||
"plt.plot (xs,[gaussian(x, 23, 5) for x in xs],'r')\n",
|
||||
"plt.show()"
|
||||
],
|
||||
"language": "python",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"metadata": {},
|
||||
"output_type": "display_data",
|
||||
"png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAEACAYAAABMEua6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlclNX+B/DPwIwiLqDgOoOiDJuhgIKoaU65oF5D2wxb\nXEIjzcy6P7PtFmZ5pfK2SCndq6ZliKWJFmJuo6blaJqUuOCCDiMqi4iCMDBzfn8cQdaBgWf27/v1\nmpezPHOeb8R85nCe5zlHxBhjIIQQYhecLF0AIYQQ4VCoE0KIHaFQJ4QQO0KhTgghdoRCnRBC7AiF\nOiGE2JFGQz0tLQ0BAQHw9fVFfHx8nddv3LiBRx55BMHBwYiIiMDJkydNUighhJDGGQx1nU6HuXPn\nIi0tDRkZGUhKSsKpU6dqbLNkyRIMGDAAJ06cwLp16/Dyyy+btGBCCCENMxjqKpUKcrkc3t7ekEgk\niI6ORkpKSo1tTp06hQcffBAA4O/vj6ysLOTm5pquYkIIIQ0yGOoajQZeXl5Vj2UyGTQaTY1tgoOD\nsXnzZgD8S+DSpUvIzs42QamEEEIaYzDURSJRow28/vrrKCwsRGhoKBISEhAaGgpnZ2fBCiSEENJ0\nYkMvSqVSqNXqqsdqtRoymazGNu3bt8fq1aurHvfu3Rt9+vSp05ZcLsf58+dbWi8hhDgUHx8fnDt3\nrulvYAaUl5ezPn36sIsXL7KysjIWHBzMMjIyamxTWFjIysrKGGOMffXVV2zatGn1ttXIrqzeu+++\na+kSWsSW6zdl7SdPMjZ6NGMVFSbbhU3/7Bmj+i3N2Ow02FMXi8VISEhAZGQkdDodYmJiEBgYiMTE\nRABAbGwsMjIyMH36dIhEIgQFBWHVqlUt+E4ixLw++QTYvRtITQUeftjS1RDScgZDHQDGjRuHcePG\n1XguNja26v6QIUNw5swZ4SsjxMRyc4EffgCWLQM+/ZRCndgHuqK0iRQKhaVLaBFbrt9UtX/1FfDo\no8CcOcDp00B6ukl2Y9M/e4DqtzWiu2M2pt+RSAQz7YqQRmm1QO/eQFoa0K8fsGQJcP48QKOHxNoY\nm53UUycO6fvvgcBAHugA8PzzwObNwPXrlq2LkJaiUCcOadcu4Mkn7z329ASGDQMOHrRcTYQIgUKd\nOKTTp4GAgJrPBQTw5wmxZRTqxOEwBpw5U3+o04lcxNZRqBOHk5fH//X0rPm8vz+FOrF9FOrE4Zw5\nwwO89tRGlaFOJ2kRW0ahThzO6dM8wGur7LnTzNHEllGoE4dT33g6wHvuNK5ObB2FOnE4lcMv9aFx\ndWLrKNSJw2lo+AXgz9NpjcSWUagTh6LVApcvAz4+9b9OPXVi6yjUiUO5cAHw8gJat67/dRpTJ7aO\nQp04FEPj6QDvwV++zHv0hNgiCnXiUAyNpwNAq1a8J08rLxJb1Wiop6WlISAgAL6+voiPj6/zel5e\nHsaOHYuQkBAEBQXh66+/NkWdhAiisZ46QOPqxLYZDHWdToe5c+ciLS0NGRkZSEpKwqlTp2psk5CQ\ngNDQUPz5559QKpX45z//iYqKCpMWTUhzNXSOenU0rk5smcFQV6lUkMvl8Pb2hkQiQXR0NFJSUmps\n0717dxQVFQEAioqK4OHhAbG40VXyCLGIs2cBPz/D21BPndgyg6Gu0Wjg5eVV9Vgmk0Gj0dTYZtas\nWTh58iR69OiB4OBgfPbZZ6aplJAWKi0FioqArl0Nb+flBdT6NSfEZhjsUotqz3hUjyVLliAkJARK\npRLnz5/H6NGjceLECbRv377OtnFxcVX3FQqFw60dSCwrJwfo3r3uRF619egBXLlinpoIqU2pVEKp\nVDb7/QZDXSqVQq1WVz1Wq9WQyWQ1tjl06BDeeustAICPjw969+6NM2fOICwsrE571UOdEHO7coUH\ndmMo1Ikl1e7wLlq0yKj3Gxx+CQsLQ2ZmJrKysqDVapGcnIyoqKga2wQEBGDXrl0AgGvXruHMmTPo\n06ePUUUQYg5NDXUPD+D2bT5cQ4itMdhTF4vFSEhIQGRkJHQ6HWJiYhAYGIjExEQAQGxsLN58803M\nmDEDwcHB0Ov1+PDDD9GpUyezFE+IMZoa6iIRH6bJyQF69zZ9XYQIScSYeZYEEIlEMNOuCKnXwoVA\nx47A6683vu3QocBHHwH332/6uggxxNjspCtKicNoak8doHF1Yrso1InDoFAnjoBCnTgMCnXiCCjU\nicOgUCeOgEKdOITbt4HycsDNrWnbU6gTW0WhThxCTg4P6iZcJA2AQp3YLgp14hCMGXoB+LY0/wux\nRRTqxCEYG+pubkBFBXDrlulqIsQUKNSJQzA21EUivn1OjulqIsQUKNSJQzA21AEaVye2iUKdOAQK\ndeIoKNSJQ6BQJ46CQp04BAp14igo1IndY4xCnTgOCnVi94qKALEYaNfOuPdJpRTqxPZQqBO715xe\nOkA9dWKbGg31tLQ0BAQEwNfXF/Hx8XVe//jjjxEaGorQ0FD069cPYrEYhYWFJimWkOZobqh3787f\nS2u7EFticOUjnU4Hf39/7Nq1C1KpFOHh4UhKSkJgYGC92//000/49NNPq9YsrbEjWvmIWMj69cDP\nPwPffWf8e9u3B7Kzmz4RGCFCE3TlI5VKBblcDm9vb0gkEkRHRyMlJaXB7b/77jtMmTKl6dUSYgbX\nrwNduzbvvV26ALm5wtZDiCkZDHWNRgMvL6+qxzKZDJoGZjkqKSnBjh078NhjjwlbISEtdP060Llz\n897buTN/PyG2QmzoRVFT5ykFsG3bNgwbNgzu7u4NbhMXF1d1X6FQQKFQNLl9QporNxfw8Wnee6mn\nTsxNqVRCqVQ2+/0GQ10qlUKtVlc9VqvVkMlk9W67YcOGRodeqoc6IeZCPXViS2p3eBctWmTU+w0O\nv4SFhSEzMxNZWVnQarVITk5GVFRUne1u3ryJ/fv3Y+LEiUbtnBBzyM3lPe7moJ46sTUGe+pisRgJ\nCQmIjIyETqdDTEwMAgMDkZiYCACIjY0FAGzZsgWRkZFo06aN6SsmxEjXr7cs1C9dErYeQkzJ4CmN\ngu6ITmkkFtK+PV/FqEMH49/77bfA9u38tEhCLEHQUxoJsXV37vAFp9u3b977u3ShMXViWyjUiV3L\nzeUHO404kasGOlBKbA2FOrFrLTlICtCBUmJ7KNSJXWvJ6YwA4OnJQ50OBxFbQaFO7FpLznwBgNat\ngbZtAZqjjtgKCnVi11o6/ALQwVJiWyjUiV1r6fALwN9P4+rEVlCoE7vW0uEXgHrqxLZQqBO7VnlK\nY0vQaY3EllCoE7smVE+dhl+IraBQJ3aNeurE0VCoE7vFGPXUieOhUCd2q7iYTw/Qtm3L2qEDpcSW\nUKgTuyXE0AtApzQS20KhTuyWEEMvAPXUiW2hUCd2S6ieuocHkJ8P6PUtb4sQU2s01NPS0hAQEABf\nX1/Ex8fXu41SqURoaCiCgoJoMWliNYTqqUskgJsbUFDQ8rYIMTWDy9npdDrMnTsXu3btglQqRXh4\nOKKiohAYGFi1TWFhIV588UXs2LEDMpkMeXl5Ji+akKYQKtSBe0Mwnp7CtEeIqRjsqatUKsjlcnh7\ne0MikSA6OhopKSk1tvnuu+/w2GOPQSaTAQA86beeWAmhhl8AOlhKbIfBUNdoNPDy8qp6LJPJoNFo\namyTmZmJgoICPPjggwgLC8M333xjmkoJMZIpeuqEWDuDwy+iJqwBVl5ejmPHjmH37t0oKSnBkCFD\nMHjwYPj6+tbZNi4uruq+QqGg8XdiUkL31CnUiTkolUoolcpmv99gqEulUqjV6qrHarW6apilkpeX\nFzw9PdGmTRu0adMGDzzwAE6cONFoqBNiakKHOh0uIuZQu8O7aNEio95vcPglLCwMmZmZyMrKglar\nRXJyMqKiompsM3HiRPz666/Q6XQoKSnB4cOH0bdvX6OKIMQU8vKEO7Dp6UmhTmyDwZ66WCxGQkIC\nIiMjodPpEBMTg8DAQCQmJgIAYmNjERAQgLFjx6J///5wcnLCrFmzKNSJVRC6p37okDBtEWJKIsbM\ns6SuSCSCmXZFCEpK+EVDJSV8/peW2rkTiI8Hdu1qeVuEGMPY7KQrSoldquylCxHoAJ3SSGwHhTqx\nS0KOpwM0pk5sB4U6sUtCjqcD90KdRhCJtaNQJ3ZJ6J66iwvQqhVw65ZwbRJiChTqxC4J3VMHaFyd\n2AYKdWKXhO6pAzSuTmwDhTqxS9RTJ46KQp3YJeqpE0dFoU7sEvXUiaOiUCd2iXrqxFFRqBO7RD11\n4qgo1Ind0emAwkKgUydh26WeOrEFFOrE7hQUAO7ugLOzsO1ST53YAgp1YndMMZ4OUE+d2AYKdWJ3\ncnNNF+rUUyfWrtFQT0tLQ0BAAHx9fREfH1/ndaVSCTc3N4SGhiI0NBTvv/++SQolpKny8oQ/SArw\nIZ3iYqC8XPi2CRGKwZWPdDod5s6di127dkEqlSI8PBxRUVEIDAyssd2IESOwdetWkxZKSFOZqqfu\n5MQPvublAd27C98+IUIw2FNXqVSQy+Xw9vaGRCJBdHQ0UlJS6mxHKxoRa2KqnjpAC1AT62cw1DUa\nDby8vKoey2QyaDSaGtuIRCIcOnQIwcHBGD9+PDIyMkxTKSFNZKqeOkDj6sT6GRx+ETVhLbABAwZA\nrVbD1dUV27dvx6RJk3D27FnBCiTEWHl5wMCBpmmbeurE2hkMdalUCrVaXfVYrVZDJpPV2KZ9+/ZV\n98eNG4c5c+agoKAAneq58iMuLq7qvkKhgEKhaGbZhDSMeurElimVSiiVyma/X8QMDIhXVFTA398f\nu3fvRo8ePTBo0CAkJSXVOFB67do1dOnSBSKRCCqVCpMnT0ZWVlbdHRm5IjYhzTVwIJCYCILine truncated
|
||||
"text": [
|
||||
"<matplotlib.figure.Figure at 0x86255c0>"
|
||||
]
|
||||
}
|
||||
],
|
||||
"prompt_number": 4
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"So what is this telling us? The blue gaussian is very narrow. It is saying that we believe x=23, and that we are very sure about that. In contrast, the red gaussian also believes that x=23, but we are much less sure about that. Our believe that x=23 is lower, and so our belief about the likely possible values for x is spread out - we think it is quite likely that x=2 or x=8, for example. The blue gaussian has almost completely eliminated 22 or 24 as possible value - their probably is almost 0.0, whereas the red curve considers them nearly as likely as 23.\n",
|
||||
"\n",
|
||||
"If we think back to the thermometer, we can consider these three curves as representing 3 thermometers. The blue curve represents a very accurate thermometer, and the red one represents a fairly inaccurate one. Green of course represents one in between the two others. Note the very powerful property the Gaussian distribution affords us - we can entirely represent both the reading and the error of a thermometer with only two numbers - the mean and the variance.\n",
|
||||
"\n",
|
||||
"The standard notation for a normal distribution is just $N(\\mu,\\sigma^2)$. I will not go into detail as to why $\\sigma^2$ is used, other than to note that $\\sigma$ is commonly called the *standard deviation*, which has enormous utility in statistics. The standard deviation is not really used in this book, so I will not address it further. The important thing to understand is that the variance ($\\sigma^2$) is a measure of the width of the curve. The curve above is notated as $N(23, 1)$, since $\\mu=23$ and $\\sigma=1$. We will use this notiation throughout the rest of the book, so learn it now.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"#### Computational Properties of the Gaussian\n",
|
||||
"\n",
|
||||
"Recall how our histogram filter worked. We had a vector (Python array) representing our belief at a certain moment in time. When we performed another measurement using the *sense()* function we had to multiply probabilities together, and when we performed the motion step using the *update()* function we had to shift and add probabilities. I've promised you that the Kalman filter uses essentially the same process, and that it uses Gaussians instead of histograms, so you might reasonable expect that we will be multipling, adding, and shifting Gaussians in the Kalman filter.\n",
|
||||
"\n",
|
||||
"A typical math book would directly launch into a multipage proof of the behavior of Gaussians under these operations, but I don't see the value in that unless you plan to do statistics. I think the math will be much more intuitive and clear if we just start developing a Kalman filter using Gaussians, and I will provide the equations for multiplying and shifting Gaussians at the appropriate time. You will then be able to develop a physical intuition for what these operations do, rather than be forced to digest a lot of fairly abstract math.\n",
|
||||
"\n",
|
||||
"The key point, which I will only assert for now, is that all the operations are very simple, and that they preserve the properties of the Gaussian. This is somewhat remarkable, in that the Gaussian is a nonlinear function, and typically if you multiply a nonlinear equation with itself you end up with a different equation. For example, the shape of $sin(x)sin(x)$ is very different from $sin(x)$. But the result of multiplying two Gaussians is yet another Gaussian. This is a fundamental discovery, and the key reason why Kalman filters are possible\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"#### Summary and Key Points"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {}
|
||||
}
|
||||
]
|
||||
{
|
||||
"metadata": {
|
||||
"name": "",
|
||||
"signature": "sha256:7987517a7cb3fbbd21339999a44f481804ef1719e02dea0e02c2456b42d2d036"
|
||||
},
|
||||
"nbformat": 3,
|
||||
"nbformat_minor": 0,
|
||||
"worksheets": [
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"#Gaussian Probabilities\n",
|
||||
"\n",
|
||||
"#### Introduction\n",
|
||||
"\n",
|
||||
"The histogram filter uses a vector or multidimensional array of probabilities to represent our belief of the system state. This is very powerful, but it has limitations. Let us list a few. First, it is multimodal, and we may not want a multimodal result. In other words, we would not want our GPS telling us that we are 50% likely to be here, 40% likely to be there, and so on. There are plenty of techniques to go from a multimodal histogram to a unimodal result, but they all involve significant processing time. This becomes worse as we extend the problem into multiple dimensions. In general for $d$ dimensions the computational run time will be $O(n^d)$. So these filters become intractable in higher dimensions. \n",
|
||||
"\n",
|
||||
"Second, the results are hard to analyze. Recall the GPS in your car. It probably has a display that tells you how much error is in the filter - it tells you are at a specific location on the earth, with for example 9 meters of error. How would you derive that information from a histogram filter? There is no clear cut way. Most, if not all of the bins in the histogram will be non-zero, and there will usually be several clusters of higher probabilities. Being multimodal, there is no clear way to derive a single error estimate. Certainly one could come up with a heuristic, but often we want mathematically precise answers, not heuristics.\n",
|
||||
"\n",
|
||||
"Third, the histogram is discrete, but we live in a continuous world. The histogram requires that you model the output of your filter as a set of discrete points. In our dog in the hallway example, we used 10 positions, which is obviously far too few positions for anything but a toy problem. For example, for a 100 meter hallway you would need 10,000 positions to model the hallway to 1cm accuracy. So each sense and update operation would entail performing calculations for 10,000 different probabilities. It gets exponentially worse as we add dimensions. If our dog was roaming in a $100x100 m^2$ courtyard, we would need 100,000,000 bins ($10,000^2$) to get 1cm accuracy.\n",
|
||||
"\n",
|
||||
"Finally, the histogram does not represent what happens in the pysical world very well. For example, in the last chapter we had this as a probability distribution: [ 0.2245871 0.06288015 0.06109133 0.0581008 0.09334062 0.2245871\n",
|
||||
" 0.06288015 0.06109133 0.0581008 0.09334062]. The largest probabilities are in position 0 and position 5. This does not fit our physical intuition at all. A dog cannot be in two places at once.\n",
|
||||
" \n",
|
||||
" \n",
|
||||
"Consider how a bimetallic thermometer works. These thermometers use a strip of bimetallic material bent into a loose coil shape, with a pointer attached to the spring. The two metals expand at different rates, so the strip will either coil tighter or uncoil as the temperature changes, and the pointer indicates the current temperature. It is in some ways a crude system, yet it works well. What kinds of errors might we expect from this system? It will not respond to rapid temperature changes, so it will always lag the current temperature a small amount. There may be frictions in the system causing the pointer to not register the correct value. And so on. Finally, when you read the dial you will rarely be exactly perpindicular to the face of the dial. Viewing the pointer against the dial from an angle will result in a small reading error - 34 might look like 35 from the left of the dial, for example. However, in total all of these effects will generally be low, and we would expect the total errors to always be small. We would probably agree that an error of 2 degrees is reasonable, but an error of 100 degrees would occur extremely rarely, if ever. Furthermore, we would expect the errors to cluster around the correct value. So in most systems we would expect the errors to equally likely to be larger or smaller than the correct value. If the correct temperature was 35 degrees, and we were told the thermometer was accurate within 2 degrees, we wouldn't be suprised to read either 33 degrees or 37 degrees. \n",
|
||||
"\n",
|
||||
"So we desire a unimodal, continuous way to represent probabilities that models how the real world works, and that is very computationally efficient to calculate. As you might guess from the chapter name, gaussian distributions provide all of these features.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"#### Probability Distributions\n",
|
||||
"\n",
|
||||
"Before we go into the math, lets look at a graph of the gaussian distribution. Don't bother reading the code yet; it is not important at this stage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"collapsed": false,
|
||||
"input": [
|
||||
"import math\n",
|
||||
"\n",
|
||||
"def gaussian (x, mu, sigma):\n",
|
||||
" ''' compute the gaussian with the specified mean(mu) and sigma'''\n",
|
||||
" return math.exp (-0.5 * (x-mu)**2 / sigma) / math.sqrt(2.*math.pi*sigma)\n"
|
||||
],
|
||||
"language": "python",
|
||||
"metadata": {},
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"collapsed": false,
|
||||
"input": [
|
||||
"%matplotlib inline\n",
|
||||
"\n",
|
||||
"xs = arange(0,10,0.1)\n",
|
||||
"ys = [gaussian (x, 5, 3) for x in xs]\n",
|
||||
"plot (xs, ys)\n",
|
||||
"show()"
|
||||
],
|
||||
"language": "python",
|
||||
"metadata": {},
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Probably this is immediately recognizable to you as a 'bell curve'. This curve is ubiquitious because under real world conditions most observations are distributed in such a manner. We will not prove the math here, but the **central limit theorem** proves that under certain conditions the arithmetic mean of independent observations will be distributed in this manner, even if the observations themselves do not have this distribution. In nonmathematical terms, this means that if you take a bunch of measurements from a sensor and use them in a filter, they are very likely to create this distribution.\n",
|
||||
"\n",
|
||||
"Before we go further, a moment for terminology. This is variously called a normal distribution, a Gaussian distribution, or a bell curve. However, other distributions also have a bell shaped curve, so that name is somewhat ambiguous, and we will not use it again."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"#### Gaussian Distributions\n",
|
||||
"\n",
|
||||
"So let us explore how gaussians work. A gaussian is a continuous probability distribution that is completely described with two parameters, the mean ($\\mu$) and the variance ($\\sigma^2$). It is defined as:\n",
|
||||
"$$ \n",
|
||||
"f(x, \\mu, \\sigma) = \\frac{1}{\\sigma\\sqrt{2\\pi}} e^{-0.5*{(x-\\mu)^2}/\\sigma^2 }\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You will not need to understand how this equation comes about, or remember it for this book, but it is useful to look at it to see how it works. Specifically, notice the term for $e$. When $x==\\mu$, the term reduces to $e^0=1$. Any other value of x will result in a smaller value for the exponent term due to the negative sign, so the curve will always be highest at $x==\\mu$.\n",
|
||||
"\n",
|
||||
"Now we will plot a gaussian centered around 23 ($\\mu=23$), with a variance of 1 ($\\sigma^2=1$). "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"collapsed": false,
|
||||
"input": [
|
||||
"xs = arange(16,30,0.1)\n",
|
||||
"ys = [gaussian (x,23,1) for x in xs]\n",
|
||||
"plt.plot (xs,ys, 'r')\n",
|
||||
"plt.axvline(23); plt.axvline(24) \n",
|
||||
"plt.show()"
|
||||
],
|
||||
"language": "python",
|
||||
"metadata": {},
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"As we expected the curve is centered around 23. The width of the curve is defined by the variance. If the variance is large than the curve will be wide, and if the variance is small the curve will be narrow.\n",
|
||||
"\n",
|
||||
"So what does this curve *mean*? Assume for a moment that we have a themometer that reads 23$\\,^{\\circ}$. We have asserted (without proof) that the distribution will be *normal*. If that is true, then this chart can be interpreted as a continuous curve depicting our belief that the temperature is any given temperature. In this curve, we assign a probability of the temperature being exactly 23$\\,^{\\circ}$ is 40%. Looking to the right, we assign the probability that the temperature is 24$\\,^{\\circ}$ is about 25%. We find 20$\\,^{\\circ}$ and 26$\\,^{\\circ}$ quite unlikely, and temperatures beyond that range extremely rare. \n",
|
||||
"\n",
|
||||
"Since this is a probability distribution it is required that the area under the curve always equals one. This should be intuitively clear - the area under the curve represents all possible occurances, which must sum to one.\n",
|
||||
"\n",
|
||||
"This leads to an important insight. If the variance is small the curve will be narrow. To keep the area == 1, the curve must also be tall. On the other hand if the variance is large the curve will be wide, and thus it will also have to be short to make the area == 1.\n",
|
||||
"\n",
|
||||
"Let us look at that:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"collapsed": false,
|
||||
"input": [
|
||||
"plt.plot (xs,[gaussian(x, 23, .2) for x in xs],'b')\n",
|
||||
"plt.plot (xs,[gaussian(x, 23, 1) for x in xs],'g')\n",
|
||||
"plt.plot (xs,[gaussian(x, 23, 5) for x in xs],'r')\n",
|
||||
"plt.show()"
|
||||
],
|
||||
"language": "python",
|
||||
"metadata": {},
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"So what is this telling us? The blue gaussian is very narrow. It is saying that we believe x=23, and that we are very sure about that. In contrast, the red gaussian also believes that x=23, but we are much less sure about that. Our believe that x=23 is lower, and so our belief about the likely possible values for x is spread out - we think it is quite likely that x=2 or x=8, for example. The blue gaussian has almost completely eliminated 22 or 24 as possible value - their probably is almost 0.0, whereas the red curve considers them nearly as likely as 23.\n",
|
||||
"\n",
|
||||
"If we think back to the thermometer, we can consider these three curves as representing 3 thermometers. The blue curve represents a very accurate thermometer, and the red one represents a fairly inaccurate one. Green of course represents one in between the two others. Note the very powerful property the Gaussian distribution affords us - we can entirely represent both the reading and the error of a thermometer with only two numbers - the mean and the variance.\n",
|
||||
"\n",
|
||||
"The standard notation for a normal distribution is just $N(\\mu,\\sigma^2)$. I will not go into detail as to why $\\sigma^2$ is used, other than to note that $\\sigma$ is commonly called the *standard deviation*, which has enormous utility in statistics. The standard deviation is not really used in this book, so I will not address it further. The important thing to understand is that the variance ($\\sigma^2$) is a measure of the width of the curve. The curve above is notated as $N(23, 1)$, since $\\mu=23$ and $\\sigma=1$. We will use this notiation throughout the rest of the book, so learn it now.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"#### Computational Properties of the Gaussian\n",
|
||||
"\n",
|
||||
"Recall how our histogram filter worked. We had a vector (Python array) representing our belief at a certain moment in time. When we performed another measurement using the *sense()* function we had to multiply probabilities together, and when we performed the motion step using the *update()* function we had to shift and add probabilities. I've promised you that the Kalman filter uses essentially the same process, and that it uses Gaussians instead of histograms, so you might reasonable expect that we will be multipling, adding, and shifting Gaussians in the Kalman filter.\n",
|
||||
"\n",
|
||||
"A typical math book would directly launch into a multipage proof of the behavior of Gaussians under these operations, but I don't see the value in that unless you plan to do statistics. I think the math will be much more intuitive and clear if we just start developing a Kalman filter using Gaussians, and I will provide the equations for multiplying and shifting Gaussians at the appropriate time. You will then be able to develop a physical intuition for what these operations do, rather than be forced to digest a lot of fairly abstract math.\n",
|
||||
"\n",
|
||||
"The key point, which I will only assert for now, is that all the operations are very simple, and that they preserve the properties of the Gaussian. This is somewhat remarkable, in that the Gaussian is a nonlinear function, and typically if you multiply a nonlinear equation with itself you end up with a different equation. For example, the shape of $sin(x)sin(x)$ is very different from $sin(x)$. But the result of multiplying two Gaussians is yet another Gaussian. This is a fundamental discovery, and the key reason why Kalman filters are possible\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"#### Summary and Key Points"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {}
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in new issue
Block a user