diff --git a/00_Preface.ipynb b/00_Preface.ipynb index fb90af0..a6b5f7e 100644 --- a/00_Preface.ipynb +++ b/00_Preface.ipynb @@ -315,7 +315,7 @@ "\n", "This book has supporting libraries for computing statistics, plotting various things related to filters, and for the various filters that we cover. This does require a strong caveat; most of the code is written for didactic purposes. It is rare that I chose the most efficient solution (which often obscures the intent of the code), and in the first parts of the book I did not concern myself with numerical stability. This is important to understand - Kalman filters in aircraft are carefully designed and implemented to be numerically stable; the naive implementation is not stable in many cases. If you are serious about Kalman filters this book will not be the last book you need. My intention is to introduce you to the concepts and mathematics, and to get you to the point where the textbooks are approachable.\n", "\n", - "Finally, this book is free. The cost for the books required to learn Kalman filtering is somewhat prohibitive even for a Silicon Valley engineer like myself; I cannot believe the are within the reach of someone in a depressed economy, or a financially struggling student. I have gained so much from free software like Python, and free books like those from Allen B. Downey [here](http://www.greenteapress.com/) [1]. It's time to repay that. So, the book is free, it is hosted on free servers, and it uses only free and open software such as IPython and MathJax to create the book. " + "Finally, this book is free. The cost for the books required to learn Kalman filtering is somewhat prohibitive even for a Silicon Valley engineer like myself; I cannot believe they are within the reach of someone in a depressed economy, or a financially struggling student. I have gained so much from free software like Python, and free books like those from Allen B. Downey [here](http://www.greenteapress.com/) [1]. It's time to repay that. So, the book is free, it is hosted on free servers, and it uses only free and open software such as IPython and MathJax to create the book. " ] }, { @@ -470,11 +470,11 @@ "\n", "I use a fair number of classes in FilterPy. I do not use inheritance or virtual functions or any of that sort of object oriented design. I use classes as a way to organize the data that the filters require. For example, the `KalmanFilter` class mentioned above stores matrices called `x`, `P`, `R`, `Q`, and more. I've seen procedural libraries for Kalman filters, and they require the programmer to maintain all of those matrices. This perhaps isn't so bad for a toy program, but start programming, say, a bank of Kalman filters and you will not enjoy having to manage all of those matrices and other associated data.\n", "\n", - "A word on variable names. I am an advocate for descriptive variable names. `R` is not, normally, descriptive. `R` is the measurement noise covariance matrix, so I could reasonably call it `measurement_noise_covariance`, and I've seen libraries do that. I've chosen not to. Why? In the end, Kalman filtering is math. To write a Kalman filter you are going to have to start by sitting down with a piece of paper and doing some math. You will be writing normal algebraic equations. Also, every Kalman filter text and source on the web uses the same linear algebra equations. You cannot read about the Kalman filter without seeing\n", + "A word on variable names. I am an advocate for descriptive variable names. In the Kalman filter literature the measurement noise covariance matrix is called `R`. `R` is not, normally, descriptive. I could reasonably call it `measurement_noise_covariance`, and I've seen libraries do that. I've chosen not to. Why? In the end, Kalman filtering is math. To write a Kalman filter you are going to have to start by sitting down with a piece of paper and doing some math. You will be writing normal algebraic equations. Also, every Kalman filter text and source on the web uses the same linear algebra equations. You cannot read about the Kalman filter without seeing\n", "\n", "$$\\dot{\\mathbf{x}} = \\mathbf{Fx} + \\mathbf{Gu}$$\n", "\n", - "in every source (a few sources use A and B instead of F and G). One of my goals in this book is to bring you to the point where you can read the original literature on Kalman filtering. I take an optimistic tone in this book - that Kalman filtering is easy to learn - and in many ways it is. However, for nontrivial problems the difficulty is not the implementation of the equations, but learning how to set up the equations so they solve your problem. In other words, every Kalman filter will implement $\\dot{\\mathbf{x}} = \\mathbf{Fx} + \\mathbf{Gu}$; the difficult part is figuring out what to put in the matrices $\\mathbf{F}$ and $\\mathbf{G}$ to make your filter work for your problem. Vast amounts of work have been done to figure out how to apply Kalman filters in various domains, and it would be tragic to not be able to read the literature and avail yourself of this research. \n", + "in every source. One of my goals in this book is to bring you to the point where you can read the original literature on Kalman filtering. I take an optimistic tone in this book - that Kalman filtering is easy to learn - and in many ways it is. However, for nontrivial problems the difficulty is not the implementation of the equations, but learning how to set up the equations so they solve your problem. In other words, every Kalman filter will implement $\\dot{\\mathbf{x}} = \\mathbf{Fx} + \\mathbf{Gu}$; the difficult part is figuring out what to put in the matrices $\\mathbf{F}$ and $\\mathbf{G}$ to make your filter work for your problem. Vast amounts of work have been done to figure out how to apply Kalman filters in various domains, and it would be tragic to not be able to read the literature and avail yourself of this research. \n", "\n", "So, like it or not you will need to learn that $\\mathbf{F}$ is the *state transition matrix* and that $\\mathbf{R}$ is the *measurement noise covariance*. Once you know that the code will become readable, and until you know that all publications and web articles on Kalman filters will be inaccessible to you. \n", "\n", @@ -502,7 +502,7 @@ " inv (dot(measurement_function, apriori_state_covariance).dot(\n", " measurement_function_transpose) + measurement_noise_covariance)))\n", "\n", - "I grant you this version has more context, but I cannot reasonable glance at this and see what math it is implementing. In particular, the linear algebra $\\mathbf{HPH}^\\mathsf{T}$ is doing something very specific - multiplying P by H and its transpose is changing the *basis* of P. It is nearly impossible to see that the Kalman gain is just a ratio of one number divided by a second number which has been converted to a different basis. If you are not solid in linear algebra perhaps that statement does not convey a lot of information to you yet, but I assure you that $\\mathbf{K} = \\mathbf{PH}^\\mathsf{T}[\\mathbf{HPH}^\\mathsf{T} + \\mathbf{R}]^{-1}$ is saying something very succinctly. There are two key pieces of information here - we are taking a ratio, and we are converting the *basis* of a matrix. I can see that in my first Python line, I cannot see that in the second line. \n", + "I grant you this version has more context, but I cannot reasonable glance at this and see what math it is implementing. In particular, the linear algebra $\\mathbf{HPH}^\\mathsf{T}$ is doing something very specific - multiplying $\\mathbf{P}$ by $\\mathbf{H}$ and its transpose is changing $\\mathbf{P}$ from world space to measurement space (we will learn what that means - it is important!) It is nearly impossible to see that the Kalman gain is just a ratio of one number divided by a second number which has been converted to a different basis. If you are not solid in linear algebra perhaps that statement does not convey a lot of information to you yet, but I assure you that $\\mathbf{K} = \\mathbf{PH}^\\mathsf{T}[\\mathbf{HPH}^\\mathsf{T} + \\mathbf{R}]^{-1}$ is saying something very succinctly. There are two key pieces of information here - we are taking a ratio, and we are converting the *basis* of a matrix. I can see that in my first Python line, I cannot see that in the second line. \n", "\n", "I will not *win* this argument, and some people will not agree with my naming choices. I will finish by stating, very truthfully, that I made two mistakes the first time I typed that second version and it took me awhile to find it. In any case, I aim for using the mathematical symbol names whenever possible, coupled with readable class and function names. So, it is `KalmanFilter.P`, not `KF.P` and not `KalmanFilter.apriori_state_covariance`. " ] diff --git a/02_Discrete_Bayes.ipynb b/02_Discrete_Bayes.ipynb index f329ce3..c1887fd 100644 --- a/02_Discrete_Bayes.ipynb +++ b/02_Discrete_Bayes.ipynb @@ -519,30 +519,22 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 1, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "sum = 1.0\n", - "probability of door = 0.1875\n", - "probability of wall = 0.0625\n" + "ename": "NameError", + "evalue": "name 'np' is not defined", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mNameError\u001b[0m Traceback (most recent call last)", + "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[0;32m 10\u001b[0m \u001b[0mnormalize\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mbelief\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 11\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 12\u001b[1;33m \u001b[0mpos_belief\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0marray\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;36m0.2\u001b[0m\u001b[1;33m]\u001b[0m \u001b[1;33m*\u001b[0m \u001b[1;36m10\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 13\u001b[0m \u001b[0mupdate\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mhallway\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mpos_belief\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mprob_correct\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;36m.75\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 14\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;31mNameError\u001b[0m: name 'np' is not defined" ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtcAAADaCAYAAABtj26qAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAG2BJREFUeJzt3X9U1vX9//EHP1IvijgZgSBMoBymEilXLJDSs2NsljO3\nknSbJdocrExl7jSKz6kUtXKHTU2oVQdZzcKdznGdZB7wqCGDziCkmamjsZkduK6mMzQXeoT394++\nXqerC/mhL3lfyP12judcvN6vF+/n9Tx6rgdvX7zfAZZlWQIAAABwyQLtLgAAAAC4UhCuAQAAAEMI\n1wAAAIAhhGsAAADAEMI1AAAAYAjhGgAAADCEcA0AAAAY0mu4rq6u1qxZsxQTE6PAwECVlZX1+k33\n79+vqVOnKiQkRDExMVq1apWRYgEAAAB/1mu4Pn36tG655RatX79eDodDAQEBPc4/efKk7rrrLkVF\nRamhoUHr16/XunXrVFRUZKxoAAAAwB8F9OcJjaGhodq0aZMefPDBC84pKSlRfn6+3G63hg8fLkla\nvXq1SkpK9Omnn156xQAAAICfMr7nuq6uTnfccYcnWEtSZmamWltbdeTIEdOnAwAAAPyG8XDtcrkU\nGRnpNXb+a5fLZfp0AAAAgN8INv0Ne9uT/XXt7e2mTw8AAAAMmLCwMK+vjV+5HjVqlM8Varfb7TkG\nAAAAXKmMh+u0tDTt3btXZ86c8YxVVVVp9OjRGjNmjOnTAQAAAH6j120hp0+fVnNzsySpq6tLR44c\nUVNTk66//nrFxsYqPz9f9fX12rlzpyTpxz/+sZ555hktWLBABQUFOnz4sJ577jk9/fTTPZ7nm5fU\nTWtoaJAkOZ3Oy3qewYSe+KInvuiJL3rii574oie+6IkveuJtsPSjp63NvV65rq+v1+TJkzV58mR1\ndHToqaee0uTJk/XUU09J+uqXFFtaWjzzr732WlVVVam1tVVOp1NLlizRihUrtHz5cgNvBQAAAPBf\nvV65njZtmrq6ui54vLS01Gds4sSJevfddy+tMgAAAGCQMb7nGgAAABiqCNcAAACAIYRrAAAAwBDC\nNQAAAGAI4RoAAAAwhHANAAAAGEK4BgAAAAwhXAMAAACGEK4BAAAAQwjXAAAAgCGEawAAAMAQwjUA\nAABgCOEaAAAAMIRwDQAAABhCuAYAAAAMIVwDAAAAhhCuAQAAAEMI1wAAAIAhhGsAAADAEMI1AAAA\nYAjhGgAAADCEcA0AAAAYQrgGAAAADCFcAwAAAIYQrgEAAABD+hSui4uLFR8fL4fDIafTqZqamh7n\nV1RU6Pbbb9e1116rG264QbNnz1Zzc7ORggEAAAB/1Wu4Li8v17Jly1RQUKCmpialp6drxowZOnr0\naLfzP/74Y82ePVvTpk1TU1OTdu7cqY6ODt19993GiwcAAAD8Sa/huqioSNnZ2Vq0aJESExO1YcMG\nRUVFqaSkpNv5TU1N6urq0tq1a5WQkKDk5GQ9/vjj+uc//6n//ve/xt8AAAAA4C96DNdnz55VY2Oj\nMjMzvcYzMzNVW1vb7ZopU6bommuu0csvv6zOzk6dOnVKmzdvVmpqqkaOHGmucgAAAMDPBFiWZV3o\nYGtrq2JiYlRdXa2MjAzP+MqVK7VlyxYdOnSo23W1tbWaPXu2Tpw4oa6uLk2aNEl/+ctfdMMNN3jN\na29v97xmTzYAAAAGg7Fjx3peh4WFeR0zfreQlpYWzZ49W9nZ2WpoaNCePXsUGhqqrKws9ZDjAQAA\ngEEvuKeD4eHhCgoKktvt9hp3u92Kiorqds1LL72k2NhYPffcc56x119/XbGxsaqrq1N6enq365xO\nZ39r75eGhoYBOc9gQk980RNf9MQXPfFFT3zRE1/0xBc98TZY+vH13Rff1OOV62HDhiklJUWVlZVe\n41VVVRcMyZZlKTDQ+9ue/7qrq6tPBQMAAACDUa/bQvLy8rR582a9+uqrOnjwoJYuXSqXy6WcnBxJ\nUn5+vqZPn+6ZP2vWLDU2NmrVqlVqbm5WY2OjsrOz9a1vfUspKSmX750AAAAANutxW4gkZWVl6fjx\n4yosLFRbW5uSkpJUUVGh2NhYSZLL5VJLS4tnfkZGhsrLy/Xss8/q+eefV0hIiNLS0rRjxw45HI7L\n904AAAAAm/UariUpNzdXubm53R4rLS31Gbv//vt1//33X1plAAAAwCBj/G4hAAAAwFBFuAYAAAAM\nIVwDAAAAhhCuAQAAAEMI1wAAAIAhhGsAAADAEMI1AAAAYAjhGgAAADCEcA0AAAAYQrgGAAAADCFc\nAwAAAIYQrgEAAABDCNcAAACAIYRrAAAAwBDCNQAAAGAI4RoAAAAwhHANAAAAGEK4BgAAAAwhXAMA\nAACGEK4BAAAAQwjXAAAAgCGEawAAAMAQwjUAAABgCOEaAAAAMIRwDQAAABjSp3BdXFys+Ph4ORwO\nOZ1O1dTU9Lrmd7/7ncaNG6cRI0YoOjpa+fn5l1wsAAAA4M+Ce5tQXl6uZcuWqaSkRBkZGdq0aZNm\nzJihjz76SLGxsd2uycvL0/bt2/Wb3/xGSUlJam9vV1tbm/HiAQAAAH/Sa7guKipSdna2Fi1aJEna\nsGGDduzYoZKSEq1Zs8Zn/uHDh/XCCy9o//79SkxM9IwnJycbLBsAAADwPz1uCzl79qwaGxuVmZnp\nNZ6Zmana2tpu1/z5z39WQkKCKioqlJCQoPj4eC1YsED/+c9/zFUNAAAA+KEAy7KsCx1sbW1VTEyM\nqqurlZGR4RlfuXKltmzZokOHDvmsycnJUVlZmW699VatW7dOkrRixQpJUl1dnQICAjxz29vbPa+b\nm5sv/d0AAAAAl9nYsWM9r8PCwryO9botpL+6urp05swZvfbaa7rpppskSa+99poSExPV0NCg2267\nzfQpAQAAAL/QY7gODw9XUFCQ3G6317jb7VZUVFS3a6KiohQcHOwJ1pJ00003KSgoSJ988skFw7XT\n6exv7f3S0NAwIOcZTOiJL3rii574oie+6IkveuKLnviiJ94GSz++vvvim3rccz1s2DClpKSosrLS\na7yqqkrp6endrsnIyNC5c+fU0tLiGWtpaVFnZ6fGjBnTn7oBAACAQaXX+1zn5eVp8+bNevXVV3Xw\n4EEtXbpULpdLOTk5kqT8/HxNnz7dM3/69OmaPHmyFi5cqKamJu3bt08LFy7U7bff7vc/hQAAAACX\notc911lZWTp+/LgKCwvV1tampKQkVVRUeO5x7XK5vK5SBwQE6J133tFjjz2mO++8Uw6HQ5mZmSoq\nKrp87wIAAADwA336hcbc3Fzl5uZ2e6y0tNRnbNSoUdq6deulVQYAAAAMMn16/DkAAACA3hGuAQAA\nAEMI1wAAAIAhhGsAAADAEMI1AAAAYAjhGgAAADCEcA0AAAAYQrgGAAAADCFcAwAAAIYQrgEAAABD\nCNcAAACAIYRrAAAAwBDCNQAAAGAI4RoAAAAwhHANAAAAGEK4BgAAAAwhXAMAAACGEK4BAAAAQwjX\nAAAAgCGEawAAAMAQwjUAAABgCOEaAAAAMIRwDQAAABhCuAYAAAAMIVwDAAAAhvQpXBcXFys+Pl4O\nh0NOp1M1NTV9+ubNzc0KDQ1VaGjoJRUJAAAADAa9huvy8nItW7ZMBQUFampqUnp6umbMmKGjR4/2\nuO7s2bOaO3eupk6dqoCAAGMFAwAAAP6q13BdVFSk7OxsLVq0SImJidqwYYOioqJUUlLS47rHH39c\nt956q+bMmSPLsowVDAAAAPirHsP12bNn1djYqMzMTK/xzMxM1dbWXnDd9u3btX37dm3cuJFgDQAA\ngCEjuKeDx44dU2dnpyIjI73GIyIi5HK5ul3T2tqqxYsXa9u2bQoJCelzIQ0NDX2eeykG6jyDCT3x\nRU980RNf9MQXPfFFT3zRE1/0xJu/92Ps2LEXPGb8biHz589Xbm6ubrvtNtPfGgAAAPBrPV65Dg8P\nV1BQkNxut9e42+1WVFRUt2t2796t6upqPfPMM5Iky7LU1dWlq666SiUlJXr44Ye7Xed0Oi+m/j47\n/xPQ5T7PYEJPfNETX/TEFz3xRU980RNf9MQXPfE2WPrR3t5+wWM9huthw4YpJSVFlZWVuu+++zzj\nVVVVmjNnTrdrPvzwQ6+vt23bptWrV6u+vl7R0dH9qRsAAAAYVHoM15KUl5en+fPnKzU1Venp6Xrx\nxRflcrmUk5MjScrPz1d9fb127twpSRo/frzX+r/97W8KDAz0GQcAAACuNL2G66ysLB0/flyFhYVq\na2tTUlKSKioqFBsbK0lyuVxqaWnp8Xtwn2sAAAAMBb2Ga0nKzc1Vbm5ut8dKS0t7XLtgwQItWLCg\n34UBAAAAg43xu4UAAAAAQxXhGgAAADCEcA0AAAAYQrgGAAAADCFcAwAAAIYQrgEAAABDCNcAAACA\nIYRrAAAAwBDCNQAAAGAI4RoAAAAwhHANAAAAGEK4BgAAAAwhXAMAAACGEK4BAAAAQwjXAAAAgCGE\nawAAAMAQwjUAAABgCOEaAAAAMIRwDQAAABhCuAYAAAAMIVwDAAAAhhCuAQAAAEMI1wAAAIAhhGsA\nAADAEMI1AAAAYEifw3VxcbHi4+PlcDjkdDpVU1Nzwbl79uzRvffeq+joaF199dVKTk5WaWmpkYIB\nAAAAf9WncF1eXq5ly5apoKBATU1NSk9P14wZM3T06NFu59fV1Sk5OVlvvfWWDhw4oNzcXC1evFhv\nvPGG0eIBAAAAfxLcl0lFRUXKzs7WokWLJEkbNmzQjh07VFJSojVr1vjMz8/P9/o6JydHu3fv1ltv\nvaV58+YZKBsAAADwP71euT579qwaGxuVmZnpNZ6Zmana2to+n6i9vV0jR47sf4UAAADAIBFgWZbV\n04TW1lbFxMSourpaGRkZnvGVK1dqy5YtOnToUK8neeedd/SjH/1ItbW1cjqdnvH29nbP6+bm5oup\nHwAAABhQY8eO9bwOCwvzOnbZ7xby17/+VT/5yU+0ceNGr2ANAAAAXGl63XMdHh6uoKAgud1ur3G3\n262oqKge19bU1Oiee+7RqlWr9POf/7zHuZc7eDc0NAzIeQYTeuKLnviiJ77oiS964oue+KInvuiJ\nt8HSj6/vvvimXsP1sGHDlJKSosrKSt13332e8aqqKs2ZM+eC66qrqzVz5kytXLlSjz32WK9F1hw5\n0eucS3FqRMSAnOfroq8OVkJ4aL/XtRw7pdbT5y5DRd4GU08AABgoA/U5LA38ZzGfw5dfn+4WkpeX\np/nz5ys1NVXp6el68cUX5XK5lJOTI+mru4PU19dr586dkr66z/U999yjRx99VPPmzZPL5ZIkBQUF\n6YYbbuj2HEuqj5l4P31wZoDOI228M1wJ4f1f13r63AD2QxoMPQEAYKAM/OewNFCfxXwOX359CtdZ\nWVk6fvy4CgsL1dbWpqSkJFVUVCg2NlaS5HK51NLS4plfVlamjo4OrVu3TuvWrfOMx8XFec0DAAAA\nriR9CteSlJubq9zc3G6PffPpi6WlpTyREQAAAEPOZb9bCAAAADBUEK4BAAAAQwjXAAAAgCGEawAA\nAMAQwjUAAABgCOEaAAAAMIRwDQAAABhCuAYAAAAMIVwDAAAAhhCuAQAAAEMI1wAAAIAhhGsAAADA\nEMI1AAAAYAjhGgAAADCEcA0AAAAYQrgGAAAADCFcAwAAAIYQrgEAAABDCNcAAACAIcF2FwD0Rcux\nU2o9fe6yn+fUiAhJUs2RE5f9XOdFXx2shPDQfq0ZqH5I9KQ7A92Ti+mHRE+6Q0980RP0BZ/DfUe4\nxqDQevqcllQfG8AznhmwM228M1wJ4f1bM/D9kOhJdwamJxfTD4medIee+KIn6As+h/uObSEAAACA\nIYRrAAAAwBDCNQAAAGBIn8J1cXGx4uPj5XA45HQ6VVNT0+P8/fv3a+rUqQoJCVFMTIxWrVplpFgA\nAADAn/UarsvLy7Vs2TIVFBSoqalJ6enpmjFjho4ePdrt/JMnT+quu+5SVFSUGhoatH79eq1bt05F\nRUXGiwcAAAD8Sa/huqioSNnZ2Vq0aJESExO1YcMGRUVFqaSkpNv5f/zjH9XR0aGysjKNHz9e9913\nnx5//HHCNQAAAK54PYbrs2fPqrGxUZmZmV7jmZmZqq2t7XZNXV2d7rjjDg0fPtxrfmtrq44cOWKg\nZAAAAMA/9Riujx07ps7OTkVGRnqNR0REyOVydbvG5XL5zD//9YXWAAAAAFeCAMuyrAsdbG1tVUxM\njKqrq5WRkeEZX7lypbZs2aJDhw75rPne976n2NhYvfLKK56xTz75RHFxcaqrq9N3vvMdz3h7e7up\n9wEAAAAMuLCwMK+ve7xyHR4erqCgILndbq9xt9utqKiobteMGjXK5wr1+fWjRo3qd8EAAADAYNFj\nuB42bJhSUlJUWVnpNV5VVaX09PRu16SlpWnv3r06c+aM1/zRo0drzJgxBkoGAAAA/FOP20IkaevW\nrZo/f76Ki4uVnp6uF198UaWlpTpw4IBiY2OVn5+v+vp67dy5U9JXt+JLTEzUtGnTVFBQoMOHDys7\nO1tPP/20li9fPiBvCgAAALBDcG8TsrKydPz4cRUWFqqtrU1JSUmqqKhQbGyspK9+SbGlpcUz/9pr\nr1VVVZUeeeQROZ1OjRw5UitWrCBYAwAA4IrX65VrAAAAAH3Tp8efXwn6+wj3K1l1dbVmzZqlmJgY\nBQYGqqyszO6SbLd27VrddtttCgsLU0REhGbNmqUDBw7YXZatNm3apOTkZIWFhSksLEzp6emqqKiw\nuyy/sXbtWgUGBmrJkiV2l2Krp59+WoGBgV5/oqOj7S7LVm1tbXrooYcUEREhh8OhCRMmqLq62u6y\nbBMXF+fzdyQwMFAzZ860uzTbnDt3Tk888YQSEhLkcDiUkJCg//u//1NnZ6fdpdnq1KlTWrZsmeLi\n4hQSEqIpU6aooaHB7rL6bUiE6/4+wv1Kd/r0ad1yyy1av369HA6HAgIC7C7Jdu+++64effRR1dXV\nadeuXQoODtb06dN14sQJu0uzTWxsrJ5//nnt27dP77//vr773e9q9uzZ+uCDD+wuzXbvvfeeXn75\nZd1yyy38+5E0btw4uVwuz5/9+/fbXZJtPv/8c02ZMkUBAQGqqKjQoUOH9MILLygiIsLu0mzz/vvv\ne/39aGxsVEBAgB544AG7S7PNmjVr9NJLL2njxo06fPiw1q9fr+LiYq1du9bu0mz18MMPq6qqSn/4\nwx/04YcfKjMzU9OnT1dra6vdpfWPNQSkpqZaixcv9hobO3aslZ+fb1NF/uOaa66xysrK7C7D73zx\nxRdWUFCQ9c4779hdil8ZOXKk9fvf/97uMmz1+eefWzfeeKO1Z88ea9q0adaSJUvsLslWTz31lDVx\n4kS7y/Ab+fn5VkZGht1l+LXCwkLruuuuszo6OuwuxTYzZ860FixY4DX24IMPWj/4wQ9sqsh+//vf\n/6zg4GDr7bff9hpPSUmxCgoKbKrq4lzxV64v5hHuwMmTJ9XV1aXrrrvO7lL8Qmdnp9588011dHTo\nzjvvtLscWy1evFhz5szR1KlTZfErK5KklpYWjR49WgkJCZo3b57+9a9/2V2SbbZt26bU1FQ98MAD\nioyM1KRJk7Rp0ya7y/IblmXp1Vdf1U9/+lMNHz7c7nJsM2PGDO3atUuHDx+WJH300UfavXu37r77\nbpsrs8+5c+fU2dnp8/dixIgRg24rb693CxnsLuYR7sDSpUs1adIkpaWl2V2Krfbv36+0tDSdOXNG\nDodDW7duVWJiot1l2ebll19WS0uLtmzZIklsCZF0++23q6ysTOPGjZPb7VZhYaHS09N14MABjRw5\n0u7yBlxLS4uKi4uVl5enJ554Qvv27fPsy3/kkUdsrs5+VVVV+ve//62f/exndpdiq1/84hf69NNP\ndfPNNys4OFjnzp1TQUGBcnJy7C7NNqGhoUpLS1NhYaEmTpyoyMhIvfHGG3rvvfc0duxYu8vrlys+\nXAP9lZeXp9raWtXU1Az58DRu3Dj9/e9/V3t7u/70pz9p7ty52r17t5xOp92lDbjDhw/rySefVE1N\njYKCgiR9dRVuqF+9/v73v+95PXHiRKWlpSk+Pl5lZWVD8hasXV1dSk1N1erVqyVJycnJam5u1qZN\nmwjX+uoH1NTUVCUlJdldiq02bNig0tJSvfnmm5owYYL27dunpUuXKi4uTgsXLrS7PNu89tprWrhw\noWJiYhQUFKSUlBTNmzdP77//vt2l9csVH64v5hHuGLqWL1+urVu3avfu3YqLi7O7HNtdddVVSkhI\nkCRNmjRJ9fX12rRpk0pLS22ubODV1dXp2LFjmjBhgmess7NTe/fu1UsvvaTTp0/rqquusrFC/xAS\nEqIJEybo448/trsUW0RHR2v8+PFeY+PGjdMnn3xiU0X+47PPPtPbb7+t4uJiu0ux3erVq1VQUKCs\nrCxJ0oQJE3TkyBGtXbt2SIfrhIQE7dmzR19++aVOnjypyMhIPfDAA7rxxhvtLq1frvg91xfzCHcM\nTUuXLlV5ebl27dqlb3/723aX45c6OzvV1dVldxm2+OEPf6gPP/xQH3zwgT744AM1NTXJ6XRq3rx5\nampqIlj/fx0dHTp48OCQvXgxZcoUHTp0yGvsH//4Bz+sS9q8ebNGjBihefPm2V2K7SzLUmCgdwQL\nDAwc8v8Tdp7D4VBkZKROnDihyspK3XvvvXaX1C9X/JVr6av/5p8/f75SU1M9j3B3uVxDdm/T6dOn\n1dzcLOmr/8I8cuSImpqadP3113uevDnUPPLII3r99de1bds2hYWFefbjh4aG6uqrr7a5Onv8+te/\n1syZMxUTE6NTp05py5Ytevfdd7Vjxw67S7PF+ft9f11ISIiuu+46nyuVQ8mKFSs0a9YsxcbG6rPP\nPtOqVav05Zdf6qGHHrK7NFssX75c6enpWrNmjbKysrRv3z5t3LhxyN9izbIsvfLKK5o7d65CQkLs\nLsd2s2fP1rPPPqv4+HiNHz9e+/bt029/+9sh++/mvMrKSnV2dmrcuHH6+OOP9atf/Uo333yzsrOz\n7S6tf2y8U8mAKi4utuLi4qzhw4dbTqfT2rt3r90l2Wb37t1WQECAFRAQYAUGBnpeZ2dn212abb7Z\ni/N/nnnmGbtLs82CBQusMWPGWMOHD7ciIiKsu+66y6qsrLS7LL/Crfgsa+7cuVZ0dLQ1bNgwa/To\n0db9999vHTx40O6ybLV9+3YrOTnZGjFihJWYmGht3LjR7pJst2vXLiswMNCqr6+3uxS/8MUXX1i/\n/OUvrbi4OMvhcFgJCQnWk08+aZ05c8bu0my1detW68Ybb7SGDx9uRUVFWUuWLLFOnjxpd1n9xuPP\nAQAAAEOu+D3XAAAAwEAhXAMAAACGEK4BAAAAQwjXAAAAgCGEawAAAMAQwjUAAABgCOEaAAAAMIRw\nDQAAABhCuAYAAAAM+X9QtfQM34Hr6AAAAABJRU5ErkJggg==\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" } ], "source": [ @@ -1665,7 +1657,8 @@ "\n", "$$P(x_i|Z) = \\frac{P(Z|x_i) P(x_i)}{P(Z)}$$\n", "\n", - "That looks ugly, but it is actually quite simple. Let's just figure out what each term on the right means. First is $P(Z|x_i)$. This is the probability for the measurement at every cell $x_i$. $P(Z)$ is just the probability of the measurement. We multiply those together. This is just the unnormalized multiplication in the `update()` function. \n", + "That looks ugly, but it is actually quite simple. Let's just figure out what each term on the right means. First is $P(Z|x_i)$. This is the probability for the measurement at every cell $x_i$. $P(x_i)$ is the *prior* - our belief before incorporating the measurements. We multiply those together. This is just the unnormalized multiplication in the `update()` function, where `belief` is our prior $P(x_i)$.\n", + " \n", "\n", " for i, val in enumerate(map_):\n", " if val == z:\n", @@ -1673,7 +1666,12 @@ " else:\n", " belief[i] *= 1.\n", "\n", - "I added the `else` here, which has no mathematical effect, to point out that every element in $x$ (called `belief` in the code) is multiplied by a probability. \n", + "I added the `else` here, which has no mathematical effect, to point out that every element in $x$ (called `belief` in the code) is multiplied by a probability. You may object that I am multiplying by a scale factor, which I am, but this scale factor is derived from the probability of the measurement being correct vs the probability being incorrect. An alternative and equivalent implementation would be\n", + "\n", + " if val == z:\n", + " belief[i] *= prob_correct\n", + " else:\n", + " belief[i] *= prob_incorrect\n", "\n", "The last term to consider is the denominator $P(Z)$. This is the probability of getting the measurement $Z$ without taking the location into account. We compute that by taking the sum of $x$, or `sum(belief)` in the code. That is how we compute the normalization! So, the `update()` function is doing nothing more than computing Bayes theorem. I could have just given you Bayes theorem and then written a function, but I doubt that would have been illuminating unless you already know Bayesian statistics. Instead, we figured out what to do just by reasoning about the situation, and so of course the resulting code ended up implementing Bayes theorem. Students spend a lot of time struggling to understand this theorem; I hope you found it relatively straightforward." ] diff --git a/04_Gaussians.ipynb b/04_Gaussians.ipynb index 53f0a36..6f8f857 100644 --- a/04_Gaussians.ipynb +++ b/04_Gaussians.ipynb @@ -776,7 +776,9 @@ "source": [ "Probably this is immediately recognizable to you as a 'bell curve'. This curve is ubiquitous because under real world conditions most observations are distributed in such a manner. In fact, this is the bell curve the student heights given earlier. \n", "\n", - "This curve is not unique to heights - a vast amount of natural phenomena exhibits this sort of distribution, including the sensors that we use in filtering problems. As we will see, it also has all the attributes that we are looking for - it represents a unimodal belief or value as a probability, it is continuous, and it is computationally efficient. We will soon discover that it also other desirable qualities that we do not yet recognize we need." + "This curve is not unique to heights - a vast amount of natural phenomena exhibits this sort of distribution, including the sensors that we use in filtering problems. As we will see, it also has all the attributes that we are looking for - it represents a unimodal belief or value as a probability, it is continuous, and it is computationally efficient. We will soon discover that it also other desirable qualities that we do not yet recognize we need.\n", + "\n", + "To further motivate you, recall the shapes of the probability distributions in the *Discrete Bayes* chapter. They were not perfect bell curves, but they were shaped somewhat like this - a tall center bar surrounded with shorter bars which get shorter the further away they are from the tall bar. We will be using Gaussians to replace the discrete probabilities used in that chapter!" ] }, {