Added discrete Wiener process Acceleration model.

Got rid of the hand-wavey 'Q is hard' text and put in a discrete
Wiener process Acceleration model for Q.

Chapter07_Kalman_Filter_Math/Kalman_Filter_Math.ipynb

@@ -1,7 +1,7 @@
 {
  "metadata": {
   "name": "",
-  "signature": "sha256:4f0493fb7c7a8813b0cda0b31e92c7dc575f7d5eb07061f42ba36214d17920ec"
+  "signature": "sha256:5e618e97b591e89d2e34bd50626e7a9afaab2de1c0ab60da49a9119975fdd7aa"
  },
  "nbformat": 3,
  "nbformat_minor": 0,
@@ -466,13 +466,26 @@
      ]
     },
     {
-     "cell_type": "code",
-     "collapsed": false,
-     "input": [],
-     "language": "python",
+     "cell_type": "markdown",
      "metadata": {},
-     "outputs": [],
-     "prompt_number": 2
+     "source": [
+      "*equations of continuous Wiener process acceleration model - just a cut paste being saved for later use when I write this*\n",
+      "\n",
+      " $$\\begin{bmatrix}\n",
+      " \\frac{1}{3}{\\Delta t}^3 & \\frac{1}{2}{\\Delta t}^2 \\\\\n",
+      " \\frac{1}{2}{\\Delta t}^2 & \\Delta t\n",
+      " \\end{bmatrix}\\tilde{q}\n",
+      " $$\n",
+      "\n",
+      "\n",
+      " $$\\begin{bmatrix}\n",
+      " \\frac{1}{20}{\\Delta t}^5 & \\frac{1}{8}{\\Delta t}^4 & \\frac{1}{6}{\\Delta t}^3 \\\\\n",
+      " \\frac{1}{8}{\\Delta t}^8 & \\frac{1}{3}{\\Delta t}^3 & \\frac{1}{2}{\\Delta t}^2  \\\\\n",
+      " \\frac{1}{6}{\\Delta t}^3 & \\frac{1}{2}{\\Delta t}^2 & \\Delta t\n",
+      " \\end{bmatrix} \\tilde{q}\n",
+      " $$\n",
+      "    "
+     ]
     }
    ],
    "metadata": {}

Chapter09_Extended_Kalman_Filters/Extended_Kalman_Filters.ipynb

@@ -1,7 +1,7 @@
 {
  "metadata": {
   "name": "",
-  "signature": "sha256:8639f3d09166c8c1ae2b55e044e278c8b7e31e19bd72545341522e3c86eb9f7b"
+  "signature": "sha256:19938ddfba204e20c435b33acc3d723467354126675c5aa78dc8200036994fad"
  },
  "nbformat": 3,
  "nbformat_minor": 0,
@@ -269,6 +269,8 @@
      "cell_type": "markdown",
      "metadata": {},
      "source": [
+      "**Author's note: this is still being heavily edited - there is a lot of duplicate material, incomplete material, and so on**\n",
+      "\n",
       "The Kalman filter that we have developed to this point is extremely good, but it is also limited. Its derivation is in the linear space, and hence it only works for linear problems. Let's be a bit more rigorous here. You can, and we have in this book, apply the Kalman filter to nonlinear problems. For example, in the g-h filter chapter we explored using a g-h filter in a problem with constant acceleration. It 'worked', in that it remained numerically stable and the filtered output did track the input, but there was always a lag. It is easy to prove that there will always be a lag when $\\mathbf{\\ddot{x}}>0$. The filter no longer produces an optimal result. If we make our time step arbitrarily small we can still handle many problems, but typically we are using Kalman filters with physical sensors and solving real-time problems. Either fast enough sensors do not exist, are prohibitively expensive, or the computation time required is excessive. It is not a workable solution.\n",
       "\n",
       "The early adopters of Kalman filters were the radar people, and this fact was not lost on them. Radar is inherently nonlinear. Radars measure the slant range to an object, and we are typically interested in the aircraft's position over the ground. We invoke Pythagoras and get the nonlinear equation:\n",
@@ -613,7 +615,7 @@
      "cell_type": "markdown",
      "metadata": {},
      "source": [
-      "We want a linear appoximation of this function so that we can use it in the Kalman filter. We will see how it is used in the Kalman filter in the next section, so don't worry about that yet. We can see that there is no single linear function (line) that gives a close approximation of this function. However, during each innovation of the Kalman filter we know it's current state, so if we linearize the function at that value we will have a close approximation. For example, suppose our current state is $x=1.5$. What would be a good linearization for this function?\n",
+      "We want a linear appoximation of this function so that we can use it in the Kalman filter. We will see how it is used in the Kalman filter in the next section, so don't worry about that yet. We can see that there is no single linear function (line) that gives a close approximation of this function. However, during each innovation (update) of the Kalman filter we know its current state, so if we linearize the function at that value we will have a close approximation. For example, suppose our current state is $x=1.5$. What would be a good linearization for this function?\n",
       "\n",
       "We can use any linear function that passes through the curve at (1.5,-0.75). For example, consider using f(x)=8x\u221212.75 as the linearization, as in the plot below."
      ]
@@ -691,7 +693,7 @@
      "source": [
       "Here we can see that this linearization is much better. It is still exactly correct at $x=1.5$, but the errors are very small as x varies. Compare the tiny error at $x=1.4$ vs the very large error at $x=1.4$ in the previous plot. This does not constitute a formal proof of correctness, but this sort of geometric depiction should be fairly convincing. Certainly it is easy to see that in this case if the line had any other slope the errors would accumulate more quickly. \n",
       "\n",
-      "To implement the extended Kalman filter we will leave the linear equations as they are, and use partial derivatives to evaluate the system matrix $\\mathbf{F}$ and the measurement matrix $\\mathbf{H}$ at the state at time t ($\\mathbf{x}_t$). Since $\\mathbf{F}$ also depends on the control input vector $\\mathbf{u}$m we will need to include that term:\n",
+      "To implement the extended Kalman filter we will leave the linear equations as they are, and use partial derivatives to evaluate the system matrix $\\mathbf{F}$ and the measurement matrix $\\mathbf{H}$ at the state at time t ($\\mathbf{x}_t$). Since $\\mathbf{F}$ also depends on the control input vector $\\mathbf{u}$ we will need to include that term:\n",
       "\n",
       "$$\n",
       "\\begin{aligned}\n",
@@ -795,7 +797,7 @@
      "cell_type": "markdown",
      "metadata": {},
      "source": [
-      "As we can see there are two minor changes to the Kalman filter equations. The first change replaces the equation $\\mathbf{x} = \\mathbf{Fx}$ with $\\mathbf{x} = f(\\mathbf{x})$. In the Kalman filter, $\\mathbf{Fx}$ is how we compute the new state based on the old state. However, in a nonlinear system we cannot use linear algebra to compute this transition. So instead we hypothesize a nonlinear function $f()$ which performs this function. Likewise, in the Kalman filter we convert the state to a measurement with the linear function $\\mathbf{Hx}$. For the extended Kalman filter we replace this with a nonlinear function $h()$, giving $\\mathbf{z}_x = h(\\mathbf{x})$.\n",
+      "As we can see there are two minor changes to the Kalman filter equations, which I have underlined. The first change replaces the equation $\\mathbf{x} = \\mathbf{Fx}$ with $\\mathbf{x} = f(\\mathbf{x})$. In the Kalman filter, $\\mathbf{Fx}$ is how we compute the new state based on the old state. However, in a nonlinear system we cannot use linear algebra to compute this transition. So instead we hypothesize a nonlinear function $f()$ which performs this function. Likewise, in the Kalman filter we convert the state to a measurement with the linear function $\\mathbf{Hx}$. For the extended Kalman filter we replace this with a nonlinear function $h()$, giving $\\mathbf{z}_x = h(\\mathbf{x})$.\n",
       "\n",
       "The only question left is how do we implement use $f()$ and $h()$ in the Kalman filter if they are nonlinear? We reach for the single tool that we have available for solving nonlinear equations - we linearize them at the point we want to evaluate the system.  For example, consider the function $f(x) = x^2 -2x$\n",
       "\n",