daviddoji/Kalman-and-Bayesian-Filters-in-Python
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Fixed heading numbers.

Browse Source 
Some sections jumped from level 2 to level 4, for example.

Also played around with generating book to HTML. I like it, but
it needs some work. The Calico table of content generator cannot
deal with different headers with the same name, for example.

I want to go to HTML so I can use Bokeh to have interactive plots.
But that means no more PDF, because Bokeh will not export to
PDF.
This commit is contained in:
Roger Labbe 2015-03-03 12:52:44 -08:00
parent e50f51987a
commit b32deadfa6
9 changed files with 167 additions and 423 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
01_g-h_filter.ipynb
View File

@ -284,7 +284,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Try Another Scale\n",
"** Try Another Scale**\n",
"\n",
"Is there any way we can improve upon this result? The obvious, first thing to try is get a better sensor. Unfortunately, your co-worker informs you that she has built 10 scales, and they all operate with about the same accuracy. You have her bring out another scale, and you weigh yourself on one, and then on the other. The first scale (A) reads \"160 lbs\", and the second (B) reads \"170 lbs\". What can we conclude about your weight?\n",
"\n",

28
05_Kalman_Filters.ipynb
View File

@ -1,7 +1,16 @@
{
"cells": [
{
"cell_type": "markdown",
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "raw",
"metadata": {},
"source": [
"[Table of Contents](http://nbviewer.ipython.org/github/rlabbe/Kalman-and-Bayesian-Filters-in-Python/blob/master/table_of_contents.ipynb)"
@ -9356,7 +9365,13 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"#####Exercise:\n",
"### Exercise: Modify Variance Values"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Modify the values of `movement_variance` and `sensor_variance` and note the effect on the filter and on the variance. Which has a larger effect on the value that variance converges to. For example, which results in a smaller variance:\n",
"\n",
" movement_variance = 40\n",
@ -19183,8 +19198,13 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"######Discussion\n",
"\n",
"### Discussion"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here we set a bad initial guess of 100. We can see that the filter never 'acquires' the signal. Note now the peak of the filter output always lags the peak of the signal by a small amount, and how the filtered signal does not come very close to capturing the high and low peaks of the input signal.\n",
"\n",
"If we recall the g-h filter chapter we can understand what is happening here. The structure of the g-h filter requires that the filter output chooses a value part way between the prediction and measurement. A varying signal like this one is always accelerating, whereas our process model assumes constant velocity, so the filter is mathematically guaranteed to always lag the input signal. \n",

31
06_Multivariate_Kalman_Filters.ipynb
View File

@ -266,7 +266,7 @@
]
},
{
"cell_type": "markdown",
"cell_type": "raw",
"metadata": {},
"source": [
"The techniques in the last chapter are very powerful, but they only work in one dimension. The gaussians represent a mean and variance that are scalars - real numbers. They provide no way to represent multidimensional data, such as the position of a dog in a field. You may retort that you could use two Kalman filters for that case, one tracks the x coordinate and the other tracks the y coordinate. That does work in some cases, but put that thought aside, because soon you will see some enormous benefits to implementing the multidimensional case.\n",
@ -16210,9 +16210,13 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### **Step 1:** Choose the State Variables and Set Initial Conditions\n",
"\n",
"\n",
"### **Step 1:** Choose the State Variables and Set Initial Conditions"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the previous chapter we tracked a dog in one dimension by using a Gaussian. The mean $(\\mu)$ represented the most likely position, and the variance ($\\sigma^2$) represented the probability distribution of the position. In that problem the position is the *state* of the system, and we call $\\mu$ the *state variable*.\n",
"\n",
"In this chapter we will be tracking both the position and velocity of the dog, so we have two state variables. It is important to understand that this is a design choice with implications and assumptions that we are not yet prepared to explore. For example, we could optionally also track acceleration, or even jerk. For now I will only state that the decision to track position and velocity contains an assumption that acceleration is zero, which of course often not true. If the object being tracked *is* accelerating, then the performance of the filter will be suboptimal, or even diverge. However, this is a good way to introduce the concept of Kalman filtering without getting bogged down into too many details, or having to deal with large matrices.\n",
@ -16261,8 +16265,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### **Step 2:** Design the State Transition Function\n",
"\n",
"### **Step 2:** Design the State Transition Function\n",
"\n",
"The next step in designing a Kalman filter is telling it how to predict the next state from the current state. We do this by providing it with equations that describe the physical model of the system. For example, for our dog tracking problem we are tracking a moving object, so we just need to provide it with the Newtonian equations for motion. If we were tracking a thrown ball we would have to provide equations for how a ball moves in a gravitational field, and perhaps include the effects of things like air drag. If we were writing a Kalman filter for a rocket we would have to tell it how the rocket responds to its thrusters and main engine. A Kalman filter for a bowling ball would incorporate the effects of friction and ball rotation. You get the idea. \n",
"\n",
@ -16349,7 +16352,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### **Step 3**: Design the Motion Function\n",
"### **Step 3**: Design the Motion Function\n",
"\n",
"The Kalman filter does not just filter data, it allows us to incorporate control inputs for systems like robots and airplanes. Consider the state transition function we wrote for the dog:\n",
"\n",
@ -16368,7 +16371,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### **Step 4**: Design the Measurement Function\n",
"### **Step 4**: Design the Measurement Function\n",
"\n",
"The Kalman filter computes the update step in what we call *measurement space*. We mostly ignored this issue in the previous chapter because of the complication it adds. In the last chapter we tracked our dog's position using a sensor that reported his position. Computing the *residual* was easy - subtract the filter's predicted position from the measurement:\n",
"\n",
@ -16414,7 +16417,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### **Step 5**: Design the Measurement Noise Matrix\n",
"### **Step 5**: Design the Measurement Noise Matrix\n",
"\n",
"The *measurement noise matrix* is a matrix that models the noise in our sensors as a covariance matrix. This can be admittedly a very difficult thing to do in practice. A complicated system may have many sensors, the correlation between them might not be clear, and usually their noise is not a pure Gaussian. For example, a sensor might be biased to read high if the temperature is high, and so the noise is not distributed equally on both sides of the mean. Later we will address this topic in detail. For now I just want you to get used to the idea of the measurement noise matrix so we will keep it deliberately simple.\n",
"\n",
@ -16433,7 +16436,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### **Step 6**: Design the Process Noise Matrix\n",
"### **Step 6**: Design the Process Noise Matrix\n",
"\n",
"What is *process noise*? Consider the motion of a thrown ball. In a vacuum and with constant gravitational force it moves in a parabola. However, if you throw the ball on the surface of the earth you will also need to model factors like rotation and air drag. However, even when you have done all of that there is usually things you cannot account for. For example, consider wind. On a windy day the ball's trajectory will differ from the computed trajectory, perhaps by a significant amount. Without wind sensors, we may have no way to model the wind. The Kalman filter models this as *process noise*, and calls it $\\mathbf{Q}$.\n",
"\n",
@ -18739,7 +18742,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Discussion\n",
"### Discussion\n",
"\n",
"As you can see, both filters produce the same results. Feel free to vary the initial guess, the measurement noise, and the process noise; so long as you make the same changes to both filters the output should be the same. This is a solid demonstration, albeit not a rigorous proof, that both filters in fact implement the same math for the 1-D case. "
]
@ -18881,7 +18884,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Exercise: Compare to a Filter That Incorporates Velocity\n",
"## Exercise: Compare to a Filter That Incorporates Velocity\n",
"\n",
"The last example did not use one of the fundamental insights of this chapter, unobserved variables. In this example velocity would the the unobserved variable. Write a Kalman filter that uses the state $\\mathbf{x}=\\begin{bmatrix}x & \\dot{x}\\end{bmatrix}^\\mathsf{T}$ and compare it against the filter in the last exercise which used the state $\\mathbf{x}=\\begin{bmatrix}x\\end{bmatrix}$."
]
@ -18901,7 +18904,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Solution\n",
"### Solution\n",
"\n",
"We've already implemented a Kalman filter for position and velocity, so I will provide the code without much comment, and then plot the result."
]
@ -19378,7 +19381,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Discussion\n",
"### Discussion\n",
"\n",
"The output of the filter that incorporates velocity into the state produces much better output than the filter that only tracks position - the output is much closer to a straight line. We've already discussed why unobserved variables increase the precision of the filter, so I will not repeat that explanation here. But the last exercise and this one is intended to trigger a train of thought:\n",
"\n",

142
08_Designing_Kalman_Filters.ipynb
View File

@ -474,10 +474,20 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"That looks correct. The slope is 1/2, as we would expect with a velocity of (2,1), and the data seems to start at near (6,4).\n",
"\n",
"##### Step 1: Choose the State Variables\n",
"\n",
"That looks correct. The slope is 1/2, as we would expect with a velocity of (2,1), and the data seems to start at near (6,4)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 1: Choose the State Variables"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As always, the first step is to choose our state variables. We are tracking in two dimensions and have a sensor that gives us a reading in each of those two dimensions, so we know that we have the two *observed variables* $x$ and $y$. If we created our Kalman filter using only those two variables the performance would not be very good because we would be ignoring the information velocity can provide to us. We will want to incorporate velocity into our equations as well. I will represent this as\n",
"\n",
"$$\\mathbf{x} = \n",
@ -492,15 +502,20 @@
"\n",
"It might be a good time to pause and address how you identify the unobserved variables. This particular example is somewhat obvious because we already worked through the 1D case in the previous chapters. Would it be so obvious if we were filtering market data, population data from a biology experiment, and so on? Probably not. There is no easy answer to this question. The first thing to ask yourself is what is the interpretation of the first and second derivatives of the data from the sensors. We do that because obtaining the first and second derivatives is mathematically trivial if you are reading from the sensors using a fixed time step. The first derivative is just the difference between two successive readings. In our tracking case the first derivative has an obvious physical interpretation: the difference between two successive positions is velocity. \n",
"\n",
"Beyond this you can start looking at how you might combine the data from two or more different sensors to produce more information. This opens up the field of *sensor fusion*, and we will be covering examples of this in later sections. For now, recognize that choosing the appropriate state variables is paramount to getting the best possible performance from your filter. "
"Beyond this you can start looking at how you might combine the data from two or more different sensors to produce more information. This opens up the field of *sensor fusion*, and we will be covering examples of this in later sections. For now, recognize that choosing the appropriate state variables is paramount to getting the best possible performance from your filter."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### **Step 2:** Design State Transition Function"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### **Step 2:** Design State Transition Function\n",
"\n",
"Our next step is to design the state transition function. Recall that the state transition function is implemented as a matrix $\\mathbf{F}$ that we multiply with the previous state of our system to get the next state, like so. \n",
"\n",
"$$\\mathbf{x}' = \\mathbf{Fx}$$\n",
@ -514,7 +529,7 @@
"y' &= (0*x) + (0* v_x) + (1*y) + (\\Delta t * v_y) \\\\\n",
"v_y &= (0*x) + (0*v_x) + (0*y) + (1 * v_y)\n",
"\\end{aligned}\n",
"$$\n"
"$$"
]
},
{
@ -553,7 +568,13 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### **Step 3**: Design the Motion Function\n",
"### **Step 3**: Design the Motion Function"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We have no control inputs to our robot (yet!), so this step is trivial - set the motion input $\\small\\mathbf{u}$ to zero. This is done for us by the class when it is created so we can skip this step, but for completeness we will be explicit."
]
},
@ -572,8 +593,13 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"##### **Step 4**: Design the Measurement Function\n",
"### **Step 4**: Design the Measurement Function"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The measurement function defines how we go from the state variables to the measurements using the equation $\\mathbf{z} = \\mathbf{Hx}$. At first this is a bit counterintuitive, after all, we use the Kalman filter to go from measurements to state. But the update step needs to compute the residual between the current measurement and the measurement represented by the prediction step. Therefore $\\textbf{H}$ is multiplied by the state $\\textbf{x}$ to produce a measurement $\\textbf{z}$. \n",
"\n",
"In this case we have measurements for (x,y), so $\\textbf{z}$ must be of dimension $2\\times 1$. Our state variable is size $4\\times 1$. We can deduce the required size for $\\textbf{H}$ by recalling that multiplying a matrix of size $m\\times n$ by $n\\times p$ yields a matrix of size $m\\times p$. Thus,\n",
@ -639,8 +665,13 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### **Step 5**: Design the Measurement Noise Matrix\n",
"\n",
"### **Step 5**: Design the Measurement Noise Matrix"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this step we need to mathematically model the noise in our sensor. For now we will make the simple assumption that the $x$ and $y$ variables are independent Gaussian processes. That is, the noise in x is not in any way dependent on the noise in y, and the noise is normally distributed about the mean. For now let's set the variance for $x$ and $y$ to be 5 for each. They are independent, so there is no covariance, and our off diagonals will be 0. This gives us:\n",
"\n",
"$$\\mathbf{R} = \\begin{bmatrix}5&0\\\\0&5\\end{bmatrix}$$\n",
@ -674,7 +705,13 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Step 6: Design the Process Noise Matrix\n",
"### Step 6: Design the Process Noise Matrix"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally, we design the process noise. We don't yet have a good way to model process noise, so for now we will assume there is a small amount of process noise, say 0.1 for each state variable. Later we will tackle this admittedly difficult topic in more detail. We have 4 state variables, so we need a $4{\\times}4$ covariance matrix:\n",
"\n",
"$$\\mathbf{Q} = \\begin{bmatrix}0.1&0&0&0\\\\0&0.1&0&0\\\\0&0&0.1&0\\\\0&0&0&0.1\\end{bmatrix}$$\n",
@ -709,8 +746,13 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### **Step 7**: Design Initial Conditions\n",
"\n",
"### **Step 7**: Design Initial Conditions"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For our simple problem we will set the initial position at (0,0) with a velocity of (0,0). Since that is a pure guess, we will set the covariance matrix $\\small\\mathbf{P}$ to a large value.\n",
"$$ \\mathbf{x} = \\begin{bmatrix}0\\\\0\\\\0\\\\0\\end{bmatrix}\\\\\n",
"\\mathbf{P} = \\begin{bmatrix}500&0&0&0\\\\0&500&0&0\\\\0&0&500&0\\\\0&0&0&500\\end{bmatrix}$$\n",
@ -26252,11 +26294,21 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Devise a way to prove that fusing the PS and wheel measurements yields a better result than using the wheel alone.\n",
"\n",
"##### Solution 1\n",
"\n",
"Force the Kalman filter to disregard the PS measurement by setting the measurement noise for the PS to a near infinite value. Re-run the filter and observe the standard deviation of the residual. "
"Devise a way to prove that fusing the PS and wheel measurements yields a better result than using the wheel alone."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Solution 1"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Force the Kalman filter to disregard the PS measurement by setting the measurement noise for the PS to a near infinite value. Re-run the filter and observe the standard deviation of the residual."
]
},
{
@ -26284,11 +26336,20 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Here we can see the error in the filter where the PS measurement is almost entirely ignored is greater than that where it is used.\n",
"\n",
"\n",
"##### Solution 2\n",
"\n",
"Here we can see the error in the filter where the PS measurement is almost entirely ignored is greater than that where it is used."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Solution 2"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This is more work, but we can write a Kalman filter that only takes one measurement."
]
},
@ -28107,7 +28168,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Step 1: Choose the State Variables"
"### Step 1: Choose the State Variables"
]
},
{
@ -28144,7 +28205,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### **Step 2:** Design State Transition Function"
"### **Step 2:** Design State Transition Function"
]
},
{
@ -28184,7 +28245,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Interlude: Test State Transition"
"### Interlude: Test State Transition"
]
},
{
@ -28731,7 +28792,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### **Step 3**: Design the Motion Function"
"### **Step 3**: Design the Motion Function"
]
},
{
@ -28745,7 +28806,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### **Step 4**: Design the Measurement Function"
"### **Step 4**: Design the Measurement Function"
]
},
{
@ -28779,7 +28840,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### **Step 5**: Design the Measurement Noise Matrix"
"### **Step 5**: Design the Measurement Noise Matrix"
]
},
{
@ -28795,7 +28856,13 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Step 6: Design the Process Noise Matrix\n",
"### Step 6: Design the Process Noise Matrix"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally, we design the process noise. As with the robot tracking example, we don't yet have a good way to model process noise. However, we are assuming a ball moving in a vacuum, so there should be no process noise. For now we will assume the process noise is 0 for each state variable. This is a bit silly - if we were in a perfect vacuum then our predictions would be perfect, and we would have no need for a Kalman filter. We will soon alter this example to be more realistic by incorporating air drag and ball spin. \n",
"\n",
"We have 5 state variables, so we need a $5{\\times}5$ covariance matrix:\n",
@ -28807,8 +28874,13 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Step 7: Design the Initial Conditions\n",
"\n",
"### Step 7: Design the Initial Conditions"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We already performed this step when we tested the state transition function. Recall that we computed the initial velocity for $x$ and $y$ using trigonometry, and set the value of $\\mathbf{x}$ with:\n",
"\n",
" omega = radians(omega)\n",

17
Appendix_B_Symbols_and_Notations.ipynb
View File

@ -264,13 +264,13 @@
"source": [
"This is just notes at this point. \n",
"\n",
"### State\n",
"## State\n",
"\n",
"$x$ (Brookner, Zarchan, Brown)\n",
"\n",
"$\\underline{x}$ Gelb)\n",
"\n",
"### State at step n\n",
"## State at step n\n",
"\n",
"$x_n$ (Brookner)\n",
"\n",
@ -280,7 +280,7 @@
"\n",
"\n",
"\n",
"### Prediction\n",
"## Prediction\n",
"\n",
"$x^-$\n",
"\n",
@ -347,7 +347,7 @@
"\\textbf{P}^-_{k+1} &= \\mathbf{\\phi}_k \\textbf{P}_k\\mathbf{\\phi}_k^T + \\textbf{Q}_{k} \\\\\n",
"\\mathbf{P}_k &= (\\mathbf{I}-\\mathbf{K}_k\\mathbf{H}_k)\\mathbf{P}^-_k\n",
"\\end{aligned}$$\n",
"## \n",
"\n",
"\n",
"### Zarchan\n",
"\n",
@ -392,15 +392,8 @@
"\\textbf{K}_k &= \\textbf{P}^-_k\\textbf{H}_k^T\\mathbf{S}_k^{-1} \\\\\n",
"\\hat{\\textbf{x}}_k &= \\hat{\\textbf{x}}^-_k +\\textbf{K}_k\\textbf{y} \\\\\n",
"\\mathbf{P}_k &= (\\mathbf{I}-\\mathbf{K}_k\\mathbf{H}_k)\\mathbf{P}^-_k\n",
"\\end{aligned}$$\n",
"## \n",
"\n"
"\\end{aligned}$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {

338
Introduction.ipynb
View File

@ -1,338 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[Table of Contents](http://nbviewer.ipython.org/github/rlabbe/Kalman-and-Bayesian-Filters-in-Python/blob/master/table_of_contents.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<center><h1>Kalman and Bayesian Filters in Python</h1></center>\n",
"<center><a href =\"http://nbviewer.ipython.org/urls/raw.github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Python/master/toc.ipynb\">Table of Contents</a></center>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#Introduction\n",
"##### Version 0.0\n",
"\n",
"**Author's note - this is obsolete, read the preface instead.**\n",
"\n",
"\n",
"The Kalman filter was introduced to the world via papers published in 1958 and 1960 by Rudolph E Kalman. This work built on work by Nobert Wiener. Kalman's early papers were extremely abstract, but researchers quickly realized that the papers described a very practical technique to filter noisy data. From then until now it has been an ongoing topic of research, and there are many books and papers devoted not only to the basics, but many specializations and extensions to the technique. If you are reading this, you have likely come across some of them.\n",
"\n",
"If you are like me, you probably find them nearly impenetrable. I find that almost all start with very abstract math, assume familiarity with notation and naming conventions that I haven't seen before, and focus heavily on proof rather than exposition and teaching. This is perhaps understandable, but it is a regrettable situation and not necessary. \n",
"\n",
"After struggling through this material for some time, things finally began making sense. A majority of my 'aha' moments were due to implementing and experimenting with various simple filters. What to make of an equation like $K_k=P_{k}^{-}H^{T}_{k}[H_k P^{-}_{k} H^{T}_{k} + R_k]^{-1}$ is initially puzzing. This is especially true when it pops out as the result of two to three pages of linear algrebra, and each variable is given only a very abstract, mathematically rigorous definition. One book I have doesn't bother to define $R_k$ despite using it throughout its 4 pages of derivation of K. If instead I tell you that K is just a scaling factor for choosing how much of a measurement and how much of a prediction to use in the filter, what K is becomes obvious (although perhaps the computation is still a bit mysterious). After implementing a few Kalman filters for toy problems and varying the values of various matrices and constants I developed both an intuitive and fairly deep understanding of how Kalman filters work. This knowledge is indispensible; it is trivial to code the handful of linear equations for a Kalman filter, and they never change. I do mean a handful - you can implement the simplest Kalman filter in 10 lines of code. \n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"** this needs a lot of editting. bored with it for now.**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"While you do need to know some basic probability and some very basic linear algebra to understand and implement Kalman filters, by and large you really do not need to understand the complicated, multi-page derivations. The end result of all the math is a small handful of equations which you can use to perform very sophisticated filtering, smoothing, and tracking. Implementing the basic equations is never difficult. Kalman filter design is much more an art than a science - implementers spend a few minutes writing the basic equations, and then a lot of time tuning the filter for their specific problem. \n",
"\n",
"I compare this to a student learning the equation for the volume of a sphere: $V(r) = \\frac{4}{3}\\pi r^3$ A student can use this equation without even understanding the functional notation $V(r)$, and certainly they do not need to be able to derive the equation via calculus. Eventually, in some domains, knowledge of the calculus will become useful, but an enormous amount of work can be done by only knowing the equation and how to apply it. Also, it is often useful to understand facts about a sphere, such as it the shape that encloses the greatest volume with the smallest surface area. You do not have to know how to prove that to make use of that information to explain the reason for why bubbles are round, or to economically fence an area for livestock. \n",
"\n",
"I argue the same is largely true with Kalman filters. In this book I will not prove the Kalman filter equations are correct, nor will I derive them. I will instead strive to build in you an physical, intuitive understanding of what the equations mean, and how they interact. Of course, if you are trying to navigate a spaceship to Mars you will require a more sophisticated understanding than I provide, and you will not view this as a useful resource. But the rest of people, who perhaps wants to track heads in video, or control a little hobby robot, or smooth some data, I think this approach should provide a lot of insight into how to create Kalman filters, and provide enough background that the standard texts are now approachable."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Prerequisites\n",
"\n",
"While Kalman filters are used in many domains, they were initially created to solve problems with missle tracking and navigation. Most of my examples will draw from physical examples such as these. However, I do this not just from a sense of history, but because I believe that helps the student form strong pysical intuitions about what the filter is doing. For example, we will learn how the Kalman filter creates and uses something called \"hidden variables\". This is normally presented in a highly abstract manner. However, it is actually quite simple. Suppose I know your position at several points in time. From that I can calculate your velocity even though I don't have any sensor that directly measures your velocity. In a Kalman filter, if I have a position sensor, the filter will generate the hidden, or unobserved variable velocity. A lot of seemingly arcane terminology suddenly becomes concrete and clear. So you should have taken a basic physics course and understand equations like $d = \\frac{a}{2} t^2 + v_0 t + d_0$\n",
"\n",
"You will need some basic calculus - you should be able to integrate velocity to get distance, or take the derivative of the distance equation above to get the velocit equation. You should be familiar with trigonometry.\n",
"\n",
"Kalman filtering depends heavily on statistics. I have a basic Kalman filtering textbook that devotes several chapters to statistics before even trying to discuss Kalman filtering. Again, I feel like this is a place where you can get a long way with a little information. So I will assume that you have had some exposure to probability. If you have seen done calculus you probably know how to perform basic probability computations. I will provide some remedial math for gaussian distributions, but I will move fairly quickly through it. \n",
"\n",
"Finally, you will probably want to have some exposure to linear algebra. I will assume that you understand matrices, arrays, and simple operations like matrix multiplication. In the more difficult moments we will use things like LU and Cholesky decomposition, and touch on things like eigenvalues and eigenvectors, but you really do not need to be able to do math at that level. \n",
"\n",
"I am writing this in IPython Notebook. I use Python because of its excellent mathematics library in the form of numpy and scipy, and its strength as a general purpose programming language. Kalman filter books that do include code invariably use Matlab. This makes sense, as any professional engineer with have access to it, and long practice with it. However, I am addressing the hobbiest, and most cannot afford to, or do not want to buy Matlab. Finally, the interactive aspect of mixing code, results, and text in one document is irrestible to me. I have read texts and papers with intriguiging graphs, and I struggle to understand how the graph was generated. With IPython Notebook nothing is hidden - the math is all revealed. If you wonder 'what would happen if I changed this parameter' - you can just type in a new value and see the results. Finally, because of the facilities of numpy and scipy you do not have to code or understand details of linear algebra - when the Kalman filter equations state that we need to find the Cholesky decomposition we will just call *numpy.linalg.cholesky*. I will spend a few words explaining what that means when we get to it, but to be honest you do not need to understand it to use the code. \n",
"\n",
"Since this is a book on doing Kalman filtering with Python I will expect that you know Python; I do not attempt to teach it. With that said, it is a very easy language to learn, and it reads much like pseudocode. If you are more familiar with another language I do not think you will have any major difficulties reading the source code. I purposefully restrict the code to the more basic features of Python to accomodate people with varying skill levels. If you use this code in your own work, feel free to use more advanced Python facilities if it strikes your fancy.\n",
"\n",
"So that is a fair amount of prerequisites, but I think you will already have most of them if you are seriously trying to solve a problem where Kalman filters are useful. "
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<style>\n",
"@import url('http://fonts.googleapis.com/css?family=Source+Code+Pro');\n",
"@import url('http://fonts.googleapis.com/css?family=Vollkorn');\n",
"@import url('http://fonts.googleapis.com/css?family=Arimo');\n",
"\n",
" div.cell{\n",
" width: 850px;\n",
" margin-left: 0% !important;\n",
" margin-right: auto;\n",
" }\n",
" div.text_cell code {\n",
" background: transparent;\n",
" color: #000000;\n",
" font-weight: 600;\n",
" font-size: 11pt;\n",
" font-style: bold;\n",
" font-family: 'Source Code Pro', Consolas, monocco, monospace;\n",
" }\n",
" h1 {\n",
" font-family: 'Open sans',verdana,arial,sans-serif;\n",
"\t}\n",
"\t\n",
" div.input_area {\n",
" background: #F6F6F9;\n",
" border: 1px solid #586e75;\n",
" }\n",
"\n",
" .text_cell_render h1 {\n",
" font-weight: 200;\n",
" font-size: 30pt;\n",
" line-height: 100%;\n",
" color:#c76c0c;\n",
" margin-bottom: 0.5em;\n",
" margin-top: 1em;\n",
" display: block;\n",
" white-space: wrap;\n",
" } \n",
" h2 {\n",
" font-family: 'Open sans',verdana,arial,sans-serif;\n",
" }\n",
" .text_cell_render h2 {\n",
" font-weight: 200;\n",
" font-size: 20pt;\n",
" font-style: italic;\n",
" line-height: 100%;\n",
" color:#c76c0c;\n",
" margin-bottom: 0.5em;\n",
" margin-top: 1.5em;\n",
" display: block;\n",
" white-space: nowrap;\n",
" } \n",
" h3 {\n",
" font-family: 'Open sans',verdana,arial,sans-serif;\n",
" }\n",
" .text_cell_render h3 {\n",
" font-weight: 300;\n",
" font-size: 18pt;\n",
" line-height: 100%;\n",
" color:#d77c0c;\n",
" margin-bottom: 0.5em;\n",
" margin-top: 2em;\n",
" display: block;\n",
" white-space: nowrap;\n",
" }\n",
" h4 {\n",
" font-family: 'Open sans',verdana,arial,sans-serif;\n",
" }\n",
" .text_cell_render h4 {\n",
" font-weight: 300;\n",
" font-size: 16pt;\n",
" color:#d77c0c;\n",
" margin-bottom: 0.5em;\n",
" margin-top: 0.5em;\n",
" display: block;\n",
" white-space: nowrap;\n",
" }\n",
" h5 {\n",
" font-family: 'Open sans',verdana,arial,sans-serif;\n",
" }\n",
" .text_cell_render h5 {\n",
" font-weight: 300;\n",
" font-style: normal;\n",
" color: #1d3b84;\n",
" font-size: 16pt;\n",
" margin-bottom: 0em;\n",
" margin-top: 1.5em;\n",
" display: block;\n",
" white-space: nowrap;\n",
" }\n",
" div.text_cell_render{\n",
" font-family: 'Arimo',verdana,arial,sans-serif;\n",
" line-height: 135%;\n",
" font-size: 125%;\n",
" width:750px;\n",
" margin-left:auto;\n",
" margin-right:auto;\n",
" text-align:justify;\n",
" text-justify:inter-word;\n",
" }\n",
" div.output_subarea.output_text.output_pyout {\n",
" overflow-x: auto;\n",
" overflow-y: scroll;\n",
" max-height: 50000px;\n",
" }\n",
" div.output_subarea.output_stream.output_stdout.output_text {\n",
" overflow-x: auto;\n",
" overflow-y: scroll;\n",
" max-height: 50000px;\n",
" }\n",
" code{\n",
" font-size: 70%;\n",
" }\n",
" .rendered_html code{\n",
" background-color: transparent;\n",
" }\n",
" ul{\n",
" margin: 2em;\n",
" }\n",
" ul li{\n",
" padding-left: 0.5em; \n",
" margin-bottom: 0.5em; \n",
" margin-top: 0.5em; \n",
" }\n",
" ul li li{\n",
" padding-left: 0.2em; \n",
" margin-bottom: 0.2em; \n",
" margin-top: 0.2em; \n",
" }\n",
" ol{\n",
" margin: 2em;\n",
" }\n",
" ol li{\n",
" padding-left: 0.5em; \n",
" margin-bottom: 0.5em; \n",
" margin-top: 0.5em; \n",
" }\n",
" ul li{\n",
" padding-left: 0.5em; \n",
" margin-bottom: 0.5em; \n",
" margin-top: 0.2em; \n",
" }\n",
" a:link{\n",
" font-weight: bold;\n",
" color:#447adb;\n",
" }\n",
" a:visited{\n",
" font-weight: bold;\n",
" color: #1d3b84;\n",
" }\n",
" a:hover{\n",
" font-weight: bold;\n",
" color: #1d3b84;\n",
" }\n",
" a:focus{\n",
" font-weight: bold;\n",
" color:#447adb;\n",
" }\n",
" a:active{\n",
" font-weight: bold;\n",
" color:#447adb;\n",
" }\n",
" .rendered_html :link {\n",
" text-decoration: underline; \n",
" }\n",
" .rendered_html :hover {\n",
" text-decoration: none; \n",
" }\n",
" .rendered_html :visited {\n",
" text-decoration: none;\n",
" }\n",
" .rendered_html :focus {\n",
" text-decoration: none;\n",
" }\n",
" .rendered_html :active {\n",
" text-decoration: none;\n",
" }\n",
" .warning{\n",
" color: rgb( 240, 20, 20 )\n",
" } \n",
" hr {\n",
" color: #f3f3f3;\n",
" background-color: #f3f3f3;\n",
" height: 1px;\n",
" }\n",
" blockquote{\n",
" display:block;\n",
" background: #fcfcfc;\n",
" border-left: 5px solid #c76c0c;\n",
" font-family: 'Open sans',verdana,arial,sans-serif;\n",
" width:680px;\n",
" padding: 10px 10px 10px 10px;\n",
" text-align:justify;\n",
" text-justify:inter-word;\n",
" }\n",
" blockquote p {\n",
" margin-bottom: 0;\n",
" line-height: 125%;\n",
" font-size: 100%;\n",
" }\n",
"</style>\n",
"<script>\n",
" MathJax.Hub.Config({\n",
" TeX: {\n",
" extensions: [\"AMSmath.js\"]\n",
" },\n",
" tex2jax: {\n",
" inlineMath: [ ['$','$'], [\"\\\\(\",\"\\\\)\"] ],\n",
" displayMath: [ ['$$','$$'], [\"\\\\[\",\"\\\\]\"] ]\n",
" },\n",
" displayAlign: 'center', // Change this to 'center' to center equations.\n",
" \"HTML-CSS\": {\n",
" styles: {'.MathJax_Display': {\"margin\": 4}}\n",
" }\n",
" });\n",
"</script>\n"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#format the book\n",
"import book_format\n",
"book_format.load_style()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.4.2"
}
},
"nbformat": 4,
"nbformat_minor": 0
}

4
code/secnum.py
View File

@ -24,7 +24,7 @@ Thanks also to:
import io
import os
import urllib2
import urllib3
from IPython.display import display_html, display_javascript
@ -42,7 +42,7 @@ def download(fname, redownload=False):
url = 'https://raw.github.com/dpsanders/ipython_extensions/master/section_numbering/' + fname
print("Downloading %s to %s" % (url, dest))
filein = urllib2.urlopen(url)
filein = urllib3.urlopen(url)
fileout = open(dest, "wb")
chunk = filein.read(1024)
while chunk:

5
pdf/html_book
View File

@ -3,7 +3,8 @@
echo "merging book..."
python merge_book.py > Kalman_and_Bayesian_Filters_in_Python.ipynb
mv -f Kalman_and_Bayesian_Filters_in_Python.ipynb ..
echo "creating pdf..."
ipython nbconvert Kalman_and_Bayesian_Filters_in_Python.ipynb
echo "creating HTML..."
#ipython nbconvert Kalman_and_Bayesian_Filters_in_Python.ipynb

23
pdf/merge_book.py
View File

@ -1,13 +1,11 @@
from __future__ import print_function
import io
from IPython.nbformat import current
import IPython.nbformat as nbformat
import sys
def remove_formatting(nb):
w = nb['worksheets']
node = w[0]
c = node['cells']
c = nb['cells']
for i in range (len(c)):
if 'input' in c[i].keys():
if c[i]['input'][0:16] == '#format the book':
@ -16,9 +14,7 @@ def remove_formatting(nb):
def remove_links(nb):
w = nb['worksheets']
node = w[0]
c = node['cells']
c = nb['cells']
for i in range (len(c)):
if 'source' in c[i].keys():
if c[i]['source'][0:19] == '[Table of Contents]':
@ -27,9 +23,7 @@ def remove_links(nb):
def remove_links_add_appendix(nb):
w = nb['worksheets']
node = w[0]
c = node['cells']
c = nb['cells']
for i in range (len(c)):
if 'source' in c[i].keys():
if c[i]['source'][0:19] == '[Table of Contents]':
@ -42,7 +36,7 @@ def merge_notebooks(filenames):
added_appendix = False
for fname in filenames:
with io.open(fname, 'r', encoding='utf-8') as f:
nb = current.read(f, u'json')
nb = nbformat.read(f, nbformat.NO_CONVERT)
remove_formatting(nb)
if not added_appendix and fname[0:8] == 'Appendix':
remove_links_add_appendix(nb)
@ -52,14 +46,13 @@ def merge_notebooks(filenames):
if merged is None:
merged = nb
else:
merged.worksheets[0].cells.extend(nb.worksheets[0].cells)
merged.metadata.name += "_merged"
merged.cells.extend(nb.cells)
#merged.metadata.name += "_merged"
print(current.writes(merged, u'json'))
print(nbformat.writes(merged, nbformat.NO_CONVERT))
if __name__ == '__main__':
#merge_notebooks(sys.argv[1:])
merge_notebooks(
['../00_Preface.ipynb',
'../01_g-h_filter.ipynb',