daviddoji
/
Kalman-and-Bayesian-Filters-in-Python
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Lots of book formatting changes. 

Changed default sizes of plots, changed the book template for
pdf creation, and lots of minor edits, most changing bold to
italic, and removing italic for emphasis.
This commit is contained in:
Roger Labbe 2015-11-26 13:08:40 -08:00
parent d2f8747c36
commit 3e65b5d2f7
19 changed files with 1069 additions and 1093 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

321
01-g-h-filter.ipynb
View File

File diff suppressed because one or more lines are too long

185
02-Discrete-Bayes.ipynb
View File

File diff suppressed because one or more lines are too long

166
04-One-Dimensional-Kalman-Filters.ipynb
View File

File diff suppressed because one or more lines are too long

198
05-Multivariate-Gaussians.ipynb
View File

File diff suppressed because one or more lines are too long

28
06-Multivariate-Kalman-Filters.ipynb
View File

@ -644,7 +644,7 @@
"\n",
"#### **Step 2:** Design the State Transition Function\n",
"\n",
"In the univariate chapter we modeled the dog's motion with\n",
"In the univariate chapter we modeled the dog's motion (the process model) with\n",
"\n",
"$$ x = v \\Delta t + x_0$$\n",
"\n",
@ -682,7 +682,7 @@
"\n",
"$$\\mathbf{A} = \\begin{bmatrix}2& 3 \\\\ 3&-1\\end{bmatrix},\\, \\mathbf{x} = \\begin{bmatrix}x\\\\y\\end{bmatrix}, \\mathbf{b}=\\begin{bmatrix}8\\\\1\\end{bmatrix}$$\n",
"\n",
"We call the set of equations that describe how the systems behaves the *process model*. We use the process model to perform the innovation, because the equations tell us what the next state will be given the current state. Kalman filters implement this using the linear equation, where $\\mathbf{\\bar{x}}$ is the *prior*, or predicted state:\n",
"We call the set of equations that describe how the systems behaves the *process model*. We use the process model to perform the *innovation*, because the equations tell us what the next state will be given the current state. Kalman filters implement this using the linear equation, where $\\mathbf{\\bar{x}}$ is the *prior*, or predicted state:\n",
"\n",
"$$\\mathbf{\\bar{x}} = \\mathbf{Fx}$$\n",
"\n",
@ -715,7 +715,7 @@
"\\mathbf{\\bar{x}} &= \\mathbf{Fx}\n",
"\\end{aligned}$$\n",
"\n",
"$\\mathbf{F}$ is often called the **state transition function**. In the `KalmanFilter` class we implement the state transition function with"
"$\\mathbf{F}$ is often called the *state transition function*. In the `KalmanFilter` class we implement the state transition function with"
]
},
{
@ -777,7 +777,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"This worked. Note that the code does not distinguish between between the *prior* and *posterior* in the variable names, so after calling predict the prior $\\bar{\\mathbf{x}}$ is stored in `KalmanFilter.x`. If we call `predict()` several times in a row the value will be updated each time. "
"This worked. Note that the code does not distinguish between between the prior and posterior in the variable names, so after calling predict the prior $\\bar{\\mathbf{x}}$ is stored in `KalmanFilter.x`. If we call `predict()` several times in a row the value will be updated each time. "
]
},
{
@ -894,7 +894,7 @@
"source": [
"#### Process Noise\n",
"\n",
"A quick review on **process noise**. A car is driving along the road with the cruise control on; it should travel at a constant speed. We model this with $x=\\dot{x}\\Delta t + x_0$. However, it is affected by a number of unknown factors. The cruise control is not perfect, and cannot maintain a constant velocity. Winds affect the car, as do hills and potholes. Passengers roll down windows, changing the drag profile of the car. \n",
"A quick review on *process noise*. A car is driving along the road with the cruise control on; it should travel at a constant speed. We model this with $x=\\dot{x}\\Delta t + x_0$. However, it is affected by a number of unknown factors. The cruise control is not perfect, and cannot maintain a constant velocity. Winds affect the car, as do hills and potholes. Passengers roll down windows, changing the drag profile of the car. \n",
"\n",
"We can model this system with the differential equation\n",
"\n",
@ -902,7 +902,7 @@
"\n",
"where $\\mathbf{Fx}$ models the state transition and $w$ is white process noise.\n",
"\n",
"To account for the unmodelled behavior in the system the filter adds a **process noise** covariance matrix $\\mathbf{Q}$ to the covariance $\\mathbf{P}$. We do not add anything to $\\mathbf{x}$ because the noise is *white* - which means that the mean of the noise will be 0. If the mean is 0, $\\mathbf{x}$ will not change."
"To account for the unmodelled behavior in the system the filter adds a process noise covariance matrix $\\mathbf{Q}$ to the covariance $\\mathbf{P}$. We do not add anything to $\\mathbf{x}$ because the noise is *white* - which means that the mean of the noise will be 0. If the mean is 0, $\\mathbf{x}$ will not change."
]
},
{
@ -911,7 +911,7 @@
"source": [
"The univariate Kalman filter used `variance = variance + process_noise` to compute the variance for the variance of the prediction step. The multivariate Kalman filter does exactly the same thing, essentially `P = P + Q`. I say 'essentially' because there are other terms unrelated to noise in the covariance equation that we will see later.\n",
"\n",
"Computing the process noise matrix can be quite demanding, and we will put it off until the Kalman math chapter. For now I will say that $\\mathbf{Q}$ equals the *expected value* of the white noise $w$, computed as $\\mathbf{Q} = E[\\mathbf{ww}^\\mathsf{T}]$. In this chapter we will focus on building an intuitive understanding on how modifying this matrix alters the behavior of the filter. FilterPy provides functions which compute $\\mathbf{Q}$ for the kinematic problems of this chapter."
"Computing the process noise matrix can be quite demanding, and we will put it off until the Kalman math chapter. For now know that $\\mathbf{Q}$ equals the expected value of the white noise $w$, computed as $\\mathbf{Q} = E[\\mathbf{ww}^\\mathsf{T}]$. In this chapter we will focus on building an intuitive understanding on how modifying this matrix alters the behavior of the filter. FilterPy provides functions which compute $\\mathbf{Q}$ for the kinematic problems of this chapter."
]
},
{
@ -953,7 +953,7 @@
"source": [
"#### **Step 4**: Design the Control Function\n",
"\n",
"The Kalman filter does not just filter data, it allows us to incorporate control inputs for systems like robots and airplanes. Suppose we are controlling a robot. At each time step we would send control signals to the robot based on its current position vs desired position. Kalman filter equations incorporate that knowledge into the filter equations, creating a predicted position based both on current velocity *and* control inputs to the drive motors. Remember, we *never* throw information away.\n",
"The Kalman filter does not just filter data, it allows us to incorporate control inputs for systems like robots and airplanes. Suppose we are controlling a robot. At each time step we would send control signals to the robot based on its current position vs desired position. Kalman filter equations incorporate that knowledge into the filter equations, creating a predicted position based both on current velocity and control inputs to the drive motors. Remember, we *never* throw information away.\n",
"\n",
"For a linear system the effect of control inputs can be described as a set of linear equations, which we can express with linear algebra as\n",
"\n",
@ -1008,7 +1008,7 @@
"\n",
"#### **Step 5**: Design the Measurement Function\n",
"\n",
"The Kalman filter computes the update step in what we call **measurement space**. We mostly ignored this issue in the univariate chapter because of the complication it adds. We tracked our dog's position using a sensor that reported his position. Computing the *residual* was easy - subtract the filter's predicted position from the measurement:\n",
"The Kalman filter computes the update step in what we call *measurement space*. We mostly ignored this issue in the univariate chapter because of the complication it adds. We tracked our dog's position using a sensor that reported his position. Computing the *residual* was easy - subtract the filter's predicted position from the measurement:\n",
"\n",
"$$ \\mathtt{residual} = \\mathtt{measured\\, \\, position} - \\mathtt{estimated\\, \\, position}$$\n",
"\n",
@ -1449,7 +1449,7 @@
"A word about my notation. I'm a programmer, and I am used to code reading `x = x + 1`. That is not an equation; the sides are not equal. If we wanted to write this in correct mathematical notation we'd write\n",
"$$x_k = x_{k-1} + 1$$\n",
"\n",
"The Kalman filter equations are traditionally *littered* with subscripts and superscripts to keep the equations mathematically consistent. This makes them very precise, but extremely hard to read. In most of the book I opt for subscriptless statements. As a programmer you should understand that I am showing you an algorithm that is to be executed step by step. I'll elaborate on this once we have a concrete example.\n",
"The Kalman filter equations are traditionally littered with subscripts and superscripts to keep the equations mathematically consistent. This makes them very precise, but extremely hard to read. In most of the book I opt for subscriptless statements. As a programmer you should understand that I am showing you an algorithm that is to be executed step by step. I'll elaborate on this once we have a concrete example.\n",
"\n",
"### Prediction Equations\n",
"\n",
@ -1603,7 +1603,7 @@
"source": [
"### Update Equations\n",
"\n",
"The update equations look messier than the predict equations, but that is mostly due to the Kalman filter computing the update in **measurement space**. This is because measurements are not *invertible*. For example, later on we will be using sensors that only give you the range to a target. It is impossible to convert a range into a position - an infinite number of positions (in a circle) will yield the same range. On the other hand, we can always compute the range (measurement) given a position(state).\n",
"The update equations look messier than the predict equations, but that is mostly due to the Kalman filter computing the update in measurement space. This is because measurements are not *invertible*. For example, later on we will be using sensors that only give you the range to a target. It is impossible to convert a range into a position - an infinite number of positions (in a circle) will yield the same range. On the other hand, we can always compute the range (measurement) given a position (state).\n",
"\n",
"Before I continue, recall that we are trying to do something very simple: choose a new estimate chosen somewhere between a measurement and a prediction, as in this chart:"
]
@ -1652,7 +1652,7 @@
"\n",
"You should recognize this $\\mathbf{ABA}^\\mathsf{T}$ form - the prediction step used $\\mathbf{FPF}^\\mathsf{T}$ to update $\\mathbf{P}$ with the state transition function. Here, we use the same form to update it with the measurement function. In a real sense the linear algebra is changing the coordinate system for us. \n",
"\n",
"Once the covariance is in measurement space we need to account for the sensor noise. This is very easy - we just add matrices. The result is variously called either the **system uncertainty** or **innovation covariance**.\n",
"Once the covariance is in measurement space we need to account for the sensor noise. This is very easy - we just add matrices. The result is variously called either the *system uncertainty* or *innovation covariance*.\n",
"\n",
"I want you to compare the equation for the system uncertainty and the covariance\n",
"\n",
@ -1683,7 +1683,7 @@
"\n",
"$$\\mu = (1-K)\\mu_\\mathtt{prior} + K\\mu_\\mathtt{z}$$\n",
"\n",
"Here $K$ is the Kalman gain, and it is a real number between 0 and 1. Examine this equation and ensure you understand how it selects a mean somewhere between the prediction and measurement. In this form the Kalman gain is essentially a *percentage* or *ratio* - if K is .9 it takes 90% of the measurement and 10% of the prediction. \n",
"Here $K$ is the Kalman gain, and it is a real number between 0 and 1. Examine this equation and ensure you understand how it selects a mean somewhere between the prediction and measurement. In this form the Kalman gain is a *percentage* or *ratio* - if K is .9 it takes 90% of the measurement and 10% of the prediction. \n",
"\n",
"For the multivariate Kalman filter $\\mathbf{K}$ is a vector, not a scalar. Here is the equation again: $\\mathbf{K} = \\mathbf{\\bar{P}H}^\\mathsf{T} \\mathbf{S}^{-1}$. Is this a *ratio*? We can think of the inverse of a matrix as linear algebra's way of doing matrix division. Division is not defined for matrices, but it is useful to think of it in this way. So we can read the equation for $\\textbf{K}$ as meaning\n",
"\n",
@ -2753,7 +2753,7 @@
"\n",
"The case with $\\mathbf{Q}$ is more dire. I hope you were skeptical when I blithely assigned a noise matrix to my prediction about the movements of a dog. Who can say what a dog will do next? The Kalman filter in my GPS doesn't know about hills, the outside winds, or my terrible driving skills. Yet the filter requires a precise number to encapsulate all of that information, and it needs to work while I drive off-road in the desert, and when a Formula One champion drives on a closed circuit track.\n",
"\n",
"These problems led some researchers and engineers to derogatorily call the Kalman filter a 'ball of mud'. In other words, it doesn't always hold together so well. Another term to know - Kalman filters can become **smug**. Their estimates are based solely on what you tell it the noises are. Those values can lead to overly confident estimates. $\\mathbf{P}$ gets smaller and smaller while the filter is actually becoming more and more inaccurate! In the worst case the filter diverges. We will see a lot of that when we start studying nonlinear filters. \n",
"These problems led some researchers and engineers to derogatorily call the Kalman filter a 'ball of mud'. In other words, it doesn't always hold together so well. Another term to know - Kalman filters can become *smug*. Their estimates are based solely on what you tell it the noises are. Those values can lead to overly confident estimates. $\\mathbf{P}$ gets smaller and smaller while the filter is actually becoming more and more inaccurate! In the worst case the filter diverges. We will see a lot of that when we start studying nonlinear filters. \n",
"\n",
"The Kalman filter is a mathematical model of the world. The output is only as accurate as that model. To make the math tractable we had to make some assumptions. We assume that the sensors and motion model have Gaussian noise. We assume that everything is linear. If that is true, the Kalman filter is *optimal* in a least squares sense. This means that there is no way to make a better estimate than what the filter gives us. However, these assumption are almost never true, and hence the model is necessarily limited, and a working filter is rarely optimal.\n",
"\n",

360
07-Kalman-Filter-Math.ipynb
View File

File diff suppressed because one or more lines are too long

228
08-Designing-Kalman-Filters.ipynb
View File

File diff suppressed because one or more lines are too long

178
10-Unscented-Kalman-Filter.ipynb
View File

File diff suppressed because one or more lines are too long

90
11-Extended-Kalman-Filters.ipynb
View File

File diff suppressed because one or more lines are too long

113
12-Particle-Filters.ipynb
View File

File diff suppressed because one or more lines are too long

52
13-Smoothing.ipynb
View File

File diff suppressed because one or more lines are too long

144
14-Adaptive-Filtering.ipynb
View File

File diff suppressed because one or more lines are too long

39
Appendix-A-Installation.ipynb
View File

@ -25,7 +25,7 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": 2,
"metadata": {
"collapsed": false
},
@ -117,6 +117,7 @@
" h5 {\n",
" font-family: 'Open sans',verdana,arial,sans-serif;\n",
" }\n",
"\n",
" .text_cell_render h5 {\n",
" font-weight: 200;\n",
" font-style: normal;\n",
@ -265,7 +266,7 @@
"<IPython.core.display.HTML object>"
]
},
"execution_count": 1,
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
@ -442,7 +443,7 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 3,
"metadata": {
"collapsed": false
},
@ -484,7 +485,7 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": 4,
"metadata": {
"collapsed": false
},
@ -498,7 +499,7 @@
"\\phi"
]
},
"execution_count": 3,
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
@ -520,7 +521,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 5,
"metadata": {
"collapsed": false
},
@ -537,7 +538,7 @@
"2⋅╲╱ φ "
]
},
"execution_count": 4,
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
@ -555,7 +556,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 6,
"metadata": {
"collapsed": false
},
@ -570,7 +571,7 @@
"(\\phi - 1)⋅⎝\\phi + 1⎠"
]
},
"execution_count": 5,
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
@ -588,7 +589,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 7,
"metadata": {
"collapsed": false
},
@ -603,7 +604,7 @@
"\\phi - 3⋅\\phi - 4"
]
},
"execution_count": 6,
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
@ -621,7 +622,7 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 8,
"metadata": {
"collapsed": false
},
@ -635,7 +636,7 @@
"8"
]
},
"execution_count": 7,
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
@ -654,7 +655,7 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 9,
"metadata": {
"collapsed": false
},
@ -668,7 +669,7 @@
"2⋅t + 2"
]
},
"execution_count": 8,
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
@ -691,7 +692,7 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": 10,
"metadata": {
"collapsed": false
},
@ -709,7 +710,7 @@
"⎣╲╱ x + z ╲╱ x + z ⎦"
]
},
"execution_count": 9,
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
@ -740,7 +741,7 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": 11,
"metadata": {
"collapsed": false
},
@ -767,7 +768,7 @@
"⎣ 6 2 ⎦"
]
},
"execution_count": 10,
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}

3
book_format.py
View File

@ -67,7 +67,7 @@ def equal_axis():
def reset_axis():
pylab.rcParams['figure.figsize'] = 11, 4
pylab.rcParams['figure.figsize'] = 11, 3
def set_figsize(x=11, y=4):
pylab.rcParams['figure.figsize'] = x, y
@ -135,7 +135,6 @@ def load_style(directory = '.', name='code/custom.css'):
if version[0] > 1 or (version[0] == 1 and version[1] >= 5):
style["axes.prop_cycle"] = "cycler('color', ['#6d904f','#013afe', '#202020','#fc4f30','#e5ae38','#A60628','#30a2da','#008080','#7A68A6','#CF4457','#188487','#E24A33'])"
style.pop("axes.color_cycle", None)
plt.rcParams.update(style)
reset_axis ()
np.set_printoptions(suppress=True)

11
code/book_plots.py
View File

@ -101,17 +101,6 @@ def plot_measurements(xs, ys=None, color='k', lw=2, label='Measurements',
"""
plt.autoscale(tight=True)
'''if ys is not None:
plt.scatter(xs, ys, marker=marker, c=c, s=s,
label=label, alpha=alpha)
if connect:
plt.plot(xs, ys, c=c, lw=1, alpha=alpha)
else:
plt.scatter(range(len(xs)), xs, marker=marker, c=c, s=s,
label=label, alpha=alpha)
if connect:
plt.plot(range(len(xs)), xs, lw=1, c=c, alpha=alpha)'''
if lines:
if ys is not None:
plt.plot(xs, ys, color=color, lw=lw, ls='--', label=label, **kwargs)

14
code/custom.css
View File

@ -4,6 +4,9 @@
@import url('http://fonts.googleapis.com/css?family=Arimo');
@import url('http://fonts.googleapis.com/css?family=Fira_sans');
.CodeMirror pre {
font-family: 'Source Code Pro', Consolas, monocco, monospace;
}
div.cell{
width: 900px;
margin-left: 0% !important;
@ -23,7 +26,7 @@
div.input_area {
background: #F6F6F9;
border: 1px solid #586e75;
border: 1px solid #586e75;
}
.text_cell_render h1 {
@ -82,6 +85,7 @@
h5 {
font-family: 'Open sans',verdana,arial,sans-serif;
}
.text_cell_render h5 {
font-weight: 200;
font-style: normal;
@ -116,7 +120,8 @@
}
code{
font-size: 70%;
font-size: 6pt;
}
.rendered_html code{
background-color: transparent;
@ -209,7 +214,8 @@
<script>
MathJax.Hub.Config({
TeX: {
extensions: ["AMSmath.js", "autobold.js"]
extensions: ["AMSmath.js"],
equationNumbers: { autoNumber: "AMS", useLabelIds: true}
},
tex2jax: {
inlineMath: [ ['$','$'], ["\\(","\\)"] ],
@ -217,7 +223,7 @@
},
displayAlign: 'center', // Change this to 'center' to center equations.
"HTML-CSS": {
scale:100,
scale:95,
availableFonts: [],
preferredFont:null,
webFont: "TeX",

8
pdf/book.tplx
View File

@ -1,10 +1,12 @@
% Inherit from report
((* extends 'report.tplx' *))
%((* extends 'report.tplx' *))
((* set cell_style = 'style_ipython.tplx' *))
((* block docclass *))
\documentclass{book}
\setcounter{chapter}{0}
\raggedbottom
((* endblock docclass *))
((* block preamble *))
@ -14,7 +16,9 @@
\title{Kalman and Bayesian Filters in Python}
\author{Roger R Labbe Jr}
((* endblock title *))
((* block markdowncell scoped *))
\setlength{\parindent}{0em}
\setlength{\parskip}{0.5em}
((( cell.source | citation2latex | strip_files_prefix | markdown2latex(extra_args=["--chapters"]) )))
((* endblock markdowncell *))

16
pdf/book6x9.tplx
View File

@ -1,7 +1,6 @@
% Inherit from report
((* extends 'report.tplx' *))
((* set cell_style = 'style_ipython.tplx' *))
((* block margins *))
\geometry{verbose,papersize={6in, 9in}, tmargin=.5in,bmargin=.5in,lmargin=.75in,rmargin=.75in}
@ -11,7 +10,11 @@
((*- endblock in_prompt -*))
((* block input scoped *))
((( cell.source | highlight2latex | indent(3) )))
$\fontsize{5}{1.5}
\selectfont$
$\noindent\rule{8cm}{0.4pt}$
((( cell.source | highlight2latex | indent(0) )))
$\noindent\rule{8cm}{0.4pt}$
((* endblock input *))
((* block output_prompt *))
@ -21,10 +24,11 @@
((* block docclass *))
\documentclass{book}
\setcounter{chapter}{0}
\raggedbottom
((* endblock docclass *))
((* block preamble *))
((* endblock preamble *))
((* block document *))
((* endblock document *))
((* block title *))
\title{Kalman and Bayesian Filters in Python}
@ -32,6 +36,10 @@
((* endblock title *))
((* block markdowncell scoped *))
\fontsize{10}{1.5}
\selectfont
\setlength{\parindent}{0em}
\setlength{\parskip}{0.5em}
((( cell.source | citation2latex | strip_files_prefix | markdown2latex(extra_args=["--chapters"]) )))
((* endblock markdowncell *))

8
pdf/merge_book.py
View File

@ -1,6 +1,6 @@
from __future__ import print_function
import io
import IPython.nbformat as nbformat
import nbformat
import sys
from formatting import *
@ -28,10 +28,8 @@ def merge_notebooks(outfile, filenames):
if __name__ == '__main__':
f = open('book.ipynb', 'w', encoding='utf-8')
'''merge_notebooks(
['../00_Preface.ipynb',
'../01_g-h_filter.ipynb',
'../Appendix_A_Installation.ipynb'])'''
'''merge_notebooks(f,
['../02-Discrete-Bayes.ipynb'])'''
merge_notebooks(f,
['../00-Preface.ipynb',