daviddoji/Kalman-and-Bayesian-Filters-in-Python
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Major rewrites due to discrete bayes changes.

Browse Source 
I've derived the x + Ky form for the univariate kalman filter.
I completely reordered material, cutting about 10 pages (pdf)
of material. I made the connection between the bayesian form
and orthogonal form more explicit.

Probably there are a lot of grammatical errors, but I wanted to get
these checked in.

I also altered the css - mainly the font.
This commit is contained in:
Roger Labbe 2016-01-18 18:16:20 -08:00
parent 5240944dd4
commit 856775e906
7 changed files with 878 additions and 1052 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

174
02-Discrete-Bayes.ipynb
View File

File diff suppressed because one or more lines are too long

1571
04-One-Dimensional-Kalman-Filters.ipynb
View File

File diff suppressed because one or more lines are too long

12
05-Multivariate-Gaussians.ipynb
View File

@ -11,7 +11,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Multivariate Gaussians - Modeling Uncertainty in Multiple Dimensions"
"# Multivariate Gaussians\n",
"\n",
"Modeling Uncertainty in Multiple Dimensions"
]
},
{
@ -285,9 +287,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The techniques in the last chapter are very powerful, but they only work with one variable or dimension. Gaussians represent a mean and variance that are scalars - real numbers. They provide no way to represent multidimensional data, such as the position of a dog in a field. You may retort that you could use two Kalman filters from the last chapter. One would track the x coordinate and the other the y coordinate. That does work, but suppose we want to track position, velocity, acceleration, and attitude. These values are related to each other, and as we learned in the g-h chapter we should never throw away information. Through one key insight we will achieve markedly better filter performance than was possible with the equations from the last chapter.\n",
"\n",
"In this chapter I will introduce you to multivariate Gaussians - Gaussians for more than one variable, and the key insight I mention above. Then, in the next chapter we will use the math from this chapter to write a complete filter in just a few lines of code. "
"The techniques in the last chapter are very powerful, but they only work with one variable or dimension. Gaussians represent a mean and variance that are scalars - real numbers. They provide no way to represent multidimensional data, such as the position of a dog in a field. You may retort that you could use two Kalman filters, one for x coordinate and the other for the y coordinate. That does work, but suppose we want to track position, velocity, acceleration, and attitude. In the last chapter we were forced to assume the dog was traveling at 1 m/s. Position and velocity are related to each other, and as we learned in the g-h chapter we should never throw away information. In this chapter we learn how to describe this relation probabilistically. Through this one key insight we will achieve markedly better filter performance than was possible with the equations from the last chapter."
]
},
{
@ -303,13 +303,13 @@
"source": [
"In the last two chapters we used Gaussians for a scalar (one dimensional) variable, expressed as $\\mathcal{N}(\\mu, \\sigma^2)$. A more formal term for this is *univariate normal*, where univariate means 'one variable'. The probability distribution of the Gaussian is known as the *univariate normal distribution*.\n",
"\n",
"What might a *multivariate normal distribution* be? *Multivariate* means multiple variables. Our goal is to be able to represent a normal distribution across multiple dimensions. I don't necessarily mean spatial dimensions - it could be position, velocity, and acceleration. Consider a two dimensional case. Let's say we believe that $x = 2$ and $y = 17$. This might be the *x* and *y* coordinates for the position of our dog, it might be the position and velocity of our dog on the x-axis, or the temperature and wind speed at our weather station. It doesn't really matter. We can see that for $N$ dimensions, we need $N$ means, which we will arrange in a column matrix (vector) like so:\n",
"What might a *multivariate normal distribution* be? *Multivariate* means multiple variables. Our goal is to be able to represent a normal distribution across multiple dimensions. I don't necessarily mean spatial dimensions - it could be position, velocity, and acceleration. Consider a two dimensional case. This might be the *x* and *y* coordinates of a robot, it might be the position and velocity of a dog on the x-axis, or milk production and feed rate at a dairy. It doesn't really matter. We can see that for $N$ dimensions, we need $N$ means, which we will arrange in a column matrix (vector) like so:\n",
"\n",
"$$\n",
"\\mu = \\begin{bmatrix}\\mu_1\\\\\\mu_2\\\\ \\vdots \\\\\\mu_n\\end{bmatrix}\n",
"$$\n",
"\n",
"Therefore for this example we would have\n",
"Let's say we believe that $x = 2$ and $y = 17$. We would have\n",
"\n",
"$$\n",
"\\mu = \\begin{bmatrix}2\\\\17\\end{bmatrix} \n",

58
06-Multivariate-Kalman-Filters.ipynb
View File

@ -11,12 +11,14 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Multivariate Kalman Filters - Working with Multiple State Variables"
"# Multivariate Kalman Filters\n",
"\n",
"Working with Multiple State Variables"
]
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": 3,
"metadata": {
"collapsed": false
},
@ -26,18 +28,57 @@
"text/html": [
"<style>\n",
"@import url('http://fonts.googleapis.com/css?family=Source+Code+Pro');\n",
"@import url('http://fonts.googleapis.com/css?family=Open+Sans');\n",
"@import url('http://fonts.googleapis.com/css?family=Vollkorn');\n",
"@import url('http://fonts.googleapis.com/css?family=Karla');\n",
"@import url('http://fonts.googleapis.com/css?family=Poppins');\n",
"@import url('http://fonts.googleapis.com/css?family=Arimo');\n",
"@import url('http://fonts.googleapis.com/css?family=Fira+sans');\n",
"@import url('http://fonts.googleapis.com/css?family=Roboto');\n",
"@import url('http://fonts.googleapis.com/css?family=Lato');\n",
"@import url('http://fonts.googleapis.com/css?family=Domine');\n",
"@import url('http://fonts.googleapis.com/css?family=Chivo');\n",
"@import url('http://fonts.googleapis.com/css?family=Cardo');\n",
"@import url('http://fonts.googleapis.com/css?family=Arvo');\n",
"@import url('http://fonts.googleapis.com/css?family=Crimson+Text');\n",
"@import url('http://fonts.googleapis.com/css?family=Ubuntu');\n",
"@import url('http://fonts.googleapis.com/css?family=Fontin');\n",
"@import url('http://fonts.googleapis.com/css?family=Lora');\n",
"@import url('http://fonts.googleapis.com/css?family=Raleway');\n",
"@import url('http://fonts.googleapis.com/css?family=Merriweather');\n",
"\n",
"\n",
".CodeMirror pre {\n",
" font-family: 'Source Code Pro', Consolas, monocco, monospace;\n",
"}\n",
" div.cell{\n",
" width: 900px;\n",
" width: 800px;\n",
" margin-left: 0% !important;\n",
" margin-right: auto;\n",
" }\n",
" div.text_cell_render{\n",
" //font-family: 'Open Sans';\n",
" //font-family: 'Karla', verdana,arial,sans-serif;\n",
" //font-family: 'Roboto', verdana,arial,sans-serif;\n",
" //font-family: 'Lato', verdana,arial,sans-serif;\n",
" //font-family: 'Domine', verdana,arial,sans-serif;\n",
" //font-family: 'Chivo', verdana,arial,sans-serif;\n",
" //font-family: 'Cardo', verdana,arial,sans-serif;\n",
" //font-family: 'Arvo', verdana,arial,sans-serif;\n",
" //font-family: 'Ubuntu';\n",
" //font-family: 'Fontin';\n",
" font-family: 'Lora';\n",
" //font-family: 'Raleway';\n",
" //font-family: 'Merriweather';\n",
" //font-family: 'Crimson Text', verdana,arial,sans-serif;\n",
" //font-family: verdana,arial,sans-serif;\n",
" //font-family: arial,sans-serif;\n",
" line-height: 130%;\n",
" font-size: 130%;\n",
" //font-weight: 800;\n",
" text-align: justify;\n",
" text-justify:inter-word;\n",
" //font-family: 'Poppins', verdana,arial,sans-serif; \n",
" }\n",
" div.text_cell code {\n",
" background: transparent;\n",
" color: #000000;\n",
@ -122,13 +163,6 @@
" display: block;\n",
" white-space: nowrap;\n",
" }\n",
" div.text_cell_render{\n",
" /*font-family: 'Vollkorn', verdana,arial,sans-serif;*/\n",
" line-height: 150%;\n",
" font-size: 130%;\n",
" text-align: justify;\n",
" text-justify:inter-word;\n",
" }\n",
" div.output_subarea.output_text.output_pyout {\n",
" overflow-x: auto;\n",
" overflow-y: scroll;\n",
@ -261,7 +295,7 @@
"<IPython.core.display.HTML object>"
]
},
"execution_count": 1,
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}

7
book_format.py
View File

@ -41,10 +41,10 @@ def test_filterpy_version():
import filterpy
from distutils.version import LooseVersion
v = filterpy.__version__
min_version = "0.1.1"
if LooseVersion(v) < LooseVersion(min_version):
if LooseVersion(v) < LooseVersion(min_version):
raise Exception("Minimum FilterPy version supported is {}.\n"
"Please install a more recent version.\n"
" ex: pip install filterpy --upgrade".format(
@ -133,7 +133,8 @@ def load_style(directory = '.', name='code/custom.css'):
style.pop("axes.color_cycle", None)
plt.rcParams.update(style)
reset_axis ()
np.set_printoptions(suppress=True)
np.set_printoptions(suppress=True,precision=3, linewidth=70,
formatter={'float':lambda x:' {:.3}'.format(x)})
styles = open(os.path.join(directory, name), 'r').read()
return HTML(styles)

58
code/custom.css
View File

@ -1,23 +1,62 @@
<style>
@import url('http://fonts.googleapis.com/css?family=Source+Code+Pro');
@import url('http://fonts.googleapis.com/css?family=Vollkorn');
@import url('http://fonts.googleapis.com/css?family=Arimo');
@import url('http://fonts.googleapis.com/css?family=Fira+sans');
@import url('http://fonts.googleapis.com/css?family=Lora');
//@import url('http://fonts.googleapis.com/css?family=Open+Sans');
//@import url('http://fonts.googleapis.com/css?family=Vollkorn');
//@import url('http://fonts.googleapis.com/css?family=Karla');
//@import url('http://fonts.googleapis.com/css?family=Poppins');
//@import url('http://fonts.googleapis.com/css?family=Arimo');
//@import url('http://fonts.googleapis.com/css?family=Roboto');
//@import url('http://fonts.googleapis.com/css?family=Lato');
//@import url('http://fonts.googleapis.com/css?family=Domine');
//@import url('http://fonts.googleapis.com/css?family=Chivo');
//@import url('http://fonts.googleapis.com/css?family=Cardo');
//@import url('http://fonts.googleapis.com/css?family=Arvo');
//@import url('http://fonts.googleapis.com/css?family=Crimson+Text');
//@import url('http://fonts.googleapis.com/css?family=Ubuntu');
//@import url('http://fonts.googleapis.com/css?family=Fontin');
//@import url('http://fonts.googleapis.com/css?family=Raleway');
//@import url('http://fonts.googleapis.com/css?family=Merriweather');
.CodeMirror pre {
font-family: 'Source Code Pro', Consolas, monocco, monospace;
}
div.cell{
width: 900px;
width: 800px;
margin-left: 0% !important;
margin-right: auto;
}
div.text_cell_render{
font-family: 'Lora';
//font-family: 'Open Sans';
//font-family: 'Karla',verdana,arial,sans-serif;
//font-family: 'Roboto',verdana,arial,sans-serif;
//font-family: 'Lato',verdana,arial,sans-serif;
//font-family: 'Domine',verdana,arial,sans-serif;
//font-family: 'Chivo',verdana,arial,sans-serif;
//font-family: 'Cardo',verdana,arial,sans-serif;
//font-family: 'Arvo',verdana,arial,sans-serif;
//font-family: 'Poppins',verdana,arial,sans-serif;
//font-family: 'Ubuntu',verdana,arial,sans-serif;
//font-family: 'Fontin',verdana,arial,sans-serif;
//font-family: 'Raleway',verdana,arial,sans-serif;
//font-family: 'Merriweather',verdana,arial,sans-serif;
//font-family: 'Crimson Text', verdana,arial,sans-serif;
//font-family: verdana,arial,sans-serif;
//font-family: arial,sans-serif;
line-height: 125%;
font-size: 130%;
text-align: justify;
text-justify:inter-word;
}
div.text_cell code {
background: transparent;
color: #000000;
font-weight: 600;
font-weight: 400;
font-size: 12pt;
font-style: bold;
//font-style: bold;
font-family: 'Source Code Pro', Consolas, monocco, monospace;
}
h1 {
@ -96,13 +135,6 @@
display: block;
white-space: nowrap;
}
div.text_cell_render{
/*font-family: 'Vollkorn', verdana,arial,sans-serif;*/
line-height: 150%;
font-size: 130%;
text-align: justify;
text-justify:inter-word;
}
div.output_subarea.output_text.output_pyout {
overflow-x: auto;
overflow-y: scroll;

50
code/kf_internal.py
View File

@ -17,7 +17,12 @@ from __future__ import (absolute_import, division, print_function,
unicode_literals)
import book_plots as bp
import filterpy.stats as stats
from math import sqrt
import matplotlib.pyplot as plt
import numpy as np
from numpy.random import randn, seed
def plot_dog_track(xs, dog, measurement_var, process_var):
N = len(xs)
@ -42,4 +47,47 @@ def print_variance(positions):
print('Variance:')
for i in range(0, len(positions), 5):
print('\t{:.4f} {:.4f} {:.4f} {:.4f} {:.4f}'.format(
*[v[1] for v in positions[i:i+5]]))
*[v[1] for v in positions[i:i+5]]))
def gaussian_vs_histogram():
seed(15)
xs = np.arange(0, 20, 0.1)
ys = np.array([stats.gaussian(x-10, 0, 2) for x in xs])
bar_ys = abs(ys + randn(len(xs)) * stats.gaussian(xs-10, 0, 10)/2)
plt.gca().bar(xs[::5]-.25, bar_ys[::5], width=0.5, color='g')
plt.plot(xs, ys, lw=3, color='k')
plt.xlim(5, 15)
class DogSimulation(object):
def __init__(self, x0=0, velocity=1,
measurement_var=0.0,
process_var=0.0):
""" x0 : initial position
velocity: (+=right, -=left)
measurement_var: variance in measurement m^2
process_var: variance in process (m/s)^2
"""
self.x = x0
self.velocity = velocity
self.meas_std = sqrt(measurement_var)
self.process_std = sqrt(process_var)
def move(self, dt=1.0):
"""Compute new position of the dog in dt seconds."""
dx = self.velocity + randn()*self.process_std
self.x += dx * dt
def sense_position(self):
""" Returns measurement of new position in meters."""
measurement = self.x + randn()*self.meas_std
return measurement
def move_and_sense(self):
""" Move dog, and return measurement of new position in meters"""
self.move()
return self.sense_position()