daviddoji/Kalman-and-Bayesian-Filters-in-Python
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Moved statistics math to Gaussians chapter.

Browse Source
This commit is contained in:
Roger Labbe 2015-11-21 12:56:56 -08:00
parent 5ae739a2c6
commit 2455d797be
1 changed files with 10 additions and 75 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

85
07-Kalman-Filter-Math.ipynb
View File

@ -120,8 +120,9 @@
" }\n",
" div.text_cell_render{\n",
" font-family: 'Fira sans', verdana,arial,sans-serif;\n",
" line-height: 125%;\n",
" font-size: 115%;\n",
" line-height: 150%;\n",
" font-size: 110%;\n",
" font-weight: 400;\n",
" text-align:justify;\n",
" text-justify:inter-word;\n",
" }\n",
@ -234,7 +235,7 @@
"<script>\n",
" MathJax.Hub.Config({\n",
" TeX: {\n",
" extensions: [\"AMSmath.js\"]\n",
" extensions: [\"AMSmath.js\", \"autobold.js\"]\n",
" },\n",
" tex2jax: {\n",
" inlineMath: [ ['$','$'], [\"\\\\(\",\"\\\\)\"] ],\n",
@ -287,7 +288,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Computing Means and Covariances\n",
"## Computing Covariances\n",
"\n",
"You will not need to do this by hand for the rest of the book, but it is important to understand the computations behind these values. It only takes a few minutes to learn."
]
@ -296,72 +297,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Computing the Mean\n",
"\n",
"So let's start with 3 variables: $a, b, c$. Let's say we have two values for each: $a = [1, 3]$, $b = [8, 7]$, and $c = [3,7]$. Another, equivalent way of writing this is that you will sometimes see is\n",
"\n",
"$$\\mathbf{x} = \\begin{bmatrix}a\\\\b\\\\c\\end{bmatrix} = \\begin{bmatrix}1 & 3\\\\8&7\\\\3&7\\end{bmatrix}$$\n",
"\n",
"You will not see that often in the Kalman filter literature, so I will adopt the alternative notation of using a subscript to indicate the $i^{th}$ value for $\\mathbf{x}$\n",
"\n",
"$$\\mathbf{x}_0 = \\begin{bmatrix}1 \\\\8\\\\3\\end{bmatrix},\\ \\ \\mathbf{x}_1 = \\begin{bmatrix}3\\\\7\\\\7\\end{bmatrix}$$\n",
"\n",
"We compute the mean as for the univariate case - sum the values and divide by the number of values. Formally\n",
"\n",
"$$ \\mu_x = \\frac{1}{n}\\sum^n_{i=1} x_i$$\n",
"\n",
"Therefore we can write:\n",
"\n",
"$$\\begin{aligned}\n",
"\\mu_a &= \\frac{1+3}{2} = 2 \\\\\n",
"\\mu_b &= \\frac{8+7}{2} = 7.5 \\\\\n",
"\\mu_c &= \\frac{3+7}{2} = 5 \n",
"\\end{aligned}$$\n",
"\n",
"Which I can express in our shorthand matrix notation as\n",
"\n",
"$$\\mu_x = \\begin{bmatrix}2\\\\7.5\\\\5\\end{bmatrix}$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Expected Value\n",
"\n",
"I could just give you the formula for the covariance but it will make more sense if you see how it is derived. To do that we first need to talk about **expected value** of a random variable. The expected value is the value we expect, on average, for the variable. \n",
"\n",
"The expected value of a random variable is the average value it would have if we took an infinite number of samples of it and then averaged those samples together. Let's say we have $x=[1,3,5]$ and each value is equally probable. What would we *expect* $x$ to have, on average?\n",
"\n",
"It would be the average of 1, 3, and 5, of course, which is 3. That should make sense; we would expect equal numbers of 1, 3, and 5 to occur, so $(1+3+5)/3=3$ is clearly the average of that infinite series of samples.\n",
"\n",
"Now suppose that each value has a different probability of happening. Say 1 has an 80% chance of occurring, 3 has an 15% chance, and 5 has only a 5% chance. In this case we compute the expected value by multiplying each value of $x$ by the percent chance of it occurring, and summing the result. So for this case we could compute\n",
"\n",
"$$E[x] = (1)(0.8) + (3)(0.15) + (5)(0.05) = 1.5$$\n",
"\n",
"Here I have introduced the standard notation $E[x]$ for the expected value of $x$.\n",
"\n",
"We can formalize this by letting $x_i$ be the $i^{th}$ value of $x$, and $p_i$ be the probability of its occurrence. This gives us\n",
"\n",
"$$E[X] = \\sum_{i=1}^n p_ix_i$$\n",
"\n",
"The value 1.5 for $x$ makes intuitive sense because x is far more like to be 1 than 3 or 5, and 3 is more likely than 5 as well.\n",
"\n",
"It is probably worth me pointing out now that if $x$ is continuous we substitute the sum for an integral, like so\n",
"\n",
"$$E[X] = \\int_{-\\infty}^\\infty xf(x)$$\n",
"\n",
"where $f(x)$ is the probability distribution function of $x$. We won't be using this equation yet, but we will be using it in the next chapter."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Computing the Covariance\n",
"\n",
"\n",
"Now we are prepared to compute the covariance matrix. The **covariance** measures how much two random variables move in the same direction, and is defined as\n",
"The **covariance** measures how much two random variables move in the same direction, and is defined as\n",
"\n",
"$$ COV(x,y) = \\frac{1}{N}\\sum_{i=1}^N (x_i - \\mu_x)(y_i - \\mu_y)$$\n",
"\n",
@ -378,13 +314,12 @@
"\\end{aligned}$$\n",
"\n",
"\n",
"Let's build some familiarity with this by calculating the covariance for the $\\mathbf{x}$ from the beginning of this section.\n",
"\n",
"Let's build some familiarity with this by calculating the covariance for a random variable $X = [a\\, b\\, c]^\\mathsf{T}$ that has this sample space:\n",
"$$\\mathbf{x}_0 = \\begin{bmatrix}1 \\\\8\\\\3\\end{bmatrix},\\ \\ \\mathbf{x}_1 = \\begin{bmatrix}3\\\\7\\\\7\\end{bmatrix}$$\n",
"\n",
"We already know the mean is\n",
"The mean of multivariate variables is computed as the mean of each univariate variable:\n",
"\n",
"$$\\mu_x = \\begin{bmatrix}2\\\\7.5\\\\5\\end{bmatrix}$$"
"$$\\mu = \\begin{bmatrix}\\frac{1+3}{2}\\\\\\frac{8+7}{2}\\\\\\frac{3+7}{2}\\end{bmatrix} = \\begin{bmatrix}2\\\\7.5\\\\5\\end{bmatrix}$$"
]
},
{
@ -2545,7 +2480,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.4.1"
"version": "3.4.3"
}
},
"nbformat": 4,