daviddoji/Kalman-and-Bayesian-Filters-in-Python
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Discussion about Python codding

Browse Source
This commit is contained in:
Roger Labbe 2014-12-01 08:50:15 -08:00
parent f6537d34a8
commit b1ce657cca
1 changed files with 65 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

69
Preface.ipynb
View File

@ -1,7 +1,7 @@
{
"metadata": {
"name": "",
"signature": "sha256:c4084a00cf2f80c65a3076d8f0c1869b6306d5b9b381ab7361f0c2dbea37a240"
"signature": "sha256:a38663b5a933c3aae3021b22b188a3a9c7dddd72d2ad0fe3b95f0cf9872b620a"
},
"nbformat": 3,
"nbformat_minor": 0,
@ -329,7 +329,7 @@
"\n",
"> Beyond that the chapters are much more in a state of flux. Reader beware. My writing methodology is to just vomit out whatever is in my head, just to get material, and then go back and think through presentation, test code, refine, and so on. Whatever is checked in in these later chapters may be wrong and not ready for your use. \n",
"\n",
"> Finally, nothing has been spell checked or proof read yet. I with IPython Notebook had spell check, but it doesn't seem to.\n"
"> Finally, nothing has been spell checked or proof read yet. I with IPython Notebook had spell check, but it doesn't seem to."
]
},
{
@ -375,7 +375,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"** author's note**. *The book is still being written, and so I am not focusing on issues like supporting multipe versions of Python. I am staying more or less on the bleeding edge of Python 3 for the time being. If you follow my suggestion of installing Anaconda all off the versioning problems will be taken care of for you, and you will not alter or affect any existing installation of Python on your machine. I am aware that telling somebody to install a specific packaging system is not a long term solution, but I can either focus on endless regression testing for every minor code change, or work on delivering the book, and then doing one sweep through it to maximize compatibility. I opt for the latter. In the meantime I welcome bug reports if the book does not work on your platform.*\n",
"** author's note**. *The book is still being written, and so I am not focusing on issues like supporting multipe versions of Python. I am staying more or less on the bleeding edge of Python 3 for the time being. If you follow my suggestion of installing Anaconda all of the versioning problems will be taken care of for you, and you will not alter or affect any existing installation of Python on your machine. I am aware that telling somebody to install a specific packaging system is not a long term solution, but I can either focus on endless regression testing for every minor code change, or work on delivering the book, and then doing one sweep through it to maximize compatibility. I opt for the latter. In the meantime I welcome bug reports if the book does not work on your platform.*\n",
"\n",
"If you want to run the notebook on your computer, which is what I recommend, then you will have to have IPython installed. I do not cover how to do that in this book; requirements change based on what other python installations you may have, whether you use a third party package like Anaconda Python, what operating system you are using, and so on. \n",
"\n",
@ -421,7 +421,68 @@
"There is an undocumented directory called **exp**. This is where I write and test code prior to putting it in the book. There is some interesting stuff in there, and feel free to look at it. As the book evolves I plan to create examples and projects, and a lot of this material will end up there. Small experiments will eventually just be deleted. If you are just interested in reading the book you can safely ignore this directory. \n",
"\n",
"\n",
"The directory **styles** contains a css file containing the style guide for the book. The default look and feel of IPython Notebook is rather plain. Work is being done on this. I have followed the examples set by books such as [Probabilistic Programming and Bayesian Methods for Hackers](http://nbviewer.ipython.org/github/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/blob/master/Chapter1_Introduction/Chapter1_Introduction.ipynb). I have also been very influenced by Professor Lorena Barba's fantastic work, [available here](https://github.com/barbagroup/CFDPython). I owe all of my look and feel to the work of these projects. \n"
"The directory **styles** contains a css file containing the style guide for the book. The default look and feel of IPython Notebook is rather plain. Work is being done on this. I have followed the examples set by books such as [Probabilistic Programming and Bayesian Methods for Hackers](http://nbviewer.ipython.org/github/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/blob/master/Chapter1_Introduction/Chapter1_Introduction.ipynb). I have also been very influenced by Professor Lorena Barba's fantastic work, [available here](https://github.com/barbagroup/CFDPython). I owe all of my look and feel to the work of these projects. "
]
},
{
"cell_type": "heading",
"level": 2,
"metadata": {},
"source": [
"Thoughts on Python and Coding Math"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"I am first and foremost a programmer. Most Kalman filtering and other engineering texts are written by mathematicians or engineers. As a result, the software is usually not production quality. I will take Paul Zarchan's book *Fundamentals of Kalman Filtering* as an example. This is a fantastic book and it belongs in your library. But the code is Fortran listing without any subroutines beyond calls to functions like `MATMUL`. This means that Kalman filters are reimplemented over and over again throughout the book. The same listing mixes simulation code with filtering code, so until you become aware of the author's naming style it can be difficult to ascertain what code is the filter and what code is the simulation. Some chapters implement the same filter in subtley different ways, and uses bold text to highlight the few lines that changed. If he needs to use Runge Kutta, that is just embedded in the code, without comments. \n",
"\n",
"There's a better way. If I want to perform an SVD I call `svd`, I do not embed an svd implementation in my code. This buys me several things. First, I don't have to reimplement SVD multiple times. I don't have to debug SVD several times, and if I do find a bud, I can fix it once and be assured that it now works across all my different projects. Finally, it is readable. It is rare that I care about the implementation of SVD in my projects.\n",
"\n",
"Now, this is a textbook on Kalman filtering, and you could reasonably point out that we *do* care about the implementation of Kalman filters. To an extent that is true, but as you will find out the code that performs the filtering amounts to 7 or so lines of code. The code to implement the math is fairly trivial. Most of the work that Kalman filters requires is the design of the matrices that get fed into the math engine. So that is how I have structured the code.\n",
"\n",
"For example, there is a class named `KalmanFilter` which implements the linear algebra for performing kalmaning filtering. To use it you will construct an object of that class, initialize various parameters, then enter a while loop where you call `KalmanFilter.predict()` and `KalmanFilter.update()` to incorporate your measurements. Thus most of your programs will be only 20-50 lines, most of that boilerplate - setting up the matrices, and then plotting and/or using the results. The possible downside of that is that the equations that perform the filtering are hidden behind functions, which we could argue is a loss in a pedagogical text. I argue the converse. I want you to learn how to use Kalman filters in the real world, for real projects, and you shouldn't be cutting and pasting established algorithms all over the place. If you want to use ode45 (Runga Kutta) you call that function, you don't reimplement it from scratch. We will do the same here.\n",
"\n",
"However, it is undeniable that you will be a bit less versed with the equations of the Kalman filter as a result. I strongly recommend looking at the source for FilterPy, and maybe even implementing your own filter from scratch to be sure you understand the concepts.\n",
"\n",
"I use a fair number of classes in FilterPy. I do not use inheritence or virtual functions or any of that sort of OO design. I use classes as a way to organize the data that the filters require. For example, the `KalmanFilter` class mentioned above stores matrices called `x`, `P`, `R`, `Q`, and more. I've seen procedural libraries for Kalman filters, and they require the programmer to maintain all of those matrices. This perhaps isn't so bad for a toy program, but start programming, say, a bank of Kalman filters and you will not enjoy having to manage all of those matrices and other associated data.\n",
"\n",
"A word on variable names. I am an advocate for descriptive variable names. `R` is not, normally, descriptive. `R` is the measurement noise covaraiance matrix, so I could reasonably call it `measurement_noise_covariance`, and I've seen libraries do that. I've chosen not to do that. Why? In the end, Kalman filtering is math. To write a Kalman filter you are going to have to start by sitting down with a piece of paper and doing some math. You will be writing normal algrebric equations. Also, every Kalman filter text and source on the web uses the same linear algebra equations. You cannot read about the Kalman filter without seeing\n",
"\n",
"$$ \\dot{\\mathbf{x}} = \\mathbf{Fx} + \\mathbf{Gu}$$\n",
"\n",
"in every source (a few sources use A and B instead of F and G). One of my goals in this book is to bring you to the point where you can read the original literature on Kalman filtering. I take an optimistic tone in this book - that Kalman filtering is easy to learn - and in many ways it is. However, for nontrivial problems the difficulty is not the implementation of the equations, but learning how to set up the equations so they solve your problem. In other words, every Kalman filter will implement $ \\dot{\\mathbf{x}} = \\mathbf{Fx} + \\mathbf{Gu}$; the difficult part is figuring out what to put in the matrices $\\mathbf{F}$ and $\\mathbf{G}$ to make your filter work for your problem. Vast amounts of work have been done to figure out how to apply Kalman filters in various domains, and it would be tragic to not be able to read the literature and avail yourself of this research. \n",
"\n",
"So, like it or not you will need to learn that $\\mathbf{F}$ is the *state transition matrix* and that $\\mathbf{R}$ is the *measurement noise covariance*. Once you know that the code will become readable, and until you know that all publications and web articles on Kalman filters will be inaccessible to you. \n",
"\n",
"Finally I feel that mathematical programming is somewhat different than regular programming; what is readable in one domain is not readable in another. `q = x + m` is opaque in a normal context. On the other hand, `x = .5*a*t**2 + v_0 * t + x_0` is to me the most readable way to write the Newtonian distance equation:\n",
"\n",
"$$ x = \\frac{1}{2}at^2 + v_0 t + x_0$$\n",
"\n",
"We could write it as \n",
"\n",
" distance = .5 * constant_acceleration * time_delta**2 + \n",
" initial_velocity * time_delta + initial_distance\n",
" \n",
"but I feel that obscures readability. This is surely debatable for this one equation; but most mathematical programs, and certainly Kalman filters, use systems of equations. I can most easily follow the code, and ensure that it does not have bugs, when it reads as close to the math as possible. Consider this equation taken from the Kalman filter:\n",
"\n",
"$$ K = PH^T[HPH^T + R]^{-1}$$\n",
"\n",
"My Python code for this would be\n",
"\n",
" K = dot3(P, H.T, inv(dot3(H,P,H.T) + R))\n",
" \n",
"It's already a bit hard to read because of the `dot` function calls (required because Python does not yet support an operator for matrix multiplication). But compare this to\n",
"\n",
" kalman_gain = dot3(apriori_state_covariance, measurement_function_transpose,\n",
" inverse (dot3(measurement_function, apriori_state_covariance,\n",
" measurement_function_transpose) +\n",
" measurement_noise_covariance))\n",
"\n",
"I grant you this version has more context, but I cannot reasonable glance at this and see what math it is implementing. In particular, the linear algebra $HPH^T$ is doing something very specific - multiplying P by H and its transverse is changing the *basis* of P. It is nearly impossible to see that the Kalman gain is just a ratio of one number divided by a second number which has been converted to a different basis. If you are not solid in linear algebra perhaps that statement does not convey a lot of information to you yet, but I assure you that $K = PH^T[HPH^T + R]^{-1}$ is saying something very succinctly. There are two key pieces of information here - we are taking a ratio, and we are converting the *basis* of a matrix. I can see that in my first Python line, I cannot see that in the second line. \n",
"\n",
"I will not *win* this argument, and some people will not agree with my naming choices. I will finish by stating, very truthfully, that I made two mistakes the first time I typed that second version and it took me awhile to find it. In any case, I aim for using the mathematical symbol names whenever possible, coupled with readable class and function names. So, it is `KalmanFilter.P`, not `KF.P` and not `KalmanFilter.apriori_state_covariance`. "
]
},
{