diff --git a/02-Discrete-Bayes.ipynb b/02-Discrete-Bayes.ipynb index 9412ed7..fcd2632 100644 --- a/02-Discrete-Bayes.ipynb +++ b/02-Discrete-Bayes.ipynb @@ -361,7 +361,7 @@ "\n", "I'm sure you've used probabilities before - as in \"the probability of rain today is 30%\". The last paragraph sounds like more of that. But Bayesian statistics was a revolution in probability because it treats probability as a belief about a single event. Let's take an example. I know that if I flip a fair coin infinitely many times I will get 50% heads and 50% tails. This is called [*frequentist statistics*](https://en.wikipedia.org/wiki/Frequentist_inference) to distinguish it from Bayesian statistics. Computations are based on the frequency in which events occur.\n", "\n", - "I flip the coin one more time and let it land. Which way do I believe it landed? Frequentist probability has nothing to say about that; it will merely state that 50% of coin flips land as heads. In some ways it is meaningless to to assign a probability to the current state of the coin. It is either heads or tails, we just don't know which. Bayes treats this as a belief about a single event - the strength of my belief or knowledge that this specific coin flip is heads is 50%.\n", + "I flip the coin one more time and let it land. Which way do I believe it landed? Frequentist probability has nothing to say about that; it will merely state that 50% of coin flips land as heads. In some ways it is meaningless to to assign a probability to the current state of the coin. It is either heads or tails, we just don't know which. Bayes treats this as a belief about a single event - the strength of my belief or knowledge that this specific coin flip is heads is 50%. Some object to the term \"belief\"; belief can imply holding something to be true without evidence. In this book it always is a measure of the strength of our knowledge. We'll learn more about this as we go.\n", "\n", "Bayesian statistics takes past information (the prior) into account. We observe that it rains 4 times every 100 days. From this I could state that the chance of rain tomorrow is 1/25. This is not how weather prediction is done. If I know it is raining today and the storm front is stalled, it is likely to rain tomorrow. Weather prediction is Bayesian.\n", "\n", @@ -1162,7 +1162,7 @@ { "data": { "text/html": [ - "" + "" ], "text/plain": [ "" @@ -1274,7 +1274,7 @@ "\n", "Lets say we get a reading of **door**, and suppose that testing shows that the sensor is 3 times more likely to be right than wrong. We should scale the probability distribution by 3 where there is a door. If we do that the result will no longer be a probability distribution, but we will learn how to correct that in a moment.\n", "\n", - "Let's look at that in Python code. Here I use the variable `z` to denote the measurement as that is a customary choice in the literature. " + "Let's look at that in Python code. Here I use the variable `z` to denote the measurement as that is a customary choice in the literature. I try to write this book from the perspective of a programmer who uses meaningful names, not a mathematician that uses one letter variables, but one goal I have is to have you reading the literature, so you need to become familiar with the naming conventions." ] }, { @@ -2059,7 +2059,7 @@ { "data": { "text/html": [ - "" + "" ], "text/plain": [ "" @@ -2123,9 +2123,13 @@ "from filterpy.discrete_bayes import normalize\n", "```\n", "\n", - "It is a bit odd to be talking about \"3 times as likely to be right as wrong\". We are working in probabilities, so let's specify the probability of the sensor being correct, and computing the scale factor from that. \n", + "It is a bit odd to be talking about \"3 times as likely to be right as wrong\". We are working in probabilities, so let's specify the probability of the sensor being correct, and computing the scale factor from that. The equation for that is\n", "\n", - "Also, the for loop is cumbersome. NumPy lets you index arrays with boolean arrays. You create a boolean array with logical operators:" + "$$scale = \\frac{prob_{correct}}{prob_{incorrect}} = \\frac{prob_{correct}} {1-prob_{correct}}$$\n", + "\n", + "\n", + "\n", + "Also, the `for` loop is cumbersome. NumPy lets you index arrays with boolean arrays. You create a boolean array with logical operators:" ] }, { @@ -2160,12 +2164,14 @@ "```python\n", "belief[hall==z] *= scale\n", "```\n", - "and only the elements which equal `z` will be multiplied by `scale`. This is roughly 100 times faster than the `for` loop because the operation is implemented in C, not Python." + "and only the elements which equal `z` will be multiplied by `scale`. This is roughly 100 times faster than the `for` loop because the operation is performed in C, not Python.\n", + "\n", + "Here is our improved version:" ] }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 10, "metadata": { "collapsed": false, "scrolled": true @@ -2947,7 +2953,7 @@ { "data": { "text/html": [ - "" + "" ], "text/plain": [ "" @@ -2981,9 +2987,13 @@ "source": [ " We can see from the output that the sum is now 1.0, and that the probability of a door vs wall is still three times larger. The result also fits our intuition that the probability of a door must be less than 0.333, and that the probability of a wall must be greater than 0.0. Finally, it should fit our intuition that we have not yet been given any information that would allow us to distinguish between any given door or wall position, so all door positions should have the same value, and the same should be true for wall positions.\n", " \n", - "This result is called the [*posterior*](https://en.wikipedia.org/wiki/Posterior_probability), which is short for *posterior probability distribution*. All this means is a probability distribution *after* incorporating the measurement information (posterior means 'after' in this context). To review, the *prior* is the probability distribution before including the measurement's information. Another term is the [*likelihood*](https://en.wikipedia.org/wiki/Likelihood_function) - this is the probability for the [*evidence*](https://en.wikipedia.org/wiki/Marginal_likelihood) (the measurement in this case) being true. That gives us this equation:\n", + "This result is called the [*posterior*](https://en.wikipedia.org/wiki/Posterior_probability), which is short for *posterior probability distribution*. All this means is a probability distribution *after* incorporating the measurement information (posterior means 'after' in this context). To review, the *prior* is the probability distribution before including the measurement's information. \n", "\n", - "$$\\mathtt{posterior} = \\frac{\\mathtt{prior}\\times \\mathtt{likelihood}}{\\mathtt{normalization}}$$ \n", + "Another term is the [*likelihood*](https://en.wikipedia.org/wiki/Likelihood_function). When we computed `belief[hall==z] *= scale` we were computing how *likely* each position was given the measurement. This is not a probability distribution because it does not sum to one. Likelihoods are not probability distributions. \n", + "\n", + "The combination of these gives us the equation\n", + "\n", + "$$\\mathtt{posterior} = \\frac{\\mathtt{likelihood} \\times \\mathtt{prior}}{\\mathtt{normalization}}$$ \n", "\n", "It is very important to learn and internalize these terms as most of the literature uses them extensively.\n", "\n", @@ -2992,7 +3002,7 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": 11, "metadata": { "collapsed": false }, @@ -3018,14 +3028,14 @@ " return normalize(likelihood * prior)\n", "```\n", "\n", - "Computation of the likelihood varies per problem. For example, the sensor might not return just 1 or 0, but a `float` between 0 and 1 indicating the probability of being in front of a door. It might use computer vision and report a blob shape that you then probabilistically match to a door. It might use sonar and return a distance reading. In each case the computation of the likelihood will be different. We will see many examples of this throughout the book.\n", + "Computation of the likelihood varies per problem. For example, the sensor might not return just 1 or 0, but a `float` between 0 and 1 indicating the probability of being in front of a door. It might use computer vision and report a blob shape that you then probabilistically match to a door. It might use sonar and return a distance reading. In each case the computation of the likelihood will be different. We will see many examples of this throughout the book, and learn how to perform these calculations.\n", "\n", "FilterPy implements `update`. Here is the previous example in a fully general form:" ] }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 12, "metadata": { "collapsed": false }, @@ -3037,7 +3047,7 @@ " 0.0625, 0.188, 0.0625])" ] }, - "execution_count": 13, + "execution_count": 12, "metadata": {}, "output_type": "execute_result" } @@ -3049,7 +3059,10 @@ " \"\"\" compute likelihood that a measurement matches\n", " positions in the hallway.\"\"\"\n", " \n", - " scale = z_prob / (1. - z_prob)\n", + " try:\n", + " scale = z_prob / (1. - z_prob)\n", + " except ZeroDivisionError:\n", + " scale = 1e8\n", "\n", " likelihood = np.ones(len(hall))\n", " likelihood[hall==z] *= scale\n", @@ -3067,11 +3080,825 @@ "\n", "Recall how quickly we were able to find an exact solution when we incorporated a series of measurements and movement updates. However, that occurred in a fictional world of perfect sensors. Might we be able to find an exact solution with noisy sensors?\n", "\n", - "Unfortunately, the answer is no. Even if the sensor readings perfectly match an extremely complicated hallway map, we cannot be 100% certain that the dog is in a specific position - there is, after all, the possibility that every sensor reading was wrong! Naturally, in a more typical situation most sensor readings will be correct, and we might be close to 100% sure of our answer, but never 100% sure. This may seem complicated, but lets go ahead and program the math.\n", + "Unfortunately, the answer is no. Even if the sensor readings perfectly match an extremely complicated hallway map, we cannot be 100% certain that the dog is in a specific position - there is, after all, a tiny possibility that every sensor reading was wrong! Naturally, in a more typical situation most sensor readings will be correct, and we might be close to 100% sure of our answer, but never 100% sure. This may seem complicated, but lets go ahead and program the math.\n", "\n", "First let's deal with the simple case - assume the movement sensor is perfect, and it reports that the dog has moved one space to the right. How would we alter our `belief` array?\n", "\n", - "I hope after a moment's thought it is clear that we should shift all the values one space to the right. If we previously thought there was a 50% chance of Simon being at position 3, then after the move to the right we should believe that there is a 50% chance he is at position 4. The hallway is circular, so we will use modulo arithmetic to perform the shift." + "I hope after a moment's thought it is clear that we should shift all the values one space to the right. If we previously thought there was a 50% chance of Simon being at position 3, then after he moved one position to the right we should believe that there is a 50% chance he is at position 4. The hallway is circular, so we will use modulo arithmetic to perform the shift." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "application/javascript": [ + "/* Put everything inside the global mpl namespace */\n", + "window.mpl = {};\n", + "\n", + "mpl.get_websocket_type = function() {\n", + " if (typeof(WebSocket) !== 'undefined') {\n", + " return WebSocket;\n", + " } else if (typeof(MozWebSocket) !== 'undefined') {\n", + " return MozWebSocket;\n", + " } else {\n", + " alert('Your browser does not have WebSocket support.' +\n", + " 'Please try Chrome, Safari or Firefox ≥ 6. ' +\n", + " 'Firefox 4 and 5 are also supported but you ' +\n", + " 'have to enable WebSockets in about:config.');\n", + " };\n", + "}\n", + "\n", + "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n", + " this.id = figure_id;\n", + "\n", + " this.ws = websocket;\n", + "\n", + " this.supports_binary = (this.ws.binaryType != undefined);\n", + "\n", + " if (!this.supports_binary) {\n", + " var warnings = document.getElementById(\"mpl-warnings\");\n", + " if (warnings) {\n", + " warnings.style.display = 'block';\n", + " warnings.textContent = (\n", + " \"This browser does not support binary websocket messages. \" +\n", + " \"Performance may be slow.\");\n", + " }\n", + " }\n", + "\n", + " this.imageObj = new Image();\n", + "\n", + " this.context = undefined;\n", + " this.message = undefined;\n", + " this.canvas = undefined;\n", + " this.rubberband_canvas = undefined;\n", + " this.rubberband_context = undefined;\n", + " this.format_dropdown = undefined;\n", + "\n", + " this.image_mode = 'full';\n", + "\n", + " this.root = $('
');\n", + " this._root_extra_style(this.root)\n", + " this.root.attr('style', 'display: inline-block');\n", + "\n", + " $(parent_element).append(this.root);\n", + "\n", + " this._init_header(this);\n", + " this._init_canvas(this);\n", + " this._init_toolbar(this);\n", + "\n", + " var fig = this;\n", + "\n", + " this.waiting = false;\n", + "\n", + " this.ws.onopen = function () {\n", + " fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n", + " fig.send_message(\"send_image_mode\", {});\n", + " fig.send_message(\"refresh\", {});\n", + " }\n", + "\n", + " this.imageObj.onload = function() {\n", + " if (fig.image_mode == 'full') {\n", + " // Full images could contain transparency (where diff images\n", + " // almost always do), so we need to clear the canvas so that\n", + " // there is no ghosting.\n", + " fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n", + " }\n", + " fig.context.drawImage(fig.imageObj, 0, 0);\n", + " };\n", + "\n", + " this.imageObj.onunload = function() {\n", + " this.ws.close();\n", + " }\n", + "\n", + " this.ws.onmessage = this._make_on_message_function(this);\n", + "\n", + " this.ondownload = ondownload;\n", + "}\n", + "\n", + "mpl.figure.prototype._init_header = function() {\n", + " var titlebar = $(\n", + " '
');\n", + " var titletext = $(\n", + " '
');\n", + " titlebar.append(titletext)\n", + " this.root.append(titlebar);\n", + " this.header = titletext[0];\n", + "}\n", + "\n", + "\n", + "\n", + "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n", + "\n", + "}\n", + "\n", + "\n", + "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n", + "\n", + "}\n", + "\n", + "mpl.figure.prototype._init_canvas = function() {\n", + " var fig = this;\n", + "\n", + " var canvas_div = $('
');\n", + "\n", + " canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n", + "\n", + " function canvas_keyboard_event(event) {\n", + " return fig.key_event(event, event['data']);\n", + " }\n", + "\n", + " canvas_div.keydown('key_press', canvas_keyboard_event);\n", + " canvas_div.keyup('key_release', canvas_keyboard_event);\n", + " this.canvas_div = canvas_div\n", + " this._canvas_extra_style(canvas_div)\n", + " this.root.append(canvas_div);\n", + "\n", + " var canvas = $('');\n", + " canvas.addClass('mpl-canvas');\n", + " canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n", + "\n", + " this.canvas = canvas[0];\n", + " this.context = canvas[0].getContext(\"2d\");\n", + "\n", + " var rubberband = $('');\n", + " rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n", + "\n", + " var pass_mouse_events = true;\n", + "\n", + " canvas_div.resizable({\n", + " start: function(event, ui) {\n", + " pass_mouse_events = false;\n", + " },\n", + " resize: function(event, ui) {\n", + " fig.request_resize(ui.size.width, ui.size.height);\n", + " },\n", + " stop: function(event, ui) {\n", + " pass_mouse_events = true;\n", + " fig.request_resize(ui.size.width, ui.size.height);\n", + " },\n", + " });\n", + "\n", + " function mouse_event_fn(event) {\n", + " if (pass_mouse_events)\n", + " return fig.mouse_event(event, event['data']);\n", + " }\n", + "\n", + " rubberband.mousedown('button_press', mouse_event_fn);\n", + " rubberband.mouseup('button_release', mouse_event_fn);\n", + " // Throttle sequential mouse events to 1 every 20ms.\n", + " rubberband.mousemove('motion_notify', mouse_event_fn);\n", + "\n", + " rubberband.mouseenter('figure_enter', mouse_event_fn);\n", + " rubberband.mouseleave('figure_leave', mouse_event_fn);\n", + "\n", + " canvas_div.on(\"wheel\", function (event) {\n", + " event = event.originalEvent;\n", + " event['data'] = 'scroll'\n", + " if (event.deltaY < 0) {\n", + " event.step = 1;\n", + " } else {\n", + " event.step = -1;\n", + " }\n", + " mouse_event_fn(event);\n", + " });\n", + "\n", + " canvas_div.append(canvas);\n", + " canvas_div.append(rubberband);\n", + "\n", + " this.rubberband = rubberband;\n", + " this.rubberband_canvas = rubberband[0];\n", + " this.rubberband_context = rubberband[0].getContext(\"2d\");\n", + " this.rubberband_context.strokeStyle = \"#000000\";\n", + "\n", + " this._resize_canvas = function(width, height) {\n", + " // Keep the size of the canvas, canvas container, and rubber band\n", + " // canvas in synch.\n", + " canvas_div.css('width', width)\n", + " canvas_div.css('height', height)\n", + "\n", + " canvas.attr('width', width);\n", + " canvas.attr('height', height);\n", + "\n", + " rubberband.attr('width', width);\n", + " rubberband.attr('height', height);\n", + " }\n", + "\n", + " // Set the figure to an initial 600x600px, this will subsequently be updated\n", + " // upon first draw.\n", + " this._resize_canvas(600, 600);\n", + "\n", + " // Disable right mouse context menu.\n", + " $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n", + " return false;\n", + " });\n", + "\n", + " function set_focus () {\n", + " canvas.focus();\n", + " canvas_div.focus();\n", + " }\n", + "\n", + " window.setTimeout(set_focus, 100);\n", + "}\n", + "\n", + "mpl.figure.prototype._init_toolbar = function() {\n", + " var fig = this;\n", + "\n", + " var nav_element = $('
')\n", + " nav_element.attr('style', 'width: 100%');\n", + " this.root.append(nav_element);\n", + "\n", + " // Define a callback function for later on.\n", + " function toolbar_event(event) {\n", + " return fig.toolbar_button_onclick(event['data']);\n", + " }\n", + " function toolbar_mouse_event(event) {\n", + " return fig.toolbar_button_onmouseover(event['data']);\n", + " }\n", + "\n", + " for(var toolbar_ind in mpl.toolbar_items) {\n", + " var name = mpl.toolbar_items[toolbar_ind][0];\n", + " var tooltip = mpl.toolbar_items[toolbar_ind][1];\n", + " var image = mpl.toolbar_items[toolbar_ind][2];\n", + " var method_name = mpl.toolbar_items[toolbar_ind][3];\n", + "\n", + " if (!name) {\n", + " // put a spacer in here.\n", + " continue;\n", + " }\n", + " var button = $('');\n", + " button.click(method_name, toolbar_event);\n", + " button.mouseover(tooltip, toolbar_mouse_event);\n", + " nav_element.append(button);\n", + " }\n", + "\n", + " // Add the status bar.\n", + " var status_bar = $('');\n", + " nav_element.append(status_bar);\n", + " this.message = status_bar[0];\n", + "\n", + " // Add the close button to the window.\n", + " var buttongrp = $('
');\n", + " var button = $('');\n", + " button.click(function (evt) { fig.handle_close(fig, {}); } );\n", + " button.mouseover('Stop Interaction', toolbar_mouse_event);\n", + " buttongrp.append(button);\n", + " var titlebar = this.root.find($('.ui-dialog-titlebar'));\n", + " titlebar.prepend(buttongrp);\n", + "}\n", + "\n", + "mpl.figure.prototype._root_extra_style = function(el){\n", + " var fig = this\n", + " el.on(\"remove\", function(){\n", + "\tfig.close_ws(fig, {});\n", + " });\n", + "}\n", + "\n", + "mpl.figure.prototype._canvas_extra_style = function(el){\n", + " // this is important to make the div 'focusable\n", + " el.attr('tabindex', 0)\n", + " // reach out to IPython and tell the keyboard manager to turn it's self\n", + " // off when our div gets focus\n", + "\n", + " // location in version 3\n", + " if (IPython.notebook.keyboard_manager) {\n", + " IPython.notebook.keyboard_manager.register_events(el);\n", + " }\n", + " else {\n", + " // location in version 2\n", + " IPython.keyboard_manager.register_events(el);\n", + " }\n", + "\n", + "}\n", + "\n", + "mpl.figure.prototype._key_event_extra = function(event, name) {\n", + " var manager = IPython.notebook.keyboard_manager;\n", + " if (!manager)\n", + " manager = IPython.keyboard_manager;\n", + "\n", + " // Check for shift+enter\n", + " if (event.shiftKey && event.which == 13) {\n", + " this.canvas_div.blur();\n", + " event.shiftKey = false;\n", + " // Send a \"J\" for go to next cell\n", + " event.which = 74;\n", + " event.keyCode = 74;\n", + " manager.command_mode();\n", + " manager.handle_keydown(event);\n", + " }\n", + "}\n", + "\n", + "mpl.figure.prototype.handle_save = function(fig, msg) {\n", + " fig.ondownload(fig, null);\n", + "}\n", + "\n", + "\n", + "mpl.find_output_cell = function(html_output) {\n", + " // Return the cell and output element which can be found *uniquely* in the notebook.\n", + " // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n", + " // IPython event is triggered only after the cells have been serialised, which for\n", + " // our purposes (turning an active figure into a static one), is too late.\n", + " var cells = IPython.notebook.get_cells();\n", + " var ncells = cells.length;\n", + " for (var i=0; i= 3 moved mimebundle to data attribute of output\n", + " data = data.data;\n", + " }\n", + " if (data['text/html'] == html_output) {\n", + " return [cell, data, j];\n", + " }\n", + " }\n", + " }\n", + " }\n", + "}\n", + "\n", + "// Register the function which deals with the matplotlib target/channel.\n", + "// The kernel may be null if the page has been refreshed.\n", + "if (IPython.notebook.kernel != null) {\n", + " IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n", + "}\n" + ], "text/plain": [ - "" + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "" + ], + "text/plain": [ + "" ] }, "metadata": {}, @@ -10484,7 +12062,7 @@ }, { "cell_type": "code", - "execution_count": 24, + "execution_count": 25, "metadata": { "collapsed": false, "scrolled": true @@ -10492,9 +12070,775 @@ "outputs": [ { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtkAAACrCAYAAACzOIB9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtYlGX+P/D3PAPIcBAPBKh4gszDYmQMkUIbntekzTI8\ntSKNpqymInZQ0l0LUXNNChO8ysLTZWrpVp6wUfFA5AZ4WHHxjPo1B0pREPMAeP/+8MfkCMoAzzMD\nzPt1XVzJPfczn3uA3nx45pl7VEIIASIiIiIiko1k7QUQERERETU2bLKJiIiIiGTGJpuIiIiISGZs\nsomIiIiIZMYmm4iIiIhIZmyyiYiIiIhkxiabbE6HDh3g4+Nj7WUQEVEDEBkZCUmScOHCBWsvhRoY\nNtlkc1QqFVQqlbWXQURENTRnzhxIkoRVq1ZZrCZ/Z1Btsckmm7N7927s3LnT2ssgIqIaskbDu2DB\nAuTm5qJNmzYWrUsNn521F0BkaR07drT2EoiIqBas8SbVnp6e8PT0tHhdavh4JpvqtfPnz0OSJPTp\n0weXLl3Ca6+9Bg8PDzg5OSEwMBAbNmwwmb93715IkgSdToeTJ08iPDwcHh4eUKvV+O9//wvg4ddk\nl5aW4l//+hd69OgBZ2dnNG3aFD179kRKSkqVa5MkCT4+PiguLkZ0dDQ6dOgAe3t7JCYmyv+FICKy\nkprmcIXly5ejZ8+eaNq0KZydnfH0009j8eLFKCsrqzT36NGjGDVqFHx8fKDRaPDYY4/B398fkyZN\nwvXr1wEAvXv3xgcffADgj+ukJUmCWq02uV5aCIHly5cjJCQEzZo1g0ajQffu3bFgwQKUlpZWql1d\nlj/qmuxNmzahd+/exjrdunXDP//5T9y4caPS3NDQUEiShHPnzuGTTz5B9+7dodFo8Morr5jxXaCG\niGeyqUG4evUqgoOD0aJFC4wdOxZXr17Fhg0bMGLECBgMBkydOtVk/qlTpxAUFIRu3bohIiICxcXF\ncHJyAoAqn2osKyvDX/7yF6SlpaFz586YOHEi7ty5g02bNmHs2LH48ccfsXz58krH3b59G3369EFR\nURFeeOEFODk5wdvbW5kvAhGRFdUkhyMiIrBmzRp4e3tDp9PB3t4emzdvxltvvQW9Xo+tW7dCku6d\n5zt69CiCgoIgSRLCwsLg6+uLkpIS5OXlYeXKlZg+fTpcXV3x+uuvAwD27duHIUOG4KmnngJwL9Ob\nNWsGACgvL8fLL7+MLVu2oHPnzhg1ahQcHR2xd+9exMbGYvfu3UhNTTXWrvCoLH/YJSr/+Mc/MHfu\nXLRs2RIjR45Es2bNoNfrERcXh82bN2P//v1wdnY2zq+4n8mTJyMjIwODBw9GWFgYXF1dZfwuUb0i\niOqxc+fOCZVKJSRJEiNHjjS57ezZs6JZs2bC0dFRXLhwQQghxJ49e4zzZ8+eXeV9dujQQXTs2NFk\nbMGCBUKlUolBgwaJ0tJS43hxcbHw8/MTkiSJjRs3mhxTUWfgwIHi5s2bcjxcIqJ6p6Y5vG7dOqFS\nqUSPHj1ESUmJcW5paano27evkCRJfPTRR8bx6dOnC0mSxPfff1+pdklJibhz547x8zlz5ghJksTK\nlSurXGtcXJxQqVRiypQp4u7duya3TZgwQUiSJD799FOT8eqyPDIyUkiSJM6fP28cO3DggFCpVKJd\nu3YiPz/fZP6YMWOEJEli8uTJJuOhoaFCpVIJb29v49eKGjdeLkINglqtxvz5803GOnbsaDzjvGbN\nGpPbPD09MXv2bLPvf/ny5VCpVFi8eDHs7P54gsfV1RXz58+HEAKff/55lccuWrQIjo6ONXg0REQN\nj7k5XJGn8+fPNzmTa2dnh4SEhIfmaVU56uzsDHt7e7PWJ4RAYmIiPD09kZCQUOns84cffggAWL16\ndZXH1yTLKx5jbGxspeu1Fy5cCEdHR6xYsQLl5eUmt6lUKrz77rto27atWXWoYePlItQgtGvXDu3b\nt680/vzzz2P+/Pk4dOiQybi/v7/ZwVxSUoIzZ87Ay8sLXbp0qXR73759AQAHDx6sdJujoyP8/PzM\nqkNE1JCZm8MV/w0NDa00t3v37vDw8MDJkyfx+++/w8nJCcOHD8cnn3yCl156CUOHDkXfvn3Rs2dP\ndO7cuUbrO3nyJC5fvozHH38ccXFxlW4XQkCj0SA3N7fSbTXN8orH2Lt370q3eXh4oHv37sjMzMTJ\nkyfRtWtXk9sDAwPNrkMNG5tsahAe9sruivGioiKTcS8vL7Pvu+LYhx2j0Wjg5uaGa9euVbrNw8PD\n7DpERA2ZuTlcVFQENzc3NGnSpMr5rVq1wm+//YaioiLjiyd//PFHzJs3D99++y3Wrl0LIQQ6dOiA\nd999FxMmTDBrfVeuXAEAnDlzxvgCyapUdX11TbO8ut8brVq1AoAqf2/U5PcTNWy8XIQahIKCgkeO\nu7m5mYzXZB/VimPz8/OrvP3mzZvGXxoP4hsUEJGtMDeH3dzcUFRUhNu3b1c532AwmMwHgGeeeQbf\nfvstrl69igMHDmDu3Lm4desWJk6cWOlywIepuL8XX3wR5eXlD/2oaneTmmZ5db83qnqMta1FDReb\nbGoQLly4UOX2SXv27AEAPP3007W+bxcXFzz++OMoKCjA8ePHK92+a9cuAIBWq611DSKihs7cHK74\nb8X4/XJycvDrr7+ic+fOxh2f7mdnZ4fAwEDExsZi5cqVEELg3//+t/F2tVoNAJWudQaALl26oFmz\nZvj555+rbKTlVPEY09LSKt3222+/IScnB87OzjW+5IUaFzbZ1CCUl5fj3XffNXkjgjNnziA5ORkO\nDg4YNWpUne5/7NixEEJg+vTpJuFcXFyM2NhYqFQqjBs3rk41iIgaMnNzuCJPY2NjTfaLLisrQ0xM\nTKU8/emnn3Dr1q1K9SrOBt//4smWLVtCCFFls69WqzF16lTk5+dj4sSJuHnzZqU5V65cwZEjR2rx\n6E3pdDoIITBv3rxKZ/jfeecd/P7774iMjDT+UUC2iddkU4Pw5JNP4j//+Q+0Wi0GDBiAK1eu4Ouv\nv0ZxcTESEhLQrl27Ot1/TEwMUlNTkZqaiu7duyMsLAylpaXYuHEjLl26hDFjxvANA4jIppmbw8OH\nD8fmzZvx1VdfoVu3bnj55ZeN+2SfOnUK/fr1Q3R0tPF+Fy5ciF27duG5555Dx44d0bRpU5w8eRJb\ntmyBs7Ozyf7bffr0gSRJ+Pjjj3H58mXj9c1TpkyBq6srZs2ahZycHHzxxRfYunUr+vbtC29vb/z2\n2284c+YM0tPT8eabb2Lx4sV1+lo8++yzmDlzJhYsWAA/Pz+Eh4fDzc0Ner0eBw8ehL+/P+bNm1en\nGtQImLvX39KlS0XHjh2Fo6OjCAgIEPv373/o3Io9Ne//kCRJ7Nixo9Z7DZJtqvhZ6t27t/jll1/E\n3/72N+Hh4SE0Go0IDAwUGzZsMJm/Z88eIUmS0Ol0D73PDh06CB8fn0rjd+7cEQsXLhT+/v7CyclJ\nuLi4iGeffVZ8+eWXVd6PSqWq8n6IGhrmOz1KTXO4wmeffSaCgoKEi4uLcHJyEk899ZT46KOPTN6L\nQAgh9Hq9GDt2rPDz8xPNmzcXzs7O4oknnhATJkwQJ06cqHS/69atE1qtVjg7OwtJkirtYV0xZ+DA\ngcLd3V00adJEtG7dWvTs2VO8//774vTp0yZzq8vyyMhIoVarK9UQQohvvvlGhIaGCjc3N+Ho6Ci6\ndu0q/vGPf5jsD14hNDT0ofdDjZNKiPue93mI9evXY/To0Vi2bBmCg4OxdOlSpKSkIDc3t8p3tzt/\n/jx8fHywY8cOPPnkk8bxFi1amOxBTFSd8+fPo2PHjggNDcXu3butvRyiRof5TtVhDhPVjlnXZCck\nJECn00Gn06Fz585ITExEq1atkJyc/NBjhBBo0aIFPDw8jB8MYCKi+oX5TkSkjGqb7NLSUmRnZ6N/\n//4m4wMGDEBGRsYjj33llVfg6emJkJAQbNy4sW4rJSIiWTHfiYiUU+2ph8uXL6O8vLzSJvSenp7G\nrc0e5OLigo8++gjBwcGws7PDd999h+HDh2PVqlWVdoF48E1EiO5XXFwMlUqF8vJy/qxQo1DVvrnW\nwnwnczCHiapXVbYr8vxey5YtMW3aNOPnTz/9NK5cuYKFCxfWeas1si3t2rVDYWGhtZdBRP8f8932\nMIeJaqfay0Xc3d2hVqsr7QNZUFBQo7cGfeaZZ3Dq1Kmar5CIiBTBfCciUk61Z7Lt7e0REBAAvV6P\noUOHGsf1ej3Cw8PNLnTo0CG0atXqkXPMfRo1KysLgHXegc+ata1d31ZrW7u+rda2dn05a9fXp9iZ\n76xdH+rbam1r17fV2nLWry7bzbpcJCYmBhEREQgMDERwcDCSk5NhMBgQFRUFAJg5cyYyMzOxc+dO\nAMCqVatgb2+PHj16QJIkfP/990hOTsbChQvr9GCIiEhezHciImWY1WQPGzYMhYWFiI+Ph8FggJ+f\nH7Zv327cQzU/Px95eXkmx8ydOxcXLlyAWq3GE088gZSUFIwcOVL+R0BERLXGfCciUobZL3yMiooy\nntl4UEpKisnnERERiIiIqNvKiIjIIpjvRETyM+vNaIiIiIiIyHxssomIiIiIZMYmm4iIiIhIZmyy\niYiIiIhkxiabiIiIiEhmbLKJiIiIiGTGJpuIiIiISGZssomIiIiIZMYmm4iIiIhIZmyyiYiIiIhk\nxiabiIiIiEhmbLKJiIiIiGTGJpuIiIiISGZmN9lJSUnw8fGBRqOBVqtFenq6WcedOnUKrq6uaNq0\naa0XSUREymG+ExHJz6wme/369YiOjsasWbNw+PBh9OrVC4MGDcLFixcfeVxpaSlGjhyJ0NBQOdZK\nREQyY74TESnDrCY7ISEBOp0OOp0OnTt3RmJiIlq1aoXk5ORHHvfOO+/A398fr776qiyLJSIieTHf\niYiUUW2TXVpaiuzsbPTv399kfMCAAcjIyHjocVu3bsW2bduwZMmSuq+SiIhkx3wnIlKOXXUTLl++\njPLycnh6epqMe3p6YteuXVUec+nSJYwfPx7fffcdnJyczF5MVlaW2XNrM19O1qxt7fq2Wtva9W21\ntrXry1G7U6dOMqxEfsx31q5P9W21trXr22ptOepXl+2K7C4yevRoTJw4EVqtFgAghFCiDBERWRjz\nnYjIPNWeyXZ3d4darUZBQYHJeEFBAby8vKo8Ji0tDfv378ecOXMA3Avhu3fvwsHBAUlJSRg3blyV\nx1WEdnUq/vIwd76crFnb2vVttba169tqbWvXl7N2UVFRne9DCcx31q4P9W21trXr22ptOetXl+3V\nNtn29vYICAiAXq/H0KFDjeN6vR7h4eFVHpOTk2Py+bfffot58+YhMzMTrVu3NmfdRESkMOY7EZFy\nqm2yASAmJgYREREIDAxEcHAwkpOTYTAYEBUVBQCYOXMmMjMzsXPnTgBAt27dTI7PzMyEJEno2rWr\nzMsnIqK6YL4TESnDrCZ72LBhKCwsRHx8PAwGA/z8/LB9+3Z4e3sDAPLz85GXl6foQomISH7MdyIi\nZZjVZANAVFSU8czGg1JSUh557JgxYzBmzJiarYyIiCyC+U5EJD9FdhchIiIiIrJlbLKJiIiIiGTG\nJpuIiIiISGZssomIiIiIZGb2Cx+JiIioYThz+ToMN8rMnn/d0QMAkH7+qtnHtHK2g6+7a43XRmQr\n2GQTERE1MoYbZZi873Itjrxt9swlf3aHr3stShDZCF4uQkREREQkMzbZREREREQyY5NNRERERCQz\nNtlERERERDJjk01EREREJDM22UREREREMmOTTUREREQkM7Ob7KSkJPj4+ECj0UCr1SI9Pf2hc3Nz\nc9GnTx94eXlBo9HA19cX7733HkpLS2VZNBERyYf5TkQkP7PejGb9+vWIjo7GsmXLEBwcjKVLl2LQ\noEHIzc2Ft7d3pfkODg6IjIxEjx490KxZMxw5cgTjxo1DeXk5FixYIPuDICKi2mG+ExEpw6wmOyEh\nATqdDjqdDgCQmJiI1NRUJCcnIz4+vtJ8X19f+Pr6Gj9v27YtXnvtNezfv1+mZRMRkRyY70REyqj2\ncpHS0lJkZ2ejf//+JuMDBgxARkaGWUVOnz6N1NRUhIaG1mqRREQkP+Y7EZFyVEII8agJBoMBbdq0\nwb59+xASEmIcj4uLw9q1a5Gbm/vQY4ODg3Hw4EHcuXMHb7zxBpYtW1ZpTlFRkfHfp06dqs1joFoq\nc26Bq+VmPZlRa83VZbC7UahoDaL6rlOnTsZ/u7m5WXElppjvjddvjh6IPXRb0RrzejTBY7d+VbQG\nUX1WXbYr2mFt2LAB169fx5EjR/DWW29hwYIFmDFjhpIlqQaulttZJoQVrUBE1sB8JyJ6tGqbbHd3\nd6jVahQUFJiMFxQUwMvL65HHtmnTBgDQpUsXlJWVYdy4cXjnnXcgSVVfpaLVas1adFZWVo3my8ma\nteWun37+KgBlm2xXV1do/drV+X4a09edtRtGfTlr339Gtz5hvjfe2sz3+l/b2vVttbac9avL9mqv\nyba3t0dAQAD0er3JuF6vR3BwsNkLKS8vN34QEZH1Md+JiJRj1uUiMTExiIiIQGBgIIKDg5GcnAyD\nwYCoqCgAwMyZM5GZmYmdO3cCANasWQNHR0d0794dDg4OyMzMRGxsLMLDw2Fvb6/coyEiohphvhMR\nKcOsJnvYsGEoLCxEfHw8DAYD/Pz8sH37duMeqvn5+cjLy/vjTu3sMH/+fJw+fRpCCLRv3x6TJ09G\ndHS0Mo+CiIhqhflORKQMs1/4GBUVZTyz8aCUlBSTz0eMGIERI0bUbWVERGQRzHciIvmZ/bbqRERE\nRERkHjbZREREREQyY5NNRERERCQzNtlERERERDJjk01EREREJDM22UREREREMmOTTUREREQkMzbZ\nREREREQyY5NNRERERCQzNtlERERERDJjk01EREREJDM22UREREREMmOTTUREREQkM7Ob7KSkJPj4\n+ECj0UCr1SI9Pf2hc/fu3YshQ4agdevWcHZ2hr+/P1JSUmRZMBERyYv5TkQkP7Oa7PXr1yM6Ohqz\nZs3C4cOH0atXLwwaNAgXL16scn5GRgaefPJJbNy4EceOHcPf//53jB8/HuvWrZN18UREVDfMdyIi\nZdiZMykhIQE6nQ46nQ4AkJiYiNTUVCQnJyM+Pr7S/JkzZ5p8HhUVhbS0NGzcuBEjRoyQYdlERCQH\n5jsRkTKqPZNdWlqK7Oxs9O/f32R8wIAByMjIMLtQcXExmjdvXvMVEhGRIpjvRETKUQkhxKMmGAwG\ntGnTBvv27UNISIhxPC4uDmvXrkVubm61RbZs2YKhQ4ciIyMDAQEBJrcVFRUZ/33q1Kmarp/q4DdH\nD8Qeuq1ojXk9muCxW78qWoOovuvUqZPx325ublZciSnme+PFfCdSXnXZrvjuIj/++CNee+01LFmy\npFIAExFRw8V8JyJ6uGqvyXZ3d4darUZBQYHJeEFBAby8vB55bHp6OgYPHoy5c+di/Pjx1S5Gq9VW\nOwcAsrKyajRfTtasLXf99PNXASh7psPV1RVav3Z1vp/G9HVn7YZRX87a95/RrU+Y7423NvO9/te2\ndn1brS1n/eqyvdom297eHgEBAdDr9Rg6dKhxXK/XIzw8/KHH7du3D2FhYYiLi8PkyZNrsGQiIrIE\n5jsp5czl6zDcKDNr7nVHDwAVfxiYr5WzHXzdXWu8NiJLMWt3kZiYGERERCAwMBDBwcFITk6GwWBA\nVFQUgHuvNs/MzMTOnTsBAHv27EFYWBgmTZqEESNGGM+SqNVquLu7K/RQiIioppjvpATDjTJM3ne5\nhkfV7Mz7kj+7w5c/clSPmdVkDxs2DIWFhYiPj4fBYICfnx+2b98Ob29vAEB+fj7y8vKM81euXImb\nN29i0aJFWLRokXG8ffv2OHv2rMwPgYiIaov5TkSkDLOabODeXqgVZzYe9OC7faWkpPAdwIiIGgjm\nOxGR/BTfXYSIiIiIyNawySYiIiIikhmbbCIiIiIimZl9TTaR3LjFExERETVWbLLJarjFExERETVW\nvFyEiIiIiEhmbLKJiIiIiGTGJpuIiIiISGZssomIiIiIZMYmm4iIiIhIZmyyiYiIiIhkVq+28DN3\nD2TumUxERERE9Vm9arK5ZzIRERERNQZmXy6SlJQEHx8faDQaaLVapKenP3Tu7du38frrr8Pf3x8O\nDg7o06ePLIslIiL5Md+JiORnVpO9fv16REdHY9asWTh8+DB69eqFQYMG4eLFi1XOLy8vh0ajweTJ\nkxEWFibrgomISD7MdyIiZZh1uUhCQgJ0Oh10Oh0AIDExEampqUhOTkZ8fHyl+U5OTkhKSgIAHDly\nBNeuXZNxyUREJBfmu3LOXL4Ow40ys+bytUZEjU+1TXZpaSmys7Px9ttvm4wPGDAAGRkZii2MiIiU\nxXxXluFGGV9rRGTDqm2yL1++jPLycnh6epqMe3p6YteuXYotTAnXr19HVtYZWe4rKytLlvuxZv2K\nMydKetTX3Nr1a8Oa33dbrW3t+nLU7tSpkwwrkZ8l872mX8eG/j0HrJtx1s5Xa9evKWac7dWWo351\n2c59somIiIiIZFbtmWx3d3eo1WoUFBSYjBcUFMDLy0uxhSnB1dUVWr92dbqPir96tFqtHEuq0TV7\nwL2/3IF7j8VcD7tm7961fzV7arKmHvU1t3b9mpD7+87a9b++nLWLiorqfB9KsGS+m/t1bCzfc8C6\nGWftfLV2fXMx42yvtpz1q8v2aptse3t7BAQEQK/XY+jQocZxvV6P8PDwOi2OanvNHlCT8OI1e0RU\nFeY7EZFyzNpdJCYmBhEREQgMDERwcDCSk5NhMBgQFRUFAJg5cyYyMzOxc+dO4zG5ubm4ffs2Ll++\njJKSEhw5cgQA4O/vr8DDICKi2mC+ExEpw6wme9iwYSgsLER8fDwMBgP8/Pywfft2eHt7AwDy8/OR\nl5dncswLL7yACxcuGD/v0aMHVCoVysvLZVw+ERHVBfOdGhtunUj1hdlvqx4VFWU8s/GglJSUSmMP\nhjIREdVPzHdqTLh1ItUX3F2EiIiIiEhmZp/JJmpMaryrSy2eUuTTiURERLaLTTbZJO7qQkREREri\n5SJERERERDLjmWzwlchERERUd9a+FJH9TP3CJht8JTIRERHVnbUvRWQ/U7+wySayMJ7pIFtk7s8Q\nX2RMRI0Fm2wiC7PlMx3W/gODrEfJnzmeWSOi+ohNNhFZjLX/wCAiosanvp7AYZNNRDahvoYwERHV\nTX09gcMmm4hsQn0NYSIiapy4TzYRERERkcx4JpuIiBot7qZDRNZidpOdlJSERYsWwWAw4E9/+hM+\n/vhjhISEPHR+Tk4O3nzzTfz8889o2bIlxo8fj9mzZ8uyaCIikk9jznfuG0xE1mJWk71+/XpER0dj\n2bJlCA4OxtKlSzFo0CDk5ubC29u70vzr16+jf//+CA0NRXZ2NnJzcxEZGQkXFxdMmzZN9gdBRES1\nw3wnIjnwWaPKzGqyExISoNPpoNPpAACJiYlITU1FcnIy4uPjK81fs2YNbt68iZUrV8LBwQFdu3ZF\nbm4uFi9ezBAmIqpHmO9EJAc+a1RZtS98LC0tRXZ2Nvr3728yPmDAAGRkZFR5zIEDB/Dcc8/BwcHB\nODZw4EBcunQJ58+fr+OSiYhIDsx3IiLlqIQQ4lETDAYD2rRpg3379plcoxcXF4e1a9ciNze30jED\nBw5E27ZtsXz5cuPY//3f/6F9+/b46aefEBQUZBwvKiqS43EQETUIbm5u1l6CEfOdiEgeVWU7t/Aj\nIiIiIpJZtU22u7s71Go1CgoKTMYLCgrg5eVV5TFeXl5VzlepVA89hoiILIv5TkSknGpf+Ghvb4+A\ngADo9XoMHTrUOK7X6xEeHl7lMT179sSMGTNw584d43V7P/zwA1q3bo327dubzK1PT50SEdkS5jsR\nkXLMulwkJiYGK1aswBdffIHjx49j6tSpMBgMiIqKAgDMnDkT/fr1M84fNWoUnJycEBkZiWPHjmHT\npk348MMPMX36dGUeBRER1QrznYhIGWZt4Tds2DAUFhYiPj4eBoMBfn5+2L59u3EP1fz8fOTl5Rnn\nN23aFHq9HpMmTUJgYCCaN2+Ot99+G9HR0co8CiIiqhXmOxGRMqrdXYSIiIiIiGqmwe0ukpSUBB8f\nH2g0Gmi1WqSnp1uk7v79+/HSSy/B29sbkiRh1apVFqkLAPPnz8czzzwDNzc3eHh44K9//SuOHTtm\nsfpJSUnw9/eHm5sb3Nzc0KtXL2zbts1i9SvMnz8fkiRhypQpFqn3/vvvQ5Ikk4/WrVtbpDZw7wxi\nZGQkPDw8oNFo4Ofnh/3791ukdseOHSs9dkmS8OKLLype++7du5g9e7bx/3MfHx/Mnj0bd+/eVbw2\nAJSUlCA6OhodOnSAk5MTQkJCkJWVZZHato75btl8ry/ZDjDfme+NM98bVJNd8fa/s2bNwuHDh9Gr\nVy8MGjQIFy9eVLx2SUkJunfvjsTERDg5OSle73779u3Dm2++iZ9++glpaWmws7NDv379cO3aNYvU\nb9u2LRYuXIhDhw4hOzsbffr0wZAhQ5CTk2OR+sC9N8D4/PPP4e/vb7GaANClSxcUFBQgPz8f+fn5\nOHr0qEXqFhUVITg4GCqVCtu3b8fx48exZMkSeHh4WKR+VlaW8THn5+fj4MGDUKlUGD58uOK1FyxY\ngOTkZHz66ac4ceIEEhMTkZSUhPnz5yteGwDGjh0LvV6P1atXIycnB/3790e/fv1gMBgsUt9WMd8t\nn+/1IdsB5jvzvRHnu2hAgoKCxIQJE0zGOnXqJGJjYy26DhcXF7Fy5UqL1rxfSUmJUKvVYsuWLVZb\nQ4sWLcRnn31mkVrXrl0Tvr6+Ys+ePSI0NFRMnjzZInXnzJkjunfvbpFaD5o5c6YICQmxSu2qzJ07\nVzRv3lzcunVL8VphYWEiMjLSZGzMmDHixRdfVLz2zZs3hZ2dndi8ebPJeEBAgJg9e7bi9W0Z8/0e\na+e7JbN8tWVvAAAF0klEQVRdCOZ7fcB8Vy7fG8yZ7Nq8/W9jVVxcjLt376J58+YWr3337l2sW7cO\nN27cQK9evSxSc/z48Rg2bBief/55i9S739mzZ9GmTRv4+Phg5MiRJi8AU9J3332HoKAgjBgxAp6e\nnujRoweWLl1qkdpV+fLLLzF69Gg0adJE8VohISFIS0vDiRMnAAD/+9//sHv3bgwePFjx2mVlZSgv\nL6/0ODUajcUuXbBFzPc/WCvfrZHtAPOd+d7I812R1l0Bly5dEiqVSuzfv99k/IMPPhBdunSx6Fqs\nfaYjPDxcBAQEiLt371qs5tGjR4WLi4uws7MTzZs3F9u2bbNI3c8++0xotVpRXl4uhBAWPdORmpoq\nvv76a3H06FGxa9cuERoaKry8vERhYaHitR0dHYVGoxGxsbHi8OHDYsWKFcLFxUUsXbpU8doP2rFj\nh5AkSRw9etRiNd977z0hSZKwt7cXkiRZ9Cxyr169xJ///Gfxyy+/iPLycrF69WqhVqstnjO2hPn+\nB0vnu7WyXQjmO/O98ec7m+xasGYIT5s2TbRp00acO3fOonVLS0vFmTNnxMGDB0VsbKxwd3cXx44d\nU7TmiRMnxGOPPSZOnjxpHLNkCD/oxo0bwsPDQyQkJChey8HBodLTibGxsaJbt26K137Qq6++KoKC\ngixW76uvvhLt2rUTGzZsEDk5OWLNmjWiRYsW4ssvv7RI/bNnz4rQ0FChUqmEvb29CAoKEqNHj7bK\n195WMN/vsUa+WyPbhWC+M99tI98bTJN9584dYWdnJ7755huT8UmTJonQ0FCLrsVaIRwdHS1at25t\nEkrW0q9fPzFu3DhFa6xYsUJIkiTs7OyMHyqVyvgX8J07dxStX5XevXuLiRMnKl6nffv24o033jAZ\nW716tXBxcVG89v1+/fVX4eDgIL744guL1Wzbtq1YsmSJydjcuXNFp06dLLYGIYT4/fffRX5+vhBC\niOHDh4uwsDCL1rclzPf6k++WyHYhmO/M9z805nxvMNdk3//2v/fT6/UIDg620qosZ+rUqVi/fj3S\n0tLQqVMnay8Hd+/exe3btxWt8fLLL+Po0aM4cuSI8UOr1WLkyJE4cuQI7O3tFa3/oFu3buH48eNo\n1aqV4rWCg4ON16xVOHHiRKW3rVZaSkoKHB0dMWLECIvV/P333yFJptEkSZLFtniqoNFo4OnpiatX\nr2LHjh0YMmSIRevbEuZ7/cl3S2Q7wHxnvv+hUee7Iq27QtavXy+aNGkili9fLnJzc8WUKVOEq6ur\nuHDhguK1S0pKxOHDh8WhQ4eEk5OTiIuLE4cPH7ZI7YkTJ4qmTZuKtLQ0kZ+fb/woKSlRvLYQQsyY\nMUPs379fnDt3Thw9elTMmDFDqNVqsWPHDovUv58ln0586623xN69e0VeXp44cOCAGDx4sHBzc7PI\n9zwzM1M4ODiI+Ph4cfr0abFhwwbh5uYmkpOTFa99vyeeeKLSjg9Ki4yMFG3bthVbt24V586dE5s2\nbRKPPfaYePvtty1Sf8eOHWL79u0iLy9P/PDDD+Kpp54SvXr1EmVlZRapb6uY75bP9/qU7UIw35nv\nyrN0vjeoJlsIIZKTk0XHjh2Fo6Oj0Gq1Ij093SJ19+zZY3wq6/6P119/XfHaVdWVJEm8//77itcW\n4t7/FB06dBCOjo7C09NT9O/fX+j1eovUflDv3r0tFsIjRowQbdq0EU2aNBHe3t7i1VdfFbm5uRap\nLYQQ27ZtE/7+/kKj0YjOnTuLTz/91GK1hRAiLS1NSJIksrKyLFq3pKRETJs2TXTo0EE4OTkJX19f\nMWvWLHH79m2L1N+wYYPw9fUVjo6OonXr1mLKlCmiuLjYIrVtHfPdsvlen7JdCOa7JTHfLZPvfFt1\nIiIiIiKZNZhrsomIiIiIGgo22UREREREMmOTTUREREQkMzbZREREREQyY5NNRERERCQzNtlERERE\nRDJjk01EREREJDM22UREREREMvt/jtAFFifUsWIAAAAASUVORK5CYII=\n", + "application/javascript": [ + "/* Put everything inside the global mpl namespace */\n", + "window.mpl = {};\n", + "\n", + "mpl.get_websocket_type = function() {\n", + " if (typeof(WebSocket) !== 'undefined') {\n", + " return WebSocket;\n", + " } else if (typeof(MozWebSocket) !== 'undefined') {\n", + " return MozWebSocket;\n", + " } else {\n", + " alert('Your browser does not have WebSocket support.' +\n", + " 'Please try Chrome, Safari or Firefox ≥ 6. ' +\n", + " 'Firefox 4 and 5 are also supported but you ' +\n", + " 'have to enable WebSockets in about:config.');\n", + " };\n", + "}\n", + "\n", + "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n", + " this.id = figure_id;\n", + "\n", + " this.ws = websocket;\n", + "\n", + " this.supports_binary = (this.ws.binaryType != undefined);\n", + "\n", + " if (!this.supports_binary) {\n", + " var warnings = document.getElementById(\"mpl-warnings\");\n", + " if (warnings) {\n", + " warnings.style.display = 'block';\n", + " warnings.textContent = (\n", + " \"This browser does not support binary websocket messages. \" +\n", + " \"Performance may be slow.\");\n", + " }\n", + " }\n", + "\n", + " this.imageObj = new Image();\n", + "\n", + " this.context = undefined;\n", + " this.message = undefined;\n", + " this.canvas = undefined;\n", + " this.rubberband_canvas = undefined;\n", + " this.rubberband_context = undefined;\n", + " this.format_dropdown = undefined;\n", + "\n", + " this.image_mode = 'full';\n", + "\n", + " this.root = $('
');\n", + " this._root_extra_style(this.root)\n", + " this.root.attr('style', 'display: inline-block');\n", + "\n", + " $(parent_element).append(this.root);\n", + "\n", + " this._init_header(this);\n", + " this._init_canvas(this);\n", + " this._init_toolbar(this);\n", + "\n", + " var fig = this;\n", + "\n", + " this.waiting = false;\n", + "\n", + " this.ws.onopen = function () {\n", + " fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n", + " fig.send_message(\"send_image_mode\", {});\n", + " fig.send_message(\"refresh\", {});\n", + " }\n", + "\n", + " this.imageObj.onload = function() {\n", + " if (fig.image_mode == 'full') {\n", + " // Full images could contain transparency (where diff images\n", + " // almost always do), so we need to clear the canvas so that\n", + " // there is no ghosting.\n", + " fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n", + " }\n", + " fig.context.drawImage(fig.imageObj, 0, 0);\n", + " };\n", + "\n", + " this.imageObj.onunload = function() {\n", + " this.ws.close();\n", + " }\n", + "\n", + " this.ws.onmessage = this._make_on_message_function(this);\n", + "\n", + " this.ondownload = ondownload;\n", + "}\n", + "\n", + "mpl.figure.prototype._init_header = function() {\n", + " var titlebar = $(\n", + " '
');\n", + " var titletext = $(\n", + " '
');\n", + " titlebar.append(titletext)\n", + " this.root.append(titlebar);\n", + " this.header = titletext[0];\n", + "}\n", + "\n", + "\n", + "\n", + "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n", + "\n", + "}\n", + "\n", + "\n", + "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n", + "\n", + "}\n", + "\n", + "mpl.figure.prototype._init_canvas = function() {\n", + " var fig = this;\n", + "\n", + " var canvas_div = $('
');\n", + "\n", + " canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n", + "\n", + " function canvas_keyboard_event(event) {\n", + " return fig.key_event(event, event['data']);\n", + " }\n", + "\n", + " canvas_div.keydown('key_press', canvas_keyboard_event);\n", + " canvas_div.keyup('key_release', canvas_keyboard_event);\n", + " this.canvas_div = canvas_div\n", + " this._canvas_extra_style(canvas_div)\n", + " this.root.append(canvas_div);\n", + "\n", + " var canvas = $('');\n", + " canvas.addClass('mpl-canvas');\n", + " canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n", + "\n", + " this.canvas = canvas[0];\n", + " this.context = canvas[0].getContext(\"2d\");\n", + "\n", + " var rubberband = $('');\n", + " rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n", + "\n", + " var pass_mouse_events = true;\n", + "\n", + " canvas_div.resizable({\n", + " start: function(event, ui) {\n", + " pass_mouse_events = false;\n", + " },\n", + " resize: function(event, ui) {\n", + " fig.request_resize(ui.size.width, ui.size.height);\n", + " },\n", + " stop: function(event, ui) {\n", + " pass_mouse_events = true;\n", + " fig.request_resize(ui.size.width, ui.size.height);\n", + " },\n", + " });\n", + "\n", + " function mouse_event_fn(event) {\n", + " if (pass_mouse_events)\n", + " return fig.mouse_event(event, event['data']);\n", + " }\n", + "\n", + " rubberband.mousedown('button_press', mouse_event_fn);\n", + " rubberband.mouseup('button_release', mouse_event_fn);\n", + " // Throttle sequential mouse events to 1 every 20ms.\n", + " rubberband.mousemove('motion_notify', mouse_event_fn);\n", + "\n", + " rubberband.mouseenter('figure_enter', mouse_event_fn);\n", + " rubberband.mouseleave('figure_leave', mouse_event_fn);\n", + "\n", + " canvas_div.on(\"wheel\", function (event) {\n", + " event = event.originalEvent;\n", + " event['data'] = 'scroll'\n", + " if (event.deltaY < 0) {\n", + " event.step = 1;\n", + " } else {\n", + " event.step = -1;\n", + " }\n", + " mouse_event_fn(event);\n", + " });\n", + "\n", + " canvas_div.append(canvas);\n", + " canvas_div.append(rubberband);\n", + "\n", + " this.rubberband = rubberband;\n", + " this.rubberband_canvas = rubberband[0];\n", + " this.rubberband_context = rubberband[0].getContext(\"2d\");\n", + " this.rubberband_context.strokeStyle = \"#000000\";\n", + "\n", + " this._resize_canvas = function(width, height) {\n", + " // Keep the size of the canvas, canvas container, and rubber band\n", + " // canvas in synch.\n", + " canvas_div.css('width', width)\n", + " canvas_div.css('height', height)\n", + "\n", + " canvas.attr('width', width);\n", + " canvas.attr('height', height);\n", + "\n", + " rubberband.attr('width', width);\n", + " rubberband.attr('height', height);\n", + " }\n", + "\n", + " // Set the figure to an initial 600x600px, this will subsequently be updated\n", + " // upon first draw.\n", + " this._resize_canvas(600, 600);\n", + "\n", + " // Disable right mouse context menu.\n", + " $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n", + " return false;\n", + " });\n", + "\n", + " function set_focus () {\n", + " canvas.focus();\n", + " canvas_div.focus();\n", + " }\n", + "\n", + " window.setTimeout(set_focus, 100);\n", + "}\n", + "\n", + "mpl.figure.prototype._init_toolbar = function() {\n", + " var fig = this;\n", + "\n", + " var nav_element = $('
')\n", + " nav_element.attr('style', 'width: 100%');\n", + " this.root.append(nav_element);\n", + "\n", + " // Define a callback function for later on.\n", + " function toolbar_event(event) {\n", + " return fig.toolbar_button_onclick(event['data']);\n", + " }\n", + " function toolbar_mouse_event(event) {\n", + " return fig.toolbar_button_onmouseover(event['data']);\n", + " }\n", + "\n", + " for(var toolbar_ind in mpl.toolbar_items) {\n", + " var name = mpl.toolbar_items[toolbar_ind][0];\n", + " var tooltip = mpl.toolbar_items[toolbar_ind][1];\n", + " var image = mpl.toolbar_items[toolbar_ind][2];\n", + " var method_name = mpl.toolbar_items[toolbar_ind][3];\n", + "\n", + " if (!name) {\n", + " // put a spacer in here.\n", + " continue;\n", + " }\n", + " var button = $('');\n", + " button.click(method_name, toolbar_event);\n", + " button.mouseover(tooltip, toolbar_mouse_event);\n", + " nav_element.append(button);\n", + " }\n", + "\n", + " // Add the status bar.\n", + " var status_bar = $('');\n", + " nav_element.append(status_bar);\n", + " this.message = status_bar[0];\n", + "\n", + " // Add the close button to the window.\n", + " var buttongrp = $('
');\n", + " var button = $('');\n", + " button.click(function (evt) { fig.handle_close(fig, {}); } );\n", + " button.mouseover('Stop Interaction', toolbar_mouse_event);\n", + " buttongrp.append(button);\n", + " var titlebar = this.root.find($('.ui-dialog-titlebar'));\n", + " titlebar.prepend(buttongrp);\n", + "}\n", + "\n", + "mpl.figure.prototype._root_extra_style = function(el){\n", + " var fig = this\n", + " el.on(\"remove\", function(){\n", + "\tfig.close_ws(fig, {});\n", + " });\n", + "}\n", + "\n", + "mpl.figure.prototype._canvas_extra_style = function(el){\n", + " // this is important to make the div 'focusable\n", + " el.attr('tabindex', 0)\n", + " // reach out to IPython and tell the keyboard manager to turn it's self\n", + " // off when our div gets focus\n", + "\n", + " // location in version 3\n", + " if (IPython.notebook.keyboard_manager) {\n", + " IPython.notebook.keyboard_manager.register_events(el);\n", + " }\n", + " else {\n", + " // location in version 2\n", + " IPython.keyboard_manager.register_events(el);\n", + " }\n", + "\n", + "}\n", + "\n", + "mpl.figure.prototype._key_event_extra = function(event, name) {\n", + " var manager = IPython.notebook.keyboard_manager;\n", + " if (!manager)\n", + " manager = IPython.keyboard_manager;\n", + "\n", + " // Check for shift+enter\n", + " if (event.shiftKey && event.which == 13) {\n", + " this.canvas_div.blur();\n", + " event.shiftKey = false;\n", + " // Send a \"J\" for go to next cell\n", + " event.which = 74;\n", + " event.keyCode = 74;\n", + " manager.command_mode();\n", + " manager.handle_keydown(event);\n", + " }\n", + "}\n", + "\n", + "mpl.figure.prototype.handle_save = function(fig, msg) {\n", + " fig.ondownload(fig, null);\n", + "}\n", + "\n", + "\n", + "mpl.find_output_cell = function(html_output) {\n", + " // Return the cell and output element which can be found *uniquely* in the notebook.\n", + " // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n", + " // IPython event is triggered only after the cells have been serialised, which for\n", + " // our purposes (turning an active figure into a static one), is too late.\n", + " var cells = IPython.notebook.get_cells();\n", + " var ncells = cells.length;\n", + " for (var i=0; i= 3 moved mimebundle to data attribute of output\n", + " data = data.data;\n", + " }\n", + " if (data['text/html'] == html_output) {\n", + " return [cell, data, j];\n", + " }\n", + " }\n", + " }\n", + " }\n", + "}\n", + "\n", + "// Register the function which deals with the matplotlib target/channel.\n", + "// The kernel may be null if the page has been refreshed.\n", + "if (IPython.notebook.kernel != null) {\n", + " IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n", + "}\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "def discrete_bayes_sim(pos, kernel, zs, z_prob_correct, sleep=0.4):\n", + " N = len(hallway)\n", + " fig = plt.figure()\n", + " for i, z in enumerate(zs):\n", + " plt.cla()\n", + " prior = predict(pos, 1, kernel)\n", + " bp.bar_plot(hallway, c='k')\n", + " bp.bar_plot(prior, ylim=(0,1.0), c='#ff8015')\n", + " plt.axvline(i % N + 0.4, lw=5)\n", + " fig.canvas.draw()\n", + " time.sleep(sleep)\n", + "\n", + " plt.cla()\n", + " likelihood = lh_hallway(hallway, z=z, z_prob=z_prob_correct)\n", + " pos = update(likelihood, prior)\n", + " bp.bar_plot(hallway, c='k')\n", + " bp.bar_plot(pos, ylim=(0,1.0))\n", + " plt.axvline(i % 10 + 0.4, lw=5)\n", + " fig.canvas.draw()\n", + " time.sleep(sleep)\n", + " \n", + "# change these numbers to alter the simulation\n", + "kernel = (.1, .8, .1)\n", + "z_prob = 1.0\n", + "\n", + "# list of perfect measurements\n", + "hallway = np.array([1, 1, 0, 0, 0, 0, 0, 0, 1, 0])\n", + "measurements = [hallway[i % len(hallway)] for i in range(25)]\n", + "pos = np.array([.1]*10)\n", + "\n", + "discrete_bayes_sim(pos, kernel, measurements, z_prob, sleep=.4)" ] }, { @@ -10567,12 +13729,12 @@ "\n", "You may be suspicious of the results above because I always passed correct sensor data into the functions. However, we are claiming that this code implements a *filter* - it should filter out bad sensor measurements. Does it do that?\n", "\n", - "To make this easy to program and visualize I will change the layout of the hallway to mostly alternating doors and hallways, and run the algorithm on 5 correct measurements:" + "To make this easy to program and visualize I will change the layout of the hallway to mostly alternating doors and hallways, and run the algorithm on 15 correct measurements:" ] }, { "cell_type": "code", - "execution_count": 50, + "execution_count": 27, "metadata": { "collapsed": false }, @@ -11344,7 +14506,7 @@ { "data": { "text/html": [ - "" + "" ], "text/plain": [ "" @@ -11359,25 +14521,20 @@ "kernel = (.1, .8, .1)\n", "prior = np.array([.1] * 10)\n", "measurements = [1, 0, 1, 0, 0]\n", - "\n", - "for m in measurements:\n", - " likelihood = lh_hallway(hallway, z=m, z_prob=.75)\n", - " posterior = update(likelihood, prior)\n", - " prior = predict(posterior, 1, kernel)\n", - "with interactive_plot():\n", - " bp.bar_plot(posterior, ylim=(0, .4))" + "z_prob = 0.75\n", + "discrete_bayes_sim(prior, kernel, measurements, z_prob, sleep=.35)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "We have identified the likely cases of having started at position 0 or 5, because we saw the following sequence of doors and walls 1,0,1,0,0. Now I inject a bad measurement:" + "We have identified the likely cases of having started at position 0 or 5, because we saw this sequence of doors and walls: 1,0,1,0,0. Now I inject a bad measurement. The next measurement should be 1, but instead we get a 0:" ] }, { "cell_type": "code", - "execution_count": 51, + "execution_count": 28, "metadata": { "collapsed": false }, @@ -12149,7 +15306,7 @@ { "data": { "text/html": [ - "" + "" ], "text/plain": [ "" @@ -12160,10 +15317,8 @@ } ], "source": [ - "likelihood = lh_hallway(hallway, z=0, z_prob=.75)\n", - "posterior = update(likelihood, prior)\n", - "with interactive_plot():\n", - " bp.bar_plot(posterior, ylim=(0, .4))" + "measurements = [1, 0, 1, 0, 0, 0]\n", + "discrete_bayes_sim(prior, kernel, measurements, z_prob, sleep=.25)" ] }, { @@ -12175,7 +15330,7 @@ }, { "cell_type": "code", - "execution_count": 55, + "execution_count": 29, "metadata": { "collapsed": false }, @@ -12947,7 +16102,7 @@ { "data": { "text/html": [ - "" + "" ], "text/plain": [ "" @@ -12966,7 +16121,8 @@ " posterior = update(likelihood, prior)\n", " prior = predict(posterior, 1, kernel)\n", " plt.subplot(3, 2, i+1)\n", - " bp.bar_plot(posterior, ylim=(0, .4), title='step {}'.format(i+1))" + " bp.bar_plot(posterior, ylim=(0, .4), title='step {}'.format(i+1))\n", + " plt.tight_layout()" ] }, { @@ -13021,7 +16177,7 @@ }, { "cell_type": "code", - "execution_count": 56, + "execution_count": 30, "metadata": { "collapsed": false }, @@ -13072,7 +16228,7 @@ }, { "cell_type": "code", - "execution_count": 57, + "execution_count": 31, "metadata": { "collapsed": false }, @@ -13124,7 +16280,7 @@ }, { "cell_type": "code", - "execution_count": 59, + "execution_count": 32, "metadata": { "collapsed": false, "scrolled": false @@ -13897,7 +17053,7 @@ { "data": { "text/html": [ - "" + "" ], "text/plain": [ "" @@ -13943,7 +17099,7 @@ }, { "cell_type": "code", - "execution_count": 60, + "execution_count": 33, "metadata": { "collapsed": false }, @@ -14715,7 +17871,7 @@ { "data": { "text/html": [ - "" + "" ], "text/plain": [ "" @@ -14760,7 +17916,7 @@ }, { "cell_type": "code", - "execution_count": 69, + "execution_count": 34, "metadata": { "collapsed": false, "scrolled": false @@ -15533,7 +18689,7 @@ { "data": { "text/html": [ - "" + "" ], "text/plain": [ "" @@ -15548,12 +18704,11 @@ " with interactive_plot():\n", " for i in range (4):\n", " random.seed(3)\n", - " plt.subplot(321+i)\n", + " plt.subplot(221+i)\n", " train_filter(148+i, kernel=[.1, .8, .1], \n", " sensor_accuracy=.8,\n", " move_distance=4, do_print=False)\n", - " plt.title ('iteration {}'.format(148+i))\n", - " plt.tight_layout()" + " plt.title ('iteration {}'.format(148+i))" ] }, { diff --git a/code/book_plots.py b/code/book_plots.py index 2c36bad..796821e 100644 --- a/code/book_plots.py +++ b/code/book_plots.py @@ -47,6 +47,7 @@ def interactive_plot(close=True, fig=None): if fig is None: fig = plt.figure() yield + plt.tight_layout() if close: end_interactive(fig)