{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "
Peter Norvig
2007, 2026
\n", "\n", "# Solving Sudoku at 400,000 puzzles per second\n", "\n", "The rules of [**Sudoku**](http://en.wikipedia.org/wiki/Sudoku) are [simple and finite](https://clip.cafe/legally-blonde-2001/the-rules-of-hair-care-are-simple-finite-s1/?srsltid=AfmBOoqLAWkreHwp-ZaP5JeYIlVYwljjgG-7aHmt-kLQ_D0q8zvxd4GG): fill in the empty squares in the puzzle so that each column, row, and 3x3 box in the solution contains all the digits from 1 to 9. For example:\n", "\n", "|Puzzle|Solution|\n", "|---|---|\n", "|![](https://upload.wikimedia.org/wikipedia/commons/thumb/e/e0/Sudoku_Puzzle_by_L2G-20050714_standardized_layout.svg/250px-Sudoku_Puzzle_by_L2G-20050714_standardized_layout.svg.png)|![](https://upload.wikimedia.org/wikipedia/commons/thumb/1/12/Sudoku_Puzzle_by_L2G-20050714_solution_standardized_layout.svg/250px-Sudoku_Puzzle_by_L2G-20050714_solution_standardized_layout.svg.png)|\n", "\n", "\n", "In 2006, I wrote a [Python program](http://norvig.com/sudoku.html) designed to solve any Sudoku puzzle. As computer security expert Ben Laurie has stated, Sudoku is \"a denial of service attack on human intellect\", and I thought maybe my program would convince people they didn't need to spend excessive time playing Sudoku. \n", "\n", "In 2007, [Peter Seibel](https://gigamonkeys.com/) invited me to publish an article in [*Code Quarterly*](https://gigamonkeys.com/code-quarterly/) magazine which would go into more detail about backtracking search and constraint propagation in general, and would feature a more efficient program. Unfortunately, the magazine [folded](https://gigamonkeys.wordpress.com/2011/10/17/end-of-the-line-for-code-quarterly/) before I could finish [the article](https://www.norvig.com/CQ/sudoku.html). In 2021 I [updated the Python Sudoku program](Sudoku.ipynb) to Jupyter notebook form with modern coding idioms, and now I do the same for the [more efficient Java Sudoku program](Sudoku.java). \n", "\n", "I asked Claude and Codex to help me update my code from Java JDK 1.6 to Java JDK 26. They were very good at that. I also asked them for suggestions to make the code faster, but they didn't have any significant ideas (although the switch to a more modern threading API did speed throughput a few percent). I guess I did a good job of writing efficient Java way back in 2007.\n", "\n", "- [**Sudoku.java**](Sudoku.java) is the Java program discussed in this notebook.\n", "- [**Sudoku.ipynb**](Sudoku.ipynb) is the original Python notebook.\n", "\n", "
\n", "TLDR: The program solves over 400,000 puzzles per second , and over 30,000 puzzles per second on the hardest puzzles. \n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Solving a Puzzle\n", "\n", "In the file [sudokus1.txt](sudokus1.txt) is the single puzzle shown at the top of the page, in this format:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "53..7....6..195....98....6.8...6...34..8.3..17...2...6.6....28....419..5....8..79\n" ] } ], "source": [ "!cat sudokus1.txt" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can solve the puzzle with the command `java Sudoku`, and display the solution with the `-grid` option:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "Puzzle 1: Solution:\n", "5 3 . | . 7 . | . . . 5 3 4 | 6 7 8 | 9 1 2 \n", "6 . . | 1 9 5 | . . . 6 7 2 | 1 9 5 | 3 4 8 \n", ". 9 8 | . . . | . 6 . 1 9 8 | 3 4 2 | 5 6 7 \n", "------+-------+------ ------+-------+------\n", "8 . . | . 6 . | . . 3 8 5 9 | 7 6 1 | 4 2 3 \n", "4 . . | 8 . 3 | . . 1 4 2 6 | 8 5 3 | 7 9 1 \n", "7 . . | . 2 . | . . 6 7 1 3 | 9 2 4 | 8 5 6 \n", "------+-------+------ ------+-------+------\n", ". 6 . | . . . | 2 8 . 9 6 1 | 5 3 7 | 2 8 4 \n", ". . . | 4 1 9 | . . 5 2 8 7 | 4 1 9 | 6 3 5 \n", ". . . | . 8 . | . 7 9 3 4 5 | 2 8 6 | 1 7 9 \n" ] } ], "source": [ "!java Sudoku -grid -nosummary sudokus1.txt" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Command Line Options\n", "\n", "Here are all the options for my `java Sudoku` command:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "usage: java Sudoku -(no)[fghnprstuv] | -[RT] | ...\n", "\n", "Options and filenames are processed left-to-right. Use '-no' to turn an option off\n", "E.g.: -v turns verify flag on, -nov turns it off. -R and -T require a number. The options:\n", "\n", " -g(rid) Print each puzzle grid and solution grid (default off)\n", " -h(elp) Print this usage message\n", " -n(aked) Run the naked pairs strategy (default on)\n", " -p(uzzle) Print summary stats for each puzzle (default off)\n", " -r(everse) Solve the reverse of each puzzle as well as each puzzle itself (default off)\n", " -s(ummary) Print per-file summary stats (default on)\n", " -t(hread) Print summary stats for each thread (default off)\n", " -v(erify) Verify each solution is valid (default on)\n", " -T Concurrently run threads (default 28)\n", " -R Repeat the solving of each puzzle times (default 1)\n", " Solve all puzzles in filename, which has one puzzle per line\n" ] } ], "source": [ "!java Sudoku -help" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Benchmark Puzzles\n", "\n", "I gathered some Sudoku puzzles from various sources:\n", "- [sudokus_hard.txt](sudokus_hard.txt): Allegedly hardest problems, form the [New Sudoku Players Forum](http://forum.enjoysudoku.com/the-hardest-sudokus-new-thread-t6539.html#p65791).\n", "- [sudokus_17.txt](sudokus_17.txt): Puzzles with exactly 17 filled-in squares (the minimum), from [Gordon Royle at the University of Western Australia](https://web.archive.org/web/20131019184812if_/http://school.maths.uwa.edu.au/~gordon/sudokumin.php).\n", "- [sudokus.txt](sudokus.txt): My collection, from various sources, and some automatically generated (by permuting rows, or swapping digits).\n", "\n", "Here are the number of puzzles in each file:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " 375 sudokus_hard.txt\n", " 49150 sudokus_17.txt\n", " 250000 sudokus.txt\n", " 299525 total\n" ] } ], "source": [ "!wc -l sudokus_hard.txt sudokus_17.txt sudokus.txt " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's run the `java Sudoku` program on all these puzzles, and show the timing results. To get even more puzzles (to lower timing variance), I use the `-r` option to include the reverse of each puzzle (the last square becomes the first, etc.) and `-R` to repeat puzzles." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Puzzles μsec KHz Threads Backtracks Name\n", "======= ====== ======= ======= ========== ====\n", " 30000 29.1 34.403 28 330.7 sudokus_hard.txt\n", " 983020 2.0 497.578 28 41.7 sudokus_17.txt\n", "1000000 2.5 402.050 28 32.5 sudokus.txt\n" ] } ], "source": [ "!java Sudoku -r -R40 sudokus_hard.txt -R10 sudokus_17.txt -R2 sudokus.txt" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "So the program is solving 34,000 puzzles per second on the hardest puzzles, and about 400,000 to 500,000 on the other files of more average-difficulty puzzles. The columns of the output are:\n", "\n", "- `Puzzles`: the total number of puzzles solved.\n", "- `μsec`: the mean time to solve a puzzle in microseconds (millionths of a second).\n", "- `KHz`: the throughput expressed as thousands of puzzles solved per second (the inverse of the μsec column).\n", "- `Threads`: the number of concurrent threads used.\n", "- `Backtracks`: the average number of times per puzzle that the search guessed wrong and had to back up.\n", "- `Name`: here the name of the file; could also be a puzzle number when -p option is on.\n", "\n", "One of the fastest known Sudoku solvers, [**tdoku**](https://github.com/t-dillon/tdoku), reports similar speed (but that was in 2020, so they probably used a slower computer than my 28-core 96 GB Mac Studio M3 Ultra).\n", "\n", "# Throughput versus Latency\n", "\n", "Note that the μsec and KHz columns are displaying **throughput**: the number of puzzles that can be solved in a given amount of time. This is different than **latency**: the time it takes to solve a single puzzle from start to end. Throughput and latency are different because there are multiple threads working in parallel. If, say, there are 28 threads working in parallel on 28 CPUs and each puzzle takes 1 second to solve, then latency would be 1 second but throughput would be 28 puzzles per second. \n", "\n", "To measure the latency, use the `-T1` option to request a single thread:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Puzzles μsec KHz Threads Backtracks Name\n", "======= ====== ======= ======= ========== ====\n", " 3000 281.9 3.547 1 292.2 sudokus_hard.txt\n", " 49151 69.3 14.430 1 40.2 sudokus_17.txt\n", " 250000 66.3 15.082 1 34.2 sudokus.txt\n" ] } ], "source": [ "!java Sudoku -T1 -R8 sudokus_hard.txt -R1 sudokus_17.txt sudokus.txt" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The average latency is about 300 microseconds for the hard puzzles and 70 microseconds for the easier puzzles." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Effect of Naked Pairs Strategy\n", "\n", "This Java program uses the so-called [naked pairs strategy](https://www.learn-sudoku.com/naked-pairs.html): If two squares in a unit share exactly the same two possible values, eliminate those values from every other square in that unit. My previous Python program did not use this strategy. The following invocation yields two lines of output, the first using the naked pairs strategy, the second without:" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Puzzles μsec KHz Threads Backtracks Name\n", "======= ====== ======= ======= ========== ====\n", "1000000 2.5 395.567 28 32.5 sudokus.txt\n", "1000000 4.2 237.418 28 107.4 sudokus.txt\n" ] } ], "source": [ "!java Sudoku -r -R2 -naked sudokus.txt -nonaked sudokus.txt" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The program is 50% faster with naked pairs compared to without. Perhaps implementing [other strategies](https://bestofsudoku.com/sudoku-strategy) could give similar speedups. However, my feeling is that the program is already fast enough, and there would probably be diminishing returns from further strategies." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Distribution of Puzzle Difficulty\n", "\n", "What's the distribution of puzzle run times, or of backtracks, across the benchmark puzzles? \n", "\n", "I can get data for individual puzzles with the `-p` option, eliminate the header lines with `tail`, and pipe the results to the file `sudata.txt`:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "!java Sudoku -T1 -p sudokus_17.txt sudokus_hard.txt | tail -n +3 > sudata.txt" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "I can import the data into pandas:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
μsecKHzBacktracks
count49527.00000049527.00000049527.000000
mean74.69540323.26210742.144939
std152.54336111.191693187.344951
min12.0000000.0990000.000000
1%18.3000001.6633000.000000
10%27.8000008.0290000.000000
25%32.80000015.4140001.000000
50%42.40000023.5990005.000000
75%64.90000030.53400024.000000
90%124.50000036.03600080.000000
99%601.05800054.546000646.740000
max10108.80000083.62600013438.000000
\n", "
" ], "text/plain": [ " μsec KHz Backtracks\n", "count 49527.000000 49527.000000 49527.000000\n", "mean 74.695403 23.262107 42.144939\n", "std 152.543361 11.191693 187.344951\n", "min 12.000000 0.099000 0.000000\n", "1% 18.300000 1.663300 0.000000\n", "10% 27.800000 8.029000 0.000000\n", "25% 32.800000 15.414000 1.000000\n", "50% 42.400000 23.599000 5.000000\n", "75% 64.900000 30.534000 24.000000\n", "90% 124.500000 36.036000 80.000000\n", "99% 601.058000 54.546000 646.740000\n", "max 10108.800000 83.626000 13438.000000" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "\n", "columns = 'Puzzles μsec KHz Threads Backtracks Puzzle Number'.split()\n", "sudata = pd.read_csv('sudata.txt', sep='\\s+', usecols=(1, 2, 4))\n", "sudata.columns = ('μsec', 'KHz', 'Backtracks')\n", "sudata.describe(percentiles=(.01, .10, .25, .50, .75, .90, .99))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The mean latency is 74 microseconds, the median is about 42 microseconds, and the maximum is about 10 milliseconds. For number of backtracks, the mean is 42, the median is only 5, at least 25% have zero or one backtrack, and the maximum is over 13,000. \n", "\n", "Here is a scatterplot:" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmAAAAGwCAYAAAAOvdliAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjYsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvq6yFwwAAAAlwSFlzAAAPYQAAD2EBqD+naQAARG9JREFUeJzt3Xl8VNX9//H3JIEhYBhIkISwBokKDSqEXSsgEbAsbW2LFoxiLdoqSxQE/FJ/Lq0g1AooLmB9SL9qS7VKv1opEhARJAQaSAUR2QIJS0QgTEBCtjm/PyDXTDZCSO5MMq/n45HHN3PnzL3n3vpl3jnnc891GGOMAAAAYJsgX3cAAAAg0BDAAAAAbEYAAwAAsBkBDAAAwGYEMAAAAJsRwAAAAGxGAAMAALBZiK870JB4PB4dOXJEYWFhcjgcvu4OAACoBmOMTp8+rejoaAUF2TM2RQCrRUeOHFH79u193Q0AAFADWVlZateunS3HIoDVorCwMEnn/wds3ry5j3sDAACqIzc3V+3bt7e+x+1AAKtFJdOOzZs3J4ABAFDP2Fk+RBE+AACAzQhgAAAANiOAAQAA2IwABgAAYDMCGAAAgM0IYAAAADYjgAEAANiMAAYAAGAzAhgAAIDNCGAAAAA2I4ABAADYjAAGAABgMx7GDQAA/FpRsUcvrd2nLQdOqnencD00+CqFBNfvMSQCGAAA8Gsvrd2nBat3y0j6fO9xSdKUhFjfduoy+TQ+fvbZZxo1apSio6PlcDj0z3/+0+t9Y4yefPJJRUdHKzQ0VIMGDdKXX37p1SY/P1+TJk1Sq1at1KxZM40ePVqHDh3yapOTk6PExES5XC65XC4lJibq1KlTXm0yMzM1atQoNWvWTK1atdLkyZNVUFBQF6cNAAAuwZYDJ2Uu/G4uvK7vfBrAvvvuO11//fVatGhRhe/PmzdPzz//vBYtWqQtW7YoKipKt956q06fPm21SUpK0vLly7Vs2TJt2LBBZ86c0ciRI1VcXGy1GTt2rNLT07Vy5UqtXLlS6enpSkxMtN4vLi7WiBEj9N1332nDhg1atmyZ3nvvPU2dOrXuTh4AAFRL707hclz43XHhdb1n/IQks3z5cuu1x+MxUVFR5tlnn7W2nTt3zrhcLvPqq68aY4w5deqUadSokVm2bJnV5vDhwyYoKMisXLnSGGPMzp07jSSzadMmq01KSoqRZHbt2mWMMWbFihUmKCjIHD582Grzt7/9zTidTuN2u6t9Dm6320i6pM8AAICqFRYVmwXJu8241zaZBcm7TWFRca3u3xff335bwZaRkaHs7GwNHTrU2uZ0OjVw4EBt3LhRkpSWlqbCwkKvNtHR0YqLi7PapKSkyOVyqW/fvlabfv36yeVyebWJi4tTdHS01WbYsGHKz89XWlpapX3Mz89Xbm6u1w8AAKhdIcFBmpIQq7d+3VdTEmLrfQG+5MfLUGRnZ0uSIiMjvbZHRkZa72VnZ6tx48Zq2bJllW1at25dbv+tW7f2alP2OC1btlTjxo2tNhWZM2eOVVfmcrnUvn37SzxLAAAQiPw2gJVwOBxer40x5baVVbZNRe1r0qasxx57TG632/rJysqqsl8AAACSHwewqKgoSSo3AnXs2DFrtCoqKkoFBQXKycmpss0333xTbv/ffvutV5uyx8nJyVFhYWG5kbHSnE6nmjdv7vUDAABwMX4bwGJiYhQVFaXk5GRrW0FBgdatW6cBAwZIkuLj49WoUSOvNkePHtWOHTusNv3795fb7dbmzZutNqmpqXK73V5tduzYoaNHj1ptVq1aJafTqfj4+Do9TwAAEHh8uhDrmTNntHfvXut1RkaG0tPTFR4erg4dOigpKUmzZ89WbGysYmNjNXv2bDVt2lRjx46VJLlcLt13332aOnWqIiIiFB4ermnTpql79+5KSEiQJHXt2lXDhw/XhAkTtHjxYknS/fffr5EjR+qaa66RJA0dOlTdunVTYmKi/vjHP+rkyZOaNm2aJkyYwKgWAACofbbdb1mBtWvXGp1fU83r55577jHGnF+K4oknnjBRUVHG6XSam2++2Wzfvt1rH3l5eWbixIkmPDzchIaGmpEjR5rMzEyvNidOnDDjxo0zYWFhJiwszIwbN87k5OR4tTl48KAZMWKECQ0NNeHh4WbixInm3Llzl3Q+LEMBAED944vvb4cxxlSRz3AJcnNz5XK55Ha7GTkDAKCe8MX3t9/WgAEAADRUBDAAAACbEcAAAABsRgADAACwGQEMAADAZgQwAAAAmxHAAAAAbEYAAwAAsBkBDAAAwGYEMAAAAJsRwAAAAGxGAAMAALAZAQwAAMBmBDAAAACbEcAAAABsRgADAACwGQEMAADAZgQwAAAAmxHAAAAAbEYAAwAAsBkBDAAAwGYEMAAAAJsRwAAAAGxGAAMAALAZAQwAAMBmBDAAAACbEcAAAABsRgADAACwGQEMAADAZgQwAAAAmxHAAAAAbEYAAwAAsBkBDAAAwGYEMAAAAJsRwAAAAGxGAAMAALAZAQwAAMBmBDAAAACbEcAAAABsRgADAACwGQEMAADAZgQwAAAAmxHAAAAAbEYAAwAAsBkBDAAAwGYEMAAAAJsRwAAAAGxGAAMAALAZAQwAAMBmBDAAAACbEcAAAABsRgADAACwGQEMAADAZgQwAAAAmxHAAAAAbObXAayoqEi/+93vFBMTo9DQUHXu3FlPP/20PB6P1cYYoyeffFLR0dEKDQ3VoEGD9OWXX3rtJz8/X5MmTVKrVq3UrFkzjR49WocOHfJqk5OTo8TERLlcLrlcLiUmJurUqVN2nCYAAAgwfh3A5s6dq1dffVWLFi3SV199pXnz5umPf/yjXnzxRavNvHnz9Pzzz2vRokXasmWLoqKidOutt+r06dNWm6SkJC1fvlzLli3Thg0bdObMGY0cOVLFxcVWm7Fjxyo9PV0rV67UypUrlZ6ersTERFvPFwAABAaHMcb4uhOVGTlypCIjI/X6669b2372s5+padOmevPNN2WMUXR0tJKSkjRjxgxJ50e7IiMjNXfuXD3wwANyu9268sor9eabb+qOO+6QJB05ckTt27fXihUrNGzYMH311Vfq1q2bNm3apL59+0qSNm3apP79+2vXrl265pprqtXf3NxcuVwuud1uNW/evJavBgAAqAu++P726xGwm266SWvWrNHu3bslSf/973+1YcMG/ehHP5IkZWRkKDs7W0OHDrU+43Q6NXDgQG3cuFGSlJaWpsLCQq820dHRiouLs9qkpKTI5XJZ4UuS+vXrJ5fLZbWpSH5+vnJzc71+AAAALibE1x2oyowZM+R2u3XttdcqODhYxcXFeuaZZ/TLX/5SkpSdnS1JioyM9PpcZGSkDh48aLVp3LixWrZsWa5Nyeezs7PVunXrcsdv3bq11aYic+bM0VNPPVXzEwQAAAHJr0fA/v73v+utt97SX//6V23dulV/+ctf9Nxzz+kvf/mLVzuHw+H12hhTbltZZdtU1P5i+3nsscfkdrutn6ysrOqcFgAACHB+PQL26KOPaubMmbrzzjslSd27d9fBgwc1Z84c3XPPPYqKipJ0fgSrTZs21ueOHTtmjYpFRUWpoKBAOTk5XqNgx44d04ABA6w233zzTbnjf/vtt+VG10pzOp1yOp2Xf6IAACCg+PUI2NmzZxUU5N3F4OBgaxmKmJgYRUVFKTk52Xq/oKBA69ats8JVfHy8GjVq5NXm6NGj2rFjh9Wmf//+crvd2rx5s9UmNTVVbrfbagMAAFBb/HoEbNSoUXrmmWfUoUMH/eAHP9C2bdv0/PPP61e/+pWk89OGSUlJmj17tmJjYxUbG6vZs2eradOmGjt2rCTJ5XLpvvvu09SpUxUREaHw8HBNmzZN3bt3V0JCgiSpa9euGj58uCZMmKDFixdLku6//36NHDmy2ndAAgAAVJdfB7AXX3xRjz/+uB588EEdO3ZM0dHReuCBB/T//t//s9pMnz5deXl5evDBB5WTk6O+fftq1apVCgsLs9rMnz9fISEhGjNmjPLy8jRkyBAtXbpUwcHBVpu3335bkydPtu6WHD16tBYtWmTfyQIAgIDh1+uA1TesAwYAQP3DOmAAAAABgAAGAABgMwIYAACAzQhgAAAANiOAAQAA2IwABgAAYDMCGAAAgM0IYAAAADYjgAEAANiMAAYAAGAzAhgAAIDNCGAAAAA2I4ABAADYjAAGAABgMwIYAACAzQhgAAAANiOAAQAA2IwABgAAYDMCGAAAgM0IYAAAADYjgAEAANiMAAYAAGAzAhgAAIDNCGAAAAA2I4ABAADYjAAGAABgMwIYAACAzQhgAAAANiOAAQAA2CzE1x0AANS9omKPXlq7T1sOnFTvTuF6aPBVCgnmb3DAVwhgABAAXlq7TwtW75aR9Pne45KkKQmxvu0UEMD48wcAAsCWAydlLvxuLrwG4DsEMAAIAL07hctx4XfHhdcAfIcpSAAIAA8NvkqSvGrAAPgOAQwAAkBIcBA1X4AfYQoSAADAZgQwAAAAmxHAAAAAbEYAAwAAsBkBDAAAwGYEMAAAAJsRwAAAAGxGAAMAALAZAQwAAMBmBDAAAACbEcAAAABsRgADAACwGQEMAADAZgQwAAAAmxHAAAAAbEYAAwAAsBkBDAAAwGYEMAAAAJsRwAAAAGxGAAMAALAZAQwAAMBmfh/ADh8+rLvuuksRERFq2rSpbrjhBqWlpVnvG2P05JNPKjo6WqGhoRo0aJC+/PJLr33k5+dr0qRJatWqlZo1a6bRo0fr0KFDXm1ycnKUmJgol8sll8ulxMREnTp1yo5TBAAAAcavA1hOTo5uvPFGNWrUSP/+97+1c+dO/elPf1KLFi2sNvPmzdPzzz+vRYsWacuWLYqKitKtt96q06dPW22SkpK0fPlyLVu2TBs2bNCZM2c0cuRIFRcXW23Gjh2r9PR0rVy5UitXrlR6eroSExPtPF0AABAgHMYY4+tOVGbmzJn6/PPPtX79+grfN8YoOjpaSUlJmjFjhqTzo12RkZGaO3euHnjgAbndbl155ZV68803dccdd0iSjhw5ovbt22vFihUaNmyYvvrqK3Xr1k2bNm1S3759JUmbNm1S//79tWvXLl1zzTXV6m9ubq5cLpfcbreaN29eC1cAAADUNV98f/v1CNgHH3ygXr166Re/+IVat26tHj166LXXXrPez8jIUHZ2toYOHWptczqdGjhwoDZu3ChJSktLU2FhoVeb6OhoxcXFWW1SUlLkcrms8CVJ/fr1k8vlstpUJD8/X7m5uV4/AAAAF+PXAWz//v165ZVXFBsbq48//li/+c1vNHnyZP3v//6vJCk7O1uSFBkZ6fW5yMhI673s7Gw1btxYLVu2rLJN69atyx2/devWVpuKzJkzx6oZc7lcat++fc1PFgAABAy/DmAej0c9e/bU7Nmz1aNHDz3wwAOaMGGCXnnlFa92DofD67Uxpty2ssq2qaj9xfbz2GOPye12Wz9ZWVnVOS0AABDg/DqAtWnTRt26dfPa1rVrV2VmZkqSoqKiJKncKNWxY8esUbGoqCgVFBQoJyenyjbffPNNueN/++235UbXSnM6nWrevLnXDwAAwMX4dQC78cYb9fXXX3tt2717tzp27ChJiomJUVRUlJKTk633CwoKtG7dOg0YMECSFB8fr0aNGnm1OXr0qHbs2GG16d+/v9xutzZv3my1SU1NldvtttoAAADUlhBfd6AqDz/8sAYMGKDZs2drzJgx2rx5s5YsWaIlS5ZIOj9tmJSUpNmzZys2NlaxsbGaPXu2mjZtqrFjx0qSXC6X7rvvPk2dOlUREREKDw/XtGnT1L17dyUkJEg6P6o2fPhwTZgwQYsXL5Yk3X///Ro5cmS174AEAACoLr8OYL1799by5cv12GOP6emnn1ZMTIwWLFigcePGWW2mT5+uvLw8Pfjgg8rJyVHfvn21atUqhYWFWW3mz5+vkJAQjRkzRnl5eRoyZIiWLl2q4OBgq83bb7+tyZMnW3dLjh49WosWLbLvZAEAQMDw63XA6hvWAQMAoP5hHTAAAIAAQAADAACwGQEMAADAZgQwAAAAmxHAAAAAbEYAAwAAsJlfrwMGAPVFUbFHL63dpy0HTqp3p3A9NPgqhQTzNy6AihHAAKAWvLR2nxas3i0j6fO9xyVJUxJifdspAH6LP88AoBZsOXBSJatamwuvAaAyBDAAqAW9O4XLceF3x4XXAFAZpiABoBY8NPgqSfKqAQOAyhDAAKAWhAQHUfMFoNpqNAXpdrt18mT5+oaTJ08qNzf3sjsFAADQkNUogN15551atmxZue3vvPOO7rzzzsvuFAAAQENWowCWmpqqwYMHl9s+aNAgpaamXnanAAAAGrIaBbD8/HwVFRWV215YWKi8vLzL7hQAAEBDVqMA1rt3by1ZsqTc9ldffVXx8fGX3SkAAICGrEZ3QT7zzDNKSEjQf//7Xw0ZMkSStGbNGm3ZskWrVq2q1Q4CAAA0NDUaAbvxxhuVkpKidu3a6Z133tGHH36oLl266IsvvtAPf/jD2u4jAABAg+IwxpiLN0N15ObmyuVyye12q3nz5r7uDgAAqAZffH/X+FFE+/bt0+9+9zuNHTtWx44dkyStXLlSX375Za11DgBqS1GxRwtX79Fdf07VwtV7VFTs8XWXAASwGgWwdevWqXv37kpNTdV7772nM2fOSJK++OILPfHEE7XaQQCoDS+t3acFq3drw97jWrB6t15au8/XXQIQwGoUwGbOnKk//OEPSk5OVuPGja3tgwcPVkpKSq11DgBqy5YDJ1VSb2EuvAYAX6lRANu+fbt++tOfltt+5ZVX6sSJE5fdKQCobb07hctx4XfHhdcA4Cs1WoaiRYsWOnr0qGJiYry2b9u2TW3btq2VjgFAbXpo8FWSzo989e4Ubr0GAF+oUQAbO3asZsyYoXfffVcOh0Mej0eff/65pk2bprvvvru2+wgAly0kOEhTEmJ93Q0AkFTDKchnnnlGHTp0UNu2bXXmzBl169ZNN998swYMGKDf/e53td1HAACABuWy1gHbv3+/tm7dKo/Hox49eig2NrD/umQdMAAA6h9ffH/XaAqyROfOndW5c2cVFxdr+/btysnJUcuWLWurbwAAAA1SjaYgk5KS9Prrr0uSiouLNXDgQPXs2VPt27fXp59+Wpv9AwAAaHBqFMD+8Y9/6Prrr5ckffjhh9q/f7927dqlpKQkzZo1q1Y7CAAA0NDUKIAdP35cUVFRkqQVK1ZozJgxuvrqq3Xfffdp+/bttdpBAACAhqZGASwyMlI7d+5UcXGxVq5cqYSEBEnS2bNnFRwcXKsdBAAAaGhqVIR/7733asyYMWrTpo0cDoduvfVWSVJqaqquvfbaWu0gAABAQ1OjAPbkk0+qe/fuyszM1C9+8Qs5nU5J56cmJ0+eXKsdBAAAaGhqtA7Y008/XeH2vXv3avny5Tp9+vRld6w+Yh0wAADqn3qzDtjy5cu9XhcXFysrK0u5ubn6/e9/XysdAwAAaKhqFMC2bdtWbltRUZGSkpK0c+fOy+4UAABAQ3ZZK+F77SgkRElJSbruuutqa5cAAAANUo2WoajMwYMHFRMTU5u7BAAAaHBqNAL2wgsvlNuWnZ2tN954Q6NGjfJ6n7siAfhaUbFHL63dpy0HTqp3p3A9NPgqhQTX6t+fAHBJanQXZHVHuRwOh/bv33/JnaqvuAsS8E8LV+/RgtW7ZSQ5JCUlXK0pCbG+7hYAP1Fv7oLMyMio7X4AQJ3ZcuCkSv7SNBdeA4AvMQYPoMHr3Slcjgu/Oy68BgBfqrW7IAHAXz00+CpJ8qoBAwBfIoABaPBCgoOo+QLgV5iCBAAAsBkBDAAAwGYEMAAAAJtRAwYAfoxFZIGGiQAGAH7spbX7rEVkP997XJK4oQBoAPgzCgD8GIvIAg0TAQwA/BiLyAINE1OQAODHWEQWaJgIYADgx1hEFmiYmIIEAACwWb0KYHPmzJHD4VBSUpK1zRijJ598UtHR0QoNDdWgQYP05Zdfen0uPz9fkyZNUqtWrdSsWTONHj1ahw4d8mqTk5OjxMREuVwuuVwuJSYm6tSpUzacFQAACDT1JoBt2bJFS5Ys0XXXXee1fd68eXr++ee1aNEibdmyRVFRUbr11lt1+vRpq01SUpKWL1+uZcuWacOGDTpz5oxGjhyp4uJiq83YsWOVnp6ulStXauXKlUpPT1diYqJt5wcAAAKIqQdOnz5tYmNjTXJyshk4cKCZMmWKMcYYj8djoqKizLPPPmu1PXfunHG5XObVV181xhhz6tQp06hRI7Ns2TKrzeHDh01QUJBZuXKlMcaYnTt3Gklm06ZNVpuUlBQjyezatavSfp07d8643W7rJysry0gybre7Nk8fwCUoLCo2C5J3m3GvbTILknebwqJiX3cJgJ9zu922f3/XixGwhx56SCNGjFBCQoLX9oyMDGVnZ2vo0KHWNqfTqYEDB2rjxo2SpLS0NBUWFnq1iY6OVlxcnNUmJSVFLpdLffv2tdr069dPLpfLalOROXPmWFOWLpdL7du3r5XzBWpLUbFHC1fv0V1/TtXC1XtUVOzxdZfqXMnCpRv2HteC1bv10tp9vu4SAJTj93dBLlu2TFu3btWWLVvKvZednS1JioyM9NoeGRmpgwcPWm0aN26sli1blmtT8vns7Gy1bt263P5bt25ttanIY489pkceecR6nZubSwiDXwnEVdRZuBRAfeDXASwrK0tTpkzRqlWr1KRJk0rbORwOr9fGmHLbyirbpqL2F9uP0+mU0+ms8jiALwViGOndKVyf7z0uIxYuBeC//HoKMi0tTceOHVN8fLxCQkIUEhKidevW6YUXXlBISIg18lV2lOrYsWPWe1FRUSooKFBOTk6Vbb755ptyx//222/Lja4B9UkgrqL+0OCrlJRwtW7q0kpJCVezcCkAv+TXI2BDhgzR9u3bvbbde++9uvbaazVjxgx17txZUVFRSk5OVo8ePSRJBQUFWrdunebOnStJio+PV6NGjZScnKwxY8ZIko4ePaodO3Zo3rx5kqT+/fvL7XZr8+bN6tOnjyQpNTVVbrdbAwYMsOt0gVoXiKuos3ApgPrArwNYWFiY4uLivLY1a9ZMERER1vakpCTNnj1bsbGxio2N1ezZs9W0aVONHTtWkuRyuXTfffdp6tSpioiIUHh4uKZNm6bu3btbRf1du3bV8OHDNWHCBC1evFiSdP/992vkyJG65pprbDxjoHYRRgDAP/l1AKuO6dOnKy8vTw8++KBycnLUt29frVq1SmFhYVab+fPnKyQkRGPGjFFeXp6GDBmipUuXKjg42Grz9ttva/LkydbdkqNHj9aiRYtsPx8AANDwOYwx5uLNUB25ublyuVxyu91q3ry5r7sDXFRRsUcvrd3nNUUZEuzXpaEAUOt88f1d70fAANRcIC5TAQD+gD91gQAWiMtUAIA/IIABAcxXy1QE4gr9AFAaU5BAAPPVMhVMfQIIdAQwIID5apkKpj4BBDqmIAHYLhBX6AeA0hgBA2C7QFyhHwBKI4ABYj0su7FCP4BARwADRFE4AMBe/IkPiKJwAIC9CGCAKAoHANiLKUhAFIUDAOxFAANEUTgAwF4EMKAB4q5OAPBvBDCgAeKuTgDwb/xJDDRA3NUJAP6NAAY0QNzVCQD+jSlIoAHirk4A8G8EMKABqsldnRTuA4B9CGBAA3I5IYrCfQCwDwEMaEAuJ0RRuA8A9mF+AWhALidEUbgPAPZhBAy4TP5UO9W7U7g+33tcRlWHqIr6TOE+ANiHAAZcprqunbqUgFfdEFVZn6n5AgB7EMCAy1R22u+9rYdqdTTsUgJede9+pN4LAHyLGjDgMpWunZKkzJNntWHvcS1YvVsvrd132fuvi7BEvRcA+BYjYMBlKj3tl3nyrDJPnpVUu2GpOnVdl4J6LwDwLQIYcJlKT/stXL3Hmi7057BUk4VaAQC1hwAG1CLCEgCgOghgQC0qHZb8aXkKAIB/IYABl6GqkMWjfQAAlSGAAZehqpDFUg8AgMowH4IGq6jYo4Wr9+iuP6dq4eo9Kir21PoxqgpZLPUAAKgMI2BosOyYAqxqiYjqFuRTKwYAgYcAhgbLjinAqkJWde9epFYMAAIPAQwNVl0sYFpWbSwRQa0YAAQeAhgarPqy2rsdQREA4F8IYGiw6ssCplUFRerDAKBhIoABPlZVUKQ+DAAaJgIYAkptjChVtY/aHrGiPgwAGiYCGAJKbYwoVbWP2h6xqqg+jGlJAKj/CGAIKLUxolR2H+9tPWSFoc0ZJ2p1xKqi+jCmJQGg/iOAIaDUxh2HpfchSZknzyrz5Flt2Htc/WLOr35fW3c0VlQfxrQkANR/BDAElNpYmqL0PrYfdsudV2i9d/hUnpISrq7TpS9YtgIA6j8CGAJKbSxNUXofN89b6xXAHA5Htfdf01qu+rK+GQCgcgQw4DL8tEe0Fq7Z6/W6usrWcnmMR0GOoIsGsvqyvhkAoHIEMOAyTLoltlxoqq6ytVzLtx1R1smzFNcDQAAggKFBq+slG6oajbrYscvWckmiuB4AAgQBDA2aL5dsuNixy9ZyeTxGL3yyh+J6AAgABDA0aL5csuFixy47elZU7FFQkIPiegAIAAQw+IWaTBVe7DNFxR4Ve4zXZzJPntXC1XtsWT3+UpeLoLgeAAIHAQx+oSZThZV9piSYvbf1kDJPnvX6TObJs1qwene19l9aTQIiy0UAACpDAINfqMlUYWWfKR3MKlKTqcjqBMSKQhojWgCAivAEX/iF3p3CrTsBq1uAXtlnSgezitSkwL06AbEkpG3Ye1wLVu/WS2v3XdIxAACBgxEw+IWaTNdV9pmyz2rsEN70/AKpxqG0zJwaTQdWp56LZzQCAKrLrwPYnDlz9P7772vXrl0KDQ3VgAEDNHfuXF1zzTVWG2OMnnrqKS1ZskQ5OTnq27evXnrpJf3gBz+w2uTn52vatGn629/+pry8PA0ZMkQvv/yy2rVrZ7XJycnR5MmT9cEHH0iSRo8erRdffFEtWrSw7XwDWU3W06rsMxUFs8rqtapb21WdgMgzGgEA1eUwxlQ1W+NTw4cP15133qnevXurqKhIs2bN0vbt27Vz5041a9ZMkjR37lw988wzWrp0qa6++mr94Q9/0Geffaavv/5aYWFhkqTf/va3+vDDD7V06VJFRERo6tSpOnnypNLS0hQcHCxJuu2223To0CEtWbJEknT//ferU6dO+vDDD6vd39zcXLlcLrndbjVv3ryWr0bgWrh6j1V/5ZCUlHB1hcGrJoXy1d13ddT1oq8AgLrhi+9vvw5gZX377bdq3bq11q1bp5tvvlnGGEVHRyspKUkzZsyQdH60KzIyUnPnztUDDzwgt9utK6+8Um+++abuuOMOSdKRI0fUvn17rVixQsOGDdNXX32lbt26adOmTerbt68kadOmTerfv7927drlNeJWFQJY3bjrz6nacKHwXZJu6tJKb/26b7l2FYWphwZfVWUoKrvvDuFN1SG8aa2NnAEA/J8vvr/9egqyLLfbLUkKDz8/tZORkaHs7GwNHTrUauN0OjVw4EBt3LhRDzzwgNLS0lRYWOjVJjo6WnFxcdq4caOGDRumlJQUuVwuK3xJUr9+/eRyubRx48ZKA1h+fr7y8/Ot17m5ubV6vjivulN7mzNOeNVgbc44IUlV3r1Ytl4s8+RZZZ48e9GlMHy5wj4AoP6rNwHMGKNHHnlEN910k+Li4iRJ2dnZkqTIyEivtpGRkTp48KDVpnHjxmrZsmW5NiWfz87OVuvWrcsds3Xr1labisyZM0dPPfVUzU8KlSo9whTfoaUmD+mitIOnqiygL7Pmqjzm4oXxpWu7SsJXZW1Lo+AeAHA56s2cycSJE/XFF1/ob3/7W7n3HA6H12tjTLltZZVtU1H7i+3nsccek9vttn6ysrIudhqoptJLOrzwyR4FOYL01q/7akpCbKVTfUFl/qc6lHNW8R1aVrm8RUkh/1u/7quf9WxXrm1RsUcLV+/RXX9O1cLVe1RU7JFUs2UzAAAoUS9GwCZNmqQPPvhAn332mdedi1FRUZLOj2C1adPG2n7s2DFrVCwqKkoFBQXKycnxGgU7duyYBgwYYLX55ptvyh3322+/LTe6VprT6ZTT6by8k0OFqjPC5DVK1rFFuccOZeXkSQ6jpISrq7W8RUV3OlY21VgXq9xTVwYAgcOvA5gxRpMmTdLy5cv16aefKiYmxuv9mJgYRUVFKTk5WT169JAkFRQUaN26dZo7d64kKT4+Xo0aNVJycrLGjBkjSTp69Kh27NihefPmSZL69+8vt9utzZs3q0+fPpKk1NRUud1uK6TBXtWp+yodjkoX0peWdvBUhQX7FaloWYvKgmBdPLeRujIACBx+HcAeeugh/fWvf9X//d//KSwszKrHcrlcCg0NlcPhUFJSkmbPnq3Y2FjFxsZq9uzZatq0qcaOHWu1ve+++zR16lRFREQoPDxc06ZNU/fu3ZWQkCBJ6tq1q4YPH64JEyZo8eLFks4vQzFy5Mhq3wGJ6rnUdbc2Z5yQx5z/vwtXy6v9pa54X9WxK3vPzrW9qCsDgMDh1wHslVdekSQNGjTIa/sbb7yh8ePHS5KmT5+uvLw8Pfjgg9ZCrKtWrbLWAJOk+fPnKyQkRGPGjLEWYl26dKm1Bpgkvf3225o8ebJ1t+To0aO1aNGiuj3BAFTdUZ6SEaaFq7+/i3HjvhNe7cvewVhah/Cm+lnPdl5Tg1Ud286pxsqwkCsABA6/DmDVWaLM4XDoySef1JNPPllpmyZNmujFF1/Uiy++WGmb8PBwvfXWWzXpZsC7lNqlikZ5Kvq8JL34yR4tXre/0lGhhwZfJY/xaPm2IzLGqG2LUAUHOdQnJqLCPlQ2wlRU7NF7Ww/ZNtVYGTvDHgDAt/w6gKF+KFuL9d7WQ16jT6XDVXzHFuVGeSoafZKkhWv2ljtW6VGhkoBVsnREVk6eJl045vg3tqh3p3A9cHOMFn+WoS0HTnoV6Ze+yzHx9c3WPio6jl3sDHsAAN8igOGyla3Fyjx5VgtW77Zelw5Xk2+Jte5KjO/YQh6P0V9SDlSr9qlDeNNyo0LLtx3xev2/mzKVm1doHW/T/hPatP+EFfj6d45QcJDD6y7HlP0nLnocAABqEwEMl62iWqzSQap0uErLzLHuSly4eo8WrNldroYr8+RZtW0RWu44P72hrV78ZI8Vun56Q9ty09T5hcVex/vqaK7X6+Agh9ddkRWFvZ/1bMfyDwCAOsW3DGqsZJHSzRkn1K9zhNq3/D40lUzxVbZgaVGxR/9Iy/IKX8EXVlLNPHlWm/afUL+YcOvZjFNuiZUcRgvX7LVWrF/4yZ5yQe2G9i28jte1TfNKj79w9Z5yU4/9O0cw+gUAqHOMgKHGStduOSRNviVWQUGOCovIyy4n4fGY8wulllK6RsvofE3UZ9MHW9vu+nNquT4EBzn0cKmFVkvXfFX0unRdWknfJe+7Jhn9AgDUNQIYaqzsXYWlpxfL3tnYq1NLvbBm7/narH0n5AptdNH9ly2E790pvNyCq31iIsoVrl/steT94G5Jat8ylAJ4AIBtCGCosarWrSp7Z2P78KZegcedV1jlviuaCiy97IR0vgasptOFFT24GwAAuxDAcEnKPn9x8i2xSsvMse5ovOvPqerdKdxrhMmo6jXdHJJXOOsQ3lRv3ten0lXqP5k68LKnCcs+uLvsawAA6hLFLrgkJSNbG/Ye1wtr9irowl2FQY4gvfDJHm3Ye1zzV+/WjiNu6zMOqcK7Gkv0jfEu1G/bIlTj39iihav3WOGr5JgLVu/WS2v3XbSfJUX2d/051dpPaX1iIryO2Scm4pKuAwAAl4MRMFySylaTL7sWmDuvSNL3xe2bM7zX2irNGKP24U0lSdGuJta6XCWLslZ39fzSo2Jlp0A37T/htf4Xq84DAHyJAIZLUrbuq/jCtGNxJUVUuRdqvXp1DNfn+yoOYakHcqzfS9eGlYStimrNKnt2Y0kwe+PzDK/QVjbUEbgAAL5EAEOlzhUU6d6l/9FXR3PVtU1zvTG+l9fIUbHHWMHGIaldy1AdP52vc0XfT/edyivUgtW7NXlIF/WLCdemjIpXuS9Rtji/9OhU6dGq8W9s8QpYJctbvLf1ULm1vUorCXUvrVW1HgoOAEBdoAYMlbp36X+Usv+ETuUVKmX/Cd279D/W8wrf+nVfOUpNOhpJh3LyrPDlDAnyeu8/B3J0xH3uko7vDAmSx2M0/o0t8hiPerRrofe2HtItf1rnVdPlkHTw5FnNX727wvDVroIFYiubSgUAwA6MgKFSXx3NrfJ1ZpmFVEtr0ihY+aVGwjym6jshK1LsMXrhkz3WQ769jn3yrPVcx9IjcRXpGN5UP+vRTsvTD1/oi0fxHVpWuoQGAAB1jREwVKprm+Zer69oEuI18nT6XOVreeUVFHm93nnULc8lBrAgh/fyFGWVPNcx+CJrSPSJiVBQkENZFx5h9MKavZLDKCnhat3UpZWSEq6mJgwAYCsCWAC72FINb4zv5TV9dygnz2sJiG5tXF7tS087FhR7Ryd3XpEOnyo/BekK9R6EbdcyVC1CG6ldy9By+yirZNSq7OhVv5hwTbklVjd1aaWHL4Srcqv2HzxlTaVOSYjl8UMAAFsxBRmgioo9Snx9c7m7A0sXojdpHKJOEc10qNRUY+laqTfG9/Iq0v/yyCnlew98XdS5Qo/atQiVHNLpc0VySLqnfyf95+BJr+O2bxmqaFcT/feQW85GQbq7X0dr1KqiIv2ygaqqVfsBALAbASxAvbR2n1fdVGWF6PEdWnrVX/Vo79L8Vbuteqqf9ojWG+N7afFnGfqy1OKr1ZVf5NGhU98HLXdeoRZ+sqdc4Xy7lk21af/51fXzizwKCQ62QlbJjQFVYd0vAIA/IYAFqIrCVu9O4eUWOPUYT5nP5XgtJbFwzV4t3Xjwos92vFSlR7/6dY7wqgczOr/cREUjXZWpTkgDAMAuFL4EqLJTcCUPvy772J9/ph/xarctM0dl1Xb4Kis4yFHuUUGZJ89W65FEAAD4IwJYgHrg5hj1iwmXM9ghZ0iQDuWc1Yuf7Cn3EO3cc95FXfkXKYyvDW1dTbye01gyZdjhwuOKSrB2FwCgvmIKMkAt/iyj1FSiUVZOnhau2Xu+IL6UcwXlq+qDHdLl5jCHKl9i4vh3Bda0Y5+YCGuq8Wc921mr11NIDwCozwhgAaio2KP3th6q8L3SBfFSxSNeNQlfzpAg9ezQUg4ZZebkedV4lZVf5NGm/Ses9blKatLiO7bQ5FtilZaZQyE9AKBeI4AFoJfW7qvyeYl14dc3dtKjt3XV/FW7tXH/nou2L7kr88VPPFq4Zq+k86vh94sJZ80uAEC9RwALAGXvbNycUflje+rKm6mZatwoRO9vq3jkraySKcayI3Ul06Y8QBsAUJ8xlBAAyt7ZWOypfA4xrEndZPLcc0Wav3q3cqt4fFGJDuFNL/p4IB6gDQCozxgBCwBlH8Nz+FTl9Venz13iUvaXqHmTRnLnVXyMDuFN9bOe7bzW9/rpDW218JPyU5YU4QMA6jMCWAAo/Rgeqe7X7aqKO69A/TtHyOGQPB6jw6fy5HA49NMb2mrSkC7l6rsmDemioCDH+SL8Di0lh1HawVMU4QMA6jWHMabuF3YKELm5uXK5XHK73WrevLmvu2Mp+9xHX3NISkq4mvotAIBf8MX3NzVgASAkOEjBQY6LN6wlzpCq/7OifgsAEOgIYAEivmML246VX+Qpt80V2sj6nfotAECgowasgStZguK9tOot/1AX2rZoojWPDNTizzKspTCo3wIABDICWANXsgSFLwv9TpwpUJPGIbVa81V2bbPSd04CAODvCGANXOklKHzF2aj2g1HpYMmirACA+oYA1kAVFXs0/+PdfnHn4939Otb6PsuubUZRPwCgPiGANTAlU3P/SMtSVhUPvK5r/TtHKDjIUWf1XqXXNqOoHwBQ3xDAGpiFq3frxbX7fNqH/p0j9OZ9feq0Jqsk1FHUDwCojwhgDUTJyNfLn/omfDlDgtSkUbCaNwlRn5iW5fpV28XyIcFB1HwBAOotAlgD4euRr54dWmrT/hNy5xXqhTV7FeQ4H5AolgcAoDwCWANQVOzR4s/2237c4CCH+nZqqb6dW2lzxokKi+IplgcAoDwWTqrnzhUUadAfP1VBsf2LTUS7muiv9/fXlIRY9YmJUMnDjkoXxffuFF7hdgAAAhkjYPVYUbFHQ/60Tofd53xy/LYtQq3fKyuKp1geAIDyCGD1VFGxR3f9OdVn4UuS1wO+KyuKp1geAIDyCGD1UFGxR+Ne26TUAzk+7UefmAifHh8AgPqKAFYPvbhmr+3hq19MuA6fylPuuSI1bxKi23u2ZToRAIAaIoDVQ+9vO2TbsYId0uQhV/OwawAAahEBrB46djrftmNFtwilhgsAgFpGAKtHioo9+uO/dym/yGPbMW/v0c62YwEAECgIYPXIn1Z+rcUbMur0GNHNnfpF7/ZKO3iKZSMAAKgjBLB64tR35/TK+rpb7T5IUp+YcC29t7eaNOY/CwAA6hLftH7uXEGRxi3ZqLRDp+v0OAO6tNJbv+5bp8cAAADncVubn0v886Y6D18SjwgCAMBOjID5uS2Z7jrdf4fwpvpZz3bUegEAYCMCWACJbu5UzJVXyGOkIMf5lexZ3wsAAPsRwAKAM8ShHu1bUmAPAICf4Nu4gfvPrMFqFdbU190AAAClMPdUxssvv6yYmBg1adJE8fHxWr9+va+7VCMOSQ8nXE34AgDADxHASvn73/+upKQkzZo1S9u2bdMPf/hD3XbbbcrMzPR116qtTVhj3XhVhJISrqawHgAAP+Uwxhhfd8Jf9O3bVz179tQrr7xibevatat+8pOfaM6cOeXa5+fnKz//++cy5ubmqn379nK73WrevHmt9KnTzI+q1a5fpxbq36U1RfUAAFyi3NxcuVyuWv3+vhhqwC4oKChQWlqaZs6c6bV96NCh2rhxY4WfmTNnjp566ik7ulepewe01xOjr/NpHwAAwKVhqOSC48ePq7i4WJGRkV7bIyMjlZ2dXeFnHnvsMbndbusnKyvLjq5aptwSq1kj4mw9JgAAuHyMgJXhcDi8Xhtjym0r4XQ65XQ67eiWl3t6R+mpn8XbflwAAFA7CGAXtGrVSsHBweVGu44dO1ZuVMxOB54d4bNjAwCAusEU5AWNGzdWfHy8kpOTvbYnJydrwIABPuoVAABoiBgBK+WRRx5RYmKievXqpf79+2vJkiXKzMzUb37zG193DQAANCAEsFLuuOMOnThxQk8//bSOHj2quLg4rVixQh07dvR11wAAQAPCOmC1yBfriAAAgMvji+9vasAAAABsRgADAACwGQEMAADAZgQwAAAAmxHAAAAAbEYAAwAAsBkBDAAAwGYEMAAAAJuxEn4tKlnTNjc318c9AQAA1VXyvW3n2vQEsFp0+vRpSVL79u193BMAAHCpTp8+LZfLZcuxeBRRLfJ4PDpy5IjCwsLkcDhqbb+5ublq3769srKyAv4RR1yL73EtvHE9vse1+B7X4ntci++VvRbGGJ0+fVrR0dEKCrKnOosRsFoUFBSkdu3a1dn+mzdvHvD/T1OCa/E9roU3rsf3uBbf41p8j2vxvdLXwq6RrxIU4QMAANiMAAYAAGAzAlg94HQ69cQTT8jpdPq6Kz7Htfge18Ib1+N7XIvvcS2+x7X4nj9cC4rwAQAAbMYIGAAAgM0IYAAAADYjgAEAANiMAAYAAGAzAlg98PLLLysmJkZNmjRRfHy81q9f7+suXZY5c+aod+/eCgsLU+vWrfWTn/xEX3/9tVcbY4yefPJJRUdHKzQ0VIMGDdKXX37p1SY/P1+TJk1Sq1at1KxZM40ePVqHDh3yapOTk6PExES5XC65XC4lJibq1KlTdX2KNTJnzhw5HA4lJSVZ2wLtOhw+fFh33XWXIiIi1LRpU91www1KS0uz3g+U61FUVKTf/e53iomJUWhoqDp37qynn35aHo/HatNQr8Vnn32mUaNGKTo6Wg6HQ//85z+93rfzvDMzMzVq1Cg1a9ZMrVq10uTJk1VQUFAXp12hqq5FYWGhZsyYoe7du6tZs2aKjo7W3XffrSNHjnjtIxCuRVkPPPCAHA6HFixY4LXd766FgV9btmyZadSokXnttdfMzp07zZQpU0yzZs3MwYMHfd21Ghs2bJh54403zI4dO0x6eroZMWKE6dChgzlz5ozV5tlnnzVhYWHmvffeM9u3bzd33HGHadOmjcnNzbXa/OY3vzFt27Y1ycnJZuvWrWbw4MHm+uuvN0VFRVab4cOHm7i4OLNx40azceNGExcXZ0aOHGnr+VbH5s2bTadOncx1111npkyZYm0PpOtw8uRJ07FjRzN+/HiTmppqMjIyzOrVq83evXutNoFyPf7whz+YiIgI869//ctkZGSYd99911xxxRVmwYIFVpuGei1WrFhhZs2aZd577z0jySxfvtzrfbvOu6ioyMTFxZnBgwebrVu3muTkZBMdHW0mTpxY59egRFXX4tSpUyYhIcH8/e9/N7t27TIpKSmmb9++Jj4+3msfgXAtSlu+fLm5/vrrTXR0tJk/f77Xe/52LQhgfq5Pnz7mN7/5jde2a6+91sycOdNHPap9x44dM5LMunXrjDHGeDweExUVZZ599lmrzblz54zL5TKvvvqqMeb8Pz6NGjUyy5Yts9ocPnzYBAUFmZUrVxpjjNm5c6eRZDZt2mS1SUlJMZLMrl277Di1ajl9+rSJjY01ycnJZuDAgVYAC7TrMGPGDHPTTTdV+n4gXY8RI0aYX/3qV17bbr/9dnPXXXcZYwLnWpT9orXzvFesWGGCgoLM4cOHrTZ/+9vfjNPpNG63u07OtypVhY4SmzdvNpKsP9AD7VocOnTItG3b1uzYscN07NjRK4D547VgCtKPFRQUKC0tTUOHDvXaPnToUG3cuNFHvap9brdbkhQeHi5JysjIUHZ2ttd5O51ODRw40DrvtLQ0FRYWerWJjo5WXFyc1SYlJUUul0t9+/a12vTr108ul8uvrt9DDz2kESNGKCEhwWt7oF2HDz74QL169dIvfvELtW7dWj169NBrr71mvR9I1+Omm27SmjVrtHv3bknSf//7X23YsEE/+tGPJAXWtSjNzvNOSUlRXFycoqOjrTbDhg1Tfn6+17S4P3G73XI4HGrRooWkwLoWHo9HiYmJevTRR/WDH/yg3Pv+eC14GLcfO378uIqLixUZGem1PTIyUtnZ2T7qVe0yxuiRRx7RTTfdpLi4OEmyzq2i8z548KDVpnHjxmrZsmW5NiWfz87OVuvWrcsds3Xr1n5z/ZYtW6atW7dqy5Yt5d4LpOsgSfv379crr7yiRx55RP/zP/+jzZs3a/LkyXI6nbr77rsD6nrMmDFDbrdb1157rYKDg1VcXKxnnnlGv/zlLyUF3n8bJew87+zs7HLHadmypRo3buyX1+bcuXOaOXOmxo4daz1cOpCuxdy5cxUSEqLJkydX+L4/XgsCWD3gcDi8Xhtjym2rryZOnKgvvvhCGzZsKPdeTc67bJuK2vvL9cvKytKUKVO0atUqNWnSpNJ2Df06lPB4POrVq5dmz54tSerRo4e+/PJLvfLKK7r77rutdoFwPf7+97/rrbfe0l//+lf94Ac/UHp6upKSkhQdHa177rnHahcI16Iidp13fbk2hYWFuvPOO+XxePTyyy9ftH1DuxZpaWlauHChtm7desn98eW1YArSj7Vq1UrBwcHlUvWxY8fKJfD6aNKkSfrggw+0du1atWvXztoeFRUlSVWed1RUlAoKCpSTk1Nlm2+++abccb/99lu/uH5paWk6duyY4uPjFRISopCQEK1bt04vvPCCQkJCrD429OtQok2bNurWrZvXtq5duyozM1NS4Px3IUmPPvqoZs6cqTvvvFPdu3dXYmKiHn74Yc2ZM0dSYF2L0uw876ioqHLHycnJUWFhoV9dm8LCQo0ZM0YZGRlKTk62Rr+kwLkW69ev17Fjx9ShQwfr39KDBw9q6tSp6tSpkyT/vBYEMD/WuHFjxcfHKzk52Wt7cnKyBgwY4KNeXT5jjCZOnKj3339fn3zyiWJiYrzej4mJUVRUlNd5FxQUaN26ddZ5x8fHq1GjRl5tjh49qh07dlht+vfvL7fbrc2bN1ttUlNT5Xa7/eL6DRkyRNu3b1d6err106tXL40bN07p6enq3LlzQFyHEjfeeGO55Uh2796tjh07Sgqc/y4k6ezZswoK8v7nOTg42FqGIpCuRWl2nnf//v21Y8cOHT161GqzatUqOZ1OxcfH1+l5VldJ+NqzZ49Wr16tiIgIr/cD5VokJibqiy++8Pq3NDo6Wo8++qg+/vhjSX56LS6pZB+2K1mG4vXXXzc7d+40SUlJplmzZubAgQO+7lqN/fa3vzUul8t8+umn5ujRo9bP2bNnrTbPPvuscblc5v333zfbt283v/zlLyu81bxdu3Zm9erVZuvWreaWW26p8Jbi6667zqSkpJiUlBTTvXt3v1puoKzSd0EaE1jXYfPmzSYkJMQ888wzZs+ePebtt982TZs2NW+99ZbVJlCuxz333GPatm1rLUPx/vvvm1atWpnp06dbbRrqtTh9+rTZtm2b2bZtm5Fknn/+ebNt2zbrzj67zrtkuYEhQ4aYrVu3mtWrV5t27drZuvRCVdeisLDQjB492rRr186kp6d7/Vuan58fUNeiImXvgjTG/64FAaweeOmll0zHjh1N48aNTc+ePa3lGuorSRX+vPHGG1Ybj8djnnjiCRMVFWWcTqe5+eabzfbt2732k5eXZyZOnGjCw8NNaGioGTlypMnMzPRqc+LECTNu3DgTFhZmwsLCzLhx40xOTo4NZ1kzZQNYoF2HDz/80MTFxRmn02muvfZas2TJEq/3A+V65ObmmilTppgOHTqYJk2amM6dO5tZs2Z5fbE21Guxdu3aCv99uOeee4wx9p73wYMHzYgRI0xoaKgJDw83EydONOfOnavL0/dS1bXIyMio9N/StWvXWvsIhGtRkYoCmL9dC4cxxlzamBkAAAAuBzVgAAAANiOAAQAA2IwABgAAYDMCGAAAgM0IYAAAADYjgAEAANiMAAYAAGAzAhgAAIDNCGAAAlanTp20YMECX3dDkvTpp5/K4XDo1KlTvu4KABsQwAD4pfHjx8vhcFg/ERERGj58uL744gvb+jBo0CAlJSXZdjwAgYMABsBvDR8+XEePHtXRo0e1Zs0ahYSEaOTIkb7ulhdjjIqKinzdDQD1DAEMgN9yOp2KiopSVFSUbrjhBs2YMUNZWVn69ttvJUkzZszQ1VdfraZNm6pz5856/PHHVVhY6LWPDz74QL169VKTJk3UqlUr3X777ZUe74033pDL5VJycrLGjx+vdevWaeHChdYo3IEDB6ypwo8//li9evWS0+nU+vXrtW/fPv34xz9WZGSkrrjiCvXu3VurV6/22n9+fr6mT5+u9u3by+l0KjY2Vq+//nqFfcnLy9OIESPUr18/nTx5UgUFBZo4caLatGmjJk2aqFOnTpozZ85lXmEAvhLi6w4AQHWcOXNGb7/9trp06aKIiAhJUlhYmJYuXaro6Ght375dEyZMUFhYmKZPny5J+uijj3T77bdr1qxZevPNN1VQUKCPPvqowv0/99xzmjNnjj7++GP169dPffr00e7duxUXF6enn35aknTllVfqwIEDkqTp06frueeeU+fOndWiRQsdOnRIP/rRj/SHP/xBTZo00V/+8heNGjVKX3/9tTp06CBJuvvuu5WSkqIXXnhB119/vTIyMnT8+PFyfXG73Ro5cqSaNGmiNWvWqFmzZnruuef0wQcf6J133lGHDh2UlZWlrKys2r7MAOxiAMAP3XPPPSY4ONg0a9bMNGvWzEgybdq0MWlpaZV+Zt68eSY+Pt563b9/fzNu3LhK23fs2NHMnz/fzJw507Rp08Z88cUXXu8PHDjQTJkyxWvb2rVrjSTzz3/+86Ln0K1bN/Piiy8aY4z5+uuvjSSTnJxcYduS/e7atctcf/315vbbbzf5+fnW+5MmTTK33HKL8Xg8Fz0uAP/HFCQAvzV48GClp6crPT1dqampGjp0qG677TYdPHhQkvSPf/xDN910k6KionTFFVfo8ccfV2ZmpvX59PR0DRkypMpj/OlPf9LixYu1YcMGde/evdp969Wrl9fr7777TtOnT1e3bt3UokULXXHFFdq1a5fVn/T0dAUHB2vgwIFV7jchIUGdO3fWO++8o8aNG1vbx48fr/T0dF1zzTWaPHmyVq1aVe2+AvA/BDAAfqtZs2bq0qWLunTpoj59+uj111/Xd999p9dee02bNm3SnXfeqdtuu03/+te/tG3bNs2aNUsFBQXW50NDQy96jB/+8IcqLi7WO++8c8l9K+3RRx/Ve++9p2eeeUbr169Xenq6unfvbvWnOn2RpBEjRmj9+vXauXOn1/aePXsqIyNDv//975WXl6cxY8bo5z//+SX1GYD/oAYMQL3hcDgUFBSkvLw8ff755+rYsaNmzZplvV8yMlbiuuuu05o1a3TvvfdWus8+ffpo0qRJGjZsmIKDg/Xoo49a7zVu3FjFxcXV6tv69es1fvx4/fSnP5V0vmatpF5Mkrp37y6Px6N169YpISGh0v08++yzuuKKKzRkyBB9+umn6tatm/Ve8+bNdccdd+iOO+7Qz3/+cw0fPlwnT55UeHh4tfoIwH8QwAD4rfz8fGVnZ0uScnJytGjRIp05c0ajRo2S2+1WZmamli1bpt69e+ujjz7S8uXLvT7/xBNPaMiQIbrqqqt05513qqioSP/+97+tIv0S/fv317///W8NHz5cISEhevjhhyWdX6g1NTVVBw4c0BVXXFFl0OnSpYvef/99jRo1Sg6HQ48//rg8Ho/1fqdOnXTPPffoV7/6lVWEf/DgQR07dkxjxozx2tdzzz2n4uJi3XLLLfr000917bXXav78+WrTpo1uuOEGBQUF6d1331VUVJRatGhxOZcYgI8wBQnAb61cuVJt2rRRmzZt1LdvX23ZskXvvvuuBg0apB//+Md6+OGHNXHiRN1www3auHGjHn/8ca/PDxo0SO+++64++OAD3XDDDbrllluUmppa4bFuvPFGffTRR3r88cf1wgsvSJKmTZum4OBgdevWTVdeeaVXfVlZ8+fPV8uWLTVgwACNGjVKw4YNU8+ePb3avPLKK/r5z3+uBx98UNdee60mTJig7777rtL9jRkzRrfccot2796tK664QnPnzlWvXr3Uu3dvHThwQCtWrFBQEP+MA/WRwxhjfN0JAACAQMKfTgAAADYjgAEAANiMAAYAAGAzAhgAAIDNCGAAAAA2I4ABAADYjAAGAABgMwIYAACAzQhgAAAANiOAAQAA2IwABgAAYLP/D4cJb2BFBFKgAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sudata.plot.scatter(x='Backtracks', y='μsec', marker='.');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This plot makes it clear that there is a strong linear relation between number of backtracks and run time. (Very roughly, one microsecond per backtrack.)\n", "\n", "One weakness of a scatter plot as a visualization tool is that there can be lots of points plotted on top of each other, and you can't tell the density of such points. A histogram portrays density better (but on only one attribute at a time):" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjoAAAGxCAYAAABr1xxGAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjYsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvq6yFwwAAAAlwSFlzAAAPYQAAD2EBqD+naQAALrRJREFUeJzt3X10VPWdx/HPJCQTgkkkIIEQgvEcH4gBlCRmg7gQK8EIuIDbUtGAT9uyCRqMolhaC7QKLZayPQ6sWFeqFuWwNWxVdnGsCrhgeUwrjVQ5DQR5MAY1gQTCZPLbP7qZOAaEC5MM9877dQ7nOPf+5nt/98sl+Xjn3jsuY4wRAACAA0WFewIAAACdhaADAAAci6ADAAAci6ADAAAci6ADAAAci6ADAAAci6ADAAAci6ADAAAci6ADAAAci6ADONyKFSvkcrmC/lxyySUaNWqUXn/99U7b7qhRo5SVlXXGcU8++aTWrFnTafM4k71798rlcumpp54K2xwAdB6CDhAhnn/+eW3evFmbNm3S8uXLFR0drfHjx+u1114L67zCHXQAOFu3cE8AQNfIyspSTk5O4PXNN9+snj176uWXX9b48ePDOLOzd/z4ccXFxcnlcoV7KgBsgjM6QISKi4tTbGysYmJiAsvmzZunvLw8JScnKzExUcOGDdNzzz2nU33378qVK5Wfn6+LLrpIF110ka655ho999xz37jNiooKxcfH67777lNLS4tcLpcaGxv1m9/8JvCx2qhRoyS1f+T25ptv6p577tEll1yi+Ph4NTc3a8+ePbr77rt1+eWXKz4+Xv3799f48eP1wQcfdNjml19+qYceekiXXXaZ3G63+vTpo1tuuUW7d+8+7Tx9Pp+mTZumiy66KPDxXlNTkx5++GFlZGQoLi5OycnJysnJ0csvv3w27QYQJpzRASKE3+9XS0uLjDH69NNPtWjRIjU2NmrKlCmBMXv37tX3v/99paenS5Lef/993X///Tpw4IAef/zxwLjHH39cP/nJTzRp0iQ99NBDSkpK0q5du7Rv377Tbv+Xv/ylZs2apblz5+qHP/yhJGnz5s268cYbVVBQoB/96EeSpMTExKD33XPPPRo7dqxefPFFNTY2KiYmRgcPHlSvXr20cOFCXXLJJfr888/1m9/8Rnl5edq5c6euvPJKSdLRo0c1YsQI7d27V48++qjy8vJ07NgxbdiwQYcOHdJVV13VYZ5ffvmlJk2apA8//FDr169Xdna2JKm8vFwvvviifvrTn+raa69VY2Ojdu3apSNHjpzLXweArmIAONrzzz9vJHX443a7zdKlS0/7Pr/fb3w+n5k/f77p1auXaW1tNcYY87e//c1ER0ebO+644xu3O3LkSHP11Vcbv99vZsyYYWJjY81LL73UYVyPHj3MtGnTTjvvqVOnnnEfW1pazMmTJ83ll19uHnzwwcDy+fPnG0nG6/We9r3V1dVGklm0aJGprq42mZmZJjMz0+zduzdoXFZWlpkwYcIZ5wLgwsIZHSBCvPDCCxo0aJAkqa6uThUVFSotLZXf79eMGTMkSW+//baefPJJbd26VQ0NDUHvr62tVUpKirxer/x+v0pLS8+4zRMnTmjChAl677339Oabb2rkyJGW533bbbd1WNbS0qKf//zneumll7Rnzx75fL7Aug8//DDw3//93/+tK664QjfddNMZt7Njxw499dRTyszM1KuvvqqLL744aP11112n3/72t5o9e7Zuvvlm5eXlqXv37pb3B0DXIugAEWLQoEEdLkbet2+fHnnkEd1555366KOPVFhYqFGjRunZZ59VWlqaYmNjtWbNGj3xxBM6fvy4JOmzzz6TJKWlpZ1xm7W1tdq/f79uuukmDR8+/Jzm3a9fvw7LysvL5fF49Oijj2rkyJHq2bOnoqKidN999wXm2TbXto/hzsTr9aqurk6LFy/uEHIk6Ve/+pXS0tK0atUq/exnP1NcXJzGjBmjRYsW6fLLLz+nfQPQ+bgYGYhgQ4YM0fHjx/XRRx/plVdeUUxMjF5//XV95zvf0fDhw4OCUZtLLrlEkvTJJ5+csX56erpee+01vfvuu5o0aZJOnDhheY6nusPqpZde0tSpU/Xkk09qzJgxuu6665STk6O6uroOcz2beUrSrFmz9L3vfU9Tp07VCy+80GF9jx49NG/ePO3evVuHDx/WsmXL9P7779vmjjUgUhF0gAhWWVkp6e+BwOVyqVu3boqOjg6sP378uF588cWg9xQWFio6OlrLli07q20UFhZq3bp12rBhg8aNG6fGxsag9W63O+gszNlwuVxyu91By9544w0dOHAgaFlRUZE++ugjvf3222esGRUVpWeeeUZlZWW66667vnH/UlJSdNddd+n222/XX//6VzU1NVmaP4Cuw0dXQITYtWuXWlpaJElHjhzRq6++Kq/Xq4kTJyojI0Njx47V4sWLNWXKFH3ve9/TkSNH9NRTT3UIFJdeeql+8IMf6Cc/+YmOHz+u22+/XUlJSaqqqlJdXZ3mzZvXYdsjRozQH/7wB918880qLCzU2rVrlZSUJEkaPHiw3n33Xb322mvq16+fEhISAndNnc64ceO0YsUKXXXVVRoyZIi2b9+uRYsWdfg4bebMmVq1apX+6Z/+SbNnz9Z1112n48ePa/369Ro3bpwKCgo61P7FL36hhIQElZSU6NixY5o1a5YkKS8vT+PGjdOQIUPUs2dPffjhh3rxxReVn5+v+Pj4s/+LANC1wn01NIDOdaq7rpKSksw111xjFi9ebE6cOBEY+x//8R/myiuvNG6321x22WVmwYIF5rnnnjOSTHV1dVDdF154weTm5pq4uDhz0UUXmWuvvdY8//zzgfVtd1191a5du0zfvn3NsGHDzGeffWaMMaaystJcf/31Jj4+3kgyI0eODJr31q1bO+zTF198Ye69917Tp08fEx8fb0aMGGE2btxoRo4cGXj/V8eWlZWZ9PR0ExMTY/r06WPGjh1rdu/ebYwJvuvqqxYtWmQkmccff9wYY8zs2bNNTk6O6dmzZ6A/Dz74oKmrqzvrvwsAXc9lzCmeBAYAAOAAXKMDAAAci6ADAAAci6ADAAAci6ADAAAci6ADAAAci6ADAAAcy3YPDGxtbdXBgweVkJBwykfDAwCAC48xRkePHlVqaqqiorruPIvtgs7Bgwc1YMCAcE8DAACcg/3795/VlwKHim2CjsfjkcfjCTzCfv/+/UpMTAxZfZ/PpzfffFOFhYWKiYkJWV07ohft6EUw+tGOXgSjH+3oRbC2fuTn5ysjI0MJCQldun3bBJ3S0lKVlpaqoaFBSUlJSkxMDHnQiY+PV2JiYsQfmPSiHb0IRj/a0Ytg9KMdvQjW1o+2gNPVl51wMTIAAHAsgg4AAHAs2wQdj8ejzMxM5ebmhnsqAADAJmwTdEpLS1VVVaWtW7eGeyoAAMAmbBN0AAAArCLoAAAAxyLoAAAAx7JN0OFiZAAAYJVtgg4XIwMAAKtsE3QAAACsIugAAADHIugAAADHIugAAADHss23l3s8Hnk8Hvn9/k7dTtbcdWr2h+6bVfcuHBuyWgAAwBrbnNHhrisAAGCVbYIOAACAVQQdAADgWAQdAADgWAQdAADgWAQdAADgWLYJOnypJwAAsMo2QYfbywEAgFW2CToAAABWEXQAAIBjEXQAAIBjEXQAAIBjEXQAAIBjEXQAAIBjEXQAAIBj2Sbo8MBAAABglW2CDg8MBAAAVtkm6AAAAFhF0AEAAI5F0AEAAI5F0AEAAI5F0AEAAI5F0AEAAI5F0AEAAI5F0AEAAI5F0AEAAI5F0AEAAI5lm6DDd10BAACrbBN0+K4rAABglW2CDgAAgFUEHQAA4FgEHQAA4FgEHQAA4FgEHQAA4FgEHQAA4FgEHQAA4FgEHQAA4FgEHQAA4FgEHQAA4FgEHQAA4FgEHQAA4FgEHQAA4FgEHQAA4FhhCzpNTU0aOHCgHn744XBNAQAAOFzYgs4TTzyhvLy8cG0eAABEgLAEnY8//li7d+/WLbfcEo7NAwCACGE56GzYsEHjx49XamqqXC6X1qxZ02HM0qVLlZGRobi4OGVnZ2vjxo1B6x9++GEtWLDgnCcNAABwNiwHncbGRg0dOlRPP/30KdevWrVKM2fO1Jw5c7Rz507dcMMNKioqUk1NjSTpv/7rv3TFFVfoiiuuOL+ZAwAAnEE3q28oKipSUVHRadcvXrxY9957r+677z5J0pIlS7Ru3TotW7ZMCxYs0Pvvv69XXnlFq1ev1rFjx+Tz+ZSYmKjHH3/8lPWam5vV3NwceN3Q0CBJ8vl88vl8Vqd/Wm213FEmZDW/WtdO2uZsx7mHGr0IRj/a0Ytg9KMdvQgW7n64jDHn/Jvd5XKpoqJCEyZMkCSdPHlS8fHxWr16tSZOnBgYV1ZWpsrKSq1fvz7o/StWrNCuXbv01FNPnXYbc+fO1bx58zosX7lypeLj48916gAAoAs1NTVpypQpqq+vV2JiYpdt1/IZnW9SV1cnv9+vlJSUoOUpKSk6fPjwOdV87LHHVF5eHnjd0NCgAQMGqLCwMKSN8vl88nq9+tG2KDW3ukJWd9fcMSGr1VXaejF69GjFxMSEezphRS+C0Y929CIY/WhHL4K19aOgoCAs2w9p0GnjcgUHBWNMh2WSdNddd52xltvtltvt7rA8JiamUw6g5laXmv2hCzp2Psg7q8d2RC+C0Y929CIY/WhHL4KFqxchvb28d+/eio6O7nD2pra2tsNZHqs8Ho8yMzOVm5t7XnUAAEDkCGnQiY2NVXZ2trxeb9Byr9er4cOHn1ft0tJSVVVVaevWredVBwAARA7LH10dO3ZMe/bsCbyurq5WZWWlkpOTlZ6ervLychUXFysnJ0f5+flavny5ampqNH369JBOHAAA4EwsB51t27YFXVDUdqHwtGnTtGLFCk2ePFlHjhzR/PnzdejQIWVlZWnt2rUaOHBg6GYNAABwFiwHnVGjRulMd6SXlJSopKTknCd1Kh6PRx6PR36/P6R1AQCAc4XtSz2t4hodAABglW2CDgAAgFUEHQAA4Fi2CTo8RwcAAFhlm6DDNToAAMAq2wQdAAAAqwg6AADAsQg6AADAsWwTdLgYGQAAWGWboMPFyAAAwCrbBB0AAACrCDoAAMCxCDoAAMCxCDoAAMCxbBN0uOsKAABYZZugw11XAADAKtsEHQAAAKsIOgAAwLEIOgAAwLEIOgAAwLFsE3S46woAAFhlm6DDXVcAAMAq2wQdAAAAqwg6AADAsQg6AADAsQg6AADAsQg6AADAsQg6AADAsQg6AADAsWwTdHhgIAAAsMo2QYcHBgIAAKtsE3QAAACsIugAAADHIugAAADHIugAAADHIugAAADHIugAAADHIugAAADHIugAAADHIugAAADHIugAAADHIugAAADHsk3Q4Us9AQCAVbYJOnypJwAAsMo2QQcAAMAqgg4AAHAsgg4AAHAsgg4AAHAsgg4AAHAsgg4AAHAsgg4AAHAsgg4AAHAsgg4AAHAsgg4AAHAsgg4AAHAsgg4AAHAsgg4AAHAsgg4AAHCsLg86R48eVW5urq655hoNHjxYzz77bFdPAQAARIhuXb3B+Ph4rV+/XvHx8WpqalJWVpYmTZqkXr16dfVUAACAw3X5GZ3o6GjFx8dLkk6cOCG/3y9jTFdPAwAARADLQWfDhg0aP368UlNT5XK5tGbNmg5jli5dqoyMDMXFxSk7O1sbN24MWv/ll19q6NChSktL0yOPPKLevXuf8w4AAACcjuWg09jYqKFDh+rpp58+5fpVq1Zp5syZmjNnjnbu3KkbbrhBRUVFqqmpCYy5+OKL9ac//UnV1dVauXKlPv3003PfAwAAgNOwfI1OUVGRioqKTrt+8eLFuvfee3XfffdJkpYsWaJ169Zp2bJlWrBgQdDYlJQUDRkyRBs2bNC3v/3tU9Zrbm5Wc3Nz4HVDQ4MkyefzyefzWZ3+abXVckeF9mO0UM6xq7TN2Y5zDzV6EYx+tKMXwehHO3oRLNz9cJnzuEDG5XKpoqJCEyZMkCSdPHlS8fHxWr16tSZOnBgYV1ZWpsrKSq1fv16ffvqpunfvrsTERDU0NCg/P18vv/yyhgwZcsptzJ07V/PmzeuwfOXKlYFrfQAAwIWtqalJU6ZMUX19vRITE7tsuyG966qurk5+v18pKSlBy1NSUnT48GFJ0ieffKJ7771XxhgZYzRjxozThhxJeuyxx1ReXh543dDQoAEDBqiwsDCkjfL5fPJ6vfrRtig1t7pCVnfX3DEhq9VV2noxevRoxcTEhHs6YUUvgtGPdvQiGP1oRy+CtfWjoKAgLNvvlNvLXa7goGCMCSzLzs5WZWXlWddyu91yu90dlsfExHTKAdTc6lKzP3RBx84HeWf12I7oRTD60Y5eBKMf7ehFsHD1IqS3l/fu3VvR0dGBszdtamtrO5zlscrj8SgzM1O5ubnnVQcAAESOkAad2NhYZWdny+v1Bi33er0aPnz4edUuLS1VVVWVtm7del51AABA5LD80dWxY8e0Z8+ewOvq6mpVVlYqOTlZ6enpKi8vV3FxsXJycpSfn6/ly5erpqZG06dPD+nEAQAAzsRy0Nm2bVvQBUVtFwpPmzZNK1as0OTJk3XkyBHNnz9fhw4dUlZWltauXauBAweGbtYAAABnwXLQGTVq1Bm/sqGkpEQlJSXnPKlT8Xg88ng88vv9Ia0LAACcq8u/6+pccY0OAACwyjZBBwAAwCqCDgAAcCzbBB2eowMAAKyyTdDhGh0AAGCVbYIOAACAVQQdAADgWLYJOlyjAwAArLJN0OEaHQAAYJVtgg4AAIBVBB0AAOBYBB0AAOBYBB0AAOBYtgk63HUFAACssk3Q4a4rAABglW2CDgAAgFUEHQAA4FgEHQAA4FgEHQAA4FgEHQAA4Fi2CTrcXg4AAKyyTdDh9nIAAGCVbYIOAACAVQQdAADgWAQdAADgWAQdAADgWAQdAADgWAQdAADgWAQdAADgWN3CPYGz5fF45PF45Pf7wz2Vc3Lp7DdCXnPvwrEhrwkAgJPY5owODwwEAABW2SboAAAAWEXQAQAAjkXQAQAAjkXQAQAAjkXQAQAAjkXQAQAAjkXQAQAAjkXQAQAAjkXQAQAAjkXQAQAAjkXQAQAAjmWboOPxeJSZmanc3NxwTwUAANiEbYIOX+oJAACssk3QAQAAsIqgAwAAHIugAwAAHIugAwAAHIugAwAAHIugAwAAHIugAwAAHIugAwAAHIugAwAAHIugAwAAHIugAwAAHIugAwAAHIugAwAAHIugAwAAHKvLg87+/fs1atQoZWZmasiQIVq9enVXTwEAAESIbl2+wW7dtGTJEl1zzTWqra3VsGHDdMstt6hHjx5dPRUAAOBwXR50+vXrp379+kmS+vTpo+TkZH3++ecEHQAAEHKWP7rasGGDxo8fr9TUVLlcLq1Zs6bDmKVLlyojI0NxcXHKzs7Wxo0bT1lr27Ztam1t1YABAyxPHAAA4Ewsn9FpbGzU0KFDdffdd+u2227rsH7VqlWaOXOmli5dquuvv17PPPOMioqKVFVVpfT09MC4I0eOaOrUqfr1r3/9jdtrbm5Wc3Nz4HVDQ4MkyefzyefzWZ3+abXVckeZkNUMqhsd2rpfrd1ZdTurvp3Qi2D0ox29CEY/2tGLYOHuh8sYc86/gV0ulyoqKjRhwoTAsry8PA0bNkzLli0LLBs0aJAmTJigBQsWSPp7eBk9erT+5V/+RcXFxd+4jblz52revHkdlq9cuVLx8fHnOnUAANCFmpqaNGXKFNXX1ysxMbHLthvSa3ROnjyp7du3a/bs2UHLCwsLtWnTJkmSMUZ33XWXbrzxxjOGHEl67LHHVF5eHnjd0NCgAQMGqLCwMKSN8vl88nq9+tG2KDW3ukJWd9fcMZKkrLnrQlbz67VDra0Xo0ePVkxMTKdswy7oRTD60Y5eBKMf7ehFsLZ+FBQUhGX7IQ06dXV18vv9SklJCVqekpKiw4cPS5L+93//V6tWrdKQIUMC1/e8+OKLGjx48Clrut1uud3uDstjYmI65QBqbnWp2R+6oNM2x1DW/HrtztJZPbYjehGMfrSjF8HoRzt6ESxcveiUu65cruBf6saYwLIRI0aotbXVck2PxyOPxyO/3x+SOTrJpbPfCGm9j39SGNJ6AACES0gfGNi7d29FR0cHzt60qa2t7XCWx6rS0lJVVVVp69at51UHAABEjpAGndjYWGVnZ8vr9QYt93q9Gj58eCg3BQAAcEaWP7o6duyY9uzZE3hdXV2tyspKJScnKz09XeXl5SouLlZOTo7y8/O1fPly1dTUaPr06SGdOAAAwJlYDjrbtm0LunK67Y6oadOmacWKFZo8ebKOHDmi+fPn69ChQ8rKytLatWs1cODA85oo1+gAAACrLAedUaNG6UyP3ikpKVFJSck5T+pUSktLVVpaqoaGBiUlJYW0NgAAcKYu//ZyAACArkLQAQAAjmWboOPxeJSZmanc3NxwTwUAANiEbYIOz9EBAABW2SboAAAAWEXQAQAAjkXQAQAAjmWboMPFyAAAwCrbBB0uRgYAAFbZJugAAABYRdABAACORdABAACORdABAACOZZugw11XAADAKtsEHe66AgAAVtkm6AAAAFhF0AEAAI5F0AEAAI5F0AEAAI5F0AEAAI5lm6DD7eUAAMAq2wQdbi8HAABW2SboAAAAWNUt3BPAhe3S2W+EtN7ehWNDWg8AgG/CGR0AAOBYBB0AAOBYBB0AAOBYBB0AAOBYBB0AAOBYtgk6PDAQAABYZZugwwMDAQCAVbYJOgAAAFYRdAAAgGMRdAAAgGMRdAAAgGMRdAAAgGMRdAAAgGMRdAAAgGMRdAAAgGMRdAAAgGMRdAAAgGPZJujwXVcAAMAq2wQdvusKAABYZZugAwAAYBVBBwAAOBZBBwAAOBZBBwAAOBZBBwAAOBZBBwAAOBZBBwAAOBZBBwAAOBZBBwAAOFa3cE8AkenS2W+EvObehWNDXhMAYG+c0QEAAI5F0AEAAI5F0AEAAI5F0AEAAI4VlqAzceJE9ezZU//8z/8cjs0DAIAIEZag88ADD+iFF14Ix6YBAEAECUvQKSgoUEJCQjg2DQAAIojloLNhwwaNHz9eqampcrlcWrNmTYcxS5cuVUZGhuLi4pSdna2NGzeGYq4AAACWWA46jY2NGjp0qJ5++ulTrl+1apVmzpypOXPmaOfOnbrhhhtUVFSkmpqa854sAACAFZafjFxUVKSioqLTrl+8eLHuvfde3XfffZKkJUuWaN26dVq2bJkWLFhgeYLNzc1qbm4OvG5oaJAk+Xw++Xw+y/VOp62WO8qErGZQ3ejQ1u3M2m11fT5fp9XuzH50Rs3OqG1H9KMdvQhGP9rRi2Dh7ofLGHPOv3FcLpcqKio0YcIESdLJkycVHx+v1atXa+LEiYFxZWVlqqys1Pr16wPL3n33XT399NP6z//8z2/cxty5czVv3rwOy1euXKn4+PhznToAAOhCTU1NmjJliurr65WYmNhl2w3pd13V1dXJ7/crJSUlaHlKSooOHz4ceD1mzBjt2LFDjY2NSktLU0VFhXJzc09Z87HHHlN5eXngdUNDgwYMGKDCwsKQNsrn88nr9epH26LU3OoKWd1dc8dIkrLmrgtZzc6uvXPOjfJ6vRo9erSufeLtkNbuin6EUttxMXr0aMXExIS8vt3Qj3b0Ihj9aEcvgrX1o6CgICzb75Qv9XS5goOCMSZo2bp1Z/9Lzu12y+12d1geExPTKQdQc6tLzf7QBZ22OYayZmfXbqsbExPTabU7sx+dobOON7uiH+3oRTD60Y5eBAtXL0J6e3nv3r0VHR0ddPZGkmprazuc5bHK4/EoMzPztGd+AAAAvi6kQSc2NlbZ2dnyer1By71er4YPH35etUtLS1VVVaWtW7eeVx0AABA5LH90dezYMe3Zsyfwurq6WpWVlUpOTlZ6errKy8tVXFysnJwc5efna/ny5aqpqdH06dNDOnEAAIAzsRx0tm3bFnRBUduFwtOmTdOKFSs0efJkHTlyRPPnz9ehQ4eUlZWltWvXauDAgaGbNQAAwFmwHHRGjRqlM92RXlJSopKSknOe1Kl4PB55PB75/f6Q1gUAAM4Vlu+6OhdcowMAAKyyTdABAACwiqADAAAcq1MeGNgZuEYHZ+vS2W+ErJY72ujn14WsHACgi9nmjA7X6AAAAKtsE3QAAACsIugAAADHIugAAADH4mJkwIJQXugsSXsXjg1pPQBAMNuc0eFiZAAAYJVtgg4AAIBVBB0AAOBYBB0AAOBYtgk6Ho9HmZmZys3NDfdUAACATdgm6HAxMgAAsMo2QQcAAMAqgg4AAHAsgg4AAHAsgg4AAHAsgg4AAHAs2wQdbi8HAABW2SbocHs5AACwyjZBBwAAwCqCDgAAcCyCDgAAcCyCDgAAcCyCDgAAcCyCDgAAcCyCDgAAcKxu4Z7A2fJ4PPJ4PPL7/eGeCoD/d+nsN0Jab+/CsSGtBwC2OaPDAwMBAIBVtgk6AAAAVhF0AACAYxF0AACAYxF0AACAYxF0AACAYxF0AACAYxF0AACAYxF0AACAYxF0AACAYxF0AACAYxF0AACAY9km6Hg8HmVmZio3NzfcUwEAADZhm6DDl3oCAACrbBN0AAAArCLoAAAAxyLoAAAAxyLoAAAAxyLoAAAAxyLoAAAAxyLoAAAAxyLoAAAAxyLoAAAAxyLoAAAAxyLoAAAAxyLoAAAAxyLoAAAAxyLoAAAAxwpL0Hn99dd15ZVX6vLLL9evf/3rcEwBAABEgG5dvcGWlhaVl5frnXfeUWJiooYNG6ZJkyYpOTm5q6cCAAAcrsvP6GzZskVXX321+vfvr4SEBN1yyy1at25dV08DAABEAMtBZ8OGDRo/frxSU1Plcrm0Zs2aDmOWLl2qjIwMxcXFKTs7Wxs3bgysO3jwoPr37x94nZaWpgMHDpzb7AEAAL6B5Y+uGhsbNXToUN1999267bbbOqxftWqVZs6cqaVLl+r666/XM888o6KiIlVVVSk9PV3GmA7vcblcp91ec3OzmpubA68bGhokST6fTz6fz+r0T6utljuq4/xCUjc6tHU7s3ZbXZ/P12m17dKPtuOhs3ttF189NqTI7sfXexHp6Ec7ehEs3P1wmVMlj7N9s8uliooKTZgwIbAsLy9Pw4YN07JlywLLBg0apAkTJmjBggXatGmTFi1apIqKCklSWVmZ8vLyNGXKlFNuY+7cuZo3b16H5StXrlR8fPy5Th0AAHShpqYmTZkyRfX19UpMTOyy7YY06Jw8eVLx8fFavXq1Jk6cGBhXVlamyspKrV+/Xi0tLRo0aJDefffdwMXI77//vnr16nXKbZzqjM6AAQNUV1cX0kb5fD55vV79aFuUmltPf4bJql1zx0iSsuaG/jqkzqq9c86N8nq9Gj16tK594u2Q1rZbP9xRRj/JadXo0aMVExMT8nm3zbkz2bEfdjg+vt6LUNZu0xXHR6i0/Qz9aj8iVVf3ojP/vYRCWz/y8vLUr1+/Lg86Ib3rqq6uTn6/XykpKUHLU1JSdPjw4b9vsFs3/eIXv1BBQYFaW1v1yCOPnDbkSJLb7Zbb7e6wPCYmplMOoOZWl5r9oQs6bXMMZc3Ort1WNyYmptNq26kfbbU7sx+dyY79sNPx8dWfRXY8PkKts34221FX9aIz/71c6DXPRqfcXv71a26MMUHLbr31Vt16662dsWkAAICAkN5e3rt3b0VHRwfO3rSpra3tcJbHKo/Ho8zMTOXm5p5XHQAAEDlCGnRiY2OVnZ0tr9cbtNzr9Wr48OHnVbu0tFRVVVXaunXredUBAACRw/JHV8eOHdOePXsCr6urq1VZWank5GSlp6ervLxcxcXFysnJUX5+vpYvX66amhpNnz49pBMHAAA4E8tBZ9u2bSooKAi8Li8vlyRNmzZNK1as0OTJk3XkyBHNnz9fhw4dUlZWltauXauBAwee10Q9Ho88Ho/8fv951QEAAJHDctAZNWrUKR/691UlJSUqKSk550mdSmlpqUpLS9XQ0KCkpKSQ1gYAAM4Ulm8vBwAA6AoEHQAA4FgEHQAA4Fi2CTo8RwcAAFhlm6DDc3QAAIBVtgk6AAAAVhF0AACAY3XKl3p2hrYHBra0tEiSGhoaQlrf5/OpqalJ/uZotYbwm2Db5tna3BSymp1du6GhQU1NTWpoaOiU2pJ9+uGPNmpq8quhoUExMTGd1o/OZMd+2OH4+HovQlm7TVccH6HS9jP0q/2IVF3di8789xIKbf04evSoJJ3xWXyh5jJdvcXz9Mknn2jAgAHhngYAADgH+/fvV1paWpdtz3ZBp7W1VQcPHlRCQoJcrtCeeRkwYID279+vxMTEkNW1I3rRjl4Eox/t6EUw+tGOXgRr60dNTY1cLpdSU1MVFdV1V87Y5qOrNlFRUZ2aBBMTEzkw/x+9aEcvgtGPdvQiGP1oRy+CJSUlhaUfXIwMAAAci6ADAAAci6Dz/9xut3784x/L7XaHeyphRy/a0Ytg9KMdvQhGP9rRi2Dh7oftLkYGAAA4W5zRAQAAjkXQAQAAjkXQAQAAjkXQAQAAjkXQAQAAjkXQkbR06VJlZGQoLi5O2dnZ2rhxY7indF4WLFig3NxcJSQkqE+fPpowYYL++te/Bo0xxmju3LlKTU1V9+7dNWrUKP3lL38JGtPc3Kz7779fvXv3Vo8ePXTrrbfqk08+CRrzxRdfqLi4WElJSUpKSlJxcbG+/PLLzt7Fc7ZgwQK5XC7NnDkzsCzSenHgwAHdeeed6tWrl+Lj43XNNddo+/btgfWR1I+Wlhb98Ic/VEZGhrp3767LLrtM8+fPV2tra2CMU/uxYcMGjR8/XqmpqXK5XFqzZk3Q+q7c75qaGo0fP149evRQ79699cADD+jkyZOdsdun9U398Pl8evTRRzV48GD16NFDqampmjp1qg4ePBhUwyn9ONOx8VXf//735XK5tGTJkqDlF1QvTIR75ZVXTExMjHn22WdNVVWVKSsrMz169DD79u0L99TO2ZgxY8zzzz9vdu3aZSorK83YsWNNenq6OXbsWGDMwoULTUJCgvnd735nPvjgAzN58mTTr18/09DQEBgzffp0079/f+P1es2OHTtMQUGBGTp0qGlpaQmMufnmm01WVpbZtGmT2bRpk8nKyjLjxo3r0v09W1u2bDGXXnqpGTJkiCkrKwssj6RefP7552bgwIHmrrvuMn/84x9NdXW1eeutt8yePXsCYyKpHz/96U9Nr169zOuvv26qq6vN6tWrzUUXXWSWLFkSGOPUfqxdu9bMmTPH/O53vzOSTEVFRdD6rtrvlpYWk5WVZQoKCsyOHTuM1+s1qampZsaMGZ3eg6/6pn58+eWX5qabbjKrVq0yu3fvNps3bzZ5eXkmOzs7qIZT+nGmY6NNRUWFGTp0qElNTTW//OUvg9ZdSL2I+KBz3XXXmenTpwctu+qqq8zs2bPDNKPQq62tNZLM+vXrjTHGtLa2mr59+5qFCxcGxpw4ccIkJSWZf//3fzfG/P0fdkxMjHnllVcCYw4cOGCioqLM//zP/xhjjKmqqjKSzPvvvx8Ys3nzZiPJ7N69uyt27awdPXrUXH755cbr9ZqRI0cGgk6k9eLRRx81I0aMOO36SOvH2LFjzT333BO0bNKkSebOO+80xkROP77+y6wr93vt2rUmKirKHDhwIDDm5ZdfNm6329TX13fK/p7JN/1yb7NlyxYjKfA/xU7tx+l68cknn5j+/fubXbt2mYEDBwYFnQutFxH90dXJkye1fft2FRYWBi0vLCzUpk2bwjSr0Kuvr5ckJScnS5Kqq6t1+PDhoP12u90aOXJkYL+3b98un88XNCY1NVVZWVmBMZs3b1ZSUpLy8vICY/7hH/5BSUlJF1z/SktLNXbsWN10001ByyOtF7///e+Vk5Ojb3/72+rTp4+uvfZaPfvss4H1kdaPESNG6A9/+IM++ugjSdKf/vQnvffee7rlllskRV4/2nTlfm/evFlZWVlKTU0NjBkzZoyam5uDPlK90NTX18vlcuniiy+WFFn9aG1tVXFxsWbNmqWrr766w/oLrRe2+/byUKqrq5Pf71dKSkrQ8pSUFB0+fDhMswotY4zKy8s1YsQIZWVlSVJg30613/v27QuMiY2NVc+ePTuMaXv/4cOH1adPnw7b7NOnzwXVv1deeUU7duzQ1q1bO6yLtF787W9/07Jly1ReXq4f/OAH2rJlix544AG53W5NnTo14vrx6KOPqr6+XldddZWio6Pl9/v1xBNP6Pbbb5cUecdHm67c78OHD3fYTs+ePRUbG3tB9kaSTpw4odmzZ2vKlCmBb+OOpH787Gc/U7du3fTAAw+ccv2F1ouIDjptXC5X0GtjTIdldjVjxgz9+c9/1nvvvddh3bns99fHnGr8hdS//fv3q6ysTG+++abi4uJOOy4SeiH9/f/EcnJy9OSTT0qSrr32Wv3lL3/RsmXLNHXq1MC4SOnHqlWr9NJLL2nlypW6+uqrVVlZqZkzZyo1NVXTpk0LjIuUfnxdV+23nXrj8/n03e9+V62trVq6dOkZxzutH9u3b9e//du/aceOHZbnE65eRPRHV71791Z0dHSHZFhbW9shRdrR/fffr9///vd65513lJaWFljet29fSfrG/e7bt69OnjypL7744hvHfPrppx22+9lnn10w/du+fbtqa2uVnZ2tbt26qVu3blq/fr1+9atfqVu3boF5RkIvJKlfv37KzMwMWjZo0CDV1NRIiqxjQ5JmzZql2bNn67vf/a4GDx6s4uJiPfjgg1qwYIGkyOtHm67c7759+3bYzhdffCGfz3fB9cbn8+k73/mOqqur5fV6A2dzpMjpx8aNG1VbW6v09PTAz9R9+/bpoYce0qWXXirpwutFRAed2NhYZWdny+v1Bi33er0aPnx4mGZ1/owxmjFjhl599VW9/fbbysjICFqfkZGhvn37Bu33yZMntX79+sB+Z2dnKyYmJmjMoUOHtGvXrsCY/Px81dfXa8uWLYExf/zjH1VfX3/B9O9b3/qWPvjgA1VWVgb+5OTk6I477lBlZaUuu+yyiOmFJF1//fUdHjXw0UcfaeDAgZIi69iQpKamJkVFBf8YjI6ODtxeHmn9aNOV+52fn69du3bp0KFDgTFvvvmm3G63srOzO3U/rWgLOR9//LHeeust9erVK2h9pPSjuLhYf/7zn4N+pqampmrWrFlat26dpAuwF2d92bJDtd1e/txzz5mqqiozc+ZM06NHD7N3795wT+2c/eu//qtJSkoy7777rjl06FDgT1NTU2DMwoULTVJSknn11VfNBx98YG6//fZT3jqalpZm3nrrLbNjxw5z4403nvL2wCFDhpjNmzebzZs3m8GDB19wtxB/3VfvujImsnqxZcsW061bN/PEE0+Yjz/+2Pz2t7818fHx5qWXXgqMiaR+TJs2zfTv3z9we/mrr75qevfubR555JHAGKf24+jRo2bnzp1m586dRpJZvHix2blzZ+Auoq7a77ZbiL/1rW+ZHTt2mLfeesukpaV1+e3l39QPn89nbr31VpOWlmYqKyuDfq42Nzc7rh9nOja+7ut3XRlzYfUi4oOOMcZ4PB4zcOBAExsba4YNGxa4DduuJJ3yz/PPPx8Y09raan784x+bvn37Grfbbf7xH//RfPDBB0F1jh8/bmbMmGGSk5NN9+7dzbhx40xNTU3QmCNHjpg77rjDJCQkmISEBHPHHXeYL774ogv28tx9PehEWi9ee+01k5WVZdxut7nqqqvM8uXLg9ZHUj8aGhpMWVmZSU9PN3Fxceayyy4zc+bMCfrl5dR+vPPOO6f8OTFt2jRjTNfu9759+8zYsWNN9+7dTXJyspkxY4Y5ceJEZ+5+B9/Uj+rq6tP+XH3nnXcCNZzSjzMdG193qqBzIfXCZYwxZ3/+BwAAwD4i+hodAADgbAQdAADgWAQdAADgWAQdAADgWAQdAADgWAQdAADgWAQdAADgWAQdAADgWAQdAADgWAQdAADgWAQdAADgWP8Habpac/DDoykAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAicAAAGxCAYAAAC5hxYeAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjYsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvq6yFwwAAAAlwSFlzAAAPYQAAD2EBqD+naQAAKIVJREFUeJzt3X90VPWd//HXECYDYZNoyBIISSCe067GCEiSUvAHUAUaERvsD7bYCC3uWTZBSbOuouy3Daxs6P7wsF0HWmy3nK1FWFZhXWWL4yI/WlB+poI53ZbdaBAJEaoZIGsYJp/vH/1mvoyJwJA7k8/NPB/n5BzunU/e93PfGZLXuXN/eIwxRgAAAJYY0NcTAAAAuBThBAAAWIVwAgAArEI4AQAAViGcAAAAqxBOAACAVQgnAADAKoQTAABgFcIJAACwCuEEAABYhXACAACsQjgBAABWIZwAcMT8+fM1evTobuvr6urk8Xgiy5s2bdKECROUmZmptLQ03XDDDfrWt74V9T3BYFCPPvqoCgsLlZqaqpEjR6qmpkbnz5+PGtfZ2al//Md/1Lhx4zR48GBdd911+vznP6+XXnopLvsIIDEG9vUEACSPvXv3as6cOZozZ47q6uo0aNAgvfvuu9q+fXtkTHt7uyZPnqz33ntPTz75pMaMGaO3335b3/nOd3TkyBG99tprkbAzf/58Pffcc1qwYIGWL1+u1NRUHTp0SO+8804f7SEAJxBOACTMnj17ZIzRD37wA2VmZkbWz58/P/Lv73//+3rrrbf05ptvqrS0VJJ01113aeTIkfrKV76in//85yovL9fu3bv105/+VEuXLtVTTz0V+f4vfvGLCdsfAPHBxzoAEqasrEyS9LWvfU3/8i//ohMnTnQb8/LLL6u4uFjjxo3TxYsXI18zZsyQx+PRjh07JEn/8R//IUmqrq5O2PwBJAbhBEDC3HnnndqyZYsuXryoBx98UHl5eSouLtbzzz8fGXPq1Cm99dZb8nq9UV/p6ekyxuj06dOSpA8++EApKSkaPnx4X+0OgDjhYx0Ajrl48WK3defOnYta/tKXvqQvfelL6ujo0BtvvKH6+nrNnTtXo0eP1sSJE5Wdna3Bgwfrn/7pn3rcRnZ2tiTpD//wDxUOh9XS0qIRI0Y4vzMA+gxHTgA45uTJk2ptbY1a98tf/rLHsT6fT5MnT9b3vvc9SdLhw4clSffee6/++7//W0OHDlVpaWm3r64rgsrLyyVJa9asidPeAOgrHDkB4BiPx6OKigo9/vjj8vl82rBhgw4dOiRJeu655/Tmm2/q/Pnzuuuuu5SXl6ePPvpI//AP/yCv16vJkydLkmpqavTCCy/ozjvv1Le//W2NGTNGnZ2dam5u1quvvqo///M/14QJE3THHXeosrJSTz31lE6dOqV7771XPp9Phw8fVlpamh5++OG+bAWAXiCcAHBMXl6eKioq9Kd/+qdqa2vT3Xffre3bt+trX/ualixZou9973v62c9+pscff1wffPCBrrvuOpWWlmr79u26+eabJUlDhgzR7t27tXLlSq1du1ZNTU0aPHiwCgoKdPfdd0fdS2XdunUaP368fvzjH2vdunUaPHiwioqK9OSTT/ZRBwA4wWOMMX09CQDuN3/+fO3YsYN7jADoNc45AQAAViGcAAAAq/CxDgAAsApHTgAAgFUIJwAAwCqEEwAAYBXX3eeks7NT77//vtLT0yOPTQcAAHYzxujs2bPKzc3VgAGXPzbiunDy/vvvKz8/v6+nAQAArsHx48eVl5d32TGuCSd+v19+vz/yYLHjx48rIyPDkdqhUEivvvqqpk+fLq/X60hNdEefE4M+xx89Tgz6nBiJ6nMwGFR+fr7S09OvONY14aS6ulrV1dUKBoPKzMxURkaGo+EkLS1NGRkZ/AeII/qcGPQ5/uhxYtDnxEh0n6/mlAxOiAUAAFYhnAAAAKu4Jpz4/X4VFRWprKysr6cCAADiyDXhpLq6Wo2Njdq/f39fTwUAAMSRa8IJAABIDoQTAABgFcIJAACwimvCCSfEAgCQHFwTTjghFgCA5OCacAIAAJID4QQAAFiFcAIAAKxCOAEAAFZxzVOJ/X6//H6/wuFwXLczeskrjtZ7Z+VMR+sBANDfuebICVfrAACQHFwTTgAAQHIgnAAAAKsQTgAAgFUIJwAAwCqEEwAAYBXXhBMe/AcAQHJwTTjhUmIAAJKDa8IJAABIDoQTAABgFcIJAACwCuEEAABYhXACAACsQjgBAABWIZwAAACruCaccBM2AACSg2vCCTdhAwAgObgmnAAAgORAOAEAAFYhnAAAAKsQTgAAgFUIJwAAwCqEEwAAYBXCCQAAsArhBAAAWIVwAgAArEI4AQAAVnFNOOHZOgAAJAfXhBOerQMAQHJwTTgBAADJgXACAACsQjgBAABWIZwAAACrEE4AAIBVCCcAAMAqhBMAAGAVwgkAALAK4QQAAFiFcAIAAKxCOAEAAFYhnAAAAKsQTgAAgFUIJwAAwCp9Fk7a29s1atQoPfroo301BQAAYKE+CycrVqzQhAkT+mrzAADAUn0STn7729/q17/+te65556+2DwAALBYzOFk165dmjVrlnJzc+XxeLRly5ZuY1avXq3CwkINGjRIJSUl2r17d9Trjz76qOrr66950gAAoP+KOZycP39eY8eO1TPPPNPj6xs3blRNTY2WLl2qw4cP64477lB5ebmam5slSf/2b/+mz372s/rsZz/bu5kDAIB+aWCs31BeXq7y8vJPff3pp5/WggUL9NBDD0mSVq1apW3btmnNmjWqr6/XG2+8oQ0bNmjTpk06d+6cQqGQMjIy9J3vfKfHeh0dHero6IgsB4NBSVIoFFIoFIp1+j3qqhMKheRLMY7U/GRtRPcZ8UOf448eJwZ9ToxE9TmW+h5jzDX/NfZ4PNq8ebMqKiokSRcuXFBaWpo2bdqk2bNnR8YtXrxYDQ0N2rlzZ9T3r1u3TkePHtXf/d3ffeo26urqtGzZsm7r169fr7S0tGudOgAASKD29nbNnTtXbW1tysjIuOzYmI+cXM7p06cVDoeVk5MTtT4nJ0ctLS3XVPOJJ55QbW1tZDkYDCo/P1/Tp0+/4s5drVAopEAgoGnTpunWFdsdqdnlaN0MR+u52aV99nq9fT2dfos+xx89Tgz6nBiJ6nPXJx9Xw9Fw0sXj8UQtG2O6rZOk+fPnX7GWz+eTz+frtt7r9TreRK/Xq45w93n2tiaixeNnh+7oc/zR48Sgz4kR7z7HUtvRS4mzs7OVkpLS7ShJa2trt6MpsfL7/SoqKlJZWVmv6gAAALs5Gk5SU1NVUlKiQCAQtT4QCGjSpEm9ql1dXa3Gxkbt37+/V3UAAIDdYv5Y59y5czp27FhkuampSQ0NDcrKylJBQYFqa2tVWVmp0tJSTZw4UWvXrlVzc7MWLlzo6MQBAED/FHM4OXDggKZOnRpZ7jpZdd68eVq3bp3mzJmjM2fOaPny5Tp58qSKi4u1detWjRo1yrlZAwCAfivmcDJlyhRd6erjqqoqVVVVXfOkeuL3++X3+xUOhx2tCwAA7NJnD/6LFeecAACQHFwTTgAAQHIgnAAAAKu4JpxwnxMAAJKDa8IJ55wAAJAcXBNOAABAciCcAAAAqxBOAACAVVwTTjghFgCA5OCacMIJsQAAJAfXhBMAAJAcCCcAAMAqhBMAAGAV14QTTogFACA5uCaccEIsAADJwTXhBAAAJAfCCQAAsArhBAAAWIVwAgAArEI4AQAAVnFNOOFSYgAAkoNrwgmXEgMAkBxcE04AAEByIJwAAACrEE4AAIBVCCcAAMAqhBMAAGAVwgkAALAK4QQAAFjFNeGEm7ABAJAcXBNOuAkbAADJwTXhBAAAJAfCCQAAsArhBAAAWIVwAgAArEI4AQAAViGcAAAAqxBOAACAVQgnAADAKoQTAABgFcIJAACwCuEEAABYxTXhhAf/AQCQHFwTTnjwHwAAycE14QQAACQHwgkAALAK4QQAAFiFcAIAAKxCOAEAAFYhnAAAAKsQTgAAgFUIJwAAwCqEEwAAYBXCCQAAsArhBAAAWIVwAgAArEI4AQAAViGcAAAAqyQ8nJw9e1ZlZWUaN26cbrnlFj377LOJngIAALDYwERvMC0tTTt37lRaWpra29tVXFys+++/X0OHDk30VAAAgIUSfuQkJSVFaWlpkqSPP/5Y4XBYxphETwMAAFgq5nCya9cuzZo1S7m5ufJ4PNqyZUu3MatXr1ZhYaEGDRqkkpIS7d69O+r1jz76SGPHjlVeXp4ee+wxZWdnX/MOAACA/iXmcHL+/HmNHTtWzzzzTI+vb9y4UTU1NVq6dKkOHz6sO+64Q+Xl5Wpubo6Mue666/SrX/1KTU1NWr9+vU6dOnXtewAAAPqVmM85KS8vV3l5+ae+/vTTT2vBggV66KGHJEmrVq3Stm3btGbNGtXX10eNzcnJ0ZgxY7Rr1y599atf7bFeR0eHOjo6IsvBYFCSFAqFFAqFYp1+j7rqhEIh+VKc/YjJqTn2B5f2GfFDn+OPHicGfU6MRPU5lvoe04sTPjwejzZv3qyKigpJ0oULF5SWlqZNmzZp9uzZkXGLFy9WQ0ODdu7cqVOnTmnw4MHKyMhQMBjUxIkT9fzzz2vMmDE9bqOurk7Lli3rtn79+vWRc1cAAIDd2tvbNXfuXLW1tSkjI+OyYx29Wuf06dMKh8PKycmJWp+Tk6OWlhZJ0nvvvacFCxbIGCNjjBYtWvSpwUSSnnjiCdXW1kaWg8Gg8vPzNX369Cvu3NUKhUIKBAKaNm2abl2x3ZGaXY7WzXC0nptd2mev19vX0+m36HP80ePEoM+Jkag+d33ycTXicimxx+OJWjbGRNaVlJSooaHhqmv5fD75fL5u671er+NN9Hq96gh7rjwwxpqIFo+fHbqjz/FHjxODPidGvPscS21HLyXOzs5WSkpK5ChJl9bW1m5HU2Ll9/tVVFSksrKyXtUBAAB2czScpKamqqSkRIFAIGp9IBDQpEmTelW7urpajY2N2r9/f6/qAAAAu8X8sc65c+d07NixyHJTU5MaGhqUlZWlgoIC1dbWqrKyUqWlpZo4caLWrl2r5uZmLVy40NGJAwCA/inmcHLgwAFNnTo1stx1suq8efO0bt06zZkzR2fOnNHy5ct18uRJFRcXa+vWrRo1alSvJur3++X3+xUOh3tVBwAA2C3mcDJlypQr3m6+qqpKVVVV1zypnlRXV6u6ulrBYFCZmZmO1gYAAPZI+LN1AAAALodwAgAArEI4AQAAVnFNOOE+JwAAJAfXhBPucwIAQHJwTTgBAADJgXACAACs4ppwwjknAAAkB9eEE845AQAgObgmnAAAgORAOAEAAFYhnAAAAKsQTgAAgFVcE064WgcAgOTgmnDC1ToAACQH14QTAACQHAgnAADAKgP7egLJYvSSVxyv+c7KmY7XBACgr3HkBAAAWIVwAgAArOKacMKlxAAAJAfXhBMuJQYAIDm4JpwAAIDkQDgBAABWIZwAAACrEE4AAIBVCCcAAMAqhBMAAGAVwgkAALCKa8IJN2EDACA5uCaccBM2AACSg2vCCQAASA6EEwAAYBXCCQAAsArhBAAAWIVwAgAArEI4AQAAViGcAAAAqxBOAACAVQgnAADAKoQTAABgFcIJAACwimvCCQ/+AwAgObgmnPDgPwAAkoNrwgkAAEgOhBMAAGAVwgkAALAK4QQAAFiFcAIAAKxCOAEAAFYhnAAAAKsQTgAAgFUIJwAAwCqEEwAAYBXCCQAAsMrAvp4Aem/0klccrffOypmO1gMAIBYcOQEAAFYhnAAAAKskPJwcP35cU6ZMUVFRkcaMGaNNmzYlegoAAMBiCT/nZODAgVq1apXGjRun1tZWjR8/Xvfcc4+GDBmS6KkAAAALJTycjBgxQiNGjJAkDRs2TFlZWfrd735HOAEAAJKu4WOdXbt2adasWcrNzZXH49GWLVu6jVm9erUKCws1aNAglZSUaPfu3T3WOnDggDo7O5Wfnx/zxAEAQP8U85GT8+fPa+zYsfrmN7+pL3/5y91e37hxo2pqarR69Wrddttt+uEPf6jy8nI1NjaqoKAgMu7MmTN68MEH9aMf/eiy2+vo6FBHR0dkORgMSpJCoZBCoVCs0+9RV51QKCRfinGk5idrO103nrWd6uun1Y1XffwefY4/epwY9DkxEtXnWOp7jDHX/JfN4/Fo8+bNqqioiKybMGGCxo8frzVr1kTW3XTTTaqoqFB9fb2k3weOadOm6U/+5E9UWVl52W3U1dVp2bJl3davX79eaWlp1zp1AACQQO3t7Zo7d67a2tqUkZFx2bGOnnNy4cIFHTx4UEuWLIlaP336dO3Zs0eSZIzR/Pnz9YUvfOGKwUSSnnjiCdXW1kaWg8Gg8vPzNX369Cvu3NUKhUIKBAKaNm2abl2x3ZGaXY7WzZAkFddtc7RuPGt31XXapX32er1x2QbocyLQ48Sgz4mRqD53ffJxNRwNJ6dPn1Y4HFZOTk7U+pycHLW0tEiSfvnLX2rjxo0aM2ZM5HyVn/70p7rlllt6rOnz+eTz+bqt93q9jjfR6/WqI+xxvKYkx+vGs3a8fwnE42eH7uhz/NHjxKDPiRHvPsdSOy5X63g80X8sjTGRdbfffrs6Oztjrun3++X3+xUOhx2ZIwAAsJOjN2HLzs5WSkpK5ChJl9bW1m5HU2JVXV2txsZG7d+/v1d1AACA3RwNJ6mpqSopKVEgEIhaHwgENGnSJCc3BQAA+qmYP9Y5d+6cjh07FlluampSQ0ODsrKyVFBQoNraWlVWVqq0tFQTJ07U2rVr1dzcrIULFzo6cQAA0D/FHE4OHDigqVOnRpa7rqSZN2+e1q1bpzlz5ujMmTNavny5Tp48qeLiYm3dulWjRo3q1UQ55wQAgOQQcziZMmWKrnRrlKqqKlVVVV3zpHpSXV2t6upqBYNBZWZmOlobAADYI+FPJQYAALgcwgkAALCKa8KJ3+9XUVGRysrK+noqAAAgjlwTTrjPCQAAycE14QQAACQHwgkAALAK4QQAAFjFNeGEE2IBAEgOrgknnBALAEBycE04AQAAyYFwAgAArEI4AQAAViGcAAAAq7gmnHC1DgAAycE14YSrdQAASA6uCScAACA5EE4AAIBVCCcAAMAqhBMAAGCVgX09Adht9JJXHKvlSzH6m885Vg4A0E+55sgJlxIDAJAcXBNOuJQYAIDk4JpwAgAAkgPhBAAAWIVwAgAArEI4AQAAViGcAAAAqxBOAACAVQgnAADAKq4JJ9yEDQCA5OCacMJN2AAASA6uCScAACA5EE4AAIBVCCcAAMAqhBMAAGAVwgkAALAK4QQAAFiFcAIAAKxCOAEAAFYhnAAAAKsQTgAAgFVcE054tg4AAMnBNeGEZ+sAAJAcXBNOAABAciCcAAAAqwzs6wkgOY1e8orjNd9ZOdPxmgCAxOPICQAAsArhBAAAWIVwAgAArEI4AQAAViGcAAAAqxBOAACAVQgnAADAKoQTAABgFcIJAACwCuEEAABYhXACAACs0ifhZPbs2br++uv1la98pS82DwAALNYn4eSRRx7RP//zP/fFpgEAgOX6JJxMnTpV6enpfbFpAABguZjDya5duzRr1izl5ubK4/Foy5Yt3casXr1ahYWFGjRokEpKSrR7924n5goAAJJAzOHk/PnzGjt2rJ555pkeX9+4caNqamq0dOlSHT58WHfccYfKy8vV3Nzc68kCAID+b2Cs31BeXq7y8vJPff3pp5/WggUL9NBDD0mSVq1apW3btmnNmjWqr6+PeYIdHR3q6OiILAeDQUlSKBRSKBSKuV5PuuqEQiH5UowjNT9Z2+m68ax9aV+drO0bYCL149mPZHfp+xnxQY8Tgz4nRqL6HEt9jzHmmv9KeDwebd68WRUVFZKkCxcuKC0tTZs2bdLs2bMj4xYvXqyGhgbt3Lkzsm7Hjh165pln9K//+q+X3UZdXZ2WLVvWbf369euVlpZ2rVMHAAAJ1N7errlz56qtrU0ZGRmXHRvzkZPLOX36tMLhsHJycqLW5+TkqKWlJbI8Y8YMHTp0SOfPn1deXp42b96ssrKyHms+8cQTqq2tjSwHg0Hl5+dr+vTpV9y5qxUKhRQIBDRt2jTdumK7IzW7HK2bIUkqrtvmaN141u6q63Rt3wCjvyrtjEufpeh5J7NL389er7evp9Mv0ePEoM+Jkag+d33ycTUcDSddPB5P1LIxJmrdtm1X/wfP5/PJ5/N1W+/1eh1votfrVUfYc+WBMdaU5HjdeNa+tK/xmnc8+4Hfi8f/EUSjx4lBnxMj3n2Opbaj4SQ7O1spKSlRR0kkqbW1tdvRlFj5/X75/X6Fw+Fe1UH/N3rJK47We2flTEfrAQAuz9H7nKSmpqqkpESBQCBqfSAQ0KRJk3pVu7q6Wo2Njdq/f3+v6gAAALvFfOTk3LlzOnbsWGS5qalJDQ0NysrKUkFBgWpra1VZWanS0lJNnDhRa9euVXNzsxYuXOjoxAEAQP8Uczg5cOCApk6dGlnuOll13rx5WrdunebMmaMzZ85o+fLlOnnypIqLi7V161aNGjXKuVkDAIB+K+ZwMmXKFF3p6uOqqipVVVVd86R6wjknAAAkhz55ts614JwTAACSg2vCCQAASA6EEwAAYBXXhBO/36+ioqJPvZMsAADoH1wTTjjnBACA5OCacAIAAJID4QQAAFiFcAIAAKzimnDCCbEAACQH14QTTogFACA5uCacAACA5EA4AQAAViGcAAAAq8T8VOK+wlOJYYPRS15xtN47K2c6Wg8A+gPXHDnhhFgAAJKDa8IJAABIDoQTAABgFcIJAACwCuEEAABYhXACAACswqXEgAWcvETZl2L0N59zrBwAJJxrjpxwKTEAAMnBNeEEAAAkB8IJAACwCuEEAABYhXACAACsQjgBAABWIZwAAACrEE4AAIBVXBNO/H6/ioqKVFZW1tdTAQAAceSacMJN2AAASA6uCScAACA5EE4AAIBVCCcAAMAqhBMAAGAVwgkAALAK4QQAAFiFcAIAAKxCOAEAAFYhnAAAAKsQTgAAgFUIJwAAwCoD+3oCV8vv98vv9yscDvf1VABXGb3kFUfrvbNypqP1euL0nKXEzBuAM1xz5IQH/wEAkBxcE04AAEByIJwAAACrEE4AAIBVCCcAAMAqhBMAAGAVwgkAALAK4QQAAFiFcAIAAKxCOAEAAFYhnAAAAKsQTgAAgFUIJwAAwCqEEwAAYBXCCQAAsEqfhJOXX35Zf/RHf6TPfOYz+tGPftQXUwAAAJYamOgNXrx4UbW1tXr99deVkZGh8ePH6/7771dWVlaipwIAACyU8CMn+/bt080336yRI0cqPT1d99xzj7Zt25boaQAAAEvFHE527dqlWbNmKTc3Vx6PR1u2bOk2ZvXq1SosLNSgQYNUUlKi3bt3R157//33NXLkyMhyXl6eTpw4cW2zBwAA/U7MH+ucP39eY8eO1Te/+U19+ctf7vb6xo0bVVNTo9WrV+u2227TD3/4Q5WXl6uxsVEFBQUyxnT7Ho/H86nb6+joUEdHR2Q5GAxKkkKhkEKhUKzT71FXnVAoJF9K9/k5UdvpuvGsfWlfnaztG2Ai9elHdO149dnp2pfWjad4vj+crJWIXiQz+pwYiepzLPU9pqe0cLXf7PFo8+bNqqioiKybMGGCxo8frzVr1kTW3XTTTaqoqFB9fb327Nmjv/3bv9XmzZslSYsXL9aECRM0d+7cHrdRV1enZcuWdVu/fv16paWlXevUAQBAArW3t2vu3Llqa2tTRkbGZcc6Gk4uXLigtLQ0bdq0SbNnz46MW7x4sRoaGrRz505dvHhRN910k3bs2BE5IfaNN97Q0KFDe9xGT0dO8vPzdfr06Svu3NUKhUIKBAKaNm2abl2x3ZGaXY7WzZAkFdc5f15NvGp31XW6tm+A0V+Vdsalz5L7+nFp7Xj12ev1xrUf8RLP/y9OuPR3htfrdazup7G9H/GS6D67VW/fH12/M/7PgQHq6Pz9JxnxeH8Eg0FlZ2dfVThx9Gqd06dPKxwOKycnJ2p9Tk6OWlpafr/BgQP193//95o6dao6Ozv12GOPfWowkSSfzyefz9dtvdfrdfzN6vV61RH+9I+YrrWmJMfrxrP2pX2N17zpR3TtePUjnu/peIrn+8PpmvQj/hLVZ7dy6v3R0emJ1IrX/5erFZdLiT95DokxJmrdfffdp/vuuy8emwYAAC7n6KXE2dnZSklJiRwl6dLa2trtaEqs/H6/ioqKVFZW1qs6AADAbo6Gk9TUVJWUlCgQCEStDwQCmjRpUq9qV1dXq7GxUfv37+9VHQAAYLeYP9Y5d+6cjh07FlluampSQ0ODsrKyVFBQoNraWlVWVqq0tFQTJ07U2rVr1dzcrIULFzo6cQAA0D/FHE4OHDigqVOnRpZra2slSfPmzdO6des0Z84cnTlzRsuXL9fJkydVXFysrVu3atSoUb2aqN/vl9/vVzgc7lUdAABgt5jDyZQpU3q8kdqlqqqqVFVVdc2T6kl1dbWqq6sVDAaVmZnpaG0AAGCPPnkqMQAAwKchnAAAAKsQTgAAgFVcE064zwkAAMnBNeGE+5wAAJAcXBNOAABAciCcAAAAq8TlwX/x0HUTtosXL0r6/aOXnRIKhdTe3q5gMKjOjnbH6kr/f55O141n7Ut762TtcIpRe3s4Ln2W3NePS2vHq89erzeu/YiXeL4/nHDp74xEPC3X9n7ES6L77Fa9fX90/c4Id6So8/89lTge74+umle6V5okeczVjLLIe++9p/z8/L6eBgAAuAbHjx9XXl7eZce4Lpx0dnbq/fffV3p6ujwejyM1g8Gg8vPzdfz4cWVkZDhSE93R58Sgz/FHjxODPidGovpsjNHZs2eVm5urAQMuf1aJaz7W6TJgwIArJq5rlZGRwX+ABKDPiUGf448eJwZ9ToxE9PlqHz/DCbEAAMAqhBMAAGAVwokkn8+n7373u/L5fH09lX6NPicGfY4/epwY9DkxbOyz606IBQAA/RtHTgAAgFUIJwAAwCqEEwAAYBXCCQAAsArhBAAAWCXpw8nq1atVWFioQYMGqaSkRLt37+7rKVmrvr5eZWVlSk9P17Bhw1RRUaH/+q//ihpjjFFdXZ1yc3M1ePBgTZkyRW+//XbUmI6ODj388MPKzs7WkCFDdN999+m9996LGvPhhx+qsrJSmZmZyszMVGVlpT766KN476J16uvr5fF4VFNTE1lHj51z4sQJfeMb39DQoUOVlpamcePG6eDBg5HX6XXvXLx4UX/5l3+pwsJCDR48WDfccIOWL1+uzs7OyBh6HLtdu3Zp1qxZys3Nlcfj0ZYtW6JeT2RPm5ubNWvWLA0ZMkTZ2dl65JFHdOHChd7vpEliGzZsMF6v1zz77LOmsbHRLF682AwZMsS8++67fT01K82YMcP85Cc/MUePHjUNDQ1m5syZpqCgwJw7dy4yZuXKlSY9Pd288MIL5siRI2bOnDlmxIgRJhgMRsYsXLjQjBw50gQCAXPo0CEzdepUM3bsWHPx4sXImC9+8YumuLjY7Nmzx+zZs8cUFxebe++9N6H729f27dtnRo8ebcaMGWMWL14cWU+PnfG73/3OjBo1ysyfP9+8+eabpqmpybz22mvm2LFjkTH0uneeeuopM3ToUPPyyy+bpqYms2nTJvMHf/AHZtWqVZEx9Dh2W7duNUuXLjUvvPCCkWQ2b94c9Xqienrx4kVTXFxspk6dag4dOmQCgYDJzc01ixYt6vU+JnU4+dznPmcWLlwYte7GG280S5Ys6aMZuUtra6uRZHbu3GmMMaazs9MMHz7crFy5MjLm448/NpmZmeYHP/iBMcaYjz76yHi9XrNhw4bImBMnTpgBAwaYn//858YYYxobG40k88Ybb0TG7N2710gyv/71rxOxa33u7Nmz5jOf+YwJBAJm8uTJkXBCj53z+OOPm9tvv/1TX6fXvTdz5kzzrW99K2rd/fffb77xjW8YY+ixEz4ZThLZ061bt5oBAwaYEydORMY8//zzxufzmba2tl7tV9J+rHPhwgUdPHhQ06dPj1o/ffp07dmzp49m5S5tbW2SpKysLElSU1OTWlpaonrq8/k0efLkSE8PHjyoUCgUNSY3N1fFxcWRMXv37lVmZqYmTJgQGfP5z39emZmZSfOzqa6u1syZM3X33XdHrafHznnppZdUWlqqr371qxo2bJhuvfVWPfvss5HX6XXv3X777frP//xP/eY3v5Ek/epXv9IvfvEL3XPPPZLocTwksqd79+5VcXGxcnNzI2NmzJihjo6OqI9Hr4XrnkrslNOnTyscDisnJydqfU5OjlpaWvpoVu5hjFFtba1uv/12FRcXS1Kkbz319N13342MSU1N1fXXX99tTNf3t7S0aNiwYd22OWzYsKT42WzYsEGHDh3S/v37u71Gj53zP//zP1qzZo1qa2v15JNPat++fXrkkUfk8/n04IMP0msHPP7442pra9ONN96olJQUhcNhrVixQl//+tcl8X6Oh0T2tKWlpdt2rr/+eqWmpva670kbTrp4PJ6oZWNMt3XobtGiRXrrrbf0i1/8ottr19LTT47paXwy/GyOHz+uxYsX69VXX9WgQYM+dRw97r3Ozk6Vlpbqr//6ryVJt956q95++22tWbNGDz74YGQcvb52Gzdu1HPPPaf169fr5ptvVkNDg2pqapSbm6t58+ZFxtFj5yWqp/Hqe9J+rJOdna2UlJRu6a61tbVbEkS0hx9+WC+99JJef/115eXlRdYPHz5cki7b0+HDh+vChQv68MMPLzvm1KlT3bb7wQcf9PufzcGDB9Xa2qqSkhINHDhQAwcO1M6dO/X9739fAwcOjOw/Pe69ESNGqKioKGrdTTfdpObmZkm8n53wF3/xF1qyZIn++I//WLfccosqKyv17W9/W/X19ZLocTwksqfDhw/vtp0PP/xQoVCo131P2nCSmpqqkpISBQKBqPWBQECTJk3qo1nZzRijRYsW6cUXX9T27dtVWFgY9XphYaGGDx8e1dMLFy5o586dkZ6WlJTI6/VGjTl58qSOHj0aGTNx4kS1tbVp3759kTFvvvmm2tra+v3P5q677tKRI0fU0NAQ+SotLdUDDzyghoYG3XDDDfTYIbfddlu3S+F/85vfaNSoUZJ4Pzuhvb1dAwZE/5lJSUmJXEpMj52XyJ5OnDhRR48e1cmTJyNjXn31Vfl8PpWUlPRuR3p1Oq3LdV1K/OMf/9g0NjaampoaM2TIEPPOO+/09dSs9Gd/9mcmMzPT7Nixw5w8eTLy1d7eHhmzcuVKk5mZaV588UVz5MgR8/Wvf73HS9jy8vLMa6+9Zg4dOmS+8IUv9HgJ25gxY8zevXvN3r17zS233NJvLwu8kkuv1jGGHjtl3759ZuDAgWbFihXmt7/9rfnZz35m0tLSzHPPPRcZQ697Z968eWbkyJGRS4lffPFFk52dbR577LHIGHocu7Nnz5rDhw+bw4cPG0nm6aefNocPH47cBiNRPe26lPiuu+4yhw4dMq+99prJy8vjUmIn+P1+M2rUKJOammrGjx8fuSwW3Unq8esnP/lJZExnZ6f57ne/a4YPH258Pp+58847zZEjR6Lq/O///q9ZtGiRycrKMoMHDzb33nuvaW5ujhpz5swZ88ADD5j09HSTnp5uHnjgAfPhhx8mYC/t88lwQo+d8+///u+muLjY+Hw+c+ONN5q1a9dGvU6veycYDJrFixebgoICM2jQIHPDDTeYpUuXmo6OjsgYehy7119/vcffxfPmzTPGJLan7777rpk5c6YZPHiwycrKMosWLTIff/xxr/fRY4wxvTv2AgAA4JykPecEAADYiXACAACsQjgBAABWIZwAAACrEE4AAIBVCCcAAMAqhBMAAGAVwgkAALAK4QQAAFiFcAIAAKxCOAEAAFb5vw5f5MJRzrRYAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sudata.hist(rwidth=0.9, column='Backtracks', log=True, bins=20);\n", "sudata.hist(rwidth=0.9, column='μsec', log=True, bins=20);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For both attributes, almost all the puzzles are in the leftmost bin (in both plots the leftmost bar is two horizontal lines above the next bin; thus more than 100 times more frequent). " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Comparing my Python and Java Sudoku Programs\n", "\n", "There are many similarities and some differences between my Java and Python Sudoku programs:\n", "- Both programs represent a puzzle as a **grid** of 81 squares, with 27 **units** (9 rows, 9 columns, 9 boxes).\n", "- They both represent uncertainty about a square's contents with a **set** of the possible digits from 1 to 9.\n", "- They both do a depth-first [search](http://en.wikipedia.org/wiki/Search_algorithm) for a solution.\n", " - They find an unfilled square with the fewest remaining possible digits, guess a digit to fill the square, and try to solve the rest of the puzzle from there. If that fails, back up and try a different guess for the square.\n", " - Each guess is placed on a new *copy* of the board. If the guess turns out to be wrong, revert to the old board.\n", "- They both use [constraint propagation](http://en.wikipedia.org/wiki/Constraint_satisfaction) to limit the search:\n", " - *The one rule:* After filling a square with a digit, eliminate the digit from all of the square's [peers](https://www.sudocue.net/guide.php#peers).\n", " - *Arc consistency:* after eliminating a possible digit from a square, check that it still has some other possible digit.\n", " - *Dual consistency:* after eliminating a possible digit from a square, check that another square in each of the square's 3 units could hold that digit.\n", " - These changes do not require a new copy of the board, because they are logical consequences, not guesses.\n", " - If a check fails, back up a level in the search.\n", "\n", "\n", "The [Python program](http://norvig.com/sudoku.html) was written for clarity and brevity. The [Java program](Sudoku.java) for efficiency. The Java program uses these tricks:\n", "- Primitive Java data types are used:\n", " - A grid is an `int[81]` array, not a hash table. \n", " - A square is an int (like `8`), not a string (like `'A9'`).\n", " - A set of possible digits is bitset represented by an int (like `0b110010100`, aka 404), not a string of digits (like `'9853'`). \n", "- Multiple puzzles are solved in parallel, in different threads.\n", "- Rather than allocating (and then garbage collecting) a new copy of the grid at each step in the depth-first search, instead we pre-allocate a `gridpool` array of grids once and for all (at the start of each thread), and then every recursive call to `search` re-uses `gridpool[level]`, copying data into that array. \n", "- The specific Sudoku strategy of [naked pairs](https://www.learn-sudoku.com/naked-pairs.html) is (optionally) implemented.\n", "- The Java compiler produces code that is inherently faster than the Python interpreter.\n", "- To warm up Java's JIT (just-in-time) code cache, I solve a few puzzles before reporting any timing results. \n", " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Is the Program Correct?\n", "\n", "How do we know this program (or any program) is correct? Traditionally, there are four kinds of evidence:\n", "- A large number of example input/output pairs that give the right answer.\n", "- Manual inspection of a smaller number of example input/output pairs.\n", "- A suite of unit tests to verify that components function properly (at least on some inputs).\n", "- A formal proof that the code is correct (or at least an outline of an argument for a partial proof).\n", "\n", "For this program:\n", "- Unless `-noverify` is given, every puzzle/solution pair is verified with the `verify` method to make sure:\n", " - Each square in the solution contains a single digit. \n", " - Each unit in the solution contains all nine digits.\n", " - Squares in the puzzle that are filled with a digit keep that same digit in the solution.\n", "- I have looked at some example solutions and double-checked some with an [online Sudoku](https://sudokuspoiler.azurewebsites.net/) program. [LGTM](https://www.dictionary.com/e/acronyms/lgtm/).\n", "- I have looked at the code carefully, but I am nowhere near a proof of correctness.\n", "- It would be nice to have a suite of unit tests." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can manually inspect these 10 \"hard\" puzzle/solution pairs:" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "Puzzle 1: Solution:\n", ". . . | . . . | . . 8 6 2 1 | 9 4 3 | 7 5 8 \n", ". . 3 | . . . | 4 . . 7 8 3 | 6 1 5 | 4 9 2 \n", ". 9 . | . 2 . | . 6 . 5 9 4 | 7 2 8 | 3 6 1 \n", "------+-------+------ ------+-------+------\n", ". . . | . 7 9 | . . . 1 4 2 | 8 7 9 | 6 3 5 \n", ". . . | . 6 1 | 2 . . 3 5 7 | 4 6 1 | 2 8 9 \n", ". 6 . | 5 . 2 | . 7 . 8 6 9 | 5 3 2 | 1 7 4 \n", "------+-------+------ ------+-------+------\n", ". . 8 | . . . | 5 . . 2 3 8 | 1 9 7 | 5 4 6 \n", ". 1 . | . . . | . 2 . 9 1 6 | 3 5 4 | 8 2 7 \n", "4 . 5 | . . . | . . 3 4 7 5 | 2 8 6 | 9 1 3 \n", "\n", "Puzzle 2: Solution:\n", ". . . | . . . | . . 2 6 3 9 | 8 4 7 | 5 1 2 \n", ". . 8 | . 1 . | 9 . . 4 7 8 | 5 1 2 | 9 6 3 \n", "5 . . | . . 3 | . 4 . 5 1 2 | 6 9 3 | 7 4 8 \n", "------+-------+------ ------+-------+------\n", ". . . | 1 . 9 | 3 . . 7 2 4 | 1 8 9 | 3 5 6 \n", ". 6 . | . 3 . | . 8 . 9 6 5 | 2 3 4 | 1 8 7 \n", ". . 3 | 7 . . | . . . 1 8 3 | 7 6 5 | 2 9 4 \n", "------+-------+------ ------+-------+------\n", ". 4 . | . . . | . . 5 8 4 7 | 9 2 1 | 6 3 5 \n", "3 . 1 | . 7 . | 8 . . 3 5 1 | 4 7 6 | 8 2 9 \n", "2 . . | . . . | . . . 2 9 6 | 3 5 8 | 4 7 1 \n", "\n", "Puzzle 3: Solution:\n", ". . 2 | . . . | 7 . . 8 3 2 | 4 1 6 | 7 9 5 \n", ". 1 . | . . . | . 6 . 4 1 7 | 9 8 5 | 2 6 3 \n", "5 . . | . . . | . 1 8 5 9 6 | 2 7 3 | 4 1 8 \n", "------+-------+------ ------+-------+------\n", ". . . | . 3 7 | . . . 9 5 1 | 8 3 7 | 6 2 4 \n", ". . . | . 4 9 | . . . 3 2 8 | 6 4 9 | 5 7 1 \n", ". . 4 | 1 . 2 | 3 . . 7 6 4 | 1 5 2 | 3 8 9 \n", "------+-------+------ ------+-------+------\n", ". . 3 | . 2 . | 9 . . 1 7 3 | 5 2 8 | 9 4 6 \n", ". 8 . | . . . | . 5 . 2 8 9 | 3 6 4 | 1 5 7 \n", "6 . . | . . . | . . 2 6 4 5 | 7 9 1 | 8 3 2 \n", "\n", "Puzzle 4: Solution:\n", ". . . | . . . | . . 7 9 6 3 | 8 1 4 | 5 2 7 \n", ". . 4 | . 2 . | 6 . . 1 5 4 | 3 2 7 | 6 8 9 \n", "8 . . | . . . | 3 1 . 8 2 7 | 9 6 5 | 3 1 4 \n", "------+-------+------ ------+-------+------\n", ". . . | . . 2 | 9 . . 3 7 1 | 4 8 2 | 9 5 6 \n", ". 4 . | . 9 . | . 3 . 6 4 5 | 7 9 1 | 8 3 2 \n", ". . 9 | 5 . 6 | . . . 2 8 9 | 5 3 6 | 7 4 1 \n", "------+-------+------ ------+-------+------\n", ". 1 . | . . . | . . 8 5 1 2 | 6 7 3 | 4 9 8 \n", ". . 6 | . 5 . | 2 . . 4 9 6 | 1 5 8 | 2 7 3 \n", "7 . . | . . . | . 6 . 7 3 8 | 2 4 9 | 1 6 5 \n", "\n", "Puzzle 5: Solution:\n", ". . 4 | . . 3 | . . . 5 9 4 | 2 6 3 | 8 7 1 \n", ". 7 . | . 8 . | . . . 6 7 1 | 5 8 4 | 3 2 9 \n", "2 . 8 | 1 . . | . . 6 2 3 8 | 1 9 7 | 5 4 6 \n", "------+-------+------ ------+-------+------\n", ". . 3 | . . . | . 9 . 7 6 3 | 4 1 8 | 2 9 5 \n", ". 8 . | . 2 . | . . . 9 8 5 | 3 2 6 | 7 1 4 \n", "1 . . | 7 . . | . . 3 1 4 2 | 7 5 9 | 6 8 3 \n", "------+-------+------ ------+-------+------\n", ". . . | . . . | 4 5 . 8 1 6 | 9 3 2 | 4 5 7 \n", ". . . | 8 . . | 9 . . 3 5 7 | 8 4 1 | 9 6 2 \n", ". . 9 | . . 5 | . . 8 4 2 9 | 6 7 5 | 1 3 8 \n", "\n", "Puzzle 6: Solution:\n", ". . 6 | . . 1 | . . . 8 2 6 | 9 7 1 | 3 5 4 \n", ". 5 . | . 3 . | . . . 7 5 4 | 8 3 6 | 1 9 2 \n", "9 . . | 4 . . | . . 7 9 1 3 | 4 2 5 | 8 6 7 \n", "------+-------+------ ------+-------+------\n", ". . 1 | . . . | . 2 . 5 7 1 | 6 4 3 | 9 2 8 \n", ". 3 . | . 9 . | . . . 2 3 8 | 1 9 7 | 5 4 6 \n", "4 . . | 5 . . | . . 1 4 6 9 | 5 8 2 | 7 3 1 \n", "------+-------+------ ------+-------+------\n", "3 . . | . . . | 6 8 . 3 4 7 | 2 1 9 | 6 8 5 \n", ". . . | 3 . . | 2 . . 1 8 5 | 3 6 4 | 2 7 9 \n", ". . 2 | . . 8 | . . 3 6 9 2 | 7 5 8 | 4 1 3 \n", "\n", "Puzzle 7: Solution:\n", ". . . | . . . | . . 3 8 6 2 | 7 1 4 | 9 5 3 \n", ". . 1 | . . 9 | . 6 . 7 4 1 | 3 5 9 | 8 6 2 \n", ". 5 . | . 8 . | 4 . . 9 5 3 | 2 8 6 | 4 7 1 \n", "------+-------+------ ------+-------+------\n", ". . . | 9 . . | . 8 . 3 7 4 | 9 2 1 | 6 8 5 \n", ". . 8 | 6 7 . | . . . 2 9 8 | 6 7 5 | 1 3 4 \n", ". 1 . | . . . | 2 . . 6 1 5 | 4 3 8 | 2 9 7 \n", "------+-------+------ ------+-------+------\n", ". . 6 | . . 7 | . 2 . 5 8 6 | 1 4 7 | 3 2 9 \n", ". 3 . | 8 . . | 5 . . 1 3 7 | 8 9 2 | 5 4 6 \n", "4 . . | . . . | . . 8 4 2 9 | 5 6 3 | 7 1 8 \n", "\n", "Puzzle 8: Solution:\n", ". . . | . . . | . . 5 7 1 4 | 9 6 3 | 2 8 5 \n", ". . 6 | . . 8 | 7 . . 9 2 6 | 5 1 8 | 7 3 4 \n", "3 . . | . . . | . 9 . 3 8 5 | 2 7 4 | 6 9 1 \n", "------+-------+------ ------+-------+------\n", ". . . | 1 . 7 | . 4 . 2 3 8 | 1 9 7 | 5 4 6 \n", ". . 7 | . . . | 8 . . 6 5 7 | 3 4 2 | 8 1 9 \n", ". 4 . | . . 6 | . . . 1 4 9 | 8 5 6 | 3 2 7 \n", "------+-------+------ ------+-------+------\n", ". 9 . | . 8 . | . . 3 4 9 2 | 7 8 5 | 1 6 3 \n", ". . 1 | 6 . . | 4 . . 8 7 1 | 6 3 9 | 4 5 2 \n", "5 . . | . 2 . | . . . 5 6 3 | 4 2 1 | 9 7 8 \n", "\n", "Puzzle 9: Solution:\n", ". . . | . . 5 | . . 3 4 2 7 | 1 6 5 | 8 9 3 \n", ". . 9 | . . . | . 4 . 5 3 9 | 2 7 8 | 6 4 1 \n", ". 8 1 | . 4 . | . . . 6 8 1 | 3 4 9 | 7 2 5 \n", "------+-------+------ ------+-------+------\n", ". . . | 7 . . | . . . 2 1 6 | 7 9 3 | 4 5 8 \n", ". . 4 | . . 2 | . . 6 3 9 4 | 5 8 2 | 1 7 6 \n", "8 . . | . 1 4 | . 3 . 8 7 5 | 6 1 4 | 9 3 2 \n", "------+-------+------ ------+-------+------\n", ". . . | . . . | 2 . . 7 5 8 | 4 3 1 | 2 6 9 \n", ". 4 . | . . 6 | . . 7 1 4 3 | 9 2 6 | 5 8 7 \n", "9 . . | . 5 . | . 1 . 9 6 2 | 8 5 7 | 3 1 4 \n", "\n", "Puzzle 10: Solution:\n", ". . . | . . 5 | . . 4 2 8 7 | 9 3 5 | 1 6 4 \n", ". 9 . | . . . | . 2 . 3 9 4 | 1 6 8 | 5 2 7 \n", ". . 6 | . 7 . | 3 . . 5 1 6 | 4 7 2 | 3 8 9 \n", "------+-------+------ ------+-------+------\n", ". . . | 7 . . | 8 . . 6 4 5 | 7 9 1 | 8 3 2 \n", ". . 8 | 6 . . | . . . 9 7 8 | 6 2 3 | 4 5 1 \n", "1 3 . | . 8 . | . . . 1 3 2 | 5 8 4 | 9 7 6 \n", "------+-------+------ ------+-------+------\n", ". . 3 | . 1 . | 6 . . 7 5 3 | 2 1 9 | 6 4 8 \n", ". 2 . | . . . | . . 5 8 2 9 | 3 4 6 | 7 1 5 \n", "4 . . | . . . | . 9 . 4 6 1 | 8 5 7 | 2 9 3 \n" ] } ], "source": [ "!head -10 sudokus_hard.txt > sudokus_hard10.txt\n", "\n", "!java Sudoku -grid -nosummary sudokus_hard10.txt" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# What's Next?\n", "\n", "There are a few things I didn't get a chance to explore; maybe you can:\n", "\n", "- Are there more algorithmic optimizations to try?\n", "- Would the program be even faster if translated to Golang or C++ or Rust?\n", "- Could you use a satisfiability prover such as DPLL to solve puzzles, as [[**tdoku**](https://github.com/t-dillon/tdoku) does?\n", "- The depth-first search makes a choice to *fill* some square with a digit. What if instead the choice was to *eliminate* a digit from the square? At first glance it seems that would be slower, because more choices would be required, but would constraint propagation make it work well?\n", "- On each recursive call, the depth-first search copies the current grid into a new grid (re-using the one in `gridpool[level]`). That requires copying an array of 81 ints. Would it be faster to instead make changes directly to the current grid, and then undo the changes when failure is detected? You would need to keep track of the changes made so that they can be undone. My guess is that this would make the code more complex and not much faster (if any), but you might want to try it.\n", "- In the theory of constraint propagation, [shaving](https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.175.7143&rep=rep1&type=pdf) means guessing a value for some variable, detecting a contradiction, and then keeping track of the fact that the value is not possible. But in our program, when we guess wrong we don't keep track of anything. Can the program be made faster by incorporating shaving?\n", "- Can you create an adversarial puzzle, where the program guesses wrong the maximal number of times? In other words, if you draw a tree of choices, what's a puzzle with a tree that has the solution in the bottom-right corner? How much time and how many backtracks does that puzzle take to solve?\n", "- What [other Sudoku strategies](https://bestofsudoku.com/sudoku-strategy) can be implemented? Can you find a suite of strategies that will solve all the puzzles with no backtracking search? \n", "- Can you develop a system to rank the difficulty of puzzles, based on the complexity of the strategies needed to solve it?\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One final word: I got around to writing this up after a friend mentioned to me \"*Hey, a while back didn't you do a Sudoku program that could solve like **a hundred puzzles a second** or something?*\" I felt like Robert Wagner when he had to explain to Dr. Evil that \"*a million dollars isn't exactly a lot of money these days.*\" \n", "\n", "\n", "\n", "*One **hundred** puzzles per second!*\n", "\n", "\n", "\n", "*A hundred isn't exactly a lot of throughput these days...*" ] } ], "metadata": { "kernelspec": { "display_name": "Python [conda env:base] *", "language": "python", "name": "conda-base-py" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.9" } }, "nbformat": 4, "nbformat_minor": 4 }