daviddoji/pytudes
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Add files via upload

Browse Source
This commit is contained in:
Peter Norvig 2018-01-10 19:18:29 -08:00 committed by GitHub
parent 26bfd0c639
commit b09dcebbf5
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 169 additions and 129 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

250
Coin Flip.ipynb
View File

@ -6,7 +6,7 @@
"source": [
"# The Devil and the Coin Flip Game\n",
"\n",
"If the Devil ever challenges me to a [fiddle contest](https://en.wikipedia.org/wiki/The_Devil_Went_Down_to_Georgia), I'm going to lose. I'd have a better chance at this contest:\n",
"If the Devil ever challenges me to a [fiddle contest](https://en.wikipedia.org/wiki/The_Devil_Went_Down_to_Georgia), I'm going down. But I heard about a contest where I'd have a better chance:\n",
"\n",
"> *You're playing a game with the devil, with your soul at stake. You're sitting at a circular table which has 4 coins, arranged in a diamond, at the 12, 3, 6, and 9 o'clock positions. You are blindfolded, and can never see the coins or the table.*\n",
"\n",
@ -18,31 +18,33 @@
"\n",
"# Analysis\n",
"\n",
"The player, being blindfolded, does not know the true state of the coins. So the player should represent what is known: the *set of possible states* of the coins. We call this a *belief state*. At the start of the game, each of the four coins could be either heads or tails, so that's 2<sup>4</sup> = 16 possibilities in the belief state:\n",
"\n",
" {HHHH, HHHT, HHTH, HHTT, HTHH, HTHT, HTTH, HTTT, THHH, THHT, THTH, THTT, TTHH, TTHT, TTTH, TTTT}\n",
"\n",
"The idea is that even though the player doesn't know for sure what the actual state of the coins is, nor what rotations the devil performs, the player can still manipulate the belief state towards the goal belief state:\n",
"\n",
" {HHHH}\n",
"\n",
"A move updates the belief state as follows: for every coin sequence in the current belief state, rotate it in every possible way, and then flip the coins specified by the position(s) in the move. Collect all these results together to form the new belief state. Solving the game means coming up with a list of moves that ends in the belief state `{'HHHH'}`. It must be a shortest possible sequence of moves, so a [breadth-first search](https://en.wikipedia.org/wiki/Breadth-first_search) is appropriate. The search space is small (just 2<sup>16</sup> possible belief states), so run time will be fast; the only issue is specifying the domain correctly. To increase the chance of getting it right, I won't try to do anything fancy, such as noticing that some coin sequences are rotational variants of other sequences.\n",
"Here are my thoughts:\n",
"- We're looking for a \"shortest sequence of moves\" that reaches a goal. That's a [shortest path search problem](https://en.wikipedia.org/wiki/Shortest_path_problem). I've done that before.\n",
"- Since the Devil gets to make moves too, you might think that this is a [minimax](https://en.wikipedia.org/wiki/Minimax) problem: that we should choose the move that leads to the shortest path, given that the Devil has the option of making moves that lead to the longest path.\n",
"- In fact, the Devil would do well to determine his best move using a minimax search.\n",
"- However, the player can't do that, because minimax only works when you know what moves the opponent is making, and the player is blinfolded; that makes it a [partially observable problem](https://en.wikipedia.org/wiki/Partially_observable_system) (in this case, not observable at all, but we still say \"partially\").\n",
"- In such problems, we don't know for sure the true state of the coins. So we should represent what *is* known: the *set of possible states* of the coins. We call this a *belief state*. At the start of the game, each of the four coins could be either heads or tails, so that's 2<sup>4</sup> = 16 possibilities in the belief state:\n",
" {HHHH, HHHT, HHTH, HHTT, HTHH, HTHT, HTTH, HTTT, \n",
" THHH, THHT, THTH, THTT, TTHH, TTHT, TTTH, TTTT}\n",
"- So we have a single-agent shortest path search in the space of belief states (not the space of physical states of the coins). We search for a path from the inital belief state to the goal belief state, `{HHHH}`.\n",
"- A move updates the belief state as follows: for every coin sequence in the current belief state, rotate it in every possible way, and then flip the coins specified by the position(s) in the move. Collect all these results together to form the new belief state. The search space is small (just 2<sup>16</sup> possible belief states), so run time will be fast; the only issue is specifying the domain correctly. \n",
"- To increase the chance of getting it right, I won't try to do anything fancy, such as noticing that some coin sequences are rotational variants of other sequences (although we'll come back to that later).\n",
"\n",
"\n",
"# Implementation Choices\n",
"\n",
"Here are the main concepts, and my implementation choices:\n",
"Here is the vocabulary of the problem, and my implementation choices:\n",
"\n",
"- `Coins`: A *coin sequence* (on the table) is represented as a `str` of four characters, such as `'HTTT'`. \n",
"- `Belief`: A *belief state* is represented as a `frozenset` of `Coins` (frozen so that it can be hashed).\n",
"- `Position`: A position is an integer index into the coin sequence; position `0` selects the `H` in `'HTTT'`\n",
"and corresponds to the 12 o'clock position; position 1 corresponds to 3 o'clock, and so on.\n",
"- `Move`: A *move* is a set of positions to flip, such as `{0, 1}`. \n",
"- `Strategy`: A strategy for playing the game is just a list of moves. Since there is no feedback while playing\n",
"- `Strategy`: A strategy for playing the game is a list of moves. Since there is no feedback while playing\n",
"(the player is blindfolded) there is no need for decision points in the strategy.\n",
"- `all_coins()`: A belief state consisting of the set of all possible coin sequences: `{'HHHH', 'HHHT', ...}`.\n",
"- `all_coins()`: returns a belief state consisting of the set of all 16 possible coin sequences: `{'HHHH', 'HHHT', ...}`.\n",
"- `rotations(coins)`: returns a set of all 4 rotations of the coin sequence.\n",
"- `update(belief, move)`: an updated belief state: all the coin sequences that could result from any rotation followed by the specified flip(s).\n",
"- `update(belief, move)`: returns an updated belief state: all the coin sequences that could result from any rotation followed by the specified flips.\n",
"- `flip(coins, move)`: flips the specified positions within the coin sequence.\n",
" (But don't flip `'HHHH'`, because the game would have already ended if this were the coin sequence.)"
]
@ -62,7 +64,7 @@
"Coins = ''.join # A coin sequence; a str: 'HHHT'.\n",
"Belief = frozenset # A set of possible coin sequences: {'HHHT', 'TTTH'}\n",
"Move = set # A set of positions to flip: {0, 1}\n",
"Strategy = list # A list of Moves: [{0, 1}, {0, 1, 2, 3}, ...]\n",
"Strategy = list # A list of Moves: [{0, 1, 2, 3}, {0, 2}, ...]\n",
"\n",
"def all_coins() -> Belief:\n",
" \"Return the belief set consisting of all possible coin sequences.\"\n",
@ -274,11 +276,8 @@
"# Search for a Solution\n",
"\n",
"The generic function `search` does a breadth-first search starting\n",
"from a `start` state, looking for a `goal` state, considering possible `actions` (a collection of moves) at each turn,\n",
"and computing the `result` of each action (`result` is a function such that `result(state, action)` returns the new state that results from executing the action in the current state). It works by keeping a queue of unexplored possibilities, where each entry in the queue is a pair consisting of a *strategy* (sequence of moves) and a *state* that that strategy leads to. We also keep a set of `explored` states, so that we don't repeat ourselves.\n",
"\n",
"Amazingly, \n",
"even though we want to search in the space of *belief states*, we can still use the generic search function that is designed for regular-old-states. The search for belief states just works, as long as we properly specify the start state, the goal state, and the means of moving between states."
"from a `start` state, looking for a `goal` state, considering possible `actions` at each turn,\n",
"and computing the `result` of each action (`result` is a function such that `result(state, action)` returns the new state that results from executing the action in the current state). `search` works by keeping a `queue` of unexplored possibilities, where each entry in the queue is a pair consisting of a *strategy* (sequence of moves) and a *state* that that strategy leads to. We also keep track of a set of `explored` states, so that we don't repeat ourselves. I've defined this function (or one just like it) multiple times before, for use in different search problems."
]
},
{
@ -292,7 +291,7 @@
"def search(start, goal, actions, result) -> Strategy:\n",
" \"Breadth-first search from start state to goal; return strategy to get to goal.\"\n",
" explored = set()\n",
" queue = deque([([], start)])\n",
" queue = deque([(Strategy(), start)])\n",
" while queue:\n",
" (strategy, state) = queue.popleft()\n",
" if state == goal:\n",
@ -300,15 +299,17 @@
" for action in actions:\n",
" state2 = result(state, action)\n",
" if state2 not in explored:\n",
" explored.add(state2)\n",
" queue.append((strategy + [action], state2))"
" queue.append((strategy + [action], state2))\n",
" explored.add(state2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `coin_search` function calls the generic `search` function to solve our specific problem:"
"Now, `search` doesn't know anything about belief states&mdash;it is designed to work on plain-old physical states of the world. But amazingly, we can still use it to search over belief states: it just works, as long as we properly specify the start state, the goal state, and the means of moving between states.\n",
"\n",
"The `coin_search` function calls `search` to solve our specific problem:"
]
},
{
@ -321,8 +322,7 @@
"source": [
"def coin_search() -> Strategy: \n",
" \"Use `search` to solve the Coin Flip problem.\"\n",
" return search(start=all_coins(), goal={'HHHH'}, \n",
" actions=powerset(range(4)), result=update)"
" return search(start=all_coins(), goal={'HHHH'}, actions=powerset(range(4)), result=update)"
]
},
{
@ -420,20 +420,20 @@
"text/plain": [
"Counter({0: 10000,\n",
" 1: 10000,\n",
" 2: 10033,\n",
" 3: 9967,\n",
" 4: 9996,\n",
" 5: 9962,\n",
" 6: 10027,\n",
" 7: 10015,\n",
" 8: 9895,\n",
" 9: 10011,\n",
" 10: 9995,\n",
" 11: 10127,\n",
" 12: 10019,\n",
" 13: 9902,\n",
" 14: 10074,\n",
" 15: 9977})"
" 2: 10105,\n",
" 3: 9895,\n",
" 4: 9817,\n",
" 5: 10166,\n",
" 6: 9928,\n",
" 7: 10089,\n",
" 8: 10028,\n",
" 9: 10082,\n",
" 10: 9991,\n",
" 11: 10105,\n",
" 12: 9976,\n",
" 13: 9970,\n",
" 14: 9948,\n",
" 15: 9900})"
]
},
"execution_count": 12,
@ -507,7 +507,43 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The starting belief set is down from 16 to 6, namely 4 heads, 3 heads, 2 adjacent heads, 2 opposite heads, 1 head, and no heads, respectively. "
"The starting belief set is down from 16 to 6, namely 4 heads, 3 heads, 2 adjacent heads, 2 opposite heads, 1 head, and no heads, respectively. \n",
"\n",
"Let's make sure we didn't break anything:"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{0, 1, 2, 3},\n",
" {0, 2},\n",
" {0, 1, 2, 3},\n",
" {0, 1},\n",
" {0, 1, 2, 3},\n",
" {0, 2},\n",
" {0, 1, 2, 3},\n",
" {0},\n",
" {0, 1, 2, 3},\n",
" {0, 2},\n",
" {0, 1, 2, 3},\n",
" {0, 1},\n",
" {0, 1, 2, 3},\n",
" {0, 2},\n",
" {0, 1, 2, 3}]"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"coin_search()"
]
},
{
@ -516,12 +552,12 @@
"source": [
"# Solutions for *N* Coins\n",
"\n",
"What if there are 3 coins on the table arranged in a triangle? Or 6 coins in a hexagon? To answer that, I'll generalize the three functions that have a \"4\" in them, `all_coins`, `rotations` and `coin_search`. I'll also generalize `flip`, which had `'HHHH'` in it. "
"What if there are 3 coins on the table arranged in a triangle? Or 6 coins in a hexagon? To answer that, I'll generalize all the functions that have a \"4\" in them: `all_coins`, `rotations` and `coin_search`, as well as `flip`, which has `'HHHH'` in it, and introduce `all_moves(N)`:"
]
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": 16,
"metadata": {
"collapsed": true
},
@ -537,8 +573,7 @@
"\n",
"def coin_search(N=4) -> Strategy: \n",
" \"Use the generic `search` function to solve the Coin Flip problem.\"\n",
" return search(start=all_coins(N), goal={'H' * N}, \n",
" actions=all_moves(N), result=update)\n",
" return search(start=all_coins(N), goal={'H' * N}, actions=all_moves(N), result=update)\n",
"\n",
"def flip(coins, move) -> Coins:\n",
" \"Flip the coins in the positions specified by the move (but leave all 'H' alone).\"\n",
@ -553,12 +588,33 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"I also need to generalize `all_moves`. To compute the set of possible moves for 4 coins, I used `powerset`, and got 16 possible moves. Now I want to know the set od canonicalized moves for any *N*. To get that, I'll look at the canonicalized set of `all_coins(N)`, and for each one pull out the positions that have an `H` in them, and flip those (the positions with a `T` should be symmetric, so we don't need them as well)."
"Let's test the new definitions and make sure we haven't broken `update` and `coin_search`:"
]
},
{
"cell_type": "code",
"execution_count": 16,
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"assert all_coins(4) == {'HHHH', 'HHHT', 'HHTT', 'HTHT', 'HTTT', 'TTTT'}\n",
"assert all_coins(5) == {'HHHHH','HHHHT', 'HHHTT','HHTHT','HHTTT', 'HTHTT', 'HTTTT', 'TTTTT'}\n",
"assert rotations('HHHHHT') == {'HHHHHT', 'HHHHTH', 'HHHTHH', 'HHTHHH', 'HTHHHH', 'THHHHH'}\n",
"assert update({'TTTTTTT'}, {3}) == {'HTTTTTT'}\n",
"assert (update(rotations('HHHHHT'), {0}) == update({'HHTHHH'}, {1}) == update({'THHHHH'}, {2})\n",
" == {'HHHHHH', 'HHHHTT', 'HHHTHT', 'HHTHHT'})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To compute the set of all possible moves for 4 coins, I used `powerset(range(4))`, and got 16 possible moves. Now I want to know the set of all *canonicalized* moves for any *N*: the moves `{0}` and `{1}` should be considered the same, since they both say \"flip one coin.\" I'll look at the canonicalized set of `all_coins(N)`, and for each one pull out the set of positions that have an `H` in them and flip those positions. (The positions with a `T` should be symmetric, so we don't need them as well.)"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"collapsed": true
},
@ -566,47 +622,33 @@
"source": [
"def all_moves(N) -> [Move]:\n",
" \"All rotationally invariant moves for a sequence of N coins.\"\n",
" return [set(i for (i, coin) in enumerate(coins) if coin == 'H')\n",
" return [set(i for i in range(N) if coins[i] == 'H')\n",
" for coins in sorted(all_coins(N))]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's test the new definitions:"
]
},
{
"cell_type": "code",
"execution_count": 17,
"execution_count": 19,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'ok'"
"[{0, 1, 2}, {0, 1}, {0}, set()]"
]
},
"execution_count": 17,
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"assert all_moves(4) == [{0, 1, 2, 3}, {0, 1, 2}, {0, 1}, {0, 2}, {0}, set()]\n",
"assert all_coins(4) == {'HHHH', 'HHHT', 'HHTT', 'HTHT', 'HTTT', 'TTTT'}\n",
"assert all_coins(5) == {'HHHHH','HHHHT', 'HHHTT','HHTHT','HHTTT', 'HTHTT', 'HTTTT', 'TTTTT'}\n",
"assert rotations('HHHHHT') == {'HHHHHT', 'HHHHTH', 'HHHTHH', 'HHTHHH', 'HTHHHH', 'THHHHH'}\n",
"assert update({'TTTTTTT'}, {3}) == {'HTTTTTT'}\n",
"assert (update(rotations('HHHHHT'), {0}) == update({'HHHHHT'}, {1 })== update({'HHHHHT'}, {2})\n",
" == {'HHHHHH', 'HHHHTT', 'HHHTHT', 'HHTHHT'})\n",
"'ok'"
"all_moves(3)"
]
},
{
"cell_type": "code",
"execution_count": 18,
"execution_count": 20,
"metadata": {},
"outputs": [
{
@ -615,7 +657,7 @@
"[{0, 1, 2, 3}, {0, 1, 2}, {0, 1}, {0, 2}, {0}, set()]"
]
},
"execution_count": 18,
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
@ -624,40 +666,18 @@
"all_moves(4)"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'HHHHHT', 'HHHHTH', 'HHHTHH', 'HHTHHH', 'HTHHHH', 'THHHHH'}"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
" rotations('HHHHHT')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With 4 coins we can flip 4, flip 3, flip 2 adjacent, flip 2 opposite, flip 1, or flip nothing.\n",
"Similarly, with 4 coins there can be 4 heads 3 heads, 2 adjacent heads, 2 opposite heads, 1 head, or no heads.\n",
"If we start with 6 coins of which one is `'T'`, and do a single flip (and it doesn't matter what position that flip is), then either the Devil flips the `'T'`, in which case we get all heads, or it can flip an `'H'` which is either adjacent to, 2 away from, or 3 away from the existing `'T'`.\n",
"\n",
"How many distinct canonical coin sequences are there for *N* coins from 1 to 10?"
]
},
{
"cell_type": "code",
"execution_count": 20,
"execution_count": 21,
"metadata": {},
"outputs": [
{
@ -666,7 +686,7 @@
"{1: 2, 2: 3, 3: 4, 4: 6, 5: 8, 6: 14, 7: 20, 8: 36, 9: 60, 10: 108}"
]
},
"execution_count": 20,
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
@ -680,14 +700,14 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"On the one hand this is encouraging; there are only 108 canonical coin sequences of length 10, far less than the 1024 non-canonical squences. On the other hand, it is discouraging; since we are searching over the belief states, that would be 2<sup>108</sup> ≌ 10<sup>32</sup> belief states, which is infeasible. However, we should be able to easily handle up to N=7, because 2<sup>20</sup> is only a million.\n",
"On the one hand this is encouraging; there are only 108 canonical coin sequences of length 10, far less than the 1,024 non-canonical squences. On the other hand, it is discouraging; since we are searching over belief states, that would be 2<sup>108</sup> ≌ 10<sup>32</sup> belief states, which is not feasible. However, we should be able to easily handle up to N=7, because 2<sup>20</sup> is only a million.\n",
"\n",
"# Solutions for 1 to 7 Coins"
]
},
{
"cell_type": "code",
"execution_count": 21,
"execution_count": 22,
"metadata": {},
"outputs": [
{
@ -716,7 +736,7 @@
" 7: None}"
]
},
"execution_count": 21,
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
@ -731,7 +751,7 @@
"source": [
"Too bad; there are no solutions for N = 3, 5, 6, or 7. \n",
"\n",
"There are solutions for N = 1, 2, 4; they have lengths 1, 3, 15, respectively. That suggests the conjecture: \n",
"There *are* solutions for N = 1, 2, 4; they have lengths 1, 3, 15, respectively. That suggests the conjecture: \n",
"\n",
"> For every *N* that is a power of 2, there will be a shortest solution of length 2<sup>*N*</sup> - 1.\n",
"\n",
@ -744,14 +764,14 @@
},
{
"cell_type": "code",
"execution_count": 22,
"execution_count": 23,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 1min 22s, sys: 288 ms, total: 1min 22s\n",
"CPU times: user 1min 22s, sys: 287 ms, total: 1min 22s\n",
"Wall time: 1min 23s\n"
]
}
@ -762,7 +782,7 @@
},
{
"cell_type": "code",
"execution_count": 23,
"execution_count": 24,
"metadata": {},
"outputs": [
{
@ -771,7 +791,7 @@
"255"
]
},
"execution_count": 23,
"execution_count": 24,
"metadata": {},
"output_type": "execute_result"
}
@ -805,7 +825,7 @@
},
{
"cell_type": "code",
"execution_count": 24,
"execution_count": 25,
"metadata": {
"collapsed": true
},
@ -833,7 +853,27 @@
},
{
"cell_type": "code",
"execution_count": 25,
"execution_count": 26,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" 0 | | HH HT TT | 0\n",
" 1 | 01 | HH HT | 1\n",
" 2 | 0 | HH TT | 2\n",
" 3 | 01 | HH | 3\n"
]
}
],
"source": [
"show(coin_search(2), 2)"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [
{
@ -869,13 +909,13 @@
"source": [
"We can see that every odd-numbered move flips all four coins to eliminate the possibility of `TTTT`, flipping it to `HHHH`. We can also see that moves 2, 4, and 6 flip two coins and have the effect of eventually eliminating the two \"two heads\" sequences from the belief state, and then move 8 eliminates the \"three heads\" and \"one heads\" sequences, while bringing back the \"two heads\" possibilities. Repeating moves 2, 4, and 6 in moves 10, 12, and 14 then re-eliminates the \"two heads\", and move 15 gets the belief state down to `{'HHHH'}`.\n",
"\n",
"You could call `show(solution8)`, but the results look bad unless you have a very wide (340 characters) screen to view it on. So I'll just show `solution8` itself, with move numbers:\n",
"You could call `show(solution8, 8)`, but the results look bad unless you have a very wide (345 characters) screen to view it on. So I'll just show `solution8` itself, with move numbers:\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 26,
"execution_count": 28,
"metadata": {},
"outputs": [
{
@ -1138,7 +1178,7 @@
" 255: {0, 1, 2, 3, 4, 5, 6, 7}}"
]
},
"execution_count": 26,
"execution_count": 28,
"metadata": {},
"output_type": "execute_result"
}