diff --git a/ipynb/Snobol.ipynb b/ipynb/Snobol.ipynb index 1c6435d..db25bdd 100644 --- a/ipynb/Snobol.ipynb +++ b/ipynb/Snobol.ipynb @@ -10,30 +10,28 @@ "\n", "> *As a student, did you ever get a bad grade on a programming assignment?* \n", "\n", - "It has been a long time since college so I've forgotten many of my assignments, but there is one I do remember. It was something like this:\n", + "I've forgotten most of my assignments, but there is one I do remember. It was something like this:\n", "\n", "## The Concordance Assignment\n", "\n", "> *Using the [`Snobol`](http://www.snobol4.org/) language, read lines of text from the standard input and print a *concordance*, which is an alphabetized list of words in the text, with the line number(s) where each word appears. Words with different capitalization (like \"A\" and \"a\") should be merged into one entry.*\n", "\n", - "After studying Snobol a bit, I realized that the intended solution was along these lines:\n", + "After studying Snobol a bit, I realized that the expected solution was along these lines:\n", "\n", - "1. Create an empty `dict`, let's call it `entries`.\n", - "2. Read the lines of text (tracking the line numbers), split them into words, and build up the string of line numbers for each word with something like\n", - "`entries[word] += line_number + \", \"`\n", + "1. Create an empty `dict` (Snobol calls these \"tables\") whose keys will be words and values will be lists of line numbers.\n", + "2. Read the lines of text (tracking the line numbers), split them into words, and build up the list of line numbers for each word.\n", "3. Convert the table into a two-dimensional `array` where each row has the two columns `[word, line_numbers]`.\n", - "4. Write a function to sort the array alphabetically (`sort` was not built-in to Snobol).\n", + "4. Write a function to sort the array alphabetically (`sort` is not built-in to Snobol).\n", "5. Write a function to print the array.\n", "\n", - "That would probably be about 40 lines of code; an easy task, the main point of which was just to get familiar with a new and different programming language. But I noticed three interesting things about Snobol:\n", + "That would be around 40 to 60 lines of code; an easy task. But I noticed three interesting things about Snobol:\n", "\n", - "* There is an *indirection* operator, `$`, so the two statements `word = \"A\"; $word = \"1\"` are equivalent to `A = \"1\"`.\n", - "* Uninitialized variables are treated as the empty string, so `$word += \"1\"` works even if we haven't seen `$word` before.\n", - "* When the program ends, the Snobol interpreter automatically\n", - "prints the values of\n", - "every variable, sorted alphabetically, as a debugging aid.\n", + "* There is an *indirection* operator, `$`, so if the variable `'word'` has the value `\"A\"`, then `'$word = i'` is the same as `'A = i'`.\n", + "* Uninitialized variables are treated as the empty string, so `'A += \"text\"'` works even if we haven't seen `'A'` before.\n", + "* When the program ends, the Snobol interpreter automatically\n", + "prints the values of every variable, sorted alphabetically, as a debugging aid.\n", "\n", - "That means I can do away with the `dict` and the `array`, eliminating steps 1, 3, 4, and 5, and just do step 2! My entire program is as follows (to make it easier to understand I use Python syntax, except for Snobol's `$word` indirection:" + "That means I could do away with the `dict` and `array` data structures, eliminating steps 1, 3, 4, and 5, and just do step 2! I ended up with a program similar to the following, but to avoid having to explain Snobol syntax, I use all Python syntax ecept for the `'$word'` indirection:" ] }, { @@ -45,9 +43,9 @@ "outputs": [], "source": [ "program = \"\"\"\n", - "for num, line in enumerate(input):\n", + "for i, line in enumerate(input):\n", " for word in re.findall(r\"\\w+\", line.upper()):\n", - " $word += str(num) + \", \"\n", + " $word += str(i) + \", \"\n", "\"\"\"" ] }, @@ -55,110 +53,173 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "That's 3 lines, not 40, and it works! Actually, it wasn't quite perfect, because the post-mortem printout included the values of the variables `num, line,` and `word`. So, reluctantly, I had to increase the program's line count by 33%:" + "That's just 3 lines, not 40 to 60! \n", + "\n", + "To test the program, I'll write a mock Snobol/Python interpreter, which at heart is just a call to the Python interpreter, `exec(program)`, except that it handles the three things I noticed about Snobol:\n", + "\n", + "* `$word` gets translated as `_context[word]`.\n", + "* It calls `exec(program, _context)`, where `_context` is a `defaultdict(str)`, so variables default to `''`.\n", + "* After the `exec` completes, the user-defined variables (but not the built-in ones) are printed." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { - "collapsed": true + "collapsed": false }, "outputs": [], - "source": [ - "program = \"\"\"\n", - "for num, line in enumerate(input):\n", - " for word in re.findall(r\"\\w+\", line.upper()):\n", - " $word += str(num) + \", \"\n", - "del num, line, word\n", - "\"\"\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To prove it works, I'll write a mock Snobol/Python interpreter, which is really just a call to the Python interpreter, `exec`, except that it does three Snobol-ish things: (1) the `context` in which the program is executed is a `defaultdict(str)`, which means that variables default to the empty string. The context is pre-loaded with builtins. (2) Any `$word` in the program is rewritten as `_context[word]`, which enables\n", - "indirection. (3) After the `exec` completes, the user-defined variables (but not the built-in ones) and their values are printed." - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [], "source": [ "from collections import defaultdict\n", "import re\n", "\n", "def snobol(program, data=''):\n", " \"\"\"A Python interpreter with three Snobol-ish features:\n", - " (1) variables default to '', (2) $word indirection, (3) post-mortem printout.\"\"\"\n", - " context = defaultdict(str, vars(__builtins__))\n", - " context.update(_context=context, re=re, input=data.splitlines())\n", - " builtins = set(context) # The builtin functions won't appear in the printout\n", - " program = re.sub(r'\\$(\\w+)', r'_context[\\1]', program) # \"$word\" => \"_context[word]\"\n", - " exec(program, context)\n", - " print('-' * 50)\n", - " for name in sorted(context):\n", - " if name not in builtins and not name.startswith('_'):\n", - " print('{:10} = {}'.format(name, context[name]))" + " (1) $word indirection; (2) variables default to ''; (3) post-mortem dump.\"\"\"\n", + " program = re.sub(r'\\$(\\w+)', r'_context[\\1]', program) # (1) \n", + " _context = defaultdict(str, vars(__builtins__)) # (2) \n", + " _context.update(re=re, input=data.splitlines(), _context=_context)\n", + " builtins = set(_context)\n", + " try:\n", + " exec(program, _context)\n", + " finally:\n", + " print('-' * 79) # (3)\n", + " for name in sorted(_context):\n", + " if not (name in builtins or name == '__builtins__'):\n", + " print('{:10} = {}'.format(name, _context[name]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Now we can run my program on some data:" + "Now we can run the program on some data:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "data = \"\"\"\n", + "There she was just a-walkin' down the street, \n", + "Singin' \"Do wah diddy diddy dum diddy do\"\n", + "Snappin' her fingers and shufflin' her feet, \n", + "Singin' \"Do wah diddy diddy dum diddy do\"\n", + "She looked good (looked good), she looked fine (looked fine)\n", + "She looked good, she looked fine and I nearly lost my mind\n", + "\"\"\"" ] }, { "cell_type": "code", "execution_count": 4, - "metadata": {}, + "metadata": { + "collapsed": false + }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "--------------------------------------------------\n", + "-------------------------------------------------------------------------------\n", "A = 1, \n", - "AND = 2, 4, \n", - "DIDDY = 1, 1, 1, 2, 2, 2, \n", - "DO = 1, 1, 2, 2, \n", + "AND = 3, 6, \n", + "DIDDY = 2, 2, 2, 4, 4, 4, \n", + "DO = 2, 2, 4, 4, \n", "DOWN = 1, \n", - "DUM = 1, 2, \n", - "FEET = 2, \n", - "FINE = 3, 3, 4, \n", - "FINGERS = 2, \n", - "GOOD = 3, 3, 4, \n", - "HER = 2, 2, \n", - "I = 4, \n", + "DUM = 2, 4, \n", + "FEET = 3, \n", + "FINE = 5, 5, 6, \n", + "FINGERS = 3, \n", + "GOOD = 5, 5, 6, \n", + "HER = 3, 3, \n", + "I = 6, \n", "JUST = 1, \n", - "LOOKED = 3, 3, 3, 3, 4, 4, \n", - "LOST = 4, \n", - "MIND = 4, \n", - "MY = 4, \n", - "NEARLY = 4, \n", - "SHE = 1, 3, 3, 4, 4, \n", - "SHUFFLIN = 2, \n", - "SINGIN = 1, 2, \n", - "SNAPPIN = 2, \n", + "LOOKED = 5, 5, 5, 5, 6, 6, \n", + "LOST = 6, \n", + "MIND = 6, \n", + "MY = 6, \n", + "NEARLY = 6, \n", + "SHE = 1, 5, 5, 6, 6, \n", + "SHUFFLIN = 3, \n", + "SINGIN = 2, 4, \n", + "SNAPPIN = 3, \n", "STREET = 1, \n", "THE = 1, \n", "THERE = 1, \n", - "WAH = 1, 2, \n", + "WAH = 2, 4, \n", + "WALKIN = 1, \n", + "WAS = 1, \n", + "i = 6\n", + "line = She looked good, she looked fine and I nearly lost my mind\n", + "word = MIND\n" + ] + } + ], + "source": [ + "snobol(program, data)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Oops! The post-mortem printout includes the variables `i`, `line`, and `word`. Reluctantly, I increased the program's line count by 33%:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "-------------------------------------------------------------------------------\n", + "A = 1, \n", + "AND = 3, 6, \n", + "DIDDY = 2, 2, 2, 4, 4, 4, \n", + "DO = 2, 2, 4, 4, \n", + "DOWN = 1, \n", + "DUM = 2, 4, \n", + "FEET = 3, \n", + "FINE = 5, 5, 6, \n", + "FINGERS = 3, \n", + "GOOD = 5, 5, 6, \n", + "HER = 3, 3, \n", + "I = 6, \n", + "JUST = 1, \n", + "LOOKED = 5, 5, 5, 5, 6, 6, \n", + "LOST = 6, \n", + "MIND = 6, \n", + "MY = 6, \n", + "NEARLY = 6, \n", + "SHE = 1, 5, 5, 6, 6, \n", + "SHUFFLIN = 3, \n", + "SINGIN = 2, 4, \n", + "SNAPPIN = 3, \n", + "STREET = 1, \n", + "THE = 1, \n", + "THERE = 1, \n", + "WAH = 2, 4, \n", "WALKIN = 1, \n", "WAS = 1, \n" ] } ], "source": [ - "data = \"\"\"\n", - "There she was just a-walkin' down the street, singin' \"Do wah diddy diddy dum diddy do\"\n", - "Snappin' her fingers and shufflin' her feet, singin' \"Do wah diddy diddy dum diddy do\"\n", - "She looked good (looked good), she looked fine (looked fine)\n", - "She looked good, she looked fine and I nearly lost my mind\n", + "program = \"\"\"\n", + "for i, line in enumerate(input):\n", + " for word in re.findall(r\"\\w+\", line.upper()):\n", + " $word += str(i) + \", \"\n", + "del i, line, word\n", "\"\"\"\n", "\n", "snobol(program, data)" @@ -168,29 +229,29 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Looks good to me! But sadly, the grader did not agree, complaining that my program was not extensible: what if I wanted to cover two or more files in one run? What if I wanted the output to have a slightly different format? I argued that [YAGNI](https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it), but the grader wasn't impressed so I got points taken off.\n", + "Looks good to me! \n", + "\n", + "But sadly, the grader for the course did not agree, complaining that my program was not extensible: what if I wanted to cover two or more files in one run? What if I wanted the output to have a slightly different format? I argued that [YAGNI](https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it), and if the requirements\n", + "changed, *then* I would write the necessary 40 or 60 lines, but there's no sense doing that until then. The grader was not impressed with my arguments and I got points taken off. \n", + "\n", + "Still, I was happy with my program. I felt like the\n", + "purpose of the assignment was to get familiar with a new programming language with some different idioms/paradigms. By using the indirection operator I learned more about \"thinking different\" than if I had written the expected program.\n", "\n", "# TFW you flunk AI\n", "\n", - "Here's another example that I had completely forgotten about until 2016, when I was cleaning out some old files and came across my old college transcript. It turns out that *I flunked an AI course!* (Or at least, didn't complete it.) This course was\n", - "offered by Prof. Richard Millward in the Cognitive Science program. I certainly remember a lot of influential material from this class: we read David Marr, we read Winston's just-published *Psychology of Computer Vision*, we read a chapter from Duda and Hart which was then only a few years old. The things I learned in that course have stuck with me for decades, but one thing that didn't stick is that my transcript said I never completed the course! I'm not sure what happened. I did an independent study with Ulf Grenander that semester; maybe I had to drop the AI course to fit that in. I don't know. \n", + "Here's another example that I had completely forgotten about until 2016, when I was cleaning out a filing cabinet and came across my old college transcript. It turns out that *I flunked an AI course!* (Or at least, didn't complete it.) This course was offered by Prof. Richard Millward in the Cognitive Science program. I certainly remember a lot of influential material from this class: we read David Marr, we read Winston's just-published *Psychology of Computer Vision*, we read a chapter from Duda and Hart which was then only a few years old. The things I learned in that course have stuck with me for decades, but one thing that didn't stick is that, according to my transcript, I never completed the course! I'm not sure what happened. I did an independent study with Ulf Grenander that semester; my best guess\n", + "is that when I started doing the independent study that would have put me over some limit, and so I had to drop the AI course. \n", "\n", - "So in both the concordance program and the Cognitive Science AI class, I had a great experience and I learned a lot, even if it wasn't well-reflected in official credit. The moral is: look for the good experiences, and don;t worry about the official credit.\n", - "\n", - "\n" + "So in both the concordance program and the Cognitive Science AI class, I had a great experience and I learned a lot, even if it wasn't well-reflected in official credit. The moral is: look for the good experiences, and don't worry about the official credit.\n" ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [] } ], "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, "language_info": { "codemirror_mode": { "name": "ipython", @@ -201,7 +262,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.5.3" + "version": "3.6.0" } }, "nbformat": 4,