259 lines
7.2 KiB
Markdown
259 lines
7.2 KiB
Markdown
\[ [Index](index.md) | [Exercise 7.6](ex7_6.md) | [Exercise 8.2](ex8_2.md) \]
|
|
|
|
# Exercise 8.1
|
|
|
|
*Objectives:*
|
|
|
|
- Learn how to customize iteration using generators
|
|
|
|
*Files Modified:* `structure.py`
|
|
|
|
*Files Created:* `follow.py`
|
|
|
|
## (a) A Simple Generator
|
|
|
|
If you ever find yourself wanting to customize iteration, you should
|
|
always think generator functions. They're easy to write---simply make
|
|
a function that carries out the desired iteration logic and uses `yield`
|
|
to emit values.
|
|
|
|
For example, try this generator that allows you to iterate over a
|
|
range of numbers with fractional steps (something not supported by
|
|
the `range()` builtin):
|
|
|
|
```python
|
|
>>> def frange(start,stop,step):
|
|
while start < stop:
|
|
yield start
|
|
start += step
|
|
|
|
>>> for x in frange(0, 2, 0.25):
|
|
print(x, end=' ')
|
|
|
|
0 0.25 0.5 0.75 1.0 1.25 1.5 1.75
|
|
>>>
|
|
```
|
|
|
|
Iterating on a generator is a one-time operation. For example, here's
|
|
what happen if you try to iterate twice:
|
|
|
|
```python
|
|
>>> f = frange(0, 2, 0.25)
|
|
>>> for x in f:
|
|
print(x, end=' ')
|
|
|
|
0 0.25 0.5 0.75 1.0 1.25 1.5 1.75
|
|
>>> for x in f:
|
|
print(x, end=' ')
|
|
|
|
>>>
|
|
```
|
|
|
|
If you want to iterate over the same sequence, you need to recreate the generator
|
|
by calling `frange()` again. Alternative, you could package everything into a class:
|
|
|
|
```python
|
|
>>> class FRange:
|
|
def __init__(self, start, stop, step):
|
|
self.start = start
|
|
self.stop = stop
|
|
self.step = step
|
|
def __iter__(self):
|
|
n = self.start
|
|
while n < self.stop:
|
|
yield n
|
|
n += self.step
|
|
|
|
>>> f = FRange(0, 2, 0.25)
|
|
>>> for x in f:
|
|
print(x, end=' ')
|
|
|
|
0 0.25 0.5 0.75 1.0 1.25 1.5 1.75
|
|
>>> for x in f:
|
|
print(x, end=' ')
|
|
|
|
0 0.25 0.5 0.75 1.0 1.25 1.5 1.75
|
|
>>>
|
|
```
|
|
|
|
## (b) Adding Iteration to Objects
|
|
|
|
If you've created a custom class, you can make it support iteration by
|
|
defining an `__iter__()` special method. `__iter__()` returns an
|
|
iterator as a result. As shown in the previous example, an easy way
|
|
to do it is to define `__iter__()` as a generator.
|
|
|
|
In earlier exercises, you defined a `Structure` base class.
|
|
Add an `__iter__()` method to this class that produces the attribute values
|
|
in order. For example:
|
|
|
|
```python
|
|
class Structure(metaclass=StructureMeta):
|
|
...
|
|
def __iter__(self):
|
|
for name in self._fields:
|
|
yield getattr(self, name)
|
|
...
|
|
```
|
|
|
|
Once you've done this, you should be able to iterate over the instance
|
|
attributes like this:
|
|
|
|
```python
|
|
>>> from stock import Stock
|
|
>>> s = Stock('GOOG', 100, 490.1)
|
|
>>> for val in s:
|
|
print(val)
|
|
GOOG
|
|
100
|
|
490.1
|
|
>>>
|
|
```
|
|
|
|
## (c) The Surprising Power of Iteration
|
|
|
|
Python uses iteration in ways you might not expect. Once you've added `__iter__()`
|
|
to the `Structure` class, you'll find that it is easy to do all sorts of new
|
|
operations. For example, conversions to sequences and unpacking:
|
|
|
|
```python
|
|
>>> s = Stock('GOOG', 100, 490.1)
|
|
>>> list(s)
|
|
['GOOG', 100, 490.1]
|
|
>>> tuple(s)
|
|
('GOOG', 100, 490.1)
|
|
>>> name, shares, price = s
|
|
>>> name
|
|
'GOOG'
|
|
>>> shares
|
|
100
|
|
>>> price
|
|
490.1
|
|
>>>
|
|
```
|
|
|
|
While we're at it, we can now add a comparison operator to our `Structure`
|
|
class:
|
|
|
|
```python
|
|
# structure.py
|
|
class Structure(metaclass=StructureMeta):
|
|
...
|
|
def __eq__(self, other):
|
|
return isinstance(other, type(self)) and tuple(self) == tuple(other)
|
|
...
|
|
```
|
|
|
|
You should now be able to compare objects:
|
|
|
|
```python
|
|
>>> a = Stock('GOOG', 100, 490.1)
|
|
>>> b = Stock('GOOG', 100, 490.1)
|
|
>>> a == b
|
|
True
|
|
>>>
|
|
```
|
|
|
|
Try running your `teststock.py` unit tests again. Everything should be passing now.
|
|
Excellent.
|
|
|
|
## (d) Monitoring a streaming data source
|
|
|
|
Generators can also be a useful way to simply produce a stream of
|
|
data. In this part, we'll explore this idea by writing a generator to
|
|
watch a log file. To start, follow the next instructions carefully.
|
|
|
|
The program `Data/stocksim.py` is a program that
|
|
simulates stock market data. As output, the program constantly writes
|
|
real-time data to a file `stocklog.csv`. In a
|
|
command window (not IDLE) go into the `Data/` directory and run this program:
|
|
|
|
```
|
|
% python3 stocksim.py
|
|
```
|
|
|
|
If you are on Windows, just locate the `stocksim.py` program and
|
|
double-click on it to run it. Now, forget about this program (just
|
|
let it run). Again, just let this program run in the background---it
|
|
will run for several hours (you shouldn't need to worry about it).
|
|
|
|
Once the above program is running, let's write a little program to
|
|
open the file, seek to the end, and watch for new output. Create a
|
|
file `follow.py` and put this code in it:
|
|
|
|
```python
|
|
# follow.py
|
|
import os
|
|
import time
|
|
f = open('Data/stocklog.csv')
|
|
f.seek(0, os.SEEK_END) # Move file pointer 0 bytes from end of file
|
|
|
|
while True:
|
|
line = f.readline()
|
|
if line == '':
|
|
time.sleep(0.1) # Sleep briefly and retry
|
|
continue
|
|
fields = line.split(',')
|
|
name = fields[0].strip('"')
|
|
price = float(fields[1])
|
|
change = float(fields[4])
|
|
if change < 0:
|
|
print('%10s %10.2f %10.2f' % (name, price, change))
|
|
```
|
|
|
|
If you run the program, you'll see a real-time stock ticker. Under the covers,
|
|
this code is kind of like the Unix `tail -f` command that's used to watch a log file.
|
|
|
|
**Note:** The use of the `readline()` method in this example is
|
|
somewhat unusual in that it is not the usual way of reading lines from
|
|
a file (normally you would just use a `for`-loop). However, in
|
|
this case, we are using it to repeatedly probe the end of the file to
|
|
see if more data has been added (`readline()` will either
|
|
return new data or an empty string).
|
|
|
|
If you look at the code carefully, the first part of the code is
|
|
producing lines of data whereas the statements at the end of the
|
|
`while` loop are consuming the data. A major feature of generator
|
|
functions is that you can move all of the data production code into a
|
|
reusable function.
|
|
|
|
Modify the code so that the file-reading is performed by
|
|
a generator function `follow(filename)`. Make it so the following code
|
|
works:
|
|
|
|
```python
|
|
>>> for line in follow('Data/stocklog.csv'):
|
|
print(line, end='')
|
|
|
|
... Should see lines of output produced here ...
|
|
```
|
|
|
|
Modify the stock ticker code so that it looks like this:
|
|
|
|
```python
|
|
for line in follow('Data/stocklog.csv'):
|
|
fields = line.split(',')
|
|
name = fields[0].strip('"')
|
|
price = float(fields[1])
|
|
change = float(fields[4])
|
|
if change < 0:
|
|
print('%10s %10.2f %10.2f' % (name, price, change))
|
|
```
|
|
|
|
**Discussion**
|
|
|
|
Something very powerful just happened here. You moved an interesting iteration pattern
|
|
(reading lines at the end of a file) into its own little function. The `follow()` function
|
|
is now this completely general purpose utility that you can use in any program. For
|
|
example, you could use it to watch server logs, debugging logs, and other similar data sources.
|
|
That's kind of cool.
|
|
|
|
\[ [Solution](soln8_1.md) | [Index](index.md) | [Exercise 7.6](ex7_6.md) | [Exercise 8.2](ex8_2.md) \]
|
|
|
|
----
|
|
`>>>` Advanced Python Mastery
|
|
`...` A course by [dabeaz](https://www.dabeaz.com)
|
|
`...` Copyright 2007-2023
|
|
|
|
. This work is licensed under a [Creative Commons Attribution-ShareAlike 4.0 International License](http://creativecommons.org/licenses/by-sa/4.0/)
|