226 lines
5.4 KiB
Markdown
226 lines
5.4 KiB
Markdown
\[ [Index](index.md) | [Exercise 8.4](ex8_4.md) | [Exercise 8.6](ex8_6.md) \]
|
|
|
|
# Exercise 8.5
|
|
|
|
*Objectives:*
|
|
|
|
- Learn about managed generators
|
|
|
|
*Files Created:* `multitask.py`, `server.py`
|
|
|
|
A generator or coroutine function can never execute without being
|
|
driven by some other code. For example, a generator used for
|
|
iteration doesn't do anything unless iteration is actually carried out
|
|
using a for-loop. Similarly, a collection of coroutines won't run
|
|
unless their `send()` method is invoked somehow.
|
|
|
|
In advanced applications of generators, it is possible to drive
|
|
generators in various unusual ways. In this exercise, we look at a
|
|
few examples.
|
|
|
|
## (a) Generators as tasks
|
|
|
|
If a file `multitask.py`, define the following code:
|
|
|
|
```python
|
|
# multitask.py
|
|
|
|
from collections import deque
|
|
|
|
tasks = deque()
|
|
def run():
|
|
while tasks:
|
|
task = tasks.popleft()
|
|
try:
|
|
task.send(None)
|
|
tasks.append(task)
|
|
except StopIteration:
|
|
print('Task done')
|
|
```
|
|
|
|
This code implements a tiny task scheduler that runs generator functions.
|
|
Try it by running it on the following functions.
|
|
|
|
```python
|
|
# multitask.py
|
|
...
|
|
|
|
def countdown(n):
|
|
while n > 0:
|
|
print('T-minus', n)
|
|
yield
|
|
n -= 1
|
|
|
|
def countup(n):
|
|
x = 0
|
|
while x < n:
|
|
print('Up we go', x)
|
|
yield
|
|
x += 1
|
|
|
|
if __name__ == '__main__':
|
|
tasks.append(countdown(10))
|
|
tasks.append(countdown(5))
|
|
tasks.append(countup(20))
|
|
run()
|
|
```
|
|
|
|
When you run this, you should see output from all of the generators
|
|
interleaved together. For example:
|
|
|
|
```python
|
|
T-minus 10
|
|
T-minus 5
|
|
Up we go 0
|
|
T-minus 9
|
|
T-minus 4
|
|
Up we go 1
|
|
T-minus 8
|
|
T-minus 3
|
|
Up we go 2
|
|
T-minus 7
|
|
T-minus 2
|
|
Up we go 3
|
|
T-minus 6
|
|
T-minus 1
|
|
Up we go 4
|
|
T-minus 5
|
|
Task done
|
|
Up we go 5
|
|
T-minus 4
|
|
Up we go 6
|
|
T-minus 3
|
|
Up we go 7
|
|
T-minus 2
|
|
Up we go 8
|
|
T-minus 1
|
|
Up we go 9
|
|
Task done
|
|
Up we go 10
|
|
Up we go 11
|
|
Up we go 12
|
|
Up we go 13
|
|
Up we go 14
|
|
Up we go 15
|
|
Up we go 16
|
|
Up we go 17
|
|
Up we go 18
|
|
Up we go 19
|
|
Task done
|
|
```
|
|
|
|
That's interesting, but not especially compelling. Move on to the next example.
|
|
|
|
## (b) Generators as Tasks Serving Network Connections
|
|
|
|
Create a file `server.py` and put the following code into it:
|
|
|
|
```python
|
|
# server.py
|
|
|
|
from socket import *
|
|
from select import select
|
|
from collections import deque
|
|
|
|
tasks = deque()
|
|
recv_wait = {} # sock -> task
|
|
send_wait = {} # sock -> task
|
|
|
|
def run():
|
|
while any([tasks, recv_wait, send_wait]):
|
|
while not tasks:
|
|
can_recv, can_send, _ = select(recv_wait, send_wait, [])
|
|
for s in can_recv:
|
|
tasks.append(recv_wait.pop(s))
|
|
for s in can_send:
|
|
tasks.append(send_wait.pop(s))
|
|
task = tasks.popleft()
|
|
try:
|
|
reason, resource = task.send(None)
|
|
if reason == 'recv':
|
|
recv_wait[resource] = task
|
|
elif reason == 'send':
|
|
send_wait[resource] = task
|
|
else:
|
|
raise RuntimeError('Unknown reason %r' % reason)
|
|
except StopIteration:
|
|
print('Task done')
|
|
```
|
|
|
|
This code is a slightly more complicated version of the task scheduler in
|
|
part (a). It will require a bit of study, but the idea is that not only
|
|
will each task yield, it will indicate a reason for doing so (receiving or
|
|
sending). Depending on the reason, the task will move over to a waiting
|
|
area. The scheduler then runs any available tasks or waits for I/O
|
|
events to occur when nothing is left to do.
|
|
|
|
It's all a bit tricky perhaps, but add the following code which implements
|
|
a simple echo server:
|
|
|
|
```python
|
|
# server.py
|
|
...
|
|
|
|
def tcp_server(address, handler):
|
|
sock = socket(AF_INET, SOCK_STREAM)
|
|
sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
|
|
sock.bind(address)
|
|
sock.listen(5)
|
|
while True:
|
|
yield 'recv', sock
|
|
client, addr = sock.accept()
|
|
tasks.append(handler(client, addr))
|
|
|
|
def echo_handler(client, address):
|
|
print('Connection from', address)
|
|
while True:
|
|
yield 'recv', client
|
|
data = client.recv(1000)
|
|
if not data:
|
|
break
|
|
yield 'send', client
|
|
client.send(b'GOT:' + data)
|
|
print('Connection closed')
|
|
|
|
if __name__ == '__main__':
|
|
tasks.append(tcp_server(('',25000), echo_handler))
|
|
run()
|
|
```
|
|
|
|
Run this server in its own terminal window. In another terminal, connect to it using a command such as `telnet` or `nc`. For example:
|
|
|
|
```
|
|
bash % nc localhost 25000
|
|
Hello
|
|
Got: Hello
|
|
World
|
|
Got: World
|
|
```
|
|
|
|
If you don't have access to `nc` or `telnet` you can also use Python itself:
|
|
|
|
```
|
|
bash % python3 -m telnetlib localhost 25000
|
|
Hello
|
|
Got: Hello
|
|
World
|
|
Got: World
|
|
```
|
|
|
|
If it's working, you should see output being echoed back to you. Not only that,
|
|
if you connect multiple clients, they'll all operate concurrently.
|
|
|
|
This tricky use of generators is not something that you would
|
|
likely have to code directly. However, they are used in certain advanced
|
|
packages such as `asyncio` that was added to the standard
|
|
library in Python 3.4.
|
|
|
|
\[ [Solution](soln8_5.md) | [Index](index.md) | [Exercise 8.4](ex8_4.md) | [Exercise 8.6](ex8_6.md) \]
|
|
|
|
----
|
|
`>>>` Advanced Python Mastery
|
|
`...` A course by [dabeaz](https://www.dabeaz.com)
|
|
`...` Copyright 2007-2023
|
|
|
|
. This work is licensed under a [Creative Commons Attribution-ShareAlike 4.0 International License](http://creativecommons.org/licenses/by-sa/4.0/)
|