System Development with Python

Week 3 :: iterators and generators

Exceptions aren't just for errors

Exception handling can be used for control flow as well

i.e. StopIteration for iterators

Iterators

Iterators are objects which support a concept of iteration over a collection

# looping over the lines in a file is done via an iterator:
  with open("file.dat") as f:
      for line in f:
          print line

  # and you can create your own
  for x in foo():
      print x

An iterable is an object which follows the Python iterator protocol

An iterator defines two required methods in order to iterate

• __iter__() returns the iterator itself
• next() returns the next item in the sequence

http://docs.python.org/2/library/stdtypes.html#iterator-types

Demonstration iterator

class CountToTen(object):
      """an iterator which returns integers from 0 to 9, inclusive"""

      def __init__(self):
          self.data = range(10)

      def __iter__(self):
          return self

      def next(self):
          try:
              return self.data.pop(0)
          except IndexError:
              raise StopIteration

  for x in CountToTen():
      print x

  # or consume the whole thing at once by converting to a list:
  list(CountToTen())
  

Now let's build an iterator

Look at the stubbed code in /examples/week-03-debugging/fib_exercise.py

Modify the run() function so it is a real iterable

The Fibonnaci sequence is defined as such:

• The first two integers in the sequence are 0 and 1
• Each member of the sequence is the sum of the previous two elements

for x in FibonacciIterator(20):
  print x
      

Where do we use iterators and why?

Iterating over resource-intensive types

Examples in the wild:

• Big Shapes

• DB Stuff

Exercise

Look at the code in /examples/week-3/iterators/yrange.py

• The yrange function is cheating by proxying it's calls to xrange
• Remove all calls to xrange
• Rewrite the yrange' next() method to be it's own range iterable

Exercise

Write a Take iterable that does the following:

• It takes a number ( num ) and a list or tuple ( a_list ) as args
• It iterates over the a_list and returns list of length num or less from it's current position
• Words are bad. Here's what example output for one implementation might look like:

  
  $ Take( 3, [0,1,2,3] )
  [0,1,2]
  [1,2,3]
  [2,3]
  [3]
  

generators

A generator is a concrete type that implements the iterator protocol.

Convert a function to a generator using the yield keyword

def count_to_10():
    for i in range(10):
        yield i

for x in count_to_10():
    print x
      

(4700 upvotes on this stackoverflow question, yield is confusing at first)

Using a generator expression to create a generator

Python list comprehensions allow you to build lists of values

my_list = [x for x in open('file.dat')]

Convert that list comprehension to a generator just by replacing '[]' with '()'

my_generator = (x for x in open('file.dat'))

Questions?