Iterators in Python

The Iterator Design Pattern

No. 21

The Iterator is one of the simplest and most frequently used of the design patterns. It allows you to move through a list or collection of data using a standard interface without having to know the details of the internal representations of that data. In addition, you can also define special iterators that perform some special processing and return only specified elements of the data collection.

The Iterator of choice in Python is the simple two method iterator:

def iter(): pass
def next(): pass

While not having a method to move to the top of a list may seem restrictive at first, it is not a serious problem in Python, because it is customary to obtain a new instance of the iterator each time you want to move through a list.

Iterators in Python

We have already seen iterators in Python all through this newsletter. The for loop always uses an iterator under the covers. Here, we go through a List.

# Iterate through an array
people = ["Fred", "Mary", "Sam"] 
for p in people:
     print (p)

which, of course, produces

Fred
Mary
Sam

You can also iterate through sets, tuples, dictionaries, and even files. These are called iterable containers and you can get an iterator from them, as the for keyword does above.

A Fibonacci iterator

But suppose you want to create a class you can iterate through that isn’t one of these built-in iterable types. To make an iterable class, it must have the methods:

__init__()
__iter__(), and
__next__()

Since these all are surrounded with double underscores, they are called “dunder methods,” and are called under the covers when you use iterators.

The __init__() is optional, but the __iter__() must return an iterator, almost always self. You terminate the iterator by raising the StopIteration exception. In this case, this happens once the return value exceed 1000. This is, of course, adjustable in the
__init­­__ method if you like.

In this example, we create a class to iterate through the Fibonacci series, where each element is the sum of the previous two.

class FiboIter():
    def __init__(self):
        self.current = 0    # initialize variables
        self.prev = 1
        self.secondLast = 0

    def __iter__(self):
        return self         # must return self

    # each iteration computes a new value
    def __next__(self):
        if self.current < 1000:   #but stops at 1000
            self.secondLast = self.prev  # copy n-1st to secondLast
            self.prev = self.current     # copy nth to prev

            # compute next x as sum of previous 2
            self.current = self.prev + self.secondLast
            return self.current
        else:
            raise StopIteration

To create and call the iterator, you create an instance and call it until it reaches past 1000. You can use either a for loop or a while loop as shown below.

fbi = FiboIter()    # create iterator
 # print out values until 1000 is exceeded
for val in fbi:
     print(val, end=" ")
print("\n")

Getting the iterator

You can get the actual iterator itself, using the iter() function and then use next() to get each successive value. This is useful if you can’t use for to run the iteration for you.

val = 0
fbi = FiboIter() 
fbit = iter(fbi)

while val < 1000:
     val = next(fbit)
     print(val, end=" ")

In both cases, the program prints out:

1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597

You can also use the iterator to pull out elements of an array to store elsewhere:

 # iterate to get elements and store them
 person = ["Fred", "Smith", "80901210"]
 pIter = iter(person)
 frname = next(pIter)
 lname = next(pIter)
 serial = next(pIter)
 print(frname, lname, serial)

You have been using iterators all the time in Python, and in this article we have looked under the hood, and showed how to create new ones. We also showed how to use the iter function to obtain the iterator used by standard Python collections.