Iterators & Generators

Shayna Kothari

Iterators

• iterator = iter(____)
• Iterators are pointers to values in iterables (iterables include lists, dictionaries, strings, etc.).
• When you call next on an iterator, the pointer moves to the next value until it reaches the end of the iterator.
• For and while loops call next on iterators repeatedly until calling next raises a StopIteration error. When you say for val in lst, for example, the loops create an iterator from the list and do this.
• This is why Python will error if you try to delete values from an iterable if that’s part of the loop’s condition!

Iterators Example

>>> lst = [1, 2]�>>> p, q = iter(lst), iter(lst)�>>> next(p)�1�>>> next(p)�2�>>> next(q)�1�>>> lst.append(3)�>>> next(p)�3�>>> next(p)�StopIterationError

q

p

Generators

• Just another kind of iterator!
• Instead of being attached to a specific iterable, to get the next value, they run a generator function.
• Generator functions are not generators -- they return generators!
• When Python sees a function with yield in it, it realizes it needs to return a generator.
• When you call next on a generator, it runs until it hits yield. It “pauses” at the place in the code where it is after it runs yield until you call next again!
• The generator will finish if you never provide it another yield.

Generators: Bookmarking

>>> def gen(x):

... print('hi!')

... i = 0

... while i < x:

... print('iteration', i)

... yield i

... i += 1

...

>>> p = gen(4)

>>> next(p)

hi!

iteration 0

0

>>> next(p)

iteration 1

1

​

Generators

def gen(n):

i = 0

while i < n:

yield i

i += 1

• Getting values from a list or another generator: use yield from or a for loop with a generator:
• for value in gen(n):� yield value
• yield from gen(n)

Recursive Generators

• Don’t yield another generator! yield is not the same as return!
• Instead, call your function to get a generator, and use that generator to get new values that you can use! Most helpful with recursive data structures where it could be useful to call the generator function on the rest.
• yield from is like calling yield in a for loop with nothing else inside

for i in some_iterator:

yield i

​

yield from some_iterator

​

=

Problem Solving Tip: When dealing with generators recursively, pretend you’re getting a list of values back from the generator function instead of a generator! (However, remember not to yield that list!)

​

Recursive Generators

def increasing_lists(lst):

'''

Generate all sublists of lst that are in increasing order.

'''

if len(lst) != 0:

yield [lst[0]]

if len(lst) > 1:

for next_lst in increasing_lists(lst[1:]):

if next_lst[0] > lst[0]:

yield [[lst[0]] + next_lst

yield next_lst

Note: we don’t use yield from here because we have an if statement inside of our loop!