A Gentle Introduction to Context Managers

Aly Sivji

@CaiusSivjus

About Me

• Aly Sivji
• ​
• Senior Analyst @ IBM Watson Health
• ​
• Grad Student @ Northwestern University
• ​
• Interests:
• Technology 🐍 | Data 📈 | Star Trek 🖖

We’ve all seen this before....

Source: Python 3 Tutorial

with Statement

• Control flow structure that encapsulates try...except...finally blocks for convenient reuse
• ​
• Results in cleaner and more readable code

Source: Python Docs

Context Manager

• Class containing methods that are executed:
1. ​
1. Before the with statement is entered __enter__
1. ​
2. After the with statement body exits __exit__

Note on __magic__ methods

• __enter__ should return an object that is assigned to the variable after as
• ​
• __exit__ is called on the original object
• ​
• If exception in __init__ or __enter__
• with block is not executed

Source: Python in the real world: Context Managers

Trace execution of with statement

class Foo:� def __enter__(self):� print('__enter__ called')� return self�� def __exit__(self, exc_type, exc_value, exc_tb):� print('__exit__ called')

>>> with Foo() as obj:�... print('inside with')

...

__enter__ called

inside with

__exit__ called

Code

REPL

Exceptions

• Once with block is entered
• __exit__ is always invoked
• Even if an Exception occurs inside the block
• ​
• Handling Exceptions
• Handle in __exit__ block and return True

Source: Python in the real world: Context Managers

Exception inside with statement

class Foo:� def __enter__(self):� print('__enter__ called')� return self�� def __exit__(self, exc_type, exc_value, exc_tb):� print('__exit__ called')

>>> with Foo() as obj:�... print('inside with')

... raise Exception('error')

...

__enter__ called

inside with

__exit__ called

----------------------------------�Exception Traceback (most recent call last)

Code

REPL

Handling exception inside with statement

class Foo2:� def __init__(self):� print('__init__ called')

​

def __enter__(self):� print('__enter__ called')� return self

def __exit__(self, exc_type, exc_value, exc_tb):� print('__exit__ called')� if exc_type:� # code to handle� print('exception handled')� return True

Code 1

Code 2

Handling exception inside with statement (2)

>>> with Foo2() as obj:�... print('inside with')

... raise Exception('error')

...

__init__ called

__enter__ called

inside with

__exit__ called

exception handled

REPL

Output

When to use Context Managers: Open/Close Files

with open('my_file.txt') as f:� data = f.readlines()

​

When to use Context Managers: Testing

import pytest��def test_zero_division():� with pytest.raises(ZeroDivisionError):� 1 / 0

​

Source: pytest Docs

When to use Context Managers: Other Use Cases

• Managing database connections
• ​
• Synchronizing access to shared resources (link)
• ​
• Logging around certain actions (lithoxyl)
• ​
• And more...! (see Additional Resources)

contextlib Package

• Part of Python Standard Library
• ​
• Create context managers using generators and the @contextmanager decorator
• ​
• ContextDecorator makes it possible to use a context manager as a function decorator

Source: Python Docs

Creating a Context Manager: Project Background

• AWS Lambda script downloads and stores tweets in a MongoDB instance on MLab
• ​
• Use context manager to query database and analyze tweets by hashtag

Source: Project Code

Using try...finally block

def download_tweets_by_hashtag_nonpythonic(hashtag):� tweets = []� try:� client = MongoClient(mlab_uri)� db = client.get_default_database()� coll = db[collection]� tweets = coll.find({"entities.hashtags.text":f"{hashtag}"})� finally:� client.close()� return tweets

Source: Project Code

Creating a Context Manager Object

class MongoCollection(object):� def __init__(self, uri, collection):� self.client = MongoClient(uri)� self.db = self.client.get_default_database()� self.collection = self.db[collection]�� def __enter__(self):� return self.collection�� def __exit__(self, exc_type, exc_value, exc_tb):� self.client.close()

Source: Project Code

Using Context Manager in with Statement

def download_tweets_by_hashtag(hashtag):�� with MongoCollection(mlab_uri, collection) as coll:� tweets = coll.find({'entities.hashtags.text':f'{hashtag}'})�� return list(tweets)

Source: Project Code

Additional Resources

• My Blog Post on Context Managers
• ​
• Python docs (1, 2)
• ​
• Python in the real world: Context Managers
• ​
• PyCon 2012 talk on Context Managers

​

Thank You

Github: alysivji

Twitter: @CaiusSivjus

Blog: https://alysivji.github.io

Slides: http://bit.ly/context-manager-chipy

​

​