Python Context Managers and the with Statement

What with guarantees, how to write __enter__ and __exit__, the contextlib shortcut, and how to swallow an exception on purpose.

with exists so that cleanup happens whatever the body does. Return early, raise, or finish normally, and the exit code still runs.

Any object with __enter__ and __exit__ can be used this way, so you can put your own resources behind the same statement.

What the two methods do

__enter__ runs first and its return value is what as binds. __exit__ runs at the end and is told which exception, if any, is in flight.

# what with actually guarantees
import io

class Tracked:
    def __init__(self):
        self.events = []
    def __enter__(self):
        self.events.append('enter')
        return self
    def __exit__(self, exc_type, exc, tb):
        self.events.append(f'exit ({exc_type.__name__ if exc_type else "no error"})')
        return False

t = Tracked()
with t as handle:
    handle.events.append('body')
print(t.events)

It prints:

['enter', 'body', 'exit (no error)']

Cleanup runs on the way out of an error

The exception carries on after __exit__ has run.

# cleanup runs even when the body raises
t2 = Tracked()
try:
    with t2:
        t2.events.append('body')
        raise ValueError('boom')
except ValueError as err:
    t2.events.append(f'caught {err}')
print(t2.events)

It prints:

['enter', 'body', 'exit (ValueError)', 'caught boom']

This is the whole point. A try/finally would do the same, but with puts the cleanup next to the resource instead of at every call site.

Returning True swallows the exception

A true return value from __exit__ means “I handled it”.

# returning True from __exit__ swallows the exception
class Swallow:
    def __enter__(self):
        return self
    def __exit__(self, exc_type, exc, tb):
        return True

with Swallow():
    raise RuntimeError('this never escapes')
print('still running')

It prints:

still running

Use this rarely and deliberately. Silently swallowing every exception is how errors go missing.

The contextlib shortcut

A generator with a single yield becomes a context manager. Everything before the yield is setup, everything after is cleanup.

# contextlib turns a generator into a context manager
import contextlib

@contextlib.contextmanager
def tag(name, out):
    out.append(f'<{name}>')
    try:
        yield out
    finally:
        out.append(f'</{name}>')

lines = []
with tag('p', lines):
    lines.append('hello')
print(''.join(lines))

It prints:

<p>hello</p>

The try/finally around the yield is what makes cleanup run when the body raises. Without it, an exception skips the rest of the generator.

suppress instead of try/except/pass

contextlib.suppress says what it means and takes one line.

# contextlib.suppress instead of try/except/pass
with contextlib.suppress(ZeroDivisionError):
    1 / 0
print('suppressed')

It prints:

suppressed

Several managers at once

Commas nest them left to right, and they close in reverse order.

# several managers on one line
buf1, buf2 = io.StringIO(), io.StringIO()
with buf1 as a, buf2 as b:
    a.write('one')
    b.write('two')
    print(a.getvalue(), b.getvalue())     # read before the block closes them
print('both closed:', buf1.closed, buf2.closed)

It prints:

one two
both closed: True True

Note the second line. Once the block ends, the files are closed, so read what you need inside.

What to remember

• __enter__ returns what as binds, __exit__ always runs.
• A true return from __exit__ swallows the exception.
• @contextlib.contextmanager plus try/finally is the short way to write one.
• Managers on one with line close in reverse order.

Locks, database transactions, temporary directories, timers and mocked settings are all context managers in the standard library. If your code has a setup and a matching teardown, it wants to be one too.