Blog

The Python Data Model, End to End

Every operator, every built-in, every piece of syntax in Python is a question asked of your object. The data model is the list of questions.

Ten parts of this series taught individual methods. This one is the idea underneath them, and it is a single sentence:

Python’s syntax is a set of questions, and dunder methods are how your object answers.

len(x) is not a function that inspects x. It is Python asking x a question — x.__len__() — and reporting the answer. Same for a + b, for i in x, with f:, x[0], if x:, str(x), x(). None of them special-case built-in types. list answers the same questions your class can.

That is why a well-written Python class stops feeling bolted on. It is not imitating a built-in. It is answering the same questions.

Everything below runs on Python 3.12 and the output is pasted from a real session.

Watching Python ask

class Probe:
    def __len__(self):
        print('  Python asked __len__')
        return 3
print(len(Probe()), bool(Probe()))
  Python asked __len__
  Python asked __len__
3 True

Two questions were asked. len() asked directly. bool() asked because there was no __bool__ and it falls back to __len__ — a non-zero length means truthy.

That fallback chain is the second thing worth knowing about the data model: most questions have a fallback. Python asks the specific question, then a more general one, then applies a default.


Creating and destroying

__new__ allocates. __init__ fills in. You will write __init__ constantly and __new__ almost never — it matters for immutable types, where there is no “after allocation” in which to assign anything.

Skip __del__. It runs at an unpredictable time, can be skipped entirely at interpreter exit, and swallows exceptions. Cleanup belongs in a context manager, below.

Representation: three questions, not two

Part 3 covered __repr__ and __str__. There is a third, and it is the one that makes your object work with f-strings:

class Temp:
    def __init__(self, c): self.c = c
    def __str__(self): return f"{self.c}C"
    def __format__(self, spec):
        if spec == 'f': return f"{self.c * 9/5 + 32:.1f}F"
        if spec == 'k': return f"{self.c + 273.15:.2f}K"
        if not spec: return str(self)
        return format(self.c, spec)

t = Temp(100)
print(f"{t} | {t:f} | {t:k} | {t:>8.1f}")
100C | 212.0F | 373.15K |    100.0

f"{t:f}" passes the string "f" to __format__ and your object decides what it means. This is exactly how datetime makes %Y work — it is not a special case in f-strings, it is datetime.__format__ choosing to hand the spec to strftime.

The last branch delegates anything it does not recognise to the underlying number, which is the delegation pattern worth copying: interpret the specs you care about, pass the rest down.

Comparison, and the one you must not forget

from functools import total_ordering
@total_ordering
class Version:
    def __init__(self, major, minor): self.major, self.minor = major, minor
    def __repr__(self): return f"Version({self.major}, {self.minor})"
    def __eq__(self, o): return (self.major, self.minor) == (o.major, o.minor)
    def __lt__(self, o): return (self.major, self.minor) < (o.major, o.minor)

v1, v2 = Version(1, 2), Version(1, 10)
print(v1 < v2, v1 >= v2, v1 != v2)
print(sorted([Version(2,0), Version(1,10), Version(1,2)]))
True False True
[Version(1, 2), Version(1, 10), Version(2, 0)]

@total_ordering fills in <=, > and >= from __eq__ and __lt__. Note Version(1, 2) < Version(1, 10) is True — comparing tuples, not strings, which is why version sorting done on strings puts 1.10 before 1.2.

!= came free: Python derives it from __eq__ unless you override it. Do not.

And the rule from Part 7 that catches everyone: define __eq__ and you must define __hash__, or your object becomes unhashable.

Attribute access

This is the layer most people never touch, and it explains @property.

class Loud:
    def __getattr__(self, name):
        return f"(no attribute {name!r}, and this ran instead)"
l = Loud()
l.real = 1
print(l.real)
print(l.missing)
1
(no attribute 'missing', and this ran instead)

__getattr__ is the fallback — it runs only when normal lookup fails. That makes it cheap and safe for proxies and lazy loading.

__getattribute__ is different and runs on every access:

class Watch:
    def __init__(self): self.x = 1
    def __getattribute__(self, name):
        if not name.startswith('_'):
            print(f'  looked up {name!r}')
        return object.__getattribute__(self, name)
w = Watch()
_ = w.x
  looked up 'x'

Note it calls object.__getattribute__ rather than self.x, which would recurse forever. The same trap applies to __setattr__:

class Frozen:
    def __init__(self, x):
        object.__setattr__(self, 'x', x)      # bypass our own __setattr__
    def __setattr__(self, name, value):
        raise AttributeError(f"{type(self).__name__} is read-only")
f = Frozen(1)
print(f.x)
try:
    f.x = 2
except AttributeError as err:
    print(type(err).__name__ + ':', err)
1
AttributeError: Frozen is read-only

__init__ has to bypass its own __setattr__ to set anything at all. This is roughly how @dataclass(frozen=True) works.

Descriptors: what @property actually is

A descriptor is an object that defines __get__ or __set__, and it controls what happens when it is accessed as a class attribute. @property is one. So are methods, classmethod and staticmethod.

Writing one directly pays off when the same validation repeats across fields:

class Positive:
    def __set_name__(self, owner, name):
        self.name = '_' + name
    def __get__(self, obj, objtype=None):
        if obj is None: return self
        return getattr(obj, self.name)
    def __set__(self, obj, value):
        if value <= 0:
            raise ValueError(f"{self.name.lstrip('_')} must be positive, got {value}")
        setattr(obj, self.name, value)

class Product:
    price = Positive()
    weight = Positive()
    def __init__(self, price, weight):
        self.price, self.weight = price, weight

p = Product(10, 2)
print(p.price, p.weight)
try:
    Product(-1, 2)
except ValueError as err:
    print(type(err).__name__ + ':', err)
10 2
ValueError: price must be positive, got -1

Two fields, one validator, and the error names the right field. __set_name__ is how the descriptor learns what it was called — Python passes it the attribute name at class creation.

As two @property pairs this is twelve lines of near-identical code. At five fields it is thirty.

Iteration

class Countdown:
    def __init__(self, n): self.n = n
    def __iter__(self):
        current = self.n
        while current > 0:
            yield current
            current -= 1

print(list(Countdown(4)))
a, b, *rest = Countdown(5)
print(a, b, rest)
[4, 3, 2, 1]
5 4 [3, 2, 1]

__iter__ as a generator is the shortest correct way to make something iterable — no separate iterator class, no __next__, no StopIteration to raise by hand.

Because it builds a fresh generator each call, the object can be iterated more than once. Return self from __iter__ and it exhausts after one pass, which is a common and confusing bug.

Unpacking works for free. So do list(), sum(), max(), in, and every comprehension.

Operators, and the reflected pair

class Metres:
    def __init__(self, v): self.v = v
    def __repr__(self): return f"Metres({self.v})"
    def __add__(self, other):
        if isinstance(other, Metres): return Metres(self.v + other.v)
        return NotImplemented
    def __radd__(self, other):
        if other == 0: return self         # makes sum() work
        return NotImplemented

print(Metres(1) + Metres(2))
print(sum([Metres(1), Metres(2), Metres(3)]))
Metres(3)
Metres(6)

For a + b, Python asks a.__add__(b) first. If that returns NotImplemented, it asks b.__radd__(a). That is how 1 + your_object can work when the integer has no idea what your type is.

The __radd__ here exists purely so sum() works — sum starts from 0 and adds, so the first operation is 0 + Metres(1), which only your __radd__ can answer.

There are also in-place forms (__iadd__ for +=). Skip them unless mutation in place is genuinely faster; without one, += falls back to __add__ and rebinds, which is usually what you want.

Context managers

class Timer:
    def __enter__(self):
        self.events = ['enter']
        return self
    def __exit__(self, exc_type, exc, tb):
        self.events.append('exit' if exc_type is None else f'exit({exc_type.__name__})')
        return False

with Timer() as t2:
    t2.events.append('body')
print(t2.events)

t3 = Timer()
try:
    with t3:
        raise ValueError('boom')
except ValueError:
    pass
print(t3.events)
['enter', 'body', 'exit']
['enter', 'exit(ValueError)']

__exit__ runs whatever happened, and is told which exception is in flight. Returning False lets it continue; returning True swallows it, which you should do rarely and deliberately.

This is where cleanup belongs — not __del__.

Class creation

class Plugin:
    registry = {}
    def __init_subclass__(cls, /, name=None, **kw):
        super().__init_subclass__(**kw)
        Plugin.registry[name or cls.__name__.lower()] = cls

class Csv(Plugin, name='csv'): pass
class Json(Plugin): pass
print(Plugin.registry)
{'csv': <class '__main__.Csv'>, 'json': <class '__main__.Json'>}

__init_subclass__ runs when someone subclasses you, and accepts keyword arguments given in the class definition. Self-registering plugins with no decorator and no metaclass.

This is the hook that removed most legitimate uses of metaclasses. If you were about to write one, check whether __init_subclass__ and __set_name__ cover it. They usually do.

Pattern matching asks a question too

match is the newest part of the data model, and your classes can answer it — but only if you tell Python what the positional slots mean.

class Point:
    __match_args__ = ('x', 'y')
    def __init__(self, x, y): self.x, self.y = x, y

def describe(p):
    match p:
        case Point(0, 0):          return 'origin'
        case Point(0, y):          return f'on the y axis at {y}'
        case Point(x, 0):          return f'on the x axis at {x}'
        case Point(x, y) if x == y: return f'on the diagonal at {x}'
        case Point(x, y):          return f'at {x},{y}'

for p in [Point(0,0), Point(0,5), Point(3,0), Point(4,4), Point(1,2)]:
    print(' ', describe(p))
  origin
  on the y axis at 5
  on the x axis at 3
  on the diagonal at 4
  at 1,2

__match_args__ is a tuple naming which attributes the positional patterns map to. Without it, positional matching is an error rather than a silent miss:

class NoMatch:
    def __init__(self, x): self.x = x
try:
    match NoMatch(1):
        case NoMatch(1): pass
except TypeError as err:
    print(type(err).__name__ + ':', err)
TypeError: NoMatch() accepts 0 positional sub-patterns (1 given)

Keyword patterns — case NoMatch(x=1) — work without it, because they name the attribute directly. And dataclasses supply it for free, in field order:

@dataclass
class DC:
    x: int
    y: int
print('dataclass gets it free:', DC.__match_args__)
match DC(1, 2):
    case DC(x=1, y=v): print(f'  matched with y={v}')
dataclass gets it free: ('x', 'y')
  matched with y=2

One caution: __match_args__ fixes an order, and reordering it later silently changes what every positional pattern means. Keyword patterns do not have that failure mode, which is a good reason to prefer them past two fields.

The async protocol is the same questions, awaited

Every protocol above has an async twin. with becomes async with and asks __aenter__ / __aexit__; for becomes async for and asks __aiter__ / __anext__.

import asyncio

class Fetcher:
    async def __aenter__(self):
        self.events = ['open']
        return self
    async def __aexit__(self, *exc):
        self.events.append('close')
        return False

class Ticker:
    def __init__(self, n): self.n = n
    def __aiter__(self):
        self.i = 0
        return self
    async def __anext__(self):
        if self.i >= self.n:
            raise StopAsyncIteration
        self.i += 1
        await asyncio.sleep(0)
        return self.i

async def main():
    async with Fetcher() as f:
        f.events.append('body')
    print(' ', f.events)
    print(' ', [x async for x in Ticker(3)])

asyncio.run(main())
  ['open', 'body', 'close']
  [1, 2, 3]

Two details differ from the synchronous versions. __aiter__ is not a coroutine — it returns the async iterator directly, and only __anext__ is awaited. And the sentinel is StopAsyncIteration, not StopIteration; raising the wrong one produces a confusing RuntimeError rather than a clean stop.

Everything else is the shape you already know.


The questions, collected

You write Python asks Falls back to
repr(x) __repr__ the default <Class object at 0x…>
str(x), print(x) __str__ __repr__
f"{x:spec}" __format__ __str__ for an empty spec
x == y __eq__ identity (is)
x != y __ne__ not __eq__
x < y __lt__ TypeError
hash(x) __hash__ id-based, unless __eq__ is defined
len(x) __len__ TypeError
if x: __bool__ __len__, then always true
x[k] __getitem__ TypeError
for i in x __iter__ __getitem__ from index 0
k in x __contains__ __iter__, then __getitem__
a + b __add__ b.__radd__(a)
x() __call__ TypeError
with x: __enter__ / __exit__ TypeError
x.missing __getattribute__ __getattr__, then AttributeError
case C(a, b) __match_args__ TypeError for positional patterns
async with x: __aenter__ / __aexit__ TypeError
async for i in x __aiter__ / __anext__ TypeError

What not to implement

The data model is large and most of it is not for you.

__del__ — unpredictable timing, skipped at exit, swallows exceptions. Use a context manager.

__getattribute__ — runs on every access, easy to make infinitely recursive, and slows the class down. __getattr__ covers nearly every real case.

Metaclasses__init_subclass__ and __set_name__ cover the common reasons.

__slots__ by default — Part 10. It has two good reasons and neither is “it seems tidier”.

And the general rule: each dunder is a promise about how your object behaves. A promise nobody needed is just more to keep true. Implement __repr__ always, __eq__ and __hash__ for value types, and add the rest when a caller actually wants to write that syntax.

The five things

  • Python’s syntax is questions; dunder methods are answers. There is no special-casing of built-in types.

  • Most questions have a fallback chain — bool to len, str to repr, iter to getitem, add to radd.

  • @property is a descriptor. Write a descriptor directly when the same validation repeats across fields.

  • Return NotImplemented from operators you cannot handle, so the other operand gets its turn.

  • Implementing a dunder is a promise. Make only the ones a caller will use.

The rest of the data model is in the language reference, and now it will read as a list of questions rather than a list of magic.

How useful was this post?

Click on a heart to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.