On this page
Description
Python 3.11 is a big release: a much faster interpreter, exception groups and except* for code that fails in several places at once, exception notes, tomllib for reading TOML, and a few typing and enum additions. It's also a release whose compatibility notes matter as much as its features - one small change to how enums format has broken real services on upgrade.
Here's that bug. A service upgrades from Python 3.10 to 3.11 and gets faster for free - CPU time on its hot paths drops by about a fifth. Then a downstream system starts rejecting its messages. The payload is built with an f-string from an enum:
class Status(str, Enum):
PAID = "paid"
body = f'{{"status": "{status}"}}' # 3.10: {"status": "paid"} 3.11: {"status": "Status.PAID"}
Python 3.11 changed how enums that mix in str (or int) format: format(), f-strings and str() now give Status.PAID, like plain enums, instead of the value (a test printed Color.RED for a (str, Enum) member). StrEnum, also new in 3.11, formats as its value.
We'll go from simple to more technical:
- Faster CPython: where the speed came from.
- Exception groups and
except*. - Exception notes with
add_note. tomllib,SelfandStrEnum- and the enum change above.
Faster CPython
The Faster CPython project's first release made Python 3.11 about 25% faster on average (10-60% across the standard benchmark suite) with no code changes:
- a specializing adaptive interpreter (PEP 659) rewrites hot bytecode into type-specialized versions (
LOAD_ATTRfor an instance attribute,BINARY_OPfor two ints) and falls back if the types change; - cheaper function calls and frames, and lazily created frame objects;
- zero-cost exceptions: a
tryblock costs nothing when no exception is raised.
It's still an interpreter - CPU-heavy loops are still far slower than Java or C - but it changed the advice from "rewrite hot paths in C" to "upgrade first". 3.12 to 3.14 continued with smaller gains and an experimental JIT.
Exception groups and except*
Concurrent code can fail in several ways at once - a TaskGroup whose two tasks both raise. Python 3.11 (PEP 654) added ExceptionGroup to carry them together, and except* to handle them by type:
try:
async with asyncio.TaskGroup() as tg:
...
except* ValueError as eg: # eg is an ExceptionGroup of just the ValueErrors
log.warning("bad input: %s", eg.exceptions)
except* KeyError as eg:
...
- Each
except*clause takes the members that match and receives them as a (sub)group; several clauses can run for one group - a group of twoValueErrors and aKeyErrorran both handlers in a test. - Unhandled members propagate as a new
ExceptionGroupwith the same message: handling onlyValueErrorletExceptionGroup('batch failed', [KeyError('sku')])escape. exceptandexcept*can't be mixed on onetry(SyntaxError: cannot have both 'except' and 'except*' on the same 'try'), and a plainexcept ValueErrordoesn't match a group containing one - which is why code around aTaskGroupneedsexcept*.
Exception notes (PEP 678): err.add_note("while parsing row 3 of orders.csv") attaches context to any exception and the traceback prints it after the message - a test's traceback ended with ValueError: invalid literal for int() ... and then the note. Better than wrapping exceptions just to add context.
tomllib, Self and StrEnum
tomllib(PEP 680) parses TOML - the format ofpyproject.toml- into dicts. It's read-only (there's nodump; use a third-party library to write), and files must be opened in binary mode: a text-mode file givesTypeError: File must be opened in binary mode, e.g. use `open('foo.toml', 'rb')`.tomllib.loads(text)parses a string.typing.Selfannotates methods that return their own class (def with_id(self, id) -> Self:), so subclasses get the right type.enum.StrEnummembers are strings that format as their value - the fix for the formatting change above. For a(str, Enum)you can't change, format.valueexplicitly. (Comparisons were never affected:Color.RED == "red"is stillTrue.)- Also in 3.11: fine-grained traceback locations (carets under the exact sub-expression that failed), and
asyncio.TaskGroupandasyncio.timeout(see the asyncio module).
Which tool for which job?
| You want to | Use |
|---|---|
Handle errors from a TaskGroup or a batch | except* SomeError as eg - eg.exceptions |
| Handle a single, ordinary exception | plain except |
| Add context to an exception without wrapping it | err.add_note("...") |
Read pyproject.toml or another TOML file | tomllib.load(open(path, "rb")) |
| Write TOML | a third-party library (tomli-w) |
| An enum whose members format as their value | StrEnum |
| A method that returns its own class | -> Self |
Common mistakes
| Mistake | What happens | Do this instead |
|---|---|---|
f-strings on (str, Enum) members after upgrading | Status.PAID instead of paid | StrEnum, or .value |
except ValueError around a TaskGroup | never matches the group | except* ValueError |
except and except* on one try | SyntaxError | one kind per try |
Expecting only one except* clause to run | other handlers run too | write each clause for its own members |
tomllib.load on a text-mode file | TypeError | open with "rb" |
| Wrapping an exception only to add context | a longer, noisier traceback | add_note |
Check yourself
Try to answer before opening each one - recalling beats re-reading.
1. A group holds two ValueErrors and a KeyError. The try has except* ValueError and except* KeyError. Which handlers run?
Show answer
Both: each except* clause takes the matching members of the group. The ValueError handler gets a group of two.
2. Why did f"{Status.PAID}" change from paid to Status.PAID after upgrading to 3.11?
Show answer
Python 3.11 changed format() and str() of mixed-in enums like class Status(str, Enum) to match plain enums. Use StrEnum, or format Status.PAID.value.
3. Why does tomllib.load(open("app.toml")) fail?
Show answer
tomllib requires a binary file: open("app.toml", "rb"). (Or read the text and use tomllib.loads.)
Key takeaways
- Upgrade to 3.11+ for speed alone - and read the "Changes in the Python API" notes first.
ExceptionGroup+except*: per-type handling, several clauses, leftovers re-raised.add_notefor context;tomllibreads TOML from binary files.(str, Enum)formatting changed in 3.11 - preferStrEnum.