, 6 min read, python, performance, startup
PEP 810's lazy keyword took my toy CLI's --version from 39 ms to bare interpreter startup. The catch: a broken import now fails where the name is first used.
The thing I wanted to try first in Python 3.15 (released October 9) was PEP 810: a lazy soft keyword for imports. Write lazy import json and the name json gets bound to a placeholder right away, but the module doesn't actually load until your code uses it. There is also a process-wide switch, -X lazy_imports=all, that makes ordinary imports lazy without touching the source at all.
That switch is tempting. You could put it in a shell alias today and feel clever. So I built a toy CLI, timed it, broke it on purpose, and ran it under pytest to see where the win is real and where it quietly costs you something.
Short version: for a CLI it's a big, cheap win. The price is that a broken import stops failing on line one and starts failing wherever the name is first used, which might be a code path nobody hits until much later.
Nothing clever. It imports asyncio, email, http.client, decimal, json and a local heavy_module (which itself imports xml.etree.ElementTree, csv and hashlib, so there's something to load). There's a --version flag that prints and exits, and a normal path that uses everything.
import sys
lazy import asyncio
lazy import email
lazy import http.client
lazy import decimal
lazy import json
lazy import heavy_module
def main() -> int:
if "--version" in sys.argv:
print("toycli 1.0.0")
return 0
data = json.dumps({"ok": True})
d = decimal.Decimal("1.5") + decimal.Decimal("2.5")
print(f"ran: {data} {d} {asyncio.__name__} {email.__name__} {http.client.__name__} {heavy_module.compute()}")
return 0
if __name__ == "__main__":
raise SystemExit(main())The eager version is the same file with the lazy words deleted. A third copy also has no lazy keywords and gets run with -X lazy_imports=all.
One thing I had wrong going in: I assumed the placeholder would survive being passed around, and only something like an attribute access would trigger the real import. Nope. The PEP says accessing a lazy object from a global variable or a module attribute reifies it, and a quick test agreed: a bare x = json at module level was enough to put json into sys.modules. Peeking through globals() or dir() doesn't trigger it. Basically any normal use of the name does.
python -X importtime prints a line for every module that gets imported, so counting lines is a rough measure of how much work a run did. On CPython 3.15.0:
--version: 193 modules. It loads everything, then prints a version string.lazy keyword, --version: 25 modules. json, decimal, asyncio, heavy_module and friends never show up. For comparison, a bare python -c pass came out at 26.lazy keyword, full run: 193 modules, same as eager. Everything gets used, so everything loads, just later.-X lazy_imports=all, --version: 25 modules.-X lazy_imports=all, full run: 124 modules.That last one surprised me. The global flag doesn't only apply to my file, it applies to every module-level import in the process, standard library included. So heavy_module's own import xml.etree.ElementTree became lazy too, and since compute() never touches ElementTree, it never loaded. That's free work saved, and also a hint about what the flag is really doing: changing import behaviour in code you didn't write.
Wall time for a fresh subprocess, median of 100 launches per case, on my laptop:
python -c pass: 11.2 ms--version: 39.4 mslazy keyword, --version: 11.1 mslazy keyword, full run: 38.5 ms-X lazy_imports=all, --version: 10.8 ms-X lazy_imports=all, full run: 31.7 msThe --version path went from about 39 ms to basically the cost of starting the interpreter. That's the whole pitch of this feature in one number. The full run with the keyword is a wash, which makes sense: deferring work you're going to do anyway just moves it. The global flag's full run is a bit faster because of those 69 modules that never loaded.
Now the part I think matters more. I misspelled the heavy module as heavy_modulee. Eager version, even with --version:
Traceback (most recent call last):
File "cli_typo_eager.py", line 9, in <module>
import heavy_modulee # typo, deliberate
^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'heavy_modulee'. Did you mean: 'heavy_module'?Fails immediately, points at the import. Normal.
Lazy version with --version: prints toycli 1.0.0, exits 0. No error at all, because nothing touched the name. Lazy version on the real path, where main() calls a do_work() function that uses it:
starting up...
Traceback (most recent call last):
File "cli_typo_lazy.py", line 10, in <module>
lazy import heavy_modulee # typo, deliberate
ImportError: lazy import of 'heavy_modulee' raised an exception during resolution
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "cli_typo_lazy.py", line 28, in <module>
raise SystemExit(main())
~~~~^^
File "cli_typo_lazy.py", line 23, in main
do_work()
~~~~~~~^^
File "cli_typo_lazy.py", line 15, in do_work
print(f"ran: {data} {asyncio.__name__} {heavy_modulee.compute()}")
^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'heavy_modulee'. Did you mean: 'heavy_module'?To be fair to CPython, this is a good traceback. It chains back to the original import line, exactly like the docs promise, so you're not left guessing. The problem isn't the message, it's the timing. A typo or a missing dependency used to blow up the moment anything ran the file. Now it can sail through --version, --help, your smoke test and half your test suite, and show up the first time someone hits the one branch that uses it. The plain-import file under -X lazy_imports=all behaved exactly the same way, which is the scary version: you didn't even write lazy anywhere.
lazy isn't a general "defer this" keyword. Put it in a function and you get:
SyntaxError: lazy import not allowed inside functionsThe whatsnew page lists class bodies and try/except/finally blocks as off limits too, plus lazy from x import * and lazy from __future__ import .... So it doesn't replace the old trick of importing inside the function that needs it. It's for the top-of-file import you'd rather pay for later.
Small thing I tripped on: the PEP describes a third mode, none, that disables laziness everywhere, which sounded like a nice kill switch. On 3.15.0, -X lazy_imports=none dies with Fatal Python error: ... expected 'all' or 'normal'. The whatsnew page only lists those two, so trust that over the PEP.
CLIs
yes. Startup is paid on every invocation, and most invocations don't touch most of what you imported. This is the textbook case.
Your library modules
yes, for the genuinely heavy imports. That's what makes test collection faster, more on that below.
Long-running servers
no, or only deliberately scoped. A server pays its import cost once at boot. Saving 30 ms at startup in exchange for an import failure that shows up mid-request instead of in the deploy log is a bad trade.
The test collection part needs a caveat I only found by trying it. I set up two pytest files and ran pytest -k test_fast so only one test was selected. When the deselected test file itself had lazy import heavy_dep at the top, heavy_dep still got imported during collection. When the same lazy import lived in a library module that the test file imported normally, heavy_dep was never loaded.
The reason is in pytest's (9.1.1) fixture discovery: it runs dir() on each test module and does a getattr on every name to check whether it's a fixture. Module attribute access reifies a lazy import, so pytest ends up loading everything a test file imports at the top. So lazy in your test files buys you nothing under pytest, but lazy in the code under test does.
For the "on, but scoped" setups there are two knobs. sys.set_lazy_imports_filter takes a callback that gets the importing module's name, the imported module's name and the fromlist, and returns True to allow laziness or False to force an eager import. __lazy_modules__ is a module-level list of module names whose plain import statements become lazy, which is handy if you still support versions older than 3.15, where it's just an ignored variable. I checked both with the misspelling trick: with a filter that only allowed names starting with heavy_, a bad import heavy_modulee deferred its error while a bad import jsonnn failed on its own line. __lazy_modules__ deferred exactly the names it listed.
So -X lazy_imports=all is fine for a quick "how much would this save me" check. For code I'm going to maintain, I'd rather pick which imports get to be lazy and keep the rest failing loudly where I can see them.1
uv python install 3.15.0, on macOS, with pytest 9.1.1 for the collection test. The exact commands and raw output are in this post's sources file. ↩
Comments