Skip to content

Speed up pytest collection by deferring heavy imports - #95

Closed
elkoled wants to merge 7 commits into
masterfrom
pr-37353
Closed

elkoled wants to merge 7 commits into
masterfrom
pr-37353

Conversation

@elkoled

@elkoled elkoled commented Feb 24, 2026

Copy link
Copy Markdown
Owner

Upstream PR: commaai/openpilot pull/37353


Closes 32611

Changes

1. Lazy imports in conftest.py

The root conftest.py imports three heavy modules at the top level:

from openpilot.common.prefix import OpenpilotPrefix
from openpilot.system.manager import manager
from openpilot.system.hardware import TICI, HARDWARE

These pull in pycapnp, numpy, hardware detection, and the full service manager — all at conftest import time, which fires during pytest --collect-only. None of these are needed until test execution time (they're only used inside fixtures).

Fix: Move all four imports into the functions that use them:

  • OpenpilotPrefix and manager → inside openpilot_function_fixture()
  • HARDWARE → inside tici_setup_fixture()
  • TICI → lazy-cached helper _is_tici() used in pytest_collection_modifyitems()

TICI still loads during collection_modifyitems, but is cached after the first call, and the other three imports are fully deferred to execution time.

2. Replace --ignore with norecursedirs in pyproject.toml

The current addopts has seven --ignore= flags for submodule directories. --ignore requires pytest to enter each directory, read its contents, then discard them. norecursedirs prevents entering them at all during filesystem traversal, which is faster for large submodule trees.

# Before
addopts = "--ignore=openpilot/ --ignore=opendbc/ --ignore=panda/ ... -Werror ..."

# After
addopts = "-Werror --strict-config --strict-markers --durations=20 --maxprocesses=8 -n auto --dist=loadgroup"
norecursedirs = ["openpilot", "opendbc", "opendbc_repo", "panda", "rednose_repo", "tinygrad_repo", "teleoprtc_repo", "msgq", "third_party", ".git", "build"]

Also added opendbc_repo, third_party, .git, and build to the exclusion list since these directories contain no test files.

Expected Impact

The primary collection bottleneck is the import chain: conftest.py → openpilot.system.hardware → pycapnp / numpy / hardware detection. Deferring these imports eliminates ~1-1.5s of module loading during collection. The norecursedirs change avoids crawling ~7 large submodule directories.

Benchmarking

To verify, run:

# Cold (drop filesystem cache first if possible)
time pytest --collect-only 2>/dev/null | tail -1

# Warm (second run, cached)
time pytest --collect-only 2>/dev/null | tail -1

sshane and others added 7 commits February 23, 2026 01:21
* disable forget for tethering

* nets

* put in wifiman

* batch

* revert

* draw
…37344)

* disable forget for tethering

* nets

* put in wifiman

* batch

* Revert "batch"

This reverts commit 9af20c1.

* clean up

* more

* more
* disable forget for tethering

* nets

* put in wifiman

* batch

* revert

* clean up
Defer heavy imports in conftest.py to speed up pytest collection

Move OpenpilotPrefix, manager, HARDWARE, and TICI imports from
module-level to first-use inside their respective fixtures/hooks.
This avoids loading the full openpilot hardware and manager modules
during test collection, which is the primary bottleneck for
pytest --collect-only performance.

Also replace --ignore flags in pyproject.toml with norecursedirs
to prevent pytest from entering submodule directories at all during
collection (--ignore requires traversing first, then discarding).
Signed-off-by: SolariSystems <solari@solari.systems>
@github-actions

Copy link
Copy Markdown

Process replay diff report

Replays driving segments through this PR and compares the behavior to master.
Please review any changes carefully to ensure they are expected.

✅ 0 changed, 66 passed, 0 errors

@github-actions

Copy link
Copy Markdown

This PR has had no activity for 24 days. It will be automatically closed in 7 days if there is no activity.

@github-actions github-actions Bot added the stale label Mar 25, 2026
@github-actions

github-actions Bot commented Apr 2, 2026

Copy link
Copy Markdown

This PR has been automatically closed due to inactivity. Feel free to re-open once activity resumes.

1 similar comment
@github-actions

github-actions Bot commented Apr 2, 2026

Copy link
Copy Markdown

This PR has been automatically closed due to inactivity. Feel free to re-open once activity resumes.

@github-actions github-actions Bot closed this Apr 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants