Skip to content

perf/tests: use more appropriate data type for package filter policy packages and add tests#10907

Merged
radoering merged 1 commit into
python-poetry:mainfrom
dhimasardinata:dhimas/cache-package-filter-policy
May 17, 2026
Merged

perf/tests: use more appropriate data type for package filter policy packages and add tests#10907
radoering merged 1 commit into
python-poetry:mainfrom
dhimasardinata:dhimas/cache-package-filter-policy

Conversation

@dhimasardinata
Copy link
Copy Markdown
Contributor

@dhimasardinata dhimasardinata commented May 17, 2026

PackageFilterPolicy normalizes installer.no-binary, installer.only-binary, and related package-filter settings once, but allows() and has_exact_package() previously scanned the normalized sequence each time they were called.

Chooser policy checks run for every candidate link, so this stores the normalized package names as an immutable frozenset for repeated membership checks.

A small local benchmark over a 1,001-entry normalized package policy:

old=0.234734s
new=0.006853s
speedup=34.3x

Pull Request Check List

  • Added tests for changed code.
  • Updated documentation for changed code. N/A, no user-facing behavior change.

Local checks:

PYTHONPATH=src uv run --with pytest --with pytest-xdist --with responses --with pytest-mock --with deepdiff python -m pytest tests/config/test_config.py tests/installation/test_chooser.py -q
uv run --with ruff python -m ruff check src/poetry/config/config.py tests/config/test_config.py
uv run --with ruff python -m ruff format --check src/poetry/config/config.py tests/config/test_config.py
PYTHONPATH=src uv run --with poetry-core --with packaging --with mypy python -m mypy src/poetry/config/config.py
git diff --check

Copy link
Copy Markdown

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • Since packages and _package_set must stay in sync, consider making packages immutable (e.g., a tuple) or otherwise preventing external mutation, so bugs aren’t introduced by code that modifies packages without updating _package_set.
  • If the policy is never mutated after initialization, using a frozenset instead of set for _package_set could better express immutability intent and slightly reduce the risk of accidental modification.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- Since `packages` and `_package_set` must stay in sync, consider making `packages` immutable (e.g., a tuple) or otherwise preventing external mutation, so bugs aren’t introduced by code that modifies `packages` without updating `_package_set`.
- If the policy is never mutated after initialization, using a `frozenset` instead of `set` for `_package_set` could better express immutability intent and slightly reduce the risk of accidental modification.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@dhimasardinata dhimasardinata force-pushed the dhimas/cache-package-filter-policy branch from 4a9eca6 to a0a8f85 Compare May 17, 2026 11:07
@dhimasardinata
Copy link
Copy Markdown
Contributor Author

Addressed Sourcery feedback in a0a8f85 by storing normalized packages as an immutable tuple and using a frozenset for repeated membership checks.

Comment thread src/poetry/config/config.py Outdated
@dhimasardinata dhimasardinata force-pushed the dhimas/cache-package-filter-policy branch from a0a8f85 to 2b2a547 Compare May 17, 2026 12:53
@radoering
Copy link
Copy Markdown
Member

A small local benchmark over a 1,001-entry normalized package policy:

I doubt that real numbers become that high. I assume that 100 is quite much and most of the times it will be less than 10. Nevertheless, frozenset feels like the more appropriate data type.

@radoering radoering force-pushed the dhimas/cache-package-filter-policy branch from 2b2a547 to 8e09ed5 Compare May 17, 2026 17:37
@radoering radoering changed the title perf: cache package filter policy packages perf/tests: use more appropriate data type for package filter policy packages and add tests May 17, 2026
@radoering radoering enabled auto-merge (squash) May 17, 2026 17:40
@radoering radoering merged commit 7df479b into python-poetry:main May 17, 2026
54 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants