Fix out-of-tree PyPI dependency resolution correctness bugs

[henryiii]

🤖 AI text below 🤖

Part of #376.

This fixes four verified correctness bugs in the out-of-tree PyPI dependency resolution (pyodide_build/out_of_tree/pypi.py and pyodide_build/cli/build.py).

Bug 1: isolation / skip_dependency_check silently dropped for dependency builds

_resolve_and_build called download_or_build_wheel(x.url, target_folder, compression_level), where the third positional argument landed in compression_level and the isolation / skip_dependency_check flags were never forwarded. The same happened during metadata resolution, where get_built_wheel(url) was called with no flags. As a result pyodide build -r reqs.txt --no-isolation -x still built dependency sdists with default isolation.

Fix: forward the flags using keyword arguments throughout the call chain. Because metadata resolution flows through Candidate -> get_metadata_for_wheel -> get_built_wheel (which has no direct access to the flags), the flags are also stored on new PyPIProvider.BUILD_ISOLATION / BUILD_SKIP_DEPENDENCY_CHECK class attributes (mirroring the existing BUILD_FLAGS / BUILD_EXPORTS pattern) and honored there.

Bug 2: PyPIProvider.find_matches only honored the last extras set

The loop over extra_requirements rebound the candidates generator each iteration, so only the last extras set was consumed. When a package was required both with and without extras (foo and foo[bar]), dependencies pulled in by the lost extras were never resolved.

Fix: union the extras across all requirements and create the candidates once. Candidate._get_dependencies evaluates each extra == "..." marker against every extra in the set, so a single candidate carrying the union is the correct choice.

Related: marker evaluation in _resolve_and_build used {"extra": ",".join(extras)}, which can never match a marker like extra == "a" when there are multiple extras. Now the marker is evaluated once per requested extra (plus the empty extra) and the requirement is included if it matches any of them.

Bug 3: UnboundLocalError for non-gz, non-whl sdist URLs

download_or_build_wheel only handled URLs ending in gz or .whl. For a .zip or .tar.bz2 sdist (still present on PyPI for older packages), wheel_path was never assigned and the later repack_zip_archive(wheel_path, ...) raised UnboundLocalError.

Fix: route any supported sdist archive format (.tar.gz, .tgz, .tar.bz2, .tbz2, .tar.xz, .txz, .tar, .zip) to the builder (the build path is format-agnostic via shutil.unpack_archive), and raise a clear error for unsupported types. The temp file used during the build now preserves the archive suffix so the format is inferred correctly. get_metadata_for_wheel was made consistent.

Bug 4: _extract_extras dropped version specifiers and mishandled multiple extras

For "pkg[extra]==1.0" the hand-rolled regex discarded ==1.0 (so the latest version was fetched), and for "pkg[a,b]" the regex \[(\w+)\] did not match commas, so no extras were extracted.

Fix: parse with packaging.requirements.Requirement so name, extras (including comma-separated) and the version specifier are all preserved. The specifier is folded back into the source-location string that flows to fetch_pypi_package. Non-requirement source locations (URLs, paths) are returned unchanged.

Tests

Added unit tests in pyodide_build/tests/test_pypi.py:

• flags propagate from _resolve_and_build into download_or_build_wheel (monkeypatched) and onto the PyPIProvider class attributes;
• find_matches with foo + foo[bar] honors the extras union; a candidate with multiple extras resolves dependencies whose markers match any single extra;
• a non-gz/non-whl URL raises a clear error (not UnboundLocalError), and a .zip sdist is routed to the builder with flags forwarded;
• _extract_extras("pkg[a,b]==1.0") yields name pkg==1.0 and extras ["a", "b"], with specifiers preserved and URLs passed through.

🤖 Generated with Claude Code

[ryanking13]

We are actually planning to remove this feature soon in #362. So let me close this. Thanks anyways!