Merge network-wrangler main fixes#5
Merged
lmz merged 8 commits intoBayAreaMetro:mainfrom Mar 27, 2026
Merged
Conversation
…429) * Fix pandera/pandas compatibility issue and dataclass mutable default - Fix dataclass mutable default issue in WranglerConfig by using Field(default_factory=...) - Fix pandera/pandas dtype error in scopes.py by changing DataFrame type annotation to pd.DataFrame for _filter_exploded_df_to_scope function parameter - This prevents pydantic from trying to introspect pandera schemas that contain list types, which pandas doesn't support as a dtype Fixes import errors when loading the network_wrangler package. * Fix prop_for_scope type annotation to avoid pandera validation at import time Change DataFrame[RoadLinksTable] to pd.DataFrame in prop_for_scope function signature to prevent pydantic from trying to introspect the pandera schema at import time. The validation still happens inside the function via validate_df_to_model(). * Simplify default_factory usage - remove unnecessary helper functions The helper functions _default_*_config() are unnecessary since the classes themselves are callable and can be used directly in Field(default_factory=...). This simplifies the code without changing functionality. * Fix RoadwayNetwork pydantic model to avoid pandera schema introspection Change DataFrame[Model] type annotations to pd.DataFrame in RoadwayNetwork class fields to prevent pydantic from trying to introspect pandera schemas at class definition time. Validation is now done in field validators using validate_df_to_model(). This fixes the 'dtype list not understood' error when importing the package. * Fix Python 3.10+ compatibility issues - Fix field_validator decorator order (classmethod before field_validator) - Fix bug in Feed.__init__ where dict iteration was missing .items() - These changes ensure compatibility with Python 3.10 through 3.12 * Update CI to test Python 3.10, 3.11, and 3.12 - Add Python 3.11 and 3.12 to test matrix - Add explicit import test that would fail before the hotfix - This ensures the hotfix is validated across all supported Python versions * Ignore UP045 linting rule to maintain Python 3.9 compatibility UP045 suggests using 'X | None' syntax which requires Python 3.10+. Since the project supports Python 3.9+, we should keep using Optional[X] for backwards compatibility. * Add Python 3.9 to CI test matrix Since the project supports Python 3.9+ (requires-python = '>=3.9'), we should test Python 3.9 in CI to ensure compatibility. * Add Python 3.13 and 3.14 to CI test matrix Test all current stable Python versions to ensure compatibility across the full range of supported versions. * Remove Python 3.14 from CI test matrix Python 3.14 is still in development and fiona/GDAL dependencies don't have wheels available yet, causing build failures. We'll add it back once it's stable and dependencies are available. * Ignore PLC0415 linting rule for intentional lazy imports PLC0415 flags imports inside functions, but many of these are intentional lazy imports to avoid circular dependencies or to only import heavy dependencies when needed. These are a common pattern in the codebase and should not be flagged. * Fix docstring capitalization (D405) Fix 'raises' to 'Raises' in docstring to match Google style convention. * Fix remaining linting errors - Fix D405: Capitalize 'returns' to 'Returns' in data.py docstring - Fix RUF022: Sort __all__ list alphabetically in __init__.py - Fix PLW1641: Add noqa comment for DBModelMixin (intentionally unhashable due to mutable dataframes) * Auto-fix remaining ruff linting issue * Fix formatting of PLR0913 rule in pyproject.toml * Fix class definition indentation for DBModelMixin * fix: remove unused PLC0415 noqa directives Ruff RUF100 errors: removed unused noqa directives for PLC0415 rule which is already disabled in pyproject.toml. All 29 instances fixed. * fix: use TypeGuard from typing_extensions for Python 3.9 compatibility - Remove duplicate TypeGuard import from typing (not available in Python 3.9) - Add 'from __future__ import annotations' to enable list[str] syntax in Python 3.9 - Simplify _islist type guard function - Update list comprehensions to use or statements for consistency * fix: use s.timespan directly after type guard check The _islist() function returns a bool (type guard), not the list itself. After the type guard check passes, we should use s.timespan directly, which mypy knows is list[str] after the type guard. Fixes TypeError: 'bool' object is not iterable in test_conflicting_managed_lane_apply * fix: pin setuptools and importlib_metadata for mkdocs compatibility Pin setuptools < 69.0 and importlib_metadata < 6.0.0 to fix EntryPoints API compatibility issue with mkdocs plugins. Fixes AttributeError: 'EntryPoints' object has no attribute 'get' in test_mkdocs_build * test: remove mkdocs build test from pytest The mkdocs build is already tested in CI via the 'mike deploy' command in the workflow. Removing the pytest test to avoid dependency issues and duplicate testing. * removed test_docs.py since it isn't needed * fix: upgrade markdown to 3.4.0+ for EntryPoints API compatibility - Upgrade markdown from 3.3.1 to >=3.4.0,<4.0 (3.4.0+ supports new EntryPoints API) - Make importlib-metadata constraint conditional for Python < 3.12 (Python 3.12+ has importlib_metadata built-in and can't be pinned) - Add importlib-metadata constraint to build-system requires Fixes AttributeError: 'EntryPoints' object has no attribute 'get' in markdown package when using Python 3.12+ * refactor: remove explicit importlib-metadata dependency Python 3.9+ has importlib.metadata in the standard library, so we don't need the importlib-metadata package as an explicit dependency. Since we upgraded markdown to 3.4.0+ which supports the new EntryPoints API, we no longer need to pin importlib-metadata for compatibility. If any package needs it, pip will install it as a transitive dependency. * refactor: remove setuptools restriction and docs from tests dependencies - Remove setuptools<69.0 restriction (no longer needed with markdown 3.4.0+) - Remove network-wrangler[docs] from tests dependencies (test_mkdocs_build was removed) These restrictions were added for EntryPoints API compatibility, but are no longer necessary after upgrading markdown to 3.4.0+ which supports the new API. * Restore docs install to testing for now Need it because using same install * docs: update CHANGELOG for hotfix/pandera-pandas-compatibility * docs: fix CHANGELOG to use version number instead of branch name * Update version name to slug * fix: update mkdocstrings members option and fix invalid escape sequence - Change members: None to members: [] in networks.md (mkdocstrings expects a list) - Fix invalid escape sequence in utils.py make_slug function (use raw string) Fixes mkdocs build error: Invalid options validation errors * fix: remove selection option from mkdocstrings config for Python 3.9 compatibility The selection.new_path_syntax option is not supported in older versions of mkdocstrings-python used on Python 3.9. Removing it to fix the build error: '__init__() got an unexpected keyword argument 'selection''
## CI and GitHub Actions - Add reusable actions: setup-python-uv (with caching), get-branch-name, build-docs, compare-benchmarks, post-coverage - Update workflows to use modern actions and cache dependencies - Simplify coverage to Python 3.13 only; fix post-coverage artifact paths - Run benchmark comparison inline in workflows instead of from a separate action - Use PR head ref/repo for checkout so auto-fix commits push to the correct branch (including forks) - Grant workflow write permissions for PRs (contents, pull-requests) where needed - Validate package version vs tag and block duplicate publishes (PyPI/TestPyPI) - Fix mkdocs config to point to current docs location - Fix duplicate documentation uploads that caused false test failures - Fix docs job to checkout code before building - Add .github/README.md documenting workflows and actions ## Python and dependency compatibility - Use TypeGuard from typing_extensions for Python 3.9 - Add `from __future__ import annotations` for Python 3.9 (e.g. list[str]) - Replace pa.String with NpString for pandas 2.2+ / Python 3.11+ (StringDtype and numpy.issubdtype) - Pin pandas <3.0 - Simplify _islist type guard and align list-comprehension style ## Linting and formatting - Lint and format fixes; PR workflow can auto-commit fixes with [skip ci] Fixes #432, #439 Co-authored-by: Cursor <cursoragent@cursor.com>
Changed continue-on-error to false for linting fixes and updated commit message to remove [skip ci] so that the final commit has the full CI run for PRs. This will yield effectively the same result, but will result in the green check mark (or x) next to PR appearing when there are formatting changes.
* chore: Deprecate Python 3.9: modernize to Python 3.10+ with PEP 604 type hints - Update minimum Python version to 3.10+ in pyproject.toml and docs - Modernize all type hints: Optional[X] → X | None, Union[X, Y] → X | Y - Update isinstance() calls and fix deprecated imports - Add strict=True to zip() calls and simplify lambdas - Remove Python 3.9-specific lint ignores Fixes #430 * style: Auto-fix code formatting and linting issues [skip ci] * fix: handle union and uniontype type hings Also: fix strict zip warnings --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Reduces apply() time from ~1s to ~260ms for 66K link networks by replacing expensive hash computation (~200ms each, called 3x per apply) with O(1) version counter checks. - Add _modification_version to DBModelMixin (auto-increments on table changes) - Update TransitSelection to use modification_version - Update TransitNetwork consistency checks to use modification_version - style: Auto-fix code formatting and linting issues FIxes #446 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Importing network_wrangler to check the version fails in CI because projectcard (a runtime dependency) is not installed at that point. Use grep + sed to extract __version__ directly from __init__.py instead. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See network-wrangler@708a017