fix(generators): apply section filter when paging localizations in download - #75
Merged
Merged
Conversation
…wnload query_localizations() built its own kwargs for get_localization_list() but never included the section filter that get_localization_count() and get_medias() already applied. When --section was passed, this caused the paginated fetch to scan localizations across the whole project (bounded by an inflated max_records) instead of just the target section, so legitimate media/localizations in the section were never matched and 0 records were downloaded. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
aidata download dataset --section <name> ...finds the correct localization/media counts during the initial query phase, but then downloads 0 images/crops when a--sectionfilter is used.Example from a real run against section
300m:Root cause
In
mbari_aidata/generators/coco_voc.py,download()selects matching media viaget_localization_count()/get_medias(), both of which correctly passsection=section_idto the Tator API.However, the actual paginated fetch of
Localizationobjects (query_localizations()→api.get_localization_list(...)) never included thesectionfilter. This caused it to page through localizations across the entire project (bounded by an inflatedmax_records), instead of the target section. Localizations were filtered afterward by checkingl.mediaagainst the section-restricted media dict, so the scan could exhaust its record budget before ever encountering a localization whose media fell inside the target section — resulting in 0 matches and no downloaded images/crops.Fix
Add the missing
sectionfilter toquery_localizations()'skwargs, matching the filtering already applied inget_localization_count()andget_medias().Testing
pytest tests/test_coco_voc_localizations_csv.py— 13 passed.localhost:8080).Made with Cursor