Skip to content

fix(generators): apply section filter when paging localizations in download - #75

Merged
danellecline merged 1 commit into
mainfrom
fix/download-section-filter
Jul 22, 2026
Merged

fix(generators): apply section filter when paging localizations in download#75
danellecline merged 1 commit into
mainfrom
fix/download-section-filter

Conversation

@danellecline

Copy link
Copy Markdown
Member

Problem

aidata download dataset --section <name> ... finds the correct localization/media counts during the initial query phase, but then downloads 0 images/crops when a --section filter is used.

Example from a real run against section 300m:

Found 38519 localizations with {..., 'section': 482}
Found 17720 media with matching direct attributes
...
Found 0 records for version [...], section 300m, and including ['Unknown']
Cropping 0 ROIs...

Root cause

In mbari_aidata/generators/coco_voc.py, download() selects matching media via get_localization_count() / get_medias(), both of which correctly pass section=section_id to the Tator API.

However, the actual paginated fetch of Localization objects (query_localizations()api.get_localization_list(...)) never included the section filter. This caused it to page through localizations across the entire project (bounded by an inflated max_records), instead of the target section. Localizations were filtered afterward by checking l.media against the section-restricted media dict, so the scan could exhaust its record budget before ever encountering a localization whose media fell inside the target section — resulting in 0 matches and no downloaded images/crops.

Fix

Add the missing section filter to query_localizations()'s kwargs, matching the filtering already applied in get_localization_count() and get_medias().

Testing

  • pytest tests/test_coco_voc_localizations_csv.py — 13 passed.
  • Full local suite: 54 passed, 4 skipped, 7 pre-existing failures unrelated to this change (require a live Tator server at localhost:8080).

Made with Cursor

…wnload

query_localizations() built its own kwargs for get_localization_list()
but never included the section filter that get_localization_count()
and get_medias() already applied. When --section was passed, this
caused the paginated fetch to scan localizations across the whole
project (bounded by an inflated max_records) instead of just the
target section, so legitimate media/localizations in the section were
never matched and 0 records were downloaded.

Co-authored-by: Cursor <cursoragent@cursor.com>
@danellecline danellecline added type/fix Bug fix scope/generators mbari_aidata/generators/ impact/low Small, localized risk status/needs-review Ready for human review labels Jul 22, 2026
@danellecline
danellecline merged commit 64afc8f into main Jul 22, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

impact/low Small, localized risk scope/generators mbari_aidata/generators/ status/needs-review Ready for human review type/fix Bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant