feat: filter verified samples in CSV export - #30
Merged
Conversation
Add scripts/export_dataset_csv.py and scripts/set_verified.py (previously untracked one-off utility scripts) and add a --verified-only flag to the export script so unverified samples can be excluded from CSV exports. Samples missing the `verified` field are treated as unverified. Includes unit tests and README documentation. Closes #29 Co-authored-by: Cursor <cursoragent@cursor.com>
danellecline
added
scope/app
Add a 'Verified only' checkbox next to Force sync in the launcher applet. Threads a new verified_only param through POST /sync -> enqueue_sync -> run_sync_job -> sync_project_to_fiftyone -> build_fiftyone_dataset_from_crops / reconcile_dataset_with_tator -> _create_sample_from_loc. When enabled, only localizations whose 'verified' attribute is truthy are included when building/reconciling the FiftyOne dataset (missing/False = excluded). Refs #29 Co-authored-by: Cursor <cursoragent@cursor.com>
Member
Author
Follow-up: added a "Verified only" checkbox to the sync launcher UIIn addition to the CSV export flag, added the equivalent filter to the web applet:
Full suite: |
…s set reconcile_dataset_with_tator now retroactively drops dataset samples whose elemental_id still exists in Tator but whose verified attribute flipped to False/missing, when verified_only is enabled. Such samples are not re-added by the add-new-samples step, since _create_sample_from_loc rejects them again for the same reason. Behavior is unchanged when verified_only is off. Refs #29 Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
--verified-onlyflag toscripts/export_dataset_csv.pyso unverifiedsamples can be excluded when exporting a dataset to CSV.
verifiedfield, or with it set toFalse, are excluded.verifiedfield at all, the flag is a no-op with a warning.scripts/set_verified.py(previously untracked) which is thecompanion script that sets
verified=Trueon samples.Closes
Closes #29
Testing
tests/test_export_dataset_csv.pycovering the filter and CSV fieldexclusion logic against a real in-memory FiftyOne dataset.
95 passed(pytest tests/).Docs
README.mddocumenting both scripts and thenew flag.