Skip to content

Refine annotation recipe & added custom routing#46

Merged
laurejt merged 3 commits intodevelopfrom
feature/refine-concept-recipe
Mar 6, 2026
Merged

Refine annotation recipe & added custom routing#46
laurejt merged 3 commits intodevelopfrom
feature/refine-concept-recipe

Conversation

@laurejt
Copy link

@laurejt laurejt commented Mar 6, 2026

Associated Issue(s): N/A

Changes in this PR

  • Added a global variable setting the number of annotations needed for a task by language
  • Added custom task routing logic
  • Added custom logic for calculating the expected number of annotations for a more accurate "source progress"

Notes

  • Custom logic assumes that the session name has a starting prefix indicating the language e.g., "pt_". The session id adds a prefix before the session name, so we determine the viable sessions (i.e, annotators) for a task by checking if the session_id contains "-[src_lang]_".

Reviewer Checklist

@laurejt laurejt requested a review from rlskoeser March 6, 2026 14:34
@laurejt laurejt self-assigned this Mar 6, 2026
@laurejt laurejt added the 👇this sprint Add Issue to ZenHub label Mar 6, 2026
Copy link

@rlskoeser rlskoeser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable to me, just might be worth a comment clarifying how the minimums were determined (I assume this is based on annotator availability)

from prodigy.types import RecipeSettingsType, StreamType
from prodigy.util import TASK_HASH_ATTR

# Mapping of languages to number of required annotations

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a comment on how they numbers were determined? (Is this based on availability of annotators for languages?)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, based on availability of annotators. Mostly just special-casing the languages that are only expected to have a single annotator.

@laurejt laurejt merged commit 58ce06e into develop Mar 6, 2026
1 check passed
@laurejt laurejt deleted the feature/refine-concept-recipe branch March 6, 2026 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

👇this sprint Add Issue to ZenHub

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants