Skip to content

For testing / staging, finalize prodigy recipe, input, and configuration for concept evaluation #49

@laurejt

Description

@laurejt
  • Recipe: Update labels to new term error typology
    • Pre: Confirm with Annie the label list
  • Ansible: Update allowed session names
    • Pre: Get list from Annie
  • Recipe: Update logic for number of required annotations by language
    • Pre: Check with Annie about annotator availability per language
    • Post: Possibly revisit task routing logic if it can be simplified
  • Recipe: Add instructions with link to handbook
    • Pre: Get term typology's working definitions from Annie
    • Pre: Create handbook google doc
  • Recipe: Add custom progress function (replace existing total_examples_target logic)
    • It may not be possible to report accurate session-level progress, so best effort within time limits
  • Recipe: Add evaluation typology to the recipe as a disclosure element

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions