You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add text_extraction as a new activity type. Where tag_extraction
generates semantic tags describing the content of a photo, text_extraction captures literal textual content visible in the
image — signage, billboards, menus, labels, street signs, documents,
text on buildings.
Motivation
Vision models already in use (qwen2-vl and similar) are capable of
reading text from images without additional providers or specialized
OCR models. The use case emerged from observing that a photo of a
billboard produces semantic tags ("billboard", "advertising") but
the actual text on the sign is never captured.
Text extraction adds a qualitatively different metadata layer that
complements tag extraction and can be run independently or in
sequence on the same project.
Scope
API
Add text_extraction as a valid value for the activity_type
field in POST /api/projects/{id}/activities
Define the prompt used for text extraction (instruct the model to
return only textual content found in the image, not a description)
Define the output format for extracted text (list of strings,
or single block, to be decided)
Apply extracted text to Lightroom metadata fields
(keywords, caption, or description — to be decided)
Documentation
Update API reference with the new activity type and output format
Update evolution document (already done in docs/)
Considerations
Activity context (issue feat: add context field to activity creation #122) is particularly useful here: the
user can guide the model ("look for billboard text",
"extract menu items") to improve extraction quality
Mixed-language text is handled naturally by multilingual models
Output quality depends on image resolution and font legibility
Acceptance criteria
POST /api/projects/{id}/activities with activity_type: "text_extraction" creates and queues
an activity successfully
Extracted text is returned in the activity results
Plugin applies extracted text to the appropriate Lightroom
metadata field
Summary
Add
text_extractionas a new activity type. Wheretag_extractiongenerates semantic tags describing the content of a photo,
text_extractioncaptures literal textual content visible in theimage — signage, billboards, menus, labels, street signs, documents,
text on buildings.
Motivation
Vision models already in use (qwen2-vl and similar) are capable of
reading text from images without additional providers or specialized
OCR models. The use case emerged from observing that a photo of a
billboard produces semantic tags ("billboard", "advertising") but
the actual text on the sign is never captured.
Text extraction adds a qualitatively different metadata layer that
complements tag extraction and can be run independently or in
sequence on the same project.
Scope
API
text_extractionas a valid value for theactivity_typefield in
POST /api/projects/{id}/activitiesreturn only textual content found in the image, not a description)
or single block, to be decided)
400withunsupported_activity_typefor unknown types(already specified in issue feat: update API paths and request shapes to new terminology, update plugin accordingly #92 — no new error code needed)
Plugin
text_extractionin the activity creation modal:consistent with the pattern defined for future activity types
in issue feat: update Lightroom plugin UI to reflect new terminology #93)
(keywords, caption, or description — to be decided)
Documentation
docs/)Considerations
user can guide the model ("look for billboard text",
"extract menu items") to improve extraction quality
Acceptance criteria
POST /api/projects/{id}/activitieswithactivity_type: "text_extraction"creates and queuesan activity successfully
metadata field
tag_extractionactivities are unaffected