You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add vision: pipe attached Slack images into the LLM #26
Both Ollama and Gemini take images natively. Slack messages with file attachments arrive on the event payload. ~30 lines of glue unlocks a whole feature class.
Use cases
"Data, what's in this screenshot?"
"Data, what's broken about this stack trace?"
"Identify this Star Trek species."
"Translate the text in this photo."
Plan
In the message handler, check message.files for image MIME types (image/png, image/jpeg, image/webp).
Fetch each via app.client.files.info + the URL with the bot token, base64 encode.
Extend the canonical message shape to allow images: [base64] on user turns.
Shipped on main as part of the work merged via PR #30.
Slack image attachments are extracted in app.js::extractMessageImages, base64-encoded, and attached to the user turn in the canonical chat shape as images: [{ mimeType, data }]. Adapters translate to Ollama's message.images and Gemini's inlineData parts. Image bytes are not persisted to convoStore.
Working live against gemma4:31b — verified with a meme: correctly identified hammer/sickle symbology and the joke premise unprompted. See CLAUDE.md "Model selection notes" for lessons learned (Slack subtype: 'file_share' gotcha, observability logging added).
Activity
Merge branch 'feature/tool-calling' — tool calling, vision, thinking,…
scarolan commentedon Jun 14, 2026
Shipped on main as part of the work merged via PR #30.
Slack image attachments are extracted in
app.js::extractMessageImages, base64-encoded, and attached to the user turn in the canonical chat shape asimages: [{ mimeType, data }]. Adapters translate to Ollama'smessage.imagesand Gemini'sinlineDataparts. Image bytes are not persisted to convoStore.Working live against gemma4:31b — verified with a meme: correctly identified hammer/sickle symbology and the joke premise unprompted. See CLAUDE.md "Model selection notes" for lessons learned (Slack
subtype: 'file_share'gotcha, observability logging added).