Repository navigation
Refactor canned triggers into native tool calls #25
Description
The regex matchers in lib/responses.js (isImageRequest, isDanceParty, isPodBayDoor, isLoveYou, isRickroll, isTikTok, dad-joke trigger) are 2018-vintage and increasingly out of place now that Data runs on native Ollama/Gemini SDKs with proper tool support.
Idea
Define a small set of tools and let the model decide when to call them:
generate_image(prompt)— calls Gemini Nano Bananatell_dad_joke()— fetches from icanhazdadjoke.comstart_dance_party()— emits the emoji rainshow_help()— returns the help textstate_asimovs_laws()— Asimov rule recitationplay_rickroll()/play_tiktok()— the button blocks
Both Ollama (tools: [...] → message.tool_calls) and Gemini (config.tools with function declarations) have native tool calling. Translation happens inside lib/chat-backends.js so handleMessage only deals with the canonical shape.
Biggest immediate win
The IMAGE_REQUEST_GUIDANCE shim where Data has to say "please use the /image command" instead of just generating the image. That whole interaction goes away — user says "Data, picture of a sehlat", Data calls the tool, image appears in the channel.
Plan
- New
lib/tools.js— tool definitions (JSON schemas + implementations). - Extend the chat-backend adapter interface:
chat({ messages, tools }) → { text, toolCalls? }. Adapters translate to/from each provider's native shape. handleMessagebecomes a small loop: call the model → iftoolCalls, execute them and feed the results back as a tool-role message → repeat until plain text. Cap iterations to prevent runaways.- Delete
isImageRequest/IMAGE_REQUEST_GUIDANCE, the love-you/pod-bay/danceparty/rickroll/tiktok matchers and their call sites inapp.js. Keepthe rules,help,dad jokematchers as fallbacks for now (or convert them too). - Tests: adapter translation of tool calls for both Ollama + Gemini, and a fake-tool-calling chat that exercises the multi-turn dispatch loop in
handleMessage.
What this unlocks downstream
Future tools (web search, weather, Slack user lookups, etc.) just slot into lib/tools.js — no new dispatch code, no new regex.
Activity
Merge branch 'feature/tool-calling' — tool calling, vision, thinking,…
scarolan commentedon Jun 14, 2026
Built, shipped, then reverted. Won't pursue further with the current model lineup.
What we learned:
gemma4 (Data's preferred backend for its strong vision) cannot reliably emit Ollama's structured
tool_callsfield. It writes calls as inline text like<call:generate_image{prompt:"..."}><tool_call|>instead. Same behavior ongemma4-openhands:latest(the agent fine-tune of the same family), so it's a family-wide wire-format / template issue, not a prompting issue.6 of the 7 tools added in PR Tool calling, vision, thinking traces, reaction UX, threaded replies #30 duplicated regex matchers that already worked.
tell_dad_joke,state_asimovs_laws,show_help,start_dance_party,play_rickroll,play_tiktok— all of these were already exact-phrase triggers that were faster, more reliable, and model-agnostic. Adding tools for them was solving a non-problem.The one tool that genuinely benefited from LLM intent extraction —
generate_image— was the unreliable one (gemma flubs the structured call). And/imageslash command already handles image gen cleanly with no LLM round-trip.The reliable alternative (qwen3.6:27b for tools) isn't vision-capable. Picking "tools work but no vision" was a worse trade than "vision works, no tools."
Outcome: tool calling code (lib/tools.js, dispatch loop, adapter translation) removed.
IMAGE_REQUEST_GUIDANCErestored — Data politely directs users to/imagewhen they ask for an image in chat. ~800 lines of code retired for net-positive reliability.If someone ever wants to revisit: pull a model that does both vision and tool calling well (e.g.
qwen2.5-vl:32bormistral-small3.1) on kepler.local first, verify it emits structuredtool_callsagainst a 2-tool test registry, then re-wire. Don't ship tools without that pre-check. Notes captured in CLAUDE.md "Model selection notes".