Skip to content

PROGRESS.md

Latest commit

 

History

History
147 lines (114 loc) · 4.93 KB

PROGRESS.md

File metadata and controls

147 lines (114 loc) · 4.93 KB
Raw

Prollama Development Progress

Current Status

Working on implementing and testing the /api/chat endpoint with tool calling support.

Completed Work

1. Generate Endpoint (src/prolog/ollama.pro)

  • Implemented generate/5 (non-streaming) and generate/6 (streaming)
  • Created generate_accumulator/3 to accumulate response field
  • Implemented process_responses/6 generic streaming processor with custom accumulator
  • Added proper error handling with status code checks for streaming

2. Chat Endpoint (src/prolog/ollama.pro)

  • Implemented chat/5 (non-streaming) - returns ResponseMessage with all fields from Result.message
  • Implemented chat/6 (streaming) - accumulates message fields throughout stream
  • Created sophisticated chat_accumulator/3 that:
    • Concatenates content and thinking fields (strings)
    • Appends images and tool_calls fields (lists)
    • Takes last value for tool_name (scalar)
  • Initializes streaming with message{role: "assistant", content: ""} to preserve tag
  • Returns complete message dict preserving all fields from stream

3. Helper Predicates

  • create_message/3 and create_message/4 - create message dicts with message tag
  • retag/3 in utils.pro - change dict tags while preserving all fields

4. Code Improvements

  • Unified streaming/non-streaming patterns using Handler == false
  • Created http_post_or_stream/4 with proper error handling
  • Added check_status/2 for consistent HTTP error reporting
  • Fixed operator precedence issues (comma=1000, arrow=1050, semicolon=1100)
  • Added status code checking for both streaming and non-streaming cases

5. Test Suite

test/generate.pro (11 tests, all passing)

  • Basic streaming/non-streaming
  • Suffix/infill with codegemma:2b
  • JSON mode and structured outputs
  • Raw mode, seed, temperature, system prompts
  • Streaming with custom handler
  • Multimodal with images (llama3.2-vision)

test/chat.pro (3 tests, passing)

  • chat_non_streaming - basic non-streaming chat
  • chat_streaming - basic streaming chat with ignore handler
  • chat_with_tools_streaming - streaming with tool calls (llama3.2)

6. Models Available

  • gemma3:270m (general)
  • codegemma:2b (suffix/infill support)
  • llama3.2 (tool calling)
  • llama3.2-vision (multimodal)
  • mistral-small3.2:24b (tool calling)

Current Work

Testing Tool Calling

Created test/chat.pro:chat_with_tools_streaming which:

  • Defines a get_weather tool with JSON schema
  • Sends request to llama3.2 with tools parameter
  • Verifies model returns tool_calls in response
  • Checks tool call structure (function.name, function.arguments)

Key Learning: assertion/1 doesn't propagate variable bindings, so bindings must happen outside assertion calls.

Next Steps

Immediate: Tool Execution Framework

Need to implement:

  1. Tool Registration: A way to map tool names to Prolog predicates

    • How should users register tools?
    • Dynamic assertion? Dict of tool_name: predicate mappings?

  2. Tool Execution: Execute tool calls and format results

    • Parse tool_call from model response
    • Call registered Prolog predicate with arguments
    • Format result as message with role: "tool", tool_name, content

  3. Agent Loop: Multi-turn conversation with tool use

    • Send initial message with tools
    • Get tool_calls from model
    • Execute tools
    • Send results back
    • Get final answer

  4. Testing: Complete tool calling workflow test

    • Register a mock get_weather tool
    • Execute the tool call from model
    • Send result back to model
    • Verify final response incorporates tool result

Technical Decisions

Message Field Accumulation

  • Content/thinking: string concatenation
  • Images/tool_calls: list append
  • tool_name: take last value
  • Based on API documentation analysis of streaming responses

Role Values

  • JSON parsing returns strings: "assistant", "user", etc.
  • Not atoms: assistant, user

Error Handling

  • Streaming and non-streaming both check status codes
  • Use check_status/2 to throw descriptive errors
  • Error format: error(http_error(Status, Response), context(_, Msg))

Important Files

  • src/prolog/ollama.pro - Main API implementation
  • src/prolog/utils.pro - Helper predicates (sha256, base64, retag)
  • test/generate.pro - Generate endpoint tests
  • test/chat.pro - Chat endpoint tests (in progress)
  • docs/ollama-api.md - API reference
  • CLAUDE.MD - Instructions about professional tone, not inventing APIs

Known Issues

None currently blocking.

API Coverage

Implemented:

  • ✅ /api/generate (streaming & non-streaming)
  • ✅ /api/chat (streaming & non-streaming)
  • ✅ /api/pull
  • ✅ /api/push
  • ✅ /api/create (including from safetensors/GGUF)
  • ✅ /api/delete
  • ✅ /api/copy
  • ✅ /api/show
  • ✅ /api/embed
  • ✅ /api/ps (list running)
  • ✅ /api/tags (list models)
  • ✅ /api/version
  • ✅ Blob operations

Not yet implemented:

  • 🔄 Tool execution framework (in progress)
  • Agent/multi-turn tool calling loops