Skip to content

docs: add Inference section to README - #2

Open
igor-susic1 wants to merge 1 commit into
masterfrom
docs/readme-inference-section
Open

docs: add Inference section to README#2
igor-susic1 wants to merge 1 commit into
masterfrom
docs/readme-inference-section

Conversation

@igor-susic1

Copy link
Copy Markdown
Owner

Adds an ## Inference section to README.md between the existing Models and Tags sections, documenting where and how kimchi runs model inference.

Covers what's actually implemented in src/ollama.ts and wired in src/cli.ts:

  • Default inference against kimchi's hosted LLM infrastructure via the setup API key
  • Native local inference through Ollama — silent, non-blocking startup probe; stale ollama provider block removed when the server is unreachable
  • Ollama endpoint precedence: http://localhost:11434
  • OpenAI-compatible surface (${host}/v1) and zero-cost reporting for Ollama models
  • Role integration — discovered models appended to builder, reviewer, and explorer pools only; orchestrator/planner/judge/researcher intentionally left untouched
  • Tier derivation by parameter size (<8B light, 8–<30B standard, ≥30B heavy, unknown→standard)
  • Observability via NODE_DEBUG=kimi:ollama

Every claim is backed by the corresponding source in src/ollama.ts / src/cli.ts; no behavior is changed.

  • Documentation update

@igor-susic1 igor-susic1 added the documentation Improvements or additions to documentation label Jun 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant