Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .konjo/scripts/dry_check.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@
import argparse
import hashlib
import json
import os
import re
import subprocess
import sys
Expand Down
8 changes: 6 additions & 2 deletions .konjo/scripts/konjo_review.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,11 @@
Exit codes: 0=APPROVED/WARNING, 1=BLOCKER, 2=API error
"""
from __future__ import annotations
import argparse, json, os, sys, time
import argparse
import json
import os
import sys
import time
from pathlib import Path

CRITIC_MODEL = "claude-opus-4-6"
Expand Down Expand Up @@ -68,7 +72,7 @@ def _call_api(diff_text: str, anthropic_module) -> dict:
usage = response.usage
print(f"[konjo-review] tokens: input={usage.input_tokens} output={usage.output_tokens} cache_read={getattr(usage, 'cache_read_input_tokens', 0)}", file=sys.stderr)
return json.loads(raw)
except (anthropic_module.RateLimitError, anthropic_module.APIStatusError) as exc:
except (anthropic_module.RateLimitError, anthropic_module.APIStatusError):
if attempt < MAX_RETRIES - 1:
delay = RETRY_BASE_DELAY * (2**attempt)
print(f"[konjo-review] retrying in {delay:.0f}s...", file=sys.stderr)
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Episodic memory engine for LLMs — persistent, associative, and beyond context windows. Hyperdimensional Computing (HDC) in Rust with Python bindings, temporal decay, semantic consolidation, and OpenAI-compatible memory middleware.

**v0.4.0** (Python) / **v0.1.0** (Rust) — 69 tests passing (1 pre-existing skip).
**v0.11.0** (Python) / **v0.1.0** (Rust) — 404 tests passing.

## Stack
Rust 2021 · rand · serde · anyhow · clap · PyO3 (optional, `--features python`) · Python 3.9+ · NumPy · asyncio · hatchling
Expand Down
11 changes: 10 additions & 1 deletion PLAN.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Kohaku — Development Plan

## Current Version: v0.9.0
## Current Version: v0.10.0

## Phase 1: Core HDC Engine (v0.1.0) ✅
- [x] Hypervector arithmetic: random, bundle, bind, permute
Expand Down Expand Up @@ -149,3 +149,12 @@ Three P2 features that turn the kohaku store into an observable, debuggable prod
- `DELETE /memories/stale?days=30&dry_run=true`
- [x] Tests: 45 new (`test_provenance.py` 14, `test_time_filter.py` 16, `test_memory_health.py` 15). Total **327 passed**.
- [x] `__init__.py` exports `ProvenanceGraph`, `ProvenanceNode`, `ProvenanceGraphResult`, `TimeFilter`, `TimelineBucket`, `apply_time_filter`, `bucket_timeline`, `filter_recent`, `MemoryHealthAnalyzer`, `MemoryHealthReport`, `DuplicatePair`, `StaleMemory`.

## Phase 13: P2 Features — Episodic Binding, Chaining, Validation (v0.11.0) ✅

- [x] `python/kohaku/episode.py` — `EpisodeStore` with role-binding. `store_episode(label, *, who, what, when, where)` binds provided role HVs into a composite via `bundle(bind(R_role, value_hv), ...)`. Fixed deterministic role HVs (`_ROLE_SEEDS`) so any two stores over the same dims share the same role space. `query_episode(*, who, what, when, where, top_k)` retrieves from any partial cue. `unbind_role(entry_id, role)` returns the original HV. 17 unit tests in `python/tests/test_episode.py`.
- [x] `python/kohaku/chaining.py` — `chain_query(memory, start_key, hops, min_similarity)` iteratively follows the highest-similarity unvisited entry's key HV. Returns `ChainResult(hops: List[HopResult], terminated_early)` with `.labels()` and `.similarities()` helpers. Terminates early on empty memory, no unvisited candidates, or `similarity < min_similarity`. 14 unit tests in `python/tests/test_chaining.py`.
- [x] `python/kohaku/validation.py` — `WriteValidator(memory, duplicate_threshold, rate_limits)` with two gates: (1) novelty — reject if nearest cosine >= threshold; (2) rate limit — per-source sliding-window deque. `validate(key_hv, source)` is read-only; `record(source)` commits the slot; `validate_and_store(...)` does both atomically. `RateLimit(max_stores, window_seconds)` validated at construction. 17 unit tests in `python/tests/test_validation.py`.
- [x] `api/main.py` — 4 new endpoints: `POST /episodes/store`, `POST /episodes/query`, `POST /chain`, `POST /memories/validate`. `RestState` gains `episodes: EpisodeStore` and `validator: WriteValidator` (pre-configured with `agent_inference` rate limit of 100/min).
- [x] `__init__.py` exports `EpisodeStore`, `EpisodeRoles`, `EpisodeResult`, `chain_query`, `ChainResult`, `HopResult`, `WriteValidator`, `RateLimit`, `ValidationResult`. Version bumped to `0.11.0`.
- [x] 46 new tests (17 episode + 14 chaining + 17 validation — 2 from chaining/validation consolidated = 46 net). Total **404 passed**.
189 changes: 182 additions & 7 deletions api/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,9 @@
if str(PY_PKG) not in sys.path:
sys.path.insert(0, str(PY_PKG))

from fastapi import Body, FastAPI, HTTPException, Query
from fastapi.responses import FileResponse, Response
from pydantic import BaseModel, Field, model_validator
from fastapi import Body, FastAPI, HTTPException, Query # noqa: E402
from fastapi.responses import FileResponse, Response # noqa: E402
from pydantic import BaseModel, Field, model_validator # noqa: E402

from kohaku import ( # noqa: E402
MemoryHealthAnalyzer,
Expand All @@ -56,25 +56,27 @@
from kohaku import ( # noqa: E402
DecayConfig,
EnrichedMemoryStore,
EnrichedRetrievalResult,
EpisodicMemory,
GraphExportConfig,
HDCRetriever,
HyperVector,
ItemMemory,
MemoryGraphExporter,
SOURCE_TRUST_WEIGHTS,
SleepConsolidator,
SleepReport,
_BACKEND,
decay_weight,
encode_text,
query as _kohaku_query,
query_with_decay,
)
from kohaku import ( # noqa: E402
EpisodeStore,
RateLimit,
WriteValidator,
chain_query,
)
from kohaku import __version__ as KOHAKU_VERSION # noqa: E402
from kohaku._pure import DIMS # noqa: E402
from kohaku._pure import HyperVector as _PyHyperVector # noqa: E402

from datetime import datetime, timezone # noqa: E402

Expand Down Expand Up @@ -308,6 +310,12 @@ def __init__(self, capacity: int = 10_000, dims: int = DIMS) -> None:
consolidation_interval_minutes=60.0,
similarity_threshold=0.85,
)
# Phase 13 P2 stores.
self.episodes = EpisodeStore(dims=dims, capacity=capacity)
self.validator = WriteValidator(
self.episodic,
rate_limits={"agent_inference": RateLimit(max_stores=100, window_seconds=60.0)},
)
self.lock = threading.Lock()
self.started_at = time.time()

Expand Down Expand Up @@ -520,6 +528,58 @@ class EnrichedQueryResponse(BaseModel):
sort: str


# ── Phase 13 P2 models ────────────────────────────────────────────────────────

class EpisodeStoreRequest(BaseModel):
label: str = Field(..., min_length=1)
who: Optional[List[float]] = None
what: Optional[List[float]] = None
when: Optional[List[float]] = None
where: Optional[List[float]] = None


class EpisodeStoreResponse(BaseModel):
entry_id: int
label: str


class EpisodeQueryRequest(BaseModel):
who: Optional[List[float]] = None
what: Optional[List[float]] = None
when: Optional[List[float]] = None
where: Optional[List[float]] = None
top_k: int = Field(5, ge=1, le=100)


class EpisodeQueryResponse(BaseModel):
results: List[Dict[str, Any]]


class ChainQueryRequest(BaseModel):
start: Union[str, List[float]]
type: InputType = "text"
hops: int = Field(3, ge=1, le=20)
min_similarity: float = Field(0.0, ge=-1.0, le=1.0)


class ChainQueryResponse(BaseModel):
hops: List[Dict[str, Any]]
terminated_early: bool


class ValidateRequest(BaseModel):
input: Union[str, List[float]]
type: InputType = "text"
source: Optional[str] = None


class ValidateResponse(BaseModel):
accepted: bool
reason: str
nearest_similarity: float
nearest_label: str


class ConsolidateRequest(BaseModel):
similarity_threshold: Optional[float] = Field(None, ge=-1.0, le=1.0)

Expand Down Expand Up @@ -1061,6 +1121,121 @@ def memories_stale_delete(
)
return analyzer.delete_stale(days=days, dry_run=dry_run)

# ── Phase 13 P2: episodic binding, chaining, validation ───────────────

@app.post("/episodes/store", response_model=EpisodeStoreResponse)
def episodes_store(req: EpisodeStoreRequest) -> EpisodeStoreResponse:
"""Store an episode bound from who / what / when / where role HVs.

Each provided role vector is binarized and bound with its fixed role HV;
the resulting bundle is stored as a single composite hypervector.
"""
def _to_hv(vals: Optional[List[float]]) -> Optional[HyperVector]:
return _vec_input_to_hv(vals) if vals is not None else None

who_hv = _to_hv(req.who)
what_hv = _to_hv(req.what)
when_hv = _to_hv(req.when)
where_hv = _to_hv(req.where)
if all(v is None for v in (who_hv, what_hv, when_hv, where_hv)):
raise HTTPException(status_code=422, detail="At least one role must be provided")
rest: RestState = app.state.rest
with rest.lock:
try:
eid = rest.episodes.store_episode(
req.label, who=who_hv, what=what_hv, when=when_hv, where=where_hv
)
except ValueError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
return EpisodeStoreResponse(entry_id=eid, label=req.label)

@app.post("/episodes/query", response_model=EpisodeQueryResponse)
def episodes_query(req: EpisodeQueryRequest) -> EpisodeQueryResponse:
"""Retrieve episodes matching a partial role cue.

Supply any subset of who / what / when / where; the query composite is
built from those roles only, enabling partial-cue retrieval.
"""
def _to_hv(vals: Optional[List[float]]) -> Optional[HyperVector]:
return _vec_input_to_hv(vals) if vals is not None else None

who_hv = _to_hv(req.who)
what_hv = _to_hv(req.what)
when_hv = _to_hv(req.when)
where_hv = _to_hv(req.where)
if all(v is None for v in (who_hv, what_hv, when_hv, where_hv)):
raise HTTPException(status_code=422, detail="At least one role must be provided")
rest: RestState = app.state.rest
with rest.lock:
try:
results = rest.episodes.query_episode(
who=who_hv, what=what_hv, when=when_hv, where=where_hv,
top_k=req.top_k,
)
except ValueError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
return EpisodeQueryResponse(
results=[
{
"entry_id": r.entry_id,
"label": r.label,
"similarity": r.similarity,
}
for r in results
]
)

@app.post("/chain", response_model=ChainQueryResponse)
def chain_endpoint(req: ChainQueryRequest) -> ChainQueryResponse:
"""Multi-hop associative chain starting from a text or vector query.

Each hop retrieves the highest-similarity unvisited entry, then follows
that entry's key HV to the next hop.
"""
if req.type == "text":
start_hv = encode_text(req.start if isinstance(req.start, str) else "")
else:
if not isinstance(req.start, list):
raise HTTPException(status_code=422, detail="start must be a list when type='vector'")
start_hv = _vec_input_to_hv(req.start)
rest: RestState = app.state.rest
with rest.lock:
result = chain_query(
rest.episodic, start_hv,
hops=req.hops,
min_similarity=req.min_similarity,
)
return ChainQueryResponse(
hops=[
{"hop": h.hop, "entry_id": h.entry_id, "label": h.label, "similarity": h.similarity}
for h in result.hops
],
terminated_early=result.terminated_early,
)

@app.post("/memories/validate", response_model=ValidateResponse)
def memories_validate(req: ValidateRequest) -> ValidateResponse:
"""Dry-run validation: check if a vector would be accepted by the write validator.

Returns accepted=True/False, rejection reason, and nearest existing entry info.
Does NOT store anything or consume a rate-limit slot.
"""
if req.type == "text":
key_hv = encode_text(req.input if isinstance(req.input, str) else "")
else:
if not isinstance(req.input, list):
raise HTTPException(status_code=422, detail="input must be a list when type='vector'")
key_hv = _vec_input_to_hv(req.input)
rest: RestState = app.state.rest
with rest.lock:
result = rest.validator.validate(key_hv, source=req.source)
return ValidateResponse(
accepted=result.accepted,
reason=result.reason,
nearest_similarity=result.nearest_similarity,
nearest_label=result.nearest_label,
)

# ── Sleep-phase consolidation ──────────────────────────────────────────
@app.post("/consolidate", response_model=ConsolidateResponse)
def consolidate_endpoint(req: ConsolidateRequest = ConsolidateRequest()) -> ConsolidateResponse:
Expand Down
6 changes: 3 additions & 3 deletions api/test_viz.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@
if str(ROOT) not in sys.path:
sys.path.insert(0, str(ROOT))

from fastapi.testclient import TestClient
from fastapi.testclient import TestClient # noqa: E402

from api.main import VizState, create_app
from kohaku._pure import DIMS
from api.main import VizState, create_app # noqa: E402
from kohaku._pure import DIMS # noqa: E402


@pytest.fixture
Expand Down
17 changes: 8 additions & 9 deletions demo/demo.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@
"""
from __future__ import annotations

import os
import sys
import tempfile
from pathlib import Path
Expand All @@ -24,14 +23,14 @@
if str(PY_PKG) not in sys.path:
sys.path.insert(0, str(PY_PKG))

import numpy as np
from rich.console import Console
from rich.panel import Panel
from rich.rule import Rule
from rich.table import Table
from rich.text import Text
import numpy as np # noqa: E402
from rich.console import Console # noqa: E402
from rich.panel import Panel # noqa: E402
from rich.rule import Rule # noqa: E402
from rich.table import Table # noqa: E402
from rich.text import Text # noqa: E402

from kohaku import (
from kohaku import ( # noqa: E402
DecayConfig,
EpisodicMemory,
HyperVector,
Expand All @@ -42,7 +41,7 @@
query_with_decay,
save,
)
from kohaku._pure import DIMS
from kohaku._pure import DIMS # noqa: E402

console = Console()

Expand Down
13 changes: 7 additions & 6 deletions demo/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,18 +42,17 @@
if str(PY_PKG) not in sys.path:
sys.path.insert(0, str(PY_PKG))

import kohaku
from kohaku import (
import kohaku # noqa: E402
from kohaku import ( # noqa: E402
DecayConfig,
EpisodicMemory,
HyperVector,
encode_text,
load,
query,
save,
)
from kohaku._pure import DIMS
from kohaku.decay import decay_weight
from kohaku._pure import DIMS # noqa: E402
from kohaku.decay import decay_weight # noqa: E402

DEMO_DIR = Path(__file__).resolve().parent
INDEX_HTML = DEMO_DIR / "index.html"
Expand Down Expand Up @@ -328,7 +327,9 @@ def do_GET(self) -> None: # noqa: N802
elif path == "/api/graph":
self._send_json(self.state.graph())
elif path == "/favicon.ico":
self.send_response(204); self._cors(); self.end_headers()
self.send_response(204)
self._cors()
self.end_headers()
else:
self._send_json({"error": f"unknown path {path!r}"}, status=404)
except Exception as e:
Expand Down
4 changes: 0 additions & 4 deletions python/kohaku.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,7 @@

from __future__ import annotations

import json
import math
import subprocess
import sys
from typing import Iterator

# ─── LCG constants (must match Rust implementation in src/hypervector.rs) ────
_LCG_MUL: int = 6_364_136_223_846_793_005
Expand Down
Loading
Loading