From 504234a6993fbdf2ea9f1d1a93f939ada0885418 Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 12:23:55 +0530 Subject: [PATCH 01/21] docs: add smart context plugin design Co-Authored-By: Claude Sonnet 4.6 --- .../2026-03-01-smart-context-plugin-design.md | 109 ++++++++++++++++++ 1 file changed, 109 insertions(+) create mode 100644 docs/plans/2026-03-01-smart-context-plugin-design.md diff --git a/docs/plans/2026-03-01-smart-context-plugin-design.md b/docs/plans/2026-03-01-smart-context-plugin-design.md new file mode 100644 index 0000000..01b68c1 --- /dev/null +++ b/docs/plans/2026-03-01-smart-context-plugin-design.md @@ -0,0 +1,109 @@ +# Smart Context Plugin Design + +**Date:** 2026-03-01 +**Status:** Approved + +## Overview + +The Smart Context plugin gives users a visual map of the topics discussed in the current chat session. Topics are rendered as a sized bubble cloud in a dedicated panel. Clicking topic bubbles toggles a semantic filter that controls which messages are injected into the LLM pipeline as history. Short instructional messages (e.g. "be concise") are identified automatically and always injected regardless of filter state. + +## Architecture + +Four logical parts: + +1. **Topic extractor** — subscribes to `pipeline.output`. After each assistant response, fires an async background LLM call (fast model) that reads the full conversation and returns a topic list. Writes the result back to the chat file via `memory.save_chat`. + +2. **Panel UI** — registers the Smart Context panel on `on_client_connected`. Renders an HTML+JS bubble cloud. Listens for `smart_context.topics_updated` to re-render after extraction. Has an on/off toggle for similarity mode. + +3. **Filter state** — the plugin holds `self.active_topics: set[str]` and `self.current_chat_id: str` in memory. `smart_context.set_filter` updates this and emits `smart_context.filter_changed` so the main chat view can apply/remove message highlights. + +4. **Context injector** — subscribes to `pipeline.context`. Always injects sticky messages. If `embedding_search` is enabled and `active_topics` is non-empty, replaces `ctx.history` with only semantically relevant messages (cosine similarity ≥ threshold). If nothing clears the threshold, falls back to full history. + +## Data Schema + +Topics are stored as a flat list on the chat file (`.memory/{chat_id}.json`), not on individual messages: + +```json +{ + "messages": [...], + "topics": [ + {"label": "LangGraph", "count": 4}, + {"label": "Python Async", "count": 2}, + {"label": "Debugging", "count": 1}, + {"label": "Instructions & Preferences", "sticky": true, "message_ids": ["a1b2", "c3d4"]} + ] +} +``` + +- Regular topics store only `label` and `count`. Message relevance is resolved at query time via vector embeddings. +- The sticky topic (`"sticky": true`) stores explicit `message_ids` because short instructional messages have no reliable embedding similarity to content topics. The LLM extraction prompt identifies and flags these explicitly. +- Topics are **fully regenerated** on each extraction (not appended), so stale topics from early in the conversation are pruned as the chat evolves. +- LLM extraction targets 3–8 topics per conversation. + +## Embedding Cache + +To avoid re-embedding messages on every filter interaction, embeddings are cached in a sidecar file: + +``` +.memory/{chat_id}.embeddings.json +``` + +Keyed by `{message_id}:{content_hash}`. When filtering is applied, only messages without a cached embedding are re-embedded. The vault's configured `embedding` model category is used. + +## Context Injection Logic + +When `pipeline.context` fires: + +1. Always collect sticky messages (by stored `message_ids`). +2. If `embedding_search` is disabled or `active_topics` is empty → pass full `ctx.history` (plus sticky, deduplicated). +3. If `embedding_search` is enabled and topics are selected: + - Embed each selected topic label. + - Score each message by max cosine similarity across selected topics. + - Collect messages with similarity ≥ `similarity_threshold`. + - If no messages clear the threshold → fall back to full history. + - Merge with sticky messages, deduplicate, restore chronological order. + - Replace `ctx.history` with the result. + +## Panel UI + +- **Bubble cloud**: topic bubbles sized proportionally by `count`. Selected bubbles glow; unselected dim to ~40% opacity. +- **Sticky section**: "Instructions & Preferences" shown as a permanently-on locked indicator above the bubble cloud, not a toggleable bubble. +- **Status line**: `"5 of 14 messages in context"` below the cloud. Reads `"All messages in context"` when no filter is active. +- **Similarity mode toggle**: on/off switch in the panel. When off, bubbles are non-interactive (visualization only). +- **Re-render trigger**: panel JS listens for `smart_context.topics_updated` event to refresh the bubble cloud after a new extraction completes. + +## Chat Highlighting + +- When `smart_context.filter_changed` is emitted, the panel JS dispatches a `CustomEvent` on `document` with the set of included message IDs. +- The main chat JS listens for this event and applies/removes a highlight class (e.g. colored left-border) to message elements matching those IDs. +- Requires message elements in the main chat view to carry `data-message-id` attributes. + +## Commands Registered + +| Command | Description | +|---|---| +| `smart_context.set_filter` | Set active topic labels (array). Empty array clears filter. | +| `smart_context.get_topics` | Return current topics for the active chat. | +| `smart_context.set_similarity_mode` | Enable or disable embedding search. | + +## Vault Configuration (`.vault.toml`) + +```toml +[plugins.smart_context] +enabled = true +similarity_threshold = 0.4 # minimum cosine similarity to include a message +embedding_search = true # set to false to disable context filtering +``` + +## Pipeline Hook Summary + +| Phase | Action | +|---|---| +| `pipeline.output` | Trigger async topic extraction via fast LLM. Write to chat file. | +| `pipeline.context` | Inject filtered history (embedding search) + always inject sticky messages. | + +## Dependencies + +- `memory` plugin — `memory.load_chat` and `memory.save_chat` for reading/writing topics. +- `sidecar.services.llm_service` — fast model for topic extraction, embedding model for similarity search. +- Main chat frontend must expose `data-message-id` on message DOM elements for highlighting. From 4b3dc770ef579a655de2f38177cca596634e1421 Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 12:34:43 +0530 Subject: [PATCH 02/21] docs: add smart context plugin implementation plan Co-Authored-By: Claude Sonnet 4.6 --- ...03-01-smart-context-implementation-plan.md | 1213 +++++++++++++++++ 1 file changed, 1213 insertions(+) create mode 100644 docs/plans/2026-03-01-smart-context-implementation-plan.md diff --git a/docs/plans/2026-03-01-smart-context-implementation-plan.md b/docs/plans/2026-03-01-smart-context-implementation-plan.md new file mode 100644 index 0000000..8f227c8 --- /dev/null +++ b/docs/plans/2026-03-01-smart-context-implementation-plan.md @@ -0,0 +1,1213 @@ +# Smart Context Plugin Implementation Plan + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Build the Smart Context plugin — LLM-extracted topic bubbles, embedding-based context filtering, sticky instructional messages, and chat message highlighting. + +**Architecture:** Two pipeline subscribers (`pipeline.output` → async topic extraction, `pipeline.context` → embedding filter). Topics stored as flat list on chat JSON. Sticky instructional messages always injected. Embedding vectors cached in `.memory/{chat_id}.embeddings.json`. Panel HTML/JS communicates back via `window.request('execute_command', ...)`. Backend events arrive as `CustomEvent` on `window`. + +**Tech Stack:** Python, LiteLLM (`aembedding`), vanilla JS, `cosine_similarity` in pure Python (no numpy). + +--- + +### Task 1: Add `embed()` to LLMService + +**Files:** +- Modify: `sidecar/services/llm_service.py` +- Modify: `sidecar/tests/test_llm_service.py` + +**Step 1: Write the failing test** + +```python +# sidecar/tests/test_llm_service.py — add inside or alongside existing tests +from unittest.mock import AsyncMock, MagicMock, patch +import pytest +from pathlib import Path +from sidecar.services.llm_service import LLMService + +@pytest.mark.asyncio +async def test_embed_returns_list_of_vectors(): + with patch("litellm.aembedding") as mock_embed: + mock_embed.return_value = MagicMock( + data=[ + {"embedding": [0.1, 0.2, 0.3]}, + {"embedding": [0.4, 0.5, 0.6]}, + ] + ) + service = LLMService( + Path("/tmp"), + {"categories": {"embedding": "openai/text-embedding-3-large"}} + ) + result = await service.embed(["hello", "world"]) + + assert len(result) == 2 + assert result[0] == [0.1, 0.2, 0.3] + assert result[1] == [0.4, 0.5, 0.6] +``` + +**Step 2: Verify it fails** + +Run: `pixi run pytest sidecar/tests/test_llm_service.py::test_embed_returns_list_of_vectors -v` +Expected: FAIL — `LLMService has no attribute 'embed'` + +**Step 3: Implement `embed()`** + +In `sidecar/services/llm_service.py`, add this method to `LLMService` after `_stream_completion` (around line 515): + +```python +async def embed( + self, + texts: List[str], + category: str = "embedding", +) -> List[List[float]]: + """Generate embeddings for a list of texts via LiteLLM.""" + from litellm import aembedding + + model_id = self.get_model_for_category(category) + if not model_id: + raise ValueError(f"No model configured for category: {category}") + + litellm_model = self._format_model_for_litellm(model_id) + self._logger.debug(f"Embedding {len(texts)} texts with {litellm_model}") + response = await aembedding(model=litellm_model, input=texts) + return [item["embedding"] for item in response.data] +``` + +**Step 4: Verify it passes** + +Run: `pixi run pytest sidecar/tests/test_llm_service.py::test_embed_returns_list_of_vectors -v` +Expected: PASS + +**Step 5: Commit** + +```bash +git add sidecar/services/llm_service.py sidecar/tests/test_llm_service.py +git commit -m "feat: add embed() method to LLMService" +``` + +--- + +### Task 2: EmbeddingCache + +**Files:** +- Create: `example-vault/plugins/smart_context/embedding_cache.py` +- Modify: `sidecar/tests/test_plugin_smart_context.py` + +**Step 1: Write failing tests** + +Add to `sidecar/tests/test_plugin_smart_context.py`: + +```python +import importlib.util +from pathlib import Path + +def _load_embedding_cache(): + base = Path(__file__).resolve().parent.parent.parent + spec = importlib.util.spec_from_file_location( + "embedding_cache", + base / "example-vault/plugins/smart_context/embedding_cache.py" + ) + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + return mod.EmbeddingCache + +def test_embedding_cache_miss_returns_none(tmp_path): + EmbeddingCache = _load_embedding_cache() + cache = EmbeddingCache(tmp_path, "chat_abc") + assert cache.get("msg1", "hello world") is None + +def test_embedding_cache_stores_and_retrieves(tmp_path): + EmbeddingCache = _load_embedding_cache() + cache = EmbeddingCache(tmp_path, "chat_abc") + cache.set("msg1", "hello world", [0.1, 0.2, 0.3]) + assert cache.get("msg1", "hello world") == [0.1, 0.2, 0.3] + +def test_embedding_cache_persists_across_instances(tmp_path): + EmbeddingCache = _load_embedding_cache() + EmbeddingCache(tmp_path, "chat_abc").set("msg1", "text", [1.0, 2.0]) + assert EmbeddingCache(tmp_path, "chat_abc").get("msg1", "text") == [1.0, 2.0] + +def test_embedding_cache_content_change_is_cache_miss(tmp_path): + EmbeddingCache = _load_embedding_cache() + cache = EmbeddingCache(tmp_path, "chat_abc") + cache.set("msg1", "original", [0.1, 0.2]) + assert cache.get("msg1", "modified") is None +``` + +**Step 2: Verify they fail** + +Run: `pixi run pytest sidecar/tests/test_plugin_smart_context.py -k "embedding_cache" -v` +Expected: FAIL — file not found + +**Step 3: Create `embedding_cache.py`** + +```python +"""Embedding cache backed by a JSON sidecar file.""" +import hashlib +import json +from pathlib import Path +from typing import Dict, List, Optional + + +class EmbeddingCache: + """Caches message embeddings in .memory/{chat_id}.embeddings.json.""" + + def __init__(self, memory_dir: Path, chat_id: str): + safe_id = "".join(c for c in chat_id if c.isalnum() or c in "-_") + self._path = memory_dir / f"{safe_id}.embeddings.json" + self._data: Dict[str, List[float]] = {} + self._load() + + def _load(self) -> None: + if self._path.exists(): + try: + with open(self._path, "r", encoding="utf-8") as f: + self._data = json.load(f) + except Exception: + self._data = {} + + def _save(self) -> None: + self._path.parent.mkdir(parents=True, exist_ok=True) + with open(self._path, "w", encoding="utf-8") as f: + json.dump(self._data, f) + + @staticmethod + def _key(message_id: str, content: str) -> str: + h = hashlib.md5(content.encode()).hexdigest()[:8] + return f"{message_id}:{h}" + + def get(self, message_id: str, content: str) -> Optional[List[float]]: + return self._data.get(self._key(message_id, content)) + + def set(self, message_id: str, content: str, embedding: List[float]) -> None: + self._data[self._key(message_id, content)] = embedding + self._save() +``` + +**Step 4: Verify all 4 tests pass** + +Run: `pixi run pytest sidecar/tests/test_plugin_smart_context.py -k "embedding_cache" -v` +Expected: 4 PASS + +**Step 5: Commit** + +```bash +git add example-vault/plugins/smart_context/embedding_cache.py sidecar/tests/test_plugin_smart_context.py +git commit -m "feat: EmbeddingCache for smart_context plugin" +``` + +--- + +### Task 3: Plugin Skeleton — Config, State, Subscriptions + +**Files:** +- Modify: `example-vault/plugins/smart_context/main.py` +- Modify: `sidecar/tests/test_plugin_smart_context.py` + +**Step 1: Write failing tests** + +```python +# sidecar/tests/test_plugin_smart_context.py +from unittest.mock import AsyncMock + +@pytest.mark.asyncio +async def test_plugin_loads_config_defaults(plugin_instance): + assert plugin_instance.similarity_threshold == 0.4 + assert plugin_instance.embedding_search is True + assert plugin_instance.active_topics == set() + +@pytest.mark.asyncio +async def test_on_load_subscribes_to_pipeline_events(plugin_instance, mock_brain): + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + await plugin_instance.on_load() + subscribed = [c[0][0] for c in mock_brain.subscribe.call_args_list] + assert "pipeline.output" in subscribed + assert "pipeline.context" in subscribed +``` + +**Step 2: Verify they fail** + +Run: `pixi run pytest sidecar/tests/test_plugin_smart_context.py::TestSmartContextPlugin::test_plugin_loads_config_defaults -v` +Expected: FAIL + +**Step 3: Rewrite `main.py`** + +```python +"""Smart Context Plugin — topic map + embedding-based context filtering.""" +import asyncio +import json +import math +import sys +from pathlib import Path +from typing import Dict, Any, List, Set + +# Make embedding_cache importable (same directory) +_plugin_dir = Path(__file__).parent +if str(_plugin_dir) not in sys.path: + sys.path.insert(0, str(_plugin_dir)) + +from embedding_cache import EmbeddingCache + +from sidecar.api.plugin_base import PluginBase +from sidecar.pipeline.events import PipelineEvents +from sidecar.pipeline.types import PipelineContext + + +def _cosine_similarity(a: List[float], b: List[float]) -> float: + dot = sum(x * y for x, y in zip(a, b)) + norm_a = math.sqrt(sum(x * x for x in a)) + norm_b = math.sqrt(sum(x * x for x in b)) + if norm_a == 0 or norm_b == 0: + return 0.0 + return dot / (norm_a * norm_b) + + +TOPIC_EXTRACTION_PROMPT = """\ +Analyze this conversation and extract the main topics discussed. +Return JSON only — no markdown, no prose. + +Format: +{ + "topics": [{"label": "Topic Name", "count": N}], + "sticky_message_ids": ["id1", "id2"] +} + +Rules: +- 3-8 topics max, 2-4 words each, Title Case +- count = approximate number of messages related to this topic +- sticky_message_ids: IDs of SHORT instructional/preference messages + (e.g. "be concise", "respond in French", "use Python 3") +- Return ONLY valid JSON""" + + +class Plugin(PluginBase): + """Smart Context Plugin.""" + + def __init__(self, plugin_dir: Path, vault_path: Path, config: Dict[str, Any] = None): + super().__init__(plugin_dir, vault_path, config) + self.panel_id = "smart-context-panel" + + self.similarity_threshold: float = float(self.config.get("similarity_threshold", 0.4)) + self.embedding_search: bool = bool(self.config.get("embedding_search", True)) + + self.active_topics: Set[str] = set() + self.current_chat_id: str = "" + + self.logger.info("Smart Context plugin initialized") + + def register_commands(self) -> None: + self.brain.register_command("smart_context.set_filter", self.set_filter, self.name) + self.brain.register_command("smart_context.get_topics", self.get_topics, self.name) + self.brain.register_command("smart_context.set_similarity_mode", self.set_similarity_mode, self.name) + + async def on_load(self) -> None: + await super().on_load() + self.subscribe(PipelineEvents.OUTPUT, self._on_pipeline_output) + self.subscribe(PipelineEvents.CONTEXT, self._on_pipeline_context) + self.logger.info("Smart Context plugin loaded") + + async def on_client_connected(self) -> None: + await self.register_panel( + panel_id=self.panel_id, + title="Smart Context", + icon="brain", + position="right", + ) + panel_file = self.plugin_dir / "panel.html" + html = panel_file.read_text(encoding="utf-8") if panel_file.exists() else \ + "

Waiting for context…

" + await self.set_panel_content(panel_id=self.panel_id, html_content=html) + + async def on_unload(self) -> None: + await self.remove_panel(self.panel_id) + await super().on_unload() + + # ========================================================================= + # Pipeline Subscribers (stubs — filled in Tasks 4 & 5) + # ========================================================================= + + async def _on_pipeline_output(self, ctx: PipelineContext) -> None: + pass + + async def _on_pipeline_context(self, ctx: PipelineContext) -> None: + pass + + # ========================================================================= + # Commands (stubs — filled in Task 6) + # ========================================================================= + + async def set_filter(self, topics: List[str] = None, **kwargs) -> Dict[str, Any]: + return {"status": "success"} + + async def get_topics(self, chat_id: str = "", **kwargs) -> Dict[str, Any]: + return {"status": "success", "topics": []} + + async def set_similarity_mode(self, enabled: bool = True, **kwargs) -> Dict[str, Any]: + return {"status": "success"} +``` + +**Step 4: Run all existing tests** + +Run: `pixi run pytest sidecar/tests/test_plugin_smart_context.py -v` +Expected: All PASS (new tests + pre-existing panel tests) + +**Step 5: Commit** + +```bash +git add example-vault/plugins/smart_context/main.py sidecar/tests/test_plugin_smart_context.py +git commit -m "feat: smart_context skeleton — config, state, subscriptions" +``` + +--- + +### Task 4: Topic Extraction (`pipeline.output` subscriber) + +**Files:** +- Modify: `example-vault/plugins/smart_context/main.py` +- Modify: `sidecar/tests/test_plugin_smart_context.py` + +**Step 1: Write failing test** + +```python +import json +from unittest.mock import AsyncMock, MagicMock, patch +from sidecar.pipeline.types import PipelineContext + +@pytest.mark.asyncio +async def test_extract_topics_parses_llm_response(plugin_instance, mock_brain): + messages = [ + {"id": "a1b2", "role": "user", "content": "be concise"}, + {"id": "c3d4", "role": "user", "content": "How does async work in Python?"}, + {"id": "e5f6", "role": "assistant", "content": "Async uses coroutines..."}, + ] + + mock_response = MagicMock() + mock_response.content = json.dumps({ + "topics": [{"label": "Python Async", "count": 2}], + "sticky_message_ids": ["a1b2"] + }) + + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + with patch("sidecar.services.llm_service.get_llm_service") as mock_get_llm: + mock_llm = MagicMock() + mock_llm.complete = AsyncMock(return_value=mock_response) + mock_get_llm.return_value = mock_llm + result = await plugin_instance._extract_topics(messages) + + labels = [t["label"] for t in result] + assert "Python Async" in labels + assert "Instructions & Preferences" in labels + sticky = next(t for t in result if t.get("sticky")) + assert sticky["message_ids"] == ["a1b2"] + assert sticky["count"] == 1 +``` + +**Step 2: Verify it fails** + +Run: `pixi run pytest sidecar/tests/test_plugin_smart_context.py::TestSmartContextPlugin::test_extract_topics_parses_llm_response -v` +Expected: FAIL — `_extract_topics` not defined + +**Step 3: Implement `_extract_topics` and `_on_pipeline_output`** + +Replace the `_on_pipeline_output` stub and add new methods in `main.py`: + +```python +async def _extract_topics(self, messages: List[Dict[str, Any]]) -> List[Dict[str, Any]]: + """Call LLM to extract topics and identify sticky messages.""" + from sidecar.services.llm_service import get_llm_service + + conversation = "\n".join( + f"[{m.get('id', '?')}] {m.get('role', 'user')}: " + f"{str(m.get('content', ''))[:200]}" + for m in messages + ) + + llm_messages = [ + {"role": "system", "content": TOPIC_EXTRACTION_PROMPT}, + {"role": "user", "content": conversation}, + ] + + llm = get_llm_service() + category = self.config.get("extraction_category", "fast") + response = await llm.complete( + messages=llm_messages, + category=category, + max_tokens=512, + temperature=0.2, + ) + + raw = response.content.strip() + if raw.startswith("```"): + raw = "\n".join(raw.split("\n")[1:]) + if raw.endswith("```"): + raw = raw[:-3] + + data = json.loads(raw) + topics: List[Dict[str, Any]] = data.get("topics", []) + sticky_ids: List[str] = data.get("sticky_message_ids", []) + + if sticky_ids: + topics.append({ + "label": "Instructions & Preferences", + "sticky": True, + "message_ids": sticky_ids, + "count": len(sticky_ids), + }) + + return topics + + +async def _on_pipeline_output(self, ctx: PipelineContext) -> None: + """After each LLM response: extract topics in background.""" + if not ctx.response: + return + chat_id = ctx.metadata.get("chat_id") + if not chat_id: + return + asyncio.create_task(self._run_topic_extraction(chat_id)) + + +async def _run_topic_extraction(self, chat_id: str) -> None: + """Background: extract topics from chat and save + notify panel.""" + try: + result = await self.brain.execute_command("memory.load_chat", chat_id=chat_id) + if result.get("status") != "success": + return + data = result.get("data", {}) + messages = data.get("messages", []) + if not messages: + return + + topics = await self._extract_topics(messages) + data["topics"] = topics + await self.brain.execute_command("memory.save_chat", chat_id=chat_id, data=data) + + self.current_chat_id = chat_id + self.emit("smart_context.topics_updated", { + "chat_id": chat_id, + "topics": topics, + "total_messages": len(messages), + }) + self.logger.info(f"Topics extracted for {chat_id}: {[t['label'] for t in topics]}") + except Exception as e: + self.logger.error(f"Topic extraction failed for {chat_id}: {e}") +``` + +**Step 4: Run tests** + +Run: `pixi run pytest sidecar/tests/test_plugin_smart_context.py -v` +Expected: All PASS + +**Step 5: Commit** + +```bash +git add example-vault/plugins/smart_context/main.py sidecar/tests/test_plugin_smart_context.py +git commit -m "feat: smart_context topic extraction via pipeline.output" +``` + +--- + +### Task 5: Context Injection (`pipeline.context` subscriber) + +**Files:** +- Modify: `example-vault/plugins/smart_context/main.py` +- Modify: `sidecar/tests/test_plugin_smart_context.py` + +**Step 1: Write failing tests** + +```python +@pytest.mark.asyncio +async def test_context_injection_passthrough_when_no_active_topics(plugin_instance, mock_brain): + plugin_instance.active_topics = set() + ctx = PipelineContext(message="hi", original_message="hi", metadata={}) + ctx.history = [{"id": "1", "role": "user", "content": "hello"}] + await plugin_instance._on_pipeline_context(ctx) + assert len(ctx.history) == 1 # unchanged + + +@pytest.mark.asyncio +async def test_context_injection_filters_by_similarity(plugin_instance, mock_brain): + plugin_instance.active_topics = {"Python Async"} + plugin_instance.embedding_search = True + plugin_instance.similarity_threshold = 0.7 + + ctx = PipelineContext( + message="more?", original_message="more?", + metadata={"chat_id": "chat_test"}, + ) + ctx.history = [ + {"id": "m1", "role": "user", "content": "How does async work in Python?"}, + {"id": "m2", "role": "user", "content": "What is the capital of France?"}, + ] + + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + mock_brain.execute_command = AsyncMock(return_value={ + "status": "success", + "data": {"messages": ctx.history, "topics": [{"label": "Python Async", "count": 1}]} + }) + with patch("sidecar.services.llm_service.get_llm_service") as mock_get_llm: + mock_llm = MagicMock() + mock_llm.embed = AsyncMock(side_effect=[ + [[0.9, 0.1]], # topic embedding + [[0.85, 0.15], [0.1, 0.95]] # m1=similar, m2=dissimilar + ]) + mock_get_llm.return_value = mock_llm + await plugin_instance._on_pipeline_context(ctx) + + assert len(ctx.history) == 1 + assert ctx.history[0]["id"] == "m1" + + +@pytest.mark.asyncio +async def test_context_injection_always_includes_sticky(plugin_instance, mock_brain): + plugin_instance.active_topics = {"Python Async"} + plugin_instance.embedding_search = True + plugin_instance.similarity_threshold = 0.99 # impossibly high + + ctx = PipelineContext(message="q", original_message="q", metadata={"chat_id": "c1"}) + ctx.history = [ + {"id": "sticky1", "role": "user", "content": "be concise"}, + {"id": "m1", "role": "user", "content": "something about python async"}, + ] + + topics_data = {"messages": ctx.history, "topics": [ + {"label": "Python Async", "count": 1}, + {"label": "Instructions & Preferences", "sticky": True, + "message_ids": ["sticky1"], "count": 1}, + ]} + + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + mock_brain.execute_command = AsyncMock( + return_value={"status": "success", "data": topics_data} + ) + with patch("sidecar.services.llm_service.get_llm_service") as mock_get_llm: + mock_llm = MagicMock() + mock_llm.embed = AsyncMock(side_effect=[ + [[1.0, 0.0]], # topic + [[0.1, 0.9], [0.0, 1.0]] # both low similarity + ]) + mock_get_llm.return_value = mock_llm + await plugin_instance._on_pipeline_context(ctx) + + assert any(m["id"] == "sticky1" for m in ctx.history) + + +@pytest.mark.asyncio +async def test_context_injection_fallback_when_threshold_met_by_nothing(plugin_instance, mock_brain): + """If nothing passes threshold (excluding sticky), return full history.""" + plugin_instance.active_topics = {"Exotic Topic"} + plugin_instance.embedding_search = True + plugin_instance.similarity_threshold = 0.99 + + ctx = PipelineContext(message="q", original_message="q", metadata={"chat_id": "c1"}) + original_history = [{"id": "m1", "role": "user", "content": "hello"}] + ctx.history = original_history.copy() + + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + mock_brain.execute_command = AsyncMock(return_value={ + "status": "success", + "data": {"messages": original_history, "topics": []} + }) + with patch("sidecar.services.llm_service.get_llm_service") as mock_get_llm: + mock_llm = MagicMock() + mock_llm.embed = AsyncMock(side_effect=[[[1.0, 0.0]], [[0.1, 0.9]]]) + mock_get_llm.return_value = mock_llm + await plugin_instance._on_pipeline_context(ctx) + + assert len(ctx.history) == 1 # full history preserved +``` + +**Step 2: Verify they fail** + +Run: `pixi run pytest sidecar/tests/test_plugin_smart_context.py -k "context_injection" -v` +Expected: All FAIL + +**Step 3: Implement shared helper + `_on_pipeline_context`** + +Add `_compute_relevant_ids` (shared by context injection and highlight computation) and replace the stub: + +```python +async def _compute_relevant_ids( + self, chat_id: str, messages: List[Dict[str, Any]], sticky_ids: Set[str] +) -> List[str]: + """ + Return IDs of messages relevant to active_topics via cosine similarity. + Sticky IDs are always included. Returns empty list if nothing passes threshold + (caller should fall back to full history). + """ + from sidecar.services.llm_service import get_llm_service + + llm = get_llm_service() + topic_embeddings = await llm.embed(list(self.active_topics)) + + cache = EmbeddingCache(self.vault_path / ".memory", chat_id) + need_embed = [] + cached: Dict[str, List[float]] = {} + + for msg in messages: + msg_id = msg.get("id", "") + content = str(msg.get("content", "")) + emb = cache.get(msg_id, content) + if emb is not None: + cached[msg_id] = emb + else: + need_embed.append(msg) + + if need_embed: + texts = [str(m.get("content", "")) for m in need_embed] + new_embeddings = await llm.embed(texts) + for msg, emb in zip(need_embed, new_embeddings): + msg_id = msg.get("id", "") + content = str(msg.get("content", "")) + cache.set(msg_id, content, emb) + cached[msg_id] = emb + + included: List[str] = [] + for msg in messages: + msg_id = msg.get("id", "") + if msg_id in sticky_ids: + included.append(msg_id) + continue + emb = cached.get(msg_id) + if emb is None: + continue + max_sim = max(_cosine_similarity(emb, t_emb) for t_emb in topic_embeddings) + if max_sim >= self.similarity_threshold: + included.append(msg_id) + + return included + + +async def _on_pipeline_context(self, ctx: PipelineContext) -> None: + """Filter ctx.history using embedding similarity to active topics.""" + if not self.active_topics or not self.embedding_search: + return + + chat_id = ctx.metadata.get("chat_id") + if not chat_id or not ctx.history: + return + + try: + result = await self.brain.execute_command("memory.load_chat", chat_id=chat_id) + data = result.get("data", {}) + topics_list = data.get("topics", []) + + sticky_ids: Set[str] = set() + for t in topics_list: + if t.get("sticky"): + sticky_ids.update(t.get("message_ids", [])) + + included = await self._compute_relevant_ids(chat_id, ctx.history, sticky_ids) + + non_sticky_included = [i for i in included if i not in sticky_ids] + if not non_sticky_included: + return # nothing passed threshold — keep full history + + included_set = set(included) + ctx.history = [m for m in ctx.history if m.get("id", "") in included_set] + + except Exception as e: + self.logger.error(f"Context injection failed: {e}") +``` + +**Step 4: Run tests** + +Run: `pixi run pytest sidecar/tests/test_plugin_smart_context.py -v` +Expected: All PASS + +**Step 5: Commit** + +```bash +git add example-vault/plugins/smart_context/main.py sidecar/tests/test_plugin_smart_context.py +git commit -m "feat: smart_context context injection with embedding similarity" +``` + +--- + +### Task 6: Commands — `set_filter`, `get_topics`, `set_similarity_mode` + +**Files:** +- Modify: `example-vault/plugins/smart_context/main.py` +- Modify: `sidecar/tests/test_plugin_smart_context.py` + +**Step 1: Write failing tests** + +```python +@pytest.mark.asyncio +async def test_set_filter_updates_active_topics_and_emits(plugin_instance, mock_brain): + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + result = await plugin_instance.set_filter(topics=["Python Async", "LangGraph"]) + assert plugin_instance.active_topics == {"Python Async", "LangGraph"} + assert result["status"] == "success" + mock_brain.emit_to_frontend.assert_called() + + +@pytest.mark.asyncio +async def test_set_filter_empty_clears(plugin_instance, mock_brain): + plugin_instance.active_topics = {"Python Async"} + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + await plugin_instance.set_filter(topics=[]) + assert plugin_instance.active_topics == set() + + +@pytest.mark.asyncio +async def test_get_topics_reads_from_chat_file(plugin_instance, mock_brain): + mock_brain.execute_command = AsyncMock(return_value={ + "status": "success", + "data": { + "messages": [{"id": "1"}], + "topics": [{"label": "Python", "count": 3}] + } + }) + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + result = await plugin_instance.get_topics(chat_id="chat_123") + assert result["status"] == "success" + assert result["topics"][0]["label"] == "Python" + assert result["total_messages"] == 1 + + +@pytest.mark.asyncio +async def test_set_similarity_mode_toggles(plugin_instance, mock_brain): + plugin_instance.embedding_search = True + result = await plugin_instance.set_similarity_mode(enabled=False) + assert plugin_instance.embedding_search is False + assert result["status"] == "success" +``` + +**Step 2: Verify they fail** + +Run: `pixi run pytest sidecar/tests/test_plugin_smart_context.py -k "test_set_filter or test_get_topics or test_set_similarity" -v` +Expected: FAIL (stubs return success but don't update state) + +**Step 3: Implement commands** + +Replace the three stub command methods in `main.py`: + +```python +async def set_filter(self, topics: List[str] = None, **kwargs) -> Dict[str, Any]: + """Set active topic filter. Empty list clears it.""" + if topics is None: + p = kwargs.get("p") or kwargs.get("params", {}) + topics = p.get("topics", []) + + self.active_topics = set(topics) + self.emit("smart_context.filter_changed", { + "active_topics": list(self.active_topics), + "chat_id": self.current_chat_id, + }) + + if self.current_chat_id and self.active_topics and self.embedding_search: + asyncio.create_task(self._emit_highlight_ids(self.current_chat_id)) + + return {"status": "success", "active_topics": list(self.active_topics)} + + +async def get_topics(self, chat_id: str = "", **kwargs) -> Dict[str, Any]: + """Return topics for the given chat.""" + if not chat_id: + p = kwargs.get("p") or kwargs.get("params", {}) + chat_id = p.get("chat_id", self.current_chat_id) + + if not chat_id: + return {"status": "success", "topics": [], "total_messages": 0} + + result = await self.brain.execute_command("memory.load_chat", chat_id=chat_id) + if result.get("status") != "success": + return {"status": "success", "topics": [], "total_messages": 0} + + data = result.get("data", {}) + return { + "status": "success", + "topics": data.get("topics", []), + "total_messages": len(data.get("messages", [])), + } + + +async def set_similarity_mode(self, enabled: bool = True, **kwargs) -> Dict[str, Any]: + """Toggle embedding-based context filtering.""" + if not isinstance(enabled, bool): + p = kwargs.get("p") or kwargs.get("params", {}) + enabled = bool(p.get("enabled", True)) + self.embedding_search = enabled + if not enabled: + self.active_topics.clear() + self.emit("smart_context.filter_changed", { + "active_topics": [], + "chat_id": self.current_chat_id, + }) + return {"status": "success", "embedding_search": self.embedding_search} + + +async def _emit_highlight_ids(self, chat_id: str) -> None: + """Background: compute filtered message IDs and emit for chat highlighting.""" + try: + result = await self.brain.execute_command("memory.load_chat", chat_id=chat_id) + data = result.get("data", {}) + messages = data.get("messages", []) + topics_list = data.get("topics", []) + + sticky_ids: Set[str] = set() + for t in topics_list: + if t.get("sticky"): + sticky_ids.update(t.get("message_ids", [])) + + included = await self._compute_relevant_ids(chat_id, messages, sticky_ids) + self.emit("smart_context.highlight_applied", {"message_ids": included}) + except Exception as e: + self.logger.error(f"Highlight computation failed: {e}") +``` + +**Step 4: Run tests** + +Run: `pixi run pytest sidecar/tests/test_plugin_smart_context.py -v` +Expected: All PASS + +**Step 5: Commit** + +```bash +git add example-vault/plugins/smart_context/main.py sidecar/tests/test_plugin_smart_context.py +git commit -m "feat: smart_context commands — set_filter, get_topics, set_similarity_mode" +``` + +--- + +### Task 7: Panel HTML + +**Files:** +- Create: `example-vault/plugins/smart_context/panel.html` +- Modify: `sidecar/tests/test_plugin_smart_context.py` + +**Step 1: Write failing test** + +```python +@pytest.mark.asyncio +async def test_on_client_connected_loads_panel_html_when_exists(plugin_instance, mock_brain): + (plugin_instance.plugin_dir / "panel.html").write_text("
test
") + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + await plugin_instance.on_client_connected() + set_calls = [ + c for c in mock_brain.emit_to_frontend.call_args_list + if c[1].get("data", {}).get("action") == "set_panel" + ] + assert any("sc-panel" in c[1]["data"]["html"] for c in set_calls) +``` + +**Step 2: Verify it fails** + +Run: `pixi run pytest sidecar/tests/test_plugin_smart_context.py::TestSmartContextPlugin::test_on_client_connected_loads_panel_html_when_exists -v` +Expected: FAIL — test file content `sc-panel` not found in HTML (fallback placeholder used instead) + +**Step 3: Create `panel.html`** + +```html + + + + + + +
+ +
+ 🔒 + Instructions always in context + 0 +
+ +
+ + +
+ +
+ Waiting for context… +
+ +
All messages in context
+ +
+ + + +``` + +**Step 4: Run tests** + +Run: `pixi run pytest sidecar/tests/test_plugin_smart_context.py -v` +Expected: All PASS + +**Step 5: Commit** + +```bash +git add example-vault/plugins/smart_context/panel.html sidecar/tests/test_plugin_smart_context.py +git commit -m "feat: smart_context bubble cloud panel HTML/JS" +``` + +--- + +### Task 8: Chat Message Highlighting (Frontend) + +**Files:** +- Modify: `src/vault/chat/chat.js` +- Modify the main CSS file (find it under `src/` — check for `style.css` or `index.css`) + +**Step 1: Locate the CSS file** + +Run: `find src -name "*.css" | head -5` + +Open the file and add the highlight class: + +```css +/* Smart Context — active filter highlight */ +.sc-highlight { + border-left: 3px solid rgba(99, 179, 237, 0.7) !important; + background: rgba(99, 179, 237, 0.05) !important; + transition: background 0.2s, border-left 0.2s; +} +``` + +**Step 2: Add listeners to `chat.js`** + +Open `src/vault/chat/chat.js`. Find the `initChatGlobals` function (or the block that sets up `window.addEventListener` calls). Add after existing listeners: + +```javascript +// Smart Context: highlight messages included in current filter +window.addEventListener('smart_context.highlight_applied', function(e) { + var messageIds = (e.detail && e.detail.message_ids) ? e.detail.message_ids : []; + + // Remove all existing highlights + document.querySelectorAll('[data-message-id].sc-highlight') + .forEach(function(el) { el.classList.remove('sc-highlight'); }); + + // Apply to matching elements + messageIds.forEach(function(id) { + var el = document.querySelector('[data-message-id="' + id + '"]'); + if (el) el.classList.add('sc-highlight'); + }); +}); + +// Clear all highlights when filter is cleared +window.addEventListener('smart_context.filter_changed', function(e) { + var active = (e.detail && e.detail.active_topics) ? e.detail.active_topics : []; + if (active.length === 0) { + document.querySelectorAll('[data-message-id].sc-highlight') + .forEach(function(el) { el.classList.remove('sc-highlight'); }); + } +}); +``` + +**Step 3: Verify with manual test** + +Start the sidecar: +```bash +pixi run sidecar --vault example-vault --ws-port 9001 --verbose +``` + +Open the UI, send a few messages on different topics, then click a topic bubble. Messages in the chat matching the filter should get a left-border highlight. Messages not matching should be unhighlighted. + +**Step 4: Commit** + +```bash +git add src/vault/chat/chat.js src/ +git commit -m "feat: chat message highlighting for smart_context filter" +``` + +--- + +### Task 9: Update `.vault.toml` and Run All Tests + +**Files:** +- Modify: `example-vault/.vault.toml` (confirm smart_context is enabled with defaults) + +**Step 1: Confirm vault config** + +The `[plugins.smart_context]` section should already exist (from before). Verify it reads: + +```toml +[plugins.smart_context] +enabled = true +``` + +Optionally add defaults explicitly: + +```toml +[plugins.smart_context] +enabled = true +similarity_threshold = 0.4 +embedding_search = true +``` + +**Step 2: Run full test suite** + +Run: `pixi run test` +Expected: All sidecar tests PASS, no regressions + +**Step 3: Run smart_context tests specifically** + +Run: `pixi run pytest sidecar/tests/test_plugin_smart_context.py -v` +Expected: All PASS + +**Step 4: Commit** + +```bash +git add example-vault/.vault.toml +git commit -m "chore: confirm smart_context vault config" +``` + +--- + +## Summary of New Files + +| File | Purpose | +|------|---------| +| `example-vault/plugins/smart_context/main.py` | Plugin: full implementation | +| `example-vault/plugins/smart_context/panel.html` | Bubble cloud UI | +| `example-vault/plugins/smart_context/embedding_cache.py` | Embedding cache | +| `sidecar/tests/test_plugin_smart_context.py` | Tests (replace existing stub tests) | + +## Modified Files + +| File | Change | +|------|--------| +| `sidecar/services/llm_service.py` | Add `embed()` method | +| `sidecar/tests/test_llm_service.py` | Add embed test | +| `src/vault/chat/chat.js` | Add highlight listeners | +| `src/` (CSS file) | Add `.sc-highlight` class | +| `example-vault/.vault.toml` | Confirm plugin config | From dcd88529c1948f87faf81cccec9851c27f537d2a Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 12:40:34 +0530 Subject: [PATCH 03/21] feat: add embed() method to LLMService Co-Authored-By: Claude Sonnet 4.6 --- sidecar/services/llm_service.py | 17 +++++++++++++++++ sidecar/tests/test_llm_service.py | 21 +++++++++++++++++++++ 2 files changed, 38 insertions(+) diff --git a/sidecar/services/llm_service.py b/sidecar/services/llm_service.py index bd4be45..0eeb264 100644 --- a/sidecar/services/llm_service.py +++ b/sidecar/services/llm_service.py @@ -514,6 +514,23 @@ async def _stream_completion( self._logger.error(f"Stream completion failed: {e}") raise + async def embed( + self, + texts: List[str], + category: str = "embedding", + ) -> List[List[float]]: + """Generate embeddings for a list of texts via LiteLLM.""" + from litellm import aembedding + + model_id = self.get_model_for_category(category) + if not model_id: + raise ValueError(f"No model configured for category: {category}") + + litellm_model = self._format_model_for_litellm(model_id) + self._logger.debug(f"Embedding {len(texts)} texts with {litellm_model}") + response = await aembedding(model=litellm_model, input=texts) + return [item["embedding"] for item in response.data] + def _format_model_for_litellm(self, model_id: str) -> str: """ Format a model ID for LiteLLM. diff --git a/sidecar/tests/test_llm_service.py b/sidecar/tests/test_llm_service.py index a8b604e..b84fd51 100644 --- a/sidecar/tests/test_llm_service.py +++ b/sidecar/tests/test_llm_service.py @@ -3,6 +3,7 @@ """ import pytest +from pathlib import Path from unittest.mock import MagicMock, AsyncMock, patch from sidecar.services.llm_service import LLMService @@ -139,6 +140,26 @@ async def test_guardrails_o1(llm_service): assert "top_p" not in args.kwargs +@pytest.mark.asyncio +async def test_embed_returns_list_of_vectors(): + with patch("litellm.aembedding") as mock_embed: + mock_embed.return_value = MagicMock( + data=[ + {"embedding": [0.1, 0.2, 0.3]}, + {"embedding": [0.4, 0.5, 0.6]}, + ] + ) + service = LLMService( + Path("/tmp"), + {"categories": {"embedding": "openai/text-embedding-3-large"}} + ) + result = await service.embed(["hello", "world"]) + + assert len(result) == 2 + assert result[0] == [0.1, 0.2, 0.3] + assert result[1] == [0.4, 0.5, 0.6] + + @pytest.mark.asyncio async def test_detect_ollama(llm_service): # Mock httpx From 09895aecacc57df79c1d5cb38f4581cd90cacea1 Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 12:46:54 +0530 Subject: [PATCH 04/21] fix: use module-level aembedding import and add negative embed test Co-Authored-By: Claude Sonnet 4.6 --- sidecar/services/llm_service.py | 4 +--- sidecar/tests/test_llm_service.py | 8 +++++++- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/sidecar/services/llm_service.py b/sidecar/services/llm_service.py index 0eeb264..8ac3de9 100644 --- a/sidecar/services/llm_service.py +++ b/sidecar/services/llm_service.py @@ -17,7 +17,7 @@ from loguru import logger import litellm -from litellm import acompletion +from litellm import acompletion, aembedding # Suppress debug info (e.g. "Provider List" link) but keep errors litellm.suppress_debug_info = True @@ -520,8 +520,6 @@ async def embed( category: str = "embedding", ) -> List[List[float]]: """Generate embeddings for a list of texts via LiteLLM.""" - from litellm import aembedding - model_id = self.get_model_for_category(category) if not model_id: raise ValueError(f"No model configured for category: {category}") diff --git a/sidecar/tests/test_llm_service.py b/sidecar/tests/test_llm_service.py index b84fd51..b3b441d 100644 --- a/sidecar/tests/test_llm_service.py +++ b/sidecar/tests/test_llm_service.py @@ -142,7 +142,7 @@ async def test_guardrails_o1(llm_service): @pytest.mark.asyncio async def test_embed_returns_list_of_vectors(): - with patch("litellm.aembedding") as mock_embed: + with patch("sidecar.services.llm_service.aembedding") as mock_embed: mock_embed.return_value = MagicMock( data=[ {"embedding": [0.1, 0.2, 0.3]}, @@ -160,6 +160,12 @@ async def test_embed_returns_list_of_vectors(): assert result[1] == [0.4, 0.5, 0.6] +@pytest.mark.asyncio +async def test_embed_raises_when_no_embedding_model(llm_service): + with pytest.raises(ValueError, match="No model configured for category: embedding"): + await llm_service.embed(["hello"]) + + @pytest.mark.asyncio async def test_detect_ollama(llm_service): # Mock httpx From 98eed68971fdd56bff5d8744470f437fdd308d16 Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 12:57:48 +0530 Subject: [PATCH 05/21] feat: EmbeddingCache for smart_context plugin Co-Authored-By: Claude Sonnet 4.6 --- .../plugins/smart_context/embedding_cache.py | 40 +++++++++++++++++++ sidecar/tests/test_plugin_smart_context.py | 33 +++++++++++++++ 2 files changed, 73 insertions(+) create mode 100644 example-vault/plugins/smart_context/embedding_cache.py diff --git a/example-vault/plugins/smart_context/embedding_cache.py b/example-vault/plugins/smart_context/embedding_cache.py new file mode 100644 index 0000000..5ef6447 --- /dev/null +++ b/example-vault/plugins/smart_context/embedding_cache.py @@ -0,0 +1,40 @@ +"""Embedding cache backed by a JSON sidecar file.""" +import hashlib +import json +from pathlib import Path +from typing import Dict, List, Optional + + +class EmbeddingCache: + """Caches message embeddings in .memory/{chat_id}.embeddings.json.""" + + def __init__(self, memory_dir: Path, chat_id: str): + safe_id = "".join(c for c in chat_id if c.isalnum() or c in "-_") + self._path = memory_dir / f"{safe_id}.embeddings.json" + self._data: Dict[str, List[float]] = {} + self._load() + + def _load(self) -> None: + if self._path.exists(): + try: + with open(self._path, "r", encoding="utf-8") as f: + self._data = json.load(f) + except Exception: + self._data = {} + + def _save(self) -> None: + self._path.parent.mkdir(parents=True, exist_ok=True) + with open(self._path, "w", encoding="utf-8") as f: + json.dump(self._data, f) + + @staticmethod + def _key(message_id: str, content: str) -> str: + h = hashlib.md5(content.encode()).hexdigest()[:8] + return f"{message_id}:{h}" + + def get(self, message_id: str, content: str) -> Optional[List[float]]: + return self._data.get(self._key(message_id, content)) + + def set(self, message_id: str, content: str, embedding: List[float]) -> None: + self._data[self._key(message_id, content)] = embedding + self._save() diff --git a/sidecar/tests/test_plugin_smart_context.py b/sidecar/tests/test_plugin_smart_context.py index 40932a8..35d9c15 100644 --- a/sidecar/tests/test_plugin_smart_context.py +++ b/sidecar/tests/test_plugin_smart_context.py @@ -109,3 +109,36 @@ async def test_on_unload_removes_panel(self, plugin_instance, mock_brain): assert remove_call is not None assert remove_call["id"] == "smart-context-panel" + + +def _load_embedding_cache(): + base = Path(__file__).resolve().parent.parent.parent + spec = importlib.util.spec_from_file_location( + "embedding_cache", + base / "example-vault/plugins/smart_context/embedding_cache.py" + ) + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + return mod.EmbeddingCache + +def test_embedding_cache_miss_returns_none(tmp_path): + EmbeddingCache = _load_embedding_cache() + cache = EmbeddingCache(tmp_path, "chat_abc") + assert cache.get("msg1", "hello world") is None + +def test_embedding_cache_stores_and_retrieves(tmp_path): + EmbeddingCache = _load_embedding_cache() + cache = EmbeddingCache(tmp_path, "chat_abc") + cache.set("msg1", "hello world", [0.1, 0.2, 0.3]) + assert cache.get("msg1", "hello world") == [0.1, 0.2, 0.3] + +def test_embedding_cache_persists_across_instances(tmp_path): + EmbeddingCache = _load_embedding_cache() + EmbeddingCache(tmp_path, "chat_abc").set("msg1", "text", [1.0, 2.0]) + assert EmbeddingCache(tmp_path, "chat_abc").get("msg1", "text") == [1.0, 2.0] + +def test_embedding_cache_content_change_is_cache_miss(tmp_path): + EmbeddingCache = _load_embedding_cache() + cache = EmbeddingCache(tmp_path, "chat_abc") + cache.set("msg1", "original", [0.1, 0.2]) + assert cache.get("msg1", "modified") is None From b55ea6b7304378b037f65a02ff199a6539bc3d5a Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 13:05:11 +0530 Subject: [PATCH 06/21] =?UTF-8?q?feat:=20smart=5Fcontext=20skeleton=20?= =?UTF-8?q?=E2=80=94=20config,=20state,=20subscriptions?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Sonnet 4.6 --- example-vault/plugins/smart_context/main.py | 140 +++++++++++++------- sidecar/tests/test_plugin_smart_context.py | 16 ++- 2 files changed, 107 insertions(+), 49 deletions(-) diff --git a/example-vault/plugins/smart_context/main.py b/example-vault/plugins/smart_context/main.py index 8d7cc8a..94efcf0 100644 --- a/example-vault/plugins/smart_context/main.py +++ b/example-vault/plugins/smart_context/main.py @@ -1,67 +1,111 @@ -""" -Smart Context Plugin -""" - +"""Smart Context Plugin — topic map + embedding-based context filtering.""" +import asyncio +import json +import math import sys from pathlib import Path -from typing import Dict, Any +from typing import Dict, Any, List, Set + +# Make embedding_cache importable (same directory) +_plugin_dir = Path(__file__).parent +if str(_plugin_dir) not in sys.path: + sys.path.insert(0, str(_plugin_dir)) -# Add tailor root to path for imports if needed, though usually available in plugin context -tailor_path = Path(__file__).resolve().parent.parent.parent.parent -if str(tailor_path) not in sys.path: - sys.path.insert(0, str(tailor_path)) +from embedding_cache import EmbeddingCache from sidecar.api.plugin_base import PluginBase -from sidecar.constants import EventType +from sidecar.pipeline.events import PipelineEvents +from sidecar.pipeline.types import PipelineContext + + +def _cosine_similarity(a: List[float], b: List[float]) -> float: + dot = sum(x * y for x, y in zip(a, b)) + norm_a = math.sqrt(sum(x * x for x in a)) + norm_b = math.sqrt(sum(x * x for x in b)) + if norm_a == 0 or norm_b == 0: + return 0.0 + return dot / (norm_a * norm_b) + + +TOPIC_EXTRACTION_PROMPT = """\ +Analyze this conversation and extract the main topics discussed. +Return JSON only — no markdown, no prose. + +Format: +{ + "topics": [{"label": "Topic Name", "count": N}], + "sticky_message_ids": ["id1", "id2"] +} + +Rules: +- 3-8 topics max, 2-4 words each, Title Case +- count = approximate number of messages related to this topic +- sticky_message_ids: IDs of SHORT instructional/preference messages + (e.g. "be concise", "respond in French", "use Python 3") +- Return ONLY valid JSON""" + class Plugin(PluginBase): - """ - Smart Context Plugin. - Renders a panel in the UI. - """ - - def __init__( - self, - plugin_dir: Path, - vault_path: Path, - config: Dict[str, Any] = None - ): + """Smart Context Plugin.""" + + def __init__(self, plugin_dir: Path, vault_path: Path, config: Dict[str, Any] = None): super().__init__(plugin_dir, vault_path, config) - self.logger.info("Smart Context plugin initialized") self.panel_id = "smart-context-panel" - + + self.similarity_threshold: float = float(self.config.get("similarity_threshold", 0.4)) + self.embedding_search: bool = bool(self.config.get("embedding_search", True)) + + self.active_topics: Set[str] = set() + self.current_chat_id: str = "" + + self.logger.info("Smart Context plugin initialized") + def register_commands(self) -> None: - """Register plugin commands.""" - # No commands to register yet - pass - + self.brain.register_command("smart_context.set_filter", self.set_filter, self.name) + self.brain.register_command("smart_context.get_topics", self.get_topics, self.name) + self.brain.register_command("smart_context.set_similarity_mode", self.set_similarity_mode, self.name) + + async def on_load(self) -> None: + await super().on_load() + self.subscribe(PipelineEvents.OUTPUT, self._on_pipeline_output) + self.subscribe(PipelineEvents.CONTEXT, self._on_pipeline_context) + self.logger.info("Smart Context plugin loaded") + async def on_client_connected(self) -> None: - """Called when frontend connects - register UI elements.""" - self.logger.info("Client connected - registering Smart Context panel") - - # Register the panel await self.register_panel( panel_id=self.panel_id, title="Smart Context", - icon="brain", - position="right" - ) - - # Set initial content - await self.set_panel_content( - panel_id=self.panel_id, - html_content="

Smart Context

Waiting for context...

" + icon="brain", + position="right", ) - - self.logger.info("Registered Smart Context panel") - - async def on_load(self) -> None: - """Called after plugin is loaded.""" - await super().on_load() - self.logger.info("Smart Context plugin loaded") + panel_file = self.plugin_dir / "panel.html" + html = panel_file.read_text(encoding="utf-8") if panel_file.exists() else \ + "

Waiting for context\u2026

" + await self.set_panel_content(panel_id=self.panel_id, html_content=html) async def on_unload(self) -> None: - """Called when plugin is unloaded.""" - self.logger.info("Smart Context plugin unloading") await self.remove_panel(self.panel_id) await super().on_unload() + + # ========================================================================= + # Pipeline Subscribers (stubs — filled in Tasks 4 & 5) + # ========================================================================= + + async def _on_pipeline_output(self, ctx: PipelineContext) -> None: + pass + + async def _on_pipeline_context(self, ctx: PipelineContext) -> None: + pass + + # ========================================================================= + # Commands (stubs — filled in Task 6) + # ========================================================================= + + async def set_filter(self, topics: List[str] = None, **kwargs) -> Dict[str, Any]: + return {"status": "success"} + + async def get_topics(self, chat_id: str = "", **kwargs) -> Dict[str, Any]: + return {"status": "success", "topics": []} + + async def set_similarity_mode(self, enabled: bool = True, **kwargs) -> Dict[str, Any]: + return {"status": "success"} diff --git a/sidecar/tests/test_plugin_smart_context.py b/sidecar/tests/test_plugin_smart_context.py index 35d9c15..407bf03 100644 --- a/sidecar/tests/test_plugin_smart_context.py +++ b/sidecar/tests/test_plugin_smart_context.py @@ -4,7 +4,7 @@ import pytest from pathlib import Path -from unittest.mock import MagicMock, patch +from unittest.mock import AsyncMock, MagicMock, patch from sidecar import constants # Import the plugin module dynamically since it's not in the main package @@ -110,6 +110,20 @@ async def test_on_unload_removes_panel(self, plugin_instance, mock_brain): assert remove_call is not None assert remove_call["id"] == "smart-context-panel" + @pytest.mark.asyncio + async def test_plugin_loads_config_defaults(self, plugin_instance): + assert plugin_instance.similarity_threshold == 0.4 + assert plugin_instance.embedding_search is True + assert plugin_instance.active_topics == set() + + @pytest.mark.asyncio + async def test_on_load_subscribes_to_pipeline_events(self, plugin_instance, mock_brain): + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + await plugin_instance.on_load() + subscribed = [c[0][0] for c in mock_brain.subscribe.call_args_list] + assert "pipeline.output" in subscribed + assert "pipeline.context" in subscribed + def _load_embedding_cache(): base = Path(__file__).resolve().parent.parent.parent From e383681322f05dfb71ff48827f8b0705ff5ad5e6 Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 13:12:34 +0530 Subject: [PATCH 07/21] feat: smart_context topic extraction via pipeline.output Co-Authored-By: Claude Sonnet 4.6 --- example-vault/plugins/smart_context/main.py | 77 ++++++++++++++++++++- sidecar/tests/test_plugin_smart_context.py | 29 ++++++++ 2 files changed, 105 insertions(+), 1 deletion(-) diff --git a/example-vault/plugins/smart_context/main.py b/example-vault/plugins/smart_context/main.py index 94efcf0..238bcdb 100644 --- a/example-vault/plugins/smart_context/main.py +++ b/example-vault/plugins/smart_context/main.py @@ -92,7 +92,82 @@ async def on_unload(self) -> None: # ========================================================================= async def _on_pipeline_output(self, ctx: PipelineContext) -> None: - pass + """After each LLM response: extract topics in background.""" + if not ctx.response: + return + chat_id = ctx.metadata.get("chat_id") + if not chat_id: + return + asyncio.create_task(self._run_topic_extraction(chat_id)) + + async def _extract_topics(self, messages: List[Dict[str, Any]]) -> List[Dict[str, Any]]: + """Call LLM to extract topics and identify sticky messages.""" + from sidecar.services.llm_service import get_llm_service + + conversation = "\n".join( + f"[{m.get('id', '?')}] {m.get('role', 'user')}: " + f"{str(m.get('content', ''))[:200]}" + for m in messages + ) + + llm_messages = [ + {"role": "system", "content": TOPIC_EXTRACTION_PROMPT}, + {"role": "user", "content": conversation}, + ] + + llm = get_llm_service() + category = self.config.get("extraction_category", "fast") + response = await llm.complete( + messages=llm_messages, + category=category, + max_tokens=512, + temperature=0.2, + ) + + raw = response.content.strip() + if raw.startswith("```"): + raw = "\n".join(raw.split("\n")[1:]) + if raw.endswith("```"): + raw = raw[:-3] + + data = json.loads(raw) + topics: List[Dict[str, Any]] = data.get("topics", []) + sticky_ids: List[str] = data.get("sticky_message_ids", []) + + if sticky_ids: + topics.append({ + "label": "Instructions & Preferences", + "sticky": True, + "message_ids": sticky_ids, + "count": len(sticky_ids), + }) + + return topics + + async def _run_topic_extraction(self, chat_id: str) -> None: + """Background: extract topics from chat and save + notify panel.""" + try: + result = await self.brain.execute_command("memory.load_chat", chat_id=chat_id) + if result.get("status") != "success": + return + data = result.get("data", {}) + messages = data.get("messages", []) + if not messages: + return + + topics = await self._extract_topics(messages) + data["topics"] = topics + await self.brain.execute_command("memory.save_chat", chat_id=chat_id, data=data) + + self.current_chat_id = chat_id + self.emit("smart_context.topics_updated", { + "chat_id": chat_id, + "topics": topics, + "total_messages": len(messages), + }) + self.logger.info(f"Topics extracted for {chat_id}: {[t['label'] for t in topics]}") + except Exception as e: + self.logger.error(f"Topic extraction failed for {chat_id}: {e}") async def _on_pipeline_context(self, ctx: PipelineContext) -> None: pass diff --git a/sidecar/tests/test_plugin_smart_context.py b/sidecar/tests/test_plugin_smart_context.py index 407bf03..094fd96 100644 --- a/sidecar/tests/test_plugin_smart_context.py +++ b/sidecar/tests/test_plugin_smart_context.py @@ -2,6 +2,7 @@ Tests for Smart Context Plugin. """ +import json import pytest from pathlib import Path from unittest.mock import AsyncMock, MagicMock, patch @@ -124,6 +125,34 @@ async def test_on_load_subscribes_to_pipeline_events(self, plugin_instance, mock assert "pipeline.output" in subscribed assert "pipeline.context" in subscribed + @pytest.mark.asyncio + async def test_extract_topics_parses_llm_response(self, plugin_instance, mock_brain): + messages = [ + {"id": "a1b2", "role": "user", "content": "be concise"}, + {"id": "c3d4", "role": "user", "content": "How does async work in Python?"}, + {"id": "e5f6", "role": "assistant", "content": "Async uses coroutines..."}, + ] + + mock_response = MagicMock() + mock_response.content = json.dumps({ + "topics": [{"label": "Python Async", "count": 2}], + "sticky_message_ids": ["a1b2"] + }) + + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + with patch("sidecar.services.llm_service.get_llm_service") as mock_get_llm: + mock_llm = MagicMock() + mock_llm.complete = AsyncMock(return_value=mock_response) + mock_get_llm.return_value = mock_llm + result = await plugin_instance._extract_topics(messages) + + labels = [t["label"] for t in result] + assert "Python Async" in labels + assert "Instructions & Preferences" in labels + sticky = next(t for t in result if t.get("sticky")) + assert sticky["message_ids"] == ["a1b2"] + assert sticky["count"] == 1 + def _load_embedding_cache(): base = Path(__file__).resolve().parent.parent.parent From 3a25ce70688c764d92dcc6a98ec51a15fe5f1976 Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 13:17:23 +0530 Subject: [PATCH 08/21] test: add coverage for _on_pipeline_output and _run_topic_extraction Co-Authored-By: Claude Sonnet 4.6 --- sidecar/tests/test_plugin_smart_context.py | 70 ++++++++++++++++++++++ 1 file changed, 70 insertions(+) diff --git a/sidecar/tests/test_plugin_smart_context.py b/sidecar/tests/test_plugin_smart_context.py index 094fd96..d057eee 100644 --- a/sidecar/tests/test_plugin_smart_context.py +++ b/sidecar/tests/test_plugin_smart_context.py @@ -2,11 +2,13 @@ Tests for Smart Context Plugin. """ +import asyncio import json import pytest from pathlib import Path from unittest.mock import AsyncMock, MagicMock, patch from sidecar import constants +from sidecar.pipeline.types import PipelineContext # Import the plugin module dynamically since it's not in the main package import importlib.util @@ -153,6 +155,74 @@ async def test_extract_topics_parses_llm_response(self, plugin_instance, mock_br assert sticky["message_ids"] == ["a1b2"] assert sticky["count"] == 1 + @pytest.mark.asyncio + async def test_on_pipeline_output_creates_extraction_task(self, plugin_instance, mock_brain): + """_on_pipeline_output should fire _run_topic_extraction as background task.""" + ctx = PipelineContext( + message="hi", original_message="hi", + metadata={"chat_id": "chat_abc"}, + response="Some LLM response", + ) + extraction_calls = [] + + async def fake_extraction(chat_id): + extraction_calls.append(chat_id) + + plugin_instance._run_topic_extraction = fake_extraction + await plugin_instance._on_pipeline_output(ctx) + # Let the event loop run the created task + await asyncio.sleep(0) + assert extraction_calls == ["chat_abc"] + + @pytest.mark.asyncio + async def test_on_pipeline_output_skips_when_no_response(self, plugin_instance): + """_on_pipeline_output should do nothing when ctx.response is falsy.""" + ctx = PipelineContext( + message="hi", original_message="hi", + metadata={"chat_id": "chat_abc"}, + ) + called = [] + + async def fake_extraction(chat_id): + called.append(chat_id) + + plugin_instance._run_topic_extraction = fake_extraction + await plugin_instance._on_pipeline_output(ctx) + await asyncio.sleep(0) + assert called == [] + + @pytest.mark.asyncio + async def test_run_topic_extraction_saves_and_emits(self, plugin_instance, mock_brain): + """_run_topic_extraction should load chat, extract topics, save, and emit.""" + messages = [{"id": "m1", "role": "user", "content": "How does async work?"}] + extracted_topics = [{"label": "Python Async", "count": 1}] + + mock_brain.execute_command = AsyncMock(return_value={ + "status": "success", + "data": {"messages": messages, "topics": []}, + }) + plugin_instance._extract_topics = AsyncMock(return_value=extracted_topics) + + emitted = [] + plugin_instance.emit = lambda event, data: emitted.append((event, data)) + + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + await plugin_instance._run_topic_extraction("chat_xyz") + + assert plugin_instance.current_chat_id == "chat_xyz" + + # execute_command called at least twice: load_chat and save_chat + assert mock_brain.execute_command.call_count >= 2 + calls_str = str(mock_brain.execute_command.call_args_list) + assert "memory.save_chat" in calls_str + + # smart_context.topics_updated event emitted with correct payload + assert any(event == "smart_context.topics_updated" for event, _ in emitted) + topics_event = next(d for ev, d in emitted if ev == "smart_context.topics_updated") + assert topics_event["chat_id"] == "chat_xyz" + assert topics_event["total_messages"] == 1 + assert topics_event["topics"] == extracted_topics + def _load_embedding_cache(): base = Path(__file__).resolve().parent.parent.parent From 97805b2ca2d32eaf1a31c0403ee14db814b78284 Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 13:19:35 +0530 Subject: [PATCH 09/21] fix: hold strong reference to background tasks; use proper mock assertion --- example-vault/plugins/smart_context/main.py | 5 ++++- sidecar/tests/test_plugin_smart_context.py | 7 +++++-- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/example-vault/plugins/smart_context/main.py b/example-vault/plugins/smart_context/main.py index 238bcdb..44936f9 100644 --- a/example-vault/plugins/smart_context/main.py +++ b/example-vault/plugins/smart_context/main.py @@ -57,6 +57,7 @@ def __init__(self, plugin_dir: Path, vault_path: Path, config: Dict[str, Any] = self.active_topics: Set[str] = set() self.current_chat_id: str = "" + self._background_tasks: Set[asyncio.Task] = set() self.logger.info("Smart Context plugin initialized") @@ -98,7 +99,9 @@ async def _on_pipeline_output(self, ctx: PipelineContext) -> None: chat_id = ctx.metadata.get("chat_id") if not chat_id: return - asyncio.create_task(self._run_topic_extraction(chat_id)) + task = asyncio.create_task(self._run_topic_extraction(chat_id)) + self._background_tasks.add(task) + task.add_done_callback(self._background_tasks.discard) async def _extract_topics(self, messages: List[Dict[str, Any]]) -> List[Dict[str, Any]]: """Call LLM to extract topics and identify sticky messages.""" diff --git a/sidecar/tests/test_plugin_smart_context.py b/sidecar/tests/test_plugin_smart_context.py index d057eee..1b47503 100644 --- a/sidecar/tests/test_plugin_smart_context.py +++ b/sidecar/tests/test_plugin_smart_context.py @@ -213,8 +213,11 @@ async def test_run_topic_extraction_saves_and_emits(self, plugin_instance, mock_ # execute_command called at least twice: load_chat and save_chat assert mock_brain.execute_command.call_count >= 2 - calls_str = str(mock_brain.execute_command.call_args_list) - assert "memory.save_chat" in calls_str + save_calls = [ + c for c in mock_brain.execute_command.call_args_list + if c.args and c.args[0] == "memory.save_chat" + ] + assert len(save_calls) == 1 # smart_context.topics_updated event emitted with correct payload assert any(event == "smart_context.topics_updated" for event, _ in emitted) From a83a63cccd21076a222246cfdb668d73aa2235b0 Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 13:22:27 +0530 Subject: [PATCH 10/21] feat: smart_context context injection with embedding similarity Implement _compute_relevant_ids() and _on_pipeline_context() to filter pipeline chat history using cosine similarity against active topic embeddings. Sticky messages are always preserved; if nothing passes the threshold the full history is kept as a fallback. Adds 4 tests (17 total). Co-Authored-By: Claude Sonnet 4.6 --- example-vault/plugins/smart_context/main.py | 80 ++++++++++++++++- sidecar/tests/test_plugin_smart_context.py | 97 +++++++++++++++++++++ 2 files changed, 176 insertions(+), 1 deletion(-) diff --git a/example-vault/plugins/smart_context/main.py b/example-vault/plugins/smart_context/main.py index 44936f9..d2cf2db 100644 --- a/example-vault/plugins/smart_context/main.py +++ b/example-vault/plugins/smart_context/main.py @@ -172,8 +172,86 @@ async def _run_topic_extraction(self, chat_id: str) -> None: except Exception as e: self.logger.error(f"Topic extraction failed for {chat_id}: {e}") + async def _compute_relevant_ids( + self, chat_id: str, messages: List[Dict[str, Any]], sticky_ids: Set[str] + ) -> List[str]: + """ + Return IDs of messages relevant to active_topics via cosine similarity. + Sticky IDs are always included. Returns empty list if nothing passes threshold + (caller should fall back to full history). + """ + from sidecar.services.llm_service import get_llm_service + + llm = get_llm_service() + topic_embeddings = await llm.embed(list(self.active_topics)) + + cache = EmbeddingCache(self.vault_path / ".memory", chat_id) + need_embed: List[Dict[str, Any]] = [] + cached: Dict[str, List[float]] = {} + + for msg in messages: + msg_id = msg.get("id", "") + content = str(msg.get("content", "")) + emb = cache.get(msg_id, content) + if emb is not None: + cached[msg_id] = emb + else: + need_embed.append(msg) + + if need_embed: + texts = [str(m.get("content", "")) for m in need_embed] + new_embeddings = await llm.embed(texts) + for msg, emb in zip(need_embed, new_embeddings): + msg_id = msg.get("id", "") + content = str(msg.get("content", "")) + cache.set(msg_id, content, emb) + cached[msg_id] = emb + + included: List[str] = [] + for msg in messages: + msg_id = msg.get("id", "") + if msg_id in sticky_ids: + included.append(msg_id) + continue + emb = cached.get(msg_id) + if emb is None: + continue + max_sim = max(_cosine_similarity(emb, t_emb) for t_emb in topic_embeddings) + if max_sim >= self.similarity_threshold: + included.append(msg_id) + + return included + async def _on_pipeline_context(self, ctx: PipelineContext) -> None: - pass + """Filter ctx.history using embedding similarity to active topics.""" + if not self.active_topics or not self.embedding_search: + return + + chat_id = ctx.metadata.get("chat_id") + if not chat_id or not ctx.history: + return + + try: + result = await self.brain.execute_command("memory.load_chat", chat_id=chat_id) + data = result.get("data", {}) + topics_list = data.get("topics", []) + + sticky_ids: Set[str] = set() + for t in topics_list: + if t.get("sticky"): + sticky_ids.update(t.get("message_ids", [])) + + included = await self._compute_relevant_ids(chat_id, ctx.history, sticky_ids) + + non_sticky_included = [i for i in included if i not in sticky_ids] + if not non_sticky_included: + return # nothing passed threshold — keep full history + + included_set = set(included) + ctx.history = [m for m in ctx.history if m.get("id", "") in included_set] + + except Exception as e: + self.logger.error(f"Context injection failed: {e}") # ========================================================================= # Commands (stubs — filled in Task 6) diff --git a/sidecar/tests/test_plugin_smart_context.py b/sidecar/tests/test_plugin_smart_context.py index 1b47503..495f05a 100644 --- a/sidecar/tests/test_plugin_smart_context.py +++ b/sidecar/tests/test_plugin_smart_context.py @@ -226,6 +226,103 @@ async def test_run_topic_extraction_saves_and_emits(self, plugin_instance, mock_ assert topics_event["total_messages"] == 1 assert topics_event["topics"] == extracted_topics + @pytest.mark.asyncio + async def test_context_injection_passthrough_when_no_active_topics(self, plugin_instance, mock_brain): + plugin_instance.active_topics = set() + ctx = PipelineContext(message="hi", original_message="hi", metadata={}) + ctx.history = [{"id": "1", "role": "user", "content": "hello"}] + await plugin_instance._on_pipeline_context(ctx) + assert len(ctx.history) == 1 # unchanged + + @pytest.mark.asyncio + async def test_context_injection_filters_by_similarity(self, plugin_instance, mock_brain): + plugin_instance.active_topics = {"Python Async"} + plugin_instance.embedding_search = True + plugin_instance.similarity_threshold = 0.7 + + ctx = PipelineContext( + message="more?", original_message="more?", + metadata={"chat_id": "chat_test"}, + ) + ctx.history = [ + {"id": "m1", "role": "user", "content": "How does async work in Python?"}, + {"id": "m2", "role": "user", "content": "What is the capital of France?"}, + ] + + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + mock_brain.execute_command = AsyncMock(return_value={ + "status": "success", + "data": {"messages": ctx.history, "topics": [{"label": "Python Async", "count": 1}]} + }) + with patch("sidecar.services.llm_service.get_llm_service") as mock_get_llm: + mock_llm = MagicMock() + mock_llm.embed = AsyncMock(side_effect=[ + [[0.9, 0.1]], # topic embedding + [[0.85, 0.15], [0.1, 0.95]] # m1=similar, m2=dissimilar + ]) + mock_get_llm.return_value = mock_llm + await plugin_instance._on_pipeline_context(ctx) + + assert len(ctx.history) == 1 + assert ctx.history[0]["id"] == "m1" + + @pytest.mark.asyncio + async def test_context_injection_always_includes_sticky(self, plugin_instance, mock_brain): + plugin_instance.active_topics = {"Python Async"} + plugin_instance.embedding_search = True + plugin_instance.similarity_threshold = 0.99 # impossibly high + + ctx = PipelineContext(message="q", original_message="q", metadata={"chat_id": "c1"}) + ctx.history = [ + {"id": "sticky1", "role": "user", "content": "be concise"}, + {"id": "m1", "role": "user", "content": "something about python async"}, + ] + + topics_data = {"messages": ctx.history, "topics": [ + {"label": "Python Async", "count": 1}, + {"label": "Instructions & Preferences", "sticky": True, + "message_ids": ["sticky1"], "count": 1}, + ]} + + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + mock_brain.execute_command = AsyncMock( + return_value={"status": "success", "data": topics_data} + ) + with patch("sidecar.services.llm_service.get_llm_service") as mock_get_llm: + mock_llm = MagicMock() + mock_llm.embed = AsyncMock(side_effect=[ + [[1.0, 0.0]], # topic + [[0.1, 0.9], [0.0, 1.0]] # both below 0.99 threshold + ]) + mock_get_llm.return_value = mock_llm + await plugin_instance._on_pipeline_context(ctx) + + assert any(m["id"] == "sticky1" for m in ctx.history) + + @pytest.mark.asyncio + async def test_context_injection_fallback_when_threshold_met_by_nothing(self, plugin_instance, mock_brain): + """If nothing passes threshold (excluding sticky), return full history.""" + plugin_instance.active_topics = {"Exotic Topic"} + plugin_instance.embedding_search = True + plugin_instance.similarity_threshold = 0.99 + + ctx = PipelineContext(message="q", original_message="q", metadata={"chat_id": "c1"}) + original_history = [{"id": "m1", "role": "user", "content": "hello"}] + ctx.history = original_history.copy() + + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + mock_brain.execute_command = AsyncMock(return_value={ + "status": "success", + "data": {"messages": original_history, "topics": []} + }) + with patch("sidecar.services.llm_service.get_llm_service") as mock_get_llm: + mock_llm = MagicMock() + mock_llm.embed = AsyncMock(side_effect=[[[1.0, 0.0]], [[0.1, 0.9]]]) + mock_get_llm.return_value = mock_llm + await plugin_instance._on_pipeline_context(ctx) + + assert len(ctx.history) == 1 # full history preserved + def _load_embedding_cache(): base = Path(__file__).resolve().parent.parent.parent From 42cbae4043765b48f0642c03ce3263f48fb2bace Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 13:25:11 +0530 Subject: [PATCH 11/21] test: fix sticky context injection test to exercise actual sticky path Co-Authored-By: Claude Sonnet 4.6 --- sidecar/tests/test_plugin_smart_context.py | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/sidecar/tests/test_plugin_smart_context.py b/sidecar/tests/test_plugin_smart_context.py index 495f05a..25b4a70 100644 --- a/sidecar/tests/test_plugin_smart_context.py +++ b/sidecar/tests/test_plugin_smart_context.py @@ -268,14 +268,16 @@ async def test_context_injection_filters_by_similarity(self, plugin_instance, mo @pytest.mark.asyncio async def test_context_injection_always_includes_sticky(self, plugin_instance, mock_brain): + """Sticky messages are kept even when their own cosine similarity is below threshold.""" plugin_instance.active_topics = {"Python Async"} plugin_instance.embedding_search = True - plugin_instance.similarity_threshold = 0.99 # impossibly high + plugin_instance.similarity_threshold = 0.7 ctx = PipelineContext(message="q", original_message="q", metadata={"chat_id": "c1"}) ctx.history = [ {"id": "sticky1", "role": "user", "content": "be concise"}, - {"id": "m1", "role": "user", "content": "something about python async"}, + {"id": "m1", "role": "user", "content": "How does async work in Python?"}, + {"id": "m2", "role": "user", "content": "What is the capital of France?"}, ] topics_data = {"messages": ctx.history, "topics": [ @@ -290,14 +292,21 @@ async def test_context_injection_always_includes_sticky(self, plugin_instance, m ) with patch("sidecar.services.llm_service.get_llm_service") as mock_get_llm: mock_llm = MagicMock() + # Topic embedding: [0.9, 0.1] + # sticky1 embedding: [0.0, 1.0] — orthogonal to topic, cosine ≈ 0.0 (below 0.7) + # m1 embedding: [0.85, 0.15] — similar to topic, cosine ≈ 0.99 (above 0.7) + # m2 embedding: [0.1, 0.95] — dissimilar to topic, cosine ≈ 0.15 (below 0.7) mock_llm.embed = AsyncMock(side_effect=[ - [[1.0, 0.0]], # topic - [[0.1, 0.9], [0.0, 1.0]] # both below 0.99 threshold + [[0.9, 0.1]], # topic embedding + [[0.0, 1.0], [0.85, 0.15], [0.1, 0.95]] # sticky1, m1, m2 ]) mock_get_llm.return_value = mock_llm await plugin_instance._on_pipeline_context(ctx) - assert any(m["id"] == "sticky1" for m in ctx.history) + ids_in_history = {m["id"] for m in ctx.history} + assert "sticky1" in ids_in_history # kept via sticky logic despite low cosine + assert "m1" in ids_in_history # kept via similarity (cosine ≈ 0.99 >= 0.7) + assert "m2" not in ids_in_history # filtered out (cosine ≈ 0.15 < 0.7) @pytest.mark.asyncio async def test_context_injection_fallback_when_threshold_met_by_nothing(self, plugin_instance, mock_brain): From 7cbd3fe5b3ab5655f498dd3e5f01ccd9cee55607 Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 13:29:39 +0530 Subject: [PATCH 12/21] =?UTF-8?q?feat:=20smart=5Fcontext=20commands=20?= =?UTF-8?q?=E2=80=94=20set=5Ffilter,=20get=5Ftopics,=20set=5Fsimilarity=5F?= =?UTF-8?q?mode?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Implements the three stub command methods and adds _emit_highlight_ids. set_filter updates active_topics and emits filter_changed; get_topics loads chat data via memory.load_chat; set_similarity_mode toggles embedding search and clears the filter when disabled. Co-Authored-By: Claude Sonnet 4.6 --- example-vault/plugins/smart_context/main.py | 72 +++++++++++++++++++-- sidecar/tests/test_plugin_smart_context.py | 38 +++++++++++ 2 files changed, 105 insertions(+), 5 deletions(-) diff --git a/example-vault/plugins/smart_context/main.py b/example-vault/plugins/smart_context/main.py index d2cf2db..8f6a060 100644 --- a/example-vault/plugins/smart_context/main.py +++ b/example-vault/plugins/smart_context/main.py @@ -89,7 +89,7 @@ async def on_unload(self) -> None: await super().on_unload() # ========================================================================= - # Pipeline Subscribers (stubs — filled in Tasks 4 & 5) + # Pipeline Subscribers # ========================================================================= async def _on_pipeline_output(self, ctx: PipelineContext) -> None: @@ -254,14 +254,76 @@ async def _on_pipeline_context(self, ctx: PipelineContext) -> None: self.logger.error(f"Context injection failed: {e}") # ========================================================================= - # Commands (stubs — filled in Task 6) + # Commands # ========================================================================= async def set_filter(self, topics: List[str] = None, **kwargs) -> Dict[str, Any]: - return {"status": "success"} + """Set active topic filter. Empty list clears it.""" + if topics is None: + p = kwargs.get("p") or kwargs.get("params", {}) + topics = p.get("topics", []) + + self.active_topics = set(topics) + self.emit("smart_context.filter_changed", { + "active_topics": list(self.active_topics), + "chat_id": self.current_chat_id, + }) + + if self.current_chat_id and self.active_topics and self.embedding_search: + task = asyncio.create_task(self._emit_highlight_ids(self.current_chat_id)) + self._background_tasks.add(task) + task.add_done_callback(self._background_tasks.discard) + + return {"status": "success", "active_topics": list(self.active_topics)} async def get_topics(self, chat_id: str = "", **kwargs) -> Dict[str, Any]: - return {"status": "success", "topics": []} + """Return topics for the given chat.""" + if not chat_id: + p = kwargs.get("p") or kwargs.get("params", {}) + chat_id = p.get("chat_id", self.current_chat_id) + + if not chat_id: + return {"status": "success", "topics": [], "total_messages": 0} + + result = await self.brain.execute_command("memory.load_chat", chat_id=chat_id) + if result.get("status") != "success": + return {"status": "success", "topics": [], "total_messages": 0} + + data = result.get("data", {}) + return { + "status": "success", + "topics": data.get("topics", []), + "total_messages": len(data.get("messages", [])), + } async def set_similarity_mode(self, enabled: bool = True, **kwargs) -> Dict[str, Any]: - return {"status": "success"} + """Toggle embedding-based context filtering.""" + if not isinstance(enabled, bool): + p = kwargs.get("p") or kwargs.get("params", {}) + enabled = bool(p.get("enabled", True)) + self.embedding_search = enabled + if not enabled: + self.active_topics.clear() + self.emit("smart_context.filter_changed", { + "active_topics": [], + "chat_id": self.current_chat_id, + }) + return {"status": "success", "embedding_search": self.embedding_search} + + async def _emit_highlight_ids(self, chat_id: str) -> None: + """Background: compute filtered message IDs and emit for chat highlighting.""" + try: + result = await self.brain.execute_command("memory.load_chat", chat_id=chat_id) + data = result.get("data", {}) + messages = data.get("messages", []) + topics_list = data.get("topics", []) + + sticky_ids: Set[str] = set() + for t in topics_list: + if t.get("sticky"): + sticky_ids.update(t.get("message_ids", [])) + + included = await self._compute_relevant_ids(chat_id, messages, sticky_ids) + self.emit("smart_context.highlight_applied", {"message_ids": included}) + except Exception as e: + self.logger.error(f"Highlight computation failed: {e}") diff --git a/sidecar/tests/test_plugin_smart_context.py b/sidecar/tests/test_plugin_smart_context.py index 25b4a70..880419d 100644 --- a/sidecar/tests/test_plugin_smart_context.py +++ b/sidecar/tests/test_plugin_smart_context.py @@ -332,6 +332,44 @@ async def test_context_injection_fallback_when_threshold_met_by_nothing(self, pl assert len(ctx.history) == 1 # full history preserved + @pytest.mark.asyncio + async def test_set_filter_updates_active_topics_and_emits(self, plugin_instance, mock_brain): + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + result = await plugin_instance.set_filter(topics=["Python Async", "LangGraph"]) + assert plugin_instance.active_topics == {"Python Async", "LangGraph"} + assert result["status"] == "success" + mock_brain.emit_to_frontend.assert_called() + + @pytest.mark.asyncio + async def test_set_filter_empty_clears(self, plugin_instance, mock_brain): + plugin_instance.active_topics = {"Python Async"} + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + await plugin_instance.set_filter(topics=[]) + assert plugin_instance.active_topics == set() + + @pytest.mark.asyncio + async def test_get_topics_reads_from_chat_file(self, plugin_instance, mock_brain): + mock_brain.execute_command = AsyncMock(return_value={ + "status": "success", + "data": { + "messages": [{"id": "1"}], + "topics": [{"label": "Python", "count": 3}] + } + }) + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + result = await plugin_instance.get_topics(chat_id="chat_123") + assert result["status"] == "success" + assert result["topics"][0]["label"] == "Python" + assert result["total_messages"] == 1 + + @pytest.mark.asyncio + async def test_set_similarity_mode_toggles(self, plugin_instance, mock_brain): + plugin_instance.embedding_search = True + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + result = await plugin_instance.set_similarity_mode(enabled=False) + assert plugin_instance.embedding_search is False + assert result["status"] == "success" + def _load_embedding_cache(): base = Path(__file__).resolve().parent.parent.parent From d2eb91adfb2c4b8481fda3eb9d3bda31549b9556 Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 13:32:02 +0530 Subject: [PATCH 13/21] test: strengthen set_similarity_mode assertions for disable side effects Co-Authored-By: Claude Sonnet 4.6 --- sidecar/tests/test_plugin_smart_context.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sidecar/tests/test_plugin_smart_context.py b/sidecar/tests/test_plugin_smart_context.py index 880419d..a64b668 100644 --- a/sidecar/tests/test_plugin_smart_context.py +++ b/sidecar/tests/test_plugin_smart_context.py @@ -365,10 +365,13 @@ async def test_get_topics_reads_from_chat_file(self, plugin_instance, mock_brain @pytest.mark.asyncio async def test_set_similarity_mode_toggles(self, plugin_instance, mock_brain): plugin_instance.embedding_search = True + plugin_instance.active_topics = {"Python Async"} with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): result = await plugin_instance.set_similarity_mode(enabled=False) assert plugin_instance.embedding_search is False + assert plugin_instance.active_topics == set() # cleared on disable assert result["status"] == "success" + mock_brain.emit_to_frontend.assert_called() # filter_changed emitted def _load_embedding_cache(): From be183fc6f7290c6ed717a17e56b7aedec454600f Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 13:36:20 +0530 Subject: [PATCH 14/21] feat: smart_context bubble cloud panel HTML/JS Co-Authored-By: Claude Sonnet 4.6 --- .../plugins/smart_context/panel.html | 166 ++++++++++++++++++ sidecar/tests/test_plugin_smart_context.py | 11 ++ 2 files changed, 177 insertions(+) create mode 100644 example-vault/plugins/smart_context/panel.html diff --git a/example-vault/plugins/smart_context/panel.html b/example-vault/plugins/smart_context/panel.html new file mode 100644 index 0000000..a748003 --- /dev/null +++ b/example-vault/plugins/smart_context/panel.html @@ -0,0 +1,166 @@ + + + + + + +
+ +
+ 🔒 + Instructions always in context + 0 +
+ +
+ + +
+ +
+ Waiting for context… +
+ +
All messages in context
+ +
+ + + diff --git a/sidecar/tests/test_plugin_smart_context.py b/sidecar/tests/test_plugin_smart_context.py index a64b668..d112d6b 100644 --- a/sidecar/tests/test_plugin_smart_context.py +++ b/sidecar/tests/test_plugin_smart_context.py @@ -373,6 +373,17 @@ async def test_set_similarity_mode_toggles(self, plugin_instance, mock_brain): assert result["status"] == "success" mock_brain.emit_to_frontend.assert_called() # filter_changed emitted + @pytest.mark.asyncio + async def test_on_client_connected_loads_panel_html_when_exists(self, plugin_instance, mock_brain): + (plugin_instance.plugin_dir / "panel.html").write_text("
test
") + with patch("sidecar.vault_brain.VaultBrain.get", return_value=mock_brain): + await plugin_instance.on_client_connected() + set_calls = [ + c for c in mock_brain.emit_to_frontend.call_args_list + if c.kwargs.get("data", {}).get("action") == "set_panel" + ] + assert any("sc-panel" in c.kwargs["data"]["html"] for c in set_calls) + def _load_embedding_cache(): base = Path(__file__).resolve().parent.parent.parent From bd6de881934edc9f063e7d83b6d9037a55e0b637 Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 13:40:33 +0530 Subject: [PATCH 15/21] fix: panel XSS via DOM manipulation; fix filter desync and double-emit Co-Authored-By: Claude Sonnet 4.6 --- .../plugins/smart_context/panel.html | 39 ++++++++++++------- 1 file changed, 25 insertions(+), 14 deletions(-) diff --git a/example-vault/plugins/smart_context/panel.html b/example-vault/plugins/smart_context/panel.html index a748003..0a980c2 100644 --- a/example-vault/plugins/smart_context/panel.html +++ b/example-vault/plugins/smart_context/panel.html @@ -95,23 +95,32 @@ var regular = scTopics.filter(function(t){ return !t.sticky; }); if (!regular.length) { - bubblesEl.innerHTML = 'No topics yet\u2026'; + bubblesEl.textContent = ''; + var placeholder = document.createElement('span'); + placeholder.className = 'sc-no-topics'; + placeholder.textContent = 'No topics yet\u2026'; + bubblesEl.appendChild(placeholder); statusEl.textContent = 'All messages in context'; return; } var maxCount = Math.max.apply(null, regular.map(function(t){ return t.count; })); + maxCount = maxCount || 1; // guard against all-zero counts var hasFilter = scSimilarity && scActive.size > 0; - bubblesEl.innerHTML = regular.map(function(t) { + bubblesEl.textContent = ''; + regular.forEach(function(t) { var scale = (0.85 + 0.3 * (t.count / maxCount)).toFixed(2); - var cls = 'sc-bubble'; - if (hasFilter) cls += scActive.has(t.label) ? ' sc-active' : ' sc-dim'; - var safeLbl = t.label.replace(/\\/g, '\\\\').replace(/'/g, "\\'"); - return '
' + - t.label + '
'; - }).join(''); + var div = document.createElement('div'); + div.className = 'sc-bubble' + + (hasFilter ? (scActive.has(t.label) ? ' sc-active' : ' sc-dim') : ''); + div.style.fontSize = scale + 'em'; + div.textContent = t.label; // safe: textContent never interprets HTML + div.addEventListener('click', function() { + window.scToggleTopic(t.label); // closure captures label directly + }); + bubblesEl.appendChild(div); + }); statusEl.textContent = hasFilter ? scActive.size + ' topic(s) selected \u2014 context filtered' @@ -133,10 +142,6 @@ scSimilarity = enabled; if (!enabled) { scActive.clear(); - window.request('execute_command', { - command: 'smart_context.set_filter', - args: { topics: [] } - }).catch(console.error); } window.request('execute_command', { command: 'smart_context.set_similarity_mode', @@ -150,7 +155,13 @@ var d = e.detail || {}; if (d.topics) { scTopics = d.topics; - scActive.clear(); + if (scActive.size > 0) { + scActive.clear(); + window.request('execute_command', { + command: 'smart_context.set_filter', + args: { topics: [] } + }).catch(function(e){ console.error('[SmartContext] clear filter:', e); }); + } scRender(); } }); From 0246f3d44334f0a7dc865d381c5a9762f23b1f2c Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 13:43:45 +0530 Subject: [PATCH 16/21] feat: chat message highlighting for smart_context filter Adds .sc-highlight CSS class and two window event listeners in chat.js (setupSmartContextEventListeners) so Smart Context filter events visually highlight matching messages via a blue left-border accent. Also fixes a variable shadowing lint issue in panel.html (e -> err in catch handler). Co-Authored-By: Claude Sonnet 4.6 --- .../plugins/smart_context/panel.html | 2 +- src/styles/styles.css | 11 +++++++ src/vault/chat/chat.js | 32 +++++++++++++++++++ 3 files changed, 44 insertions(+), 1 deletion(-) diff --git a/example-vault/plugins/smart_context/panel.html b/example-vault/plugins/smart_context/panel.html index 0a980c2..ccca5cc 100644 --- a/example-vault/plugins/smart_context/panel.html +++ b/example-vault/plugins/smart_context/panel.html @@ -160,7 +160,7 @@ window.request('execute_command', { command: 'smart_context.set_filter', args: { topics: [] } - }).catch(function(e){ console.error('[SmartContext] clear filter:', e); }); + }).catch(function(err){ console.error('[SmartContext] clear filter:', err); }); } scRender(); } diff --git a/src/styles/styles.css b/src/styles/styles.css index f3107bd..3d3cbd5 100644 --- a/src/styles/styles.css +++ b/src/styles/styles.css @@ -3995,4 +3995,15 @@ input:checked+.slider:before { .themes-grid { grid-template-columns: 1fr; } +} + +/* ======================================== + SMART CONTEXT — CHAT MESSAGE HIGHLIGHTING + ======================================== */ + +/* Applied to chat messages included in the current Smart Context filter */ +.sc-highlight { + border-left: 3px solid rgba(99, 179, 237, 0.7) !important; + background: rgba(99, 179, 237, 0.05) !important; + transition: background 0.2s, border-left 0.2s; } \ No newline at end of file diff --git a/src/vault/chat/chat.js b/src/vault/chat/chat.js index 38b9086..b107d37 100644 --- a/src/vault/chat/chat.js +++ b/src/vault/chat/chat.js @@ -35,6 +35,9 @@ export function initChatGlobals() { // Setup streaming event listeners setupStreamEventListeners(); + // Setup Smart Context highlight event listeners + setupSmartContextEventListeners(); + window.__chatGlobalsInitialized = true; console.log('[Chat] Global listeners initialized'); } @@ -861,6 +864,35 @@ function setupStreamEventListeners() { }); } +/** + * Setup event listeners for Smart Context chat highlighting + */ +function setupSmartContextEventListeners() { + // Highlight messages included in the current Smart Context filter + window.addEventListener('smart_context.highlight_applied', function(e) { + var messageIds = (e.detail && e.detail.message_ids) ? e.detail.message_ids : []; + + // Remove all existing highlights + document.querySelectorAll('[data-message-id].sc-highlight') + .forEach(function(el) { el.classList.remove('sc-highlight'); }); + + // Apply to matching elements + messageIds.forEach(function(id) { + var el = document.querySelector('[data-message-id="' + id + '"]'); + if (el) el.classList.add('sc-highlight'); + }); + }); + + // Clear all highlights when filter is cleared + window.addEventListener('smart_context.filter_changed', function(e) { + var active = (e.detail && e.detail.active_topics) ? e.detail.active_topics : []; + if (active.length === 0) { + document.querySelectorAll('[data-message-id].sc-highlight') + .forEach(function(el) { el.classList.remove('sc-highlight'); }); + } + }); +} + /** * Update streaming content without adding toolbar (for in-progress streaming) */ From 3ec3a6dc308b1dea2583faec4949c35f7f157086 Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 13:45:01 +0530 Subject: [PATCH 17/21] fix: rename shadowed catch parameter e to err in panel.html --- example-vault/plugins/smart_context/panel.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/example-vault/plugins/smart_context/panel.html b/example-vault/plugins/smart_context/panel.html index ccca5cc..4f89cac 100644 --- a/example-vault/plugins/smart_context/panel.html +++ b/example-vault/plugins/smart_context/panel.html @@ -135,7 +135,7 @@ window.request('execute_command', { command: 'smart_context.set_filter', args: { topics: Array.from(scActive) } - }).catch(function(e){ console.error('[SmartContext] set_filter:', e); }); + }).catch(function(err){ console.error('[SmartContext] set_filter:', err); }); }; window.scToggleSimilarity = function(enabled) { From 29276bdc170ae27175753b4c3881495276a9e13c Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 13:46:52 +0530 Subject: [PATCH 18/21] style: use arrow functions and const in setupSmartContextEventListeners --- src/vault/chat/chat.js | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/src/vault/chat/chat.js b/src/vault/chat/chat.js index b107d37..7176b55 100644 --- a/src/vault/chat/chat.js +++ b/src/vault/chat/chat.js @@ -869,26 +869,26 @@ function setupStreamEventListeners() { */ function setupSmartContextEventListeners() { // Highlight messages included in the current Smart Context filter - window.addEventListener('smart_context.highlight_applied', function(e) { - var messageIds = (e.detail && e.detail.message_ids) ? e.detail.message_ids : []; + window.addEventListener('smart_context.highlight_applied', (e) => { + const messageIds = e.detail?.message_ids ?? []; // Remove all existing highlights document.querySelectorAll('[data-message-id].sc-highlight') - .forEach(function(el) { el.classList.remove('sc-highlight'); }); + .forEach((el) => el.classList.remove('sc-highlight')); // Apply to matching elements - messageIds.forEach(function(id) { - var el = document.querySelector('[data-message-id="' + id + '"]'); + messageIds.forEach((id) => { + const el = document.querySelector(`[data-message-id="${id}"]`); if (el) el.classList.add('sc-highlight'); }); }); // Clear all highlights when filter is cleared - window.addEventListener('smart_context.filter_changed', function(e) { - var active = (e.detail && e.detail.active_topics) ? e.detail.active_topics : []; + window.addEventListener('smart_context.filter_changed', (e) => { + const active = e.detail?.active_topics ?? []; if (active.length === 0) { document.querySelectorAll('[data-message-id].sc-highlight') - .forEach(function(el) { el.classList.remove('sc-highlight'); }); + .forEach((el) => el.classList.remove('sc-highlight')); } }); } From 0ea6f1b66255288bdc4050251848c0351bd12836 Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 14:52:09 +0530 Subject: [PATCH 19/21] chore: explicit smart_context vault config defaults Co-Authored-By: Claude Sonnet 4.6 --- example-vault/.vault.toml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/example-vault/.vault.toml b/example-vault/.vault.toml index 9b97fd7..fbb879b 100644 --- a/example-vault/.vault.toml +++ b/example-vault/.vault.toml @@ -46,4 +46,6 @@ enabled = true enabled = true [plugins.smart_context] -enabled = true \ No newline at end of file +enabled = true +similarity_threshold = 0.4 +embedding_search = true \ No newline at end of file From 164cb4cd0dfb9904b9339e207aaba0db1b31f7bf Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 14:54:57 +0530 Subject: [PATCH 20/21] fix: prevent smart_context topic extraction from interfering with memory_fallback tests The smart_context plugin registers a pipeline.output subscriber that spawns a background asyncio task (_run_topic_extraction) after every LLM response. In the test environment the memory plugin is live, so memory.load_chat succeeds and _run_topic_extraction proceeds to call LLMService.complete with the topic extraction prompt. This overwrites mock_instance.complete.call_args, making the test's assertion about which messages were sent to the chat completion call fail. Fix: patch _run_topic_extraction to a no-op AsyncMock on the smart_context plugin instance before the test's chat_send calls, so background extraction cannot consume mock calls or corrupt call_args. Co-Authored-By: Claude Sonnet 4.6 --- sidecar/tests/test_memory_fallback.py | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/sidecar/tests/test_memory_fallback.py b/sidecar/tests/test_memory_fallback.py index a0e944d..38c8259 100644 --- a/sidecar/tests/test_memory_fallback.py +++ b/sidecar/tests/test_memory_fallback.py @@ -49,6 +49,13 @@ async def test_memory_fallback_linear_chat(example_vault_path): # Verify Keyring/others if needed, but mostly we care about LLM being mocked + # Prevent smart_context from making extra LLM calls during topic extraction. + # _run_topic_extraction is a background asyncio task that fires after each + # pipeline output. It calls memory.load_chat then LLMService.complete, which + # consumes mock calls and corrupts call_args assertions later in this test. + if "smart_context" in brain.plugins: + brain.plugins["smart_context"]._run_topic_extraction = AsyncMock(return_value=None) + # SIMULATE MISSING PLUGIN # Manually remove chat_branches from the loaded plugins if "chat_branches" in brain.plugins: From 91d5228cf8e6aa41499081bb99d0e15958463f3d Mon Sep 17 00:00:00 2001 From: ARC345 Date: Sun, 1 Mar 2026 14:59:31 +0530 Subject: [PATCH 21/21] fix: fence stripping, message count status line, batch embedding cache writes Co-Authored-By: Claude Sonnet 4.6 --- .../plugins/smart_context/embedding_cache.py | 4 ++++ example-vault/plugins/smart_context/main.py | 2 ++ .../plugins/smart_context/panel.html | 19 ++++++++++++++++--- sidecar/tests/test_plugin_smart_context.py | 4 +++- 4 files changed, 25 insertions(+), 4 deletions(-) diff --git a/example-vault/plugins/smart_context/embedding_cache.py b/example-vault/plugins/smart_context/embedding_cache.py index 5ef6447..647fd62 100644 --- a/example-vault/plugins/smart_context/embedding_cache.py +++ b/example-vault/plugins/smart_context/embedding_cache.py @@ -36,5 +36,9 @@ def get(self, message_id: str, content: str) -> Optional[List[float]]: return self._data.get(self._key(message_id, content)) def set(self, message_id: str, content: str, embedding: List[float]) -> None: + """Store an embedding. Call save() to persist.""" self._data[self._key(message_id, content)] = embedding + + def save(self) -> None: + """Flush all cached embeddings to disk.""" self._save() diff --git a/example-vault/plugins/smart_context/main.py b/example-vault/plugins/smart_context/main.py index 8f6a060..00cc160 100644 --- a/example-vault/plugins/smart_context/main.py +++ b/example-vault/plugins/smart_context/main.py @@ -132,6 +132,7 @@ async def _extract_topics(self, messages: List[Dict[str, Any]]) -> List[Dict[str raw = "\n".join(raw.split("\n")[1:]) if raw.endswith("```"): raw = raw[:-3] + raw = raw.strip() # remove any leftover whitespace/newlines after fence removal data = json.loads(raw) topics: List[Dict[str, Any]] = data.get("topics", []) @@ -206,6 +207,7 @@ async def _compute_relevant_ids( content = str(msg.get("content", "")) cache.set(msg_id, content, emb) cached[msg_id] = emb + cache.save() # flush all new embeddings in one write included: List[str] = [] for msg in messages: diff --git a/example-vault/plugins/smart_context/panel.html b/example-vault/plugins/smart_context/panel.html index 4f89cac..daa91ff 100644 --- a/example-vault/plugins/smart_context/panel.html +++ b/example-vault/plugins/smart_context/panel.html @@ -84,6 +84,7 @@ var scTopics = []; var scActive = new Set(); var scSimilarity = true; +var scTotalMessages = 0; function scRender() { var bubblesEl = document.getElementById('sc-bubbles'); @@ -122,9 +123,10 @@ bubblesEl.appendChild(div); }); - statusEl.textContent = hasFilter - ? scActive.size + ' topic(s) selected \u2014 context filtered' - : 'All messages in context'; + if (!hasFilter) { + statusEl.textContent = 'All messages in context'; + } + // When hasFilter is true, status is updated by smart_context.highlight_applied listener } window.scToggleTopic = function(label) { @@ -155,6 +157,7 @@ var d = e.detail || {}; if (d.topics) { scTopics = d.topics; + scTotalMessages = d.total_messages || 0; if (scActive.size > 0) { scActive.clear(); window.request('execute_command', { @@ -166,6 +169,16 @@ } }); +// Update status line with message count when highlighting is applied +window.addEventListener('smart_context.highlight_applied', function(e) { + var d = e.detail || {}; + var ids = d.message_ids || []; + var statusEl = document.getElementById('sc-status'); + if (statusEl && scTotalMessages > 0) { + statusEl.textContent = ids.length + ' of ' + scTotalMessages + ' messages in context'; + } +}); + // Relay filter_changed -> chat highlighting window.addEventListener('smart_context.filter_changed', function(e) { window.dispatchEvent(new CustomEvent('smart_context.highlight_request', { diff --git a/sidecar/tests/test_plugin_smart_context.py b/sidecar/tests/test_plugin_smart_context.py index d112d6b..e56238f 100644 --- a/sidecar/tests/test_plugin_smart_context.py +++ b/sidecar/tests/test_plugin_smart_context.py @@ -408,7 +408,9 @@ def test_embedding_cache_stores_and_retrieves(tmp_path): def test_embedding_cache_persists_across_instances(tmp_path): EmbeddingCache = _load_embedding_cache() - EmbeddingCache(tmp_path, "chat_abc").set("msg1", "text", [1.0, 2.0]) + cache = EmbeddingCache(tmp_path, "chat_abc") + cache.set("msg1", "text", [1.0, 2.0]) + cache.save() assert EmbeddingCache(tmp_path, "chat_abc").get("msg1", "text") == [1.0, 2.0] def test_embedding_cache_content_change_is_cache_miss(tmp_path):