diff --git a/README.md b/README.md index 0584a28c..3db516b1 100644 --- a/README.md +++ b/README.md @@ -6,10 +6,10 @@ ![OpenAI Compatible](https://img.shields.io/badge/API-OpenAI--compatible-111827.svg) ![Tests](https://github.com/makeitdouble/jin_core/actions/workflows/tests.yml/badge.svg) -**JIN Core** is a local-first AI orchestration runtime. Instead of treating models as simple stateless chat boxes, it splits them into dedicated runtime roles (reasoning, service, translation) inside a single, seamless browser interface — with no frontend build step required. +**JIN Core Engine** is a local AI runtime for OpenAI-compatible models with visible memory, visible reasoning traces, and inspectable session state. ### 3-Layer Memory -JIN doesn't just store logs. It uses short-term continuity to dynamically guide conversation strategy: +JIN uses short-term continuity to dynamically guide conversation strategy: * **L1 (Live Facts):** Actionable session state kept in active process memory. * **L2 (Patterns):** Tracks interaction loops and repetition counters to adapt prompts on the fly. @@ -17,16 +17,32 @@ JIN doesn't just store logs. It uses short-term continuity to dynamically guide *Every memory update is captured as a versioned snapshot with diff highlights, fully inspectable in the right-side timeline panel.* +## UI Preview + +### Runtime Workspace ![JIN Core Engine runtime UI](ui/static/images/jin-core-redesign.jpg) -![highlight](ui/static/images/highlight.png) +Main runtime view: chat, live reasoning, telemetry, and inspectable memory panels in one browser workspace. + +### Think Citations + +![Think citation highlighting](ui/static/images/think-highlight.jpg) + +Think citation highlighting shows where reasoning quotes rules, runtime memory, or restored session context. + +### Memory Timeline + +![Runtime memory snapshot timeline](ui/static/images/runtime-highlight.png) + +Runtime memory snapshots can be stepped through visually, with new or changed facts highlighted in the sidebar. ## Capabilities - A chat room that feels alive: answers stream in as they are written, thinking stays visually separate from the final reply, and you can stop a generation the moment it drifts. - A visible short-term memory: JIN keeps a compact sense of what this session is about, what changed, and what still feels unresolved. - A memory timeline you can inspect: step through snapshots and see which facts or patterns were added instead of guessing what the assistant remembered. +- Citation highlighting inside think blocks: rule fragments, runtime memory, and restored session context are softly highlighted after a thinking block completes, then reappear on hover. - A calmer loop breaker: when the conversation starts repeating itself, JIN can notice the pattern and change strategy instead of giving the same polite answer again. - A built-in search move: the model can ask the runtime to search, then answer from trusted results without dumping raw tool syntax into the chat. - A session save and restore path: saying something like "save the session" or "that's all for today" triggers a compact L3 memory digest. The browser stores it locally and replays it on reconnect so the next session starts from context rather than blank slate. @@ -59,6 +75,8 @@ The memory layer can also surface compact pattern signals. When the session star Each memory update is also stored as a per-session snapshot. The UI can step backward and forward through those snapshots, replaying lightweight diff highlights so the user can see which memory keys or values were added or changed during the conversation. +Completed thinking blocks are also scanned for direct citations from trusted prompt context. Rule matches, indexed runtime-memory matches, and restored session-memory matches are highlighted with separate colors so the user can see which injected source shaped the reasoning without interrupting streaming. + If generation is aborted, the runtime captures the partial answer and schedules an interrupted memory update. The memory summarizer is instructed to mark the turn as incomplete and not treat it as resolved. When the user signals the end of a session — explicitly or through natural closing phrases — the brain emits a `REMEMBER_SESSION` action. The runtime builds a compact L3 digest from the current snapshot history and sends it to the browser for local storage. On the next connection, the browser sends the digest back as part of the bootstrap payload and the runtime injects it as trusted session context before the first turn. @@ -531,8 +549,6 @@ The frontend uses vanilla JavaScript and Tailwind from CDN. The current input be The following capabilities are planned but not yet implemented. -**Think-block rule citation highlighting.** After a thinking block completes, the UI scans the text for phrases matching known injected prompt sections (core identity, mode rules, runtime memory, session memory). Matched fragments are highlighted with a soft color; hovering shows a tooltip with the source name, type, layer, and full rule text. This makes prompt influence visible during debugging without affecting the streaming display. - **Long-term facts layer (L4).** A cross-session key-fact store extracted from completed turns by the service model, stored as JSON, and retrieved via keyword scoring before each brain call. Facts carry category, relevance, confidence, and mention count. A deduplication pass prevents drift from accumulating near-duplicate entries. The top-N retrieved facts are injected into the brain prompt as low-priority background context. No vector search or embedding index; heuristic scoring only for MVP. **User and JIN (LX layer) profiles.** A periodic distillation of session snapshots into two versioned JSON files: `user_profile.json` (stable preferences, recurring themes, friction points, open projects) and `jin_profile.json` (emergent behavioral biases, voice tendencies, avoidances). Profiles are built from snapshot archives in batches, not in real time. They are injected as soft background context, not as hard identity constraints. Old profile versions are kept for rollback. diff --git a/app.py b/app.py index cdc81837..11b11f94 100644 --- a/app.py +++ b/app.py @@ -45,6 +45,9 @@ from runtime.behavior_contract import ( get_behavior_contract, ) +from utils.rule_citations import ( + get_rule_citation_registry, +) STATUS_CHECK_TIMEOUT = getattr( config, @@ -297,6 +300,33 @@ async def api_behavior_contract(): return get_behavior_contract() +@app.get("/api/debug/rule-citations") +async def api_debug_rule_citations(): + + enabled = bool( + getattr( + config, + "DEBUG_RULE_CITATIONS", + True, + ) + ) + + if not enabled: + return { + "enabled": False, + "version": "disabled", + "fragmentCount": 0, + "fragments": [], + } + + registry = get_rule_citation_registry() + + return { + "enabled": True, + **registry, + } + + # --------------------------------------------------------- # DEV ENTRYPOINT # --------------------------------------------------------- diff --git a/config.example.py b/config.example.py index 7d30d1f4..2380714d 100644 --- a/config.example.py +++ b/config.example.py @@ -3,6 +3,7 @@ USE_SERVICE_AS_BRAIN = True TRANSLATION_ENABLED = False TRANSLATE_RESPONSE = False +DEBUG_RULE_CITATIONS = True CHAT_ENDPOINT = "/v1/chat/completions" MODELS_ENDPOINT = "/v1/models" @@ -78,7 +79,6 @@ SEARCH_TIMEOUT = 100.0 - # --------------------------------------------------------- # TRANSLATOR MODEL # --------------------------------------------------------- diff --git a/ui/static/images/highlight.png b/ui/static/images/runtime-highlight.png similarity index 100% rename from ui/static/images/highlight.png rename to ui/static/images/runtime-highlight.png diff --git a/ui/static/images/think-highlight.jpg b/ui/static/images/think-highlight.jpg new file mode 100644 index 00000000..7335f0a1 Binary files /dev/null and b/ui/static/images/think-highlight.jpg differ diff --git a/ui/static/js/chat.js b/ui/static/js/chat.js index 72f3f793..40fae752 100644 --- a/ui/static/js/chat.js +++ b/ui/static/js/chat.js @@ -8,9 +8,17 @@ const streamMessages = const STREAM_FRAME_WARNING_MS = 12; const STREAM_NEAR_BOTTOM_PX = 72; +const THINK_RULE_CITATIONS_ENDPOINT = + "/api/debug/rule-citations"; +const THINK_RULE_WORKER_URL = + "/static/js/think-rule-worker.js?v=rule-citations-3"; let streamFrameScheduled = false; +let thinkRuleCitationWorker = null; +let thinkRuleCitationRegistryPromise = null; +let nextThinkRuntimeCitationIndex = 0; const deferredRuntimeActionsAfterResponse = []; +const activeThinkRuleCitationJobs = new Map(); const jinInputLoopState = { previousInput: "", @@ -192,6 +200,840 @@ function requestStreamFrame(callback) { } +function loadThinkRuleCitationRegistry() { + + if (thinkRuleCitationRegistryPromise) { + return thinkRuleCitationRegistryPromise; + } + + thinkRuleCitationRegistryPromise = fetch( + THINK_RULE_CITATIONS_ENDPOINT, + { + cache: "no-store", + } + ) + .then((response) => { + if (!response.ok) { + throw new Error( + `Rule citation registry failed: ${response.status}` + ); + } + + return response.json(); + }) + .catch((error) => { + if (isStreamDebugEnabled()) { + console.warn( + "[think-rules] disabled", + error + ); + } + + return { + enabled: false, + fragments: [], + }; + }); + + return thinkRuleCitationRegistryPromise; + +} + +function getThinkRuleCitationWorker() { + + if (thinkRuleCitationWorker) { + return thinkRuleCitationWorker; + } + + if (!window.Worker) { + return null; + } + + thinkRuleCitationWorker = + new Worker( + THINK_RULE_WORKER_URL + ); + + thinkRuleCitationWorker.onmessage = + handleThinkRuleWorkerMessage; + + thinkRuleCitationWorker.onerror = (event) => { + if (isStreamDebugEnabled()) { + console.warn( + "[think-rules] worker error", + event.message + ); + } + }; + + return thinkRuleCitationWorker; + +} + +function thinkRuleLevelRank(level) { + + if (level === "exact") { + return 3; + } + + if (level === "near") { + return 2; + } + + return 1; + +} + +function thinkCitationSourcePriority(match) { + + if ( + match + && match.sourceType === "rule" + ) { + return 2; + } + + if ( + match + && match.sourceType === "runtime" + ) { + return 1; + } + + if ( + match + && match.sourceType === "session" + ) { + return 0; + } + + return -1; + +} + +function resolveThinkRuleOverlaps(matches) { + + const seen = new Set(); + const sorted = [...matches] + .filter((match) => { + if ( + !match + || match.end <= match.start + ) { + return false; + } + + const key = [ + match.start, + match.end, + match.level, + match.constantName, + match.sourceText, + ].join("|"); + + if (seen.has(key)) { + return false; + } + + seen.add(key); + return true; + }) + .sort((left, right) => { + const priorityDelta = + thinkCitationSourcePriority( + right + ) + - thinkCitationSourcePriority( + left + ); + + if (priorityDelta) { + return priorityDelta; + } + + const levelDelta = + thinkRuleLevelRank( + right.level + ) + - thinkRuleLevelRank( + left.level + ); + + if (levelDelta) { + return levelDelta; + } + + if (right.score !== left.score) { + return right.score - left.score; + } + + return ( + (right.end - right.start) + - (left.end - left.start) + ); + }); + + const selected = []; + + sorted.forEach((match) => { + const overlaps = + selected.some( + (selectedMatch) => ( + match.start < selectedMatch.end + && match.end > selectedMatch.start + ) + ); + + if (!overlaps) { + selected.push( + match + ); + } + }); + + return selected.sort( + (left, right) => left.start - right.start + ); + +} + +function buildThinkRuleTitle( + match, + matchedText +) { + + const score = + Math.round( + Number( + match.score || 0 + ) * 100 + ); + + const label = + match.sourceType === "runtime" + ? "RUNTIME" + : match.sourceType === "session" + ? "SESSION" + : "RULE"; + + return [ + `${label} - ${match.constantName || "unknown"} - ${match.level || "match"} - ${score}%`, + `source: ${match.source || "rules"}`, + `layer: ${match.layer || "base"}`, + `matched: "${matchedText}"`, + `${match.sourceType === "rule" ? "rule" : "memory"}: "${match.titleText || match.sourceText || ""}"`, + ].join("\n"); + +} + +function getThinkCitationClassName(match) { + + const sourceClass = + match.sourceType === "runtime" + ? "runtime" + : match.sourceType === "session" + ? "session" + : "rule"; + + return [ + "think-rule-hit", + `think-citation-${sourceClass}`, + match.level || "near", + ].join(" "); + +} + +function splitThinkCitationTextFragments(text) { + + const runtimeModel = + window.JinRuntime + && window.JinRuntime.memoryModel; + + const lines = + runtimeModel + && typeof runtimeModel.splitMemoryTextLines === "function" + ? runtimeModel.splitMemoryTextLines( + text + ) + : String(text || "") + .replace(/\\n/g, "\n") + .split(/\r?\n+/) + .map(line => line.trim()) + .filter(Boolean); + + return lines + .map((line) => { + const cleanedLine = + runtimeModel + && typeof runtimeModel.stripRuntimeMemoryMeta === "function" + ? runtimeModel.stripRuntimeMemoryMeta( + line + ) + : line; + + return String(cleanedLine || "").trim(); + }) + .filter(Boolean); + +} + +function buildMemoryCitationFragments( + memoryText, + options +) { + + const { + source, + sourceType, + citationType, + layer, + idPrefix, + defaultConstantName, + } = options; + + const fragments = []; + const seen = new Set(); + + splitThinkCitationTextFragments( + memoryText + ).forEach((line, index) => { + const separatorIndex = + line.indexOf(":"); + const key = + separatorIndex > 0 + ? line.slice( + 0, + separatorIndex + ).trim() + : defaultConstantName; + const value = + separatorIndex > 0 + ? line.slice( + separatorIndex + 1 + ).trim() + : line; + + [ + line, + value, + ].forEach((sourceText, variantIndex) => { + const normalized = + sourceText + .toLowerCase() + .replace(/\s+/g, " ") + .trim(); + + if ( + !normalized + || normalized.length < 24 + || seen.has( + normalized + ) + ) { + return; + } + + seen.add( + normalized + ); + + fragments.push( + { + id: `${idPrefix}:${index}:${variantIndex}`, + source, + sourceType, + citationType, + layer, + constantName: key || defaultConstantName, + sourceText, + titleText: line, + minScore: 0.72, + } + ); + }); + }); + + return fragments; + +} + +function getRuntimeCitationSnapshot( + snapshotIndex +) { + + const runtimeApi = + window.JinRuntime + && window.JinRuntime.runtime; + + if ( + runtimeApi + && typeof runtimeApi.getRuntimeMemorySnapshot === "function" + ) { + return ( + runtimeApi.getRuntimeMemorySnapshot( + snapshotIndex + ) + || null + ); + } + + const storage = + window.JinRuntime + && window.JinRuntime.storage; + + if ( + storage + && typeof storage.readLatestRuntimeMemory === "function" + ) { + const latestRuntime = + storage.readLatestRuntimeMemory(); + + if ( + latestRuntime + && latestRuntime.runtime_snapshot + ) { + return latestRuntime.runtime_snapshot; + } + + return latestRuntime || null; + } + + return null; + +} + +function getRuntimeCitationTextFromSnapshot( + snapshot +) { + + return String( + ( + snapshot + && ( + snapshot.raw_memory + || snapshot.runtime_memory + || ( + snapshot.runtime_snapshot + && snapshot.runtime_snapshot.raw_memory + ) + ) + ) + || "" + ).trim(); + +} + +function buildRuntimeCitationFragments( + snapshotIndex +) { + + const snapshot = + getRuntimeCitationSnapshot( + snapshotIndex + ); + const runtimeMemory = + getRuntimeCitationTextFromSnapshot( + snapshot + ); + + if (!runtimeMemory) { + return []; + } + + return buildMemoryCitationFragments( + runtimeMemory, + { + source: `runtimeSnapshot[${snapshotIndex}]`, + sourceType: "runtime", + citationType: "runtime_citation", + layer: "runtime", + idPrefix: `runtime:${snapshotIndex}`, + defaultConstantName: "runtime_memory", + } + ); + +} + +function buildSessionCitationFragments() { + + const storage = + window.JinRuntime + && window.JinRuntime.storage; + + if ( + !storage + || typeof storage.readLatestSavedSessionMemory !== "function" + ) { + return []; + } + + const savedSession = + storage.readLatestSavedSessionMemory(); + + if ( + !savedSession + || savedSession.explicit_save !== true + ) { + return []; + } + + const sessionMemory = + String( + savedSession.session_memory || "" + ).trim(); + + if (!sessionMemory) { + return []; + } + + return buildMemoryCitationFragments( + sessionMemory, + { + source: "latestSavedSessionMemory", + sourceType: "session", + citationType: "session_citation", + layer: "session", + idPrefix: "session", + defaultConstantName: "session_memory", + } + ); + +} + +function renderThinkRuleHighlights(job) { + + const element = + job.element; + + if ( + !element + || element.dataset.thinkId !== job.thinkId + ) { + return; + } + + const text = + job.text; + const matches = + resolveThinkRuleOverlaps( + job.matches + ); + + if (!matches.length) { + return false; + } + + const fragment = + document.createDocumentFragment(); + let cursor = 0; + + matches.forEach((match) => { + const start = Math.max( + 0, + Math.min( + text.length, + match.start + ) + ); + const end = Math.max( + start, + Math.min( + text.length, + match.end + ) + ); + + if (start > cursor) { + fragment.appendChild( + document.createTextNode( + text.slice( + cursor, + start + ) + ) + ); + } + + const matchedText = + text.slice( + start, + end + ); + const span = + document.createElement("span"); + + span.className = + getThinkCitationClassName( + match + ); + span.textContent = + matchedText; + span.title = + buildThinkRuleTitle( + match, + matchedText + ); + span.setAttribute( + "aria-label", + span.title + ); + span.style.setProperty( + "--think-match-score", + String( + Math.max( + 0, + Math.min( + 1, + Number( + match.score || 0 + ) + ) + ) + ) + ); + + fragment.appendChild( + span + ); + + cursor = end; + }); + + if (cursor < text.length) { + fragment.appendChild( + document.createTextNode( + text.slice( + cursor + ) + ) + ); + } + + element.replaceChildren( + fragment + ); + element.classList.add( + "has-rule-highlights" + ); + element.__jinThinkTextNode = null; + + updateThinkExpandedHeight( + element + ); + + job.matches = + matches; + + return true; + +} + +function pulseThinkRuleHighlights(job) { + + const element = + job.element; + + if ( + !element + || element.dataset.thinkId !== job.thinkId + ) { + return; + } + + if (element.__jinThinkRulePulseTimer) { + clearTimeout( + element.__jinThinkRulePulseTimer + ); + } + + element.classList.remove( + "is-rule-highlight-revealing" + ); + + void element.offsetWidth; + + element.classList.add( + "is-rule-highlight-revealing" + ); + + element.__jinThinkRulePulseTimer = setTimeout( + () => { + element.classList.remove( + "is-rule-highlight-revealing" + ); + element.__jinThinkRulePulseTimer = null; + }, + 5000 + ); + +} + +function handleThinkRuleWorkerMessage(event) { + + const data = + event.data + || {}; + const thinkId = + data.thinkId; + const job = + activeThinkRuleCitationJobs.get( + thinkId + ); + + if ( + !job + || !job.element + || job.element.dataset.thinkId !== thinkId + ) { + return; + } + + if ( + data.type === "ruleMatchesChunk" + ) { + job.matches = resolveThinkRuleOverlaps( + [ + ...job.matches, + ...(data.matches || []), + ] + ); + return; + } + + if ( + data.type === "ruleMatchesDone" + ) { + job.done = true; + if ( + renderThinkRuleHighlights( + job + ) + ) { + requestAnimationFrame( + () => pulseThinkRuleHighlights( + job + ) + ); + } + activeThinkRuleCitationJobs.delete( + thinkId + ); + } + +} + +function startThinkRuleCitationAnalysis( + messageId, + stream +) { + + if ( + !stream + || !stream.group + || !stream.group.createdThinking + || !stream.group.thinkContent + || !stream.thinking.trim() + ) { + return; + } + + const thinkContent = + stream.group.thinkContent; + const thinkId = + String( + messageId + ); + const text = + stream.thinking; + const runtimeCitationIndex = + Number.isInteger( + stream.runtimeCitationIndex + ) + ? stream.runtimeCitationIndex + : nextThinkRuntimeCitationIndex++; + + stream.runtimeCitationIndex = + runtimeCitationIndex; + + thinkContent.dataset.thinkId = + thinkId; + thinkContent.dataset.runtimeCitationIndex = + String( + runtimeCitationIndex + ); + thinkContent.__jinThinkRawText = + text; + + activeThinkRuleCitationJobs.set( + thinkId, + { + thinkId, + element: thinkContent, + text, + runtimeCitationIndex, + matches: [], + done: false, + } + ); + + loadThinkRuleCitationRegistry() + .then((registry) => { + const currentJob = + activeThinkRuleCitationJobs.get( + thinkId + ); + + if ( + !currentJob + || !registry.enabled + || !Array.isArray( + registry.fragments + ) + || thinkContent.dataset.thinkId !== thinkId + ) { + activeThinkRuleCitationJobs.delete( + thinkId + ); + return; + } + + const fragments = [ + ...registry.fragments, + ...buildRuntimeCitationFragments( + currentJob.runtimeCitationIndex + ), + ...buildSessionCitationFragments(), + ]; + + if (!fragments.length) { + activeThinkRuleCitationJobs.delete( + thinkId + ); + return; + } + + const worker = + getThinkRuleCitationWorker(); + + if (!worker) { + activeThinkRuleCitationJobs.delete( + thinkId + ); + return; + } + + worker.postMessage( + { + type: "analyzeThinkRules", + thinkId, + text, + fragments, + } + ); + }); + +} + function shouldAutoScroll() { @@ -310,6 +1152,10 @@ function flushStreamFrame() { stream.pendingThinking ); + updateThinkExpandedHeight( + stream.group.thinkContent + ); + stream.pendingThinking = ""; @@ -811,6 +1657,50 @@ function fadeRuntimeAction( // CREATE STREAM GROUP +function updateThinkExpandedHeight( + thinkContent +) { + + if (!thinkContent) { + return; + } + + thinkContent.style.setProperty( + "--jin-think-expanded-height", + `${thinkContent.scrollHeight}px` + ); + +} + +let thinkResizeFrame = null; + +window.addEventListener( + "resize", + () => { + + if (thinkResizeFrame) { + return; + } + + thinkResizeFrame = requestAnimationFrame( + () => { + + thinkResizeFrame = null; + + document + .querySelectorAll( + ".jin-think-content" + ) + .forEach( + updateThinkExpandedHeight + ); + + } + ); + + } +); + function createStreamGroup( role, contextSnapshot = null @@ -833,42 +1723,80 @@ function createStreamGroup( thinkWrapper.className = "jin-think-wrapper"; - const thinkHeader = - document.createElement("button"); - - thinkHeader.className = - "jin-think-header"; - - thinkHeader.innerHTML = - `▼ <think>`; - const thinkContent = document.createElement("div"); thinkContent.className = "jin-think-content"; + thinkContent.setAttribute( + "role", + "button" + ); + + thinkContent.setAttribute( + "tabindex", + "0" + ); + + thinkContent.setAttribute( + "aria-expanded", + "true" + ); + + thinkContent.setAttribute( + "aria-label", + "Toggle thinking block" + ); + let collapsed = false; - thinkHeader.onclick = () => { + const setCollapsed = (nextCollapsed) => { collapsed = - !collapsed; + nextCollapsed; - thinkContent.style.display = + thinkContent.classList.toggle( + "is-collapsed", collapsed - ? "none" - : "block"; + ); - thinkHeader.innerHTML = + thinkContent.setAttribute( + "aria-expanded", collapsed - ? `▶ <think>` - : `▼ <think>`; + ? "false" + : "true" + ); }; - thinkWrapper.appendChild( - thinkHeader + thinkContent.addEventListener( + "click", + () => { + setCollapsed( + !collapsed + ); + } + ); + + thinkContent.addEventListener( + "keydown", + (event) => { + + if ( + event.key !== "Enter" + && event.key !== " " + ) { + return; + } + + event.preventDefault(); + + setCollapsed( + !collapsed + ); + + } ); thinkWrapper.appendChild( @@ -959,6 +1887,9 @@ function ensureStreamGroup( stream.group.thinkContent = realGroup.thinkContent; + stream.group.thinkContent.dataset.thinkId = + stream.messageId; + stream.group.messageRow = realGroup.messageRow; @@ -996,6 +1927,7 @@ function startStreamMessage( messageId, { role, + messageId, context: contextSnapshot, group, thinking: "", @@ -1182,6 +2114,11 @@ function finishStreamMessage( ); } + startThinkRuleCitationAnalysis( + messageId, + stream + ); + } streamMessages.delete( diff --git a/ui/static/js/runtime/runtime.js b/ui/static/js/runtime/runtime.js index a73aa7c4..83c50a25 100644 --- a/ui/static/js/runtime/runtime.js +++ b/ui/static/js/runtime/runtime.js @@ -473,6 +473,22 @@ window.JinRuntime.runtime = { init() { return true; }, + getRuntimeMemorySnapshots() { + return runtimeMemoryHistory.snapshots; + }, + getRuntimeMemorySnapshot(index) { + const numericIndex = + Number(index); + + if ( + !Number.isInteger(numericIndex) + || numericIndex < 0 + ) { + return null; + } + + return runtimeMemoryHistory.snapshots[numericIndex] || null; + }, handleRuntimeMemoryMessage, renderRuntimeMemorySnapshot, renderDiffs: renderRuntimeDiffs, diff --git a/ui/static/js/think-rule-worker.js b/ui/static/js/think-rule-worker.js new file mode 100644 index 00000000..938a414f --- /dev/null +++ b/ui/static/js/think-rule-worker.js @@ -0,0 +1,872 @@ +const MIN_MATCH_CHARS = 24; +const MIN_MATCH_TOKENS = 4; +const MIN_RARE_TOKENS = 3; +const MIN_CONSECUTIVE_TOKEN_MATCH = 3; +const FRAGMENT_BATCH_SIZE = 8; + +const STOP_WORDS = new Set([ + "a", + "an", + "and", + "are", + "as", + "at", + "be", + "by", + "for", + "from", + "i", + "if", + "in", + "into", + "is", + "it", + "my", + "not", + "of", + "on", + "or", + "that", + "the", + "this", + "to", + "use", + "when", + "with", +]); + +function isLetterOrNumber(char) { + try { + return /[\p{L}\p{N}]/u.test( + char + ); + } catch (error) { + return /[a-z0-9]/i.test( + char + ); + } +} + +function normalizeWithMap(text) { + const output = []; + const map = []; + + function pushSpace(index) { + if (!output.length) { + return; + } + + if ( + output[output.length - 1] === " " + ) { + return; + } + + output.push(" "); + map.push(index); + } + + const source = String( + text || "" + ); + + for ( + let index = 0; + index < source.length; + index += 1 + ) { + const rawChar = source[index]; + const normalizedChars = rawChar.normalize + ? rawChar.normalize("NFKC").toLowerCase() + : rawChar.toLowerCase(); + + for (const char of normalizedChars) { + if ( + isLetterOrNumber( + char + ) + ) { + output.push( + char + ); + map.push( + index + ); + } else { + pushSpace( + index + ); + } + } + } + + while ( + output.length + && output[output.length - 1] === " " + ) { + output.pop(); + map.pop(); + } + + return { + text: output.join(""), + map, + }; +} + +function normalizeText(text) { + return normalizeWithMap( + text + ).text; +} + +function tokenizeNormalizedText(text) { + if (!text) { + return []; + } + + return text + .split(" ") + .filter(Boolean); +} + +function tokenizeOriginalText(text) { + const normalized = + normalizeWithMap( + text + ); + + const tokens = []; + let start = 0; + + while ( + start < normalized.text.length + ) { + while ( + normalized.text[start] === " " + ) { + start += 1; + } + + if ( + start >= normalized.text.length + ) { + break; + } + + let end = start; + + while ( + end < normalized.text.length + && normalized.text[end] !== " " + ) { + end += 1; + } + + tokens.push( + { + text: normalized.text.slice( + start, + end + ), + start: normalized.map[start], + end: normalized.map[end - 1] + 1, + } + ); + + start = end + 1; + } + + return tokens; +} + +function countRareTokens(tokens) { + const seen = new Set(); + + tokens.forEach((token) => { + if ( + token.length >= 4 + && !STOP_WORDS.has(token) + ) { + seen.add( + token + ); + } + }); + + return seen.size; +} + +function isUsablePhrase(tokens) { + const phrase = tokens.join(" "); + + return ( + tokens.length >= MIN_MATCH_TOKENS + && phrase.length >= MIN_MATCH_CHARS + && countRareTokens(tokens) >= MIN_RARE_TOKENS + ); +} + +function findAllNormalizedRanges( + haystack, + needle, + originalText +) { + const ranges = []; + + if ( + !haystack.text + || !needle + ) { + return ranges; + } + + let index = haystack.text.indexOf( + needle + ); + + while (index !== -1) { + const endIndex = + index + needle.length - 1; + + ranges.push( + expandMatchedRange( + originalText, + { + start: haystack.map[index], + end: haystack.map[endIndex] + 1, + } + ) + ); + + index = haystack.text.indexOf( + needle, + index + Math.max( + 1, + Math.floor( + needle.length / 2 + ) + ) + ); + } + + return ranges; +} + +function expandMatchedRange( + text, + range +) { + const source = + String( + text || "" + ); + + let start = + range.start; + let end = + range.end; + + const pairs = [ + ["<", ">"], + ["`", "`"], + ['"', '"'], + ["'", "'"], + ]; + + let expanded = true; + + while (expanded) { + expanded = false; + + for (const [left, right] of pairs) { + if ( + start > 0 + && end < source.length + && source[start - 1] === left + && source[end] === right + ) { + start -= 1; + end += 1; + expanded = true; + } + } + } + + return { + start, + end, + }; +} + +function buildMatch( + fragment, + range, + score, + level +) { + return { + start: range.start, + end: range.end, + score, + level, + constantName: fragment.constantName, + source: fragment.source, + sourceType: fragment.sourceType, + citationType: fragment.citationType, + layer: fragment.layer, + sourceText: fragment.sourceText, + titleText: fragment.titleText || fragment.sourceText, + }; +} + +function sourcePriority( + match +) { + if ( + match + && match.sourceType === "rule" + ) { + return 2; + } + + if ( + match + && match.sourceType === "runtime" + ) { + return 1; + } + + if ( + match + && match.sourceType === "session" + ) { + return 0; + } + + return -1; +} + +function findExactMatches( + fragment, + thinkNormalized, + originalText +) { + const sourceTokens = tokenizeNormalizedText( + normalizeText( + fragment.sourceText + ) + ); + + if ( + !isUsablePhrase( + sourceTokens + ) + ) { + return []; + } + + const matches = []; + const fullNeedle = sourceTokens.join(" "); + const fullRanges = findAllNormalizedRanges( + thinkNormalized, + fullNeedle, + originalText + ); + + fullRanges.forEach((range) => { + matches.push( + buildMatch( + fragment, + range, + 1, + "exact" + ) + ); + }); + + const maxWindowSize = Math.min( + 14, + sourceTokens.length - 1 + ); + + for ( + let size = maxWindowSize; + size >= MIN_MATCH_TOKENS; + size -= 1 + ) { + for ( + let start = 0; + start <= sourceTokens.length - size; + start += 1 + ) { + const phraseTokens = sourceTokens.slice( + start, + start + size + ); + + if ( + !isUsablePhrase( + phraseTokens + ) + ) { + continue; + } + + const needle = phraseTokens.join(" "); + const ranges = findAllNormalizedRanges( + thinkNormalized, + needle, + originalText + ); + + ranges.forEach((range) => { + matches.push( + buildMatch( + fragment, + range, + 0.92, + "exact" + ) + ); + }); + } + } + + return matches; +} + +function tokenOverlapScore( + sourceTokens, + thinkTokens +) { + const sourceCounts = new Map(); + + sourceTokens.forEach((token) => { + sourceCounts.set( + token, + ( + sourceCounts.get( + token + ) + || 0 + ) + 1 + ); + }); + + let overlap = 0; + let rareOverlap = 0; + + thinkTokens.forEach((token) => { + const count = + sourceCounts.get( + token + ) + || 0; + + if (!count) { + return; + } + + overlap += 1; + + if ( + token.length >= 4 + && !STOP_WORDS.has(token) + ) { + rareOverlap += 1; + } + + if (count === 1) { + sourceCounts.delete( + token + ); + } else { + sourceCounts.set( + token, + count - 1 + ); + } + }); + + const score = + overlap + / Math.max( + sourceTokens.length, + thinkTokens.length + ); + + return { + score, + rareOverlap, + }; +} + +function findBestConsecutiveTokenRun( + sourceTokens, + thinkTokens +) { + let best = { + sourceStart: 0, + thinkStart: 0, + length: 0, + }; + + for ( + let sourceStart = 0; + sourceStart < sourceTokens.length; + sourceStart += 1 + ) { + for ( + let thinkStart = 0; + thinkStart < thinkTokens.length; + thinkStart += 1 + ) { + let length = 0; + + while ( + sourceStart + length < sourceTokens.length + && thinkStart + length < thinkTokens.length + && sourceTokens[sourceStart + length] + === thinkTokens[thinkStart + length] + ) { + length += 1; + } + + if ( + length > best.length + ) { + best = { + sourceStart, + thinkStart, + length, + }; + } + } + } + + return best; +} + +function isLikelyStructuredLabelRange( + text +) { + return /(?:^|[\s*"`])[_A-Z0-9]{3,}(?:_[A-Z0-9]+)+\s*:/u.test( + text + ); +} + +function findTokenWindowMatches( + fragment, + thinkTokens, + originalText +) { + const sourceTokens = tokenizeNormalizedText( + normalizeText( + fragment.sourceText + ) + ); + + const matches = []; + const maxSourceWindow = Math.min( + 14, + sourceTokens.length + ); + + for ( + let sourceSize = maxSourceWindow; + sourceSize >= MIN_MATCH_TOKENS; + sourceSize -= 1 + ) { + for ( + let sourceStart = 0; + sourceStart <= sourceTokens.length - sourceSize; + sourceStart += 1 + ) { + const sourceWindow = sourceTokens.slice( + sourceStart, + sourceStart + sourceSize + ); + + if ( + !isUsablePhrase( + sourceWindow + ) + ) { + continue; + } + + const minThinkSize = Math.max( + MIN_MATCH_TOKENS, + sourceSize - 1 + ); + const maxThinkSize = Math.min( + thinkTokens.length, + sourceSize + 1 + ); + + for ( + let thinkSize = maxThinkSize; + thinkSize >= minThinkSize; + thinkSize -= 1 + ) { + for ( + let thinkStart = 0; + thinkStart <= thinkTokens.length - thinkSize; + thinkStart += 1 + ) { + const thinkWindow = thinkTokens.slice( + thinkStart, + thinkStart + thinkSize + ); + const thinkWindowTokens = thinkWindow.map( + (token) => token.text + ); + const result = tokenOverlapScore( + sourceWindow, + thinkWindowTokens + ); + const bestRun = findBestConsecutiveTokenRun( + sourceWindow, + thinkWindowTokens + ); + + if ( + result.score < 0.72 + || result.rareOverlap < MIN_RARE_TOKENS + || bestRun.length < MIN_CONSECUTIVE_TOKEN_MATCH + ) { + continue; + } + + const runStartToken = + thinkWindow[bestRun.thinkStart]; + const runEndToken = + thinkWindow[ + bestRun.thinkStart + + bestRun.length + - 1 + ]; + const matchedOriginalText = + originalText.slice( + runStartToken.start, + runEndToken.end + ); + + if ( + isLikelyStructuredLabelRange( + matchedOriginalText + ) + ) { + continue; + } + + const level = + result.score >= 0.82 + ? "near" + : "compressed"; + + matches.push( + buildMatch( + fragment, + expandMatchedRange( + originalText, + { + start: runStartToken.start, + end: runEndToken.end, + } + ), + Number( + result.score.toFixed( + 2 + ) + ), + level + ) + ); + } + } + } + } + + return matches; +} + +function levelRank(level) { + if (level === "exact") { + return 3; + } + + if (level === "near") { + return 2; + } + + return 1; +} + +function resolveOverlaps(matches) { + const sorted = [...matches].sort( + (left, right) => { + const priorityDelta = + sourcePriority( + right + ) + - sourcePriority( + left + ); + + if (priorityDelta) { + return priorityDelta; + } + + const levelDelta = + levelRank( + right.level + ) + - levelRank( + left.level + ); + + if (levelDelta) { + return levelDelta; + } + + if (right.score !== left.score) { + return right.score - left.score; + } + + return ( + (right.end - right.start) + - (left.end - left.start) + ); + } + ); + + const selected = []; + + sorted.forEach((match) => { + if ( + match.end <= match.start + || selected.some( + (selectedMatch) => ( + match.start < selectedMatch.end + && match.end > selectedMatch.start + ) + ) + ) { + return; + } + + selected.push( + match + ); + }); + + return selected.sort( + (left, right) => left.start - right.start + ); +} + +function analyzeThinkRules( + payload +) { + const { + thinkId, + text, + fragments, + } = payload; + + const thinkNormalized = + normalizeWithMap( + text + ); + const thinkTokens = + tokenizeOriginalText( + text + ); + + let index = 0; + + function processBatch() { + const batchMatches = []; + const end = Math.min( + index + FRAGMENT_BATCH_SIZE, + fragments.length + ); + + for ( + ; + index < end; + index += 1 + ) { + const fragment = + fragments[index]; + + batchMatches.push( + ...findExactMatches( + fragment, + thinkNormalized, + text + ) + ); + + batchMatches.push( + ...findTokenWindowMatches( + fragment, + thinkTokens, + text + ) + ); + } + + const resolvedMatches = + resolveOverlaps( + batchMatches + ); + + if (resolvedMatches.length) { + self.postMessage( + { + type: "ruleMatchesChunk", + thinkId, + matches: resolvedMatches, + } + ); + } + + if ( + index < fragments.length + ) { + setTimeout( + processBatch, + 0 + ); + return; + } + + self.postMessage( + { + type: "ruleMatchesDone", + thinkId, + } + ); + } + + processBatch(); +} + +self.onmessage = (event) => { + const data = + event.data + || {}; + + if ( + data.type !== "analyzeThinkRules" + ) { + return; + } + + analyzeThinkRules( + data + ); +}; diff --git a/ui/templates/index.html b/ui/templates/index.html index 6b5ecc44..18181972 100644 --- a/ui/templates/index.html +++ b/ui/templates/index.html @@ -730,31 +730,7 @@ opacity: 0.92; } - .jin-think-header { - /* Строка заголовка . */ - display: flex; - align-items: center; - gap: 8px; - - /* Приглушённый цвет заголовка think. */ - color: rgba(203, 213, 225, 0.72); - - /* Размер подписи think. */ - font-size: 11px; - - /* Плавность подсветки при hover. */ - transition: color 0.2s ease; - } - - .jin-think-header:hover { - /* Заголовок think становится читабельнее при наведении. */ - color: rgba(226, 232, 240, 0.96); - } - .jin-think-content { - /* Отступ между заголовком и телом мысли. */ - margin-top: 8px; - /* Внутренние отступы think-плашки. */ padding: 14px 18px; @@ -771,6 +747,11 @@ /* Сохраняет переносы текста think. */ white-space: pre-wrap; + /* Блок сам является переключателем сворачивания. */ + cursor: pointer; + overflow: hidden; + max-height: var(--jin-think-expanded-height, 65vh); + /* Масса think-плашки. Регулируется --jin-chat-panel-think. */ background: var(--jin-chat-panel-think); @@ -787,6 +768,96 @@ box-shadow: 0 8px 26px rgba(0, 0, 0, 0.28), inset 0 0 18px rgba(255, 255, 255, 0.025); + + /* Плавное сворачивание без исчезновения из ленты. */ + transition: + max-height 0.24s ease, + border-color 0.2s ease, + background-color 0.2s ease, + box-shadow 0.2s ease; + will-change: max-height; + } + + .jin-think-content:hover { + border-color: rgba(148, 163, 184, 0.24); + background: rgba(12, 12, 14, 0.52); + } + + .jin-think-content:focus-visible { + outline: 1px solid rgba(94, 234, 212, 0.42); + outline-offset: 3px; + } + + .jin-think-content.is-collapsed { + max-height: calc(1lh + 28px); + border-color: rgba(148, 163, 184, 0.11); + box-shadow: + 0 6px 20px rgba(0, 0, 0, 0.22), + inset 0 -18px 18px rgba(12, 12, 14, 0.42); + } + + .think-rule-hit { + color: inherit; + text-shadow: none; + transition: + color 1s ease, + text-shadow 1s ease; + } + + .jin-think-content.is-rule-highlight-revealing .think-citation-rule.exact, + .jin-think-content.has-rule-highlights:hover .think-citation-rule.exact { + color: #fff7df; + text-shadow: + 0 0 8px rgba(255, 245, 220, 0.55), + 0 0 18px rgba(255, 200, 130, 0.28); + } + + .jin-think-content.is-rule-highlight-revealing .think-citation-rule.near, + .jin-think-content.has-rule-highlights:hover .think-citation-rule.near { + color: #efe2c8; + text-shadow: 0 0 8px rgba(255, 235, 200, 0.28); + } + + .jin-think-content.is-rule-highlight-revealing .think-citation-rule.compressed, + .jin-think-content.has-rule-highlights:hover .think-citation-rule.compressed { + color: #d8cdb9; + text-shadow: 0 0 6px rgba(255, 235, 200, 0.16); + } + + .jin-think-content.is-rule-highlight-revealing .think-citation-runtime.exact, + .jin-think-content.has-rule-highlights:hover .think-citation-runtime.exact { + color: rgba(134,239,172,0.96); + text-shadow: 0 0 10px rgba(34,197,94,0.35); + } + + .jin-think-content.is-rule-highlight-revealing .think-citation-runtime.near, + .jin-think-content.has-rule-highlights:hover .think-citation-runtime.near { + color: rgba(134,239,172,0.86); + text-shadow: 0 0 8px rgba(34,197,94,0.26); + } + + .jin-think-content.is-rule-highlight-revealing .think-citation-runtime.compressed, + .jin-think-content.has-rule-highlights:hover .think-citation-runtime.compressed { + color: rgba(134,239,172,0.76); + text-shadow: 0 0 6px rgba(34,197,94,0.18); + } + + .jin-think-content.is-rule-highlight-revealing .think-citation-session.exact, + .jin-think-content.has-rule-highlights:hover .think-citation-session.exact { + color: rgba(216,180,254,0.94); + text-shadow: 0 0 10px rgba(168,85,247,0.32); + } + + .jin-think-content.is-rule-highlight-revealing .think-citation-session.near, + .jin-think-content.has-rule-highlights:hover .think-citation-session.near { + color: rgba(216,180,254,0.84); + text-shadow: 0 0 8px rgba(168,85,247,0.24); + } + + .jin-think-content.is-rule-highlight-revealing .think-citation-session.compressed, + .jin-think-content.has-rule-highlights:hover .think-citation-session.compressed { + color: rgba(216,180,254,0.74); + text-shadow: 0 0 6px rgba(168,85,247,0.16); } #chat-history .border-l.border-slate-500.pl-4.text-xs.text-zinc-300.italic.leading-relaxed.whitespace-pre-wrap { @@ -1405,10 +1476,6 @@