Skip to content

Add J-Space Contrast Lab MVP #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
arthurpanhku merged 1 commit into from
Aug 11, 2026
Merged

Add J-Space Contrast Lab MVP #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,18 @@ python -m j7scope_serve --backend hf \

<p align="center"><img src="docs/screenshots/compare.png" width="840" alt="Compare view: cross-trace rigor strip + aligned zh/en sessions" /></p>

**Contrast Lab · 成对证据显微镜** —— 在 Compare 之上加入整对 permutation
null、同语言上限、bootstrap CI、单概念 rank 轨迹和答案身份混杂检查。指标由
[`j7scope/contrast.py`](j7scope/contrast.py) 离线写入 `align.json`，浏览器只展示；连接
`hf` GPU sidecar 后，还可从同一页面发起一次 activation patch，并把当前证据、干预
结果与完整 provenance 导出为 Claim Card。

```bash
# 为任意两条已有 Trace v1 生成/更新 Contrast Lab 成对证据
python experiments/build_contrast.py trace-a trace-b \
--trace-root results/traces --mode cross_language
```

**Gallery** —— 所有 trace 的入口（读 `traces/index.json`，平行对自动归为一张对比卡）。

<p align="center"><img src="docs/screenshots/gallery.png" width="840" alt="J-Space Gallery landing page" /></p>
Expand Down Expand Up @@ -185,12 +197,14 @@ j7scope/
│ ├── fitting.py # J-lens 拟合 + 残差捕获（sidecar 复用）
│ ├── patching.py # M2 的跨语言 activation patching
│ ├── metrics.py # CKA / SVCCA / overlap 计算
│ ├── contrast.py # 成对 null / CI / 轨迹 / 混杂检查 + provenance
│ └── viz.py # 双语对照的读出可视化页面
├── apps/
│ ├── serve/ # ★ 实时 J-Space 旁路（sidecar）
│ │ ├── j7scope_serve/ # OpenAI 兼容代理 + SSE 侧信道（纯标准库）
│ │ ├── viewer/index.html # 自包含的实时视图
│ │ └── integrations/opencode/ # provider 配置 + 插件
│ ├── site/contrast.html # Contrast Lab（离线证据 + 可选 GPU 干预）
│ └── web/ # React/Vite J-Space Explorer（离线 artifact 浏览）
├── experiments/build_demo_run.py # 生成前端开发用 demo artifact（非真实实验）
├── notebooks/walkthrough_zh_en.ipynb
Expand Down
6 changes: 6 additions & 0 deletions apps/serve/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,10 +109,16 @@ trace at capture time; the viewer only displays it. Trace Schema v1 lives in
| `/v1/chat/completions` | POST | OpenAI chat, streaming or not; drives generation |
| `/v1/models` | GET | one model entry (harnesses probe this) |
| `/jspace/stream` | GET | SSE: one J-space event per generated token |
| `/jspace/interventions` | POST | one activation-patching probe (`hf` backend only) |
| `/`, `/replay.html`, `/compare.html`, `/live.html`, `/assets/*` | GET | the static platform ([`../site`](../site)) |
| `/traces/*` | GET | recorded Trace v1 files (when `--traces`/`--record` set) |
| `/health` | GET | backend / model / layer / viewer count |

`/health.capabilities.activation_patch` tells Contrast Lab whether the connected
sidecar can run a causal A/B probe. The intervention endpoint accepts
`source_prompt`, `target_prompt`, `patch_layer`, optional source/target positions,
and `topk`; it is deliberately unavailable on the synthetic mock backend.

### Side-channel event

Each token emits (see [`protocol.py`](j7scope_serve/protocol.py)):
Expand Down
56 changes: 56 additions & 0 deletions apps/serve/j7scope_serve/app.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,11 @@ def do_GET(self): # noqa: N802
"layer": self.backend.layer,
"is_demo": getattr(self.backend, "is_demo", False),
"viewers": self.bus.subscriber_count(),
"capabilities": {
"activation_patch": bool(
getattr(self.backend, "supports_intervention", False)
),
},
})
if path == "/v1/models":
return self._json({
Expand All @@ -150,6 +155,8 @@ def do_POST(self): # noqa: N802
path = self.path.split("?", 1)[0]
if path == "/v1/chat/completions":
return self._chat_completions()
if path == "/jspace/interventions":
return self._run_intervention()
return self._json({"error": "not found"}, status=404)

# ---- handlers ---------------------------------------------------------
Expand Down Expand Up @@ -287,6 +294,55 @@ def _chat_completions(self) -> None:
"total_tokens": seq},
})

def _run_intervention(self) -> None:
if not getattr(self.backend, "supports_intervention", False):
return self._json({
"error": "activation patching requires the hf backend on a GPU sidecar",
"capability": "activation_patch",
}, status=409)
body = self._read_body()
source_prompt = body.get("source_prompt")
target_prompt = body.get("target_prompt")
if not isinstance(source_prompt, str) or not source_prompt.strip():
return self._json({"error": "source_prompt is required"}, status=400)
if not isinstance(target_prompt, str) or not target_prompt.strip():
return self._json({"error": "target_prompt is required"}, status=400)
if len(source_prompt) > 20_000 or len(target_prompt) > 20_000:
return self._json({"error": "prompt exceeds 20,000 characters"}, status=400)
try:
patch_layer = int(body.get("patch_layer"))
source_position = int(body.get("source_position", -1))
target_position = int(body.get("target_position", -1))
topk = min(100, max(1, int(body.get("topk", 20))))
except (TypeError, ValueError):
return self._json({"error": "intervention positions and layers must be integers"}, status=400)
if patch_layer < 0 or patch_layer >= self.backend.layer:
return self._json({
"error": f"patch_layer must be between 0 and {self.backend.layer - 1}"
}, status=400)
try:
result = self.backend.intervene(
source_prompt=source_prompt,
target_prompt=target_prompt,
patch_layer=patch_layer,
source_position=source_position,
target_position=target_position,
topk=topk,
)
except (ValueError, IndexError) as exc:
return self._json({"error": str(exc)}, status=400)
except Exception as exc:
print(f" [intervention] failed: {exc!r}")
return self._json({
"error": "activation patching failed; inspect the sidecar log"
}, status=500)
raw_readout = result.pop("readout")
result["readout"] = protocol.bucket_readout(
raw_readout,
per_lang=self.server.per_lang, # type: ignore[attr-defined]
)
result["is_demo"] = False
return self._json(result)

def _maybe_record(self, buffered, messages) -> None:
record_dir = self.server.record_dir # type: ignore[attr-defined]
Expand Down
40 changes: 40 additions & 0 deletions apps/serve/j7scope_serve/backends.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ class Backend:
layer: int
is_demo: bool = False
jacobian_estimator: str = "not_applicable"
supports_intervention: bool = False

def generate(self, messages: Sequence[dict], **params) -> Iterator[Step]:
raise NotImplementedError
Expand Down Expand Up @@ -153,6 +154,8 @@ class HFBackend(Backend):
the model weights available. Structured to mirror MockBackend's Step stream.
"""

supports_intervention = True

def __init__(self, model_name: str = "Qwen/Qwen2.5-7B-Instruct", layer: int = 18,
topk: int = 24, max_new_tokens: int = 256, device: str = None,
jacobian_corpus: Sequence[str] = None, n_probes: int = 16,
Expand Down Expand Up @@ -391,6 +394,43 @@ def generate(self, messages: Sequence[dict], **params) -> Iterator[Step]:

cur = next_id.view(1, 1)

def intervene(
self,
*,
source_prompt: str,
target_prompt: str,
patch_layer: int,
source_position: int = -1,
target_position: int = -1,
topk: int = 20,
) -> dict:
"""Run one activation-patching A/B probe on the loaded GPU model."""
if self._model is None:
self.load()
from j7scope.patching import patch_and_readout

result = patch_and_readout(
self._jlens,
source_prompt,
target_prompt,
patch_layer=patch_layer,
src_pos=source_position,
tgt_pos=target_position,
k=topk,
)
return {
"kind": "activation_patch",
"model": self.model_name,
"model_revision": self.model_revision_resolved,
"lens_layer": self.layer,
"patch_layer": patch_layer,
"source_position": source_position,
"target_position": target_position,
"jacobian_sha1": self.jacobian_sha1,
"readout": result["readout"],
"next_token": result["next_token"],
}


def make_backend(kind: str, **kw) -> Backend:
if kind == "mock":
Expand Down
63 changes: 63 additions & 0 deletions apps/site/assets/jspace.css
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,3 +105,66 @@ button:hover { border-color: var(--accent); }
.export { display: flex; gap: 8px; flex-wrap: wrap; }

footer { margin-top: 22px; color: var(--muted); font-size: 11px; line-height: 1.6; }

/* Contrast Lab */
.lab-toolbar { display: flex; align-items: center; gap: 10px; flex-wrap: wrap;
padding: 10px 12px; margin-bottom: 12px; background: var(--panel); border: 1px solid var(--line); border-radius: 10px; }
.lab-toolbar label { color: var(--muted); font-size: 12px; }
.lab-toolbar .spacer { flex: 1; }
.lab-tabs { display: flex; gap: 3px; padding: 3px; background: var(--panel-2); border-radius: 8px; }
.lab-tab { border-color: transparent; background: transparent; color: var(--muted); }
.lab-tab.active { background: var(--panel); border-color: var(--line); color: var(--ink); }
.lab-summary { display: flex; align-items: center; gap: 8px; color: var(--muted); font-size: 11px; margin-bottom: 8px; flex-wrap: wrap; }
.lab-pair { display: grid; grid-template-columns: 1fr 1fr; gap: 12px; }
.lab-trace { min-width: 0; padding: 13px; background: var(--panel); border: 1px solid var(--line); border-radius: 12px; }
.lab-trace-head { display: flex; justify-content: space-between; gap: 8px; align-items: baseline; }
.lab-trace-head span { color: var(--muted); font-size: 10px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.lab-trace .stream { max-height: 92px; overflow: auto; margin: 10px 0; background: var(--panel-2); }
.lab-token-readout { display: grid; grid-template-columns: 1fr 1fr; gap: 8px; }
.lab-token-readout .col { padding: 10px; background: var(--panel-2); }
.lab-token-readout .rows { gap: 4px; }
.lab-token-readout .row { grid-template-columns: minmax(50px, auto) 1fr 34px; gap: 6px; }
.lab-token-readout .row:nth-child(n+5) { display: none; }
.lab-evidence { display: grid; grid-template-columns: minmax(0, 1.6fr) minmax(230px, .7fr); gap: 12px; margin-top: 12px; }
.lab-evidence h2, .lab-mode-panel h2, .lab-trajectory h2 { font-size: 14px; margin: 0 0 8px; }
.lab-evidence .rigor { margin: 0; }
.lab-calibration, .lab-mode-panel, .lab-trajectory, .lab-provenance { padding: 14px 16px; background: var(--panel); border: 1px solid var(--line); border-radius: 12px; }
.lab-calibration dl { margin: 0; display: grid; gap: 10px; }
.lab-calibration dl div { display: flex; justify-content: space-between; gap: 10px; padding-bottom: 8px; border-bottom: 1px solid var(--line); }
.lab-calibration dl div:last-child { border: 0; padding: 0; }
.lab-calibration dt { color: var(--muted); font-size: 11px; }
.lab-calibration dd { margin: 0; font-weight: 650; font-variant-numeric: tabular-nums; }
.lab-mode-panel { margin-top: 12px; }
.lab-mode-panel .caption, .lab-section-head span { color: var(--muted); font-size: 11px; }
.intervention-form { display: flex; gap: 10px; align-items: end; flex-wrap: wrap; margin-top: 12px; }
.intervention-form label { display: grid; gap: 4px; color: var(--muted); font-size: 11px; }
.intervention-form input { width: 94px; padding: 5px 7px; color: var(--ink); background: var(--panel-2); border: 1px solid var(--line); border-radius: 6px; }
.intervention-result { margin-top: 12px; color: var(--muted); font-size: 12px; }
.intervention-result pre, .lab-provenance pre { overflow: auto; max-height: 280px; color: var(--ink); background: var(--panel-2); padding: 10px; border-radius: 8px; }
.confound { display: flex; align-items: center; gap: 12px; padding: 12px; border: 1px solid var(--line); border-radius: 9px; }
.confound span { color: var(--muted); }
.confound.controlled { border-color: #29513c; }.confound.controlled b { color: var(--good); }
.confound.confounded { border-color: #6a4720; }.confound.confounded b { color: var(--warn); }
.lab-trajectory { margin-top: 12px; }
.lab-section-head { display: flex; justify-content: space-between; gap: 12px; align-items: start; }
.trajectory-chart { min-height: 220px; margin-top: 8px; }
.trajectory-chart svg { display: block; width: 100%; height: auto; overflow: visible; }
.trajectory-chart text { fill: var(--muted); font-size: 10px; }
.trajectory-chart .grid { stroke: var(--line); stroke-dasharray: 3 4; }
.trajectory-chart .cursor { stroke: var(--ink); stroke-width: 1; opacity: .35; }
.trajectory-chart line.series-a { stroke: var(--en); stroke-width: 2.5; }
.trajectory-chart circle.series-a { fill: var(--en); }
.trajectory-chart line.series-b { stroke: var(--zh); stroke-width: 2.5; }
.trajectory-chart circle.series-b { fill: var(--zh); }
.lab-provenance { margin-top: 12px; color: var(--muted); }
.lab-provenance summary { cursor: pointer; }
@media (max-width: 760px) {
.lab-pair, .lab-evidence { grid-template-columns: 1fr; }
.lab-token-readout { grid-template-columns: 1fr 1fr; }
.lab-toolbar .spacer { display: none; }
}
@media (max-width: 440px) {
.lab-token-readout { grid-template-columns: 1fr; }
.lab-tabs { width: 100%; }
.lab-tab { flex: 1; padding-inline: 4px; }
}
2 changes: 2 additions & 0 deletions apps/site/compare.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
<h1><a href="./">J7Scope</a> <span class="sub">Compare · 双生对齐</span></h1>
<nav>
<a href="./">← Gallery</a>
<a id="labLink" href="contrast.html">Contrast Lab</a>
<a href="https://github.com/arthurpanhku/j7scope">GitHub</a>
</nav>
</header>
Expand Down Expand Up @@ -51,6 +52,7 @@ <h2 style="font-size:14px;margin:16px 0 4px">跨会话 · Cross-trace rigor <spa
var J = window.JSpace, $ = function (id) { return document.getElementById(id); };
var params = new URLSearchParams(location.search), group = params.get("group");
if (!group) { location.href = "./"; return; }
$("labLink").href = "contrast.html?group=" + encodeURIComponent(group);

var A = null, B = null, MAP = null, PR = null, pos = 0, playing = false, timer = null;

Expand Down
Loading
Loading