bug(evaluation): ON arm stops capturing after #1401 because the literal `eval:` scope does not exist

[Fengzdadi]

Describe the bug

The SWE-bench Pro console in evaluation/ passes POWERCONTEXT_CODEX_SCOPE_ID=eval:{run_id}:{arm} to Codex (evaluation/src/powercontext_eval/powercontext_sut.py:1146), but never creates that scope. Since #1401, an explicit scope must already exist on the Server, and scope IDs are generated by the Server (scp_…).

The Codex UserPromptSubmit hook resolves the scope through /v1/scope-bindings/resolve, gets a 404, and exits 0 without output. It neither captures nor recalls, so on current master the ON arm behaves like the OFF arm, and validate_treatment rejects it because prompt_sources == 0.

A related check can't be satisfied either. mcp_requests counts docker logs lines containing /mcp (powercontext_sut.py:2664). The Server runs uvicorn with access_log=False (src/powercontext/server/cli.py:266), and its own access log line does not include the request path. So the OFF check mcp_requests == 0 always passes without verifying anything, and web/reporting.py:252-257 requires mcp_requests > 0 for the ON arm, which these logs can't provide. I verified the log output locally but don't know how it interacts with the deployment behind the published report, so this part may need your input.

Proposed fix: create one scope per arm through POST /v1/scopes after Server readiness, with idempotency_key=eval:{run_id}:{arm}, and use the returned ID for Codex, the evidence query, validate_treatment and the report check. I have a PR ready for the scope part. How to count mcp_requests is a design choice I'd leave to you.

Steps to reproduce

This reproduces the Codex hook and Server path without running a SWE-bench task:

1. Start a Server from the checkout under test with an empty data directory:
   POWERCONTEXT_HOME=<tmp> uv run powercontext server run --host 127.0.0.1 --port 8000
   (the plugin reads its Server URL from .mcp.json, which points at port 8000).
2. Pipe a UserPromptSubmit payload into the hook with POWERCONTEXT_CODEX_SCOPE_ID=eval:repro:on:
   echo '{"hook_event_name":"UserPromptSubmit","prompt":"We decided to use OceanBase as the storage backend.","cwd":"<tmp>","session_id":"s1"}' | uv run --frozen --project integrations/codex/plugins/powercontext python integrations/codex/plugins/powercontext/hooks/recall.py
3. Count captured sources with the evaluation harness's own query against <tmp>/powercontext.db:
   SELECT COUNT(*) FROM pc_sources WHERE scope_id = 'eval:repro:on';
4. Control: create a scope with POST /v1/scopes, pass the returned scp_… ID instead, and repeat.

For the /mcp check: send an MCP initialize request to POST /mcp/ and search the Server output for /mcp.

Expected behavior

The ON arm captures the prompt into a scope the harness owns, and the treatment evidence reflects real plugin activity.

Actual behavior

Source	Scope passed to the hook	Captured sources
4e0f78ad (parent of #1401)	literal eval:…:on	1
259bee3a (#1401)	literal eval:…:on	0
master 49bb27f8	literal eval:…:on	0
master 49bb27f8 (control)	API-created scp_…	1

With the literal scope, resolve_scope_id raises ScopeBindingStatusError (HTTP 404) and main() returns 0 without output.

For the /mcp check, the initialize request returned 200 and the Server output contained 0 lines with /mcp.

The published SWE-bench Pro result (#1409, 2026-08-31) predates #1401 (2026-09-03), and literal scopes still captured before #1401, so the scope part does not affect the published numbers.

Environment

• PowerContext version: master 49bb27f8
• Python version: 3.13 (via uv)
• OS: macOS (arm64)
• Server: SQLite, no generation or embedding model configured (capture does not need one)

Are you willing to submit a PR to fix this bug?

• Yes, I would like to submit a PR.