docs: fix benchmark repository references

README.md

@@ -116,7 +116,7 @@ frontier_agent/  generic loop, scheduling, registries, AgentBus, observers
 plugins/tools/   web, shell, file, sandbox, and team tool implementations
 workflows/       ReAct and Agent Team pipelines, profiles, prompts, observers
 apodex/          terminal CLI/TUI, approvals, sessions, traces, and Docker path
-benchmarks/      public benchmark harness plus standalone FrontierSearchBench
+benchmarks/      public harness plus bundled FrontierSearchBench/FrontierChallenge
 ```
 
 More detail: [framework architecture](docs/framework.md),

benchmarks/README.md

@@ -9,13 +9,13 @@ The benchmark tree has three deliberately separate areas:
   benchmarks such as GDPval, HLE, OfficeQA, BrowseComp, and FrontierScience,
   together with their registry, runner, judges, dataset tooling, and local
   result directories.
-- [`frontier_search_bench/`](frontier_search_bench/) is the independently
-  maintained [FrontierSearchBench](https://github.com/ApodexAI/frontier-search-bench)
-  source tree. It is kept at the top level so it remains visible and can be
-  synced to its future standalone open-source repository.
-- [`frontierchallenge/`](frontierchallenge/) contains FrontierChallenge's
-  standalone Harbor runtime, 97-task registry, taxonomy, image recipes,
-  documentation, and Hugging Face dataset setup flow.
+- [`frontier_search_bench/`](frontier_search_bench/) contains
+  [FrontierSearchBench](frontier_search_bench/): 41 verifiable deep-search
+  queries, their official scorers, and batch evaluation tooling. Answer
+  collection is integrated through the adapter under [`public/`](public/).
+- [`frontierchallenge/`](frontierchallenge/) contains
+  [FrontierChallenge](frontierchallenge/)'s Harbor runtime, 97-task registry,
+  taxonomy, image recipes, documentation, and Hugging Face dataset setup flow.
 
 The public evaluation harness runs one question per Python subprocess. Runs are
 independently reproducible, resumable, and protected from a single hung task by
@@ -40,7 +40,7 @@ Each dataset registers its default workflow and scoring implementation. Passing
 | `frontier_science_olympiad` | FrontierScience | `stateful-react-agent` | LLM judge |
 | `deepsearchqa` | DeepSearchQA | `stateful-react-agent` | LLM judge |
 | `widesearch` | WideSearch | `stateful-react-agent` | Structural F1 |
-| `frontier_search` | FrontierSearchBench | `stateful-react-agent` | Official external batch scorer |
+| `frontier_search` | FrontierSearchBench | `stateful-react-agent` | Bundled post-collection batch scorer |
 | `officeqa` | OfficeQA | `stateful-react-agent` | Official deterministic reward |
 | `officeqa_full` | OfficeQA-Full | `stateful-react-agent` | Official deterministic reward |
 | `gdpval` | GDPval | `stateful-react-agent` | Deterministic deliverable validation |
@@ -108,8 +108,8 @@ benchmarks/
 │   ├── scripts/               dataset download and standardization
 │   ├── datasets/              local source data (gitignored)
 │   └── results/               local run artifacts (gitignored)
-├── frontier_search_bench/     standalone benchmark source and official scorers
-└── frontierchallenge/         standalone scientific-workflow benchmark runtime
+├── frontier_search_bench/     FrontierSearchBench queries and official scorers
+└── frontierchallenge/         FrontierChallenge scientific-workflow runtime
 ```
 
 ## Add a benchmark

benchmarks/frontier_search_bench/README.md

@@ -2,7 +2,7 @@
 
 Apodex's benchmark for deep search / deep research. The goal is to compare AI platforms side by side on challenging tasks that are long-horizon, retrieval-heavy, and verifiable.
 
-The repository holds two things:
+This directory holds two things:
 
 1. **Verifiable benchmark queries** (`queries/verifiable.json`) — 41 evaluation inputs whose answers are unique or programmatically checkable (numbers, entities, enumerable sets, …).
 2. **Evaluation scripts** (`eval/verifiable/`) — auto-scoring of model answers (41/41 implemented).
@@ -13,7 +13,12 @@ The repository holds two things:
 queries/verifiable.json  ──►  (external collection: platform answers → unified JSON)  ──►  eval/verifiable/
 ```
 
-Answer collection does not happen in this repository; the eval scripts only consume the **unified JSON** interface below.
+The benchmark implementation in this directory does not collect answers; its
+eval scripts consume the **unified JSON** interface below. FrontierAgent's
+collection adapter lives in
+[`../public/families/frontier_search.py`](../public/families/frontier_search.py),
+and the complete collect/export/score workflow is documented in
+[`../../docs/eval-frontier-search.md`](../../docs/eval-frontier-search.md).
 
 **Query file** (`queries/verifiable.json`, 41 items) — each entry looks like:
 
@@ -47,10 +52,10 @@ The eval scripts read `report_content` and fall back to `response`. This is the
 > python3 -c 'import json,pathlib;p=pathlib.Path;src=json.loads(p("queries/verifiable.json").read_text(encoding="utf-8"));p("queries/answer_template.json").write_text(json.dumps([{"id":e["id"],"query":e["query"],"report_content":"","response":""} for e in src],ensure_ascii=False,indent=2)+"\n",encoding="utf-8")'
 > ```
 
-## Repository layout
+## Directory layout
 
 ```
-frontier-search-bench/
+benchmarks/frontier_search_bench/
 ├── queries/
 │   └── verifiable.json           # 41 items
 └── eval/
@@ -63,7 +68,10 @@ frontier-search-bench/
 
 Requires **Python 3.10+**.
 
+From the FrontierAgent repository root:
+
 ```bash
+cd benchmarks/frontier_search_bench
 pip install -r eval/verifiable/requirements.txt
 cp eval/verifiable/.env.example eval/verifiable/.env   # fill in OPENROUTER_API_KEY
 ```
@@ -72,7 +80,9 @@ See [`eval/verifiable/README.md`](eval/verifiable/README.md) for details.
 
 ## Quickstart
 
-Prerequisite: unified JSON answer files prepared per the data contract above.
+Prerequisite: unified JSON answer files prepared per the data contract above,
+with the working directory set to `benchmarks/frontier_search_bench/` as shown
+in the environment setup.
 
 **1. Score a single query** (full instructions in [`eval/verifiable/README.md`](eval/verifiable/README.md)):
 

benchmarks/frontier_search_bench/UPSTREAM.md

@@ -1,29 +1,17 @@
-# Upstream provenance
+# Maintenance notes
 
-This directory is imported from
-[`ApodexAI/frontier-search-bench`](https://github.com/ApodexAI/frontier-search-bench)
-with Git subtree.
+This directory is maintained directly in FrontierAgent at
+`benchmarks/frontier_search_bench`. It contains the canonical query set and
+official scorers; it is not synchronized from a separate public repository.
 
-- Imported upstream commit: `0a5323b9823f8ee05486bbae11ca96999e5d5af9`
-- FrontierAgent prefix: `benchmarks/frontier_search_bench`
-- Import mode: squashed subtree
-
-This directory documents itself as if answer collection happened elsewhere,
-because upstream has no collector. In FrontierAgent it does: the adapter in
-`benchmarks/public/families/frontier_search.py` feeds the shared subprocess
-runner, and [`docs/eval-frontier-search.md`](../../docs/eval-frontier-search.md)
-owns the collect / export / score workflow. Read that instead of inferring a FrontierAgent
-workflow from the upstream README.
-
-Keep FrontierAgent adapters outside this directory when possible. Pull a future
-upstream release from the repository root with:
-
-```bash
-git subtree pull \
-  --prefix=benchmarks/frontier_search_bench \
-  https://github.com/ApodexAI/frontier-search-bench.git main --squash
-```
+Keep FrontierAgent-specific collection adapters outside this directory when
+possible. The adapter in
+[`../public/families/frontier_search.py`](../public/families/frontier_search.py)
+feeds the shared subprocess runner, while
+[`../../docs/eval-frontier-search.md`](../../docs/eval-frontier-search.md) owns
+the collect/export/score workflow.
 
 The official scorers contain ground truth. Do not expose this directory to the
-agent process during a scored collection run; see
-`docs/eval-frontier-search.md` for the isolation requirement.
+agent process during a scored collection run; see the
+[FrontierSearchBench evaluation guide](../../docs/eval-frontier-search.md) for
+the isolation requirement.

benchmarks/frontier_search_bench/eval/verifiable/README.md

@@ -1,6 +1,7 @@
 # Verifiable Eval — auto-scoring scripts
 
-Programmatic scoring for the 41 queries in `queries/verifiable.json` at the repository root.
+Programmatic scoring for the 41 queries in `queries/verifiable.json` at the
+FrontierSearchBench directory root.
 
 ## Directory layout
 
@@ -88,7 +89,9 @@ cp .env.example .env
 # edit .env and fill in OPENROUTER_API_KEY=sk-...
 ```
 
-`.env` may live in `eval/verifiable/`, its parent directory, or the repository root — all three locations are loaded automatically.
+Place `.env` at `eval/verifiable/.env`, the location consistently supported by
+the batch runner and all query scorers. Alternatively, export
+`OPENROUTER_API_KEY` and `OPENROUTER_BASE_URL` in the environment.
 
 The code reads two environment variables: `OPENROUTER_API_KEY` (required) and `OPENROUTER_BASE_URL` (optional, default `https://openrouter.ai/api/v1`). Models are called through the OpenAI SDK.
 

benchmarks/frontier_search_bench/eval/verifiable/scorers/query_15/auto_scorer.py

@@ -72,7 +72,7 @@ def get_client():
         base_url=base_url,
         api_key=api_key,
         default_headers={
-            "HTTP-Referer": "https://github.com/ApodexAI/frontier-search-bench",
+            "HTTP-Referer": "https://github.com/ApodexAI/FrontierAgent",
             "X-Title": "frontier-search-bench",
         },
     )

benchmarks/frontier_search_bench/eval/verifiable/scorers/query_38/auto_scorer.py

@@ -231,7 +231,7 @@ def get_client():
         base_url=base_url,
         api_key=api_key,
         default_headers={
-            "HTTP-Referer": "https://github.com/ApodexAI/frontier-search-bench",
+            "HTTP-Referer": "https://github.com/ApodexAI/FrontierAgent",
             "X-Title": "frontier-search-bench",
         },
     )

benchmarks/frontierchallenge/site/README.md

@@ -1,30 +1,32 @@
 # FrontierChallenge website
 
 This directory contains the dependency-free static website for the benchmark.
-It lives in the public FrontierChallenge repository so website content, task
-documentation, and leaderboard links can evolve together.
+It lives in FrontierAgent alongside FrontierChallenge's task documentation and
+leaderboard links so they can evolve together.
 
 Published preview: <https://urban-chainsaw-mnormyp.pages.github.io/>
 
 ## Preview locally
 
-From the repository root, run:
+From `benchmarks/frontierchallenge/`, run:
 
 ```sh
 python3 -m http.server 8000 --directory site
 ```
 
 Then visit <http://localhost:8000>. Stop the server with `Ctrl-C`.
 
-## Build for hosting
+## Build the Worker bundle
 
 ```sh
 python3 site/build_site.py
 ```
 
-The generated `site/dist/` directory is intentionally ignored by Git and is
-recreated for each hosting build. GitHub Pages deployment is defined in
-`.github/workflows/pages.yml`.
+The generated Worker module under `site/dist/` is intentionally ignored by Git
+and recreated for each bundle. The separate GitHub Pages deployment is defined
+in the repository-level
+[`frontierchallenge-pages.yml`](../../../.github/workflows/frontierchallenge-pages.yml)
+workflow.
 
 ## Paper figures
 

benchmarks/public/README.md

@@ -5,11 +5,11 @@ public benchmarks, including BrowseComp, HLE, OfficeQA, GDPval, and
 FrontierScience. It owns the registry, dataset adapters, judges, subprocess
 runner, download/standardization scripts, and gitignored local artifacts.
 
-The independently maintained FrontierSearchBench source is intentionally kept
-outside this package at [`../frontier_search_bench/`](../frontier_search_bench/).
-The adapter in `families/frontier_search.py` lets the shared runner collect its
-answers without mixing FrontierAgent-specific runtime code into that source
-tree.
+The bundled [FrontierSearchBench](../frontier_search_bench/) queries and scorers
+are kept in a sibling directory. The adapter in
+`families/frontier_search.py` lets the shared runner collect its answers while
+keeping FrontierAgent-specific runtime code out of the benchmark
+implementation.
 
 Run a one-question smoke evaluation after installing the eval dependencies and
 placing the selected dataset under `datasets/`:

benchmarks/public/runner/score_frontier_search.py

@@ -41,7 +41,7 @@ def _scorer_env() -> dict[str, str]:
             env["OPENROUTER_BASE_URL"] = env.get(url_var) or _DEFAULT_BASE_URL
             return env
 
-    # No credentials anywhere. Leave the key empty so the upstream scorer's
+    # No credentials anywhere. Leave the key empty so the bundled scorer's
     # own "OPENROUTER_API_KEY not set" check reports it.
     env["OPENROUTER_API_KEY"] = ""
     env.setdefault("OPENROUTER_BASE_URL", _DEFAULT_BASE_URL)

docs/eval-frontier-search.md

@@ -12,19 +12,19 @@ around the checkout. That is why it has its own page — [`eval.md`](eval.md) ow
 everything the benchmarks share (installation, judge preflight, datasets, the
 runner's options, results), and this page owns only what is specific here.
 
-The benchmark source lives at
-[`benchmarks/frontier_search_bench/`](../benchmarks/frontier_search_bench/) as a
-squashed Git subtree of
-[`ApodexAI/frontier-search-bench`](https://github.com/ApodexAI/frontier-search-bench);
-see [`UPSTREAM.md`](../benchmarks/frontier_search_bench/UPSTREAM.md) for the
-import provenance and the `git subtree pull` command. FrontierAgent's adapter is
-deliberately outside that tree, in `benchmarks/public/families/frontier_search.py`,
-so the subtree stays syncable. The upstream README states that answer collection
-does not happen in its repository — inside FrontierAgent it does, through the
-shared subprocess runner described below.
+The canonical benchmark implementation lives at
+[`benchmarks/frontier_search_bench/`](../benchmarks/frontier_search_bench/).
+FrontierAgent's collection adapter is deliberately kept outside that directory,
+in
+[`benchmarks/public/families/frontier_search.py`](../benchmarks/public/families/frontier_search.py),
+so the query/scorer implementation remains separate from the shared runtime.
+See its [maintenance notes](../benchmarks/frontier_search_bench/UPSTREAM.md) for
+the ownership and isolation boundaries. The benchmark implementation consumes
+unified JSON answer files; inside FrontierAgent, the shared subprocess runner
+collects and exports those answers as described below.
 
 No dataset download is needed. The 41 queries and all 41 official scorers are
-bundled with the subtree.
+bundled with FrontierAgent.
 
 ## Evaluation isolation requirement
 
@@ -46,7 +46,7 @@ and prints a warning into the run's own output; it is not evidence of isolation.
 
 ## Credentials
 
-The wrapper maps `JUDGE_API_KEY` and `JUDGE_BASE_URL` onto the upstream scorer's
+The wrapper maps `JUDGE_API_KEY` and `JUDGE_BASE_URL` onto the scorer's
 `OPENROUTER_API_KEY` and `OPENROUTER_BASE_URL`, always as a pair from one
 provider — set `OPENROUTER_*` directly and the `JUDGE_*` values are left alone.
 
@@ -81,7 +81,7 @@ interrupted collection continues where it stopped.
 
 ## 2. Export
 
-Export the run into the upstream unified JSON contract. For a multi-run
+Export the run into the benchmark's unified JSON contract. For a multi-run
 evaluation, export each `run_<n>` separately as one model/run input.
 
 ```bash
@@ -98,8 +98,9 @@ of 41 would report a headline score for the 12 easiest. Finish the run, or pass
 ## 3. Score
 
 Run all 41 official scorers. The wrapper reuses `JUDGE_API_KEY` and
-`JUDGE_BASE_URL`, executes a temporary copy of the upstream tree, and preserves
-aggregate plus per-query artifacts under the requested result directory.
+`JUDGE_BASE_URL`, executes a temporary copy of the benchmark implementation,
+and preserves aggregate plus per-query artifacts under the requested result
+directory.
 
 ```bash
 uv run python -m benchmarks.public.runner.score_frontier_search \
@@ -128,9 +129,9 @@ an explicit step.
 
 ## Scoring internals
 
-Upstream's own documentation is the reference for what the scorers do:
+The bundled scorer documentation is the reference for what the scorers do:
 [`eval/verifiable/README.md`](../benchmarks/frontier_search_bench/eval/verifiable/README.md)
 covers the CLI, the three-stage pipeline, normalization, and the meaning of
 coverage. The unified JSON answer contract it consumes — and which
-`export_frontier_search` produces — is documented in the subtree's
+`export_frontier_search` produces — is documented in the benchmark's
 [`README.md`](../benchmarks/frontier_search_bench/README.md).

docs/framework.md

@@ -10,7 +10,7 @@ plugins, and benchmark evaluation. The framework layer has no dependency on
 frontier_agent/   generic loop, scheduling, registries, AgentBus, observers
 plugins/tools/   tool implementations and sandbox policy
 workflows/       pipeline specs, profiles, prompts, workflow-owned observers
-benchmarks/      public benchmark harness plus standalone FrontierSearchBench
+benchmarks/      public harness plus bundled FrontierSearchBench/FrontierChallenge
 ```
 
 ## Runtime flow