Skip to content

Reefine: close the loop with the person who asked - #468

Merged
BobbyZhouZijian merged 42 commits into
mainfrom
feat/reefine-loop
Sep 16, 2026
Merged

BobbyZhouZijian merged 42 commits into
mainfrom
feat/reefine-loop

Conversation

@quao627

@quao627 quao627 commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

Closes #435

What Reefine is, after this branch

Reefine is Reef's built-in recipe for growing a coding harness from plain-language asks. A person asks in their reef-pi session; the service writes the change as a versioned harness release; the session reports the result and walks the person through review, install and setup. No GPU, no benchmark: the served model proposes, a health floor keeps the tree working, and the person is the judge of whether the change is what they asked for.

The loop, end to end

  1. Ask. /reef-harness <what it should do> in the session. With a UI the session model first thinks the request through and asks up to three clarifying questions (options plus free text) with the reef_ask_user tool, then files the request with reef_file_request. --direct, a headless session, or reef-pi harness "<text>" from the shell file it as is.
  2. Watch. Filing prints a link, /reef/harness/requests/<id>/page?scenario=…&token=…, a page that refreshes itself and shows queued, proposing, evaluating, then the result. The session footer shows the step's phase and elapsed time; reef-pi harness … --wait does the same in the shell.
  3. The step, on the service. The proposer restates the request, lists its triggers and states and where each comes from, lists what only the person can provide as requires items (with a prompt sentence for setup), then writes the entries: skills, rules, agent commands or pi extensions. A second call reviews the entries against the request and records what is covered and what is not. The candidate runs one health episode ([health]: run a shell command, answer with its output); selection: floor publishes it when the floor holds. The current release is not re-run.
  4. The result, in the chat. A persistent message in the session (and one line in the shell): published, or ready but waiting for review because it changes an extension, or nothing changed and why (the proposer's own failure reason included), always with Details: /reef-versions <step>. Uncovered points from the review are listed. Background completion opens no confirmation or input dialog, so the person can continue chatting with the agent.
  5. Promote, install, set up. The completion notice names the commands to use when ready: /reef-versions <step> promote for a pending release, or /reef-versions <step> install for a published step. These explicit actions open the confirmation and setup flow; an explicit promote also offers to install its new head. The shell equivalents are reef-pi update and reef-pi setup. An env item asks for the value and stores it in <install root>/.reef-harness-env (mode 0600, injected into the agent's environment, never in the tree); a permission or service item shows its check and runs it after a confirm. Then /reload. Session start repeats the offer for anything still pending or unmet.

The pieces

Area What changed
Engine (reef/train/cordis_backend, reef/recipe/cordis.py) selection: floor with floor_score (candidate-only evaluation, sides=("candidate",)); StepProposal(mutations, notes) recorded as proposal_notes; the entries keyword so a proposer can update or remove a rules entry; refused requires recorded with their reason; StepProgress exposed by the backend for the request page.
Method (reef/recipe/reefine) Design-first request prompt; review call; requires salvaged (an env check that is not a variable name is normalized) and refused items recorded; undeclared_env for variables an extension reads without declaring; one health task replaces the three arithmetic tasks; larger reply budgets for reasoning models; the failure reason recorded when a step produces no change.
Service (reef/service) GET /reef/harness/requests/{id}/page with a responsive request/progress/outcome layout, README branding, dark theme and explicit next actions; both page routes accept ?scenario=&token= for browser links; the release page shows Design, Review, the floor result, refused requires and prompts; harness-evolve profile folded into reefine (the old name is an alias).
Client (reef/harness/adapters/pi/*.ts, reef/harness/client/wrapper.py) The two tools and the clarify flow; the watch with a timeout, persisted request ids, a durable custom-message report and non-blocking completion notices; explicit /reef-versions <step> install; session-start hints; reef-pi harness --wait, page, update, setup --json/--set/--run, --help; env warnings and setup offers at session start.
Core (reef/core/requirements.py) requires items gain an optional prompt.

Configuration

reef serve --recipe reefine --inference.upstream-url … --inference.upstream-model … (port 8901, token reef-local, state under .reef/reefine/). Defaults the recipe sets: training-mode: manual, requests: true, version_check: true, review_kinds: [code_extension], selection: floor, the [health] task. REEF_PROPOSER_MAX_TOKENS / REEF_PROPOSER_TIMEOUT_S override the model-call budgets.

Verification

The branch carries origin/main as of #489 (merge commits 91b2d817 and 0a436cb8; #461's plan call and #460's review rows are kept, see the "Merging" notes in the PR thread; the second merge touched only the READMEs). Checks on the merged branch:

  • tests/reef_service: 3260 passed, 67 skipped. Three modules fail to collect here because ray is not installed in this environment (test_multi_scenario_bridge, test_sao_bridge, test_slime_bridge); test_profile_builds_without_tutorial_or_training_imports was deselected because its python -I subprocess imports the editable install of the main checkout, not the branch.
  • mypy: clean (290 files). pre-commit run --all-files: clean. node --check on both pi extensions: clean. README pairing (check_readme_i18n.py): synchronized. Docs site link and contract checks: clean.
  • Live, against an OpenRouter endpoint (anthropic/claude-sonnet-5) on a second service (port 8902):
    • on the merged code, the request "when a task is finished, text me a one-line summary at my phone number using iMessage" took three proposer calls (plan 7 s, entries 36 s, review 24 s, all stop), produced a rules entry plus an extension, declared REEF_IMESSAGE_PHONE (env, with a prompt) and messages-automation (permission, with a check and a prompt), passed the health floor with one episode, recorded a complete review, and settled pending; reef-pi harness … --wait printed the result, the page link and the next steps;
    • before the merge, the same shape of request was walked through the wrapper: reef-pi setup --json listed both items unmet, reef-pi update refused with exit 3 and the list, setup --set stored the number in .reef-harness-env (mode 0600), setup --mark checked the permission off, update installed the head, and a headless session saw the stored variable in its environment and REEF_HARNESS_WRAPPER exported; a rules request settled selected with one episode and its request page showed proposing then the result; an extension release was promoted, installed and its tool used in a headless session.
  • Not yet run live: the in-session confirms (promote, install, setup, /reload) added in the last round need a TTY; they are covered by the node tests with a stubbed wrapper. That is the first thing to try when testing this branch by hand.

Non-blocking completion follow-up (d2288244)

Background completion previously opened a modal even while the agent was busy. Completion now reports the result and the commands to run when ready; /reef-versions <step> install starts installation explicitly, and pending releases still require promotion. No wire or persisted-format changes.

  • Regression reproduced before the fix: both interactive busy/idle cases opened dialogs. After the fix, the focused requests/version-check/entry suites passed: 82 passed. Coverage includes non-blocking selected/pending results in busy, idle and headless sessions, explicit install/setup, and rejecting installation of unpublished steps.
  • pre-commit run --all-files: passed. python -m mypy: passed (295 source files).
  • GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_SYSTEM=/dev/null python -m pytest tests/ -n 8 --dist loadfile -q --durations=5: 4283 passed, 63 skipped, 5 failed. Four network/timeout cases passed when rerun serially on both the original PR and this change. test_profile_builds_without_tutorial_or_training_imports fails on the original PR too: its isolated subprocess uses the main checkout's editable install in the reused environment.
  • The UI regression uses the existing Node harness; no live TTY session was exercised for this follow-up.

Request page redesign (baad00f4)

The request page previously gave the instruction, identifiers, progress and result nearly equal visual weight. It now highlights the request and next action, with a four-stage progress strip, status badges, separate outcome and change cards, and expandable session metadata and machine requirements. Desktop uses two columns; phones stack the cards. Light and dark themes use the original README Reef logo and its matching colors. Pending, rejected and skipped steps label their mutations as proposed changes; published and pending results show the explicit install or promote command.

The route, authentication/query propagation, five-second refresh until settlement, and stored data remain compatible. The page remains self-contained, with no JavaScript, external assets or new dependencies. API documentation and the existing request-page tests were updated.

  • Request-page, release-page and authentication suites: 35 passed, including the original README SVG, escaped content, pending/promoted states and the live request route.
  • Browser checks: 72 views across nine states, light/dark themes and widths 320/390/768/1440; no horizontal overflow or page errors. Inspected desktop and phone screenshots; also verified keyboard expansion and long content on a 320px screen.
  • pre-commit run --all-files: passed. python -m mypy: passed (295 source files).
  • GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_SYSTEM=/dev/null NO_PROXY='*' python -m pytest tests/ -n 4 --dist loadfile -q --durations=5: 4287 passed, 63 skipped, 1 failed. The sole failure is the previously reproduced environment issue in test_profile_builds_without_tutorial_or_training_imports: its isolated subprocess imports the main checkout's editable installation. The focused suite above also covers the logo correction added after full-suite collection.

CI sampling regression (d3fd7461)

The Python 3.11 package job failed because the backlog test could select an earlier sample with two unread records before training began. The test helper now matches all required metrics within one sample; the backlog test requires both two unread records and active training. A regression supplies both the earlier and running samples to verify the selection. Production behavior, dependencies and CI checks are unchanged.

  • Reproduced the old selection deterministically: it returned training/execution/active=0 despite a later running sample being available.
  • tests/reef_service/test_async_pipeline_core.py: 17 passed, followed by 10 consecutive successful runs (170 tests).
  • pre-commit run --all-files: passed.

Naming and compatibility follow-up (302aba97)

Harness code, UI, CLI output, tests, and documentation now use check, validation, evaluation, and result for the corresponding concepts. AGENTS.md records this preference for identifiers and serialized keys, while preserving conventional technical names such as neural-network gates.

New reviews and settled proposals write result; evaluation metadata uses evaluation, evaluation_sides, evaluation_task_count, and evaluation_context, and the active phase is evaluating. Readers accept the earlier review, metric, phase, and rollback-context names. Harness manifests and native inspection retain the gate alias, native inspection accepts verdicts as well as results, and renamed public classes/helpers retain import aliases. New code and displayed labels use the preferred names.

  • Full local suite: 4302 passed, 63 skipped with the previously reproduced editable-install environment failure (test_profile_builds_without_tutorial_or_training_imports) excluded. Remote package CI runs that test normally.
  • Final focused install/proposal/regression suite: 83 passed. Compatibility coverage includes old/new review keys, evaluation-side fields, rollback state, and native inspection names.
  • pre-commit run --all-files, mypy (295 files), Node syntax, docs links and docs contracts: passed. Final wording-only corrections also passed pre-commit.
  • Request-page browser checks: 72 views, with no overflow or page errors.
  • Remote CI for 302aba97: all required checks passed, including Python 3.10/3.11/3.12 and combined coverage. The first source 3.10 run timed out with two tests unfinished and no reported assertion failures; both tests passed locally in 5 seconds, and rerunning the failed jobs passed without code changes. The hang was not reproduced or diagnosed.

Automatic session install recovery (17e4013b)

A running session selects releases through the service/scenario saved at launch, while its wrapper previously reread the installation's model binding and scenario. Reusing that directory for another service or scenario could make update --release <id> query the wrong catalog and report that the release did not exist. A missing model binding also stopped updates despite the session retaining a valid connection.

Setup and update for the session's own installation now automatically use that session's service URL, scenario, and token. The resolved scenario is carried into the install-script request, so a successful install restores the correct configuration on disk without a restart or an additional confirmation. Pi's setup value/check calls also pass the selected release ID throughout the flow. Explicit commands targeting another installation remain independent. Recovery never borrows credentials from a replaced binding or substitutes another release when the requested ID is absent; the missing-release diagnostic still identifies the catalog queried.

This replaces the earlier mismatch-only refusal. The reporter's exact environment remains unavailable; automatic recovery covers the reproduced configuration mismatch and missing-binding cases.

  • Six recovery cases failed against the refusal-only behavior and pass after the change: update and setup writes with a different disk service, different disk scenario, or missing model binding. Additional cases verify no fallback to another catalog/head, no reuse of another service's token, and independent updates of a different install directory.
  • Wrapper, requests extension, and version-check suites: 153 passed before the final two boundary tests were added. Final full local suite: 4310 passed, 63 skipped, including both added tests, excluding only the previously documented editable-install environment failure in test_profile_builds_without_tutorial_or_training_imports.
  • Real Git/LFS and HTTP diagnostic: promote a release, replace the disk connection with a wrong address/scenario, then update using the active session. The generated install script names the original session's service, scenario, and selected release. Vendor install execution was stubbed for this diagnostic; focused wrapper tests execute their test install script.
  • pre-commit run --all-files and mypy (295 files): passed.
  • Remote PR checks for 17e4013b: all passed, including Python 3.10/3.11/3.12, combined coverage, lint, docs, Docker, and harness smoke tests.

Since the first draft

  • bd562630: the clarifying questions decide what gets built and never ask for setup values (a phone number, a credential, an account, a permission, or which app to use when the request names one); those are collected by reef-pi setup after the install. Options must be concrete and mutually exclusive; a default the model can settle is stated in the request instead of asked.
  • 5442ec4d, 7817f962: a request the service no longer knows (two polls in a row answer 404, as after a scenario reset) ends the session's watch with one notice and is dropped from the stored requests; reef-pi harness --wait prints the same line and exits 1.
  • 0a436cb8: origin/main up to docs: align README benchmark result links #489 merged (README wording only).

Not in this branch

AI assistance

Claude Code implemented this branch under the author's direction, in parallel worktrees per area, with live runs on this machine against an OpenRouter endpoint. Codex changed background completion to non-blocking notices, added the explicit install action, redesigned the request page with the README logo, and updated regression tests and documentation at the user's request. Codex also corrected the periodic-metrics test race reported by package CI and added its regression test. Codex also unified harness naming, updated the shared naming preference, and retained compatibility reads and aliases. Codex also reproduced stale session/install configuration failures and implemented automatic setup/update recovery using the active session and added catalog diagnostics. The author is responsible for the change.

🤖 Generated with Claude Code

…and refused requires

The release page renders, when the row carries them, a Design section after
Why (proposal_notes.design), a Review section after What changed (the
review verdict, the covered and uncovered points, and the variables a
written extension reads that no requires item names), the floor gate's
passed, failed, floor_score and gate_sides in Verdict, and the items the
step refused (training_request.refused_requires and
proposal_notes.refused_requires) with their reason under Setup. A row
without them renders as before. The HTTP API reference documents the new
step metrics and the page's sections.
…p the name as an alias

`reef serve --recipe harness-evolve` loaded the tutorial's proposer from the
checkout through PYTHONPATH and listened on 8900 without a token; issue #435
(item 8) folds it into the built-in reefine profile. `profile_names()` now
returns ("reefine",), `--recipe harness-evolve` resolves to the reefine
profile and the launcher prints one line on stderr saying so, the
unknown-profile error lists reefine and the former name, and the checkout
PYTHONPATH hook (`_PROFILE_METHODS`, `REEF_CHECKOUT`) goes away because the
reefine proposer ships in the wheel. The `harness_evolve` recipe kind and its
batch ids are unchanged.

Docs: the CLI reference and the harness guide name reefine (port 8901, token
reef-local, state under .reef/reefine/) and the alias; the root READMEs'
deployment section walks through the Reefine flow (serve, scenario, install,
`reef-pi harness` or `/reef-harness`, `/reef-versions`).
…d refused requires

- evolution.selection: floor gates the candidate alone against
  evolution.floor_score (default 1.0, positive, only with floor);
  CordisBackend.evaluate takes sides=, leaves the current_* keys out and
  records gate_sides when the pair is not run; settle_step carries the
  failure manifest over untouched when the committed tree was not gated;
  recheck_every with floor is a RecipeConfigError
- StepProposal(mutations, notes): prepare_step records _bounded(notes)
  under proposal_notes when non-empty; empty mutations skip as no proposal
- entries keyword: the step's snapshot entries reach a proposer whose
  signature names it, through _CallableProposer as well
- training_request.refused_requires lists what _proposer_requires dropped
  with the reason, present only when non-empty; the log lines stay
…reef-pi --wait, page and help

/reef-harness hands a request to the session model first, which asks what
is unclear with reef_ask_user and files it with reef_file_request; --direct
or a headless session files as is. A watch then polls the catalog and
reports the step's verdict with the next action, session start says the
commands exist and counts the releases awaiting review, and
/reef-versions <step> prints the design and what the review left
uncovered. version_check.ts warns once per unset env variable the
installed release requires.

reef-pi harness --wait prints the same verdict lines and exits 0, 1 or 2;
reef-pi page fetches a step's page into the cache and opens it; reef-pi
--help lists the subcommands before pi's help; doctor lists the releases
awaiting review. The API reference names the tools as reef's own.
… and a health floor

- ReefineRecipe defaults selection to floor; the profile and the tutorial
  deployment carry the one [health] task and say what the floor checks;
  the built-in scorer knows that task alone, and the evolve-your-harness
  shim grades its own three answers over the built-in final_assistant_text
  and forwards entries
- REQUEST_PROMPT asks for the design first (a restatement, the triggers
  and states with their sources and the explicit toggle rule, what the
  user must provide as requires with one example per kind), then the
  entries, complete and nothing more; the listing names every entry id
  from entries, so a rules entry updates by id
- a request returns StepProposal(mutations, notes): the design (1500
  chars), the review call's {verdict, covered, uncovered} (60 s, 1024
  tokens; a failure leaves it out and never blocks), refused_requires
  with parse_requires' reason after an env check is brought to the one
  variable it names, and undeclared_env from the process.env reads of a
  written extension; the failure path still returns a bare Mutation
- tutorials/reefine/run.py reads passed/failed beside wins/losses/ties
  and won means met the floor; the user guide and the tutorial README
  describe the floor, the notes on the step page and the ask-to-install
  flow, with the historical rows marked as run under selection: always
…s progress

A request whose proposer call failed (a thinking model spending the 4096
token reply budget on its reasoning, say) settled as "skipped: no proposal",
and the session only saw "produced no change (no proposal)"; the reason was
in the step record alone, and between the filing and the verdict the footer
never moved.

- evolution.py: _ask returns the reply and, on failure, a one-line reason
  (how long the call took, the reply budget and the error, with a
  REEF_PROPOSER_MAX_TOKENS hint for a reply without text). A request whose
  call fails or whose reply gives nothing to apply returns a StepProposal
  without mutations and the reason under notes["failure"], which the backend
  records for the skipped step; the failure path still returns None. The
  default reply budgets are 16384 for a request, 4096 for a failure step and
  2048 for the review; the override keeps working.
- requests.ts: the skipped line quotes the failure; the selected, rejected
  and skipped lines end with "Details: /reef-versions <step>."; the footer
  says queued until the request's record shows a step took it, then running
  with the elapsed time; the filing texts say a step usually takes one to
  three minutes.
- wrapper.py: the skipped line quotes the failure too, and --wait says once
  when the step started.
- release_page.py: a "proposer failure" row in the Verdict table.
- The docs, the tutorial README and run.sh follow.
GET /reef/harness/requests/{record_id}/page shows where the step for a
filed request stands (queued, proposing, gating, running, settling) and
reloads itself every five seconds until the catalog row that names the
request lands, then its verdict, its mutations, what the review left
uncovered and a link to the version page. The harness backend exposes
StepProgress (request id, phase, start, step record, the gate's episode
count) as CordisBackend.step_progress, set in prepare_step and evaluate
and cleared at settlement, abort, a skip or a failure.

Both page routes take the scenario and the token as query parameters
when the headers are absent, since a browser opening a link sends none;
the token then sits in the URL, so a deployment that hands out such
links is a local one. Every other route stays header-only.

Reefine's review call gets 8192 reply tokens: a reasoning model spends
the budget on its reasoning first and 2048 came back empty live.
… watch that survives a restart

A person filed a request through /reef-harness; the step settled four
minutes later as a pending release; no report appeared in the session,
and nothing said a pending release waits for a promote. The report was
one ctx.ui.notify line, which the next status line overwrote, and one
hung catalog read stalled every later tick of the watch.

- requests.ts: every filing answers with the request's page link
  (/reef/harness/requests/<id>/page?scenario=...&token=...) in the tool
  result and the accepted notice; the report is appended to the session
  as a custom message (customType reef-harness, no turn) beside the
  notice; the pending line says the release is not installed until
  promoted, with the promote command and the step's page link; every
  fetch carries an abort signal with a 10 s deadline
  (REEF_HARNESS_FETCH_MS); the filed requests are kept in
  .reef-harness-requests.json beside the release file (newest ten, none
  older than a day) until reported, and session_start reports a stored
  request whose row the catalog holds or re-arms the watch for one it
  does not; the session start line says how to see and promote what
  waits; /reef-versions <step> prints the step's page link with the
  query parameters beside the curl.
- wrapper.py: harness prints "watch it here: <link>" after the accepted
  line; --wait and doctor print the same pending sentence with the
  step's page link.
- The tests and the two guides follow.
… and the setup

requests.ts: after the report of a settle seen live, a selected release is offered for install
(Install release <id8> now?) and a pending one for promote first (Promote release <id8> now?, with
the step page link), then install; /reef-versions <step> promote offers the install too. The install
runs reef-pi update --release <id> through the wrapper (REEF_HARNESS_WRAPPER, else reef-pi beside
the release file; without either the commands are named), then the setup loop: setup --json lists the
items, each unmet one is asked once (input for env, confirm for a check) and handed to setup --set /
setup --run, one line per item, the ones left unmet named at the end; an update the wrapper refuses
for unmet items (exit 3) runs the loop first and then the update again. The last line is Installed
release <id8>. Type /reload to load it now. Headless nothing is asked.

version_check.ts: with a UI and a wrapper on disk, an unmet head is offered as Set up release <id8>
now? with the list, the setup loop runs, then the update offer follows; the update runs reef-pi update
through the wrapper when one exists (the bash pipeline stays as the fallback) and ends with the reload
line. Headless, and without a wrapper, the messages stay as they were.

The node runners stub pi.exec per argv (TEST_EXEC), the confirms (TEST_CONFIRM list) and the inputs.
…d --wait hands over each next step

A requires item gains an optional prompt, one sentence telling the person
what to enter or grant: parse_requires keeps it (stripped, at most 200
characters, dropped when blank), and the route and the backend screen it
as they screen a check. The values the person gives for env items live in
<install root>/.reef-harness-env (mode 0600, written by setup alone); the
wrapper sets each variable in the agent's environment unless the shell
sets it, exports REEF_HARNESS_WRAPPER, and counts an env item met when the
environment or the env file holds its variable. reef-pi setup asks for an
unmet env item's value (without echo for a credential-looking name) and
shows the prompt; --json, --set NAME=VALUE and --run NAME serve scripts
and the extensions one item at a time. reef-pi update fetches the install
script and runs it for this install root, refusing with exit 3 while an
item is unmet. On a terminal, harness --wait asks to install a selected
release (setup prompts, then update) or to promote and install a pending
one; without one it prints the commands. A reinstall leaves the env file
and any other file the tree does not own alone (covered by a test).
# Conflicts:
#	docs/user-guide/evolve-your-harness.rst
@github-actions github-actions Bot added area: core Core contracts and scenario lifecycle area: docs User and developer documentation area: harness Harness integration, recipes, and evolution surfaces area: service Service APIs, deployment, and command-line interfaces labels Sep 14, 2026
@github-actions

Copy link
Copy Markdown

Merge Oncall: @BobbyZhouZijian

The merge oncall will route reviews and keep this pull request moving toward a clear decision.

# Conflicts:
#	README.i18n.yaml
#	docs/user-guide/recipes/reefine.rst
#	reef/harness/adapters/pi/version_check.ts
#	reef/harness/client/wrapper.py
#	reef/recipe/cordis.py
#	reef/recipe/reefine/evolution.py
#	reef/service/deploy/orchestrator.py
#	reef/service/profiles/harness-evolve.yaml
#	reef/service/request_service.py
#	reef/train/cordis_backend/__init__.py
#	reef/train/cordis_backend/backend.py
#	reef/train/cordis_backend/strategies.py
#	tests/reef_service/test_harness_example.py
#	tests/reef_service/test_reefine_recipe.py
#	tests/reef_service/test_reefine_tutorial.py
#	tests/reef_service/test_selection.py
#	tests/reef_service/test_version_check.py
@quao627

quao627 commented Sep 14, 2026

Copy link
Copy Markdown
Contributor Author

Merging notes (origin/main up to #432, merge commit 91b2d81):

  • Engine: main's shape wins (TrajectoryItem, CandidateBackend + ProposalGate + StepRecords, contracts.py, plugin factories with .build(candidate_backend), candidate_backend naming). The branch's floor policy is re-expressed as FloorPlugin + FloorPluginFactory; StepProposal, entries, refused_requires, StepProgress and the request page keep their contracts.
  • Proposer: feat(reefine): a request gets a plan call, then the tool a step needs beside its rule #461's plan call (PLAN_PROMPT, the {plan} section, the platform sentence) sits inside the branch's design-first request prompt. Budgets after the merge: request 65536 tokens / 600 s (main's), review 16384 / 120 s, failure path 8192 / 60 s; REEF_PROPOSER_MAX_TOKENS and REEF_PROPOSER_TIMEOUT_S still override all of them.
  • Client: fix(harness): put the harness binary's directory first on PATH for the session #453's PATH line and feat(harness): say when a release waits for a person's review #460's doctor rows are kept (the review rows now carry the ?scenario=&token= page link); feat(harness): say when a release waits for a person's review #460's per-row notice in version_check.ts was dropped because requests.ts already prints one session-start line naming the steps and the promote command.
  • Service: main's handler naming and startup report; the branch's page routes, query-parameter auth and the harness-evolve alias notice on top. reef/service/profiles/harness-evolve.yaml stays removed.
  • README: main's newer text with the branch's Reefine deployment section re-applied; README.i18n.yaml regenerated.

quao627 and others added 6 commits September 15, 2026 09:53
# Conflicts:
#	README.i18n.yaml
#	README.md
#	README.zh.md
Report background verdicts without opening promote, install, or setup dialogs. Add an explicit versions install action and retain confirmation for user-initiated changes.

AI assistance: Codex implemented and tested this change at the user’s direction.
Highlight the request, progress, outcome and explicit next actions in a responsive layout with light and dark themes. Reuse the README Reef logo and preserve authenticated links and refresh behavior.

AI assistance: Codex implemented and verified the request page redesign under the user’s direction.
@blacksmith-sh

This comment has been minimized.

A periodic sample can report two unread records before training starts. Match backlog and active training in the same sample, and cover stale pre-training samples explicitly to prevent the package CI race.

AI assistance: Codex diagnosed the CI failure, updated the test helper and callers, and verified the regression.
Rename harness code, records, UI, and docs around checks, evaluations, and results. Keep legacy reads and public aliases for existing records and callers, and document the naming preference in AGENTS.md.

Implemented with Codex under the author's direction.
@BobbyZhouZijian BobbyZhouZijian changed the title [DO NOT MERGE] Reefine: close the loop with the person who asked Reefine: close the loop with the person who asked Sep 15, 2026
Check setup and update calls against the active session before querying or writing an installation that now points to another service or scenario. Include the queried catalog location when a requested release is absent, with URL credentials redacted.

Implemented with Codex under the author's direction.
Resolve setup and update through the active session's service, scenario, and token for its own installation, so replaced or missing disk bindings recover automatically. Carry the resolved scenario into the install request and pin setup actions to the requested release. Preserve independent installs and refuse to substitute a missing release.

Implemented with Codex under the author's direction.
Extract the document wrapper, design tokens, logo and status vocabulary
into page_chrome, then rebuild the version page on it: a branded header,
a hero carrying the step's result, cards per section, automatic light and
dark themes and a responsive layout, as the request page already had.

The version page now reads a commit time as a date rather than a Unix
timestamp, names each state and evaluation metric in the words a person
reads instead of the record's own keys, and links the neighbouring steps
and the served head, carrying the scenario and token it was opened with
so a browser can follow them. The request page keeps its markup byte for
byte and loses its duplicated stylesheet.

Implemented with opencode under the author's direction.
Escape during /reef-harness clarification could not cancel: it fell from
a question's select into the free text input, recorded "no answer", and
the request was filed anyway. Escape, the new "Cancel this request"
option, and no text in the free text answer now all drop the request, and
the model is told not to file it. The dialogs carry the turn's abort
signal, which was ignored before.

While a step runs, an animated spinner sits above the input box with the
phase and the time in the step, and ctrl+r expands it in place with the
request, the evaluation's episodes, the step record and the page link.
The step keeps running in the background either way, leaving the session
free. The phase comes from a new JSON route, requests/{id}/progress,
which answers the reading the request page already rendered as HTML; a
service without it leaves the spinner at queued.

Includes the author's earlier work marking the installed step in
/reef-versions.

Implemented with opencode under the author's direction.
Resolve the two README.i18n.yaml and docs/reference/configuration.rst
conflicts against the changes since the last merge (#495):
- configuration.rst: keep the floor selection and floor_score rows this
  branch adds alongside main's verifier_reward evaluate row.
- README.i18n.yaml: regenerate hashes for the merged README pair.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: core Core contracts and scenario lifecycle area: docs User and developer documentation area: harness Harness integration, recipes, and evolution surfaces area: service Service APIs, deployment, and command-line interfaces area: training Training execution, runtimes, algorithms, and pipeline reliability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Reefine: close the loop with the person who asked

2 participants