Skip to content

Fix the GLM run's failures; weigh cost, noise and predictions in self-edits - #60

Merged
yogthos merged 3 commits into
mainfrom
na2k-run-bugs
Sep 28, 2026
Merged

yogthos merged 3 commits into
mainfrom
na2k-run-bugs

Conversation

@yogthos

@yogthos yogthos commented Sep 28, 2026

Copy link
Copy Markdown
Owner

The endless-flight run on GLM-5.3 surfaced seven bugs, all fixed here. The worst: oversight woke the supervisor on levels rather than changes, spent its whole pass budget in 24 minutes, and left the last two hours unsupervised. plan also took a JSON-encoded list as a literal path, which withheld done until the owner ran out of turns. A board owner could close its own task with task close and skip review.

The rest applies the RRSI and DSPy review to the self-edit loop:

  • verdicts now carry token cost, a deadband measured from the tally's own spread, and a floor at the best kept result
  • an experiment names the version it tests and the signal it expects to move
  • a side-call prompt that stops asking for what its parser reads is refused
  • gates can record the value they compare at every check
  • unproductive project edits and the reviews written under an open change reach the supervisor's brief
  • the arena can contrast a passed and a failed run of one task and flags recurring edits that name their subject or drop a guard

The unrunnable repl manifest is removed.

Full suite green locally (2860 tests).

Oversight woke the supervisor on levels, so after one real pass it spent the
rest of its 12-pass budget in 24 minutes on the same unmet-gate count and was
absent for the two hours that needed it. It now wakes on what changed since
the last pass that spent a call.

GLM sends array arguments as JSON strings: plan took one entry as the path
["src/a.clj"] and withheld done for it until the branch ran out of turns, and
split saw no parts. Such strings are decoded, and list syntax left in a plan
entry is refused as not a path.

A board owner could close its dispatched task with task close, which skipped
review, the critic and acceptance. That close is refused and points at done.

Also: the contradiction check judges a denial per sentence; an invalid JSON
escape is named with its column instead of "No matching clause: 38"; the
repetition scan covers replies that end in a parse error and counts a short
phrase repeated far past the floor; the verify pass reads the judge's reply
without its reasoning.
…e-call formats

A verdict now carries tokens per turn on both sides and applies RRSI's cost
rule: a gain that added more tokens than it earned is costly, a flat change
that cut them is cheaper. The deadband comes from each side's own spread
instead of a fixed 0.15, and a result under the best a kept change reached
is below-best. An experiment can name the version it tests and the signal it
expects to move; the verdict settles the prediction and the lever memory
keeps both. manifest gets a versions action and the drift block shows why
the latest version was made.

A side call's prompt that stops asking for what its parser reads is refused
(gates.edn :reply-formats), and a judge reply with no verdict line is marked
unread instead of passing silently. Gates may declare :measure; every check
of one is kept in gate_checks with its value, fired or not, so a threshold
can be fitted from both sides of its cut.
…en promotions

The repl manifest could not run as a loop and its enforcement lives in
phases.edn, so it and its cells are gone. A project edit that has been in
force for failed runs and no shipped one is offered to the supervisor for
removal, like a dead gate. An open experiment is shown with what the reviews
said while it was in force, not only its fitness number.

The arena records where each run's database is kept, pairs a passed and a
failed run of one task and asks what the passing one did differently
(ARENA_CONTRAST), and flags a recurring edit that names its subject or drops
a guard its template had, before anyone promotes it.
@yogthos
yogthos merged commit c16d36f into main Sep 28, 2026
1 check passed
@yogthos
yogthos deleted the na2k-run-bugs branch September 28, 2026 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant