Repository navigation
feat: learning loops - price list, lessons from corrections, what it learns, eval candidates (09) - #56
Conversation
…t it learns, eval candidates Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…from the ratings (09 task 1) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ecorded on the candidate (09 task 2) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…tored by the remember tap (09 task 3) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…reception's corrections, tasks to look at (09 task 5) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…f picked cases with a minimum share (09 task 6) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…s, no lesson with a file-reading answer or one that raises the class, kept lessons counted from stored items, rating notes with personal data held back (review of 09) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
| if kept: | ||
| result["memory"] = kept | ||
| result["buttons"] = [*result["buttons"], *(f"forget:{m}" for m in kept)][:12] | ||
| lesson = decision.get("lesson") |
There was a problem hiding this comment.
Lessons are lost when the decision after a correction is a confirm. The lesson block runs only on paths that reach the end of _act. The confirm paths (return result | {"decision": "confirm", ...} at ~L453 and ~L465) and the rule-1 auto-start (~L460) return early. A Differently is often followed by another "I understand it as … Shall I start?". That happens whenever the corrected task has confidence below θ, or when the model replies confirm. In those cases the model's lesson is dropped without any sign. Every test (corrected(), test_a_lesson_from_a_personal_batch…) has the second reply be a confident task, so this path is never tested.
Suggestion: move the offer into a small helper and apply it on every return path, or explicitly decide that lessons only accompany final decisions and say so in the plan. Also add a test where the decision after Differently is a low-confidence task.
| | 03f | Antigravity CLI connector (`agy`, Google login; replaced Gemini CLI) — design and plan: `task-03f-antigravity-cli/` (`plan-03f.md`) | 03 | done | | ||
| | 05 | REST intake and Telegram bot (tasks per project), HIL notifier with one-tap answers, dashboard: HIL queue, rating queue, exceptions, tasks. The operator's everyday work (submit, answer, rate) must not need the command line: the owner found the `ooat` commands very unfriendly (2026-10-05). Design: `task-05-operator-ux/design.md` (`ooat serve`, web app, chat bridge; plans 05a–05c, with 01c) | 04 | design | | ||
| | 07 | Project workspace: agents read and write the files of a project's folders through OOAT's own MCP file tools (no commands; containers later) — design: `task-07-project-workspace/design.md` (ADR 0017); plans 07a–07c | 04, 05 | 07a plan | | ||
| | 09 | Learning loops: the price list from ratings feeds the Gate's EU, a lesson from a correction (tap Remember), What it learns on the Usage page, eval candidates and a regression run — plan: `task-09-learning/plan-09.md` | 04, 08 | plan | |
There was a problem hiding this comment.
Nit: this PR implements 09, but the status column still says plan. Should it be done (or whatever status the owner wants)?
| if value is not None: # spec §6 with no HIL or silent-error term measured yet (plan 09 decision 4) | ||
| candidate["eu_usd"] = value * p - estimate | ||
| candidates = [candidate] | ||
| if value is not None and candidate["eu_usd"] < 0: |
There was a problem hiding this comment.
This matches spec §6/§11 (V·p − C, Beta prior ×5). Two things to note:
- The rule just got stricter. With the card prior 0.7 (
cap.general.complete_task), a task now closes oncevalue < 1.43 × estimate. Before, it closed only whenvalue < estimate. That is intended, but operators with valued tasks will see more "not worth its cost" closures from day one. - The loop can lock itself out. Tasks closed here never get a contract or a rating, so after a few rejections
p_acceptfor that task type can only stay where it is. It can never recover, and the type stays closed for good. Consider a floor (spec §6q_min), shadow mode (spec §11), or at least showing "closed by EU" counts in What it learns so the operator can see it happening.
| return self._close(task, data_class, budget, "CLOSED_ABSTAINED", "Not run: no permitted route.", missing) | ||
| candidates = [{"topology": "T2", "model_usd": estimate}] | ||
| if value is not None and estimate > value: | ||
| p = learning.p_accept(self._ledger.events(types=list(learning.EVENTS)), WORKER_CAPABILITY, |
There was a problem hiding this comment.
Minor (efficiency): every Gate run reads all TASK_SUBMITTED/CONTRACT_ISSUED/TASK_CLOSED/TASK_RATED events from the ledger and also reloads the card from disk. That's fine for F1 because it matches how θ is computed, but it grows linearly with the ledger on the submit path. Worth a TODO before the Postgres backend lands.
|
Advisory review — verdict: changes suggested (one functional bug, otherwise solid).
I could not run |
Plan 09 (in
tasks/f1-skeleton/task-09-learning/plan-09.md) and its implementation: OOAT learns from its own ledger.p_acceptwith the candidate, pluseu_usdwhen the task has a value, and closes a task whose expected utility is negative as not worth its cost.m:). A remembered lesson or open point keeps the class of the decision it comes from, and the tap endpoint requires the token's cap to cover that class. A lesson is not offered with a file-reading answer, when it looks like a secret, or when its text would raise the class.GET /api/v1/learning(scope read), shown as a section of the Usage page. It covers the price list, decision thresholds, the reception's corrections and quick cancels, forgotten items, and tasks to look at. Task ids and notes are shown only within the reader's cap, and a rating note with personal data is held back.evals/candidates.pyexports corrections and rejected tasks to a file outside the repository only.evals/reception/run.pyreads a folder of picked cases and gets--min-expected.remembertap,lessoninbody_reception_decidedand the reception schema (plan 09 decision 6, approved with the plan).Tests: 1469 passed, 7 skipped; web check clean, node tests 35/35. A final review was run; its findings are fixed in f969e0f.
🤖 Generated with Claude Code