Skip to content

feat: learning loops - price list, lessons from corrections, what it learns, eval candidates (09) - #56

Merged
pegner-ag merged 9 commits into
mainfrom
feat/09-learning
Oct 19, 2026
Merged

pegner-ag merged 9 commits into
mainfrom
feat/09-learning

Conversation

@pegner-ag

Copy link
Copy Markdown
Owner

Plan 09 (in tasks/f1-skeleton/task-09-learning/plan-09.md) and its implementation: OOAT learns from its own ledger.

  • Learning price list (spec §11): a Beta mean of acceptance per capability × task type, computed from rated runs and never stored. The Gate records p_accept with the candidate, plus eu_usd when the task has a value, and closes a task whose expected utility is negative as not worth its cost.
  • A lesson from a correction: after a Differently with text, the reception may offer a lesson (preference, fact or glossary). It is stored only on the operator's Remember tap, on the web and in the chat bridge (m:). A remembered lesson or open point keeps the class of the decision it comes from, and the tap endpoint requires the token's cap to cover that class. A lesson is not offered with a file-reading answer, when it looks like a secret, or when its text would raise the class.
  • What it learns: GET /api/v1/learning (scope read), shown as a section of the Usage page. It covers the price list, decision thresholds, the reception's corrections and quick cancels, forgotten items, and tasks to look at. Task ids and notes are shown only within the reader's cap, and a rating note with personal data is held back.
  • Eval candidates: evals/candidates.py exports corrections and rejected tasks to a file outside the repository only. evals/reception/run.py reads a folder of picked cases and gets --min-expected.
  • Spec/schema: the remember tap, lesson in body_reception_decided and the reception schema (plan 09 decision 6, approved with the plan).

Tests: 1469 passed, 7 skipped; web check clean, node tests 35/35. A final review was run; its findings are fixed in f969e0f.

🤖 Generated with Claude Code

pegner-ag and others added 9 commits October 19, 2026 00:34
…t it learns, eval candidates

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…from the ratings (09 task 1)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ecorded on the candidate (09 task 2)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…tored by the remember tap (09 task 3)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…reception's corrections, tasks to look at (09 task 5)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…f picked cases with a minimum share (09 task 6)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…s, no lesson with a file-reading answer or one that raises the class, kept lessons counted from stored items, rating notes with personal data held back (review of 09)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@pegner-ag
pegner-ag merged commit dd1831c into main Oct 19, 2026
4 checks passed
if kept:
result["memory"] = kept
result["buttons"] = [*result["buttons"], *(f"forget:{m}" for m in kept)][:12]
lesson = decision.get("lesson")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lessons are lost when the decision after a correction is a confirm. The lesson block runs only on paths that reach the end of _act. The confirm paths (return result | {"decision": "confirm", ...} at ~L453 and ~L465) and the rule-1 auto-start (~L460) return early. A Differently is often followed by another "I understand it as … Shall I start?". That happens whenever the corrected task has confidence below θ, or when the model replies confirm. In those cases the model's lesson is dropped without any sign. Every test (corrected(), test_a_lesson_from_a_personal_batch…) has the second reply be a confident task, so this path is never tested.

Suggestion: move the offer into a small helper and apply it on every return path, or explicitly decide that lessons only accompany final decisions and say so in the plan. Also add a test where the decision after Differently is a low-confidence task.

| 03f | Antigravity CLI connector (`agy`, Google login; replaced Gemini CLI) — design and plan: `task-03f-antigravity-cli/` (`plan-03f.md`) | 03 | done |
| 05 | REST intake and Telegram bot (tasks per project), HIL notifier with one-tap answers, dashboard: HIL queue, rating queue, exceptions, tasks. The operator's everyday work (submit, answer, rate) must not need the command line: the owner found the `ooat` commands very unfriendly (2026-10-05). Design: `task-05-operator-ux/design.md` (`ooat serve`, web app, chat bridge; plans 05a–05c, with 01c) | 04 | design |
| 07 | Project workspace: agents read and write the files of a project's folders through OOAT's own MCP file tools (no commands; containers later) — design: `task-07-project-workspace/design.md` (ADR 0017); plans 07a–07c | 04, 05 | 07a plan |
| 09 | Learning loops: the price list from ratings feeds the Gate's EU, a lesson from a correction (tap Remember), What it learns on the Usage page, eval candidates and a regression run — plan: `task-09-learning/plan-09.md` | 04, 08 | plan |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this PR implements 09, but the status column still says plan. Should it be done (or whatever status the owner wants)?

if value is not None: # spec §6 with no HIL or silent-error term measured yet (plan 09 decision 4)
candidate["eu_usd"] = value * p - estimate
candidates = [candidate]
if value is not None and candidate["eu_usd"] < 0:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This matches spec §6/§11 (V·p − C, Beta prior ×5). Two things to note:

  1. The rule just got stricter. With the card prior 0.7 (cap.general.complete_task), a task now closes once value < 1.43 × estimate. Before, it closed only when value < estimate. That is intended, but operators with valued tasks will see more "not worth its cost" closures from day one.
  2. The loop can lock itself out. Tasks closed here never get a contract or a rating, so after a few rejections p_accept for that task type can only stay where it is. It can never recover, and the type stays closed for good. Consider a floor (spec §6 q_min), shadow mode (spec §11), or at least showing "closed by EU" counts in What it learns so the operator can see it happening.

return self._close(task, data_class, budget, "CLOSED_ABSTAINED", "Not run: no permitted route.", missing)
candidates = [{"topology": "T2", "model_usd": estimate}]
if value is not None and estimate > value:
p = learning.p_accept(self._ledger.events(types=list(learning.EVENTS)), WORKER_CAPABILITY,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor (efficiency): every Gate run reads all TASK_SUBMITTED/CONTRACT_ISSUED/TASK_CLOSED/TASK_RATED events from the ledger and also reloads the card from disk. That's fine for F1 because it matches how θ is computed, but it grows linearly with the ledger on the submit path. Worth a TODO before the Postgres backend lands.

@claude

claude Bot commented Oct 19, 2026

Copy link
Copy Markdown

Advisory review — verdict: changes suggested (one functional bug, otherwise solid).

  1. Bug: lessons are dropped on the confirm path (reception/__init__.py _act). A lesson is offered only when _act runs to its end. The confirm and rule-1 auto-start paths return early, so a correction followed by another Shall I start? silently loses the lesson. Every test makes the decision after Differently a confident task, so this path is never tested. See inline.
  2. The Gate's EU check is stricter and can lock itself out (gate.py). The formula matches spec §6 and §11 (V·p − C, prior ×5, lazy abstention counts and justified doesn't, as in §11). Two consequences: with prior 0.7, tasks now close once V < 1.43·C; and tasks closed by EU are never rated, so a low p_accept can't recover. Consider q_min, shadow mode, or a visible count. See inline.
  3. Security invariants: no issues found.
    • The tap endpoint now checks remember/todo against the cap of the last decision's class.
    • A lesson is never offered when it looks like a secret, would raise the class, or is above the channel cap. When stored, it keeps the decision's class.
    • Storing a lesson needs a human tap (actor.kind = hil), in line with ADR 0018.
    • /learning shows task ids and notes only within the cap, and PII-scanned notes are held back. The aggregate counts cover the whole ledger, as plan decision 7 says.
    • evals/candidates.py refuses to write inside the repo and filters by --max-class.
    • Nothing writes to cost_card.observed: the cards keep {"n": 0}.
    • The ledger is still append-only.
  4. Schemas: the tap pattern, lesson in body_reception_decided and in reception.schema.json are consistent with the code. The p_accept and eu_usd candidate fields already existed. The PR says the spec and schema change was approved with plan 09 (decision 6); please confirm the owner signed it off, per CLAUDE.md.
  5. Docs: docs/description.md and docs/chat-bridge.md describe the new state. Nit: tasks/f1-skeleton/README.md still lists 09 as plan.
  6. Minor: the Gate scans the ledger on every submit (same pattern as θ), which is fine for F1.

I could not run pytest in this environment. I am relying on the PR's reported 1469 passed / 7 skipped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant