Skip to content

feat(protos): let agents improve themselves through scored candidate runs without exposing the yardstick - #611

Merged
yordis merged 10 commits into
mainfrom
yordis/feat-candidate-search-contracts
Oct 22, 2026
Merged

feat(protos): let agents improve themselves through scored candidate runs without exposing the yardstick#611
yordis merged 10 commits into
mainfrom
yordis/feat-candidate-search-contracts

Conversation

@yordis

@yordis yordis commented Oct 21, 2026 •
edited by blacksmith-sh Bot
Loading

Copy link
Copy Markdown
Member
  • Agents should get better at a task by trying variations of themselves, but each attempt must not churn agent revisions or bypass the existing proposal and verdict path.
  • A score is only trustworthy if the agent being judged can never see or reach the yardstick, including over the network.
  • Every attempt, run and measurement needs to stay an immutable fact that later decisions can point back to as it was at that moment.
  • What an agent learns should outlive a single task, so later work can build on it through skills and memory under existing hierarchy and policy.
  • Who may approve an agent's change must stay a human decision by default, with charter changes never delegated to an agent.

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

yordis added 2 commits October 21, 2026 04:08
@yordis
feat(protos): let agents improve themselves through scored candidate …
616c453 
…runs without exposing the yardstick

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>
@yordis
feat(protos): keep every improvement attempt and its measurement as i…
66c498c 
…mmutable, point-in-time facts

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>
@cursor

cursor Bot commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown

PR Summary

Medium Risk
Large new contract surface spanning evaluation isolation, egress, and verdict policy; enforcement is not in this PR, but mistaken future dispatch/query behavior could leak scores or widen network reach.

Overview
Introduces ADR#0072 and wires protobuf contracts for a recorded-candidate improvement loop: trials are immutable candidates (deltas over a base revision within an EditableSurface), runs pin candidate_ref on sessions/workflows, and promotions cite origin_candidate_ref on proposals—without minting revisions per trial.

Evaluation gets its own streams (versioned rubrics with withheld vocabulary, bindings, scored results tied to SessionPositionRef, blind comparisons with label→session mapping kept out of improver-facing APIs). A new score-free GetRunHistory query resolves labels server-side and can elide content as ELISION_REASON_WITHHELD_VOCABULARY.

Policy adds egress permissions (default deny; evaluation/rubric stores never reachable under evaluation) and verdict authorities (default human; charter class cannot delegate to agents). Library skills/memories move content to claim checks with promotion provenance back to proposals/candidates.

ADR#0031 now documents candidate_ref on the execution plan; session commands/events project it like checkout_ref, plus FORK_REASON_PROBE. buf.gen.yaml also excludes evaluation and library from Rust codegen alongside existing agent/session packages. SkillVersionRef.skill_id and EvaluationResultRef.evaluation_result_id change to typed id messages in place.

Reviewed by Cursor Bugbot for commit 8bde207. Bugbot is set up for automated code reviews on this repo. Configure here.

@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 109 files, which is 9 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: f9d89fb2-8fdc-4561-bce8-ef12f3a02dab
📥 Commits

Reviewing files that changed from the base of the PR and between 62c66d8 and 8bde207.

📒 Files selected for processing: 109
  • buf.gen.yaml
  • docs/adr/0031-agent-implementation-and-session-plan.md
  • docs/adr/0072-agents-improve-through-recorded-candidates.md
  • docs/adr/index.md
  • proto/trogonai/agents/agents/v1alpha1/dependencies.proto
  • proto/trogonai/agents/agents/v1alpha1/editable_surface.proto
  • proto/trogonai/agents/agents/v1alpha1/implementation.proto
  • proto/trogonai/agents/candidates/v1alpha1/candidate.proto
  • proto/trogonai/agents/candidates/v1alpha1/candidate_id.proto
  • proto/trogonai/agents/candidates/v1alpha1/candidate_recorded.proto
  • proto/trogonai/agents/candidates/v1alpha1/candidate_ref.proto
  • proto/trogonai/agents/candidates/v1alpha1/claim_check.proto
  • proto/trogonai/agents/candidates/v1alpha1/events.proto
  • proto/trogonai/agents/candidates/v1alpha1/faults.proto
  • proto/trogonai/agents/candidates/v1alpha1/record_candidate.proto
  • proto/trogonai/agents/proposals/v1alpha1/proposal.proto
  • proto/trogonai/agents/proposals/v1alpha1/proposal_id.proto
  • proto/trogonai/evaluation/bindings/v1alpha1/bind_rubric.proto
  • proto/trogonai/evaluation/bindings/v1alpha1/binding.proto
  • proto/trogonai/evaluation/bindings/v1alpha1/claim_check.proto
  • proto/trogonai/evaluation/bindings/v1alpha1/evaluation_binding_id.proto
  • proto/trogonai/evaluation/bindings/v1alpha1/evaluation_result_id.proto
  • proto/trogonai/evaluation/bindings/v1alpha1/evaluation_result_recorded.proto
  • proto/trogonai/evaluation/bindings/v1alpha1/events.proto
  • proto/trogonai/evaluation/bindings/v1alpha1/faults.proto
  • proto/trogonai/evaluation/bindings/v1alpha1/record_evaluation_result.proto
  • proto/trogonai/evaluation/bindings/v1alpha1/rubric_bound.proto
  • proto/trogonai/evaluation/bindings/v1alpha1/rubric_unbound.proto
  • proto/trogonai/evaluation/bindings/v1alpha1/score.proto
  • proto/trogonai/evaluation/bindings/v1alpha1/unbind_rubric.proto
  • proto/trogonai/evaluation/comparisons/v1alpha1/candidate_label.proto
  • proto/trogonai/evaluation/comparisons/v1alpha1/claim_check.proto
  • proto/trogonai/evaluation/comparisons/v1alpha1/close_comparison.proto
  • proto/trogonai/evaluation/comparisons/v1alpha1/comparison_closed.proto
  • proto/trogonai/evaluation/comparisons/v1alpha1/comparison_id.proto
  • proto/trogonai/evaluation/comparisons/v1alpha1/comparison_opened.proto
  • proto/trogonai/evaluation/comparisons/v1alpha1/comparison_run_enrolled.proto
  • proto/trogonai/evaluation/comparisons/v1alpha1/enroll_comparison_run.proto
  • proto/trogonai/evaluation/comparisons/v1alpha1/events.proto
  • proto/trogonai/evaluation/comparisons/v1alpha1/faults.proto
  • proto/trogonai/evaluation/comparisons/v1alpha1/open_comparison.proto
  • proto/trogonai/evaluation/rubrics/v1alpha1/add_rubric_version.proto
  • proto/trogonai/evaluation/rubrics/v1alpha1/claim_check.proto
  • proto/trogonai/evaluation/rubrics/v1alpha1/events.proto
  • proto/trogonai/evaluation/rubrics/v1alpha1/faults.proto
  • proto/trogonai/evaluation/rubrics/v1alpha1/register_rubric.proto
  • proto/trogonai/evaluation/rubrics/v1alpha1/retire_rubric.proto
  • proto/trogonai/evaluation/rubrics/v1alpha1/rubric_id.proto
  • proto/trogonai/evaluation/rubrics/v1alpha1/rubric_registered.proto
  • proto/trogonai/evaluation/rubrics/v1alpha1/rubric_retired.proto
  • proto/trogonai/evaluation/rubrics/v1alpha1/rubric_version_added.proto
  • proto/trogonai/evaluation/rubrics/v1alpha1/rubric_version_ref.proto
  • proto/trogonai/library/memories/v1alpha1/claim_check.proto
  • proto/trogonai/library/memories/v1alpha1/events.proto
  • proto/trogonai/library/memories/v1alpha1/faults.proto
  • proto/trogonai/library/memories/v1alpha1/memory_id.proto
  • proto/trogonai/library/memories/v1alpha1/memory_kind.proto
  • proto/trogonai/library/memories/v1alpha1/memory_provenance.proto
  • proto/trogonai/library/memories/v1alpha1/memory_superseded.proto
  • proto/trogonai/library/memories/v1alpha1/memory_written.proto
  • proto/trogonai/library/memories/v1alpha1/supersede_memory.proto
  • proto/trogonai/library/memories/v1alpha1/write_memory.proto
  • proto/trogonai/library/skills/ids/v1alpha1/skill_id.proto
  • proto/trogonai/library/skills/v1alpha1/add_skill_version.proto
  • proto/trogonai/library/skills/v1alpha1/archive_skill.proto
  • proto/trogonai/library/skills/v1alpha1/claim_check.proto
  • proto/trogonai/library/skills/v1alpha1/events.proto
  • proto/trogonai/library/skills/v1alpha1/faults.proto
  • proto/trogonai/library/skills/v1alpha1/register_skill.proto
  • proto/trogonai/library/skills/v1alpha1/skill_archived.proto
  • proto/trogonai/library/skills/v1alpha1/skill_provenance.proto
  • proto/trogonai/library/skills/v1alpha1/skill_registered.proto
  • proto/trogonai/library/skills/v1alpha1/skill_version_added.proto
  • proto/trogonai/policy/egress_permissions/v1alpha1/clear_egress_permission.proto
  • proto/trogonai/policy/egress_permissions/v1alpha1/egress_permission_cleared.proto
  • proto/trogonai/policy/egress_permissions/v1alpha1/egress_permission_initialized.proto
  • proto/trogonai/policy/egress_permissions/v1alpha1/egress_permission_set.proto
  • proto/trogonai/policy/egress_permissions/v1alpha1/events.proto
  • proto/trogonai/policy/egress_permissions/v1alpha1/faults.proto
  • proto/trogonai/policy/egress_permissions/v1alpha1/initialize_egress_permission.proto
  • proto/trogonai/policy/egress_permissions/v1alpha1/permission_policy.proto
  • proto/trogonai/policy/egress_permissions/v1alpha1/set_egress_permission.proto
  • proto/trogonai/policy/verdict_authorities/v1alpha1/authority_policy.proto
  • proto/trogonai/policy/verdict_authorities/v1alpha1/clear_verdict_authority.proto
  • proto/trogonai/policy/verdict_authorities/v1alpha1/events.proto
  • proto/trogonai/policy/verdict_authorities/v1alpha1/faults.proto
  • proto/trogonai/policy/verdict_authorities/v1alpha1/initialize_verdict_authority.proto
  • proto/trogonai/policy/verdict_authorities/v1alpha1/set_verdict_authority.proto
  • proto/trogonai/policy/verdict_authorities/v1alpha1/verdict_authority_cleared.proto
  • proto/trogonai/policy/verdict_authorities/v1alpha1/verdict_authority_initialized.proto
  • proto/trogonai/policy/verdict_authorities/v1alpha1/verdict_authority_set.proto
  • proto/trogonai/session/sessions/queries/v1alpha1/candidate_label.proto
  • proto/trogonai/session/sessions/queries/v1alpha1/comparison_id.proto
  • proto/trogonai/session/sessions/queries/v1alpha1/get_run_history.proto
  • proto/trogonai/session/sessions/queries/v1alpha1/history_item.proto
  • proto/trogonai/session/sessions/queries/v1alpha1/page_cursor.proto
  • proto/trogonai/session/sessions/queries/v1alpha1/query_error.proto
  • proto/trogonai/session/sessions/queries/v1alpha1/run_history_item.proto
  • proto/trogonai/session/sessions/v1alpha1/candidate_id.proto
  • proto/trogonai/session/sessions/v1alpha1/candidate_ref.proto
  • proto/trogonai/session/sessions/v1alpha1/create_child_session.proto
  • proto/trogonai/session/sessions/v1alpha1/create_session.proto
  • proto/trogonai/session/sessions/v1alpha1/fork_session.proto
  • proto/trogonai/session/sessions/v1alpha1/recover_session.proto
  • proto/trogonai/session/sessions/v1alpha1/session_forked.proto
  • proto/trogonai/session/sessions/v1alpha1/session_id.proto
  • proto/trogonai/session/sessions/v1alpha1/session_position_ref.proto
  • proto/trogonai/session/sessions/v1alpha1/session_started.proto
  • proto/trogonai/workflows/definitions/v1alpha1/workflow_definition.proto

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

cursor[bot]
cursor Bot reviewed Oct 21, 2026
View reviewed changes

@cursor cursor Bot left a comment •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread proto/trogonai/session/sessions/v1alpha1/session_started.proto
Comment thread proto/trogonai/agents/candidates/v1alpha1/candidate.proto
Comment thread proto/trogonai/agents/candidates/v1alpha1/candidate.proto
@yordis
refactor(protos): drop reserved field numbers since nothing has shipp…
e05d6a9 
…ed against these contracts

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>
cursor[bot]
cursor Bot reviewed Oct 21, 2026
View reviewed changes

@cursor cursor Bot left a comment •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread proto/trogonai/session/sessions/queries/v1alpha1/get_run_history.proto
@yordis
fix(protos): reject ambiguous or empty candidates and keep run-histor…
c7b6b15 
…y paging stable

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>
cursor[bot]
cursor Bot reviewed Oct 21, 2026
View reviewed changes

@cursor cursor Bot left a comment •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread proto/trogonai/agents/candidates/v1alpha1/candidate.proto
@yordis
fix(protos): refuse candidates naming skills or memory kinds that adm…
441d14c 
…ission would reject

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>
cursor[bot]
cursor Bot reviewed Oct 21, 2026
View reviewed changes

@cursor cursor Bot left a comment •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread proto/trogonai/session/sessions/queries/v1alpha1/get_run_history.proto
@yordis
fix(protos): let run-history scans resume across every run of a compa…
e99ca53 
…rison

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>
cursor[bot]
cursor Bot reviewed Oct 21, 2026
View reviewed changes

@cursor cursor Bot left a comment •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread proto/trogonai/session/sessions/queries/v1alpha1/run_history_item.proto
@yordis
fix(protos): keep interrupted assistant turns and tool calls distingu…
888e57e 
…ishable in run history

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>
cursor[bot]
cursor Bot reviewed Oct 21, 2026
View reviewed changes

@cursor cursor Bot left a comment •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread proto/trogonai/agents/candidates/v1alpha1/candidate.proto
Comment thread proto/trogonai/session/sessions/queries/v1alpha1/run_history_item.proto
Comment thread proto/trogonai/session/sessions/queries/v1alpha1/get_run_history.proto
@yordis
fix(protos): refuse untyped settings edits and unattributed run messa…
e17436b 
…ges, and name missing comparisons

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>
cursor[bot]
cursor Bot reviewed Oct 21, 2026
View reviewed changes

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit e17436b. Configure here.

Comment thread proto/trogonai/evaluation/bindings/v1alpha1/score.proto
yordis added 2 commits October 21, 2026 06:35
@yordis
fix(protos): refuse criterion scores that omit their measurement
f60af4e 
Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>
@yordis
feat(protos): let improver instructions carry their attribution and k…
8bde207 
…eep grading vocabulary out of what the judged reads

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>
@yordis
yordis merged commit 84eca3b into main Oct 22, 2026
18 checks passed
@yordis
yordis deleted the yordis/feat-candidate-search-contracts branch October 22, 2026 14:11
@yordis yordis added the buf:skip:breaking Skip the Buf breaking change check label Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cursor cursor[bot] cursor[bot] left review comments

Assignees

No one assigned

Labels

buf:skip:breaking Skip the Buf breaking change check

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@yordis