Skip to content

Commit 7acea04

Browse files
authored
feat(results): signal auto-merged remote pushes
Adds an auto_merged_remote sync signal for diverged result-branch auto-merges, hard-deprecates the next-only backup_and_force_push policy, and documents the release-channel learning.
1 parent dae788e commit 7acea04

17 files changed

Lines changed: 268 additions & 106 deletions

File tree

‎CONCEPTS.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,3 +33,11 @@ Shared domain vocabulary for this project — entities, named processes, and sta
3333
**Gate policy** — The explicit rule that decides whether repeated attempts pass CI, such as `all_attempts_successful`, `any_attempt_successful`, `attempt_success_rate_at_least`, or `mean_pass_rate_at_least`. Without a repeat-run gate policy, AgentV preserves the normal single-run gate behavior and treats repeat statistics as report data.
3434

3535
**Flaky eval outcome** — A repeat-run aggregate whose attempts disagree, or whose failure classification points at verifier, infrastructure, or timeout instability rather than a stable model-quality failure.
36+
37+
## Release Channels
38+
39+
**Stable release** — A package publication channel whose surfaces are treated as compatibility commitments for normal users.
40+
41+
**Next tag** — A prerelease package channel used to validate upcoming AgentV surfaces before they become stable compatibility commitments.
42+
43+
Next-tag-only surfaces may be hard-corrected before stable release when preserving them would encode an unsafe or misleading contract. Stable-release surfaces need an explicit compatibility or migration strategy.

‎apps/cli/src/commands/results/remote.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -156,7 +156,7 @@ export interface ResultsPublishOverrides {
156156
readonly remote?: string;
157157
readonly auto_push?: boolean;
158158
readonly require_push?: boolean;
159-
readonly push_conflict_policy?: 'block' | 'backup_and_force_push';
159+
readonly push_conflict_policy?: 'block';
160160
}
161161

162162
const REMOTE_RUN_PREFIX = 'remote::';

‎apps/dashboard/src/lib/project-sync-status.test.ts‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -205,6 +205,21 @@ describe('buildProjectSyncFeedback', () => {
205205
expect(feedback.message).toContain('pulled remote results');
206206
});
207207

208+
it('surfaces auto-merged remote changes in successful sync feedback', () => {
209+
const feedback = buildProjectSyncFeedback({
210+
configured: true,
211+
available: true,
212+
sync_status: 'clean',
213+
auto_merged_remote: true,
214+
push_performed: true,
215+
run_count: 2,
216+
});
217+
218+
expect(feedback.kind).toBe('success');
219+
expect(feedback.message).toContain('Merged remote (auto)');
220+
expect(feedback.message).toContain('pushed local results');
221+
});
222+
208223
it('keeps blocked sync feedback explicit', () => {
209224
expect(
210225
buildProjectSyncFeedback({

‎apps/dashboard/src/lib/project-sync-status.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -334,6 +334,7 @@ export function buildProjectSyncFeedback(status: RemoteStatusResponse): {
334334
const actions = [
335335
status.commit_created ? 'committed pending metadata' : undefined,
336336
status.pull_performed ? 'pulled remote results' : undefined,
337+
status.auto_merged_remote ? 'Merged remote (auto)' : undefined,
337338
status.push_performed ? 'pushed local results' : undefined,
338339
].filter((action): action is string => action !== undefined);
339340

‎apps/dashboard/src/lib/types.ts‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -471,7 +471,7 @@ export interface RemoteStatusResponse {
471471
local_dir?: string;
472472
path?: string;
473473
auto_push?: boolean;
474-
push_conflict_policy?: 'block' | 'backup_and_force_push';
474+
push_conflict_policy?: 'block';
475475
branch_prefix?: string;
476476
run_count?: number;
477477
last_synced_at?: string;
@@ -500,6 +500,7 @@ export interface RemoteStatusResponse {
500500
pull_performed?: boolean;
501501
push_performed?: boolean;
502502
commit_created?: boolean;
503+
auto_merged_remote?: boolean;
503504
target_branch?: string;
504505
remote_commit?: string;
505506
local_commit?: string;

‎apps/web/src/content/docs/docs/tools/dashboard.mdx‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -295,7 +295,7 @@ projects:
295295
push_conflict_policy: block
296296
```
297297

298-
`results.repo.remote` is the Git remote URL AgentV fetches and pushes. `results.repo.path: .` stores completed run artifacts on a dedicated branch of the source repository without checking out that branch in the source worktree. AgentV manages the local Git remote alias for that URL, so the normal config stays portable across machines. When `results.repo.remote` is omitted, `results.repo.path` means an existing local Git checkout whose object database and refs AgentV should write to, and the branch defaults to `agentv/results/v1`. AgentV creates the branch automatically on first publish and commits only AgentV result paths into it. `sync.auto_push: false` keeps the result commit local; set it to `true` to push the branch best-effort after each completed run. `sync.require_push: true` is for CI workflows where a push failure should fail the command after local artifacts are written. `sync.push_conflict_policy` defaults to `block`; the `backup_and_force_push` value is deprecated and no longer force-pushes. Non-fast-forward result branch pushes are auto-merged with artifact-aware Git merge drivers and pushed as a fast-forward, so the canonical results branch is never force-pushed or rewritten. Genuine overlay conflicts route to a timestamped temp branch plus a GitHub compare link for a human merge instead.
298+
`results.repo.remote` is the Git remote URL AgentV fetches and pushes. `results.repo.path: .` stores completed run artifacts on a dedicated branch of the source repository without checking out that branch in the source worktree. AgentV manages the local Git remote alias for that URL, so the normal config stays portable across machines. When `results.repo.remote` is omitted, `results.repo.path` means an existing local Git checkout whose object database and refs AgentV should write to, and the branch defaults to `agentv/results/v1`. AgentV creates the branch automatically on first publish and commits only AgentV result paths into it. `sync.auto_push: false` keeps the result commit local; set it to `true` to push the branch best-effort after each completed run. `sync.require_push: true` is for CI workflows where a push failure should fail the command after local artifacts are written. `sync.push_conflict_policy` defaults to `block`; the removed `backup_and_force_push` value is rejected with migration guidance because AgentV never force-pushes result branches. Non-fast-forward result branch pushes are auto-merged with artifact-aware Git merge drivers and pushed as a fast-forward, so the canonical results branch is never force-pushed or rewritten. Genuine overlay conflicts route to a timestamped temp branch plus a GitHub compare link for a human merge instead.
299299

300300
For a separate results repository, use `results.repo.remote` and an optional managed clone `results.repo.path`:
301301

@@ -425,7 +425,7 @@ After sync, newly fetched remote runs appear in the list with a **remote** sourc
425425
- Safe uncommitted changes under the configured results repo's owned result and metadata paths, such as remote tag overlays under `metadata/runs/**`, are committed and pushed when `sync.auto_push: true`.
426426
- A local results repo that is ahead is pushed when `sync.auto_push: true` and the committed paths are all under `.agentv/results/**`.
427427
- Dirty non-results files, dirty metadata plus remote changes, unresolved conflicts, missing upstream branches, non-results commits ahead, and rejected pushes are blocked instead of reset.
428-
- Non-fast-forward result branch pushes never force-push. AgentV runs a bounded fetch → merge → push loop that absorbs concurrent remote writes with a real merge commit using artifact-aware Git merge drivers (union for the append-only `index.jsonl`, a JSON-union driver for tag and feedback overlays), so the common append-mostly case auto-merges and pushes as a fast-forward. The `sync.push_conflict_policy: backup_and_force_push` value is deprecated and no longer force-pushes; it now auto-merges like the default and emits a one-time deprecation notice.
428+
- Non-fast-forward result branch pushes never force-push. AgentV runs a bounded fetch → merge → push loop that absorbs concurrent remote writes with a real merge commit using artifact-aware Git merge drivers (union for the append-only `index.jsonl`, a JSON-union driver for tag and feedback overlays), so the common append-mostly case auto-merges and pushes as a fast-forward. When Dashboard sync absorbs concurrent remote changes this way, the success feedback includes **Merged remote (auto)**. The removed `sync.push_conflict_policy: backup_and_force_push` value is rejected with migration guidance; remove the field or set it to `block`.
429429
- When a genuine overlay conflict cannot be auto-merged, AgentV does not touch the canonical branch. It pushes the local work to a fresh timestamped `agentv/results-sync/<timestamp>-<branch-slug>-<random>` branch and reports `needs_human_merge` with a `pending_merge` block (temp branch, target branch, and a GitHub compare URL when the remote is on GitHub). The toolbar shows a **Pending merge** card: open the link to merge the branch into the canonical target on GitHub (GitHub's pull request is the conflict surface — AgentV builds no merge UI), then click **I merged it — resync**. That resumes canonical sync by fast-forward-pulling the merged target. A premature click is a safe no-op — local work stays intact and the next sync re-creates a temp branch.
430430

431431
When sync is blocked, Dashboard keeps the local clone intact and shows the `block_reason`, `dirty_paths` or `conflicted_paths`, `git_status`, and a compact `git_diff_summary` so you can resolve the results repo manually before syncing again.

‎apps/web/src/content/docs/docs/tools/results.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -239,7 +239,7 @@ The CLI contract is deliberately narrow: `agentv results` manages local result a
239239

240240
Use these supported remote workflows instead:
241241

242-
- **Automatic publishing:** configure `projects[].results` or top-level `results`; new `agentv eval` and `agentv pipeline bench` runs publish completed artifacts after the run completes. Use `repo.remote` with `repo.path: .` and `repo.branch: agentv/results/v1` to store primary result records on a dedicated branch of the source repo without requiring a machine-local Git remote name. AgentV reserves `agentv/results/v1` for primary results and `agentv/artifacts/v1` for heavy artifact payloads. When `index.jsonl` rows point trace or transcript payloads at `agentv/artifacts/v1`, automatic publishing stores those bytes on that artifact branch in the same remote and publishes pointer keys such as `runs/<run-path>/<pointer.path>`. The configured results branch remains the metadata/control plane (`index.jsonl`, `benchmark.json`, tags, and pointers) instead of duplicating canonical trace/transcript payload bodies. Local pre-publish run workspaces can still contain those files beside the manifest so local tools keep working. Mutable run tags are stored as `tags.json` with a `tag_revision`; there is no tag event log in the normal results layout. `results.repo.path` without `results.repo.remote` means an existing local Git checkout, distinct from `workspace.repos[].repo`, which is a portable repository identity. AgentV manages any local Git remote alias internally. Set `sync.auto_push: true` to push after publish, or `sync.require_push: true` in CI to fail when that push fails. Non-fast-forward result branch pushes never force-push: AgentV auto-merges concurrent remote writes with artifact-aware Git merge drivers (a union driver for the append-only `index.jsonl`, a JSON-union driver for tag and feedback overlays) and pushes the merge as a fast-forward, and routes a genuine overlay conflict to a timestamped `agentv/results-sync/...` branch plus a GitHub compare/PR link for a human merge. `sync.push_conflict_policy: backup_and_force_push` is deprecated and no longer force-pushes — it now auto-merges like the default `block` and emits a one-time deprecation notice. While an eval is still running, [WIP checkpoints](/docs/tools/wip-checkpoints/) can keep partial run output durable on `agentv/wip/...` branches when auto-push is enabled.
242+
- **Automatic publishing:** configure `projects[].results` or top-level `results`; new `agentv eval` and `agentv pipeline bench` runs publish completed artifacts after the run completes. Use `repo.remote` with `repo.path: .` and `repo.branch: agentv/results/v1` to store primary result records on a dedicated branch of the source repo without requiring a machine-local Git remote name. AgentV reserves `agentv/results/v1` for primary results and `agentv/artifacts/v1` for heavy artifact payloads. When `index.jsonl` rows point trace or transcript payloads at `agentv/artifacts/v1`, automatic publishing stores those bytes on that artifact branch in the same remote and publishes pointer keys such as `runs/<run-path>/<pointer.path>`. The configured results branch remains the metadata/control plane (`index.jsonl`, `benchmark.json`, tags, and pointers) instead of duplicating canonical trace/transcript payload bodies. Local pre-publish run workspaces can still contain those files beside the manifest so local tools keep working. Mutable run tags are stored as `tags.json` with a `tag_revision`; there is no tag event log in the normal results layout. `results.repo.path` without `results.repo.remote` means an existing local Git checkout, distinct from `workspace.repos[].repo`, which is a portable repository identity. AgentV manages any local Git remote alias internally. Set `sync.auto_push: true` to push after publish, or `sync.require_push: true` in CI to fail when that push fails. Non-fast-forward result branch pushes never force-push: AgentV auto-merges concurrent remote writes with artifact-aware Git merge drivers (a union driver for the append-only `index.jsonl`, a JSON-union driver for tag and feedback overlays) and pushes the merge as a fast-forward, and routes a genuine overlay conflict to a timestamped `agentv/results-sync/...` branch plus a GitHub compare/PR link for a human merge. The removed `sync.push_conflict_policy: backup_and_force_push` value is rejected with migration guidance; remove the field or set it to `block`. While an eval is still running, [WIP checkpoints](/docs/tools/wip-checkpoints/) can keep partial run output durable on `agentv/wip/...` branches when auto-push is enabled.
243243
- **Manual Dashboard sync:** run `agentv dashboard`, open the project, and use **Sync Project**.
244244
- **Manual API sync:** while Dashboard is running, call `GET /api/projects/:projectId/remote/status` or `POST /api/projects/:projectId/remote/sync` for project-scoped automation. Single-project sessions also expose `GET /api/remote/status` and `POST /api/remote/sync`.
245245
- **Git escape hatch:** for advanced recovery, inspect or repair the configured `projects[].results.repo.path` clone with `git` directly, then sync again.

‎docs/adr/2026-06-24-no-force-push-results-sync.md‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -69,9 +69,11 @@ all of it and is safe: a premature OK just pulls a target lacking the local work
6969
re-diverges on the next push, and re-creates a temp branch — no data loss, no
7070
force push.
7171

72-
`backup_and_force_push` is **deprecated, not removed**: the config value still
73-
validates but now auto-merges like the default and emits a one-time deprecation
74-
notice, so shipped surfaces referencing it keep working.
72+
`backup_and_force_push` is **hard-deprecated/removed** from supported config:
73+
the value shipped only on the `next` npm tag before stable release, so AgentV
74+
now rejects it with migration guidance instead of preserving a compatibility
75+
alias. Remove the field or set `sync.push_conflict_policy: block`; AgentV never
76+
force-pushes result branches.
7577

7678
## Consequences
7779

@@ -112,8 +114,9 @@ notice, so shipped surfaces referencing it keep working.
112114
Delivered in phases under epic av-raf (all non-breaking):
113115

114116
- Phase 0 — `.gitattributes` + `agentv-json` merge driver registration (#1506).
115-
- Phase 1 — bounded `fetch → merge → push` loop replacing the force-push path;
116-
`backup_and_force_push` deprecated (#1506).
117+
- Phase 1 — bounded `fetch → merge → push` loop replacing the force-push path
118+
(#1506); `backup_and_force_push` hard-deprecated before stable release
119+
(#1510).
117120
- Phase 2 — temp-branch fallback + `confirm-merge` (OK-to-resync) API (#1507).
118121
- Phase 3 — Dashboard **Pending merge** card with the GitHub link + resync button
119122
(#1508).
Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
---
2+
title: Hard-correct next-tag-only surfaces before stable release
3+
date: 2026-06-25
4+
category: conventions
5+
module: Release compatibility
6+
problem_type: convention
7+
component: development_workflow
8+
severity: medium
9+
applies_when:
10+
- Removing or renaming a config value, wire field, CLI flag, or public API surface
11+
- Deciding whether a shipped-looking surface needs backward compatibility
12+
tags: [release-channel, compatibility, deprecation, config-schema]
13+
---
14+
15+
# Hard-correct next-tag-only surfaces before stable release
16+
17+
## Context
18+
19+
AgentV briefly exposed `results.sync.push_conflict_policy: backup_and_force_push`
20+
on the npm `next` tag while replacing force-push results sync with a no-force
21+
merge loop. Treating that as a stable shipped surface would have kept a
22+
misleading compatibility alias around even though the value contradicted the new
23+
product invariant: AgentV never force-pushes result branches.
24+
25+
## Guidance
26+
27+
When checking whether a config value or public surface has shipped, distinguish
28+
release channels:
29+
30+
- Stable npm releases require normal compatibility handling: preserve behavior,
31+
soft-deprecate, or provide an explicit migration path.
32+
- `next`-only releases can be hard-corrected before the surface reaches stable,
33+
especially when preserving the surface would encode a dangerous or misleading
34+
contract.
35+
36+
For removed config values, make the correction explicit:
37+
38+
```yaml
39+
results:
40+
sync:
41+
# Remove unsupported aliases and use the stable default.
42+
push_conflict_policy: block
43+
```
44+
45+
If existing local registries or generated config may contain the removed value,
46+
either reject it with migration guidance or drop it during a registry migration
47+
that rewrites the supported shape on the next save.
48+
49+
## Why This Matters
50+
51+
Pre-release tags are useful for discovering wrong API names and unsafe contracts.
52+
If every `next` exposure becomes permanent compatibility debt, the project loses
53+
the ability to correct those mistakes before stable release. The compatibility
54+
bar should protect stable users without forcing unsafe pre-release names into
55+
the long-term schema.
56+
57+
## When to Apply
58+
59+
- A value, flag, or field appeared only on npm `next` or another prerelease
60+
channel.
61+
- The replacement behavior is already stable and safer.
62+
- Keeping the old surface would confuse users about current behavior or
63+
preserve a hazardous name.
64+
65+
## Examples
66+
67+
`backup_and_force_push` should not remain a supported
68+
`results.sync.push_conflict_policy` value after the force-push implementation is
69+
removed. Even though it appeared on a published `next` tarball, the stable
70+
migration is to remove the field or set it to `block`; AgentV's actual behavior
71+
is a no-force-push merge loop.
72+
73+
## Related
74+
75+
- docs/adr/2026-06-24-no-force-push-results-sync.md

‎packages/core/src/evaluation/loaders/config-loader.ts‎

Lines changed: 8 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ export type ExecutionDefaults = {
3535
readonly pool_slots?: number;
3636
};
3737

38-
export type ResultPushConflictPolicy = 'block' | 'backup_and_force_push';
38+
export type ResultPushConflictPolicy = 'block';
3939

4040
export type ResultsConfig = {
4141
readonly mode?: 'github';
@@ -782,21 +782,20 @@ export function parseResultsConfig(raw: unknown, configPath: string): ResultsCon
782782
logWarning(`Invalid results.sync.require_push in ${configPath}, expected boolean`);
783783
return undefined;
784784
}
785-
if (
786-
syncObj.push_conflict_policy !== undefined &&
787-
syncObj.push_conflict_policy !== 'block' &&
788-
syncObj.push_conflict_policy !== 'backup_and_force_push'
789-
) {
785+
if (syncObj.push_conflict_policy === 'backup_and_force_push') {
790786
logWarning(
791-
`Invalid results.sync.push_conflict_policy in ${configPath}, expected 'block' or 'backup_and_force_push'`,
787+
`results.sync.push_conflict_policy: 'backup_and_force_push' in ${configPath} is no longer supported. Remove the field or set it to 'block'; AgentV never force-pushes result branches.`,
792788
);
793789
return undefined;
794790
}
791+
if (syncObj.push_conflict_policy !== undefined && syncObj.push_conflict_policy !== 'block') {
792+
logWarning(`Invalid results.sync.push_conflict_policy in ${configPath}, expected 'block'`);
793+
return undefined;
794+
}
795795
sync = {
796796
...(typeof syncObj.auto_push === 'boolean' && { auto_push: syncObj.auto_push }),
797797
...(typeof syncObj.require_push === 'boolean' && { require_push: syncObj.require_push }),
798-
...((syncObj.push_conflict_policy === 'block' ||
799-
syncObj.push_conflict_policy === 'backup_and_force_push') && {
798+
...(syncObj.push_conflict_policy === 'block' && {
800799
push_conflict_policy: syncObj.push_conflict_policy,
801800
}),
802801
};

0 commit comments

Comments
 (0)