Repository navigation
[feat] Record what a session, step or run delivered #1387
Copy link
Copy link
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
Example
A learning names the session it came from:
$ teamai contribute --file notes.md --title "Pod startup timeouts" --session-id 7c1e…---
title: Pod startup timeouts
author: alice
date: 2026-10-24
tags: [k8s, startup, troubleshooting]
session: { platform: claude-code, sessionId: 7c1e… }
---A learning's id is its path under learnings/ without .md:
learnings/pod-startup-timeouts-2026-10-24-k3x9q2.md → pod-startup-timeouts-2026-10-24-k3x9q2
learnings/k8s/pod-startup-timeouts-2026-10-24-k3x9q2.md → k8s/pod-startup-timeouts-2026-10-24-k3x9q2
A session's outcomes stay on this machine, as dashboard events:
{"type":"session_outcome","sessionId":"7c1e…","tool":"claude","runMode":"attended",
"outcomes":[{"kind":"commit","sha":"9f2c…"},{"kind":"pr","url":"https://github.com/acme/app/pull/42"},
{"kind":"learning","id":"pod-startup-timeouts-2026-10-24-k3x9q2"}]}A step's outcomes come from what the run already records, plus what the step's own commands report. Inside a workflow attempt, contribute appends to the attempt's outcome file:
$ echo "$TEAMAI_WORKFLOW_ATTEMPT"
r3f9…/a2
$ teamai contribute --file "$FILE" --title "Retry backoff"
$ cat <dataHome>/workflows/runs/r3f9…/outcomes/a2.jsonl
{"kind":"learning","id":"retry-backoff-2026-10-24-p0x1za"}The run keeps prs; its union is derived:
// workflows/runs/r3f9….json on teamai-reports, end record
{ "run": "r3f9…", "runMode": "unattended", "prs": ["https://github.com/acme/app/pull/43"],
"steps": [ { "step": "fix", "pass": 1, "outcomes": [ { "kind": "pr", "url": "https://github.com/acme/app/pull/43" } ] },
{ "step": "close", "pass": 1, "outcomes": [ { "kind": "learning", "id": "retry-backoff-2026-10-24-p0x1za" } ] } ] }The usage report and digest carry counts only, per run mode and outcome kind:
runMode outcome sessions tokens/session interventions/session
attended pr 12 410k 6.1
attended learning 5 220k 3.0
attended none 31 95k 2.2
unattended pr 20 380k 0.4
not recorded none 7 120k 1.9
# stats/alice.yaml on teamai-reports, beside today's counters
outcomes:
attended:
pr: { sessions: 12, tokens: { input: …, output: … }, interventions: 73 }
learning: { sessions: 5, tokens: { input: …, output: … }, interventions: 15 }
none: { sessions: 31, tokens: { input: …, output: … }, interventions: 68 }
unattended:
pr: { sessions: 20, tokens: { input: …, output: … }, interventions: 8 }
notRecorded:
none: { sessions: 7, tokens: { input: …, output: … }, interventions: 13 }A session can be in several rows (a PR and a learning), so the rows overlap and their totals are not additive.
Behavior
| Kind | Session | Step pass | Run |
|---|---|---|---|
commit |
commits authored by the member's git identity on the session's checkout, in gitAtStart.head..<gitHead of its last Stop event>, once the session is finished (#1385) |
commits on the step's branch | derived: union of steps |
pr |
a PR open for the checkout's branch that contains one of the session's commits. Looked up once, when the usage report folds a finished session, and cached. A session with no commit gets none | the run's delivery receipt for that step and pass (deliver: pr) |
prs, as today |
learning |
contribute for the session --session-id names (else the tool's session variable): the learning's id. Also inside a workflow attempt: the session it names is not the attempt's |
contribute run inside the attempt (TEAMAI_WORKFLOW_ATTEMPT set): appended to the attempt's outcome file, put on the pass when the attempt settles |
derived: union of steps |
| Data | Where it lives | Leaves the machine |
|---|---|---|
| Session outcomes with ids, shas and URLs | local dashboard events | no |
| A step's reported outcomes | <dataHome>/workflows/runs/<run>/outcomes/<attempt>.jsonl, then the pass |
the pass's outcomes, with the run record |
| PR lookups | <dataHome>/dashboard/outcome-prs.json, by session |
no |
Counts per (runMode, outcome kind): sessions, tokens, interventions |
stats/<member>.yaml, under outcomes |
yes, as today's counts do |
| Step outcomes | the run's end record | yes, as run records do today |
A learning's session |
its frontmatter on teamai-learnings |
yes, with the learning |
A session, step or run can have several outcomes, or none. none is the empty list, shown as its own row.
Acceptance criteria
teamai contributewritessession: { platform, sessionId }into the learning's frontmatter before the learning is queued, for the session it already resolves (--session-id, else the tool's session variable). The platform is resolved asscore --sessionresolves it: the session variable's tool family, else the session's captures.- A learning's id is its path under
learnings/without.md, so a namespaced learning's id carries its namespace (k8s/…). A root learning's id is its file name without.md. - Session outcomes are dashboard events, folded with the session's other events. They are never written into captures. A session with no capture (a tool teamai has no session adapter for) still gets its outcomes, so the per-mode report counts it.
- The Stop dashboard event records
gitHead, so a session's commit range is known without reading git at fold time. - With
TEAMAI_WORKFLOW_ATTEMPTset (feat(workflow): add session discovery and managed public leases #1304),contributeappends{"kind":"learning","id":…}to<dataHome>/workflows/runs/<run>/outcomes/<attempt>.jsonl. Both drivers read that file when the attempt settles and put its entries on the pass'soutcomes. The step's result file andworkflow reportare unchanged. Covered by a test per driver. - Step outcomes also come from
run.deliveries(bystepandpass). A run record gains no second PR field next toprs. - Outcome rows count the same sessions [feat] Run mode: tell attended runs from unattended ones #1384 counts as the member's: sessions with
launchedByare left out, and step outcomes live on run records only.launchedByis read for the session an outcome belongs to, not for the process that writes it. - The usage report has counts only, under
outcomes: { <runMode | notRecorded>: { <kind | none>: { sessions, tokens: { input, output }, interventions } } }. An attended session is added once it is finished (6 hours idle). An unattended session is added whenteamai score --drainreports usage at the end of its job ([feat] Publish and score sessions by run mode #1385), since the drain is where unattended sessions finish, so an ephemeral runner's sessions are counted. A per-session reported flag in a new baseline file,reported-outcomes.jsonbeside today's baselines, keeps it from being added twice. No session id, commit sha or PR URL is published. - Outcome rows use [feat] Run mode: tell attended runs from unattended ones #1384's
runMode; records without it arenot recorded. This issue adds the outcome dimension and does not re-split the existing metrics. - The PR lookup uses
providerForRemotefor the checkout's origin, with the credentialsdeliver: pruses, and is skipped without them. A lookup that fails leavesprout; it never fails the hook or the report.
Definition of done
docs/guide/reporting.md,docs/guide/dashboard.mdand their zh-CN pages, with the examples above.docs/guide/knowledge.mdand its zh-CN page: one line on thesessionfield.skill-data/share/references/doc-template.md: thesessionfield in the frontmatter table (the share workflow writes learnings).contributehelp text:--session-idnow also records the session in the learning, and inside a workflow attempt the learning is recorded on the step.skill-data/core/references/commands.mdregenerated withnpx vitest run commands-reference -u.- Real-CLI record: one session with a commit and a learning, and the usage report and
digestthat show them; one workflow step that runscontribute, and its pass'soutcomes.
Out of scope
- Issues and specs as outcome kinds (later).
- Cost in currency; tokens only, as today.
- A pointer from a learning to the archived copy of its session. Readers find it from
sessionwithfindArchived.
Notes
contribute:src/contribute.ts:141-275. It queues the content at:228and uses the session id only formarkContributedat:268-270. File names:generateFilename(:120-130); the namespaced path is built at:210. Frontmatter helpers:parseFrontmatter/stringifyFrontmatter(src/utils/frontmatter.ts:73,:93), assrc/maintenance/confidence.ts:96-139already uses them.- Platform resolution for a session id: reuse
sessionOf's pattern (src/session-flow/scoring.ts:57-76).agentSessionFromEnv(src/utils/session-id.ts:81) gives the id and the tool family, andAgentSession.platformForToolmaps the family to a session platform. - Session subject shape:
SessionSubjectSchema(src/scorers/label.ts:13). - Start head:
addGitStateAtStart(src/dashboard-collector.ts:2500-2504) putsgitAtStartonsession_start. Folding:aggregateSessionMetrics(src/dashboard-collector.ts:2082). - Captures describe a session's start and are swept a day after the session file is gone (
src/session-flow/captures.ts:204-217), so they are not the place for outcomes. They exist only for tools with a session adapter (src/session-flow/adapters/platform-facts.ts); Pi, Muse, Copilot and others are not captured today. - Run records:
buildEndRecord(src/workflow/records.ts) takesprsfromrun.deliveries. Receipts carrystepandpass(DeliveryReceiptSchema,src/workflow/delivery-schema.ts). - Why an outcome file and not the step's result: the local driver's prompt tool writes its whole result file at the end (
TEAMAI_WORKFLOW_RESULT, set increateLocalToolAdapter,src/workflow/drivers/local/local-tool-adapter.ts), which would overwrite a linecontributeadded, and command steps and skill-driven steps do not use that file. The attempt marker reaches every process of an attempt in both drivers (attemptEnv,src/workflow/attempt-marker.ts, feat(workflow): add session discovery and managed public leases #1304). - PR provider:
providerForRemote(src/providers/registry.ts:234), asdeliver: prpicks it (originOf,src/workflow/pr-delivery.ts). - Usage aggregation:
src/team-push.ts:933-985, baselinesreportedBaselines(:817).stats/<user>.yamlis counts only by design (src/types.ts:1743-1771). - The implementer decides whether session outcomes are one
session_outcomeevent or fields on the session's last event, and whatcontributedoes when neither the session variable nor a capture names the platform.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request
Activity
SaulMoro commentedon Oct 25, 2026
Rewrote the body. The usage report carries counts only, per run mode and outcome kind, because
stats/<user>.yamlis counts only by design (src/types.ts:1726-1754). Ids, shas and URLs stay local as dashboard events.Those events also cover sessions with no capture, so a tool teamai has no session adapter for still counts.
The session subject is spelled
{ platform, sessionId }, as labels spell it (src/scorers/label.ts:13), and the frontmatterarchivefield is dropped.Step outcomes come from
run.deliveriesnext toprs, and a session'sproutcome needs one of its own commits. The split by mode stays with #1384.SaulMoro commentedon Oct 25, 2026
<dataHome>/workflows/runs/<run>/outcomes/<attempt>.jsonl, whichcontributeappends to whenTEAMAI_WORKFLOW_ATTEMPTis set. The tool's result file would overwrite it, and command steps and skill-driven steps do not use that file.learnings/without.md, so a namespaced learning keeps its namespace.gitHeadfor the commit range, and where PR lookups are cached. Outcome rows count only the member's own sessions.SaulMoro commentedon Oct 25, 2026
Said when an unattended session's outcomes are counted. That happens when
teamai score --drainreports usage at the end of its job (#1385), because the drain is where unattended sessions finish. Before, nothing counted them on a runner that never runspull. Attended sessions are still counted at 6 hours idle.Updated the
UserStatsline numbers, which moved onmain.SaulMoro commentedon Oct 26, 2026
session-close(#1388) runscontribute --session-idinside a workflow attempt. Thelearningrow now says the session outcome goes to the session--session-idnames, even inside an attempt, and thatlaunchedByis read for the session an outcome belongs to, not for the process that writes it. Without this, attended learnings would drop out of the report once #1388 lands.feat(contribute): record the session and step a learning came from
feat(workflow): record what each step pass delivered
feat(dashboard): record the head a Stop saw, and keep session outcome…
feat(report): count what finished sessions delivered, per run mode
feat(digest): show what finished sessions delivered, per run mode
feat(dashboard): show what each step pass delivered in a run's detail
16 remaining items
feat(dashboard): show what a live session delivered so far
feat(dashboard): show what finished sessions delivered on the overview
docs: describe session outcomes in the reporting, dashboard and knowl…
fix(contribute): keep a contribution when its step outcome cannot be …
fix(report): look up a session's pull request on the branch it last s…
refactor(outcomes): derive outcome shapes from their owners
docs: say where a session's commits and pull request show, and list p…
fix(report): count a finished session with the usage the report count…
fix(report): find the commits of a member whose git email has a + tag
fix(report): read a session's commits in its repository once its work…
feat(reporting): record what sessions and workflow steps delivered, a…
SaulMoro commentedon Oct 27, 2026
Status: #1471 is merged (bd3adf9).
contributewritessession: { platform, sessionId }into the learning and records the learning as an outcome of that session. Inside a workflow attempt it also appends to the attempt's outcome file, which settles onto the pass with the step's deliveries. Stop events carrygitHeadandgitBranch. The usage report folds each finished session once (reported-outcomes.json) intooutcomes.<runMode | notRecorded>.<kind | none>, anddigest, the dashboard and the guides show it.Left: counting an unattended session when
teamai score --drainreports usage at the end of its job. #1478 (59b2c07) runs the usage report from the drain, but the outcome fold still finishes a session only after 6 hours idle (isFinishedinsrc/session-outcomes.ts) and does not read the drain's mark. A runner gone before then never reports those outcomes, asdocs/guide/reporting.mdnow says. The fix also changes that sentence and its zh-CN page, and needs a real-CLI drain that shows an unattended session's outcome instats/<member>.yaml.This stays open until that lands.
feat(reporting): count an unattended session's outcomes at the drain …
feat(reporting): count what unattended sessions delivered at the drai…