Skip to content

Commit d201568

Browse files
christsoclaude
andcommitted
docs(concepts): define Tags as the promptfoo map; note manual tags removed (av-qsxw)
Adds a "Tags" entry (promptfoo-shaped Record<string,string> map; reserved `experiment` key; Dashboard Tags-tab grouping; the sole tags concept after the manual per-run chips were removed) and notes on the Experiment entry that experiment is expressed as `tags.experiment` with no top-level field. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 142f30a commit d201568

1 file changed

Lines changed: 3 additions & 1 deletion

File tree

‎CONCEPTS.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,9 @@ Shared domain vocabulary for this project — entities, named processes, and sta
2020

2121
**Wrapper eval** — Eval YAML whose main job is to import task suites and bind top-level runtime policy such as target selection, repeat count, timeout, budget, and thresholds. Wrapper evals may live under an `experiments/` directory, but that path is an optional user-owned convention and AgentV does not infer behavior from it. A wrapper that imports suites with `type: suite` does not define parent `workspace`; imported suites own task environment.
2222

23-
**Experiment** — A string metadata/run-grouping label such as `baseline`, `candidate`, `with_skills`, or `without_skills`. It is not a runtime-policy object and not a result path namespace. Runtime policy belongs in top-level eval fields or target objects; the experiment label is recorded in `summary.json` and `index.jsonl` for Dashboard grouping and comparison. Lifecycle setup belongs in `workspace.hooks` or `targets[].hooks`, not in a separate experiment artifact.
23+
**Experiment** — A string metadata/run-grouping label such as `baseline`, `candidate`, `with_skills`, or `without_skills`. It is not a runtime-policy object and not a result path namespace. Experiment is expressed as the reserved `tags.experiment` key (see **Tags**); there is no top-level `experiment` field. Runtime policy belongs in top-level eval fields or target objects; the experiment label is recorded in `summary.json` and `index.jsonl` for Dashboard grouping and comparison. Lifecycle setup belongs in `workspace.hooks` or `targets[].hooks`, not in a separate experiment artifact.
24+
25+
**Tags** — A promptfoo-shaped `Record<string,string>` map authored on an eval (or project config / `--tag key=value`) that labels a run with structured facets such as `experiment`, `team`, or `env`. The reserved `experiment` key feeds the experiment namespace. The resolved map is recorded in `summary.json` `metadata.tags` and every `index.jsonl` row, and the Dashboard "Tags" tab groups and compares runs by any tag key. This is the only "tags" concept: the earlier free-form manual per-run tag chips have been removed. (Suite-level `tags` may still be authored as a string list, which is a selection construct for `select.tags` / `--tag name` filtering rather than run metadata.)
2426

2527
**Workspace** — The task environment an eval prepares for the agent: repositories, templates, fixture files, and lifecycle hooks. It is not prompt input; use `input` for instructions and `workspace.repos[]` for multi-repo workspaces the agent can inspect or modify through tools.
2628

0 commit comments

Comments
 (0)