Skip to content

[RefImpl] Commit a sample OTLP trace from the reference Guardian, and test the exporter against it #266

Closed

Description

@rocklambros

Target runtime or language

TypeScript on Bun, reference-implementations/agt/, on top of the OpenTelemetry exporter from PR #244.

What this proves

A committed sample of what the reference Guardian's trace export actually looks like, plus a test that holds the exporter to the mapping it claims to follow.

Today the only way to see ACS trace output is to run the Guardian and drive traffic through it. A committed sample lets an evaluator open the output without running anything, and gives us a fixed thing to test against. It also closes a gap the review of PR #261 found. If the exporter stops emitting an attribute the mapping requires or makes conditional, every existing test still passes. A test over a real sample makes that kind of drift fail.

What to add, after PR #244 merges:

  1. A generator, reference-implementations/agt/scripts/make-sample-trace.ts, run as bun run sample:trace. It starts the Guardian in-process with ACS_OTEL_FILE pointed at a temp file, drives a fixed set of synthetic requests through the two live hook methods, runs the audit-log converter from PR Export Guardian traces as OTLP to a file and over HTTP #244, and writes the result to the sample path. The set covers these cases:
    • an allowed steps/toolCallRequest and its steps/toolCallResult, linked by acs.request_id_ref
    • a denied request with reasoning, reason_codes, and policy_references where one entry lacks a rule_id (exercises the "" padding)
    • a request without capability (acs.capability must be absent) and one with it (present)
    • a JSON-RPC error response (acs.decision.failure, class guardian_error)
    • a decision finished after a short negotiated timeout (class timeout)
    • a Guardian-unreachable step from the host audit log through the converter (class unreachable, reported_by observed_agent)
  2. The sample, reference-implementations/agt/samples/trace/acs-trace-sample.jsonl, in OTLP/JSON lines, exactly as the exporter writes it. It holds synthetic data only: no real paths, URLs, hostnames or credentials, because acs.provenance.source_id and acs.reasoning go out as written.
  3. A test, reference-implementations/agt/test/sample-trace.test.ts, with two halves:
    • The sample matches the mapping. Every span name is a step_to_span value. Every required attribute is present. Every conditional attribute is present exactly when the generator's request carried its source. The policy and provenance arrays follow the padding rule. Every failure class and posture is a value decision_failure_event.attribute_values lists.
    • The sample is current. Running the generator again gives the same spans and attributes once span ids, trace ids and timestamps are normalized. A change to the exporter or the mapping then has to come with a regenerated sample.
  4. Docs.
    • The Guardian README gets a short section pointing at the sample, with the one-line command that regenerates it.
    • The root README's trace paragraph links it.
    • Both keep the existing statement that this covers two hook methods and is not an ACS-Trace conformance claim.

Order. This lands after PR #244, the PR for issue #228, and PR #261. The sample is meant to show the decision-event fields and the conditional attributes, so generating it earlier would bake in output that is about to change.

Do you intend to implement this yourself

Yes, I want this assigned to me

Current Priority Scope

Feeds the runnable Guardian reference implementation

Activity

  1. self-assigned this
    on Oct 12, 2026
  2. moved this from Needs triage to Accepted in ACS Project Trackeron Oct 12, 2026
  3. moved this from Accepted to In progress in ACS Project Trackeron Oct 15, 2026
  4. added a commit that references this issue on Oct 16, 2026
    c883fd7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

priority:P1Maintainers onlyscope:in-focusFeeds the ninety-day committed outcome. Maintainers onlystatus:acceptedIn the backlog. Maintainers onlytype:refimplReference implementation or adapter workworkstream:coding-agentsOwning workstream. Maintainers only

Type

No type

Projects

Relationships

None yet

Development

No branches or pull requests

Issue actions