Skip to content

Add the Scribe starter kit and landing page - #1

Merged
michaelshimeles merged 2 commits into
mainfrom
build/starter-kit
Aug 11, 2026
Merged

Add the Scribe starter kit and landing page#1
michaelshimeles merged 2 commits into
mainfrom
build/starter-kit

Conversation

@michaelshimeles

@michaelshimeles michaelshimeles commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Starter kit (packages/kit): telemetry contract (AgentInteractionRecordV1, attribute namespaces, W3C traceparent helpers), versioned scrub catalog where every pattern ships with tested examples, corpus writer enforcing policy → scrub → validate → put with idempotent per-turn R2 keys and the non-configurable sensitive-capability override, and an offline evaluation adapter around pinned @dynatrace-oss/dt-eval-lib@0.0.15-alpha (Vercel AI Gateway via openai provider + baseUrl, deterministic stub judge for PRs, fail-closed retry policy, hard/soft gates, JSON + JUnit artifacts, R2 run history + pinned baseline).
  • scribe CLI: eval validate, eval run, doctor, verify-recipes, install-recipe, gen-collector; exit codes 2 schema / 3 config / 4 gate / 5 drift; runs on Node 24 native TS with no build step.
  • Collector: collector/otel-collector.yaml generated from the scrub catalog (RE2-safe variants; doctor fails on drift), plus Docker Compose for the standalone-host deployment.
  • Recipes: kit-core, evals, collector manifests with SHA-256 checksums recorded in consumers' .scribe.json.
  • Skills + CI: diagnose-a-turn and compare-eval-runs skills; GitHub Actions running typecheck, tests, build, and the deterministic eval gate with JUnit reporting.
  • Landing page (apps/web): Vite + React + Kumo, monochrome design-system tokens only, simplified pillar copy.

See PLAN.md for the full 90-day plan this implements (Days 1–60 core, canary primitives for Days 61–90).

Test plan

  • pnpm check — typecheck clean across workspace
  • pnpm test — 122 tests / 8 suites, including: every scrub-catalog example removed, sensitive-capability override drops bodies, retry exhaustion fails closed, gate boundaries (0.50 floor, 0.10 decline), CLI exit codes, recipe drift detection, collector-config sync
  • pnpm build — landing page production build
  • CLI exercised end to end: doctor all green, stub eval gate passes (exit 0), gate-failing fixtures exit 4, install-recipe + verify-recipes roundtrip clean
  • CI green on this PR (first workflow run)

Made with Cursor


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith with what you need. Autofix is disabled.

Telemetry contract (AgentInteractionRecordV1, namespaces, traceparent),
versioned scrub catalog with tested examples, corpus writer with the
non-configurable sensitive-capability override, offline eval adapter
around pinned dt-eval-lib (gateway + deterministic stub judges,
fail-closed retries, gates, JUnit), scribe CLI with exit-code contract,
catalog-generated Collector config, checksummed recipes with drift
detection, operational skills, CI with the deterministic eval gate, and
the Kumo landing page.

Co-authored-by: Cursor <cursoragent@cursor.com>
@greptile-apps

greptile-apps Bot commented Aug 11, 2026

Copy link
Copy Markdown

Greptile Summary

Scribe adds a telemetry collection and evaluation kit, deployment guidance, CI automation, and a landing page. The earlier collector-ingress concern was disproved: the Collector accepts valid bearer-authenticated OTLP requests but rejects unauthenticated requests, and its published OTLP ports are loopback-only. The earlier CI hardening concern was also disproved: all actions are commit-SHA pinned and workflow permissions are explicitly limited.

Confidence Score: 5/5

No blocking failure remains; the change is safe to merge.

No accepted blocking findings remain after exercising the previously reported Collector and CI failure paths.

T-Rex T-Rex Logs

What T-Rex did

  • A read-only OTLP ingress configuration check was executed against collector/docker-compose.yml and collector/otel-collector.yaml, and all assertions passed with exit code 0, confirming no unauthenticated exposure.
  • An OTLP ingress configuration check script was authored and its execution output captured, recording the exact command, working directory, assertions, results, and exit code.
  • A CI workflow security validation script was authored and executed against the CI workflow; it pins four SHAs and enforces least-privilege permissions, with the run reporting FIXED_OR_INVALID.
  • An executable harness started the Collector with prior and current configurations and sent matching OTLP/HTTP requests; the prior configuration allowed unauthenticated access, the current rejects unauthenticated requests (401) and accepts bearer-token authenticated requests (200), and the regression suite passed.
  • The executable harness directly started the actual Collector binary against both revisions and demonstrated the updated receiver boundary with no security defect detected.

View all artifacts

T-Rex Ran code and verified through T-Rex

Reviews (2): Last reviewed commit: "Harden collector ingress and CI per secu..." | Re-trigger Greptile

Comment thread collector/docker-compose.yml Outdated
Comment thread .github/workflows/ci.yml Outdated
Collector: bearer-token auth required on both OTLP receivers (generated
into the config), compose publishes ports on 127.0.0.1 only behind the
TLS proxy, posture verified by kit tests. CI: actions pinned to full
commit SHAs with least-privilege GITHUB_TOKEN permissions.

Co-authored-by: Cursor <cursoragent@cursor.com>
@michaelshimeles
michaelshimeles merged commit ab032e9 into main Aug 11, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant