Add the Scribe starter kit and landing page - #1
Merged
Conversation
Telemetry contract (AgentInteractionRecordV1, namespaces, traceparent), versioned scrub catalog with tested examples, corpus writer with the non-configurable sensitive-capability override, offline eval adapter around pinned dt-eval-lib (gateway + deterministic stub judges, fail-closed retries, gates, JUnit), scribe CLI with exit-code contract, catalog-generated Collector config, checksummed recipes with drift detection, operational skills, CI with the deterministic eval gate, and the Kumo landing page. Co-authored-by: Cursor <cursoragent@cursor.com>
Greptile SummaryScribe adds a telemetry collection and evaluation kit, deployment guidance, CI automation, and a landing page. The earlier collector-ingress concern was disproved: the Collector accepts valid bearer-authenticated OTLP requests but rejects unauthenticated requests, and its published OTLP ports are loopback-only. The earlier CI hardening concern was also disproved: all actions are commit-SHA pinned and workflow permissions are explicitly limited. Confidence Score: 5/5No blocking failure remains; the change is safe to merge. No accepted blocking findings remain after exercising the previously reported Collector and CI failure paths.
What T-Rex did
Reviews (2): Last reviewed commit: "Harden collector ingress and CI per secu..." | Re-trigger Greptile |
Collector: bearer-token auth required on both OTLP receivers (generated into the config), compose publishes ports on 127.0.0.1 only behind the TLS proxy, posture verified by kit tests. CI: actions pinned to full commit SHAs with least-privilege GITHUB_TOKEN permissions. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
packages/kit): telemetry contract (AgentInteractionRecordV1, attribute namespaces, W3C traceparent helpers), versioned scrub catalog where every pattern ships with tested examples, corpus writer enforcing policy → scrub → validate → put with idempotent per-turn R2 keys and the non-configurable sensitive-capability override, and an offline evaluation adapter around pinned@dynatrace-oss/dt-eval-lib@0.0.15-alpha(Vercel AI Gateway viaopenaiprovider +baseUrl, deterministic stub judge for PRs, fail-closed retry policy, hard/soft gates, JSON + JUnit artifacts, R2 run history + pinned baseline).scribeCLI:eval validate,eval run,doctor,verify-recipes,install-recipe,gen-collector; exit codes 2 schema / 3 config / 4 gate / 5 drift; runs on Node 24 native TS with no build step.collector/otel-collector.yamlgenerated from the scrub catalog (RE2-safe variants;doctorfails on drift), plus Docker Compose for the standalone-host deployment.kit-core,evals,collectormanifests with SHA-256 checksums recorded in consumers'.scribe.json.apps/web): Vite + React + Kumo, monochrome design-system tokens only, simplified pillar copy.See
PLAN.mdfor the full 90-day plan this implements (Days 1–60 core, canary primitives for Days 61–90).Test plan
pnpm check— typecheck clean across workspacepnpm test— 122 tests / 8 suites, including: every scrub-catalog example removed, sensitive-capability override drops bodies, retry exhaustion fails closed, gate boundaries (0.50 floor, 0.10 decline), CLI exit codes, recipe drift detection, collector-config syncpnpm build— landing page production builddoctorall green, stub eval gate passes (exit 0), gate-failing fixtures exit 4,install-recipe+verify-recipesroundtrip cleanMade with Cursor
Need help on this PR? Tag
@codesmithwith what you need. Autofix is disabled.