feat(ts-pivot): TypeScript test harness + Eval functions + behavioral fixtures#4
feat(ts-pivot): TypeScript test harness + Eval functions + behavioral fixtures#4JacobPEvans-personal wants to merge 10 commits into
Conversation
Pilot pivot mirroring dryvist/cc-edge-pack-template#3 (TS port). All Cribl pack content (default/, data/) carries over from VisiCore unchanged; the test/build tooling switches to TypeScript + Vitest + Biome + nix-devenv. Pack changes (the user-visible behavior): - default/pipelines/claude-code-otel/conf.yml — added Eval function setting sourcetype='claude:code:otel', index='claude-code', datatype='claude-code-otel' - default/pipelines/claude-code-session-logs/conf.yml — added Eval function setting sourcetype='claude:code:session_logs', index='claude-code', datatype='claude-code-session-logs' Test harness (synced from cc-edge-pack-template@feature/ts-pivot): - tests/cribl-client.ts — TS client (CriblClient, runRouteFlow, assertRequiredFields, capture primitives) - tests/parse-filter.ts, global-setup.ts, test-helpers.ts, routes.test.ts, pipelines.test.ts, vitest.config.ts, tsconfig.json - tests/package.json + pnpm-lock.yaml (vitest 2.1, typescript 5.6, biome 2, tar 7, yaml 2, tsx 4) - tests/generate-fixtures.ts — committed helper for regenerating expected fixtures via live Cribl (run via `pnpm exec tsx generate-fixtures.ts ...`) Fixtures (the actual proof the pack works): - tests/fixtures/claude-code-otel/sample.json — sourced from data/samples/sanitized_event_api_request.json (5 OTLP events) - tests/fixtures/claude-code-otel/sample.expected.json — generated by running through live Cribl, trimmed to {sourcetype, index, datatype, _raw, _time} - tests/fixtures/claude-code-session-logs/sample.json — sourced from data/samples/sanitized_assistant.log (1 assistant turn) - tests/fixtures/claude-code-session-logs/sample.expected.json — same flow Tooling (mirrors template): - biome.jsonc, flake.nix, .envrc, .gitignore, Makefile (npm/pnpm scripts) - .release-please-manifest.json + release-please-config.json (initial 0.0.0) Workflows: - .github/workflows/test.yml — caller for cc-edge-pack-template's reusable workflow, pinned to @feature/ts-pivot until that PR merges - .github/workflows/release.yml — caller for the template release workflow (pinned to @main since release isn't exercised in this PR) - .github/workflows/release-please.yml — caller for JacobPEvans inherited workflow Docs: - CLAUDE.md — rewritten for pilot scope (was a stale copy of the template's CLAUDE.md from Python era) - README.md — added Installation + Usage anchors at top to satisfy readme validator; documented Eval pipeline tagging (sourcetype/index/datatype); removed the misleading 1.0.0 release notes line (no release exists yet, and per user direction the agent does NOT tag versions — release-please proposes them via PR for human approval) Verification (locally, with `nix develop github:JacobPEvans/nix-devenv?dir=shells/typescript`): - pnpm run typecheck: tsc --noEmit returns 0 - biome check: 14 files, 0 errors - pnpm run test (vitest run): 11/11 passed - 7 structural route tests - 2 dynamic route flow tests (one per route — both produce non-empty output) - 2 fixture-driven pipeline tests (partial-match + required-fields assertion) Sequencing note: the test caller workflow is temporarily pinned to `cc-edge-pack-template@feature/ts-pivot` so this PR's CI can run before the template PR merges. Flip to `@main` immediately after dryvist/cc-edge-pack-template#3 lands. (claude)
…amed input dryvist exposes only the generic GH_APP_ID + GH_APP_PRIVATE_KEY org secrets; the wrap pattern aliases them to whatever the inherited workflow expects at the boundary. Anyone reading the dryvist repos sees only the generic name. (claude)
There was a problem hiding this comment.
Pull request overview
Ports the pack’s test/build harness from Python/pytest to TypeScript/Vitest (mirroring cc-edge-pack-template TS pivot) and updates the pack behavior so both pipelines set Splunk-canonical routing fields (sourcetype, index, datatype). This aligns runtime behavior with fixture-driven tests and adds reproducible fixtures generated from a live Cribl instance.
Changes:
- Replace Python-based tests with a TypeScript/Vitest harness (client, global setup, route/pipeline suites, fixture generator).
- Add/adjust Edge pack configuration: inputs, routes, pipelines (Eval tagging), pack logo, and sample catalog/data.
- Add repo tooling + automation (Biome, nix flake/direnv, release-please config, updated workflows/Makefile/docs).
Reviewed changes
Copilot reviewed 57 out of 59 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/vitest.config.ts | Adds Vitest config with globalSetup + sequential execution. |
| tests/tsconfig.json | Adds strict TS config for the test harness. |
| tests/test_routes.py | Removes legacy Python route structure tests. |
| tests/test_pipelines.py | Removes legacy Python fixture-driven pipeline tests. |
| tests/test-helpers.ts | Adds helpers to construct a per-file Cribl client + read installed pack id. |
| tests/routes.test.ts | Adds TS route structure + dynamic route-flow tests. |
| tests/requirements.txt | Removes Python test dependencies list. |
| tests/pipelines.test.ts | Adds TS fixture discovery + pipeline execution assertions. |
| tests/parse-filter.ts | Adds small parser for simple field=='value' route filters. |
| tests/package.json | Adds isolated pnpm workspace for tests (vitest/tsx/biome/typescript). |
| tests/global-setup.ts | Adds Vitest global setup/teardown to install pack into Cribl once per run. |
| tests/generate-fixtures.ts | Adds committed script to regenerate expected fixtures via live Cribl. |
| tests/fixtures/claude-code-session-logs/sample.json | Adds session-logs pipeline input fixture. |
| tests/fixtures/claude-code-session-logs/sample.expected.json | Adds expected output fixture asserting canonical fields. |
| tests/fixtures/claude-code-otel/sample.json | Adds OTEL pipeline input fixture. |
| tests/fixtures/claude-code-otel/sample.expected.json | Adds expected output fixture asserting canonical fields. |
| tests/cribl_client.py | Removes legacy Python Cribl management API client. |
| tests/cribl-client.ts | Adds TS Cribl management API client + pack tarball builder + assertions. |
| tests/conftest.py | Removes legacy pytest fixtures for Cribl lifecycle. |
| tests/README.md | Removes legacy Python test harness documentation. |
| release-please-config.json | Adds release-please configuration for automated versioning/changelog. |
| package.json | Sets actual pack id/metadata/tags for cc-edge-claude-code-io. |
| flake.nix | Adds Nix flake devshell delegating to nix-devenv TypeScript shell. |
| default/samples.yml | Populates Cribl UI sample catalog entries for included sample data. |
| default/pipelines/route.yml | Replaces placeholder routes with real routes keyed by datatype. |
| default/pipelines/claude-code-session-logs/conf.yml | Adds pipeline that tags session-log events with canonical fields. |
| default/pipelines/claude-code-otel/conf.yml | Adds pipeline that tags OTEL events with canonical fields. |
| default/pack.yml | Sets pack logo (base64 image). |
| default/inputs.yml | Defines file monitor + OTLP inputs with datatype metadata. |
| data/samples/viOToM.json | Adds sanitized OTEL sample payload (tool_result). |
| data/samples/smpLil.json | Adds sanitized OTEL sample payload (api_error). |
| data/samples/nuJv6j.json | Adds sanitized OTEL sample payload (user_prompt). |
| data/samples/mSfzuP.json | Adds sanitized OTEL sample payload (tool_decision). |
| data/samples/lCrlnQ.json | Adds sanitized OTEL sample payload (lines_of_code.count). |
| data/samples/kkcx4D.json | Adds sanitized session-log sample payload (queue-operation). |
| data/samples/k0h8NP.json | Adds sanitized session-log sample payload (file-history-snapshot). |
| data/samples/hO8QQb.json | Adds sanitized session-log sample payload (assistant turn). |
| data/samples/go9A8j.json | Adds sanitized OTEL sample payload (api_request). |
| data/samples/bw3bbR.json | Adds sanitized OTEL sample payload (commit.count). |
| data/samples/bCOM2K.json | Adds sanitized session-log sample payload (user tool_result). |
| data/samples/aukWzL.json | Adds sanitized OTEL sample payload (active_time.total). |
| data/samples/QtXzUr.json | Adds sanitized OTEL sample payload (token.usage). |
| data/samples/QS0mDG.json | Adds sanitized OTEL sample payload (pull_request.count). |
| data/samples/OsXDpa.json | Adds sanitized session-log sample payload (system stop_hook_summary). |
| data/samples/Io4OFW.json | Adds sanitized OTEL sample payload (session.count). |
| data/samples/DBvKLd.json | Adds sanitized OTEL sample payload (cost.usage). |
| data/samples/CBxGTZ.json | Adds sanitized OTEL sample payload (code_edit_tool.decision). |
| data/samples/36cP1M.json | Adds sanitized session-log sample payload (progress/agent_progress). |
| biome.jsonc | Adds repo-wide Biome config (excluding pack contents + fixtures). |
| README.md | Rewrites docs for this specific pack (install/usage/setup/reference). |
| Makefile | Updates make targets for pnpm/Vitest/Biome + removes Python venv flow. |
| CLAUDE.md | Updates AI assistant guidance to this pack’s scope + TS harness workflow. |
| .release-please-manifest.json | Adds initial release-please manifest version. |
| .gitignore | Switches ignores from Python venv to Node/TS + direnv/nix artifacts. |
| .github/workflows/test.yml | Switches CI test job to reusable workflow in the TS template repo. |
| .github/workflows/release.yml | Switches CI release job to reusable workflow in the TS template repo. |
| .github/workflows/release-please.yml | Adds workflow calling org-wide release-please reusable workflow. |
| .envrc | Enables direnv flake integration. |
Files not reviewed (1)
- tests/pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Pilot was scaffolded before the template added scripts/, .editorconfig, lefthook.yml, the slim biome.jsonc, and the latest cribl-client.ts + routes.test.ts (existsSync guard, response.ok check). Brings pilot in line with template@main so CI can resolve scripts/validate-pack-structure.sh and the rest of the chain. (claude)
- README.md: fix double-quote typo in metadata example - CLAUDE.md: routes filter on top-level datatype (set by input metadata), not _metadata.datatype - generate-fixtures.ts: doc comment now matches actual usage (pnpm exec tsx, not node --experimental-strip-types) - release-please-config.json: package-name -> cc-edge-claude-code-io (was leftover template placeholder) - .release-please-manifest.json: align with package.json @ 1.0.0 (claude)
|
Closing in favor of #5, which is a minimal-scope rebuild from the now-proven template baseline (cc-edge-pack-template#8). This PR ballooned to 64 files with substantial pack content I authored alongside the harness — circular validation. The new PR keeps only the load-bearing test-infra + tooling + workflow + pack-identity changes; real claude-code-otel and claude-code-session-logs pipelines land in follow-up PRs. |
Summary
Pilot TS pivot mirroring dryvist/cc-edge-pack-template#3 (TS port). All Cribl pack content (`default/`, `data/`) carries over from VisiCore unchanged; the test/build tooling switches to TypeScript + Vitest + Biome + nix-devenv.
What changed
Pack behavior (the user-visible part)
(Index value follows the vendor/software convention you specified.)
Test harness (synced from cc-edge-pack-template@feature/ts-pivot)
Fixtures (the actual proof)
Tooling
Workflows
Docs
Verification
Locally via `nix develop github:JacobPEvans/nix-devenv?dir=shells/typescript`:
Sequencing
After CI green:
No version tag in this PR. Release timing is yours to decide.