Skip to content

feat(ts-pivot): TypeScript test harness + Eval functions + behavioral fixtures#4

Closed
JacobPEvans-personal wants to merge 10 commits into
mainfrom
feat/ts-pivot
Closed

feat(ts-pivot): TypeScript test harness + Eval functions + behavioral fixtures#4
JacobPEvans-personal wants to merge 10 commits into
mainfrom
feat/ts-pivot

Conversation

@JacobPEvans-personal
Copy link
Copy Markdown
Member

Summary

Pilot TS pivot mirroring dryvist/cc-edge-pack-template#3 (TS port). All Cribl pack content (`default/`, `data/`) carries over from VisiCore unchanged; the test/build tooling switches to TypeScript + Vitest + Biome + nix-devenv.

What changed

Pack behavior (the user-visible part)

  • Both pipelines (`claude-code-otel`, `claude-code-session-logs`) now run an Eval function that sets the Splunk-canonical fields per dryvist convention:
Pipeline sourcetype index datatype
`claude-code-otel` `claude:code:otel` `claude-code` `claude-code-otel`
`claude-code-session-logs` `claude:code:session_logs` `claude-code` `claude-code-session-logs`

(Index value follows the vendor/software convention you specified.)

Test harness (synced from cc-edge-pack-template@feature/ts-pivot)

  • `tests/cribl-client.ts` — TS Cribl management API client
  • `tests/{parse-filter,global-setup,test-helpers}.ts` — supporting modules
  • `tests/{routes,pipelines}.test.ts` — Vitest suites
  • `tests/{package.json,tsconfig.json,vitest.config.ts,pnpm-lock.yaml}`
  • `tests/generate-fixtures.ts` — committed helper for regenerating expected fixtures (run via `pnpm exec tsx generate-fixtures.ts `)

Fixtures (the actual proof)

  • `tests/fixtures/claude-code-otel/sample.json` — sourced from `data/samples/sanitized_event_api_request.json` (5 OTLP events)
  • `tests/fixtures/claude-code-otel/sample.expected.json` — generated by running through live Cribl, trimmed to `{sourcetype, index, datatype, _raw, _time}`
  • `tests/fixtures/claude-code-session-logs/sample.{json,expected.json}` — same flow with `sanitized_assistant.log`

Tooling

  • `biome.jsonc`, `flake.nix` (delegates to `JacobPEvans/nix-devenv?dir=shells/typescript`), `.envrc`, updated `Makefile` + `.gitignore`
  • `.release-please-manifest.json` + `release-please-config.json` (initial 0.0.0)

Workflows

  • `.github/workflows/test.yml` — caller for `cc-edge-pack-template/.github/workflows/cribl-pack-test.yml`, pinned to `@feature/ts-pivot` until that PR merges
  • `.github/workflows/release.yml` — caller for the template release workflow
  • `.github/workflows/release-please.yml` — caller for inherited `JacobPEvans/.github/.github/workflows/_release-please.yml@main`

Docs

  • `CLAUDE.md` — rewritten for pilot scope (was stale Python-era template copy)
  • `README.md` — added Installation + Usage anchors at top, documented the Eval pipeline tagging, removed the misleading `1.0.0` release-notes line (no release exists yet)

Verification

Locally via `nix develop github:JacobPEvans/nix-devenv?dir=shells/typescript`:

  • `pnpm run typecheck` — `tsc --noEmit` returns 0
  • `biome check` — 14 files, 0 errors
  • `pnpm run test` — 11/11 passed:
    • 7 structural route tests
    • 2 dynamic route flow tests (default + jZR327, both fire correct pipeline + produce non-empty output)
    • 2 fixture-driven pipeline tests (partial-match against expected fixtures + required-fields assertion proves `sourcetype`/`index`/`datatype` set correctly)

Sequencing

After CI green:

  1. Merge dryvist/cc-edge-pack-template#3 first (so the workflow URL resolves cleanly at `@main`)
  2. Update this PR: flip `uses: ...@feature/ts-pivot` -> `...@main` in `.github/workflows/test.yml`
  3. CI re-runs green
  4. Merge this PR
  5. Merge dryvist/.github#1 last (drops the old workflow URLs)

No version tag in this PR. Release timing is yours to decide.

Pilot pivot mirroring dryvist/cc-edge-pack-template#3 (TS port). All Cribl
pack content (default/, data/) carries over from VisiCore unchanged; the
test/build tooling switches to TypeScript + Vitest + Biome + nix-devenv.

Pack changes (the user-visible behavior):

- default/pipelines/claude-code-otel/conf.yml — added Eval function setting
  sourcetype='claude:code:otel', index='claude-code', datatype='claude-code-otel'
- default/pipelines/claude-code-session-logs/conf.yml — added Eval function
  setting sourcetype='claude:code:session_logs', index='claude-code',
  datatype='claude-code-session-logs'

Test harness (synced from cc-edge-pack-template@feature/ts-pivot):

- tests/cribl-client.ts — TS client (CriblClient, runRouteFlow,
  assertRequiredFields, capture primitives)
- tests/parse-filter.ts, global-setup.ts, test-helpers.ts, routes.test.ts,
  pipelines.test.ts, vitest.config.ts, tsconfig.json
- tests/package.json + pnpm-lock.yaml (vitest 2.1, typescript 5.6, biome 2,
  tar 7, yaml 2, tsx 4)
- tests/generate-fixtures.ts — committed helper for regenerating expected
  fixtures via live Cribl (run via `pnpm exec tsx generate-fixtures.ts ...`)

Fixtures (the actual proof the pack works):

- tests/fixtures/claude-code-otel/sample.json — sourced from
  data/samples/sanitized_event_api_request.json (5 OTLP events)
- tests/fixtures/claude-code-otel/sample.expected.json — generated by
  running through live Cribl, trimmed to {sourcetype, index, datatype, _raw,
  _time}
- tests/fixtures/claude-code-session-logs/sample.json — sourced from
  data/samples/sanitized_assistant.log (1 assistant turn)
- tests/fixtures/claude-code-session-logs/sample.expected.json — same flow

Tooling (mirrors template):

- biome.jsonc, flake.nix, .envrc, .gitignore, Makefile (npm/pnpm scripts)
- .release-please-manifest.json + release-please-config.json (initial 0.0.0)

Workflows:

- .github/workflows/test.yml — caller for cc-edge-pack-template's reusable
  workflow, pinned to @feature/ts-pivot until that PR merges
- .github/workflows/release.yml — caller for the template release workflow
  (pinned to @main since release isn't exercised in this PR)
- .github/workflows/release-please.yml — caller for JacobPEvans inherited
  workflow

Docs:

- CLAUDE.md — rewritten for pilot scope (was a stale copy of the template's
  CLAUDE.md from Python era)
- README.md — added Installation + Usage anchors at top to satisfy readme
  validator; documented Eval pipeline tagging (sourcetype/index/datatype);
  removed the misleading 1.0.0 release notes line (no release exists yet,
  and per user direction the agent does NOT tag versions — release-please
  proposes them via PR for human approval)

Verification (locally, with `nix develop github:JacobPEvans/nix-devenv?dir=shells/typescript`):

- pnpm run typecheck: tsc --noEmit returns 0
- biome check: 14 files, 0 errors
- pnpm run test (vitest run): 11/11 passed
  - 7 structural route tests
  - 2 dynamic route flow tests (one per route — both produce non-empty output)
  - 2 fixture-driven pipeline tests (partial-match + required-fields assertion)

Sequencing note: the test caller workflow is temporarily pinned to
`cc-edge-pack-template@feature/ts-pivot` so this PR's CI can run before the
template PR merges. Flip to `@main` immediately after
dryvist/cc-edge-pack-template#3 lands.

(claude)
…amed input

dryvist exposes only the generic GH_APP_ID + GH_APP_PRIVATE_KEY org secrets;
the wrap pattern aliases them to whatever the inherited workflow expects at
the boundary. Anyone reading the dryvist repos sees only the generic name.

(claude)
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Ports the pack’s test/build harness from Python/pytest to TypeScript/Vitest (mirroring cc-edge-pack-template TS pivot) and updates the pack behavior so both pipelines set Splunk-canonical routing fields (sourcetype, index, datatype). This aligns runtime behavior with fixture-driven tests and adds reproducible fixtures generated from a live Cribl instance.

Changes:

  • Replace Python-based tests with a TypeScript/Vitest harness (client, global setup, route/pipeline suites, fixture generator).
  • Add/adjust Edge pack configuration: inputs, routes, pipelines (Eval tagging), pack logo, and sample catalog/data.
  • Add repo tooling + automation (Biome, nix flake/direnv, release-please config, updated workflows/Makefile/docs).

Reviewed changes

Copilot reviewed 57 out of 59 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
tests/vitest.config.ts Adds Vitest config with globalSetup + sequential execution.
tests/tsconfig.json Adds strict TS config for the test harness.
tests/test_routes.py Removes legacy Python route structure tests.
tests/test_pipelines.py Removes legacy Python fixture-driven pipeline tests.
tests/test-helpers.ts Adds helpers to construct a per-file Cribl client + read installed pack id.
tests/routes.test.ts Adds TS route structure + dynamic route-flow tests.
tests/requirements.txt Removes Python test dependencies list.
tests/pipelines.test.ts Adds TS fixture discovery + pipeline execution assertions.
tests/parse-filter.ts Adds small parser for simple field=='value' route filters.
tests/package.json Adds isolated pnpm workspace for tests (vitest/tsx/biome/typescript).
tests/global-setup.ts Adds Vitest global setup/teardown to install pack into Cribl once per run.
tests/generate-fixtures.ts Adds committed script to regenerate expected fixtures via live Cribl.
tests/fixtures/claude-code-session-logs/sample.json Adds session-logs pipeline input fixture.
tests/fixtures/claude-code-session-logs/sample.expected.json Adds expected output fixture asserting canonical fields.
tests/fixtures/claude-code-otel/sample.json Adds OTEL pipeline input fixture.
tests/fixtures/claude-code-otel/sample.expected.json Adds expected output fixture asserting canonical fields.
tests/cribl_client.py Removes legacy Python Cribl management API client.
tests/cribl-client.ts Adds TS Cribl management API client + pack tarball builder + assertions.
tests/conftest.py Removes legacy pytest fixtures for Cribl lifecycle.
tests/README.md Removes legacy Python test harness documentation.
release-please-config.json Adds release-please configuration for automated versioning/changelog.
package.json Sets actual pack id/metadata/tags for cc-edge-claude-code-io.
flake.nix Adds Nix flake devshell delegating to nix-devenv TypeScript shell.
default/samples.yml Populates Cribl UI sample catalog entries for included sample data.
default/pipelines/route.yml Replaces placeholder routes with real routes keyed by datatype.
default/pipelines/claude-code-session-logs/conf.yml Adds pipeline that tags session-log events with canonical fields.
default/pipelines/claude-code-otel/conf.yml Adds pipeline that tags OTEL events with canonical fields.
default/pack.yml Sets pack logo (base64 image).
default/inputs.yml Defines file monitor + OTLP inputs with datatype metadata.
data/samples/viOToM.json Adds sanitized OTEL sample payload (tool_result).
data/samples/smpLil.json Adds sanitized OTEL sample payload (api_error).
data/samples/nuJv6j.json Adds sanitized OTEL sample payload (user_prompt).
data/samples/mSfzuP.json Adds sanitized OTEL sample payload (tool_decision).
data/samples/lCrlnQ.json Adds sanitized OTEL sample payload (lines_of_code.count).
data/samples/kkcx4D.json Adds sanitized session-log sample payload (queue-operation).
data/samples/k0h8NP.json Adds sanitized session-log sample payload (file-history-snapshot).
data/samples/hO8QQb.json Adds sanitized session-log sample payload (assistant turn).
data/samples/go9A8j.json Adds sanitized OTEL sample payload (api_request).
data/samples/bw3bbR.json Adds sanitized OTEL sample payload (commit.count).
data/samples/bCOM2K.json Adds sanitized session-log sample payload (user tool_result).
data/samples/aukWzL.json Adds sanitized OTEL sample payload (active_time.total).
data/samples/QtXzUr.json Adds sanitized OTEL sample payload (token.usage).
data/samples/QS0mDG.json Adds sanitized OTEL sample payload (pull_request.count).
data/samples/OsXDpa.json Adds sanitized session-log sample payload (system stop_hook_summary).
data/samples/Io4OFW.json Adds sanitized OTEL sample payload (session.count).
data/samples/DBvKLd.json Adds sanitized OTEL sample payload (cost.usage).
data/samples/CBxGTZ.json Adds sanitized OTEL sample payload (code_edit_tool.decision).
data/samples/36cP1M.json Adds sanitized session-log sample payload (progress/agent_progress).
biome.jsonc Adds repo-wide Biome config (excluding pack contents + fixtures).
README.md Rewrites docs for this specific pack (install/usage/setup/reference).
Makefile Updates make targets for pnpm/Vitest/Biome + removes Python venv flow.
CLAUDE.md Updates AI assistant guidance to this pack’s scope + TS harness workflow.
.release-please-manifest.json Adds initial release-please manifest version.
.gitignore Switches ignores from Python venv to Node/TS + direnv/nix artifacts.
.github/workflows/test.yml Switches CI test job to reusable workflow in the TS template repo.
.github/workflows/release.yml Switches CI release job to reusable workflow in the TS template repo.
.github/workflows/release-please.yml Adds workflow calling org-wide release-please reusable workflow.
.envrc Enables direnv flake integration.
Files not reviewed (1)
  • tests/pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread README.md Outdated
Comment thread CLAUDE.md Outdated
Comment thread tests/generate-fixtures.ts Outdated
Comment thread tests/cribl-client.ts
Comment thread package.json
Comment thread release-please-config.json
Comment thread .release-please-manifest.json Outdated
Comment thread .github/workflows/test.yml
Pilot was scaffolded before the template added scripts/, .editorconfig,
lefthook.yml, the slim biome.jsonc, and the latest cribl-client.ts +
routes.test.ts (existsSync guard, response.ok check). Brings pilot in line
with template@main so CI can resolve scripts/validate-pack-structure.sh and
the rest of the chain.

(claude)
- README.md: fix double-quote typo in metadata example
- CLAUDE.md: routes filter on top-level datatype (set by input metadata),
  not _metadata.datatype
- generate-fixtures.ts: doc comment now matches actual usage
  (pnpm exec tsx, not node --experimental-strip-types)
- release-please-config.json: package-name -> cc-edge-claude-code-io
  (was leftover template placeholder)
- .release-please-manifest.json: align with package.json @ 1.0.0

(claude)
@JacobPEvans-personal
Copy link
Copy Markdown
Member Author

Closing in favor of #5, which is a minimal-scope rebuild from the now-proven template baseline (cc-edge-pack-template#8). This PR ballooned to 64 files with substantial pack content I authored alongside the harness — circular validation. The new PR keeps only the load-bearing test-infra + tooling + workflow + pack-identity changes; real claude-code-otel and claude-code-session-logs pipelines land in follow-up PRs.

@JacobPEvans-personal JacobPEvans-personal deleted the feat/ts-pivot branch April 26, 2026 21:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants