feat: TypeScript test harness + Eval functions + behavioral fixtures by JacobPEvans-personal · Pull Request #3 · dryvist/cc-edge-claude-code-io

JacobPEvans-personal · 2026-04-26T00:48:58Z

Summary

Pilot TS pivot mirroring dryvist/cc-edge-pack-template#3 (TS port). All Cribl pack content (`default/`, `data/`) carries over from VisiCore unchanged; the test/build tooling switches to TypeScript + Vitest + Biome + nix-devenv.

What changed

Pack behavior (the user-visible part)

Both pipelines (`claude-code-otel`, `claude-code-session-logs`) now run an Eval function that sets the Splunk-canonical fields per dryvist convention:

Pipeline	sourcetype	index	datatype
`claude-code-otel`	`claude:code:otel`	`claude-code`	`claude-code-otel`
`claude-code-session-logs`	`claude:code:session_logs`	`claude-code`	`claude-code-session-logs`

(Index value follows the vendor/software convention you specified.)

Test harness (synced from cc-edge-pack-template@feature/ts-pivot)

`tests/cribl-client.ts` — TS Cribl management API client
`tests/{parse-filter,global-setup,test-helpers}.ts` — supporting modules
`tests/{routes,pipelines}.test.ts` — Vitest suites
`tests/{package.json,tsconfig.json,vitest.config.ts,pnpm-lock.yaml}`
`tests/generate-fixtures.ts` — committed helper for regenerating expected fixtures (run via `pnpm exec tsx generate-fixtures.ts `)

Fixtures (the actual proof)

`tests/fixtures/claude-code-otel/sample.json` — sourced from `data/samples/sanitized_event_api_request.json` (5 OTLP events)
`tests/fixtures/claude-code-otel/sample.expected.json` — generated by running through live Cribl, trimmed to `{sourcetype, index, datatype, _raw, _time}`
`tests/fixtures/claude-code-session-logs/sample.{json,expected.json}` — same flow with `sanitized_assistant.log`

Tooling

`biome.jsonc`, `flake.nix` (delegates to `JacobPEvans/nix-devenv?dir=shells/typescript`), `.envrc`, updated `Makefile` + `.gitignore`
`.release-please-manifest.json` + `release-please-config.json` (initial 0.0.0)

Workflows

`.github/workflows/test.yml` — caller for `cc-edge-pack-template/.github/workflows/cribl-pack-test.yml`, pinned to `@feature/ts-pivot` until that PR merges
`.github/workflows/release.yml` — caller for the template release workflow
`.github/workflows/release-please.yml` — caller for inherited `JacobPEvans/.github/.github/workflows/_release-please.yml@main`

Docs

`CLAUDE.md` — rewritten for pilot scope (was stale Python-era template copy)
`README.md` — added Installation + Usage anchors at top, documented the Eval pipeline tagging, removed the misleading `1.0.0` release-notes line (no release exists yet)

Verification

Locally via `nix develop github:JacobPEvans/nix-devenv?dir=shells/typescript`:

`pnpm run typecheck` — `tsc --noEmit` returns 0
`biome check` — 14 files, 0 errors
`pnpm run test` — 11/11 passed:
- 7 structural route tests
- 2 dynamic route flow tests (default + jZR327, both fire correct pipeline + produce non-empty output)
- 2 fixture-driven pipeline tests (partial-match against expected fixtures + required-fields assertion proves `sourcetype`/`index`/`datatype` set correctly)

Sequencing

After CI green:

Merge dryvist/cc-edge-pack-template#3 first (so the workflow URL resolves cleanly at `@main`)
Update this PR: flip `uses: ...@feature/ts-pivot` -> `...@main` in `.github/workflows/test.yml`
CI re-runs green
Merge this PR
Merge dryvist/.github#1 last (drops the old workflow URLs)

No version tag in this PR. Release timing is yours to decide.

@main

Pilot pivot mirroring dryvist/cc-edge-pack-template#3 (TS port). All Cribl pack content (default/, data/) carries over from VisiCore unchanged; the test/build tooling switches to TypeScript + Vitest + Biome + nix-devenv. Pack changes (the user-visible behavior): - default/pipelines/claude-code-otel/conf.yml — added Eval function setting sourcetype='claude:code:otel', index='claude-code', datatype='claude-code-otel' - default/pipelines/claude-code-session-logs/conf.yml — added Eval function setting sourcetype='claude:code:session_logs', index='claude-code', datatype='claude-code-session-logs' Test harness (synced from cc-edge-pack-template@feature/ts-pivot): - tests/cribl-client.ts — TS client (CriblClient, runRouteFlow, assertRequiredFields, capture primitives) - tests/parse-filter.ts, global-setup.ts, test-helpers.ts, routes.test.ts, pipelines.test.ts, vitest.config.ts, tsconfig.json - tests/package.json + pnpm-lock.yaml (vitest 2.1, typescript 5.6, biome 2, tar 7, yaml 2, tsx 4) - tests/generate-fixtures.ts — committed helper for regenerating expected fixtures via live Cribl (run via `pnpm exec tsx generate-fixtures.ts ...`) Fixtures (the actual proof the pack works): - tests/fixtures/claude-code-otel/sample.json — sourced from data/samples/sanitized_event_api_request.json (5 OTLP events) - tests/fixtures/claude-code-otel/sample.expected.json — generated by running through live Cribl, trimmed to {sourcetype, index, datatype, _raw, _time} - tests/fixtures/claude-code-session-logs/sample.json — sourced from data/samples/sanitized_assistant.log (1 assistant turn) - tests/fixtures/claude-code-session-logs/sample.expected.json — same flow Tooling (mirrors template): - biome.jsonc, flake.nix, .envrc, .gitignore, Makefile (npm/pnpm scripts) - .release-please-manifest.json + release-please-config.json (initial 0.0.0) Workflows: - .github/workflows/test.yml — caller for cc-edge-pack-template's reusable workflow, pinned to @feature/ts-pivot until that PR merges - .github/workflows/release.yml — caller for the template release workflow (pinned to @main since release isn't exercised in this PR) - .github/workflows/release-please.yml — caller for JacobPEvans inherited workflow Docs: - CLAUDE.md — rewritten for pilot scope (was a stale copy of the template's CLAUDE.md from Python era) - README.md — added Installation + Usage anchors at top to satisfy readme validator; documented Eval pipeline tagging (sourcetype/index/datatype); removed the misleading 1.0.0 release notes line (no release exists yet, and per user direction the agent does NOT tag versions — release-please proposes them via PR for human approval) Verification (locally, with `nix develop github:JacobPEvans/nix-devenv?dir=shells/typescript`): - pnpm run typecheck: tsc --noEmit returns 0 - biome check: 14 files, 0 errors - pnpm run test (vitest run): 11/11 passed - 7 structural route tests - 2 dynamic route flow tests (one per route — both produce non-empty output) - 2 fixture-driven pipeline tests (partial-match + required-fields assertion) Sequencing note: the test caller workflow is temporarily pinned to `cc-edge-pack-template@feature/ts-pivot` so this PR's CI can run before the template PR merges. Flip to `@main` immediately after dryvist/cc-edge-pack-template#3 lands. (claude)

Copilot

Pull request overview

Ports the pack repo’s build/test tooling to a TypeScript + Vitest + Biome + nix-devenv harness, and updates the pack pipelines/routes to set Splunk-canonical fields (sourcetype, index, datatype) with fixture-backed behavioral verification.

Changes:

Replaces the Python-based test harness with a TypeScript/Vitest harness (client, global setup, route/pipeline suites, fixture generator).
Adds two pipelines + routes/inputs that tag events via Eval and route by datatype.
Introduces repo tooling for formatting/linting (Biome), dev shell (Nix flake/direnv), and release automation (release-please), plus updated docs.

Reviewed changes

Copilot reviewed 57 out of 59 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
tests/vitest.config.ts	Adds Vitest configuration (single-worker + global setup).
tests/tsconfig.json	Adds strict TypeScript compiler settings for test harness.
tests/test_routes.py	Removes legacy Python route tests.
tests/test_pipelines.py	Removes legacy Python pipeline fixture tests.
tests/test-helpers.ts	Adds per-test-file Cribl client helper + pack id accessor.
tests/routes.test.ts	Adds TS route structure tests + dynamic route-flow tests.
tests/requirements.txt	Removes Python test dependency list.
tests/pipelines.test.ts	Adds TS fixture-driven pipeline tests (optional required-fields skip).
tests/parse-filter.ts	Adds simple filter parser for dynamic route-flow testing.
tests/package.json	Adds Node/TS dev dependencies and scripts for tests.
tests/global-setup.ts	Adds Vitest global setup/teardown to install pack into Cribl once per run.
tests/generate-fixtures.ts	Adds committed helper to generate `.expected.json` fixtures via live Cribl.
tests/fixtures/claude-code-session-logs/sample.json	Adds session-logs input fixture.
tests/fixtures/claude-code-session-logs/sample.expected.json	Adds expected output fixture asserting canonical field tagging.
tests/fixtures/claude-code-otel/sample.json	Adds OTEL input fixture (5 events).
tests/fixtures/claude-code-otel/sample.expected.json	Adds expected output fixture asserting canonical field tagging.
tests/cribl_client.py	Removes legacy Python Cribl API client.
tests/cribl-client.ts	Adds TypeScript Cribl management API client + tarball builder + assertions.
tests/conftest.py	Removes pytest fixtures used by the legacy harness.
tests/README.md	Removes legacy Python test harness documentation.
release-please-config.json	Adds release-please configuration.
.release-please-manifest.json	Adds release-please manifest with initial version state.
package.json	Updates pack metadata (name/tags/etc.) and version.
flake.nix	Adds Nix flake delegating to shared TypeScript dev shell.
.envrc	Enables `direnv` flake usage.
default/samples.yml	Populates Cribl UI sample catalog entries for included sample files.
default/pipelines/route.yml	Replaces placeholder routes with `datatype`-based routing to the two pipelines.
default/pipelines/claude-code-session-logs/conf.yml	Adds Eval tagging for session logs pipeline.
default/pipelines/claude-code-otel/conf.yml	Adds Eval tagging for OTEL pipeline.
default/pack.yml	Adds embedded base64 logo.
default/inputs.yml	Adds file monitor + OTLP source definitions with `datatype` metadata.
data/samples/viOToM.json	Adds OTEL sample payload for Cribl UI samples.
data/samples/smpLil.json	Adds OTEL sample payload for Cribl UI samples.
data/samples/nuJv6j.json	Adds OTEL sample payload for Cribl UI samples.
data/samples/mSfzuP.json	Adds OTEL sample payload for Cribl UI samples.
data/samples/lCrlnQ.json	Adds OTEL sample payload for Cribl UI samples.
data/samples/kkcx4D.json	Adds session-log sample payload for Cribl UI samples.
data/samples/k0h8NP.json	Adds session-log sample payload for Cribl UI samples.
data/samples/hO8QQb.json	Adds session-log sample payload for Cribl UI samples.
data/samples/go9A8j.json	Adds OTEL sample payload for Cribl UI samples.
data/samples/bw3bbR.json	Adds OTEL sample payload for Cribl UI samples.
data/samples/bCOM2K.json	Adds session-log sample payload for Cribl UI samples.
data/samples/aukWzL.json	Adds OTEL sample payload for Cribl UI samples.
data/samples/QtXzUr.json	Adds OTEL sample payload for Cribl UI samples.
data/samples/QS0mDG.json	Adds OTEL sample payload for Cribl UI samples.
data/samples/OsXDpa.json	Adds session-log sample payload for Cribl UI samples.
data/samples/Io4OFW.json	Adds OTEL sample payload for Cribl UI samples.
data/samples/DBvKLd.json	Adds OTEL sample payload for Cribl UI samples.
data/samples/CBxGTZ.json	Adds OTEL sample payload for Cribl UI samples.
data/samples/36cP1M.json	Adds session-log sample payload for Cribl UI samples.
biome.jsonc	Adds Biome formatter/linter configuration.
README.md	Rewrites docs for this specific pack (install/usage/setup/reference).
Makefile	Updates developer commands to use pnpm/Vitest/Biome + keeps pack build/validate targets.
CLAUDE.md	Updates AI assistant guidance for this specific pack and new TS harness.
.gitignore	Switches ignores from Python venv/pytest to Node/direnv artifacts.
.github/workflows/test.yml	Updates CI to call TS-pivot reusable workflow (pinned to feature branch).
.github/workflows/release.yml	Updates CI to call release workflow from template repo.
.github/workflows/release-please.yml	Adds release-please workflow caller.

Files not reviewed (1)

tests/pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-26T00:52:29Z

+- **Auth:** None
+- **OTLP Version:** 1.3.1
+- **Max active connections:** 1000
+- **Metadata:** `datatype == ''claude-code-otel'`


The OTLP input metadata line has mismatched quoting: datatype == ''claude-code-otel'. This looks like a typo and may confuse readers; it should be a single-quoted value like datatype == 'claude-code-otel' (or match the exact route filter syntax used elsewhere).

Suggested change

- **Metadata:** `datatype == ''claude-code-otel'`

- **Metadata:** `datatype == 'claude-code-otel'`

Copilot · 2026-04-26T00:52:30Z

+
+## License
+---
+This Pack uses the following license: [`Apache 2.0`](https://github.com/criblio/appscope/blob/master/LICENSE)


The License section links to an unrelated repository (criblio/appscope). Since this pack includes its own LICENSE file, the README should link to that (or to the correct canonical license reference for this repository) to avoid misrepresenting the licensing terms.

Suggested change

This Pack uses the following license: [`Apache 2.0`](https://github.com/criblio/appscope/blob/master/LICENSE)

This Pack uses the following license: [`Apache 2.0`](LICENSE)

Copilot · 2026-04-26T00:52:30Z

+    const response = await fetch(url, init);
+    const text = await response.text();
+
+    // Try JSON first; fall back to NDJSON; fall back to raw text.
+    if (text.length === 0) return null;
+    try {


CriblClient.call() doesn’t check response.ok before attempting to parse the response body. For non-2xx responses, callers will treat error payloads (or HTML/text) as successful results, which can hide failures and make tests flaky/hard to debug. Please fail fast when !response.ok (include status + response text), similar to login().

Copilot · 2026-04-26T00:52:30Z

+const REQUIRED_FIELDS: Record<PackType, readonly string[]> = {
+  edge: ['sourcetype', 'index'],
+  stream: ['host', 'source', '_time'],
+};


assertRequiredFields() documents that Edge packs should have Splunk-canonical routing fields, but REQUIRED_FIELDS.edge only checks sourcetype and index. This PR’s pipelines also set datatype and routes depend on it, so fixture cases without an .expected.json won’t assert datatype is present. Consider adding datatype to the Edge required-fields list to make the smoke-test path enforce the intended contract.

Copilot · 2026-04-26T00:52:30Z

+  const absInput = resolve(inputPath);
+  const filename = absInput.split('/').pop();
+  if (filename === undefined) {
+    throw new Error(`Could not derive filename from path '${absInput}'`);
+  }
+  const expectedPath = join(dirname(absInput), `${filename.replace(/\.json$/, '')}.expected.json`);


generate-fixtures.ts derives filename by splitting on '/', which breaks on Windows paths (and is generally less robust). Use path.basename(absInput) instead so fixture generation works cross-platform.

Copilot · 2026-04-26T00:52:30Z

+  const events = JSON.parse(await readFile(absInput, 'utf-8')) as CriblEvent[];
+  console.log(`Loaded ${events.length} input event(s) from ${absInput}`);


generate-fixtures.ts assumes the input fixture JSON parses to CriblEvent[]. Elsewhere (e.g. pipelines.test.ts) fixtures can be a single object or an array; this script will log events.length and pass a non-array to saveSample() if given a single-object fixture. Consider normalizing to an array (and validating the parsed type) before continuing.

Copilot · 2026-04-26T00:52:31Z

+    ".": {
+      "package-name": "cc-edge-pack-template",
+      "changelog-path": "CHANGELOG.md",
+      "extra-files": ["package.json"]
+    }


release-please-config.json still uses package-name: "cc-edge-pack-template", which doesn’t match this pack’s actual package.json name (cc-edge-claude-code-io). This will produce incorrect release metadata (and can confuse downstream automation). Update the configured package name to match the pack.

Copilot · 2026-04-26T00:52:31Z

+  "name": "cc-edge-claude-code-io",
+  "version": "1.0.0",
  "minLogStreamVersion": "4.17.0",


package.json sets version to 1.0.0, but this PR also introduces a release-please manifest at 0.0.0 and the PR description states this is an initial pre-release with no releases cut yet. This mismatch will cause release-please/changelog state to be inconsistent. Align package.json version with .release-please-manifest.json (or update the manifest/config if 1.0.0 is intended).

…amed input dryvist exposes only the generic GH_APP_ID + GH_APP_PRIVATE_KEY org secrets; the wrap pattern aliases them to whatever the inherited workflow expects at the boundary. Anyone reading the dryvist repos sees only the generic name. (claude)

JacobPEvans-personal · 2026-04-26T12:58:35Z

Recreating on feat/ts-pivot branch (org ruleset rejects feature/ prefix). Same content + commits. Will reopen as a new PR shortly.

Copilot AI review requested due to automatic review settings April 26, 2026 00:49

Copilot started reviewing on behalf of JacobPEvans-personal April 26, 2026 00:49 View session

Copilot AI reviewed Apr 26, 2026

View reviewed changes

JacobPEvans-personal closed this Apr 26, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: TypeScript test harness + Eval functions + behavioral fixtures#3

feat: TypeScript test harness + Eval functions + behavioral fixtures#3
JacobPEvans-personal wants to merge 2 commits into
mainfrom
feature/ts-pivot

JacobPEvans-personal commented Apr 26, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 26, 2026

Uh oh!

Copilot AI Apr 26, 2026

Uh oh!

Copilot AI Apr 26, 2026

Uh oh!

Copilot AI Apr 26, 2026

Uh oh!

Copilot AI Apr 26, 2026

Uh oh!

Copilot AI Apr 26, 2026

Uh oh!

Copilot AI Apr 26, 2026

Uh oh!

Copilot AI Apr 26, 2026

Uh oh!

JacobPEvans-personal commented Apr 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	- Metadata: `datatype == ''claude-code-otel'`
	- Metadata: `datatype == 'claude-code-otel'`

	This Pack uses the following license: [`Apache 2.0`](https://github.com/criblio/appscope/blob/master/LICENSE)
	This Pack uses the following license: [`Apache 2.0`](LICENSE)

		const events = JSON.parse(await readFile(absInput, 'utf-8')) as CriblEvent[];
		console.log(`Loaded ${events.length} input event(s) from ${absInput}`);

Conversation

JacobPEvans-personal commented Apr 26, 2026

Summary

What changed

Pack behavior (the user-visible part)

Test harness (synced from cc-edge-pack-template@feature/ts-pivot)

Fixtures (the actual proof)

Tooling

Workflows

Docs

Verification

Sequencing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

Uh oh!

JacobPEvans-personal commented Apr 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants