From bca3e105afe438f35699efbb879a88021dc8a167 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 13:46:03 -0500 Subject: [PATCH 01/26] Add TypeScript implementation design spec Co-Authored-By: Claude Fable 5 --- ...-07-04-typescript-implementation-design.md | 206 ++++++++++++++++++ 1 file changed, 206 insertions(+) create mode 100644 .claude/superpowers/specs/2026-07-04-typescript-implementation-design.md diff --git a/.claude/superpowers/specs/2026-07-04-typescript-implementation-design.md b/.claude/superpowers/specs/2026-07-04-typescript-implementation-design.md new file mode 100644 index 0000000..f93aa61 --- /dev/null +++ b/.claude/superpowers/specs/2026-07-04-typescript-implementation-design.md @@ -0,0 +1,206 @@ +# TypeScript Implementation Design + +**Date:** 2026-07-04 +**Status:** Draft — pending review +**Goal:** Add a TypeScript implementation of esque to the monorepo, published to npm as `esque-ts`, behaviorally identical to the JVM and Python implementations and validated by the existing black-box compatibility test harness. + +## Decisions Made + +| Decision | Choice | Rationale | +|----------|--------|-----------| +| ES client | Official `@elastic/elasticsearch` (^9) | Consistent with JVM and Python (both use official clients); handles auth mechanisms, retries, and typing. | +| npm package name | `esque-ts` | Mirrors the `esque-py` PyPI naming convention. | +| Snapshot publishing | Real npm registry, `dev` dist-tag | Prerelease versions (`X.Y.Z-dev.N`) are invisible to normal installs and semver ranges; closest match to the JVM `-SNAPSHOT` / Python TestPyPI flow without a second registry. | +| Toolchain | Lean Node-native: npm + `tsc` (strict) + `node:test` + Biome | Fewest moving parts; Biome is the ruff analog (one tool for lint + format); no bundler needed for a small library + CLI. | +| Module format | ESM-only (`"type": "module"`), Node >= 22 | Modern default; no CJS consumers expected for a CLI-first tool. | +| Directory | `implementations/typescript/` | Full platform name, consistent with `implementations/python/`. | + +## Repository Placement + +``` +implementations/typescript/ +├── package.json # name: esque-ts, type: module, bin: {"esque": "dist/cli.js"}, engines: {"node": ">=22"} +├── package-lock.json +├── tsconfig.json # strict: true, NodeNext modules, outDir dist/ +├── biome.json # lint + format; line width 120 (matches ruff config) +├── src/ +│ ├── configuration.ts # EsqueConfiguration +│ ├── esque.ts # Esque orchestrator + verifyStateIntegrity +│ ├── cli.ts # commander entrypoint (shebang line) +│ ├── migration/ +│ │ ├── model.ts # MigrationFile, MigrationFileMetadata, MigrationFileContents, +│ │ │ # MigrationFileRequestDefinition, numeric version comparison +│ │ ├── template.ts # MigrationTemplateResolver: #{varName} validation + substitution +│ │ └── loader.ts # MigrationFileLoader: discovery, parsing, canonical checksum +│ └── elasticsearch/ +│ ├── documents.ts # INDEX_DEFINITION, index name, wrapper-object document types +│ ├── operations.ts # RestClientOperations over @elastic/elasticsearch +│ └── lock.ts # ElasticsearchDocumentLock +└── tests/ + ├── model.test.ts # version ordering (1.9.0 < 1.10.0, padding) + ├── checksum.test.ts # canonical checksum reference vectors + ├── template.test.ts # validation + substitution across all request fields + └── integrity.test.ts # verifyStateIntegrity error scenarios via mock client +``` + +**Runtime dependencies:** `@elastic/elasticsearch` (^9), `commander`, `yaml`. +**Dev dependencies:** `typescript`, `@biomejs/biome`, `tsx`, `@types/node`. + +## Behavioral Contract + +The implementation follows the esque-new-language-implementation skill guide exactly: same nine +building blocks, same ES document wrapper-object shapes (`{"migration": {...}}`, `{"lock": {...}}`), +camelCase field names in ES, lock doc id `lock:`, `.esque` index definition, +`refresh=true` on record creation, and identical CLI option surface. Only TypeScript-specific +design points are documented below. + +## TypeScript-Specific Design + +### Canonical checksum + +`JSON.stringify` does not sort keys, so `loader.ts` includes a small canonical-serialization +helper: + +1. Build `{"requests": [request.toCanonicalDict(), ...]}` from the **resolved** requests; + `toCanonicalDict()` returns only non-null fields with camelCase keys. +2. Recursively drop `null`/`undefined` values. +3. Serialize with keys sorted alphabetically at every level, compact separators (no spaces). +4. UTF-8 encode → `node:crypto` MD5 → `Buffer.readInt32BE(0)` (first 4 bytes as big-endian + signed 32-bit int). + +`checksum.test.ts` pins the same reference vectors as the Python `test_checksum.py`, which +guarantees cross-implementation equality; the compatibility harness verifies it end-to-end. + +### Async model and lifecycle + +- All ES-touching methods are `async`; the CLI awaits `execute()`. +- `Esque` constructor is `(client: Client, configuration: EsqueConfiguration, properties: Record = {})` + and only stores references / instantiates collaborators — no I/O at construction time + (required so unit tests can construct with a mock client). +- `Esque` implements `close()` (try unlock → swallow "not held" errors, warn on others; then + close the client) and `Symbol.asyncDispose` delegating to `close()` so `await using` works. +- `verifyStateIntegrity` and `verifyRecordIntegrity` are TS-`private` methods on `Esque`, using + `configuration.migrationKey` directly. Unit tests access them via index access + (`esque["verifyStateIntegrity"](...)`) on an instance built with a mock client — TS `private` + is compile-time only, so this works without `any` casts beyond the index expression. + +### Version comparison + +`model.ts` exports a comparator: split versions on `.`, compare segment-by-segment numerically, +pad the shorter version with zeros (`1.9.0 < 1.10.0`, `2.1 == 2.1.0` for ordering purposes). +Files sort by this comparator after loading. + +### Distributed lock + +- ES side identical to other implementations: `op_type=create` on doc id `lock:`, + poll every 100ms via `setTimeout` until the deadline (`lockTimeoutMinutes`, default 5). +- `tryLock(timeoutMinutes): Promise`, `unlock(): Promise` (deletes the ES doc, + releases the local guard in `finally`). +- `doLock()` returns `false` on any error (no ConflictError differentiation yet — same known + TODO as the other implementations). +- The JVM wraps the ES lock in a local `ReentrantLock` for thread safety; Node is + single-threaded, so the local guard is a simple held-flag maintained for API parity and to + make `unlock()` without `tryLock()` an error. + +### Error semantics + +- CLI exits 1 on any error, message to `stderr` (matching JVM/Python). +- Template validation collects **all** missing variables before throwing. +- `checkMigrationIndexExists` treats 404 as `false`; `createMigrationIndex` treats 409 + (already exists) as success. + +## CLI + +`commander`, identical options to Clikt/Click: + +``` +--es-url TEXT required +--migrations-dir TEXT required # CLI prepends "file:" before building EsqueConfiguration +--migration-key TEXT required +--migration-user TEXT optional +--lock-timeout-minutes N default 5 +--property key=value repeatable; split on first "=" +``` + +`package.json` declares `"bin": {"esque": "dist/cli.js"}`; `cli.ts` starts with +`#!/usr/bin/env node`. Consumers can run `npx esque-ts` or install globally. + +## Compatibility Harness Registration + +Add to `tests/implementations.yml`: + +```yaml +typescript: + invocation: direct + command: ["npm", "exec", "--prefix", "implementations/typescript", "--", "tsx", "implementations/typescript/src/cli.ts"] +``` + +Running via `tsx` means the harness never depends on a possibly-stale `dist/` build — only +`npm ci` in `implementations/typescript/` is required beforehand (analogous to `uv run`'s +auto-sync for Python). The 17 parametrized scenarios run automatically against the new +implementation, and the non-parametrized `test_cross_implementation_record_equivalency` picks +it up too — taking the CI compat matrix from 35 to 52 tests (17 × 3 + 1). + +Per existing project policy, different-language implementations are **not** interchangeable +against the same `migrationKey`; the harness tests each implementation independently against +the shared behavioral contract. + +## Versioning + +New `.github/version_typescript.sh`, mirroring `version_python.sh` but emitting strict SemVer: + +- Exact tag `X.Y.Z` → `X.Y.Z` (release) +- Otherwise (tag + N commits) → `X.Y.Z-dev.N` (prerelease) +- No tags → `0.0.0-dev.` + +npm constraints honored: exactly three numeric segments, prerelease as `-dev.N` (PEP 440's +fourth-segment `.devN` is invalid SemVer), versions immutable once published. Note: +`X.Y.Z-dev.N` sorts *before* the released `X.Y.Z`; this is harmless because the `dev` +dist-tag and normal semver ranges shield consumers from prereleases, and it matches the +ordering semantics of PEP 440 `.devN`. + +## CI + +New `typescript` job in `.github/workflows/ci.yml`, following the `python` job pattern: + +1. `actions/checkout` (with `fetch-depth: 0` + `fetch-tags` for git describe) +2. `actions/setup-node` — Node 24 +3. `npm ci` (in `implementations/typescript/`) +4. Lint/format check: `npx biome ci src/ tests/` +5. Typecheck: `npx tsc --noEmit` +6. Unit tests: `node --import tsx --test tests/` (runs the `.test.ts` files directly, no build + step; same invocation used by the pre-commit hook and the `npm test` script) +7. Build: `npx tsc` +8. Version: patch `package.json` via + `npm version --no-git-tag-version "$(.github/version_typescript.sh)"` +9. Publish: `npm publish --tag dev` for non-release builds; `npm publish` (implicit `latest`) + on GitHub release events. Auth via new `NPM_TOKEN` repository secret. + +`compatibility-tests` job: add `needs: typescript` and an `npm ci` step for +`implementations/typescript/` before running pytest. + +## Repo Housekeeping + +- **Pre-commit hook** (`.githooks/pre-commit`): insert `[3] TypeScript checks` (biome check, + `tsc --noEmit`, `node --test`) and renumber compat tests to `[4]`. +- **Dev container**: add Node 24 (devcontainer feature or apt setup). +- **CLAUDE.md**: add TypeScript to repository structure, build commands, code conventions, and + registered implementations; update compat test count (35 → 52); fix the stale claim that the + Python implementation uses httpx — it uses the official `elasticsearch` client + (`elasticsearch>=9.0.0`). + +## Testing Strategy + +Matches the Python precedent exactly: + +- **Unit tests** (`node:test`, no ES): the four complex areas — version ordering, canonical + checksum, template resolution, integrity verification (with a mock ES client). +- **Integration**: the black-box compatibility harness in `tests/` is the integration layer. + No TypeScript-specific integration tests. + +## Out of Scope + +- Rollback/undo, FAILED migration records, ConflictError differentiation — existing known + TODOs shared by all implementations. +- CJS build output, Bun/Deno support. +- npm provenance/OIDC publishing (can be added later; requires workflow permission changes). From 37564a5ef583b0b245500aaabeff01c359a6c127 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 14:19:53 -0500 Subject: [PATCH 02/26] Add TypeScript implementation plan Co-Authored-By: Claude Fable 5 --- .../2026-07-04-typescript-implementation.md | 1810 +++++++++++++++++ 1 file changed, 1810 insertions(+) create mode 100644 .claude/superpowers/plans/2026-07-04-typescript-implementation.md diff --git a/.claude/superpowers/plans/2026-07-04-typescript-implementation.md b/.claude/superpowers/plans/2026-07-04-typescript-implementation.md new file mode 100644 index 0000000..dbe2083 --- /dev/null +++ b/.claude/superpowers/plans/2026-07-04-typescript-implementation.md @@ -0,0 +1,1810 @@ +# TypeScript Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add a TypeScript implementation of esque at `implementations/typescript/`, published to npm as `esque-ts`, behaviorally identical to the JVM/Python implementations and validated by the existing compatibility harness. + +**Architecture:** Module-for-module mirror of `implementations/python/src/esque/` (the most recent reference port). ESM-only Node >= 22 package; official `@elastic/elasticsearch` client; all ES-touching code async. The compat harness invokes the CLI via `tsx` directly against `src/` so no build step is needed for tests. + +**Tech Stack:** TypeScript (strict), npm, `tsc` build, `node:test` + `tsx` for unit tests, Biome for lint/format, `commander` CLI, `yaml` parser. + +**Spec:** `.claude/superpowers/specs/2026-07-04-typescript-implementation-design.md` + +**Reference implementation:** `implementations/python/src/esque/` — when in doubt about behavior or error messages, mirror it exactly. Error message *strings* should match Python's (the compat harness greps stderr in some scenarios and consistency helps debugging). + +**Note on commits:** the repo pre-commit hook runs JVM checks + Python checks + the full compat suite (~4 minutes). That is expected; don't bypass with `--no-verify`. Commit at the end of each task, not each step. + +--- + +## Task 0: Verify local prerequisites + +**Files:** none + +- [ ] **Step 1: Check Node and npm are available and recent enough** + +Run: `node --version && npm --version` +Expected: Node >= 22.x. If Node is missing or too old, stop and report — do not attempt to install system-wide tooling without asking. + +--- + +## Task 1: Project scaffolding + +**Files:** +- Create: `implementations/typescript/package.json` +- Create: `implementations/typescript/tsconfig.json` +- Create: `implementations/typescript/tsconfig.build.json` +- Create: `implementations/typescript/biome.json` +- Create: `implementations/typescript/.gitignore` +- Create: `implementations/typescript/src/` and `implementations/typescript/tests/` directories + +- [ ] **Step 1: Create `implementations/typescript/package.json`** + +```json +{ + "name": "esque-ts", + "version": "0.0.0", + "description": "Esque (Elasticsearch Stateful Query Executor) — migration management for Elasticsearch, like Flyway for ES clusters", + "license": "Apache-2.0", + "type": "module", + "engines": { + "node": ">=22" + }, + "bin": { + "esque": "dist/cli.js" + }, + "exports": { + ".": "./dist/esque.js" + }, + "files": [ + "dist" + ], + "repository": { + "type": "git", + "url": "git+https://github.com/loesak/esque.git" + }, + "scripts": { + "build": "tsc -p tsconfig.build.json", + "typecheck": "tsc --noEmit", + "lint": "biome ci src tests", + "format": "biome format --write src tests", + "test": "node --import tsx --test tests/*.test.ts", + "prepublishOnly": "npm run build" + }, + "dependencies": { + "@elastic/elasticsearch": "^9.0.0", + "commander": "^14.0.0", + "yaml": "^2.7.0" + }, + "devDependencies": { + "@biomejs/biome": "^2.0.0", + "@types/node": "^22.0.0", + "tsx": "^4.19.0", + "typescript": "^5.7.0" + } +} +``` + +(The `version` field is a placeholder — CI patches it via `npm version --no-git-tag-version` before publishing, mirroring how `version_python.sh` output is sed-ed into `pyproject.toml`.) + +- [ ] **Step 2: Create `implementations/typescript/tsconfig.json`** (typecheck config — covers src AND tests) + +```json +{ + "compilerOptions": { + "target": "es2023", + "module": "nodenext", + "moduleResolution": "nodenext", + "strict": true, + "noUncheckedIndexedAccess": true, + "forceConsistentCasingInFileNames": true, + "skipLibCheck": true, + "noEmit": true + }, + "include": ["src", "tests"] +} +``` + +- [ ] **Step 3: Create `implementations/typescript/tsconfig.build.json`** (build config — src only, emits `dist/`) + +```json +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "noEmit": false, + "outDir": "dist", + "rootDir": "src", + "declaration": true, + "sourceMap": true + }, + "include": ["src"] +} +``` + +- [ ] **Step 4: Create `implementations/typescript/biome.json`** + +```json +{ + "$schema": "./node_modules/@biomejs/biome/configuration_schema.json", + "files": { + "includes": ["src/**", "tests/**"] + }, + "formatter": { + "enabled": true, + "indentStyle": "space", + "indentWidth": 2, + "lineWidth": 120 + }, + "linter": { + "enabled": true, + "rules": { + "recommended": true + } + }, + "javascript": { + "formatter": { + "quoteStyle": "double" + } + } +} +``` + +(If `biome ci` later complains about the config shape, run `npx biome migrate --write` — the schema key names occasionally shift between Biome majors; accept whatever the migration produces.) + +- [ ] **Step 5: Create `implementations/typescript/.gitignore`** + +``` +node_modules/ +dist/ +``` + +- [ ] **Step 6: Install dependencies** + +Run: `cd implementations/typescript && npm install` +Expected: `package-lock.json` created, no errors. `package-lock.json` MUST be committed (CI uses `npm ci`). + +- [ ] **Step 7: Sanity-check the toolchain** + +Run: `cd implementations/typescript && npx tsc --noEmit && npx biome ci src tests 2>&1 || true` +Expected: `tsc` succeeds trivially (no inputs yet is OK, or "No inputs were found" — if that error appears, create an empty placeholder `src/configuration.ts` containing only `export {};`; it gets real content in Task 2). Biome may report "no files" — fine. + +- [ ] **Step 8: Commit** + +```bash +git add implementations/typescript +git commit -m "TypeScript implementation: project scaffolding" +``` + +--- + +## Task 2: Configuration and migration model (TDD) + +**Files:** +- Create: `implementations/typescript/src/configuration.ts` +- Create: `implementations/typescript/src/migration/model.ts` +- Test: `implementations/typescript/tests/model.test.ts` + +- [ ] **Step 1: Write the failing test** — port of `implementations/python/tests/test_model.py` + +`implementations/typescript/tests/model.test.ts`: + +```typescript +import assert from "node:assert/strict"; +import { test } from "node:test"; +import type { MigrationFile } from "../src/migration/model.js"; +import { compareMigrationFiles } from "../src/migration/model.js"; + +function file(version: string, description = "Test"): MigrationFile { + return { + metadata: { filename: `V${version}__${description}.yml`, version, description, checksum: 0 }, + contents: { requests: [] }, + }; +} + +test("numeric segment ordering: 1.9.0 < 1.10.0", () => { + assert.ok(compareMigrationFiles(file("1.9.0"), file("1.10.0")) < 0); +}); + +test("major ordering", () => { + assert.ok(compareMigrationFiles(file("1.0.0"), file("2.0.0")) < 0); +}); + +test("minor ordering", () => { + assert.ok(compareMigrationFiles(file("1.0.0"), file("1.1.0")) < 0); +}); + +test("patch ordering", () => { + assert.ok(compareMigrationFiles(file("1.0.0"), file("1.0.1")) < 0); +}); + +test("unequal segment count: 1.0 < 1.0.1", () => { + assert.ok(compareMigrationFiles(file("1.0"), file("1.0.1")) < 0); +}); + +test("equal versions compare as 0", () => { + assert.equal(compareMigrationFiles(file("1.0.0"), file("1.0.0")), 0); +}); + +test("sort order", () => { + const files = [file("1.10.0"), file("2.0.0"), file("1.9.0"), file("1.0.0")]; + files.sort(compareMigrationFiles); + assert.deepEqual( + files.map((f) => f.metadata.version), + ["1.0.0", "1.9.0", "1.10.0", "2.0.0"], + ); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd implementations/typescript && npm test` +Expected: FAIL — cannot find module `../src/migration/model.js` + +- [ ] **Step 3: Write `implementations/typescript/src/configuration.ts`** + +```typescript +export interface EsqueConfiguration { + readonly migrationKey: string; + readonly migrationUser: string | null; + readonly migrationDirectory: string; + readonly lockTimeoutMinutes: number; +} + +export function createEsqueConfiguration(options: { + migrationKey: string; + migrationUser?: string | null; + migrationDirectory?: string; + lockTimeoutMinutes?: number; +}): EsqueConfiguration { + return { + migrationKey: options.migrationKey, + migrationUser: options.migrationUser ?? null, + migrationDirectory: options.migrationDirectory ?? "file:es.migration", + lockTimeoutMinutes: options.lockTimeoutMinutes ?? 5, + }; +} +``` + +(Defaults mirror the Python `EsqueConfiguration` dataclass: `migration_directory="file:es.migration"`, `lock_timeout_minutes=5`.) + +- [ ] **Step 4: Write `implementations/typescript/src/migration/model.ts`** + +```typescript +export interface MigrationFileRequestDefinition { + readonly method: string; + readonly path: string; + readonly contentType: string | null; + readonly params: Readonly> | null; + readonly body: string | null; +} + +export interface CanonicalRequest { + method: string; + path: string; + body?: string; + contentType?: string; + params?: Record; +} + +// Only non-null fields, camelCase keys — this shape feeds the checksum. +export function toCanonicalDict(request: MigrationFileRequestDefinition): CanonicalRequest { + const d: CanonicalRequest = { method: request.method, path: request.path }; + if (request.body !== null) { + d.body = request.body; + } + if (request.contentType !== null) { + d.contentType = request.contentType; + } + if (request.params !== null) { + d.params = { ...request.params }; + } + return d; +} + +export interface MigrationFileMetadata { + readonly filename: string; + readonly version: string; + readonly description: string; + readonly checksum: number; +} + +export interface MigrationFileContents { + readonly requests: readonly MigrationFileRequestDefinition[]; +} + +export interface MigrationFile { + readonly metadata: MigrationFileMetadata; + readonly contents: MigrationFileContents; +} + +// Numeric per-segment comparison; shorter versions padded with zeros (1.9.0 < 1.10.0, 1.0 < 1.0.1). +// Ties broken by description, mirroring the Python MigrationFile.__lt__. +export function compareMigrationFiles(a: MigrationFile, b: MigrationFile): number { + const av = a.metadata.version.split(".").map(Number); + const bv = b.metadata.version.split(".").map(Number); + const len = Math.max(av.length, bv.length); + for (let i = 0; i < len; i++) { + const diff = (av[i] ?? 0) - (bv[i] ?? 0); + if (diff !== 0) { + return diff; + } + } + if (a.metadata.description < b.metadata.description) { + return -1; + } + if (a.metadata.description > b.metadata.description) { + return 1; + } + return 0; +} +``` + +- [ ] **Step 5: Run test to verify it passes** + +Run: `cd implementations/typescript && npm test` +Expected: PASS (7 tests) + +- [ ] **Step 6: Lint, format, typecheck** + +Run: `cd implementations/typescript && npm run format && npx biome ci src tests && npm run typecheck` +Expected: all clean + +- [ ] **Step 7: Commit** + +```bash +git add implementations/typescript +git commit -m "TypeScript implementation: configuration and migration model" +``` + +--- + +## Task 3: Template resolver (TDD) + +**Files:** +- Create: `implementations/typescript/src/migration/template.ts` +- Test: `implementations/typescript/tests/template.test.ts` + +- [ ] **Step 1: Write the failing test** — port of `implementations/python/tests/test_template.py` + +`implementations/typescript/tests/template.test.ts`: + +```typescript +import assert from "node:assert/strict"; +import { test } from "node:test"; +import type { MigrationFile, MigrationFileRequestDefinition } from "../src/migration/model.js"; +import { MigrationTemplateResolver } from "../src/migration/template.js"; + +function req(partial: Partial & { method: string; path: string }): MigrationFileRequestDefinition { + return { contentType: null, params: null, body: null, ...partial }; +} + +function fileOf(...requests: MigrationFileRequestDefinition[]): MigrationFile { + return { + metadata: { filename: "V1.0.0__Test.yml", version: "1.0.0", description: "Test", checksum: 0 }, + contents: { requests }, + }; +} + +test("validate passes when all vars present", () => { + const r = req({ method: "PUT", path: "/#{indexName}" }); + new MigrationTemplateResolver({ indexName: "my-index" }).validate([fileOf(r)]); +}); + +test("validate throws on missing var", () => { + const r = req({ method: "PUT", path: "/#{missing}" }); + assert.throws(() => new MigrationTemplateResolver({}).validate([fileOf(r)]), /missing/); +}); + +test("validate collects all missing vars", () => { + const r = req({ method: "PUT", path: "/#{a}", body: "#{b}" }); + assert.throws( + () => new MigrationTemplateResolver({}).validate([fileOf(r)]), + (error: Error) => error.message.includes("a") && error.message.includes("b"), + ); +}); + +test("validate checks body, params, and contentType", () => { + const r = req({ method: "POST", path: "/", contentType: "#{ct}", params: { k: "#{v}" }, body: "#{body}" }); + assert.throws( + () => new MigrationTemplateResolver({}).validate([fileOf(r)]), + (error: Error) => error.message.includes("ct") && error.message.includes("v") && error.message.includes("body"), + ); +}); + +test("resolve substitutes path", () => { + const r = req({ method: "PUT", path: "/#{indexName}" }); + const result = new MigrationTemplateResolver({ indexName: "my-index" }).resolve(r); + assert.equal(result.path, "/my-index"); +}); + +test("resolve substitutes body", () => { + const r = req({ method: "POST", path: "/", body: '{"index": "#{name}"}' }); + const result = new MigrationTemplateResolver({ name: "test" }).resolve(r); + assert.equal(result.body, '{"index": "test"}'); +}); + +test("resolve substitutes params values", () => { + const r = req({ method: "GET", path: "/", params: { q: "#{query}" } }); + const result = new MigrationTemplateResolver({ query: "value" }).resolve(r); + assert.deepEqual(result.params, { q: "value" }); +}); + +test("resolve substitutes contentType", () => { + const r = req({ method: "PUT", path: "/", contentType: "#{ct}" }); + const result = new MigrationTemplateResolver({ ct: "application/json" }).resolve(r); + assert.equal(result.contentType, "application/json"); +}); + +test("resolve does not substitute method", () => { + const r = req({ method: "PUT", path: "/index" }); + const result = new MigrationTemplateResolver({}).resolve(r); + assert.equal(result.method, "PUT"); +}); + +test("resolve handles no template vars", () => { + const r = req({ method: "DELETE", path: "/index", body: '{"key": "value"}' }); + const result = new MigrationTemplateResolver({}).resolve(r); + assert.equal(result.path, "/index"); + assert.equal(result.body, '{"key": "value"}'); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd implementations/typescript && npm test` +Expected: FAIL — cannot find module `../src/migration/template.js` + +- [ ] **Step 3: Write `implementations/typescript/src/migration/template.ts`** + +```typescript +import type { MigrationFile, MigrationFileContents, MigrationFileRequestDefinition } from "./model.js"; + +const PLACEHOLDER_PATTERN = /#\{([a-zA-Z0-9._-]+)\}/g; + +export class MigrationTemplateResolver { + private readonly properties: Readonly>; + + constructor(properties: Readonly>) { + this.properties = properties; + } + + // Collects ALL missing variables before throwing. + validate(files: readonly MigrationFile[]): void { + const missing = new Set(); + for (const file of files) { + for (const request of file.contents.requests) { + const texts = [ + request.path, + request.contentType ?? "", + request.body ?? "", + ...Object.values(request.params ?? {}), + ]; + for (const text of texts) { + for (const match of text.matchAll(PLACEHOLDER_PATTERN)) { + const key = match[1]; + if (key !== undefined && !(key in this.properties)) { + missing.add(key); + } + } + } + } + } + if (missing.size > 0) { + throw new Error( + `migration files reference template variables with no matching properties: ${[...missing].join(", ")}`, + ); + } + } + + // Substitutes #{varName} in path, contentType, params values, body — NOT method. + resolve(definition: MigrationFileRequestDefinition): MigrationFileRequestDefinition { + return { + method: definition.method, + path: this.substitute(definition.path), + contentType: definition.contentType !== null ? this.substitute(definition.contentType) : null, + params: + definition.params !== null + ? Object.fromEntries(Object.entries(definition.params).map(([k, v]) => [k, this.substitute(v)])) + : null, + body: definition.body !== null ? this.substitute(definition.body) : null, + }; + } + + resolveContents(contents: MigrationFileContents): MigrationFileContents { + return { requests: contents.requests.map((r) => this.resolve(r)) }; + } + + private substitute(text: string): string { + return text.replace(PLACEHOLDER_PATTERN, (_match, key: string) => { + const value = this.properties[key]; + if (value === undefined) { + throw new Error(`unresolved template variable '#{${key}}' — was validate() called?`); + } + return value; + }); + } +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd implementations/typescript && npm test` +Expected: PASS (7 model + 10 template tests) + +- [ ] **Step 5: Lint, format, typecheck** + +Run: `cd implementations/typescript && npm run format && npx biome ci src tests && npm run typecheck` +Expected: all clean + +- [ ] **Step 6: Commit** + +```bash +git add implementations/typescript +git commit -m "TypeScript implementation: template resolver" +``` + +--- + +## Task 4: Canonical checksum (TDD) + +**Files:** +- Create: `implementations/typescript/src/migration/loader.ts` (checksum half only; `MigrationFileLoader` added in Task 5) +- Test: `implementations/typescript/tests/checksum.test.ts` + +- [ ] **Step 1: Write the failing test** — port of `implementations/python/tests/test_checksum.py` + +`implementations/typescript/tests/checksum.test.ts`: + +```typescript +import assert from "node:assert/strict"; +import { test } from "node:test"; +import { calculateChecksum } from "../src/migration/loader.js"; +import type { MigrationFileRequestDefinition } from "../src/migration/model.js"; + +function req(partial: Partial & { method: string; path: string }): MigrationFileRequestDefinition { + return { contentType: null, params: null, body: null, ...partial }; +} + +function checksum(requests: MigrationFileRequestDefinition[]): number { + return calculateChecksum({ requests }); +} + +test("result is a signed 32-bit integer", () => { + const result = checksum([req({ method: "PUT", path: "/test-index" })]); + assert.ok(Number.isInteger(result)); + assert.ok(result >= -(2 ** 31) && result <= 2 ** 31 - 1); +}); + +test("deterministic", () => { + const requests = [req({ method: "PUT", path: "/test", body: '{"settings": {}}' })]; + assert.equal(checksum(requests), checksum(requests)); +}); + +test("null fields excluded", () => { + const r1 = req({ method: "PUT", path: "/index" }); + const r2 = req({ method: "PUT", path: "/index", body: null, contentType: null, params: null }); + assert.equal(checksum([r1]), checksum([r2])); +}); + +test("different content differs", () => { + const r1 = [req({ method: "PUT", path: "/index-a" })]; + const r2 = [req({ method: "PUT", path: "/index-b" })]; + assert.notEqual(checksum(r1), checksum(r2)); +}); + +test("cross-implementation reference vector", () => { + // MUST equal the Python/JVM value for the identical input. -991565970 was generated from + // the Python reference implementation via MigrationFileLoader.calculate_checksum for + // [MigrationFileRequestDefinition(method="PUT", path="/test-index")]. + const result = checksum([req({ method: "PUT", path: "/test-index" })]); + assert.equal(result, -991565970); +}); + +test("multiple requests", () => { + const r1 = req({ method: "PUT", path: "/index" }); + const r2 = req({ method: "POST", path: "/_aliases", body: "{}" }); + const combined = checksum([r1, r2]); + assert.notEqual(combined, checksum([r1])); + assert.notEqual(combined, checksum([r2])); +}); +``` + +The `-991565970` vector hard-pins cross-implementation checksum equality in a unit test (the compat harness's `test_cross_implementation_record_equivalency` also verifies it end-to-end). + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd implementations/typescript && npm test` +Expected: FAIL — cannot find module `../src/migration/loader.js` + +- [ ] **Step 3: Write the checksum half of `implementations/typescript/src/migration/loader.ts`** + +```typescript +import { createHash } from "node:crypto"; +import type { MigrationFileContents } from "./model.js"; +import { toCanonicalDict } from "./model.js"; + +type CanonicalValue = string | number | boolean | null | CanonicalValue[] | { [key: string]: CanonicalValue | undefined }; + +// Canonical JSON: keys sorted alphabetically at every level, null/undefined object values +// dropped, compact separators. Matches Python's +// json.dumps(remove_nulls(data), sort_keys=True, separators=(",", ":"), ensure_ascii=False). +function canonicalJson(value: CanonicalValue): string { + if (value === null || typeof value !== "object") { + return JSON.stringify(value); + } + if (Array.isArray(value)) { + return `[${value.map((item) => canonicalJson(item)).join(",")}]`; + } + const entries = Object.entries(value) + .filter((entry): entry is [string, CanonicalValue] => entry[1] !== null && entry[1] !== undefined) + .sort(([a], [b]) => (a < b ? -1 : 1)); + return `{${entries.map(([k, v]) => `${JSON.stringify(k)}:${canonicalJson(v)}`).join(",")}}`; +} + +// Canonical algorithm (identical across all implementations): +// canonical JSON of {"requests": [...]} → UTF-8 → MD5 → first 4 bytes as big-endian signed int32. +export function calculateChecksum(contents: MigrationFileContents): number { + const data: CanonicalValue = { requests: contents.requests.map((r) => toCanonicalDict(r) as CanonicalValue) }; + const digest = createHash("md5").update(canonicalJson(data), "utf8").digest(); + return digest.readInt32BE(0); +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd implementations/typescript && npm test` +Expected: PASS (including the cross-implementation reference vector) + +- [ ] **Step 5: Lint, format, typecheck** + +Run: `cd implementations/typescript && npm run format && npx biome ci src tests && npm run typecheck` +Expected: all clean + +- [ ] **Step 6: Commit** + +```bash +git add implementations/typescript +git commit -m "TypeScript implementation: canonical checksum" +``` + +--- + +## Task 5: Migration file loader + +**Files:** +- Modify: `implementations/typescript/src/migration/loader.ts` (append loader class to the checksum code from Task 4) + +No dedicated unit test (mirrors Python, where file loading is covered by the compat harness). The checksum tests from Task 4 must keep passing. + +- [ ] **Step 1: Append to `implementations/typescript/src/migration/loader.ts`** + +Add these imports at the top (merging with the existing ones): + +```typescript +import { readdirSync, readFileSync, statSync } from "node:fs"; +import { join } from "node:path"; +import { parse } from "yaml"; +import type { MigrationFile, MigrationFileRequestDefinition } from "./model.js"; +import { compareMigrationFiles } from "./model.js"; +import type { MigrationTemplateResolver } from "./template.js"; +``` + +Then append: + +```typescript +const FILE_NAME_PATTERN = /^V((\d+\.?)+)__(\w+)\.yml$/; + +export class MigrationFileLoader { + private readonly migrationDirectory: string; + private readonly templateResolver: MigrationTemplateResolver; + + constructor(migrationDirectory: string, templateResolver: MigrationTemplateResolver) { + this.migrationDirectory = migrationDirectory; + this.templateResolver = templateResolver; + } + + load(): MigrationFile[] { + const dir = resolveDirectoryPath(this.migrationDirectory); + const rawFiles: MigrationFile[] = []; + for (const name of readdirSync(dir)) { + const filePath = join(dir, name); + if (!statSync(filePath).isFile()) { + continue; + } + const match = FILE_NAME_PATTERN.exec(name); + if (match === null) { + continue; + } + rawFiles.push(readRawFile(filePath, name, match)); + } + rawFiles.sort(compareMigrationFiles); + this.templateResolver.validate(rawFiles); + return rawFiles.map((file) => this.resolveFile(file)); + } + + private resolveFile(file: MigrationFile): MigrationFile { + const resolvedContents = this.templateResolver.resolveContents(file.contents); + return { + metadata: { ...file.metadata, checksum: calculateChecksum(resolvedContents) }, + contents: resolvedContents, + }; + } +} + +function resolveDirectoryPath(directory: string): string { + if (directory.startsWith("file:")) { + return directory.slice("file:".length); + } + throw new Error(`unsupported migration directory scheme in '${directory}'. supported schemes: 'file:'`); +} + +function readRawFile(filePath: string, filename: string, match: RegExpExecArray): MigrationFile { + const version = match[1]; + const description = match[3]; + if (version === undefined || description === undefined) { + throw new Error(`invalid migration filename: ${filename}`); + } + const data = parse(readFileSync(filePath, "utf8")) as { requests: Record[] }; + return { + metadata: { filename, version, description, checksum: 0 }, + contents: { requests: data.requests.map((raw) => parseRequest(raw)) }, + }; +} + +function parseRequest(raw: Record): MigrationFileRequestDefinition { + return { + method: String(raw.method), + path: String(raw.path), + contentType: "contentType" in raw ? String(raw.contentType) : null, + params: + "params" in raw + ? Object.fromEntries( + Object.entries(raw.params as Record).map(([k, v]) => [String(k), String(v)]), + ) + : null, + body: "body" in raw ? String(raw.body) : null, + }; +} +``` + +- [ ] **Step 2: Run all tests to verify nothing broke** + +Run: `cd implementations/typescript && npm test` +Expected: PASS (all model, template, checksum tests) + +- [ ] **Step 3: Lint, format, typecheck** + +Run: `cd implementations/typescript && npm run format && npx biome ci src tests && npm run typecheck` +Expected: all clean + +- [ ] **Step 4: Commit** + +```bash +git add implementations/typescript +git commit -m "TypeScript implementation: migration file loader" +``` + +--- + +## Task 6: ES document types and index definition + +**Files:** +- Create: `implementations/typescript/src/elasticsearch/documents.ts` + +Data-only module; covered indirectly by Task 9 unit tests and the compat harness. + +- [ ] **Step 1: Write `implementations/typescript/src/elasticsearch/documents.ts`** + +Mirrors `implementations/python/src/esque/elasticsearch/documents.py` exactly — same index settings/mappings, same wrapper-object shapes, camelCase field names, `installedBy` omitted when null. + +```typescript +export const MIGRATION_INDEX = ".esque"; +export const LOCK_ID_PREFIX = "lock"; + +export const INDEX_DEFINITION = { + settings: { + index: { + number_of_shards: "1", + auto_expand_replicas: "0-all", + refresh_interval: "1s", + }, + }, + mappings: { + properties: { + lock: { properties: { date: { type: "date" } } }, + migration: { + properties: { + checksum: { type: "long" }, + description: { type: "keyword" }, + executionTime: { type: "long" }, + filename: { type: "keyword" }, + installedOn: { type: "date" }, + migrationKey: { type: "keyword" }, + order: { type: "long" }, + version: { type: "keyword" }, + }, + }, + }, + }, +} as const; + +export interface MigrationRecord { + readonly migrationKey: string; + readonly order: number; + readonly filename: string; + readonly version: string; + readonly description: string; + readonly checksum: number; + readonly installedBy: string | null; + readonly installedOn: string; // ISO-8601 UTC + readonly executionTime: number; +} + +// Wrapper-object serialization — mirrors JVM @JsonTypeInfo(As.WRAPPER_OBJECT). +export function migrationRecordToDocument(record: MigrationRecord): { migration: Record } { + const doc: Record = { + migrationKey: record.migrationKey, + order: record.order, + filename: record.filename, + version: record.version, + description: record.description, + checksum: record.checksum, + installedOn: record.installedOn, + executionTime: record.executionTime, + }; + if (record.installedBy !== null) { + doc.installedBy = record.installedBy; + } + return { migration: doc }; +} + +export function migrationRecordFromDocument(source: Record): MigrationRecord { + const raw = source.migration as Record; + return { + migrationKey: String(raw.migrationKey), + order: Number(raw.order), + filename: String(raw.filename), + version: String(raw.version), + description: String(raw.description), + checksum: Number(raw.checksum), + installedBy: raw.installedBy !== undefined && raw.installedBy !== null ? String(raw.installedBy) : null, + installedOn: String(raw.installedOn), + executionTime: Number(raw.executionTime), + }; +} + +export function migrationLockToDocument(date: Date): { lock: { date: string } } { + return { lock: { date: date.toISOString() } }; +} +``` + +- [ ] **Step 2: Typecheck and lint** + +Run: `cd implementations/typescript && npm run format && npx biome ci src tests && npm run typecheck` +Expected: all clean + +- [ ] **Step 3: Commit** + +```bash +git add implementations/typescript +git commit -m "TypeScript implementation: ES document types and index definition" +``` + +--- + +## Task 7: REST client operations + +**Files:** +- Create: `implementations/typescript/src/elasticsearch/operations.ts` + +Covered by the compat harness (mirrors Python, which has no unit tests for this module). + +- [ ] **Step 1: Write `implementations/typescript/src/elasticsearch/operations.ts`** + +```typescript +import type { Client } from "@elastic/elasticsearch"; +import { errors } from "@elastic/elasticsearch"; +import type { MigrationFile, MigrationFileRequestDefinition } from "../migration/model.js"; +import type { MigrationRecord } from "./documents.js"; +import { + INDEX_DEFINITION, + LOCK_ID_PREFIX, + MIGRATION_INDEX, + migrationLockToDocument, + migrationRecordFromDocument, + migrationRecordToDocument, +} from "./documents.js"; + +export class RestClientOperations { + private readonly client: Client; + private readonly migrationKey: string; + + constructor(client: Client, migrationKey: string) { + this.client = client; + this.migrationKey = migrationKey; + } + + async close(): Promise { + await this.client.close(); + } + + async checkMigrationIndexExists(): Promise { + return await this.client.indices.exists({ index: MIGRATION_INDEX }); + } + + async createMigrationIndex(): Promise { + try { + await this.client.indices.create({ + index: MIGRATION_INDEX, + settings: INDEX_DEFINITION.settings, + mappings: INDEX_DEFINITION.mappings, + }); + } catch (error) { + if (isAlreadyExistsError(error)) { + return; // another process created it first — safe + } + throw error; + } + } + + async createLockRecord(): Promise { + await this.client.index({ + index: MIGRATION_INDEX, + id: `${LOCK_ID_PREFIX}:${this.migrationKey}`, + document: migrationLockToDocument(new Date()), + op_type: "create", + }); + } + + async deleteLockRecord(): Promise { + await this.client.delete({ + index: MIGRATION_INDEX, + id: `${LOCK_ID_PREFIX}:${this.migrationKey}`, + }); + } + + async getMigrationRecords(): Promise { + const response = await this.client.search>({ + index: MIGRATION_INDEX, + query: { bool: { filter: [{ term: { "migration.migrationKey": this.migrationKey } }] } }, + size: 10000, + }); + const records = response.hits.hits.map((hit) => migrationRecordFromDocument(hit._source as Record)); + records.sort((a, b) => a.order - b.order); + return records; + } + + async getMigrationRecordForMigrationFile(file: MigrationFile): Promise { + const response = await this.client.search>({ + index: MIGRATION_INDEX, + query: { + bool: { + filter: [ + { term: { "migration.migrationKey": this.migrationKey } }, + { term: { "migration.filename": file.metadata.filename } }, + ], + }, + }, + }); + const hits = response.hits.hits; + if (hits.length > 1) { + throw new Error( + `found more than one migration record for file [${file.metadata.filename}] and migration key [${this.migrationKey}]`, + ); + } + const first = hits[0]; + if (first !== undefined) { + return migrationRecordFromDocument(first._source as Record); + } + return null; + } + + async executeMigrationDefinition(definition: MigrationFileRequestDefinition): Promise { + const headers: Record = {}; + if (definition.contentType !== null) { + headers["content-type"] = definition.contentType; + } + await this.client.transport.request( + { + method: definition.method, + path: definition.path, + querystring: definition.params ?? undefined, + body: definition.body ?? undefined, + }, + { headers }, + ); + } + + async createMigrationRecord(record: MigrationRecord): Promise { + if (record.migrationKey !== this.migrationKey) { + throw new Error("migration record migration key must match operational migration key"); + } + await this.client.index({ + index: MIGRATION_INDEX, + document: migrationRecordToDocument(record), + refresh: true, // without it, reads immediately after won't see the record + }); + } +} + +function isAlreadyExistsError(error: unknown): boolean { + if (!(error instanceof errors.ResponseError)) { + return false; + } + const body = error.body as { error?: { type?: string } } | undefined; + return body?.error?.type === "resource_already_exists_exception"; +} +``` + +(If the client's `transport.request` signature differs in the installed v9 minor — e.g. `querystring` typing — adapt the call but keep the behavior: params go in the URL query string, body sent raw with the given content-type header, method never templated.) + +- [ ] **Step 2: Typecheck and lint** + +Run: `cd implementations/typescript && npm run format && npx biome ci src tests && npm run typecheck` +Expected: all clean + +- [ ] **Step 3: Commit** + +```bash +git add implementations/typescript +git commit -m "TypeScript implementation: REST client operations" +``` + +--- + +## Task 8: Distributed lock + +**Files:** +- Create: `implementations/typescript/src/elasticsearch/lock.ts` + +- [ ] **Step 1: Write `implementations/typescript/src/elasticsearch/lock.ts`** + +```typescript +import { setTimeout as sleep } from "node:timers/promises"; +import type { RestClientOperations } from "./operations.js"; + +const IDLE_BETWEEN_TRIES_MS = 100; + +// Thrown by unlock() when the lock is not held — expected during Esque.close() after a clean run. +export class LockNotHeldError extends Error {} + +// Distributed lock via ES op_type=create. The JVM wraps this in a local ReentrantLock for +// thread safety; Node is single-threaded, so a held-flag suffices — it exists to make +// unlock() without tryLock() a detectable error, and to avoid deleting another process's +// lock document from a process that never acquired it. +export class ElasticsearchDocumentLock { + private readonly operations: RestClientOperations; + private held = false; + + constructor(operations: RestClientOperations) { + this.operations = operations; + } + + async tryLock(timeoutMinutes: number): Promise { + const deadline = Date.now() + timeoutMinutes * 60_000; + for (;;) { + if (await this.doLock()) { + this.held = true; + return true; + } + if (Date.now() >= deadline) { + return false; + } + await sleep(IDLE_BETWEEN_TRIES_MS); + } + } + + async unlock(): Promise { + if (!this.held) { + throw new LockNotHeldError("cannot release un-acquired lock"); + } + this.held = false; + try { + await this.operations.deleteLockRecord(); + } catch (error) { + throw new Error("Failed to release mutex", { cause: error }); + } + } + + private async doLock(): Promise { + try { + await this.operations.createLockRecord(); + return true; + } catch { + // TODO: differentiate ConflictError (lock exists) from other failures + return false; + } + } +} +``` + +- [ ] **Step 2: Typecheck and lint** + +Run: `cd implementations/typescript && npm run format && npx biome ci src tests && npm run typecheck` +Expected: all clean + +- [ ] **Step 3: Commit** + +```bash +git add implementations/typescript +git commit -m "TypeScript implementation: distributed document lock" +``` + +--- + +## Task 9: Esque orchestrator (TDD on integrity checks) + +**Files:** +- Create: `implementations/typescript/src/esque.ts` +- Test: `implementations/typescript/tests/integrity.test.ts` + +- [ ] **Step 1: Write the failing test** — port of `implementations/python/tests/test_integrity.py` + +`implementations/typescript/tests/integrity.test.ts`: + +```typescript +import assert from "node:assert/strict"; +import { test } from "node:test"; +import type { Client } from "@elastic/elasticsearch"; +import { createEsqueConfiguration } from "../src/configuration.js"; +import type { MigrationRecord } from "../src/elasticsearch/documents.js"; +import { Esque } from "../src/esque.js"; +import type { MigrationFile } from "../src/migration/model.js"; + +const SENTINEL_DATE = "2026-01-01T00:00:00.000Z"; + +// Constructor only stores references — no ES calls happen at init time — so an empty +// object stands in for the client. +function esque(migrationKey = "test-key"): Esque { + return new Esque({} as unknown as Client, createEsqueConfiguration({ migrationKey })); +} + +function verify(instance: Esque, files: MigrationFile[], history: MigrationRecord[]): void { + // TS `private` is compile-time only; element access is the sanctioned escape hatch for tests. + instance["verifyStateIntegrity"](files, history); +} + +function file(version: string, description = "Test", checksum = 42): MigrationFile { + return { + metadata: { filename: `V${version}__${description}.yml`, version, description, checksum }, + contents: { requests: [] }, + }; +} + +function record(f: MigrationFile, order: number, overrides: Partial = {}): MigrationRecord { + return { + migrationKey: "test-key", + order, + filename: f.metadata.filename, + version: f.metadata.version, + description: f.metadata.description, + checksum: f.metadata.checksum, + installedBy: null, + installedOn: SENTINEL_DATE, + executionTime: 0, + ...overrides, + }; +} + +test("passes with no history", () => { + verify(esque(), [file("1.0.0"), file("1.1.0")], []); +}); + +test("passes with complete matching history", () => { + const f1 = file("1.0.0"); + const f2 = file("1.1.0"); + verify(esque(), [f1, f2], [record(f1, 0), record(f2, 1)]); +}); + +test("passes with partial history", () => { + const f1 = file("1.0.0"); + const f2 = file("1.1.0"); + verify(esque(), [f1, f2], [record(f1, 0)]); +}); + +test("throws when more records than files", () => { + const f = file("1.0.0"); + assert.throws(() => verify(esque(), [], [record(f, 0)]), /more migrations/); +}); + +test("throws on gap in history", () => { + const f1 = file("1.0.0"); + const f2 = file("1.1.0"); + const f3 = file("1.2.0"); + assert.throws(() => verify(esque(), [f1, f2, f3], [record(f1, 0), record(f3, 2)]), /corrupt/); +}); + +test("throws on checksum mismatch", () => { + const f = file("1.0.0", "Test", 42); + assert.throws(() => verify(esque(), [f], [record(f, 0, { checksum: 999 })]), /integrity/); +}); + +test("throws on version mismatch", () => { + const f = file("1.0.0"); + assert.throws(() => verify(esque(), [f], [record(f, 0, { version: "9.9.9" })]), /integrity/); +}); + +test("throws on description mismatch", () => { + const f = file("1.0.0", "Original"); + assert.throws(() => verify(esque(), [f], [record(f, 0, { description: "Modified" })]), /integrity/); +}); + +test("throws on migration key mismatch", () => { + const f = file("1.0.0"); + assert.throws(() => verify(esque(), [f], [record(f, 0, { migrationKey: "other-key" })]), /integrity/); +}); + +test("throws when file missing for record", () => { + const f = file("1.0.0"); + const orphan = record(file("1.0.0", "Ghost"), 0); + assert.throws(() => verify(esque(), [f], [orphan]), /could not find/); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd implementations/typescript && npm test` +Expected: FAIL — cannot find module `../src/esque.js` + +- [ ] **Step 3: Write `implementations/typescript/src/esque.ts`** + +```typescript +import type { Client } from "@elastic/elasticsearch"; +import type { EsqueConfiguration } from "./configuration.js"; +import type { MigrationRecord } from "./elasticsearch/documents.js"; +import { ElasticsearchDocumentLock, LockNotHeldError } from "./elasticsearch/lock.js"; +import { RestClientOperations } from "./elasticsearch/operations.js"; +import { MigrationFileLoader } from "./migration/loader.js"; +import type { MigrationFile } from "./migration/model.js"; +import { MigrationTemplateResolver } from "./migration/template.js"; + +export class Esque { + private readonly configuration: EsqueConfiguration; + private readonly migrationLoader: MigrationFileLoader; + private readonly operations: RestClientOperations; + private readonly lock: ElasticsearchDocumentLock; + + constructor(client: Client, configuration: EsqueConfiguration, properties: Record = {}) { + this.configuration = configuration; + this.migrationLoader = new MigrationFileLoader( + configuration.migrationDirectory, + new MigrationTemplateResolver(properties), + ); + this.operations = new RestClientOperations(client, configuration.migrationKey); + this.lock = new ElasticsearchDocumentLock(this.operations); + } + + async close(): Promise { + try { + await this.lock.unlock(); + } catch (error) { + if (!(error instanceof LockNotHeldError)) { + console.warn("failed to release a execution lock. you may need to manually delete the lock document yourself"); + } + } + try { + await this.operations.close(); + } catch { + console.warn("failed to close client. this is likely not an issue"); + } + } + + async [Symbol.asyncDispose](): Promise { + await this.close(); + } + + async execute(): Promise { + try { + await this.initialize(); + const files = this.migrationLoader.load(); + const history = await this.operations.getMigrationRecords(); + this.verifyStateIntegrity(files, history); + await this.runMigrations(files); + } catch (error) { + throw new Error("Failed to run esque execution", { cause: error }); + } + } + + private async initialize(): Promise { + if (!(await this.operations.checkMigrationIndexExists())) { + await this.operations.createMigrationIndex(); + } + } + + private verifyStateIntegrity(files: MigrationFile[], history: MigrationRecord[]): void { + if (history.length > files.length) { + throw new Error( + "the migration records are showing more migrations than the local system defines. " + + "did you refactor your files or use an incorrect migration key?", + ); + } + const last = history[history.length - 1]; + if (last !== undefined && history.length !== last.order + 1) { + throw new Error("the migration records seem to be corrupt as some records appear to be missing."); + } + for (const record of history) { + this.verifyRecordIntegrity(record, files); + } + } + + private verifyRecordIntegrity(record: MigrationRecord, files: MigrationFile[]): void { + const companion = files.find((f) => f.metadata.filename === record.filename); + if (companion === undefined) { + throw new Error( + `could not find migration file matching migration history record by filename [${record.filename}]`, + ); + } + if ( + record.order !== files.indexOf(companion) || + record.version !== companion.metadata.version || + record.description !== companion.metadata.description || + record.checksum !== companion.metadata.checksum || + record.migrationKey !== this.configuration.migrationKey + ) { + throw new Error( + `could not verify integrity of migration history record for filename [${record.filename}]. ` + + "did you refactor your migration scripts after a previous execution?", + ); + } + } + + private async runMigrations(files: MigrationFile[]): Promise { + try { + for (const file of files) { + try { + if (await this.lock.tryLock(this.configuration.lockTimeoutMinutes)) { + const existing = await this.operations.getMigrationRecordForMigrationFile(file); + if (existing === null) { + const start = performance.now(); + await this.runMigrationForFile(file); + const elapsedMs = Math.round(performance.now() - start); + await this.operations.createMigrationRecord({ + migrationKey: this.configuration.migrationKey, + order: files.indexOf(file), + filename: file.metadata.filename, + version: file.metadata.version, + description: file.metadata.description, + checksum: file.metadata.checksum, + installedBy: this.configuration.migrationUser, + installedOn: new Date().toISOString(), + executionTime: elapsedMs, + }); + } + } else { + throw new Error("failed to acquire lock"); + } + } catch (error) { + throw new Error(`Failed to execute queries in migration file [${file.metadata.filename}]`, { + cause: error, + }); + } finally { + try { + await this.lock.unlock(); + } catch (error) { + if (!(error instanceof LockNotHeldError)) { + throw error; + } + } + } + } + } catch (error) { + throw new Error("failed to run migrations", { cause: error }); + } + } + + private async runMigrationForFile(file: MigrationFile): Promise { + for (const [position, definition] of file.contents.requests.entries()) { + try { + await this.operations.executeMigrationDefinition(definition); + } catch (error) { + throw new Error( + `Failed to execute query in position [${position}] in migration file [${file.metadata.filename}]`, + { cause: error }, + ); + } + } + } +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd implementations/typescript && npm test` +Expected: PASS (all suites; integrity adds 10 tests) + +- [ ] **Step 5: Lint, format, typecheck** + +Run: `cd implementations/typescript && npm run format && npx biome ci src tests && npm run typecheck` +Expected: all clean. If Biome's `noConsole`-family rules flag the `console.warn` calls in `close()`, suppress per-line with `// biome-ignore lint/suspicious/noConsole: intentional operator-facing warning` (adjust rule name to what Biome reports). + +- [ ] **Step 6: Commit** + +```bash +git add implementations/typescript +git commit -m "TypeScript implementation: Esque orchestrator with integrity verification" +``` + +--- + +## Task 10: CLI + +**Files:** +- Create: `implementations/typescript/src/cli.ts` + +- [ ] **Step 1: Write `implementations/typescript/src/cli.ts`** + +```typescript +#!/usr/bin/env node +import { Client } from "@elastic/elasticsearch"; +import { Command, InvalidArgumentError } from "commander"; +import { createEsqueConfiguration } from "./configuration.js"; +import { Esque } from "./esque.js"; + +function collectProperty(value: string, previous: Record): Record { + const separator = value.indexOf("="); + if (separator <= 0) { + throw new InvalidArgumentError(`must be in key=value format, got: '${value}'`); + } + previous[value.slice(0, separator)] = value.slice(separator + 1); + return previous; +} + +const program = new Command() + .name("esque") + .description("Run Elasticsearch migrations.") + .requiredOption("--es-url ", "Elasticsearch URL (e.g. http://localhost:9200)") + .requiredOption("--migrations-dir ", "Path to directory containing migration YAML files") + .requiredOption("--migration-key ", "Unique key scoping this migration set") + .option("--migration-user ", "User to record on each migration record") + .option( + "--lock-timeout-minutes ", + "Lock acquisition timeout in minutes", + (value: string) => Number.parseInt(value, 10), + 5, + ) + .option("--property ", "Template substitution property as key=value (repeatable)", collectProperty, {}); + +program.parse(); + +const opts = program.opts<{ + esUrl: string; + migrationsDir: string; + migrationKey: string; + migrationUser?: string; + lockTimeoutMinutes: number; + property: Record; +}>(); + +const configuration = createEsqueConfiguration({ + migrationKey: opts.migrationKey, + migrationUser: opts.migrationUser ?? null, + migrationDirectory: `file:${opts.migrationsDir}`, + lockTimeoutMinutes: opts.lockTimeoutMinutes, +}); + +const esque = new Esque(new Client({ node: opts.esUrl }), configuration, opts.property); +try { + await esque.execute(); +} catch (error) { + console.error(`Error: ${error instanceof Error ? error.message : String(error)}`); + process.exitCode = 1; +} finally { + await esque.close(); +} +``` + +- [ ] **Step 2: Smoke-test help output** + +Run: `cd implementations/typescript && npx tsx src/cli.ts --help` +Expected: usage text listing `--es-url`, `--migrations-dir`, `--migration-key`, `--migration-user`, `--lock-timeout-minutes`, `--property`; exit 0. + +- [ ] **Step 3: Smoke-test error path (no ES running)** + +Run: `cd implementations/typescript && npx tsx src/cli.ts --es-url=http://localhost:1 --migrations-dir=../../tests/fixtures/single --migration-key=smoke; echo "exit=$?"` +Expected: a line starting with `Error:` on stderr and `exit=1`. + +- [ ] **Step 4: Lint, format, typecheck** + +Run: `cd implementations/typescript && npm run format && npx biome ci src tests && npm run typecheck` +Expected: all clean (same `noConsole` caveat as Task 9 for `console.error`). + +- [ ] **Step 5: Verify the build produces a runnable CLI** + +Run: `cd implementations/typescript && npm run build && node dist/cli.js --help` +Expected: same usage text; exit 0. + +- [ ] **Step 6: Commit** + +```bash +git add implementations/typescript +git commit -m "TypeScript implementation: CLI entrypoint" +``` + +--- + +## Task 11: Register with the compatibility harness + +**Files:** +- Modify: `tests/implementations.yml` + +- [ ] **Step 1: Add the typescript entry to `tests/implementations.yml`** + +The file becomes: + +```yaml +implementations: + jvm: + invocation: gradle + gradle_dir: "implementations/jvm" + task: "run" + python: + invocation: direct + command: ["uv", "run", "--project", "implementations/python", "esque"] + typescript: + invocation: direct + command: ["npm", "exec", "--prefix", "implementations/typescript", "--", "tsx", "implementations/typescript/src/cli.ts"] +``` + +(`direct` commands run with `cwd = ROOT_DIR` — see `tests/helpers.py` `run()` — so both the `--prefix` and the `src/cli.ts` path are repo-root-relative. `npm exec --prefix` resolves the locally installed `tsx` from the implementation's `node_modules`.) + +- [ ] **Step 2: Run the compat suite against typescript only** + +Run: `cd tests && uv run pytest . -v -k "typescript"` +Expected: 17 passed (Docker must be running). Failures here are real behavior differences — debug the TypeScript implementation against the Python reference, don't touch the harness. + +- [ ] **Step 3: Run the cross-implementation equivalency test (now includes typescript)** + +Run: `cd tests && uv run pytest . -v -k "cross_implementation"` +Expected: 1 passed — proves record fields INCLUDING CHECKSUMS match across jvm, python, and typescript. + +- [ ] **Step 4: Run the full suite** + +Run: `cd tests && uv run pytest . -q` +Expected: 52 passed (17 × 3 + 1) + +- [ ] **Step 5: Commit** + +```bash +git add tests/implementations.yml +git commit -m "Register TypeScript implementation with compatibility harness" +``` + +--- + +## Task 12: Version script + +**Files:** +- Create: `.github/version_typescript.sh` (mode 755) + +- [ ] **Step 1: Write `.github/version_typescript.sh`** + +```bash +#!/bin/bash +# Produces a SemVer version from git tags — mirrors version_python.sh but for npm. +# Exact tag X.Y.Z → X.Y.Z (release); otherwise → X.Y.Z-dev.N (prerelease). + +GIT_DESCRIBE=$(git describe --tags 2>/dev/null) + +if [[ $GIT_DESCRIBE =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then + echo "$GIT_DESCRIBE" +elif [[ $GIT_DESCRIBE =~ ^([0-9]+\.[0-9]+\.[0-9]+)-([0-9]+)-g[0-9a-f]+ ]]; then + echo "${BASH_REMATCH[1]}-dev.${BASH_REMATCH[2]}" +else + echo "0.0.0-dev.$(git rev-list --count HEAD 2>/dev/null || echo 0)" +fi +``` + +- [ ] **Step 2: Make it executable and verify output is valid SemVer** + +Run: `chmod +x .github/version_typescript.sh && .github/version_typescript.sh` +Expected: something like `X.Y.Z-dev.N` (or `0.0.0-dev.N` if no tags reachable). Verify npm accepts it: + +Run: `cd implementations/typescript && npm version --no-git-tag-version "$(../../.github/version_typescript.sh)" && git checkout -- package.json package-lock.json` +Expected: npm prints a `v...` version and exits 0; the checkout restores the placeholder `0.0.0`. + +- [ ] **Step 3: Commit** + +```bash +git add .github/version_typescript.sh +git commit -m "Add git-tag-based version script for TypeScript" +``` + +--- + +## Task 13: CI workflow + +**Files:** +- Modify: `.github/workflows/ci.yml` + +- [ ] **Step 1: Add the `typescript` job** (after the `python` job, following its pattern) + +```yaml + typescript: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + with: + fetch-depth: 0 + - uses: actions/setup-node@v5 + with: + node-version: 24 + registry-url: https://registry.npmjs.org + - working-directory: implementations/typescript + run: npm ci + - working-directory: implementations/typescript + run: npx biome ci src tests + - working-directory: implementations/typescript + run: npm run typecheck + - working-directory: implementations/typescript + run: npm test + - working-directory: implementations/typescript + run: npm version --no-git-tag-version "$(../../.github/version_typescript.sh)" + - working-directory: implementations/typescript + run: npm run build + - if: github.event_name != 'release' + working-directory: implementations/typescript + run: npm publish --tag dev + env: + NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }} + - if: github.event_name == 'release' + working-directory: implementations/typescript + run: npm publish + env: + NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }} +``` + +(Check the current major of `actions/setup-node` on the marketplace and use it — the repo just bumped checkout to v7 and setup-java to v5, so use whatever is latest, not blindly `v5`.) + +- [ ] **Step 2: Update the `compatibility-tests` job** + +Change `needs: [jvm, python]` to `needs: [jvm, python, typescript]` and add Node setup + install after the uv setup steps: + +```yaml + - uses: actions/setup-node@v5 + with: + node-version: 24 + - working-directory: implementations/typescript + run: npm ci +``` + +- [ ] **Step 3: Validate workflow syntax** + +Run: `uvx --from yamllint yamllint -d relaxed .github/workflows/ci.yml || python3 -c "import yaml,sys; yaml.safe_load(open('.github/workflows/ci.yml')); print('OK')"` +Expected: no syntax errors / `OK`. + +- [ ] **Step 4: Commit** + +```bash +git add .github/workflows/ci.yml +git commit -m "Add TypeScript job to CI" +``` + +**Post-merge manual step (for the user, note it in the final report):** create the `NPM_TOKEN` repository secret (an npm automation token with publish rights for `esque-ts`) before the first CI publish run. + +--- + +## Task 14: Pre-commit hook and dev container + +**Files:** +- Modify: `.githooks/pre-commit` +- Modify: `.devcontainer/Dockerfile` + +- [ ] **Step 1: Update `.githooks/pre-commit`** — insert TypeScript checks as step 3, renumber compat to 4 + +Replace the body after the `echo "Running pre-commit checks..."` line so the checks read: + +```sh +echo "[1/4] JVM: format, lint, test" +(cd implementations/jvm && ./gradlew ktfmtCheck detekt test) + +echo "[2/4] Python: format, lint, typecheck" +(cd implementations/python && uv run ruff check src/esque/ && uv run ruff format --check src/esque/ && uv run pyright src/esque/) + +echo "[3/4] TypeScript: format, lint, typecheck, test" +(cd implementations/typescript && npx biome ci src tests && npm run typecheck && npm test) + +echo "[4/4] Compatibility tests" +(cd tests && uv run pytest . -q) + +echo "Pre-commit checks passed." +``` + +- [ ] **Step 2: Add Node to `.devcontainer/Dockerfile`** — after the "install python things" block (still as USER ubuntu), add: + +```dockerfile +# install node things +ARG NODE_VERSION=24 +RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh | bash \ + && source ${HOME}/.nvm/nvm.sh \ + && nvm install ${NODE_VERSION} \ + && nvm alias default ${NODE_VERSION} +``` + +- [ ] **Step 3: Verify the hook passes end-to-end** + +Run: `.githooks/pre-commit` +Expected: all four sections pass (compat tests take ~3 minutes). + +- [ ] **Step 4: Commit** + +```bash +git add .githooks/pre-commit .devcontainer/Dockerfile +git commit -m "Add TypeScript checks to pre-commit hook and Node to dev container" +``` + +--- + +## Task 15: Documentation (CLAUDE.md) + +**Files:** +- Modify: `CLAUDE.md` + +- [ ] **Step 1: Apply these updates to `CLAUDE.md`:** + +1. **Project Overview** — implementations line becomes: + `- **Implementations:** JVM (Kotlin 2.4.0, Java 21) · Python 3.14 · TypeScript (Node 22+)` and add + `- **TypeScript published to:** npm as \`esque-ts\`` +2. **Repository Structure** — add a `typescript/` block under `implementations/` mirroring the python block (package.json, biome.json, tsconfig, `src/` module list, `tests/` list); add `version_typescript.sh` next to the other version scripts. +3. **Build and Development → Prerequisites** — add `Node.js 22+ (24 recommended) and npm`. +4. **Add a "TypeScript Commands (run from `implementations/typescript/`)" section:** + +```bash +cd implementations/typescript + +# install dependencies +npm install + +# format code +npm run format + +# check formatting and lint +npx biome ci src tests + +# typecheck +npm run typecheck + +# run unit tests +npm test + +# build (emits dist/) +npm run build + +# run the CLI +npx tsx src/cli.ts --help +``` + +5. **CI/CD section** — add the `typescript` job description: Biome + tsc + node:test + build + publish; version from `.github/version_typescript.sh` (SemVer: `X.Y.Z` on exact tag, `X.Y.Z-dev.N` otherwise) patched into `package.json`; non-release builds publish to npm under the `dev` dist-tag, releases to `latest`; secret `NPM_TOKEN`. Update compat-tests line: `needs jvm + python + typescript; runs 52 pytest scenarios via testcontainers.` +6. **Add a "TypeScript Code Conventions" section** (mirroring the Python one): `src/` layout matching the module structure, ESM-only strict TypeScript, Biome for lint/format (line width 120), official `@elastic/elasticsearch` client, `commander` CLI with the same option names, `yaml` for parsing, unit tests via `node:test` run through `tsx`. +7. **Python Code Conventions — fix stale httpx claim**: replace the `**httpx**: ES REST calls (not elasticsearch-py, ...)` bullet with `**elasticsearch**: official Python ES client (>=9) — handles auth mechanisms, retries, and typed responses`. +8. **Registered Implementations** — update the yaml snippet to include the typescript entry (same content as Task 11). +9. **Testing → Compatibility Test Harness** — mention TypeScript runs via `tsx` with no build step; update `17 scenarios` phrasing to `17 parametrized scenarios + 1 cross-implementation equivalency test`. +10. **Pre-commit hook description** at top of Repository Structure: `[1] JVM checks [2] Python checks [3] TypeScript checks [4] compat tests`. + +- [ ] **Step 2: Commit** + +```bash +git add CLAUDE.md +git commit -m "Document TypeScript implementation in CLAUDE.md" +``` + +--- + +## Task 16: Final verification + +**Files:** none + +- [ ] **Step 1: Full unit test + lint sweep for typescript** + +Run: `cd implementations/typescript && npx biome ci src tests && npm run typecheck && npm test && npm run build` +Expected: all pass. + +- [ ] **Step 2: Full compatibility suite** + +Run: `cd tests && uv run pytest . -v` +Expected: 52 passed. + +- [ ] **Step 3: Confirm clean tree and review the branch diff** + +Run: `git status --short && git log --oneline master..HEAD` +Expected: clean tree; commits for scaffolding, model, template, checksum, loader, documents, operations, lock, orchestrator, CLI, harness registration, version script, CI, hook/devcontainer, docs. + +- [ ] **Step 4: Report completion** — use superpowers:verification-before-completion, then superpowers:finishing-a-development-branch. Remind the user about the `NPM_TOKEN` secret. \ No newline at end of file From 2c20a487333c56d54601f51de624c9367279be8c Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 14:34:23 -0500 Subject: [PATCH 03/26] TypeScript implementation: project scaffolding Co-Authored-By: Claude Fable 5 --- implementations/typescript/.gitignore | 2 + implementations/typescript/biome.json | 23 + implementations/typescript/package-lock.json | 902 ++++++++++++++++++ implementations/typescript/package.json | 42 + .../typescript/src/configuration.ts | 1 + .../typescript/tsconfig.build.json | 11 + implementations/typescript/tsconfig.json | 13 + 7 files changed, 994 insertions(+) create mode 100644 implementations/typescript/.gitignore create mode 100644 implementations/typescript/biome.json create mode 100644 implementations/typescript/package-lock.json create mode 100644 implementations/typescript/package.json create mode 100644 implementations/typescript/src/configuration.ts create mode 100644 implementations/typescript/tsconfig.build.json create mode 100644 implementations/typescript/tsconfig.json diff --git a/implementations/typescript/.gitignore b/implementations/typescript/.gitignore new file mode 100644 index 0000000..b947077 --- /dev/null +++ b/implementations/typescript/.gitignore @@ -0,0 +1,2 @@ +node_modules/ +dist/ diff --git a/implementations/typescript/biome.json b/implementations/typescript/biome.json new file mode 100644 index 0000000..66b79d6 --- /dev/null +++ b/implementations/typescript/biome.json @@ -0,0 +1,23 @@ +{ + "$schema": "./node_modules/@biomejs/biome/configuration_schema.json", + "files": { + "includes": ["src/**", "tests/**"] + }, + "formatter": { + "enabled": true, + "indentStyle": "space", + "indentWidth": 2, + "lineWidth": 120 + }, + "linter": { + "enabled": true, + "rules": { + "recommended": true + } + }, + "javascript": { + "formatter": { + "quoteStyle": "double" + } + } +} diff --git a/implementations/typescript/package-lock.json b/implementations/typescript/package-lock.json new file mode 100644 index 0000000..3a634f3 --- /dev/null +++ b/implementations/typescript/package-lock.json @@ -0,0 +1,902 @@ +{ + "name": "esque-ts", + "version": "0.0.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "esque-ts", + "version": "0.0.0", + "license": "Apache-2.0", + "dependencies": { + "@elastic/elasticsearch": "^9.0.0", + "commander": "^14.0.0", + "yaml": "^2.7.0" + }, + "bin": { + "esque": "dist/cli.js" + }, + "devDependencies": { + "@biomejs/biome": "^2.0.0", + "@types/node": "^22.0.0", + "tsx": "^4.19.0", + "typescript": "^5.7.0" + }, + "engines": { + "node": ">=22" + } + }, + "node_modules/@biomejs/biome": { + "version": "2.5.2", + "resolved": "https://registry.npmjs.org/@biomejs/biome/-/biome-2.5.2.tgz", + "integrity": "sha512-VQ3RCqr7JmDIX+w6stWYl+g/3bYofN3q2wDBHUKKc/c7i5QWrFKFBZYCYPWTE6agsUPMIZZe6/CMmVUfUAhkKA==", + "dev": true, + "license": "MIT OR Apache-2.0", + "bin": { + "biome": "bin/biome" + }, + "engines": { + "node": ">=14.21.3" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/biome" + }, + "optionalDependencies": { + "@biomejs/cli-darwin-arm64": "2.5.2", + "@biomejs/cli-darwin-x64": "2.5.2", + "@biomejs/cli-linux-arm64": "2.5.2", + "@biomejs/cli-linux-arm64-musl": "2.5.2", + "@biomejs/cli-linux-x64": "2.5.2", + "@biomejs/cli-linux-x64-musl": "2.5.2", + "@biomejs/cli-win32-arm64": "2.5.2", + "@biomejs/cli-win32-x64": "2.5.2" + } + }, + "node_modules/@biomejs/cli-darwin-arm64": { + "version": "2.5.2", + "resolved": "https://registry.npmjs.org/@biomejs/cli-darwin-arm64/-/cli-darwin-arm64-2.5.2.tgz", + "integrity": "sha512-e7P3P7EkwFc/KiX2AHw4YDLIBOMfG9CPCAwy52k5Bp0dfhkozx9hf6wCmIr2QeXy2XeccJ3V/Sg+hDmzYEqxSg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-darwin-x64": { + "version": "2.5.2", + "resolved": "https://registry.npmjs.org/@biomejs/cli-darwin-x64/-/cli-darwin-x64-2.5.2.tgz", + "integrity": "sha512-ymzMvjC1Jg0b9K0D26ZdARqFQXs7MocfLC5FOCGfkC0Ss+ACUJkX5364ZM5nT4NLZanHRZNVrZEy+Ibwcvux/g==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-linux-arm64": { + "version": "2.5.2", + "resolved": "https://registry.npmjs.org/@biomejs/cli-linux-arm64/-/cli-linux-arm64-2.5.2.tgz", + "integrity": "sha512-t7sseOmqND57uUWTwlawU6BYj+J06T/9EkydzBhkrgw/FK3QVhjU2wsJR0frljrKZ0/I8A/rYw7284QgqjQfIQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-linux-arm64-musl": { + "version": "2.5.2", + "resolved": "https://registry.npmjs.org/@biomejs/cli-linux-arm64-musl/-/cli-linux-arm64-musl-2.5.2.tgz", + "integrity": "sha512-w+ANG0ZvTu9IeEg9QnstoOnk6L0fpwJifW6aHR18+cb5Z39bkANItYjAfMrnvce5tmMK+IQ6nPX7/kQFdam5iw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-linux-x64": { + "version": "2.5.2", + "resolved": "https://registry.npmjs.org/@biomejs/cli-linux-x64/-/cli-linux-x64-2.5.2.tgz", + "integrity": "sha512-M/lOZrewzTCRDINbjhQ1gYYru37KlD3kJBQwwKCG0ckz5E9IZwIoJ3X0wBwRXA+yBDIwWUuPBHS67HzJY4dTfA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-linux-x64-musl": { + "version": "2.5.2", + "resolved": "https://registry.npmjs.org/@biomejs/cli-linux-x64-musl/-/cli-linux-x64-musl-2.5.2.tgz", + "integrity": "sha512-VArNLAzND063tF+XY0yPyM+DyahpzOMzOAvb7qs259nhjJWRjvjZdssuA+Rfl+l07+NOesKZ0Xu2yFrXyBMtzw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-win32-arm64": { + "version": "2.5.2", + "resolved": "https://registry.npmjs.org/@biomejs/cli-win32-arm64/-/cli-win32-arm64-2.5.2.tgz", + "integrity": "sha512-kbjFFKyZlzYnAuw7sRy5qDoFG6zrP40UK08oPQsWK0ct3NMnGSt+Bs1iviEEyEIP57N5MrykGXdO/wRiaR4lww==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-win32-x64": { + "version": "2.5.2", + "resolved": "https://registry.npmjs.org/@biomejs/cli-win32-x64/-/cli-win32-x64-2.5.2.tgz", + "integrity": "sha512-4InchVpdVmdkkkgjQqKpgvyu+VPnoF/7RPSw5YATgEVpt2j72wcCAeV5TwaE9ZGJUZWZn7v2CwSAj6CrMJEx8A==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@elastic/elasticsearch": { + "version": "9.4.2", + "resolved": "https://registry.npmjs.org/@elastic/elasticsearch/-/elasticsearch-9.4.2.tgz", + "integrity": "sha512-H9myMlLUeotkZhZ4ppinoMGDFxmW3lY8/s+4TIk1vFHyCvWU1Ej4T7azX5buCzemyFApgN0ywnEuvOtpel2VZg==", + "license": "Apache-2.0", + "dependencies": { + "@elastic/transport": "^9.3.5", + "tslib": "^2.4.0" + }, + "engines": { + "node": ">=20" + }, + "peerDependencies": { + "apache-arrow": "18.x - 21.x" + }, + "peerDependenciesMeta": { + "apache-arrow": { + "optional": true + } + } + }, + "node_modules/@elastic/transport": { + "version": "9.3.7", + "resolved": "https://registry.npmjs.org/@elastic/transport/-/transport-9.3.7.tgz", + "integrity": "sha512-L38Ax21uF2OPUmCRWycZ/dZdMYf7gMrtClcxvVrqJVFmn8ET2M++GYmFGJpLqOHS1beATxOXLWe7y2ijSQz/ng==", + "license": "Apache-2.0", + "dependencies": { + "@opentelemetry/api": "1.x", + "@opentelemetry/core": "2.x", + "debug": "^4.4.1", + "hpagent": "^1.2.0", + "ms": "^2.1.3", + "secure-json-parse": "^4.0.0", + "tslib": "^2.8.1", + "undici": "^7.19.1" + }, + "engines": { + "node": ">=20" + } + }, + "node_modules/@esbuild/aix-ppc64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.1.tgz", + "integrity": "sha512-Svl7tq8k/08+p6CXPpRjQ1fKX+1odH/BQbb48fV6fj3CWHhsoIOoY87w1oHXm0qEpkIK3ZfVgp0hed3XBXzXMQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "aix" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.28.1.tgz", + "integrity": "sha512-0k2F129Xdio1TdJfzJ8sy1Q47vUD2NnwdhiAf7drUN1EBTfPf4hsFCtmMgu/6m8JSzsBrlmVjudMBQqOfG8usQ==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.28.1.tgz", + "integrity": "sha512-34EGEbCIAgosYz6goLcopX6Mo7NyGv9tfwEM2/7Ce2VcVRk568iSvniGWcUXIy7wEDR1wzolcxcriFVrWYcwBg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.28.1.tgz", + "integrity": "sha512-dbwY7ltSMDWsRatcRpCnES4F+im88OCUgGZjy52shC7GqHRE/cYlxNbB4Z4UpJswpcc4Qxd2oE/ufM0p61IKng==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.28.1.tgz", + "integrity": "sha512-TZbWkQY7kvTAXbXUT7uVACR5cMHsDiSz9z7ZKAX/RTq/WJEk3QyRr0wZpNhBDX+/0CtdqUIJlOiodQcta6tY3Q==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.28.1.tgz", + "integrity": "sha512-zfdzgK9ACBNZLI/CyHTOx81SyNbM6YXn7rxSgX97VjyiPl9W1i4Ka4fgKECEoFCKGpvBj5qArWIGgQjOwkgskQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.28.1.tgz", + "integrity": "sha512-wG2EA8ENdEI0qhkSZMjfqrdY+ziCYCPMmtZjjIwOmXFjmyzEHn+UUxk5of+SYsjtfs3VpnlC7QLzSI5hY/rOAw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.28.1.tgz", + "integrity": "sha512-i7dZ9vQgnvSCzi/rYCXNgtF/U+eKZNJBzu3eTQbRgHnM7tNSizLOkRFAl3qzVc/Op/u5YkHHa4pf/3DOYHthLQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.28.1.tgz", + "integrity": "sha512-qVXBOHQS+d5Y722GwJzJUtOLlX7km3CraOaGormF1pDtPd2C/l1SHRPgjLunLGe51Sh5YYWKMFDyV4SxgMQYTQ==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.28.1.tgz", + "integrity": "sha512-yHs+0uc8+nvEAfAfxrWQKK5peSNzBc4PegcMO0EJ2hT71uA7vB8Ihg2e77R2P7SG5uYjPbHlLLmve4LLLRCf0g==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ia32": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.28.1.tgz", + "integrity": "sha512-d1z4ZuP0ajrfz/FhGT4vv278rX8KnPPJx8i5+AtK7TYbx9Le9F1hyzurZpkEyjkGa9dUGhQow4C1NmeGvqxN2w==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-loong64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.28.1.tgz", + "integrity": "sha512-M5sRjUVZrkm1OAPR3dlOYzNmN+loZKGVi1VUQGrwuqLcbR6qeAz+famMhjASeH3YVKvZz+zT1jlh/keC3Rj/lg==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-mips64el": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.28.1.tgz", + "integrity": "sha512-mRObBZeHh2OxcBFPWE/FjylkRgZdYuiTR3vaTozquCGOH14iP9oN4x4Ge81CoIDYQrXmIxpFumJBu5MtZpnQJQ==", + "cpu": [ + "mips64el" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ppc64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.28.1.tgz", + "integrity": "sha512-slScBsMAb3GFDcdrCgLwZtPYRoH2H/youv10QiZyRjmsP48fznoveWytSgCI/R0ZcUgpc0ZhIUEx6LHts8yrfQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-riscv64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.28.1.tgz", + "integrity": "sha512-kw0owk1o0GFETUJyW0jc0G4Yzs0BHZn0JDZ8JRT088vjJYX777BAs1fDGxAC+q831qOs2DTC96mNsG2opdfyyQ==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-s390x": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.28.1.tgz", + "integrity": "sha512-/lAIjX8aYFRByhh6L5rYtPEDRqa9de/4V/juOXcta5frjvzXO4/sqEtyytse0g3zZFuWu5cDN0MkLz2qRDD2Ag==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.28.1.tgz", + "integrity": "sha512-u/anNYF2mmVOEDwLtnQ1wOr3EZ9sTNGLWrsYGYwHWzGA3Si84IOkHXlbWTD1NB+9/1lcnweYKO54uhxZydNzfA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.28.1.tgz", + "integrity": "sha512-oks0DYbLwWMmaakTsCb+zL4E+aHRVLom9IJZOAthMQEPiQmydXHkziYEsGYRx0uNV/IjEKGAV941JzH02pflqw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.28.1.tgz", + "integrity": "sha512-aeL6lAnN89Hz43Mlh1G8ARasbuoYvSITDEx0tHh5b7jJnHcssqgjy9Yx430GDpmCa6OyrKoS0aNRjKundRizGg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.28.1.tgz", + "integrity": "sha512-MEFJe5C3R8pwXdZ5Y21oo6m7ePiS0d9pWucn99O/wvyJZChoIQKrQDxKrGeW8F5+T0okTHesAmDeiHDTIq0V/Q==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.28.1.tgz", + "integrity": "sha512-i/ZLIOafE0Z8cI/XANJAixoJL/uRAoS2xOA3rb0xN+KK0K177cMAsQYkzHtBrtMXAKuAc7HGgcWiZ/sRC1Nxgw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openharmony-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.28.1.tgz", + "integrity": "sha512-ge+Z7EXFNt2BO1oAMsVpiQ8EwndV9i1xXerAeTIK7AtPs3bKFXQM7nlRxDSIUIMeueR1CNXxqztLzdNeReKBJg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/sunos-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.28.1.tgz", + "integrity": "sha512-BEjgtECkL3vY+SaSQ6nzVfiALUeFxpawyp8Jmf5PtYhf1Ug40N1h/hxlhts+f1FvSvarEigdxS3BlSMI2PJLcQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "sunos" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.28.1.tgz", + "integrity": "sha512-lCv9eK/H6ZJWbE7bh2nw54CZ9M2nupBxJcTsdk/QQnWkdSjKGuxmmH8/GWrlT1eMmZfn4dGcCjRte397WqfQXA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-ia32": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.28.1.tgz", + "integrity": "sha512-zvb/mB2bSCoJOpoCBgYKKpX6YM6mJBlBUVUtVj41DlZJVEB6/0CKlRYxP5wWl1C1ILiCoAU5wZZ4q1P3qeS6Eg==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.28.1.tgz", + "integrity": "sha512-bm4Mowrv+GXMlpWX++EcXw/iLyd1o3+bJkC2DkWXYVvgZCqD/bSj9ctZeAMC3cIxgjRVR2Dufaiu4YPxr5gW1A==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@opentelemetry/api": { + "version": "1.9.1", + "resolved": "https://registry.npmjs.org/@opentelemetry/api/-/api-1.9.1.tgz", + "integrity": "sha512-gLyJlPHPZYdAk1JENA9LeHejZe1Ti77/pTeFm/nMXmQH/HFZlcS/O2XJB+L8fkbrNSqhdtlvjBVjxwUYanNH5Q==", + "license": "Apache-2.0", + "engines": { + "node": ">=8.0.0" + } + }, + "node_modules/@opentelemetry/core": { + "version": "2.9.0", + "resolved": "https://registry.npmjs.org/@opentelemetry/core/-/core-2.9.0.tgz", + "integrity": "sha512-m2nckMT80NnmjTYSPjJQObBJ+8dgkoajEOUbznL8AHZ3T3yHRk2P7gI1PhEBc1+lOnrYE9UWrWHqJDsmqjmNbw==", + "license": "Apache-2.0", + "dependencies": { + "@opentelemetry/semantic-conventions": "^1.29.0" + }, + "engines": { + "node": "^18.19.0 || >=20.6.0" + }, + "peerDependencies": { + "@opentelemetry/api": ">=1.0.0 <1.10.0" + } + }, + "node_modules/@opentelemetry/semantic-conventions": { + "version": "1.41.1", + "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.41.1.tgz", + "integrity": "sha512-/UhIkaZgPutTFmQ7RnIJGgDXZmtEJ7Dvi86xNTFWcnRxVRNk/aotsqDJYeEvDP+FSMB2SdW+pQzNMcWP0rwuNA==", + "license": "Apache-2.0", + "engines": { + "node": ">=14" + } + }, + "node_modules/@types/node": { + "version": "22.20.0", + "resolved": "https://registry.npmjs.org/@types/node/-/node-22.20.0.tgz", + "integrity": "sha512-QWlFW2wf3nTjC13/DqRnBpR4ZO36VJH/JVBkA/vcnmbTBNQIlnObqyqZE1tUR7+Ni23Lda8R1BxMfbXRpCUx5g==", + "dev": true, + "license": "MIT", + "dependencies": { + "undici-types": "~6.21.0" + } + }, + "node_modules/commander": { + "version": "14.0.3", + "resolved": "https://registry.npmjs.org/commander/-/commander-14.0.3.tgz", + "integrity": "sha512-H+y0Jo/T1RZ9qPP4Eh1pkcQcLRglraJaSLoyOtHxu6AapkjWVCy2Sit1QQ4x3Dng8qDlSsZEet7g5Pq06MvTgw==", + "license": "MIT", + "engines": { + "node": ">=20" + } + }, + "node_modules/debug": { + "version": "4.4.3", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz", + "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==", + "license": "MIT", + "dependencies": { + "ms": "^2.1.3" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/esbuild": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.28.1.tgz", + "integrity": "sha512-HrJrvZv5ayxBzPfwphOoNzkzOIIlifzk0KJrGK2c8R4+LKpMtpYLQeUdjnwjWv/LZlkH2laZk+4w78pi99D4Vw==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "bin": { + "esbuild": "bin/esbuild" + }, + "engines": { + "node": ">=18" + }, + "optionalDependencies": { + "@esbuild/aix-ppc64": "0.28.1", + "@esbuild/android-arm": "0.28.1", + "@esbuild/android-arm64": "0.28.1", + "@esbuild/android-x64": "0.28.1", + "@esbuild/darwin-arm64": "0.28.1", + "@esbuild/darwin-x64": "0.28.1", + "@esbuild/freebsd-arm64": "0.28.1", + "@esbuild/freebsd-x64": "0.28.1", + "@esbuild/linux-arm": "0.28.1", + "@esbuild/linux-arm64": "0.28.1", + "@esbuild/linux-ia32": "0.28.1", + "@esbuild/linux-loong64": "0.28.1", + "@esbuild/linux-mips64el": "0.28.1", + "@esbuild/linux-ppc64": "0.28.1", + "@esbuild/linux-riscv64": "0.28.1", + "@esbuild/linux-s390x": "0.28.1", + "@esbuild/linux-x64": "0.28.1", + "@esbuild/netbsd-arm64": "0.28.1", + "@esbuild/netbsd-x64": "0.28.1", + "@esbuild/openbsd-arm64": "0.28.1", + "@esbuild/openbsd-x64": "0.28.1", + "@esbuild/openharmony-arm64": "0.28.1", + "@esbuild/sunos-x64": "0.28.1", + "@esbuild/win32-arm64": "0.28.1", + "@esbuild/win32-ia32": "0.28.1", + "@esbuild/win32-x64": "0.28.1" + } + }, + "node_modules/fsevents": { + "version": "2.3.3", + "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz", + "integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^8.16.0 || ^10.6.0 || >=11.0.0" + } + }, + "node_modules/hpagent": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/hpagent/-/hpagent-1.2.0.tgz", + "integrity": "sha512-A91dYTeIB6NoXG+PxTQpCCDDnfHsW9kc06Lvpu1TEe9gnd6ZFeiBoRO9JvzEv6xK7EX97/dUE8g/vBMTqTS3CA==", + "license": "MIT", + "engines": { + "node": ">=14" + } + }, + "node_modules/ms": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", + "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", + "license": "MIT" + }, + "node_modules/secure-json-parse": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/secure-json-parse/-/secure-json-parse-4.1.0.tgz", + "integrity": "sha512-l4KnYfEyqYJxDwlNVyRfO2E4NTHfMKAWdUuA8J0yve2Dz/E/PdBepY03RvyJpssIpRFwJoCD55wA+mEDs6ByWA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/fastify" + }, + { + "type": "opencollective", + "url": "https://opencollective.com/fastify" + } + ], + "license": "BSD-3-Clause" + }, + "node_modules/tslib": { + "version": "2.8.1", + "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz", + "integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==", + "license": "0BSD" + }, + "node_modules/tsx": { + "version": "4.23.0", + "resolved": "https://registry.npmjs.org/tsx/-/tsx-4.23.0.tgz", + "integrity": "sha512-eUdUIaCr963q2h5u3+QwvYp0+eqPvn+egeqZUm0hwERCqqx1E3kK5ehbGCvqSE5MQAULr67ww0cA3jKc3YkM1w==", + "dev": true, + "license": "MIT", + "dependencies": { + "esbuild": "~0.28.0" + }, + "bin": { + "tsx": "dist/cli.mjs" + }, + "engines": { + "node": ">=18.0.0" + }, + "optionalDependencies": { + "fsevents": "~2.3.3" + } + }, + "node_modules/typescript": { + "version": "5.9.3", + "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz", + "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", + "dev": true, + "license": "Apache-2.0", + "bin": { + "tsc": "bin/tsc", + "tsserver": "bin/tsserver" + }, + "engines": { + "node": ">=14.17" + } + }, + "node_modules/undici": { + "version": "7.28.0", + "resolved": "https://registry.npmjs.org/undici/-/undici-7.28.0.tgz", + "integrity": "sha512-cRZYrTDwWznlnRiPjggAGxZXanty6M8RV1ff8Wm4LWXBp7/IG8v5DnOm74DtUBp9OONpK75YlPnIjQqX0dBDtA==", + "license": "MIT", + "engines": { + "node": ">=20.18.1" + } + }, + "node_modules/undici-types": { + "version": "6.21.0", + "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz", + "integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/yaml": { + "version": "2.9.0", + "resolved": "https://registry.npmjs.org/yaml/-/yaml-2.9.0.tgz", + "integrity": "sha512-2AvhNX3mb8zd6Zy7INTtSpl1F15HW6Wnqj0srWlkKLcpYl/gMIMJiyuGq2KeI2YFxUPjdlB+3Lc10seMLtL4cA==", + "license": "ISC", + "bin": { + "yaml": "bin.mjs" + }, + "engines": { + "node": ">= 14.6" + }, + "funding": { + "url": "https://github.com/sponsors/eemeli" + } + } + } +} diff --git a/implementations/typescript/package.json b/implementations/typescript/package.json new file mode 100644 index 0000000..974be77 --- /dev/null +++ b/implementations/typescript/package.json @@ -0,0 +1,42 @@ +{ + "name": "esque-ts", + "version": "0.0.0", + "description": "Esque (Elasticsearch Stateful Query Executor) — migration management for Elasticsearch, like Flyway for ES clusters", + "license": "Apache-2.0", + "type": "module", + "engines": { + "node": ">=22" + }, + "bin": { + "esque": "dist/cli.js" + }, + "exports": { + ".": "./dist/esque.js" + }, + "files": [ + "dist" + ], + "repository": { + "type": "git", + "url": "git+https://github.com/loesak/esque.git" + }, + "scripts": { + "build": "tsc -p tsconfig.build.json", + "typecheck": "tsc --noEmit", + "lint": "biome ci src tests", + "format": "biome format --write src tests", + "test": "node --import tsx --test tests/*.test.ts", + "prepublishOnly": "npm run build" + }, + "dependencies": { + "@elastic/elasticsearch": "^9.0.0", + "commander": "^14.0.0", + "yaml": "^2.7.0" + }, + "devDependencies": { + "@biomejs/biome": "^2.0.0", + "@types/node": "^22.0.0", + "tsx": "^4.19.0", + "typescript": "^5.7.0" + } +} diff --git a/implementations/typescript/src/configuration.ts b/implementations/typescript/src/configuration.ts new file mode 100644 index 0000000..cb0ff5c --- /dev/null +++ b/implementations/typescript/src/configuration.ts @@ -0,0 +1 @@ +export {}; diff --git a/implementations/typescript/tsconfig.build.json b/implementations/typescript/tsconfig.build.json new file mode 100644 index 0000000..b9198b5 --- /dev/null +++ b/implementations/typescript/tsconfig.build.json @@ -0,0 +1,11 @@ +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "noEmit": false, + "outDir": "dist", + "rootDir": "src", + "declaration": true, + "sourceMap": true + }, + "include": ["src"] +} diff --git a/implementations/typescript/tsconfig.json b/implementations/typescript/tsconfig.json new file mode 100644 index 0000000..98e5a2e --- /dev/null +++ b/implementations/typescript/tsconfig.json @@ -0,0 +1,13 @@ +{ + "compilerOptions": { + "target": "es2023", + "module": "nodenext", + "moduleResolution": "nodenext", + "strict": true, + "noUncheckedIndexedAccess": true, + "forceConsistentCasingInFileNames": true, + "skipLibCheck": true, + "noEmit": true + }, + "include": ["src", "tests"] +} From 028bf011cded416fd7e4fa22f0d7d9c041399925 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 14:43:32 -0500 Subject: [PATCH 04/26] TypeScript implementation: configuration and migration model Co-Authored-By: Claude Fable 5 --- .../typescript/src/configuration.ts | 21 +++++- .../typescript/src/migration/model.ts | 67 +++++++++++++++++++ .../typescript/tests/model.test.ts | 44 ++++++++++++ 3 files changed, 131 insertions(+), 1 deletion(-) create mode 100644 implementations/typescript/src/migration/model.ts create mode 100644 implementations/typescript/tests/model.test.ts diff --git a/implementations/typescript/src/configuration.ts b/implementations/typescript/src/configuration.ts index cb0ff5c..5be4f47 100644 --- a/implementations/typescript/src/configuration.ts +++ b/implementations/typescript/src/configuration.ts @@ -1 +1,20 @@ -export {}; +export interface EsqueConfiguration { + readonly migrationKey: string; + readonly migrationUser: string | null; + readonly migrationDirectory: string; + readonly lockTimeoutMinutes: number; +} + +export function createEsqueConfiguration(options: { + migrationKey: string; + migrationUser?: string | null; + migrationDirectory?: string; + lockTimeoutMinutes?: number; +}): EsqueConfiguration { + return { + migrationKey: options.migrationKey, + migrationUser: options.migrationUser ?? null, + migrationDirectory: options.migrationDirectory ?? "file:es.migration", + lockTimeoutMinutes: options.lockTimeoutMinutes ?? 5, + }; +} diff --git a/implementations/typescript/src/migration/model.ts b/implementations/typescript/src/migration/model.ts new file mode 100644 index 0000000..a393eb1 --- /dev/null +++ b/implementations/typescript/src/migration/model.ts @@ -0,0 +1,67 @@ +export interface MigrationFileRequestDefinition { + readonly method: string; + readonly path: string; + readonly contentType: string | null; + readonly params: Readonly> | null; + readonly body: string | null; +} + +export interface CanonicalRequest { + method: string; + path: string; + body?: string; + contentType?: string; + params?: Record; +} + +// Only non-null fields, camelCase keys — this shape feeds the checksum. +export function toCanonicalDict(request: MigrationFileRequestDefinition): CanonicalRequest { + const d: CanonicalRequest = { method: request.method, path: request.path }; + if (request.body !== null) { + d.body = request.body; + } + if (request.contentType !== null) { + d.contentType = request.contentType; + } + if (request.params !== null) { + d.params = { ...request.params }; + } + return d; +} + +export interface MigrationFileMetadata { + readonly filename: string; + readonly version: string; + readonly description: string; + readonly checksum: number; +} + +export interface MigrationFileContents { + readonly requests: readonly MigrationFileRequestDefinition[]; +} + +export interface MigrationFile { + readonly metadata: MigrationFileMetadata; + readonly contents: MigrationFileContents; +} + +// Numeric per-segment comparison; shorter versions padded with zeros (1.9.0 < 1.10.0, 1.0 < 1.0.1). +// Ties broken by description, mirroring the Python MigrationFile.__lt__. +export function compareMigrationFiles(a: MigrationFile, b: MigrationFile): number { + const av = a.metadata.version.split(".").map(Number); + const bv = b.metadata.version.split(".").map(Number); + const len = Math.max(av.length, bv.length); + for (let i = 0; i < len; i++) { + const diff = (av[i] ?? 0) - (bv[i] ?? 0); + if (diff !== 0) { + return diff; + } + } + if (a.metadata.description < b.metadata.description) { + return -1; + } + if (a.metadata.description > b.metadata.description) { + return 1; + } + return 0; +} diff --git a/implementations/typescript/tests/model.test.ts b/implementations/typescript/tests/model.test.ts new file mode 100644 index 0000000..097e00e --- /dev/null +++ b/implementations/typescript/tests/model.test.ts @@ -0,0 +1,44 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import type { MigrationFile } from "../src/migration/model.js"; +import { compareMigrationFiles } from "../src/migration/model.js"; + +function file(version: string, description = "Test"): MigrationFile { + return { + metadata: { filename: `V${version}__${description}.yml`, version, description, checksum: 0 }, + contents: { requests: [] }, + }; +} + +test("numeric segment ordering: 1.9.0 < 1.10.0", () => { + assert.ok(compareMigrationFiles(file("1.9.0"), file("1.10.0")) < 0); +}); + +test("major ordering", () => { + assert.ok(compareMigrationFiles(file("1.0.0"), file("2.0.0")) < 0); +}); + +test("minor ordering", () => { + assert.ok(compareMigrationFiles(file("1.0.0"), file("1.1.0")) < 0); +}); + +test("patch ordering", () => { + assert.ok(compareMigrationFiles(file("1.0.0"), file("1.0.1")) < 0); +}); + +test("unequal segment count: 1.0 < 1.0.1", () => { + assert.ok(compareMigrationFiles(file("1.0"), file("1.0.1")) < 0); +}); + +test("equal versions compare as 0", () => { + assert.equal(compareMigrationFiles(file("1.0.0"), file("1.0.0")), 0); +}); + +test("sort order", () => { + const files = [file("1.10.0"), file("2.0.0"), file("1.9.0"), file("1.0.0")]; + files.sort(compareMigrationFiles); + assert.deepEqual( + files.map((f) => f.metadata.version), + ["1.0.0", "1.9.0", "1.10.0", "2.0.0"], + ); +}); From 354c00f76c84e926cc251574cf5d31b0ad027bd9 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 14:53:15 -0500 Subject: [PATCH 05/26] TypeScript implementation: template resolver Co-Authored-By: Claude Fable 5 --- .../typescript/src/migration/template.ts | 67 ++++++++++++++++ .../typescript/tests/model.test.ts | 5 ++ .../typescript/tests/template.test.ts | 80 +++++++++++++++++++ 3 files changed, 152 insertions(+) create mode 100644 implementations/typescript/src/migration/template.ts create mode 100644 implementations/typescript/tests/template.test.ts diff --git a/implementations/typescript/src/migration/template.ts b/implementations/typescript/src/migration/template.ts new file mode 100644 index 0000000..2394112 --- /dev/null +++ b/implementations/typescript/src/migration/template.ts @@ -0,0 +1,67 @@ +import type { MigrationFile, MigrationFileContents, MigrationFileRequestDefinition } from "./model.js"; + +const PLACEHOLDER_PATTERN = /#\{([a-zA-Z0-9._-]+)\}/g; + +export class MigrationTemplateResolver { + private readonly properties: Readonly>; + + constructor(properties: Readonly>) { + this.properties = properties; + } + + // Collects ALL missing variables before throwing. + validate(files: readonly MigrationFile[]): void { + const missing = new Set(); + for (const file of files) { + for (const request of file.contents.requests) { + const texts = [ + request.path, + request.contentType ?? "", + request.body ?? "", + ...Object.values(request.params ?? {}), + ]; + for (const text of texts) { + for (const match of text.matchAll(PLACEHOLDER_PATTERN)) { + const key = match[1]; + if (key !== undefined && !(key in this.properties)) { + missing.add(key); + } + } + } + } + } + if (missing.size > 0) { + throw new Error( + `migration files reference template variables with no matching properties: ${[...missing].join(", ")}`, + ); + } + } + + // Substitutes #{varName} in path, contentType, params values, body — NOT method. + resolve(definition: MigrationFileRequestDefinition): MigrationFileRequestDefinition { + return { + method: definition.method, + path: this.substitute(definition.path), + contentType: definition.contentType !== null ? this.substitute(definition.contentType) : null, + params: + definition.params !== null + ? Object.fromEntries(Object.entries(definition.params).map(([k, v]) => [k, this.substitute(v)])) + : null, + body: definition.body !== null ? this.substitute(definition.body) : null, + }; + } + + resolveContents(contents: MigrationFileContents): MigrationFileContents { + return { requests: contents.requests.map((r) => this.resolve(r)) }; + } + + private substitute(text: string): string { + return text.replace(PLACEHOLDER_PATTERN, (_match, key: string) => { + const value = this.properties[key]; + if (value === undefined) { + throw new Error(`unresolved template variable '#{${key}}' — was validate() called?`); + } + return value; + }); + } +} diff --git a/implementations/typescript/tests/model.test.ts b/implementations/typescript/tests/model.test.ts index 097e00e..33959ea 100644 --- a/implementations/typescript/tests/model.test.ts +++ b/implementations/typescript/tests/model.test.ts @@ -42,3 +42,8 @@ test("sort order", () => { ["1.0.0", "1.9.0", "1.10.0", "2.0.0"], ); }); + +test("description tiebreak when versions equal", () => { + assert.ok(compareMigrationFiles(file("1.0.0", "Alpha"), file("1.0.0", "Beta")) < 0); + assert.ok(compareMigrationFiles(file("1.0.0", "Beta"), file("1.0.0", "Alpha")) > 0); +}); diff --git a/implementations/typescript/tests/template.test.ts b/implementations/typescript/tests/template.test.ts new file mode 100644 index 0000000..17ea08a --- /dev/null +++ b/implementations/typescript/tests/template.test.ts @@ -0,0 +1,80 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import type { MigrationFile, MigrationFileRequestDefinition } from "../src/migration/model.js"; +import { MigrationTemplateResolver } from "../src/migration/template.js"; + +function req( + partial: Partial & { method: string; path: string }, +): MigrationFileRequestDefinition { + return { contentType: null, params: null, body: null, ...partial }; +} + +function fileOf(...requests: MigrationFileRequestDefinition[]): MigrationFile { + return { + metadata: { filename: "V1.0.0__Test.yml", version: "1.0.0", description: "Test", checksum: 0 }, + contents: { requests }, + }; +} + +test("validate passes when all vars present", () => { + const r = req({ method: "PUT", path: "/#{indexName}" }); + new MigrationTemplateResolver({ indexName: "my-index" }).validate([fileOf(r)]); +}); + +test("validate throws on missing var", () => { + const r = req({ method: "PUT", path: "/#{missing}" }); + assert.throws(() => new MigrationTemplateResolver({}).validate([fileOf(r)]), /missing/); +}); + +test("validate collects all missing vars", () => { + const r = req({ method: "PUT", path: "/#{a}", body: "#{b}" }); + assert.throws( + () => new MigrationTemplateResolver({}).validate([fileOf(r)]), + (error: Error) => error.message.includes("a") && error.message.includes("b"), + ); +}); + +test("validate checks body, params, and contentType", () => { + const r = req({ method: "POST", path: "/", contentType: "#{ct}", params: { k: "#{v}" }, body: "#{body}" }); + assert.throws( + () => new MigrationTemplateResolver({}).validate([fileOf(r)]), + (error: Error) => error.message.includes("ct") && error.message.includes("v") && error.message.includes("body"), + ); +}); + +test("resolve substitutes path", () => { + const r = req({ method: "PUT", path: "/#{indexName}" }); + const result = new MigrationTemplateResolver({ indexName: "my-index" }).resolve(r); + assert.equal(result.path, "/my-index"); +}); + +test("resolve substitutes body", () => { + const r = req({ method: "POST", path: "/", body: '{"index": "#{name}"}' }); + const result = new MigrationTemplateResolver({ name: "test" }).resolve(r); + assert.equal(result.body, '{"index": "test"}'); +}); + +test("resolve substitutes params values", () => { + const r = req({ method: "GET", path: "/", params: { q: "#{query}" } }); + const result = new MigrationTemplateResolver({ query: "value" }).resolve(r); + assert.deepEqual(result.params, { q: "value" }); +}); + +test("resolve substitutes contentType", () => { + const r = req({ method: "PUT", path: "/", contentType: "#{ct}" }); + const result = new MigrationTemplateResolver({ ct: "application/json" }).resolve(r); + assert.equal(result.contentType, "application/json"); +}); + +test("resolve does not substitute method", () => { + const r = req({ method: "PUT", path: "/index" }); + const result = new MigrationTemplateResolver({}).resolve(r); + assert.equal(result.method, "PUT"); +}); + +test("resolve handles no template vars", () => { + const r = req({ method: "DELETE", path: "/index", body: '{"key": "value"}' }); + const result = new MigrationTemplateResolver({}).resolve(r); + assert.equal(result.path, "/index"); + assert.equal(result.body, '{"key": "value"}'); +}); From 57ba78f260592a913e29519fcdc689373c246bc3 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 15:02:10 -0500 Subject: [PATCH 06/26] TypeScript implementation: use Object.hasOwn for template property lookups Co-Authored-By: Claude Fable 5 --- .../typescript/src/migration/template.ts | 4 ++-- .../typescript/tests/template.test.ts | 17 +++++++++++++++++ 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/implementations/typescript/src/migration/template.ts b/implementations/typescript/src/migration/template.ts index 2394112..0e18a5e 100644 --- a/implementations/typescript/src/migration/template.ts +++ b/implementations/typescript/src/migration/template.ts @@ -23,7 +23,7 @@ export class MigrationTemplateResolver { for (const text of texts) { for (const match of text.matchAll(PLACEHOLDER_PATTERN)) { const key = match[1]; - if (key !== undefined && !(key in this.properties)) { + if (key !== undefined && !Object.hasOwn(this.properties, key)) { missing.add(key); } } @@ -57,7 +57,7 @@ export class MigrationTemplateResolver { private substitute(text: string): string { return text.replace(PLACEHOLDER_PATTERN, (_match, key: string) => { - const value = this.properties[key]; + const value = Object.hasOwn(this.properties, key) ? this.properties[key] : undefined; if (value === undefined) { throw new Error(`unresolved template variable '#{${key}}' — was validate() called?`); } diff --git a/implementations/typescript/tests/template.test.ts b/implementations/typescript/tests/template.test.ts index 17ea08a..9fdd179 100644 --- a/implementations/typescript/tests/template.test.ts +++ b/implementations/typescript/tests/template.test.ts @@ -78,3 +78,20 @@ test("resolve handles no template vars", () => { assert.equal(result.path, "/index"); assert.equal(result.body, '{"key": "value"}'); }); + +test("validate treats Object.prototype member names as missing vars", () => { + const r = req({ method: "PUT", path: "/#{toString}" }); + assert.throws(() => new MigrationTemplateResolver({}).validate([fileOf(r)]), /toString/); +}); + +test("resolve handles adjacent placeholders", () => { + const r = req({ method: "PUT", path: "/#{a}#{b}" }); + const result = new MigrationTemplateResolver({ a: "x", b: "y" }).resolve(r); + assert.equal(result.path, "/xy"); +}); + +test("resolve inserts property values containing dollar patterns literally", () => { + const r = req({ method: "POST", path: "/", body: "#{v}" }); + const result = new MigrationTemplateResolver({ v: "cost: $& and $1" }).resolve(r); + assert.equal(result.body, "cost: $& and $1"); +}); From c3a9da982b57d0e8b04d88da0497fe45cbe12e15 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 15:11:01 -0500 Subject: [PATCH 07/26] TypeScript implementation: canonical checksum Co-Authored-By: Claude Fable 5 --- .../typescript/src/migration/loader.ts | 27 ++++++++++ .../typescript/tests/checksum.test.ts | 53 +++++++++++++++++++ 2 files changed, 80 insertions(+) create mode 100644 implementations/typescript/src/migration/loader.ts create mode 100644 implementations/typescript/tests/checksum.test.ts diff --git a/implementations/typescript/src/migration/loader.ts b/implementations/typescript/src/migration/loader.ts new file mode 100644 index 0000000..e74d357 --- /dev/null +++ b/implementations/typescript/src/migration/loader.ts @@ -0,0 +1,27 @@ +import { createHash } from "node:crypto"; +import type { MigrationFileContents } from "./model.js"; +import { toCanonicalDict } from "./model.js"; + +// Canonical JSON: keys sorted alphabetically at every level, null/undefined object values +// dropped, compact separators. Matches Python's +// json.dumps(remove_nulls(data), sort_keys=True, separators=(",", ":"), ensure_ascii=False). +function canonicalJson(value: unknown): string { + if (value === null || typeof value !== "object") { + return JSON.stringify(value); + } + if (Array.isArray(value)) { + return `[${value.map((item) => canonicalJson(item)).join(",")}]`; + } + const entries = Object.entries(value as Record) + .filter((entry) => entry[1] !== null && entry[1] !== undefined) + .sort(([a], [b]) => (a < b ? -1 : 1)); + return `{${entries.map(([k, v]) => `${JSON.stringify(k)}:${canonicalJson(v)}`).join(",")}}`; +} + +// Canonical algorithm (identical across all implementations): +// canonical JSON of {"requests": [...]} → UTF-8 → MD5 → first 4 bytes as big-endian signed int32. +export function calculateChecksum(contents: MigrationFileContents): number { + const data = { requests: contents.requests.map((r) => toCanonicalDict(r)) }; + const digest = createHash("md5").update(canonicalJson(data), "utf8").digest(); + return digest.readInt32BE(0); +} diff --git a/implementations/typescript/tests/checksum.test.ts b/implementations/typescript/tests/checksum.test.ts new file mode 100644 index 0000000..1392a28 --- /dev/null +++ b/implementations/typescript/tests/checksum.test.ts @@ -0,0 +1,53 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import { calculateChecksum } from "../src/migration/loader.js"; +import type { MigrationFileRequestDefinition } from "../src/migration/model.js"; + +function req( + partial: Partial & { method: string; path: string }, +): MigrationFileRequestDefinition { + return { contentType: null, params: null, body: null, ...partial }; +} + +function checksum(requests: MigrationFileRequestDefinition[]): number { + return calculateChecksum({ requests }); +} + +test("result is a signed 32-bit integer", () => { + const result = checksum([req({ method: "PUT", path: "/test-index" })]); + assert.ok(Number.isInteger(result)); + assert.ok(result >= -(2 ** 31) && result <= 2 ** 31 - 1); +}); + +test("deterministic", () => { + const requests = [req({ method: "PUT", path: "/test", body: '{"settings": {}}' })]; + assert.equal(checksum(requests), checksum(requests)); +}); + +test("null fields excluded", () => { + const r1 = req({ method: "PUT", path: "/index" }); + const r2 = req({ method: "PUT", path: "/index", body: null, contentType: null, params: null }); + assert.equal(checksum([r1]), checksum([r2])); +}); + +test("different content differs", () => { + const r1 = [req({ method: "PUT", path: "/index-a" })]; + const r2 = [req({ method: "PUT", path: "/index-b" })]; + assert.notEqual(checksum(r1), checksum(r2)); +}); + +test("cross-implementation reference vector", () => { + // MUST equal the Python/JVM value for the identical input. -991565970 was generated from + // the Python reference implementation via MigrationFileLoader.calculate_checksum for + // [MigrationFileRequestDefinition(method="PUT", path="/test-index")]. + const result = checksum([req({ method: "PUT", path: "/test-index" })]); + assert.equal(result, -991565970); +}); + +test("multiple requests", () => { + const r1 = req({ method: "PUT", path: "/index" }); + const r2 = req({ method: "POST", path: "/_aliases", body: "{}" }); + const combined = checksum([r1, r2]); + assert.notEqual(combined, checksum([r1])); + assert.notEqual(combined, checksum([r2])); +}); From 2ef4deb6d04ca9a247852384543dec0d754bd9c7 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 15:22:40 -0500 Subject: [PATCH 08/26] TypeScript implementation: migration file loader Co-Authored-By: Claude Fable 5 --- .../typescript/src/migration/loader.ts | 85 ++++++++++++++++++- .../typescript/tests/checksum.test.ts | 13 +++ 2 files changed, 96 insertions(+), 2 deletions(-) diff --git a/implementations/typescript/src/migration/loader.ts b/implementations/typescript/src/migration/loader.ts index e74d357..3ce83c1 100644 --- a/implementations/typescript/src/migration/loader.ts +++ b/implementations/typescript/src/migration/loader.ts @@ -1,10 +1,15 @@ import { createHash } from "node:crypto"; -import type { MigrationFileContents } from "./model.js"; -import { toCanonicalDict } from "./model.js"; +import { readdirSync, readFileSync, statSync } from "node:fs"; +import { join } from "node:path"; +import { parse } from "yaml"; +import type { MigrationFile, MigrationFileContents, MigrationFileRequestDefinition } from "./model.js"; +import { compareMigrationFiles, toCanonicalDict } from "./model.js"; +import type { MigrationTemplateResolver } from "./template.js"; // Canonical JSON: keys sorted alphabetically at every level, null/undefined object values // dropped, compact separators. Matches Python's // json.dumps(remove_nulls(data), sort_keys=True, separators=(",", ":"), ensure_ascii=False). +// Only ever fed the closed shape produced by toCanonicalDict — do not reuse for arbitrary JSON. function canonicalJson(value: unknown): string { if (value === null || typeof value !== "object") { return JSON.stringify(value); @@ -25,3 +30,79 @@ export function calculateChecksum(contents: MigrationFileContents): number { const digest = createHash("md5").update(canonicalJson(data), "utf8").digest(); return digest.readInt32BE(0); } + +const FILE_NAME_PATTERN = /^V((\d+\.?)+)__(\w+)\.yml$/; + +export class MigrationFileLoader { + private readonly migrationDirectory: string; + private readonly templateResolver: MigrationTemplateResolver; + + constructor(migrationDirectory: string, templateResolver: MigrationTemplateResolver) { + this.migrationDirectory = migrationDirectory; + this.templateResolver = templateResolver; + } + + load(): MigrationFile[] { + const dir = resolveDirectoryPath(this.migrationDirectory); + const rawFiles: MigrationFile[] = []; + for (const name of readdirSync(dir)) { + const filePath = join(dir, name); + if (!statSync(filePath).isFile()) { + continue; + } + const match = FILE_NAME_PATTERN.exec(name); + if (match === null) { + continue; + } + rawFiles.push(readRawFile(filePath, name, match)); + } + rawFiles.sort(compareMigrationFiles); + this.templateResolver.validate(rawFiles); + return rawFiles.map((file) => this.resolveFile(file)); + } + + private resolveFile(file: MigrationFile): MigrationFile { + const resolvedContents = this.templateResolver.resolveContents(file.contents); + return { + metadata: { ...file.metadata, checksum: calculateChecksum(resolvedContents) }, + contents: resolvedContents, + }; + } +} + +function resolveDirectoryPath(directory: string): string { + if (directory.startsWith("file:")) { + return directory.slice("file:".length); + } + throw new Error(`unsupported migration directory scheme in '${directory}'. supported schemes: 'file:'`); +} + +function readRawFile(filePath: string, filename: string, match: RegExpExecArray): MigrationFile { + const version = match[1]; + const description = match[3]; + if (version === undefined || description === undefined || version.split(".").some((s) => s === "")) { + // Empty segment (e.g. trailing dot in "V1.__X.yml") — fail loud, mirroring Python's + // int("") ValueError, instead of silently treating it as 0. + throw new Error(`invalid migration filename: ${filename}`); + } + const data = parse(readFileSync(filePath, "utf8")) as { requests: Record[] }; + return { + metadata: { filename, version, description, checksum: 0 }, + contents: { requests: data.requests.map((raw) => parseRequest(raw)) }, + }; +} + +function parseRequest(raw: Record): MigrationFileRequestDefinition { + return { + method: String(raw.method), + path: String(raw.path), + contentType: "contentType" in raw ? String(raw.contentType) : null, + params: + "params" in raw + ? Object.fromEntries( + Object.entries(raw.params as Record).map(([k, v]) => [String(k), String(v)]), + ) + : null, + body: "body" in raw ? String(raw.body) : null, + }; +} diff --git a/implementations/typescript/tests/checksum.test.ts b/implementations/typescript/tests/checksum.test.ts index 1392a28..10f6cb1 100644 --- a/implementations/typescript/tests/checksum.test.ts +++ b/implementations/typescript/tests/checksum.test.ts @@ -44,6 +44,19 @@ test("cross-implementation reference vector", () => { assert.equal(result, -991565970); }); +test("cross-implementation reference vector with all fields", () => { + // 509953348 was generated from the Python reference implementation for this exact input; + // exercises alphabetical key ordering across all five canonical fields plus non-ASCII. + const r = req({ + method: "POST", + path: "/x", + body: '{"a": "ü"}', + contentType: "application/json", + params: { q: "1" }, + }); + assert.equal(checksum([r]), 509953348); +}); + test("multiple requests", () => { const r1 = req({ method: "PUT", path: "/index" }); const r2 = req({ method: "POST", path: "/_aliases", body: "{}" }); From 9192a467ba25b7d0f07711cb8e6aa2014c051d0a Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 15:32:53 -0500 Subject: [PATCH 09/26] TypeScript implementation: fail loudly on malformed migration YAML Co-Authored-By: Claude Fable 5 --- .../typescript/src/migration/loader.ts | 33 ++++++++--- .../typescript/tests/loader.test.ts | 56 +++++++++++++++++++ 2 files changed, 80 insertions(+), 9 deletions(-) create mode 100644 implementations/typescript/tests/loader.test.ts diff --git a/implementations/typescript/src/migration/loader.ts b/implementations/typescript/src/migration/loader.ts index 3ce83c1..15a2fa6 100644 --- a/implementations/typescript/src/migration/loader.ts +++ b/implementations/typescript/src/migration/loader.ts @@ -85,24 +85,39 @@ function readRawFile(filePath: string, filename: string, match: RegExpExecArray) // int("") ValueError, instead of silently treating it as 0. throw new Error(`invalid migration filename: ${filename}`); } - const data = parse(readFileSync(filePath, "utf8")) as { requests: Record[] }; + const data = parse(readFileSync(filePath, "utf8")) as { requests?: unknown }; + if (!Array.isArray(data?.requests)) { + throw new Error(`migration file [${filename}] must contain a 'requests' list`); + } return { metadata: { filename, version, description, checksum: 0 }, - contents: { requests: data.requests.map((raw) => parseRequest(raw)) }, + contents: { requests: data.requests.map((raw: unknown) => parseRequest(raw)) }, }; } -function parseRequest(raw: Record): MigrationFileRequestDefinition { +function parseRequest(raw: unknown): MigrationFileRequestDefinition { + if (!isPlainObject(raw)) { + throw new Error("migration request definition must be a mapping"); + } + if (!("method" in raw) || !("path" in raw)) { + throw new Error("migration request definition missing required field 'method' or 'path'"); + } + let params: Record | null = null; + if ("params" in raw) { + if (!isPlainObject(raw.params)) { + throw new Error("migration request 'params' must be a mapping of string keys to values"); + } + params = Object.fromEntries(Object.entries(raw.params).map(([k, v]) => [String(k), String(v)])); + } return { method: String(raw.method), path: String(raw.path), contentType: "contentType" in raw ? String(raw.contentType) : null, - params: - "params" in raw - ? Object.fromEntries( - Object.entries(raw.params as Record).map(([k, v]) => [String(k), String(v)]), - ) - : null, + params, body: "body" in raw ? String(raw.body) : null, }; } + +function isPlainObject(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} diff --git a/implementations/typescript/tests/loader.test.ts b/implementations/typescript/tests/loader.test.ts new file mode 100644 index 0000000..6a26620 --- /dev/null +++ b/implementations/typescript/tests/loader.test.ts @@ -0,0 +1,56 @@ +import assert from "node:assert/strict"; +import { mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { test } from "node:test"; +import { MigrationFileLoader } from "../src/migration/loader.js"; +import { MigrationTemplateResolver } from "../src/migration/template.js"; + +function loaderFor(files: Record): MigrationFileLoader { + const dir = mkdtempSync(join(tmpdir(), "esque-loader-test-")); + for (const [name, content] of Object.entries(files)) { + writeFileSync(join(dir, name), content); + } + return new MigrationFileLoader(`file:${dir}`, new MigrationTemplateResolver({})); +} + +test("loads and orders valid migration files", () => { + const loader = loaderFor({ + "V1.10.0__B.yml": "requests:\n - method: PUT\n path: /b\n", + "V1.9.0__A.yml": "requests:\n - method: PUT\n path: /a\n", + "ignored.txt": "not a migration", + }); + const files = loader.load(); + assert.deepEqual( + files.map((f) => f.metadata.version), + ["1.9.0", "1.10.0"], + ); + assert.ok(files.every((f) => Number.isInteger(f.metadata.checksum) && f.metadata.checksum !== 0)); +}); + +test("throws on trailing-dot version in filename", () => { + const loader = loaderFor({ "V1.__X.yml": "requests:\n - method: PUT\n path: /x\n" }); + assert.throws(() => loader.load(), /invalid migration filename/); +}); + +test("throws when requests key is missing", () => { + const loader = loaderFor({ "V1.0.0__X.yml": "notrequests: []\n" }); + assert.throws(() => loader.load(), /must contain a 'requests' list/); +}); + +test("throws when request is missing method or path", () => { + const loader = loaderFor({ "V1.0.0__X.yml": "requests:\n - path: /x\n" }); + assert.throws(() => loader.load(), /missing required field/); +}); + +test("throws when params is not a mapping", () => { + const loader = loaderFor({ + "V1.0.0__X.yml": "requests:\n - method: PUT\n path: /x\n params:\n - 1\n - 2\n", + }); + assert.throws(() => loader.load(), /'params' must be a mapping/); +}); + +test("rejects non-file: directory scheme", () => { + const loader = new MigrationFileLoader("s3://bucket/migrations", new MigrationTemplateResolver({})); + assert.throws(() => loader.load(), /unsupported migration directory scheme/); +}); From 317286776912d802104aa4a034d7d53cdc777b21 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 15:46:14 -0500 Subject: [PATCH 10/26] TypeScript implementation: ES document types and index definition Co-Authored-By: Claude Fable 5 --- .../typescript/src/elasticsearch/documents.ts | 78 +++++++++++++++++++ 1 file changed, 78 insertions(+) create mode 100644 implementations/typescript/src/elasticsearch/documents.ts diff --git a/implementations/typescript/src/elasticsearch/documents.ts b/implementations/typescript/src/elasticsearch/documents.ts new file mode 100644 index 0000000..2d5c678 --- /dev/null +++ b/implementations/typescript/src/elasticsearch/documents.ts @@ -0,0 +1,78 @@ +export const MIGRATION_INDEX = ".esque"; +export const LOCK_ID_PREFIX = "lock"; + +export const INDEX_DEFINITION = { + settings: { + index: { + number_of_shards: "1", + auto_expand_replicas: "0-all", + refresh_interval: "1s", + }, + }, + mappings: { + properties: { + lock: { properties: { date: { type: "date" } } }, + migration: { + properties: { + checksum: { type: "long" }, + description: { type: "keyword" }, + executionTime: { type: "long" }, + filename: { type: "keyword" }, + installedOn: { type: "date" }, + migrationKey: { type: "keyword" }, + order: { type: "long" }, + version: { type: "keyword" }, + }, + }, + }, + }, +} as const; + +export interface MigrationRecord { + readonly migrationKey: string; + readonly order: number; + readonly filename: string; + readonly version: string; + readonly description: string; + readonly checksum: number; + readonly installedBy: string | null; + readonly installedOn: string; // ISO-8601 UTC + readonly executionTime: number; +} + +// Wrapper-object serialization — mirrors JVM @JsonTypeInfo(As.WRAPPER_OBJECT). +export function migrationRecordToDocument(record: MigrationRecord): { migration: Record } { + const doc: Record = { + migrationKey: record.migrationKey, + order: record.order, + filename: record.filename, + version: record.version, + description: record.description, + checksum: record.checksum, + installedOn: record.installedOn, + executionTime: record.executionTime, + }; + if (record.installedBy !== null) { + doc.installedBy = record.installedBy; + } + return { migration: doc }; +} + +export function migrationRecordFromDocument(source: Record): MigrationRecord { + const raw = source.migration as Record; + return { + migrationKey: String(raw.migrationKey), + order: Number(raw.order), + filename: String(raw.filename), + version: String(raw.version), + description: String(raw.description), + checksum: Number(raw.checksum), + installedBy: raw.installedBy !== undefined && raw.installedBy !== null ? String(raw.installedBy) : null, + installedOn: String(raw.installedOn), + executionTime: Number(raw.executionTime), + }; +} + +export function migrationLockToDocument(date: Date): { lock: { date: string } } { + return { lock: { date: date.toISOString() } }; +} From b4a92cff6219bfbd266fb0810b807f334e235d95 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 16:21:10 -0500 Subject: [PATCH 11/26] TypeScript implementation: fail loudly on malformed ES migration records Co-Authored-By: Claude Fable 5 --- .../typescript/src/elasticsearch/documents.ts | 24 ++++++--- .../typescript/tests/documents.test.ts | 50 +++++++++++++++++++ 2 files changed, 66 insertions(+), 8 deletions(-) create mode 100644 implementations/typescript/tests/documents.test.ts diff --git a/implementations/typescript/src/elasticsearch/documents.ts b/implementations/typescript/src/elasticsearch/documents.ts index 2d5c678..1ecd971 100644 --- a/implementations/typescript/src/elasticsearch/documents.ts +++ b/implementations/typescript/src/elasticsearch/documents.ts @@ -58,18 +58,26 @@ export function migrationRecordToDocument(record: MigrationRecord): { migration: return { migration: doc }; } +function requireField(raw: Record, field: string): unknown { + const value = raw[field]; + if (value === undefined || value === null) { + throw new Error(`malformed migration record in .esque index: missing field '${field}'`); + } + return value; +} + export function migrationRecordFromDocument(source: Record): MigrationRecord { const raw = source.migration as Record; return { - migrationKey: String(raw.migrationKey), - order: Number(raw.order), - filename: String(raw.filename), - version: String(raw.version), - description: String(raw.description), - checksum: Number(raw.checksum), + migrationKey: String(requireField(raw, "migrationKey")), + order: Number(requireField(raw, "order")), + filename: String(requireField(raw, "filename")), + version: String(requireField(raw, "version")), + description: String(requireField(raw, "description")), + checksum: Number(requireField(raw, "checksum")), installedBy: raw.installedBy !== undefined && raw.installedBy !== null ? String(raw.installedBy) : null, - installedOn: String(raw.installedOn), - executionTime: Number(raw.executionTime), + installedOn: String(requireField(raw, "installedOn")), + executionTime: Number(requireField(raw, "executionTime")), }; } diff --git a/implementations/typescript/tests/documents.test.ts b/implementations/typescript/tests/documents.test.ts new file mode 100644 index 0000000..f5cb1d4 --- /dev/null +++ b/implementations/typescript/tests/documents.test.ts @@ -0,0 +1,50 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import { + migrationLockToDocument, + migrationRecordFromDocument, + migrationRecordToDocument, +} from "../src/elasticsearch/documents.js"; + +test("round-trips a full migration record", () => { + const record = { + migrationKey: "default", + order: 0, + filename: "V1.0.0__CreateFirstIndex.yml", + version: "1.0.0", + description: "CreateFirstIndex", + checksum: -123456789, + installedBy: "aaron", + installedOn: "2026-06-13T12:00:00.000Z", + executionTime: 42, + }; + const doc = migrationRecordToDocument(record); + assert.deepEqual(migrationRecordFromDocument(doc), record); +}); + +test("omits installedBy from the document when null", () => { + const doc = migrationRecordToDocument({ + migrationKey: "default", + order: 0, + filename: "V1.0.0__CreateFirstIndex.yml", + version: "1.0.0", + description: "CreateFirstIndex", + checksum: 1, + installedBy: null, + installedOn: "2026-06-13T12:00:00.000Z", + executionTime: 1, + }); + assert.equal("installedBy" in doc.migration, false); +}); + +test("throws on missing required field", () => { + assert.throws( + () => migrationRecordFromDocument({ migration: { migrationKey: "k", order: 0 } }), + /missing field 'filename'/, + ); +}); + +test("migrationLockToDocument wraps an ISO date under lock.date", () => { + const date = new Date("2026-06-13T12:00:00.000Z"); + assert.deepEqual(migrationLockToDocument(date), { lock: { date: "2026-06-13T12:00:00.000Z" } }); +}); From 99249ca1e2b61a5f55507b5886ec16a1769e83ef Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 16:26:37 -0500 Subject: [PATCH 12/26] TypeScript implementation: REST client operations Co-Authored-By: Claude Fable 5 --- .../src/elasticsearch/operations.ts | 134 ++++++++++++++++++ 1 file changed, 134 insertions(+) create mode 100644 implementations/typescript/src/elasticsearch/operations.ts diff --git a/implementations/typescript/src/elasticsearch/operations.ts b/implementations/typescript/src/elasticsearch/operations.ts new file mode 100644 index 0000000..f4427d5 --- /dev/null +++ b/implementations/typescript/src/elasticsearch/operations.ts @@ -0,0 +1,134 @@ +import type { Client } from "@elastic/elasticsearch"; +import { errors } from "@elastic/elasticsearch"; +import type { MigrationFile, MigrationFileRequestDefinition } from "../migration/model.js"; +import type { MigrationRecord } from "./documents.js"; +import { + INDEX_DEFINITION, + LOCK_ID_PREFIX, + MIGRATION_INDEX, + migrationLockToDocument, + migrationRecordFromDocument, + migrationRecordToDocument, +} from "./documents.js"; + +export class RestClientOperations { + private readonly client: Client; + private readonly migrationKey: string; + + constructor(client: Client, migrationKey: string) { + this.client = client; + this.migrationKey = migrationKey; + } + + async close(): Promise { + await this.client.close(); + } + + async checkMigrationIndexExists(): Promise { + return await this.client.indices.exists({ index: MIGRATION_INDEX }); + } + + async createMigrationIndex(): Promise { + try { + await this.client.indices.create({ + index: MIGRATION_INDEX, + settings: INDEX_DEFINITION.settings, + mappings: INDEX_DEFINITION.mappings, + }); + } catch (error) { + if (isAlreadyExistsError(error)) { + return; // another process created it first — safe + } + throw error; + } + } + + async createLockRecord(): Promise { + await this.client.index({ + index: MIGRATION_INDEX, + id: `${LOCK_ID_PREFIX}:${this.migrationKey}`, + document: migrationLockToDocument(new Date()), + op_type: "create", + }); + } + + async deleteLockRecord(): Promise { + await this.client.delete({ + index: MIGRATION_INDEX, + id: `${LOCK_ID_PREFIX}:${this.migrationKey}`, + }); + } + + async getMigrationRecords(): Promise { + const response = await this.client.search>({ + index: MIGRATION_INDEX, + query: { bool: { filter: [{ term: { "migration.migrationKey": this.migrationKey } }] } }, + size: 10000, + }); + const records = response.hits.hits.map((hit) => + migrationRecordFromDocument(hit._source as Record), + ); + records.sort((a, b) => a.order - b.order); + return records; + } + + async getMigrationRecordForMigrationFile(file: MigrationFile): Promise { + const response = await this.client.search>({ + index: MIGRATION_INDEX, + query: { + bool: { + filter: [ + { term: { "migration.migrationKey": this.migrationKey } }, + { term: { "migration.filename": file.metadata.filename } }, + ], + }, + }, + }); + const hits = response.hits.hits; + if (hits.length > 1) { + throw new Error( + `found more than one migration record for file [${file.metadata.filename}] and migration key [${this.migrationKey}]`, + ); + } + const first = hits[0]; + if (first !== undefined) { + return migrationRecordFromDocument(first._source as Record); + } + return null; + } + + async executeMigrationDefinition(definition: MigrationFileRequestDefinition): Promise { + const headers: Record = {}; + if (definition.contentType !== null) { + headers["content-type"] = definition.contentType; + } + await this.client.transport.request( + { + method: definition.method, + path: definition.path, + querystring: definition.params ?? undefined, + body: definition.body ?? undefined, + }, + { headers }, + ); + } + + async createMigrationRecord(record: MigrationRecord): Promise { + if (record.migrationKey !== this.migrationKey) { + throw new Error("migration record migration key must match operational migration key"); + } + await this.client.index({ + index: MIGRATION_INDEX, + document: migrationRecordToDocument(record), + refresh: true, // without it, reads immediately after won't see the record + }); + } +} + +function isAlreadyExistsError(error: unknown): boolean { + if (!(error instanceof errors.ResponseError)) { + return false; + } + const body = error.body as { error?: { type?: string } } | undefined; + return body?.error?.type === "resource_already_exists_exception"; +} From 310972d3cde4a80bcdcd04ad51e2d5fcec97efb9 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 16:37:13 -0500 Subject: [PATCH 13/26] TypeScript implementation: distributed document lock Co-Authored-By: Claude Fable 5 --- .../typescript/src/elasticsearch/lock.ts | 56 ++++++++++++ implementations/typescript/tests/lock.test.ts | 89 +++++++++++++++++++ 2 files changed, 145 insertions(+) create mode 100644 implementations/typescript/src/elasticsearch/lock.ts create mode 100644 implementations/typescript/tests/lock.test.ts diff --git a/implementations/typescript/src/elasticsearch/lock.ts b/implementations/typescript/src/elasticsearch/lock.ts new file mode 100644 index 0000000..dcc263f --- /dev/null +++ b/implementations/typescript/src/elasticsearch/lock.ts @@ -0,0 +1,56 @@ +import { setTimeout as sleep } from "node:timers/promises"; +import type { RestClientOperations } from "./operations.js"; + +const IDLE_BETWEEN_TRIES_MS = 100; + +// Thrown by unlock() when the lock is not held — expected during Esque.close() after a clean run. +export class LockNotHeldError extends Error {} + +// Distributed lock via ES op_type=create. The JVM wraps this in a local ReentrantLock for +// thread safety; Node is single-threaded, so a held-flag suffices — it exists to make +// unlock() without tryLock() a detectable error, and to avoid deleting another process's +// lock document from a process that never acquired it. +export class ElasticsearchDocumentLock { + private readonly operations: RestClientOperations; + private held = false; + + constructor(operations: RestClientOperations) { + this.operations = operations; + } + + async tryLock(timeoutMinutes: number): Promise { + const deadline = Date.now() + timeoutMinutes * 60_000; + for (;;) { + if (await this.doLock()) { + this.held = true; + return true; + } + if (Date.now() >= deadline) { + return false; + } + await sleep(IDLE_BETWEEN_TRIES_MS); + } + } + + async unlock(): Promise { + if (!this.held) { + throw new LockNotHeldError("cannot release un-acquired lock"); + } + this.held = false; + try { + await this.operations.deleteLockRecord(); + } catch (error) { + throw new Error("Failed to release mutex", { cause: error }); + } + } + + private async doLock(): Promise { + try { + await this.operations.createLockRecord(); + return true; + } catch { + // TODO: differentiate ConflictError (lock exists) from other failures + return false; + } + } +} diff --git a/implementations/typescript/tests/lock.test.ts b/implementations/typescript/tests/lock.test.ts new file mode 100644 index 0000000..9604802 --- /dev/null +++ b/implementations/typescript/tests/lock.test.ts @@ -0,0 +1,89 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import { ElasticsearchDocumentLock, LockNotHeldError } from "../src/elasticsearch/lock.js"; +import type { RestClientOperations } from "../src/elasticsearch/operations.js"; + +function fakeOperations(overrides: Partial = {}): RestClientOperations { + return { + createLockRecord: async () => {}, + deleteLockRecord: async () => {}, + ...overrides, + } as RestClientOperations; +} + +test("tryLock succeeds immediately when createLockRecord succeeds", async () => { + const lock = new ElasticsearchDocumentLock(fakeOperations()); + assert.equal(await lock.tryLock(1), true); +}); + +test("unlock deletes the lock record after a successful tryLock", async () => { + let deleted = false; + const lock = new ElasticsearchDocumentLock( + fakeOperations({ + deleteLockRecord: async () => { + deleted = true; + }, + }), + ); + await lock.tryLock(1); + await lock.unlock(); + assert.equal(deleted, true); +}); + +test("unlock throws LockNotHeldError when called without a prior successful tryLock", async () => { + const lock = new ElasticsearchDocumentLock(fakeOperations()); + await assert.rejects(() => lock.unlock(), LockNotHeldError); +}); + +test("unlock throws LockNotHeldError again after a first successful unlock", async () => { + const lock = new ElasticsearchDocumentLock(fakeOperations()); + await lock.tryLock(1); + await lock.unlock(); + await assert.rejects(() => lock.unlock(), LockNotHeldError); +}); + +test("tryLock retries after a failed acquisition and eventually succeeds", async () => { + let attempts = 0; + const lock = new ElasticsearchDocumentLock( + fakeOperations({ + createLockRecord: async () => { + attempts += 1; + if (attempts < 3) { + throw new Error("lock exists"); + } + }, + }), + ); + const acquired = await lock.tryLock(1); + assert.equal(acquired, true); + assert.equal(attempts, 3); +}); + +test("tryLock returns false when the timeout elapses before acquisition", async () => { + const lock = new ElasticsearchDocumentLock( + fakeOperations({ + createLockRecord: async () => { + throw new Error("lock exists"); + }, + }), + ); + // timeoutMinutes=0 means the deadline is already in the past after the first failed attempt. + const acquired = await lock.tryLock(0); + assert.equal(acquired, false); +}); + +test("unlock wraps deleteLockRecord failures and still marks the lock as released", async () => { + const lock = new ElasticsearchDocumentLock( + fakeOperations({ + deleteLockRecord: async () => { + throw new Error("network error"); + }, + }), + ); + await lock.tryLock(1); + await assert.rejects(() => lock.unlock(), /Failed to release mutex/); + // Even though deleteLockRecord failed, the local held-flag was already cleared before the + // attempt (matching the plan's unlock() ordering), so a second unlock() call throws + // LockNotHeldError, not another "Failed to release mutex". + await assert.rejects(() => lock.unlock(), LockNotHeldError); +}); From e8cf843e00473d831e832f74c741501dcf8fda71 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 16:50:17 -0500 Subject: [PATCH 14/26] TypeScript implementation: Esque orchestrator with integrity verification Wires configuration, migration loader, ES operations, and the distributed lock into the main Esque class, mirroring the Python orchestrator's execution flow and integrity-check logic method-for-method. Co-Authored-By: Claude Fable 5 --- implementations/typescript/src/esque.ts | 163 ++++++++++++++++++ .../typescript/tests/integrity.test.ts | 97 +++++++++++ 2 files changed, 260 insertions(+) create mode 100644 implementations/typescript/src/esque.ts create mode 100644 implementations/typescript/tests/integrity.test.ts diff --git a/implementations/typescript/src/esque.ts b/implementations/typescript/src/esque.ts new file mode 100644 index 0000000..7c16658 --- /dev/null +++ b/implementations/typescript/src/esque.ts @@ -0,0 +1,163 @@ +import type { Client } from "@elastic/elasticsearch"; +import type { EsqueConfiguration } from "./configuration.js"; +import type { MigrationRecord } from "./elasticsearch/documents.js"; +import { ElasticsearchDocumentLock, LockNotHeldError } from "./elasticsearch/lock.js"; +import { RestClientOperations } from "./elasticsearch/operations.js"; +import { MigrationFileLoader } from "./migration/loader.js"; +import type { MigrationFile } from "./migration/model.js"; +import { MigrationTemplateResolver } from "./migration/template.js"; + +export class Esque { + private readonly configuration: EsqueConfiguration; + private readonly migrationLoader: MigrationFileLoader; + private readonly operations: RestClientOperations; + private readonly lock: ElasticsearchDocumentLock; + + constructor(client: Client, configuration: EsqueConfiguration, properties: Record = {}) { + this.configuration = configuration; + this.migrationLoader = new MigrationFileLoader( + configuration.migrationDirectory, + new MigrationTemplateResolver(properties), + ); + this.operations = new RestClientOperations(client, configuration.migrationKey); + this.lock = new ElasticsearchDocumentLock(this.operations); + } + + async close(): Promise { + try { + await this.lock.unlock(); + } catch (error) { + if (!(error instanceof LockNotHeldError)) { + console.warn("failed to release a execution lock. you may need to manually delete the lock document yourself"); + } + } + try { + await this.operations.close(); + } catch { + console.warn("failed to close client. this is likely not an issue"); + } + } + + async [Symbol.asyncDispose](): Promise { + await this.close(); + } + + async execute(): Promise { + try { + await this.initialize(); + const files = this.migrationLoader.load(); + const history = await this.operations.getMigrationRecords(); + this.verifyStateIntegrity(files, history); + await this.runMigrations(files); + } catch (error) { + throw new Error("Failed to run esque execution", { cause: error }); + } + } + + private async initialize(): Promise { + if (!(await this.operations.checkMigrationIndexExists())) { + await this.operations.createMigrationIndex(); + } + } + + private verifyStateIntegrity(files: MigrationFile[], history: MigrationRecord[]): void { + if (history.length > files.length) { + throw new Error( + "the migration records are showing more migrations than the local system defines. " + + "did you refactor your files or use an incorrect migration key?", + ); + } + const last = history[history.length - 1]; + if (last !== undefined && history.length !== last.order + 1) { + throw new Error("the migration records seem to be corrupt as some records appear to be missing."); + } + for (const record of history) { + this.verifyRecordIntegrity(record, files); + } + } + + private verifyRecordIntegrity(record: MigrationRecord, files: MigrationFile[]): void { + const companion = files.find((f) => f.metadata.filename === record.filename); + if (companion === undefined) { + throw new Error( + `could not find migration file matching migration history record by filename [${record.filename}]`, + ); + } + if ( + record.order !== files.indexOf(companion) || + record.version !== companion.metadata.version || + record.description !== companion.metadata.description || + record.checksum !== companion.metadata.checksum || + record.migrationKey !== this.configuration.migrationKey + ) { + throw new Error( + `could not verify integrity of migration history record for filename [${record.filename}]. ` + + "did you refactor your migration scripts after a previous execution?", + ); + } + } + + private async runMigrations(files: MigrationFile[]): Promise { + try { + for (const file of files) { + try { + if (await this.lock.tryLock(this.configuration.lockTimeoutMinutes)) { + const existing = await this.operations.getMigrationRecordForMigrationFile(file); + if (existing === null) { + const start = performance.now(); + await this.runMigrationForFile(file); + const elapsedMs = Math.round(performance.now() - start); + await this.operations.createMigrationRecord({ + migrationKey: this.configuration.migrationKey, + order: files.indexOf(file), + filename: file.metadata.filename, + version: file.metadata.version, + description: file.metadata.description, + checksum: file.metadata.checksum, + installedBy: this.configuration.migrationUser, + installedOn: new Date().toISOString(), + executionTime: elapsedMs, + }); + } + } else { + throw new Error("failed to acquire lock"); + } + } catch (error) { + throw new Error(`Failed to execute queries in migration file [${file.metadata.filename}]`, { + cause: error, + }); + } finally { + await this.unlockIgnoringNotHeld(); + } + } + } catch (error) { + throw new Error("failed to run migrations", { cause: error }); + } + } + + // Releasing a lock we never acquired (e.g. tryLock() failed) is expected — swallow just that + // case. Split out from runMigrations' finally block so the rethrow isn't a direct statement + // inside a finally clause (which would unsafely shadow the original error's control flow). + private async unlockIgnoringNotHeld(): Promise { + try { + await this.lock.unlock(); + } catch (error) { + if (!(error instanceof LockNotHeldError)) { + throw error; + } + } + } + + private async runMigrationForFile(file: MigrationFile): Promise { + for (const [position, definition] of file.contents.requests.entries()) { + try { + await this.operations.executeMigrationDefinition(definition); + } catch (error) { + throw new Error( + `Failed to execute query in position [${position}] in migration file [${file.metadata.filename}]`, + { cause: error }, + ); + } + } + } +} diff --git a/implementations/typescript/tests/integrity.test.ts b/implementations/typescript/tests/integrity.test.ts new file mode 100644 index 0000000..eb3052a --- /dev/null +++ b/implementations/typescript/tests/integrity.test.ts @@ -0,0 +1,97 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import type { Client } from "@elastic/elasticsearch"; +import { createEsqueConfiguration } from "../src/configuration.js"; +import type { MigrationRecord } from "../src/elasticsearch/documents.js"; +import { Esque } from "../src/esque.js"; +import type { MigrationFile } from "../src/migration/model.js"; + +const SENTINEL_DATE = "2026-01-01T00:00:00.000Z"; + +// Constructor only stores references — no ES calls happen at init time — so an empty +// object stands in for the client. +function esque(migrationKey = "test-key"): Esque { + return new Esque({} as unknown as Client, createEsqueConfiguration({ migrationKey })); +} + +function verify(instance: Esque, files: MigrationFile[], history: MigrationRecord[]): void { + // TS `private` is compile-time only; element access is the sanctioned escape hatch for tests. + // biome-ignore lint/complexity/useLiteralKeys: dot notation would be a compile error on a private member. + instance["verifyStateIntegrity"](files, history); +} + +function file(version: string, description = "Test", checksum = 42): MigrationFile { + return { + metadata: { filename: `V${version}__${description}.yml`, version, description, checksum }, + contents: { requests: [] }, + }; +} + +function record(f: MigrationFile, order: number, overrides: Partial = {}): MigrationRecord { + return { + migrationKey: "test-key", + order, + filename: f.metadata.filename, + version: f.metadata.version, + description: f.metadata.description, + checksum: f.metadata.checksum, + installedBy: null, + installedOn: SENTINEL_DATE, + executionTime: 0, + ...overrides, + }; +} + +test("passes with no history", () => { + verify(esque(), [file("1.0.0"), file("1.1.0")], []); +}); + +test("passes with complete matching history", () => { + const f1 = file("1.0.0"); + const f2 = file("1.1.0"); + verify(esque(), [f1, f2], [record(f1, 0), record(f2, 1)]); +}); + +test("passes with partial history", () => { + const f1 = file("1.0.0"); + const f2 = file("1.1.0"); + verify(esque(), [f1, f2], [record(f1, 0)]); +}); + +test("throws when more records than files", () => { + const f = file("1.0.0"); + assert.throws(() => verify(esque(), [], [record(f, 0)]), /more migrations/); +}); + +test("throws on gap in history", () => { + const f1 = file("1.0.0"); + const f2 = file("1.1.0"); + const f3 = file("1.2.0"); + assert.throws(() => verify(esque(), [f1, f2, f3], [record(f1, 0), record(f3, 2)]), /corrupt/); +}); + +test("throws on checksum mismatch", () => { + const f = file("1.0.0", "Test", 42); + assert.throws(() => verify(esque(), [f], [record(f, 0, { checksum: 999 })]), /integrity/); +}); + +test("throws on version mismatch", () => { + const f = file("1.0.0"); + assert.throws(() => verify(esque(), [f], [record(f, 0, { version: "9.9.9" })]), /integrity/); +}); + +test("throws on description mismatch", () => { + const f = file("1.0.0", "Original"); + assert.throws(() => verify(esque(), [f], [record(f, 0, { description: "Modified" })]), /integrity/); +}); + +test("throws on migration key mismatch", () => { + const f = file("1.0.0"); + assert.throws(() => verify(esque(), [f], [record(f, 0, { migrationKey: "other-key" })]), /integrity/); +}); + +test("throws when file missing for record", () => { + const f = file("1.0.0"); + const orphan = record(file("1.0.0", "Ghost"), 0); + assert.throws(() => verify(esque(), [f], [orphan]), /could not find/); +}); From 04a771c2ba1fb90ba0842aeea561481ac66cf9d7 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 17:03:26 -0500 Subject: [PATCH 15/26] TypeScript implementation: fix lock-release error shadowing in runMigrations runMigrations previously released the lock in a finally block, which meant a lock-release failure would unconditionally overwrite whatever the try/catch was about to throw (or discard a successful migration entirely). Execution and release outcomes are now captured separately: a release failure never masks a real execution error (and is at least warned about if both fail), and a release-only failure after a successful migration now produces a clear, file-scoped error instead of an unrelated one. Also adds the missing test for the order-mismatch branch of verifyRecordIntegrity's OR condition, which no existing test exercised. Co-Authored-By: Claude Fable 5 --- implementations/typescript/src/esque.ts | 44 ++++++++++++------- .../typescript/tests/integrity.test.ts | 10 +++++ 2 files changed, 38 insertions(+), 16 deletions(-) diff --git a/implementations/typescript/src/esque.ts b/implementations/typescript/src/esque.ts index 7c16658..5be81bb 100644 --- a/implementations/typescript/src/esque.ts +++ b/implementations/typescript/src/esque.ts @@ -100,6 +100,7 @@ export class Esque { private async runMigrations(files: MigrationFile[]): Promise { try { for (const file of files) { + let executionError: unknown; try { if (await this.lock.tryLock(this.configuration.lockTimeoutMinutes)) { const existing = await this.operations.getMigrationRecordForMigrationFile(file); @@ -123,11 +124,35 @@ export class Esque { throw new Error("failed to acquire lock"); } } catch (error) { - throw new Error(`Failed to execute queries in migration file [${file.metadata.filename}]`, { + executionError = new Error(`Failed to execute queries in migration file [${file.metadata.filename}]`, { cause: error, }); - } finally { - await this.unlockIgnoringNotHeld(); + } + + // Never let a release failure silently replace an execution failure (or its success) — + // capture both separately instead of releasing in a `finally` that could throw over + // whatever the try/catch above was about to produce. + let releaseError: unknown; + try { + await this.lock.unlock(); + } catch (error) { + if (!(error instanceof LockNotHeldError)) { + releaseError = error; + } + } + + if (executionError !== undefined) { + if (releaseError !== undefined) { + console.warn( + `failed to release execution lock for migration file [${file.metadata.filename}] after a migration failure. you may need to manually delete the lock document yourself`, + ); + } + throw executionError; + } + if (releaseError !== undefined) { + throw new Error(`Failed to release execution lock after migration file [${file.metadata.filename}]`, { + cause: releaseError, + }); } } } catch (error) { @@ -135,19 +160,6 @@ export class Esque { } } - // Releasing a lock we never acquired (e.g. tryLock() failed) is expected — swallow just that - // case. Split out from runMigrations' finally block so the rethrow isn't a direct statement - // inside a finally clause (which would unsafely shadow the original error's control flow). - private async unlockIgnoringNotHeld(): Promise { - try { - await this.lock.unlock(); - } catch (error) { - if (!(error instanceof LockNotHeldError)) { - throw error; - } - } - } - private async runMigrationForFile(file: MigrationFile): Promise { for (const [position, definition] of file.contents.requests.entries()) { try { diff --git a/implementations/typescript/tests/integrity.test.ts b/implementations/typescript/tests/integrity.test.ts index eb3052a..d7383b6 100644 --- a/implementations/typescript/tests/integrity.test.ts +++ b/implementations/typescript/tests/integrity.test.ts @@ -95,3 +95,13 @@ test("throws when file missing for record", () => { const orphan = record(file("1.0.0", "Ghost"), 0); assert.throws(() => verify(esque(), [f], [orphan]), /could not find/); }); + +test("throws when a record's order does not match the file's position", () => { + const f1 = file("1.0.0"); + const f2 = file("1.1.0"); + // f2 sits at index 1 in `files`, but the record claims order 0. Using order 0 (rather than a + // trailing-gap value) keeps this past the earlier "gap in history" check so it actually + // exercises verifyRecordIntegrity's order comparison. + const swapped = record(f2, 0); + assert.throws(() => verify(esque(), [f1, f2], [swapped]), /integrity/); +}); From 0aeebda1b451f2c12b6eaa731a3968b4323844e9 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 17:09:24 -0500 Subject: [PATCH 16/26] TypeScript implementation: CLI entrypoint Co-Authored-By: Claude Fable 5 --- implementations/typescript/src/cli.ts | 70 +++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) create mode 100644 implementations/typescript/src/cli.ts diff --git a/implementations/typescript/src/cli.ts b/implementations/typescript/src/cli.ts new file mode 100644 index 0000000..56abbe3 --- /dev/null +++ b/implementations/typescript/src/cli.ts @@ -0,0 +1,70 @@ +#!/usr/bin/env node +import { Client } from "@elastic/elasticsearch"; +import { Command, InvalidArgumentError } from "commander"; +import { createEsqueConfiguration } from "./configuration.js"; +import { Esque } from "./esque.js"; + +function collectProperty(value: string, previous: Record): Record { + const separator = value.indexOf("="); + if (separator <= 0) { + throw new InvalidArgumentError(`must be in key=value format, got: '${value}'`); + } + previous[value.slice(0, separator)] = value.slice(separator + 1); + return previous; +} + +function formatErrorChain(error: unknown): string { + if (!(error instanceof Error)) { + return String(error); + } + const messages = [error.message]; + let cause = error.cause; + while (cause instanceof Error) { + messages.push(cause.message); + cause = cause.cause; + } + return messages.join(" -> "); +} + +const program = new Command() + .name("esque") + .description("Run Elasticsearch migrations.") + .requiredOption("--es-url ", "Elasticsearch URL (e.g. http://localhost:9200)") + .requiredOption("--migrations-dir ", "Path to directory containing migration YAML files") + .requiredOption("--migration-key ", "Unique key scoping this migration set") + .option("--migration-user ", "User to record on each migration record") + .option( + "--lock-timeout-minutes ", + "Lock acquisition timeout in minutes", + (value: string) => Number.parseInt(value, 10), + 5, + ) + .option("--property ", "Template substitution property as key=value (repeatable)", collectProperty, {}); + +program.parse(); + +const opts = program.opts<{ + esUrl: string; + migrationsDir: string; + migrationKey: string; + migrationUser?: string; + lockTimeoutMinutes: number; + property: Record; +}>(); + +const configuration = createEsqueConfiguration({ + migrationKey: opts.migrationKey, + migrationUser: opts.migrationUser ?? null, + migrationDirectory: `file:${opts.migrationsDir}`, + lockTimeoutMinutes: opts.lockTimeoutMinutes, +}); + +const esque = new Esque(new Client({ node: opts.esUrl }), configuration, opts.property); +try { + await esque.execute(); +} catch (error) { + console.error(`Error: ${formatErrorChain(error)}`); + process.exitCode = 1; +} finally { + await esque.close(); +} From 76a1ddba73faacf82986f8846893f55581b78a66 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 17:20:20 -0500 Subject: [PATCH 17/26] TypeScript implementation: fix CLI crash on bad URL and hang on invalid lock timeout Co-Authored-By: Claude Fable 5 --- implementations/typescript/src/cli.ts | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/implementations/typescript/src/cli.ts b/implementations/typescript/src/cli.ts index 56abbe3..199e23c 100644 --- a/implementations/typescript/src/cli.ts +++ b/implementations/typescript/src/cli.ts @@ -36,7 +36,13 @@ const program = new Command() .option( "--lock-timeout-minutes ", "Lock acquisition timeout in minutes", - (value: string) => Number.parseInt(value, 10), + (value: string) => { + const parsed = Number.parseInt(value, 10); + if (Number.isNaN(parsed)) { + throw new InvalidArgumentError(`must be an integer, got: '${value}'`); + } + return parsed; + }, 5, ) .option("--property ", "Template substitution property as key=value (repeatable)", collectProperty, {}); @@ -59,12 +65,13 @@ const configuration = createEsqueConfiguration({ lockTimeoutMinutes: opts.lockTimeoutMinutes, }); -const esque = new Esque(new Client({ node: opts.esUrl }), configuration, opts.property); +let esque: Esque | undefined; try { + esque = new Esque(new Client({ node: opts.esUrl }), configuration, opts.property); await esque.execute(); } catch (error) { console.error(`Error: ${formatErrorChain(error)}`); process.exitCode = 1; } finally { - await esque.close(); + await esque?.close(); } From 621ccca4f7df6af25bf98f81124fd03982d0db47 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 17:29:02 -0500 Subject: [PATCH 18/26] Register TypeScript implementation with compatibility harness Co-Authored-By: Claude Fable 5 --- tests/implementations.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/implementations.yml b/tests/implementations.yml index 57b1739..f4eadd2 100644 --- a/tests/implementations.yml +++ b/tests/implementations.yml @@ -6,3 +6,6 @@ implementations: python: invocation: direct command: ["uv", "run", "--project", "implementations/python", "esque"] + typescript: + invocation: direct + command: ["npm", "exec", "--prefix", "implementations/typescript", "--", "tsx", "implementations/typescript/src/cli.ts"] From e9b8cd40b99bb4f75e8cf607b61c5fcec72acfde Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 17:41:45 -0500 Subject: [PATCH 19/26] Add git-tag-based version script for TypeScript Co-Authored-By: Claude Fable 5 --- .github/version_typescript.sh | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100755 .github/version_typescript.sh diff --git a/.github/version_typescript.sh b/.github/version_typescript.sh new file mode 100755 index 0000000..0ab8bc5 --- /dev/null +++ b/.github/version_typescript.sh @@ -0,0 +1,13 @@ +#!/bin/bash +# Produces a SemVer version from git tags — mirrors version_python.sh but for npm. +# Exact tag X.Y.Z → X.Y.Z (release); otherwise → X.Y.Z-dev.N (prerelease). + +GIT_DESCRIBE=$(git describe --tags 2>/dev/null) + +if [[ $GIT_DESCRIBE =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then + echo "$GIT_DESCRIBE" +elif [[ $GIT_DESCRIBE =~ ^([0-9]+\.[0-9]+\.[0-9]+)-([0-9]+)-g[0-9a-f]+ ]]; then + echo "${BASH_REMATCH[1]}-dev.${BASH_REMATCH[2]}" +else + echo "0.0.0-dev.$(git rev-list --count HEAD 2>/dev/null || echo 0)" +fi From 7aa151c5557c7f438a0dc544d8f4a14cb7614186 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 17:50:00 -0500 Subject: [PATCH 20/26] Add TypeScript job to CI Co-Authored-By: Claude Fable 5 --- .github/workflows/ci.yml | 42 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ad2532b..cb36bc5 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -63,9 +63,44 @@ jobs: env: UV_PUBLISH_TOKEN: ${{ secrets.PYPI_TOKEN }} + typescript: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + with: + fetch-depth: 0 + - uses: actions/setup-node@v6 + with: + node-version: 24 + registry-url: https://registry.npmjs.org + - working-directory: implementations/typescript + run: npm ci + - working-directory: implementations/typescript + run: npx biome ci src tests + - working-directory: implementations/typescript + run: npm run typecheck + - working-directory: implementations/typescript + run: npm test + - working-directory: implementations/typescript + run: npm version --no-git-tag-version "$(../../.github/version_typescript.sh)" + - working-directory: implementations/typescript + run: npm run build + - working-directory: implementations/typescript + run: node dist/cli.js --help + - if: github.event_name != 'release' + working-directory: implementations/typescript + run: npm publish --tag dev + env: + NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }} + - if: github.event_name == 'release' + working-directory: implementations/typescript + run: npm publish + env: + NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }} + compatibility-tests: runs-on: ubuntu-latest - needs: [jvm, python] + needs: [jvm, python, typescript] steps: - uses: actions/checkout@v7 with: @@ -78,5 +113,10 @@ jobs: with: python-version: "3.14" - run: uv sync --project implementations/python + - uses: actions/setup-node@v6 + with: + node-version: 24 + - working-directory: implementations/typescript + run: npm ci - working-directory: tests run: uv run pytest . -v From 369be12b5cccf1460b3a1458c034717427a7a192 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 18:00:40 -0500 Subject: [PATCH 21/26] Add TypeScript checks to pre-commit hook and Node to dev container Co-Authored-By: Claude Fable 5 --- .devcontainer/Dockerfile | 7 +++++++ .githooks/pre-commit | 9 ++++++--- 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile index 176e468..f7543eb 100644 --- a/.devcontainer/Dockerfile +++ b/.devcontainer/Dockerfile @@ -24,6 +24,13 @@ RUN curl -s "https://get.sdkman.io" | bash \ # install python things RUN curl -LsSf https://astral.sh/uv/install.sh | sh +# install node things +ARG NODE_VERSION=24 +RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.5/install.sh | bash \ + && source ${HOME}/.nvm/nvm.sh \ + && nvm install ${NODE_VERSION} \ + && nvm alias default ${NODE_VERSION} + # install github cli RUN DEBIAN_FRONTEND=noninteractive \ && (type -p wget >/dev/null || (apt update && apt install wget -y)) \ diff --git a/.githooks/pre-commit b/.githooks/pre-commit index 76d5908..9e86e7f 100755 --- a/.githooks/pre-commit +++ b/.githooks/pre-commit @@ -10,13 +10,16 @@ fi echo "Running pre-commit checks..." -echo "[1/3] JVM: format, lint, test" +echo "[1/4] JVM: format, lint, test" (cd implementations/jvm && ./gradlew ktfmtCheck detekt test) -echo "[2/3] Python: format, lint, typecheck" +echo "[2/4] Python: format, lint, typecheck" (cd implementations/python && uv run ruff check src/esque/ && uv run ruff format --check src/esque/ && uv run pyright src/esque/) -echo "[3/3] Compatibility tests" +echo "[3/4] TypeScript: format, lint, typecheck, test" +(cd implementations/typescript && npx biome ci src tests && npm run typecheck && npm test) + +echo "[4/4] Compatibility tests" (cd tests && uv run pytest . -q) echo "Pre-commit checks passed." From 07eaf32a990a3da3d119de3e20b1e80493d72327 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 18:16:53 -0500 Subject: [PATCH 22/26] Fix pre-commit hook: use npm run lint to avoid npx biome false-pass Co-Authored-By: Claude Fable 5 --- .githooks/pre-commit | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.githooks/pre-commit b/.githooks/pre-commit index 9e86e7f..cb0a977 100755 --- a/.githooks/pre-commit +++ b/.githooks/pre-commit @@ -16,8 +16,9 @@ echo "[1/4] JVM: format, lint, test" echo "[2/4] Python: format, lint, typecheck" (cd implementations/python && uv run ruff check src/esque/ && uv run ruff format --check src/esque/ && uv run pyright src/esque/) +# Requires a one-time `npm install` in implementations/typescript/ (no auto-sync like uv/gradlew). echo "[3/4] TypeScript: format, lint, typecheck, test" -(cd implementations/typescript && npx biome ci src tests && npm run typecheck && npm test) +(cd implementations/typescript && npm run lint && npm run typecheck && npm test) echo "[4/4] Compatibility tests" (cd tests && uv run pytest . -q) From d2661a0381e69eda8a9c136eb89e9822a328e2b7 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 18:23:10 -0500 Subject: [PATCH 23/26] Document TypeScript implementation in CLAUDE.md Adds the TypeScript implementation to the project overview, repository structure, build commands, CI/CD, code conventions, and compatibility test harness sections. Also corrects a stale claim that the Python implementation uses httpx instead of the official elasticsearch client. Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 114 +++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 91 insertions(+), 23 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 6b1c6cf..7d49408 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -9,10 +9,11 @@ Specs live in `.claude/superpowers/specs/` named `YYYY-MM-DD--design.md`. **Esque** (**E**lasticsearch **S**tateful **Qu**ery **E**xecutor) is a migration management library for Elasticsearch, similar to Flyway but for ES clusters. It executes pre-defined queries in order, tracks which have been applied, validates integrity, and supports distributed locking for safe concurrent execution. - **License:** Apache 2.0 -- **Implementations:** JVM (Kotlin 2.4.0, Java 21) · Python 3.14 +- **Implementations:** JVM (Kotlin 2.4.0, Java 21) · Python 3.14 · TypeScript (Node 22+) - **Target:** Elasticsearch 9+ (ES 9.4.x REST API) - **JVM published to:** Maven Central as `org.loesak.esque:esque` - **Python published to:** PyPI as `esque-py` +- **TypeScript published to:** npm as `esque-ts` ## Repository Structure @@ -20,10 +21,11 @@ Specs live in `.claude/superpowers/specs/` named `YYYY-MM-DD--design.md`. esque/ ├── setup-hooks.sh # One-time dev setup: activates git pre-commit hook ├── .githooks/ -│ └── pre-commit # [1] JVM checks [2] Python checks [3] compat tests +│ └── pre-commit # [1] JVM checks [2] Python checks [3] TypeScript checks [4] compat tests ├── .github/ │ ├── version_jvm.sh # Git-tag-based version for JVM (X.Y.Z or X.Y.Z-...-SNAPSHOT) │ ├── version_python.sh # PEP 440 version for Python (X.Y.Z or X.Y.Z.devN) +│ ├── version_typescript.sh # SemVer version for TypeScript (X.Y.Z or X.Y.Z-dev.N) │ └── workflows/ │ └── ci.yml # lint + build + publish + compatibility-tests ├── .devcontainer/ # Dev container (Ubuntu, Zulu JDK 21) @@ -52,26 +54,48 @@ esque/ │ │ ├── MigrationFileLoader.kt │ │ ├── MigrationTemplateResolver.kt │ │ └── model/MigrationFile.kt -│ └── python/ # Python implementation -│ ├── pyproject.toml # uv project: click, httpx, pyyaml; hatchling build -│ ├── src/esque/ -│ │ ├── configuration.py # EsqueConfiguration dataclass -│ │ ├── esque.py # Main orchestrator + verify_integrity -│ │ ├── cli.py # Click CLI entrypoint -│ │ ├── __main__.py # python -m esque shim +│ ├── python/ # Python implementation +│ │ ├── pyproject.toml # uv project: click, elasticsearch, pyyaml; hatchling build +│ │ ├── src/esque/ +│ │ │ ├── configuration.py # EsqueConfiguration dataclass +│ │ │ ├── esque.py # Main orchestrator + verify_integrity +│ │ │ ├── cli.py # Click CLI entrypoint +│ │ │ ├── __main__.py # python -m esque shim +│ │ │ ├── elasticsearch/ +│ │ │ │ ├── documents.py # INDEX_DEFINITION, constants +│ │ │ │ ├── operations.py # ES REST calls +│ │ │ │ └── lock.py # Distributed lock (op_type=create polling) +│ │ │ └── migration/ +│ │ │ ├── model.py # MigrationRequest, MigrationFile +│ │ │ ├── template.py # #{varName} validation and substitution +│ │ │ └── loader.py # File discovery, parsing, checksum +│ │ └── tests/ +│ │ ├── test_model.py # Version ordering +│ │ ├── test_checksum.py # Canonical checksum algorithm +│ │ ├── test_template.py # Template validation and substitution +│ │ └── test_integrity.py # verify_integrity error scenarios +│ └── typescript/ # TypeScript implementation +│ ├── package.json # npm project: commander, @elastic/elasticsearch, yaml; tsc build +│ ├── src/ +│ │ ├── configuration.ts # EsqueConfiguration type +│ │ ├── esque.ts # Main orchestrator + verifyIntegrity +│ │ ├── cli.ts # commander CLI entrypoint │ │ ├── elasticsearch/ -│ │ │ ├── documents.py # INDEX_DEFINITION, constants -│ │ │ ├── operations.py # ES REST calls -│ │ │ └── lock.py # Distributed lock (op_type=create polling) +│ │ │ ├── documents.ts # INDEX_DEFINITION, constants +│ │ │ ├── operations.ts # ES REST calls +│ │ │ └── lock.ts # Distributed lock (op_type=create polling) │ │ └── migration/ -│ │ ├── model.py # MigrationRequest, MigrationFile -│ │ ├── template.py # #{varName} validation and substitution -│ │ └── loader.py # File discovery, parsing, checksum +│ │ ├── model.ts # MigrationRequest, MigrationFile +│ │ ├── template.ts # #{varName} validation and substitution +│ │ └── loader.ts # File discovery, parsing, checksum │ └── tests/ -│ ├── test_model.py # Version ordering -│ ├── test_checksum.py # Canonical checksum algorithm -│ ├── test_template.py # Template validation and substitution -│ └── test_integrity.py # verify_integrity error scenarios +│ ├── model.test.ts # Version ordering +│ ├── checksum.test.ts # Canonical checksum algorithm +│ ├── template.test.ts # Template validation and substitution +│ ├── integrity.test.ts # verifyIntegrity error scenarios +│ ├── loader.test.ts # File discovery and parsing +│ ├── lock.test.ts # Distributed lock behavior +│ └── documents.test.ts # ES document (de)serialization └── tests/ # Black-box compatibility test harness ├── pyproject.toml # uv project: pytest, testcontainers, httpx, pyyaml ├── implementations.yml # Registered implementations with invocation config @@ -93,6 +117,7 @@ esque/ - Java 21 (Zulu distribution recommended) - uv (Python package manager — `curl -LsSf https://astral.sh/uv/install.sh | sh`) +- Node.js 22+ (24 recommended) and npm - Docker (for integration tests and compatibility tests via testcontainers) ### First-time setup @@ -150,6 +175,33 @@ uv run pyright esque/ uv run esque --help ``` +### TypeScript Commands (run from `implementations/typescript/`) + +```bash +cd implementations/typescript + +# install dependencies (required once — no auto-sync like uv/gradlew) +npm install + +# format code +npm run format + +# check formatting and lint +npm run lint + +# typecheck +npm run typecheck + +# run unit tests +npm test + +# build (emits dist/) +npm run build + +# run the CLI +npx tsx src/cli.ts --help +``` + ### Compatibility Tests (run from `tests/`) ```bash @@ -193,7 +245,8 @@ A single **`ci.yml`** handles everything — checks, publishing, and compatibili - **Triggers**: push to `master` · PRs to `master` · published GitHub releases - **`jvm`**: ktfmtCheck + detekt + build + publish on every build. vanniktech plugin routes automatically — `*-SNAPSHOT` versions go to OSSRH snapshots, release versions go to Maven Central staging. - **`python`**: ruff + pyright + build + publish on every build. Version is computed by `.github/version_python.sh` (PEP 440: `X.Y.Z` on exact tag, `X.Y.Z.devN` otherwise) and patched into `pyproject.toml` before building. Non-release builds publish to TestPyPI (`TEST_PYPI_TOKEN`); release builds publish to PyPI (`PYPI_TOKEN`). -- **`compatibility-tests`**: needs `jvm` + `python`; runs 34 pytest scenarios via testcontainers. +- **`typescript`**: Biome + tsc + node:test + build + a compiled dist/cli.js smoke test, then publish on every build. Version from `.github/version_typescript.sh` (SemVer: `X.Y.Z` on exact tag, `X.Y.Z-dev.N` otherwise) patched into `package.json` before building. Non-release builds publish to npm under the `dev` dist-tag (`NPM_TOKEN` secret); release builds publish to `latest`. +- **`compatibility-tests`**: needs `jvm` + `python` + `typescript`; runs 52 pytest scenarios (17 × 3 implementations + 1 cross-implementation equivalency test) via testcontainers. ## Architecture @@ -303,10 +356,21 @@ Uses ES `op_type=create` for cross-process atomicity. The JVM also wraps this wi - **Package layout**: `src/esque/` with modules mirroring the JVM structure — `esque.py` (orchestrator), `configuration.py`, `cli.py`, `elasticsearch/` (documents, operations, lock), `migration/` (model, template, loader) - **Entry point**: `esque.cli:main`; `__main__.py` is a thin shim for `python -m esque` - **Strict typing**: all functions annotated; `cast()` used where isinstance-narrowing produces Unknown; `field(default_factory=lambda: [])` instead of `field(default_factory=list)` to satisfy pyright strict -- **httpx**: ES REST calls (not elasticsearch-py, to avoid client version compatibility issues) +- **elasticsearch**: official Python ES client (>=9) — handles auth mechanisms, retries, and typed responses - **PyYAML**: migration file parsing - **Click**: CLI with the same option names as the JVM Clikt interface +## TypeScript Code Conventions + +- **Package layout**: `src/` mirrors the module structure used by JVM/Python — `esque.ts` (orchestrator), `configuration.ts`, `cli.ts`, `elasticsearch/` (documents, operations, lock), `migration/` (model, template, loader) +- **Module system**: ESM-only (`"type": "module"` in package.json), strict TypeScript, Node.js 22+ +- **Formatting/Linting**: [Biome](https://biomejs.dev/) — one tool for both, analogous to ruff. Run `npm run format` to auto-format, `npm run lint` (or `npx biome ci src tests`, once dependencies are installed) to check. +- **Type checking**: TypeScript in `strict` mode with `noUncheckedIndexedAccess` +- **@elastic/elasticsearch**: official TypeScript ES client (same choice as Python and JVM — needed for auth mechanisms, retries, and typed responses; a plain HTTP client was considered and rejected for the same reasons Python rejected it) +- **commander**: CLI framework with the same option names as the Python Click / JVM Clikt interfaces +- **yaml**: migration file parsing +- **Unit tests**: `node:test`, run via `tsx` (no build step required) — covers version ordering, template resolution, canonical checksum (including a pinned cross-implementation reference vector), integrity verification, distributed lock behavior, and ES document (de)serialization + ## Testing ### JVM Integration Tests @@ -329,7 +393,8 @@ Lives in `tests/` as a standalone uv project. Each test invokes an implementatio - **Fixture**: one session-scoped ES container (`ElasticSearchContainer`), cleaned between tests with `DELETE /.esque` and `DELETE /test-*` - **Parametrized**: every test function is parametrized over `all_implementations()` which reads `tests/implementations.yml` -- **Adding a new implementation**: add an entry to `implementations.yml` with `invocation: direct` and a `command` list; tests run automatically +- **Scenario count**: 17 parametrized scenarios × 3 implementations + 1 cross-implementation equivalency test = 52 pytest cases +- **Adding a new implementation**: add an entry to `implementations.yml` with `invocation: direct` and a `command` list; tests run automatically. TypeScript runs via `tsx` directly against `src/cli.ts` with no build step required (only `npm ci` beforehand needed), analogous to how `uv run` auto-syncs for Python. ### Registered Implementations (`tests/implementations.yml`) @@ -342,11 +407,14 @@ implementations: python: invocation: direct command: ["uv", "run", "--project", "implementations/python", "esque"] + typescript: + invocation: direct + command: ["npm", "exec", "--prefix", "implementations/typescript", "--", "tsx", "implementations/typescript/src/cli.ts"] ``` ## Known TODOs in Code -- Differentiate lock creation failure vs. lock-already-exists (JVM `RestClientOperations`) +- Differentiate lock creation failure vs. lock-already-exists (present in all three implementations: JVM `RestClientOperations`, Python `lock.py`, TypeScript `lock.ts`) - Configurable lock timeout for long-running queries (`Esque.kt`) - Consider writing "FAILED" migration records (`Esque.kt`) - Rollback/undo capability From 889802d865296c4a5e991d24da03793fecbef460 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 18:29:41 -0500 Subject: [PATCH 24/26] Fix stale 'both implementations' language in Architecture section now that TypeScript exists Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 7d49408..04cd067 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -252,7 +252,7 @@ A single **`ci.yml`** handles everything — checks, publishing, and compatibili ### Execution Flow -Both implementations perform the same sequence: +All three implementations perform the same sequence: 1. **Initialize** — Create the `.esque` index in ES if it doesn't exist 2. **Load** — Discover and parse YAML migration files from the migrations directory @@ -284,14 +284,14 @@ Both implementations perform the same sequence: ### Checksum Algorithm -Both implementations must produce identical checksums for the same resolved migration content: +All three implementations must produce identical checksums for the same resolved migration content: 1. Serialize the resolved request list as JSON: `{"requests": [{...}, ...]}` with keys sorted alphabetically and null fields omitted 2. Encode as UTF-8 3. Compute MD5 digest 4. Take the first 4 bytes interpreted as a big-endian signed 32-bit integer -This is the canonical algorithm since Phase 3. The JVM uses `JSON_MAPPER_CANONICAL` (Jackson with `ORDER_MAP_ENTRIES_BY_KEYS` + `NON_NULL`). Python uses `json.dumps(sort_keys=True, separators=(',', ':'))` after recursively removing None values. +This is the canonical algorithm since Phase 3. The JVM uses `JSON_MAPPER_CANONICAL` (Jackson with `ORDER_MAP_ENTRIES_BY_KEYS` + `NON_NULL`). Python uses `json.dumps(sort_keys=True, separators=(',', ':'))` after recursively removing None values. TypeScript uses a hand-written `canonicalJson` serializer (sorted keys, null/undefined object values dropped, compact separators) before MD5-hashing. ### ES Document Structure @@ -331,7 +331,7 @@ V{VERSION}__{DESCRIPTION}.yml ### Distributed Locking -Uses ES `op_type=create` for cross-process atomicity. The JVM also wraps this with a local `ReentrantLock` for thread safety. Python polls at 100ms intervals. Both default to a 5-minute timeout. +Uses ES `op_type=create` for cross-process atomicity. The JVM also wraps this with a local `ReentrantLock` for thread safety. Python polls at 100ms intervals. TypeScript polls at 100ms intervals like Python, using a boolean held-flag instead of a real mutex since Node is single-threaded. All three default to a 5-minute timeout. ## JVM Code Conventions From a60ba35b30b7fac8b812507b55807441d617c923 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 18:38:07 -0500 Subject: [PATCH 25/26] Restructure TypeScript docs to match JVM/Python section pattern Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 04cd067..0acd0ca 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -9,7 +9,7 @@ Specs live in `.claude/superpowers/specs/` named `YYYY-MM-DD--design.md`. **Esque** (**E**lasticsearch **S**tateful **Qu**ery **E**xecutor) is a migration management library for Elasticsearch, similar to Flyway but for ES clusters. It executes pre-defined queries in order, tracks which have been applied, validates integrity, and supports distributed locking for safe concurrent execution. - **License:** Apache 2.0 -- **Implementations:** JVM (Kotlin 2.4.0, Java 21) · Python 3.14 · TypeScript (Node 22+) +- **Implementations:** JVM (Kotlin 2.4.0, Java 21) · Python 3.14 · TypeScript 5.x (Node 22+) - **Target:** Elasticsearch 9+ (ES 9.4.x REST API) - **JVM published to:** Maven Central as `org.loesak.esque:esque` - **Python published to:** PyPI as `esque-py` @@ -180,7 +180,7 @@ uv run esque --help ```bash cd implementations/typescript -# install dependencies (required once — no auto-sync like uv/gradlew) +# install dependencies npm install # format code @@ -238,6 +238,11 @@ Version is derived from git tags via `version.sh`: - **Formatting/Linting**: [ruff](https://docs.astral.sh/ruff/) — Black-compatible, line-length 120. Run `uv run ruff format esque/` to auto-format. - **Type checking**: [pyright](https://github.com/microsoft/pyright) in `strict` mode + [ty](https://github.com/astral-sh/ty) with all warn-level rules escalated to errors. All code must be fully annotated. +### TypeScript Code Style and Typing + +- **Formatting/Linting**: [Biome](https://biomejs.dev/) — one tool for both, analogous to ruff. Run `npm run format` to auto-format, `npm run lint` to check (requires `npm install` once beforehand). +- **Type checking**: TypeScript in `strict` mode with `noUncheckedIndexedAccess`. Run `npm run typecheck`. + ### CI/CD A single **`ci.yml`** handles everything — checks, publishing, and compatibility tests: @@ -363,13 +368,10 @@ Uses ES `op_type=create` for cross-process atomicity. The JVM also wraps this wi ## TypeScript Code Conventions - **Package layout**: `src/` mirrors the module structure used by JVM/Python — `esque.ts` (orchestrator), `configuration.ts`, `cli.ts`, `elasticsearch/` (documents, operations, lock), `migration/` (model, template, loader) -- **Module system**: ESM-only (`"type": "module"` in package.json), strict TypeScript, Node.js 22+ -- **Formatting/Linting**: [Biome](https://biomejs.dev/) — one tool for both, analogous to ruff. Run `npm run format` to auto-format, `npm run lint` (or `npx biome ci src tests`, once dependencies are installed) to check. -- **Type checking**: TypeScript in `strict` mode with `noUncheckedIndexedAccess` +- **Module system**: ESM-only (`"type": "module"` in package.json), Node.js 22+ - **@elastic/elasticsearch**: official TypeScript ES client (same choice as Python and JVM — needed for auth mechanisms, retries, and typed responses; a plain HTTP client was considered and rejected for the same reasons Python rejected it) - **commander**: CLI framework with the same option names as the Python Click / JVM Clikt interfaces - **yaml**: migration file parsing -- **Unit tests**: `node:test`, run via `tsx` (no build step required) — covers version ordering, template resolution, canonical checksum (including a pinned cross-implementation reference vector), integrity verification, distributed lock behavior, and ES document (de)serialization ## Testing @@ -387,6 +389,19 @@ Live in `implementations/python/tests/`. Pure unit tests (no ES), covering the m Run via `uv run pytest` from `implementations/python/`. +### TypeScript Unit Tests + +Live in `implementations/typescript/tests/`. Pure unit tests (no ES), covering the most complex logic: +- `model.test.ts` — numeric version ordering (`1.9.0 < 1.10.0`) +- `template.test.ts` — `#{varName}` validation and substitution across all request fields +- `checksum.test.ts` — canonical checksum algorithm properties, including a pinned cross-implementation reference vector +- `loader.test.ts` — migration file discovery, ordering, and fail-loud validation of malformed YAML +- `lock.test.ts` — distributed lock acquisition/release/timeout behavior +- `documents.test.ts` — ES document (de)serialization, including fail-loud validation of malformed records +- `integrity.test.ts` — all `verifyStateIntegrity` error scenarios + +Run via `npm test` from `implementations/typescript/` (uses `node:test` via `tsx`, no build step required). + ### Compatibility Test Harness Lives in `tests/` as a standalone uv project. Each test invokes an implementation as a subprocess via its CLI, then queries ES directly via httpx to verify state. From 166971f374007f800965843d9b5914eca13b73e0 Mon Sep 17 00:00:00 2001 From: Aaron Loes Date: Sat, 4 Jul 2026 18:44:38 -0500 Subject: [PATCH 26/26] Fix stale verify_integrity method name and add TypeScript compat test example Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 0acd0ca..6da358e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -58,7 +58,7 @@ esque/ │ │ ├── pyproject.toml # uv project: click, elasticsearch, pyyaml; hatchling build │ │ ├── src/esque/ │ │ │ ├── configuration.py # EsqueConfiguration dataclass -│ │ │ ├── esque.py # Main orchestrator + verify_integrity +│ │ │ ├── esque.py # Main orchestrator + _verify_state_integrity │ │ │ ├── cli.py # Click CLI entrypoint │ │ │ ├── __main__.py # python -m esque shim │ │ │ ├── elasticsearch/ @@ -73,12 +73,12 @@ esque/ │ │ ├── test_model.py # Version ordering │ │ ├── test_checksum.py # Canonical checksum algorithm │ │ ├── test_template.py # Template validation and substitution -│ │ └── test_integrity.py # verify_integrity error scenarios +│ │ └── test_integrity.py # _verify_state_integrity error scenarios │ └── typescript/ # TypeScript implementation │ ├── package.json # npm project: commander, @elastic/elasticsearch, yaml; tsc build │ ├── src/ │ │ ├── configuration.ts # EsqueConfiguration type -│ │ ├── esque.ts # Main orchestrator + verifyIntegrity +│ │ ├── esque.ts # Main orchestrator + verifyStateIntegrity │ │ ├── cli.ts # commander CLI entrypoint │ │ ├── elasticsearch/ │ │ │ ├── documents.ts # INDEX_DEFINITION, constants @@ -92,7 +92,7 @@ esque/ │ ├── model.test.ts # Version ordering │ ├── checksum.test.ts # Canonical checksum algorithm │ ├── template.test.ts # Template validation and substitution -│ ├── integrity.test.ts # verifyIntegrity error scenarios +│ ├── integrity.test.ts # verifyStateIntegrity error scenarios │ ├── loader.test.ts # File discovery and parsing │ ├── lock.test.ts # Distributed lock behavior │ └── documents.test.ts # ES document (de)serialization @@ -213,6 +213,7 @@ uv run pytest . -v # run against a specific implementation only uv run pytest . -v -k "jvm" uv run pytest . -v -k "python" +uv run pytest . -v -k "typescript" ``` ### GPG Signing (JVM) @@ -385,7 +386,7 @@ Live in `implementations/python/tests/`. Pure unit tests (no ES), covering the m - `test_model.py` — numeric version ordering (`1.9.0 < 1.10.0`) - `test_checksum.py` — canonical checksum algorithm properties - `test_template.py` — `#{varName}` validation and substitution across all request fields -- `test_integrity.py` — all `verify_integrity` error scenarios +- `test_integrity.py` — all `_verify_state_integrity` error scenarios Run via `uv run pytest` from `implementations/python/`.