Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,28 @@ commands.
- [docs/configuration.md](docs/configuration.md) — env vars + strategy file layout
- [docs/test-specification.md](docs/test-specification.md) — test plan (236 tests)

## E2E live tests — preflight gate

`npm run test:e2e-live` runs against the testnet Nostr relay, L3 aggregator,
IPFS gateway, Fulcrum, and Market API. Before any container is spawned, the
suite probes those services via `@unicitylabs/infra-probe` and aborts up-front
if any are unreachable — saving 10–15-minute container-spawn cycles that would
otherwise surface infra outages as opaque test timeouts.

| Env var | Default | Effect |
|---|---|---|
| `TRADER_E2E_SKIP_PREFLIGHT` | unset | Set to `1` to bypass the gate entirely (escape hatch when iterating on TS/test-only changes that don't need infra) |
| `TRADER_E2E_PREFLIGHT_STRICT` | unset | Set to `1` to also fail on `degraded` (default warns and proceeds — e2e timeouts absorb mild slowness) |
| `TRADER_E2E_PREFLIGHT_NETWORK` | `testnet` | One of `testnet`, `mainnet`, `dev`. Other values throw at startup |
| `TRADER_E2E_PREFLIGHT_TIMEOUT_MS` | `30000` | Per-probe ceiling. Must be a positive finite number; NaN/0/negative throw at startup |

Ad-hoc probing without invoking vitest:

```sh
npm run preflight # pretty-printed
npm run preflight:json # one-line JSON for scripting
```

## Status

Restored from `pre-trader-cut-v1` tag of agentic-hosting (Phase b decoupling).
Expand Down
69 changes: 69 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@
"test:watch": "vitest",
"test:coverage": "vitest run --coverage",
"test:e2e-live": "vitest run --config vitest.e2e-live.config.ts",
"preflight": "unicity-infra-probe --network testnet",
"preflight:json": "unicity-infra-probe --network testnet --format json",
"lint": "eslint .",
"typecheck": "tsc --noEmit && tsc --noEmit -p tsconfig.test.json"
},
Expand All @@ -28,6 +30,7 @@
"@types/uuid": "^10.0.0",
"@typescript-eslint/eslint-plugin": "^8.24.0",
"@typescript-eslint/parser": "^8.24.0",
"@unicitylabs/infra-probe": "^0.3.0",
"@vitest/coverage-v8": "^3.0.0",
"eslint": "^9.20.0",
"tsup": "^8.4.0",
Expand Down
14 changes: 14 additions & 0 deletions test/e2e-live/global-setup.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
/**
* Vitest globalSetup for the e2e-live suite.
*
* Runs ONCE before any test file. If the preflight throws, vitest aborts
* the entire run before spawning any Docker containers — saving the
* 10-15-minute round trip we'd otherwise eat on a relay outage or
* unreachable aggregator.
*/

import { runPreflight } from './preflight.js';

export async function setup(): Promise<void> {
await runPreflight();
}
17 changes: 17 additions & 0 deletions test/e2e-live/infra-probe.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
/**
* Ambient declaration for @unicitylabs/infra-probe — the package ships
* pure-ESM .mjs without bundled .d.ts. Only typed to the surface our
* preflight uses; the full report shape is re-typed locally in preflight.ts
* with `as Report` so changes upstream surface as type errors there.
*/
declare module '@unicitylabs/infra-probe' {
export interface ProbeOptions {
network?: 'testnet' | 'mainnet' | 'dev';
only?: string[];
timeoutMs?: number;
aggregatorApiKey?: string;
}
export function runProbes(options?: ProbeOptions): Promise<unknown>;
export function exitCodeForReport(report: unknown): number;
export const SERVICES: readonly string[];
}
158 changes: 158 additions & 0 deletions test/e2e-live/preflight.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
/**
* Preflight infrastructure check for the e2e-live suite.
*
* Wraps `@unicitylabs/infra-probe` to verify that every Unicity Network
* service the e2e-live tests depend on (Nostr relay, L3 Aggregator, IPFS
* gateway, L1 Fulcrum, Market API) is reachable and functional BEFORE
* spawning any Docker containers or burning faucet quota.
*
* Failure modes the probe catches that the e2e suite would otherwise hit
* as opaque timeouts 5-15 minutes deep into a run:
* - Nostr relay silently dropping kind:30078 / kind:1059 publishes
* (the symptom of the 2026-04-30 outage that motivated this gate)
* - Aggregator API key rejection / rate-limit
* - IPFS gateway 5xx
* - Fulcrum chain tip stale (L1 stuck) → token-create races
* - Market API down → no intent discovery at all
*
* Environment knobs:
* TRADER_E2E_SKIP_PREFLIGHT=1 — bypass the gate entirely (escape hatch)
* TRADER_E2E_PREFLIGHT_STRICT=1 — also fail on `degraded` (default: warn-only)
* TRADER_E2E_PREFLIGHT_NETWORK — override the network (default: testnet)
* TRADER_E2E_PREFLIGHT_TIMEOUT_MS — per-probe ceiling (default: 30000)
*/

import { runProbes } from '@unicitylabs/infra-probe';

interface Check {
readonly name: string;
readonly status: 'pass' | 'warn' | 'fail';
readonly latencyMs: number;
readonly message: string;
}

interface Service {
readonly service: string;
readonly endpoint: string;
readonly status: 'healthy' | 'degraded' | 'unreachable' | 'error';
readonly latencyMs: number;
readonly checks: Check[];
readonly error?: string;
}

interface Report {
readonly services: Service[];
readonly summary: {
readonly total: number;
readonly healthy: number;
readonly degraded: number;
readonly unreachable: number;
};
}

function statusIcon(status: Service['status']): string {
if (status === 'healthy') return '✓';
if (status === 'degraded') return '⚠';
return '✗';
}

function checkIcon(status: Check['status']): string {
if (status === 'pass') return '✓';
if (status === 'warn') return '⚠';
return '✗';
}

function logReport(report: Report): void {
for (const svc of report.services) {
console.log(
`[preflight] ${statusIcon(svc.status)} ${svc.service.padEnd(11)} ${svc.endpoint} (${svc.status}, ${svc.latencyMs}ms)`,
);
if (svc.error) {
console.log(`[preflight] error: ${svc.error}`);
}
for (const c of svc.checks) {
const icon = checkIcon(c.status);
console.log(`[preflight] ${icon} ${c.name.padEnd(20)} ${c.latencyMs}ms ${c.message}`);
}
}
const { total, healthy, degraded, unreachable } = report.summary;
console.log(
`[preflight] summary: ${healthy}/${total} healthy, ${degraded} degraded, ${unreachable} unreachable`,
);
}

export async function runPreflight(): Promise<void> {
if (process.env['TRADER_E2E_SKIP_PREFLIGHT'] === '1') {
console.log('[preflight] SKIPPED (TRADER_E2E_SKIP_PREFLIGHT=1)');
return;
}

// Validate network enum locally rather than blind-cast. The upstream
// `runProbes` would also throw on an unknown network, but tightening here
// makes the contract local and immune to upstream silent enum extensions.
const VALID_NETWORKS = ['testnet', 'mainnet', 'dev'] as const;
type Network = (typeof VALID_NETWORKS)[number];
const rawNetwork = process.env['TRADER_E2E_PREFLIGHT_NETWORK'] ?? 'testnet';
if (!(VALID_NETWORKS as readonly string[]).includes(rawNetwork)) {
throw new Error(
`Preflight: invalid TRADER_E2E_PREFLIGHT_NETWORK="${rawNetwork}". ` +
`Must be one of: ${VALID_NETWORKS.join(', ')}.`,
);
}
const network = rawNetwork as Network;

// Validate timeoutMs. `Number('abc')` returns NaN; `Number('-1')` returns -1;
// `Number('0')` returns 0. All of these would propagate to the upstream
// probe's setTimeout and either fire immediately (NaN coerces to 1ms in Node)
// or never fire (0 = no timeout in setTimeout's contract). Both produce
// misleading "preflight failed" results from a typo. Reject loudly.
const rawTimeoutMs = process.env['TRADER_E2E_PREFLIGHT_TIMEOUT_MS'] ?? '30000';
const timeoutMs = Number(rawTimeoutMs);
if (!Number.isFinite(timeoutMs) || timeoutMs <= 0) {
throw new Error(
`Preflight: invalid TRADER_E2E_PREFLIGHT_TIMEOUT_MS="${rawTimeoutMs}". ` +
`Must be a positive finite number (milliseconds).`,
);
}

const strict = process.env['TRADER_E2E_PREFLIGHT_STRICT'] === '1';

console.log(
`[preflight] probing ${network} infrastructure (timeout=${timeoutMs}ms, strict=${strict})...`,
);
const startedAt = Date.now();

const report = (await runProbes({ network, timeoutMs })) as Report;
const elapsed = Date.now() - startedAt;

logReport(report);
console.log(`[preflight] completed in ${elapsed}ms`);

const { unreachable, degraded } = report.summary;
const downServices = report.services
.filter((s) => s.status === 'unreachable' || s.status === 'error')
.map((s) => `${s.service}=${s.status}`);
const slowServices = report.services
.filter((s) => s.status === 'degraded')
.map((s) => `${s.service}=${s.status}`);

if (unreachable > 0) {
throw new Error(
`Preflight failed: ${unreachable} service(s) unreachable [${downServices.join(', ')}]. ` +
`Set TRADER_E2E_SKIP_PREFLIGHT=1 to bypass (not recommended — tests will likely hang).`,
);
}

if (degraded > 0) {
if (strict) {
throw new Error(
`Preflight failed (strict mode): ${degraded} service(s) degraded [${slowServices.join(', ')}]. ` +
`Unset TRADER_E2E_PREFLIGHT_STRICT or set TRADER_E2E_SKIP_PREFLIGHT=1 to bypass.`,
);
}
console.warn(
`[preflight] WARNING: ${degraded} service(s) degraded [${slowServices.join(', ')}] — ` +
`tests may be slow or intermittently fail. Set TRADER_E2E_PREFLIGHT_STRICT=1 to fail-fast on this.`,
);
}
}
6 changes: 6 additions & 0 deletions vitest.e2e-live.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,12 @@ import { defineConfig } from 'vitest/config';
export default defineConfig({
test: {
include: ['test/e2e-live/**/*.test.ts'],
// Run @unicitylabs/infra-probe before any test file. Aborts the run
// up-front if the testnet Nostr relay / aggregator / IPFS / Fulcrum /
// market is unreachable, instead of consuming a 10-15-minute container
// spawn cycle to discover the same failure as an opaque timeout.
// Bypass: TRADER_E2E_SKIP_PREFLIGHT=1.
globalSetup: ['./test/e2e-live/global-setup.ts'],
testTimeout: 180_000,
hookTimeout: 300_000,
pool: 'forks',
Expand Down