Skip to content

Replace the openai SDK with a lean fetch-based client. - #2

Merged
expilu merged 2 commits into
mainfrom
feature/remove-openai-sdk-dependency
Sep 26, 2026
Merged

expilu merged 2 commits into
mainfrom
feature/remove-openai-sdk-dependency

Conversation

@expilu

@expilu expilu commented Sep 26, 2026

Copy link
Copy Markdown
Owner

Drop the openai dependency — lean fetch-based generateText client

Summary

smart-decisions now talks to OpenAI-compatible endpoints through ~100 lines of its own
transport code instead of the openai SDK. The library ships zero runtime dependencies.

The public API is unchanged: choice() keeps its shape and signature. Question gains two
optional provider settings: maxRetries (default 2) and timeoutMs (default 600000).

What changed

New transport — src/utils/llms/generate-text.ts

  • Single non-streaming POST {apiBaseUrl}/chat/completions with Authorization: Bearer and a
    JSON body including stream: false — the same wire format the SDK produced (verified by
    capturing the exact request).
  • Configurable retries (default 2, SDK-style counting: total attempts = maxRetries + 1) with
    exponential backoff (500ms doubling, capped at 8s), honoring retry-after-ms / retry-after
    and the x-should-retry header. Retryable statuses: 408, 409, 429, ≥ 500.
  • Per-attempt timeout (default 10 min) via AbortSignal.timeout. Timeouts are deliberately
    not retried
    — doubling a 10-minute wait for a one-token decision is worse than failing fast.
  • Errors are plain Errors carrying the HTTP status, a truncated response body, and the
    original failure as cause.
  • User-Agent: smart-decisions/<version> (src/version.ts, kept in sync with package.json
    by a test).

Repo structure

  • src/utils/{time,network,error,text}/ — eight focused helpers (sleep, backoffDelay,
    numericHeader, retryDelayFromHeaders, shouldRetryStatus, isTimeoutError, messageOf,
    truncate).
  • src/types/ — seven granular internal types describing the chat-completions
    request/response shapes.
  • House doc standard applied repo-wide: self-documenting JSDoc with @param, @returns,
    @throws and @example on every exported symbol.

Tooling

  • tsconfig.json is now a repo-wide typecheck project (src + test + examples, noEmit);
    emitting lives in tsconfig.build.json. New typecheck script, wired into prepublishOnly.
    Tests and expectTypeOf assertions are now enforced by the compiler.
  • .vscode watch task now runs the full-repo typecheck (no dist/ churn on save).

Tests

  • The transport is tested through the global fetch with real Response objects and fake
    timers; system1-choice tests re-wired the same way; new unit suites for every helper and
    type; a version-sync test.
  • 23 files / 69 tests, 100% lines / branches / functions / statements (thresholds enforced).

Behavior deltas vs the SDK

openai SDK (before) now
Retries 2 fixed, incl. timeouts configurable (default 2), timeouts excluded
Timeout 10 min per request configurable, 10 min per attempt (default)
Errors APIError subclasses plain Error (status + body + cause)
Headers OpenAI/JS x.y.z, X-Stainless-* telemetry smart-decisions/x.y.z, nothing else
Dependencies openai none

Verification

  • typecheck, lint, format:check, build, test with enforced 100% coverage
  • Live end-to-end: npm run example against llama.cpp (movie @ 0.9991, matches the
    README's documented numbers)
  • Old-SDK vs new-transport equivalence on the live server: same answer and same token
    ranking; the small logprob deltas were proven to be server-side run-to-run noise via a
    same-client control (two identical requests through one transport also wobble)
  • Captured wire request: identical method, path, headers and body contract
  • npm pack — 51 files, 47.3 kB, zero openai references in dist or lockfile

@expilu expilu self-assigned this Sep 26, 2026
@expilu
expilu merged commit 8a2c933 into main Sep 26, 2026
2 of 3 checks passed
@expilu
expilu deleted the feature/remove-openai-sdk-dependency branch September 26, 2026 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant