Skip to content

feat(ai,agent): route model calls through cloudflare ai gateway for token+cost observability - #165

Merged
prisis merged 6 commits into
alphafrom
feat/ai-gateway-observability
Jul 22, 2026
Merged

prisis merged 6 commits into
alphafrom
feat/ai-gateway-observability

Conversation

@prisis

@prisis prisis commented Jul 22, 2026 •

Copy link
Copy Markdown
Member

Tier 0 — Cloudflare AI Gateway integration

Opt-in routing of Workers AI model calls through Cloudflare AI Gateway so LLM calls get token + dollar-cost observability out of the box.

  • resolveAiGateway(env) helper in @lunora/ai (dependency-free); create-ai derives the Workers AI gateway from env.
  • Threaded through @lunora/agent (generate.ts, voice-do.ts) and codegen so app ctx.ai calls route through too.
  • New generation-span attributes (additive; absent when the gateway didn't run): gen_ai.usage.cost (USD), gen_ai.response.cached, cf.aig.log_id — read from providerMetadata.gateway / cf-aig-* response headers.
  • Opt-in env: LUNORA_AI_GATEWAY_ACCOUNT_ID + LUNORA_AI_GATEWAY_ID (+ optional LUNORA_AI_GATEWAY_TOKEN). Unset ⇒ unchanged direct-provider behavior.

Verify: ai 104 / agent 283 / codegen 838 tests pass; tsc + eslint clean.

Follow-ups (tracked): RAG embed spans don't carry cost yet (needs post-hoc attributes on the server LunoraTracer — folds into the trace-fusion work); confirm the exact gateway cost field against a live gateway; auto-thread cf-aig-metadata at model construction.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Added opt-in Cloudflare AI Gateway routing for AI requests, automatically inferred from Worker environment when not explicitly configured.
    • AI initialization and stubs now receive Worker env to enable correct runtime gateway behavior.
    • Enhanced telemetry with AI Gateway metadata (cache status, request cost, and log IDs) when available.
  • Documentation
    • Updated AI configuration guidance to clarify gateway precedence (explicit gateway vs env-derived routing) and how to enable environment-based routing.

…oken+cost observability

Add opt-in Cloudflare AI Gateway routing so LLM calls get token +
dollar-cost telemetry out of the box. `resolveAiGateway(env)` reads the
`LUNORA_AI_GATEWAY_*` env vars (dependency-free) and both `create-ai`'s
Workers AI provider and the agent's model construction route through the
gateway when configured; unset = unchanged (direct provider). The agent
OTLP generation spans additively carry `gen_ai.usage.cost`,
`gen_ai.response.cached`, and `cf.aig.log_id` when the gateway computed
them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0197KjhmBDB3PXAc6TXiPqZ9
@netlify

netlify Bot commented Jul 22, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for lunorash ready!

Name Link
🔨 Latest commit 15c6707
🔍 Latest deploy log https://app.netlify.com/projects/lunorash/deploys/6a60ed3c4e4aaa0008f6eca6
😎 Deploy Preview https://deploy-preview-165--lunorash.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions

Copy link
Copy Markdown
Contributor

Thank you for following the naming conventions! 🙏

@coderabbitai

coderabbitai Bot commented Jul 22, 2026 •

Copy link
Copy Markdown

Review Change Stack

Important

Review skipped

Review was skipped due to path filters

⛔ Files ignored due to path filters (1)
  • packages/vite/__tests__/studio-plugin.test.ts is excluded by !**/__tests__/**, !**/*.test.ts and included by packages/**

CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including **/dist/** will override the default block on the dist directory, by removing the pattern from both the lists.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: c417b6ee-bdba-4e62-85c4-add948c73491

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Walkthrough

Adds opt-in Cloudflare AI Gateway routing for Workers AI through environment-based configuration, propagates Worker environments into AI construction paths, and records gateway cache, cost, and log identifiers in OTLP telemetry.

Changes

AI Gateway routing and telemetry

Layer / File(s) Summary
Gateway resolution contract
packages/ai/src/gateway.ts, packages/ai/src/index.ts, packages/ai/src/types.ts
Defines gateway environment variables, resolution metadata, conditional authorization and correlation headers, public exports, and environment-based routing documentation.
Provider gateway wiring
packages/ai/src/create-ai.ts, packages/agent/src/generate.ts, packages/agent/src/voice-do.ts, packages/codegen/src/emit.ts
Prioritizes explicit gateway options, derives gateway routing from env when available, applies routing to provider and binding calls, and passes Worker environment values through AI construction paths.
Gateway telemetry span attributes
packages/agent/src/telemetry/common.ts, packages/agent/src/telemetry/otlp.ts
Extracts gateway cache, cost, and log identifiers from model results and conditionally adds them to language-model OTLP spans.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Worker
  participant createAi
  participant resolveAiGateway
  participant WorkersAI
  participant OTLP
  Worker->>createAi: AI binding and env
  createAi->>resolveAiGateway: derive gateway configuration
  resolveAiGateway-->>createAi: gateway option or undefined
  createAi->>WorkersAI: construct model provider
  WorkersAI-->>OTLP: model result with gateway metadata
  OTLP->>OTLP: emit cache, cost, and log ID attributes
Loading

Possibly related PRs

  • anolilab/lunora#160: Both modify OTLP language-model span emission, with this PR adding Cloudflare AI Gateway attributes.
  • anolilab/lunora#162: Both modify executeLanguageModelCall span attribute construction.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description has useful details, but it doesn't follow the required template sections or include linked issues, checklist, notes, or CLA text. Reformat it to the template and add Summary, Linked issues, Test plan, Checklist, Notes for reviewers, and the CLA statement.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly states the main change: routing AI model calls through Cloudflare AI Gateway for observability.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/ai-gateway-observability

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

Thank you for confirming the Contributor License Agreement! 🙏

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/ai/src/create-ai.ts`:
- Around line 60-70: The resolved gateway used to construct workersai must also
be applied to ai.run() options. In createAi, reuse the resolved gateway from
resolveGatewayOption(gateway, env) and merge it into runOptions only when
runOptions.gateway is absent, preserving any caller-provided gateway.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: c25cd11a-52b4-4c37-91b9-fefb0cb40c83

📥 Commits

Reviewing files that changed from the base of the PR and between d71f8f1 and 881ea89.

⛔ Files ignored due to path filters (4)
  • packages/agent/__tests__/telemetry/otlp.test.ts is excluded by !**/__tests__/**, !**/*.test.ts and included by packages/**
  • packages/ai/__tests__/build-provider.test.ts is excluded by !**/__tests__/**, !**/*.test.ts and included by packages/**
  • packages/ai/__tests__/gateway.test.ts is excluded by !**/__tests__/**, !**/*.test.ts and included by packages/**
  • packages/codegen/__tests__/run-codegen.test.ts is excluded by !**/__tests__/**, !**/*.test.ts and included by packages/**
📒 Files selected for processing (9)
  • packages/agent/src/generate.ts
  • packages/agent/src/telemetry/common.ts
  • packages/agent/src/telemetry/otlp.ts
  • packages/agent/src/voice-do.ts
  • packages/ai/src/create-ai.ts
  • packages/ai/src/gateway.ts
  • packages/ai/src/index.ts
  • packages/ai/src/types.ts
  • packages/codegen/src/emit.ts

Comment thread packages/ai/src/create-ai.ts Resolved
@codspeed

codspeed Bot commented Jul 22, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 4 untouched benchmarks
⏩ 157 skipped benchmarks1


Comparing feat/ai-gateway-observability (15c6707) with alpha (948d27f)2

Open in CodSpeed

Footnotes

  1. 157 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on alpha (7f16ac1) during the generation of this report, so 948d27f was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

CodeRabbit #165: the resolved AI Gateway was applied to the AI-SDK model
path (createWorkersAI) but ai.run() forwarded runOptions unchanged, so raw
Workers AI binding calls skipped gateway routing (and its token+cost
telemetry) unless every caller passed `gateway` manually. Hoist the resolved
gateway and merge it into run options when the caller didn't set one.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0197KjhmBDB3PXAc6TXiPqZ9
The studio-plugin ETag test used `expect.hasAssertions()` but legitimately
makes zero assertions when @lunora/studio isn't built (the 501 path taken in
the affected-test CI env), so it failed there. This surfaced on any PR that
makes @lunora/vite "affected" (e.g. a @lunora/codegen change). Replace it with
an unconditional `expect([200, 501]).toContain(statusCode)` that holds in both
the built and unbuilt environments and keeps ≥1 assertion on the 501 path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0197KjhmBDB3PXAc6TXiPqZ9
@prisis
prisis merged commit fc26783 into alpha Jul 22, 2026
12 checks passed
@prisis
prisis deleted the feat/ai-gateway-observability branch July 22, 2026 16:18
prisis added a commit that referenced this pull request Jul 28, 2026
From better-auth-ui #195, #193, #169, #166, #74 and #337.

#195 — the organization limits are now published by uiConfig(). better-auth
enforces organizationLimit / allowUserToCreateOrganization server-side and
exposes no endpoint that reports them, so a UI can only find out by being
refused. The create form is disabled with the reason instead. A function-form
allowUserToCreateOrganization is a per-request decision the endpoint cannot
evaluate, so it reports allowed and lets the server refuse — the alternative
hides the button from everyone because one user might be denied.

#166 — the 2FA card no longer vanishes for an OAuth-only account; it explains
that a password is needed first. A setting that is simply absent reads as
'this app doesn't support 2FA'. Only a successful accounts read that found no
credential row counts: while loading, or on error, the card behaves as
before — the flow gate's rule, don't hide what you cannot reason about.

#74 — the TOTP key is shown beside the otpauth:// URI. Most authenticators
reject a pasted URI, so a desktop app or a broken camera had no way in.

#193 — createActiveMemberController resolves the signed-in user's role in the
active organization, which otherwise means fetching the org and finding
yourself in members at every call site. A rendering hint, not authorization.

#337 — organization.showSlug hides the slug field; create already derives one
from the name. #169 was already supported by that same fallback.

#165 (organization custom fields) is not done: it needs an additionalFields
design that spans every card, not a flag.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using our GitHub Discussions tab for questions.

@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Sep 9, 2026

This branch was previously deployed

1 inactive deployment
benchmarks — 15c67078 Deployed Jul 22, 2026 by prisis via Benchmarks #870
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant