Repository navigation
feat(ai,agent): route model calls through cloudflare ai gateway for token+cost observability - #165
Conversation
…oken+cost observability Add opt-in Cloudflare AI Gateway routing so LLM calls get token + dollar-cost telemetry out of the box. `resolveAiGateway(env)` reads the `LUNORA_AI_GATEWAY_*` env vars (dependency-free) and both `create-ai`'s Workers AI provider and the agent's model construction route through the gateway when configured; unset = unchanged (direct provider). The agent OTLP generation spans additively carry `gen_ai.usage.cost`, `gen_ai.response.cached`, and `cf.aig.log_id` when the gateway computed them. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0197KjhmBDB3PXAc6TXiPqZ9
✅ Deploy Preview for lunorash ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Thank you for following the naming conventions! 🙏 |
|
Important Review skippedReview was skipped due to path filters ⛔ Files ignored due to path filters (1)
CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
WalkthroughAdds opt-in Cloudflare AI Gateway routing for Workers AI through environment-based configuration, propagates Worker environments into AI construction paths, and records gateway cache, cost, and log identifiers in OTLP telemetry. ChangesAI Gateway routing and telemetry
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant Worker
participant createAi
participant resolveAiGateway
participant WorkersAI
participant OTLP
Worker->>createAi: AI binding and env
createAi->>resolveAiGateway: derive gateway configuration
resolveAiGateway-->>createAi: gateway option or undefined
createAi->>WorkersAI: construct model provider
WorkersAI-->>OTLP: model result with gateway metadata
OTLP->>OTLP: emit cache, cost, and log ID attributes
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Thank you for confirming the Contributor License Agreement! 🙏 |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/ai/src/create-ai.ts`:
- Around line 60-70: The resolved gateway used to construct workersai must also
be applied to ai.run() options. In createAi, reuse the resolved gateway from
resolveGatewayOption(gateway, env) and merge it into runOptions only when
runOptions.gateway is absent, preserving any caller-provided gateway.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: c25cd11a-52b4-4c37-91b9-fefb0cb40c83
⛔ Files ignored due to path filters (4)
packages/agent/__tests__/telemetry/otlp.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/ai/__tests__/build-provider.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/ai/__tests__/gateway.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/codegen/__tests__/run-codegen.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**
📒 Files selected for processing (9)
packages/agent/src/generate.tspackages/agent/src/telemetry/common.tspackages/agent/src/telemetry/otlp.tspackages/agent/src/voice-do.tspackages/ai/src/create-ai.tspackages/ai/src/gateway.tspackages/ai/src/index.tspackages/ai/src/types.tspackages/codegen/src/emit.ts
Merging this PR will not alter performance
Comparing Footnotes
|
CodeRabbit #165: the resolved AI Gateway was applied to the AI-SDK model path (createWorkersAI) but ai.run() forwarded runOptions unchanged, so raw Workers AI binding calls skipped gateway routing (and its token+cost telemetry) unless every caller passed `gateway` manually. Hoist the resolved gateway and merge it into run options when the caller didn't set one. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0197KjhmBDB3PXAc6TXiPqZ9
The studio-plugin ETag test used `expect.hasAssertions()` but legitimately makes zero assertions when @lunora/studio isn't built (the 501 path taken in the affected-test CI env), so it failed there. This surfaced on any PR that makes @lunora/vite "affected" (e.g. a @lunora/codegen change). Replace it with an unconditional `expect([200, 501]).toContain(statusCode)` that holds in both the built and unbuilt environments and keeps ≥1 assertion on the 501 path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0197KjhmBDB3PXAc6TXiPqZ9
From better-auth-ui #195, #193, #169, #166, #74 and #337. #195 — the organization limits are now published by uiConfig(). better-auth enforces organizationLimit / allowUserToCreateOrganization server-side and exposes no endpoint that reports them, so a UI can only find out by being refused. The create form is disabled with the reason instead. A function-form allowUserToCreateOrganization is a per-request decision the endpoint cannot evaluate, so it reports allowed and lets the server refuse — the alternative hides the button from everyone because one user might be denied. #166 — the 2FA card no longer vanishes for an OAuth-only account; it explains that a password is needed first. A setting that is simply absent reads as 'this app doesn't support 2FA'. Only a successful accounts read that found no credential row counts: while loading, or on error, the card behaves as before — the flow gate's rule, don't hide what you cannot reason about. #74 — the TOTP key is shown beside the otpauth:// URI. Most authenticators reject a pasted URI, so a desktop app or a broken camera had no way in. #193 — createActiveMemberController resolves the signed-in user's role in the active organization, which otherwise means fetching the org and finding yourself in members at every call site. A rendering hint, not authorization. #337 — organization.showSlug hides the slug field; create already derives one from the name. #169 was already supported by that same fallback. #165 (organization custom fields) is not done: it needs an additionalFields design that spans every card, not a flag. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Tier 0 — Cloudflare AI Gateway integration
Opt-in routing of Workers AI model calls through Cloudflare AI Gateway so LLM calls get token + dollar-cost observability out of the box.
resolveAiGateway(env)helper in@lunora/ai(dependency-free);create-aiderives the Workers AIgatewayfrom env.@lunora/agent(generate.ts,voice-do.ts) and codegen so appctx.aicalls route through too.gen_ai.usage.cost(USD),gen_ai.response.cached,cf.aig.log_id— read fromproviderMetadata.gateway/cf-aig-*response headers.LUNORA_AI_GATEWAY_ACCOUNT_ID+LUNORA_AI_GATEWAY_ID(+ optionalLUNORA_AI_GATEWAY_TOKEN). Unset ⇒ unchanged direct-provider behavior.Verify: ai 104 / agent 283 / codegen 838 tests pass; tsc + eslint clean.
Follow-ups (tracked): RAG embed spans don't carry cost yet (needs post-hoc attributes on the server
LunoraTracer— folds into the trace-fusion work); confirm the exact gateway cost field against a live gateway; auto-threadcf-aig-metadataat model construction.🤖 Generated with Claude Code
Summary by CodeRabbit
envto enable correct runtime gateway behavior.gatewayvsenv-derived routing) and how to enable environment-based routing.