Skip to content

feat: 修复排行榜指标并增加请求缓存详情 - #1383

Merged
ding113 merged 2 commits into
devfrom
leaderboard-cache-coefficient-success
Aug 2, 2026
Merged

feat: 修复排行榜指标并增加请求缓存详情#1383
ding113 merged 2 commits into
devfrom
leaderboard-cache-coefficient-success

Conversation

@ding113

@ding113 ding113 commented Aug 2, 2026

Copy link
Copy Markdown
Owner

摘要

  • 修复缓存效果调度器将 JavaScript Date 直接交给 Drizzle/postgres.js 编码导致聚合持续失败的问题,恢复 provider_cache_effectiveness 数据产出。
  • 为 provider 及 redirected model 明细接入缓存系数,并在 billing_model_source=redirected 下展示最终执行模型的成功率;仅在没有可计数 outcome 时返回不可用。
  • 在请求详情 Summary 和 Logic Trace 中统一展示实际缓存率、理论缓存率和本请求缓存系数,并保留原始 token provenance tooltip。
  • Prefix ID 与物理 Session ID 支持点击跳转到使用记录;现有 sessionId 筛选同时支持 canonical identity 和 physical Session ID 的精确过滤。
  • 扩展 usage-log API、普通表、虚拟表和 CSV/XLSX 导出契约,并补齐五种语言文案。

关键实现

  • 原生缓存聚合 SQL 使用 ISO string + ::timestamptz,避免 postgres.js 的 Date 参数编码错误。
  • 请求级缓存指标由持久化 token 字段确定性派生,不新增数据库列或 Redis 依赖。
  • 请求缓存系数为单请求 raw coefficient,不应用 provider 窗口级置信折扣。
  • model 缓存系数按 providerId + normalized redirected model 聚合;original 模型口径保持 null
  • instrumentation 日志安全展开首层 cause 的 name/code/message,避免再次丢失真实底层错误。
  • 无数据库迁移、无新 Pod、无外部组件。

验证

  • bun run lint
  • bun run lint:fix
  • bun run typecheck
  • bun run test:847 个 test files 通过,2 个跳过;8233 个 tests 通过,13 个跳过
  • bun run build
  • git diff --check

Production build 仅保留仓库既有的 Edge Runtime Node API warnings。

部署说明

部署后仍需执行一次有界历史回填,并连续观察至少两个 5 分钟调度周期,确认聚合表持续增长、排行榜 provider/model 系数恢复且不再出现 cache effectiveness tick failure。

Greptile Summary

The PR restores cache-effectiveness aggregation and expands request-level cache, model-success, and session-identity reporting across repository, API, dashboard, and export paths.

  • Derives and displays actual/theoretical cache rates and per-request cache coefficients.
  • Preserves Prefix ID versus physical Session ID semantics through ledger fallback, filtering, navigation, and exports.
  • Corrects redirected-model leaderboard metrics and improves underlying scheduler error diagnostics.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the previously reported ledger fallback identity loss is corrected across all relevant query and mapping paths, and ledger population and backfill preserve the required identity kind.

Important Files Changed

Filename Overview
src/repository/usage-logs.ts Propagates identity-kind and cache-metric data consistently through message-request, ledger fallback, readonly, and slim-row paths.
src/lib/usage-logs/export/columns.ts Separates canonical Prefix ID from physical Session ID using the preserved identity kind.
src/lib/cache-effectiveness/request-metrics.ts Centralizes deterministic derivation of request-level cache rates, coefficients, and availability.
src/repository/provider-cache-effectiveness.ts Adjusts cache-effectiveness aggregation inputs to avoid timestamp encoding failures.
src/repository/leaderboard.ts Extends provider and redirected-model leaderboard metrics with corrected success-rate and cache-coefficient handling.
src/app/[locale]/dashboard/logs/_components/error-details-dialog/components/CachePerformance.tsx Presents request cache measurements and token provenance with localized availability explanations.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  MR[message_request identity and cache fields] --> Trigger[Ledger trigger/backfill]
  Trigger --> UL[usage_ledger]
  MR --> Repo[Usage log repository]
  UL --> Repo
  Repo --> API[Usage-log API]
  Repo --> Export[CSV/XLSX export]
  API --> UI[Summary, Logic Trace, and tables]
Loading

Reviews (2): Last reviewed commit: "fix: address cache metric review finding..." | Re-trigger Greptile

Context used (3)

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@coderabbitai

coderabbitai Bot commented Aug 2, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

本次变更新增请求级缓存指标,支持 Prefix ID 与物理 Session ID,更新日志详情和导出展示,并调整模型排行榜成功率及缓存系数逻辑。

Changes

缓存与会话可观测性

Layer / File(s) Summary
缓存指标计算与聚合
src/lib/cache-effectiveness/*, src/repository/provider-cache-effectiveness.ts, src/instrumentation.ts, tests/unit/lib/cache-effectiveness/*
新增缓存指标派生、provider/model 缓存系数聚合、时间窗口处理和调度器错误描述。
日志数据契约与导出
src/repository/usage-logs.ts, src/lib/usage-logs/export/*, src/lib/api/..., src/lib/api-client/...
日志数据增加会话身份和缓存字段。详情导出分别写入 Prefix ID 与物理 Session ID。
日志详情缓存与会话展示
src/app/.../error-details-dialog/*, messages/*/dashboard.json
详情页展示缓存性能指标。Prefix ID 和物理 Session ID 使用日志筛选链接。各语言文案同步更新。
排行榜成功率与缓存系数
src/repository/leaderboard.ts, src/app/.../leaderboard/*
redirected 模型保留可计数成功率,并合并模型缓存系数。无可计数结果时使用 no_countable_outcomes

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 24.44% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed 标题准确概括了排行榜指标修复和请求缓存详情新增,且与主要变更一致。
Description check ✅ Passed 描述详细说明了调度器修复、缓存指标、会话筛选、导出契约和本地化文案等变更。
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch leaderboard-cache-coefficient-success

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Comment thread src/repository/usage-logs.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b1fb5905b9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/repository/usage-logs.ts Outdated
Comment thread src/repository/leaderboard.ts
Comment thread src/lib/cache-effectiveness/request-metrics.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/app/[locale]/dashboard/logs/_components/error-details-dialog/components/LogicTraceTab.tsx (1)

400-511: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

将 CachePerformance 展示到所有 Logic Trace 请求流程。

LogicTraceTab 只在 summary 中有条件地渲染 CachePerformance;Discovery 模式直接返回 DiscoveryTraceView,普通决策链、Affinity Hit、replay/重试/hedge 也未渲染。如果需求是统一展示缓存性能指标,需将 CachePerformance 拆到 Session Reuse 流程之外,或为 Discovery/普通流程添加独立展示区域。

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@src/app/`[locale]/dashboard/logs/_components/error-details-dialog/components/LogicTraceTab.tsx
around lines 400 - 511, 将 CachePerformance 从 isSessionReuseFlow 的 StepCard
内容中提取为所有 Logic Trace 流程共享的展示区域,确保 DiscoveryTraceView、普通决策链、Affinity Hit 以及
replay/重试/hedge 流程也能显示缓存性能指标;复用现有缓存指标属性和紧凑布局,避免仅在 session reuse 分支渲染。
🧹 Nitpick comments (2)
src/app/[locale]/dashboard/logs/_components/error-details-dialog/components/LogicTraceTab.tsx (1)

91-105: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

buildLogsFilterHref 与 SummaryTab.tsx 中的实现完全重复。

此函数体与 SummaryTab.tsx 第 110-114 行新增的 buildLogsFilterHref 完全相同。两处独立维护同一段逻辑,后续修改容易只改一处而遗漏另一处。
提取为共享工具函数,放在该目录下的公共文件中,两个标签页共同引用。

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@src/app/`[locale]/dashboard/logs/_components/error-details-dialog/components/LogicTraceTab.tsx
around lines 91 - 105, 提取 LogicTraceTab 中的 buildLogsFilterHref
为该目录下的共享工具函数,并移除两处重复实现;更新 LogicTraceTab 和 SummaryTab 共同引用该函数,保持现有 sessionId
查询参数及返回路径行为不变。
src/app/[locale]/dashboard/logs/_components/error-details-dialog/components/SummaryTab.tsx (1)

108-128: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

buildLogsFilterHref 与 LogicTraceTab.tsx 中的实现完全重复。

此函数体与 LogicTraceTab.tsx 第 101-105 行新增的 buildLogsFilterHref 完全相同。两处独立维护同一段逻辑,后续修改容易只改一处而遗漏另一处。
提取为共享工具函数,放在该目录下的公共文件中,两个标签页共同引用。

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@src/app/`[locale]/dashboard/logs/_components/error-details-dialog/components/SummaryTab.tsx
around lines 108 - 128, Extract the duplicated buildLogsFilterHref logic from
SummaryTab and LogicTraceTab into a shared utility in their common directory,
then import and use that utility in both tabs. Preserve the existing sessionId
query construction and generated dashboard logs URL exactly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/instrumentation.ts`:
- Around line 294-314: Update describeSchedulerError so the errorCause value
truncates Error causes’ message to 500 characters, matching the existing
non-Error cause handling; preserve the current undefined behavior when no cause
is present.

In `@src/repository/leaderboard.ts`:
- Around line 787-807: Align the cache-coefficient lookup in the leaderboard
model rows with the model field used by getProviderModelCacheCoefficients for
redirected data. Update modelCacheKey and the corresponding lookup in the second
affected block (around the additional cacheCoefficientBp assignment) to use the
same original-model source and normalization as the coefficient aggregation,
preventing valid coefficients from becoming null.

---

Outside diff comments:
In
`@src/app/`[locale]/dashboard/logs/_components/error-details-dialog/components/LogicTraceTab.tsx:
- Around line 400-511: 将 CachePerformance 从 isSessionReuseFlow 的 StepCard
内容中提取为所有 Logic Trace 流程共享的展示区域,确保 DiscoveryTraceView、普通决策链、Affinity Hit 以及
replay/重试/hedge 流程也能显示缓存性能指标;复用现有缓存指标属性和紧凑布局,避免仅在 session reuse 分支渲染。

---

Nitpick comments:
In
`@src/app/`[locale]/dashboard/logs/_components/error-details-dialog/components/LogicTraceTab.tsx:
- Around line 91-105: 提取 LogicTraceTab 中的 buildLogsFilterHref
为该目录下的共享工具函数,并移除两处重复实现;更新 LogicTraceTab 和 SummaryTab 共同引用该函数,保持现有 sessionId
查询参数及返回路径行为不变。

In
`@src/app/`[locale]/dashboard/logs/_components/error-details-dialog/components/SummaryTab.tsx:
- Around line 108-128: Extract the duplicated buildLogsFilterHref logic from
SummaryTab and LogicTraceTab into a shared utility in their common directory,
then import and use that utility in both tabs. Preserve the existing sessionId
query construction and generated dashboard logs URL exactly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 3ee58996-7802-4e89-b1d8-68213de5238f

📥 Commits

Reviewing files that changed from the base of the PR and between 5dd805e and b1fb590.

📒 Files selected for processing (37)
  • messages/en/dashboard.json
  • messages/ja/dashboard.json
  • messages/ru/dashboard.json
  • messages/zh-CN/dashboard.json
  • messages/zh-TW/dashboard.json
  • src/app/[locale]/dashboard/leaderboard/_components/leaderboard-view.tsx
  • src/app/[locale]/dashboard/leaderboard/_components/success-rate-display.ts
  • src/app/[locale]/dashboard/logs/_components/error-details-dialog.test.tsx
  • src/app/[locale]/dashboard/logs/_components/error-details-dialog/components/CachePerformance.test.tsx
  • src/app/[locale]/dashboard/logs/_components/error-details-dialog/components/CachePerformance.tsx
  • src/app/[locale]/dashboard/logs/_components/error-details-dialog/components/LogicTraceTab.tsx
  • src/app/[locale]/dashboard/logs/_components/error-details-dialog/components/SummaryTab.tsx
  • src/app/[locale]/dashboard/logs/_components/error-details-dialog/components/index.ts
  • src/app/[locale]/dashboard/logs/_components/error-details-dialog/index.tsx
  • src/app/[locale]/dashboard/logs/_components/error-details-dialog/types.ts
  • src/app/[locale]/dashboard/logs/_components/usage-logs-table.test.tsx
  • src/app/[locale]/dashboard/logs/_components/usage-logs-table.tsx
  • src/app/[locale]/dashboard/logs/_components/virtualized-logs-table.test.tsx
  • src/app/[locale]/dashboard/logs/_components/virtualized-logs-table.tsx
  • src/instrumentation.ts
  • src/lib/api-client/v1/openapi-types.gen.ts
  • src/lib/api/v1/schemas/usage-logs.ts
  • src/lib/cache-effectiveness/request-metrics.ts
  • src/lib/cache-effectiveness/service.ts
  • src/lib/usage-logs/export/columns.ts
  • src/repository/leaderboard.ts
  • src/repository/provider-cache-effectiveness.ts
  • src/repository/usage-logs.ts
  • tests/unit/api/leaderboard-route.test.ts
  • tests/unit/dashboard/leaderboard-success-rate-display.test.ts
  • tests/unit/lib/cache-effectiveness/request-metrics.test.ts
  • tests/unit/lib/cache-effectiveness/service.test.ts
  • tests/unit/repository/leaderboard-cache-coefficient.test.ts
  • tests/unit/repository/leaderboard-provider-metrics.test.ts
  • tests/unit/repository/usage-logs-sessionid-filter.test.ts
  • tests/unit/usage-logs/export-csv.test.ts
  • tests/unit/usage-logs/export-xlsx.test.ts

Comment thread src/instrumentation.ts Outdated
Comment thread src/repository/leaderboard.ts
@github-actions github-actions Bot added the size/XL Extra Large PR (> 1000 lines) label Aug 2, 2026
Comment thread src/repository/provider-cache-effectiveness.ts
Comment thread src/repository/usage-logs.ts Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

This XL PR adds substantial leaderboard, usage-log, export, and i18n surface area. I found two high-confidence logic issues in the new code paths: one in the new model-level cache-coefficient aggregation, and one in the new ledger-backed session-identity mapping.

PR Size: XL

  • Lines changed: 1494
  • Files changed: 37
  • Split suggestion: Separate the leaderboard/cache-effectiveness backend work from the logs/export/session-identity UI contract changes.
  • Split suggestion: Keep the translation catalog updates in a follow-up PR so behavioral review is not mixed with 100+ locale-string edits.

Issues Found

Category Critical High Medium Low
Logic/Bugs 0 2 0 0
Security 0 0 0 0
Error Handling 0 0 0 0
Types 0 0 0 0
Comments/Docs 0 0 0 0
Tests 0 0 0 0
Simplification 0 0 0 0

Critical Issues (Must Fix)

  • None.

High Priority Issues (Should Fix)

  • src/repository/provider-cache-effectiveness.ts:204 — the new model-level cache-coefficient query slices scheduler rollups by window_end, which can pull traffic from outside the requested daily/custom leaderboard window.
  • src/repository/usage-logs.ts:661 — ledger-backed rows hard-code sessionIdentityKind to null, so the new Prefix ID / Session ID split breaks for historical or ledger-only data.

Review Coverage

  • Logic and correctness
  • Security (OWASP Top 10)
  • Error handling
  • Type safety
  • Documentation accuracy
  • Test coverage
  • Code clarity

Automated review by Codex AI

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Reviewed PR #1383, applied the size/XL label, posted 2 inline comments, and submitted the summary review.
  • Reported src/repository/provider-cache-effectiveness.ts:204 for model-level cache coefficients leaking scheduler windows across daily/custom leaderboard boundaries.
  • Reported src/repository/usage-logs.ts:661 for hard-coding sessionIdentityKind to null on ledger-backed rows, which breaks the new Prefix ID / Session ID split for historical/ledger-only data.
  • Summary review is now on the PR with the required size stats, severity table, split suggestions, and coverage checklist.

@ding113

ding113 commented Aug 2, 2026

Copy link
Copy Markdown
Owner Author

Review follow-up for 3ae38cf:

  • Addressed every validated inline finding: ledger identity propagation, Redis payload v4, neutral not_recorded semantics, bounded Error.cause logging, key-scoped cache fields, and the missing Prefix Affinity compact cache panel.
  • Extracted buildLogsFilterHref so Summary and Logic Trace use the same URLSearchParams contract.
  • The Logic Trace scope is intentional: the approved requirement replaces the cache hint in session/prefix reuse steps. Prefix Affinity now has the same compact metrics and identity links; Discovery, ordinary selection, replay, retry, and hedge steps do not claim session/prefix reuse and therefore do not render a duplicate cache panel.
  • Two inline findings were resolved with contract evidence rather than code changes: redirected model keys already share the final executed model source, and provider cache coefficients intentionally use completed scheduler-window attribution.

Validation: 847 test files / 8243 tests passed, plus lint, lint:fix, typecheck, OpenAPI check/lint, production build, and git diff --check.

@ding113
ding113 merged commit a602f84 into dev Aug 2, 2026
24 of 25 checks passed
@github-project-automation github-project-automation Bot moved this from Backlog to Done in Claude Code Hub Roadmap Aug 2, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3ae38cf0c0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +745 to +748
const modelCacheCoefficientsPromise =
billingModelSource === "redirected"
? getProviderModelCacheCoefficients(resolveLeaderboardWindow(period, timezone, dateRange))
: Promise.resolve(new Map());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Attach handlers to both concurrent leaderboard queries

When the usage-ledger model query rejects or remains pending while this already-started coefficient query rejects, execution has not reached the later await, so the coefficient rejection can be reported as unhandled. registerCrashDiagnostics exits the process on unhandled rejections, meaning a database failure while loading this leaderboard can terminate the entire replica rather than only fail the request. Attach handlers immediately, such as by awaiting both operations through Promise.all; the same pattern is repeated in the cache-hit branch at lines 889-925.

Useful? React with 👍 / 👎.

Comment on lines +350 to +352
<Link
href={buildLogsFilterHref(identity.value)}
className="text-xs font-mono break-all underline-offset-2 hover:underline"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep reserved physical session links queryable

For a client-supplied physical session ID beginning with pfx: or sid:, buildPublicSessionIdentity replaces the canonical identity with a sid:<hash> value, so this list contains both the hash and the raw physical ID. The new link filters using that raw value, but buildUsageLogConditions intentionally treats reserved values as canonical-only and omits the physical session_id comparison; clicking the displayed physical ID therefore returns no matching logs. Link reserved source rows through their canonical identity or add an explicitly disambiguated physical-session filter.

Useful? React with 👍 / 👎.

@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

🧪 测试结果

测试类型 状态
代码质量
单元测试
集成测试
API 测试

总体结果: ✅ 所有测试通过

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XL Extra Large PR (> 1000 lines)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant