Skip to content

[for binary building]#69977

Open
winoros wants to merge 25 commits into
pingcap:masterfrom
winoros:preview-ru-v4
Open

[for binary building]#69977
winoros wants to merge 25 commits into
pingcap:masterfrom
winoros:preview-ru-v4

Conversation

@winoros

@winoros winoros commented Jul 22, 2026

Copy link
Copy Markdown
Member

What problem does this PR solve?

Issue Number: close #xxx

Problem Summary:

What changed and how does it work?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

Summary by CodeRabbit

  • New Features
    • Added EXPLAIN ANALYZE FORMAT='RU' preview resource-unit reporting, including RU demo modeling and status-only diagnostics.
    • Added optional read-billing demo support (tidb_enable_read_billing_demo) with new statement-summary INFORMATION_SCHEMA tables (base-units + status, current + history), plus RU demo general-log and Prometheus observability.
  • Bug Fixes
    • Improved RU evidence capture (including close/early-error handling and expected-task accounting) and runtime byte tracking.
    • Enhanced hash-join runtime stats with accurate hash-table row accounting.
  • Documentation
    • Expanded RU/read-billing design documentation and updated RU-related test case maps.
  • Tests
    • Added extensive RU/read-billing coverage for explain output, metrics, and general-log emission behavior.

@winoros winoros added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 22, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jul 22, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign bb7133, nolouch, terry1purcell, windtalker for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. sig/planner SIG: Planner labels Jul 22, 2026
@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The PR adds preview RU explain and read-billing collection, including v4 formula evaluation, runtime evidence, mutation tracking, bounded statement-summary storage, INFORMATION_SCHEMA readers, metrics, general-log completion records, and extensive planner, executor, session, and storage tests.

Changes

Preview RU model and execution flow

Layer / File(s) Summary
RU formula, gating, and rendering
pkg/planner/core/*, pkg/types/explain_format.go
Adds FORMAT='RU' validation, runtime-evidence estimation, operator/unit calculation, status handling, and RU result rendering.
Runtime evidence collection
pkg/executor/internal/exec/*, pkg/util/execdetails/*, pkg/executor/join/*, pkg/util/chunk/*
Records runtime bytes, expected cop tasks, scan details, hash-table rows, and prewrite-region metrics.
Statement lifecycle and mutation tracking
pkg/session/*, pkg/sessionctx/stmtctx/stmtctx.go
Adds statement-local KV mutation recording, pipelined/retry handling, early-error status-only recording, and the read-billing-demo session variable.

Statement summaries and observability

Layer / File(s) Summary
Structured statement-summary data
pkg/util/stmtsummary/*, pkg/util/stmtsummary/v2/*
Adds bounded base-unit/status aggregation, JSON-stable records, status-only handling, current/history readers, and summary-table output.
Information Schema and planner wiring
pkg/infoschema/*, pkg/executor/stmtsummary.go, pkg/executor/builder.go, pkg/planner/core/*mem_table*
Registers read-billing base-unit and status tables, including history and cluster variants, and routes them through statement-summary readers.
Metrics and completion logs
pkg/metrics/*, pkg/executor/adapter.go
Adds ExplainRU/read-billing metrics, statement finalization integration, prepared-statement identity handling, and single-shot completion logging.

Runtime-stat correctness

Layer / File(s) Summary
Coprocessor close-time accounting
pkg/distsql/select_result.go, pkg/distsql/select_result_test.go
Aggregates scan details and expected tasks across responses, applies unconsumed stats after response closure, and preserves close errors.
Join runtime statistics
pkg/executor/join/*
Tracks hash-table state rows with overflow-safe clone and merge behavior for both hash join implementations.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related PRs

Suggested reviewers: yudongusa

Poem

A rabbit counts bytes in the breeze,
RU rows hop through trees.
Logs sparkle, summaries glow—
Stats know where the numbers go.

🚥 Pre-merge checks | ✅ 2 | ❌ 3

❌ Failed checks (2 warnings, 1 inconclusive)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description is just the template with placeholders; it lacks filled issue, problem, changes, tests, side effects, and release note details. Fill in the issue number, problem summary, implementation details, test checklist, side effects, documentation impacts, and release note.
Docstring Coverage ⚠️ Warning Docstring coverage is 13.04% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title is too vague and doesn't describe the main change in the PR. Use a concise title that names the primary change, such as the new preview RU/read-billing demo support.
✅ Passed checks (2 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
pkg/session/session.go (1)

2473-2476: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value

Use types.ExplainFormatRU here instead of repeating "ru".
StmtCtx.ExplainFormat is already lowercased, so this stays semantically identical and removes the literal duplication.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/session/session.go` around lines 2473 - 2476, Update the ExplainFormat
comparison in the previewKVMutationEnabled expression to use the existing
types.ExplainFormatRU constant instead of the literal "ru"; preserve the current
case-insensitive comparison and all surrounding conditions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/util/stmtsummary/statement_summary.go`:
- Around line 769-780: Update initReadBillingDemoStatusOnly to initialize
ssbd.isInternal from sei.IsInternal before the status-only summary aggregation
runs, preserving correct classification for the first internal statement while
leaving subsequent AND-based updates unchanged.

---

Nitpick comments:
In `@pkg/session/session.go`:
- Around line 2473-2476: Update the ExplainFormat comparison in the
previewKVMutationEnabled expression to use the existing types.ExplainFormatRU
constant instead of the literal "ru"; preserve the current case-insensitive
comparison and all surrounding conditions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 08a31ecd-3764-46fd-bc46-6fb3087feff9

📥 Commits

Reviewing files that changed from the base of the PR and between 955fd65 and 8c4feed.

📒 Files selected for processing (77)
  • .agents/skills/tidb-test-guidelines/references/distsql-case-map.md
  • .agents/skills/tidb-test-guidelines/references/executor-case-map.md
  • .agents/skills/tidb-test-guidelines/references/metrics-case-map.md
  • .agents/skills/tidb-test-guidelines/references/planner-case-map.md
  • .agents/skills/tidb-test-guidelines/references/session-case-map.md
  • .agents/skills/tidb-test-guidelines/references/util-case-map.md
  • docs/design/2026-07-01-read-billing-demo-ru-model.md
  • docs/design/2026-07-08-write-billing-units-weights-design.md
  • docs/design/2026-07-10-preview-ru-tidb-kv-mutation.md
  • docs/design/2026-07-22-preview-ru-resource-formula-plan.md
  • pkg/distsql/BUILD.bazel
  • pkg/distsql/select_result.go
  • pkg/distsql/select_result_test.go
  • pkg/executor/BUILD.bazel
  • pkg/executor/adapter.go
  • pkg/executor/adapter_internal_test.go
  • pkg/executor/adapter_test.go
  • pkg/executor/builder.go
  • pkg/executor/compiler.go
  • pkg/executor/explain_test.go
  • pkg/executor/explainfor_test.go
  • pkg/executor/internal/exec/BUILD.bazel
  • pkg/executor/internal/exec/executor.go
  • pkg/executor/internal/exec/executor_test.go
  • pkg/executor/join/hash_join_stats.go
  • pkg/executor/join/hash_join_v1.go
  • pkg/executor/join/hash_join_v2.go
  • pkg/executor/join/hash_table_v1.go
  • pkg/executor/join/hash_table_v1_test.go
  • pkg/executor/join/join_stats_test.go
  • pkg/executor/stmtsummary.go
  • pkg/infoschema/cluster.go
  • pkg/infoschema/tables.go
  • pkg/metrics/BUILD.bazel
  • pkg/metrics/explain_ru.go
  • pkg/metrics/metrics.go
  • pkg/metrics/metrics_internal_test.go
  • pkg/metrics/ru_v2.go
  • pkg/planner/core/BUILD.bazel
  • pkg/planner/core/common_plans.go
  • pkg/planner/core/common_plans_test.go
  • pkg/planner/core/explain_ru.go
  • pkg/planner/core/logical_plan_builder.go
  • pkg/planner/core/operator/logicalop/logical_mem_table.go
  • pkg/planner/core/pb_to_plan.go
  • pkg/planner/core/planbuilder.go
  • pkg/session/BUILD.bazel
  • pkg/session/session.go
  • pkg/session/tidb_test.go
  • pkg/session/txn.go
  • pkg/sessionctx/stmtctx/stmtctx.go
  • pkg/sessionctx/vardef/tidb_vars.go
  • pkg/sessionctx/variable/session.go
  • pkg/sessionctx/variable/sysvar.go
  • pkg/types/explain_format.go
  • pkg/util/chunk/chunk.go
  • pkg/util/chunk/chunk_test.go
  • pkg/util/execdetails/execdetails_test.go
  • pkg/util/execdetails/runtime_stats.go
  • pkg/util/execdetails/ruv2_metrics.go
  • pkg/util/stmtsummary/BUILD.bazel
  • pkg/util/stmtsummary/evicted.go
  • pkg/util/stmtsummary/read_billing.go
  • pkg/util/stmtsummary/read_billing_test.go
  • pkg/util/stmtsummary/reader.go
  • pkg/util/stmtsummary/statement_summary.go
  • pkg/util/stmtsummary/statement_summary_test.go
  • pkg/util/stmtsummary/v2/BUILD.bazel
  • pkg/util/stmtsummary/v2/column.go
  • pkg/util/stmtsummary/v2/column_test.go
  • pkg/util/stmtsummary/v2/logger.go
  • pkg/util/stmtsummary/v2/reader.go
  • pkg/util/stmtsummary/v2/reader_test.go
  • pkg/util/stmtsummary/v2/record.go
  • pkg/util/stmtsummary/v2/record_test.go
  • pkg/util/stmtsummary/v2/stmtsummary.go
  • pkg/util/stmtsummary/v2/stmtsummary_test.go

Comment on lines +769 to +780
func (ssbd *stmtSummaryByDigest) initReadBillingDemoStatusOnly(sei *StmtExecInfo) {
ssbd.cumulative = *newReadBillingDemoStatusOnlyStats(sei)
ssbd.schemaName = sei.SchemaName
ssbd.digest = sei.Digest
ssbd.planDigest = sei.PlanDigest
if sei.StmtCtx != nil {
ssbd.stmtType = sei.StmtCtx.StmtType
}
ssbd.normalizedSQL = formatSQL(sei.NormalizedSQL)
ssbd.history = list.New()
ssbd.initialized = true
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -nP 'func \(ssbd \*stmtSummaryByDigest\) init\(' -A 25 pkg/util/stmtsummary/statement_summary.go | rg -n 'isInternal'

Repository: pingcap/tidb

Length of output: 150


🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE='pkg/util/stmtsummary/statement_summary.go'

# Map the relevant methods first.
ast-grep outline "$FILE" --view expanded | sed -n '1,220p'

echo '--- initReadBillingDemoStatusOnly ---'
rg -n 'initReadBillingDemoStatusOnly|func \(ssbd \*stmtSummaryByDigest\) init\(' -A 20 -B 5 "$FILE"

echo '--- AddReadBillingDemoStatusOnly path ---'
rg -n 'AddReadBillingDemoStatusOnly|addReadBillingDemoStatusOnly|isInternal' -A 20 -B 10 "$FILE"

Repository: pingcap/tidb

Length of output: 20816


🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE='pkg/util/stmtsummary/statement_summary.go'

# Read the small relevant slices directly with line numbers.
sed -n '430,520p' "$FILE" | cat -n
echo '---'
sed -n '740,790p' "$FILE" | cat -n
echo '---'
sed -n '1160,1210p' "$FILE" | cat -n

Repository: pingcap/tidb

Length of output: 7553


🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE='pkg/util/stmtsummary/statement_summary.go'

# Show the normal initializer and the special initializer side by side.
rg -n 'func \(ssbd \*stmtSummaryByDigest\) init\(' -A 40 -B 5 "$FILE" || true
echo '---'
rg -n 'func \(ssbd \*stmtSummaryByDigest\) initReadBillingDemoStatusOnly\(' -A 25 -B 5 "$FILE" || true
echo '---'
rg -n 'summary\.isInternal =|ssbd\.isInternal =|IsInternal' -A 5 -B 5 "$FILE" || true

Repository: pingcap/tidb

Length of output: 4840


initReadBillingDemoStatusOnly should copy isInternal.

The status-only path does summary.isInternal = summary.isInternal && sei.IsInternal, so a newly allocated summary stays false unless this initializer sets it first. That misclassifies the first internal statement recorded here.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/util/stmtsummary/statement_summary.go` around lines 769 - 780, Update
initReadBillingDemoStatusOnly to initialize ssbd.isInternal from sei.IsInternal
before the status-only summary aggregation runs, preserving correct
classification for the first internal statement while leaving subsequent
AND-based updates unchanged.

@winoros
winoros force-pushed the preview-ru-v4 branch 2 times, most recently from 2d092b3 to a1b928b Compare July 23, 2026 16:37

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/design/2026-07-22-preview-ru-resource-formula-plan.md`:
- Line 27: Update the Milestone 1 progress entry to say “seven-weight” instead
of “six-weight,” preserving the listed semantic weights and all other milestone
details.

In `@pkg/executor/join/hash_join_v2.go`:
- Around line 96-112: The hashStateRows method currently saturates overflow to
math.MaxUint64, allowing an invalid count to be recorded by the hash_state_rows
path around the caller at lines 1347-1349. Change hashStateRows and its caller
to propagate an overflow/failure status, suppress hash_state_rows when overflow
occurs, and record only the existing bounded invalid reason instead of billing
the saturated value.

In `@pkg/session/tidb_test.go`:
- Around line 420-429: Update the test around dmlMetrics and commitMetrics to
use a test-only resource-manager write-count injection path that modifies
statement snapshots without emitting the global Prometheus metric; if
unavailable, isolate and restore the process-wide counter using existing test
infrastructure. Preserve the assertions verifying independent snapshots and the
expected counts.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 01576011-b7f3-493d-ba5d-463784474f78

📥 Commits

Reviewing files that changed from the base of the PR and between 2d092b3 and a1b928b.

📒 Files selected for processing (31)
  • .agents/skills/tidb-test-guidelines/references/distsql-case-map.md
  • .agents/skills/tidb-test-guidelines/references/executor-case-map.md
  • .agents/skills/tidb-test-guidelines/references/metrics-case-map.md
  • .agents/skills/tidb-test-guidelines/references/planner-case-map.md
  • .agents/skills/tidb-test-guidelines/references/session-case-map.md
  • .agents/skills/tidb-test-guidelines/references/util-case-map.md
  • docs/design/2026-07-01-read-billing-demo-ru-model.md
  • docs/design/2026-07-10-preview-ru-tidb-kv-mutation.md
  • docs/design/2026-07-22-preview-ru-resource-formula-plan.md
  • pkg/distsql/select_result_test.go
  • pkg/executor/adapter.go
  • pkg/executor/adapter_internal_test.go
  • pkg/executor/explain_test.go
  • pkg/executor/join/hash_join_stats.go
  • pkg/executor/join/hash_join_v1.go
  • pkg/executor/join/hash_join_v2.go
  • pkg/executor/join/hash_table_v1.go
  • pkg/executor/join/hash_table_v1_test.go
  • pkg/executor/join/join_stats_test.go
  • pkg/infoschema/tables.go
  • pkg/metrics/explain_ru.go
  • pkg/metrics/metrics_internal_test.go
  • pkg/planner/core/BUILD.bazel
  • pkg/planner/core/common_plans_test.go
  • pkg/planner/core/explain_ru.go
  • pkg/session/tidb_test.go
  • pkg/util/execdetails/execdetails_test.go
  • pkg/util/execdetails/runtime_stats.go
  • pkg/util/stmtsummary/read_billing.go
  • pkg/util/stmtsummary/read_billing_test.go
  • pkg/util/stmtsummary/v2/record_test.go
🚧 Files skipped from review as they are similar to previous changes (23)
  • .agents/skills/tidb-test-guidelines/references/session-case-map.md
  • pkg/executor/join/hash_table_v1.go
  • pkg/executor/join/hash_table_v1_test.go
  • pkg/executor/join/join_stats_test.go
  • .agents/skills/tidb-test-guidelines/references/util-case-map.md
  • docs/design/2026-07-10-preview-ru-tidb-kv-mutation.md
  • .agents/skills/tidb-test-guidelines/references/executor-case-map.md
  • pkg/infoschema/tables.go
  • pkg/metrics/metrics_internal_test.go
  • pkg/util/stmtsummary/v2/record_test.go
  • pkg/executor/join/hash_join_stats.go
  • pkg/util/stmtsummary/read_billing_test.go
  • pkg/executor/join/hash_join_v1.go
  • pkg/distsql/select_result_test.go
  • pkg/metrics/explain_ru.go
  • docs/design/2026-07-01-read-billing-demo-ru-model.md
  • pkg/executor/adapter_internal_test.go
  • pkg/util/execdetails/runtime_stats.go
  • pkg/util/stmtsummary/read_billing.go
  • pkg/executor/explain_test.go
  • pkg/planner/core/common_plans_test.go
  • pkg/executor/adapter.go
  • pkg/planner/core/explain_ru.go

Comment thread docs/design/2026-07-22-preview-ru-resource-formula-plan.md
Comment on lines +96 to +112
func (htc *hashTableContext) hashStateRows() uint64 {
var rows uint64
if htc == nil || htc.hashTable == nil {
return 0
}
for _, table := range htc.hashTable.tables {
if table != nil && table.rowData != nil {
validRows := table.rowData.validKeyCount()
if math.MaxUint64-rows < validRows {
return math.MaxUint64
}
rows += validRows
}
}
return rows
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Reject hash-state overflow instead of saturating it.

The v4 contract requires overflow rejection/fail-closed behavior, but this returns math.MaxUint64 and Line 1348 records it as a valid hash-state count. Propagate an overflow status so hash_state_rows is suppressed with a bounded invalid reason rather than billing an invalid saturated value.

Also applies to: 1347-1349

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/executor/join/hash_join_v2.go` around lines 96 - 112, The hashStateRows
method currently saturates overflow to math.MaxUint64, allowing an invalid count
to be recorded by the hash_state_rows path around the caller at lines 1347-1349.
Change hashStateRows and its caller to propagate an overflow/failure status,
suppress hash_state_rows when overflow occurs, and record only the existing
bounded invalid reason instead of billing the saturated value.

Comment thread pkg/session/tidb_test.go

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/design/2026-07-22-preview-ru-resource-formula-plan.md`:
- Around line 30-35: Normalize all progress-audit timestamps to a single
timezone, preferably UTC, and reorder the checklist entries chronologically by
their actual instants. Preserve the existing audit details and completion
statuses while ensuring each subsequent entry is later than the preceding one.

In `@pkg/executor/adapter_test.go`:
- Around line 745-774: Strengthen the assertions in the point-lookup
completion-entry test around rawUnits so every unit has op_class equal to
"kv_point_lookup", rather than skipping other classes. Preserve the existing
field and count validations, ensuring the test fails when any unrelated
operation class is emitted.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 0e77f174-1a3c-45d1-a9f1-cad04b773f18

📥 Commits

Reviewing files that changed from the base of the PR and between a1b928b and 6dce456.

📒 Files selected for processing (32)
  • .agents/skills/tidb-test-guidelines/references/distsql-case-map.md
  • .agents/skills/tidb-test-guidelines/references/executor-case-map.md
  • .agents/skills/tidb-test-guidelines/references/metrics-case-map.md
  • .agents/skills/tidb-test-guidelines/references/planner-case-map.md
  • .agents/skills/tidb-test-guidelines/references/session-case-map.md
  • .agents/skills/tidb-test-guidelines/references/util-case-map.md
  • docs/design/2026-07-01-read-billing-demo-ru-model.md
  • docs/design/2026-07-10-preview-ru-tidb-kv-mutation.md
  • docs/design/2026-07-22-preview-ru-resource-formula-plan.md
  • pkg/distsql/select_result_test.go
  • pkg/executor/adapter.go
  • pkg/executor/adapter_internal_test.go
  • pkg/executor/adapter_test.go
  • pkg/executor/explain_test.go
  • pkg/executor/join/hash_join_stats.go
  • pkg/executor/join/hash_join_v1.go
  • pkg/executor/join/hash_join_v2.go
  • pkg/executor/join/hash_table_v1.go
  • pkg/executor/join/hash_table_v1_test.go
  • pkg/executor/join/join_stats_test.go
  • pkg/infoschema/tables.go
  • pkg/metrics/explain_ru.go
  • pkg/metrics/metrics_internal_test.go
  • pkg/planner/core/BUILD.bazel
  • pkg/planner/core/common_plans_test.go
  • pkg/planner/core/explain_ru.go
  • pkg/session/tidb_test.go
  • pkg/util/execdetails/execdetails_test.go
  • pkg/util/execdetails/runtime_stats.go
  • pkg/util/stmtsummary/read_billing.go
  • pkg/util/stmtsummary/read_billing_test.go
  • pkg/util/stmtsummary/v2/record_test.go
🚧 Files skipped from review as they are similar to previous changes (27)
  • .agents/skills/tidb-test-guidelines/references/metrics-case-map.md
  • .agents/skills/tidb-test-guidelines/references/session-case-map.md
  • pkg/executor/join/hash_table_v1.go
  • .agents/skills/tidb-test-guidelines/references/planner-case-map.md
  • pkg/executor/join/hash_table_v1_test.go
  • .agents/skills/tidb-test-guidelines/references/distsql-case-map.md
  • pkg/util/stmtsummary/read_billing_test.go
  • pkg/executor/join/hash_join_v1.go
  • pkg/executor/join/hash_join_v2.go
  • pkg/planner/core/BUILD.bazel
  • pkg/util/stmtsummary/v2/record_test.go
  • pkg/metrics/explain_ru.go
  • pkg/executor/adapter_internal_test.go
  • .agents/skills/tidb-test-guidelines/references/util-case-map.md
  • docs/design/2026-07-10-preview-ru-tidb-kv-mutation.md
  • pkg/util/execdetails/execdetails_test.go
  • pkg/infoschema/tables.go
  • pkg/metrics/metrics_internal_test.go
  • pkg/executor/join/hash_join_stats.go
  • pkg/session/tidb_test.go
  • docs/design/2026-07-01-read-billing-demo-ru-model.md
  • pkg/executor/adapter.go
  • pkg/distsql/select_result_test.go
  • pkg/util/stmtsummary/read_billing.go
  • pkg/executor/explain_test.go
  • pkg/planner/core/explain_ru.go
  • pkg/planner/core/common_plans_test.go

Comment on lines +30 to +35
- [x] (2026-07-22 20:55Z) Completed the implementation Ready profile: `make bazel_prepare`, all six prescribed failpoint-wrapped package test sets, the tagged metrics test, `make lint`, and `git diff --check` passed; the HashJoin set additionally ran `TestHashRowContainer` for V1 NAAJ state coverage.
- [x] (2026-07-22 22:10Z) A completion audit found and fixed a v4 migration regression: the write constructor had stopped consulting the existing statement mutation snapshot's `Pipelined` flag. Pipelined DML and COMMIT now retain mutation evidence but fail the write-request component closed with `pipelined_tikv_payload_unsupported`; focused DML and COMMIT regressions cover the restored gate.
- [x] (2026-07-22 04:25+08:00) The final completion audit found that the ordering gate rejected residual scalar functions but did not prove that column `ByItems` belonged to the direct child schema. Sort/TopN now validate the executed flat child, including Projection expression/schema alignment, and a regression was observed failing before the fix and passing afterward.
- [x] (2026-07-22 04:30+08:00) Recompleted the Ready gate after the ordering fix: the full planner/core and executor target sets passed with failpoint cleanup, the earlier same-state join/session/statement-summary/metrics runs remained applicable, `make lint` passed after the code edit, and `git diff --check` plus API/range review found no remaining mismatch.
- [x] (2026-07-22 05:35+08:00) A requirement-by-requirement completion audit found that the private weight container omitted its specified immutable version, so a test-only calibrated set could publish a total under the production uncalibrated label. Added version ownership and validation to the weight container, propagated its active version to every output, and observed the focused regression fail before the fix and pass afterward.
- [x] (2026-07-22 05:45+08:00) Recompleted the full Ready gate from the current worktree: `make bazel_prepare` produced no generated diff; all six failpoint-wrapped target groups, the HashJoin NAAJ coverage, and tagged metrics tests passed; `make lint`, `git diff --check`, API-boundary searches, and the baseline-to-HEAD scope review passed. Failpoint refcount returned to zero after every run.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Keep the progress audit chronological.

These entries mix Z and +08:00; for example, 2026-07-22 04:25+08:00 is earlier than the preceding 2026-07-22 20:55Z. Normalize the timestamps or reorder the entries so the audit trail remains trustworthy.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/design/2026-07-22-preview-ru-resource-formula-plan.md` around lines 30 -
35, Normalize all progress-audit timestamps to a single timezone, preferably
UTC, and reorder the checklist entries chronologically by their actual instants.
Preserve the existing audit details and completion statuses while ensuring each
subsequent entry is later than the preceding one.

Comment on lines +745 to +774
entries := recorded.TakeAll()
var completionEntries []observer.LoggedEntry
for _, entry := range entries {
if entry.Message == "GENERAL_LOG_RU_UNITS" {
completionEntries = append(completionEntries, entry)
}
}
require.Len(t, completionEntries, 1)
requireReadBillingDemoGeneralLogIdentity(t, completionEntries[0], tc.querySQL)
rawUnits, ok := completionEntries[0].ContextMap()["units"].([]any)
require.True(t, ok)
pointUnits := 0
for _, rawUnit := range rawUnits {
unit, ok := rawUnit.(map[string]any)
require.True(t, ok)
if unit["op_class"] != "kv_point_lookup" {
continue
}
pointUnits++
require.Equal(t, "tikv", unit["site"])
require.Equal(t, tc.operatorKind, unit["operator_kind"])
require.Equal(t, "read_request_count", unit["unit"])
require.Equal(t, "ruv2_metrics", unit["input_source"])
require.Equal(t, "all", unit["input_side"])
require.Equal(t, float64(tc.requests), unit["value"])
}
require.Equal(t, 1, pointUnits)
})
}
})

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Test doesn't actually verify "point lookup rpc only".

The loop at lines 757-770 skips (continue) any unit whose op_class != "kv_point_lookup" and only checks pointUnits == 1 (line 771). It never asserts that rawUnits contains only point-lookup units, so a regression that emits extra, unrelated op_class entries for a point-get query would pass silently, despite the test name promising "point lookup rpc only".

🧪 Proposed fix to enforce exclusivity
 				require.Equal(t, 1, pointUnits)
+				require.Len(t, rawUnits, pointUnits, "expected units to contain only kv_point_lookup entries")
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
entries := recorded.TakeAll()
var completionEntries []observer.LoggedEntry
for _, entry := range entries {
if entry.Message == "GENERAL_LOG_RU_UNITS" {
completionEntries = append(completionEntries, entry)
}
}
require.Len(t, completionEntries, 1)
requireReadBillingDemoGeneralLogIdentity(t, completionEntries[0], tc.querySQL)
rawUnits, ok := completionEntries[0].ContextMap()["units"].([]any)
require.True(t, ok)
pointUnits := 0
for _, rawUnit := range rawUnits {
unit, ok := rawUnit.(map[string]any)
require.True(t, ok)
if unit["op_class"] != "kv_point_lookup" {
continue
}
pointUnits++
require.Equal(t, "tikv", unit["site"])
require.Equal(t, tc.operatorKind, unit["operator_kind"])
require.Equal(t, "read_request_count", unit["unit"])
require.Equal(t, "ruv2_metrics", unit["input_source"])
require.Equal(t, "all", unit["input_side"])
require.Equal(t, float64(tc.requests), unit["value"])
}
require.Equal(t, 1, pointUnits)
})
}
})
entries := recorded.TakeAll()
var completionEntries []observer.LoggedEntry
for _, entry := range entries {
if entry.Message == "GENERAL_LOG_RU_UNITS" {
completionEntries = append(completionEntries, entry)
}
}
require.Len(t, completionEntries, 1)
requireReadBillingDemoGeneralLogIdentity(t, completionEntries[0], tc.querySQL)
rawUnits, ok := completionEntries[0].ContextMap()["units"].([]any)
require.True(t, ok)
pointUnits := 0
for _, rawUnit := range rawUnits {
unit, ok := rawUnit.(map[string]any)
require.True(t, ok)
if unit["op_class"] != "kv_point_lookup" {
continue
}
pointUnits++
require.Equal(t, "tikv", unit["site"])
require.Equal(t, tc.operatorKind, unit["operator_kind"])
require.Equal(t, "read_request_count", unit["unit"])
require.Equal(t, "ruv2_metrics", unit["input_source"])
require.Equal(t, "all", unit["input_side"])
require.Equal(t, float64(tc.requests), unit["value"])
}
require.Equal(t, 1, pointUnits)
require.Len(t, rawUnits, pointUnits, "expected units to contain only kv_point_lookup entries")
})
}
})
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/executor/adapter_test.go` around lines 745 - 774, Strengthen the
assertions in the point-lookup completion-entry test around rawUnits so every
unit has op_class equal to "kv_point_lookup", rather than skipping other
classes. Preserve the existing field and count validations, ensuring the test
fails when any unrelated operation class is emitted.

@ti-chi-bot

ti-chi-bot Bot commented Jul 24, 2026

Copy link
Copy Markdown

[FORMAT CHECKER NOTIFICATION]

Notice: To remove the do-not-merge/invalid-title label, please follow title format, for example pkg [, pkg2, pkg3]: what is changed or *: what is changed.

The title description (the part after :) must be between 1 and 320 characters. Please check if your title exceeds this limit.

📖 For more info, you can check the "Contribute Code" section in the development guide.


Notice: To remove the do-not-merge/needs-linked-issue label, please provide the linked issue number on one line in the PR body, for example: Issue Number: close #123 or Issue Number: ref #456.

📖 For more info, you can check the "Contribute Code" section in the development guide.


Notice: To remove the do-not-merge/needs-tests-checked label, please finished the tests then check the finished items in description.

For example:

Tests <!-- At least one of them must be included. -->

- [x] Unit test
- [ ] Integration test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No code

‼️ Must keep the HTML comments <!-- At least one of them must be included. -->

📖 For more info, you can check the "Contribute Code" section in the development guide.

@ti-chi-bot

ti-chi-bot Bot commented Jul 24, 2026

Copy link
Copy Markdown

@winoros: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-mysql-client-test 89d0d65 link true /test pull-mysql-client-test
idc-jenkins-ci-tidb/check_dev 89d0d65 link true /test check-dev
pull-lightning-integration-test 89d0d65 link true /test pull-lightning-integration-test
pull-integration-e2e-test 89d0d65 link true /test pull-integration-e2e-test
idc-jenkins-ci-tidb/mysql-test 89d0d65 link true /test mysql-test
pull-build-next-gen 89d0d65 link true /test pull-build-next-gen
pull-br-integration-test 89d0d65 link true /test pull-br-integration-test
idc-jenkins-ci-tidb/build 89d0d65 link true /test build
pull-mysql-client-test-next-gen 89d0d65 link true /test pull-mysql-client-test-next-gen
idc-jenkins-ci-tidb/check_dev_2 89d0d65 link true /test check-dev2
pull-integration-realcluster-test-next-gen 89d0d65 link true /test pull-integration-realcluster-test-next-gen
idc-jenkins-ci-tidb/unit-test 89d0d65 link true /test unit-test
pull-unit-test-next-gen 89d0d65 link true /test pull-unit-test-next-gen

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. do-not-merge/invalid-title do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked release-note-none Denotes a PR that doesn't merit a release note. sig/planner SIG: Planner size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants