Phase 3: Build consistency audit harness and replay diagnostics

[inaniloquentee]

Background

Split from #6 Phase 3: Consistency Audit Harness.

This issue turns the Phase 1/2 metadata and telemetry surfaces into a reusable audit/replay harness. It should consume the existing or in-flight Phase 1 records instead of redefining them, and it should use the Phase 2 linear_logp decision/fallback telemetry where available.

Scope

• Build a shared audit result path for active-token dlogp, ratio0, clipfrac0, approx_kl0, and percentile summaries, reusing Phase 1 diagnostic definitions where they already exist.
• Add a read-only teacher-forcing replay path that does not resample, regenerate, run optimizer steps, or mutate model state.
• Add replay/debug dumps for selected rollout batches.
• Add per-rank diagnostics for distributed-capable audit runs, without introducing new distributed operator sweeps here.
• Validate existing metadata and fingerprints before drift attribution:
  • checkpoint or weight version
  • tokenizer and token IDs
  • active masks and padding/layout semantics
  • sampling settings
  • position/cache metadata
  • quantization settings
  • rollout ID, session ID, model name, and weight-update status
• Validate requested-vs-actual provenance and reject undeclared fallback in strict cases.
• Add batch-invariance replay cases for the same sample alone vs mixed batch, padding/packing variants, active-token-density variants, and dynamic-sampling keep/drop variants.
• Add a lightweight result-cube output indexed by normalized axes such as batch layout, dtype, router/cache policy, TP/SP/CP, logp backend, and deterministic policy.

Non-Goals

• Do not add or replace operator kernels.
• Do not redo Phase 1 metadata schemas, capability records, or basic execution-decision plumbing.
• Do not redo Phase 2 linear_logp productionization or strict fallback checks.
• Do not run TP/SP/CP distributed operator attribution; that belongs to Phase 5.
• Do not treat the Blackwell mismatch baseline in [Experiment] Build Blackwell train-inference mismatch baseline for VIME post-training #24 as completed by this harness. [Experiment] Build Blackwell train-inference mismatch baseline for VIME post-training #24 may consume this harness later, but remains a separate experiment.

Acceptance Criteria

• A zero-update single-GPU run reports max_abs_dlogp within the RL-Kernel per-dtype tolerance over active response/action tokens.
• Known metadata mismatches are caught before operator drift is blamed.
• Same-sample dlogp remains within tolerance across the covered batch-layout replay cases declared by the contract.
• Audit mode is read-only and does not change training behavior.
• Reports include enough requested-vs-actual provenance to explain native, optimized, reference, and fallback paths.

Dependencies

• Roadmap: RL-Kernel integration roadmap for vime #6.
• Phase 1 surfaces: Phase 1: add RL-Kernel fast/consistency mode parsing #7, Phase 1: capability-aware execution decisions and numeric contracts #8, Phase 1: consistency metadata, Data Buffer, and provenance fingerprints #10, Phase 1: add native audit-only dlogp diagnostics #15 and their corresponding PRs.
• Phase 2 linear_logp surfaces: Phase 2: Productionize linear_logp backend support and telemetry #20, Phase 2: Add linear_logp strict-mode checks and fallback tests #21 and their corresponding PRs.