This repository is the working root for Chess Meta-control (CMC)—research that combines large-scale human chess analytics in human_analytics/ with a neural meta-controller in lmcos/ (learned metacontrol over search). The chronological experiment log is labnotebook.md; stable analysis write-ups live under reports/. This README is written so that a reader (or an AI agent doing literature search) can recover intent, formal objectives, training protocols, and connections to prior work without re-deriving them from the code alone.
Objective (control layer). Build a meta-controller that manages the trade-off between thinking (expanding a Leela/lc0 search tree) and acting (playing a move). At the control layer this is an optimal stopping problem: is the move-quality we might discover worth the compute we are about to spend? The lmcos stack implements this as representation learning first (GNN over search trees), then offline fitted-Q / advantage regression on teacher traces—not yet full self-play PPO at production scale.
Primary code locations
| Path | Role |
|---|---|
chess_analysis/ |
DuckDB, figures; Slidev deck lives under human_analytics/presentations/ |
chess_analysis/human_analytics/ |
Human analytics entry points (movetime_analysis.py, …) and utils/ library |
chess_analysis/human_analytics/metacontrol/ |
Modular tree-search export (refactored from lmcos/) |
chess_analysis/human_analytics/slurm/scripts/ |
Pipeline CLIs (preprocess, engine eval, joins) |
chess_analysis/human_analytics/slurm/ |
Shell/Sbatch orchestration that calls slurm/scripts/*.py |
chess_analysis/labnotebook.md |
Chronological log (Description · Rationale · Status · Reference) |
chess_analysis/reports/ |
Stable analysis reports (R-* refs) |
chess_analysis/lmcos/ |
Tree encoder, offline controller training; src/ (cts package), analysis/ (cts.analysis), slurm/ (stage scripts + slurm/configs/ run YAMLs) |
chess_analysis/human_analytics/presentations/lmcos-overview/ |
Slidev deck: motivation, method, human validation |
For environment setup, Stockfish paths, and notebook entry points, see this file and chess_analysis/human_analytics/README.md. The latter documents code layout (human_analytics/ vs slurm/scripts/), the behavioral pipeline, and figure conventions.
Pipeline / DuckDB (preprocess.py): DuckDB spill and staged parquet files follow one directory per step (work_dir for get_games and process_moves, staging_dir for shard extract and merge into moves). After parquets land, merge only builds moves; process_moves builds processed_moves / processed_moves_nonzero. preprocess.sh runs both. Defaults live in preprocess.py main() config, not **kwargs plumbing—see chess_analysis/human_analytics/README.md §4.
The motivating picture is a planning system that keeps explicit tree search as scaffolding but replaces hand-written search-control rules with neural decision modules. At a high level:
- A meta-controller chooses: act in the real environment (play a move) vs plan inside an internal search tree.
- The act path uses a policy at the root of the current tree; the plan path uses a planning head over a learned tree representation to choose planning operations (navigation, expansion, evaluation) that update the tree through a learned world model and value feedback, before committing to a real move.
- In principle, such a system could be trained in an AlphaZero-style loop with self-play.
The implementation is deliberately narrower: meta-control of search only (when to keep expanding vs when to halt and play), on teacher-generated search trees and offline targets. Trajectories are snapshots of a growing search (e.g. from Leela / lc0-style search on positions sampled from Lichess with simple filters). Supervision is derived from counterfactual value-of-computation: comparing halting at each expansion step to continuing, under a defined continue cost and halt rewards from the search state (see §3).
Central empirical questions include: Can a simple halt/continue policy learn (near-)optimal control given a TreeNN encoding? Which encoding or cost architecture (linear vs budget-aware) supports learning? How does behavior relate to human time allocation and engine-based VOC (value of computation) from the behavioral track?
A future layer is a full planning head (which node to expand, etc.) on the same representation; see labnotebook.md and (R-LMCOS-OVERVIEW) for the roadmap.
Work under chess_analysis/human_analytics/ treats chess as a natural experiment in resource allocation: move time is heavy-tailed; remaining clock and position complexity both predict thinking time, with a stable VOC effect (prospective engine gain vs shallow eval) and characteristic ply-stage “arc” of deliberation. Slides in human_analytics/presentations/lmcos-overview/ connect this to resource-rational meta-control: humans adapt budgets to time pressure and to estimated benefit of search.
The lmcos line asks the complementary question: if we teach a network the statistics of a search tree, can it approximate the stopping rule implied by a formal cost–benefit model? That links behavioral VOC curves to machine metareasoning on trees.
Along one search episode, let snapshots be (s_0,\ldots,s_{K-1}) with step index (t) over expansions. The halt reward at (t), written (h_t), scores the quality of the move selected if the agent stops at (t). The project moved from a regret formulation to an absolute full-reference target: (h_t) is driven by the value of the best move at snapshot (t) under the full teacher search (e.g. (Q)-full), not by difference to the final best move only. That avoids degenerate small margins on intermediate steps (see (R-ARCH-LMCOS), 2026-04-05).
Let continue cost be (c > 0) per expansion (possibly extended to linear vs power cumulative costs via planning_cost.PlanningCostConfig). If the agent halts at step (j), a natural return is
[
R(j) = h_j - C(j),
]
where (C(j)) is the cumulative planning cost paid for continues before stopping (e.g. (C(j) = c\cdot j) for a linear per-step cost with (j) continues before halt at the same index in the lab’s indexing conventions—implementation details in planning_cost.py and controller_oracle.py).
The offline oracle for “halt vs continue” is obtained by backward dynamic programming on the finite horizon (see compute_oracle_policy in controller_oracle.py):
- At the last index, the value is the halt payoff.
- For each earlier (t), compare halt now vs continue (pay incremental cost, transition to (t+1)).
Actions are typically encoded with 0 = continue, 1 = halt. The optimal stop step is (\arg\max_j R(j)) under the same cost model.
Role of (c). The lab found that for real data, very small (c) (e.g. order (10^{-3})) yields a meaningfully adaptive oracle (nontrivial continuation rate), while larger (c) can collapse the oracle to immediate halt—so the same dataset supports different “economies of thought” depending on the cost scale.
Define Q-style targets: [ Q_{\mathrm{halt}}(s_t) = h_t, \qquad Q_{\mathrm{continue}}(s_t) = -c + V^(s_{t+1}), ] with (V^) the oracle value of following the optimal policy from the next snapshot (or the DP value in tabular form on the chain). The compute advantage (scalar actually trained in the fitted-Q / advantage path) is [ A_{\mathrm{compute}}(s_t) = Q_{\mathrm{continue}}(s_t) - Q_{\mathrm{halt}}(s_t). ] A greedy policy continues iff (A_{\mathrm{compute}}(s_t) > 0). Training minimizes MSE to Bellman-derived (A_{\mathrm{compute}}) to avoid “common-mode” value fitting that matches levels but not the decision boundary (see (R-ARCH-LMCOS), fitted-Q and advantage-only sections).
The scalar-(c) model ignores literal tree size and remaining time budget. The budgeted extension (budgeted_controller_oracle.py) augments the state with node count (N_t) and remaining planning budget (T_t). A step cost combines:
- Maintenance (superlinear in tree size by default), e.g. [ c_{\mathrm{maint}}(N) = s \left(\frac{N}{N_0}\right)^\eta ] with learnable/defaults for (s, N_0, \eta).
- Time cost from a discrete budget consumed each step, e.g. using a power-law-style incremental cost
[
c_{\mathrm{time}}(T) = \lambda\Big[ (T-1+\tau)^{-(p-1)} - (T+\tau)^{-(p-1)} \Big]
]
(see
time_costin code; when budget is exhausted, a timeout value is used).
The Bellman backup matches the same halt-vs-continue pattern with state-dependent continue cost. Packed episodes augment raw trajectories with synthetic starting budgets across buckets so the controller sees diverse ((N,T)) (manifest format v2 in the lab). The trained head often uses concat((z_t, N_t, T_t)) with encoder embedding (z_t).
episode_difficulty.py defines metrics (halt reward range, optimal vs second-best return gap, entropy over per-stop returns, regret of “halt at 0,” curvature of halt-reward differences) to characterize whether stopping time matters. Analysis showed many episodes have nearly flat return landscapes, motivating filtering (e.g. by halt reward range) before retraining.
Search trees are tensorized for GPU batching (tensorizer.py): a flat-forest layout with parent/child pointers, packed into fat shards for I/O efficiency. A TreeNN-style model (GNN.py, TreeMHA.py) runs rounds of message passing in topological order (not one synchronous blur of all neighbors):
- Upward sweep (child → parent): children aggregate via multi-head attention; each parent updates with a GRU (“what did my children discover?”).
- Downward sweep (parent → child): the parent summary is linearly projected and broadcast; each child updates with the same GRU (“given global context, how should my local move-vector change?”).
Slot encodings (sinusoidal / learned) disambiguate child order (canonical UCI ordering) for per-edge prediction heads—central to child-WDL pretraining, where every parent→child edge in a batch contributes a cross-entropy signal (not just the root), giving dense gradients for subtree structure.
- Mode (research): dynamic growth — run a full oracle search (e.g. large node budget), then take a prefix of the expansion sequence as input and consolidate deep statistics from the full tree as supervised targets (prefix / deep targets in
cts_pretrain.py). - Targets: Scalar value backups and, after fixes in 2026-04-10, search-consolidated per-edge WDL targets (visit-weighted, perspective-correct) stored as
edge_wdl_targets, not raw value-head slices at a node in isolation. - Prefix derivation:
derive_pretrain_prefixes.pycan subsample variable-size prefixes from existing fixed full trees without re-querying the engine (see (R-ARCH-LMCOS)).
Large-scale flow: generate many .pt PretrainExample / raw examples (cluster) → pack to shards → pretrain encoder (e.g. child-WDL) → pack controller episodes (with budget augmentation) → train halt/continue head. Job templates and run YAMLs live under lmcos/slurm/ (slurm/configs/<stage>/). Slurm stdout/stderr go to flat slurm/logs/; per-run metrics YAML, curve PNGs, and comparison plots go to flat slurm/outputs/<stage>/ (tracked in git). Stage 4 ablation configs are submitted via ./slurm/4_supervised_controller/submit_configs.sh (glob all YAMLs in slurm/configs/4_supervised_controller/). See lmcos/slurm/README.md and (R-LMCOS-STAGE4).
The project has been refactored into a modular structure under analysis/metacontrol/ to enforce strict decoupling and didactic clarity:
core/: Fundamental data structures (tree.py,tensorizer.py) and schemas (schemas.py).data/: Pipeline logic for tree generation (generator.py), meta-control DP derivation (targets_mc.py), and GNN target computation (targets_gnn.py).tutorials/: Didactic notebooks and generation scripts demonstrating the full pipeline.tests/: Comprehensive integration and unit tests for the pipeline.
The new TreeSearch class in generator.py provides a clean, method-based API for tree growth, while targets_mc.py and targets_gnn.py separate the derivation of training targets for the controller and the GNN respectively.
Root sampling and single-tree export (2026-05):
- Lichess roots:
analysis/metacontrol/scripts/sample.py(ChessSampler) writes filtered FEN rows; an example row is kept at/scratch/gpfs/GRIFFITHS/hl4291/data/metacontrol_example.csvwhen integration tests run. - One-tree pipeline:
analysis/metacontrol/scripts/generate_and_profile.pyreads that CSV (or--csv), runsTreeSearch+ targets +TreeTensorizer, and saves/scratch/gpfs/GRIFFITHS/hl4291/data/trees/example_tree_00001.ptby default. Use--no-profilefor wall time only; otherwise it prints cumulativecProfilestats and writesgenerate_and_profile.pstatsalongside the shard. .ptlayout vsysagiv: Legacy controller shards on the cluster useformat="cts_budgeted_controller_episode_shard_v4"with RL replay pointer tensors (trajectory_node_ptr,episode_step_ptr, …) and flattenedtarget_advantagesacross many trajectories. Metacontrol’s Phase~1 export usesformat="metacontrol_single_tree_v1", the same five-widenode_features(value, WDL,wdl_varpad),edge_childasint32, and omits trajectory pointers. It adds explicit tensorsmc_halt_rewards,mc_dp_values,edge_wdl_targets, and per-snapshottarget_advantages;oracle_valueshere are per-node GNN consolidated values (not the legacy per-episode oracle scalars). Seeanalysis/metacontrol/README.mdandanalysis/metacontrol/migration.md.
The pipeline now features a generalized engine provider system that supports both LC0 and Stockfish (and any other UCI-compatible engine).
UciExpansionProvider: Centralizes subprocess management and UCI communication.LC0ExpansionProvider: Parses specialized statistics likeVerboseMoveStatsand WDL.StockfishExpansionProvider: Handles standard UCI info and usesmultipvrank-based priors.
This abstraction ensures that the TreeSearch logic remains engine-agnostic, allowing research to focus on the search dynamics rather than engine-specific parsing.
- Mathematical Parity and Perspective Correctness:
The pipeline enforces strict sign and perspective consistency. Historical discrepancies in teacher targets (where winning positions were sometimes recorded with negative values) have been resolved.
- Expansion: Engine Q-values (reported from parent perspective) are flipped (
-Q) when stored in child nodes to maintain local consistency. - Backpropagation: Values are flipped again during path traversal, restoring the original engine intent at the parent edge.
- WDL Targets: Normalized WDL probabilities are flipped (
win <-> loss) at each level to ensure they always reflect the side-to-move. - Parity Tests: A dedicated suite (
test_legacy_parity.py) ensures that new generation logic matches the mathematical core of the legacy system while correcting its sign conventions.
- Expansion: Engine Q-values (reported from parent perspective) are flipped (
The project maintains a comprehensive test suite (analysis/metacontrol/tests/) that covers unit, engine, and integration scenarios (run pytest analysis/metacontrol/tests -m "not integration" for a fast slice; include test_sampler.py for DuckDB smoke tests):
- Core Logic & Target Derivation:
test_targets_mc.pyverifies the DP algorithm, including a simulation demonstrating that when a tree expansion discovers a mate-in-2, the backward DP properly assigns a massive positive "continue advantage" to earlier snapshots.test_targets_gnn.pyensures target consolidation is accurate, validating that terminal checkmate states correctly map to a WDL of(1.0, 0.0, 0.0)from the parent's perspective.
- Provider Accuracy (
test_providers.py): Engine-invariant tests for both LC0 and Stockfish. These confirm that terminal states (like Fool's Mate or Stalemate) yield strictly consistent WDL evaluations regardless of which side is checkmated, validating the localChildPerspectiveinvariant. - Search Efficacy (
test_search_quality.py): End-to-end validation using real engine binaries to confirm the search process correctly identifies tactical wins (Scholar's Mate, Back Rank Mate, Arabian Mate, Damiano's Mate, Morphy's Puzzle, Philidor's Smothered Sequence). It explicitly tests the PUCT exploration mechanics, for instance showing that a quiet mate-in-2 might be overlooked with a lowc_puctbut is confidently surfaced whenc_puctis raised. - Divergence Analysis: Documented evidence proving that the fixed pipeline is structurally superior to the legacy system, which previously avoided winning moves due to a sign-inversion bug.
The pipeline has been "hardened" for production-scale data generation:
- Perspective Parity: Confirmed via
test_legacy_mates.pythat the pipeline correctly implementsChildPerspective = -ParentQ, fixing the "search blindness" of the legacy code. - Engine Robustness: Implemented dynamic
MultiPVcapping and explicit UCI flushing to prevent engine segmentation faults (notably in Stockfish 15) and ensure stable communication. - Engine-Invariant Testing: Both LC0 and Stockfish are verified through the same parameterized test suites for both terminal states and deep tactical searches.
Run the full validation suite:
pytest analysis/metacontrol/tests/core/test_providers.py analysis/metacontrol/tests/data/test_search_quality.py analysis/metacontrol/tests/data/test_pipeline.py analysis/metacontrol/tests/data/test_generator.pysupervised_branch.py / cts_rl.py implement PPO (and baselines) on frozen-encoder or unfrozen encoders, with Bernoulli halt/continue, value head, and diagnostics (approx kl, clipfrac, explained variance). A rollout collection bug was fixed so batch size is ( \texttt{num_envs} \times \texttt{rollout_steps} ) (regression test in test_supervised_branch.py).
scripts/rl_sanity_checks.py validates sign of updates and logit drift on toy tasks—used to rule out gradient wiring bugs when long runs are flat.
scripts/train_fitted_q_controller.py fits scalar (A_{\mathrm{compute}}) (or budget-aware MLP on features) with MSE to DP targets, with greedy evaluation and optional diagnostics JSONL. Materialization caches for frozen-encoder root embeddings avoid repeating heavy encoding each epoch when manifests/checkpoints are unchanged.
probe_controller_representation.py: Train a small MLP/linear probe on frozen root embeddings to predict oracle action or regress advantage—tests whether the representation carries the halting signal without full RL.controller_representation_control.py: Trivial or oracle-derived features (e.g. oracle-action-now one-hot) to isolate PPO vs representation; includes one-step bandit and supervised checks.
- PPO on real trees can be stable but over-search relative to the offline oracle at low (c); validation return can plateau early.
- Unfreezing the encoder in one large run did not improve validation vs frozen.
- Fitted advantage on frozen embeddings can get return near oracle but poor exact stop-step / sign unless data are filtered to nontrivial episodes; async vs sync encoders can differ on filtered data.
- Budgeted packing and training are the current intended path for state-aware costs.
These are hypothesis-generating outcomes; see labnotebook.md and (R-A0) for numbers and run IDs.
| Resource | Content |
|---|---|
analysis/presentations/lmcos-overview/ |
Motivation, architecture slides, human clock/VOC figures |
labnotebook.md |
Chronological experiment log |
reports/ |
Stable R-* analysis reports |
Analysis notebooks mentioned in the lab (regret_landscape.ipynb, episode_difficulty_analysis.ipynb) live alongside packed diagnostics on analysis machines.
The project sits at the intersection of several named research areas. Useful query families and paper clusters:
- Metareasoning / rational metareasoning (Horvitz, Russell & Wefald, Han & Stewart): deciding when to stop deliberation; anytime algorithms; value of computation as an explicit object.
- Optimal stopping on finite horizons: backward induction, threshold policies, regret vs reward parameterizations (connections to §3).
- GNNs on trees / graphs for search: TreeLSTM, GAT, graph transformers, UCT + learned value (AlphaZero, MuZero, Sampled MuZero); neural MCTS and search policy learning.
- Reinforcement learning for budgeted or horizon problems: PPO, REINFORCE, contextual bandits (one-step diagnostics in this repo); Offline RL and imitation of oracles; Fitted Q-Iteration / Bellman regression for discrete control on fixed datasets.
- Chess and game AI: MCTS + NN (Leela, Stockfish NNUE as non-learned control); adaptive search time in engines.
- Cognitive / psychophysics of chess (if linking to human data): time pressure, depth of search vs task difficulty; the behavioral scripts formalize VOC and clock elasticity.
Contrast this work: offline teacher trees, no full self-play loop yet, explicit cost models (linear or budgeted) with DP oracles rather than end-game outcome only.
- Environment: hl4291 della uses
/home/hl4291/venvfor stage-4 jobs (VENV_DIR); ysagiv jobs may usects_supervisedconda. Activate locally withsource .venv/bin/activatefrom repo root when available. - Tests:
lmcos/tests/cover plumbing, oracles, fitted-Q, probes; runpytest tests/fromlmcos/withPYTHONPATHset. - Sync: When copying to clusters, the lab notes using
rsync -avRto avoid sparse directory mistakes.
For day-to-day commands: human_analytics/README.md (DuckDB ETL, figures); lmcos/slurm/README.md (pipeline Slurm); labnotebook.md (experiments).
| Area | Lead | Scope |
|---|---|---|
| Model development and analysis | Yotam | lc0 tree generation, GNN architecture (asynchronous sweeps, child-decodability pretraining), metacontroller training (pipeline design and implementation) and mech interp |
| Validation & psychology | Jordan | Baselines, human alignment (move time, clock regimes), controller ablations, diagnostics |
- Data path — Leela/lc0 trees → tensorization → GNN forward pass (stages 1–3 under
lmcos/slurm/). - Supervised encoder — prefix vs oracle WDL; frozen
tree_encodercheckpoint. - Offline meta-control — fitted advantage / budgeted oracle on materialized caches; greedy regret eval.
- Psychometric validation — compare learned stopping to human Lichess move-time and engine VOC.
Current production focus is stage 3–4 on ysagiv read-only caches (hl4291 della, VENV_DIR=/home/hl4291/venv).
| Concept | Module / path |
|---|---|
| Tree generation & encoder pretrain CLI | cts.data.build_tree |
| Controller episode pack / materialize | cts.data.preprocess_mc |
| Fitted-Q controller train + metrics plots | cts.train.controller_train |
| Slurm stage wrappers + run YAMLs | lmcos/slurm/, lmcos/slurm/configs/ |
| Post-hoc budgeted-run analysis | cts.analysis.analyze_budgeted_controller_run |
| Human analytics pipeline & figures | human_analytics/ (see human_analytics/README.md) |
Last updated 2026-06-04: experiment log at labnotebook.md + reports/; see (R-LMCOS-STAGE4) for the 2026-05-29 controller harness.