chore: release v0.2.0 - #7
Merged
Merged
Conversation
The previous PR added args.hand_mode references but forgot the argparse entry, so any
train_board_ocr invocation crashed with AttributeError before training could start. Wire
--hand-mode {classification,regression} through with the same default as the function signature.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…kill (#6) * feat(train): DDP + preload cache + backbone sweep + train-backbones skill Ship a bigger BoardOCR training toolkit while adding multi-GPU support. - DDP: torchrun-launched multi-GPU training via new dist_utils helpers. setup_distributed()/is_main()/all_reduce_mean() no-op outside torchrun so the same script runs on the 1-GPU dev box and the 8-GPU A100 target. SyncBatchNorm conversion for BN-heavy backbones; DistributedSampler with per-epoch shuffle; metrics reduced globally at epoch end; ckpt saved on rank 0 only. - Preload cache: HFCaptureDataset preload option decodes + resizes + pads every device rendering into an on-disk .npy at init and memory-maps it on subsequent runs. Fingerprint-keyed path handles --limit/config variations; atomic tmp-then-rename write; rank 0 builds + barrier so DDP workers share the mmap. - Backbone sweep: BoardOCR gains 7 new backbones (mobilenet_v3_large, efficientnet_b0/b1, convnext_atto/femto/pico/nano) alongside the existing small/tiny pair via a shared timm feature-wrapper for the sub-tiny convnext family. - Orchestration: scripts/train_backbones.sh runs the sweep in sequential, DDP, or PARALLEL_GPU=1 mode (one backbone per GPU wave). Adds RESUME_INCOMPLETE for pick-up-where-interrupted reruns. scripts/train_multi_gpu.sh is a torchrun one-shot for a single backbone. scripts/autotune.py probes VRAM/CPU and prints eval-able env vars sized for the detected hardware. - Skill: .claude/skills/train-backbones/SKILL.md drives the launch flow through AskUserQuestion (mode/image_size/epochs) + autotune, hiding the env-var incantation. - Data pipeline speedups (Fable-guided): cv2.imdecode replaces PIL (GIL- free WebP), INTER_AREA on downscale, IMREAD_COLOR_RGB skips a cvtColor copy, on-device metric accumulation cuts one sync per step, channels_last + bf16 autocast + fused AdamW when CUDA. Known followup: origin/master's hand_mode (classification/regression) and hf_config dispatcher on train_board_ocr were dropped by the merge — both should be re-integrated in a follow-up PR before the regression path is used in earnest. bump version to 0.2.0 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore: bump mito-train to 0.2.0 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(datasets): serialize cache builds with fcntl locks for parallel sweeps When multiple independent processes run in PARALLEL_GPU mode, they may race on the same cache path. Use fcntl advisory locks on a sibling .lock file to serialize the build: first process builds, others wait and then find the cache already exists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(backbones): add comprehensive backbone selection guide Documents all 9 supported backbones with parameter counts, model sizes across precisions, deployment-specific recommendations, detailed architecture specs, and a decision tree for backbone selection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(backbones): add deployment details and accuracy guidance Expand the backbone selection guide with: - Cloudflare Workers (self-hosted ONNX) and Workers AI deployment options - 2024+ smartphone capabilities (iPhone 15/16, Snapdragon 8 Gen 3/4, Pixel 9, midrange Android) - Cross-platform inference time comparison table - Accuracy requirements by use case (casual viewing, playback, analysis, official publication) - Cell accuracy vs sfen_full relationship and practical trade-offs - Factors affecting accuracy and real-world implementation compromises Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(backbones): clarify Cloudflare Workers ONNX deployment options Split Workers deployment guidance into three routes: workers-wonnx with WebGPU (recommended), Workers AI catalog (BYOM pending), and CPU-only (impractical). Add WebGPU inference benchmarks, 2026 pricing breakdown, and mobilenet_v3_small case study; update deployment table with realistic performance expectations. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(wandb): disable auto-collected system metrics Suppress auto-collected GPU/CPU/mem/disk metrics that clutter the Charts tab, and simplify the initialization logic. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: add OCR scaling outlook analysis from sweep results Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: document backbone sweep results and hand head bottleneck Results from 9-backbone sweep (50 epochs, image_size=224) show convnext_tiny reaches 0.716 sfen_full. Identify hand head as primary bottleneck: adjacent count misses (19-way CE issue) + distribution bias, not architecture limits. Reorder priorities to hand regression-head first. Mark count=10 gap resolved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: update hand distribution analysis with 2026-07-12 measurements Refresh TRAINING_PLAN.md and ocr-scaling-outlook.md with empirical data from current `ultemica/piyoshogi` OCR dataset. Updated slot label frequencies (65.68% zero labels), piece-wise non-zero rates, and class weight calculations. Confirms prior count=10 data gap is resolved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: add val distribution gap analysis from 2026-07-12 measurements Identifies val label imbalance and missing high-count entries limiting hand head evaluation. Proposes per-count recall and synthetic validation alternatives. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: tidy up wandb chart labels and extend board_ocr v3 plan - rename bare wandb.log keys (step, epoch, epoch_frac, lr) to train/* or step/* prefixed variants so nothing lands in wandb's default "Charts" section anymore. - extend TRAINING_PLAN.md with the v3 board_ocr recipe (HF 18k + synth 10k merge, class-weight-clip-max 15.0, epochs 50, cosine scheduler). - record convnext_nano sweep completion (run nyrsx25c, val sfen_full 0.660) in docs/ocr-scaling-outlook.md and update the Pareto verdict. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(lint): drop unused os import from train_board_ocr.py ruff F401 flagged mito_train/training/train_board_ocr.py:20. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(train): configure wandb to log experiments locally Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(devcontainer): increase shm_size to 32gb Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release v0.2.0
Ships the accumulated develop work to production.
Highlights
train-backbonesskill (feat(train): DDP + preload cache + backbone sweep + train-backbones skill #6)scripts/train_backbones.shsweep runner with sequential / DDP / parallel-GPU modesscripts/autotune.pydocs/backbones.md,docs/ocr-scaling-outlook.md, updatedTRAINING_PLAN.md--hand-modeCLI flag (fix(board_ocr): expose --hand-mode CLI flag #5)Version
0.2.0🤖 Generated with Claude Code