Skip to content

Add end-to-end offline tutorial notebook - #2

Closed
jsp2195 wants to merge 2 commits into
generate-complete-llm-training-stack-repofrom
codex/generate-complete-llm-training-stack-repo
Closed

Add end-to-end offline tutorial notebook#2
jsp2195 wants to merge 2 commits into
generate-complete-llm-training-stack-repofrom
codex/generate-complete-llm-training-stack-repo

Conversation

@jsp2195

@jsp2195 jsp2195 commented Feb 19, 2026

Copy link
Copy Markdown
Owner

Motivation

  • Provide a single, runnable Jupyter tutorial that demonstrates the repository's full offline workflow end-to-end (tokenizer → pretrain → eval → SFT → optional DPO → inference) using only local toy data and in-repo APIs.
  • Make it easy for users to exercise the stack on CPU or GPU with modest default runtimes and clear knobs to shorten/lengthen runs.

Description

  • Add exactly one file: notebooks/end_to_end_tutorial.ipynb, containing labeled sections for Setup, Tokenizer training, Pretraining (tiny GPT), Evaluation (perplexity + probes), SFT, optional DPO (toggle), and a minimal greedy generation demo.
  • The notebook performs robust repo-root detection, imports from src/llmstack modules, writes artifacts under artifacts/ and runs/, sets a deterministic seed via seed_everything, and auto-selects device/precision (bf16/fp16 on CUDA, fp32 on CPU).
  • Runtime knobs (PRETRAIN_STEPS, SFT_STEPS, DPO_STEPS, RUN_DPO) are centralized and modest by default (pretrain ~80 steps, SFT ~30, DPO ~20) to keep CPU runs feasible, and all paths are relative to the repository root.
  • The notebook is offline-first and does not download external models or datasets, and it uses the repo's tokenizer wrapper, DataModule, Trainer, evaluate_perplexity, run_probes, and post-training helpers where appropriate.

Testing

  • Confirmed the notebook file was written and is valid JSON by parsing it with a short Python snippet that printed nbformat, nbformat_minor, and the cell count (script executed successfully).
  • Displayed the notebook contents to verify the expected sections and that the primary runtime knobs and output paths (artifacts/ and runs/) are present.
  • No automated unit tests in tests/ were modified, and CI configuration was left untouched by this change; the notebook is intended for local interactive execution (offline) and was validated for offline-only behavior in the added cells.

Codex Task

@jsp2195 jsp2195 closed this Feb 19, 2026
@jsp2195
jsp2195 deleted the codex/generate-complete-llm-training-stack-repo branch February 19, 2026 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant