Cancer research · biomedical computing · reproducible machine learning
Authored by Paul Skeffington · Compiled with OpenAI
An open, methods-focused notebook for research across all cancer types, including molecular oncology, pathology/imaging, cancer epidemiology, computational biology and translational research. The current worked case uses a breast cancer teaching book and synthetic paired-specimen ML, with TCGA-BRCA as the current data-source example under consideration. Neither TCGA nor breast cancer limits this repository's research scope. Start with a research question, then build a documented, reproducibly checked analysis.
Choose your route: Task-first navigation · Learn by subject · Research resources · Computational oncology book. Start with the outcome you need, then use the linked method, example and verification evidence.
Machine-readable access: Agent discovery (llms.txt) · Indexing and retrieval contract — stable topic routes, provenance, source boundaries and RAG guidance.
Quick navigation: Start with your task · Browse learning tracks · Research scope · Research workflow · Run an example · Progress and limitations
This repository is a modular field guide for biomedical students and early-career researchers. Read one section to answer a current question, then follow prerequisites or related links when useful; there is no required course order. Each subject overview points to focused lessons, runnable examples, expected outputs, and limitations. Start with the task table below or browse by topic.
Start with the statistical method decision matrix, complete synthetic breast cancer study, or graph-method alignment and teaching gallery. Reusable plotting functions and their regression tests are documented in the R resource index. Sources and completed verification steps are distinguished from tests still awaiting execution in the acceptance ledger.
The Computational Oncology: A Clinician–Scientist's Guide is a developing ten-part, 60-chapter ebook with six drafted chapters, a publication roadmap, machine-readable chapter manifest, and section-linked reference registry. The full manuscript remains in development and has not undergone peer review.
Choose the route that matches what you need; no weekly sequence is required. Each topic can be used on its own, and the final column gives optional next steps.
| I want to… | Start with | Then explore |
|---|---|---|
| Choose a cancer research question | Cancer research scope — all cancer types, data modalities and study designs | Independent study/data-access contract and protocol template |
| Understand the current breast cancer example | Breast cancer book — 12 chapters and 60 gene profiles | Gene lookup and pathways |
| Learn Rust from zero | Rust foundations course — variables, borrowing, object-oriented patterns, Cargo and tests | Research-oriented Rust engineering and source-backed practices |
| Learn Git/GitHub from zero | Zero-to-push Git and GitHub tutorial — setup, sign-in, essential commands and safe first push | Branches, pull requests, errors and research repository safety |
| Learn introductory biomedical statistics | Data-type-first statistics guide — quantitative vs qualitative, counts, ordinal, survival, a test-selection flowchart and practice workbook | Every chart object: real paper and runnable R, recent methods evidence and R tools |
| Understand study types in real papers | Study designs and published examples — nine readings connecting sampling, timing, assignment and analysis | Source register and probability/sampling |
| Decide which analysis to run | Choosing an analysis: clinical research guide — measurement, design, estimands, and four worked examples | Study protocol and analysis record |
| Learn research computing | Computing by subject — R, data structure, joins, missingness and statistics | Worked analysis |
| See how ML actually trains | Visual ML data-flow guide — splits, preprocessing and held-out predictions | ML methods and plotting gallery |
| Run and review an experiment | Paired-specimen ML walkthrough | Comparison, tuning and validation evidence |
| Design reproducible research | Research workflow — scope, inputs, checks and reporting | Roadmap and gate evidence |
New to version control? Follow the complete command-line guide to install Git, sign in to GitHub, clone a repository or create your own, and upload your first changes. The basic cycle is:
git status
git switch -c docs/my-change # in an existing repository
# Edit README.md, then review and save the change:
git diff
git add README.md
git commit -m "docs: update README"
git push -u origin docs/my-changeThen open a pull request on GitHub for review and merging. These commands assume you have already cloned a repository, configured your Git identity and signed in with push access. Commit saves locally; push uploads; a pull request proposes a merge. Never commit credentials, patient-linked records or restricted research data. Start with the beginner guide if any prerequisite is unfamiliar.
Current worked-case question: Can invented assay measurements distinguish a recorded specimen category for people who were not used to train the model? This illustrates one oncology research workflow, not the only supported question. A different cancer, data modality or endpoint requires a separately defined study and evaluation protocol. This is not a clinical diagnostic or prognosis task.
People → specimens → measurements → preparation → training → validation → predictions → interpretation
- Know the unit of analysis. For the current example, read the breast cancer research narrative and data identifiers and joins. Specimens from the same person must stay together in evaluation splits.
- Prepare measurements without leakage. Use the visual data-flow guide to understand training-only imputation, scaling and feature selection.
- Compare methods fairly. Follow the paired assay comparison and model-tuning guide before interpreting scores.
- Explain and reproduce the result. Inspect the seven-figure gallery, evidence review and analysis record template.
Use these subject tracks as standalone references. Start with the overview for a map of the material, then open the lesson that matches your question.
| Track | Start here | Useful when you need to… |
|---|---|---|
| Biology and cancer context | Cancer research scope · Genomics resources | frame a biological question, interpret gene or pathway material, and distinguish examples from study findings |
| Command line and files | Unix and Bash | inspect files, automate a repeatable run, and understand command output and exit codes |
| Data and biomedical statistics | Computing subjects · Statistics decision guide | represent observations, join tables, handle missing data, choose an introductory method, and explain uncertainty |
| Research workflow | Research workflow · Study protocol template | define a study question, units, data access, analysis plan, and reproducible record |
| Machine learning | ML overview | understand models, compare methods, check leakage, evaluate predictions, and interpret limits |
| Figures and communication | Visualization resources | choose a plot, label it clearly, and communicate what the display does and does not show |
| Programming for biomedical work | Computing subjects · Rust course | learn a language or follow a runnable, research-oriented example |
| Git and collaboration | Git/GitHub tutorial | save changes safely, work on a branch, and propose a reviewable contribution |
- Read its question, prerequisites, and expected output before running code.
- Use only the stated teaching data or data you are authorized to analyze; patient data requires source-specific review and approval.
- Compare results with the expected output, change one assumption, and explain what changed.
- Record software versions, data provenance, choices, and limitations with the analysis record.
- Follow the overview links when a concept is unfamiliar. A guide does not mean every optional method has been executed or reviewed for every study.
Run the existing synthetic paired-specimen example from the repository root. Use Python and the pinned modeling requirements:
python3 -m venv .venv
.venv/bin/python -m pip install -r 04_machine_learning/requirements-python.txt
.venv/bin/python 04_machine_learning/examples/breast_cancer_feature_selection.py /tmp/pauls-notes-assay-exampleThis creates a new directory with a fitted pipeline, held-out assessment, feature-review table, environment versions and SHA-256 manifest. The output path must be new and outside the repository; choose another path if it already exists. The 60 people and 48 anonymous features in this example are invented teaching data, not TCGA patient records or the book's 60 named genes.
Variable-only synthetic ML: The configurable oncology teaching lab lets you change numeric generator, nested person-grouping, feature-selection and regularization settings in JSON and produce a new auditable research-only report. This remains simulated breast-specimen classification, not an adapter for TCGA or any other participant dataset.
Prefer a lighter introduction? Start with the computing walkthrough. Want an additional programming language? See Rust inference, native Rust training and the R/Python methods guide. Each guide states what has actually been executed.
| Area | Go to | What you'll find |
|---|---|---|
| Biology and genomics | Genomics resources | The book, gene biology, annotations and pathway context |
| Computing foundations | Unix and Bash · Computing subjects | Shell scripts, data types, reshaping, missingness and statistics |
| Machine learning | ML home | Algorithms, feature selection, comparisons, validation, Python, R and Rust |
| Visualization | R plotting tools · Annotated heatmaps and z-score tails · ML figure gallery | Reusable plots, annotations, gene-expression tails and interpretation |
| Research evidence | Biomedical coding · Rust engineering handbook · Literature resources | Source matrices, programming and verification rules, methods and quality checks |
| Writing and reproducibility | LaTeX and bibliography · Research workflow | Manuscripts, citations, provenance and execution records |
For a task-by-task index, use the research resource guide or complete resource directory.
- Milestones and evidence: Dated status dashboard and machine-readable gate record distinguish delivered software, draft code, data-access blockers, test evidence and scientific limits. Rust foundations, the book-aligned handbook, G3 inference parity (R3) and the Python configurable synthetic runner (R4) are delivered within bounded educational/software scope. Public-data privacy protections (D0) remain in draft review and TCGA-BRCA real-study admission (D1) is blocked.
- Established in this repository: synthetic software demonstrations for person-grouped model comparison, tuning, plotting, robustness, repeated-cohort assessment, conditional uncertainty, and a matched-objective Rust training check. See the roadmap gate ledger for what each test established and what it did not.
- Still separate work: the new G4 synthetic 24-month survival protocol and data dictionary are frozen without running a G4 survival model. G5 published-result reproduction requires its own source-specific protocol. Consult the learning roadmap rather than treating a passing test as evidence of clinical validity.
- Cancer data and privacy: Source, file-level access, licensing/consent, privacy and publication review are study-specific for every cancer dataset. The TCGA-BRCA example uses the GDC's open/controlled distinction; its metadata-only adapter does not authorize that dataset or other providers. Keep participant-linked records, controlled data and credentials out of this public repository. See cancer research scope and data/access contract.
- Model use: current examples are research and education only, not validated for any cancer diagnosis, screening, therapy selection, prognosis or clinical decision support. Synthetic classification results are not findings about human tumors.
This is the second major learning track in Paul's Notes: designing repeatable, inspectable agent-assisted workflows rather than relying on one-off prompts. The principles apply to Codex, hosted LLMs, and locally deployed models; model capability, tool access, and execution results must be verified separately.
| Component | Practical contract | Evidence to retain |
|---|---|---|
| Agent templates | Define objective, context, inputs, permitted tools, constraints, output schema, stop conditions and success criteria before execution. Keep task instructions versioned. | Template version, parameters, generated plan and final artifact |
| Reusable skills | Package one bounded capability per skill with a clear trigger, prerequisites, procedure, examples and validation. Prefer composable skills over monolithic agent prompts. | Skill manifest, example fixtures, compatibility notes and tests |
| Agent orchestration | Assign an owner, explicit dependencies and handoff contracts to each task. Use least-privilege tool access, bounded retries and escalation on blocked gates. | Task ID, decisions, tool/action log, blocker and handoff record |
| Retrieval and RAG | Index canonical sources with stable paths, provenance and freshness markers. Treat retrieved content as untrusted data; cite or link the exact evidence. | Retrieval query, source versions, cited passages and evaluation results |
| Engineering quality | Require small diffs, format/lint/unit/integration tests, secret scanning, human review for destructive actions and release acceptance gates. | Test commands and outcomes, review notes, commit and PR links |
| Model portability | Separate task specification from provider-specific message formats, tool schemas and inference settings. Test each adapter against the same fixtures. | Capability matrix, adapter version, portability tests and known limits |
id: AGENT-001
title: Bounded repository documentation improvement
objective: Improve one documented workflow without changing runtime behavior
inputs:
- target_repository
- source_paths
- acceptance_criteria
allowed_actions:
- read_source
- edit_documentation
- run_documentation_checks
constraints:
- no_secrets_or_patient_data
- no_unreviewed_destructive_actions
- do_not_claim_unrun_tests
outputs:
- changed_files
- validation_report
- unresolved_risks
acceptance:
- links_resolve
- examples_match_repository
- diff_reviewed
- provenance_recordedBest-practice execution cycle: scope the task → select a template/skill → inspect canonical sources → plan a small change → execute within permissions → run deterministic checks → report evidence and limitations → obtain review before release. Failed checks are blockers, not successes to be described away.
Repository-specific safeguards: keep research-source licensing and privacy boundaries intact; never send controlled biomedical data, credentials or participant-linked records to an unapproved model endpoint. An agent's successful code execution is not proof of scientific validity, clinical readiness or deployment safety.
Scholarly evidence: Literature matrix — authors, methods, limitations and research priorities.
Related reading: Machine-readable indexing and RAG contract · Reproducible research workflow · Research record template · Contributing and validation · Git/GitHub foundations.
Use the study protocol to plan a study, the data dictionary to define records and measurements, and the analysis record to document each execution.