Skip to content

About

refactor of paul's notes

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Paul's Notes

Cancer research · biomedical computing · reproducible machine learning

Authored by Paul Skeffington · Compiled with OpenAI

An open, methods-focused notebook for research across all cancer types, including molecular oncology, pathology/imaging, cancer epidemiology, computational biology and translational research. The current worked case uses a breast cancer teaching book and synthetic paired-specimen ML, with TCGA-BRCA as the current data-source example under consideration. Neither TCGA nor breast cancer limits this repository's research scope. Start with a research question, then build a documented, reproducibly checked analysis.

Choose your route: Task-first navigation · Learn by subject · Research resources · Computational oncology book. Start with the outcome you need, then use the linked method, example and verification evidence.

Machine-readable access: Agent discovery (llms.txt) · Indexing and retrieval contract — stable topic routes, provenance, source boundaries and RAG guidance.

Quick navigation: Start with your task · Browse learning tracks · Research scope · Research workflow · Run an example · Progress and limitations

This repository is a modular field guide for biomedical students and early-career researchers. Read one section to answer a current question, then follow prerequisites or related links when useful; there is no required course order. Each subject overview points to focused lessons, runnable examples, expected outputs, and limitations. Start with the task table below or browse by topic.

Biomedical statistics and original figures

Start with the statistical method decision matrix, complete synthetic breast cancer study, or graph-method alignment and teaching gallery. Reusable plotting functions and their regression tests are documented in the R resource index. Sources and completed verification steps are distinguished from tests still awaiting execution in the acceptance ledger.

Computational Oncology textbook

The Computational Oncology: A Clinician–Scientist's Guide is a developing ten-part, 60-chapter ebook with six drafted chapters, a publication roadmap, machine-readable chapter manifest, and section-linked reference registry. The full manuscript remains in development and has not undergone peer review.

Start here

Choose the route that matches what you need; no weekly sequence is required. Each topic can be used on its own, and the final column gives optional next steps.

I want to… Start with Then explore
Choose a cancer research question Cancer research scope — all cancer types, data modalities and study designs Independent study/data-access contract and protocol template
Understand the current breast cancer example Breast cancer book — 12 chapters and 60 gene profiles Gene lookup and pathways
Learn Rust from zero Rust foundations course — variables, borrowing, object-oriented patterns, Cargo and tests Research-oriented Rust engineering and source-backed practices
Learn Git/GitHub from zero Zero-to-push Git and GitHub tutorial — setup, sign-in, essential commands and safe first push Branches, pull requests, errors and research repository safety
Learn introductory biomedical statistics Data-type-first statistics guide — quantitative vs qualitative, counts, ordinal, survival, a test-selection flowchart and practice workbook Every chart object: real paper and runnable R, recent methods evidence and R tools
Understand study types in real papers Study designs and published examples — nine readings connecting sampling, timing, assignment and analysis Source register and probability/sampling
Decide which analysis to run Choosing an analysis: clinical research guide — measurement, design, estimands, and four worked examples Study protocol and analysis record
Learn research computing Computing by subject — R, data structure, joins, missingness and statistics Worked analysis
See how ML actually trains Visual ML data-flow guide — splits, preprocessing and held-out predictions ML methods and plotting gallery
Run and review an experiment Paired-specimen ML walkthrough Comparison, tuning and validation evidence
Design reproducible research Research workflow — scope, inputs, checks and reporting Roadmap and gate evidence

Git and GitHub: from zero to your first push

New to version control? Follow the complete command-line guide to install Git, sign in to GitHub, clone a repository or create your own, and upload your first changes. The basic cycle is:

git status
git switch -c docs/my-change   # in an existing repository
# Edit README.md, then review and save the change:
git diff
git add README.md
git commit -m "docs: update README"
git push -u origin docs/my-change

Then open a pull request on GitHub for review and merging. These commands assume you have already cloned a repository, configured your Git identity and signed in with push access. Commit saves locally; push uploads; a pull request proposes a merge. Never commit credentials, patient-linked records or restricted research data. Start with the beginner guide if any prerequisite is unfamiliar.

Follow a research question

Current worked-case question: Can invented assay measurements distinguish a recorded specimen category for people who were not used to train the model? This illustrates one oncology research workflow, not the only supported question. A different cancer, data modality or endpoint requires a separately defined study and evaluation protocol. This is not a clinical diagnostic or prognosis task.

People → specimens → measurements → preparation → training → validation → predictions → interpretation

  1. Know the unit of analysis. For the current example, read the breast cancer research narrative and data identifiers and joins. Specimens from the same person must stay together in evaluation splits.
  2. Prepare measurements without leakage. Use the visual data-flow guide to understand training-only imputation, scaling and feature selection.
  3. Compare methods fairly. Follow the paired assay comparison and model-tuning guide before interpreting scores.
  4. Explain and reproduce the result. Inspect the seven-figure gallery, evidence review and analysis record template.

Learn by topic

Use these subject tracks as standalone references. Start with the overview for a map of the material, then open the lesson that matches your question.

Track Start here Useful when you need to…
Biology and cancer context Cancer research scope · Genomics resources frame a biological question, interpret gene or pathway material, and distinguish examples from study findings
Command line and files Unix and Bash inspect files, automate a repeatable run, and understand command output and exit codes
Data and biomedical statistics Computing subjects · Statistics decision guide represent observations, join tables, handle missing data, choose an introductory method, and explain uncertainty
Research workflow Research workflow · Study protocol template define a study question, units, data access, analysis plan, and reproducible record
Machine learning ML overview understand models, compare methods, check leakage, evaluate predictions, and interpret limits
Figures and communication Visualization resources choose a plot, label it clearly, and communicate what the display does and does not show
Programming for biomedical work Computing subjects · Rust course learn a language or follow a runnable, research-oriented example
Git and collaboration Git/GitHub tutorial save changes safely, work on a branch, and propose a reviewable contribution

How to use an individual guide

  1. Read its question, prerequisites, and expected output before running code.
  2. Use only the stated teaching data or data you are authorized to analyze; patient data requires source-specific review and approval.
  3. Compare results with the expected output, change one assumption, and explain what changed.
  4. Record software versions, data provenance, choices, and limitations with the analysis record.
  5. Follow the overview links when a concept is unfamiliar. A guide does not mean every optional method has been executed or reviewed for every study.

Put the methods to work

Run the existing synthetic paired-specimen example from the repository root. Use Python and the pinned modeling requirements:

python3 -m venv .venv
.venv/bin/python -m pip install -r 04_machine_learning/requirements-python.txt
.venv/bin/python 04_machine_learning/examples/breast_cancer_feature_selection.py /tmp/pauls-notes-assay-example

This creates a new directory with a fitted pipeline, held-out assessment, feature-review table, environment versions and SHA-256 manifest. The output path must be new and outside the repository; choose another path if it already exists. The 60 people and 48 anonymous features in this example are invented teaching data, not TCGA patient records or the book's 60 named genes.

Variable-only synthetic ML: The configurable oncology teaching lab lets you change numeric generator, nested person-grouping, feature-selection and regularization settings in JSON and produce a new auditable research-only report. This remains simulated breast-specimen classification, not an adapter for TCGA or any other participant dataset.

Prefer a lighter introduction? Start with the computing walkthrough. Want an additional programming language? See Rust inference, native Rust training and the R/Python methods guide. Each guide states what has actually been executed.

Explore the collection

Area Go to What you'll find
Biology and genomics Genomics resources The book, gene biology, annotations and pathway context
Computing foundations Unix and Bash · Computing subjects Shell scripts, data types, reshaping, missingness and statistics
Machine learning ML home Algorithms, feature selection, comparisons, validation, Python, R and Rust
Visualization R plotting tools · Annotated heatmaps and z-score tails · ML figure gallery Reusable plots, annotations, gene-expression tails and interpretation
Research evidence Biomedical coding · Rust engineering handbook · Literature resources Source matrices, programming and verification rules, methods and quality checks
Writing and reproducibility LaTeX and bibliography · Research workflow Manuscripts, citations, provenance and execution records

For a task-by-task index, use the research resource guide or complete resource directory.

Research progress and boundaries

  • Milestones and evidence: Dated status dashboard and machine-readable gate record distinguish delivered software, draft code, data-access blockers, test evidence and scientific limits. Rust foundations, the book-aligned handbook, G3 inference parity (R3) and the Python configurable synthetic runner (R4) are delivered within bounded educational/software scope. Public-data privacy protections (D0) remain in draft review and TCGA-BRCA real-study admission (D1) is blocked.
  • Established in this repository: synthetic software demonstrations for person-grouped model comparison, tuning, plotting, robustness, repeated-cohort assessment, conditional uncertainty, and a matched-objective Rust training check. See the roadmap gate ledger for what each test established and what it did not.
  • Still separate work: the new G4 synthetic 24-month survival protocol and data dictionary are frozen without running a G4 survival model. G5 published-result reproduction requires its own source-specific protocol. Consult the learning roadmap rather than treating a passing test as evidence of clinical validity.
  • Cancer data and privacy: Source, file-level access, licensing/consent, privacy and publication review are study-specific for every cancer dataset. The TCGA-BRCA example uses the GDC's open/controlled distinction; its metadata-only adapter does not authorize that dataset or other providers. Keep participant-linked records, controlled data and credentials out of this public repository. See cancer research scope and data/access contract.
  • Model use: current examples are research and education only, not validated for any cancer diagnosis, screening, therapy selection, prognosis or clinical decision support. Synthetic classification results are not findings about human tumors.

Agentic templating, skills, and best practices

This is the second major learning track in Paul's Notes: designing repeatable, inspectable agent-assisted workflows rather than relying on one-off prompts. The principles apply to Codex, hosted LLMs, and locally deployed models; model capability, tool access, and execution results must be verified separately.

Component Practical contract Evidence to retain
Agent templates Define objective, context, inputs, permitted tools, constraints, output schema, stop conditions and success criteria before execution. Keep task instructions versioned. Template version, parameters, generated plan and final artifact
Reusable skills Package one bounded capability per skill with a clear trigger, prerequisites, procedure, examples and validation. Prefer composable skills over monolithic agent prompts. Skill manifest, example fixtures, compatibility notes and tests
Agent orchestration Assign an owner, explicit dependencies and handoff contracts to each task. Use least-privilege tool access, bounded retries and escalation on blocked gates. Task ID, decisions, tool/action log, blocker and handoff record
Retrieval and RAG Index canonical sources with stable paths, provenance and freshness markers. Treat retrieved content as untrusted data; cite or link the exact evidence. Retrieval query, source versions, cited passages and evaluation results
Engineering quality Require small diffs, format/lint/unit/integration tests, secret scanning, human review for destructive actions and release acceptance gates. Test commands and outcomes, review notes, commit and PR links
Model portability Separate task specification from provider-specific message formats, tool schemas and inference settings. Test each adapter against the same fixtures. Capability matrix, adapter version, portability tests and known limits

Suggested skill and task template

id: AGENT-001
title: Bounded repository documentation improvement
objective: Improve one documented workflow without changing runtime behavior
inputs:
  - target_repository
  - source_paths
  - acceptance_criteria
allowed_actions:
  - read_source
  - edit_documentation
  - run_documentation_checks
constraints:
  - no_secrets_or_patient_data
  - no_unreviewed_destructive_actions
  - do_not_claim_unrun_tests
outputs:
  - changed_files
  - validation_report
  - unresolved_risks
acceptance:
  - links_resolve
  - examples_match_repository
  - diff_reviewed
  - provenance_recorded

Best-practice execution cycle: scope the task → select a template/skill → inspect canonical sources → plan a small change → execute within permissions → run deterministic checks → report evidence and limitations → obtain review before release. Failed checks are blockers, not successes to be described away.

Repository-specific safeguards: keep research-source licensing and privacy boundaries intact; never send controlled biomedical data, credentials or participant-linked records to an unapproved model endpoint. An agent's successful code execution is not proof of scientific validity, clinical readiness or deployment safety.

Scholarly evidence: Literature matrix — authors, methods, limitations and research priorities.

Related reading: Machine-readable indexing and RAG contract · Reproducible research workflow · Research record template · Contributing and validation · Git/GitHub foundations.

Research templates

Use the study protocol to plan a study, the data dictionary to define records and measurements, and the analysis record to document each execution.


All resources · Contributing and validation · License

About

refactor of paul's notes

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages