Wjy dev

.github/workflows/ci.yml

@@ -0,0 +1,95 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  test:
+    runs-on: self-hosted
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.9", "3.10", "3.11", "3.12"]
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e ".[all]"
+
+      - name: Run tests
+        run: python -m pytest tests/ -v --tb=short
+
+  lint:
+    runs-on: self-hosted
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e ".[all]"
+          pip install flake8 mypy
+
+      - name: flake8
+        run: python -m flake8 scratchv/ scratchv_dag/ tests/
+
+      - name: mypy
+        run: python -m mypy scratchv/ scratchv_dag/ --ignore-missing-imports
+
+  coverage:
+    runs-on: self-hosted
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e ".[all]"
+          pip install pytest-cov
+
+      - name: Run tests with coverage
+        run: python -m pytest tests/ --cov=scratchv --cov=scratchv_dag --cov-report=term --cov-report=xml
+
+      - name: Upload coverage to Codecov
+        uses: codecov/codecov-action@v5
+        with:
+          files: ./coverage.xml
+          fail_ci_if_error: false
+
+  smoke:
+    runs-on: self-hosted
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e ".[all]"
+
+      - name: Smoke test - DSL compilation
+        run: |
+          python -m scratchv examples/simple_add.dsl -o /tmp/simple_add.s --dump-ir
+          python -m scratchv examples/relu_test.dsl -o /tmp/relu.s --optimize all
+          python -m scratchv examples/matmul_test.dsl -o /tmp/matmul.s --optimize all

.gitignore

@@ -5,6 +5,9 @@ dist/
 build/
 *.s
 *.o
+*.ll
 models/
 venv/
 .venv/
+.claude/
+scratchv.egg-info/

CHANGELOG.md

@@ -0,0 +1,49 @@
+# Changelog
+
+## [0.3.0] — 2026-05-18
+
+### Added
+- `scratchv_dag/`: standalone LLVM-style SelectionDAG infrastructure package
+  - `sdnode.py`: SDNode, SDValue, MVT, SelectionDAG container
+  - `selection_dag.py`: DAGBuilder, DAGCombiner, DAGScheduler pipeline
+  - `cache.py`: 4 MB L1 cache simulator (set-associative, LRU, write-back)
+  - `allocator.py`: Buddy-system memory allocator with cache-line alignment and scratchpad
+- `docs/developer_guide.md`: guide for extending ScratchV with new ops and passes
+- `Makefile`: standard dev targets (install, test, clean, lint, docs)
+- `CHANGELOG.md`, `CONTRIBUTING.md`: project metadata files
+- `pyproject.toml`: classifiers, readme field, license field
+
+### Changed
+- Consolidated `scratchv/codegen/` and `scratchv/memory/` into re-export shims over `scratchv_dag/`
+- Python requirement lowered to 3.8 with full compatibility fixes
+- `pyproject.toml` version bumped to 0.3.0
+- `.gitignore` extended for `.ll` files and `.claude/`
+
+## [0.2.0] — 2026-05-15
+
+### Added
+- LLVM IR backend (`llvm_codegen.py`)
+- Advanced optimizations: peephole, muladd fusion, LICM
+- Verification framework: ONNX Runtime comparison, numpy reference, DSL interpreter
+- TinyFive adapter for assembly verification and profiling
+- CLI options: `--backend`, `--optimize`, `--verify`, `--reg-alloc`
+- Documentation: optimization guide, verification guide
+
+### Changed
+- Instruction selector supports all major ops (add, sub, mul, div, neg, exp,
+  relu, gelu, softmax, maxpool, matmul, dot)
+- Register allocator: greedy mode (LRU-based) added alongside naive
+
+## [0.1.0] — 2026-05-01
+
+### Added
+- Initial IR: types (Value, Instruction, BasicBlock, Function, Program)
+- ONNX parser: Add, Mul, Sub, Div, MatMul, ReLU, GELU, Softmax, MaxPool
+- DSL parser for fast iteration without ONNX dependency
+- IR builder with chainable API
+- IR printer for debugging
+- Instruction selector: IR → RISC-V pseudo-instructions
+- Register allocator: naive (spill-all) mode
+- Assembly emitter: GAS-syntax output
+- Constant folding and dead code elimination passes
+- CLI entry point with `-o`, `--dump-ir` flags

CONTRIBUTING.md

@@ -0,0 +1,62 @@
+# Contributing to ScratchV
+
+Thanks for your interest! This is an educational compiler project, and
+contributions of all kinds — code, docs, bug reports, teaching materials —
+are very welcome.
+
+## Quick Start
+
+```bash
+git clone https://github.com/kinsomwang/ScratchV
+cd ScratchV
+pip install -e .            # install in editable mode
+pip install tinyfive        # optional: assembly verification
+pytest tests/ -v            # run all tests
+```
+
+## Code Style
+
+- **Python version**: 3.8+ compatible (no `|` union syntax in annotations
+  unless guarded by `from __future__ import annotations`; no
+  `dataclass(slots=True)`).
+- **Type hints**: annotate all public functions and methods.
+- **Docstrings**: Google or NumPy style is fine — keep them short but useful.
+- **No `__pycache__`**: they're gitignored; just don't commit them.
+
+## Pull Request Process
+
+1. **Open an issue** first to discuss the change you'd like to make.
+2. Make your changes on a feature branch (`git checkout -b feat/my-thing`).
+3. Add or update tests in `tests/`.
+4. Run `pytest tests/` — all tests must pass.
+5. Run `make check` if available (lint + test).
+6. Open a PR with a clear title and description.
+
+## Adding a New IR Opcode
+
+1. Add the opcode to `scratchv/ir/types.py` → `OpCode` enum.
+2. (Optional) Add a builder method in `scratchv/ir/builder.py`.
+3. Add a selection handler in `scratchv/backend/instruction_select.py`.
+4. Add an LLVM codegen handler in `scratchv/backend/llvm_codegen.py`.
+5. Add a test case in `tests/`.
+6. Run `pytest` to verify.
+
+## Adding a New Optimization Pass
+
+1. Create `scratchv/optimizer/my_pass.py`.
+2. Implement a class with a `run(program) → int` method (returns number of
+   transformations applied).
+3. Register it in `scratchv/main.py` → `run_optimizer()`.
+4. Add test cases (positive: should transform; negative: should not).
+5. Run `pytest` to verify.
+
+## Documentation
+
+- User-facing docs go in `docs/`.
+- Inline code comments are for *why* not *what*.
+- The README is the single source of truth for project-wide docs.
+
+## Code of Conduct
+
+Be respectful, assume good faith, and remember that this is a learning project.
+Help others level up.

Makefile

@@ -0,0 +1,52 @@
+# ScratchV developer makefile
+.POSIX:
+
+.PHONY: install test clean lint check docs examples
+
+# ── Installation ──────────────────────────────────────────────────────────────
+
+install:
+	pip install -e .
+	pip install -e ".[all]" 2>/dev/null || pip install -e .
+
+# ── Testing ───────────────────────────────────────────────────────────────────
+
+test:
+	python3 -m pytest tests/ -v --tb=short
+
+test-coverage:
+	python3 -m pytest tests/ --cov=scratchv --cov=scratchv_dag --cov-report=term
+
+# ── Lint ──────────────────────────────────────────────────────────────────────
+
+lint:
+	-python3 -m flake8 scratchv/ scratchv_dag/ tests/ 2>/dev/null || echo "install flake8: pip install flake8"
+	-python3 -m mypy scratchv/ scratchv_dag/ --ignore-missing-imports 2>/dev/null || echo "install mypy: pip install mypy"
+
+# ── Clean ─────────────────────────────────────────────────────────────────────
+
+clean:
+	find . -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null
+	find . -type f -name '*.pyc' -delete
+	rm -rf .pytest_cache
+	rm -rf scratchv.egg-info scratchv_dag.egg-info
+	rm -rf dist build
+	rm -f output.s output.ll
+
+# ── Checks (runs before PR) ───────────────────────────────────────────────────
+
+check: clean test
+
+# ── Quick examples ────────────────────────────────────────────────────────────
+
+examples:
+	@echo "=== DSL examples ==="
+	python3 -m scratchv examples/simple_add.dsl -o /tmp/simple_add.s --dump-ir
+	python3 -m scratchv examples/relu_test.dsl -o /tmp/relu.s --optimize all
+	python3 -m scratchv examples/matmul_test.dsl -o /tmp/matmul.s --optimize all
+
+# ── Build docs preview (if pandoc is available) ────────────────────────────────
+
+docs:
+	@echo "Documentation is markdown — no build required."
+	@ls docs/*.md