A shape-specializing tensor compiler you can inspect end to end
Trace a compact Python-shaped tensor program into SSA, optimize it, emit native C or WGSL, verify it against an interpreter, and examine every stage in the browser.
def layer(x, w, b):
h = x @ w
return relu(h + b)TensorForge is an educational and experimental compiler stack implemented in C++20. Its distinguishing feature is observability: the CLI and browser expose traced and optimized IR, graph snapshots, tuning decisions, generated C, generated WGSL, numerical diagnostics, and measured execution. Static input shapes let the compiler specialize loops and dispatch plans without requiring shape annotations in source.
Important
TensorForge is not a general Python compiler, production inference runtime, or production-certified sandbox. The language is a deliberately small, pure tensor-dataflow subset. Server-side native execution compiles request-derived C and therefore requires a carefully isolated deployment boundary.
- What is implemented
- Architecture
- Quick start
- CLI reference
- ForgeScript language
- End-to-end examples
- Compiler pipeline
- Execution backends
- Autotuning, caching, and fingerprints
- Browser playground
- HTTP and WebSocket API
- Configuration
- Docker and deployment
- Testing, fuzzing, and numerical checks
- Performance methodology
- Security model and limitations
- Project structure
- Troubleshooting
- Contributing and community
- License status
| Capability | Status | Implementation notes |
|---|---|---|
| Python-shaped tensor language | Implemented | One pure function; assignments; one return; static f32 tensors |
| Shape inference | Implemented | NumPy-style elementwise broadcasting, batched matmul, reductions |
| SSA tracing and diagnostics | Implemented | Source-spanned syntax, trace, and shape errors |
| Optimization | Implemented | Dead-code elimination and conservative elementwise fusion |
| Reference execution | Implemented | C++ interpreter used as the correctness oracle |
| Native CPU execution | Implemented | Shape-specialized C, host compiler, shared object, dlopen |
| CPU autotuning | Implemented | Elementwise unroll and matmul tile candidates; persistent JSON cache |
| WGSL lowering | Implemented | Elementwise, reductions, and tiled matmul shader generation |
| Native GPU execution | Not implemented | --backend gpu simulates the generated dispatch plan on the host; it does not submit to a physical GPU |
| Numerical audit | Implemented | Per-op f32 versus f64 analysis with a configurable error budget |
| Differential fuzzing | Implemented | Random programs compared with the interpreter |
| Web application | Implemented | Next.js 15, React 19, Monaco editor, pipeline/DAG views, leaderboard |
| Service API | Implemented | REST and WebSocket server with SQLite, auth, limits, and origin checks |
| Multi-instance coordination | Not implemented | Rate limits and leaderboard notifications are process-local |
flowchart LR
S[ForgeScript source + input shapes] --> L[Lexer]
L --> P[Recursive-descent parser]
P --> T[Abstract tracer]
T --> IR[Shaped f32 SSA graph]
IR --> D1[Dead-code elimination]
D1 --> F[Elementwise fusion]
F --> D2[Cleanup DCE]
D2 --> I[Reference interpreter]
D2 --> C[Shape-specialized C]
C --> CC[Host C compiler]
CC --> SO[Shared object + dlopen]
SO --> CPU[Native CPU execution]
D2 --> W[WGSL + buffer/dispatch plan]
W --> GS[Host-side dispatch simulation]
D2 --> A[Autotuner]
A <--> K[(Per-machine JSON cache)]
D2 --> API[REST / WebSocket service]
API <--> DB[(SQLite)]
API --> UI[Next.js observatory]
The C++ compiler, CLI, and server are built from repository sources. SQLite and threads are linked for the server; native CPU kernels additionally depend on a host C compiler and dynamic loading support at runtime.
- CMake 3.20+
- A C++20 compiler
- SQLite development headers
- A C compiler available at runtime (
cc,clang,gcc, orFORGE_CC) - Node.js 22 and npm for frontend development
- Docker with Compose for the integrated stack
cmake -S cpp -B cpp/build -DCMAKE_BUILD_TYPE=Release
cmake --build cpp/build -j
ctest --test-dir cpp/build --output-on-failurecpp/build/forge compile examples/mlp.fs --shapes 8x16,16x12,12
cpp/build/forge run examples/mlp.fs --shapes 8x16,16x12,12 --backend cpu--shapes 8x16,16x12,12 assigns [8,16], [16,12], and [12] to the function's three parameters. Inputs are deterministic pseudorandom f32 tensors derived from --seed; the CLI currently does not accept user-provided tensor values.
cp .env.example .env
# Replace AUTH_SECRET with a unique value of at least 32 characters.
docker compose up -d --buildOpen http://localhost/playground. The bundled nginx listener is HTTP-only and intended for local use; public deployment requirements are documented below.
forge compile <file> --shapes <spec> Print IR snapshots and graph summary
forge run <file> --shapes <spec> Execute on cpu, gpu simulation, or interpreter
forge bench <file> --shapes <spec> Autotune and report repeated CPU timing
forge wgsl <file> --shapes <spec> Print generated WGSL modules
forge check <file> --shapes <spec> Attribute f32/f64 numerical error by operation
forge fuzz [options] Differentially test generated programs
| Option | Applies to | Default | Meaning |
|---|---|---|---|
--shapes <spec> |
File commands | Required | Comma-separated parameter shapes; dimensions use x |
--backend <name> |
run |
cpu |
cpu, gpu, or interp |
--iters <n> |
bench |
50 |
Timed iterations; 0 is normalized to one |
--count <n> |
fuzz |
100 |
Number of generated programs |
--seed <n> |
Input generation/fuzzing | 1 |
Reproducible base seed |
--budget <x> |
check |
1e-6 |
Maximum accepted relative output error |
--json |
check |
Off | Emit the numerical report as JSON |
--emit-source |
CPU run/bench |
Off | Print generated C |
-h, --help |
All | — | Show built-in help |
check exits with status 1 if measured output error exceeds the budget. Invocation errors use status 2; compilation or execution failures use status 1.
ForgeScript is parsed directly; it is not executed as Python.
program := one function definition
body := assignments followed by one return
values := statically shaped f32 tensors and finite scalar literals
| Form | Semantics |
|---|---|
a + b, a - b, a * b, a / b |
Right-aligned NumPy-style broadcasting |
a @ b, matmul(a, b), a.matmul(b) |
Rank-2-or-higher matmul with broadcast batch dimensions |
-x, neg(x) |
Elementwise negation |
relu(x), exp(x), sigmoid(x) |
Elementwise unary operations; method form is also accepted |
sum(x), mean(x), max(x) |
Reduction over all dimensions |
x.sum(axis=-1, keepdims=True) |
Axis reduction; negative axes and boolean keepdims supported |
1.0, 2, 3e-2 |
Finite scalar f32 constants that broadcast |
Comments and parenthesized multiline expressions are accepted. Tabs, imports, control flow, comparisons, classes, loops, lambdas, floor division, exponentiation syntax, mutation, and arbitrary Python calls are rejected. Errors include source line and column information.
Inspect elementwise fusion
cpp/build/forge compile examples/fusion.fs --shapes 64x64,64x64The example contains add, multiply, ReLU, subtract, and sigmoid. The fusion pass should represent the chain as one fused elementwise expression where single-use and broadcasting constraints permit it. Inspect the emitted stage instead of relying on a fixed node count across future compiler changes.
Compare execution paths
cpp/build/forge run examples/mlp.fs --shapes 8x16,16x12,12 --backend interp
cpp/build/forge run examples/mlp.fs --shapes 8x16,16x12,12 --backend cpu
cpp/build/forge run examples/mlp.fs --shapes 8x16,16x12,12 --backend gpuAll use deterministic inputs for the same seed. The gpu path validates WGSL lowering and simulates dispatch arithmetic on the host; only cpu runs a generated native shared object.
Audit unstable and stable softmax
cpp/build/forge check examples/softmax.fs --shapes 64x64,64x16 --budget 1e-5
cpp/build/forge check examples/softmax_stable.fs --shapes 64x64,64x16 --budget 1e-5
cpp/build/forge check examples/softmax_stable.fs --shapes 64x64,64x16 --budget 1e-5 --jsonThe first example intentionally exposes f32 exponential overflow; the second subtracts the column maximum before exponentiation. The explicit 1e-5 budget is appropriate for this example's accumulated matmul/reduction error; the CLI default remains 1e-6.
- Lex and parse — a recursive-descent parser creates a small AST and rejects unsupported Python constructs early.
- Trace — expressions are abstractly evaluated into shaped SSA nodes. Shapes and
f32dtype are attached to every value. - DCE — nodes that cannot reach the return value are removed.
- Elementwise fusion — compatible single-use pointwise chains are nested into one expression without recomputing shared producers.
- Cleanup DCE — nodes absorbed by fusion are removed.
- Lower — optimized SSA is interpreted, emitted as shape-specialized C, or lowered into WGSL modules and a buffer plan.
- Tune and execute — CPU candidates are measured, a configuration is cached, and the resulting shared object is invoked.
The core pipeline records trace, dce, and fusion snapshots containing IR text, display-graph JSON, and summaries. Tuned execution adds autotune and codegen; WebSocket compilation can also add wgsl and execute events.
The C++ interpreter is the semantic reference. It executes SSA directly and is used by server benchmarks to verify generated CPU output before a leaderboard entry can be stored.
The CPU backend emits C99 with static extents, broadcast-aware indexing, blocked batched matmul, reductions, and fused scalar expressions. It invokes FORGE_CC or tries cc, clang, then gcc with -O2 -shared -fPIC -std=c99, loads the temporary shared object using dlopen, and calls forge_entry.
This is JIT-like runtime compilation, but it is implemented through a host C toolchain rather than an embedded compiler library. The runtime compiler is therefore mandatory, including inside the server container.
The WGSL backend emits compute shaders for fused elementwise nodes, reductions, and tiled batched matmul, including workgroup geometry and buffer bindings.
Caution
Native GPU submission is not present in this C++ repository. GpuKernel::run is a sequential host-side simulation of the generated shader plan. GPU_DISPATCH_THRESHOLD is 65,536 total input elements and is exposed through should_use_gpu, but the CLI's explicit --backend gpu path does not auto-fallback to CPU based on this threshold.
The CPU tuner varies only dimensions relevant to the graph:
- elementwise unroll: base
4, plus1,2, and8when applicable; - cubic matmul tiles: base
32³, plus16³,64³, and128³when applicable; - asymmetric matmul tiles:
16×128×16and64×64×16.
Each viable candidate receives two warmups and seven timed runs; the median determines the winner. The tuning key is cpu-<hardware-id>-<structural-hash>, where hardware identity includes OS, architecture, logical core count, and the CPU brand on macOS. Cache writes use a temporary sibling file followed by rename.
- CLI cache:
FORGE_CACHE_DIR, else$HOME/.cache/tensorforge/kernels, else/tmp/tensorforge-kernels. - Server cache:
FORGE_KERNEL_CACHE_DIR(default/tmp/forge-kernel-cache; Compose overrides it to the persistent volume).
The browser additionally computes a frontend-only compiler fingerprint such as TF1-… from normalized source, shapes, operation mix, observed stage sequence, graph-node evidence, and estimated input bytes. It is useful for sharing a compact workload profile, but it is not the compiler's structural hash, a security digest, or a guarantee that source cannot be inferred.
The Next.js 15 / React 19 application provides:
- Monaco source editing and JSON shape input;
- bundled MLP, stable softmax, fusion, and layer-normalization examples;
- streamed stage chronology over
/ws/compile; - DAG and IR/codegen inspection;
- CPU benchmark requests and measured runtime display;
- copy/export compiler fingerprints;
- a SQLite-backed public leaderboard.
For frontend-only development:
npm --prefix frontend ci
FORGE_SERVER_INTERNAL_URL=http://127.0.0.1:8080 npm --prefix frontend run devThe current live-compile client defaults its WebSocket URL to ws://127.0.0.1:8080/ws/compile; set NEXT_PUBLIC_FORGE_WS_URL when the browser cannot reach that address or when using HTTPS (wss://…).
| Method | Path | Success | Purpose |
|---|---|---|---|
GET |
/health, /api/health |
200 |
Process liveness plus SQLite round-trip |
POST |
/api/compile |
200 |
Return optimized stages, WGSL when lowerable, category, node/FLOP estimates |
POST |
/api/benchmark |
200 |
Verify CPU output, tune, time, and conditionally record |
GET |
/api/leaderboard |
200 |
Public correct submissions; category and limit filters |
POST |
/api/auth/register |
200 |
Register and issue a seven-day bearer token |
POST |
/api/auth/login |
200 |
Authenticate and issue a seven-day bearer token |
GET |
/api/auth/me |
200 |
Resolve the current bearer token |
| WebSocket | /ws/compile |
101 |
One compile message in; stage events and { "done": true } out |
| WebSocket | /ws/leaderboard |
101 |
Connected/update/ping notifications |
Malformed JSON or request fields return 400; valid requests containing syntax, trace, shape, or lowering errors return 422; authentication failures return 401; origin failures return 403; duplicates return 409; rate limits return 429; missing writable benchmark cache returns 503. HTTP bodies are capped at 4 MiB by the server, while bundled nginx sets client_max_body_size 2m.
const response = await fetch('http://127.0.0.1:8080/api/compile', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
source: 'def f(x, w):\n return relu(x @ w)\n',
input_shapes: [[8, 16], [16, 12]],
}),
});
const payload = await response.json();
if (!response.ok) throw new Error(payload.error ?? `HTTP ${response.status}`);
console.log(payload.stages);http POST :8080/api/benchmark \
source='def f(x, w):
return relu(x @ w)
' \
input_shapes:='[[8,16],[16,12]]'
http GET :8080/api/leaderboard category==matmul limit==20Create a POST request to http://127.0.0.1:8080/api/compile, select Body → raw → JSON, and use:
{
"source": "def f(x, w):\n return relu(x @ w)\n",
"input_shapes": [[8, 16], [16, 12]]
}For /api/auth/me, choose Authorization → Bearer Token and paste the token returned by register or login.
const socket = new WebSocket('ws://127.0.0.1:8080/ws/compile');
socket.addEventListener('open', () => {
socket.send(JSON.stringify({
source: 'def f(x):\n return relu(x)\n',
input_shapes: [[64, 64]],
}));
});
socket.addEventListener('message', ({ data }) => console.log(JSON.parse(data)));Browser Origin must exactly match PUBLIC_ORIGIN; non-browser clients may omit Origin.
| Variable | Default | Validation / operational note |
|---|---|---|
AUTH_SECRET |
None; required | At least 32 characters; the known .env.example placeholder is rejected |
FORGE_SERVER_BIND |
0.0.0.0:8080 |
IPv4 literal/host form with port 0..65535; port 0 supports tests |
DATABASE_URL |
tensorforge.db |
Bare path or SQLite/file URL; schema is migrated at startup |
FORGE_KERNEL_CACHE_DIR |
/tmp/forge-kernel-cache |
Server tuning records; should be writable and persistent |
FORGE_CACHE_DIR |
Platform fallback | CLI tuning cache only |
PUBLIC_ORIGIN |
http://localhost |
Exact http:// or https:// browser origin, no trailing slash or newlines |
ADMIN_EMAILS |
Empty | Comma-separated normalized emails granted admin at registration |
RATE_LIMIT_COMPILE_PER_MIN |
30 |
Positive uint32; shared with registration and compile WebSocket |
RATE_LIMIT_BENCHMARK_PER_MIN |
10 |
Positive uint32; also used by login |
MAX_TENSOR_ELEMENTS |
16777216 |
Bounds each input and total input elements |
MAX_BENCH_WALL_MS |
2000 |
Positive benchmark timing budget |
SOCKET_TIMEOUT_SECONDS |
30 |
Receive/send timeout, allowed range 1..3600 |
MAX_CONNECTIONS |
64 |
Concurrent connection cap, allowed range 1..10000; excess receives 503 |
FORGE_CC |
Auto-detect | One compiler executable name/path only; arguments are not accepted |
FORGE_SERVER_INTERNAL_URL |
http://forge-server:8080 |
Next.js server-side API rewrite target |
NEXT_PUBLIC_FORGE_WS_URL |
Browser default | Public compile WebSocket URL for the frontend |
Additional request limits are fixed in code: source ≤ 64 KiB, ≤ 16 inputs, and rank ≤ 4. Empty dimensions and non-positive/non-integer dimensions are rejected.
cp .env.example .env
# Generate and set a unique secret, for example with:
openssl rand -base64 32
docker compose up -d --build
docker compose psThe stack contains:
- nginx 1.27 Alpine on port 80, proxying
/api,/ws, and the frontend; - Next.js standalone frontend on port 3000;
- C++ server on port 8080 with SQLite and the kernel cache in
forgedata.
Both application images run as non-root users. Compose drops capabilities, enables no-new-privileges, caps PIDs/memory/CPU, and bounds server connections. The server runtime intentionally contains GCC and development libc because native kernels compile per workload.
Warning
deploy/nginx.conf serves plain HTTP. A public deployment must terminate TLS externally, route both HTTP and WebSocket traffic, and set PUBLIC_ORIGIN to the exact externally visible HTTPS origin (for example, https://forge.example.com, with no trailing slash). The included stack does not provision certificates or HTTPS.
SQLite, process-local rate limiting, and process-local leaderboard polling make this a single-server topology. Do not scale forge-server horizontally without introducing shared coordination and deciding how tuning cache and SQLite writes will be handled.
ctest --test-dir cpp/build --output-on-failure
npm --prefix frontend ci
npm --prefix frontend run build
npm --prefix frontend audit --omit=dev --audit-level=high
cpp/build/forge fuzz --count 500 --seed 1CMake registers exactly five CTest suites:
core_correctnesscore_numericcpu_jit_correctnessgpu_wgslserver_api
They cover core parsing/tracing/optimization behavior and numeric analysis, native CPU correctness, WGSL generation, and server behavior over real sockets. CI builds C++ in Release mode, runs all five suites, builds the frontend with Node.js 22, and performs a production dependency audit with npm audit --omit=dev --audit-level=high.
forge fuzz generates random programs and compares native CPU output with the interpreter. It currently does not execute a physical GPU. Preserve a failing seed when reporting a mismatch.
TensorForge intentionally ships no repository-wide benchmark claim. Results depend on shapes, operation mix, compiler, CPU, thermal state, cache state, background load, and whether tuning was cached.
- CLI
benchperforms one untimed run, then reports arithmetic mean wall time across--iters; GFLOP/s uses the graph's internal FLOP estimate and counts each multiply-accumulate as two arithmetic operations. - Autotuning uses median candidate time after warmups.
- Server benchmarking reports medians: up to five interpreter samples and up to fifty compiled-kernel samples, both constrained by
MAX_BENCH_WALL_MS. - Server “speedup” is compiled CPU median versus the in-process interpreter baseline, not versus an external framework or vendor library.
- Correctness is checked before a leaderboard row is recorded; absolute or relative error must be ≤
1e-3.
For reproducible comparisons, record revision, OS/architecture, CPU, C compiler and version, build type, exact source/shapes/seed, iteration count, cache-hit state, chosen configuration, and system load. Run multiple fresh processes and report distributions rather than a single best result.
Implemented controls include:
- required HMAC secret, minimum 32 characters, and rejection of the published placeholder;
- PBKDF2-SHA256 salted password hashes and constant-time hash/signature comparison;
- seven-day HMAC-SHA256 bearer tokens with expiry and account re-resolution;
- exact CORS and WebSocket origin checks for browser clients;
- source, body, tensor, rank, input-count, benchmark-time, socket-time, and connection limits;
- per-identity in-memory rate limiting with
Retry-Aftermetadata; - SQLite prepared operations/migrations, non-root containers, dropped capabilities, and proxy security headers;
- correctness verification before leaderboard insertion.
Operational boundaries:
- Native CPU execution compiles request-derived code. Although generated C comes from validated IR rather than direct source interpolation, treat the compiler/server as a high-risk execution service and isolate it from sensitive networks and credentials.
- TLS, firewalling, host hardening, backups, secret rotation, monitoring, and abuse detection are operator responsibilities.
X-Forwarded-Foridentity handling is meaningful only behind a trusted proxy; do not expose an arrangement that lets clients spoof trusted forwarding metadata.- Rate-limit state and leaderboard notifications are not shared across processes.
- Leaderboard entries deliberately expose submitted source and shapes publicly.
- Tokens are bearer credentials; there is no refresh/revocation endpoint beyond removing the account or rotating
AUTH_SECRET. - The project has no claim of security audit, production certification, side-channel resistance, or complete resource isolation.
Report vulnerabilities privately as described in SECURITY.md.
cpp/
├── CMakeLists.txt C++20 targets and five CTest suites
├── Dockerfile Multi-stage non-root server image
├── src/
│ ├── cli/ forge command-line driver
│ ├── core/ lexer, parser, tracer, SSA, passes, interpreter, audit
│ ├── cpu/ C emitter, runtime compiler, autotuner, cache
│ ├── gpu/ WGSL generation and host dispatch simulation
│ └── server/ HTTP, WebSocket, auth, SQLite, limits, API handlers
└── tests/ core, CPU, GPU/WGSL, and server suites
frontend/ Next.js 15 / React 19 observatory and leaderboard
examples/ fusion, MLP, reductions, unstable/stable softmax
deploy/nginx.conf HTTP reverse proxy configuration
.github/workflows/ci.yml C++ tests, frontend build, production npm audit
docker-compose.yml Single-host integrated stack
CONTRIBUTING.md Development and pull-request guidance
SECURITY.md Private reporting and deployment boundary
No working host C compiler found
Install a C compiler or set FORGE_CC to one executable path. Do not include command-line arguments. The compiler must support -O2 -shared -fPIC -std=c99 and link libm.
Server rejects AUTH_SECRET or PUBLIC_ORIGIN
Use a unique secret of at least 32 characters; the example placeholder is intentionally refused. Use an exact origin such as http://localhost or https://forge.example.com, without a trailing slash, path, or newline.
Playground cannot connect to port 8080
Confirm forge_server is healthy, then set NEXT_PUBLIC_FORGE_WS_URL to a browser-reachable ws:// or wss:// URL. The frontend's fallback points to 127.0.0.1:8080, which is not correct for every remote deployment.
Shape error or wrong number of shapes
Provide exactly one shape group per function parameter. Matmul requires both operands to have rank ≥ 2, equal inner dimensions, and broadcastable batch dimensions. Use keepdims=True when a reduced axis must broadcast back into the original tensor.
Benchmark always tunes or returns 503
Ensure FORGE_KERNEL_CACHE_DIR is writable. A read-only or unavailable cache prevents the benchmark endpoint from proceeding when no fallback cache can be opened. For CLI tuning, inspect FORGE_CACHE_DIR or the documented platform fallback.
Expected real GPU acceleration
The repository emits WGSL and simulates the dispatch plan on the host. It does not include a native WebGPU runtime or device submission path, so GPU timings are not available from the CLI/server.
Read CONTRIBUTING.md for prerequisites and checks, follow CODE_OF_CONDUCT.md, and use the repository's issue and pull-request templates. Keep changes focused, add tests for behavior changes, document security implications, and do not commit .env, databases, caches, generated builds, or secrets.
Because no repository URL is encoded in this checkout, this README intentionally uses only relative community links and does not fabricate issue, workflow, or maintainer URLs.
No license has been selected or included in this repository. Until a license is added by the copyright holder, the absence of a license generally means no permission is granted to copy, modify, or redistribute the code beyond rights provided by applicable law. Contributors and users should not infer a license from package metadata or project visibility.