TPT GPU is an open-source, hardware-agnostic, full-stack GPU compute platform designed for AI/ML workloads. It features TPT Script — an AI-native programming language with a minimal, orthogonal API surface that LLMs can reason over without truncation.
- Complete Standard Library — 200+ orthogonal operations covering tensors, neural networks, optimization, and distributed computing
- Production-Ready Compiler — Lexer, parser, type checker with tensor shape inference, and dual codegen (Rust + TPTIR)
- LLM Inference Runtime —
GpuInferenceEnginewith arch-template dispatch (LLaMA 3, Mistral, Qwen2, Phi-3, Gemma 2), sliding-window KV cache, and automatic vendor routing (CUDA → ROCm → Metal → TPTIR) - Shared Model Registry — GGUF models stored once in
~/.tpt/models/and shared across all TPT tools - IDE Support — Full LSP server, VS Code extension, formatter, and linter
- Browser Playground — Try TPT Script live in your browser, no install required:
tools/tpt-playground/ - Framework Integration — PyTorch and JAX backends with seamless dispatch
- AI-Assisted Kernel Generation — Automated kernel optimization and generation tools
- Comprehensive Documentation — 17 tutorials, complete language spec, and API reference
# Clone the repository
git clone https://github.com/tpt-solutions/tpt-gpu.git
cd tpt-gpu
# Build the TPT Script compiler
cd layer7_tptb
cargo build --release -p tpt-gpu-script-cli
# The binary is at: target/release/tpt-gpu-scriptCreate hello.tpts:
import tpt
@doc("Compute the ReLU activation function")
fn relu(x: Tensor[f32, n]) -> Tensor[f32, n] {
return tpt.relu(x)
}
Compile and check:
# Type-check
tpt check hello.tpts
# Compile to Rust + TPTIR
tpt compile hello.tpts -o output/
# List all available operations
tpt ops
# Get docs for an operation
tpt docs matmulLooking for a specific end-to-end scenario (training loop, LLM inference, benchmarking, vendor certification)? See docs/use-cases.md.
- Rust toolchain >= 1.75 (
rustup update) - Cargo workspace support
- Optional: VS Code for IDE features
# Build all Rust layers
cargo build --release
# Build specific components
cd layer7_tptb
cargo build --release -p tpt-gpu-script-cli # CLI tool
cargo build --release -p tpt-gpu-script-lsp # LSP server
cargo build --release -p tpt-gpu-script-format # Formatter/linter
# Run tests
cargo test --workspace
# Build with simulation mode (no hardware required)
cd layer5_tptp
cargo build --features sim- Statically typed with tensor shape inference
- Minimal API — ~200 orthogonal operations (vs PyTorch's ~2000)
- AI-native — Every operation has machine-readable metadata (
@doc,@constraint,@complexity) - Dual compilation — Host functions → Rust, GPU kernels → TPTIR
- Rich annotations —
@requires_gpu,@distributed,@deploy, and more
- Architecture-agnostic dispatch — Add new model architectures by registering one
ArchTemplate - Supported architectures — LLaMA 3, Mistral, Qwen2, Phi-3, Gemma 2 (GGUF format)
- Sliding-window KV cache — Autoregressive decoding with overflow eviction
- Automatic vendor routing — CUDA → ROCm → Metal → TPTIR fallback
- Shared model registry — Models downloaded once to
~/.tpt/models/via HuggingFace
- Fast compilation — Parallel Rust implementation
- Structured errors — Error codes, locations, suggestions, and auto-fixes
- Introspection API —
tpt.introspect.list_operations(),get_schema(),validate_code() - LSP support — Completions, hover, diagnostics, go-to-definition
- Three-tier allocator — Slab → Buddy → Fallback
- Priority scheduler — With aging to prevent starvation
- Optimized kernels — GEMM, Attention, Conv2D, Conv3D, normalization layers
- AI-guided optimization — Automated kernel tuning and generation
- PyTorch dispatch — Seamless backend integration
- JAX integration — XLA-compatible primitives
- HuggingFace support — Model loading and inference
- Distributed training — FSDP and pipeline parallelism
TPT GPU is organized into 7 independent layers with well-defined FFI/API boundaries:
layer1_isa/ SystemVerilog ISA — 32-bit fixed-length, 9-stage SIMT pipeline
layer2_tptd/ Kernel drivers — Linux DRM, Windows WDM, macOS DriverKit
layer3_tptc/ TPTIR compiler — MLIR-compatible dialect (C++ + Rust)
layer4_tptr/ GPU runtime — allocator, scheduler, kernel launch, LLM inference (Rust)
layer5_tptp/ GPU primitives — GEMM, Attention, Conv2D (TPTIR + Rust)
layer6_tptf/ Framework backends — PyTorch, JAX integration (Python + Rust)
layer7_tptb/ TPT Script compiler — lexer → parser → type checker → codegen (Rust)
Development flow: TPT Script (L7) → TPTIR (L3) → TPT ISA (L1) via Runtime (L4)
| Tool | Description | Command |
|---|---|---|
tpt-gpu-script |
CLI compiler | tpt-gpu-script check, tpt-gpu-script compile, tpt-gpu-script run |
tpt-gpu-lsp |
Language Server | IDE integration |
tpt-gpu-fmt |
Formatter/Linter | tpt-gpu-fmt fmt, tpt-gpu-fmt lint |
model-registry |
Shared GGUF model registry | tpt-gpu-models add/list/fetch |
kernel-generator |
AI-assisted kernel gen | Spec → TPTIR → validate → benchmark |
kernel-optimizer |
Auto-tuning | Grid → hill-climb → AI-guided search |
TPT GPU components are published to crates.io for easy integration:
[dependencies]
tpt-gpu-script-core = "1.0" # TPT Script compiler
tpt-gpu-primitives = "1.0" # GPU primitives
tpt-gpu-runtime = "1.0" # RuntimePublish commands:
cd layer7_tptb/tptb-core && cargo publish
cd layer5_tptp/tptp-core && cargo publish
cd layer4_tptr/tptr-core && cargo publish- Complete standard library
- Production-ready compiler
- LLM inference runtime with KV cache
- Shared GGUF model registry
- IDE support (LSP, VS Code extension)
- Framework integration (PyTorch, JAX)
- AI-assisted kernel generation
- REPL for interactive development
- Enhanced error recovery
- Performance profiling tools
- Expanded hardware support
- Custom silicon support
- Advanced distributed computing
- Web-based compiler playground
- TPT Script as recommended API
This project accepts bug reports and feature requests via GitHub Issues. Pull requests are not accepted at this time.
| Document | Description |
|---|---|
| User Guide | Complete TPT Script language reference |
| Language Spec | Formal language specification (51KB) |
| Tutorials | 17 hands-on tutorials from basics to advanced |
| Architecture | Developer guide and build instructions |
| Model Registry | Shared GGUF model registry format |
Please see SECURITY.md for security policies and reporting vulnerabilities.
TPT GPU is dual-licensed under your choice of the MIT License or the Apache License 2.0 with LLVM Exception.
See LICENSE-MIT and LICENSE-APACHE for the full license text.
- Rust Community — For the amazing ecosystem and tooling
- MLIR Project — For compiler infrastructure inspiration
- Open Source Contributors — For making this project possible