Skip to content
gladia-research-groupPublic

About

A framework for encrypted CKKS transformer inference on GPU. We implement complex slots utilization for decoding, packing awareness for sparse bootstrap placement, and a python API. Perseus supports encrypted language modelling, image and sentence classification, and image generation.

Topics

Resources

Contributing

Stars

28 stars

Watchers

0 watching

Forks

Repository files navigation

Note

News. Perseus now supports flow matching! Gemma-2-2B and SANA-Sprint 0.6B turn a text prompt into a 768 × 768 image fully encrypted, in 267 s on one GPU, with the perseus-t2i command (examples/sana).

Sapienza University of Rome            George Mason University            Paradigma

Perseus

A full-stack system for homomorphic model inference

📄 Paper   ·   🏁 Encrypted GPT-2 Speedrun

Results · Runtime · Reproduce · Install · Python API · Citation

Perseus runs transformers under fully homomorphic encryption on a single GPU, so the server computes on data it never sees in the clear. Models, packings, nonlinear approximations and bootstrap placement are plain Python; each block is compiled into a program that a Rust + CUDA CKKS runtime replays (its kernels descend from FIDESlib32bits, our 32-bit GPU port of FIDESlib).

Perseus, encrypted, end to end on one RTX PRO 6000 with 128-bit security, and the peak GPU memory of each run (Llama-3-8B and Gemma-2-2B stream their weights from host memory, GPT-2 small keeps its weights on the GPU). Llama-3-8B prefill of a 128-token prompt 53.6 s (85.1 GiB), decode of one token at a 512-token context 6.86 s (52.4 GiB when decoding 16 tokens). Gemma-2-2B prefill 29.7 s (80.5 GiB), decode 5.49 s per token at a 512-token context (63.5 GiB when decoding 16 tokens). GPT-2 small prefill 6.04 s (59.3 GiB), decode 1.55 s per token at a 512-token context (63.3 GiB when decoding 16 tokens). ViT-B/16 17.7 s per image (68.1 GiB). BERT-base 7.36 s for 128 tokens (57.9 GiB). ResNet-20 0.321 s per CIFAR-10 image (35.5 GiB). Whisper base.en encoder over one 10-s audio window 19.2 s (70.0 GiB), decode 1.39 s per token of the transcript (56.9 GiB). SANA-Sprint 0.6B, with its Gemma-2-2B text encoder, 267 s for one 768 × 768 image from a text prompt (76.7 GiB). Perseus, encrypted, end to end on one RTX PRO 6000 with 128-bit security, and the peak GPU memory of each run (Llama-3-8B and Gemma-2-2B stream their weights from host memory, GPT-2 small keeps its weights on the GPU). Llama-3-8B prefill of a 128-token prompt 53.6 s (85.1 GiB), decode of one token at a 512-token context 6.86 s (52.4 GiB when decoding 16 tokens). Gemma-2-2B prefill 29.7 s (80.5 GiB), decode 5.49 s per token at a 512-token context (63.5 GiB when decoding 16 tokens). GPT-2 small prefill 6.04 s (59.3 GiB), decode 1.55 s per token at a 512-token context (63.3 GiB when decoding 16 tokens). ViT-B/16 17.7 s per image (68.1 GiB). BERT-base 7.36 s for 128 tokens (57.9 GiB). ResNet-20 0.321 s per CIFAR-10 image (35.5 GiB). Whisper base.en encoder over one 10-s audio window 19.2 s (70.0 GiB), decode 1.39 s per token of the transcript (56.9 GiB). SANA-Sprint 0.6B, with its Gemma-2-2B text encoder, 267 s for one 768 × 768 image from a text prompt (76.7 GiB).

What's inside

  • Python models, compiled. GPT-2, Llama-3, BERT, ViT, ResNet-20 on CIFAR-10 and SANA-Sprint (with its Gemma-2 text encoder) are Python programs on perseus.impl (python/perseus/examples/), with readable packings, nonlinearities and layers. Each block is recorded once on a CPU twin, compiled with its plan into a program, and replayed by the runtime with no Python on the critical path, from one device arena sized at setup.
  • In-kernel key regeneration. Half of every key-switching key is regenerated on the GPU from a ChaCha12 seed instead of being stored.
  • Complex Cachemir packing. Two real vectors per ciphertext, in the complex slots of CKKS.
  • Packing- and magnitude-aware bootstrap placement. GPT-2 runs 416 bootstraps per token, against 584–894 for DaCapo, Orion and Fhelipe.
  • CKKS -> TFHE -> CKKS lookup tables. perseus.impl.tfhe.tfhe_lut evaluates a sign or any lookup table on chosen slots by programmable bootstrapping (crates pc-tfhe, pc-gpu tfhe), as one planned refresh: exact decisions where a polynomial needs high degree (ReLU's sign, argmax, thresholds). A bootstrap takes 1.77 ms (40k/s batched) at TFHE-rs's 2-bit failure target (p_fail 2^-139), 3.8× TFHE-rs on the same GPU.
  • Security. Every key estimated against the standard lattice attacks and the 2026 ring attacks (ePrint 2026/279, 2026/366): all at or above 128 bits; scripts and raw results in security/.

Results

One NVIDIA RTX PRO 6000, 128-bit security. Prior results on their own hardware, or replicated on ours where the code is released.

Encrypted Llama-3-8B. Prefill of a 128-token prompt, seconds. MOAI 3592.0 on H200, NEXUS 829.0 on 4 A100, Euston 484.0, STIP 406.6 (composed timings, amortized over a batch); Odin 366.4 on H100; Cerium 347.5 (replicated on RTX PRO 6000); Sylph 116.0 (composed); Perseus 53.6 on RTX PRO 6000, open source. Decode of one token, seconds per token at a context size. AR-HE 1569.0 at 32, ROSETTA 311.0 at 512, Cachemir 97.0 at 512, Sylph 31.0 at 128 (all composed timings); Perseus 6.86 at 512 on RTX PRO 6000, open source. Encrypted Llama-3-8B. Prefill of a 128-token prompt, seconds. MOAI 3592.0 on H200, NEXUS 829.0 on 4 A100, Euston 484.0, STIP 406.6 (composed timings, amortized over a batch); Odin 366.4 on H100; Cerium 347.5 (replicated on RTX PRO 6000); Sylph 116.0 (composed); Perseus 53.6 on RTX PRO 6000, open source. Decode of one token, seconds per token at a context size. AR-HE 1569.0 at 32, ROSETTA 311.0 at 512, Cachemir 97.0 at 512, Sylph 31.0 at 128 (all composed timings); Perseus 6.86 at 512 on RTX PRO 6000, open source.
Encrypted Gemma-2-2B. Perseus prefill of a 128-token prompt 29.7 s, decode of one token 5.49 s at a 512-token context, on RTX PRO 6000, open source. Encrypted Gemma-2-2B. Perseus prefill of a 128-token prompt 29.7 s, decode of one token 5.49 s at a 512-token context, on RTX PRO 6000, open source.
Encrypted GPT-2 small. AR-HE 128-token prefill 18520.0 s, decode 544.0 s per token at context 33. ROSETTA decode 78.6 s at 512 (composed). EncryptedLLM decode 68.0 s at 128. Perseus prefill 6.04 s, decode 1.55 s per token at context 512, on RTX PRO 6000, open source. Encrypted GPT-2 small. AR-HE 128-token prefill 18520.0 s, decode 544.0 s per token at context 33. ROSETTA decode 78.6 s at 512 (composed). EncryptedLLM decode 68.0 s at 128. Perseus prefill 6.04 s, decode 1.55 s per token at context 512, on RTX PRO 6000, open source.
Encrypted ViT inference of one image, seconds. FIONA, ViT-S/16, 374.4 on a 48-core CPU (amortized over a batch of 32, retrained polynomial model); ATLAS, ViT-B/16, 2259.0 on RTX 4090; Perseus, ViT-B/16, 17.7 on RTX PRO 6000, open source. Encrypted ViT inference of one image, seconds. FIONA, ViT-S/16, 374.4 on a 48-core CPU (amortized over a batch of 32, retrained polynomial model); ATLAS, ViT-B/16, 2259.0 on RTX 4090; Perseus, ViT-B/16, 17.7 on RTX PRO 6000, open source.
Encrypted BERT inference of one sentence, seconds per sentence. THOR, BERT-base, 128 tokens, 602.0 on A100; POLARIS, BERT-Mini, 83.0 on A100; Cerium, BERT-base, 128 tokens, 43.6 (replicated on RTX PRO 6000); Perseus, BERT-base, 128 tokens, 7.36 on RTX PRO 6000, open source. Encrypted BERT inference of one sentence, seconds per sentence. THOR, BERT-base, 128 tokens, 602.0 on A100; POLARIS, BERT-Mini, 83.0 on A100; Cerium, BERT-base, 128 tokens, 43.6 (replicated on RTX PRO 6000); Perseus, BERT-base, 128 tokens, 7.36 on RTX PRO 6000, open source.
Encrypted ResNet-20 inference of one 32 × 32 CIFAR-10 image, seconds, and top-1 on the CIFAR-10 test set. Orion 618.2 on one CPU thread, open source; Cheddar 0.72 on RTX 5090, 91.95%, library released, ResNet-20 not; Cerium 0.456 on B200 (0.620 on H100, from the paper), 91.4%; Perseus 0.321 on RTX PRO 6000, 92.5% on the first 1,000 test images (the same as its float64 model), open source. Encrypted ResNet-20 inference of one 32 × 32 CIFAR-10 image, seconds, and top-1 on the CIFAR-10 test set. Orion 618.2 on one CPU thread, open source; Cheddar 0.72 on RTX 5090, 91.95%, library released, ResNet-20 not; Cerium 0.456 on B200 (0.620 on H100, from the paper), 91.4%; Perseus 0.321 on RTX PRO 6000, 92.5% on the first 1,000 test images (the same as its float64 model), open source.
Encrypted Whisper base.en speech to text. Perseus encoder over one 10-s audio window (clips up to 8 s) 19.2 s, decode 1.39 s per token, WER 4.87% on 64 LibriSpeech test-clean clips (its float64 model 4.74%), on RTX PRO 6000, open source. Encrypted Whisper base.en speech to text. Perseus encoder over one 10-s audio window (clips up to 8 s) 19.2 s, decode 1.39 s per token, WER 4.87% on 64 LibriSpeech test-clean clips (its float64 model 4.74%), on RTX PRO 6000, open source.
Encrypted generation of one image from a text prompt, seconds. Perseus, SANA-Sprint 0.6B with its Gemma-2-2B text encoder, 768 × 768, 2 steps, 267 on RTX PRO 6000, encrypted end to end, open source. Encrypted generation of one image from a text prompt, seconds. Perseus, SANA-Sprint 0.6B with its Gemma-2-2B text encoder, 768 × 768, 2 steps, 267 on RTX PRO 6000, encrypted end to end, open source.

Times exclude key generation, encryption and decryption; each model runs its default plan (python/perseus/examples/defaults.py).

Runtime

perseus-core is the CKKS runtime in crates/, written in Rust and CUDA. The table times its primitives on one RTX PRO 6000, and the two commands under it reproduce the table.

perseus-core CKKS primitives on one RTX PRO 6000, device milliseconds per call for one ciphertext and per ciphertext in a batch of 8. Addition 0.014 and 0.054, multiplication by a scalar 0.017 and 0.037, by a plaintext 0.015 and 0.037, relinearized multiplication 0.356 and 0.415, rescale 0.047 and 0.065, rotation 0.284 and 0.358, forward and inverse NTT 0.057 and 0.089. Bootstraps, dense 12.2 and 12.8, StC first 10.5 and 10.7, sparse 512 slots 10.7 and 11.9, sparse 256 slots 10.0 and 11.4, one slot 5.8 and 6.9. Ring 2^16, 51 + 13 primes in 32-bit words, log QP 1734, dnum 4, 128-bit security. perseus-core CKKS primitives on one RTX PRO 6000, device milliseconds per call for one ciphertext and per ciphertext in a batch of 8. Addition 0.014 and 0.054, multiplication by a scalar 0.017 and 0.037, by a plaintext 0.015 and 0.037, relinearized multiplication 0.356 and 0.415, rescale 0.047 and 0.065, rotation 0.284 and 0.358, forward and inverse NTT 0.057 and 0.089. Bootstraps, dense 12.2 and 12.8, StC first 10.5 and 10.7, sparse 512 slots 10.7 and 11.9, sparse 256 slots 10.0 and 11.4, one slot 5.8 and 6.9. Ring 2^16, 51 + 13 primes in 32-bit words, log QP 1734, dnum 4, 128-bit security.

PC_BENCH_CHAIN=n32a4 cargo run --release -p pc-tools -F pc-gpu/mont --bin pc-bench -- modmul --k 1,8 --reps 20   # leaf operations
PC_BENCH_CHAIN=n32a4 cargo run --release -p pc-tools --bin pc-bench -- bts --all --k 1,8 --reps 20 --keys packed  # bootstraps

Reproduce

After the install, each script prints one row's time and its accuracy against the float64 model (NODE/MEMS bind it to the GPU's NUMA node); each runs the model's default plan (python/perseus/examples/defaults.py).

GPU=0 NODE=1 MEMS=1,2 bash scripts/repro/llama_prefill128.sh   # Llama-3-8B, 128-token prefill
GPU=0 bash scripts/repro/llama_decode.sh                        # Llama-3-8B, decode up to a 512-token context
GPU=0 bash scripts/repro/gpt2_decode_ctx.sh                     # GPT-2, decode up to a 512-token context
GPU=0 bash scripts/repro/gpt2_prefill128.sh                     # GPT-2, 128-token prefill to the first token
GPU=0 bash scripts/repro/bert_base.sh                           # BERT-base, 128 tokens (SST-2)
GPU=0 bash scripts/repro/vit_base.sh                            # ViT-B/16, four EuroSAT images
GPU=0 bash scripts/repro/resnet20.sh                            # ResNet-20, 64 CIFAR-10 images

Llama-3-8B needs access to meta-llama/Llama-3.1-8B; Llama, BERT and ViT were measured on a 96 GB GPU.

Layout

  • crates/ the Rust + CUDA runtime
    • pc-core the CUDA-free CKKS mathematics (parameters, encoding, keys, bootstrap constants); pc-shadow the level/degree/scale engine the planner, the recorder and the runtime share; pc-config the typed run configuration.
    • pc-cuda, pc-kernels, pc-cublas, pc-gpu the device runtime (NTT, key switching, bootstrap routes, PCMM/CCMM, the plaintext store, one frozen device arena).
    • pc-program a recorded block + its plan -> a compiled program, and its replay.
    • pc-py the Python extension perseus._pc; pc-client the CUDA-free client (keys, encryption, the wire format); pc-tools benchmarks and cross-checks.
  • python/perseus/
    • impl/ the Python implementation of encrypted linear layers, attention, normalization and activations, the recorder and the planned drivers.
    • plan/ the planner, which reads a recorded forward pass, simulates how levels are spent, places the bootstraps and picks each one's type. It also holds our ports of the DaCapo, Orion and Fhelipe planners.
    • examples/{gpt2,llama,bert,vit,resnet,sana}/ the models on perseus.impl, with their runners and planning scripts (make_plan.sh); defaults.py the plan each runs by default.
    • calibrate/, export.py, hub.py the fitting of the polynomial approximations, the export of HuggingFace weights, the model download.
    • client/, serve/ the client that holds the secret key and the server that runs on its key bundle.
    • configs/model/approximation/ the fitted approximations of every model.
  • graphs/ the recorded forward passes the plans are computed from (the Llama ones are not in the repository: graphs/LLAMA.txt lists them and how to record them).
  • bootstrap_placements/ one directory per plan, each with the recipe that regenerates it and the runtime switches it was made for (PLAN_CMD.txt).
  • scripts/repro/ one script per result; scripts/make_plans.sh replays every plan's recipe; scripts/utils/{dacapo,orion}_upstream/ run the released DaCapo and Orion tools.
  • notebooks/ a custom model on the runtime's operations and the client-server split.
  • tools/ the repo gate (ci.sh) and the route-table tool.
  • assets/ the logos and the paper's result tables (paper-tables.tex, results-table.tex, rendered by render.sh).

Python API

perseus.impl is where models are written, as encrypted layers in Python. The recorder runs them once on a CPU twin (no GPU and no values, only the op stream, the plaintexts and the levels), the runtime compiles them per block with the plan, and the planned drivers replay them token after token. python/perseus/examples/gpt2 is GPT-2 written with it; its runner is the shortest way in.

source scripts/local_env.sh                     # the weights and the reference outputs (README step 1)
.venv/bin/python -m perseus.examples.gpt2.run_decode --tokens 4 --argmax          # the default plan
.venv/bin/python -m perseus.examples.gpt2.run_decode --fake --tokens 1 --argmax \
    --capture graphs/my_capture                                                  # record for the planner (CPU)

The runtime's operations are also callable one at a time, which is how a new model starts (notebooks/custom_encrypted_model.ipynb).

import numpy as np
from perseus import _pc

cfg = _pc.Config.preset("n32_gpt2_decode")             # the 32-bit chain
keys = _pc.KeySource.generate([1, 2, 4, 8], seed=0)     # secret key + the rotations the model uses
sess = _pc.Session(cfg, keys, pool=32, pt_slots=8).complete_setup()
fhe = sess.fhe
x = np.zeros(fhe.slots); x[:16] = np.random.default_rng(0).standard_normal(16)
y = fhe.rotate_and_sum(fhe.square(fhe.encrypt(x, 0)), 1, 16)   # sum of squares into slot 0
print(np.real(fhe.decrypt_slots_complex(y)[0]), np.sum(x ** 2))
sess.close()

Branching on encrypted values goes through perseus.impl.tfhe: register a table (register_sign, register_lut) and the slots it reads (register_slots, with the copies a packing replicates), call tfhe_lut(ops, x, table, slots) in a stage, and give the planned session tfhe=<values per pass> (setup_session registers the tables). python -m perseus.examples.resnet.run --plan bootstrap_placements/resnet20_tfhe_argmax --tfhe --tfhe-argmax runs ResNet-20 with its ReLU signs and an encrypted argmax head that way.

A new model subclasses perseus.impl.driver.ImplModel and lists its stages, the masks each step needs and its weights; python/perseus/impl/__init__.py describes the modules. The client (perseus.client) and the server (perseus.serve) split the key holder from the GPU (notebooks/client_server.ipynb).

Pipeline

The complete sequence for GPT-2 on the 32-bit parameters, from the weights to an encrypted run. Llama-3 follows the same steps with python/perseus/examples/llama/make_plan.sh. The BERT and ViT plans ship ready to run. Every file a step produces is already in the repository, so any step can be skipped.

0. Install

You need the following.

  • Linux, an NVIDIA GPU of compute capability 12.0 (Blackwell; the kernels are built for sm_120) and CUDA 13.2 (CUDA_HOME, default /usr/local/cuda-13.2).
  • A GPU with enough memory for the run. Decoding peaks at 63.1 GiB with GPT-2 and 52.4 GiB with Llama-3-8B, and the Llama-3-8B prefill needs the most, 85.1 GiB (the first table lists every peak, measured on a 96 GB GPU).
  • The Rust toolchain (rustup; rust-toolchain.toml pins the version) and Python 3.12.
git clone https://github.com/gladia-research-group/perseus
cd perseus
uv venv --python 3.12 .venv && uv pip install --python .venv/bin/python -e ".[hf,examples,dev]"
.venv/bin/maturin develop --release              # builds the runtime and installs perseus._pc

The planner, the weight export and the approximation fitting run without the extension. scripts/local_env.sh sets where the data lives, and a variable you export yourself always wins.

variable what it points at default
PERSEUS_DATA where the weights, the calibration text and the reference outputs live .cache/
WEIGHTS_PATH the exported weights (weights.bin.zip) under PERSEUS_DATA
ALL_BLOCKS_IO_DIR the reference outputs that encrypted runs are checked against under PERSEUS_DATA

The CKKS parameters and the runtime switches come from the plan (its PLAN_CMD.txt) and the typed configuration (perseus._pc.Config).

1. Export the weights, fit the approximations, compute the reference outputs

.venv/bin/perseus-export --model openai-community/gpt2 --out .cache/models/openai-community/gpt2
.venv/bin/python -c 'import numpy as np; from omegaconf import OmegaConf; from perseus.calibrate import data; \
    np.save(".cache/pools/openwebtext_gpt2.npy", np.asarray(data.load_token_pool("openai-community/gpt2", \
    OmegaConf.load("python/perseus/configs/dataset/openwebtext.yaml"))))'
.venv/bin/perseus-calibrate model=gpt2 dataset=openwebtext
.venv/bin/python scripts/utils/gen_gpt2_oracle.py --model openai-community/gpt2 \
    --pool .cache/pools/openwebtext_gpt2.npy --out .cache/oracle/gpt2/all_blocks_io

The export packs the weights for the runtime. The calibration fits a polynomial approximation to every nonlinear function (LayerNorm, softmax, GELU and the encrypted argmax) on the value ranges it sees on OpenWebText. The results use configs fitted the same way for their plans, each named in its plan's PLAN_CMD.txt (gpt2_base_n32_ctx512s for the 512-token decode). The last command runs the plaintext model on a fixed piece of text and saves its inputs and output logits, the reference outputs that encrypted runs are compared against.

A model fine-tuned for encryption, such as the HEAT GPT-2, is exported on top of its base model with --checkpoint (a model.pt file or a HuggingFace repository id).

.venv/bin/perseus-export --model openai-community/gpt2 \
    --checkpoint gladia/heat-gpt2-small-openwebtext --tag heat --out .cache/models/gladia/heat-gpt2
.venv/bin/python scripts/utils/gen_gpt2_oracle.py --model openai-community/gpt2 \
    --checkpoint gladia/heat-gpt2-small-openwebtext \
    --pool .cache/pools/openwebtext_gpt2.npy --out .cache/oracle/gpt2_heat/all_blocks_io

2. Record a forward pass

The planner reads a record of one encrypted forward pass, with every encrypted value, how its data is laid out and the largest magnitude it reaches. The first token runs without a plan on the CPU twin (the numpy model of the runtime, values included) and is recorded block by block, the encrypted argmax included.

source scripts/local_env.sh
.venv/bin/python -m perseus.examples.gpt2.run_decode --fake --tokens 1 --argmax \
    --capture graphs/gpt2_decode

A recording belongs to one approximation config and one set of runtime switches, and changing either means recording again.

3. Plan

The planner runs on the CPU, in seconds per plan (the comparison tools take minutes). Each plan directory's PLAN_CMD.txt holds its recipe, and make_plans.sh replays them.

bash scripts/make_plans.sh gpt2_decode                # the GPT-2 decode plan
bash scripts/make_plans.sh                            # every plan in the repository

python -m perseus.plan --help lists the planner's options, among them the safety margin on recorded magnitudes (--mag-safety), the accuracy target (--err-target), the largest scaling setting (--cf-max), the sizes of the cheaper bootstrap (--sparse-slots), removing bootstraps that turn out redundant (--prune) and the placement algorithm (--placer {min_cut,ilp,orion,dacapo,fhelipe}, where ilp is the exact algorithm). The 32-bit plans estimate each bootstrap's error from measurements (python/perseus/plan/data/bts_accuracy_n32.json), the 64-bit plans from a formula.

The comparison plans of the placement study come from the released tools. DaCapo needs its compiler, hecate-opt, built once (LLVM/MLIR 18); Orion's solver is downloaded and patched on first use.

bash scripts/utils/dacapo_upstream/build_hecate.sh
bash scripts/make_plans.sh python/dacapo python/orion python/fhelipe

4. Run encrypted

source scripts/local_env.sh
DECODE=".venv/bin/python -m perseus.examples.gpt2.run_decode --tokens 16 --argmax"
$DECODE                                                        # the default plan
CTX=1024 bash scripts/repro/gpt2_decode_ctx.sh                 # up to GPT-2's 1024-token context (its reference: the script)

Generation feeds each token back without decrypting it. The encrypted argmax picks the token, its embedding is computed from the ciphertext, and the next position starts from there (run_generate, which checks the tokens against greedy decoding of the plaintext model).

.venv/bin/python -m perseus.examples.gpt2.run_generate --prompt 3 --tokens 4

The runtime runs planned only. Every block is a program compiled from its recording and its plan, and a plan that does not bind fails at compile time. The decode runs are teacher-forced (each position's input is the reference token's embedding); the encrypted argmax runs on every position. A plan that binds at every cache length (SM_PERIODIC=2 in its runtime= line, the 1024-context plan) records every position at setup and compiles one program per structure class.

A run passes when it prints [decode] PASS, which means that at every position the reference token stays among the top few of the encrypted output, and the divergence from the plaintext output stays small (a broken run shows a KL above 50). Matching the plaintext top token is reported but not required. Time a run on an otherwise idle machine, with the process pinned to the GPU's NUMA node (numactl --cpunodebind=<node> --membind=<node>).

Encrypted GPT-2 Speedrun

A public leaderboard for encrypted GPT-2 small on one GPU, in two tracks, Prefill-128 and Decode-8, at 128-bit security and a fixed accuracy. It lives on the speedrun branch: the rules, the harness that times entries and the records. Every entry is a pull request against that branch.

Citation

If you use Perseus, please cite the paper, arXiv:2611.04488:

@misc{zirilli2026perseus,
  title         = {Perseus: A Full-Stack System for Homomorphic Model Inference},
  author        = {Zirilli, Alessandro and Marincione, Davide and Kornaropoulos, Evgenios M. and Ateniese, Giuseppe and Rodol{\`a}, Emanuele},
  year          = {2026},
  eprint        = {2611.04488},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CR},
  doi           = {10.48550/arXiv.2611.04488},
  url           = {https://arxiv.org/abs/2611.04488}
}

License

Perseus is released under the Business Source License 1.1, free for any non-commercial use (research, academic, evaluation); commercial use requires prior written permission from the authors. On the Change Date (2030-08-18) the license converts to GPLv2.

GLADIA Research Group

Perseus slew Medusa without ever looking at her, only at her reflection in a polished shield. Perseus runs language models the same way, looking only at the ciphertext.

About

A framework for encrypted CKKS transformer inference on GPU. We implement complex slots utilization for decoding, packing awareness for sparse bootstrap placement, and a python API. Perseus supports encrypted language modelling, image and sentence classification, and image generation.

Topics

Resources

Contributing

Stars

28 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages