Skip to content

About

Tokenizer-free byte-level language model research: vocab=256, 1.58-bit ternary quantization, and a hand-assembled x86-64 kernel executed from raw memory. Hand-written backprop verified by gradient check at 2.9e-6.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

1 Commit

Folders and files

Repository files navigation

ElectricMind

A different kind of AI: no tokenizer, no vocabulary, no encoder. It reads the raw bytes — the literal 1s and 0s of any file — and the lowest layer of its compute is machine code written directly into RAM and executed by the CPU.

Most language models start by encoding: they chop text into "tokens" from a fixed vocabulary (a tokenizer / BPE encoder) and look each token up in an embedding table. That tokenizer is a human-designed front-end bolted onto the network. It's why models can't see spelling, struggle with code/binary, and need a different vocabulary per language.

ElectricMind has none of that. The input alphabet is the 256 possible byte values — full stop. Every file on your computer, in every language, every binary format, every image, every executable, is already a sequence of bytes. The model learns directly on that stream. There is nothing between the electricity that stores a byte in RAM and the model's first layer.

The two ideas, both real and both built here

  1. No encoder (byte-native). The model's only alphabet is 0x00–0xFF. It learns to predict the next byte. This is a real 2024–2026 research frontier (Meta's Byte Latent Transformer, MambaByte, bGPT — see research/). We build the smallest honest version: a from-scratch model in pure NumPy, no PyTorch, no TensorFlow. The math is written by hand; nothing is a black box.

  2. Down to the electric (bare-metal compute). The heavy math (matrix multiply) runs through NumPy → OpenBLAS → AVX SIMD instructions on the CPU — that is the electric layer; @ compiles to vector instructions. And to make "down to the electric" literal rather than a figure of speech, ElectricMind includes a component whose kernel is hand-assembled x86-64 machine code, poked byte-by-byte into executable memory and called as a function via ctypes + Windows VirtualAlloc. No compiler. Just the opcodes.

What this is NOT (honesty first)

  • Not a competitor to GPT. It's a small, legible, different architecture.
  • Not magic. "Works without an encoder" is true and meaningful; "AI made of pure electricity beyond physics" is not a thing — every claim here is buildable and verified with runnable code.
  • Not GPU-dependent. CPU-first. A GPU path is researched and noted, but the prototype proves the idea on the CPU you already have.

Layout

core/        the byte-native model (numpy), tokenizer-free data pipeline
prototype/   runnable demos: train on bytes, generate bytes, the electric kernel
bench/       speed + learning-curve measurements (evidence, not claims)
research/    cited research reports (byte models, bare-metal inference)
docs/        design decisions, the honest handoff

Status

See docs/HANDOFF.md for the verified, evidence-backed state at any time. Built on Windows 11, Python 3.14 + NumPy 2.4 (OpenBLAS), no external ML framework.

About

Tokenizer-free byte-level language model research: vocab=256, 1.58-bit ternary quantization, and a hand-assembled x86-64 kernel executed from raw memory. Hand-written backprop verified by gradient check at 2.9e-6.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages