OpenMythos ported to Apple Silicon MLX — with CONFIRMED 4x depth extrapolation!
Disclaimer: OpenMythos is an independent, community-driven theoretical reconstruction. Not affiliated with Anthropic.
Trained on RavenX-Sec security data, M4 Max 128GB:
n_loops= 1: loss=10.3155
n_loops= 2: loss=10.2770 ← trained here
n_loops= 4: loss=10.2448 ← BETTER (extrapolated!)
n_loops= 8: loss=10.2380 ← BEST (4x training depth!)
n_loops=16: loss=10.2488 ← slight degradation
n_loops=32: loss=10.2644 ← ACT halting would fix this
Train at 2 loops → optimal at 8 loops = 4x depth extrapolation. More loops = deeper reasoning without additional parameters.
This is the first demonstration of RDT depth extrapolation on Apple Silicon and on security domain data.
A pure MLX implementation of the OpenMythos architecture — a Recurrent-Depth Transformer (RDT) that uses the same weights looped T times for deeper reasoning without parameter growth. Now includes MoDA (Mixture-of-Depths Attention) where each layer attends to ALL preceding layers' outputs.
Original: github.com/DeadByDawn101/OpenMythos (PyTorch)
Input → [Prelude] → [Recurrent Block × T loops] → [Coda] → Output
↑_____________↓
Same weights, more loops = deeper thinking
Layer N attends to: [current sequence] + [Layer 1..N-1 outputs]
= The model can SEE ITS OWN THINKING from previous layers!
On June 9, 2026, Anthropic released the Claude Mythos 5 / Fable 5 System Card and the Claude Mythos Preview System Card. Their findings validate EVERY core concept in OpenMythos.
We were already on the same path — building independently on Apple Silicon what Anthropic built at $100M+ scale.
| OpenMythos Concept | Anthropic's Parallel | System Card Section |
|---|---|---|
| Depth Extrapolation (train 2 → test 8) | "Substantial post-training and fine-tuning" — iterative refinement across multiple model snapshots | §1.1, §1.1.4 |
| Thinking Toggle (OFF/LOW/MED/HIGH) | "Chain-of-thought controllability" — testing reasoning trace control and monitorability | §6.5.5 (Mythos 5) |
| Multi-Trajectory Reasoning (GRAM best-of-N) | Multi-Agent Harnesses — exploring multiple reasoning paths simultaneously | §8.15 (Mythos 5) |
| Progressive Training Rounds (12 rounds) | "Different snapshots taken at various points during training" with iterative evaluation | §1.4 (both cards) |
| Builder/Breaker (OpenSelfRevise) | Adversarial safety testing — "destructive or reckless actions in pursuit of goals" | §4.3.1 / §6.3.1 |
| Safety at Model + Harness Level | Classifier fallback architecture: Fable 5 falls back to Opus 4.8 when classifiers trigger | §1.5 (Mythos 5) |
| Thinking Block Stripping ( removal) | "Reasoning text is denser and more difficult to interpret" — monitoring and controlling thought traces | §6.5.5.1 (Mythos 5) |
| Agent Memory + Provenance | "Automated offline monitoring" of behavior patterns across sessions | §4.2.1.2 / §6.2.1.2 |
| Cyber as Top Domain | "Most capable model on cyber tasks" — "autonomously discover and exploit zero-day vulnerabilities" | §3 (both cards) |
| Dynamic Profiles (mode switching) | Safety classifier routing — different capability levels per task type | §1.5 (Mythos 5) |
On iterative refinement: "Different snapshots of the model are taken at various points during the training process" — exactly our progressive round training methodology.
On cyber capabilities: "Mythos 5 is the most capable model we have evaluated on cyber tasks... scores far ahead of Claude Opus 4.8" — our RavenX-CyberAgent operates in this same space.
On reasoning control: "Chain-of-thought monitorability evaluations" — validates our thinking toggle and
<think>block management.
On safety architecture: "Fable 5's cybersecurity classifiers are effective at detecting cyber use and cause the model to fall back to Opus 4.8" — same concept as our Dynamic Profiles and harness-level safety.
On model awareness: "Evaluation awareness is significant and not always verbalized" — critical insight for our Evaluations framework design.
We built the open-source version of the same methodology Anthropic uses for their most powerful model.
ANTHROPIC (closed, $100M+): OPENMYTHOS (open, 1 MacBook):
Iterative post-training → Progressive rounds (12)
Chain-of-thought control → Thinking toggle (OFF→HIGH)
Multi-agent harnesses → GRAM multi-trajectory
Classifier-based fallback → Dynamic Profiles
White-box activation analysis → Core AI Debugger (Apple)
Adversarial safety testing → Builder/Breaker (OpenSelfRevise)
Cyber as top capability domain → RavenX-CyberAgent
Safety at model + system level → Dual-level memory (model + harness)
The name isn't a coincidence. The methodology isn't a coincidence. We were on the same path.
import mlx.core as mx
from open_mythos_mlx import OpenMythos, mythos_1b
cfg = mythos_1b()
model = OpenMythos(cfg)
ids = mx.random.randint(0, cfg.vocab_size, (1, 32))
logits = model(ids, n_loops=8) # more loops = deeper reasoningThe breakthrough for stable RDT training on MLX:
for t in range(n_loops):
if t < n_loops - 1:
x = mx.stop_gradient(x) # detach history
out = self.recurrent(x + 0.1 * e)
x = 0.5 * x + 0.5 * outOnly backpropagate through the last loop iteration — prevents gradient explosion through time while maintaining depth extrapolation.
- "RDT-Distilled Security Reasoning in MoE Transformers" — Train RDT on security data, distill deep traces into production MoE model
- "RDT-to-MoE Reasoning Transfer" — General technique for transferring variable-depth reasoning to fixed-depth architectures
- Original PyTorch: OpenMythos by kyegomez
- MLX Port + Depth Extrapolation: @DeadByDawn101 / RavenX LLC
MIT
Become a sponsor on Tribe.run.