From b583e9c3acb73097bb35a7dbe603be7072fc5bc3 Mon Sep 17 00:00:00 2001 From: ZacharyZcR Date: Wed, 29 Jul 2026 02:23:47 +0800 Subject: [PATCH 1/6] docs: define the inference research mission --- README.md | 67 +++++++++++++++++++++++++++++++++++++------------------ 1 file changed, 45 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index a17696c3..0d2f481e 100644 --- a/README.md +++ b/README.md @@ -12,12 +12,22 @@ English · 简体中文 · 繁體中文 · Italiano

-**Tiny engine, immense model.** Run **GLM-5.2 (744B-parameter MoE)** on a consumer machine with ~25 GB of RAM — in pure C, with zero dependencies, by streaming experts from disk. - -Colibrì is a lightweight, quality-preserving MoE runtime that treats VRAM, RAM, -and storage as one managed memory hierarchy. Insufficient fast memory may reduce -speed, but the default policy **never silently changes model precision or router -semantics**. +**Tiny engine, immense model.** Run **GLM-5.2 (744B-parameter MoE)** on a +consumer machine with ~25 GB of RAM — in pure C, with zero dependencies, by +streaming experts from disk. + +> **Colibrì is an experimental inference engine and research platform.** Its +> primary goal is to pursue inference-side performance across the entire +> software/hardware boundary — model formats, memory hierarchy, storage I/O, +> placement, scheduling, kernels, speculation, and CPU/GPU overlap — so large +> models depend less on scarce hardware and cost less to run. + +Colibrì treats VRAM, RAM, and storage as one managed memory hierarchy. It is +deliberately a place to test aggressive systems ideas, not a production runtime +with an SLA. Experiments must earn their place through reproducible end-to-end +measurements, and the default policy **never silently changes model precision +or router semantics**. Insufficient fast memory may reduce speed; it must not +quietly redefine the model. ``` $ ./coli chat @@ -50,15 +60,26 @@ brightness is routing heat, and every expert routed in a turn flashes white. Hov as a 3-D galaxy — 13,260 characterised experts, 1,041 replicated specialists clustering by topic (poetry, law, Chinese, SQL…). Position is measured routing affinity, not a learned embedding. Drag to spin.

-## The vision +## The research mission + +Frontier inference should not require datacenter-class hardware by default. +Colibrì's research target is simple: **reduce the hardware dependency and total +cost of inference by optimizing every part of the inference path that evidence +shows is limiting it**. + +That includes changing how weights are represented and moved, deciding what +lives in VRAM, RAM, or storage, overlapping heterogeneous compute, reducing +launch and synchronization overhead, exploiting sparsity and reuse, and testing +new decoding algorithms. Nothing is protected merely because it is conventional; +nothing is adopted merely because a microbenchmark looks fast. The deciding +result is end-to-end inference on real machines, with correctness and quality +measured alongside throughput, latency, memory, and cost. -Frontier models should not be sealed inside datacenters. colibrì exists so that -**anyone curious enough can open one up**: run a 744B-parameter mind on hardware -you already own, watch every expert fire in real time, and change the code that -does it. Not renting intelligence behind an API — *holding* it: probing it, -measuring it, improving it. Every optimisation in this project started with -someone measuring something on their own machine; the engine is deliberately -small enough that the next one can come from you. +The practical consequence is accessibility: run a 744B-parameter model on +hardware you already own, watch every expert fire in real time, and change the +code that does it. Not renting intelligence behind an API — *holding* it: +probing it, measuring it, improving it. The engine is deliberately small enough +that the next useful optimization can come from anyone willing to measure it. ## The idea @@ -244,9 +265,10 @@ so put it on a disk with the room, ideally a fast one: > ⚠️ Use the **gs64** container above, not the older per-row int4 mirrors > (`mateogrgic/…`, `jlnsrk/…`): those measure ~9pp worse on quality and are the -> root cause of the think-mode loops and never-terminating generations in -> [#455](https://github.com/JustVugg/colibri/issues/455) — the gs64 container -> cured every failing case there. The MTP head must also be **int8, not int4** +> root cause of the original think-mode loops and never-terminating generations +> in [#455](https://github.com/JustVugg/colibri/issues/455). The gs64 container +> fixed those controlled per-row A/Bs, but it is not a general repetition or +> EOS-starvation guard. The MTP head must also be **int8, not int4** > (int4 → 0% draft acceptance, [#8](https://github.com/JustVugg/colibri/issues/8)): > `ls -l /out-mtp-*` — int8 (correct) is `3527131672 / 5366238584 / 1065950496`. @@ -287,11 +309,12 @@ and the optional API gateway. ## What's next -- **Algorithmic research is active.** The current hierarchy is LRU + a learned - pin set; the next step is under way — smarter placement and scheduling, - overlap of CPU and GPU expert execution, and routing-aware speculation. - Everything lands the way this project always works: measured, reviewed, and - merged in the open. +- **Inference-systems research is the product.** The current hierarchy is LRU + + a learned pin set; active work spans model formats, compression, placement, + scheduling, I/O, CPU/GPU kernels, heterogeneous overlap, KV state, and + routing-aware speculation. The objective is lower hardware requirements and + lower cost per useful token. Everything lands the way this project works: + measured end to end, reviewed, and developed in the open. - **More open models.** The tiering algorithm is model-agnostic: any MoE with routed experts can be staged the same way. GLM-5.2 and OLMoE run today; support for more open-weight families — **Kimi K2** (Moonshot AI), From 22b505a248c65aac63e0bd48b4c97144d3e689e6 Mon Sep 17 00:00:00 2001 From: ZacharyZcR Date: Wed, 29 Jul 2026 02:24:38 +0800 Subject: [PATCH 2/6] docs: remove the minimum-RAM headline --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 0d2f481e..44fe511a 100644 --- a/README.md +++ b/README.md @@ -12,9 +12,9 @@ English · 简体中文 · 繁體中文 · Italiano

-**Tiny engine, immense model.** Run **GLM-5.2 (744B-parameter MoE)** on a -consumer machine with ~25 GB of RAM — in pure C, with zero dependencies, by -streaming experts from disk. +**Tiny engine, immense model.** Explore **GLM-5.2 (744B-parameter MoE)** across +consumer and heterogeneous hardware — in pure C, with zero engine dependencies, +by treating storage, RAM, and VRAM as one inference hierarchy. > **Colibrì is an experimental inference engine and research platform.** Its > primary goal is to pursue inference-side performance across the entire From e42541cb82df810430cbe55fc8e8b8d6c1d0b37c Mon Sep 17 00:00:00 2001 From: ZacharyZcR Date: Wed, 29 Jul 2026 02:27:22 +0800 Subject: [PATCH 3/6] docs: align translated project positioning --- README.it.md | 68 +++++++++++++++++++++++++++++++++++++++++-------- README.zh-CN.md | 46 ++++++++++++++++++++++++++++----- README.zh-TW.md | 46 ++++++++++++++++++++++++++++----- 3 files changed, 136 insertions(+), 24 deletions(-) diff --git a/README.it.md b/README.it.md index 09292357..3d16a62c 100644 --- a/README.it.md +++ b/README.it.md @@ -6,12 +6,21 @@ English · 简体中文 · 繁體中文 · Italiano

-**Motore piccolo, modello immenso.** Esegui **GLM-5.2 (744 miliardi di parametri, MoE)** su un computer consumer con ~25 GB di RAM — in C puro, zero dipendenze, caricando gli expert dal disco in streaming. - -Colibrì è un runtime MoE leggero e che preserva la qualità: tratta VRAM, RAM e -disco come un'unica gerarchia di memoria gestita. Se la memoria veloce non basta -il modello rallenta, ma la policy predefinita **non cambia mai silenziosamente la -precisione del modello né la semantica del router**. +**Motore piccolo, modello immenso.** Esplora **GLM-5.2 (MoE da 744 miliardi di +parametri)** su hardware consumer ed eterogeneo — in C puro, senza dipendenze +del motore, trattando storage, RAM e VRAM come un'unica gerarchia di inferenza. + +> **Colibrì è un motore di inferenza sperimentale e una piattaforma di ricerca.** +> Il suo obiettivo principale è migliorare le prestazioni di inferenza lungo +> l'intero confine software/hardware — formati dei modelli, gerarchia di memoria, +> I/O dello storage, piazzamento, scheduling, kernel, speculazione e sovrapposizione +> CPU/GPU — affinché i grandi modelli dipendano meno da hardware raro e costino meno. + +Colibrì è intenzionalmente un luogo dove verificare idee di sistema aggressive, +non un runtime di produzione con SLA. Gli esperimenti devono dimostrare il proprio +valore con misure end-to-end riproducibili; la policy predefinita **non cambia mai +silenziosamente la precisione del modello né la semantica del router**. Una memoria +veloce insufficiente può ridurre la velocità, ma non ridefinire il modello di nascosto. ``` $ ./coli chat @@ -46,6 +55,26 @@ misurato degli expert come una galassia 3D — 13.260 expert caratterizzati, replicabili che si raggruppano per argomento (poesia, legge, cinese, SQL…). La posizione deriva dall'affinità di routing misurata, non da un embedding appreso. Trascinare per ruotare.

+## La missione di ricerca + +L'inferenza di frontiera non dovrebbe richiedere per forza hardware da datacenter. +L'obiettivo di Colibrì è semplice: **ridurre la dipendenza dall'hardware e il costo +totale dell'inferenza, ottimizzando ogni parte del percorso che le misure indicano +come limitante**. + +Questo comprende cambiare il modo in cui i pesi sono rappresentati e spostati, +decidere cosa risiede in VRAM, RAM o storage, sovrapporre calcolo eterogeneo, +ridurre i costi di avvio e sincronizzazione, sfruttare sparsità e riuso e verificare +nuovi algoritmi di decoding. La convenzione non protegge una tecnica; un microbenchmark +veloce non basta ad adottarla. Decide l'inferenza end-to-end su macchine reali, +misurando correttezza e qualità insieme a throughput, latenza, memoria e costo. + +Il risultato pratico è l'accessibilità: eseguire un modello da 744B sull'hardware +che già possiedi, osservare ogni expert in tempo reale e modificare il codice che +lo rende possibile. Non noleggiare intelligenza dietro un'API, ma possederla, +analizzarla, misurarla e migliorarla. Il motore resta volutamente abbastanza piccolo +perché la prossima ottimizzazione utile possa arrivare da chiunque sia disposto a misurarla. + ## L'idea Un modello Mixture-of-Experts da 744B attiva solo ~40B parametri per token — e @@ -194,14 +223,20 @@ resta in `c/` — è un'installazione editabile dal clone, non un wheel). ### 2. Scarica il modello -Un container **GLM-5.2 int4** pre-convertito è su Hugging Face — **usa la -versione con le teste MTP int8**. Pesa circa **372 GB**, quindi mettilo su un +Un container **GLM-5.2 int4** pre-convertito è su Hugging Face — usa la build +**group-scaled (gs64) con la testa MTP int8**. Pesa circa **372 GB**, quindi mettilo su un disco che abbia lo spazio, meglio se veloce: **https://huggingface.co/mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp** -> ⚠️ Il mirror originale contiene teste MTP int4 → accettazione dei draft allo 0% -> ([#8](https://github.com/JustVugg/colibri/issues/8)). Verifica la tua versione: +> ⚠️ Usa il container **gs64** qui sopra, non i vecchi mirror int4 per-row +> (`mateogrgic/…`, `jlnsrk/…`): misurano circa 9 punti percentuali in meno sulla +> qualità e causavano i loop in think-mode e le generazioni senza termine originali +> di [#455](https://github.com/JustVugg/colibri/issues/455). Il container gs64 ha +> corretto quegli A/B per-row controllati, ma non è una protezione generale contro +> ripetizioni o EOS starvation. Anche la testa MTP deve essere **int8, non int4** +> (int4 → 0% di accettazione dei draft, +> [#8](https://github.com/JustVugg/colibri/issues/8)): > `ls -l /out-mtp-*` — int8 (corretto) è `3527131672 / 5366238584 / 1065950496`. Oppure converti tu stesso dalla sorgente FP8 — un unico comando riprendibile che @@ -238,6 +273,19 @@ e per il gateway API opzionale. | Draft forzati da grammatica (output strutturato) | [docs/grammar-draft.md](docs/grammar-draft.md) | | Inventario delle variabili d'ambiente | [docs/ENVIRONMENT.md](docs/ENVIRONMENT.md) | +## Prossimi passi + +- **La ricerca sui sistemi di inferenza è il prodotto.** La gerarchia attuale usa + LRU e un insieme appreso di expert fissati; il lavoro attivo copre formati, + compressione, piazzamento, scheduling, I/O, kernel CPU/GPU, sovrapposizione + eterogenea, stato KV e speculazione consapevole del routing. L'obiettivo è + ridurre i requisiti hardware e il costo per token utile, con risultati misurati + end-to-end, revisionati e sviluppati apertamente. +- **Più modelli aperti.** L'algoritmo di tiering è indipendente dal modello: + qualsiasi MoE con expert instradati può essere organizzato allo stesso modo. + GLM-5.2 e OLMoE funzionano già; **Kimi K2**, **Qwen3 MoE** e **MiniMax** sono + nella roadmap. + ## Sostenere il progetto colibrì è nato come progetto di una sola persona su un portatile con 12 core diff --git a/README.zh-CN.md b/README.zh-CN.md index 12e57e5b..0229f735 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -6,11 +6,16 @@ English · 简体中文 · 繁體中文 · Italiano

-**小巧引擎，庞大模型。**只需约 25 GB 内存，就能在消费级电脑上运行 **GLM-5.2（744B 参数的 MoE）**——以零依赖的纯 C 实现，从磁盘流式加载专家。 +**小巧引擎，庞大模型。**在消费级与异构硬件上探索 **GLM-5.2（744B 参数的 +MoE）**——以引擎零依赖的纯 C 实现，将存储、RAM 与 VRAM 视为统一的推理层级。 -Colibrì 是一套轻量、保持模型质量的 MoE 运行时，将 VRAM、RAM -与存储设备视为统一管理的内存层级。高速内存不足可能降低速度， -但默认策略**绝不会在未告知的情况下改变模型精度或路由语义**。 +> **Colibrì 是一个实验性推理引擎与研究平台。**它的首要目标是在完整的软硬件边界上 +> 追求推理侧性能——模型格式、内存层级、存储 I/O、放置、调度、内核、推测解码以及 +> CPU/GPU 重叠执行——让大模型减少对稀缺硬件的依赖，并降低运行成本。 + +Colibrì 刻意用于验证激进的系统思路，而不是提供 SLA 的生产运行时。实验必须通过 +可复现的端到端测量证明价值；默认策略**绝不会在未告知的情况下改变模型精度或路由语义**。 +高速内存不足可以降低速度，但不能悄悄重新定义模型。 ``` $ ./coli chat @@ -43,6 +48,20 @@ VRAM／RAM／磁盘层级条，以及角落的实时迷你大脑。

呈现为 3D 星系——共 13,260 个已分析专家，其中 1,041 个可复现的专门专家会按主题聚集 （诗歌、法律、中文、SQL……）。位置取自实测路由亲和度，而非学习出的嵌入向量。拖拽即可旋转。

+## 研究使命 + +前沿模型推理不该默认要求数据中心级硬件。Colibrì 的研究目标很简单： +**优化证据表明受限的每一段推理路径，降低推理的硬件依赖与总成本**。 + +这包括改变权重的表示与移动方式，决定哪些内容常驻 VRAM、RAM 或存储，重叠异构计算， +降低启动与同步开销，利用稀疏性与复用，并验证新的解码算法。传统做法不是免责理由， +微基准快也不是采用理由；最终依据是在真实机器上的端到端推理，同时测量正确性、质量、 +吞吐、延迟、内存与成本。 + +它最终带来的是可及性：在已有硬件上运行 744B 模型，实时观察每个专家，并直接修改实现。 +不是从 API 租用智能，而是持有、探测、测量和改进它。引擎刻意保持足够小，让任何愿意测量 +的人都可能贡献下一项有效优化。 + ## 核心概念 744B 的专家混合（Mixture-of-Experts）模型，每个 token 只会激活约 40B 参数—— @@ -176,12 +195,16 @@ git clone https://github.com/JustVugg/colibri && cd colibri/c ### 2. 获取模型 Hugging Face 上已有预转换的 **GLM-5.2 int4** 容器——请务必使用 -**含 int8 MTP head 的版本**。它约为 **372 GB**，请放在空间足够的磁盘上，最好是快盘： +**含 int8 MTP head 的 group-scaled（gs64）版本**。它约为 **372 GB**，请放在空间足够的磁盘上，最好是快盘： **https://huggingface.co/mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp** -> ⚠️ 原始镜像使用 int4 MTP head → 草稿接受率为 0% ->（[#8](https://github.com/JustVugg/colibri/issues/8)）。请检查你的版本： +> ⚠️ 请使用上面的 **gs64** 容器，不要使用较旧的 per-row int4 镜像 +>（`mateogrgic/…`、`jlnsrk/…`）：后者质量实测低约 9 个百分点，也是 +> [#455](https://github.com/JustVugg/colibri/issues/455) 最初 think-mode 循环与生成不终止的根因。 +> gs64 修复了受控的 per-row A/B 问题，但不是通用的重复或 EOS starvation 防护。 +> MTP head 也必须是 **int8，而非 int4**（int4 的草稿接受率为 0%， +> [#8](https://github.com/JustVugg/colibri/issues/8)）： > `ls -l /out-mtp-*`——正确的 int8 大小为 `3527131672 / 5366238584 / 1065950496`。 你也可以自行从 FP8 源转换——只需一条可断点续传的命令，且任何时候都不需要 @@ -217,6 +240,15 @@ COLI_MODEL=/nvme/glm52_i4 ./coli doctor # 只读就绪检查 | 语法强制草稿（结构化输出） | [docs/grammar-draft.md](docs/grammar-draft.md) | | 环境变量完整清单 | [docs/ENVIRONMENT.md](docs/ENVIRONMENT.md) | +## 下一步 + +- **推理系统研究就是产品。**当前层级采用 LRU 与学习型固定集；正在研究模型格式、压缩、 + 放置、调度、I/O、CPU/GPU 内核、异构重叠、KV 状态与路由感知推测。目标是降低硬件要求 + 和每个有效 token 的成本，所有成果都以端到端测量为准、经审查并公开开发。 +- **支持更多开放模型。**层级算法与模型无关，任何带路由专家的 MoE 都能用相同方式分层。 + GLM-5.2 与 OLMoE 目前可用；**Kimi K2**、**Qwen3 MoE**、**MiniMax** 等开放权重模型 + 已列入路线图。 + ## 支持项目 colibrì 最初由一人使用 12 核心、25 GB RAM 的笔记本开发； diff --git a/README.zh-TW.md b/README.zh-TW.md index 4fe6e632..6ad5a972 100644 --- a/README.zh-TW.md +++ b/README.zh-TW.md @@ -6,11 +6,16 @@ English · 简体中文 · 繁體中文 · Italiano

-**小巧引擎，龐大模型。**只要約 25 GB 記憶體，就能在消費級電腦上執行 **GLM-5.2（744B 參數的 MoE）**——以零相依套件的純 C 實作，從硬碟串流載入專家。 +**小巧引擎，龐大模型。**在消費級與異質硬體上探索 **GLM-5.2（744B 參數的 +MoE）**——以引擎零相依套件的純 C 實作，將儲存、RAM 與 VRAM 視為統一的推論階層。 -Colibrì 是一套輕量、維持品質的 MoE 執行環境，將 VRAM、RAM -與儲存裝置視為統一管理的記憶體階層。高速記憶體不足可能降低速度， -但預設策略**絕不會在未告知的情況下改變模型精度或路由語意**。 +> **Colibrì 是一個實驗性推論引擎與研究平台。**它的首要目標是在完整的軟硬體邊界上 +> 追求推論側效能——模型格式、記憶體階層、儲存 I/O、配置、排程、核心、推測解碼以及 +> CPU/GPU 重疊執行——讓大型模型減少對稀缺硬體的依賴，並降低執行成本。 + +Colibrì 刻意用於驗證激進的系統構想，而不是提供 SLA 的生產執行環境。實驗必須透過 +可重現的端到端測量證明價值；預設策略**絕不會在未告知的情況下改變模型精度或路由語意**。 +高速記憶體不足可以降低速度，但不能悄悄重新定義模型。 ``` $ ./coli chat @@ -43,6 +48,20 @@ VRAM／RAM／硬碟層級長條，以及角落的即時迷你大腦。

呈現為 3D 星系——共 13,260 個已分析專家，其中 1,041 個可重現的專門專家會按主題聚集 （詩歌、法律、中文、SQL……）。位置取自實測路由親和度，而非學習出的嵌入向量。拖曳即可旋轉。

+## 研究使命 + +前沿模型推論不該預設要求資料中心級硬體。Colibrì 的研究目標很簡單： +**最佳化證據顯示受限的每一段推論路徑，降低推論的硬體依賴與總成本**。 + +這包括改變權重的表示與移動方式，決定哪些內容常駐 VRAM、RAM 或儲存，重疊異質運算， +降低啟動與同步開銷，利用稀疏性與重用，並驗證新的解碼演算法。傳統做法不是免責理由， +微基準快也不是採用理由；最終依據是在真實機器上的端到端推論，同時測量正確性、品質、 +吞吐、延遲、記憶體與成本。 + +它最終帶來的是可及性：在既有硬體上執行 744B 模型，即時觀察每個專家，並直接修改實作。 +不是從 API 租用智慧，而是持有、探測、測量和改進它。引擎刻意維持足夠小，讓任何願意測量 +的人都可能貢獻下一項有效最佳化。 + ## 核心概念 744B 的專家混合（Mixture-of-Experts）模型，每個 token 只會啟用約 40B 參數—— @@ -176,12 +195,16 @@ git clone https://github.com/JustVugg/colibri && cd colibri/c ### 2. 取得模型 Hugging Face 上已有預先轉換的 **GLM-5.2 int4** 容器——請務必使用 -**含 int8 MTP head 的版本**。它約為 **372 GB**，請放在空間足夠的硬碟上，最好是快碟： +**含 int8 MTP head 的 group-scaled（gs64）版本**。它約為 **372 GB**，請放在空間足夠的硬碟上，最好是快碟： **https://huggingface.co/mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp** -> ⚠️ 原始鏡像使用 int4 MTP head → 草稿接受率為 0% ->（[#8](https://github.com/JustVugg/colibri/issues/8)）。請檢查你的版本： +> ⚠️ 請使用上面的 **gs64** 容器，不要使用較舊的 per-row int4 鏡像 +>（`mateogrgic/…`、`jlnsrk/…`）：後者品質實測低約 9 個百分點，也是 +> [#455](https://github.com/JustVugg/colibri/issues/455) 最初 think-mode 迴圈與生成不終止的根因。 +> gs64 修復了受控的 per-row A/B 問題，但不是通用的重複或 EOS starvation 防護。 +> MTP head 也必須是 **int8，而非 int4**（int4 的草稿接受率為 0%， +> [#8](https://github.com/JustVugg/colibri/issues/8)）： > `ls -l /out-mtp-*`——正確的 int8 大小為 `3527131672 / 5366238584 / 1065950496`。 你也可以自行從 FP8 來源轉換——只需一條可續傳的指令，且任何時候都不需要 @@ -217,6 +240,15 @@ COLI_MODEL=/nvme/glm52_i4 ./coli doctor # 唯讀就緒檢查 | 文法強制草稿（結構化輸出） | [docs/grammar-draft.md](docs/grammar-draft.md) | | 環境變數完整清單 | [docs/ENVIRONMENT.md](docs/ENVIRONMENT.md) | +## 下一步 + +- **推論系統研究就是產品。**目前階層採用 LRU 與學習型固定集；正在研究模型格式、壓縮、 + 配置、排程、I/O、CPU/GPU 核心、異質重疊、KV 狀態與路由感知推測。目標是降低硬體要求 + 和每個有效 token 的成本，所有成果都以端到端測量為準、經審查並公開開發。 +- **支援更多開放模型。**階層演算法與模型無關，任何帶路由專家的 MoE 都能用相同方式分層。 + GLM-5.2 與 OLMoE 目前可用；**Kimi K2**、**Qwen3 MoE**、**MiniMax** 等開放權重模型 + 已列入路線圖。 + ## 支持專案 colibrì 最初是由一人使用 12 核心、25 GB RAM 的筆電開發； From 04bdc86ca283b743484c043b60ea62381e0d425d Mon Sep 17 00:00:00 2001 From: ZacharyZcR Date: Wed, 29 Jul 2026 02:28:14 +0800 Subject: [PATCH 4/6] docs: highlight inference engine techniques --- README.it.md | 17 +++++++++++++++++ README.md | 17 +++++++++++++++++ README.zh-CN.md | 15 +++++++++++++++ README.zh-TW.md | 15 +++++++++++++++ 4 files changed, 64 insertions(+) diff --git a/README.it.md b/README.it.md index 3d16a62c..41d68584 100644 --- a/README.it.md +++ b/README.it.md @@ -75,6 +75,23 @@ lo rende possibile. Non noleggiare intelligenza dietro un'API, ma possederla, analizzarla, misurarla e migliorarla. Il motore resta volutamente abbastanza piccolo perché la prossima ottimizzazione utile possa arrivare da chiunque sia disposto a misurarla. +## Punti di forza tecnici + +- **Una gerarchia, non una soglia di memoria.** VRAM, RAM e NVMe sono livelli di + piazzamento degli stessi pesi; poca memoria veloce cambia la velocità, non il modello. +- **Un JIT per i pesi.** Il calore di routing misurato alimenta una LRU per layer, + un hot-store appreso e il prefetch del layer successivo senza caricare tutti gli expert. +- **L'I/O fa parte del motore.** Unione degli expert per batch, letture sovrapposte + al calcolo, `O_DIRECT` e striping pesato su due SSD ottimizzano direttamente lo streaming. +- **Esecuzione eterogenea.** CPU, CUDA, Metal, memoria NUMA e residenza parziale o + completa degli expert condividono un runtime e si combinano in base alla macchina. +- **Stato compresso senza cambiare modello.** Validazione token-exact, stato MLA KV + 57× più piccolo, conversazioni persistenti e DSA fedele vincolano l'ottimizzazione + alla correttezza. +- **La speculazione deve meritarsi il costo.** MTP nativo e draft vincolati da + grammatica sono misurati end-to-end e si disattivano quando l'accettazione non + ripaga la verifica. + ## L'idea Un modello Mixture-of-Experts da 744B attiva solo ~40B parametri per token — e diff --git a/README.md b/README.md index 44fe511a..d8a66229 100644 --- a/README.md +++ b/README.md @@ -81,6 +81,23 @@ code that does it. Not renting intelligence behind an API — *holding* it: probing it, measuring it, improving it. The engine is deliberately small enough that the next useful optimization can come from anyone willing to measure it. +## Technical highlights + +- **One hierarchy, not one memory threshold.** VRAM, RAM, and NVMe are placement + tiers for the same weights; limited fast memory changes speed, not model semantics. +- **A JIT for weights.** Measured routing heat drives a per-layer LRU, a learned + pinned hot-store, and one-layer-ahead prefetch instead of loading every expert. +- **I/O is part of the engine.** Batched expert unions, overlapped reads and + compute, `O_DIRECT`, and weighted dual-SSD striping attack the streaming path + rather than pretending storage latency is free. +- **Heterogeneous execution.** CPU, CUDA, Metal, NUMA memory, and partial or full + expert residency share one runtime and can be combined according to the machine. +- **Compressed state without a different model.** Token-exact forward validation, + 57× smaller MLA KV state, persistent warm conversations, and faithful DSA keep + optimization tied to correctness. +- **Speculation that must earn its keep.** Native MTP and grammar-forced drafts + are measured end to end and can be disabled when acceptance does not repay verification. + ## The idea A 744B Mixture-of-Experts model activates only ~40B parameters per token — and diff --git a/README.zh-CN.md b/README.zh-CN.md index 0229f735..0d96e903 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -62,6 +62,21 @@ VRAM／RAM／磁盘层级条，以及角落的实时迷你大脑。

不是从 API 租用智能，而是持有、探测、测量和改进它。引擎刻意保持足够小，让任何愿意测量 的人都可能贡献下一项有效优化。 +## 技术亮点 + +- **统一层级，而非单一内存门槛。**VRAM、RAM 与 NVMe 是同一份权重的不同放置层级； + 高速内存不足只影响速度，不改变模型语义。 +- **权重的 JIT。**实测路由热度驱动逐层 LRU、学习型热门专家固定区和提前一层的预取， + 无需加载所有专家。 +- **I/O 本身就是引擎的一部分。**专家批次并集、读算重叠、`O_DIRECT` 与加权双 SSD + 分流直接优化流式路径，而不是假装存储延迟不存在。 +- **异构执行。**CPU、CUDA、Metal、NUMA 内存以及专家的部分或全部常驻共用一个运行时， + 可按机器条件组合。 +- **压缩状态，不篡改模型。**逐 token 精确的前向验证、缩小 57 倍的 MLA KV 状态、 + 持久化热会话与忠实 DSA，让优化始终受正确性约束。 +- **必须证明收益的推测解码。**原生 MTP 与语法强制草稿均接受端到端测量； + 接受率无法覆盖验证成本时可以关闭。 + ## 核心概念 744B 的专家混合（Mixture-of-Experts）模型，每个 token 只会激活约 40B 参数—— diff --git a/README.zh-TW.md b/README.zh-TW.md index 6ad5a972..ac679d2b 100644 --- a/README.zh-TW.md +++ b/README.zh-TW.md @@ -62,6 +62,21 @@ VRAM／RAM／硬碟層級長條，以及角落的即時迷你大腦。

不是從 API 租用智慧，而是持有、探測、測量和改進它。引擎刻意維持足夠小，讓任何願意測量 的人都可能貢獻下一項有效最佳化。 +## 技術亮點 + +- **統一階層，而非單一記憶體門檻。**VRAM、RAM 與 NVMe 是同一份權重的不同配置階層； + 高速記憶體不足只影響速度，不改變模型語意。 +- **權重的 JIT。**實測路由熱度驅動逐層 LRU、學習型熱門專家固定區和提前一層的預先載入， + 無需載入所有專家。 +- **I/O 本身就是引擎的一部分。**專家批次聯集、讀算重疊、`O_DIRECT` 與加權雙 SSD + 分流直接最佳化串流路徑，而不是假裝儲存延遲不存在。 +- **異質執行。**CPU、CUDA、Metal、NUMA 記憶體以及專家的部分或全部常駐共用一個執行環境， + 可依機器條件組合。 +- **壓縮狀態，不竄改模型。**逐 token 精確的前向驗證、縮小 57 倍的 MLA KV 狀態、 + 持久化熱會話與忠實 DSA，讓最佳化始終受正確性約束。 +- **必須證明收益的推測解碼。**原生 MTP 與文法強制草稿均接受端到端測量； + 接受率無法覆蓋驗證成本時可以關閉。 + ## 核心概念 744B 的專家混合（Mixture-of-Experts）模型，每個 token 只會啟用約 40B 參數—— From 4de7f4c61fa377baef5fc39a391af09c37f96857 Mon Sep 17 00:00:00 2001 From: ZacharyZcR Date: Wed, 29 Jul 2026 02:33:30 +0800 Subject: [PATCH 5/6] docs: publish open inference experiments --- README.it.md | 36 ++++++++++++++++++++++++++++++++---- README.md | 36 ++++++++++++++++++++++++++++++++---- README.zh-CN.md | 33 ++++++++++++++++++++++++++++----- README.zh-TW.md | 33 ++++++++++++++++++++++++++++----- 4 files changed, 120 insertions(+), 18 deletions(-) diff --git a/README.it.md b/README.it.md index 41d68584..88401717 100644 --- a/README.it.md +++ b/README.it.md @@ -75,23 +75,51 @@ lo rende possibile. Non noleggiare intelligenza dietro un'API, ma possederla, analizzarla, misurarla e migliorarla. Il motore resta volutamente abbastanza piccolo perché la prossima ottimizzazione utile possa arrivare da chiunque sia disposto a misurarla. -## Punti di forza tecnici +## Tecniche fondamentali e risultati misurati - **Una gerarchia, non una soglia di memoria.** VRAM, RAM e NVMe sono livelli di piazzamento degli stessi pesi; poca memoria veloce cambia la velocità, non il modello. - **Un JIT per i pesi.** Il calore di routing misurato alimenta una LRU per layer, un hot-store appreso e il prefetch del layer successivo senza caricare tutti gli expert. + Aiuta sui carichi ripetibili, ma la cronologia può sovradattarsi e il prefetch può + perdere su alcuni host: sono policy da misurare, non promesse. - **L'I/O fa parte del motore.** Unione degli expert per batch, letture sovrapposte - al calcolo, `O_DIRECT` e striping pesato su due SSD ottimizzano direttamente lo streaming. + al calcolo, `O_DIRECT` e striping pesato su due SSD ottimizzano direttamente lo + streaming. `O_DIRECT` dipende dal disco e il doppio SSD richiede più A/B end-to-end. - **Esecuzione eterogenea.** CPU, CUDA, Metal, memoria NUMA e residenza parziale o - completa degli expert condividono un runtime e si combinano in base alla macchina. + completa degli expert condividono un runtime; la combinazione utile dipende da + calcolo, banda, residenza e carico. - **Stato compresso senza cambiare modello.** Validazione token-exact, stato MLA KV 57× più piccolo, conversazioni persistenti e DSA fedele vincolano l'ottimizzazione - alla correttezza. + alla correttezza. Sono proprietà di memoria, latenza e correttezza, non una + promessa generale di throughput. - **La speculazione deve meritarsi il costo.** MTP nativo e draft vincolati da grammatica sono misurati end-to-end e si disattivano quando l'accettazione non ripaga la verifica. +## Ipotesi aperte, esperimenti e partecipazione + +Colibrì considera ogni ottimizzazione un'ipotesi finché un A/B end-to-end +controllato non dimostra il contrario. Le domande principali sono: + +| ipotesi | evidenza attuale | esperimento ancora necessario | +|---|---|---| +| La cronologia di routing può piazzare gli expert meglio di una semplice LRU | i pin appresi migliorano carichi ripetuti, ma possono sovradattarsi al prompt | A/B cross-session su set esclusi: codice, chat, multilingua e contesti lunghi | +| Più SSD possono trasformare banda indipendente in velocità di decode | routing pesato mirror/split implementato e validato; il modello di banda è solido | GLM-5.2 a cache fredda, uno contro due dischi su controller indipendenti reali | +| Un planner hardware-aware può avvicinarsi automaticamente alla configurazione migliore | oggi rileva budget RAM/VRAM e diversi backend | confrontare il piano generato con sweep controllati su laptop, workstation, NUMA e multi-GPU | +| Rappresentazioni lossless o a qualità limitata possono ridurre abbastanza il movimento dei pesi | esistono ablation di formato e quantizzazione con gate di qualità | riprodurre insieme qualità, byte mossi, latenza e costo per token utile, non solo il rapporto di compressione | +| La speculazione routing-aware può convenire prima della residenza quasi completa | MTP e draft grammaticali funzionano, ma MTP ha anche perso il 32% intorno all'85% di expert hit | mappare il pareggio tra accettazione, hit rate, batch union e profondità del draft | +| La sovrapposizione CPU/GPU può nascondere trasferimenti e sincronizzazione | esistono risultati positivi CUDA e Metal, ma CPU veloci e bassa residenza possono annullarli | profili per fase e A/B a variabile singola su PCIe, memoria unificata e piena residenza | + +Per contribuire, scegli una riga e pubblica anche i risultati negativi. Registra +hardware, commit, container del modello, comando esatto, prompt, stato cache, +throughput, TTFT, expert hit, byte letti e controllo qualità; cambia una sola +variabile, ripeti e allega i log grezzi. Parti da +[CONTRIBUTING.md](CONTRIBUTING.md), confronta il +[protocollo di benchmark](docs/benchmarks.md), quindi +[apri una issue di esperimento](https://github.com/JustVugg/colibri/issues/new). +Un fallimento controllato vale più di un numero veloce senza spiegazione. + ## L'idea Un modello Mixture-of-Experts da 744B attiva solo ~40B parametri per token — e diff --git a/README.md b/README.md index d8a66229..5d965062 100644 --- a/README.md +++ b/README.md @@ -81,23 +81,51 @@ code that does it. Not renting intelligence behind an API — *holding* it: probing it, measuring it, improving it. The engine is deliberately small enough that the next useful optimization can come from anyone willing to measure it. -## Technical highlights +## Core techniques and measured findings - **One hierarchy, not one memory threshold.** VRAM, RAM, and NVMe are placement tiers for the same weights; limited fast memory changes speed, not model semantics. - **A JIT for weights.** Measured routing heat drives a per-layer LRU, a learned pinned hot-store, and one-layer-ahead prefetch instead of loading every expert. + It wins on repeatable workloads; history can overfit, and lookahead can lose on + some hosts, so both remain measurable policies rather than promises. - **I/O is part of the engine.** Batched expert unions, overlapped reads and compute, `O_DIRECT`, and weighted dual-SSD striping attack the streaming path - rather than pretending storage latency is free. + rather than pretending storage latency is free. `O_DIRECT` is drive-dependent, + and dual-SSD still needs broader end-to-end community A/Bs. - **Heterogeneous execution.** CPU, CUDA, Metal, NUMA memory, and partial or full - expert residency share one runtime and can be combined according to the machine. + expert residency share one runtime and can be combined according to the machine; + the profitable combination depends on compute, bandwidth, residency, and workload. - **Compressed state without a different model.** Token-exact forward validation, 57× smaller MLA KV state, persistent warm conversations, and faithful DSA keep - optimization tied to correctness. + optimization tied to correctness. These are memory, latency, and correctness + properties — not a blanket throughput claim. - **Speculation that must earn its keep.** Native MTP and grammar-forced drafts are measured end to end and can be disabled when acceptance does not repay verification. +## Open hypotheses, experiments, and how to help + +Colibrì treats an optimization as a hypothesis until a controlled end-to-end A/B +shows otherwise. These are the main questions now: + +| hypothesis | evidence so far | experiment still needed | +|---|---|---| +| Routing history can place experts better than plain LRU | learned pins improve repeated workloads, but can overfit a prompt | held-out, cross-session A/Bs across coding, chat, multilingual, and long-context workloads | +| Multiple SSDs can turn independent bandwidth into decode speed | weighted mirror/split routing is implemented and validated; the bandwidth model is sound | cold-cache one-drive vs two-drive GLM-5.2 runs on real, independent controllers | +| A hardware-aware planner can approach each machine's best configuration automatically | RAM/VRAM budgets and several backends are detected today | compare the generated plan with a controlled parameter sweep across laptops, workstations, NUMA hosts, and multi-GPU systems | +| Lossless or quality-bounded representations can reduce weight movement enough to matter | format and quantization ablations exist, with correctness/quality gates | reproduce quality, bytes moved, latency, and cost per useful token together — not compression ratio alone | +| Routing-aware speculation can pay before near-full residency | MTP and grammar drafts work, but MTP has also measured a 32% loss around 85% expert hit | map the break-even surface across acceptance, expert hit rate, batch union, and draft depth | +| CPU/GPU overlap can hide transfer and synchronization rather than merely move the bottleneck | CUDA and Metal wins exist, but fast CPUs and low residency can erase them | per-stage profiles and one-variable A/Bs across PCIe, unified-memory, and full-resident machines | + +Want to help? Pick one row and publish the negative results too. Record the +hardware, commit, model/container, exact command, prompt, cache state, throughput, +TTFT, expert hit rate, bytes read, and quality check; change one variable, repeat +the run, and attach raw logs. Start with +[CONTRIBUTING.md](CONTRIBUTING.md), compare against +[the benchmark protocol](docs/benchmarks.md), then +[open an experiment issue](https://github.com/JustVugg/colibri/issues/new). +A well-controlled failure is more valuable here than an unexplained fast number. + ## The idea A 744B Mixture-of-Experts model activates only ~40B parameters per token — and diff --git a/README.zh-CN.md b/README.zh-CN.md index 0d96e903..3c72c3ab 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -62,21 +62,44 @@ VRAM／RAM／磁盘层级条，以及角落的实时迷你大脑。

不是从 API 租用智能，而是持有、探测、测量和改进它。引擎刻意保持足够小，让任何愿意测量 的人都可能贡献下一项有效优化。 -## 技术亮点 +## 核心技术与实测结论 - **统一层级，而非单一内存门槛。**VRAM、RAM 与 NVMe 是同一份权重的不同放置层级； 高速内存不足只影响速度，不改变模型语义。 - **权重的 JIT。**实测路由热度驱动逐层 LRU、学习型热门专家固定区和提前一层的预取， - 无需加载所有专家。 + 无需加载所有专家。它在可重复负载上有收益，但历史可能过拟合，预取在部分主机上也可能 + 负优化，因此它们是需要测量的策略，不是性能承诺。 - **I/O 本身就是引擎的一部分。**专家批次并集、读算重叠、`O_DIRECT` 与加权双 SSD - 分流直接优化流式路径，而不是假装存储延迟不存在。 + 分流直接优化流式路径，而不是假装存储延迟不存在。`O_DIRECT` 取决于磁盘，双 SSD + 仍需要更多社区端到端 A/B。 - **异构执行。**CPU、CUDA、Metal、NUMA 内存以及专家的部分或全部常驻共用一个运行时， - 可按机器条件组合。 + 可按机器条件组合；最佳组合取决于算力、带宽、驻留率与负载。 - **压缩状态，不篡改模型。**逐 token 精确的前向验证、缩小 57 倍的 MLA KV 状态、 - 持久化热会话与忠实 DSA，让优化始终受正确性约束。 + 持久化热会话与忠实 DSA，让优化始终受正确性约束。这些是内存、延迟和正确性属性， + 不是笼统的吞吐承诺。 - **必须证明收益的推测解码。**原生 MTP 与语法强制草稿均接受端到端测量； 接受率无法覆盖验证成本时可以关闭。 +## 开放猜想、实验与参与方式 + +在受控的端到端 A/B 证明之前，Colibrì 将每项优化都视为猜想。当前主要问题如下： + +| 猜想 | 当前证据 | 仍需完成的实验 | +|---|---|---| +| 路由历史能比普通 LRU 更好地放置专家 | 学习型固定区能改善重复负载，但也会对 prompt 过拟合 | 在代码、对话、多语言和长上下文负载上做留出集、跨会话 A/B | +| 多块 SSD 能将独立带宽转化为解码速度 | 加权镜像／分片路由已实现并通过校验，带宽模型成立 | 在独立控制器的真实磁盘上做冷缓存、单盘与双盘 GLM-5.2 对照 | +| 硬件感知规划器能自动接近每台机器的最优配置 | 当前已检测 RAM/VRAM 预算与多个后端 | 将自动方案与参数扫描对比，覆盖笔记本、工作站、NUMA 和多 GPU 主机 | +| 无损或质量受控的表示能充分减少权重搬运 | 已有格式与量化消融，并设置正确性／质量门槛 | 同时复现质量、搬运字节、延迟和每个有效 token 成本，而非只看压缩率 | +| 路由感知推测能在接近全驻留前盈利 | MTP 与语法草稿可用，但 MTP 在约 85% expert hit 时也实测过 -32% | 绘制接受率、命中率、批次并集与草稿深度的盈亏边界 | +| CPU/GPU 重叠能隐藏传输与同步，而非仅转移瓶颈 | CUDA 与 Metal 有成功数据，但强 CPU 和低驻留率会抹平收益 | 在 PCIe、统一内存与全驻留机器上做逐阶段 profile 和单变量 A/B | + +想参与就任选一行，负结果也请公开。请记录硬件、commit、模型容器、完整命令、prompt、 +缓存状态、吞吐、TTFT、expert hit、读取字节数与质量检查；每次只改一个变量，重复运行并附上 +原始日志。先阅读 [CONTRIBUTING.md](CONTRIBUTING.md) 和 +[benchmark 协议](docs/benchmarks.md)，然后 +[创建实验 issue](https://github.com/JustVugg/colibri/issues/new)。 +在这里，一个受控的失败比一个无法解释的高数字更有价值。 + ## 核心概念 744B 的专家混合（Mixture-of-Experts）模型，每个 token 只会激活约 40B 参数—— diff --git a/README.zh-TW.md b/README.zh-TW.md index ac679d2b..34222e0d 100644 --- a/README.zh-TW.md +++ b/README.zh-TW.md @@ -62,21 +62,44 @@ VRAM／RAM／硬碟層級長條，以及角落的即時迷你大腦。

不是從 API 租用智慧，而是持有、探測、測量和改進它。引擎刻意維持足夠小，讓任何願意測量 的人都可能貢獻下一項有效最佳化。 -## 技術亮點 +## 核心技術與實測結論 - **統一階層，而非單一記憶體門檻。**VRAM、RAM 與 NVMe 是同一份權重的不同配置階層； 高速記憶體不足只影響速度，不改變模型語意。 - **權重的 JIT。**實測路由熱度驅動逐層 LRU、學習型熱門專家固定區和提前一層的預先載入， - 無需載入所有專家。 + 無需載入所有專家。它在可重複負載上有收益，但歷史可能過度擬合，預先載入在部分主機上 + 也可能負最佳化，因此它們是需要測量的策略，不是效能承諾。 - **I/O 本身就是引擎的一部分。**專家批次聯集、讀算重疊、`O_DIRECT` 與加權雙 SSD - 分流直接最佳化串流路徑，而不是假裝儲存延遲不存在。 + 分流直接最佳化串流路徑，而不是假裝儲存延遲不存在。`O_DIRECT` 取決於磁碟，雙 SSD + 仍需要更多社群端到端 A/B。 - **異質執行。**CPU、CUDA、Metal、NUMA 記憶體以及專家的部分或全部常駐共用一個執行環境， - 可依機器條件組合。 + 可依機器條件組合；最佳組合取決於算力、頻寬、常駐率與負載。 - **壓縮狀態，不竄改模型。**逐 token 精確的前向驗證、縮小 57 倍的 MLA KV 狀態、 - 持久化熱會話與忠實 DSA，讓最佳化始終受正確性約束。 + 持久化熱會話與忠實 DSA，讓最佳化始終受正確性約束。這些是記憶體、延遲和正確性屬性， + 不是籠統的吞吐承諾。 - **必須證明收益的推測解碼。**原生 MTP 與文法強制草稿均接受端到端測量； 接受率無法覆蓋驗證成本時可以關閉。 +## 開放猜想、實驗與參與方式 + +在受控的端到端 A/B 證明之前，Colibrì 將每項最佳化都視為猜想。目前主要問題如下： + +| 猜想 | 目前證據 | 仍需完成的實驗 | +|---|---|---| +| 路由歷史能比普通 LRU 更好地配置專家 | 學習型固定區能改善重複負載，但也會對 prompt 過度擬合 | 在程式碼、對話、多語言和長上下文負載上做留出集、跨會話 A/B | +| 多顆 SSD 能將獨立頻寬轉化為解碼速度 | 加權鏡像／分片路由已實作並通過驗證，頻寬模型成立 | 在獨立控制器的真實磁碟上做冷快取、單碟與雙碟 GLM-5.2 對照 | +| 硬體感知規劃器能自動接近每台機器的最佳配置 | 目前已偵測 RAM/VRAM 預算與多個後端 | 將自動方案與參數掃描對比，涵蓋筆電、工作站、NUMA 和多 GPU 主機 | +| 無損或品質受控的表示能充分減少權重搬運 | 已有格式與量化消融，並設置正確性／品質門檻 | 同時重現品質、搬運位元組、延遲和每個有效 token 成本，而非只看壓縮率 | +| 路由感知推測能在接近全常駐前獲利 | MTP 與文法草稿可用，但 MTP 在約 85% expert hit 時也實測過 -32% | 繪製接受率、命中率、批次聯集與草稿深度的盈虧邊界 | +| CPU/GPU 重疊能隱藏傳輸與同步，而非僅轉移瓶頸 | CUDA 與 Metal 有成功數據，但強 CPU 和低常駐率會抹平收益 | 在 PCIe、統一記憶體與全常駐機器上做逐階段 profile 和單變數 A/B | + +想參與就任選一行，負結果也請公開。請記錄硬體、commit、模型容器、完整指令、prompt、 +快取狀態、吞吐、TTFT、expert hit、讀取位元組數與品質檢查；每次只改一個變數，重複執行並附上 +原始日誌。先閱讀 [CONTRIBUTING.md](CONTRIBUTING.md) 和 +[benchmark 協議](docs/benchmarks.md)，然後 +[建立實驗 issue](https://github.com/JustVugg/colibri/issues/new)。 +在這裡，一個受控的失敗比一個無法解釋的高數字更有價值。 + ## 核心概念 744B 的專家混合（Mixture-of-Experts）模型，每個 token 只會啟用約 40B 參數—— From 00400e20e3c9aa1096bd897492f995dfafa18ef5 Mon Sep 17 00:00:00 2001 From: ZacharyZcR Date: Wed, 29 Jul 2026 02:35:13 +0800 Subject: [PATCH 6/6] docs: add Discord community link --- README.it.md | 3 +++ README.md | 3 +++ README.zh-CN.md | 2 ++ README.zh-TW.md | 2 ++ 4 files changed, 10 insertions(+) diff --git a/README.it.md b/README.it.md index 88401717..94545467 100644 --- a/README.it.md +++ b/README.it.md @@ -3,6 +3,7 @@

+ Discord · English · 简体中文 · 繁體中文 · Italiano

@@ -340,6 +341,8 @@ Se ti è utile: - ⭐ metti una stella al repository e condividilo; - 🐛 apri issue con i numeri di benchmark del tuo hardware — i datapoint fanno avanzare questo progetto più di qualsiasi altra cosa; +- 💬 entra nella [comunità Discord](https://discord.gg/fpQxKnRb) per discutere + esperimenti, risultati hardware e direzioni di ricerca; - 💬 contattaci via GitHub issues per sponsorizzare lo sviluppo o donare hardware. ## Struttura del repository diff --git a/README.md b/README.md index 5d965062..d820a6ea 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,7 @@

Website · + Discord · English · 简体中文 · 繁體中文 · Italiano

@@ -373,6 +374,8 @@ today its numbers come from a community of real machines. If it's useful to you: - ⭐ star the repo and share it; - 🐛 open issues with benchmark numbers from your hardware — datapoints move this project more than anything else; +- 💬 join the [Discord community](https://discord.gg/fpQxKnRb) to discuss + experiments, hardware results, and research directions; - 💬 reach out via GitHub issues to sponsor development or donate hardware. ## Repo layout diff --git a/README.zh-CN.md b/README.zh-CN.md index 3c72c3ab..47d3136f 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -3,6 +3,7 @@

+ Discord · English · 简体中文 · 繁體中文 · Italiano

@@ -294,6 +295,7 @@ colibrì 最初由一人使用 12 核心、25 GB RAM 的笔记本开发； - ⭐ 为仓库加星并分享； - 🐛 以 issue 提交你的硬件 benchmark 数据——实测数据比任何其他事都更能推动项目； +- 💬 加入 [Discord 社区](https://discord.gg/fpQxKnRb)，讨论实验、硬件数据与研究方向； - 💬 若想赞助开发或捐赠硬件，请通过 GitHub issues 联系。 ## 仓库结构 diff --git a/README.zh-TW.md b/README.zh-TW.md index 34222e0d..815f931d 100644 --- a/README.zh-TW.md +++ b/README.zh-TW.md @@ -3,6 +3,7 @@

+ Discord · English · 简体中文 · 繁體中文 · Italiano

@@ -294,6 +295,7 @@ colibrì 最初是由一人使用 12 核心、25 GB RAM 的筆電開發； - ⭐ 為儲存庫加星並分享； - 🐛 以 issue 提交你的硬體 benchmark 數據——實測資料比任何其他事都更能推動專案； +- 💬 加入 [Discord 社群](https://discord.gg/fpQxKnRb)，討論實驗、硬體數據與研究方向； - 💬 若想贊助開發或捐贈硬體，請透過 GitHub issues 聯絡。 ## 儲存庫結構