feat: add local-cluster MoE, dense sharding, and WebGPU workers

[gauravsaini]

Summary

This PR adds a progressive local-cluster execution path for consumer Macs and browser/WebGPU devices:

• coordinator/worker registration and distributed MoE expert routing
• native batch-union expert RPC for disk-backed workers
• contiguous dense MLP activation sharding across local nodes
• browser/mobile WebGPU expert workers over WebSocket
• native TCP-to-WebSocket activation proxy
• WebGPU expert weight exporter for readable floating-point checkpoints
• browser worker UI and protocol/dispatch tests

The coordinator keeps token generation, routing, attention, and KV state local. Remote workers execute assigned expert FFNs or dense MLP ranges.

Validation

• native make test
• 73 Python tests
• 17 web tests
• TypeScript/Vite production build
• Python and JavaScript syntax checks

Scope and follow-ups

The WebGPU path currently uses f32 buffers and is intended for a trusted LAN. TLS/authentication, WebRTC transport, quantized WebGPU weight formats, and broader dense/tensor-parallel sharding can follow as separate changes.

Not yet tested (sorry for being gpu poor) :

Real C coordinator → dense worker COLIDN01 end-to-end
Dense shard output equivalence against local computation
Real C → WebGPU browser integration
WebGPU FFN numerical correctness against a CPU reference
Exporter tests
Actual iPhone/Android/WebGPU hardware smoke test

[gauravsaini]

@JustVugg: Alla vostra attenzione 😀

[JustVugg]

Heads-up: c/glm.c was just split into c/colibri.c + header modules (#391, now merged into dev). This PR touches the old glm.c, so it will need a rebase onto current dev with its changes moved into colibri.c (or the relevant extracted header: quant.h, sample.h, kv_persist.h, telemetry.h, grammar.h). The make glm target still works (alias of make colibri), so no build scripts break. Apologies for the churn — ping me if the rebase gets thorny and I can help place the hunks.

[JustVugg]

Rebase needed: this has been in conflict against dev for a while and today's merges (#298, #192) widened it. It bundles three big features (local-cluster MoE, dense sharding, WebGPU workers) — honestly it would land much more easily as three separate PRs, each rebased on current dev. Would you be up for splitting it? The cluster piece especially has overlap with the direction in #431/#432 worth reconciling.

[JustVugg]

This bundles three big features (local-cluster MoE, dense sharding, WebGPU workers) in +1408 lines and conflicts with current dev. Honestly it'll land far more easily split into three PRs, each rebased on current dev (CI green) with its own correctness gate: token-exact on the CPU path where applicable, and a coherent-output datapoint for the WebGPU workers. The cluster piece also overlaps #431/#432 — worth reconciling. Would you split it? I'll review each fast. Thanks @gauravsaini.

[gauravsaini]

Sure , will try to do it in coming week if that's ok.

[JustVugg]

Thanks for this, and apologies for the slow first response — it's a big PR from a first-time contributor and it deserved an answer sooner.

Two blockers before content review, one mechanical and one about scope.

Mechanical: the PR modifies c/glm.c, which no longer exists — #391 renamed the engine to c/colibri.c plus header modules (make glm remains only as an alias). So this needs a forward port onto a moved file rather than a rebase. Also, the workflows here have never run on this PR: GitHub holds runs from contributors whose first PR hasn't merged yet, so there are no CI results on it at all yet. That's on me and I'll approve them once it's rebased and there's something meaningful to test.

Scope: this bundles three substantial and largely independent features — local-cluster MoE, dense sharding, and WebGPU workers — at ~1,400 lines across 9 files including a new Python module and browser worker code. Reviewed as one unit, the honest answer is that I cannot merge it: any one of the three failing would block the other two, and I have no way to validate a distributed cluster path or a WebGPU path. Split into three PRs, each becomes assessable on its own terms, and the strongest one can land while the others get discussed.

If I had to pick where to start: dense sharding is closest to the engine's existing concerns and easiest to validate against the current tier code. The cluster and WebGPU pieces raise design questions worth their own threads — particularly how a cluster path interacts with the expert cache and the RAM budget, which the engine now enforces against measured RSS (#403).

Not a rejection. It is a request to make the work reviewable, and a warning that as one 1,400-line bundle touching a file that no longer exists, it will keep sitting here.

[gauravsaini]

#550 - Here is the one just for dense sharding targetted for dev and I'll still send the remaining two independently and close this one
I was building something similar over browser and you've made me revive the effort on that
https://github.com/gauravsaini/tensorbend-p2p

[JustVugg]

Closing this as superseded — and thank you for doing exactly what was asked: you split it into three reviewable PRs (#550 dense MLP sharding, #551 distributed expert workers, #552 WebGPU workers). That's the right shape, and each can now be assessed on its own terms. Full credit for the work; it lives on in the three split PRs, which is where review will continue. Reopen if I've missed something that isn't covered by the split.