Notes from a community-owned AI mesh.
Release notes, tutorials for distributed llama.cpp inference, architecture deep-dives, and arguments for why AI infrastructure should be owned by the people who use it — not the people who rent it.
llama.cpp RPC backend: distributed inference across multiple machines
The complete guide to running one model across several computers with llama.cpp's RPC backend: building with GGML_RPC=ON, starting rpc-server on each worker, the --rpc host1,host2 comma list, offload with -ngl, and the failure modes nobody warns you about (version skew, the 512-alignment assert, the weakest-hop tax, and no auth on the wire).
llama.cpp tensor-split: running one model across multiple GPUs
What --tensor-split and --split-mode actually do, when to use layer split vs row split, how to size the ratios for uneven GPUs, and a straight answer to the question everyone asks: does llama.cpp really support tensor parallelism? Real flags, real log output, honest limits.
Announcing SharedLLM v0.1.0 — public alpha
The first public alpha of SharedLLM is live. Pre-built llama.cpp RPC binaries for macOS, Linux, and Windows; a three-container integration stack; and the full coordinator + worker handshake under AGPL-3.0. Here's what works, what doesn't, and how to try it.
Splitting Llama across two MacBook Pros with llama.cpp RPC
A step-by-step walkthrough of setting up llama.cpp's RPC backend across two machines: building with GGML_RPC=ON, running rpc-server on the worker, pointing llama-server at it, and watching tensors travel over TCP. Real commands from a working setup.
SharedLLM vs Petals vs Exo vs Kalavai: distributed LLM networks in 2026
Five projects claim to let you run large language models across a mesh of consumer machines. Three of them actually work. Here's an honest, opinionated comparison — including the project I'm building — across backend maturity, governance, privacy posture, and what happens when the coordinator goes down.