llama.cpp vs vLLM vs SGLang: Choosing an Inference Engine You Can Self-Host
Three open-source engines, three different jobs. What each one is built for, the ideas that actually separate them, and how to decide for your own workload.
Software Engineer. I build and write about real-world systems.
I work on backend systems and software architecture. I care about how systems are designed, how teams work together, and how new tools like AI fit into everyday engineering work.
I write about what I learn along the way. The goal is to share ideas that are useful no matter what you are building.
Recent Posts
What Actually Happens When You Quantize a Language Model
How shrinking a model's numeric precision cuts memory and speeds up inference, and exactly what you trade away to get there.
Where AI Actually Speeds Up Engineering Work (And Where It Doesn't)
A plain, honest look at this by task type, not another take on whether AI will replace engineers.