Tiny-vLLM – high performance LLM inference engine in C++ and CUDA

Tiny-vLLM is an open-source project that rebuilds vLLM's core inference logic in C++ and CUDA rather than Python. The practical value is twofold: it is a legible codebase for anyone trying to understand how paged attention and continuous batching actually work at the systems level, and it is a minimal base for researchers who want to experiment without dragging in the full Python dependency tree. The performance story is unverified — there is no published benchmark comparing it against the original vLLM or llama.cpp on representative workloads. Reservation: if you need production LLM serving today, this is not it. The original vLLM has years of optimization work and active community behind it. This project looks like a learning vehicle and a research sandbox, which is genuinely useful but not what the title implies at first glance. Treat it as a study tool or a starting point, not a drop-in replacement. -> Best for: ML researcher or AI engineer studying inference engine architecture