BerriAI/litellm

Drop-in Python SDK and proxy gateway that routes to 100+ LLM APIs using a single OpenAI-shaped interface. The pitch is real: swap Anthropic for Bedrock for Azure without touching your application code. What earns it a pick over the crowded gateway space is the operational layer that ships with it — cost tracking per request, per key budgets, load balancing across providers, and guardrails that fire before the LLM call, not after. If you are running a multi-tenant SaaS where different customers want different models, or you are sick of getting surprise bills when one provider's latency tanks and your retry logic hammers another, this handles both. Reservation: the self-hosted proxy adds meaningful infra overhead; the managed cloud tier pricing is opaque until you dig. But as a pure SDK it is zero-overhead to adopt. -> Best for: SaaS team of 2-5 building LLM-backed features who need provider flexibility without rewriting their inference layer