LiteLLM: Unified AI Gateway for 100+ Language Models

LiteLLM: Unified AI Gateway for 100+ Language Models 🔌

#

LiteLLM is an open-source AI gateway that provides a single unified interface to call 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure, and more. You can use it as a Python SDK for direct integration or deploy it as a centralized proxy server (AI Gateway).

💡 Explanation in a nutshell

#

What is LiteLLM?

• Unified AI gateway for 100+ LLM providers
• Single interface compatible with OpenAI
• Can be used as Python SDK or proxy server

Main advantages:

• One API for multiple language models
• OpenAI-compatible - swap providers without rewriting code
• Production-ready with virtual keys, spend tracking and load balancing
• Very low latency: 8ms P95 at 1k RPS

Essential for teams wanting flexibility in their AI systems.

More information at the link 👇

GitHub - BerriAI/litellm: Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and …

github.com ↗