OpenRouter and LiteLLM: one interface for every model

Repo: BerriAI/litellm · open source with a separate enterprise licence · and OpenRouter, a hosted service (not open source)

Almost every automation I build calls a language model, and rarely the same one: a strong model for writing, a cheap one for classification, a vision model for chart images. Hard-coding each provider's SDK gets messy fast. Two tools solve that from different directions.

OpenRouter: one key, hundreds of models

OpenRouter is a hosted API that routes an OpenAI-style request to many providers' models. You change a model name; your code doesn't change.

It's what the GammaJuice pipeline uses for its AI commentary, and it's the AI integration behind AffRank. Practical upsides:

• One bill and one key instead of a dozen provider accounts.
• Fast model swaps for comparing quality and cost on the same prompt.
• Fallbacks when a provider is down.

The trade-off: it's a third party in the middle of your traffic, so treat it like any vendor — check its data policies and keep secrets out of prompts.

LiteLLM: the gateway you run yourself

LiteLLM is an open-source AI gateway that gives one OpenAI-format interface to 100+ providers, either as a Python library or as a self-hosted proxy. The proxy adds what teams need: virtual keys, spend tracking, guardrails, load balancing and an admin dashboard. It can also act as a gateway for MCP servers.

It's on my toolbox list for situations where routing has to stay in-house — client work with data constraints, or when several services need shared budgets and keys.

How I choose

The habit that matters more than the router

Whichever you use, log the model name with every request and validate every response. When output quality changes, the first question is "which model answered?" — and the second is "did the code accept something it shouldn't have?" I covered the validation side in structured outputs.