Building AI features without locking into a single provider

When I added AI briefings to FinReview, I made a decision that had nothing to do with which model produced the best output: I would not hardcode a provider.

The temptation to hardcode is real. OpenAI's API is well-documented, widely used, and straightforward to integrate. Writing import openai at the top of a file and calling openai.chat.completions.create is a ten-minute implementation. It works immediately.

It also creates a dependency that is harder to remove than it was to add.

I had watched the LLM market move for long enough to be skeptical of any specific provider's long-term position. In the twelve months before I built FinReview, the practical ranking of models by capability had shifted multiple times. Models that were state-of-the-art when a project started were noticeably behind by the time it shipped. Building on a specific model name meant either accepting that the application would use a progressively less capable model, or accepting recurring code changes to update it.

The alternative was to treat the model selection as configuration rather than code.

FinReview externalises two values as environment variables: AI_MODEL_ENDPOINT and AI_MODEL_NAME. The insight service constructs its API call from those values. In production, they point to OpenRouter with a free-tier Gemma model as default. In a self-hosted environment, they can point to a local Ollama instance running a different model entirely. No code change required — only a configuration change.

OpenRouter was the right choice for the endpoint because it provides a single API surface compatible with OpenAI's request format, backed by dozens of provider models. Switching from Gemma to Mistral to Claude to GPT-4 is a one-line environment variable change. OpenRouter handles routing, billing aggregation, and fallback between providers. The application code is unaware of which underlying provider is serving the request.

The insight service also implements a model fallback chain. If the configured model returns a 429 rate limit or a 404, the service retries with an alternative model before falling back to a static informational message. The fallback message is not an error — it is a graceful degradation that keeps the application functional even when the AI layer is unavailable. This is consistent with the broader principle that AI features should enhance the application rather than become a dependency for its basic operation.

One thing I would do differently is provide clearer documentation of which free-tier models are reliably available at any given time. The free-tier model landscape changes frequently, and a deployment that worked perfectly in testing can behave differently a month later if a model is deprecated or rate limits change. Configuration flexibility solves the technical problem of switching providers; it does not solve the operational problem of knowing which provider to switch to. That knowledge requires staying current with the model ecosystem, which is itself a discipline.

The broader principle here extends beyond AI providers.

Any external dependency that changes faster than your release cycle is a candidate for abstraction. AI models change faster than almost any other dependency in software today. Building the abstraction at the start, when it costs almost nothing, is significantly cheaper than building it after the application has hardcoded provider assumptions throughout the codebase.

I do not expect today's best model to remain the best model next year. Good architecture anticipates change rather than resisting it.