Skip to main content
View as Markdown

AI Providers

The AI provider is the LLM (and embedding) backend that powers every AI capability. You select it with environment variables — never in the app schema. Setting AI_PROVIDER is the master switch that activates the entire AI layer; leaving it unset keeps AI dormant.

AI_PROVIDER=anthropic
AI_API_KEY=sk-ant-...
AI_MODEL=claude-sonnet-4-5

Supported Providers

Provider AI_PROVIDER value API key required Base URL required Notes
Anthropic anthropic Yes No Claude models. Default model: claude-sonnet-4-5.
OpenAI openai Yes No GPT / o-series models. Default model: gpt-4o.
Mistral mistral Yes No Mistral / Codestral. Default model: mistral-large-latest.
Google Gemini google (alias gemini) Yes No Gemini models. Default model: gemini-2.0-flash.
Ollama ollama No Yes Local, self-hosted. No key needed. Default model: llama3.1.
OpenAI-compatible openai-compatible Yes Yes Any endpoint that speaks the OpenAI API. Set AI_MODEL explicitly.

Core Environment Variables

Variable Description Example
AI_PROVIDER Provider identifier. Master switch — unset disables all AI. anthropic
AI_API_KEY API key for cloud providers. Not used by Ollama. sk-ant-...
AI_BASE_URL Endpoint URL. Required for Ollama and OpenAI-compatible; cloud providers use built-in defaults. http://localhost:11434
AI_MODEL Default LLM model. Falls back to the provider's recommended default when unset. claude-sonnet-4-5
AI_TEMPERATURE Default sampling temperature, 0–1 inclusive. 0.7
AI_MAX_TOKENS Default maximum output tokens (positive integer). 4096
AI_EMBEDDING_MODEL Embedding model for RAG. Defaults to the provider's embedding model. text-embedding-3-small
AI_EMBEDDING_DIMENSIONS Vector dimensions (must match the embedding model's output). 1536

An empty or whitespace-only value is treated identically to unset — operators who blank out AI_PROVIDER intend to disable it, not supply an invalid value. The server boots cleanly with AI disabled rather than throwing a parse error.

Provider-Specific Key & URL Aliases

In addition to the generic AI_API_KEY / AI_BASE_URL, each cloud provider accepts a conventional alias env var. The generic AI_* value always takes precedence; the alias is the fallback.

Provider API-key alias Base-URL alias
Anthropic ANTHROPIC_API_KEY —
OpenAI OPENAI_API_KEY —
Mistral MISTRAL_API_KEY —
Google Gemini GOOGLE_API_KEY —
Ollama — (no key) OLLAMA_BASE_URL
OpenAI-compatible (uses AI_API_KEY) (uses AI_BASE_URL)

Default Models

When AI_MODEL is unset, each provider resolves to a sensible default:

Provider Default model
Anthropic claude-sonnet-4-5
OpenAI gpt-4o
Mistral mistral-large-latest
Google Gemini gemini-2.0-flash
Ollama llama3.1
OpenAI-compatible none — set AI_MODEL

OpenAI-compatible endpoints point at an arbitrary backend, so there is no universal default model — you must set AI_MODEL explicitly.

Model Validation

For providers with a known catalogue (Anthropic, OpenAI, Mistral, Google), Sovrium emits a non-fatal startup warning when AI_MODEL (or an agent's model override) does not match a recognized model — typically a cross-provider mix-up (gpt-4o on anthropic) or a typo (claud-sonet). This is a warning, not an error; the server still starts.

Ollama and OpenAI-compatible providers have open-ended catalogues (locally installed models, arbitrary endpoints), so model-name validation is skipped entirely for them.

Configuration Examples

# Self-hosted, zero-cost, frugal-by-default: local Ollama.
AI_PROVIDER=ollama
AI_BASE_URL=http://localhost:11434     # or OLLAMA_BASE_URL
AI_MODEL=llama3.1
AI_EMBEDDING_MODEL=nomic-embed-text
# Anthropic with a temperature override.
AI_PROVIDER=anthropic
ANTHROPIC_API_KEY=sk-ant-...           # or AI_API_KEY
AI_MODEL=claude-sonnet-4-5
AI_TEMPERATURE=0.3
AI_MAX_TOKENS=4096
# An OpenAI-compatible gateway (vLLM, LiteLLM, Azure OpenAI, etc.).
AI_PROVIDER=openai-compatible
AI_BASE_URL=https://my-gateway.internal/v1
AI_API_KEY=...
AI_MODEL=my-deployed-model             # required — no default