The gateway supports 8 LLM providers. You can configure multiple providers simultaneously and control which one handles each request through a three-step resolution process.

Provider configuration

Define providers in the providers: section of rivano.yaml:

gateway:
  providers:
    openai:
      apiKey: "${OPENAI_API_KEY}"
      isPrimary: true
    anthropic:
      apiKey: "${ANTHROPIC_API_KEY}"
      isFallback: true
    google:
      apiKey: "${GOOGLE_API_KEY}"
    ollama:
      baseUrl: "http://localhost:11434"

Supported providers

Provider key	Default base URL	Auth header	Notes
`openai`	`https://api.openai.com/v1`	`Authorization: Bearer`
`anthropic`	`https://api.anthropic.com`	`x-api-key`	Uses Anthropic’s own header format
`google`	`https://generativelanguage.googleapis.com`	`Authorization: Bearer`	Gemini models
`azure`	—	`Authorization: Bearer`	Requires `baseUrl` pointing to your deployment
`bedrock`	—	`Authorization: Bearer`	Requires `baseUrl` with AWS endpoint
`mistral`	`https://api.mistral.ai/v1`	`Authorization: Bearer`
`cohere`	`https://api.cohere.com/v1`	`Authorization: Bearer`
`ollama`	`http://localhost:11434`	—	No auth required; set `baseUrl` for remote Ollama

Provider resolution order

When a request arrives, the gateway selects the target provider using this three-step process:

1. X-Rivano-Provider header

Set the X-Rivano-Provider request header to route to a specific provider by name:

curl http://localhost:8080/openai/v1/chat/completions \
  -H "Authorization: Bearer sk-..." \
  -H "X-Rivano-Provider: anthropic" \
  -H "Content-Type: application/json" \
  -d '{"model": "claude-3-5-sonnet-20241022", "messages": [...]}'

2. URL path prefix

Use a provider-specific path prefix. The gateway strips the prefix before forwarding to the provider:

# Routes to Anthropic
curl http://localhost:8080/anthropic/v1/messages \
  -H "x-api-key: sk-ant-..." \
  -d '...'

# Routes to Google
curl http://localhost:8080/google/v1/models/gemini-pro:generateContent \
  -H "Authorization: Bearer ..." \
  -d '...'

Path prefix	Routes to
`/openai/v1/...`	OpenAI
`/anthropic/v1/...`	Anthropic
`/google/v1/...`	Google
`/azure/v1/...`	Azure OpenAI
`/bedrock/v1/...`	Amazon Bedrock
`/mistral/v1/...`	Mistral
`/cohere/v1/...`	Cohere
`/ollama/v1/...`	Ollama

3. Default (primary) provider

If neither a header nor a path prefix is present, the request goes to the provider marked isPrimary: true. If no provider is marked primary, the first configured provider is used.

Auth header mapping

The gateway translates your application’s auth header to the format the target provider expects. You do not need to change your application’s auth header when routing between providers.

Provider	Expects	Gateway behavior
Anthropic	`x-api-key: sk-ant-...`	Uses the `apiKey` from `rivano.yaml`, not the incoming header
All others	`Authorization: Bearer sk-...`	Uses the `apiKey` from `rivano.yaml`

💡

Your application only needs to authenticate with the gateway (or not at all in development). The gateway substitutes its own configured API keys when forwarding to providers — your provider keys never need to be distributed to application instances.

Failover

Mark one provider as isFallback: true to enable automatic failover:

gateway:
  providers:
    openai:
      apiKey: "${OPENAI_API_KEY}"
      isPrimary: true
    anthropic:
      apiKey: "${ANTHROPIC_API_KEY}"
      isFallback: true
      baseUrl: "https://api.anthropic.com"

When the primary provider returns a 5xx error or times out, the gateway retries the request against the fallback provider. The failover is transparent — your application sees a successful response.

Azure OpenAI

Azure requires a deployment-specific base URL:

gateway:
  providers:
    azure:
      apiKey: "${AZURE_OPENAI_API_KEY}"
      baseUrl: "https://your-resource.openai.azure.com/openai/deployments/gpt-4o"

Ollama (local models)

Point baseUrl at your Ollama instance. No apiKey is needed:

gateway:
  providers:
    ollama:
      baseUrl: "http://localhost:11434"

Then route requests via the path prefix:

curl http://localhost:8080/ollama/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "llama3.2", "messages": [...]}'

Gateway Configuration — Full provider config field reference
Gateway Middleware — Middleware that runs before the provider is called
SDK Providers — Manage provider config via the API

Gateway Providers