Understanding six concepts gives you the full mental model for working with Rivano. Everything else — SDK methods, CLI commands, gateway config — maps back to these primitives.

Agents

An agent is a named, versioned AI workload. It represents a distinct use case in your system: contract-summarizer, support-classifier, code-reviewer.

Agents are defined in rivano.yaml and deployed with rivano deploy. Every deployment creates a new version, so you can roll back instantly. When a request passes through the gateway tagged with an agent name, all traces, costs, and policy evaluations are attributed to that agent.

agents:
  - name: contract-summarizer
    description: Summarizes legal contracts
    modelProvider: openai
    modelName: gpt-4o
    environment: production

An agent is not running code — it is a logical grouping. The same application code can produce requests attributed to different agents based on the X-Rivano-Agent header you pass to the gateway.

Policies

A policy is a declarative rule that fires on every proxied request or response. Policies have three parts:

Phase — request (before the LLM sees the prompt) or response (after the LLM replies)
Condition — what to check: injection_score, pii_detected, token_count, or model_name
Action — what to do: block (reject the request), redact (scrub the content), or warn (log and continue)

Policies are evaluated in order. A block action short-circuits the pipeline — the LLM is never called, and a 400 response is returned to your application.

# Example: block requests with high injection risk
name: block-injection
onPhase: request
conditionType: injection_score
conditionConfig:
  threshold: 0.7
action: block

💡

Use rivano policies apply-template foundational to apply a curated starter pack: block injection, redact PII from responses, warn on large prompts, and block SSN patterns.

Traces

A trace is a complete record of one proxied request. It contains:

Spans — individual steps in the pipeline (policy evaluation, provider call, response processing)
Metadata — agent name, session ID, environment, timestamp
Metrics — latency, token counts (input/output), estimated cost
Scores — quality evaluations run post-response

Traces are queryable by agent, status, environment, session, and time range. They are the primary unit of observability in Rivano.

Trace
├── span: policy/block-injection (pass, 1ms)
├── span: provider/openai (200 OK, 312ms)
│   ├── model: gpt-4o
│   ├── input_tokens: 142
│   └── output_tokens: 87
└── span: policy/redact-pii-response (pass, 2ms)

Tenants

A tenant is a fully isolated organization within Rivano. All data — agents, traces, policies, API keys, billing — is scoped to a tenant. Row-level security in the database prevents any data from crossing tenant boundaries.

When you sign up for Rivano, you create a tenant. Multi-tenant support is available for building AI SaaS products.

Teams

A team is a group of users within a tenant. Teams let you:

Scope agent access — a team can see only the agents assigned to it
Assign roles — admin, member, or custom roles
Organize large engineering orgs — one team per product area, one team per customer

A user can belong to multiple teams. Teams are created and managed via the SDK, CLI, or dashboard.

Providers

A provider is an LLM backend. Rivano supports routing to any of these providers:

Provider	Path prefix	Auth header
OpenAI	`/openai/v1`	`Authorization: Bearer`
Anthropic	`/anthropic/v1`	`x-api-key`
Google (Gemini)	`/google/v1`	`Authorization: Bearer`
Azure OpenAI	`/azure/v1`	`Authorization: Bearer`
Amazon Bedrock	`/bedrock/v1`	`Authorization: Bearer`
Mistral	`/mistral/v1`	`Authorization: Bearer`
Cohere	`/cohere/v1`	`Authorization: Bearer`
Ollama (local)	`/ollama/v1`	—

You configure providers in rivano.yaml or via the dashboard. Mark one provider as isPrimary to receive traffic when no provider hint is specified. Mark others as isFallback for automatic failover.

gateway:
  providers:
    openai:
      apiKey: "sk-..."
    anthropic:
      apiKey: "sk-ant-..."

How it all fits together

rivano.yaml         ──► rivano deploy  ──► Control Plane (agents, policies)
                                                 │
Your App ──► Rivano Gateway  ────────────────────┘
               │    ▲
               │    └── evaluates policies
               ▼
          LLM Provider
               │
               └──► Trace recorded ──► Dashboard / SDK / CLI

Your application talks to the gateway. The gateway loads policies from the control plane, evaluates each request, calls the LLM provider, records a trace, and returns the response. All of this is invisible to your application — it sees a normal LLM API response.

Quickstart — See these concepts in action
SDK Agents — Manage agents programmatically
SDK Policies — Create and update policies
SDK Traces — Query trace data
Gateway Providers — Configure provider routing

Core Concepts