AnythingLLM with Claude, GPT and Gemini

Connect in three fields

AnythingLLM has a Generic OpenAI provider for any OpenAI-compatible endpoint. Point it at avots.

Setup

LLM Provider:         Generic OpenAI
Base URL:             https://api.avots.ai/openai/v1
API Key:              av_mcp_YOUR_KEY
Chat Model Name:      anthropic/claude-sonnet-4.6
Token context window: 200000
Max Tokens:           4096

Generic OpenAI has no model auto-discovery, type the exact Chat Model Name by hand. Set the Token context window and Max Tokens correctly, wrong values silently truncate or error. This configures the chat LLM only, embeddings are a separate provider.

Example request

curl https://api.avots.ai/openai/v1/chat/completions \
  -H "Authorization: Bearer av_mcp_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-4.6",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

The same key also unlocks MCP tools

Your av_mcp_ key is not only for chat. avots also runs as an MCP server, so AnythingLLM can call avots creative tools, image, video, music, talking avatars, on the same key and the same balance.

MCP setup guide →

Models and prices

Set the model field to any id below, or a lazy name like claude / gpt / gemini that maps to a sensible default.

* Price in ⚡ per 1,000,000 tokens, input / output (1000⚡ ≈ $1). Free models need a positive balance. Full list via GET /openai/v1/models.

Questions

Other clients