AI Endpoint Configuration

LibreChat supports multiple AI providers and custom endpoints. Configure them using environment variables and the librechat.yaml file.

Endpoint Selection

ENDPOINTS

string

Comma-separated list of enabled endpoints

ENDPOINTS=openAI,assistants,azureOpenAI,google,anthropic

OpenAI

OPENAI_API_KEY

string

required

OpenAI API key. Set to user_provided to require users to provide their own keys.

OPENAI_API_KEY=sk-xxx
# Or require user-provided keys:
OPENAI_API_KEY=user_provided

OPENAI_MODELS

string

Comma-separated list of available OpenAI models

OPENAI_MODELS=gpt-5,gpt-5-codex,gpt-5-mini,gpt-5-nano,o3-pro,o3,o4-mini,gpt-4.1,gpt-4.1-mini,gpt-4.1-nano,o3-mini,o1-pro,o1,gpt-4o,gpt-4o-mini

DEBUG_OPENAI

boolean

default:"false"

Enable OpenAI debug logging

DEBUG_OPENAI=false

TITLE_CONVO

boolean

default:"true"

Enable automatic conversation title generation

TITLE_CONVO=false

OPENAI_TITLE_MODEL

string

default:"gpt-4o-mini"

Model to use for conversation title generation

OPENAI_TITLE_MODEL=gpt-4o-mini

OPENAI_SUMMARIZE

boolean

default:"false"

Enable conversation summarization

OPENAI_SUMMARIZE=true

OPENAI_SUMMARY_MODEL

string

default:"gpt-4o-mini"

Model to use for summarization

OPENAI_SUMMARY_MODEL=gpt-4o-mini

OPENAI_FORCE_PROMPT

boolean

default:"false"

Force system prompt inclusion

OPENAI_FORCE_PROMPT=true

OPENAI_REVERSE_PROXY

string

Reverse proxy URL for OpenAI API

OPENAI_REVERSE_PROXY=https://your-proxy.com/v1

OPENAI_ORGANIZATION

string

OpenAI organization ID

OPENAI_ORGANIZATION=org-xxx

Assistants API

ASSISTANTS_API_KEY

string

API key for OpenAI Assistants

ASSISTANTS_API_KEY=user_provided
# Or use specific key:
ASSISTANTS_API_KEY=sk-xxx

ASSISTANTS_BASE_URL

string

Custom base URL for Assistants API

ASSISTANTS_BASE_URL=https://api.openai.com/v1

ASSISTANTS_MODELS

string

Comma-separated list of models available for Assistants

ASSISTANTS_MODELS=gpt-4o,gpt-4o-mini,gpt-3.5-turbo-0125

Assistants Configuration (YAML)

librechat.yaml

endpoints:
  assistants:
    disableBuilder: false         # Disable Assistants Builder Interface
    pollIntervalMs: 3000          # Polling interval for updates
    timeoutMs: 180000             # Timeout for operations
    
    # Filter assistants by ID
    supportedIds:
      - "asst_supportedAssistantId1"
      - "asst_supportedAssistantId2"
    # OR exclude specific assistants
    # excludedIds:
    #   - "asst_excludedAssistantId"
    
    # Only show user-created assistants
    # privateAssistants: false
    
    # Models that support retrieval
    retrievalModels:
      - "gpt-4-turbo-preview"
    
    # Available capabilities
    capabilities:
      - "code_interpreter"
      - "retrieval"
      - "actions"
      - "tools"
      - "image_vision"

Anthropic

ANTHROPIC_API_KEY

string

Anthropic API key

ANTHROPIC_API_KEY=user_provided
# Or use specific key:
ANTHROPIC_API_KEY=sk-ant-xxx

ANTHROPIC_MODELS

string

Comma-separated list of available Anthropic models

ANTHROPIC_MODELS=claude-sonnet-4-6,claude-opus-4-6,claude-opus-4-20250514,claude-sonnet-4-20250514,claude-3-7-sonnet-20250219,claude-3-5-sonnet-20241022,claude-3-5-haiku-20241022

ANTHROPIC_REVERSE_PROXY

string

Reverse proxy URL for Anthropic API

ANTHROPIC_REVERSE_PROXY=https://your-proxy.com/v1

Anthropic via Google Vertex AI

ANTHROPIC_USE_VERTEX

boolean

default:"false"

Use Anthropic models through Google Vertex AI

ANTHROPIC_USE_VERTEX=true

ANTHROPIC_VERTEX_REGION

string

default:"us-east5"

Vertex AI region for Anthropic models

ANTHROPIC_VERTEX_REGION=us-east5

Anthropic Configuration (YAML)

librechat.yaml

endpoints:
  anthropic:
    streamRate: 20                # Stream rate limiting (ms)
    titleModel: claude-3.5-haiku  # Model for titles
    
    # Vertex AI Configuration
    vertex:
      region: "us-east5"          # Vertex region
      # serviceKeyFile: "/path/to/service-account.json"
      # projectId: "${VERTEX_PROJECT_ID}"
      
      # Model mappings
      models:
        claude-opus-4.5:
          deploymentName: claude-opus-4-5@20251101
        claude-sonnet-4:
          deploymentName: claude-sonnet-4-20250514
        claude-3.7-sonnet:
          deploymentName: claude-3-7-sonnet-20250219

Google

GOOGLE_KEY

string

Google API key (for Gemini API / AI Studio)

GOOGLE_KEY=user_provided
# Or use specific key:
GOOGLE_KEY=AIzaSyXXX

GOOGLE_REVERSE_PROXY

string

Reverse proxy URL for Google API

GOOGLE_REVERSE_PROXY=https://your-proxy.com/v1

GOOGLE_AUTH_HEADER

boolean

default:"false"

Pass API key in Authorization header instead of X-goog-api-key

Use this if your reverse proxy doesn’t support X-goog-api-key header

GOOGLE_AUTH_HEADER=true

GOOGLE_MODELS

string

Comma-separated list of available Google models

# Gemini API (AI Studio):
GOOGLE_MODELS=gemini-3.1-pro-preview,gemini-3.1-pro-preview-customtools,gemini-2.5-pro,gemini-2.5-flash,gemini-2.5-flash-lite,gemini-2.0-flash,gemini-2.0-flash-lite

# Vertex AI:
GOOGLE_MODELS=gemini-3.1-pro-preview,gemini-2.5-pro,gemini-2.5-flash,gemini-2.0-flash-001,gemini-2.0-flash-lite-001

GOOGLE_TITLE_MODEL

string

default:"gemini-2.0-flash-lite-001"

Model to use for title generation

GOOGLE_TITLE_MODEL=gemini-2.0-flash-lite-001

Vertex AI Configuration

GOOGLE_LOC

string

default:"us-central1"

Google Cloud region for Vertex AI

GOOGLE_LOC=us-central1

GOOGLE_CLOUD_LOCATION

string

default:"global"

Alternative region for Gemini Image Generation

GOOGLE_CLOUD_LOCATION=global

GOOGLE_SERVICE_KEY_FILE

string

Path to Google Cloud service account JSON file

GOOGLE_SERVICE_KEY_FILE=/path/to/service-account.json

Google Safety Settings

GOOGLE_SAFETY_*

string

Safety settings for both Vertex AI and Gemini APIOptions: BLOCK_NONE, BLOCK_ONLY_HIGH, BLOCK_MEDIUM_AND_ABOVE, BLOCK_LOW_AND_ABOVE

For Vertex AI: BLOCK_NONE requires either allowlist access or monthly invoiced billingFor Gemini API: BLOCK_NONE is available by default

GOOGLE_SAFETY_SEXUALLY_EXPLICIT=BLOCK_ONLY_HIGH
GOOGLE_SAFETY_HATE_SPEECH=BLOCK_ONLY_HIGH
GOOGLE_SAFETY_HARASSMENT=BLOCK_ONLY_HIGH
GOOGLE_SAFETY_DANGEROUS_CONTENT=BLOCK_ONLY_HIGH
GOOGLE_SAFETY_CIVIC_INTEGRITY=BLOCK_ONLY_HIGH

Gemini Image Generation

GEMINI_API_KEY

string

Dedicated Gemini API key for image generation (falls back to GOOGLE_KEY)

GEMINI_API_KEY=AIzaSyXXX

GEMINI_IMAGE_MODEL

string

default:"gemini-2.5-flash-image"

Vertex AI model for image generation

GEMINI_IMAGE_MODEL=gemini-2.5-flash-image

AWS Bedrock

BEDROCK_AWS_DEFAULT_REGION

string

required

Default AWS region for Bedrock

BEDROCK_AWS_DEFAULT_REGION=us-east-1

BEDROCK_AWS_ACCESS_KEY_ID

string

AWS access key ID

BEDROCK_AWS_ACCESS_KEY_ID=AKIAXXXXX

BEDROCK_AWS_SECRET_ACCESS_KEY

string

AWS secret access key

BEDROCK_AWS_SECRET_ACCESS_KEY=xxxxx

BEDROCK_AWS_SESSION_TOKEN

string

AWS session token (for temporary credentials)

BEDROCK_AWS_SESSION_TOKEN=xxxxx

BEDROCK_AWS_MODELS

string

Comma-separated list of Bedrock model IDs

If omitted, all supported model IDs will be included

BEDROCK_AWS_MODELS=anthropic.claude-sonnet-4-6,anthropic.claude-opus-4-6-v1,anthropic.claude-3-5-sonnet-20240620-v1:0,meta.llama3-1-8b-instruct-v1:0

# Cross-region inference:
BEDROCK_AWS_MODELS=us.anthropic.claude-sonnet-4-6,global.anthropic.claude-opus-4-6-v1

Bedrock Configuration (YAML)

librechat.yaml

endpoints:
  bedrock:
    # Model configuration
    models:
      - "anthropic.claude-3-7-sonnet-20250219-v1:0"
      - "anthropic.claude-3-5-sonnet-20241022-v2:0"
    
    # Inference profiles
    inferenceProfiles:
      "us.anthropic.claude-sonnet-4-20250514-v1:0": "${BEDROCK_INFERENCE_PROFILE_CLAUDE_SONNET}"
      "anthropic.claude-3-7-sonnet-20250219-v1:0": "arn:aws:bedrock:us-west-2:123456789012:application-inference-profile/abc123"
    
    # Guardrails
    guardrailConfig:
      guardrailIdentifier: "your-guardrail-id"
      guardrailVersion: "1"
      trace: "enabled"  # enabled, enabled_full, or disabled

Unsupported Bedrock Models:

ai21.j2-mid-v1 (no streaming support)
ai21.j2-ultra-v1, cohere.command-text-v14, cohere.command-light-text-v14 (no conversation history support)

Azure OpenAI

Azure OpenAI environment variables are DEPRECATED. Use librechat.yaml configuration instead.

librechat.yaml

endpoints:
  azureOpenAI:
    # Configuration goes here
    # See librechat.yaml documentation

Custom Endpoints

Define custom OpenAI-compatible endpoints in librechat.yaml:

Groq Example

librechat.yaml

endpoints:
  custom:
    - name: 'groq'
      apiKey: '${GROQ_API_KEY}'
      baseURL: 'https://api.groq.com/openai/v1/'
      models:
        default:
          - 'llama3-70b-8192'
          - 'llama3-8b-8192'
          - 'mixtral-8x7b-32768'
          - 'gemma-7b-it'
        fetch: false
      titleConvo: true
      titleModel: 'mixtral-8x7b-32768'
      modelDisplayLabel: 'groq'

Mistral AI Example

librechat.yaml

endpoints:
  custom:
    - name: 'Mistral'
      apiKey: '${MISTRAL_API_KEY}'
      baseURL: 'https://api.mistral.ai/v1'
      models:
        default: ['mistral-tiny', 'mistral-small', 'mistral-medium']
        fetch: true
      titleConvo: true
      titleModel: 'mistral-tiny'
      modelDisplayLabel: 'Mistral'
      dropParams: ['stop', 'user', 'frequency_penalty', 'presence_penalty']

OpenRouter Example

librechat.yaml

endpoints:
  custom:
    - name: 'OpenRouter'
      apiKey: '${OPENROUTER_KEY}'
      baseURL: 'https://openrouter.ai/api/v1'
      headers:
        x-librechat-body-parentmessageid: '{{LIBRECHAT_BODY_PARENTMESSAGEID}}'
      models:
        default: ['meta-llama/llama-3-70b-instruct']
        fetch: true
      titleConvo: true
      titleModel: 'meta-llama/llama-3-70b-instruct'
      dropParams: ['stop']
      modelDisplayLabel: 'OpenRouter'

Known Endpoints (API Keys Only)

These endpoints can be configured with just API keys:

ANYSCALE_API_KEY=xxxxx

Agents Endpoint

librechat.yaml

endpoints:
  agents:
    recursionLimit: 50              # Default recursion depth
    maxRecursionLimit: 100          # Maximum recursion depth
    disableBuilder: false           # Disable builder interface
    maxCitations: 30                # Max total citations
    maxCitationsPerFile: 7          # Max citations per file
    minRelevanceScore: 0.45         # Minimum relevance threshold
    
    # Available capabilities
    capabilities:
      - "deferred_tools"
      - "execute_code"
      - "file_search"
      - "actions"
      - "tools"

Next Steps

librechat.yaml

Complete YAML configuration reference

Environment Variables

All environment variables

Authentication

Set up OAuth and SSO

File Storage

Configure file storage

Get Started

Core Features

Configuration

Deployment

User Guide

Administration

AI Endpoint Configuration

AI Endpoint Configuration

Endpoint Selection

OpenAI

Assistants API

Assistants Configuration (YAML)

Anthropic

Anthropic via Google Vertex AI

Anthropic Configuration (YAML)

Google

Vertex AI Configuration

Google Safety Settings

Gemini Image Generation

AWS Bedrock

Bedrock Configuration (YAML)

Azure OpenAI

Custom Endpoints

Groq Example

Mistral AI Example

OpenRouter Example

Known Endpoints (API Keys Only)

Agents Endpoint

Next Steps

librechat.yaml

Environment Variables

Authentication

File Storage

Build docs developers (and LLMs) love

Get Started

Core Features

Configuration

Deployment

User Guide

Administration

​AI Endpoint Configuration

​Endpoint Selection

​OpenAI

​Assistants API

​Assistants Configuration (YAML)

​Anthropic

​Anthropic via Google Vertex AI

​Anthropic Configuration (YAML)

​Google

​Vertex AI Configuration

​Google Safety Settings

​Gemini Image Generation

​AWS Bedrock

​Bedrock Configuration (YAML)

​Azure OpenAI

​Custom Endpoints

​Groq Example

​Mistral AI Example

​OpenRouter Example

​Known Endpoints (API Keys Only)

​Agents Endpoint

​Next Steps

librechat.yaml

Environment Variables

Authentication

File Storage

Build docs developers (and LLMs) love

AI Endpoint Configuration

Endpoint Selection

OpenAI

Assistants API

Assistants Configuration (YAML)

Anthropic

Anthropic via Google Vertex AI

Anthropic Configuration (YAML)

Google

Vertex AI Configuration

Google Safety Settings

Gemini Image Generation

AWS Bedrock

Bedrock Configuration (YAML)

Azure OpenAI

Custom Endpoints

Groq Example

Mistral AI Example

OpenRouter Example

Known Endpoints (API Keys Only)

Agents Endpoint

Next Steps