Overview
LLM Gateway provides detailed usage analytics for every API request, including token usage, costs, latency, errors, and routing metadata. All data is stored and can be queried via the Analytics API.
What’s Tracked
Every request generates a comprehensive log entry:
apps/api/src/routes/logs.ts
interface LogEntry {
// Identifiers
id: string;
requestId: string;
organizationId: string;
projectId: string;
apiKeyId: string;
// Timestamps and Performance
createdAt: Date;
duration: number; // milliseconds
// Model Information
requestedModel: string;
requestedProvider: string | null;
usedModel: string;
usedProvider: string;
// Request/Response Data
messages: any; // Full conversation history
content: string | null; // Response content
reasoningContent: string | null; // Extended thinking
responseSize: number; // bytes
// Token Usage
promptTokens: string | null;
completionTokens: string | null;
totalTokens: string | null;
reasoningTokens: string | null;
imageInputTokens: string | null;
imageOutputTokens: string | null;
// Costs (USD)
cost: number | null;
inputCost: number | null;
outputCost: number | null;
requestCost: number | null;
imageInputCost: number | null;
imageOutputCost: number | null;
estimatedCost: boolean | null;
// Request Parameters
temperature: number | null;
maxTokens: number | null;
topP: number | null;
frequencyPenalty: number | null;
presencePenalty: number | null;
reasoningEffort: string | null;
reasoningMaxTokens: number | null;
responseFormat: any | null;
tools: any | null;
toolChoice: any | null;
toolResults: any | null;
// Status and Errors
finishReason: string | null;
unifiedFinishReason: string | null;
hasError: boolean | null;
errorDetails: any | null;
canceled: boolean | null;
// Flags
streamed: boolean | null;
cached: boolean | null;
// Metadata
customHeaders: any | null;
mode: "api-keys" | "credits" | "hybrid";
usedMode: "api-keys" | "credits";
source: string | null;
routingMetadata: any | null;
}
Querying Logs
Retrieve logs via the REST API:
curl https://api.llmgateway.io/logs \
-H "Authorization: Bearer YOUR_API_KEY" \
-G \
--data-urlencode "projectId=proj_abc123" \
--data-urlencode "startDate=2024-01-01" \
--data-urlencode "limit=50"
import requests
response = requests.get(
"https://api.llmgateway.io/logs",
headers={"Authorization": f"Bearer {api_key}"},
params={
"projectId": "proj_abc123",
"startDate": "2024-01-01",
"limit": 50
}
)
logs = response.json()["logs"]
for log in logs:
print(f"{log['usedModel']}: {log['totalTokens']} tokens, ${log['cost']}")
const response = await fetch(
'https://api.llmgateway.io/logs?' + new URLSearchParams({
projectId: 'proj_abc123',
startDate: '2024-01-01',
limit: '50'
}),
{
headers: {
'Authorization': `Bearer ${apiKey}`
}
}
);
const { logs } = await response.json();
logs.forEach(log => {
console.log(`${log.usedModel}: ${log.totalTokens} tokens, $${log.cost}`);
});
Query Parameters
Filter logs using these parameters:
apps/api/src/routes/logs.ts
const querySchema = z.object({
// Filtering
apiKeyId: z.string().optional(),
providerKeyId: z.string().optional(),
projectId: z.string().optional(),
orgId: z.string().optional(),
// Date Range
startDate: z.string().optional(), // ISO string
endDate: z.string().optional(),
// Model/Provider
model: z.string().optional(), // e.g., "gpt-4o"
provider: z.string().optional(), // e.g., "openai"
// Status
finishReason: z.string().optional(),
unifiedFinishReason: z.string().optional(),
// Source Tracking
source: z.string().optional(),
// Custom Headers
customHeaderKey: z.string().optional(),
customHeaderValue: z.string().optional(),
// Pagination
cursor: z.string().optional(),
limit: z.number().int().min(1).max(100).optional(), // default: 50
orderBy: z.enum(["createdAt_asc", "createdAt_desc"]).optional() // default: desc
});
Example Filters
Date Range
?startDate=2024-01-01&endDate=2024-01-31
Custom Header
?customHeaderKey=user-id&customHeaderValue=123
Logs use cursor-based pagination:
{
"logs": [...],
"pagination": {
"nextCursor": "log_xyz789",
"hasMore": true,
"limit": 50
}
}
Fetch the next page:
curl "https://api.llmgateway.io/logs?cursor=log_xyz789&limit=50"
Cost Tracking
LLM Gateway calculates costs for all requests:
apps/gateway/src/lib/costs.ts
export function calculateCosts(params: {
promptTokens: number;
completionTokens: number;
reasoningTokens?: number;
cachedTokens?: number;
imageInputCount?: number;
imageOutputCount?: number;
model: ModelDefinition;
provider: ProviderModelMapping;
}) {
const inputCost = (params.promptTokens / 1_000_000) * (provider.inputPrice ?? 0);
const outputCost = (params.completionTokens / 1_000_000) * (provider.outputPrice ?? 0);
const cachedInputCost = (params.cachedTokens ?? 0) / 1_000_000 * (provider.cachedInputPrice ?? 0);
const requestCost = provider.requestPrice ?? 0;
return {
inputCost,
outputCost,
cachedInputCost,
requestCost,
totalCost: inputCost + outputCost + cachedInputCost + requestCost
};
}
Cost Breakdown
{
"usage": {
"prompt_tokens": 150,
"completion_tokens": 50,
"total_tokens": 200,
"reasoning_tokens": 1200,
"cost_usd_total": 0.00045,
"cost_usd_input": 0.00015,
"cost_usd_output": 0.0003,
"cost_usd_cached_input": 0.0,
"cost_usd_request": 0.0
}
}
Costs are calculated using real-time pricing from each provider’s documentation.
Source Tracking
Track where requests originate:
curl https://api.llmgateway.io/v1/chat/completions \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "x-source: my-app/v1.0" \
-d '{...}'
Query by source:
curl "https://api.llmgateway.io/logs?source=my-app/v1.0"
Add custom metadata to requests:
curl https://api.llmgateway.io/v1/chat/completions \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "x-llmgateway-user-id: user_123" \
-H "x-llmgateway-session-id: sess_456" \
-d '{...}'
Query by custom headers:
curl "https://api.llmgateway.io/logs?customHeaderKey=user-id&customHeaderValue=user_123"
Unique Models & Providers
Get a list of all models and providers you’ve used:
curl "https://api.llmgateway.io/logs/unique-models?projectId=proj_abc123"
Response:
{
"models": [
"gpt-4o",
"gpt-4o-mini",
"claude-3-5-sonnet-20241022",
"gemini-2.0-flash-001"
],
"providers": [
"openai",
"anthropic",
"google-ai-studio"
]
}
Data Retention
Log retention depends on your plan:
- Free Plan: 3 days
- Pro Plan: 90 days
- Enterprise Plan: Custom retention
Data storage costs $0.01 per 1M tokens. This is automatically deducted from your credits when retention is enabled.
Privacy and Security
Control what data is stored:
// Set retention level in organization settings
organization.retentionLevel = "none" | "metadata" | "retain";
- none - Don’t store request/response data
- metadata - Store metadata only (no message content)
- retain - Store full request/response data
Fetch Individual Logs
Get a specific log by ID:
curl "https://api.llmgateway.io/logs/log_abc123" \
-H "Authorization: Bearer YOUR_API_KEY"
Response:
{
"log": {
"id": "log_abc123",
"requestId": "req_xyz789",
"usedModel": "openai/gpt-4o",
"usedProvider": "openai",
"totalTokens": "200",
"cost": 0.00045,
"duration": 1234,
"messages": [...],
"content": "The capital of France is Paris.",
"routingMetadata": {...}
}
}
Dashboard Analytics
View analytics in the web dashboard:
- Usage over time - Token and request trends
- Cost breakdown - By model, provider, and project
- Latency metrics - P50, P95, P99 latencies
- Error rates - Track failed requests
- Top models - Most used models
- Top users - By API key
Analytics data is updated in real-time. Refresh the dashboard to see the latest metrics.
Export Analytics
Export logs to CSV or JSON:
# Fetch all logs and save to file
curl "https://api.llmgateway.io/logs?projectId=proj_abc123&limit=100" \
-H "Authorization: Bearer YOUR_API_KEY" \
> logs.json