Models
ADK-TS provides a provider-agnostic interface for working with Large Language Models (LLMs), enabling seamless integration with multiple providers including OpenAI, Google (Gemini/Vertex AI), Anthropic, and more.
Architecture
BaseLlm Interface
All LLM providers extend from BaseLlm, which defines a unified interface:
import { BaseLlm } from "@iqai/adk";
import type { LlmRequest, LlmResponse } from "@iqai/adk";
export abstract class BaseLlm {
/**
* The name of the LLM (e.g., "gpt-4o" or "gemini-2.5-flash")
*/
model: string;
/**
* Returns list of supported models in regex for LLMRegistry
*/
static supportedModels(): string[];
/**
* Generates content from the given request.
* Yields one or more LlmResponse objects.
*/
async *generateContentAsync(
llmRequest: LlmRequest,
stream?: boolean,
): AsyncGenerator<LlmResponse, void, unknown>;
}
Supported Providers
OpenAI
Supports GPT-4, GPT-4o, GPT-3.5-turbo, and all OpenAI models:
import { LlmAgent } from "@iqai/adk";
// Using model string (automatically resolved)
const agent = new LlmAgent({
name: "openai_agent",
model: "gpt-4o",
instruction: "You are a helpful assistant",
});
// Supported models:
// - gpt-4o, gpt-4o-mini
// - gpt-4, gpt-4-turbo
// - gpt-3.5-turbo
// - o1-preview, o1-mini
Set the OPENAI_API_KEY environment variable to use OpenAI models.
Google (Gemini & Vertex AI)
Supports Gemini and Vertex AI models:
// Gemini models
const geminiAgent = new LlmAgent({
name: "gemini_agent",
model: "gemini-2.5-flash",
});
// Supported Gemini models:
// - gemini-2.5-flash, gemini-2.5-pro
// - gemini-2.0-flash
// - gemini-1.5-pro, gemini-1.5-flash
// Vertex AI models
const vertexAgent = new LlmAgent({
name: "vertex_agent",
model: "gemini-1.5-pro",
});
Set the GOOGLE_API_KEY environment variable for Gemini, or configure Google Cloud credentials for Vertex AI.
Anthropic
Supports Claude models:
const claudeAgent = new LlmAgent({
name: "claude_agent",
model: "claude-3-5-sonnet",
});
// Supported models:
// - claude-3-5-sonnet
// - claude-3-opus, claude-3-sonnet, claude-3-haiku
Set the ANTHROPIC_API_KEY environment variable to use Claude models.
Vercel AI SDK
Integrate with any provider supported by the Vercel AI SDK:
import { openai } from "@ai-sdk/openai";
import { LlmAgent } from "@iqai/adk";
const agent = new LlmAgent({
name: "ai_sdk_agent",
model: openai("gpt-4o"),
});
LLM Registry
The LLMRegistry automatically resolves model strings to provider instances:
import { LLMRegistry } from "@iqai/adk";
// Register a custom provider
class CustomLlm extends BaseLlm {
static override supportedModels(): string[] {
return ["custom-.*"]; // Regex pattern
}
protected async *generateContentAsyncImpl(
llmRequest: LlmRequest,
stream?: boolean,
): AsyncGenerator<LlmResponse, void, unknown> {
// Implementation
}
}
// Register the provider
LLMRegistry.register(CustomLlm);
// Now you can use it
const agent = new LlmAgent({
name: "my_agent",
model: "custom-model-v1", // Automatically uses CustomLlm
});
LlmRequest
LlmRequest is the standardized input format for all LLM providers:
import { LlmRequest } from "@iqai/adk";
const request = new LlmRequest({
model: "gpt-4o",
contents: [
{
role: "user",
parts: [{ text: "Hello, how are you?" }],
},
],
config: {
temperature: 0.7,
maxOutputTokens: 1000,
topP: 0.9,
},
toolsDict: {
search: searchTool,
calculate: calculatorTool,
},
});
// Append instructions
request.appendInstructions([
"Be concise and accurate",
"Use tools when appropriate",
]);
// Append tools
request.appendTools([newTool]);
Request Configuration
interface GenerateContentConfig {
/** System instruction for the model */
systemInstruction?: string;
/** Temperature (0.0 to 2.0) */
temperature?: number;
/** Maximum tokens to generate */
maxOutputTokens?: number;
/** Top-p sampling */
topP?: number;
/** Top-k sampling */
topK?: number;
/** Response format (for structured output) */
responseFormat?: {
type: "json_object" | "text";
};
/** Stop sequences */
stopSequences?: string[];
}
LlmResponse
LlmResponse is the standardized output format:
import { LlmResponse } from "@iqai/adk";
interface LlmResponse {
/** Response content with parts */
content?: {
role: string;
parts: Part[];
};
/** Text content (extracted from parts) */
text?: string;
/** Whether response is partial (streaming) */
partial?: boolean;
/** Usage metadata */
usageMetadata?: {
promptTokenCount: number;
candidatesTokenCount: number;
totalTokenCount: number;
};
}
Streaming
All providers support streaming responses:
import { Runner } from "@iqai/adk";
const runner = new Runner({
appName: "my-app",
agent,
sessionService,
});
for await (const event of runner.runAsync({
userId: "user-123",
sessionId: session.id,
newMessage: { parts: [{ text: "Tell me a story" }] },
})) {
// Events stream in real-time
if (event.text) {
process.stdout.write(event.text);
}
}
Streaming is enabled by default. The framework handles streaming differences across providers automatically.
Function Calling
ADK-TS normalizes function calling across providers:
import { BaseTool, FunctionDeclaration } from "@iqai/adk";
class WeatherTool extends BaseTool {
constructor() {
super({
name: "get_weather",
description: "Get current weather for a location",
});
}
getDeclaration(): FunctionDeclaration {
return {
name: this.name,
description: this.description,
parameters: {
type: "object",
properties: {
location: {
type: "string",
description: "City name or coordinates",
},
},
required: ["location"],
},
};
}
async runAsync(args: Record<string, any>): Promise<any> {
// Implementation
return { temperature: 72, condition: "sunny" };
}
}
Function calling is automatically translated to each provider’s format (OpenAI tools, Gemini function calling, Claude tool use, etc.).
Model Configuration
Configure model behavior at the agent level:
const agent = new LlmAgent({
name: "configured_agent",
model: "gpt-4o",
config: {
temperature: 0.3,
maxOutputTokens: 2000,
topP: 0.95,
stopSequences: ["END"],
},
});
Context Caching
Optimize costs with context caching (supported by select providers):
import { ContextCacheConfig } from "@iqai/adk";
const agent = new LlmAgent({
name: "cached_agent",
model: "gemini-2.5-flash",
cacheConfig: new ContextCacheConfig({
ttlSeconds: 3600, // Cache for 1 hour
}),
});
Error Handling
Handle provider-specific errors gracefully:
try {
for await (const event of runner.runAsync(params)) {
// Process event
}
} catch (error) {
if (error.message.includes("rate limit")) {
// Handle rate limiting
} else if (error.message.includes("invalid_api_key")) {
// Handle authentication errors
} else {
// Handle other errors
}
}
Model Fallbacks
Implement automatic failover with the ModelFallbackPlugin:
import { ModelFallbackPlugin } from "@iqai/adk";
const agent = new LlmAgent({
name: "resilient_agent",
model: "gpt-4o",
plugins: [
new ModelFallbackPlugin({
fallbackModels: ["gpt-4o-mini", "gemini-2.5-flash"],
}),
],
});
Best Practices
-
Use Model Strings: Prefer model string identifiers over direct provider instances for flexibility.
-
Set Appropriate Temperatures: Use lower temperatures (0.1-0.3) for factual tasks, higher (0.7-1.0) for creative tasks.
-
Monitor Token Usage: Track
usageMetadata to optimize costs and performance.
-
Implement Fallbacks: Use multiple models for production resilience.
-
Leverage Context Caching: Reduce costs for repetitive system instructions and large contexts.
- Agents - Create and configure agents
- Tools - Extend model capabilities with tools
- Flows - Understand the request processing pipeline