#atom

API Gateway Service for Large Language Models

Core Idea: OpenRouter is a unified API gateway that provides simplified access to multiple large language models (LLMs) through a standardized interface, enabling developers to switch between or combine different AI models without changing their integration code.

Key Elements

Platform Architecture

• Unified API Interface: Standardized endpoints compatible with OpenAI's format
• Model Routing: Intelligently directs requests to appropriate models
• Cost Management: Pay-per-use billing with potential cost savings over direct model access
• Multi-Provider Support: Single integration point for numerous AI providers

Supported Models

OpenRouter provides access to models from multiple providers, including:

• Anthropic: Claude 3 Opus, Claude 3 Sonnet, Claude 3 Haiku
• Google: Gemini 1.5 Pro, Gemini 1.5 Flash, Gemini 1.0 Pro
• OpenAI: GPT-4o, GPT-4 Turbo, GPT-3.5 Turbo
• Mistral AI: Mistral Large, Mistral Medium, Mistral Small
• Meta: Llama 3 70B, Llama 3 8B
• Various open-source models via hosted instances

Integration Methods

API Requests

// Example API request
const response = await fetch('https://openrouter.ai/api/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': 'Bearer ' + OPENROUTER_API_KEY,
    'HTTP-Referer': 'https://your-site.com', // Required for attribution
  },
  body: JSON.stringify({
    model: 'anthropic/claude-3-opus', // Specific model or 'openrouter/auto' for automatic selection
    messages: [
      { role: 'user', content: 'What are the key principles of good API design?' }
    ]
  })
});

SDK Integration

// Using the OpenRouter SDK
import { OpenRouter } from 'openrouter';

const openrouter = new OpenRouter({
  apiKey: 'your-api-key',
  referer: 'https://your-site.com'
});

const response = await openrouter.chat.completions.create({
  model: 'google/gemini-1.5-pro',
  messages: [
    { role: 'user', content: 'Explain quantum computing in simple terms' }
  ]
});

Key Features

• Model Fallbacks: Configure backup models if primary choice is unavailable
• Rate Limiting: Managed rate limits across providers
• Context Handling: Automatically manages context windows for different models
• Customizable Routing: Rules-based routing based on prompt characteristics
• Usage Analytics: Track consumption across different models and providers

Practical Applications

Multi-Model Applications

Developing applications that leverage multiple models for different use cases:

// Different models for different tasks
async function processRequest(type, content) {
  const modelMap = {
    'creative': 'anthropic/claude-3-opus',
    'factual': 'google/gemini-1.5-pro',
    'coding': 'openai/gpt-4o',
    'summarization': 'mistral/mistral-medium'
  };
  
  const model = modelMap[type] || 'openrouter/auto';
  
  return openrouter.chat.completions.create({
    model: model,
    messages: [{ role: 'user', content }]
  });
}

Cost Optimization

Implementing cost-effective model selection based on task complexity:

function selectModelByComplexity(prompt) {
  // Simple complexity heuristic based on prompt length
  const complexity = prompt.length > 500 ? 'high' : 'low';
  
  return complexity === 'high' 
    ? 'anthropic/claude-3-opus'  // More capable but expensive model
    : 'mistral/mistral-small';   // Less expensive model for simpler tasks
}

Hybrid AI Systems

Creating systems that combine different AI models' strengths:

async function enhancedResponse(userQuery) {
  // Generate initial response with creative model
  const initialResponse = await openrouter.chat.completions.create({
    model: 'anthropic/claude-3-opus',
    messages: [{ role: 'user', content: userQuery }]
  });
  
  // Fact-check with analytical model
  const factCheckPrompt = `Fact check this response to "${userQuery}": ${initialResponse.choices[0].message.content}`;
  const factCheck = await openrouter.chat.completions.create({
    model: 'google/gemini-1.5-pro',
    messages: [{ role: 'user', content: factCheckPrompt }]
  });
  
  return {
    response: initialResponse.choices[0].message.content,
    factCheck: factCheck.choices[0].message.content
  };
}

Additional Connections

• Broader Context: LLM API Integration Patterns (patterns for working with AI APIs)
• Applications: AI Model Selection Strategies (choosing the right model for different tasks)
• See Also: AI Cost Optimization (techniques for managing AI API expenses)

References

1. OpenRouter Documentation - https://openrouter.ai/docs
2. OpenRouter GitHub - https://github.com/openrouter-dev

#ai #llm #api-gateway #claude #gpt #gemini #mistral

Connections:

Sources:

• From: Worklog n8n.guiom.dev