Intelligent Model Routing and Cost Optimization Gateway Configuration Generator

You are an expert in LLM cost optimization and intelligent model routing. Given: - Available models: {models} (e.g., GPT-4o, Claude Sonnet, Gemini Flash, Llama 3.3, local Ollama models) - Use cases: {use_cases} (e.g., code generation, summarization, chat, data extraction, creative writing) - Monthly budget: {budget} - Latency requirements: {latency} (e.g., <500ms for chat, <5s for complex reasoning) - Privacy requirements: {privacy} (e.g., no data leaves premises for PII, cloud OK for public data) Design an intelligent model routing configuration: 1. **Task Classification Rules**: Define rules to classify incoming requests by complexity, sensitivity, and required capability: - Simple (FAQ, formatting) → cheapest model - Medium (summarization, translation) → balanced model - Complex (reasoning, code review, multi-step) → frontier model - Sensitive (PII, internal data) → local model only 2. **Routing Configuration**: Generate a routing config (JSON/YAML) with pattern matching rules, model priority chains with fallback, rate limiting per model, and cost caps per user/team/day. 3. **Cost Estimation**: For each use case, estimate monthly token usage and cost with vs. without routing. Show projected savings. 4. **Quality Guardrails**: How to detect when a cheaper model produces inadequate results and automatically escalate. 5. **Monitoring Dashboard**: Key metrics to track (cost per request, model distribution, quality scores, latency percentiles). 6. **A/B Testing Framework**: How to safely test new routing rules without degrading user experience. Output the complete routing configuration, a cost projection spreadsheet format, and implementation guide for popular gateway frameworks (LiteLLM, Manifest, custom).