智能模型路由与成本优化网关配置生成器

You are an expert in LLM cost optimization and intelligent model routing.

Given:

• Available models: {models} (e.g., GPT-4o, Claude Sonnet, Gemini Flash, Llama 3.3, local Ollama models)
• Use cases: {use_cases} (e.g., code generation, summarization, chat, data extraction, creative writing)
• Monthly budget: {budget}
• Latency requirements: {latency} (e.g., <500ms for chat, <5s for complex reasoning)
• Privacy requirements: {privacy} (e.g., no data leaves premises for PII, cloud OK for public data)

Design an intelligent model routing configuration:

1. Task Classification Rules: Define rules to classify incoming requests by complexity, sensitivity, and required capability:

   • Simple (FAQ, formatting) → cheapest model
   • Medium (summarization, translation) → balanced model
   • Complex (reasoning, code review, multi-step) → frontier model
   • Sensitive (PII, internal data) → local model only

2. Routing Configuration: Generate a routing config (JSON/YAML) with pattern matching rules, model priority chains with fallback, rate limiting per model, and cost caps per user/team/day.

3. Cost Estimation: For each use case, estimate monthly token usage and cost with vs. without routing. Show projected savings.

4. Quality Guardrails: How to detect when a cheaper model produces inadequate results and automatically escalate.

5. Monitoring Dashboard: Key metrics to track (cost per request, model distribution, quality scores, latency percentiles).

6. A/B Testing Framework: How to safely test new routing rules without degrading user experience.

Output the complete routing configuration, a cost projection spreadsheet format, and implementation guide for popular gateway frameworks (LiteLLM, Manifest, custom).