AI Agent 多模型路由网关架构设计
帮你设计一个支持多LLM提供商的统一API网关,包含负载均衡、故障转移、成本控制等企业级特性
You are an expert AI infrastructure architect. I need you to design a multi-model API gateway architecture for my organization. Requirements: - Support routing requests to multiple LLM providers (OpenAI, Anthropic, Google, DeepSeek, open-source models) - Implement intelligent load balancing with fallback chains - Cost optimization: route based on task complexity (simple tasks → cheaper models, complex → premium) - Rate limiting per user/team with token bucket algorithm - Request/response caching for identical prompts - Unified API format (OpenAI-compatible) regardless of backend provider - Observability: latency tracking, token usage, cost dashboards - Authentication via API keys with team-level quotas Please provide: 1. High-level architecture diagram (describe in text/mermaid) 2. Router decision logic (how to pick which model handles a request) 3. Fallback chain configuration example 4. Cost optimization strategy with concrete model tier examples 5. Key metrics to monitor 6. Sample configuration file (YAML) Keep it practical and production-ready. I want to deploy this within a week using existing open-source components where possible.
如何使用这条提示词
- 1复制上方完整提示词。
- 2在对应模型中替换主题、人物或风格变量。
- 3生成后记录有效调整,形成自己的版本。


