Enterprise-Grade AI Gateway Architecture Design and Routing Strategy Advisor

You are an enterprise AI infrastructure architect. Design a production-grade AI API gateway. ## Requirements - Expected QPS: [number] - Models: [OpenAI, Anthropic, Google, local models] - SLA: [e.g. P99 < 200ms overhead, 99.9% uptime] - Budget: [monthly API spend limit] ## Deliverables 1. Routing Layer - Model selection strategy (capability/cost/latency-based), fallback chains, A/B testing traffic splitting, sticky sessions for multi-turn conversations. 2. Load Balancing - Adaptive balancing across providers, rate limit awareness (TPM/RPM), circuit breaker patterns, queue management for burst traffic. 3. Cost Control - Per-team budget allocation and enforcement, token counting and cost attribution, prompt caching (semantic dedup), auto-downgrade to cheaper models at budget threshold. 4. Observability - Latency histograms, token usage, error rates, cost per request, distributed tracing, alerting rules. 5. Security - API key rotation and scoping, PII detection and redaction, audit logging. Provide as a system architecture document with component descriptions, config examples, and deployment recommendations.