Back to prompt library
Text · General-purpose LLMEnterprise-Grade AI Gateway Multi-Tenant Architecture Design PromptPW
CreatorPrompt2 Editorial TeamCurated by PromptWhisper
TextGeneral-purpose LLMAI & Agents

Enterprise-Grade AI Gateway Multi-Tenant Architecture Design Prompt

Design an enterprise-grade AI gateway architecture supporting multi-tenancy, guardrails, MCP registration, and intelligent routing.

12Views
Full promptReplace variables in braces, then use it directly

You are an enterprise AI platform architect. Design a production-grade AI Gateway with the following capabilities: Requirements: - Multi-tenant isolation with per-tenant rate limits and quotas - Model routing: intelligent routing across multiple LLM providers (OpenAI, Anthropic, Google, local models) - MCP (Model Context Protocol) server registry and discovery - Guardrails: input/output content filtering, PII detection, token budget enforcement - Observability: request tracing, cost tracking per tenant, latency dashboards Please provide: 1. **System Architecture Diagram** (describe in Mermaid syntax) 2. **API Gateway Layer**: Authentication, rate limiting, request transformation 3. **Router Design**: Cost-aware routing, fallback chains, A/B testing support 4. **MCP Registry**: How to register, discover, and version MCP servers 5. **Guardrail Pipeline**: Pre-processing and post-processing filter chain design 6. **Multi-tenancy Model**: Database schema for tenants, API keys, usage quotas 7. **Deployment**: Kubernetes manifests or Docker Compose for the core services 8. **Cost Optimization**: Token caching, semantic dedup, prompt compression strategies Output a complete technical design document with architecture decisions and trade-offs explained.

4/17/2026

How to use this prompt

  1. 1Copy the complete prompt above.
  2. 2Replace the topic, subject, or style variables.
  3. 3Save effective changes to build your own version.