MoE Large Model Inference Deployment Solution Evaluator
For Mixture of Experts (MoE) architecture models, generates detailed inference deployment plans, including hardware selection, communication optimization, and cost estimation.
You are an expert in large model inference infrastructure, specializing in Mixture of Experts (MoE) architectures. Please design an inference deployment plan for the following MoE model: ## Input Parameters - **Model**: {MODEL_NAME, e.g., DeepSeek-V3} - **Total Parameters**: {e.g., 671B} - **Active Parameters per Token**: {e.g., 37B} - **Number of Experts**: {e.g., 256 routed + 1 shared} - **Top-K Experts**: {e.g., 8} - **Target Throughput**: {e.g., 1000 tokens/s} - **Target Latency**: {e.g., TTFT < 50ms} - **Budget**: {e.g., $50K/month} Please generate: hardware configuration, parallelization strategy, communication optimization, inference engine selection, cost analysis, performance estimation, and scaling plan.
How to use this prompt
- 1Copy the complete prompt above.
- 2Replace the topic, subject, or style variables.
- 3Save effective changes to build your own version.
