LLM Inference Service Gray Release and Traffic Switching Runbook Generator

You are a senior MLOps/SRE engineer specializing in LLM inference service deployments. Given the following service architecture: - **Current model**: {current_model} - **New model**: {new_model} - **Infrastructure**: {infra_details} - **Traffic volume**: {qps} requests/second - **SLA requirements**: {sla} Generate a **production-ready deployment runbook**: ## Phase 1: Pre-deployment Checklist - Model weights downloaded and verified (sha256) - Benchmark results on staging (throughput, latency, quality) - A/B test evaluation criteria defined - Rollback procedure documented and tested - Monitoring dashboards configured ## Phase 2: Canary Deployment (1-5% traffic) - Deployment commands (Helm/kubectl/docker) - Health check endpoints and expected responses - Key metrics to monitor (TTFT, ITL, throughput, error rate, GPU utilization) - Duration: minimum observation window - Go/No-go criteria with specific thresholds ## Phase 3: Progressive Rollout - Traffic split schedule: 5% -> 25% -> 50% -> 100% - Minimum soak time per stage - Automated quality comparison (side-by-side eval) - Cost comparison (tokens/second/GPU) ## Phase 4: Full Rollout and Cleanup - Old model decommission steps - Cache warming strategy - Documentation updates ## Emergency Rollback Procedure - Single-command rollback - Traffic drain procedure - Post-mortem template ## Monitoring and Alerting - Prometheus/Grafana query templates for key metrics - PagerDuty/Slack alert rules - Anomaly detection thresholds Output as executable markdown with copy-pasteable commands.