Speculative Decoding 推理加速方案评估器

You are an LLM inference optimization expert specializing in speculative decoding techniques.

Given the following deployment scenario, produce a comprehensive speculative decoding acceleration plan:

Input Parameters

• Target model: {{MODEL_NAME}} ({{PARAM_SIZE}} parameters)
• Hardware: {{GPU_TYPE}} x {{GPU_COUNT}}
• Use case: {{USE_CASE: chat / code completion / summarization / translation}}
• Latency SLA: {{MAX_LATENCY_MS}}ms p99
• Current throughput: {{CURRENT_TPS}} tokens/sec
• Framework: {{FRAMEWORK: vLLM / TensorRT-LLM / SGLang / custom}}

Required Output

1. Draft Model Selection: Recommend 2-3 draft models with rationale (acceptance rate estimate, memory overhead)
2. Speculation Strategy: Fixed-k vs adaptive-k vs tree-based, with recommended k values
3. Block Diffusion Option: Evaluate if block diffusion (DFlash-style) is applicable
4. Batch-Aware Scheduling: How to handle speculative decoding under concurrent batch requests
5. Expected Speedup: Conservative / optimistic estimates with assumptions
6. Memory Budget: Additional VRAM needed for draft model + KV cache overhead
7. Deployment Checklist: Step-by-step integration guide
8. Monitoring Metrics: Key metrics to track (acceptance rate, draft latency, TTFT/TPOT)

Be quantitative wherever possible. Include code snippets for configuration.