端侧AI应用性能优化检查清单

You are an edge AI deployment specialist. Generate a comprehensive performance optimization checklist for deploying AI models on edge devices.

Target Device Profile

• Device type: [e.g., smartphone, Raspberry Pi, embedded board, browser]
• Hardware specs: [e.g., 8GB RAM, Snapdragon 8 Gen 3, Apple M-series, WebGPU]
• Model type: [e.g., LLM, vision model, speech recognition]
• Model size: [e.g., 3B parameters, 500MB]
• Latency requirement: [e.g., <100ms first token, real-time inference]

Generate Optimization Checklist:

Phase 1: Model Compression

• Quantization strategy (INT8/INT4/GPTQ/AWQ/GGUF)
• Knowledge distillation from larger teacher model
• Pruning (structured vs unstructured)
• Vocabulary reduction for target use case
• LoRA/QLoRA fine-tuning for task-specific optimization

Phase 2: Inference Engine Selection

Compare and recommend from: LiteRT-LM, llama.cpp, MLC-LLM, ONNX Runtime, TensorRT, Core ML

• Benchmark template for each engine
• Platform compatibility matrix

Phase 3: Runtime Optimization

• KV-cache management and memory pooling
• Speculative decoding configuration
• Batch scheduling for concurrent requests
• Context window sliding strategy
• Prefill/decode phase optimization

Phase 4: System-Level Tuning

• Thermal throttling mitigation
• Power consumption profiling
• Memory mapping and swap configuration
• GPU/NPU scheduling priorities

Phase 5: Measurement & Validation

• Benchmark script template (tokens/sec, TTFT, memory peak)
• Quality regression test suite
• A/B comparison framework

For each item, provide:

1. Why it matters
2. How to implement (concrete commands/code)
3. Expected improvement range
4. Trade-offs to consider