On-Device LLM Deployment Performance Comparison Report Generator
Generate performance benchmark reports for LLM inference on edge devices, covering latency, throughput, memory usage, and quantization strategy comparisons.
You are an edge AI deployment specialist. Generate a comprehensive benchmark report for deploying LLMs on edge devices. ## Test Configuration - Target device: [Raspberry Pi 5 / iPhone 16 / Android flagship / Mac Mini M4] - Models to evaluate: [list models, e.g. Gemma-4-E2B, Phi-4-mini, Qwen3-1.5B] - Use cases: [chat / code completion / tool calling / vision] ## Report Structure 1. Quantization Impact Analysis - Compare FP16, INT8, INT4, MXFP4 for each model across model size, RAM usage, tokens/sec, and quality metrics. 2. Inference Engine Comparison - Compare llama.cpp, LiteRT-LM, mistral.rs, MLX, ONNX Runtime on cold start time, first token latency, sustained throughput, peak memory, GPU/NPU utilization. 3. Battery and Thermal Analysis (mobile) - Power consumption per 1K tokens, thermal throttling onset, sustained vs burst performance. 4. Recommendations - Best model-engine-quantization combo per use case, memory-constrained strategies, when to use on-device vs cloud fallback. Format as a professional benchmark report with markdown tables.
How to use this prompt
- 1Copy the complete prompt above.
- 2Replace the topic, subject, or style variables.
- 3Save effective changes to build your own version.
