On-Device AI Model Deployment and Optimization Guide Generator

You are an expert in on-device AI deployment and model optimization. Help me deploy an AI model to run efficiently on edge devices. ## My Setup: - Target device: [smartphone / Raspberry Pi / embedded board - specify] - Hardware specs: [CPU/GPU/NPU, RAM, storage] - Model type: [LLM / vision / speech - specify] - Base model: [model name and size] - Latency requirement: [max acceptable inference time] - Memory budget: [max RAM usage] ## Please generate a complete deployment guide covering: ### 1. Model Optimization - Quantization strategy (INT8/INT4/mixed-precision) with expected quality-speed tradeoffs - Knowledge distillation options if the model is too large - Layer pruning and architecture search recommendations - Specific commands using tools like llama.cpp, ONNX Runtime, TensorRT, Core ML, LiteRT ### 2. Runtime Configuration - Optimal inference engine for the target platform - Thread/batch configuration - Memory mapping and KV-cache optimization ### 3. Integration Code - Minimal working example to load and run the optimized model - Streaming output handling and error handling ### 4. Benchmarking - How to measure tokens/sec, time-to-first-token, memory peak - Comparison table template: original vs optimized model ### 5. Production Checklist - Model versioning and OTA update strategy - Privacy considerations for on-device inference Provide concrete commands and code, not just theory.