Apple Silicon 本地模型部署性能调优助手
帮助你在 Mac 上优化本地 LLM 推理性能,包括内存管理、批处理配置和 SSD 缓存策略
You are an expert in deploying and optimizing LLM inference on Apple Silicon Macs (M1/M2/M3/M4 series). Help me optimize my local model deployment. Context: - Hardware: [describe your Mac model, RAM, SSD] - Model: [model name and size, e.g. Llama 3 70B Q4] - Framework: [mlx-lm / llama.cpp / ollama / other] - Use case: [chat / batch processing / API server / coding assistant] Please provide: 1. **Memory optimization**: Quantization level recommendations, KV cache settings, and memory-mapped loading strategies for my hardware 2. **Batch processing config**: Optimal continuous batching parameters, max concurrent requests, and queue management 3. **SSD caching strategy**: How to configure SSD-based KV cache offloading for models that exceed unified memory 4. **Performance benchmarks**: Expected tokens/sec for my setup, and specific flags/settings to maximize throughput 5. **Monitoring**: Commands and tools to monitor GPU utilization, memory pressure, and thermal throttling 6. **Comparison**: Trade-offs between different inference frameworks for my specific use case Provide concrete terminal commands and config snippets I can copy-paste. Flag any settings that risk system instability.
如何使用这条提示词
- 1复制上方完整提示词。
- 2在对应模型中替换主题、人物或风格变量。
- 3生成后记录有效调整,形成自己的版本。


