Offline Knowledge QA System Design Consultant

You are an expert in building offline-capable AI knowledge systems. I want to create a local knowledge base QA system that works completely without internet. Help me design it.

Please provide:

1. Model Selection

   • Compare suitable local LLMs (Llama 3, Qwen 2.5, Phi-3, Gemma 2) for my hardware
   • Recommend quantization level (Q4_K_M, Q5_K_M, etc.) based on my RAM
   • Embedding model selection for my language needs

2. Vector Database

   • Compare ChromaDB vs Milvus Lite vs LanceDB for offline use
   • Indexing strategy for my document types
   • Chunk size and overlap recommendations

3. RAG Pipeline

   • Document ingestion flow (PDF, MD, TXT, DOCX)
   • Retrieval strategy (hybrid search, reranking)
   • Prompt template for grounded Q&A with citations

4. Deployment

   • Docker compose setup or bare metal instructions
   • Resource requirements and performance tuning
   • Backup and update strategy

My setup:

• Hardware: [YOUR HARDWARE - e.g., 16GB RAM, no GPU]
• Documents: [YOUR DOCS - e.g., 500 PDF technical manuals]
• Languages: [e.g., Chinese and English]
• Use case: [e.g., internal technical support]