RAG System Knowledge Base Quality Assessment and Optimization Report Generator
Input your RAG system configuration and sample queries to automatically generate a retrieval quality assessment report, including recall analysis, chunking strategy suggestions, and reranking optimization plans.
You are a RAG System Quality Auditor. Analyze my RAG (Retrieval-Augmented Generation) setup and generate a comprehensive optimization report. ## My Current Setup - **Document types**: [PDF/HTML/Markdown/etc.] - **Chunking strategy**: [fixed-size/semantic/recursive] - **Chunk size**: [N tokens], overlap: [M tokens] - **Embedding model**: [model name] - **Vector DB**: [Pinecone/Weaviate/Chroma/etc.] - **Reranker**: [yes/no, model if yes] - **Top-K retrieval**: [K] ## Sample Queries That Perform Poorly 1. [Query 1] → Expected answer: [X], Got: [Y] 2. [Query 2] → Expected answer: [X], Got: [Y] ## Please Generate 1. **Diagnosis**: Root cause analysis for each failed query 2. **Chunking Optimization**: Recommend chunk size, overlap, and strategy 3. **Retrieval Pipeline**: Suggest hybrid search, query expansion, or HyDE 4. **Reranking Strategy**: Whether to add/change reranker 5. **Evaluation Framework**: RAGAS-compatible test cases 6. **Implementation Plan**: Step-by-step migration path Format as a structured report with severity ratings (Critical/High/Medium/Low).
How to use this prompt
- 1Copy the complete prompt above.
- 2Replace the topic, subject, or style variables.
- 3Save effective changes to build your own version.
