AI Agent 记忆系统压力测试与评估框架

You are an AI systems evaluation expert specializing in agent memory architectures. Design a comprehensive stress test and evaluation framework for an AI agent memory system.

Memory System Under Test:

• Type: [vector DB / knowledge graph / hybrid / file-based]
• Agent Framework: [e.g., LangChain, CrewAI, custom]
• Context Window: [token limit]
• Persistence: [ephemeral / session / long-term]

Generate the Following Test Suites:

Suite 1: Retrieval Accuracy

• Design 10 test cases with planted facts at varying recency
• Include distractor information to test precision
• Measure: Recall@K, Precision@K, MRR

Suite 2: Forgetting Curve

• Simulate conversations of increasing length (100, 500, 1000, 5000 turns)
• Plant critical facts at turn N, query at turn N+X
• Measure: At what distance does recall drop below 80%?

Suite 3: Contradiction Handling

• Introduce conflicting information at different timestamps
• Test whether the agent uses the most recent vs most frequent information
• Measure: Temporal consistency score

Suite 4: Cross-Session Continuity

• Define 5 facts in Session A, query in Session B
• Measure: Cross-session recall rate

Suite 5: Context Window Efficiency

• Measure how much of the context window is used for memory vs new input
• Test compression strategies
• Measure: Useful token ratio

Suite 6: Adversarial Injection

• Attempt to overwrite memories with conflicting injections
• Test memory isolation between users/sessions
• Measure: Injection resistance score

Output Format:

For each suite, provide:

1. Test script pseudocode
2. Expected baseline metrics
3. Scoring rubric (pass/warn/fail thresholds)
4. Automated evaluation criteria

Now generate the complete framework for: [describe your agent memory system]