AI Agent Anomaly Behavior Monitoring and Alert Rule Generator

You are an AI safety and observability engineer. Design a comprehensive anomaly detection and alerting ruleset for monitoring AI agent behavior in production. ## Agent System Context - **Agent Framework**: {framework} (e.g., LangChain, CrewAI, AutoGen, custom) - **Deployment Scale**: {scale} (e.g., 100 concurrent agents) - **Tool Access**: {tools} (e.g., code execution, web browsing, file system, API calls) - **Risk Tolerance**: {risk_level} (low/medium/high) ## Generate Monitoring Rules For: ### 1. Token & Cost Anomalies - Per-task token budget thresholds (input/output separately) - Cost spike detection (rolling average comparison) - Context window utilization alerts - Unusual model switching patterns ### 2. Tool Call Anomalies - Repeated failed tool calls (loop detection) - Unusual tool call frequency or ordering - Dangerous tool call patterns (e.g., recursive file deletion) - Unauthorized tool access attempts - Tool call latency degradation ### 3. Behavioral Anomalies - Task completion time outliers - Agent stuck in reasoning loops (same output patterns) - Goal drift detection (task divergence from original intent) - Hallucination indicators in structured output - Unexpected conversation length ### 4. Security Boundary Monitoring - Prompt injection attempt detection - Data exfiltration patterns (sensitive data in outputs) - Privilege escalation attempts - Sandbox escape indicators - PII leakage in logs or outputs ### 5. System Health - Memory usage per agent session - Queue depth and processing delays - Error rate by agent type and task category - Upstream API availability and degradation ## Output Format For each rule, provide: - Rule name and ID - Detection logic (pseudocode or query) - Severity level (info/warning/critical) - Recommended action (alert/throttle/kill/escalate) - False positive mitigation strategy - Example Prometheus/Grafana alert rule or equivalent