Local Debugging Agent

A multi-agent debugging system demonstrating advanced AI agent design patterns for DevOps troubleshooting.

Key Features

• Context Engineering - Implements the 5-file pattern for persistent context management, clear prompt and too
• Multi-Agent Architecture - Supervisor + collector pattern where sub-agents only gather information
• Mock DevOps Tools - Simulated logs, metrics, runbooks, and remediation actions

📐 For detailed architecture diagrams, see ARCHITECTURE.md

Why Context Engineering?

This demo explores a fundamental challenge in building reliable AI agents: how to manage context effectively.

The Problem

Real-world agents face three pain points:

1. Observations can be huge - Logs, metrics, and web pages easily blow past context limits
2. Performance degrades - Model, goal tracking, and reasoning quality drops greatly with very long contexts, even if technically supported
3. Long inputs are expensive - Every token costs to transmit and process, even with caching

Many systems use aggressive truncation or compression. But here's the fundamental issue: an agent must predict the next action based on all prior state. You can't reliably know which observation becomes critical 10 steps later. Any irreversible compression carries risk.

The Solution: File System as External Memory and Sub-agent Architecture(like how claude, manus, and other advanced agents do)

This demo treats the file system as the ultimate context:

Property	Benefit
Unlimited size	No token limits on stored information
Persistent	Survives across turns, sessions, even restarts, gives agent a chance to learn from past experiences and predicable behavior
Directly operable	Agent reads and writes on demand
Structured	Organized files, not a blob of text

The context window becomes volatile RAM; the file system becomes persistent disk. The agent learns to externalize its working memory.

Restorable Compression

Our compression is always restorable:

• Full data → artifacts/ directory (JSON files with complete query results)
• Wire summary → context window (compact findings + artifact path reference)
• Need the details? Read the artifact file

This pattern appears throughout the collectors: write everything to disk, return only what's needed for the next decision.

Sub-Agents for Context Hygiene

The supervisor + collector pattern serves a second purpose: keeping the main agent's context clean.

• Each collector runs in its own context window
• Collectors return wire summaries (not raw data) to the supervisor
• The supervisor's context stays focused on decision-making

The Vision: Predictable, Self-Learning Agents

This architecture enables agents that:

• Stay predictable - Clear separation between data gathering and decision making
• Learn from experience - sessions.md stores past investigations with symptoms, root causes, and resolutions
• Grow with use - Each completed investigation makes future debugging faster
• Never get lost - Precious context preserved for decisions, not consumed by raw data

The cross-session learning is real: in our demos, the second occurrence of an issue pattern gets diagnosed significantly faster because the agent references what worked before.

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                     SUPERVISOR AGENT                             │
│  - Makes decisions based on collected information                │
│  - Determines root causes and recommends actions                 │
│  - Maintains context via planning files                          │
│  - Executes remediation and records sessions                     │
└──────────────────────┬──────────────────────────────────────────┘
                       │ calls as tools
       ┌───────────────┼───────────────┐
       ▼               ▼               ▼
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│    LOGS     │ │   METRICS   │ │  KNOWLEDGE  │
│  COLLECTOR  │ │  COLLECTOR  │ │  COLLECTOR  │
│  (info only)│ │  (info only)│ │  (info only)│
└─────────────┘ └─────────────┘ └─────────────┘
       │               │               │
       └───────────────┴───────────────┘
                       │
                       ▼
              ┌───────────────┐
              │   artifacts/  │
              │  (JSON files) │
              └───────────────┘

Sub-Agent Design

The collector sub-agents follow a strict "info only" pattern:

• They ONLY gather and report data
• They DO NOT make decisions or recommendations
• They write full data to artifacts/ and return wire summaries
• They return structured findings for the supervisor to analyze

This separation ensures:

1. Clear responsibility boundaries
2. The supervisor has full context for decision-making
3. Single-writer pattern for planning files (only supervisor writes)
4. Easier debugging and testing of each component

Context Engineering

Implements a 5-file pattern for persistent context management:

File	Purpose
task_plan.md	Phase tracking (5 phases), investigation goals, decisions, errors
findings.md	Research cache from collectors, organized by category (Logs, Metrics, Knowledge Base)
progress.md	Session log, activity tracking, error tracking with attempt counts, this is for debugging, no in the context of the agent
TODO.md	Next 3-7 immediate actions, rewritten each reasoning cycle
sessions.md	Past completed investigations for cross-session knowledge sharing

Key Principles

1. Filesystem as External Memory - Context window is volatile; files persist across turns
2. Single-Writer Pattern - Only supervisor modifies planning files; collectors write to artifacts/
3. 2-Action Rule - After every 2 collection actions, remind supervisor to call update_findings() to persist observations
4. 3-Strike Protocol - Track errors per tool; after 3 failures of the same tool, suggest alternative approach or escalate
5. Read Before Decide - Re-read task_plan.md before decision tools (attention manipulation via recency bias)
6. Cross-Session Learning - sessions.md stores past investigations to inform future troubleshooting

How Context Engineering Works

┌─────────────────────────────────────────────────────────────┐
│                    HOOK LIFECYCLE                            │
├─────────────────────────────────────────────────────────────┤
│ AgentInitializedEvent                                        │
│   └─> Load task_plan.md, TODO.md, findings.md, sessions.md  │
│       into system prompt                                     │
├─────────────────────────────────────────────────────────────┤
│ BeforeToolCallEvent                                          │
│   └─> If decision tool: re-read task_plan.md (recency bias) │
├─────────────────────────────────────────────────────────────┤
│ AfterToolCallEvent                                           │
│   ├─> Increment action counter (2-Action Rule)              │
│   ├─> If error: track in error_attempts (3-Strike Protocol) │
│   └─> If 2 actions: remind to call update_findings()        │
├─────────────────────────────────────────────────────────────┤
│ AfterInvocationEvent                                         │
│   └─> Update progress.md with activity log                  │
└─────────────────────────────────────────────────────────────┘

Installation

# Clone or create the project
cd local-debugging-agent

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
pip install -e .

# Or with development dependencies
pip install -e ".[dev]"

Configuration

Create a .env file with your Anthropic API key:

cp .env.example .env
# Edit .env and add your key:
# ANTHROPIC_API_KEY=your-api-key-here

Usage

Run Demo Scenarios

# Log spike investigation (database issues, you can try to run twice to see how the previous session really help the agent to solve the similar problems with confidence)
python -m src.main log_spike

# Performance degradation (same issue like above, but with different prompt )
python -m src.main performance

Reset Environment

After running the agent, planning files and artifacts accumulate. Use the reset script to clean up:

# Interactive mode - shows current state and asks what to delete
python reset_env.py

# Quick reset of planning files + artifacts
python reset_env.py --all

# Preview what would be deleted (no actual deletion)
python reset_env.py --dry-run --all

# Reset only planning files
python reset_env.py --planning

# Reset only artifacts
python reset_env.py --artifacts

# Include Python cache cleanup
python reset_env.py --all --cache

Project Structure

local-debugging-agent/
├── src/
│   ├── main.py                    # Entry point with CLI
│   ├── models/
│   │   └── anthropic_config.py    # Anthropic API configuration
│   ├── agents/
│   │   ├── supervisor_agent.py    # Decision-making orchestrator
│   │   ├── logs_collector.py      # Log data collector (sub-agent)
│   │   ├── metrics_collector.py   # Metrics data collector (sub-agent)
│   │   └── knowledge_collector.py # Runbook/docs collector (sub-agent)
│   ├── hooks/
│   │   └── context_engineering_hook.py  # 5-file pattern hooks
│   ├── tools/
│   │   ├── logs_tools.py          # Log query tools (search_logs, get_error_logs, etc.)
│   │   ├── metrics_tools.py       # Metrics tools (get_performance_metrics, etc.)
│   │   ├── runbook_tools.py       # Knowledge tools (search_runbooks, etc.)
│   │   ├── remediation_tools.py   # Remediation tools (execute_remediation, etc.)
│   │   ├── supervisor_tools.py    # Planning file tools (update_findings, etc.)
│   │   └── response_utils.py      # Response formatting utilities
│   ├── context/
│   │   ├── file_manager.py        # Planning file utilities
│   │   └── templates/             # File templates (5 .md files)
│   └── utils/
│       └── demo_output.py         # Demo output formatting
├── mock_data/
│   ├── logs_data/                 # Log scenario JSON files
│   └── metrics_data/              # Metrics scenario JSON files
├── planning_files/                # Runtime planning files (gitignored)
├── artifacts/                     # Collector output JSONs (gitignored)
├── scenarios/                     # Demo scenario scripts
├── tests/                         # Unit and integration tests
└── reset_env.py                   # Environment cleanup script

Tools Reference

Log Tools (src/tools/logs_tools.py)

Tool	Description	Key Parameters
search_logs	Search logs by pattern and filters	pattern, time_range, log_level, service
get_error_logs	Get pre-aggregated error summaries	since, service, severity
analyze_log_patterns	Find recurring patterns with frequency	time_window, min_occurrences
get_recent_logs	Get latest log entries (newest first)	limit, service, include_levels

Metrics Tools (src/tools/metrics_tools.py)

Tool	Description	Key Parameters
get_performance_metrics	Get response time and throughput	metric_type, time_range, service
get_error_rates	Get error rate percentages	time_window, service
get_resource_metrics	Get CPU, memory, disk, GC utilization	resource_type, service, time_window
analyze_trends	Analyze time-series with leak detection	metric_name, anomaly_threshold

Knowledge Tools (src/tools/runbook_tools.py)

Tool	Description	Key Parameters
search_runbooks	Find runbooks by keyword	query, severity_filter
get_troubleshooting_guide	Get full runbook content	runbook_key
get_known_issues	Find documented bugs/limitations	status_filter, query

Remediation Tools (src/tools/remediation_tools.py)

Tool	Description	Key Parameters
get_available_remediations	Get remediation tools for a runbook	runbook_id
execute_remediation	Execute a remediation action (mock)	tool_key, target_service

Available Remediation Actions:

• restart_pods - Rolling restart (risk: low)
• restart_auth_service - Restart auth pods (risk: low)
• rotate_logs - Force log rotation (risk: low)
• flush_dns_cache - Clear DNS cache (risk: low)

Supervisor Tools (src/tools/supervisor_tools.py)

Tool	Description	Key Parameters
update_todo	Rewrite TODO.md with next actions	actions, current_phase
update_findings	Record findings to findings.md	category, summary, key_observations
record_session	Save investigation to sessions.md	issue_type, root_cause, resolution
update_phase	Update phase status in task_plan.md	phase_number, new_status
execute_recommended_remediation	Execute recommended remediation for a runbook	runbook_id, issue_summary

Response Format

All tools support a response_format parameter:

• "detailed" (default) - Returns full data
• "concise" - Returns truncated data (5-10 entries max, summaries only)

Mock Data

mock_data/
├── logs_data/
│   ├── log_spike_scenario.json      # 16 log entries with 5-min failure cascade
│   └── error_patterns.json          # 5 aggregated error patterns with counts
└── metrics_data/
    ├── perf_degradation_scenario.json  # 7-phase timeline (90 min degradation)
    ├── response_times.json             # Response time percentiles with baselines
    ├── resource_usage.json             # CPU, memory, GC, thread metrics
    ├── error_rates.json                # Error rate progression with thresholds
    └── trends.json                     # Anomaly detection and leak indicators

Mock Scenarios

1. Log Spike Investigation

Simulates a database connectivity issue with a 5-minute failure cascade:

Phase	Time	Event
Normal	14:20-14:21	Info logs, normal queries
Warning	14:21-14:22	Slow queries, connection pool at 90%
Error	14:23-14:24	Timeouts, pool exhausted (50/50 connections)
Critical	14:24-14:25	OutOfMemoryError, circuit breaker opens

Error patterns detected:

• Database connection timeout (15 occurrences)
• Connection pool exhaustion (8 occurrences)
• OutOfMemoryError (3 critical)
• Circuit breaker triggered (2 events)

2. Performance Degradation

Simulates a memory leak with gradual degradation over 90 minutes:

Phase	Time	Response Time	Memory	Status
normal	14:00	150ms (P99: 280ms)	45%	healthy
early_warning	14:15	280ms (P99: 450ms)	55%	warning
degrading	14:30	450ms (P99: 850ms)	65%	degraded
degraded	14:45	750ms (P99: 1.5s)	75%	critical
critical	15:00	1.2s (P99: 3.5s)	85%	critical
severe	15:15	2.5s (P99: 8s)	92%	failure
failure	15:30	5s (P99: 15s)	98%	failure

Key indicators:

• Heap grows from 900MB to 1,966MB (2GB max)
• GC pause time: 180ms → 15,000ms (83x increase)
• Leak rate: 12% per hour
• Estimated OOM: 15:45:00Z

Customization

Adding New Tools

Create tools in src/tools/ using the @tool decorator:

from strands import tool

@tool
def my_custom_tool(parameter: str, response_format: str = "detailed") -> dict:
    """Tool description for the agent."""
    result = {"status": "success", "data": "..."}
    # Use response_utils for consistent formatting
    return result

Adding New Collectors

Create collectors in src/agents/ following the pattern in logs_collector.py:

1. Create an async function decorated with @tool
2. Call relevant tools to gather data
3. Write full results to artifacts/ directory
4. Return a wire summary (not full data)

Modifying Mock Data

Edit JSON files in mock_data/ to change the scenarios. Key structures:

Log entry:

{
  "timestamp": "2024-01-15T14:20:00Z",
  "level": "INFO|WARN|ERROR|CRITICAL",
  "service": "web-service",
  "message": "Log message here"
}

Metrics entry:

{
  "timestamp": "2024-01-15T14:00:00Z",
  "phase": "normal|warning|critical",
  "response_time_ms": {"p50": 120, "p90": 145, "p99": 280},
  "memory_percent": 45,
  "cpu_percent": 25
}

Technical Details

Strands SDK Integration

• Uses strands.models.anthropic.AnthropicModel for direct API access
• Sub-agents implemented as @tool decorated async functions
• Hooks via HookProvider class with event callbacks

Context Engineering Hooks

Event	Action
AgentInitializedEvent	Load planning context (task_plan, TODO, findings, sessions) into system prompt
BeforeToolCallEvent	Read task_plan.md before decision tools (attention manipulation via recency bias)
AfterToolCallEvent	Track 2-Action Rule counter, log errors for 3-Strike Protocol
AfterInvocationEvent	Update progress.md with activity log
MessageAddedEvent	Track message flow for debugging

Investigation Workflow

The supervisor follows a standard investigation flow:

1. Check past sessions - Review sessions.md for similar issues
2. Collect data - Dispatch collectors for logs, metrics, knowledge
3. Update findings - Call update_findings() after collecting (2-Action Rule)
4. Analyze and correlate - Identify root cause from findings
5. Execute remediation - Call execute_recommended_remediation()
6. Record session - Call record_session() with learnings and actions

License

MIT