Skip to content

Security: ram-ai-kumar/ai-orchestrator

Security

docs/SECURITY.md

Security & Zero Trust Architecture

Executive Summary

AI Orchestrator implements a defense-in-depth security posture following Zero Trust Architecture (ZTA) principles. All code execution occurs in isolated environments with strict access controls, comprehensive audit logging, and continuous validation at each stage of the pipeline.

Zero Trust Principles

1. Never Trust, Always Verify

Implementation: Every operation is validated before execution:

  • Input Validation: All user inputs sanitized before processing
  • Command Validation: Only whitelisted commands allowed in sandbox
  • Patch Validation: All generated code reviewed before application
  • Output Validation: Test results validated before approval

2. Least Privilege Access

Implementation: Each component operates with minimum required permissions:

  • Docker Containers: Run without privileged mode, no host filesystem access
  • Git Worktrees: Isolated branches, no direct main branch access
  • LLM Access: Local-only, no external API keys or credentials
  • Filesystem Access: Read-only for codebase, write-only for worktrees

3. Assume Compromise

Implementation: System designed to operate even if components are compromised:

  • Sandbox Isolation: Compromised container cannot affect host
  • Worktree Isolation: Compromised worktree cannot affect main branch
  • State Immutability: State passed by value, not reference
  • Audit Trail: All actions logged for forensics

Security Architecture Layers

Layer 1: Input Validation

Location: app/security/input_validation.py (to be implemented)

Controls:

  • Task description length limits
  • Path traversal prevention
  • Command injection prevention
  • XSS prevention for web interface

Validation Rules:

MAX_TASK_LENGTH = 10000
ALLOWED_PATHS = ["/workspace"]
FORBIDDEN_PATTERNS = ["../", "~", "/etc", "/proc"]

Layer 2: LLM Security

Model Selection:

  • Local models only (no external APIs)
  • No API keys or credentials stored
  • Models run in user-controlled environment

Prompt Injection Protection:

  • System prompts enforce output format constraints
  • Output parsing validates structure
  • No arbitrary code execution from LLM output

Supply Chain Security:

  • Models pulled from trusted Ollama registry
  • Model signatures verified (future enhancement)
  • Model versioning and pinning

Layer 3: Vector Database Security

Location: app/retrieval/vector_store.py

Controls:

  • Local ChromaDB instance (no network exposure)
  • File-based storage with filesystem permissions
  • No authentication required (local-only)
  • Document metadata sanitization

Access Control:

# ChromaDB runs locally with filesystem permissions
# Only orchestrator process has read/write access
persist_directory="./data/chroma"
chmod 700 ./data/chroma

Layer 4: Code Generation Security

Location: app/agents/coder.py

Controls:

  • Output format restricted to git diff patches
  • No full file generation (diffs only)
  • No arbitrary command execution
  • Patch syntax validation before application

Patch Validation:

# Validate patch format before application
def validate_patch(patch_content):
    required_headers = ["--- a/", "+++ b/", "@@"]
    return all(header in patch_content for header in required_headers)

Layer 5: Git Worktree Isolation

Location: app/tools/git_worktree.py

Controls:

  • Patches applied to isolated worktrees, not main branch
  • Worktrees created in dedicated directory (.worktrees-*)
  • Automatic cleanup after completion
  • No direct push to main branch

Isolation Benefits:

  • Main branch never touched during execution
  • Easy rollback by deleting worktree
  • Parallel development without conflicts
  • Audit trail of all worktree operations

Layer 6: Docker Sandbox Security

Location: app/sandbox/docker_runner.py

Controls:

  • Non-privileged containers (no --privileged flag)
  • Resource limits enforced (CPU: 100k, Memory: 512MB)
  • No host filesystem access (volume mount only)
  • Network isolation (default bridge network)
  • Automatic container cleanup

Container Security Profile:

container = client.containers.run(
    image,
    command="tail -f /dev/null",
    volumes={workspace: {"bind": "/workspace", "mode": "rw"}},
    detach=True,
    mem_limit="512m",
    cpu_quota=100000,
    # No --privileged
    # No --network=host
    # No --cap-add
)

Layer 7: Command Whitelisting

Location: app/sandbox/command_whitelist.py

Controls:

  • Regex-based pattern matching
  • Explicit allowlist (no denylist approach)
  • No shell command execution (direct exec)
  • Command argument validation

Allowed Commands:

ALLOWED_COMMANDS = [
    r"python.*\.py",
    r"pytest.*",
    r"npm test",
    r"cargo test",
    r"make test",
]

Layer 8: Code Review Security

Location: app/agents/reviewer.py

Controls:

  • Automated security review by LLM
  • Checks for:
    • SQL injection patterns
    • Command injection patterns
    • XSS vulnerabilities
    • Hardcoded credentials
    • Insecure dependencies
  • Approval required before patch application

Review Criteria:

  • Correctness: Does the code accomplish the task?
  • Security: Are there obvious vulnerabilities?
  • Edge cases: Are error conditions handled?
  • Completeness: Is the implementation complete?

Layer 9: Output Validation

Location: app/security/output_validation.py (to be implemented)

Controls:

  • Test result validation
  • Exit code checking
  • Output sanitization
  • Error message filtering

Validation Rules:

def validate_test_result(result):
    # Check exit code
    if result["exit_code"] not in [0, 1]:
        raise SecurityException("Unexpected exit code")

    # Check output size
    if len(result["output"]) > MAX_OUTPUT_SIZE:
        raise SecurityException("Output too large")

    # Check for sensitive data
    if contains_secrets(result["output"]):
        raise SecurityException("Sensitive data in output")

Threat Model

Threat Actors

  1. Malicious User: Attempts to inject malicious code or commands
  2. Compromised LLM: Model generates malicious output
  3. Container Escape: Attacker escapes Docker container
  4. Supply Chain Attack: Malicious model or dependency
  5. Insider Threat: Authorized user abuses privileges

Mitigation Strategies

Threat Mitigation Control Layer
Malicious User Input validation, command whitelisting 1, 7
Compromised LLM Output validation, code review 4, 8
Container Escape Non-privileged, resource limits 6
Supply Chain Local models, signature verification 2
Insider Threat Audit logging, access control 9

Audit Logging

Location: app/core/state.py (logs field)

Logging Events:

  • Task submission
  • Plan generation
  • Context retrieval
  • Patch generation
  • Worktree creation
  • Patch application
  • Test execution
  • Review decisions
  • Step progression

Log Format:

state["logs"].append(f"Coder generated patch for step {state['current_step']}")
state["logs"].append(f"Reviewer approved patch")
state["logs"].append(f"Test result: {result['success']}")

Log Retention:

  • In-memory during execution
  • Persistent storage (future enhancement)
  • Immutable log entries
  • Tamper-evident logging (future enhancement)

Access Control

Role-Based Access Control (RBAC)

Roles (to be implemented):

  • Admin: Full system access
  • Developer: Submit tasks, view results
  • Reviewer: Approve/reject patches
  • Auditor: View logs only

Permissions:

PERMISSIONS = {
    "admin": ["*"],
    "developer": ["task:submit", "result:view"],
    "reviewer": ["patch:approve", "patch:reject"],
    "auditor": ["log:view"],
}

Authentication

Current: None (local execution)

Future Enhancements:

  • API key authentication
  • OAuth 2.0 integration
  • MFA for sensitive operations
  • Session management

Compliance Alignment

SOC 2 Type II

Trust Principles:

  • Security: Implemented via ZTA controls
  • Availability: Resource limits prevent DoS
  • Processing Integrity: Code review and testing
  • Confidentiality: Local processing, no data exfiltration
  • Privacy: No personal data processed

Evidence Collection:

  • Audit logs for all operations
  • Access logs for authentication
  • Change logs for code modifications
  • Performance metrics for availability

ISO 27001

Controls Implemented:

  • A.8.1: Asset inventory (codebase tracking)
  • A.9.1: Access control (RBAC)
  • A.12.1: Operations security (sandboxing)
  • A.14.1: System acquisition (supply chain security)
  • A.16.1: Incident management (logging and monitoring)

NIST Cybersecurity Framework

Functions:

  • Identify: Asset management, risk assessment
  • Protect: Access control, awareness training
  • Detect: Anomaly detection, logging
  • Respond: Incident response planning
  • Recover: Backup and recovery procedures

Data Protection

Data at Rest

  • Codebase: Git repository with encryption at rest (future)
  • Vector DB: File-based storage with filesystem permissions
  • Logs: Local filesystem with restricted access
  • Worktrees: Temporary, auto-deleted after completion

Data in Transit

  • Local Execution: No network transmission
  • Future: TLS 1.3 for remote operations
  • Future: Certificate pinning for external APIs

Data Privacy

  • No personal data processed
  • No PII in codebase
  • No telemetry or analytics
  • Local-only processing

Incident Response

Incident Types

  1. Security Incident: Successful attack or compromise
  2. Policy Violation: Deviation from security policies
  3. System Failure: Unavailability or degradation
  4. Data Breach: Unauthorized data access

Response Procedures

Detection:

  • Log monitoring and alerting
  • Anomaly detection in test results
  • Resource usage monitoring

Containment:

  • Isolate affected containers
  • Stop graph execution
  • Preserve forensic evidence

Eradication:

  • Remove malicious code
  • Patch vulnerabilities
  • Update security controls

Recovery:

  • Restore from clean state
  • Verify system integrity
  • Resume operations

Lessons Learned:

  • Post-incident review
  • Update security policies
  • Improve detection capabilities

Security Testing

Current Testing

  • Manual code review
  • Unit tests for critical components (future)
  • Integration tests (future)

Future Testing

  • Static Analysis: Bandit, Semgrep
  • Dependency Scanning: pip-audit, Snyk
  • Container Scanning: Trivy, Docker Scout
  • Penetration Testing: External security assessment
  • Fuzz Testing: Input validation testing

Security Configuration

Docker Security

# docker-compose.yml (future)
services:
  orchestrator:
    security_opt:
      - no-new-privileges:true
    read_only: true
    tmpfs:
      - /tmp
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - DAC_OVERRIDE

Git Security

# Prevent pushing to main branch
git config --global branch.main.pushRemote
# Enable commit signing
git config commit.gpgsign true

Ollama Security

# Restrict network access
# Run in isolated network
# Enable model verification

Monitoring and Alerting

Current Monitoring

  • In-memory logging
  • Console output

Future Enhancements

  • Metrics: Prometheus endpoint
  • Logs: ELK stack or Loki
  • Alerts: PagerDuty integration
  • Dashboards: Grafana visualization

Security Metrics

  • Failed authentication attempts
  • Blocked commands (whitelist violations)
  • Rejected patches (security concerns)
  • Container escape attempts
  • Resource limit violations

Future Security Enhancements

Short Term (0-3 months)

  1. Implement input validation module
  2. Implement output validation module
  3. Add comprehensive audit logging
  4. Implement RBAC system
  5. Add static analysis to CI/CD

Medium Term (3-6 months)

  1. Implement TLS for remote operations
  2. Add certificate pinning
  3. Implement anomaly detection
  4. Add security scanning pipeline
  5. Implement secret scanning

Long Term (6-12 months)

  1. Implement formal security certification (SOC 2, ISO 27001)
  2. Add third-party security audits
  3. Implement bug bounty program
  4. Add threat hunting capabilities
  5. Implement security information and event management (SIEM)

Security Contacts

References

There aren't any published security advisories