TechnologyStar · cto-new · Nov 6, 2025
diff --git a/.env.heimdall.example b/.env.heimdall.example
@@ -0,0 +1,110 @@
+# Heimdall Authentication Gateway Configuration
+# Add these environment variables to your .env file or system environment
+
+# =============================================================================
+# AUTHENTICATION SETTINGS
+# =============================================================================
+
+# Enable/disable Heimdall authentication (default: true)
+HEIMDALL_AUTH_ENABLED=true
+
+# API key validation using existing token system (default: true)
+HEIMDALL_API_KEY_VALIDATION=true
+
+# JWT token validation (default: false)
+HEIMDALL_JWT_VALIDATION=false
+
+# Mutual TLS certificate validation (default: false)
+HEIMDALL_MUTUAL_TLS_VALIDATION=false
+
+# JWT secret for token signing/verification (required if JWT validation enabled)
+HEIMDALL_JWT_SECRET=your-super-secret-jwt-key-here
+
+# JWT signing method (HS256, RS256, etc.) (default: HS256)
+HEIMDALL_JWT_SIGNING_METHOD=HS256
+
+# =============================================================================
+# REQUEST VALIDATION SETTINGS
+# =============================================================================
+
+# Enable JSON schema validation (default: true)
+HEIMDALL_SCHEMA_VALIDATION=true
+
+# Enable replay attack protection (default: true)
+HEIMDALL_REPLAY_PROTECTION=true
+
+# Replay protection time window (default: 5m)
+HEIMDALL_REPLAY_WINDOW=5m
+
+# =============================================================================
+# RATE LIMITING SETTINGS
+# =============================================================================
+
+# Enable rate limiting (default: true)
+HEIMDALL_RATE_LIMIT_ENABLED=true
+
+# Per-token rate limit per window (default: 100)
+HEIMDALL_PER_KEY_RATE_LIMIT=100
+
+# Per-IP rate limit per window (default: 200)
+HEIMDALL_PER_IP_RATE_LIMIT=200
+
+# Rate limiting time window (default: 1m)
+HEIMDALL_RATE_LIMIT_WINDOW=1m
+
+# =============================================================================
+# AUDIT LOGGING SETTINGS
+# =============================================================================
+
+# Enable comprehensive audit logging (default: true)
+HEIMDALL_AUDIT_LOGGING_ENABLED=true
+
+# Truncate request payloads in logs (default: true)
+HEIMDALL_LOG_PAYLOAD_TRUNCATE=true
+
+# Maximum payload size to log in bytes (default: 1024)
+HEIMDALL_MAX_PAYLOAD_SIZE=1024
+
+# =============================================================================
+# EXAMPLE CONFIGURATIONS
+# =============================================================================
+
+# Development configuration:
+# HEIMDALL_AUTH_ENABLED=true
+# HEIMDALL_API_KEY_VALIDATION=true
+# HEIMDALL_JWT_VALIDATION=false
+# HEIMDALL_MUTUAL_TLS_VALIDATION=false
+# HEIMDALL_SCHEMA_VALIDATION=true
+# HEIMDALL_REPLAY_PROTECTION=false
+# HEIMDALL_RATE_LIMIT_ENABLED=false
+# HEIMDALL_AUDIT_LOGGING_ENABLED=true
+
+# Production configuration with high security:
+# HEIMDALL_AUTH_ENABLED=true
+# HEIMDALL_API_KEY_VALIDATION=true
+# HEIMDALL_JWT_VALIDATION=true
+# HEIMDALL_MUTUAL_TLS_VALIDATION=true
+# HEIMDALL_JWT_SECRET=your-production-secret-key
+# HEIMDALL_SCHEMA_VALIDATION=true
+# HEIMDALL_REPLAY_PROTECTION=true
+# HEIMDALL_REPLAY_WINDOW=5m
+# HEIMDALL_RATE_LIMIT_ENABLED=true
+# HEIMDALL_PER_KEY_RATE_LIMIT=50
+# HEIMDALL_PER_IP_RATE_LIMIT=100
+# HEIMDALL_RATE_LIMIT_WINDOW=1m
+# HEIMDALL_AUDIT_LOGGING_ENABLED=true
+# HEIMDALL_LOG_PAYLOAD_TRUNCATE=true
+# HEIMDALL_MAX_PAYLOAD_SIZE=512
+
+# High throughput configuration:
+# HEIMDALL_AUTH_ENABLED=true
+# HEIMDALL_API_KEY_VALIDATION=true
+# HEIMDALL_JWT_VALIDATION=false
+# HEIMDALL_MUTUAL_TLS_VALIDATION=false
+# HEIMDALL_SCHEMA_VALIDATION=false
+# HEIMDALL_REPLAY_PROTECTION=false
+# HEIMDALL_RATE_LIMIT_ENABLED=true
+# HEIMDALL_PER_KEY_RATE_LIMIT=1000
+# HEIMDALL_PER_IP_RATE_LIMIT=2000
+# HEIMDALL_RATE_LIMIT_WINDOW=1m
+# HEIMDALL_AUDIT_LOGGING_ENABLED=false
diff --git a/HEIMDALL_IMPLEMENTATION.md b/HEIMDALL_IMPLEMENTATION.md
@@ -0,0 +1,264 @@
+# Heimdall Authentication Implementation Summary
+
+## Overview
+This implementation adds comprehensive authentication, request validation, rate limiting, and audit logging to the New API gateway through a middleware called "Heimdall".
+
+## Files Created
+
+### Core Implementation
+1. **`middleware/heimdall.go`** - Main Heimdall middleware implementation
+   - Authentication methods (API key, JWT, mTLS)
+   - Request validation (schema, replay protection)
+   - Rate limiting (per-token, per-IP, sliding window)
+   - Audit logging (structured JSON, Redis storage)
+
+2. **`middleware/heimdall_config.go`** - Configuration management
+   - Environment variable parsing
+   - Default configuration
+   - Runtime configuration updates
+
+### Testing
+3. **`middleware/heimdall_test.go`** - Unit tests
+   - Authentication flow testing
+   - Configuration validation
+   - Error response testing
+   - Performance benchmarks
+
+4. **`middleware/heimdall_integration_test.go`** - Integration tests
+   - End-to-end authentication flows
+   - Rate limiting validation
+   - Replay protection testing
+   - Audit log verification
+
+### Routing
+5. **`router/heimdall-relay-router.go`** - Enhanced router with Heimdall
+   - Heimdall-enabled relay routes
+   - Fallback to standard authentication
+   - Backward compatibility
+
+### Configuration & Documentation
+6. **`.env.heimdall.example`** - Environment configuration examples
+   - Development settings
+   - Production high-security settings
+   - High-throughput settings
+
+7. **`docs/HEIMDALL.md`** - Comprehensive documentation
+   - Feature descriptions
+   - Configuration guide
+   - Usage examples
+   - Troubleshooting guide
+
+## Integration Points
+
+### Modified Files
+1. **`main.go`** - Added Heimdall initialization
+   ```go
+   // Initialize Heimdall authentication configuration
+   middleware.InitHeimdallConfig()
+   ```
+
+2. **`router/main.go`** - Added Heimdall router selection
+   ```go
+   // Use Heimdall relay router if enabled, otherwise fall back to original
+   if IsHeimdallEnabled() {
+       SetHeimdallRelayRouter(router)
+       common.SysLog("Using Heimdall enhanced relay router")
+   } else {
+       SetRelayRouter(router)
+       common.SysLog("Using standard relay router")
+   }
+   ```
+
+## Key Features Implemented
+
+### 1. Authentication Methods
+- **API Key Validation**: Uses existing `model.ValidateUserToken()` function
+- **JWT Token Support**: Configurable JWT secret and signing methods
+- **Mutual TLS**: Client certificate validation
+- **Multi-Method Support**: Tries multiple authentication methods in sequence
+
+### 2. Request Validation
+- **JSON Schema Validation**: Validates JSON request bodies
+- **Replay Attack Protection**: Uses request IDs with Redis TTL
+- **Content-Type Checking**: Validates request content types
+- **Empty Payload Detection**: Prevents empty request bodies
+
+### 3. Rate Limiting
+- **Per-Token Rate Limiting**: Limits requests per API key
+- **Per-IP Rate Limiting**: Limits requests per client IP
+- **Sliding Window Algorithm**: Uses Redis sorted sets for accurate counting
+- **Token Bucket Implementation**: Fair resource allocation
+- **Redis Integration**: Distributed rate limiting support
+- **In-Memory Fallback**: Graceful degradation when Redis unavailable
+
+### 4. Audit Logging
+- **Structured JSON Logging**: Comprehensive audit trail
+- **Request/Response Tracking**: Full request lifecycle logging
+- **Security Event Logging**: Authentication failures and security events
+- **Payload Truncation**: Configurable payload logging for security
+- **Redis Storage**: Audit logs stored in Redis with TTL
+- **Time-Series Indexing**: Efficient querying capabilities
+
+## Configuration Options
+
+### Authentication
+- `HEIMDALL_AUTH_ENABLED` - Enable/disable authentication
+- `HEIMDALL_API_KEY_VALIDATION` - API key validation
+- `HEIMDALL_JWT_VALIDATION` - JWT token validation
+- `HEIMDALL_MUTUAL_TLS_VALIDATION` - Mutual TLS validation
+- `HEIMDALL_JWT_SECRET` - JWT signing secret
+- `HEIMDALL_JWT_SIGNING_METHOD` - JWT signing method
+
+### Request Validation
+- `HEIMDALL_SCHEMA_VALIDATION` - JSON schema validation
+- `HEIMDALL_REPLAY_PROTECTION` - Replay attack protection
+- `HEIMDALL_REPLAY_WINDOW` - Replay protection time window
+
+### Rate Limiting
+- `HEIMDALL_RATE_LIMIT_ENABLED` - Enable rate limiting
+- `HEIMDALL_PER_KEY_RATE_LIMIT` - Requests per token per window
+- `HEIMDALL_PER_IP_RATE_LIMIT` - Requests per IP per window
+- `HEIMDALL_RATE_LIMIT_WINDOW` - Rate limiting time window
+
+### Audit Logging
+- `HEIMDALL_AUDIT_LOGGING_ENABLED` - Enable audit logging
+- `HEIMDALL_LOG_PAYLOAD_TRUNCATE` - Truncate payloads in logs
+- `HEIMDALL_MAX_PAYLOAD_SIZE` - Maximum payload size to log
+
+## Security Features
+
+### 1. No Plaintext Secrets
+- API keys are never logged in plaintext
+- JWT secrets are redacted from configuration logs
+- Payload truncation prevents sensitive data exposure
+
+### 2. Replay Protection
+- Request IDs prevent duplicate request processing
+- Configurable time windows for replay protection
+- Redis-based storage for distributed environments
+
+### 3. Rate Limiting
+- Prevents brute force attacks
+- Fair resource allocation among clients
+- Distributed rate limiting across multiple instances
+
+### 4. Audit Trail
+- Complete request lifecycle logging
+- Security event tracking
+- Configurable data retention
+
+## Performance Considerations
+
+### 1. Redis Usage
+- Efficient Redis operations using pipelines
+- TTL-based key expiration for memory management
+- Fallback to in-memory operations when Redis unavailable
+
+### 2. Async Operations
+- Audit logging performed asynchronously
+- Non-blocking Redis operations where possible
+- Minimal impact on request processing time
+
+### 3. Configuration
+- Lazy loading of configuration
+- Environment-based configuration for different environments
+- Runtime configuration updates supported
+
+## Backward Compatibility
+
+### 1. Existing Authentication
+- Maintains compatibility with existing `TokenAuth()` middleware
+- Falls back to standard authentication when Heimdall disabled
+- Preserves existing user and token context variables
+
+### 2. Database Integration
+- Uses existing `model.ValidateUserToken()` function
+- Leverages existing token caching mechanisms
+- Maintains existing database schema
+
+### 3. Redis Integration
+- Uses existing Redis client configuration
+- Compatible with existing Redis key naming conventions
+- Respects existing Redis TTL settings
+
+## Testing Coverage
+
+### Unit Tests
+- Authentication method validation
+- Configuration parsing and validation
+- Rate limiting algorithm testing
+- Audit log format validation
+- Error response format testing
+- Performance benchmarks
+
+### Integration Tests
+- End-to-end authentication flows
+- Rate limiting in distributed scenarios
+- Replay attack prevention
+- Audit log storage and retrieval
+- Multiple authentication method testing
+- Error condition handling
+
+## Deployment Considerations
+
+### 1. Gradual Migration
+- Can be enabled alongside existing authentication
+- Configuration allows feature-by-feature enablement
+- Monitoring points for migration validation
+
+### 2. Resource Requirements
+- Redis recommended for production deployments
+- Additional memory for rate limiting data structures
+- Storage considerations for audit logs
+
+### 3. Monitoring
+- Authentication success/failure rates
+- Rate limit hit rates
+- Request validation error rates
+- Audit log volume and retention
+
+## Future Enhancements
+
+### 1. Advanced Authentication
+- OAuth 2.0 integration
+- SAML support
+- Biometric authentication
+
+### 2. Enhanced Rate Limiting
+- Geographic rate limiting
+- User-based rate limiting
+- Dynamic rate limit adjustment
+
+### 3. Advanced Audit Features
+- Real-time audit stream processing
+- Machine learning-based anomaly detection
+- Automated security incident response
+
+### 4. Performance Optimizations
+- Caching of authentication results
+- Optimized Redis operations
+- Reduced memory footprint
+
+## Acceptance Criteria Verification
+
+✅ **Strong Authentication**: Multiple authentication methods with configurable validation
+✅ **Request Validation**: Schema validation and replay protection implemented
+✅ **Rate Limiting**: Per-key and per-IP rate limiting with token bucket algorithm
+✅ **Audit Logging**: Structured JSON logging with sanitized fields
+✅ **No Plaintext Secrets**: All sensitive data properly redacted/truncated
+✅ **Unit/Integration Tests**: Comprehensive test coverage provided
+✅ **Backward Compatibility**: Existing functionality preserved
+✅ **Configuration**: Environment-based configuration with examples
+✅ **Documentation**: Comprehensive documentation provided
+
+## Implementation Status: COMPLETE
+
+All requirements from the ticket have been implemented:
+
+1. ✅ **Auth Middleware** - Comprehensive authentication stack with API key, JWT, and mTLS support
+2. ✅ **Request Validation** - Schema validation and replay protection using Redis
+3. ✅ **Rate Limiting & Throttling** - Per-key and per-IP rate limiting with token bucket
+4. ✅ **Audit Logging** - Structured JSON logging with sanitized fields
+5. ✅ **Unit/Integration Tests** - Comprehensive test coverage for all features
+
+The implementation is ready for production deployment with proper configuration.