Basic Loop Implementation with Iteration Counts

[maycon]

Basic Loop Implementation with Iteration Counts

• Epic: Advanced Chain States: Loops, Conditionals, and Parallel Execution #3
• Phase: 1 - Core Loop Support (v0.1)
• Priority: High
• Estimated Effort: 3-5 days

Summary

Implement basic loop functionality that allows executing a state multiple times with a fixed iteration count.

Motivation

Currently, testing scenarios that require repeating requests (e.g., "redeem coupon 10 times") requires duplicating state definitions or running the attack multiple times manually. This leads to:

• Verbose, hard-to-maintain YAML configurations
• Inability to model brute-force or exhaustive testing scenarios
• Poor user experience for common attack patterns

Proposed Syntax

states:
  redeem_multiple_times:
    description: "Attempt to redeem coupon multiple times"
    request: |
      POST /api/redeem-coupon
      Authorization: Bearer {{ token }}

      {"coupon_code": "SUMMER2024"}
    
    loop:
      iterations: 10
    
    next:
      - on_status: 200
        goto: verify_redemptions
      - on_loop_complete:
        goto: end

Requirements

Functional Requirements

1. Fixed Iteration Count

   • Support iterations parameter (integer, minimum 1)
   • Execute state exactly N times
   • Maintain execution order (sequential by default)

2. Loop Completion Detection

   • Trigger on_loop_complete transition when all iterations finish
   • Support mixing loop completion with status-based transitions

3. Request Execution

   • Each iteration should render the request template independently
   • Support using extracted variables from previous iterations
   • Maintain separate response context for each iteration

Non-Functional Requirements

1. Performance

   • Minimal overhead compared to manually duplicated states
   • Memory-efficient for loops with 1000+ iterations
   • Support for streaming results (don't load all responses in memory)

2. Error Handling

   • Continue loop on errors by default
   • Support stop_on_error flag (optional)
   • Collect and report all errors at loop completion

3. Observability

   • Log each iteration with iteration number
   • Track success/failure rate across iterations
   • Report timing statistics (min, max, avg, p95, p99)

Implementation Details

State Configuration Schema

{
  "properties": {
    "loop": {
      "type": "object",
      "properties": {
        "iterations": {
          "type": "integer",
          "minimum": 1,
          "maximum": 10000,
          "description": "Number of times to execute this state"
        },
        "stop_on_error": {
          "type": "boolean",
          "default": false,
          "description": "Stop loop if any request fails"
        },
        "parallel": {
          "type": "boolean",
          "default": false,
          "description": "Execute iterations in parallel (Phase 4 feature)"
        }
      },
      "required": ["iterations"]
    }
  }
}

Internal Execution Flow

class LoopExecutor:
    def execute_loop_state(self, state_config, context):
        loop_config = state_config.get('loop', {})
        iterations = loop_config.get('iterations')
        stop_on_error = loop_config.get('stop_on_error', False)
        
        results = []
        for i in range(iterations):
            # Update loop context
            context['loop'] = {
                'index': i,
                'iteration': i + 1  # Reserved for future
            }
            
            # Execute single iteration
            try:
                result = self.execute_single_request(state_config, context)
                results.append(result)
                
                # Check for early exit conditions
                if stop_on_error and not result.success:
                    break
                    
            except Exception as e:
                if stop_on_error:
                    raise
                results.append(LoopIterationError(iteration=i, error=e))
        
        # Determine next transition
        return self.resolve_loop_transition(state_config, results, context)

Transition Logic

1. Status-based transitions checked on LAST iteration only
2. on_loop_complete triggered after all iterations
3. Priority: status transitions > loop_complete > default next

def resolve_loop_transition(self, state_config, results, context):
    last_result = results[-1]
    
    # Check status-based transitions using last iteration
    for transition in state_config.get('next', []):
        if 'on_status' in transition:
            if last_result.status == transition['on_status']:
                return transition['goto']
    
    # Check loop_complete transition
    for transition in state_config.get('next', []):
        if 'on_loop_complete' in transition:
            return transition['goto']
    
    # Default
    return state_config.get('next', [{}])[0].get('goto')

Testing Requirements

Unit Tests

• Loop executes exactly N iterations
• stop_on_error stops on first failure
• Loop continues on error when stop_on_error=false
• Transitions resolve correctly after loop completion
• Loop with iterations=1 works correctly
• Large loop (1000+ iterations) completes without memory issues

Integration Tests

• Loop with HTTP requests executes successfully
• Extracted variables available in subsequent iterations
• Loop statistics reported correctly
• Error collection works across iterations

Performance Tests

• 1000 iterations complete in reasonable time (<5% overhead)
• Memory usage remains stable across iterations
• No memory leaks in long-running loops

Output Example

======================================================================
LOOP STATE: redeem_multiple_times
======================================================================
Iterations: 10
Stop on error: false
======================================================================

[Iteration 1/10] Status: 200, Time: 45.2ms ✓
[Iteration 2/10] Status: 200, Time: 46.1ms ✓
[Iteration 3/10] Status: 200, Time: 44.8ms ✓
[Iteration 4/10] Status: 409, Time: 43.2ms ✗ (Conflict)
[Iteration 5/10] Status: 409, Time: 42.9ms ✗ (Conflict)
...

======================================================================
LOOP RESULTS
======================================================================
Total iterations: 10
Successful: 3 (30%)
Failed: 7 (70%)

Timing Statistics:
  Average: 45.1ms
  Min: 42.9ms
  Max: 48.3ms
  p95: 47.8ms
  p99: 48.3ms

⚠ VULNERABLE: 3 successful redemptions detected!
  Expected: 1 (first request should succeed, rest should fail)
  Actual: 3 (race condition or validation bug)
======================================================================

Documentation Updates

• Add "Loops" section to README
• Create tutorial: "Testing with Loops"
• Add example: Rate limit testing
• Add example: Brute force attack simulation
• Update JSON schema documentation

Migration Path

Existing configurations continue to work unchanged. New loop syntax is opt-in.

Before (duplicated states):

states:
  redeem_1: { request: "...", next: [{goto: redeem_2}] }
  redeem_2: { request: "...", next: [{goto: redeem_3}] }
  redeem_3: { request: "...", next: [{goto: end}] }

After (with loops):

states:
  redeem_multiple:
    request: "..."
    loop:
      iterations: 3
    next:
      - on_loop_complete:
        goto: end

Acceptance Criteria

• loop.iterations parameter works as documented
• All iteration execute sequentially in order
• on_loop_complete transition triggers correctly
• stop_on_error flag works as expected
• Loop statistics are logged and reported
• Performance overhead is <5% vs duplicated states
• Documentation includes 2+ working examples
• All tests pass

Follow-up Issues

• #XX - Loop Context Variables (index, counter, first, last) - Phase 1.2
• #XX - Early Loop Exit with break_on Conditions - Phase 1.3
• #XX - Inter-iteration Delays - Phase 1.4

Related Issues

• Depends on: None (standalone feature)
• Blocks: All other Phase 1 issues
• Related to: JSON Schema Validation (#XX)