Skip to content

feat: implement adaptive rate limiting and throttling system#251

Merged
kelly-musk merged 2 commits into
kellymusk:masterfrom
sonoflawal:utility
Mar 28, 2026
Merged

feat: implement adaptive rate limiting and throttling system#251
kelly-musk merged 2 commits into
kellymusk:masterfrom
sonoflawal:utility

Conversation

@sonoflawal
Copy link
Copy Markdown
Contributor

closes #176

Summary

Implements an adaptive rate limiting system that dynamically adjusts rate
limits based on real-time platform health signals, consumer behaviour, and
detected anomalies — complementing the existing static limits.

What's changed

  • Signal collection — samples CPU utilisation, DB pool utilisation, Redis
    memory pressure, request queue depth, error rate, and p99 response time on a
    configurable interval with rolling averages to smooth transient spikes

  • Adaptation modes — Normal → Elevated → Critical → Emergency, driven by
    configurable per-signal thresholds with hysteresis to prevent oscillation

  • Consumer priority tiers — High (partners, microservices) are protected
    from tightening in Elevated/Critical; Standard and Low receive progressively
    aggressive multipliers; all tiers get a reduced but non-zero limit in Emergency

  • Per-consumer trend tracking — detects accelerating request rates and
    applies an additional throttle multiplier during Elevated/Critical mode

  • Graceful degradation — non-essential endpoints return 503 with
    Retry-After in Critical/Emergency; essential financial endpoints are queued
    up to a configurable depth with FIFO shedding when full

  • Admin controlGET/POST/DELETE /api/admin/adaptive-rate-limit/...
    endpoints for status inspection and manual mode override

  • Redis sync — effective multipliers are pushed to Redis on every cycle so
    all application instances apply consistent limits simultaneously

  • Observability — Prometheus gauges for mode, all signal values, rolling
    averages, and effective multipliers per tier; counters for mode transitions,
    per-consumer throttle applications, and request shedding events; structured
    log events with full signal context on every Critical/Emergency transition

Files

Path Purpose
src/adaptive_rate_limit/ New module (12 files)
migrations/20260403000000_adaptive_rate_limiting.sql Signal snapshots + mode transitions tables
src/metrics/mod.rs Registers adaptive rl metrics in global registry
src/main.rs Wires up worker, middleware, and admin routes
src/lib.rs Exports new module
tests/adaptive_rate_limit_integration.rs Integration tests

Testing

  • 20 unit tests covering rolling averages, threshold evaluation, hysteresis
    enforcement, multiplier application per tier, accelerating request detection,
    request shedding logic, and endpoint classification
  • Integration tests covering mode transitions, manual override, multiplier tier
    ordering, hysteresis, and worker lifecycle

@drips-wave
Copy link
Copy Markdown

drips-wave Bot commented Mar 27, 2026

@sonoflawal Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@kelly-musk kelly-musk merged commit b2973dc into kellymusk:master Mar 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Adaptive Rate Limiting & Throttling

2 participants