🛡️ Sentinel: [CRITICAL] Fix Rate Limiter Bypass / IP Spoofing via X-Forwarded-For (#285)

* 🛡️ Sentinel: [CRITICAL] Fix Rate Limiter Bypass / IP Spoofing via X-Forwarded-For

Adds ProxyHeadersMiddleware properly scoped with TRUSTED_PROXIES to correctly extract the true client IP while avoiding trivial IP spoofing by untrusted proxies. Separated from ALLOWED_HOSTS which should only be used for TrustedHostMiddleware domain-matching.

Co-authored-by: lgcorzo <46710567+lgcorzo@users.noreply.github.com>

* 🛡️ Sentinel: [CRITICAL] Fix Rate Limiter Bypass / IP Spoofing via X-Forwarded-For

Adds ProxyHeadersMiddleware properly scoped with TRUSTED_PROXIES to correctly extract the true client IP while avoiding trivial IP spoofing by untrusted proxies. Separated from ALLOWED_HOSTS which should only be used for TrustedHostMiddleware domain-matching. Fixes CI failures.

Co-authored-by: lgcorzo <46710567+lgcorzo@users.noreply.github.com>

---------

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>

Author: lgcorzo

.jules/sentinel.md

@@ -57,3 +57,18 @@
 **Vulnerability:** The `/predict` endpoint enforced a maximum row limit but did not limit the number of columns, allowing an attacker to submit wide payloads causing memory exhaustion and Algorithmic DoS.
 **Learning:** When validating tabular or matrix-like data inputs, limits must be applied to all dimensions (both rows and columns) to properly bound memory usage.
 **Prevention:** Always define and enforce strict limits on both row and column counts for incoming data structures.
+
+## 2026-12-06 - Rate Limiter Bypass / DoS via Proxy IP Masking
+**Vulnerability:** The in-memory rate limiter relies on `request.client.host` which yields the proxy's IP address instead of the real client IP.
+**Learning:** When an application is deployed behind a load balancer or reverse proxy, `request.client.host` limits all users connecting through the proxy together, enabling an attacker to inadvertently block all users or bypass rate limiting.
+**Prevention:** Always use `X-Forwarded-For` (or equivalent proxy headers) to extract the actual client IP for rate limiting and auditing.
+
+## 2026-12-07 - Rate Limiter Bypass / IP Spoofing via X-Forwarded-For
+**Vulnerability:** Manually parsing the `X-Forwarded-For` header to determine the client IP allows an attacker to trivially spoof their IP address, bypassing the rate limiter and potentially causing DoS.
+**Learning:** Blindly trusting `X-Forwarded-For` without validating that the immediate connection comes from a trusted proxy allows IP spoofing.
+**Prevention:** Always use `ProxyHeadersMiddleware` (or equivalent secure middleware) configured with a strict list of trusted proxies (`trusted_hosts`) to securely extract the real client IP.
+
+## 2026-12-08 - Misconfigured ProxyHeadersMiddleware (Trusting ALLOWED_HOSTS)
+**Vulnerability:** `ProxyHeadersMiddleware` was configured using `ALLOWED_HOSTS` instead of a separate list of trusted proxy IPs. Since `ALLOWED_HOSTS` usually contains domain names (or `*`), this causes the middleware to either fail or blindly trust all `X-Forwarded-For` headers, leading to IP Spoofing.
+**Learning:** `TrustedHostMiddleware` uses domain names to validate the HTTP `Host` header, whereas `ProxyHeadersMiddleware` requires the IP addresses of trusted upstream proxies to securely parse `X-Forwarded-For`. Reusing the same variable conflates these two distinct security mechanisms.
+**Prevention:** Always define a separate `TRUSTED_PROXIES` configuration variable (defaulting to `127.0.0.1`) specifically for `ProxyHeadersMiddleware`.

src/regression_model_template/controller/kafka_app.py

@@ -16,6 +16,7 @@
 from fastapi import FastAPI, HTTPException, Request
 from fastapi.middleware.cors import CORSMiddleware
 from fastapi.middleware.trustedhost import TrustedHostMiddleware
+from uvicorn.middleware.proxy_headers import ProxyHeadersMiddleware
 from pydantic import BaseModel, field_validator
 
 from regression_model_template.core.schemas import InputsSchema, Outputs
@@ -40,6 +41,7 @@
 # Security Configuration
 ALLOWED_HOSTS = os.getenv("ALLOWED_HOSTS", "*").split(",")
 ALLOWED_ORIGINS = os.getenv("ALLOWED_ORIGINS", "*").split(",")
+TRUSTED_PROXIES = os.getenv("TRUSTED_PROXIES", "127.0.0.1").split(",")
 
 # Configure logging
 logging.basicConfig(level=logging.INFO, format=LOGGING_FORMAT)
@@ -53,6 +55,8 @@
 )
 
 # Security Middlewares
+app.add_middleware(ProxyHeadersMiddleware, trusted_hosts=TRUSTED_PROXIES)  # type: ignore[arg-type]
+
 app.add_middleware(TrustedHostMiddleware, allowed_hosts=ALLOWED_HOSTS)
 
 app.add_middleware(

tests/controller/test_kafka_app_dos.py

@@ -68,3 +68,50 @@ def test_prediction_request_inconsistent_lengths():
     with pytest.raises(ValidationError) as excinfo:
         PredictionRequest(input_data=input_data)
     assert "All columns must have the same length" in str(excinfo.value)
+
+
+@pytest.mark.asyncio
+async def test_predict_rate_limiter_uses_proxy_middleware():
+    """Test that ProxyHeadersMiddleware correctly modifies the ASGI scope for trusted proxies."""
+    from regression_model_template.controller.kafka_app import app
+    from uvicorn.middleware.proxy_headers import ProxyHeadersMiddleware
+
+    proxy_middleware_config = next(m for m in app.user_middleware if m.cls == ProxyHeadersMiddleware)
+    assert proxy_middleware_config.kwargs["trusted_hosts"] == ["127.0.0.1"]
+
+    async def mock_app(scope, receive, send):
+        pass  # We just want to inspect the scope after middleware
+
+    middleware_instance = ProxyHeadersMiddleware(app=mock_app, trusted_hosts=["10.0.0.1"])
+
+    # Simulate request from a trusted proxy (10.0.0.1) passing a spoofed client (203.0.113.1)
+    scope = {
+        "type": "http",
+        "client": ("10.0.0.1", 12345),
+        "headers": [(b"x-forwarded-for", b"203.0.113.1, 198.51.100.1")],
+    }
+
+    async def mock_receive():
+        return {"type": "http.request"}
+
+    async def mock_send(msg):
+        pass
+
+    await middleware_instance(scope, mock_receive, mock_send)
+
+    # Assert the middleware successfully updated the client IP since it trusted the proxy
+    # Note: Uvicorn's ProxyHeadersMiddleware sets the client IP to the right-most IP in X-Forwarded-For
+    # that is NOT a trusted proxy. Here we only trusted 10.0.0.1, so it picked 198.51.100.1.
+    assert scope["client"][0] == "198.51.100.1"
+
+    # Simulate request from an untrusted proxy (192.168.1.1)
+    scope_untrusted = {
+        "type": "http",
+        "client": ("192.168.1.1", 12345),
+        "headers": [(b"x-forwarded-for", b"203.0.113.1")],
+    }
+
+    await middleware_instance(scope_untrusted, mock_receive, mock_send)
+
+    # Assert the middleware ignored the header and kept the untrusted IP
+    assert scope_untrusted["client"][0] == "192.168.1.1"