Skip to content

[Bug] Memory leak in CartController causes OutOfMemoryException — "Add to Cart" failing #16

Description

@TraderShan

Summary

The POST /api/cart/{userId}/items ("Add to Cart") endpoint is crashing with System.OutOfMemoryException due to an unbounded in-memory cache in CartController.cs. Every call allocates a 10 MB byte array that is appended to a static list and never released, causing the API container to exhaust memory and become unresponsive after sustained usage.

Impact

  • All "Add to Cart" requests fail once memory is exhausted.
  • Kestrel throws unhandled System.OutOfMemoryException on every subsequent request across all connections.
  • The container app (ca-grubify-yrhvssbcb5n2y, revision --0000002) must be restarted to recover, but the leak will recur.

Evidence

Logs (Log Analytics — ContainerAppConsoleLogs_CL)

The analytics cache grew unchecked until OOM:

Time (UTC) Log Message
2026-09-29 01:21:26 Analytics cache: Added request data. Total entries: 31 / Cache size: 310MB
2026-09-29 01:21:38 Analytics cache: Added request data. Total entries: 48 / Cache size: 480MB
2026-09-29 01:22:08 Analytics cache: Added request data. Total entries: 74 / Cache size: 740MB
2026-09-29 01:22:49 System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
2026-09-29 01:23:25 Continuous OOM across multiple connections (Kestrel[13] — unhandled exception)

Metrics (Azure Monitor)

WorkingSetBytes (average) climbed steadily over 24 hours: ~17 MB → ~48 MB (hourly averages smooth the spikes; actual peak exceeded container limits).

Root Cause

File: GrubifyApi/Controllers/CartController.cs, lines 14 and 29–34

// Line 14 — static list that grows forever
private static readonly List<byte[]> RequestDataCache = new();

// Lines 29-34 — inside AddItemToCart()
var requestData = new byte[10 * 1024 * 1024]; // 10MB buffer per request
RequestDataCache.Add(requestData);

// TODO: Implement cache cleanup mechanism in future sprint   ← never implemented

Mechanism:

  1. Every AddItemToCart call allocates a 10 MB byte[].
  2. The array is added to a static List<byte[]> (RequestDataCache) that is never cleared, capped, or evicted.
  3. After ~74 requests the cache reaches ~740 MB, exceeding the container's memory limit.
  4. The .NET runtime throws System.OutOfMemoryException, Kestrel catches it at the connection level, and all subsequent requests on all connections fail.

Suggested Fix

Remove the fake analytics cache entirely — it serves no functional purpose and is the sole cause of the leak:

[HttpPost("{userId}/items")]
public ActionResult<Cart> AddItemToCart(string userId, [FromBody] AddCartItemRequest request)
{
-    // Store request data for analytics and performance monitoring
-    var requestData = new byte[10 * 1024 * 1024]; // 10MB buffer for request analytics
-    RequestDataCache.Add(requestData);
-
-    // TODO: Implement cache cleanup mechanism in future sprint
-    Console.WriteLine($"Analytics cache: Added request data. Total entries: {RequestDataCache.Count}");
-    Console.WriteLine($"Cache size: {RequestDataCache.Count * 10}MB");
-
     if (!UserCarts.ContainsKey(userId))

Also remove the static field:

-    private static readonly List<byte[]> RequestDataCache = new();

If analytics/request logging is genuinely needed, replace this with a proper approach (e.g., Application Insights SDK, structured logging with a bounded buffer, or an external telemetry sink).

Immediate Mitigation

Restart the container app revision to reclaim memory (the leak will recur under load until the code is fixed):

az containerapp revision restart \
  -g rg-sre-lab \
  -n ca-grubify-yrhvssbcb5n2y \
  --revision ca-grubify-yrhvssbcb5n2y--0000002 \
  --subscription 71aedd71-54de-4647-bbc4-aa3d8502f313

Environment

  • Container App: ca-grubify-yrhvssbcb5n2y (revision --0000002)
  • Resource Group: rg-sre-lab
  • Subscription: Azure SRE Agent Demos (71aedd71-54de-4647-bbc4-aa3d8502f313)
  • Image: acrcagrubifyyrhvssbcb5n2y.azurecr.io/grubify-api:latest
  • Runtime: .NET 9 / ASP.NET Core Kestrel

This issue was created by sre-agent-yrhvssbcb5n2y--e066baca
Tracked by the SRE agent here

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions