Summary
The POST /api/cart/{userId}/items ("Add to Cart") endpoint is crashing with System.OutOfMemoryException due to an unbounded in-memory cache in CartController.cs. Every call allocates a 10 MB byte array that is appended to a static list and never released, causing the API container to exhaust memory and become unresponsive after sustained usage.
Impact
- All "Add to Cart" requests fail once memory is exhausted.
- Kestrel throws unhandled
System.OutOfMemoryException on every subsequent request across all connections.
- The container app (
ca-grubify-yrhvssbcb5n2y, revision --0000002) must be restarted to recover, but the leak will recur.
Evidence
Logs (Log Analytics — ContainerAppConsoleLogs_CL)
The analytics cache grew unchecked until OOM:
| Time (UTC) |
Log Message |
| 2026-09-29 01:21:26 |
Analytics cache: Added request data. Total entries: 31 / Cache size: 310MB |
| 2026-09-29 01:21:38 |
Analytics cache: Added request data. Total entries: 48 / Cache size: 480MB |
| 2026-09-29 01:22:08 |
Analytics cache: Added request data. Total entries: 74 / Cache size: 740MB |
| 2026-09-29 01:22:49 |
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown. |
| 2026-09-29 01:23:25 |
Continuous OOM across multiple connections (Kestrel[13] — unhandled exception) |
Metrics (Azure Monitor)
WorkingSetBytes (average) climbed steadily over 24 hours: ~17 MB → ~48 MB (hourly averages smooth the spikes; actual peak exceeded container limits).
Root Cause
File: GrubifyApi/Controllers/CartController.cs, lines 14 and 29–34
// Line 14 — static list that grows forever
private static readonly List<byte[]> RequestDataCache = new();
// Lines 29-34 — inside AddItemToCart()
var requestData = new byte[10 * 1024 * 1024]; // 10MB buffer per request
RequestDataCache.Add(requestData);
// TODO: Implement cache cleanup mechanism in future sprint ← never implemented
Mechanism:
- Every
AddItemToCart call allocates a 10 MB byte[].
- The array is added to a
static List<byte[]> (RequestDataCache) that is never cleared, capped, or evicted.
- After ~74 requests the cache reaches ~740 MB, exceeding the container's memory limit.
- The .NET runtime throws
System.OutOfMemoryException, Kestrel catches it at the connection level, and all subsequent requests on all connections fail.
Suggested Fix
Remove the fake analytics cache entirely — it serves no functional purpose and is the sole cause of the leak:
[HttpPost("{userId}/items")]
public ActionResult<Cart> AddItemToCart(string userId, [FromBody] AddCartItemRequest request)
{
- // Store request data for analytics and performance monitoring
- var requestData = new byte[10 * 1024 * 1024]; // 10MB buffer for request analytics
- RequestDataCache.Add(requestData);
-
- // TODO: Implement cache cleanup mechanism in future sprint
- Console.WriteLine($"Analytics cache: Added request data. Total entries: {RequestDataCache.Count}");
- Console.WriteLine($"Cache size: {RequestDataCache.Count * 10}MB");
-
if (!UserCarts.ContainsKey(userId))
Also remove the static field:
- private static readonly List<byte[]> RequestDataCache = new();
If analytics/request logging is genuinely needed, replace this with a proper approach (e.g., Application Insights SDK, structured logging with a bounded buffer, or an external telemetry sink).
Immediate Mitigation
Restart the container app revision to reclaim memory (the leak will recur under load until the code is fixed):
az containerapp revision restart \
-g rg-sre-lab \
-n ca-grubify-yrhvssbcb5n2y \
--revision ca-grubify-yrhvssbcb5n2y--0000002 \
--subscription 71aedd71-54de-4647-bbc4-aa3d8502f313
Environment
- Container App:
ca-grubify-yrhvssbcb5n2y (revision --0000002)
- Resource Group:
rg-sre-lab
- Subscription: Azure SRE Agent Demos (
71aedd71-54de-4647-bbc4-aa3d8502f313)
- Image:
acrcagrubifyyrhvssbcb5n2y.azurecr.io/grubify-api:latest
- Runtime: .NET 9 / ASP.NET Core Kestrel
This issue was created by sre-agent-yrhvssbcb5n2y--e066baca
Tracked by the SRE agent here
Summary
The
POST /api/cart/{userId}/items("Add to Cart") endpoint is crashing withSystem.OutOfMemoryExceptiondue to an unbounded in-memory cache inCartController.cs. Every call allocates a 10 MB byte array that is appended to a static list and never released, causing the API container to exhaust memory and become unresponsive after sustained usage.Impact
System.OutOfMemoryExceptionon every subsequent request across all connections.ca-grubify-yrhvssbcb5n2y, revision--0000002) must be restarted to recover, but the leak will recur.Evidence
Logs (Log Analytics —
ContainerAppConsoleLogs_CL)The analytics cache grew unchecked until OOM:
Analytics cache: Added request data. Total entries: 31/Cache size: 310MBAnalytics cache: Added request data. Total entries: 48/Cache size: 480MBAnalytics cache: Added request data. Total entries: 74/Cache size: 740MBSystem.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.Kestrel[13]— unhandled exception)Metrics (Azure Monitor)
WorkingSetBytes(average) climbed steadily over 24 hours: ~17 MB → ~48 MB (hourly averages smooth the spikes; actual peak exceeded container limits).Root Cause
File:
GrubifyApi/Controllers/CartController.cs, lines 14 and 29–34Mechanism:
AddItemToCartcall allocates a 10 MBbyte[].static List<byte[]>(RequestDataCache) that is never cleared, capped, or evicted.System.OutOfMemoryException, Kestrel catches it at the connection level, and all subsequent requests on all connections fail.Suggested Fix
Remove the fake analytics cache entirely — it serves no functional purpose and is the sole cause of the leak:
Also remove the static field:
If analytics/request logging is genuinely needed, replace this with a proper approach (e.g., Application Insights SDK, structured logging with a bounded buffer, or an external telemetry sink).
Immediate Mitigation
Restart the container app revision to reclaim memory (the leak will recur under load until the code is fixed):
Environment
ca-grubify-yrhvssbcb5n2y(revision--0000002)rg-sre-lab71aedd71-54de-4647-bbc4-aa3d8502f313)acrcagrubifyyrhvssbcb5n2y.azurecr.io/grubify-api:latestThis issue was created by sre-agent-yrhvssbcb5n2y--e066baca
Tracked by the SRE agent here