[Bug] Model.UnloadAsync retains process and CUDA memory after model unload

[natke]

Describe the issue

After loading qwen3.5-2b-text-cuda-gpu:1, attempting inference, and calling IModel.UnloadAsync(), most model-related process and CUDA device memory remains allocated.

IsLoadedAsync() returns false, but measured memory remains close to the peak allocation. The expected behavior is for unloading to release memory that is no longer needed.

In a reproducible run, inference failed with a CUDA allocation error after the model loaded. Unloading afterward retained approximately 98% of the peak process-private allocation and approximately 99% of the peak CUDA allocation. Exact values vary; the issue is the retained proportion after unload.

To reproduce

1. Use Microsoft.AI.Foundry.Local version 2.0.1.
2. Initialize the manager and register CUDAExecutionProvider.
3. Download and select qwen3.5-2b-text-cuda-gpu:1.
4. Record process-private and CUDA device memory.
5. Call LoadAsync() and attempt a chat completion.
6. Call UnloadAsync(), then force managed collection and finalization.
7. Confirm IsLoadedAsync() returns false and measure memory again.

Observed behavior:

MODEL_LOADED_AFTER_UNLOAD=False
RETENTION kind=PROCESS_PRIVATE retained_percent=approximately 98
RETENTION kind=GPU_DEVICE_USED retained_percent=approximately 99

Urgency

Retained allocations prevent applications from reliably recovering memory after unloading a model, especially after an inference allocation failure.

System information

• Platform and architecture: Windows X64
• OS version: Windows 11
• Installation type: Released package/binary
• Foundry Local version: Microsoft.AI.Foundry.Local 2.0.1
• API or surface area: C# SDK
• Hardware acceleration/backend: WebGPU/CUDA
• Backend/runtime version: ONNX Runtime 1.28.0; ONNX Runtime GenAI 0.15.2