Memory usage with ReadSample

[Ciaran3]

Hi all,
Thanks for the great work on keeping this library maintained! Sorry I have not contributed in a long time..
I have a question regarding using ReadSample on the source reader - with asynchronous mode.
Essentially: (https://docs.microsoft.com/en-us/windows/win32/medfound/using-the-source-reader-in-asynchronous-mode)
I'm just capturing video frames, so I am not selecting the audio stream.

I've read your discussion on the topic:
microsoft/Windows-classic-samples#108
https://social.msdn.microsoft.com/Forums/en-US/c2b39d7b-5203-492c-9663-75d1601f82c7/memory-leak-imfsourcereaderreadsample-method?forum=mediafoundationdevelopment

I'm using critical section and releasing the sample (=nil), but still the memory usage seems to keep going up.. Even after the source reader is flushed and closed the memory usage does not seem to free up completely.

Is this still a known memory issue with media foundation? Or perhaps I'm still missing something.
If the source reader is caching the samples, I would expect these to be released at least on Flush.

Admittedly i'm using version: 2.6.1, so upgrading to 3.0.1 will be my next step, in case some calls have been updated to fix this.

[FactoryXCode]

Hi,
To fix this issue, I implemented MsgWaitForMultipleObjects to give the
ReadSample method more time to organize it's sample pool with in the OnReadSample function, alternatively you can use the Sleep() method.

See: Example MfAudioClipEx (function TAudioClipClass.OnReadSample)

Although I think you need to implement a callback function from IMFMediaEvent, that reports when a sample is processed, before processing the next one, as usual in a-synchronous mode. I think function IMFMediaEvent.GetStatus(out phrStatus: HRESULT) could do the job.

Right now I'm working on a WASAPI component that should render RAW/WAV formats. (See: Examples\MFComponents\WasApiRenderFile.pas. That is not working yet.

Greeting, Tony.

[Ciaran3]

Hi Tony,

Thanks for your reply. Unfortunately, I haven't had any luck with that solution either.
I've even tried testing with the synchronous ReadSample instead of asynchronous, to narrow down the issue - but still no luck completely freeing up the memory.

This example below is the memory in use, when reading a single video frame sample.

• Memory usage before calling ReadSample: 10 MB
• After ReadSample: 39 MB (Makes sense, the sample is in memory)
• After releasing the sample and flushing the source reader: 40 MB (What??)
• After destroying the IMFSourceReader: 16 MB

As you can see, the memory is nearly freed up after destroying the source reader, but not before..
A real shame if the source reader needs to be closed to free up the memory. A use case of requesting multiple video thumbnails, at different positions in the video will be much slower if the source reader needs to be closed and reopened each time.
Unless Media Foundation does it own memory clean up at a certain point? But I cant find any details on that, nor anywhere where I can specify the cache size.

Edit - I can put together a sample project demonstrating this, if that would be of use.

[FactoryXCode]

I agree it's a weird issue. I asked MS about the cpu usage, but that is another thing. However, a debugger like MadExcept does not report any issues about memory leaks, nor does mstrace. Did you close com and MF ?

function CloseMF(): HResult; begin // Shutdown MF Result := MFShutdown(); // Shutdown COM CoUninitialize(); end;

[Ciaran3]

Yeah I guess its not exactly a leak, as the memory is freed eventually, just not when I want (flushing the source) 😞
I'll rework my asynchronous code into the demo project and see if that resolves the issue. Perhaps its an issue with my current implementation 🤞

[Ciaran3]

Hi Tony,

I've added a sample frame capture project here:
https://github.com/Ciaran3/MfPack/tree/Master/MfPack/Samples/MfFrameCapture

Let me know if you see any ways to improve it. Ideally, I'll update it to have an asynchronous capture example as the memory issue does not seem as bad with this synchronous code.

If you think this sample project is of use for MfPack, I can open a pull request 👍

[Ciaran3]

Ah no - its not the D2D1 units (although that does cause confusion).
Its actually RemoveAllBuffers that is the cause - remove that an no error is raised. I remember running into that issue last time I tried that call, so I just dont use it. Even without it memory usage still seems pretty good in my local copy.

[FactoryXCode]

That is good news 👍
Still weird this exception only occurs while reading samples from .wmf files.

[FactoryXCode]

Ah no - its not the D2D1 units (although that does cause confusion)

The MfPack D2D1 API has a different unit scope (WinApi.DirectX.D2D1) to prevent name mangling with the APi included with Delphi (WinApi.D2D1) . Since 2020 D2D1 had major updates that Embarcadero did not implement. There is no way those versions can be mixed unless both are declared, which is not a good idea. Also, older Delphi versions (<2019) don't have this API available. Remember when writing code, it must be possible to compile from Delphi XE2 and up.

[Ciaran3]

Also, older Delphi versions (<2019) don't have this API available. Remember when writing code, it must be possible to compile from Delphi XE2 and up.

ah yes, that does make it tricky. Do you have multiple Delphi versions to check compatibility? Or you just know by now the areas to check? In my own copy I'm just using the WinApi.D2D1 in the SampleConverter unit - but I guess that might cause issues elsewhere if it is missing major updates.

[FactoryXCode]

Do you have multiple Delphi versions to check compatibility?

Yes I do. But usual I check with XE7, that version, in my opinion, is also safe to check for older versions.
Further, you can see at the date in the units, when an API is introduced. For instance the Embarcadero API's D2D1 and DXGI are not working with some samples, like MfThumbnails. Embarcadero has a bad reputation to maintain the API's they release or there is no API at all (like Media Foundation now and DirectShow in the past) or the API is incomplete (WinMM API).

but I guess that might cause issues elsewhere if it is missing major updates.

All API's included with MfPack are up to date with the latest SDK. When Microsoft is releasing a new sample, it almost always need the latest SDK So there's a risk Delphi is missing parts or a whole API.

In the meanwhile I added png and jpg functionality for saving to file. These API's where introduced in Delphi 2009.

I checked the MFSample interface, but according to the vtables in MFPlat.dll, there is nothing wrong in sequence or declaration. So, the RemoveBuffers issue only affects VFW right? Because I did test with AVI and MP4, and there it will not raise any pointer exception.