Improve memory management

[Spacechild1]

EEL2's memory management is rather dodgy. When the script accesses a memory address and the page hasn't been allocated, it will just cmalloc() it on the fly! Turns out EEL2 is not magic after all...

I've added printf statements in the ELL2 code to track all calls to calloc() and free().

I've used the following test code:

(
DynGenDef(\memTest, "
@init
buf = 0;
i = 0;

@sample
v = buf[i];
i += 1;
i = min(i, 80000);
(i % 1024) == 0 ? print(i);
out0 = 0;
").send;
)

{ DynGen.ar(1, \memTest) }.play;

This is the output:

click me

DynGen
EEL2Adapter
EEL2Adapter: init
malloc() __newBlock_align()
malloc() __newBlock_align()
malloc() __newBlock_align()
free() freeBlocks
malloc() __newBlock_align()
malloc() __newBlock_align()
free() freeBlocks
EEL2Adapter: init done
EEL2Adapter: @init section
EEL2Adapter: @init section done
__NSEEL_RAMAlloc
EnterMutex
calloc() block
memused: 524288
LeaveMutex
0
1024
2048
3072
4096
5120
6144
7168
8192
9216
10240
11264
12288
13312
14336
15360
16384
17408
18432
19456
20480
21504
22528
23552
24576
25600
26624
27648
28672
29696
30720
31744
32768
33792
34816
35840
36864
37888
38912
39936
40960
41984
43008
44032
45056
46080
47104
48128
49152
50176
51200
52224
53248
54272
55296
56320
57344
58368
59392
60416
61440
62464
63488
64512
__NSEEL_RAMAlloc
EnterMutex
calloc() block
memused: 1048576
LeaveMutex
65536
66560
67584
68608
69632
70656
71680
72704
73728
74752
75776
76800
77824
78848
79872
~DynGen
delete VM
~EEL2Adapter
free() freeBlocks
free() freeBlocks
free() __growbuf_resize
NSEEL_VM_FreeRAM
free() block
free() block
free() freeBlocks

As you can see, calloc() is called the first time we access the memory at index 0. This will allocate a "page" of 65536 items (= 512 KB). Once we reach buffer index 65536, the next page is allocated.

Obviously, this is all but realtime safe.

Also, I don't understand why __NSEEL_RAMAlloc needs to call the global mutex. All operations are local to the VM. The only global state is the memory size counter (which could be made atomic). Funnily enough, NSEEL_VM_freeRAM does not use the mutex at all (but NSEEL_VM_freeRAMIfCodeRequested does.)

---

I see the following potential improvements:

1. do not lock the global mutex when allocating RAM for the VM

2. allow to set an allocator function that is used for allocating memory pages. Two possible strategies:

   1. use our own global memory pool, possibly protecting it with a spinlock. Actually, since we always (de)allocate the same number of bytes, this could be simple freelist with a number of preallocated nodes.
   2. use RTAlloc and RTFree.
      This would be probably be the nicer option, but we'd have to be careful that these functions are called on the correct thread. AFAICT blocks should only be allocated on the RT thread (via __NSEEL_RAMAlloc in one of the code sections) whereas blocks might be freed on either the RT thread (via NSEEL_VM_freeRAMIfCodeRequested) or on the NRT thread (via NSEEL_VM_freeRAM when the VM is freed). EEL2 would have to tell our allocator in which context a block is freed.

Obviously, all of this can only be done by maintaining our own patched version of EEL2.

---

In the meantime, one thing we could do is pre-allocate one page of memory after we've created the VM, so at least we don't have to hit the memory allocator for first 65536 items. (This can be done with the NSEEL_VM_preallocram function.) The downside is that every DynGen instance would use 512 KB of memory. That's quite a lot of memory if you have many instances. On the other hand, most of that memory will never be touched and modern machines have plenty of RAM. I just leave this up for discussion.

Of course, none of this is urgent. I just wanted to put everything down so I don't forget it.

[capital-G]

Thanks for looking into this! :)

Maybe we can also use something discussed like in #93 (comment) - e.g. // @memorySize: 512kb? Though this size is most likely to be srate dependent :/

The interesting thing seems to me: alloc is only called when the memory is actually accessed? so a delayed init? This could actually be useful for us since we are running at this point in the RT thread and could catch this by using RTAlloc?

[Spacechild1]

Maybe we can also use something discussed like in #93 (comment) - e.g. // @memorySize: 512kb? Though this size is most likely to be srate dependent :/

Yes, but I think this would put too much burden on the user. Also, the required amount of memory might only be known at runtime.

The interesting thing seems to me: alloc is only called when the memory is actually accessed? so a delayed init?

Yes, the memory is only allocated on demand.

This could actually be useful for us since we are running at this point in the RT thread and could catch this by using RTAlloc?

Yes, that's what I suggested in the PR description. Using RTAlloc would be the most transparent solution. The user only has to make sure that the Server has enough RT memory. (With supercollider/supercollider#7111 this should be less of an issue since scsynth would use 256 MB of RT memory by default.) The only issue with using RTAlloc is that we have to zero the memory. I have to measure how long it takes to memset 512 KB of (paged-in) memory. The absolute minimum would be 8 us, but realistically it's probably quite a bit more than that. This could become a problem if dozens of instances happen to allocate at the same time.

---

The alternative would be to use our own memory pool. This would also be faster because we can use a simple (lockfree) pool allocator. (ELL2 memory blocks always have the same size, namely 65536 * sizeof(double).) To allocate a block, we only have to pop an item from the freelist, which is always O(1). Also, we could clear the blocks on the NRT thread when we return them to the freelist, so that our allocation function would return blocks that are already zeroed. Finally, we would never run out of memory because in the worst case we can get new blocks from the heap.

The big question would be: how many blocks should we preallocate? Should this be settable by the user? If yes, how?

[Spacechild1]

Just a note to myself: I want to look if we can make the actual VM creation real-time safe by deferring all memory allocations to our RT memory allocator. Ideally, we can get rid of the sync argument entirely and always create/destroy the VM synchronously on the RT thread. I need to have a closer look at the EEL2 implementation to see if this is actually possible/practical.