Client compilation error

[nnurlan008]

Hi,

I am trying to compile the client using make -j on bks_module/remoteswap/client.
However, I get the following error:

/users/Nurlan/Atlas/atlas-runtime/bks_module/remoteswap/client/rswap_rdma.h:29:10: fatal error: linux/lightnvm.h: No such file or directory
29 | #include <linux/lightnvm.h>
| ^~~~~~~~~~~~~~~~~~
compilation terminated.

Any hint on this?

Update: solved by editing the KDIR line on makefile of client.

Thanks

[nnurlan008]

Thanks for sharing this work. I have reproduced the work. When I try to run unique_ptr_test, I get an error

/users/Nurlan/Atlas/atlas-runtime/tests/runtime/unique_ptr/unique_ptr_test.cc:52: Failure Expected: (object_last) != (buffer + kNumObjects - 1), actual: 0x7ffff2dfdf80 vs 0x7ffff2dfdf80 The last object should be evacuated

And then the test fails. If I let the max possible memory for group, then the program gets killed.
What could cause this error?

[lscat11]

Hi, could you check if the kernel swapping is working properly? Here is a simple array traversal example, and try to run it under the 512 MB memory limit to see if it works:

#include <iostream>
#include <random>
#include <vector>

int main() {
    const size_t kTotalData = 1UL << 30; // 1GB
    std::vector<int> array(kTotalData/sizeof(int));
    
    std::srand(0);
    
    for(size_t i = 0; i < array.size(); ++i) {
        array[i] = std::rand();
    }

    std::cout << "Check `RssAnon` in `/proc/<pid>/status` and press any key to continue..." << std::endl;
    char ch;
    std::cin >> ch;
    
    int r = 0;
    for(auto & x : array){
        r ^= x;
    }
    
    std::cout << "Result: " << r << std::endl;
    
    return 0;
}

[nnurlan008]

Thanks for your reply. I ran the code you shared. On the 512MB limit, it gets killed (but no specific message under dmesg related to OOM). If I increase the memory limit on cgroup, the code runs successfully.

dmesg:

What could be the reason for not utilizing the kernel's swap properly?

Update:
I tested the code on 5.15.0-122-generic (default kernel on ubuntu 22.04), the kernel swap works fine. However, when I switched to kernel 5.14.0-rc5, the kernel swapping does not work -> code execution gets killed.

[lscat11]

Maybe you can check the following points :

1. check whether the RDMA connection between the cpu server and memory server is normal, and make sure the remoteswap(atlas-runtime/bks_module/remoteswap/client and atlas-runtime/bks_module/remoteswap/server) is set up properly
2. adjust the RMGRID_NR_PCORE variable in linux-5.14-rc5/include/linux/adc_macros.h to match the number of your server's physical cpu cores

[nnurlan008]

The connection works fine as I have verified it with ib_read_bw.
Once I update the RMGRID_NR_PCORE variable in linux-5.14-rc5/include/linux/adc_macros.h, now I am getting the following errors:

The code crashes with 'Bus error'. Is there any other file I should update based on my system?

Thanks in advance