[Feature]: Dirty memory size triggered CKPT

Summary (Provide a general summary of the issue in the Title above):(required)
- Leverage the statistics of dirty key ratio to trigger the CKPT
- Base on the dirty key ratio and current memory foot print to calculate the actually dirty memory ratio to the total memory limit
- If the dirty memory ratio is bigger than 20%, then trigger a checkpoint
- Remove original checkpoint trigger when memory is full
Detailed Description (Provide a detailed description of the change or addition you are proposing):(required)
Software version (the version of the software you're running):(required)
e.g. 0.0.1
Possible Implementation(suggest an idea for implementing addition or change)(optional)

Yesterday, I spend some time to investigate several CI and nightly tests fails, so did not have updates for this work item.

We have more work to do, since liuyl and I have new idea to control the frequency of the NotifyCKPT should be triggered

Each ccshard will trigger CKPT individually, so we need to prevent the spam by check the request_ckpt_, but the mutex acquire too expensive, we change the type of request_ckpt_ as std::atomic, and using the release/acquire to guarantee the memory order.
Before all DataSyncTask are finished on a core, the dirty memory footprint can be still high on that core, and the Ckpt will be triggered unnecessarily, we should avoid this issue.
- Option 1: We keep track of the DataSyncTask for a core using a reference count, new Ckpt will be triggered only if the reference count is zero
- Option 2: Define a minimum checkpoint interval to control the frequency

Navigation Menu