Are You Sure You Want to Use MMAP in Your Database Management System? (A. Crotty, CIDR 2022)

[benclmnt]

Link to paper

Main Claim

mmap is not a suitable replacement for a traditional buffer pool that manually handles read and write syscalls. This is despite the seemingly less complexity and more efficiency benefits that mmap offers.

Main problems

1. Transparent Paging: mmap might flush dirty page at any time, without regards to whether the page contains any changes from uncommitted transaction.
   • (3.1) To ensure transactional safety, DB needs to implement either Copy-on-Write (maintaining an extra copy of the database file as private workspace) with WAL, or Shadow Paging.
   • (3.3) To ensure data integrity, DB needs to validate checksum on every page access. Even worse, there is no mechanism to ensure that pages is not corrupted before writing to secondary storage.
2. (3.2) Since page cache might be full at anytime, read-only queries can trigger unintended page fault, causing IO spikes.
3. (3.2) mmap doesn't support asynchronous reads.
4. (3.4) mmap-based file I/O suffers from page table contention, single-threaded page eviction and TLB shootdowns.

[benclmnt]

Shadow paging

DB maintains separate primary and shadow copies of the database. To perform an update, the DBMS copies the affected pages from primary to shadow copy, where it then applies the necessary changes. Committing the changes involves flushing the modified shadow copy and swapping the primary and shadow copy.

This is similar to "old-master-new-master" processing

[benclmnt]

How mmap works

1. Program receives a pointer to mmap-ed file contents
2. OS reserves part of the program's virtual address space.
3. On file access,
4. OS attempts to retrieve the page.
5. Since the page is not in the memory, it triggers a page fault and OS loads the file into a physical memory page.
6. OS adds the mapping <virtual address, physical address> into page table
7. Initiating CPU also loads this mapping into its local TLB.

[benclmnt]

TLB shootdowns

When OS evict pages, it needs to remove the mappings from both the page table and each CPU core's TLB. Current CPUs do not provide coherence for remote TLBs, so the OS has to issue an expensive inter-processor interrupt to flush them, called a TLB shootdown.