-
Notifications
You must be signed in to change notification settings - Fork 4
Expand file tree
/
Copy pathlimitations.html
More file actions
146 lines (145 loc) · 6.31 KB
/
limitations.html
File metadata and controls
146 lines (145 loc) · 6.31 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
<!DOCTYPE html>
<html lang="en">
<head>
<link rel="stylesheet" type="text/css" href="styles/style.css">
<title>OpenFAM: A library for programming Fabric-Attached Memory</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<header>
<h1>OpenFAM Reference Implementation</h1>
</header>
<section>
<nav>
<ul>
<li><a href="index.html">Home</a></li>
<li><a href="release_notes.html">Release Notes</a></li>
<li><a href="limitations.html">Design Choices</a></li>
<li><a href="errors.html">Exceptions and Error Codes</a></li>
<li><a href="services.html">Services</a></li>
<li><a href="config_files.html">Configuration Files</a></li>
</ul>
<hr>
Initialization and Finalization
<ul>
<li><a href="fam_initialize.html">fam_initialize</a></li>
<li><a href="fam_finalize.html">fam_finalize</a></li>
<li><a href="fam_abort.html">fam_abort</a></li>
</ul>
<hr>
Options & Query
<ul>
<li><a href="fam_list_options.html">fam_list_options</a></li>
<li><a href="fam_get_option.html">fam_get_option</a></li>
<li><a href="fam_lookup.html">fam_lookup</a></li>
<li><a href="fam_stat.html">fam_stat</a></li>
</ul>
<hr>
Memory Allocation
<ul>
<li><a href="fam_create_region.html">fam_create_region</a></li>
<li><a href="fam_destroy_region.html">fam_destroy_region</a></li>
<li><a href="fam_resize_region.html">fam_resize_region</a></li>
<li><a href="fam_allocate.html">fam_allocate</a></li>
<li><a href="fam_deallocate.html">fam_deallocate</a></li>
<li><a href="fam_change_permissions.html">fam_change_permissions</a></li>
<li><a href="fam_close.html">fam_close</a></li>
</ul>
<hr>
Memory Map
<ul>
<li><a href="fam_map.html">fam_map</a></li>
<li><a href="fam_unmap.html">fam_unmap</a></li>
</ul>
<hr>
Data Path Operations
<ul>
<li><a href="fam_get.html">fam_get</a></li>
<li><a href="fam_put.html">fam_put</a></li>
<li><a href="fam_gather.html">fam_gather</a></li>
<li><a href="fam_scatter.html">fam_scatter</a></li>
<li><a href="fam_copy.html">fam_copy</a></li>
<li><a href="fam_copy_wait.html">fam_copy_wait</a></li>
<li><a href="fam_backup.html">fam_backup</a></li>
<li><a href="fam_backup_wait.html">fam_backup_wait</a></li>
<li><a href="fam_restore.html">fam_restore</a></li>
<li><a href="fam_restore_wait.html">fam_restore_wait</a></li>
<li><a href="fam_delete_backup.html">fam_delete_backup</a></li>
<li><a href="fam_delete_backup_wait.html">fam_delete_backup_wait</a></li>
</ul>
<hr>
Atomics
<ul>
<li><a href="fam_set.html">fam_set</a></li>
<li><a href="fam_add.html">fam_add</a></li>
<li><a href="fam_subtract.html">fam_subtract</a></li>
<li><a href="fam_min.html">fam_min</a></li>
<li><a href="fam_max.html">fam_max</a></li>
<li><a href="fam_and.html">fam_and</a></li>
<li><a href="fam_or.html">fam_or</a></li>
<li><a href="fam_xor.html">fam_xor</a></li>
<li><a href="fam_fetch_TYPE.html">fam_fetch_TYPE</a></li>
<li><a href="fam_swap.html">fam_swap</a></li>
<li><a href="fam_compare_swap.html">fam_compare_swap</a></li>
<li><a href="fam_fetch_add.html">fam_fetch_add</a></li>
<li><a href="fam_fetch_subtract.html">fam_fetch_subtract</a></li>
<li><a href="fam_fetch_min.html">fam_fetch_min</a></li>
<li><a href="fam_fetch_and.html">fam_fetch_and</a></li>
<li><a href="fam_fetch_max.html">fam_fetch_max</a></li>
<li><a href="fam_fetch_or.html">fam_fetch_or</a></li>
<li><a href="fam_fetch_xor.html">fam_fetch_xor</a></li>
</ul>
<hr>
Ordering
<ul>
<li><a href="fam_barrier_all.html">fam_barrier_all</a></li>
<li><a href="fam_fence.html">fam_fence</a></li>
<li><a href="fam_quiet.html">fam_quiet</a></li>
<li><a href="fam_progress.html">fam_progress</a></li>
<li><a href="fam_context_open.html">fam_context_open</a></li>
<li><a href="fam_context_close.html">fam_context_close</a></li>
</ul>
<hr>
</nav>
<article>
<h1>Design Choices</h1>
Table 1 describes the current design limits within the OpenFAM reference implementation.
Since the maximum FAM size in the reference implementation is bound by the amount of memory available
at the memory server, these parameters should not pose undue limitations on memory size, while enabling
developers to build reasonable sized applications using the reference implementation.
<table>
<caption>Table 1: Data set limits per memory server in the reference implementation</caption>
<tbody>
<tr><th>Item</th><th>Limit</th></tr>
<tr><td>Maximum FAM size</td><td>128 TB</td></tr>
<tr><td>Minimum region size</td><td>8 GB</td></tr>
<tr><td>Maximum region size</td><td>1 TB</td></tr>
<tr><td>Maximum number of regions</td><td>16 K regions</td></tr>
<tr><td>Minimum data Item size</td><td>128 bytes</td></tr>
<tr><td>Maximum data Item size</td><td>512 GB</td></tr>
<tr><td>Maximum number of data items within a region</td><td>~8 B data Items</td></tr>
</tbody>
</table>
As mentioned in <cite><a href="https://link.springer.com/chapter/10.1007%2F978-3-030-04918-8_5">The OpenFAM API: A Programming Model for Disaggregated Persistent Memory</a></cite>,
the reference implementation uses libfabric
to handle the underlying memory traffic.
The implementation supports the complete API with exceptions highlighted in Table 2.
<table>
<caption>Table 2: OpenFAM APIs currently not available in the reference implementation</caption>
<tbody>
<tr><th>API</th><th>Memory Server</th><th>Shared Memory</th><th>Comment</th></tr>
<tr><td class="code">fam_map</td><td>NO</td><td>YES</td><td>Libfabric currently does not support mmap APIs</td></tr>
<tr><td class="code">fam_unmap</td><td>NO</td><td>YES</td><td>Libfabric currently does not support mmap APIs</td></tr>
<tr><td class="code">fam_fence</td><td>YES (only with sockets provider)</td><td>NO</td><td>Fence operation is not supported by CXI, PSM2 and Verbs libfabric provider.<br/>Support for other providers and Shared Memory model will be added in future</td></tr>
</tbody>
</table>
Since the underlying libfabric providers do not support 128-bit compare and swap (CAS) operations, CAS operations have been implemented
in software.
</article>
</section>
<footer>
<p>Copyright 2021-23, Hewlett Packard Enterprise Development Co, LLP</p>
</footer>
</body>
</html>