-
Notifications
You must be signed in to change notification settings - Fork 4
Expand file tree
/
Copy pathfam_initialize.html
More file actions
304 lines (301 loc) · 11.4 KB
/
fam_initialize.html
File metadata and controls
304 lines (301 loc) · 11.4 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
<!DOCTYPE html>
<html lang="en">
<head>
<link rel="stylesheet" type="text/css" href="styles/style.css">
<title>OpenFAM: A library for programming Fabric-Attached Memory</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<header>
<h1>OpenFAM Reference Implementation</h1>
</header>
<section>
<nav>
<ul>
<li><a href="index.html">Home</a></li>
<li><a href="release_notes.html">Release Notes</a></li>
<li><a href="limitations.html">Design Choices</a></li>
<li><a href="errors.html">Exceptions and Error Codes</a></li>
<li><a href="services.html">Services</a></li>
<li><a href="config_files.html">Configuration Files</a></li>
</ul>
<hr>
Initialization and Finalization
<ul>
<li><a href="fam_initialize.html">fam_initialize</a></li>
<li><a href="fam_finalize.html">fam_finalize</a></li>
<li><a href="fam_abort.html">fam_abort</a></li>
</ul>
<hr>
Options & Query
<ul>
<li><a href="fam_list_options.html">fam_list_options</a></li>
<li><a href="fam_get_option.html">fam_get_option</a></li>
<li><a href="fam_lookup.html">fam_lookup</a></li>
<li><a href="fam_stat.html">fam_stat</a></li>
</ul>
<hr>
Memory Allocation
<ul>
<li><a href="fam_create_region.html">fam_create_region</a></li>
<li><a href="fam_destroy_region.html">fam_destroy_region</a></li>
<li><a href="fam_resize_region.html">fam_resize_region</a></li>
<li><a href="fam_allocate.html">fam_allocate</a></li>
<li><a href="fam_deallocate.html">fam_deallocate</a></li>
<li><a href="fam_change_permissions.html">fam_change_permissions</a></li>
<li><a href="fam_close.html">fam_close</a></li>
</ul>
<hr>
Memory Map
<ul>
<li><a href="fam_map.html">fam_map</a></li>
<li><a href="fam_unmap.html">fam_unmap</a></li>
</ul>
<hr>
Data Path Operations
<ul>
<li><a href="fam_get.html">fam_get</a></li>
<li><a href="fam_put.html">fam_put</a></li>
<li><a href="fam_gather.html">fam_gather</a></li>
<li><a href="fam_scatter.html">fam_scatter</a></li>
<li><a href="fam_copy.html">fam_copy</a></li>
<li><a href="fam_copy_wait.html">fam_copy_wait</a></li>
<li><a href="fam_backup.html">fam_backup</a></li>
<li><a href="fam_backup_wait.html">fam_backup_wait</a></li>
<li><a href="fam_restore.html">fam_restore</a></li>
<li><a href="fam_restore_wait.html">fam_restore_wait</a></li>
<li><a href="fam_delete_backup.html">fam_delete_backup</a></li>
<li><a href="fam_delete_backup_wait.html">fam_delete_backup_wait</a></li>
</ul>
<hr>
Atomics
<ul>
<li><a href="fam_set.html">fam_set</a></li>
<li><a href="fam_add.html">fam_add</a></li>
<li><a href="fam_subtract.html">fam_subtract</a></li>
<li><a href="fam_min.html">fam_min</a></li>
<li><a href="fam_max.html">fam_max</a></li>
<li><a href="fam_and.html">fam_and</a></li>
<li><a href="fam_or.html">fam_or</a></li>
<li><a href="fam_xor.html">fam_xor</a></li>
<li><a href="fam_fetch_TYPE.html">fam_fetch_TYPE</a></li>
<li><a href="fam_swap.html">fam_swap</a></li>
<li><a href="fam_compare_swap.html">fam_compare_swap</a></li>
<li><a href="fam_fetch_add.html">fam_fetch_add</a></li>
<li><a href="fam_fetch_subtract.html">fam_fetch_subtract</a></li>
<li><a href="fam_fetch_min.html">fam_fetch_min</a></li>
<li><a href="fam_fetch_and.html">fam_fetch_and</a></li>
<li><a href="fam_fetch_max.html">fam_fetch_max</a></li>
<li><a href="fam_fetch_or.html">fam_fetch_or</a></li>
<li><a href="fam_fetch_xor.html">fam_fetch_xor</a></li>
</ul>
<hr>
Ordering
<ul>
<li><a href="fam_barrier_all.html">fam_barrier_all</a></li>
<li><a href="fam_fence.html">fam_fence</a></li>
<li><a href="fam_quiet.html">fam_quiet</a></li>
<li><a href="fam_progress.html">fam_progress</a></li>
<li><a href="fam_context_open.html">fam_context_open</a></li>
<li><a href="fam_context_close.html">fam_context_close</a></li>
</ul>
<hr>
</nav>
<article>
<h1 class="code">openfam::fam::fam_initialize</h1>
<p>Initialize the OpenFAM library.</p>
<h2>Synopsis</h2>
<p class="code">void fam_initialize(const char *groupName, Fam_Options *options);</p>
<h2>Description</h2>
<p>This method is required to be the first method called when a
processing element uses the OpenFAM library. It initializes internal
data structures within the library that allow the processing element
to participate in the application. Normally, called by all PEs at the
start of the program.</p>
<h2>Input Arguments</h2>
<table>
<tbody>
<tr><th>Name</th><th>Description</th></tr>
<tr><td>groupName</td><td>Name that defines an application group. All cooperating PEs must provide the same group name.</td></tr>
<tr><td>options</td><td>Pre-defined options that a processing element provides to the OpenFAM library.</td></tr>
</tbody>
</table>
<p><code>fam_initialize()</code> takes a defined object of type <code>Fam_Options</code> as input</p>
<pre>
typedef struct {
char *defaultRegionName;
char *cisServer;
char *grpcPort;
char *libfabricProvider;
char *famThreadModel;
char *cisInterfaceType;
char *openFamModel;
char *famContextModel;
char *numConsumer;
char *runtime;
char *fam_default_memory_type;
char *if_device;
} Fam_Options;
<br>Two additional fields have been added in Fam_Options to achieve better performance with Slingshot interconnect.
<br>More details can be found <a href="fam_initialize_internal.html">here</a>.
</pre>
<table>
<tbody>
<tr><th>Name</th><th>Description</th><th>Default Value</th></tr>
<tr>
<td class="code">defaultRegionName</td><td>Default region name. Currently ignored by the implementation.</td>
<td>"default"</td>
</tr>
<tr>
<td class="code">cisServer</td><td>Client Interface Server to be used by OpenFam APIs. The value is
given as cisServer IPaddress string.</td>
<td>"127.0.0.1"</td>
</tr>
<tr><td class="code">grpcPort</td><td>Port to be used by OpenFam client interface service operations.</td><td>"8787"</td></tr>
<tr><td class="code">libfabricProvider</td><td>Libfabric provider to be used by OpenFam libfabric data path
operations. The supported options are "sockets", "verbs;ofi_rxm" and "psm2".</td>
<td>"sockets"</td>
</tr>
<tr><td class="code">famThreadModel</td><td>Threading model used. Options are:.
<ul><li>"FAM_THREAD_SERIALIZE" - With this option, the application can be multi-threaded. However concurrent invocation of OpenFAM APIs invocation from multiple threads is not allowed.</li>
<li>"FAM_THREAD_MULTIPLE" - With this option, the application can be multi-threaded and any threads may invoke OpenFAM APIs. </li></ul>
</td>
<td>"FAM_THREAD_SERIALIZE"</td>
</tr>
<tr><td class="code">cisInterfaceType</td><td>Communication interface type to be used between processing element and Client interface service.
Supported types are "rpc" and "direct".</td>
<td>"rpc"</td>
</tr>
<tr>
<td class="code">
openFamModel
</td>
<td>
Type of OpenFAM memory allocator model implementation used by OpenFAM API.
Supported allocator implementations are:
<ul><li>"memory_server" – Memory Server implementation that can be used in a
standard cluster.</li>
<li>"shared_memory" – Shared Memory implementation that is usable in
large shared memory scale-up machines.</li></ul>
</td>
<td>
"memory_server"
</td>
</tr>
<tr><td class="code">famContextModel</td><td>Context model used. Currently ignored by the implementation.</td>
<td>"FAM_CONTEXT_DEFAULT"</td>
</tr>
<tr>
<td class="code">
numConsumer
</td>
<td>
Number of consumer threads for shared memory model using the nvmm allocator.
</td>
<td>
"1"
</td>
</tr>
<tr>
<td class="code">
runtime
</td>
<td>
FAM runtime environment used. Supported values are "PMIX",
"PMI2", "NONE".
</td>
<td>
"PMIX"
</td>
</tr>
<tr>
<td class="code">
fam_default_memory_type
</td>
<td>
Memory Type to be used for FAM region creation. Supported values are "PERSISTENT",
"VOLATILE".
</td>
<td>
"VOLATILE"
</td>
</tr>
<tr>
<td class="code">
if_device
</td>
<td>
Interface device used by the PE for communication. Supported values are "ib0", "ib1", etc for verbs, "cxi0". "cxi1" for cxi device.
</td>
<td>
"ib0" for infiniband, "cxi0" for cxi device.
</td>
</tr>
</tbody>
</table>
<p> </p>
<p>The application should set the options value to zero to use the
default option values.</p>
<p class="code">Fam_Options *fm = (Fam_Options*)malloc(sizeof(Fam_Options));</p>
<p class="code">memset((void *)fm, 0, sizeof(Fam_Options));</p>
<p> </p>
<h2>Return Values</h2>
<p>None. Throws <code>Fam_Exception</code> on error.</p>
<h2>Fam Error Numbers</h2>
<table>
<tbody>
<tr><th>Error</th><th>Description</th></tr>
<tr><td class="code">FAM_ERR_INVALID</td><td>API called with incorrect parameters.</td></tr>
<tr><td class="code">FAM_ERR_LIBFABRIC</td><td>Libfabric error occurred.</td></tr>
<tr><td class="code">FAM_ERR_RPC</td><td>Communication error from grpc layer.</td></tr>
<tr><td class="code">FAM_ERR_RPC_CLIENT_NOTFOUND</td><td>RPC service not available.</td></tr>
<tr><td class="code">FAM_ERR_MEMSERV_LIST_EMPTY</td><td>Memory service not initialized.</td></tr>
</tbody></table>
<h3>Notes</h3>
<p>Additional fields may be specified in options in the future to support
parameters or other variables that have traditionally been provided to
the runtime via environment variables.</p>
<p>The following options are not supported in current
implementation and should always be set to zero to use the default
values.<div class="code"> defaultRegionName and famContextModel</div>.
<p>Alternatively, the application can pass these options using configuration files. See <a href="config_files.html">Configuration Files</a> for details.
However, <code>fam_initialize()</code> routine will give precedence to the options passed as <code>Fam_Options</code> argument over the options present in configuration files. </p>
<h2 id="e1">Example</h2>
<pre>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <fam/fam.h>
#include <fam/fam_exception.h>
using namespace std;
using namespace openfam;
int main(void) {
fam *myFam = new fam();
Fam_Options *fm = (Fam_Options*)malloc(sizeof(Fam_Options));
memset((void *)fm, 0, sizeof(Fam_Options));
// Set specific options that are needed by the implementation
fm->cisServer = strdup("10.0.0.25");
fm->libfabricProvider = strdup("sockets");
try {
myFam->fam_initialize("myApplication", fm);
printf("FAM initialized\n");
} catch (Fam_Exception& e) {
printf("FAM Initialization failed: %s\n", e.fam_error_msg());
myFam->fam_abort(-1); // abort the program
// note that fam_abort currently returns after signaling
// so we must terminate with the same value
return -1;
}
myFam->fam_finalize("myApplication");
printf("FAM finalized\n");
return 0;
}
</pre>
</article>
</section>
<footer>
<p>Copyright 2021-2023, Hewlett Packard Enterprise Development Co, LLP</p>
</footer>
</body>
</html>