diff --git a/rocfile/docs/async.rst b/hipfile/docs/async.rst similarity index 69% rename from rocfile/docs/async.rst rename to hipfile/docs/async.rst index 9b49d77c..8d831c5b 100644 --- a/rocfile/docs/async.rst +++ b/hipfile/docs/async.rst @@ -1,7 +1,7 @@ Asynchronous API ================ -Asynchronous I/O is currently not supported in rocFile. +Asynchronous I/O is currently not supported in hipFile. API calls from the async section of the reference manual will fail and return error codes. diff --git a/rocfile/docs/batch.rst b/hipfile/docs/batch.rst similarity index 65% rename from rocfile/docs/batch.rst rename to hipfile/docs/batch.rst index e3f7bffa..b0bd5696 100644 --- a/rocfile/docs/batch.rst +++ b/hipfile/docs/batch.rst @@ -1,7 +1,7 @@ Batch API ========= -Batch IO operations are currently not supported in rocFile. +Batch IO operations are currently not supported in hipFile. API calls from the batch section of the reference manual will fail and return error codes. diff --git a/hipfile/docs/core.rst b/hipfile/docs/core.rst index c1ff2b0d..ab7816ed 100644 --- a/hipfile/docs/core.rst +++ b/hipfile/docs/core.rst @@ -29,3 +29,18 @@ Any of the parameters can be ignored by passing in a NULL pointer. We do not provide a hipFile equivalent to cuFile's ``cuFileGetVersion()`` via the ``hipify`` tool. This is because any logic involving the obtained version number would be platform-specific and have to be customized regardless. + +Parameter Getters and Setters +----------------------------- +Like cuFile, hipFile includes a set of functions for getting and setting library +parameters. These comprise several API calls that get/set parameters +of a particular type based on an enum selector (e.g., ``hipFileGetParameterSizeT()``). +These API calls are all unsupported at this time and will return errors +if called. + +* ``hipFileError_t hipFileGetParameterSizeT(hipFileSizeTConfigParameter_t param, size_t *value)`` +* ``hipFileError_t hipFileGetParameterBool(hipFileBoolConfigParameter_t param, bool *value)`` +* ``hipFileError_t hipFileGetParameterString(hipFileStringConfigParameter_t param, char *desc_str, int len)`` +* ``hipFileError_t hipFileSetParameterSizeT(hipFileSizeTConfigParameter_t param, size_t value)`` +* ``hipFileError_t hipFileSetParameterBool(hipFileBoolConfigParameter_t param, bool value)`` +* ``hipFileError_t hipFileSetParameterString(hipFileStringConfigParameter_t param, const char *desc_str)`` diff --git a/rocfile/docs/driver.rst b/hipfile/docs/driver.rst similarity index 59% rename from rocfile/docs/driver.rst rename to hipfile/docs/driver.rst index a9fc8d74..ad40fdbb 100644 --- a/rocfile/docs/driver.rst +++ b/hipfile/docs/driver.rst @@ -4,7 +4,7 @@ Driver API Basic API Usage --------------- -The rocFile driver API is used to initialize and shut down the "driver" (i.e., +The hipFile driver API is used to initialize and shut down the "driver" (i.e., the library state). The driver is reference counted, and calls to open and close the driver simply increase and decrease the reference count. When the reference count drops to zero, the driver's data structures are cleaned up, @@ -19,23 +19,23 @@ drivers open in a single process. The basic driver API calls: -* ``rocFileError_t rocFileDriverOpen(void)`` -* ``rocFileError_t rocFileDriverClose(void)`` -* ``int64_t rocFileUseCount(void)`` +* ``hipFileError_t hipFileDriverOpen(void)`` +* ``hipFileError_t hipFileDriverClose(void)`` +* ``int64_t hipFileUseCount(void)`` -The open call should be called before making any other rocFile API calls -and the close call should be called when all rocFile API calls are complete. -The ``rocFileUseCount()`` call can be used to check the current reference +The open call should be called before making any other hipFile API calls +and the close call should be called when all hipFile API calls are complete. +The ``hipFileUseCount()`` call can be used to check the current reference count of the driver. Note that some API calls will automatically open the driver and bump the reference count if it has not yet been explicitly opened: -* ``rocFileHandleRegister()`` -* ``rocFileBufRegister()`` +* ``hipFileHandleRegister()`` +* ``hipFileBufRegister()`` Also note that this "implicit opening" will not be tracked, so it's up the -application to add an extra call to ``rocFileDriverClose()`` to fully +application to add an extra call to ``hipFileDriverClose()`` to fully close the driver. These API calls will only increment the driver count if they succeed. @@ -44,24 +44,24 @@ if the intent is to completely shut down the driver, as simply matching open and close calls may not reduce the reference count to zero. These API calls have been coded to behave like cuFile's calls, regarding -initialization and errors, but there is no guarantee that rocFile's driver +initialization and errors, but there is no guarantee that hipFile's driver calls will match unpublished cuFile API behaviour. Driver Getters and Setters -------------------------- -These API calls exist in rocFile, but currently have no effect and will return +These API calls exist in hipFile, but currently have no effect and will return an error: -* ``rocFileError_t rocFileDriverGetProperties(rocFileDriverProps_t *props)`` -* ``rocFileError_t rocFileDriverSetPollMode(bool poll, size_t poll_threshold_size)`` -* ``rocFileError_t rocFileDriverSetMaxDirectIOSize(size_t max_direct_io_size)`` -* ``rocFileError_t rocFileDriverSetMaxCacheSize(size_t max_cache_size)`` -* ``rocFileError_t rocFileDriverSetMaxPinnedMemSize(size_t max_pinned_size)`` +* ``hipFileError_t hipFileDriverGetProperties(hipFileDriverProps_t *props)`` +* ``hipFileError_t hipFileDriverSetPollMode(bool poll, size_t poll_threshold_size)`` +* ``hipFileError_t hipFileDriverSetMaxDirectIOSize(size_t max_direct_io_size)`` +* ``hipFileError_t hipFileDriverSetMaxCacheSize(size_t max_cache_size)`` +* ``hipFileError_t hipFileDriverSetMaxPinnedMemSize(size_t max_pinned_size)`` -These API calls also exist in rocFile, but have no effect, cannot be used to set +These API calls also exist in hipFile, but have no effect, cannot be used to set driver properties, and will return an error: -* ``rocFileError_t rocFileSetParameterSizeT(rocFileSizeTConfigParameter_t param, size_t value)`` -* ``rocFileError_t rocFileSetParameterBool(rocFileBoolConfigParameter_t param, bool value)`` -* ``rocFileError_t rocFileSetParameterString(rocFileStringConfigParameter_t param, const char *desc_str)`` +* ``hipFileError_t hipFileSetParameterSizeT(hipFileSizeTConfigParameter_t param, size_t value)`` +* ``hipFileError_t hipFileSetParameterBool(hipFileBoolConfigParameter_t param, bool value)`` +* ``hipFileError_t hipFileSetParameterString(hipFileStringConfigParameter_t param, const char *desc_str)`` diff --git a/rocfile/docs/errors.rst b/hipfile/docs/errors.rst similarity index 51% rename from rocfile/docs/errors.rst rename to hipfile/docs/errors.rst index 6c4c8b74..44e3392f 100644 --- a/rocfile/docs/errors.rst +++ b/hipfile/docs/errors.rst @@ -1,62 +1,62 @@ Errors and Error Handling ========================= -Functions that return a rocFileOpError_t struct +Functions that return a hipFileOpError_t struct ----------------------------------------------- Errors are handled identically to cuFile. Most API calls return -a ``rocFileError_t`` struct, which includes ``rocFileOpError_t`` -field for returning rocFile error codes, and a ``hipError_t`` +a ``hipFileError_t`` struct, which includes ``hipFileOpError_t`` +field for returning hipFile error codes, and a ``hipError_t`` field for returning GPU error codes. These fields should be checked -for ``rocFileSuccess`` and ``hipSuccess``, respectively. Any other +for ``hipFileSuccess`` and ``hipSuccess``, respectively. Any other values indicate an error. :: - typedef struct __ROCFILE_NODISCARD rocFileError { - rocFileOpError_t err; //!< Errors related to rocFile or the GPU IO driver + typedef struct __ROCFILE_NODISCARD hipFileError { + hipFileOpError_t err; //!< Errors related to hipFile or the GPU IO driver hipError_t hip_drv_err; //!< Errors related to the GPU driver - } rocFileError_t; + } hipFileError_t; Note that the struct is marked with the ``[[nodiscard]]`` attribute so in C++17 / C23 or greater, the compiler will complain if you do not check error values. -If a GPU driver error occurs the ``rocFileOpError_t`` value will be -``rocFileHipDriverError`` and the ``hipError_t`` field will be set to +If a GPU driver error occurs the ``hipFileOpError_t`` value will be +``hipFileHipDriverError`` and the ``hipError_t`` field will be set to the appropriate HIP driver error value. -When any other rocFile error is returned, the ``rocFileOpError_t`` field will be -set to the appropriate rocFile error and the ``hipError_t`` field will +When any other hipFile error is returned, the ``hipFileOpError_t`` field will be +set to the appropriate hipFile error and the ``hipError_t`` field will be set to ``hipSuccess``. -Several helper macros are included in ``rocfile.h`` that help with error checking: +Several helper macros are included in ``hipfile.h`` that help with error checking: * ``IS_ROCFILE_ERR()`` * ``ROCFILE_ERRSTR()`` * ``IS_HIP_DRV_ERR()`` * ``HIP_DRV_ERR()`` -See the error section of the rocFile reference manual for documentation for +See the error section of the hipFile reference manual for documentation for each of these macros. Functions that return an integer value -------------------------------------- -Several read and write functions (e.g., ``rocFileRead()``) return a ``ssize_t`` value. +Several read and write functions (e.g., ``hipFileRead()``) return a ``ssize_t`` value. Like the POSIX ``read(3)`` call, these API calls return negative values for errors. -Unlike the POSIX call, however, which only returns -1 on errors, the rocFile IO -calls return a value that reflects the ``rocFileOpError_t`` or ``hipError_t`` +Unlike the POSIX call, however, which only returns -1 on errors, the hipFile IO +calls return a value that reflects the ``hipFileOpError_t`` or ``hipError_t`` value that would have been returned. ``hipError_t`` and its values are defined in ``hip/hip_runtime_api.h``. The enum -values are assigned integers up to ~1000. ``rocFileOpError_t`` enum values are +values are assigned integers up to ~1000. ``hipFileOpError_t`` enum values are assigned values of 5000+. -When rocFile IO calls that return a ``size_t`` fail, the returned value is the -negative of the ``rocFileOpError_t`` value (rocFile errors) or the negative of +When hipFile IO calls that return a ``size_t`` fail, the returned value is the +negative of the ``hipFileOpError_t`` value (hipFile errors) or the negative of the ``hipError_t`` value (GPU driver errors) that would normally have been returned -via the ``rocFileError_t`` struct. +via the ``hipFileError_t`` struct. Other functions --------------- -* ``rocFileOpStatusError()`` returns a string that corresponds to a ``rocFileOpError_t`` value. It cannot fail. -* ``rocFileUseCount()`` returns the reference count of the library. It returns -1 on errors. +* ``hipFileOpStatusError()`` returns a string that corresponds to a ``hipFileOpError_t`` value. It cannot fail. +* ``hipFileUseCount()`` returns the reference count of the library. It returns -1 on errors. diff --git a/hipfile/docs/file.rst b/hipfile/docs/file.rst new file mode 100644 index 00000000..5eb5d1de --- /dev/null +++ b/hipfile/docs/file.rst @@ -0,0 +1,40 @@ +File Handle API +=============== + +File Handles +------------ +hipFile files are accessed via an opaque ``hipFileHandle_t`` pointer +obtained from ``hipFileHandleRegister()``. This registration API call +takes a ``hipFileDescr_t`` struct, which contains the filesystem file +descriptor to be used for hipFile IO. This hipFile handle must be closed +using ``hipFileHandleDeregister()`` to avoid leaking resources. + +Ideally, ``hipFileDriverOpen()`` should be used to initialize the driver +before registering hipFile handles. hipFile will automatically perform +this initialization for the caller, though, if the driver has not been +initialized when ``hipFileHandleRegister()`` is called. See the driver +section of the documentation for a more thorough discussion of this. + +Buffers +------- +Memory buffers that will be used with multiple hipFile IO operations +should be registered via ``hipFileBufRegister()``. If this is not +done, a temporary internal buffer will be used for IO, though this +may not be as performant as using registered buffers. + +When registering a buffer, no special handle or pointer is returned +to the caller. Instead, the registered buffer will be tracked internally +and used for IO when appropriate. When no longer necessary for IO, +registered buffers should be freed using ``hipFileBufDeregister()``. + +As in ``hipFileHandleRegister()``, the driver will automatically be +initialized by ``hipFileBufRegister()`` if it has not already. + +IO Operations +------------- +hipFile read and write operations are performed using ``hipFileRead()`` +and ``hipFileWrite()``, respectively. These API calls take a hipFile +handle, a buffer (registered or not), the size of the IO operation +in bytes, and the file and buffer offsets. If using a registered buffer, +the buffer pointer should be the one that was registered and not +a pointer inside that buffer. diff --git a/hipfile/docs/introduction.rst b/hipfile/docs/introduction.rst index e4efdd86..9aed003d 100644 --- a/hipfile/docs/introduction.rst +++ b/hipfile/docs/introduction.rst @@ -1,11 +1,20 @@ Introduction ============ +In recent years, the demand for high-performance data movement between +storage and GPU memory has grown rapidly, driven by the increasing +scale of AI training, scientific simulations, and data analytics +workloads. In response to this evolving need within heterogeneous +computing environments, AMD introduces hipFile, a cuFile-equivalent +API framework designed to enable direct data paths between NVMe +devices and GPU memory, significantly reducing CPU overhead and +improving IO throughput. -hipFile is an AMD equivalent to NVIDIA's cuFile API. It is intendended -to be a drop-in, easily HIPify-able replacement for cuFile. Like other -HIP libraries, it transparently maps hipFile API calls to AMD's rocFile -or NVIDIA's cuFile based on whether `__HIP_PLATFORM_AMD__` or `__HIP_PLATFORM_NVIDIA__` -were set when building the library, respectively. - -The documentation for hipFile is somewhat sparse as it's a very thin layer. -For API details, see AMD's rocFile or NVIDIA's cuFile API documentation. +hipFile provides developers with an interface for performing +high-performance IO operations between storage devices and AMD +GPUs. By bypassing traditional CPU memory staging buffers, +hipFile allows applications to achieve lower latency and higher +bandwidth when transferring large datasets into GPU memory. This +direct data path integration complements the broader ROCm stack +- seamlessly interoperating with HIP kernels, HIP streams, and +RDMA-enabled storage systems - to support end-to-end acceleration +for IO-bound workflows. diff --git a/rocfile/docs/misc_api.rst b/hipfile/docs/misc_api.rst similarity index 100% rename from rocfile/docs/misc_api.rst rename to hipfile/docs/misc_api.rst diff --git a/hipfile/docs/rdma.rst b/hipfile/docs/rdma.rst new file mode 100644 index 00000000..38e77a5e --- /dev/null +++ b/hipfile/docs/rdma.rst @@ -0,0 +1,6 @@ +Userspace RDMA API +================== + +RDMA is currently not supported in hipFile. + +Setting RDMA options in hipFile API calls will have no effect. diff --git a/rocfile/docs/config_build.rst b/rocfile/docs/config_build.rst deleted file mode 100644 index 028825ce..00000000 --- a/rocfile/docs/config_build.rst +++ /dev/null @@ -1,5 +0,0 @@ -Configuration and Build -======================= - -Please see the README.md in the project root for instructions -for building rocFile. diff --git a/rocfile/docs/core.rst b/rocfile/docs/core.rst deleted file mode 100644 index d5ae3fbf..00000000 --- a/rocfile/docs/core.rst +++ /dev/null @@ -1,42 +0,0 @@ -Core API -======== - -API Versioning --------------- -rocFile uses (`semantic versioning `_) so library -releases have major, minor, and patch versions. The soversion number -on POSIX systems matches the major API number, so we guarantee that minor -and patch releases will not break the ABI. - -These version numbers are defined in ``rocfile.h``: -* ``ROCFILE_VERSION_MAJOR`` -* ``ROCFILE_VERSION_MINOR`` -* ``ROCFILE_VERSION_PATCH`` - -These symbols can be used to ifdef around API differences. They are always set -to integer values and will never contain text strings like "-patch1". - -rocFile also includes an API call for determining the library version at -runtime: - -This API call can be used to get the individual components of the version -number. - - ``rocFileError_t rocFileGetVersion(unsigned *major, unsigned *minor, unsigned *patch)`` - -Any of the parameters can be ignored by passing in a NULL pointer. - -Parameter Getters and Setters ------------------------------ -Like cuFile, rocFile includes a set of functions for getting and setting library -parameters. These comprise several API calls that get/set parameters -of a particular type based on an enum selector (e.g., ``rocFileGetParameterSizeT()``). -These API calls are all unsupported at this time and will return errors -if called. - -* ``rocFileError_t rocFileGetParameterSizeT(rocFileSizeTConfigParameter_t param, size_t *value)`` -* ``rocFileError_t rocFileGetParameterBool(rocFileBoolConfigParameter_t param, bool *value)`` -* ``rocFileError_t rocFileGetParameterString(rocFileStringConfigParameter_t param, char *desc_str, int len)`` -* ``rocFileError_t rocFileSetParameterSizeT(rocFileSizeTConfigParameter_t param, size_t value)`` -* ``rocFileError_t rocFileSetParameterBool(rocFileBoolConfigParameter_t param, bool value)`` -* ``rocFileError_t rocFileSetParameterString(rocFileStringConfigParameter_t param, const char *desc_str)`` diff --git a/rocfile/docs/file.rst b/rocfile/docs/file.rst deleted file mode 100644 index ec7f930c..00000000 --- a/rocfile/docs/file.rst +++ /dev/null @@ -1,40 +0,0 @@ -File Handle API -=============== - -File Handles ------------- -rocFile files are accessed via an opaque ``rocFileHandle_t`` pointer -obtained from ``rocFileHandleRegister()``. This registration API call -takes a ``rocFileDescr_t`` struct, which contains the filesystem file -descriptor to be used for rocFile IO. This rocFile handle must be closed -using ``rocFileHandleDeregister()`` to avoid leaking resources. - -Ideally, ``rocFileDriverOpen()`` should be used to initialize the driver -before registering rocFile handles. rocFile will automatically perform -this initialization for the caller, though, if the driver has not been -initialized when ``rocFileHandleRegister()`` is called. See the driver -section of the documentation for a more thorough discussion of this. - -Buffers -------- -Memory buffers that will be used with multiple rocFile IO operations -should be registered via ``rocFileBufRegister()``. If this is not -done, a temporary internal buffer will be used for IO, though this -may not be as performant as using registered buffers. - -When registering a buffer, no special handle or pointer is returned -to the caller. Instead, the registered buffer will be tracked internally -and used for IO when appropriate. When no longer necessary for IO, -registered buffers should be freed using ``rocFileBufDeregister()``. - -As in ``rocFileHandleRegister()``, the driver will automatically be -initialized by ``rocFileBufRegister()`` if it has not already. - -IO Operations -------------- -rocFile read and write operations are performed using ``rocFileRead()`` -and ``rocFileWrite()``, respectively. These API calls take a rocFile -handle, a buffer (registered or not), the size of the IO operation -in bytes, and the file and buffer offsets. If using a registered buffer, -the buffer pointer should be the one that was registered and not -a pointer inside that buffer. diff --git a/rocfile/docs/introduction.rst b/rocfile/docs/introduction.rst deleted file mode 100644 index 18505f8f..00000000 --- a/rocfile/docs/introduction.rst +++ /dev/null @@ -1,20 +0,0 @@ -Introduction -============ -In recent years, the demand for high-performance data movement between -storage and GPU memory has grown rapidly, driven by the increasing -scale of AI training, scientific simulations, and data analytics -workloads. In response to this evolving need within heterogeneous -computing environments, AMD introduces rocFile, a cuFile-equivalent -API framework designed to enable direct data paths between NVMe -devices and GPU memory, significantly reducing CPU overhead and -improving IO throughput. - -rocFile provides developers with an interface for performing -high-performance IO operations between storage devices and AMD -GPUs. By bypassing traditional CPU memory staging buffers, -hipFile allows applications to achieve lower latency and higher -bandwidth when transferring large datasets into GPU memory. This -direct data path integration complements the broader ROCm stack -- seamlessly interoperating with HIP kernels, HIP streams, and -RDMA-enabled storage systems - to support end-to-end acceleration -for IO-bound workflows. diff --git a/rocfile/docs/nvidia_compat.rst b/rocfile/docs/nvidia_compat.rst deleted file mode 100644 index 23d57fe2..00000000 --- a/rocfile/docs/nvidia_compat.rst +++ /dev/null @@ -1,5 +0,0 @@ -NVIDIA cuFile Compatibility -=========================== - -rocfile.h/cufile.h Differences ------------------------------- diff --git a/rocfile/docs/rdma.rst b/rocfile/docs/rdma.rst deleted file mode 100644 index 60b52777..00000000 --- a/rocfile/docs/rdma.rst +++ /dev/null @@ -1,6 +0,0 @@ -Userspace RDMA API -================== - -RDMA is currently not supported in rocFile. - -Setting RDMA options in rocFile API calls will have no effect.