diff --git a/docs/architecture/ADDON-FUZZING.md b/docs/architecture/ADDON-FUZZING.md index f2bda383c3..a591e780bb 100644 --- a/docs/architecture/ADDON-FUZZING.md +++ b/docs/architecture/ADDON-FUZZING.md @@ -43,11 +43,9 @@ libFuzzer). ## Scope Scope is **every QVAC native addon** — all packages whose `package.json` -declares `"addon": true` — **except the ONNX Runtime addons** (`onnx`, -`ocr-onnx`), which are out of scope for this effort. The in-scope addons are -C++20 Bare addons built with CMake + vcpkg on the centrally-pinned -**clang-22 / libc++** toolchain, linked against `ggml` / `llama.cpp` / -`whisper.cpp` / `stable-diffusion.cpp`. +declares `"addon": true`. The in-scope addons are C++20 Bare addons built with +CMake + vcpkg on the centrally-pinned **clang-22 / libc++** toolchain, linked +against `ggml` / `llama.cpp` / `whisper.cpp` / `stable-diffusion.cpp`. ### Addon inventory @@ -67,9 +65,6 @@ C++20 Bare addons built with CMake + vcpkg on the centrally-pinned | `diffusion-cpp` | stable-diffusion.cpp | image / video generation | ✅ | | `fabric` | shared ggml + llama.cpp runtime host | infra (no direct input parsing) | ❌ | -The ONNX Runtime addons (`onnx`, `ocr-onnx`) are intentionally excluded from -this effort and are not listed above. - Fuzzing targets **pure parse/transform functions** that consume attacker-influenceable bytes. It deliberately does **not** target full `runJob` / end-to-end inference paths (non-deterministic, backend-stateful, and @@ -357,7 +352,7 @@ rest come from vcpkg is what would put two Abseils in one link. `ports/abseil/` in `qvac-registry-vcpkg`, at version `20260526.0`. It replaces the previous onnxruntime-pinned port (`version-string: "onnxruntime"`, Abseil `20240722.0`) **at HEAD only**: both old entries stay in `versions/a-/abseil.json`, -so `ocr-onnx` keeps resolving what it resolves today from its pinned baseline. The +so anything on an older baseline keeps resolving what it resolves today. The manifest asks for `abseil[asan]` with `"version>=": "20260526.0"`, so the floor is stated where the dependency is declared rather than left to the baseline. @@ -429,9 +424,9 @@ headers carry no stability promise. A FuzzTest bump should re-check the closure above (`rg '#include "(re2|util)/' fuzztest-src/fuzztest/`) before bumping the port. -Nothing else resolves `re2` from this registry: `packages/onnx` and -`packages/translation-nmtcpp` both route it to the Microsoft registry through -their `vcpkg-configuration.json` allowlists. +Nothing else resolves `re2` from this registry: `packages/translation-nmtcpp` +routes it to the Microsoft registry through its `vcpkg-configuration.json` +allowlist. ### The `fuzztest` registry port diff --git a/docs/repository-layout.md b/docs/repository-layout.md index 8fe356269d..9d611ef672 100644 --- a/docs/repository-layout.md +++ b/docs/repository-layout.md @@ -21,7 +21,6 @@ Legend: | langdetect-text | Language detection library providing an interface for detecting the language of given text | Addon | | llm-llamacpp | Native C++ addon for running Large Language Models (LLMs) via `qvac-fabric-llm.cpp` | Addon | | ocr-ggml | Optical Character Recognition (OCR) addon (EasyOCR pipeline) on the GGML backend | Addon | -| onnx | Bare addon for ONNX Runtime session management | Addon | | rag | JavaScript library for Retrieval-Augmented Generation (RAG) with document ingestion, vector search, and LLM integration | Addon | | transcription-parakeet | Speech-to-text (ASR) and Sortformer speaker-diarization addon using NVIDIA Parakeet models | Addon | | transcription-whispercpp | Whisper-based audio transcription addon via `qvac-ext-lib-whisper.cpp` | Addon | diff --git a/packages/fabric/README.md b/packages/fabric/README.md index 7736a8a68f..d3061c42e0 100644 --- a/packages/fabric/README.md +++ b/packages/fabric/README.md @@ -7,9 +7,9 @@ npm dependency and dynamically link against it, so the multi-hundred-megabyte llama/ggml runtime is **built once** and **loaded once per process** instead of being statically embedded into every addon. -It is modeled on [`@qvac/onnx`](../onnx) and follows the same -npm + `prebuilds/` + `include_bare_module(... PREBUILD)` + companion-`.bare` -pattern. See [INTEGRATION.md](./INTEGRATION.md) for the consumer guide. +It follows the npm + `prebuilds/` + `include_bare_module(... PREBUILD)` + +companion-`.bare` pattern. See [INTEGRATION.md](./INTEGRATION.md) for the +consumer guide. ## What it ships diff --git a/packages/ocr-ggml/.agent/knowledge/vcpkg-management.md b/packages/ocr-ggml/.agent/knowledge/vcpkg-management.md index 7e8691d02b..8a39f53d39 100644 --- a/packages/ocr-ggml/.agent/knowledge/vcpkg-management.md +++ b/packages/ocr-ggml/.agent/knowledge/vcpkg-management.md @@ -279,22 +279,6 @@ Triplets that genuinely differ from the shared defaults stay in the package's ow - **parakeet / tts-onnx**: a `x64-linux.cmake` variant that adds `-Wno-array-bounds`, plus their macOS/iOS triplets. Local linux triplets chainload the shared toolchain via `${CMAKE_CURRENT_LIST_DIR}/../../../../vcpkg-overlays/toolchains/linux-clang.cmake`. -- **onnx**: keeps `vcpkg-override-triplets/triplets/` (apple/unix/android), layered - ahead of the shared folder. - -#### ONNX Packages — Release-Only Triplets - -Location: `packages/onnx/vcpkg-override-triplets/triplets/` and the local -`vcpkg/triplets/` of `tts-onnx`, `transcription-parakeet`. - -These local triplets set `VCPKG_BUILD_TYPE release` to halve build time and disk -usage, and platform-specific deployment targets (e.g., iOS 13.3): - -```cmake -set(VCPKG_BUILD_TYPE release) -set(VCPKG_LIBRARY_LINKAGE static) -``` - ## Overlay Ports Overlay ports let a package override a registry port with a local version. They are declared in `vcpkg-configuration.json`: diff --git a/packages/onnx/.gitignore b/packages/onnx/.gitignore deleted file mode 100644 index fdacfb7e69..0000000000 --- a/packages/onnx/.gitignore +++ /dev/null @@ -1,12 +0,0 @@ -.idea/ -.vs/ -build/ -prebuilds/ -node_modules/ -dist/ -package-lock.json - -# Added by lint-cpp -.clang-format -.clang-tidy -.valgrind.supp diff --git a/packages/onnx/CHANGELOG.md b/packages/onnx/CHANGELOG.md deleted file mode 100644 index cc32bae2f7..0000000000 --- a/packages/onnx/CHANGELOG.md +++ /dev/null @@ -1,141 +0,0 @@ -# Changelog - -## [0.15.1] - 2026-06-22 - -### Changed - -- Windows prebuilds now link the static Visual C++ runtime (`/MT`) instead of - importing `vcruntime140.dll`, `msvcp140.dll`, or UCRT DLLs from the MSVC - redistributable. Shared monorepo `vcpkg-overlays/triplets/{x64,arm64}-windows.cmake` - build vcpkg dependencies with a static CRT; addon CMake no longer links - `msvcrt.lib`, which had forced the dynamic runtime. `INTEGRATION.md` now - documents that consumer addons must not link `msvcrt.lib` for the same reason. - No public API change. - -## Pull Requests - -- [#2722](https://github.com/tetherto/qvac/pull/2722) - QVAC-21100: Switch to static C/C++ windows runtimes - -## [0.15.0] - 2026-05-11 - -### Changed - -- Updated package's directory structure to be more consistent with the remaning QVAC packages. - -## [0.14.1] - 2026-04-21 - -### Fixed - -- Patched ONNX Runtime 1.24.2 CoreML EP to fix `!model_path.empty()` crash when loading models with external data files (`.onnx_data` / `.onnx.data`) on macOS/iOS. Root cause: `TensorProtoWithExternalDataToTensorProto` passed the full model file path instead of its parent directory to `ReadExternalDataForTensor`. Backport of [microsoft/onnxruntime#28062](https://github.com/microsoft/onnxruntime/pull/28062). Upstream issue: [microsoft/onnxruntime#28005](https://github.com/microsoft/onnxruntime/issues/28005). - -### Changed - -- Pinned onnxruntime dependency to `>= 1.24.2#5` in `vcpkg.json` to pick up the CoreML fix from `qvac-registry-vcpkg`. - -## [0.14.0] - 2026-03-30 - -### Changed - -- Upgraded the bundled ONNX Runtime toolchain from 1.22.0 to 1.24.2 by pinning `onnx` to the updated `qvac-registry-vcpkg` baseline -- Restored the default XNNPack-enabled build against the compatible ONNX Runtime 1.24.2 registry revision -- Refreshed package metadata and release documentation to match the new runtime baseline and release version - -## [0.13.3] - 2026-03-18 - -### Changed - -- Logging adjustments - - -## [0.13.2] - 2026-03-16 - -### Fixed - -- Session creation fallback chain now catches `std::exception` (including `std::bad_alloc`) instead of only `Ort::Exception`, fixing DirectML OOM failures on Windows CI machines without a real GPU - -## [0.13.1] - 2026-03-16 - -### Added - -- Fallback chain in `OnnxSession` constructor: if session init fails with a non-CPU provider (e.g. DirectML OOM on CI machines without a real GPU), automatically retries with CPU-only configuration - -## [0.13.0] - 2026-03-13 - -### Added - -- `runRaw()` method on `OnnxSession` returning `std::vector` for zero-copy output access, avoiding the `memcpy` into `OutputTensor` on every inference call -- `inputName(size_t)` and `outputName(size_t)` accessors on `IOnnxSession` and `OnnxSession` for direct O(1) access to cached input/output names without querying the ORT API - -### Changed - -- `IOnnxSession` interface now requires `inputName()` and `outputName()` pure virtual methods -- Cached `Ort::MemoryInfo` as a class member instead of recreating it on every `run()` call -- Refactored `run()` to delegate to `runRaw()` internally (no behavior change for existing callers) - - -## [0.12.12] - 2026-03-12 - -### Fixed - -- Windows runtime loading: export `OrtGetApiBase` from the bare module using `/EXPORT` linker flag (a `.def` file overrides `WINDOWS_EXPORT_ALL_SYMBOLS`, suppressing the `bare_*`/`napi_*` auto-exports and causing DLL initialization failure) -- macOS runtime loading: set `INSTALL_NAME_DIR` to `@rpath` so that consumer addons can resolve the companion `qvac__onnx@0.bare` via their `@loader_path` rpath entries (cmake-bare's default empty install_name caused dyld to skip rpath search entirely) - - -## [0.12.10] - 2026-03-12 - -### Fixed - -- Windows C++20 clang-cl build: replaced legacy `OrtSessionOptionsAppendExecutionProvider_DML` C API with generic `AppendExecutionProvider("DML")`, removing `#include ` which pulled in the Windows SDK and caused `byte` ambiguity with `std::byte` - - -## [0.12.8] - 2026-03-11 - -### Added - -- Android logger - -### Fixed - -- Added exception handler for com.ms.internal.nhwc schemas issue - - -## [0.12.1] - 2026-03-05 - -### Fixed - -- Failed CI sanity checks -- CI build errors for android, osx, and ios - - -## [0.12.0] - 2026-03-04 - -### Added - -- Full bare addon architecture with C++ binding layer and JavaScript API -- New JS API: `configureEnvironment()`, `getAvailableProviders()`, `createSession()`, `getInputInfo()`, `getOutputInfo()`, `run()`, `destroySession()` -- New C++ headers: `OnnxConfig.hpp` (configuration enums/structs) -- INTEGRATION.md consumer guide - -### Changed - -- Refactored from header-only interface library (`add_library(INTERFACE)`) to bare addon module (`add_bare_module(EXPORTS)`) -- CMake minimum version raised to 3.25 -- XNNPack execution provider enabled by default - -### Fixed - -- Crash issue in session management -- Protobuf build errors -- Build errors encountered by consumer addons -- Package linked as dynamic (not static) for proper runtime behavior - ---- - -### Categories - -- **Added** for new features -- **Changed** for changes in existing functionality -- **Deprecated** for soon-to-be removed features -- **Removed** for now removed features -- **Fixed** for any bug fixes -- **Security** in case of vulnerabilities diff --git a/packages/onnx/CMakeLists.txt b/packages/onnx/CMakeLists.txt deleted file mode 100644 index caadaeb0cd..0000000000 --- a/packages/onnx/CMakeLists.txt +++ /dev/null @@ -1,323 +0,0 @@ -cmake_minimum_required(VERSION 3.25) - -option(BUILD_TESTING "Build tests" OFF) -option(ENABLE_XNNPACK "Enable XNNPack execution provider" ON) -option(QVAC_ONNX_ANDROID_LOG "Enable Android logcat debug logging" OFF) - -if(BUILD_TESTING) - list(APPEND VCPKG_MANIFEST_FEATURES "tests") -endif() - -if(ENABLE_XNNPACK) - list(APPEND VCPKG_MANIFEST_FEATURES "xnnpack") -endif() - -find_package(cmake-bare REQUIRED PATHS node_modules/cmake-bare) -find_package(cmake-vcpkg REQUIRED PATHS node_modules/cmake-vcpkg) - -# Package-local override triplets (apple/unix/android) take precedence; the -# shared monorepo triplets supply the static-CRT windows triplets. -set(VCPKG_OVERLAY_TRIPLETS "${CMAKE_CURRENT_SOURCE_DIR}/vcpkg-override-triplets/triplets;${CMAKE_CURRENT_SOURCE_DIR}/../../vcpkg-overlays/triplets;${VCPKG_OVERLAY_TRIPLETS}") - -set(VCPKG_INSTALL_OPTIONS --clean-after-build) - -file(READ "${CMAKE_CURRENT_SOURCE_DIR}/package.json" _pkg_json) -string(JSON _pkg_version GET "${_pkg_json}" "version") - -project(qvac-onnx VERSION ${_pkg_version} LANGUAGES C CXX) - -find_path(VCPKG_INSTALLED_PATH share/lint-cpp/.clang-format REQUIRED) - -include(CTest) - -set(CMAKE_CXX_STANDARD 20) -set(CMAKE_CXX_STANDARD_REQUIRED ON) -set(CMAKE_CXX_EXTENSIONS OFF) -set(CMAKE_POSITION_INDEPENDENT_CODE ON) - -# Patch cmake config files for cmake 4.x compatibility -foreach(_cmake4_fix_file - "${VCPKG_INSTALLED_PATH}/share/onnxruntime/onnxruntimeTargets.cmake" - "${VCPKG_INSTALLED_PATH}/share/protobuf/protobuf-targets.cmake") - if(EXISTS "${_cmake4_fix_file}") - file(READ "${_cmake4_fix_file}" _cmake4_fix_content) - # Strip LINK_ONLY generator expressions. - # Use [^";] to exclude " (property value boundary) and ; (entry separator) - # while still allowing nested genexes like $<$:Dbghelp>. - # Build pattern with string(CONCAT) to prevent cmake genex evaluation. - if(_cmake4_fix_content MATCHES "LINK_ONLY:") - string(CONCAT _lo_re "\\\\?" "\\$" [=[]=]) - string(REGEX REPLACE "${_lo_re}" "\\1" - _cmake4_fix_content "${_cmake4_fix_content}") - endif() - file(WRITE "${_cmake4_fix_file}" "${_cmake4_fix_content}") - endif() -endforeach() - -find_package(unofficial-pthreadpool CONFIG QUIET) -find_package(onnxruntime CONFIG REQUIRED) - -set(QVAC_ONNX_ORT_TARGET onnxruntime::onnxruntime) - -add_bare_module(qvac-onnx EXPORTS) - -target_sources(${qvac-onnx} PRIVATE addon/binding.cpp) -target_include_directories(${qvac-onnx} PRIVATE src) -target_link_libraries(${qvac-onnx} PRIVATE ${QVAC_ONNX_ORT_TARGET}) - -if(QVAC_ONNX_ANDROID_LOG) - target_compile_definitions(${qvac-onnx} PRIVATE QVAC_ONNX_ENABLE_ANDROID_LOG) -endif() - -# Override install_name so that consumers using @loader_path rpath can resolve -# this library as a companion dependency (cmake-bare sets INSTALL_NAME_DIR "" -# which produces a bare filename install_name that dyld cannot resolve via rpath). -if(APPLE) - set_target_properties(${qvac-onnx}_module PROPERTIES - INSTALL_NAME_DIR "@rpath" - BUILD_WITH_INSTALL_NAME_DIR ON) -endif() - -# Version script -if(UNIX AND NOT APPLE) - target_link_options(${qvac-onnx}_module PRIVATE - -Wl,--version-script=${CMAKE_CURRENT_SOURCE_DIR}/symbols.map) -elseif(APPLE) - target_link_options(${qvac-onnx}_module PRIVATE - -Wl,-exported_symbols_list,${CMAKE_CURRENT_SOURCE_DIR}/exports.txt) -elseif(WIN32) - # Use /EXPORT instead of a .def file to avoid overriding the auto-generated - # exports from WINDOWS_EXPORT_ALL_SYMBOLS (which exports bare_* and napi_*). - target_link_options(${qvac-onnx}_module PRIVATE - /EXPORT:OrtGetApiBase) -endif() - -# Strip .bare module -if(CMAKE_STRIP AND NOT WIN32) - if(APPLE) - set(_strip_flags -x) - else() - set(_strip_flags --strip-unneeded) - endif() - add_custom_command( - TARGET ${qvac-onnx}_module POST_BUILD - COMMAND ${CMAKE_STRIP} ${_strip_flags} $ - COMMENT "Stripping ${qvac-onnx}_module") -endif() - -# --- Export headers and cmake config for consumer addons --- -add_library(qvac-onnx-headers INTERFACE) -set_target_properties(qvac-onnx-headers PROPERTIES EXPORT_NAME headers) -target_include_directories(qvac-onnx-headers INTERFACE - $ - $ - $) - -if(QVAC_ONNX_ANDROID_LOG) - target_compile_definitions(qvac-onnx-headers INTERFACE QVAC_ONNX_ENABLE_ANDROID_LOG) -endif() - -# --- Installs --- -install(DIRECTORY src/qvac-onnx DESTINATION include FILES_MATCHING PATTERN "*.hpp") - -install(DIRECTORY ${VCPKG_INSTALLED_PATH}/include/ DESTINATION include - PATTERN "gtest" EXCLUDE PATTERN "gmock" EXCLUDE PATTERN "pybind11" EXCLUDE - PATTERN "benchmark" EXCLUDE PATTERN "Eigen" EXCLUDE PATTERN "eigen3" EXCLUDE - PATTERN "unsupported" EXCLUDE PATTERN "signature_of_eigen3_matrix_library" EXCLUDE - PATTERN "absl" EXCLUDE PATTERN "google" EXCLUDE PATTERN "nlohmann" EXCLUDE - PATTERN "boost" EXCLUDE PATTERN "flatbuffers" EXCLUDE PATTERN "onnx" EXCLUDE - PATTERN "cxxopts.hpp" EXCLUDE PATTERN "utf8_range.h" EXCLUDE - PATTERN "utf8_validity.h" EXCLUDE PATTERN "experiments-config.h" EXCLUDE) - -# Mobile: also install static libs and cmake configs for static linking -if(ANDROID OR (APPLE AND CMAKE_SYSTEM_NAME STREQUAL "iOS")) - install(DIRECTORY ${VCPKG_INSTALLED_PATH}/lib/ DESTINATION lib - FILES_MATCHING PATTERN "*.a" PATTERN "*.lib" - PATTERN "pkgconfig" EXCLUDE PATTERN "libprotoc*" EXCLUDE PATTERN "libbenchmark*" EXCLUDE) - - install(CODE [=[ - if(NOT WIN32) - find_program(_strip strip) - if(_strip) - file(GLOB _libs "${CMAKE_INSTALL_PREFIX}/lib/*.a") - foreach(_lib IN LISTS _libs) - if(APPLE) - execute_process(COMMAND "${_strip}" -x "${_lib}" ERROR_QUIET) - else() - execute_process(COMMAND "${_strip}" --strip-unneeded "${_lib}" ERROR_QUIET) - endif() - endforeach() - endif() - endif() - ]=]) - - install(DIRECTORY ${VCPKG_INSTALLED_PATH}/share/ DESTINATION share - PATTERN "gtest" EXCLUDE PATTERN "pybind11" EXCLUDE PATTERN "benchmark" EXCLUDE - PATTERN "cxxopts" EXCLUDE PATTERN "lint-cpp" EXCLUDE PATTERN "pkgconfig" EXCLUDE - PATTERN "vcpkg-cmake" EXCLUDE PATTERN "vcpkg-cmake-config" EXCLUDE - PATTERN "vcpkg-boost" EXCLUDE PATTERN "vcpkg.spdx.json" EXCLUDE - PATTERN "vcpkg_abi_info.txt" EXCLUDE PATTERN "copyright" EXCLUDE) - - # Patch shipped protobuf cmake configs - install(CODE [=[ - set(_dir "${CMAKE_INSTALL_PREFIX}/share/protobuf") - set(_f "${_dir}/protobuf-targets.cmake") - if(EXISTS "${_f}") - file(READ "${_f}" _c) - string(REPLACE - "# Create imported target protobuf::protoc\nadd_executable(protobuf::protoc IMPORTED)\n" - "" _c "${_c}") - string(REGEX REPLACE - "# Create imported target protobuf::libprotoc\nadd_library\\(protobuf::libprotoc STATIC IMPORTED\\)\n\nset_target_properties\\(protobuf::libprotoc PROPERTIES\n[^)]*\\)\n" - "" _c "${_c}") - string(REPLACE " protobuf::protoc" "" _c "${_c}") - string(REPLACE " protobuf::libprotoc" "" _c "${_c}") - file(WRITE "${_f}" "${_c}") - endif() - set(_f "${_dir}/protobuf-targets-release.cmake") - if(EXISTS "${_f}") - file(READ "${_f}" _c) - string(REGEX REPLACE - "# Import target .protobuf::protoc. for configuration .[^.]*.\nset_property\\(TARGET protobuf::protoc[^\n]*\nset_target_properties\\(protobuf::protoc[^\n]*\n[^\n]*\n[^\n]*\n\nlist\\(APPEND _cmake_import_check_targets protobuf::protoc \\)\nlist\\(APPEND _cmake_import_check_files_for_protobuf::protoc[^\n]*\n" - "" _c "${_c}") - string(REGEX REPLACE - "# Import target .protobuf::libprotoc. for configuration .[^.]*.\nset_property\\(TARGET protobuf::libprotoc[^\n]*\nset_target_properties\\(protobuf::libprotoc[^\n]*\n[^\n]*\n[^\n]*\n[^\n]*\n\nlist\\(APPEND _cmake_import_check_targets protobuf::libprotoc \\)\nlist\\(APPEND _cmake_import_check_files_for_protobuf::libprotoc[^\n]*\n" - "" _c "${_c}") - file(WRITE "${_f}" "${_c}") - endif() - ]=]) - - # Patch shipped flatbuffers cmake configs - install(CODE [=[ - set(_dir "${CMAKE_INSTALL_PREFIX}/share/flatbuffers") - file(REMOVE "${_dir}/FlatcTargets.cmake" "${_dir}/FlatcTargets-release.cmake") - ]=]) - - # Patch shipped onnxruntime cmake configs - install(CODE [=[ - set(_dir "${CMAKE_INSTALL_PREFIX}/share/onnxruntime") - set(_f "${_dir}/onnxruntimeTargets.cmake") - if(EXISTS "${_f}") - file(READ "${_f}" _c) - string(REGEX REPLACE - "# Create imported target onnxruntime::onnxruntime\nadd_library\\(onnxruntime::onnxruntime SHARED IMPORTED\\)\n\nset_target_properties\\(onnxruntime::onnxruntime PROPERTIES\n[^)]*\\)\n" - "" _c "${_c}") - string(REPLACE " onnxruntime::onnxruntime " " " _c "${_c}") - file(WRITE "${_f}" "${_c}") - endif() - set(_f "${_dir}/onnxruntimeTargets-release.cmake") - if(EXISTS "${_f}") - file(READ "${_f}" _c) - string(REGEX REPLACE - "# Import target .onnxruntime::onnxruntime. for configuration .[^.]*.\nset_property\\(TARGET onnxruntime::onnxruntime APPEND[^\n]*\nset_target_properties\\(onnxruntime::onnxruntime PROPERTIES\n[^)]*\\)\n\nlist\\(APPEND _cmake_import_check_targets onnxruntime::onnxruntime \\)\nlist\\(APPEND _cmake_import_check_files_for_onnxruntime::onnxruntime[^\n]*\n" - "" _c "${_c}") - file(WRITE "${_f}" "${_c}") - endif() - ]=]) - - # Add missing find_dependency to shipped onnxruntimeConfig - install(CODE [=[ - set(_f "${CMAKE_INSTALL_PREFIX}/share/onnxruntime/onnxruntimeConfig.cmake") - if(EXISTS "${_f}") - file(READ "${_f}" _c) - string(REPLACE - "include(\"\${CMAKE_CURRENT_LIST_DIR}/onnxruntimeTargets.cmake\")" - "find_dependency(unofficial-pthreadpool CONFIG)\ninclude(\"\${CMAKE_CURRENT_LIST_DIR}/onnxruntimeTargets.cmake\")" - _c "${_c}") - file(WRITE "${_f}" "${_c}") - endif() - ]=]) - - # Patch shipped Eigen3 cmake configs — remove non-existent include dir - install(CODE [=[ - set(_f "${CMAKE_INSTALL_PREFIX}/share/eigen3/Eigen3Targets.cmake") - if(EXISTS "${_f}") - file(READ "${_f}" _c) - string(REGEX REPLACE - "set_target_properties\\(Eigen3::Eigen PROPERTIES\n INTERFACE_INCLUDE_DIRECTORIES[^\n]*\n\\)\n" - "" - _c "${_c}") - file(WRITE "${_f}" "${_c}") - endif() - ]=]) - - # Strip LINK_ONLY from shipped cmake configs for cmake 4.x consumers - install(CODE [=[ - foreach(_lo_file - "${CMAKE_INSTALL_PREFIX}/share/onnxruntime/onnxruntimeTargets.cmake" - "${CMAKE_INSTALL_PREFIX}/share/protobuf/protobuf-targets.cmake") - if(EXISTS "${_lo_file}") - file(READ "${_lo_file}" _c) - string(CONCAT _lo_re "\\\\?" "\\$" [==[]==]) - string(REGEX REPLACE "${_lo_re}" "\\1" _c "${_c}") - file(WRITE "${_lo_file}" "${_c}") - endif() - endforeach() - ]=]) -endif() - -# --- qvac-onnx cmake package config --- -install(TARGETS qvac-onnx-headers EXPORT qvac-onnxTargets) -install(EXPORT qvac-onnxTargets NAMESPACE qvac-onnx:: DESTINATION share/qvac-onnx/cmake) - -include(CMakePackageConfigHelpers) -configure_package_config_file( - cmake/qvac-onnxConfig.cmake.in - ${CMAKE_CURRENT_BINARY_DIR}/qvac-onnxConfig.cmake - INSTALL_DESTINATION share/qvac-onnx/cmake) -write_basic_package_version_file( - ${CMAKE_CURRENT_BINARY_DIR}/qvac-onnxConfigVersion.cmake - VERSION ${PROJECT_VERSION} COMPATIBILITY SameMajorVersion) -install(FILES - ${CMAKE_CURRENT_BINARY_DIR}/qvac-onnxConfig.cmake - ${CMAKE_CURRENT_BINARY_DIR}/qvac-onnxConfigVersion.cmake - DESTINATION share/qvac-onnx/cmake) - -# Android-specific -function(strip_libraries LIB_DIR LIB_NAMES) - foreach(LIB_NAME IN LISTS LIB_NAMES) - set(LIB_PATH "${LIB_DIR}/${LIB_NAME}") - execute_process(COMMAND ${LLVM_STRIP} --strip-debug "${LIB_PATH}" ERROR_VARIABLE STRIP_ERROR) - message("-- Stripping ${LIB_PATH} errorCode: ${STRIP_ERROR}") - endforeach() -endfunction() - -if(ANDROID) - find_package(Vulkan REQUIRED) - target_link_libraries(${qvac-onnx} PRIVATE ${Vulkan_LIBRARY} log) - find_program(LLVM_STRIP llvm-strip) - if(NOT LLVM_STRIP) - message(WARNING "-- llvm-strip not found. Skipping library stripping for Android build.") - else() - set(LIBRARIES_TO_STRIP_IN_LIB_DIR libonnx.a libprotobuf.a) - set(VCPKG_INSTALLED_DIR "${CMAKE_BINARY_DIR}/_vcpkg/${VCPKG_TARGET_TRIPLET}") - strip_libraries("${VCPKG_INSTALLED_DIR}/lib" "${LIBRARIES_TO_STRIP_IN_LIB_DIR}") - endif() -endif() - -if(WIN32) - target_compile_options(${qvac-onnx} PRIVATE "/utf-8") - target_compile_definitions(${qvac-onnx} PUBLIC WIN32_LEAN_AND_MEAN NOMINMAX NOGDI) -endif() - -if(BUILD_TESTING) - find_package(GTest CONFIG REQUIRED) - include(GoogleTest) - enable_testing() - - add_executable(unit_tests - tests/tensor_types_test.cpp tests/config_test.cpp tests/interface_test.cpp) - target_include_directories(unit_tests PRIVATE src) - target_link_libraries(unit_tests PRIVATE GTest::gmock_main) - gtest_discover_tests(unit_tests) - - add_executable(ort_tests - tests/runtime_test.cpp tests/type_conversions_test.cpp - tests/session_options_test.cpp tests/session_test.cpp - tests/addon_shared_runtime_test.cpp) - target_include_directories(ort_tests PRIVATE src) - target_compile_definitions(ort_tests PRIVATE - TEST_FIXTURES_DIR="${CMAKE_CURRENT_SOURCE_DIR}/tests/fixtures") - target_link_libraries(ort_tests PRIVATE GTest::gmock_main ${QVAC_ONNX_ORT_TARGET}) - gtest_discover_tests(ort_tests) -endif() diff --git a/packages/onnx/INTEGRATION.md b/packages/onnx/INTEGRATION.md deleted file mode 100644 index f87916e3ab..0000000000 --- a/packages/onnx/INTEGRATION.md +++ /dev/null @@ -1,632 +0,0 @@ -# Integrating @qvac/onnx into a Consumer Addon - -This guide covers all steps needed for an ONNX-based consumer addon to depend on and use `@qvac/onnx`. It uses `@qvac/ocr-onnx` as a concrete reference implementation throughout. - -> **Note:** the reference package `@qvac/ocr-onnx` (`packages/ocr-onnx`) was retired and removed from the monorepo (QVAC-22515) — OCR now runs on the GGML backend via `packages/ocr-ggml`. The integration snippets below remain valid as the worked example for future ONNX consumers; browse the removed package in git history (tag `ocr-onnx-v0.7.2`). - -## Overview - -`@qvac/onnx` is distributed as an **npm package** (bare addon). It ships everything a consumer addon needs to build against ONNX Runtime: - -- **qvac-onnx C++ headers** (`prebuilds/include/qvac-onnx/`) — header-only `OnnxSession`, `OnnxRuntime`, config types, tensor types -- **ONNX Runtime headers** (`prebuilds/include/onnxruntime/`) — public ORT C/C++ API headers -- **CMake config** (`prebuilds/share/qvac-onnx/`) — `find_package(qvac-onnx)` exposes: - - `qvac-onnx::headers` — compile-time headers (always available) - - `qvac-onnx::qvac-onnx-static` — static ORT linking (mobile builds only, when `prebuilds/share/onnxruntime/` exists) -- **Prebuilt `.bare` shared library** (`prebuilds//qvac__onnx.bare`) — exports `OrtGetApiBase` and EP registration symbols; desktop consumers dynamically link against this -- **JS API** — `configureEnvironment`, `getAvailableProviders`, `createSession`, `run`, `destroySession`, etc. (see [README.md](./README.md)) - -### Desktop vs Mobile - -- **Desktop** (Linux, macOS, Windows): Consumer addons dynamically link against `@qvac/onnx.bare` via `include_bare_module`. ORT symbols (`OrtGetApiBase`, etc.) are resolved at runtime from the shared `.bare`. This means ORT is loaded once per process, regardless of how many ONNX-based addons are loaded. -- **Mobile** (Android, iOS): Controlled by the `MOBILE_DYNAMIC_LINK` CMake option (default `ON`). When `ON`, mobile builds use the same dynamic linking as desktop. When `OFF`, consumer addons statically link via `qvac-onnx::qvac-onnx-static`, which transitively provides `onnxruntime::onnxruntime_static`. - -Consumer addons do **not** need `onnxruntime` in their own `vcpkg.json`. The ONNX Runtime comes bundled with `@qvac/onnx`. - ---- - -## Step 1 — npm dependency - -Add `@qvac/onnx` to the consumer's `package.json`: - -```json -{ - "dependencies": { - "@qvac/onnx": "^0.13.3" - }, - "devDependencies": { - "cmake-bare": "^1.7.5", - "cmake-vcpkg": "^1.1.0" - } -} -``` - -After `npm install`, the headers, prebuilt `.bare` shared library, and cmake configs are available under `node_modules/@qvac/onnx/prebuilds/`. - -**How `ocr-onnx` does it:** `@qvac/ocr-onnx` depends on `"@qvac/onnx": "^0.13.3"` in its `dependencies`. It also depends on `@qvac/infer-base` (shared base class for ONNX inference addons), `@qvac/error`, and `@qvac/response` for the JS layer, and `opencv4` (via vcpkg) for image processing in C++. - ---- - -## Step 2 — vcpkg manifest (`vcpkg.json`) - -The consumer's `vcpkg.json` only needs its own addon-specific dependencies. ONNX Runtime and its transitive dependencies are provided by `@qvac/onnx` via npm. - -```json -{ - "name": "my-consumer-addon", - "version": "1.0.0", - "dependencies": [ - { - "name": "inference-addon-cpp", - "version>=": "1.0.0" - }, - { - "name": "lint-cpp", - "version>=": "1.4.4" - } - ], - "features": { - "tests": { - "description": "Build tests", - "dependencies": ["gtest"] - } - } -} -``` - -Add any addon-specific vcpkg dependencies here. Do **not** add `onnxruntime`. - -**How `ocr-onnx` does it:** Its `vcpkg.json` lists `opencv4` (with specific features: `jpeg`, `png`, `quirc`, `tiff`, `webp`), `inference-addon-cpp`, and `lint-cpp`. No `onnxruntime` — that comes from `@qvac/onnx`. - ---- - -## Step 3 — vcpkg registry configuration (`vcpkg-configuration.json`) - -Ensure the consumer's `vcpkg-configuration.json` includes the Tether registry as default and the Microsoft registry for any upstream packages the addon itself needs: - -```json -{ - "default-registry": { - "kind": "git", - "baseline": "", - "repository": "https://github.com/tetherto/qvac-registry-vcpkg.git" - }, - "registries": [ - { - "kind": "git", - "baseline": "8c901fe2b0e69a542d02810d4089505fd0c480d8", - "repository": "https://github.com/microsoft/vcpkg", - "packages": [ - "gtest" - ] - } - ] -} -``` - -Add only the Microsoft registry packages your addon directly depends on. Packages previously required for onnxruntime (flatbuffers, re2, abseil, eigen3, etc.) are no longer needed here — they ship with `@qvac/onnx`. - -**How `ocr-onnx` does it:** Its `vcpkg-configuration.json` lists the Microsoft registry packages for `opencv4` and its transitive dependencies (libjpeg-turbo, libpng, libwebp, zlib, etc.) plus `gtest`. The list is large because OpenCV pulls in many upstream deps, but none are onnxruntime-related. - ---- - -## Step 4 — CMakeLists.txt - -### Finding @qvac/onnx - -A single `find_package` call discovers headers and cmake targets: - -```cmake -cmake_minimum_required(VERSION 3.25) - -find_package(cmake-bare REQUIRED PATHS node_modules/cmake-bare) -find_package(cmake-vcpkg REQUIRED PATHS node_modules/cmake-vcpkg) - -project(my-consumer-addon VERSION 1.0.0 LANGUAGES C CXX) - -set(CMAKE_CXX_STANDARD 20) -set(CMAKE_CXX_STANDARD_REQUIRED ON) -set(CMAKE_CXX_EXTENSIONS OFF) -set(CMAKE_POSITION_INDEPENDENT_CODE ON) - -# --- Find @qvac/onnx (provides headers + cmake targets) --- -set(qvac-onnx_DIR "${CMAKE_CURRENT_SOURCE_DIR}/node_modules/@qvac/onnx/prebuilds/share/qvac-onnx/cmake") -find_package(qvac-onnx CONFIG REQUIRED) - -# --- Define bare addon --- -add_bare_module(my-consumer-addon EXPORTS) - -target_sources(${my-consumer-addon} PRIVATE addon/binding.cpp) - -# Route ONNX session logs through JsLogger -target_compile_definitions(${my-consumer-addon} PRIVATE JS_LOGGER) -``` - -**How `ocr-onnx` does it:** It sets the cmake config path explicitly via `set(qvac-onnx_DIR ...)` rather than passing `PATHS` to `find_package`: - -```cmake -set(qvac-onnx_DIR "${CMAKE_CURRENT_SOURCE_DIR}/node_modules/@qvac/onnx/prebuilds/share/qvac-onnx/cmake") -find_package(qvac-onnx CONFIG REQUIRED) -``` - -### Linking — desktop vs mobile - -Consumer addons must use platform-conditional linking. The `MOBILE_DYNAMIC_LINK` CMake option (default `ON`) controls whether mobile builds use dynamic or static linking: - -```cmake -option(MOBILE_DYNAMIC_LINK "Use dynamic linking for ONNX Runtime on mobile" ON) - -if((ANDROID OR (APPLE AND CMAKE_SYSTEM_NAME STREQUAL "iOS")) AND NOT MOBILE_DYNAMIC_LINK) - # Mobile (static): each addon embeds ONNX Runtime - target_link_libraries(${my-consumer-addon} PRIVATE - qvac-onnx::qvac-onnx-static - ) -else() - # Desktop and mobile (dynamic): link against @qvac/onnx.bare shared module - include_bare_module("@qvac/onnx" qvac_onnx_target PREBUILD) - - # Headers for compile-time (OnnxSession.hpp, onnxruntime_cxx_api.h, etc.) - target_link_libraries(${my-consumer-addon} PRIVATE - qvac-onnx::headers - ) - - # Dynamic link — adds DT_NEEDED: qvac__onnx@0.bare - target_link_libraries(${my-consumer-addon}_module PRIVATE - ${qvac_onnx_target}_module - ) - - # Install @qvac/onnx.bare as companion library alongside the consumer .bare - bare_target(host) - bare_module_target("." _unused NAME addon_name) - install(FILES $ - DESTINATION ${host}/${addon_name} - RENAME qvac__onnx@0.bare) -endif() -``` - -**How `ocr-onnx` does it:** The `ocr-onnx` CMakeLists.txt uses exactly this pattern. Dynamic linking is the default path for both desktop and mobile. The `else()` branch handles desktop + dynamic mobile, while `MOBILE_DYNAMIC_LINK=OFF` falls back to static linking for mobile targets: - -```cmake -if((ANDROID OR (APPLE AND CMAKE_SYSTEM_NAME STREQUAL "iOS")) AND NOT MOBILE_DYNAMIC_LINK) - target_link_libraries( - ${inference-addon-onnx-ocr-fasttext} - PRIVATE - ${OpenCV_LIBS} - qvac-onnx::qvac-onnx-static - ) -else() - include_bare_module("@qvac/onnx" qvac_onnx_target PREBUILD) - - target_link_libraries( - ${inference-addon-onnx-ocr-fasttext} - PRIVATE - ${OpenCV_LIBS} - qvac-onnx::headers - ) - target_link_libraries( - ${inference-addon-onnx-ocr-fasttext}_module - PRIVATE - ${qvac_onnx_target}_module - ) - - bare_target(host) - bare_module_target("." _unused NAME addon_name) - install(FILES $ - DESTINATION ${host}/${addon_name} - RENAME qvac__onnx@0.bare) -endif() -``` - -Note: `ocr-onnx` also links `${OpenCV_LIBS}` alongside the onnx targets — this is addon-specific. The onnx linking pattern is the same for all consumers. - -**How it works at runtime (desktop / dynamic mobile):** - -1. Consumer addon `.bare` has `DT_NEEDED: qvac__onnx@0.bare` -2. The dynamic linker resolves this via RPATH to the companion directory -3. If `qvac__onnx@0.bare` is already loaded (by another addon) → reuses it (SONAME match) -4. ORT symbols (`OrtGetApiBase`, etc.) resolve from the single loaded instance -5. All consumer addons share one ORT in memory - -**CMake targets:** - -| Target | Description | When available | -|--------|-------------|----------------| -| `qvac-onnx::headers` | Compile-time headers only (qvac-onnx + ORT public API) | Always | -| `qvac-onnx::qvac-onnx-static` | Headers + `onnxruntime::onnxruntime_static` | Mobile builds only (when `prebuilds/share/onnxruntime/` exists) | - -### Symbol visibility - -Consumer addons on desktop do **not** need a `symbols.map` or version script for ORT symbols. ORT symbols are resolved at runtime from the shared `@qvac/onnx.bare`, not statically linked into each consumer. - -Consumer addons typically use a standard visibility map that exports only `bare_*` and `napi_*` symbols: - -``` -{ - global: - bare_*; - napi_*; - local: - *; -}; -``` - -On mobile with static linking, symbol visibility is handled automatically by the platform's default linking behavior. - -### Platform-specific additions - -```cmake -# Android: Vulkan + log -if(ANDROID) - find_package(Vulkan REQUIRED) - target_link_libraries(${my-consumer-addon} PRIVATE ${Vulkan_LIBRARY} log) -endif() - -# Windows: UTF-8, lean headers -if(WIN32) - target_compile_options(${my-consumer-addon} PRIVATE "/utf-8") - target_compile_definitions(${my-consumer-addon} PUBLIC - WIN32_LEAN_AND_MEAN NOMINMAX NOGDI) -endif() -``` - -> **Do not link `msvcrt.lib`.** The shared `vcpkg-overlays/triplets/{arm64,x64}-windows.cmake` -> build dependencies with the static MSVC runtime, and the bare-make win32 toolchain -> compiles the addon with the static runtime too. Linking `msvcrt.lib` (the dynamic -> CRT import library) reintroduces a runtime dependency on `vcruntime140.dll` / -> `msvcp140.dll`. Make sure the consumer addon also picks up the shared triplets via -> `VCPKG_OVERLAY_TRIPLETS` (see `vcpkg-management.md`). - -**How `ocr-onnx` does it:** Uses the exact same Android (Vulkan + log) and Windows (/utf-8, lean headers) blocks, plus Android-specific library stripping for OpenCV. - ---- - -## Step 5 — JS-side: pre-loading @qvac/onnx - -Consumer addons that dynamically link against `@qvac/onnx.bare` **must** pre-load it in their `binding.js` before calling `require.addon()`. This ensures the bare runtime has registered the `.bare` module before the dynamic linker tries to resolve it (required for Windows delay-load): - -```js -// Pre-load @qvac/onnx so its .bare module is registered with the bare runtime -// before our addon triggers Windows delay-load resolution of qvac__onnx@0.bare -// (bare_module_find requires modules to be already loaded). -require('@qvac/onnx') - -module.exports = require.addon() -``` - -**How `ocr-onnx` does it:** Its `binding.js` does exactly this — `require('@qvac/onnx')` followed by `module.exports = require.addon()`. This is the only place where `ocr-onnx` references the `@qvac/onnx` JS module. All ONNX inference in `ocr-onnx` happens through the C++ API (see Step 6). - ---- - -## Step 6 — C++ usage - -### Include headers - -All headers live under the `qvac-onnx/` include prefix: - -```cpp -#include // Concrete session (header-only, pulls in ORT) -#include // Abstract interface (ORT-free) -#include // Environment singleton, configure(), getAvailableProviders() -#include // SessionConfig, EnvironmentConfig, enums -#include // TensorInfo, InputTensor, OutputTensor, TensorType -``` - -### Configure the environment (optional) - -The environment is process-wide. Call `configure()` before any session is created to customize logging: - -```cpp -#include - -onnx_addon::EnvironmentConfig envCfg; -envCfg.loggingLevel = onnx_addon::LoggingLevel::INFO; -envCfg.loggingId = "my-addon"; - -onnx_addon::OnnxRuntime::configure(envCfg); // throws if instance() already called -``` - -If `configure()` is never called, defaults are used (`ERROR` level, `"qvac-onnx"` id). - -### Query available execution providers - -```cpp -auto providers = onnx_addon::OnnxRuntime::getAvailableProviders(); -// e.g. {"CPUExecutionProvider", "XnnpackExecutionProvider"} -``` - -### Create and run a session - -```cpp -#include -#include - -// Configure session -onnx_addon::SessionConfig config; -config.provider = onnx_addon::ExecutionProvider::AUTO_GPU; -config.optimization = onnx_addon::GraphOptimizationLevel::EXTENDED; -config.intraOpThreads = 4; -config.interOpThreads = 2; -config.enableMemoryPattern = true; -config.enableCpuMemArena = true; -config.enableXnnpack = true; -config.executionMode = onnx_addon::ExecutionMode::SEQUENTIAL; - -// Create session (automatic fallback: requested config → no XNNPACK → CPU-only) -onnx_addon::OnnxSession session("path/to/model.onnx", config); - -// Inspect model -auto inputs = session.getInputInfo(); // std::vector -auto outputs = session.getOutputInfo(); // std::vector - -// Direct name access (cached, avoids ORT API calls) -const std::string& inputName = session.inputName(0); -const std::string& outputName = session.outputName(0); - -// Prepare input tensor -onnx_addon::InputTensor input; -input.name = inputs[0].name; -input.shape = {1, 3, 224, 224}; -input.type = onnx_addon::TensorType::FLOAT32; -input.data = myFloatData.data(); -input.dataSize = myFloatData.size() * sizeof(float); - -// Run inference (deep copy of outputs) -auto results = session.run(input); - -// Access output -const auto& output = results[0]; -auto floatData = output.as(); // typed pointer access -``` - -### Zero-copy inference with `runRaw()` - -For performance-critical pipelines, `runRaw()` returns raw `Ort::Value` objects directly, avoiding the deep copy that `run()` performs: - -```cpp -// Returns std::vector — zero-copy output -auto ortValues = session.runRaw(input); - -// Access tensor data directly from ORT memory -auto typeInfo = ortValues[0].GetTypeInfo(); -auto tensorInfo = typeInfo.GetTensorTypeAndShapeInfo(); -const float* data = ortValues[0].GetTensorData(); -``` - -`runRaw()` is only available on `OnnxSession` (not `IOnnxSession`) since it exposes ORT types. - -### How `ocr-onnx` uses the C++ API - -`ocr-onnx` uses `@qvac/onnx` exclusively through the C++ header-only API — it does **not** call the JS API for inference. All pipeline steps use `runRaw()` for zero-copy performance. The integration pattern is: - -1. **`PipelineConfig` holds an `onnx_addon::SessionConfig`** — the JS layer passes provider/optimization/thread configuration down to C++ via the config struct. Defaults to `CPU` provider. - -2. **Each pipeline step owns an `onnx_addon::OnnxSession`** — there are four step classes that each create and hold their own session: - - `StepDetectionInference` — CRAFT text detection (EasyOCR mode) - - `StepRecognizeText` — CTC text recognition (EasyOCR mode) - - `StepDoctrDetection` — DBNet text detection (DocTR mode) - - `StepDoctrRecognition` — PARSeq/CRNN recognition (DocTR mode) - -3. **Sessions are constructed with model path + shared config:** - ```cpp - // In Pipeline constructor - stepDetection_ = std::make_unique( - pathDetector, config.sessionConfig, config.magRatio); - ``` - -4. **Each step includes ``** and uses the session member directly for inference, calling `session_.runRaw()` with prepared `InputTensor` objects built from OpenCV `cv::Mat` data. - -5. **Windows session lifetime workaround** — On Windows, all four pipeline steps defer session destruction to avoid an ORT global-state crash during process teardown: - ```cpp - #if defined(_WIN32) || defined(_WIN64) - ~StepDetectionInference() { deferWindowsSessionLeak(std::move(session_)); } - #endif - ``` - The `deferWindowsSessionLeak()` function moves the session to a leaked pointer that the OS reclaims on process exit, avoiding a crash in ORT's global cleanup. - -6. **Two pipeline modes** — Pipeline supports `EASYOCR` mode (3 steps: detect → bounding boxes → recognize) and `DOCTR` mode (2 steps: detect → recognize). Both modes share the same `SessionConfig`. - -### Use the abstract interface for decoupling - -If your addon wants to avoid pulling ONNX Runtime headers into every translation unit, use the abstract interface: - -```cpp -#include // No ORT dependency - -class MyPipeline { - std::unique_ptr session_; -public: - void setSession(std::unique_ptr s) { - session_ = std::move(s); - } - std::vector infer(const onnx_addon::InputTensor& in) { - return session_->run(in); - } -}; -``` - -Then construct the concrete session in the translation unit that links ORT: - -```cpp -#include -pipeline.setSession(std::make_unique(path, config)); -``` - -Note: `ocr-onnx` does not use the abstract interface — it includes `OnnxSession.hpp` directly in each pipeline step and stores `onnx_addon::OnnxSession` as a direct member. This is simpler when all translation units already link ORT headers, and allows access to `runRaw()` which is not part of the abstract interface. - -### Shared ORT runtime singleton - -`OnnxSession` internally uses `OnnxRuntime::instance()` — a process-wide Meyers singleton that creates a single `Ort::Env`. Multiple sessions across different consumer addons share the same runtime environment. You do not need to manage `Ort::Env` yourself. - ---- - -## Step 7 — JS-side usage (optional) - -If the consumer addon needs to call the `@qvac/onnx` JS API directly (rather than only using the C++ headers): - -```js -const onnx = require('@qvac/onnx') - -// Optional: configure environment before first session -onnx.configureEnvironment({ - loggingLevel: 'info', // 'verbose' | 'info' | 'warning' | 'error' | 'fatal' - loggingId: 'my-addon' -}) - -// Query available execution providers -const providers = onnx.getAvailableProviders() -// e.g. ['CPUExecutionProvider', 'XnnpackExecutionProvider'] - -// Create session -const handle = onnx.createSession('/path/to/model.onnx', { - provider: 'auto_gpu', - optimization: 'extended', - intraOpThreads: 4, - interOpThreads: 2, - enableXnnpack: true, - enableMemoryPattern: true, - enableCpuMemArena: true, - executionMode: 'sequential' -}) - -const inputInfo = onnx.getInputInfo(handle) -const outputInfo = onnx.getOutputInfo(handle) - -const results = onnx.run(handle, [{ - name: inputInfo[0].name, - shape: [1, 3, 224, 224], - type: 'float32', - data: new Float32Array(1 * 3 * 224 * 224) -}]) - -// results: [{ name, shape, type, data: Float32Array }] - -onnx.destroySession(handle) -``` - -**How `ocr-onnx` uses the JS API:** It does **not** use the JS API for inference at all. The only JS-side interaction is the `require('@qvac/onnx')` pre-load in `binding.js` (see Step 5). All session creation and inference happens through the C++ API in the pipeline steps. The JS layer (`ONNXOcr` class in `index.js`) passes configuration parameters down to C++ via `createInstance` / `runJob` bindings, and receives results via callbacks. - -The config parameters mapped from JS to `onnx_addon::SessionConfig` in the C++ binding layer (`AddonJs.hpp`) are: - -| JS parameter | C++ field | Mapping | -|-------------|-----------|---------| -| `useGPU` | `sessionConfig.provider` | `true` → `AUTO_GPU`, `false` → `CPU` | -| `graphOptimization` | `sessionConfig.optimization` | `"basic"`, `"extended"`, `"all"`, `"disable"` | -| `enableXnnpack` | `sessionConfig.enableXnnpack` | boolean | -| `enableCpuMemArena` | `sessionConfig.enableCpuMemArena` | boolean | -| `intraOpThreads` | `sessionConfig.intraOpThreads` | integer | - ---- - -## Step 8 — Build - -```bash -npm install # Resolves @qvac/onnx + devDependencies (cmake-bare, cmake-vcpkg) -npm run build # bare-make generate && bare-make build && bare-make install -``` - ---- - -## Thread pool configuration - -ONNX Runtime uses two thread pools per session: - -| Setting | What it controls | Default | -|---------|-----------------|---------| -| `intraOpThreads` | Parallelism **within** a single operator (e.g. matrix multiply) | `0` (all cores) | -| `interOpThreads` | Parallelism **between** independent operators in the graph | `0` (all cores) | -| `executionMode` | Whether independent operators run in parallel or sequentially | `"sequential"` | - -- **Sequential mode** (default): Operators run one at a time. Only intra-op parallelism is used. This is the safest default and recommended for most workloads. -- **Parallel mode**: Independent operators can run concurrently. Requires `interOpThreads > 1` to be effective. Useful for models with many independent branches. - -### Memory options - -| Setting | What it controls | Default | -|---------|-----------------|---------| -| `enableMemoryPattern` | Reuse memory allocations based on execution patterns | `true` | -| `enableCpuMemArena` | Use a memory arena for CPU allocations to reduce malloc overhead | `true` | - -DirectML on Windows automatically disables memory patterns and forces sequential mode — this is handled internally by the session options builder. - ---- - -## `ocr-onnx` integration summary - -Here is how `@qvac/ocr-onnx` integrates `@qvac/onnx` end-to-end: - -``` -JS layer (index.js) -│ ONNXOcr extends ONNXBase -│ Passes config params (useGPU, timeout, graphOptimization, -│ enableXnnpack, intraOpThreads, enableCpuMemArena, etc.) to C++ -│ -├── binding.js -│ require('@qvac/onnx') ← pre-load for Windows delay-load -│ require.addon() ← loads ocr-onnx .bare addon -│ -├── ocr-fasttext.js -│ OcrFasttextInterface wraps native binding calls -│ (createInstance, runJob, activate, destroy, cancel) -│ -└── C++ layer (addon/) - │ - ├── addon/AddonJs.hpp - │ Maps JS params → PipelineConfig with onnx_addon::SessionConfig - │ (useGPU → provider, graphOptimization → optimization, etc.) - │ - └── pipeline/ - ├── Pipeline.cpp - │ Constructs steps, passes onnx_addon::SessionConfig to each - │ Two modes: EASYOCR (3 steps) or DOCTR (2 steps) - │ - ├── StepDetectionInference ← owns onnx_addon::OnnxSession - │ #include - │ CRAFT text detection model — uses runRaw() for zero-copy - │ - ├── StepRecognizeText ← owns onnx_addon::OnnxSession - │ #include - │ CTC text recognition model — uses runRaw() for zero-copy - │ - ├── StepDoctrDetection ← owns onnx_addon::OnnxSession - │ #include - │ DBNet detection model — uses runRaw() for zero-copy - │ - └── StepDoctrRecognition ← owns onnx_addon::OnnxSession - #include - PARSeq/CRNN recognition model — uses runRaw() for zero-copy -``` - -Key takeaways from the `ocr-onnx` integration: -- All ONNX inference is C++-only; the JS API is not used for inference -- All pipeline steps use `runRaw()` for zero-copy output, avoiding `memcpy` of output tensors -- Each pipeline step creates its own `OnnxSession` from the shared `SessionConfig` -- The `binding.js` pre-load is essential for cross-platform module resolution -- OpenCV is the only additional native dependency (`vcpkg.json`) -- The `MOBILE_DYNAMIC_LINK` option allows switching between dynamic and static ORT linking on mobile -- Windows requires deferred session destruction (`deferWindowsSessionLeak`) to avoid ORT global-state crashes - ---- - -## Checklist - -| # | Step | What to verify | -|---|------|----------------| -| 1 | `package.json` | `@qvac/onnx` `^0.13.3` in `dependencies`; `cmake-bare` and `cmake-vcpkg` in `devDependencies` | -| 2 | `vcpkg.json` | `onnxruntime` is **not** listed (it ships with `@qvac/onnx`); only addon-specific deps remain | -| 3 | `vcpkg-configuration.json` | Tether registry as default; Microsoft registry only for addon-specific upstream packages | -| 4 | `CMakeLists.txt` | `find_package(qvac-onnx ...)`, platform guard with `qvac-onnx::headers` + `include_bare_module` (desktop/dynamic) or `qvac-onnx::qvac-onnx-static` (static mobile); `JS_LOGGER` defined | -| 5 | `binding.js` | `require('@qvac/onnx')` **before** `require.addon()` (Windows delay-load) | -| 6 | Companion lib | Desktop: `qvac__onnx@0.bare` installed in `prebuilds///` | -| 7 | C++ sources | Include `` instead of raw `` | -| 8 | Build | `npm run build` succeeds; `readelf -d` shows `NEEDED qvac__onnx@0.bare` (desktop) | - -## Supported Platforms - -| Platform | Execution Provider | Triplet | -|----------|--------------------|---------| -| Linux | XNNPack, CPU | `x64-linux` | -| macOS | CoreML, XNNPack, CPU | `arm64-osx` | -| Windows | DirectML, XNNPack, CPU | (default MSVC) | -| Android | NNAPI, XNNPack, CPU | `arm64-android` | -| iOS | CoreML, XNNPack, CPU | `arm64-ios` | -| iOS Sim | CoreML, XNNPack, CPU | `arm64-ios-simulator`, `x64-ios-simulator` | diff --git a/packages/onnx/LICENSE b/packages/onnx/LICENSE deleted file mode 100644 index f433b1a53f..0000000000 --- a/packages/onnx/LICENSE +++ /dev/null @@ -1,177 +0,0 @@ - - Apache License - Version 2.0, January 2004 - http://www.apache.org/licenses/ - - TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION - - 1. Definitions. - - "License" shall mean the terms and conditions for use, reproduction, - and distribution as defined by Sections 1 through 9 of this document. - - "Licensor" shall mean the copyright owner or entity authorized by - the copyright owner that is granting the License. - - "Legal Entity" shall mean the union of the acting entity and all - other entities that control, are controlled by, or are under common - control with that entity. For the purposes of this definition, - "control" means (i) the power, direct or indirect, to cause the - direction or management of such entity, whether by contract or - otherwise, or (ii) ownership of fifty percent (50%) or more of the - outstanding shares, or (iii) beneficial ownership of such entity. - - "You" (or "Your") shall mean an individual or Legal Entity - exercising permissions granted by this License. - - "Source" form shall mean the preferred form for making modifications, - including but not limited to software source code, documentation - source, and configuration files. - - "Object" form shall mean any form resulting from mechanical - transformation or translation of a Source form, including but - not limited to compiled object code, generated documentation, - and conversions to other media types. - - "Work" shall mean the work of authorship, whether in Source or - Object form, made available under the License, as indicated by a - copyright notice that is included in or attached to the work - (an example is provided in the Appendix below). - - "Derivative Works" shall mean any work, whether in Source or Object - form, that is based on (or derived from) the Work and for which the - editorial revisions, annotations, elaborations, or other modifications - represent, as a whole, an original work of authorship. For the purposes - of this License, Derivative Works shall not include works that remain - separable from, or merely link (or bind by name) to the interfaces of, - the Work and Derivative Works thereof. - - "Contribution" shall mean any work of authorship, including - the original version of the Work and any modifications or additions - to that Work or Derivative Works thereof, that is intentionally - submitted to Licensor for inclusion in the Work by the copyright owner - or by an individual or Legal Entity authorized to submit on behalf of - the copyright owner. For the purposes of this definition, "submitted" - means any form of electronic, verbal, or written communication sent - to the Licensor or its representatives, including but not limited to - communication on electronic mailing lists, source code control systems, - and issue tracking systems that are managed by, or on behalf of, the - Licensor for the purpose of discussing and improving the Work, but - excluding communication that is conspicuously marked or otherwise - designated in writing by the copyright owner as "Not a Contribution." - - "Contributor" shall mean Licensor and any individual or Legal Entity - on behalf of whom a Contribution has been received by Licensor and - subsequently incorporated within the Work. - - 2. Grant of Copyright License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - copyright license to reproduce, prepare Derivative Works of, - publicly display, publicly perform, sublicense, and distribute the - Work and such Derivative Works in Source or Object form. - - 3. Grant of Patent License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - (except as stated in this section) patent license to make, have made, - use, offer to sell, sell, import, and otherwise transfer the Work, - where such license applies only to those patent claims licensable - by such Contributor that are necessarily infringed by their - Contribution(s) alone or by combination of their Contribution(s) - with the Work to which such Contribution(s) was submitted. If You - institute patent litigation against any entity (including a - cross-claim or counterclaim in a lawsuit) alleging that the Work - or a Contribution incorporated within the Work constitutes direct - or contributory patent infringement, then any patent licenses - granted to You under this License for that Work shall terminate - as of the date such litigation is filed. - - 4. Redistribution. You may reproduce and distribute copies of the - Work or Derivative Works thereof in any medium, with or without - modifications, and in Source or Object form, provided that You - meet the following conditions: - - (a) You must give any other recipients of the Work or - Derivative Works a copy of this License; and - - (b) You must cause any modified files to carry prominent notices - stating that You changed the files; and - - (c) You must retain, in the Source form of any Derivative Works - that You distribute, all copyright, patent, trademark, and - attribution notices from the Source form of the Work, - excluding those notices that do not pertain to any part of - the Derivative Works; and - - (d) If the Work includes a "NOTICE" text file as part of its - distribution, then any Derivative Works that You distribute must - include a readable copy of the attribution notices contained - within such NOTICE file, excluding those notices that do not - pertain to any part of the Derivative Works, in at least one - of the following places: within a NOTICE text file distributed - as part of the Derivative Works; within the Source form or - documentation, if provided along with the Derivative Works; or, - within a display generated by the Derivative Works, if and - wherever such third-party notices normally appear. The contents - of the NOTICE file are for informational purposes only and - do not modify the License. You may add Your own attribution - notices within Derivative Works that You distribute, alongside - or as an addendum to the NOTICE text from the Work, provided - that such additional attribution notices cannot be construed - as modifying the License. - - You may add Your own copyright statement to Your modifications and - may provide additional or different license terms and conditions - for use, reproduction, or distribution of Your modifications, or - for any such Derivative Works as a whole, provided Your use, - reproduction, and distribution of the Work otherwise complies with - the conditions stated in this License. - - 5. Submission of Contributions. Unless You explicitly state otherwise, - any Contribution intentionally submitted for inclusion in the Work - by You to the Licensor shall be under the terms and conditions of - this License, without any additional terms or conditions. - Notwithstanding the above, nothing herein shall supersede or modify - the terms of any separate license agreement you may have executed - with Licensor regarding such Contributions. - - 6. Trademarks. This License does not grant permission to use the trade - names, trademarks, service marks, or product names of the Licensor, - except as required for reasonable and customary use in describing the - origin of the Work and reproducing the content of the NOTICE file. - - 7. Disclaimer of Warranty. Unless required by applicable law or - agreed to in writing, Licensor provides the Work (and each - Contributor provides its Contributions) on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied, including, without limitation, any warranties or conditions - of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A - PARTICULAR PURPOSE. You are solely responsible for determining the - appropriateness of using or redistributing the Work and assume any - risks associated with Your exercise of permissions under this License. - - 8. Limitation of Liability. In no event and under no legal theory, - whether in tort (including negligence), contract, or otherwise, - unless required by applicable law (such as deliberate and grossly - negligent acts) or agreed to in writing, shall any Contributor be - liable to You for damages, including any direct, indirect, special, - incidental, or consequential damages of any character arising as a - result of this License or out of the use or inability to use the - Work (including but not limited to damages for loss of goodwill, - work stoppage, computer failure or malfunction, or any and all - other commercial damages or losses), even if such Contributor - has been advised of the possibility of such damages. - - 9. Accepting Warranty or Additional Liability. While redistributing - the Work or Derivative Works thereof, You may choose to offer, - and charge a fee for, acceptance of support, warranty, indemnity, - or other liability obligations and/or rights consistent with this - License. However, in accepting such obligations, You may act only - on Your own behalf and on Your sole responsibility, not on behalf - of any other Contributor, and only if You agree to indemnify, - defend, and hold each Contributor harmless for any liability - incurred by, or claims asserted against, such Contributor by reason - of your accepting any such warranty or additional liability. - - END OF TERMS AND CONDITIONS diff --git a/packages/onnx/NOTICE b/packages/onnx/NOTICE deleted file mode 100644 index e4ed18d5bc..0000000000 --- a/packages/onnx/NOTICE +++ /dev/null @@ -1,17 +0,0 @@ -@qvac/onnx -Copyright 2026 Tether Data, S.A. de C.V. - -This product includes third-party components under their -respective licenses. @qvac/onnx itself is licensed under -Apache-2.0; bundled dependencies are governed by the licenses -listed below. - -========================================================================= -C++ Dependencies -========================================================================= - ---- mit (MIT License) --- - - onnxruntime - https://github.com/microsoft/onnxruntime - diff --git a/packages/onnx/README.md b/packages/onnx/README.md deleted file mode 100644 index 585ec31e2c..0000000000 --- a/packages/onnx/README.md +++ /dev/null @@ -1,283 +0,0 @@ -# @qvac/onnx - -Bare addon providing ONNX Runtime session management for QVAC inference addons. Links ONNX Runtime (via vcpkg) and exposes both a C++ header-only library and a JavaScript API. On desktop platforms, `@qvac/onnx.bare` is the single shared library containing ORT — consumer addons dynamically link against it so ORT is loaded exactly once per process. On mobile (Android/iOS), consumer addons can statically link ORT via the bundled static libraries, or dynamically link (controlled by the `MOBILE_DYNAMIC_LINK` CMake option). Has no dependency on `inference-addon-cpp`. - -## Architecture - -``` -┌─────────────────────────────────────────────────────┐ -│ JS API (binding.js → addon/binding.cpp) │ -│ configureEnvironment · getAvailableProviders │ -│ createSession · run · getInputInfo · getOutputInfo │ -│ destroySession │ -└──────────────────────┬──────────────────────────────┘ - │ -┌──────────────────────▼──────────────────────────────┐ -│ C++ Headers (header-only, namespace onnx_addon) │ -│ │ -│ OnnxSession.hpp Concrete session │ -│ IOnnxSession.hpp Abstract interface (no ORT)│ -│ OnnxRuntime.hpp Process-wide Ort::Env │ -│ OnnxSessionOptionsBuilder.hpp EP / thread config │ -│ OnnxConfig.hpp SessionConfig, enums │ -│ OnnxTensor.hpp TensorInfo, I/O tensors │ -│ OnnxTypeConversions.hpp ORT ↔ addon type mapping │ -│ Logger.hpp Logging (stdout or JS) │ -│ AndroidLog.hpp Android logcat logging │ -└──────────────────────┬──────────────────────────────┘ - │ -┌──────────────────────▼──────────────────────────────┐ -│ ONNX Runtime (via vcpkg) │ -│ Desktop: exported from qvac__onnx.bare (shared) │ -│ Mobile: dynamic (default) or static linking │ -│ + XNNPack EP · CoreML · NNAPI · DirectML │ -└─────────────────────────────────────────────────────┘ -``` - -**Key design points:** - -- **Single ORT load** — On desktop, `@qvac/onnx.bare` exports `OrtGetApiBase` and EP registration symbols. Consumer addons dynamically link against it (`DT_NEEDED: qvac__onnx@0.bare`), so ORT is loaded exactly once per process via ELF SONAME deduplication. -- **Single Ort::Env** — `OnnxRuntime` is a Meyers singleton. All sessions across all consumer addons in the same process share one environment. -- **Header-only C++** — Consumer addons include `` and link `qvac-onnx::headers` (compile-time headers only). ORT symbols are resolved at runtime from the shared `.bare`. -- **Abstract interface** — `IOnnxSession` lets consumers decouple pipeline code from ONNX Runtime headers. -- **Mobile linking** — Controlled by `MOBILE_DYNAMIC_LINK` CMake option (default `ON`). When `ON`, mobile builds use the same dynamic linking as desktop. When `OFF`, consumer addons statically link via `qvac-onnx::qvac-onnx-static` (which transitively provides `onnxruntime::onnxruntime_static`). -- **Automatic fallback chain** — Session construction retries on failure: requested config → without XNNPACK → CPU-only. This ensures models load even when an EP is unavailable. - -## JS API - -### `configureEnvironment(config?)` - -Configures the process-wide ONNX Runtime environment. Must be called **before** the first `createSession()`. Calling after initialization throws. - -| Field | Type | Default | Description | -|-------|------|---------|-------------| -| `loggingLevel` | `string` | `"error"` | `"verbose"`, `"info"`, `"warning"`, `"error"`, `"fatal"` | -| `loggingId` | `string` | `"qvac-onnx"` | Identifier used in ORT log messages | - -### `getAvailableProviders() → string[]` - -Returns the list of execution providers compiled into this build (for example `["CPUExecutionProvider"]` or `["CPUExecutionProvider", "XnnpackExecutionProvider"]`, depending on how the addon was built). - -### `createSession(modelPath, config?) → handle` - -Creates an ONNX Runtime inference session for the given model file. - -| Field | Type | Default | Description | -|-------|------|---------|-------------| -| `provider` | `string` | `"auto_gpu"` | `"cpu"`, `"auto_gpu"`, `"nnapi"`, `"coreml"`, `"directml"` | -| `optimization` | `string` | `"extended"` | `"disable"`, `"basic"`, `"extended"`, `"all"` | -| `intraOpThreads` | `number` | `0` | Intra-op parallelism thread count (0 = auto) | -| `interOpThreads` | `number` | `0` | Inter-op parallelism thread count (0 = auto) | -| `enableXnnpack` | `boolean` | `false` | Enable XNNPack execution provider for CPU | -| `enableMemoryPattern` | `boolean` | `true` | Enable memory pattern optimization | -| `enableCpuMemArena` | `boolean` | `true` | Enable CPU memory arena | -| `executionMode` | `string` | `"sequential"` | `"sequential"` or `"parallel"` | - -### `getInputInfo(handle) → Array<{name, shape, type}>` - -Returns input tensor metadata for the session. - -### `getOutputInfo(handle) → Array<{name, shape, type}>` - -Returns output tensor metadata for the session. - -### `run(handle, inputs) → Array<{name, shape, type, data}>` - -Runs inference. Each input: `{name: string, shape: number[], type: string, data: TypedArray}`. - -Supported tensor types and corresponding TypedArrays: -- `float32` → `Float32Array` -- `float16` → Special handling (no native JS TypedArray) -- `int64` → `BigInt64Array` -- `int32` → `Int32Array` -- `int8` → `Int8Array` -- `uint8` → `Uint8Array` - -Returns array of outputs: `{name: string, shape: number[], type: string, data: TypedArray}` where `data` contains the inference results as the appropriate TypedArray type. - -### `destroySession(handle)` - -Destroys the session and frees resources. - -## C++ API - -All headers are under the `qvac-onnx/` include prefix (source in `src/qvac-onnx/`). - -### Headers - -| Header | Description | -|--------|-------------| -| `OnnxSession.hpp` | Concrete session — load model, run inference, inspect I/O, zero-copy `runRaw()` | -| `IOnnxSession.hpp` | Abstract interface (no ORT dependency) for virtual dispatch | -| `OnnxRuntime.hpp` | Singleton `Ort::Env`; `configure()` and `getAvailableProviders()` | -| `OnnxConfig.hpp` | `SessionConfig`, `EnvironmentConfig`, enums, `providerToString()`, `optimizationToString()` | -| `OnnxTensor.hpp` | `TensorInfo`, `InputTensor`, `OutputTensor`, `TensorType`, `tensorTypeSize()` | -| `OnnxSessionOptionsBuilder.hpp` | Builds `Ort::SessionOptions` from `SessionConfig` | -| `OnnxTypeConversions.hpp` | Maps ORT element types to `TensorType` | -| `Logger.hpp` | Logging via stdout or JS (controlled by `JS_LOGGER` define) | -| `AndroidLog.hpp` | Android logcat logging (controlled by `QVAC_ONNX_ENABLE_ANDROID_LOG` define) | - -### Configuration types - -```cpp -// Environment (process-wide, one-time) -onnx_addon::EnvironmentConfig envCfg; -envCfg.loggingLevel = onnx_addon::LoggingLevel::ERROR; // VERBOSE, INFO, WARNING, ERROR, FATAL -envCfg.loggingId = "my-addon"; -onnx_addon::OnnxRuntime::configure(envCfg); // before first session - -// Session (per-model) -onnx_addon::SessionConfig config; -config.provider = onnx_addon::ExecutionProvider::CPU; // CPU, AUTO_GPU, NNAPI, CoreML, DirectML -config.optimization = onnx_addon::GraphOptimizationLevel::ALL; // DISABLE, BASIC, EXTENDED, ALL -config.intraOpThreads = 4; -config.interOpThreads = 2; -config.enableMemoryPattern = true; -config.enableCpuMemArena = true; -config.enableXnnpack = false; // Default: false (must be explicitly enabled) -config.executionMode = onnx_addon::ExecutionMode::SEQUENTIAL; // SEQUENTIAL, PARALLEL -``` - -### OnnxSession - -Concrete session class inheriting from `IOnnxSession`. Header-only, requires ONNX Runtime linked by the consuming target. Non-copyable, movable. - -```cpp -#include - -// Construction — loads model with automatic fallback chain: -// 1. Try with requested config (may include GPU EP + XNNPACK) -// 2. If XNNPACK enabled and init fails, retry without XNNPACK -// 3. If a non-CPU provider was requested and init fails, retry CPU-only -onnx_addon::OnnxSession session("model.onnx", config); - -// Introspection -auto inputs = session.getInputInfo(); // std::vector -auto outputs = session.getOutputInfo(); // std::vector -const std::string& name = session.inputName(0); // cached, no ORT API call -const std::string& out = session.outputName(0); // cached, no ORT API call -bool valid = session.isValid(); // true if session loaded successfully -const std::string& path = session.modelPath(); - -// Run inference — returns deep-copied OutputTensors -auto results = session.run(input); // single input, all outputs -auto results = session.run(inputs); // multiple inputs, all outputs -auto results = session.run(inputs, outputNames); // multiple inputs, specific outputs - -// Run inference — returns raw Ort::Values (zero-copy, OnnxSession-only, not in IOnnxSession) -auto ortValues = session.runRaw(input); // single input, all outputs -auto ortValues = session.runRaw(inputs); // multiple inputs, all outputs -auto ortValues = session.runRaw(inputs, outputNames); // multiple inputs, specific outputs -``` - -The `runRaw()` methods return `std::vector` directly from ORT, avoiding the deep copy that `run()` performs. Use `runRaw()` in performance-critical pipelines where you can work with ORT values directly. - -### IOnnxSession (abstract interface) - -For consumers that want to decouple pipeline code from ONNX Runtime headers: - -```cpp -#include // No ORT dependency - -class IOnnxSession { - virtual std::vector getInputInfo() const = 0; - virtual std::vector getOutputInfo() const = 0; - virtual const std::string& inputName(size_t index) const = 0; - virtual const std::string& outputName(size_t index) const = 0; - virtual std::vector run(const InputTensor& input) = 0; - virtual std::vector run(const std::vector& inputs) = 0; - virtual std::vector run(const std::vector& inputs, - const std::vector& outputNames) = 0; - virtual bool isValid() const = 0; - virtual const std::string& modelPath() const = 0; -}; -``` - -### Tensor types - -```cpp -// TensorType enum: FLOAT32, FLOAT16, INT64, INT32, INT8, UINT8 - -struct TensorInfo { std::string name; std::vector shape; TensorType type; }; - -struct InputTensor { - std::string name; - std::vector shape; - TensorType type = TensorType::FLOAT32; - const void* data = nullptr; // Caller owns memory - size_t dataSize = 0; // Size in bytes -}; - -struct OutputTensor { - std::string name; - std::vector shape; - TensorType type; - std::vector data; // Addon owns copy - size_t elementCount() const; - template const T* as() const; // Const typed access - template T* asMutable(); // Mutable typed access -}; - -size_t tensorTypeSize(TensorType type); // FLOAT32=4, FLOAT16=2, INT64=8, INT32=4, INT8=1, UINT8=1 -``` - -### Quick example - -```cpp -#include - -onnx_addon::SessionConfig config; -config.provider = onnx_addon::ExecutionProvider::CPU; - -onnx_addon::OnnxSession session("model.onnx", config); - -auto inputs = session.getInputInfo(); -auto outputs = session.getOutputInfo(); - -onnx_addon::InputTensor input; -input.name = inputs[0].name; -input.shape = {1, 3, 224, 224}; -input.type = onnx_addon::TensorType::FLOAT32; -input.data = floatData.data(); // Replace with actual data pointer -input.dataSize = floatData.size() * sizeof(float); - -auto results = session.run(input); -const float* out = results[0].as(); -``` - -## Consumer Addon Integration - -ONNX-based consumer addons get `@qvac/onnx` via npm. This single dependency provides the C++ headers, ONNX Runtime headers, CMake targets, and — on mobile (when static linking) — static libraries. On desktop, ORT symbols are resolved at runtime from the shared `@qvac/onnx.bare` (installed as a companion library). Consumer addons do **not** need `onnxruntime` in their own `vcpkg.json`. - -See **[INTEGRATION.md](./INTEGRATION.md)** for a step-by-step guide covering `package.json`, `vcpkg.json`, `CMakeLists.txt`, symbol visibility, and platform-specific setup (the worked example references the retired `@qvac/ocr-onnx` addon, preserved in git history). - -## Building - -```bash -npm run build # bare-make generate && bare-make build && bare-make install -``` - -## Running C++ tests - -```bash -npm run test:cpp # build with -D BUILD_TESTING=ON, then ctest -``` - -Two test binaries are produced: -- `unit_tests` — pure C++ tests for tensor types, config, and the abstract interface (no ORT dependency) -- `ort_tests` — tests that link ORT: runtime singleton, type conversions, session options, session lifecycle, and the shared-runtime addon scenario - -## Platform Support - -| Platform | Execution Providers | Triplet | -|----------|-------------------|---------| -| Linux | XNNPack, CPU | `x64-linux` | -| macOS | CoreML, XNNPack, CPU | `arm64-osx` | -| Windows | DirectML, XNNPack, CPU | (default MSVC) | -| Android | NNAPI, XNNPack, CPU | `arm64-android` | -| iOS | CoreML, XNNPack, CPU | `arm64-ios` | -| iOS Sim | CoreML, XNNPack, CPU | `arm64-ios-simulator`, `x64-ios-simulator` | - -## License - -Apache-2.0 diff --git a/packages/onnx/addon/binding.cpp b/packages/onnx/addon/binding.cpp deleted file mode 100644 index 67f06e1a34..0000000000 --- a/packages/onnx/addon/binding.cpp +++ /dev/null @@ -1,520 +0,0 @@ -#include - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include -#include - -// NOLINTBEGIN(cppcoreguidelines-macro-usage) -#define JSCHECK(call) \ - if ((call) != 0) throw std::runtime_error("js API call failed") - -#define CATCH \ - catch (const std::exception& e) { \ - js_throw_error(env, "QVAC_ONNX_ERROR", e.what()); \ - return nullptr; \ - } \ - catch (...) { \ - js_throw_error(env, "QVAC_ONNX_ERROR", "Unknown error"); \ - return nullptr; \ - } -// NOLINTEND(cppcoreguidelines-macro-usage) - -namespace { - -// --------------------------------------------------------------------------- -// Session storage -// --------------------------------------------------------------------------- - -std::mutex sessionsMtx; -std::unordered_map> sessions; -uint64_t nextSessionId = 0; - -// --------------------------------------------------------------------------- -// JS helpers (thin wrappers over js.h) -// --------------------------------------------------------------------------- - -std::vector getArgs(js_env_t* env, js_callback_info_t* info) { - size_t argc = 0; - JSCHECK(js_get_callback_info(env, info, &argc, nullptr, nullptr, nullptr)); - std::vector args(argc); - JSCHECK(js_get_callback_info(env, info, &argc, args.data(), nullptr, - nullptr)); - return args; -} - -js_value_t* jsString(js_env_t* env, std::string_view str) { - js_value_t* result = nullptr; - JSCHECK(js_create_string_utf8(env, reinterpret_cast(str.data()), - str.size(), &result)); - return result; -} - -std::string fromJsString(js_env_t* env, js_value_t* value) { - size_t len = 0; - JSCHECK(js_get_value_string_utf8(env, value, nullptr, 0, &len)); - std::string result(len, '\0'); - JSCHECK(js_get_value_string_utf8( - env, value, reinterpret_cast(result.data()), len, nullptr)); - return result; -} - -js_value_t* jsNumber(js_env_t* env, int64_t val) { - js_value_t* result = nullptr; - JSCHECK(js_create_int64(env, val, &result)); - return result; -} - -int32_t fromJsInt32(js_env_t* env, js_value_t* value) { - int32_t result = 0; - JSCHECK(js_get_value_int32(env, value, &result)); - return result; -} - -int64_t fromJsInt64(js_env_t* env, js_value_t* value) { - int64_t result = 0; - JSCHECK(js_get_value_int64(env, value, &result)); - return result; -} - -js_value_t* jsObject(js_env_t* env) { - js_value_t* result = nullptr; - JSCHECK(js_create_object(env, &result)); - return result; -} - -void jsPropSet(js_env_t* env, js_value_t* obj, const char* key, - js_value_t* val) { - JSCHECK(js_set_named_property(env, obj, key, val)); -} - -js_value_t* jsPropGet(js_env_t* env, js_value_t* obj, const char* key) { - js_value_t* result = nullptr; - JSCHECK(js_get_named_property(env, obj, key, &result)); - return result; -} - -bool jsIsUndefinedOrNull(js_env_t* env, js_value_t* value) { - bool undef = false; - bool null = false; - JSCHECK(js_is_undefined(env, value, &undef)); - JSCHECK(js_is_null(env, value, &null)); - return undef || null; -} - -bool jsIsObject(js_env_t* env, js_value_t* value) { - bool result = false; - JSCHECK(js_is_object(env, value, &result)); - return result; -} - -js_value_t* jsArray(js_env_t* env, size_t len) { - js_value_t* result = nullptr; - JSCHECK(js_create_array_with_length(env, len, &result)); - return result; -} - -uint32_t jsArrayLen(js_env_t* env, js_value_t* arr) { - uint32_t result = 0; - JSCHECK(js_get_array_length(env, arr, &result)); - return result; -} - -js_value_t* jsArrayGet(js_env_t* env, js_value_t* arr, uint32_t index) { - js_value_t* result = nullptr; - JSCHECK(js_get_element(env, arr, index, &result)); - return result; -} - -void jsArraySet(js_env_t* env, js_value_t* arr, uint32_t index, - js_value_t* val) { - JSCHECK(js_set_element(env, arr, index, val)); -} - -/// Get optional string property from a JS object. Returns empty optional -/// if the property is undefined/null. -std::optional jsOptString(js_env_t* env, js_value_t* obj, - const char* key) { - auto* val = jsPropGet(env, obj, key); - if (jsIsUndefinedOrNull(env, val)) return std::nullopt; - return fromJsString(env, val); -} - -/// Get optional int32 property from a JS object. -std::optional jsOptInt32(js_env_t* env, js_value_t* obj, - const char* key) { - auto* val = jsPropGet(env, obj, key); - if (jsIsUndefinedOrNull(env, val)) return std::nullopt; - return fromJsInt32(env, val); -} - -/// Get optional bool property from a JS object. -std::optional jsOptBool(js_env_t* env, js_value_t* obj, - const char* key) { - auto* val = jsPropGet(env, obj, key); - if (jsIsUndefinedOrNull(env, val)) return std::nullopt; - bool result = false; - JSCHECK(js_get_value_bool(env, val, &result)); - return result; -} - -/// Create a JS TypedArray by copying data from a span. -template -js_value_t* jsTypedArray(js_env_t* env, const T* data, size_t count) { - constexpr js_typedarray_type_t arrayType = [] { - if constexpr (std::is_same_v) return js_float32array; - else if constexpr (std::is_same_v) return js_bigint64array; - else if constexpr (std::is_same_v) return js_int32array; - else if constexpr (std::is_same_v) return js_int8array; - else if constexpr (std::is_same_v) return js_uint8array; - else if constexpr (std::is_same_v) return js_float64array; - else { static_assert(sizeof(T) == 0, "Unsupported typed array type"); } - }(); - - size_t byteLen = count * sizeof(T); - js_value_t* arrayBuffer = nullptr; - void* bufferData = nullptr; - JSCHECK(js_create_arraybuffer(env, byteLen, &bufferData, &arrayBuffer)); - std::memcpy(bufferData, data, byteLen); - - js_value_t* typedArray = nullptr; - JSCHECK(js_create_typedarray(env, arrayType, count, arrayBuffer, 0, - &typedArray)); - return typedArray; -} - -// --------------------------------------------------------------------------- -// Domain helpers -// --------------------------------------------------------------------------- - -onnx_addon::OnnxSession& getSession(js_env_t* env, js_value_t* handle) { - auto id = static_cast(fromJsInt64(env, handle)); - std::scoped_lock lock{sessionsMtx}; - auto found = sessions.find(id); - if (found == sessions.end()) { - throw std::invalid_argument("Invalid session handle"); - } - return *found->second; -} - -std::string tensorTypeToString(onnx_addon::TensorType type) { - switch (type) { - case onnx_addon::TensorType::FLOAT32: return "float32"; - case onnx_addon::TensorType::FLOAT16: return "float16"; - case onnx_addon::TensorType::INT64: return "int64"; - case onnx_addon::TensorType::INT32: return "int32"; - case onnx_addon::TensorType::INT8: return "int8"; - case onnx_addon::TensorType::UINT8: return "uint8"; - default: return "unknown"; - } -} - -onnx_addon::TensorType stringToTensorType(std::string_view str) { - if (str == "float32") return onnx_addon::TensorType::FLOAT32; - if (str == "float16") return onnx_addon::TensorType::FLOAT16; - if (str == "int64") return onnx_addon::TensorType::INT64; - if (str == "int32") return onnx_addon::TensorType::INT32; - if (str == "int8") return onnx_addon::TensorType::INT8; - if (str == "uint8") return onnx_addon::TensorType::UINT8; - return onnx_addon::TensorType::FLOAT32; -} - -onnx_addon::ExecutionProvider stringToProvider(std::string_view str) { - if (str == "cpu") return onnx_addon::ExecutionProvider::CPU; - if (str == "auto_gpu") return onnx_addon::ExecutionProvider::AUTO_GPU; - if (str == "nnapi") return onnx_addon::ExecutionProvider::NNAPI; - if (str == "coreml") return onnx_addon::ExecutionProvider::CoreML; - if (str == "directml") return onnx_addon::ExecutionProvider::DirectML; - return onnx_addon::ExecutionProvider::AUTO_GPU; -} - -onnx_addon::GraphOptimizationLevel stringToOptimization( - std::string_view str) { - if (str == "disable") return onnx_addon::GraphOptimizationLevel::DISABLE; - if (str == "basic") return onnx_addon::GraphOptimizationLevel::BASIC; - if (str == "extended") return onnx_addon::GraphOptimizationLevel::EXTENDED; - if (str == "all") return onnx_addon::GraphOptimizationLevel::ALL; - return onnx_addon::GraphOptimizationLevel::EXTENDED; -} - -onnx_addon::LoggingLevel stringToLoggingLevel(std::string_view str) { - if (str == "verbose") return onnx_addon::LoggingLevel::VERBOSE; - if (str == "info") return onnx_addon::LoggingLevel::INFO; - if (str == "warning") return onnx_addon::LoggingLevel::WARNING; - if (str == "error") return onnx_addon::LoggingLevel::ERROR; - if (str == "fatal") return onnx_addon::LoggingLevel::FATAL; - return onnx_addon::LoggingLevel::WARNING; -} - -onnx_addon::ExecutionMode stringToExecutionMode(std::string_view str) { - if (str == "parallel") return onnx_addon::ExecutionMode::PARALLEL; - return onnx_addon::ExecutionMode::SEQUENTIAL; -} - -js_value_t* buildTensorInfoArray( - js_env_t* env, const std::vector& infos) { - auto* arr = jsArray(env, infos.size()); - for (uint32_t i = 0; i < infos.size(); ++i) { - auto* obj = jsObject(env); - jsPropSet(env, obj, "name", jsString(env, infos[i].name)); - jsPropSet(env, obj, "type", - jsString(env, tensorTypeToString(infos[i].type))); - - auto* shape = jsArray(env, infos[i].shape.size()); - for (uint32_t j = 0; j < infos[i].shape.size(); ++j) { - jsArraySet(env, shape, j, jsNumber(env, infos[i].shape[j])); - } - jsPropSet(env, obj, "shape", shape); - - jsArraySet(env, arr, i, obj); - } - return arr; -} - -js_value_t* buildOutputTypedArray(js_env_t* env, - const onnx_addon::OutputTensor& output) { - size_t count = output.elementCount(); - switch (output.type) { - case onnx_addon::TensorType::FLOAT32: - return jsTypedArray(env, output.as(), count); - case onnx_addon::TensorType::INT64: - return jsTypedArray(env, output.as(), count); - case onnx_addon::TensorType::INT32: - return jsTypedArray(env, output.as(), count); - case onnx_addon::TensorType::INT8: - return jsTypedArray(env, output.as(), count); - case onnx_addon::TensorType::UINT8: - return jsTypedArray(env, output.data.data(), count); - default: - return jsTypedArray(env, output.as(), count); - } -} - -// --------------------------------------------------------------------------- -// Exported functions -// --------------------------------------------------------------------------- - -auto configureEnvironment(js_env_t* env, js_callback_info_t* info) - -> js_value_t* try { - auto args = getArgs(env, info); - onnx_addon::EnvironmentConfig cfg{}; - - if (!args.empty() && jsIsObject(env, args[0]) && - !jsIsUndefinedOrNull(env, args[0])) { - auto* configObj = args[0]; - - auto level = jsOptString(env, configObj, "loggingLevel"); - if (level) cfg.loggingLevel = stringToLoggingLevel(*level); - - auto id = jsOptString(env, configObj, "loggingId"); - if (id) cfg.loggingId = *id; - } - - onnx_addon::OnnxRuntime::configure(cfg); - return nullptr; -} -CATCH - -auto getAvailableProviders(js_env_t* env, js_callback_info_t* info) - -> js_value_t* try { - (void)info; - auto providers = onnx_addon::OnnxRuntime::getAvailableProviders(); - auto* arr = jsArray(env, providers.size()); - for (uint32_t i = 0; i < providers.size(); ++i) { - jsArraySet(env, arr, i, jsString(env, providers[i])); - } - return arr; -} -CATCH - -auto createSession(js_env_t* env, js_callback_info_t* info) - -> js_value_t* try { - auto args = getArgs(env, info); - if (args.empty()) { - throw std::invalid_argument("Expected at least 1 argument: modelPath"); - } - - auto modelPath = fromJsString(env, args[0]); - onnx_addon::SessionConfig config{}; - - // Parse optional config object - if (args.size() > 1 && jsIsObject(env, args[1]) && - !jsIsUndefinedOrNull(env, args[1])) { - auto* configObj = args[1]; - - auto provider = jsOptString(env, configObj, "provider"); - if (provider) config.provider = stringToProvider(*provider); - - auto optimization = jsOptString(env, configObj, "optimization"); - if (optimization) config.optimization = stringToOptimization(*optimization); - - auto intraOp = jsOptInt32(env, configObj, "intraOpThreads"); - if (intraOp) config.intraOpThreads = *intraOp; - - auto interOp = jsOptInt32(env, configObj, "interOpThreads"); - if (interOp) config.interOpThreads = *interOp; - - auto xnnpack = jsOptBool(env, configObj, "enableXnnpack"); - if (xnnpack) config.enableXnnpack = *xnnpack; - - auto memPattern = jsOptBool(env, configObj, "enableMemoryPattern"); - if (memPattern) config.enableMemoryPattern = *memPattern; - - auto cpuMemArena = jsOptBool(env, configObj, "enableCpuMemArena"); - if (cpuMemArena) config.enableCpuMemArena = *cpuMemArena; - - auto execMode = jsOptString(env, configObj, "executionMode"); - if (execMode) config.executionMode = stringToExecutionMode(*execMode); - } - - auto session = - std::make_unique(modelPath, config); - - std::scoped_lock lock{sessionsMtx}; - auto id = nextSessionId++; - sessions.emplace(id, std::move(session)); - return jsNumber(env, static_cast(id)); -} -CATCH - -auto getInputInfo(js_env_t* env, js_callback_info_t* info) - -> js_value_t* try { - auto args = getArgs(env, info); - if (args.empty()) { - throw std::invalid_argument("Expected 1 argument: session handle"); - } - auto& session = getSession(env, args[0]); - return buildTensorInfoArray(env, session.getInputInfo()); -} -CATCH - -auto getOutputInfo(js_env_t* env, js_callback_info_t* info) - -> js_value_t* try { - auto args = getArgs(env, info); - if (args.empty()) { - throw std::invalid_argument("Expected 1 argument: session handle"); - } - auto& session = getSession(env, args[0]); - return buildTensorInfoArray(env, session.getOutputInfo()); -} -CATCH - -auto run(js_env_t* env, js_callback_info_t* info) -> js_value_t* try { - auto args = getArgs(env, info); - if (args.size() < 2) { - throw std::invalid_argument( - "Expected 2 arguments: session handle, inputs array"); - } - - auto& session = getSession(env, args[0]); - auto* inputsArr = args[1]; - uint32_t inputCount = jsArrayLen(env, inputsArr); - - std::vector inputs; - inputs.reserve(inputCount); - - for (uint32_t i = 0; i < inputCount; ++i) { - auto* inputObj = jsArrayGet(env, inputsArr, i); - - onnx_addon::InputTensor input; - input.name = fromJsString(env, jsPropGet(env, inputObj, "name")); - - auto typeStr = fromJsString(env, jsPropGet(env, inputObj, "type")); - input.type = stringToTensorType(typeStr); - - auto* shapeArr = jsPropGet(env, inputObj, "shape"); - uint32_t shapeLen = jsArrayLen(env, shapeArr); - input.shape.reserve(shapeLen); - for (uint32_t j = 0; j < shapeLen; ++j) { - input.shape.push_back(fromJsInt64(env, jsArrayGet(env, shapeArr, j))); - } - - // Extract raw data pointer from TypedArray - auto* dataValue = jsPropGet(env, inputObj, "data"); - void* dataPtr = nullptr; - size_t dataLen = 0; - JSCHECK(js_get_typedarray_info(env, dataValue, nullptr, &dataPtr, &dataLen, - nullptr, nullptr)); - input.data = dataPtr; - input.dataSize = dataLen * onnx_addon::tensorTypeSize(input.type); - - inputs.push_back(std::move(input)); - } - - auto outputs = session.run(inputs); - - auto* outputArr = jsArray(env, outputs.size()); - for (uint32_t i = 0; i < outputs.size(); ++i) { - auto* obj = jsObject(env); - jsPropSet(env, obj, "name", jsString(env, outputs[i].name)); - jsPropSet(env, obj, "type", - jsString(env, tensorTypeToString(outputs[i].type))); - - auto* shape = jsArray(env, outputs[i].shape.size()); - for (uint32_t j = 0; j < outputs[i].shape.size(); ++j) { - jsArraySet(env, shape, j, jsNumber(env, outputs[i].shape[j])); - } - jsPropSet(env, obj, "shape", shape); - jsPropSet(env, obj, "data", buildOutputTypedArray(env, outputs[i])); - - jsArraySet(env, outputArr, i, obj); - } - - return outputArr; -} -CATCH - -auto destroySession(js_env_t* env, js_callback_info_t* info) - -> js_value_t* try { - auto args = getArgs(env, info); - if (args.empty()) { - throw std::invalid_argument("Expected 1 argument: session handle"); - } - - auto id = static_cast(fromJsInt64(env, args[0])); - std::scoped_lock lock{sessionsMtx}; - if (sessions.erase(id) == 0) { - throw std::invalid_argument("Invalid session handle"); - } - return nullptr; -} -CATCH - -} // namespace - -js_value_t* qvacOnnxExports(js_env_t* env, js_value_t* exports) { -// NOLINTBEGIN(cppcoreguidelines-macro-usage) -#define V(name, fn) \ - { \ - js_value_t* val; \ - if (js_create_function(env, name, -1, fn, nullptr, &val) != 0) { \ - return nullptr; \ - } \ - if (js_set_named_property(env, exports, name, val) != 0) { \ - return nullptr; \ - } \ - } - - V("configureEnvironment", configureEnvironment) - V("getAvailableProviders", getAvailableProviders) - V("createSession", createSession) - V("getInputInfo", getInputInfo) - V("getOutputInfo", getOutputInfo) - V("run", run) - V("destroySession", destroySession) -#undef V -// NOLINTEND(cppcoreguidelines-macro-usage) - - return exports; -} - -BARE_MODULE(qvac_onnx, qvacOnnxExports) diff --git a/packages/onnx/binding.js b/packages/onnx/binding.js deleted file mode 100644 index cea46308c0..0000000000 --- a/packages/onnx/binding.js +++ /dev/null @@ -1 +0,0 @@ -module.exports = require.addon() diff --git a/packages/onnx/ci/remove-brew-llvm.sh b/packages/onnx/ci/remove-brew-llvm.sh deleted file mode 100755 index 51eced6717..0000000000 --- a/packages/onnx/ci/remove-brew-llvm.sh +++ /dev/null @@ -1,28 +0,0 @@ -#!/usr/bin/env bash -set -e - -echo "Before cleanup — current clang path and version:" -which clang++ -clang++ --version || true - -echo "Checking for any Homebrew LLVM installations..." -for pkg in $(brew list --formula | grep '^llvm@' || true); do - echo "Removing $pkg to prevent conflicts" - brew uninstall --ignore-dependencies "$pkg" || true -done - -echo "Cleaning up brew environment..." -brew cleanup -s || true -hash -r || true - -echo "After cleanup — verifying Apple clang:" -which clang++ -xcrun --find clang++ -clang++ --version || true - -if which clang++ | grep -q "/opt/homebrew/"; then - echo "Still using Homebrew clang++ — aborting build!" - exit 1 -fi - -echo "Homebrew LLVM removed successfully. Apple Clang is now active." diff --git a/packages/onnx/cmake/qvac-onnxConfig.cmake.in b/packages/onnx/cmake/qvac-onnxConfig.cmake.in deleted file mode 100644 index 00d6d772b1..0000000000 --- a/packages/onnx/cmake/qvac-onnxConfig.cmake.in +++ /dev/null @@ -1,19 +0,0 @@ -@PACKAGE_INIT@ - -include("${CMAKE_CURRENT_LIST_DIR}/qvac-onnxTargets.cmake") - -# Mobile builds ship static libs — provide qvac-onnx::qvac-onnx-static -if(EXISTS "${PACKAGE_PREFIX_DIR}/share/onnxruntime") - list(PREPEND CMAKE_PREFIX_PATH "${PACKAGE_PREFIX_DIR}") - - include(CMakeFindDependencyMacro) - find_dependency(onnxruntime CONFIG) - - if(NOT TARGET qvac-onnx::qvac-onnx-static) - add_library(qvac-onnx::qvac-onnx-static INTERFACE IMPORTED) - set_target_properties(qvac-onnx::qvac-onnx-static PROPERTIES - INTERFACE_LINK_LIBRARIES "qvac-onnx::headers;onnxruntime::onnxruntime") - endif() -endif() - -check_required_components(qvac-onnx) diff --git a/packages/onnx/exports.txt b/packages/onnx/exports.txt deleted file mode 100644 index d381a087d6..0000000000 --- a/packages/onnx/exports.txt +++ /dev/null @@ -1,4 +0,0 @@ -_bare_* -_napi_* -_OrtGetApiBase -_OrtSessionOptionsAppendExecutionProvider_* diff --git a/packages/onnx/package.json b/packages/onnx/package.json deleted file mode 100644 index 518921a0c4..0000000000 --- a/packages/onnx/package.json +++ /dev/null @@ -1,47 +0,0 @@ -{ - "name": "@qvac/onnx", - "version": "0.15.1", - "description": "Bare addon for ONNX Runtime session management", - "addon": true, - "engines": { - "bare": ">=1.19.3" - }, - "scripts": { - "build": "bare-make generate && bare-make build && bare-make install", - "build:pack": "mkdir -p dist && npm pack --pack-destination dist", - "mobile:copy-prebuilds": "cp -r prebuilds/android-arm64 prebuilds/android-ia32 || echo 'Warning: Failed to copy prebuilds to android-ia32'; cp -r prebuilds/android-arm64 prebuilds/android-arm || echo 'Warning: Failed to copy prebuilds to android-arm'; cp -r prebuilds/android-arm64 prebuilds/android-x64 || echo 'Warning: Failed to copy prebuilds to android-x64'; cp -r prebuilds/ios-arm64 prebuilds/ios-arm64-simulator 2>/dev/null || echo 'iOS prebuilds already present'; cp -r prebuilds/ios-arm64 prebuilds/ios-x64-simulator 2>/dev/null || echo 'iOS prebuilds already present'", - "lint": "standard", - "lint:fix": "standard --fix", - "lint-cpp": "clang-tidy -p build $(find addon -name '*.cpp')", - "test:cpp:build": "bare-make generate -D BUILD_TESTING=ON && bare-make build", - "test:cpp:run": "cd build && ctest --output-on-failure", - "test:cpp": "npm run test:cpp:build && npm run test:cpp:run" - }, - "files": [ - "binding.js", - "prebuilds/*/*.bare", - "prebuilds/*/*.bare.exports", - "prebuilds/include", - "prebuilds/share/qvac-onnx", - "LICENSE", - "NOTICE" - ], - "exports": { - ".": "./binding.js", - "./package": "./package.json" - }, - "author": "Tether", - "license": "Apache-2.0", - "repository": { - "type": "git", - "url": "git+https://github.com/tetherto/qvac.git", - "directory": "packages/onnx" - }, - "bugs": "https://github.com/tetherto/qvac/issues", - "homepage": "https://qvac.tether.io", - "devDependencies": { - "cmake-bare": "^1.7.5", - "cmake-vcpkg": "^1.1.0", - "standard": "^17.0.0" - } -} diff --git a/packages/onnx/release-notes/v0.14.0.md b/packages/onnx/release-notes/v0.14.0.md deleted file mode 100644 index 54869af9bb..0000000000 --- a/packages/onnx/release-notes/v0.14.0.md +++ /dev/null @@ -1,13 +0,0 @@ -# QVAC ONNX Addon v0.14.0 Release Notes - -This release upgrades the shared ONNX Runtime layer used by QVAC addons from 1.22.0 to 1.24.2. It keeps the existing addon API intact while moving the package onto the newer custom vcpkg baseline that will be published to npm with this release. - -## Changes - -### Upgrade shared ONNX Runtime baseline to 1.24.2 - -The addon now resolves ONNX Runtime 1.24.2 from the QVAC vcpkg registry baseline instead of the older 1.22.0 toolchain. This keeps desktop and mobile builds aligned on the newer runtime and refreshes the platform-specific DirectML, CoreML, NNAPI, and XNNPack support shipped with the addon. - -## Backward Compatibility - -No JavaScript or C++ API changes are intended in this release. Existing consumers should only need to update to `@qvac/onnx@^0.14.0` and rebuild against the refreshed prebuild set. diff --git a/packages/onnx/src/qvac-onnx/AndroidLog.hpp b/packages/onnx/src/qvac-onnx/AndroidLog.hpp deleted file mode 100644 index a722745ef4..0000000000 --- a/packages/onnx/src/qvac-onnx/AndroidLog.hpp +++ /dev/null @@ -1,19 +0,0 @@ -#pragma once - -// Android logcat logging for ONNX session diagnostics. -// Disabled by default. Enable at build time with: -// -D QVAC_ONNX_ANDROID_LOG=ON (CMake) -// or -DQVAC_ONNX_ENABLE_ANDROID_LOG (compiler flag) - -#if defined(__ANDROID__) && defined(QVAC_ONNX_ENABLE_ANDROID_LOG) -#include -#define ONNX_ALOG_TAG "QVAC_ONNX" -// NOLINTBEGIN(cppcoreguidelines-macro-usage) -#define ONNX_ALOG(fmt, ...) \ - __android_log_print(ANDROID_LOG_INFO, ONNX_ALOG_TAG, fmt, ##__VA_ARGS__) -// NOLINTEND(cppcoreguidelines-macro-usage) -#else -// NOLINTBEGIN(cppcoreguidelines-macro-usage) -#define ONNX_ALOG(fmt, ...) ((void)0) -// NOLINTEND(cppcoreguidelines-macro-usage) -#endif diff --git a/packages/onnx/src/qvac-onnx/IOnnxSession.hpp b/packages/onnx/src/qvac-onnx/IOnnxSession.hpp deleted file mode 100644 index a05ea871f8..0000000000 --- a/packages/onnx/src/qvac-onnx/IOnnxSession.hpp +++ /dev/null @@ -1,44 +0,0 @@ -#pragma once - -#include -#include - -#include "OnnxTensor.hpp" - -namespace onnx_addon { - -/** - * Abstract interface for ONNX sessions. - * This interface has no ONNX Runtime dependency - consumers can use it - * for type-erased access to sessions via virtual dispatch. - */ -class IOnnxSession { - public: - virtual ~IOnnxSession() = default; - - // Model introspection - [[nodiscard]] virtual std::vector getInputInfo() const = 0; - [[nodiscard]] virtual std::vector getOutputInfo() const = 0; - - // Direct access to cached input/output names (avoids ORT API queries) - [[nodiscard]] virtual const std::string &inputName(size_t index) const = 0; - [[nodiscard]] virtual const std::string &outputName(size_t index) const = 0; - - // Run inference - single input, all outputs - virtual std::vector run(const InputTensor& input) = 0; - - // Run inference - multiple inputs, all outputs - virtual std::vector run(const std::vector& inputs) = 0; - - // Run inference - multiple inputs, specific outputs - virtual std::vector run(const std::vector& inputs, - const std::vector& outputNames) = 0; - - // Check if session is valid and ready - [[nodiscard]] virtual bool isValid() const = 0; - - // Get the model path - [[nodiscard]] virtual const std::string& modelPath() const = 0; -}; - -} // namespace onnx_addon diff --git a/packages/onnx/src/qvac-onnx/Logger.hpp b/packages/onnx/src/qvac-onnx/Logger.hpp deleted file mode 100644 index daa2ea96ef..0000000000 --- a/packages/onnx/src/qvac-onnx/Logger.hpp +++ /dev/null @@ -1,70 +0,0 @@ -#pragma once - -#include - -namespace onnx_addon::logger { - -enum class Priority : int { - ERROR = 0, - WARNING = 1, - INFO = 2, - DEBUG = 3, - OFF = 4 -}; - -static constexpr std::string_view to_string(Priority priority) noexcept { - switch (priority) { - case Priority::INFO: return "INFO"; - case Priority::DEBUG: return "DEBUG"; - case Priority::WARNING: return "WARNING"; - case Priority::ERROR: return "ERROR"; - case Priority::OFF: return "OFF"; - default: return "UNKNOWN"; - } -} - -} // namespace onnx_addon::logger - -// When JS_LOGGER is defined, consumer addons provide JS logging via -// inference-addon-cpp. This requires inference-addon-cpp -// in the include path and is an opt-in compile-time dependency for consumers. -#ifdef JS_LOGGER -#include -#ifndef QLOG -// NOLINTBEGIN(cppcoreguidelines-macro-usage) -#define QLOG(prio, msg) \ - qvac_lib_inference_addon_cpp::logger::JsLogger::log( \ - static_cast( \ - static_cast(prio)), \ - msg) -// NOLINTEND(cppcoreguidelines-macro-usage) -#endif -#else -// Standalone fallback: log to stdout -#ifndef QLOG -#include -// NOLINTBEGIN(cppcoreguidelines-macro-usage) -#define QLOG(prio, msg) \ - do { \ - std::cout << "[" \ - << onnx_addon::logger::to_string( \ - static_cast(prio)) \ - << "]: " << msg << std::endl; \ - } while (0) -// NOLINTEND(cppcoreguidelines-macro-usage) -#endif -#endif - -#ifndef NDEBUG -#ifndef QLOG_DEBUG -// NOLINTBEGIN(cppcoreguidelines-macro-usage) -#define QLOG_DEBUG(msg) QLOG(onnx_addon::logger::Priority::DEBUG, msg) -// NOLINTEND(cppcoreguidelines-macro-usage) -#endif -#else -#ifndef QLOG_DEBUG -// NOLINTBEGIN(cppcoreguidelines-macro-usage) -#define QLOG_DEBUG(msg) ((void)0) -// NOLINTEND(cppcoreguidelines-macro-usage) -#endif -#endif diff --git a/packages/onnx/src/qvac-onnx/OnnxConfig.hpp b/packages/onnx/src/qvac-onnx/OnnxConfig.hpp deleted file mode 100644 index 69036e2a63..0000000000 --- a/packages/onnx/src/qvac-onnx/OnnxConfig.hpp +++ /dev/null @@ -1,74 +0,0 @@ -#pragma once - -#include - -namespace onnx_addon { - -enum class ExecutionProvider { - CPU, - AUTO_GPU, // Auto-select based on platform (NNAPI/CoreML/DirectML) - NNAPI, // Android - CoreML, // Apple - DirectML // Windows -}; - -enum class GraphOptimizationLevel { - DISABLE, - BASIC, - EXTENDED, - ALL -}; - -enum class LoggingLevel { - VERBOSE, - INFO, - WARNING, - ERROR, - FATAL -}; - -enum class ExecutionMode { - SEQUENTIAL, - PARALLEL -}; - -struct EnvironmentConfig { - LoggingLevel loggingLevel = - LoggingLevel::ERROR; // Suppress ORT EP node-assignment warnings - std::string loggingId = "qvac-onnx"; -}; - -struct SessionConfig { - ExecutionProvider provider = ExecutionProvider::AUTO_GPU; - GraphOptimizationLevel optimization = GraphOptimizationLevel::EXTENDED; - int intraOpThreads = 0; // 0 = auto (use all available cores) - int interOpThreads = 0; // 0 = auto - bool enableMemoryPattern = true; - bool enableCpuMemArena = true; - bool enableXnnpack = - false; // XNNPack EP (opt-in: may cause node fallback warnings) - ExecutionMode executionMode = ExecutionMode::SEQUENTIAL; -}; - -inline std::string providerToString(ExecutionProvider provider) { - switch (provider) { - case ExecutionProvider::CPU: return "CPU"; - case ExecutionProvider::AUTO_GPU: return "AUTO_GPU"; - case ExecutionProvider::NNAPI: return "NNAPI"; - case ExecutionProvider::CoreML: return "CoreML"; - case ExecutionProvider::DirectML: return "DirectML"; - } - return "UNKNOWN"; -} - -inline std::string optimizationToString(GraphOptimizationLevel level) { - switch (level) { - case GraphOptimizationLevel::DISABLE: return "DISABLE"; - case GraphOptimizationLevel::BASIC: return "BASIC"; - case GraphOptimizationLevel::EXTENDED: return "EXTENDED"; - case GraphOptimizationLevel::ALL: return "ALL"; - } - return "UNKNOWN"; -} - -} // namespace onnx_addon diff --git a/packages/onnx/src/qvac-onnx/OnnxRuntime.hpp b/packages/onnx/src/qvac-onnx/OnnxRuntime.hpp deleted file mode 100644 index ad92918047..0000000000 --- a/packages/onnx/src/qvac-onnx/OnnxRuntime.hpp +++ /dev/null @@ -1,101 +0,0 @@ -#pragma once - -#include - -#include -#include -#include -#include -#include - -#include "Logger.hpp" -#include "OnnxConfig.hpp" - -namespace onnx_addon { - -/** - * Process-wide singleton for the ONNX Runtime environment. - * All OnnxSession instances share this single Ort::Env. - * - * ONNX Runtime recommends one Ort::Env per process. Creating multiple - * environments wastes memory and prevents shared thread pools. - * - * Thread-safe: Meyers singleton with guaranteed static init ordering. - * - * Call configure() before the first instance() call to customize logging - * level and identifier. If not called, defaults are used. - */ -class OnnxRuntime { - public: - OnnxRuntime(const OnnxRuntime&) = delete; - OnnxRuntime& operator=(const OnnxRuntime&) = delete; - OnnxRuntime(OnnxRuntime&&) = delete; - OnnxRuntime& operator=(OnnxRuntime&&) = delete; - - /// Configure the environment before first use. Throws if already initialized. - static void configure(const EnvironmentConfig& cfg) { - std::scoped_lock lock{configMtx_()}; - if (initialized_()) { - throw std::runtime_error( - "OnnxRuntime::configure() must be called before the first " - "instance() call"); - } - pendingConfig_() = cfg; - } - - static OnnxRuntime& instance() { - static OnnxRuntime inst{resolveConfig_()}; - return inst; - } - - Ort::Env& env() { return env_; } - - /// Returns the list of available execution providers from ONNX Runtime. - static std::vector getAvailableProviders() { - return Ort::GetAvailableProviders(); - } - - private: - explicit OnnxRuntime(const EnvironmentConfig& cfg) - : env_(toOrtLevel(cfg.loggingLevel), cfg.loggingId.c_str()) { - std::scoped_lock lock{configMtx_()}; - initialized_() = true; - QLOG(logger::Priority::INFO, "[OnnxRuntime] Singleton environment created"); - } - ~OnnxRuntime() = default; - - static OrtLoggingLevel toOrtLevel(LoggingLevel level) { - switch (level) { - case LoggingLevel::VERBOSE: return ORT_LOGGING_LEVEL_VERBOSE; - case LoggingLevel::INFO: return ORT_LOGGING_LEVEL_INFO; - case LoggingLevel::WARNING: return ORT_LOGGING_LEVEL_WARNING; - case LoggingLevel::ERROR: return ORT_LOGGING_LEVEL_ERROR; - case LoggingLevel::FATAL: return ORT_LOGGING_LEVEL_FATAL; - } - return ORT_LOGGING_LEVEL_WARNING; - } - - static EnvironmentConfig resolveConfig_() { - std::scoped_lock lock{configMtx_()}; - if (pendingConfig_()) return *pendingConfig_(); - return EnvironmentConfig{}; - } - - // Static storage via function-local statics to avoid SIOF - static std::mutex& configMtx_() { - static std::mutex mtx; - return mtx; - } - static std::optional& pendingConfig_() { - static std::optional cfg; - return cfg; - } - static bool& initialized_() { - static bool init = false; - return init; - } - - Ort::Env env_; -}; - -} // namespace onnx_addon diff --git a/packages/onnx/src/qvac-onnx/OnnxSession.hpp b/packages/onnx/src/qvac-onnx/OnnxSession.hpp deleted file mode 100644 index d23a194469..0000000000 --- a/packages/onnx/src/qvac-onnx/OnnxSession.hpp +++ /dev/null @@ -1,422 +0,0 @@ -#pragma once - -#include - -#include -#include -#include -#include -#include -#include - -#include "AndroidLog.hpp" -#include "IOnnxSession.hpp" -#include "OnnxConfig.hpp" -#include "OnnxRuntime.hpp" -#include "OnnxSessionOptionsBuilder.hpp" -#include "OnnxTensor.hpp" -#include "OnnxTypeConversions.hpp" - -namespace onnx_addon { - -/** - * Concrete ONNX session implementation (header-only). - * Inherits from IOnnxSession so that consumers can use virtual dispatch. - * Requires ONNX Runtime to be linked by the consuming target. - */ -class OnnxSession : public IOnnxSession { - public: - // Constructor - loads model from file path - inline explicit OnnxSession(const std::string& modelPath, - const SessionConfig& config = {}); - - ~OnnxSession() override = default; - - // Non-copyable - OnnxSession(const OnnxSession&) = delete; - OnnxSession& operator=(const OnnxSession&) = delete; - - // Movable - OnnxSession(OnnxSession&&) noexcept = default; - OnnxSession& operator=(OnnxSession&&) noexcept = default; - - // Model introspection - [[nodiscard]] inline std::vector getInputInfo() const override; - [[nodiscard]] inline std::vector getOutputInfo() const override; - - // Direct access to cached input/output names (avoids ORT API queries) - [[nodiscard]] inline const std::string & - inputName(size_t index) const override; - [[nodiscard]] inline const std::string & - outputName(size_t index) const override; - - // Run inference - single input, all outputs - inline std::vector run(const InputTensor& input) override; - - // Run inference - multiple inputs, all outputs - inline std::vector run( - const std::vector& inputs) override; - - // Run inference - multiple inputs, specific outputs - inline std::vector run( - const std::vector& inputs, - const std::vector& outputNames) override; - - // Run inference returning raw ORT values (zero-copy output). - // Only available on OnnxSession (not IOnnxSession) since it exposes ORT - // types. - inline std::vector runRaw(const InputTensor &input); - inline std::vector runRaw(const std::vector &inputs); - inline std::vector - runRaw(const std::vector &inputs, - const std::vector &outputNames); - - // Check if session is valid and ready - [[nodiscard]] inline bool isValid() const override; - - // Get the model path - [[nodiscard]] inline const std::string& modelPath() const override; - - private: - std::string modelPath_; - std::unique_ptr session_; - Ort::AllocatorWithDefaultOptions allocator_; - Ort::MemoryInfo memoryInfo_ = - Ort::MemoryInfo::CreateCpu(OrtArenaAllocator, OrtMemTypeDefault); - std::vector inputNames_; - std::vector outputNames_; - std::vector outputNamePtrs_; - Ort::RunOptions runOptions_; - - // Create an ORT tensor from a single InputTensor (avoids duplication) - inline Ort::Value createInputOrtValue(const InputTensor &input); - - // Platform-aware session construction (Windows needs wide strings) - static inline std::unique_ptr createOrtSession( - Ort::Env& env, const std::string& path, - const Ort::SessionOptions& options); -}; - -// --------------------------------------------------------------------------- -// Inline implementation -// --------------------------------------------------------------------------- - -inline std::unique_ptr OnnxSession::createOrtSession( - Ort::Env& env, const std::string& path, - const Ort::SessionOptions& options) { -#if defined(_WIN32) || defined(_WIN64) - std::wstring widePath(path.begin(), path.end()); - return std::make_unique(env, widePath.c_str(), options); -#else - return std::make_unique(env, path.c_str(), options); -#endif -} - -inline OnnxSession::OnnxSession(const std::string& modelPath, - const SessionConfig& config) - : modelPath_(modelPath) { - QLOG(logger::Priority::INFO, - std::string("[OnnxSession] Loading model: ") + modelPath); - ONNX_ALOG("[OnnxSession] Loading model: %s", modelPath.c_str()); - - auto& env = OnnxRuntime::instance().env(); - Ort::SessionOptions sessionOptions = buildSessionOptions(config); - - // Create the session with fallback chain: - // 1. Try with requested config (may include GPU EP + XNNPACK) - // 2. If XNNPACK enabled and init fails, retry without XNNPACK - // 3. If a non-CPU provider was requested and init fails, retry CPU-only - try { - session_ = createOrtSession(env, modelPath, sessionOptions); - } catch (const std::exception &e) { - bool retried = false; - - // Retry without XNNPACK (e.g. NHWC schema conflicts) - if (config.enableXnnpack) { - QLOG(logger::Priority::WARNING, - std::string("[OnnxSession] Session init failed: ") + e.what() + - ", retrying without XNNPACK"); - ONNX_ALOG( - "[OnnxSession] Session init failed: %s, retrying without XNNPACK", - e.what()); - try { - SessionConfig fallbackConfig = config; - fallbackConfig.enableXnnpack = false; - session_ = createOrtSession(env, modelPath, - buildSessionOptions(fallbackConfig)); - retried = true; - QLOG(logger::Priority::INFO, - "[OnnxSession] Session created without XNNPACK"); - ONNX_ALOG("[OnnxSession] Session created without XNNPACK"); - } catch (const std::exception &) { - // Fall through to CPU-only retry below - } - } - - // Retry with CPU-only (e.g. DirectML OOM on machines without a real GPU) - if (!retried && config.provider != ExecutionProvider::CPU) { - QLOG(logger::Priority::WARNING, - std::string("[OnnxSession] Session init failed: ") + e.what() + - ", retrying with CPU-only"); - ONNX_ALOG("[OnnxSession] Session init failed: %s, retrying with CPU-only", - e.what()); - try { - SessionConfig cpuConfig = config; - cpuConfig.provider = ExecutionProvider::CPU; - cpuConfig.enableXnnpack = false; - session_ = - createOrtSession(env, modelPath, buildSessionOptions(cpuConfig)); - retried = true; - QLOG(logger::Priority::INFO, - "[OnnxSession] Session created with CPU fallback"); - ONNX_ALOG("[OnnxSession] Session created with CPU fallback"); - } catch (const std::exception &) { - // All retries exhausted - } - } - - if (!retried) { - throw; - } - } - - // Cache input/output names - const size_t numInputs = session_->GetInputCount(); - inputNames_.reserve(numInputs); - for (size_t i = 0; i < numInputs; ++i) { - auto namePtr = session_->GetInputNameAllocated(i, allocator_); - inputNames_.emplace_back(namePtr.get()); - } - - const size_t numOutputs = session_->GetOutputCount(); - outputNames_.reserve(numOutputs); - outputNamePtrs_.reserve(numOutputs); - for (size_t i = 0; i < numOutputs; ++i) { - auto namePtr = session_->GetOutputNameAllocated(i, allocator_); - outputNames_.emplace_back(namePtr.get()); - } - for (const auto &name : outputNames_) { - outputNamePtrs_.push_back(name.c_str()); - } - - QLOG(logger::Priority::INFO, - std::string("[OnnxSession] Session ready, ") + - std::to_string(numInputs) + " input(s), " + - std::to_string(numOutputs) + " output(s)"); - ONNX_ALOG("[OnnxSession] Session ready, %zu input(s), %zu output(s)", - numInputs, numOutputs); -} - -inline std::vector OnnxSession::getInputInfo() const { - std::vector infos; - const size_t numInputs = session_->GetInputCount(); - infos.reserve(numInputs); - - for (size_t i = 0; i < numInputs; ++i) { - TensorInfo info; - info.name = inputNames_[i]; - - auto typeInfo = session_->GetInputTypeInfo(i); - auto tensorInfo = typeInfo.GetTensorTypeAndShapeInfo(); - - info.shape = tensorInfo.GetShape(); - info.type = fromOnnxType(tensorInfo.GetElementType()); - - infos.push_back(std::move(info)); - } - - return infos; -} - -inline std::vector OnnxSession::getOutputInfo() const { - std::vector infos; - const size_t numOutputs = session_->GetOutputCount(); - infos.reserve(numOutputs); - - for (size_t i = 0; i < numOutputs; ++i) { - TensorInfo info; - info.name = outputNames_[i]; - - auto typeInfo = session_->GetOutputTypeInfo(i); - auto tensorInfo = typeInfo.GetTensorTypeAndShapeInfo(); - - info.shape = tensorInfo.GetShape(); - info.type = fromOnnxType(tensorInfo.GetElementType()); - - infos.push_back(std::move(info)); - } - - return infos; -} - -inline const std::string &OnnxSession::inputName(size_t index) const { - return inputNames_[index]; -} - -inline const std::string &OnnxSession::outputName(size_t index) const { - return outputNames_[index]; -} - -inline std::vector OnnxSession::run(const InputTensor& input) { - return run(std::vector{input}); -} - -inline std::vector OnnxSession::run( - const std::vector& inputs) { - return run(inputs, outputNames_); -} - -inline std::vector OnnxSession::run( - const std::vector& inputs, - const std::vector& outputNames) { - auto ortOutputs = runRaw(inputs, outputNames); - - // Convert ORT outputs to OutputTensor (deep copy) - std::vector outputs; - outputs.reserve(ortOutputs.size()); - - for (size_t i = 0; i < ortOutputs.size(); ++i) { - OutputTensor output; - output.name = outputNames[i]; - - auto& ortOutput = ortOutputs[i]; - auto typeInfo = ortOutput.GetTypeInfo(); - auto tensorInfo = typeInfo.GetTensorTypeAndShapeInfo(); - - output.shape = tensorInfo.GetShape(); - output.type = fromOnnxType(tensorInfo.GetElementType()); - - // Calculate data size and copy - size_t elementCount = output.elementCount(); - size_t elementSize = tensorTypeSize(output.type); - size_t dataSize = elementCount * elementSize; - - output.data.resize(dataSize); - const void* srcData = ortOutput.GetTensorRawData(); - std::memcpy(output.data.data(), srcData, dataSize); - - outputs.push_back(std::move(output)); - } - - return outputs; -} - -inline Ort::Value OnnxSession::createInputOrtValue(const InputTensor &input) { - switch (input.type) { - case TensorType::FLOAT32: - return Ort::Value::CreateTensor( - memoryInfo_, - const_cast(static_cast(input.data)), - input.dataSize / sizeof(float), input.shape.data(), input.shape.size()); - case TensorType::INT64: - return Ort::Value::CreateTensor( - memoryInfo_, - const_cast(static_cast(input.data)), - input.dataSize / sizeof(int64_t), input.shape.data(), - input.shape.size()); - case TensorType::INT32: - return Ort::Value::CreateTensor( - memoryInfo_, - const_cast(static_cast(input.data)), - input.dataSize / sizeof(int32_t), input.shape.data(), - input.shape.size()); - case TensorType::UINT8: - return Ort::Value::CreateTensor( - memoryInfo_, - const_cast(static_cast(input.data)), - input.dataSize / sizeof(uint8_t), input.shape.data(), - input.shape.size()); - case TensorType::INT8: - return Ort::Value::CreateTensor( - memoryInfo_, - const_cast(static_cast(input.data)), - input.dataSize / sizeof(int8_t), input.shape.data(), - input.shape.size()); - default: - return Ort::Value::CreateTensor( - memoryInfo_, - const_cast(static_cast(input.data)), - input.dataSize / sizeof(float), input.shape.data(), input.shape.size()); - } -} - -inline std::vector OnnxSession::runRaw(const InputTensor &input) { - if (!isValid()) { - QLOG(logger::Priority::ERROR, - std::string( - "[OnnxSession] Run failed: session is not valid for model ") + - modelPath_); - throw std::runtime_error("OnnxSession is not valid"); - } - QLOG_DEBUG(std::string("[OnnxSession] Running inference on ") + modelPath_ + - " with 1 input(s)"); - - const char *inputNamePtr = input.name.c_str(); - Ort::Value inputTensor = createInputOrtValue(input); - - return session_->Run(runOptions_, &inputNamePtr, &inputTensor, 1, - outputNamePtrs_.data(), outputNamePtrs_.size()); -} - -inline std::vector -OnnxSession::runRaw(const std::vector &inputs) { - return runRaw(inputs, outputNames_); -} - -inline std::vector -OnnxSession::runRaw(const std::vector &inputs, - const std::vector &outputNames) { - if (!isValid()) { - QLOG(logger::Priority::ERROR, - std::string( - "[OnnxSession] Run failed: session is not valid for model ") + - modelPath_); - throw std::runtime_error("OnnxSession is not valid"); - } - QLOG_DEBUG(std::string("[OnnxSession] Running inference on ") + modelPath_ + - " with " + std::to_string(inputs.size()) + " input(s)"); - - // Prepare input tensors - std::vector inputTensors; - inputTensors.reserve(inputs.size()); - - std::vector inputNamePtrs; - inputNamePtrs.reserve(inputs.size()); - - for (const auto &input : inputs) { - inputNamePtrs.push_back(input.name.c_str()); - inputTensors.push_back(createInputOrtValue(input)); - } - - // Use cached output name pointers when requesting all outputs - const char *const *outPtrs; - size_t outCount; - std::vector customOutputNamePtrs; - - if (&outputNames == &outputNames_) { - // Common path: requesting all outputs — use cached pointers - outPtrs = outputNamePtrs_.data(); - outCount = outputNamePtrs_.size(); - } else { - customOutputNamePtrs.reserve(outputNames.size()); - for (const auto &name : outputNames) { - customOutputNamePtrs.push_back(name.c_str()); - } - outPtrs = customOutputNamePtrs.data(); - outCount = customOutputNamePtrs.size(); - } - - // Run inference - return session_->Run(runOptions_, inputNamePtrs.data(), inputTensors.data(), - inputTensors.size(), outPtrs, outCount); -} - -inline bool OnnxSession::isValid() const { - return session_ != nullptr; -} - -inline const std::string& OnnxSession::modelPath() const { - return modelPath_; -} - -} // namespace onnx_addon diff --git a/packages/onnx/src/qvac-onnx/OnnxSessionOptionsBuilder.hpp b/packages/onnx/src/qvac-onnx/OnnxSessionOptionsBuilder.hpp deleted file mode 100644 index 933109d09b..0000000000 --- a/packages/onnx/src/qvac-onnx/OnnxSessionOptionsBuilder.hpp +++ /dev/null @@ -1,191 +0,0 @@ -#pragma once - -#include - -#include -#include - -#include "AndroidLog.hpp" -#include "OnnxConfig.hpp" -#include "Logger.hpp" - -#ifdef __ANDROID__ -#include -#endif - -namespace onnx_addon { - -// Try to append XNNPack execution provider if available and enabled. -// Does NOT downgrade the caller's optimization level. If XNNPACK is -// incompatible with the chosen optimization (e.g. EXTENDED triggers -// NhwcTransformer conflicts), OnnxSession's constructor catches the -// ORT exception and retries without XNNPACK automatically. -inline void tryAppendXnnpack(Ort::SessionOptions& sessionOptions) { - try { - const auto providers = Ort::GetAvailableProviders(); - const bool available = - std::find(providers.begin(), providers.end(), - "XnnpackExecutionProvider") != providers.end(); - if (available) { - sessionOptions.AppendExecutionProvider("XNNPACK", {}); - QLOG(logger::Priority::INFO, "[OnnxSession] XNNPack EP appended"); - ONNX_ALOG("[OnnxSession] XNNPack EP appended"); - } else { - QLOG(logger::Priority::DEBUG, "[OnnxSession] XNNPack EP not available"); - ONNX_ALOG("[OnnxSession] XNNPack EP not available"); - } - } catch (const std::exception& e) { - QLOG(logger::Priority::WARNING, - std::string("[OnnxSession] Failed to append XNNPack: ") + e.what()); - ONNX_ALOG("[OnnxSession] Failed to append XNNPack: %s", e.what()); - } -} - -// Build session options based on config -inline Ort::SessionOptions buildSessionOptions(const SessionConfig& config) { - Ort::SessionOptions sessionOptions; - - QLOG(logger::Priority::DEBUG, - std::string("[OnnxSession] buildSessionOptions - provider=") + - providerToString(config.provider) + - ", optimization=" + optimizationToString(config.optimization) + - ", enableXnnpack=" + (config.enableXnnpack ? "true" : "false")); - ONNX_ALOG("[OnnxSession] buildSessionOptions - provider=%s, optimization=%s, xnnpack=%s", - providerToString(config.provider).c_str(), - optimizationToString(config.optimization).c_str(), - config.enableXnnpack ? "true" : "false"); - - // Set graph optimization level (using global ONNX Runtime enum values) - switch (config.optimization) { - case GraphOptimizationLevel::DISABLE: - sessionOptions.SetGraphOptimizationLevel( - ::GraphOptimizationLevel::ORT_DISABLE_ALL); - break; - case GraphOptimizationLevel::BASIC: - sessionOptions.SetGraphOptimizationLevel( - ::GraphOptimizationLevel::ORT_ENABLE_BASIC); - break; - case GraphOptimizationLevel::EXTENDED: - sessionOptions.SetGraphOptimizationLevel( - ::GraphOptimizationLevel::ORT_ENABLE_EXTENDED); - break; - case GraphOptimizationLevel::ALL: - sessionOptions.SetGraphOptimizationLevel( - ::GraphOptimizationLevel::ORT_ENABLE_ALL); - break; - } - - // Execution mode - sessionOptions.SetExecutionMode( - config.executionMode == ExecutionMode::PARALLEL - ? ::ExecutionMode::ORT_PARALLEL - : ::ExecutionMode::ORT_SEQUENTIAL); - - // Memory options - if (!config.enableMemoryPattern) { - sessionOptions.DisableMemPattern(); - } - if (!config.enableCpuMemArena) { - sessionOptions.DisableCpuMemArena(); - } - - // CPU-only mode - if (config.provider == ExecutionProvider::CPU) { - QLOG(logger::Priority::DEBUG, "[OnnxSession] CPU-only mode"); - if (config.enableXnnpack) { - tryAppendXnnpack(sessionOptions); - } - sessionOptions.SetIntraOpNumThreads(config.intraOpThreads); - sessionOptions.SetInterOpNumThreads(config.interOpThreads); - return sessionOptions; - } - - // Try to set up GPU provider - const auto providers = Ort::GetAvailableProviders(); - -#ifdef __ANDROID__ - if (config.provider == ExecutionProvider::AUTO_GPU || - config.provider == ExecutionProvider::NNAPI) { - try { - const bool nnapiAvailable = - std::find(providers.begin(), providers.end(), - "NnapiExecutionProvider") != providers.end(); - - if (nnapiAvailable) { - // NNAPI does not register com.ms.internal.nhwc schemas, same - // issue as XNNPACK, so we must drop to BASIC. - //TODO: confirm with testing - sessionOptions.SetGraphOptimizationLevel( - ::GraphOptimizationLevel::ORT_ENABLE_BASIC); - uint32_t nnapiFlags = NNAPI_FLAG_USE_FP16 | NNAPI_FLAG_CPU_DISABLED; - Ort::ThrowOnError(OrtSessionOptionsAppendExecutionProvider_Nnapi( - sessionOptions, nnapiFlags)); - QLOG(logger::Priority::INFO, "[OnnxSession] NNAPI EP appended (optimization set to BASIC)"); - ONNX_ALOG("[OnnxSession] NNAPI EP appended (optimization set to BASIC)"); - } else { - QLOG(logger::Priority::WARNING, "[OnnxSession] NNAPI EP not available, falling back to CPU"); - ONNX_ALOG("[OnnxSession] NNAPI EP not available, falling back to CPU"); - } - } catch (const std::exception& e) { - QLOG(logger::Priority::WARNING, - std::string("[OnnxSession] Failed to append NNAPI: ") + e.what()); - ONNX_ALOG("[OnnxSession] Failed to append NNAPI: %s", e.what()); - } - } - -#elif defined(__APPLE__) - if (config.provider == ExecutionProvider::AUTO_GPU || - config.provider == ExecutionProvider::CoreML) { - try { - const bool coremlAvailable = - std::find(providers.begin(), providers.end(), - "CoreMLExecutionProvider") != providers.end(); - - if (coremlAvailable) { - sessionOptions.AppendExecutionProvider("CoreML"); - QLOG(logger::Priority::INFO, "[OnnxSession] CoreML EP appended"); - } else { - QLOG(logger::Priority::WARNING, "[OnnxSession] CoreML EP not available, falling back to CPU"); - } - } catch (const std::exception& e) { - QLOG(logger::Priority::WARNING, - std::string("[OnnxSession] Failed to append CoreML: ") + e.what()); - } - } - -#elif defined(_WIN32) || defined(_WIN64) - if (config.provider == ExecutionProvider::AUTO_GPU || - config.provider == ExecutionProvider::DirectML) { - try { - const bool dmlAvailable = - std::find(providers.begin(), providers.end(), - "DmlExecutionProvider") != providers.end(); - - if (dmlAvailable) { - sessionOptions.SetExecutionMode(::ExecutionMode::ORT_SEQUENTIAL); - sessionOptions.DisableMemPattern(); - sessionOptions.AppendExecutionProvider("DML", {{"device_id", "0"}}); - QLOG(logger::Priority::INFO, "[OnnxSession] DirectML EP appended"); - } else { - QLOG(logger::Priority::WARNING, "[OnnxSession] DirectML EP not available, falling back to CPU"); - } - } catch (const std::exception& e) { - QLOG(logger::Priority::WARNING, - std::string("[OnnxSession] Failed to append DirectML: ") + e.what()); - } - } -#endif - - // XNNPack as CPU fallback accelerator alongside GPU providers - if (config.enableXnnpack) { - tryAppendXnnpack(sessionOptions); - } - - // Set threading options (applies to CPU fallback as well) - sessionOptions.SetIntraOpNumThreads(config.intraOpThreads); - sessionOptions.SetInterOpNumThreads(config.interOpThreads); - - return sessionOptions; -} - -} // namespace onnx_addon diff --git a/packages/onnx/src/qvac-onnx/OnnxTensor.hpp b/packages/onnx/src/qvac-onnx/OnnxTensor.hpp deleted file mode 100644 index bae449af19..0000000000 --- a/packages/onnx/src/qvac-onnx/OnnxTensor.hpp +++ /dev/null @@ -1,83 +0,0 @@ -#pragma once - -#include -#include -#include - -namespace onnx_addon { - -enum class TensorType { - FLOAT32, - FLOAT16, - INT64, - INT32, - INT8, - UINT8 -}; - -struct TensorInfo { - std::string name; - std::vector shape; - TensorType type; -}; - -struct InputTensor { - std::string name; - std::vector shape; - TensorType type = TensorType::FLOAT32; - const void* data = nullptr; // Raw pointer to data (caller owns memory) - size_t dataSize = 0; // Size in bytes -}; - -struct OutputTensor { - std::string name; - std::vector shape; - TensorType type; - std::vector data; // Owned copy of output data - - // Get element count from shape - [[nodiscard]] size_t elementCount() const { - if (shape.empty()) { - return 0; - } - size_t count = 1; - for (const auto dim : shape) { - count *= static_cast(dim); - } - return count; - } - - // Get data as typed pointer (const) - template - [[nodiscard]] const T* as() const { - return reinterpret_cast(data.data()); - } - - // Get data as typed pointer (mutable) - template - [[nodiscard]] T* asMutable() { - return reinterpret_cast(data.data()); - } -}; - -// Helper to get size of tensor element type in bytes -inline size_t tensorTypeSize(TensorType type) { - switch (type) { - case TensorType::FLOAT32: - return 4; - case TensorType::FLOAT16: - return 2; - case TensorType::INT64: - return 8; - case TensorType::INT32: - return 4; - case TensorType::INT8: - return 1; - case TensorType::UINT8: - return 1; - default: - return 0; - } -} - -} // namespace onnx_addon diff --git a/packages/onnx/src/qvac-onnx/OnnxTypeConversions.hpp b/packages/onnx/src/qvac-onnx/OnnxTypeConversions.hpp deleted file mode 100644 index a5641de61b..0000000000 --- a/packages/onnx/src/qvac-onnx/OnnxTypeConversions.hpp +++ /dev/null @@ -1,49 +0,0 @@ -#pragma once - -#include - -#include "OnnxTensor.hpp" - -namespace onnx_addon { - -// Convert our TensorType to ONNX element type -inline ONNXTensorElementDataType toOnnxType(TensorType type) { - switch (type) { - case TensorType::FLOAT32: - return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT; - case TensorType::FLOAT16: - return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT16; - case TensorType::INT64: - return ONNX_TENSOR_ELEMENT_DATA_TYPE_INT64; - case TensorType::INT32: - return ONNX_TENSOR_ELEMENT_DATA_TYPE_INT32; - case TensorType::INT8: - return ONNX_TENSOR_ELEMENT_DATA_TYPE_INT8; - case TensorType::UINT8: - return ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT8; - default: - return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT; - } -} - -// Convert ONNX element type to our TensorType -inline TensorType fromOnnxType(ONNXTensorElementDataType onnxType) { - switch (onnxType) { - case ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT: - return TensorType::FLOAT32; - case ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT16: - return TensorType::FLOAT16; - case ONNX_TENSOR_ELEMENT_DATA_TYPE_INT64: - return TensorType::INT64; - case ONNX_TENSOR_ELEMENT_DATA_TYPE_INT32: - return TensorType::INT32; - case ONNX_TENSOR_ELEMENT_DATA_TYPE_INT8: - return TensorType::INT8; - case ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT8: - return TensorType::UINT8; - default: - return TensorType::FLOAT32; - } -} - -} // namespace onnx_addon diff --git a/packages/onnx/symbols.map b/packages/onnx/symbols.map deleted file mode 100644 index bc76775b4a..0000000000 --- a/packages/onnx/symbols.map +++ /dev/null @@ -1,9 +0,0 @@ -{ - global: - bare_*; - napi_*; - OrtGetApiBase; - OrtSessionOptionsAppendExecutionProvider_*; - local: - *; -}; diff --git a/packages/onnx/tests/addon_shared_runtime_test.cpp b/packages/onnx/tests/addon_shared_runtime_test.cpp deleted file mode 100644 index 3796158f05..0000000000 --- a/packages/onnx/tests/addon_shared_runtime_test.cpp +++ /dev/null @@ -1,243 +0,0 @@ -#include - -#include -#include -#include - -#include "qvac-onnx/OnnxRuntime.hpp" -#include "qvac-onnx/OnnxSession.hpp" - -namespace oa = onnx_addon; -namespace logger = onnx_addon::logger; - -#ifndef TEST_FIXTURES_DIR -#error "TEST_FIXTURES_DIR must be defined at compile time" -#endif - -static std::string fixturePath(const std::string& name) { - return std::string(TEST_FIXTURES_DIR) + "/" + name; -} - -// --------------------------------------------------------------------------- -// Simulated consumer addon A: identity-based normalizer -// --------------------------------------------------------------------------- - -class AddonNormalizer { - public: - explicit AddonNormalizer(const std::string& modelPath) - : session_(modelPath, oa::SessionConfig{ - .provider = oa::ExecutionProvider::CPU, - .enableXnnpack = false}) { - QLOG(logger::Priority::INFO, "[AddonNormalizer] Initialized with model: " + modelPath); - } - - std::vector normalize(const std::vector& input) { - QLOG(logger::Priority::DEBUG, - "[AddonNormalizer] Running normalize with " + - std::to_string(input.size()) + " elements"); - oa::InputTensor tensor{.name = "X", - .shape = {1, static_cast(input.size())}, - .type = oa::TensorType::FLOAT32, - .data = input.data(), - .dataSize = input.size() * sizeof(float)}; - auto results = session_.run(tensor); - const float* out = results[0].as(); - QLOG(logger::Priority::DEBUG, "[AddonNormalizer] Normalize complete"); - return {out, out + results[0].elementCount()}; - } - - oa::OnnxRuntime& runtime() { return oa::OnnxRuntime::instance(); } - - bool isValid() const { return session_.isValid(); } - - private: - oa::OnnxSession session_; -}; - -// --------------------------------------------------------------------------- -// Simulated consumer addon B: arithmetic combiner using add model -// --------------------------------------------------------------------------- - -class AddonCombiner { - public: - explicit AddonCombiner(const std::string& modelPath) - : session_(modelPath, oa::SessionConfig{ - .provider = oa::ExecutionProvider::CPU, - .enableXnnpack = false}) { - QLOG(logger::Priority::INFO, "[AddonCombiner] Initialized with model: " + modelPath); - } - - std::vector combine(const std::vector& a, - const std::vector& b) { - QLOG(logger::Priority::DEBUG, - "[AddonCombiner] Running combine with " + - std::to_string(a.size()) + " elements per input"); - std::vector inputs = { - {.name = "A", - .shape = {1, static_cast(a.size())}, - .type = oa::TensorType::FLOAT32, - .data = a.data(), - .dataSize = a.size() * sizeof(float)}, - {.name = "B", - .shape = {1, static_cast(b.size())}, - .type = oa::TensorType::FLOAT32, - .data = b.data(), - .dataSize = b.size() * sizeof(float)}}; - auto results = session_.run(inputs); - const float* out = results[0].as(); - QLOG(logger::Priority::DEBUG, "[AddonCombiner] Combine complete"); - return {out, out + results[0].elementCount()}; - } - - oa::OnnxRuntime& runtime() { return oa::OnnxRuntime::instance(); } - - bool isValid() const { return session_.isValid(); } - - private: - oa::OnnxSession session_; -}; - -// --------------------------------------------------------------------------- -// Tests: both addons share a single OnnxRuntime instance -// --------------------------------------------------------------------------- - -class AddonSharedRuntimeTest : public ::testing::Test { - protected: - void SetUp() override { - normalizer_ = std::make_unique( - fixturePath("identity.onnx")); - combiner_ = std::make_unique( - fixturePath("add.onnx")); - } - - std::unique_ptr normalizer_; - std::unique_ptr combiner_; -}; - -TEST_F(AddonSharedRuntimeTest, BothAddonsAreValid) { - EXPECT_TRUE(normalizer_->isValid()); - EXPECT_TRUE(combiner_->isValid()); -} - -TEST_F(AddonSharedRuntimeTest, BothAddonsShareSameRuntimeInstance) { - EXPECT_EQ(&normalizer_->runtime(), &combiner_->runtime()); -} - -TEST_F(AddonSharedRuntimeTest, BothAddonsShareSameEnv) { - auto& env1 = normalizer_->runtime().env(); - auto& env2 = combiner_->runtime().env(); - EXPECT_EQ(&env1, &env2); -} - -TEST_F(AddonSharedRuntimeTest, NormalizerProducesCorrectResults) { - std::vector input = {1.0f, 2.0f, 3.0f, 4.0f}; - auto result = normalizer_->normalize(input); - ASSERT_EQ(result.size(), 4); - EXPECT_FLOAT_EQ(result[0], 1.0f); - EXPECT_FLOAT_EQ(result[1], 2.0f); - EXPECT_FLOAT_EQ(result[2], 3.0f); - EXPECT_FLOAT_EQ(result[3], 4.0f); -} - -TEST_F(AddonSharedRuntimeTest, CombinerProducesCorrectResults) { - std::vector a = {1.0f, 2.0f, 3.0f, 4.0f}; - std::vector b = {10.0f, 20.0f, 30.0f, 40.0f}; - auto result = combiner_->combine(a, b); - ASSERT_EQ(result.size(), 4); - EXPECT_FLOAT_EQ(result[0], 11.0f); - EXPECT_FLOAT_EQ(result[1], 22.0f); - EXPECT_FLOAT_EQ(result[2], 33.0f); - EXPECT_FLOAT_EQ(result[3], 44.0f); -} - -TEST_F(AddonSharedRuntimeTest, InterleavedInferenceSharesSingleRuntime) { - std::vector data = {5.0f, 6.0f, 7.0f, 8.0f}; - - // Alternate between addons to confirm shared runtime stays stable - auto r1 = normalizer_->normalize(data); - auto r2 = combiner_->combine(data, data); - auto r3 = normalizer_->normalize(data); - auto r4 = combiner_->combine(r1, r3); - - EXPECT_FLOAT_EQ(r1[0], 5.0f); - EXPECT_FLOAT_EQ(r2[0], 10.0f); - EXPECT_FLOAT_EQ(r3[0], 5.0f); - EXPECT_FLOAT_EQ(r4[0], 10.0f); - - // Runtime pointer is still the same after all interleaved operations - EXPECT_EQ(&normalizer_->runtime(), &combiner_->runtime()); -} - -TEST_F(AddonSharedRuntimeTest, ConcurrentInferenceSharesSingleRuntime) { - std::vector data = {1.0f, 2.0f, 3.0f, 4.0f}; - - auto normFuture = std::async(std::launch::async, [&]() { - std::vector results; - for (int i = 0; i < 10; ++i) { - results = normalizer_->normalize(data); - } - return results; - }); - - auto combFuture = std::async(std::launch::async, [&]() { - std::vector results; - for (int i = 0; i < 10; ++i) { - results = combiner_->combine(data, data); - } - return results; - }); - - auto normResult = normFuture.get(); - auto combResult = combFuture.get(); - - // Verify results are still correct after concurrent execution - ASSERT_EQ(normResult.size(), 4); - EXPECT_FLOAT_EQ(normResult[0], 1.0f); - EXPECT_FLOAT_EQ(normResult[3], 4.0f); - - ASSERT_EQ(combResult.size(), 4); - EXPECT_FLOAT_EQ(combResult[0], 2.0f); - EXPECT_FLOAT_EQ(combResult[3], 8.0f); - - // Singleton is still the same instance - EXPECT_EQ(&normalizer_->runtime(), &combiner_->runtime()); -} - -// --------------------------------------------------------------------------- -// Test: addons created at different times still share the same runtime -// --------------------------------------------------------------------------- - -TEST(AddonSharedRuntimeLifecycleTest, LateCreatedAddonSharesRuntime) { - auto normalizer = std::make_unique( - fixturePath("identity.onnx")); - - // Capture the runtime address before creating the second addon - const auto* runtimeBefore = &normalizer->runtime(); - - auto combiner = std::make_unique( - fixturePath("add.onnx")); - - EXPECT_EQ(runtimeBefore, &combiner->runtime()); -} - -TEST(AddonSharedRuntimeLifecycleTest, RuntimeSurvivesAddonDestruction) { - const oa::OnnxRuntime* runtimeAddr = nullptr; - - { - AddonNormalizer normalizer(fixturePath("identity.onnx")); - runtimeAddr = &normalizer.runtime(); - } - // normalizer is destroyed, but singleton lives on - - AddonCombiner combiner(fixturePath("add.onnx")); - EXPECT_EQ(runtimeAddr, &combiner.runtime()); - - // The combiner still works correctly - std::vector a = {1.0f, 2.0f, 3.0f, 4.0f}; - std::vector b = {4.0f, 3.0f, 2.0f, 1.0f}; - auto result = combiner.combine(a, b); - EXPECT_FLOAT_EQ(result[0], 5.0f); - EXPECT_FLOAT_EQ(result[1], 5.0f); - EXPECT_FLOAT_EQ(result[2], 5.0f); - EXPECT_FLOAT_EQ(result[3], 5.0f); -} diff --git a/packages/onnx/tests/config_test.cpp b/packages/onnx/tests/config_test.cpp deleted file mode 100644 index 53d5f829ee..0000000000 --- a/packages/onnx/tests/config_test.cpp +++ /dev/null @@ -1,75 +0,0 @@ -#include - -#include "qvac-onnx/OnnxConfig.hpp" - -using namespace onnx_addon; - -TEST(SessionConfigTest, Defaults) { - SessionConfig config; - EXPECT_EQ(config.provider, ExecutionProvider::AUTO_GPU); - EXPECT_EQ(config.optimization, GraphOptimizationLevel::EXTENDED); - EXPECT_EQ(config.intraOpThreads, 0); - EXPECT_EQ(config.interOpThreads, 0); - EXPECT_TRUE(config.enableMemoryPattern); - EXPECT_TRUE(config.enableCpuMemArena); - EXPECT_FALSE(config.enableXnnpack); - EXPECT_EQ(config.executionMode, ExecutionMode::SEQUENTIAL); -} - -TEST(SessionConfigTest, CustomValues) { - SessionConfig config{ - .provider = ExecutionProvider::CPU, - .optimization = GraphOptimizationLevel::ALL, - .intraOpThreads = 4, - .interOpThreads = 2, - .enableMemoryPattern = false, - .enableCpuMemArena = false, - .enableXnnpack = false, - .executionMode = ExecutionMode::PARALLEL}; - - EXPECT_EQ(config.provider, ExecutionProvider::CPU); - EXPECT_EQ(config.optimization, GraphOptimizationLevel::ALL); - EXPECT_EQ(config.intraOpThreads, 4); - EXPECT_EQ(config.interOpThreads, 2); - EXPECT_FALSE(config.enableMemoryPattern); - EXPECT_FALSE(config.enableCpuMemArena); - EXPECT_FALSE(config.enableXnnpack); - EXPECT_EQ(config.executionMode, ExecutionMode::PARALLEL); -} - -TEST(ExecutionProviderTest, AllValues) { - EXPECT_NE(ExecutionProvider::CPU, ExecutionProvider::AUTO_GPU); - EXPECT_NE(ExecutionProvider::NNAPI, ExecutionProvider::CoreML); - EXPECT_NE(ExecutionProvider::CoreML, ExecutionProvider::DirectML); -} - -TEST(GraphOptimizationLevelTest, AllValues) { - EXPECT_NE(GraphOptimizationLevel::DISABLE, GraphOptimizationLevel::BASIC); - EXPECT_NE(GraphOptimizationLevel::BASIC, GraphOptimizationLevel::EXTENDED); - EXPECT_NE(GraphOptimizationLevel::EXTENDED, GraphOptimizationLevel::ALL); -} - -TEST(LoggingLevelTest, AllValues) { - EXPECT_NE(LoggingLevel::VERBOSE, LoggingLevel::INFO); - EXPECT_NE(LoggingLevel::INFO, LoggingLevel::WARNING); - EXPECT_NE(LoggingLevel::WARNING, LoggingLevel::ERROR); - EXPECT_NE(LoggingLevel::ERROR, LoggingLevel::FATAL); -} - -TEST(ExecutionModeTest, AllValues) { - EXPECT_NE(ExecutionMode::SEQUENTIAL, ExecutionMode::PARALLEL); -} - -TEST(EnvironmentConfigTest, Defaults) { - EnvironmentConfig cfg; - EXPECT_EQ(cfg.loggingLevel, LoggingLevel::ERROR); - EXPECT_EQ(cfg.loggingId, "qvac-onnx"); -} - -TEST(EnvironmentConfigTest, CustomValues) { - EnvironmentConfig cfg{ - .loggingLevel = LoggingLevel::VERBOSE, - .loggingId = "my-app"}; - EXPECT_EQ(cfg.loggingLevel, LoggingLevel::VERBOSE); - EXPECT_EQ(cfg.loggingId, "my-app"); -} diff --git a/packages/onnx/tests/fixtures/add.onnx b/packages/onnx/tests/fixtures/add.onnx deleted file mode 100644 index e5c8bea433..0000000000 Binary files a/packages/onnx/tests/fixtures/add.onnx and /dev/null differ diff --git a/packages/onnx/tests/fixtures/identity.onnx b/packages/onnx/tests/fixtures/identity.onnx deleted file mode 100644 index ffb9e96e50..0000000000 Binary files a/packages/onnx/tests/fixtures/identity.onnx and /dev/null differ diff --git a/packages/onnx/tests/fixtures/identity_int64.onnx b/packages/onnx/tests/fixtures/identity_int64.onnx deleted file mode 100644 index 8647b99033..0000000000 Binary files a/packages/onnx/tests/fixtures/identity_int64.onnx and /dev/null differ diff --git a/packages/onnx/tests/fixtures/multi_output.onnx b/packages/onnx/tests/fixtures/multi_output.onnx deleted file mode 100644 index e1d9d04f22..0000000000 Binary files a/packages/onnx/tests/fixtures/multi_output.onnx and /dev/null differ diff --git a/packages/onnx/tests/interface_test.cpp b/packages/onnx/tests/interface_test.cpp deleted file mode 100644 index 319ef3f7d1..0000000000 --- a/packages/onnx/tests/interface_test.cpp +++ /dev/null @@ -1,195 +0,0 @@ -#include -#include - -#include "qvac-onnx/IOnnxSession.hpp" - -using namespace onnx_addon; - -// Mock implementation of IOnnxSession for testing virtual dispatch -class MockOnnxSession : public IOnnxSession { - public: - explicit MockOnnxSession(std::string path) : path_(std::move(path)) {} - - [[nodiscard]] std::vector getInputInfo() const override { - return inputInfo_; - } - - [[nodiscard]] std::vector getOutputInfo() const override { - return outputInfo_; - } - - [[nodiscard]] const std::string &inputName(size_t index) const override { - return inputInfo_[index].name; - } - - [[nodiscard]] const std::string &outputName(size_t index) const override { - return outputInfo_[index].name; - } - - std::vector run(const InputTensor& input) override { - return run(std::vector{input}); - } - - std::vector run( - const std::vector& /*inputs*/) override { - runCallCount_++; - return outputs_; - } - - std::vector run( - const std::vector& inputs, - const std::vector& /*outputNames*/) override { - return run(inputs); - } - - [[nodiscard]] bool isValid() const override { return valid_; } - - [[nodiscard]] const std::string& modelPath() const override { return path_; } - - // Test helpers - void setValid(bool v) { valid_ = v; } - void setInputInfo(std::vector info) { inputInfo_ = std::move(info); } - void setOutputInfo(std::vector info) { - outputInfo_ = std::move(info); - } - void setOutputs(std::vector out) { outputs_ = std::move(out); } - int runCallCount() const { return runCallCount_; } - - private: - std::string path_; - bool valid_ = true; - std::vector inputInfo_; - std::vector outputInfo_; - std::vector outputs_; - int runCallCount_ = 0; -}; - -TEST(IOnnxSessionTest, VirtualDispatchModelPath) { - MockOnnxSession mock("test_model.onnx"); - IOnnxSession& iface = mock; - - EXPECT_EQ(iface.modelPath(), "test_model.onnx"); -} - -TEST(IOnnxSessionTest, VirtualDispatchIsValid) { - MockOnnxSession mock("model.onnx"); - IOnnxSession& iface = mock; - - EXPECT_TRUE(iface.isValid()); - mock.setValid(false); - EXPECT_FALSE(iface.isValid()); -} - -TEST(IOnnxSessionTest, VirtualDispatchGetInputInfo) { - MockOnnxSession mock("model.onnx"); - IOnnxSession& iface = mock; - - TensorInfo info; - info.name = "input_0"; - info.shape = {1, 3, 224, 224}; - info.type = TensorType::FLOAT32; - mock.setInputInfo({info}); - - auto inputs = iface.getInputInfo(); - ASSERT_EQ(inputs.size(), 1); - EXPECT_EQ(inputs[0].name, "input_0"); - EXPECT_EQ(inputs[0].shape, (std::vector{1, 3, 224, 224})); - EXPECT_EQ(inputs[0].type, TensorType::FLOAT32); -} - -TEST(IOnnxSessionTest, VirtualDispatchGetOutputInfo) { - MockOnnxSession mock("model.onnx"); - IOnnxSession& iface = mock; - - TensorInfo info; - info.name = "output_0"; - info.shape = {1, 1000}; - info.type = TensorType::FLOAT32; - mock.setOutputInfo({info}); - - auto outputs = iface.getOutputInfo(); - ASSERT_EQ(outputs.size(), 1); - EXPECT_EQ(outputs[0].name, "output_0"); - EXPECT_EQ(outputs[0].shape, (std::vector{1, 1000})); -} - -TEST(IOnnxSessionTest, VirtualDispatchRunSingleInput) { - MockOnnxSession mock("model.onnx"); - IOnnxSession& iface = mock; - - OutputTensor out; - out.name = "Y"; - out.shape = {1, 4}; - out.type = TensorType::FLOAT32; - out.data.resize(4 * sizeof(float)); - mock.setOutputs({out}); - - float data[] = {1.0f, 2.0f, 3.0f, 4.0f}; - InputTensor input{ - .name = "X", - .shape = {1, 4}, - .type = TensorType::FLOAT32, - .data = data, - .dataSize = sizeof(data)}; - - auto results = iface.run(input); - ASSERT_EQ(results.size(), 1); - EXPECT_EQ(results[0].name, "Y"); - EXPECT_EQ(mock.runCallCount(), 1); -} - -TEST(IOnnxSessionTest, VirtualDispatchRunMultipleInputs) { - MockOnnxSession mock("model.onnx"); - IOnnxSession& iface = mock; - mock.setOutputs({}); - - float data[] = {1.0f}; - std::vector inputs = { - {.name = "A", - .shape = {1}, - .type = TensorType::FLOAT32, - .data = data, - .dataSize = sizeof(float)}, - {.name = "B", - .shape = {1}, - .type = TensorType::FLOAT32, - .data = data, - .dataSize = sizeof(float)}}; - - iface.run(inputs); - EXPECT_EQ(mock.runCallCount(), 1); -} - -TEST(IOnnxSessionTest, VirtualDispatchRunWithOutputNames) { - MockOnnxSession mock("model.onnx"); - IOnnxSession& iface = mock; - mock.setOutputs({}); - - float data[] = {1.0f}; - std::vector inputs = { - {.name = "X", - .shape = {1}, - .type = TensorType::FLOAT32, - .data = data, - .dataSize = sizeof(float)}}; - - iface.run(inputs, {"output_0"}); - EXPECT_EQ(mock.runCallCount(), 1); -} - -TEST(IOnnxSessionTest, PolymorphicDeletion) { - // Ensure deleting through base pointer works (virtual destructor) - auto* mock = new MockOnnxSession("model.onnx"); - IOnnxSession* iface = mock; - delete iface; // Should not leak -} - -TEST(IOnnxSessionTest, MultipleImplementations) { - MockOnnxSession mock1("model_a.onnx"); - MockOnnxSession mock2("model_b.onnx"); - - std::vector sessions = {&mock1, &mock2}; - - EXPECT_EQ(sessions[0]->modelPath(), "model_a.onnx"); - EXPECT_EQ(sessions[1]->modelPath(), "model_b.onnx"); -} diff --git a/packages/onnx/tests/runtime_test.cpp b/packages/onnx/tests/runtime_test.cpp deleted file mode 100644 index 42c9df7f69..0000000000 --- a/packages/onnx/tests/runtime_test.cpp +++ /dev/null @@ -1,46 +0,0 @@ -#include - -#include "qvac-onnx/OnnxRuntime.hpp" - -namespace oa = onnx_addon; - -TEST(OnnxRuntimeTest, SingletonReturnsSameInstance) { - auto& inst1 = oa::OnnxRuntime::instance(); - auto& inst2 = oa::OnnxRuntime::instance(); - EXPECT_EQ(&inst1, &inst2); -} - -TEST(OnnxRuntimeTest, EnvReturnsSameReference) { - auto& env1 = oa::OnnxRuntime::instance().env(); - auto& env2 = oa::OnnxRuntime::instance().env(); - EXPECT_EQ(&env1, &env2); -} - -TEST(OnnxRuntimeTest, EnvIsNotNull) { - auto& env = oa::OnnxRuntime::instance().env(); - // Ort::Env is valid if we can query available providers through it - // (there's no direct "isValid" on Env, but GetAvailableProviders uses the - // global ORT API which requires a valid environment to be initialized) - auto providers = Ort::GetAvailableProviders(); - EXPECT_FALSE(providers.empty()); - // CPUExecutionProvider is always available - EXPECT_NE(std::find(providers.begin(), providers.end(), - "CPUExecutionProvider"), - providers.end()); -} - -TEST(OnnxRuntimeTest, GetAvailableProviders) { - auto providers = oa::OnnxRuntime::getAvailableProviders(); - EXPECT_FALSE(providers.empty()); - EXPECT_NE(std::find(providers.begin(), providers.end(), - "CPUExecutionProvider"), - providers.end()); -} - -TEST(OnnxRuntimeTest, ConfigureThrowsAfterInit) { - // Ensure the singleton is initialized first - (void)oa::OnnxRuntime::instance(); - // Now configure() must throw since the environment is already created - oa::EnvironmentConfig cfg; - EXPECT_THROW(oa::OnnxRuntime::configure(cfg), std::runtime_error); -} diff --git a/packages/onnx/tests/session_options_test.cpp b/packages/onnx/tests/session_options_test.cpp deleted file mode 100644 index 0d43c60ee7..0000000000 --- a/packages/onnx/tests/session_options_test.cpp +++ /dev/null @@ -1,122 +0,0 @@ -#include - -#include "qvac-onnx/OnnxSessionOptionsBuilder.hpp" - -// Do not use "using namespace onnx_addon" here because the ORT C API -// defines a global GraphOptimizationLevel typedef that collides with -// onnx_addon::GraphOptimizationLevel. -namespace oa = onnx_addon; - -TEST(SessionOptionsTest, CpuProviderReturnsValidOptions) { - oa::SessionConfig config{.provider = oa::ExecutionProvider::CPU}; - auto options = oa::buildSessionOptions(config); - // If we get here without throwing, the options are valid - SUCCEED(); -} - -TEST(SessionOptionsTest, AutoGpuProviderReturnsValidOptions) { - oa::SessionConfig config{.provider = oa::ExecutionProvider::AUTO_GPU}; - auto options = oa::buildSessionOptions(config); - SUCCEED(); -} - -TEST(SessionOptionsTest, DefaultConfigReturnsValidOptions) { - oa::SessionConfig config; - auto options = oa::buildSessionOptions(config); - SUCCEED(); -} - -TEST(SessionOptionsTest, AllOptimizationLevels) { - for (auto level : {oa::GraphOptimizationLevel::DISABLE, - oa::GraphOptimizationLevel::BASIC, - oa::GraphOptimizationLevel::EXTENDED, - oa::GraphOptimizationLevel::ALL}) { - oa::SessionConfig config{ - .provider = oa::ExecutionProvider::CPU, .optimization = level}; - auto options = oa::buildSessionOptions(config); - SUCCEED(); - } -} - -TEST(SessionOptionsTest, CustomThreadCounts) { - oa::SessionConfig config{ - .provider = oa::ExecutionProvider::CPU, - .intraOpThreads = 2, - .interOpThreads = 1}; - auto options = oa::buildSessionOptions(config); - SUCCEED(); -} - -TEST(SessionOptionsTest, XnnpackEnabledDoesNotThrow) { - oa::SessionConfig config{ - .provider = oa::ExecutionProvider::CPU, .enableXnnpack = true}; - EXPECT_NO_THROW(oa::buildSessionOptions(config)); -} - -TEST(SessionOptionsTest, XnnpackDisabledDoesNotThrow) { - oa::SessionConfig config{ - .provider = oa::ExecutionProvider::CPU, .enableXnnpack = false}; - EXPECT_NO_THROW(oa::buildSessionOptions(config)); -} - -TEST(SessionOptionsTest, TryAppendXnnpackDoesNotThrow) { - Ort::SessionOptions options; - EXPECT_NO_THROW(oa::tryAppendXnnpack(options)); -} - -TEST(SessionOptionsTest, XnnpackWithAutoGpuDoesNotThrow) { - oa::SessionConfig config{ - .provider = oa::ExecutionProvider::AUTO_GPU, .enableXnnpack = true}; - EXPECT_NO_THROW(oa::buildSessionOptions(config)); -} - -TEST(SessionOptionsTest, AllProvidersDoNotThrow) { - for (auto provider : - {oa::ExecutionProvider::CPU, oa::ExecutionProvider::AUTO_GPU, - oa::ExecutionProvider::NNAPI, oa::ExecutionProvider::CoreML, - oa::ExecutionProvider::DirectML}) { - oa::SessionConfig config{.provider = provider}; - EXPECT_NO_THROW(oa::buildSessionOptions(config)); - } -} - -TEST(SessionOptionsTest, ExecutionModeSequential) { - oa::SessionConfig config{ - .provider = oa::ExecutionProvider::CPU, - .executionMode = oa::ExecutionMode::SEQUENTIAL}; - EXPECT_NO_THROW(oa::buildSessionOptions(config)); -} - -TEST(SessionOptionsTest, ExecutionModeParallel) { - oa::SessionConfig config{ - .provider = oa::ExecutionProvider::CPU, - .executionMode = oa::ExecutionMode::PARALLEL}; - EXPECT_NO_THROW(oa::buildSessionOptions(config)); -} - -TEST(SessionOptionsTest, DisableMemoryPattern) { - oa::SessionConfig config{ - .provider = oa::ExecutionProvider::CPU, - .enableMemoryPattern = false}; - EXPECT_NO_THROW(oa::buildSessionOptions(config)); -} - -TEST(SessionOptionsTest, DisableCpuMemArena) { - oa::SessionConfig config{ - .provider = oa::ExecutionProvider::CPU, - .enableCpuMemArena = false}; - EXPECT_NO_THROW(oa::buildSessionOptions(config)); -} - -TEST(SessionOptionsTest, AllConfigFieldsCombined) { - oa::SessionConfig config{ - .provider = oa::ExecutionProvider::CPU, - .optimization = oa::GraphOptimizationLevel::ALL, - .intraOpThreads = 2, - .interOpThreads = 1, - .enableMemoryPattern = false, - .enableCpuMemArena = false, - .enableXnnpack = false, - .executionMode = oa::ExecutionMode::PARALLEL}; - EXPECT_NO_THROW(oa::buildSessionOptions(config)); -} diff --git a/packages/onnx/tests/session_test.cpp b/packages/onnx/tests/session_test.cpp deleted file mode 100644 index 60aaf21139..0000000000 --- a/packages/onnx/tests/session_test.cpp +++ /dev/null @@ -1,380 +0,0 @@ -#include - -#include -#include -#include -#include - -#include "qvac-onnx/OnnxSession.hpp" - -// Do not use "using namespace onnx_addon" because ORT C API defines a global -// GraphOptimizationLevel typedef that collides with onnx_addon's. -namespace oa = onnx_addon; - -// Resolve fixture paths relative to the test source directory. -// CMAKE_CURRENT_SOURCE_DIR is passed via -D at compile time. -#ifndef TEST_FIXTURES_DIR -#error "TEST_FIXTURES_DIR must be defined at compile time" -#endif - -static std::string fixturePath(const std::string& name) { - return std::string(TEST_FIXTURES_DIR) + "/" + name; -} - -// --------------------------------------------------------------------------- -// Construction -// --------------------------------------------------------------------------- - -TEST(OnnxSessionTest, ConstructWithIdentityModel) { - oa::OnnxSession session(fixturePath("identity.onnx"), - {.provider = oa::ExecutionProvider::CPU}); - EXPECT_TRUE(session.isValid()); -} - -TEST(OnnxSessionTest, ModelPathIsStored) { - auto path = fixturePath("identity.onnx"); - oa::OnnxSession session(path, {.provider = oa::ExecutionProvider::CPU}); - EXPECT_EQ(session.modelPath(), path); -} - -TEST(OnnxSessionTest, InvalidPathThrows) { - EXPECT_THROW(oa::OnnxSession("/nonexistent/model.onnx", - {.provider = oa::ExecutionProvider::CPU}), - Ort::Exception); -} - -TEST(OnnxSessionTest, MoveConstruction) { - oa::OnnxSession original(fixturePath("identity.onnx"), - {.provider = oa::ExecutionProvider::CPU}); - ASSERT_TRUE(original.isValid()); - - oa::OnnxSession moved(std::move(original)); - EXPECT_TRUE(moved.isValid()); - EXPECT_EQ(moved.modelPath(), fixturePath("identity.onnx")); -} - -TEST(OnnxSessionTest, MoveAssignment) { - oa::OnnxSession a(fixturePath("identity.onnx"), - {.provider = oa::ExecutionProvider::CPU}); - oa::OnnxSession b(fixturePath("add.onnx"), - {.provider = oa::ExecutionProvider::CPU}); - - b = std::move(a); - EXPECT_TRUE(b.isValid()); - EXPECT_EQ(b.modelPath(), fixturePath("identity.onnx")); -} - -// --------------------------------------------------------------------------- -// Introspection - Identity model (1 input, 1 output, float [1,4]) -// --------------------------------------------------------------------------- - -class IdentitySessionTest : public ::testing::Test { - protected: - void SetUp() override { - session_ = std::make_unique(fixturePath("identity.onnx"), - oa::SessionConfig{ - .provider = oa::ExecutionProvider::CPU, - .enableXnnpack = false}); - } - std::unique_ptr session_; -}; - -TEST_F(IdentitySessionTest, GetInputInfo) { - auto inputs = session_->getInputInfo(); - ASSERT_EQ(inputs.size(), 1); - EXPECT_EQ(inputs[0].name, "X"); - EXPECT_EQ(inputs[0].type, oa::TensorType::FLOAT32); - ASSERT_EQ(inputs[0].shape.size(), 2); - EXPECT_EQ(inputs[0].shape[0], 1); - EXPECT_EQ(inputs[0].shape[1], 4); -} - -TEST_F(IdentitySessionTest, GetOutputInfo) { - auto outputs = session_->getOutputInfo(); - ASSERT_EQ(outputs.size(), 1); - EXPECT_EQ(outputs[0].name, "Y"); - EXPECT_EQ(outputs[0].type, oa::TensorType::FLOAT32); - ASSERT_EQ(outputs[0].shape.size(), 2); - EXPECT_EQ(outputs[0].shape[0], 1); - EXPECT_EQ(outputs[0].shape[1], 4); -} - -TEST_F(IdentitySessionTest, RunSingleInput) { - float data[] = {1.0f, 2.0f, 3.0f, 4.0f}; - oa::InputTensor input{.name = "X", - .shape = {1, 4}, - .type = oa::TensorType::FLOAT32, - .data = data, - .dataSize = sizeof(data)}; - - auto results = session_->run(input); - ASSERT_EQ(results.size(), 1); - EXPECT_EQ(results[0].name, "Y"); - EXPECT_EQ(results[0].type, oa::TensorType::FLOAT32); - ASSERT_EQ(results[0].shape, (std::vector{1, 4})); - ASSERT_EQ(results[0].elementCount(), 4); - - const float* out = results[0].as(); - EXPECT_FLOAT_EQ(out[0], 1.0f); - EXPECT_FLOAT_EQ(out[1], 2.0f); - EXPECT_FLOAT_EQ(out[2], 3.0f); - EXPECT_FLOAT_EQ(out[3], 4.0f); -} - -TEST_F(IdentitySessionTest, RunMultipleInputsOverload) { - float data[] = {5.0f, 6.0f, 7.0f, 8.0f}; - std::vector inputs = { - {.name = "X", - .shape = {1, 4}, - .type = oa::TensorType::FLOAT32, - .data = data, - .dataSize = sizeof(data)}}; - - auto results = session_->run(inputs); - ASSERT_EQ(results.size(), 1); - - const float* out = results[0].as(); - EXPECT_FLOAT_EQ(out[0], 5.0f); - EXPECT_FLOAT_EQ(out[3], 8.0f); -} - -TEST_F(IdentitySessionTest, RunWithSpecificOutputNames) { - float data[] = {10.0f, 20.0f, 30.0f, 40.0f}; - std::vector inputs = { - {.name = "X", - .shape = {1, 4}, - .type = oa::TensorType::FLOAT32, - .data = data, - .dataSize = sizeof(data)}}; - - auto results = session_->run(inputs, {"Y"}); - ASSERT_EQ(results.size(), 1); - EXPECT_EQ(results[0].name, "Y"); - - const float* out = results[0].as(); - EXPECT_FLOAT_EQ(out[0], 10.0f); -} - -TEST_F(IdentitySessionTest, RunMultipleTimes) { - for (int i = 0; i < 5; ++i) { - float data[] = {static_cast(i), 0.0f, 0.0f, 0.0f}; - oa::InputTensor input{.name = "X", - .shape = {1, 4}, - .type = oa::TensorType::FLOAT32, - .data = data, - .dataSize = sizeof(data)}; - - auto results = session_->run(input); - EXPECT_FLOAT_EQ(results[0].as()[0], static_cast(i)); - } -} - -// --------------------------------------------------------------------------- -// Add model (2 inputs, 1 output, float [1,4]) -// --------------------------------------------------------------------------- - -class AddSessionTest : public ::testing::Test { - protected: - void SetUp() override { - session_ = std::make_unique( - fixturePath("add.onnx"), - oa::SessionConfig{.provider = oa::ExecutionProvider::CPU, - .enableXnnpack = false}); - } - std::unique_ptr session_; -}; - -TEST_F(AddSessionTest, GetInputInfo) { - auto inputs = session_->getInputInfo(); - ASSERT_EQ(inputs.size(), 2); - EXPECT_EQ(inputs[0].name, "A"); - EXPECT_EQ(inputs[1].name, "B"); -} - -TEST_F(AddSessionTest, RunAddsInputs) { - float a[] = {1.0f, 2.0f, 3.0f, 4.0f}; - float b[] = {10.0f, 20.0f, 30.0f, 40.0f}; - - std::vector inputs = { - {.name = "A", - .shape = {1, 4}, - .type = oa::TensorType::FLOAT32, - .data = a, - .dataSize = sizeof(a)}, - {.name = "B", - .shape = {1, 4}, - .type = oa::TensorType::FLOAT32, - .data = b, - .dataSize = sizeof(b)}}; - - auto results = session_->run(inputs); - ASSERT_EQ(results.size(), 1); - EXPECT_EQ(results[0].name, "C"); - - const float* out = results[0].as(); - EXPECT_FLOAT_EQ(out[0], 11.0f); - EXPECT_FLOAT_EQ(out[1], 22.0f); - EXPECT_FLOAT_EQ(out[2], 33.0f); - EXPECT_FLOAT_EQ(out[3], 44.0f); -} - -// --------------------------------------------------------------------------- -// Multi-output model (1 input, 2 outputs: identity_out + relu_out) -// --------------------------------------------------------------------------- - -class MultiOutputSessionTest : public ::testing::Test { - protected: - void SetUp() override { - session_ = std::make_unique( - fixturePath("multi_output.onnx"), - oa::SessionConfig{.provider = oa::ExecutionProvider::CPU, - .enableXnnpack = false}); - } - std::unique_ptr session_; -}; - -TEST_F(MultiOutputSessionTest, GetOutputInfo) { - auto outputs = session_->getOutputInfo(); - ASSERT_EQ(outputs.size(), 2); - EXPECT_EQ(outputs[0].name, "identity_out"); - EXPECT_EQ(outputs[1].name, "relu_out"); -} - -TEST_F(MultiOutputSessionTest, RunAllOutputs) { - float data[] = {-2.0f, -1.0f, 0.0f, 3.0f}; - oa::InputTensor input{.name = "X", - .shape = {1, 4}, - .type = oa::TensorType::FLOAT32, - .data = data, - .dataSize = sizeof(data)}; - - auto results = session_->run(input); - ASSERT_EQ(results.size(), 2); - - // Identity output: same as input - const float* identity = results[0].as(); - EXPECT_FLOAT_EQ(identity[0], -2.0f); - EXPECT_FLOAT_EQ(identity[1], -1.0f); - EXPECT_FLOAT_EQ(identity[2], 0.0f); - EXPECT_FLOAT_EQ(identity[3], 3.0f); - - // Relu output: max(0, x) - const float* relu = results[1].as(); - EXPECT_FLOAT_EQ(relu[0], 0.0f); - EXPECT_FLOAT_EQ(relu[1], 0.0f); - EXPECT_FLOAT_EQ(relu[2], 0.0f); - EXPECT_FLOAT_EQ(relu[3], 3.0f); -} - -TEST_F(MultiOutputSessionTest, RunSelectiveOutput) { - float data[] = {-1.0f, 0.0f, 1.0f, 2.0f}; - std::vector inputs = { - {.name = "X", - .shape = {1, 4}, - .type = oa::TensorType::FLOAT32, - .data = data, - .dataSize = sizeof(data)}}; - - // Request only relu_out - auto results = session_->run(inputs, {"relu_out"}); - ASSERT_EQ(results.size(), 1); - EXPECT_EQ(results[0].name, "relu_out"); - - const float* relu = results[0].as(); - EXPECT_FLOAT_EQ(relu[0], 0.0f); - EXPECT_FLOAT_EQ(relu[3], 2.0f); -} - -// --------------------------------------------------------------------------- -// Int64 Identity model -// --------------------------------------------------------------------------- - -TEST(OnnxSessionInt64Test, RunInt64Identity) { - oa::OnnxSession session(fixturePath("identity_int64.onnx"), - {.provider = oa::ExecutionProvider::CPU, - .enableXnnpack = false}); - - auto inputs_info = session.getInputInfo(); - ASSERT_EQ(inputs_info.size(), 1); - EXPECT_EQ(inputs_info[0].type, oa::TensorType::INT64); - - int64_t data[] = {100, 200, 300}; - oa::InputTensor input{.name = "X", - .shape = {1, 3}, - .type = oa::TensorType::INT64, - .data = data, - .dataSize = sizeof(data)}; - - auto results = session.run(input); - ASSERT_EQ(results.size(), 1); - EXPECT_EQ(results[0].type, oa::TensorType::INT64); - - const int64_t* out = results[0].as(); - EXPECT_EQ(out[0], 100); - EXPECT_EQ(out[1], 200); - EXPECT_EQ(out[2], 300); -} - -// --------------------------------------------------------------------------- -// Shared singleton env: multiple sessions use the same OnnxRuntime -// --------------------------------------------------------------------------- - -TEST(OnnxSessionSharedEnvTest, MultipleSessionsShareEnv) { - oa::OnnxSession session1(fixturePath("identity.onnx"), - {.provider = oa::ExecutionProvider::CPU, - .enableXnnpack = false}); - oa::OnnxSession session2(fixturePath("add.onnx"), - {.provider = oa::ExecutionProvider::CPU, - .enableXnnpack = false}); - oa::OnnxSession session3(fixturePath("multi_output.onnx"), - {.provider = oa::ExecutionProvider::CPU, - .enableXnnpack = false}); - - EXPECT_TRUE(session1.isValid()); - EXPECT_TRUE(session2.isValid()); - EXPECT_TRUE(session3.isValid()); - - // Run each to verify they all work independently - float data[] = {1.0f, 2.0f, 3.0f, 4.0f}; - oa::InputTensor input{.name = "X", - .shape = {1, 4}, - .type = oa::TensorType::FLOAT32, - .data = data, - .dataSize = sizeof(data)}; - - auto r1 = session1.run(input); - EXPECT_EQ(r1[0].as()[0], 1.0f); - - auto r3 = session3.run(input); - EXPECT_EQ(r3.size(), 2); -} - -// --------------------------------------------------------------------------- -// Virtual dispatch through oa::IOnnxSession interface -// --------------------------------------------------------------------------- - -TEST(OnnxSessionInterfaceTest, UseViaBasePointer) { - auto session = std::make_unique( - fixturePath("identity.onnx"), - oa::SessionConfig{.provider = oa::ExecutionProvider::CPU, - .enableXnnpack = false}); - - oa::IOnnxSession* iface = session.get(); - - EXPECT_TRUE(iface->isValid()); - EXPECT_EQ(iface->modelPath(), fixturePath("identity.onnx")); - - auto info = iface->getInputInfo(); - ASSERT_EQ(info.size(), 1); - EXPECT_EQ(info[0].name, "X"); - - float data[] = {42.0f, 0.0f, 0.0f, 0.0f}; - oa::InputTensor input{.name = "X", - .shape = {1, 4}, - .type = oa::TensorType::FLOAT32, - .data = data, - .dataSize = sizeof(data)}; - - auto results = iface->run(input); - EXPECT_FLOAT_EQ(results[0].as()[0], 42.0f); -} diff --git a/packages/onnx/tests/tensor_types_test.cpp b/packages/onnx/tests/tensor_types_test.cpp deleted file mode 100644 index 9ab56646c8..0000000000 --- a/packages/onnx/tests/tensor_types_test.cpp +++ /dev/null @@ -1,87 +0,0 @@ -#include - -#include "qvac-onnx/OnnxTensor.hpp" - -using namespace onnx_addon; - -TEST(TensorTypeTest, TensorTypeSizes) { - EXPECT_EQ(tensorTypeSize(TensorType::FLOAT32), 4); - EXPECT_EQ(tensorTypeSize(TensorType::FLOAT16), 2); - EXPECT_EQ(tensorTypeSize(TensorType::INT64), 8); - EXPECT_EQ(tensorTypeSize(TensorType::INT32), 4); - EXPECT_EQ(tensorTypeSize(TensorType::INT8), 1); - EXPECT_EQ(tensorTypeSize(TensorType::UINT8), 1); -} - -TEST(OutputTensorTest, ElementCount) { - OutputTensor tensor; - tensor.shape = {2, 3, 4}; - tensor.type = TensorType::FLOAT32; - EXPECT_EQ(tensor.elementCount(), 24); -} - -TEST(OutputTensorTest, ElementCountScalar) { - OutputTensor tensor; - tensor.shape = {1}; - tensor.type = TensorType::FLOAT32; - EXPECT_EQ(tensor.elementCount(), 1); -} - -TEST(OutputTensorTest, ElementCountEmpty) { - OutputTensor tensor; - tensor.type = TensorType::FLOAT32; - EXPECT_EQ(tensor.elementCount(), 0); -} - -TEST(OutputTensorTest, TypedAccess) { - OutputTensor tensor; - tensor.shape = {2}; - tensor.type = TensorType::FLOAT32; - tensor.data.resize(2 * sizeof(float)); - - float* ptr = tensor.asMutable(); - ptr[0] = 1.0f; - ptr[1] = 2.0f; - - const auto& constTensor = tensor; - EXPECT_FLOAT_EQ(constTensor.as()[0], 1.0f); - EXPECT_FLOAT_EQ(constTensor.as()[1], 2.0f); -} - -TEST(OutputTensorTest, Int64TypedAccess) { - OutputTensor tensor; - tensor.shape = {3}; - tensor.type = TensorType::INT64; - tensor.data.resize(3 * sizeof(int64_t)); - - int64_t* ptr = tensor.asMutable(); - ptr[0] = 100; - ptr[1] = 200; - ptr[2] = 300; - - EXPECT_EQ(tensor.as()[0], 100); - EXPECT_EQ(tensor.as()[1], 200); - EXPECT_EQ(tensor.as()[2], 300); -} - -TEST(InputTensorTest, Defaults) { - InputTensor input; - EXPECT_EQ(input.type, TensorType::FLOAT32); - EXPECT_EQ(input.data, nullptr); - EXPECT_EQ(input.dataSize, 0); - EXPECT_TRUE(input.name.empty()); - EXPECT_TRUE(input.shape.empty()); -} - -TEST(TensorInfoTest, Construction) { - TensorInfo info; - info.name = "input_0"; - info.shape = {1, 3, 224, 224}; - info.type = TensorType::FLOAT32; - - EXPECT_EQ(info.name, "input_0"); - EXPECT_EQ(info.shape.size(), 4); - EXPECT_EQ(info.shape[0], 1); - EXPECT_EQ(info.shape[3], 224); - EXPECT_EQ(info.type, TensorType::FLOAT32); -} diff --git a/packages/onnx/tests/type_conversions_test.cpp b/packages/onnx/tests/type_conversions_test.cpp deleted file mode 100644 index f62011c862..0000000000 --- a/packages/onnx/tests/type_conversions_test.cpp +++ /dev/null @@ -1,78 +0,0 @@ -#include - -#include "qvac-onnx/OnnxTypeConversions.hpp" - -namespace oa = onnx_addon; - -TEST(TypeConversionsTest, ToOnnxTypeFloat32) { - EXPECT_EQ(oa::toOnnxType(oa::TensorType::FLOAT32), ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT); -} - -TEST(TypeConversionsTest, ToOnnxTypeFloat16) { - EXPECT_EQ(oa::toOnnxType(oa::TensorType::FLOAT16), - ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT16); -} - -TEST(TypeConversionsTest, ToOnnxTypeInt64) { - EXPECT_EQ(oa::toOnnxType(oa::TensorType::INT64), ONNX_TENSOR_ELEMENT_DATA_TYPE_INT64); -} - -TEST(TypeConversionsTest, ToOnnxTypeInt32) { - EXPECT_EQ(oa::toOnnxType(oa::TensorType::INT32), ONNX_TENSOR_ELEMENT_DATA_TYPE_INT32); -} - -TEST(TypeConversionsTest, ToOnnxTypeInt8) { - EXPECT_EQ(oa::toOnnxType(oa::TensorType::INT8), ONNX_TENSOR_ELEMENT_DATA_TYPE_INT8); -} - -TEST(TypeConversionsTest, ToOnnxTypeUint8) { - EXPECT_EQ(oa::toOnnxType(oa::TensorType::UINT8), ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT8); -} - -TEST(TypeConversionsTest, FromOnnxTypeFloat) { - EXPECT_EQ(oa::fromOnnxType(ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT), - oa::TensorType::FLOAT32); -} - -TEST(TypeConversionsTest, FromOnnxTypeFloat16) { - EXPECT_EQ(oa::fromOnnxType(ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT16), - oa::TensorType::FLOAT16); -} - -TEST(TypeConversionsTest, FromOnnxTypeInt64) { - EXPECT_EQ(oa::fromOnnxType(ONNX_TENSOR_ELEMENT_DATA_TYPE_INT64), - oa::TensorType::INT64); -} - -TEST(TypeConversionsTest, FromOnnxTypeInt32) { - EXPECT_EQ(oa::fromOnnxType(ONNX_TENSOR_ELEMENT_DATA_TYPE_INT32), - oa::TensorType::INT32); -} - -TEST(TypeConversionsTest, FromOnnxTypeInt8) { - EXPECT_EQ(oa::fromOnnxType(ONNX_TENSOR_ELEMENT_DATA_TYPE_INT8), oa::TensorType::INT8); -} - -TEST(TypeConversionsTest, FromOnnxTypeUint8) { - EXPECT_EQ(oa::fromOnnxType(ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT8), - oa::TensorType::UINT8); -} - -TEST(TypeConversionsTest, FromOnnxTypeUnknownDefaultsToFloat32) { - EXPECT_EQ(oa::fromOnnxType(ONNX_TENSOR_ELEMENT_DATA_TYPE_DOUBLE), - oa::TensorType::FLOAT32); - EXPECT_EQ(oa::fromOnnxType(ONNX_TENSOR_ELEMENT_DATA_TYPE_STRING), - oa::TensorType::FLOAT32); - EXPECT_EQ(oa::fromOnnxType(ONNX_TENSOR_ELEMENT_DATA_TYPE_BOOL), - oa::TensorType::FLOAT32); -} - -TEST(TypeConversionsTest, RoundTripAllTypes) { - // Verify that toOnnxType -> fromOnnxType is identity for all supported types - EXPECT_EQ(oa::fromOnnxType(oa::toOnnxType(oa::TensorType::FLOAT32)), oa::TensorType::FLOAT32); - EXPECT_EQ(oa::fromOnnxType(oa::toOnnxType(oa::TensorType::FLOAT16)), oa::TensorType::FLOAT16); - EXPECT_EQ(oa::fromOnnxType(oa::toOnnxType(oa::TensorType::INT64)), oa::TensorType::INT64); - EXPECT_EQ(oa::fromOnnxType(oa::toOnnxType(oa::TensorType::INT32)), oa::TensorType::INT32); - EXPECT_EQ(oa::fromOnnxType(oa::toOnnxType(oa::TensorType::INT8)), oa::TensorType::INT8); - EXPECT_EQ(oa::fromOnnxType(oa::toOnnxType(oa::TensorType::UINT8)), oa::TensorType::UINT8); -} diff --git a/packages/onnx/vcpkg-configuration.json b/packages/onnx/vcpkg-configuration.json deleted file mode 100644 index 5c87f809d7..0000000000 --- a/packages/onnx/vcpkg-configuration.json +++ /dev/null @@ -1,40 +0,0 @@ -{ - "default-registry": { - "kind": "git", - "baseline": "5c6f9e7c57d49c0aaf40f85d791e7ab85512b132", - "repository": "https://github.com/tetherto/qvac-registry-vcpkg.git" - }, - "registries": [ - { - "kind": "git", - "baseline": "8c901fe2b0e69a542d02810d4089505fd0c480d8", - "repository": "https://github.com/microsoft/vcpkg", - "packages": [ - "directx-headers", - "flatbuffers", - "fp16", - "fxdiv", - "boost-config", - "boost-mp11", - "cxxopts", - "date", - "dlpack", - "ms-gsl", - "nlohmann-json", - "optional-lite", - "psimd", - "re2", - "safeint", - "boost-cmake", - "boost-headers", - "boost-uninstall", - "vcpkg-boost", - "mimalloc", - "wil", - "gtest", - "utf8-range", - "zlib" - ] - } - ] -} diff --git a/packages/onnx/vcpkg-override-triplets/triplets/arm64-ios-simulator.cmake b/packages/onnx/vcpkg-override-triplets/triplets/arm64-ios-simulator.cmake deleted file mode 100644 index a9ee44299d..0000000000 --- a/packages/onnx/vcpkg-override-triplets/triplets/arm64-ios-simulator.cmake +++ /dev/null @@ -1,3 +0,0 @@ -include("${CMAKE_CURRENT_LIST_DIR}/../../../../vcpkg-overlays/triplets/arm64-ios-simulator.cmake") - -set(VCPKG_OSX_DEPLOYMENT_TARGET 16.3) diff --git a/packages/onnx/vcpkg-override-triplets/triplets/arm64-ios.cmake b/packages/onnx/vcpkg-override-triplets/triplets/arm64-ios.cmake deleted file mode 100644 index 2b04ea3c08..0000000000 --- a/packages/onnx/vcpkg-override-triplets/triplets/arm64-ios.cmake +++ /dev/null @@ -1,3 +0,0 @@ -include("${CMAKE_CURRENT_LIST_DIR}/../../../../vcpkg-overlays/triplets/arm64-ios.cmake") - -set(VCPKG_OSX_DEPLOYMENT_TARGET 16.3) diff --git a/packages/onnx/vcpkg-override-triplets/triplets/arm64-osx.cmake b/packages/onnx/vcpkg-override-triplets/triplets/arm64-osx.cmake deleted file mode 100644 index 04124c92b3..0000000000 --- a/packages/onnx/vcpkg-override-triplets/triplets/arm64-osx.cmake +++ /dev/null @@ -1,3 +0,0 @@ -include("${CMAKE_CURRENT_LIST_DIR}/../../../../vcpkg-overlays/triplets/arm64-osx.cmake") - -set(VCPKG_OSX_DEPLOYMENT_TARGET 13.3) diff --git a/packages/onnx/vcpkg-override-triplets/triplets/x64-ios-simulator.cmake b/packages/onnx/vcpkg-override-triplets/triplets/x64-ios-simulator.cmake deleted file mode 100644 index 3edadb2156..0000000000 --- a/packages/onnx/vcpkg-override-triplets/triplets/x64-ios-simulator.cmake +++ /dev/null @@ -1,3 +0,0 @@ -include("${CMAKE_CURRENT_LIST_DIR}/../../../../vcpkg-overlays/triplets/x64-ios-simulator.cmake") - -set(VCPKG_OSX_DEPLOYMENT_TARGET 16.3) diff --git a/packages/onnx/vcpkg-override-triplets/triplets/x64-linux.cmake b/packages/onnx/vcpkg-override-triplets/triplets/x64-linux.cmake deleted file mode 100644 index 7182c3e081..0000000000 --- a/packages/onnx/vcpkg-override-triplets/triplets/x64-linux.cmake +++ /dev/null @@ -1,4 +0,0 @@ -include("${CMAKE_CURRENT_LIST_DIR}/../../../../vcpkg-overlays/triplets/x64-linux.cmake") - -set(VCPKG_C_FLAGS "-fPIC -Wno-array-bounds") -set(VCPKG_CXX_FLAGS "-fPIC -stdlib=libc++ -Wno-array-bounds") diff --git a/packages/onnx/vcpkg-override-triplets/triplets/x64-osx.cmake b/packages/onnx/vcpkg-override-triplets/triplets/x64-osx.cmake deleted file mode 100644 index 13595a87aa..0000000000 --- a/packages/onnx/vcpkg-override-triplets/triplets/x64-osx.cmake +++ /dev/null @@ -1,5 +0,0 @@ -include("${CMAKE_CURRENT_LIST_DIR}/../../../../vcpkg-overlays/triplets/x64-osx.cmake") - -set(VCPKG_OSX_DEPLOYMENT_TARGET 13.3) -set(VCPKG_CXX_FLAGS "-Wno-array-bounds") -set(VCPKG_C_FLAGS "-Wno-array-bounds") diff --git a/packages/onnx/vcpkg.json b/packages/onnx/vcpkg.json deleted file mode 100644 index cd8ba1024a..0000000000 --- a/packages/onnx/vcpkg.json +++ /dev/null @@ -1,64 +0,0 @@ -{ - "name": "qvac-onnx", - "version": "1.0.0", - "dependencies": [ - { - "name": "onnxruntime", - "platform": "!(android | osx | ios | windows)", - "version>=": "1.24.2#6" - }, - { - "name": "onnxruntime", - "features": [ - "dml-ep" - ], - "platform": "windows", - "version>=": "1.24.2#6" - }, - { - "name": "onnxruntime", - "features": [ - "nnapi-ep" - ], - "platform": "android", - "version>=": "1.24.2#6" - }, - { - "name": "onnxruntime", - "features": [ - "coreml-ep" - ], - "platform": "osx | ios", - "version>=": "1.24.2#6" - }, - { - "name": "qvac-lint-cpp", - "version>=": "1.4.4#3" - } - ], - "overrides": [ - { - "name": "flatbuffers", - "version": "23.5.26" - } - ], - "features": { - "xnnpack": { - "description": "Enable XNNPack execution provider for optimized CPU inference", - "dependencies": [ - { - "name": "onnxruntime", - "features": [ - "xnnpack-ep" - ] - } - ] - }, - "tests": { - "description": "Build tests", - "dependencies": [ - "gtest" - ] - } - } -}