Skip to content

Return MLX errors to ORT instead of exiting the process - #2

Open
altunenes wants to merge 1 commit into
onnxruntime:mainfrom
altunenes:ReturnMLXerrorstoORT
Open

altunenes wants to merge 1 commit into
onnxruntime:mainfrom
altunenes:ReturnMLXerrorstoORT

Conversation

@altunenes

Copy link
Copy Markdown

Hey, thanks for this project! I started exploring/experimenting it with some ASR models (Parakeet TDT, Sortformer), and the results look great so far :-) . While testing, I hit a crash: when an input is invalid, the whole app exits instead of getting an error back.

based on my experiments, the cause is mlx-c's default error handler, which prints the message and calls exit(-1). The EP never replaces it, so its own error handling never gets a chance to run. I just add a handler that just logs the message... The failing mlx-c call then returns a non-zero status, and ORT gets a normal ORT_EP_FAIL.

if you want to reproduce: run the Parakeet TDT encoder with an input that's too long (T=45000). The CPU EP returns a broadcast error. The MLX EP currently exits with status 255. With this change it returns ExecutionProviderFailure, and the same session keeps working and still matches CPU afterwards.

Btw my simple tests:
For 40 sec audio:
CPU: 2.82 s
TDT, encoder on MLX, decoder on CPU = 0.64 s
TDT, all on MLX = 0.89 s

4.5 sec sortformer audio=
cpu= 15 secs
mx= 5.0 secs!

strange:
Nemotron streaming (for 40 secs audio)
cpu = 10.5 secs
mlx= 10.3 secs

@CLAassistant

CLAassistant commented Sep 18, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants