Skip to content

fix(gguf): safely propagate runtime errors for unknown architectures#2106

Open
glaziermag wants to merge 1 commit intoEricLBuehler:masterfrom
glaziermag:fix-gguf-architecture-panic
Open

fix(gguf): safely propagate runtime errors for unknown architectures#2106
glaziermag wants to merge 1 commit intoEricLBuehler:masterfrom
glaziermag:fix-gguf-architecture-panic

Conversation

@glaziermag
Copy link
Copy Markdown
Contributor

Fixes Issue #2098: Handle unknown GGUF Architecture panics

Description

Hi there! This PR adds a small fix for #2098, where the GGUF loader would panic if it encountered an unsupported architecture string (like gemma4), bypassing the normal error handling.

I've swapped the .unwrap() call to map the anyhow::Result into a candle_core::Error::Msg. Hopefully, this will allow the application to gracefully return an error rather than completely crashing the thread.


Local Testing

To make sure this behaves properly natively, I tested it locally using mistralrs-server and a dummy gemma4 GGUF file.

How I tested it natively:

$ RUST_BACKTRACE=1 cargo run -p mistralrs-server -- gguf -m . -f dummy_gemma4.gguf

Before this PR (Panic):

   Compiling mistralrs-server v0.8.1
    Finished `dev` profile [optimized + debuginfo] target(s)
     Running `target/debug/mistralrs-server gguf -m . -f dummy_gemma4.gguf`
thread 'main' panicked at mistralrs-core/src/gguf/content.rs:151:22:
called `Result::unwrap()` on an `Err` value: Unknown GGUF architecture `gemma4`
stack backtrace:
   0: __rustc::rust_begin_unwind
...

After this PR (Graceful Error):

   Compiling mistralrs-server v0.8.1
    Finished `dev` profile [optimized + debuginfo] target(s)
     Running `target/debug/mistralrs-server gguf -m . -f dummy_gemma4.gguf`
 INFO mistralrs_core::pipeline::gguf: Prompt chunk size is 1024.
Error: Unknown GGUF architecture `gemma4`

Verification

I also ran cargo test --workspace to make sure this isolated change doesn't accidentally cause any regressions in the surrounding tensor or pipeline tests. Everything seems to be passing smoothly.

Please let me know if there's a better way to structure this error mapping or if anything else is needed here. Thanks for your time reviewing!

@glaziermag glaziermag marked this pull request as ready for review April 14, 2026 00:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant