Add parakeet_capi_transcribe_pcm_logits: expose CTC log-probs - #57
Merged
Conversation
The C-API only ever returned decoded text/timestamps/JSON, even though CTCDecoder::forward already computes a log-softmaxed [T, vocab+1] matrix internally. External decoder stacks that do their own LM fusion (e.g. pyctcdecode + KenLM + hotwords) need that matrix directly instead of this library's own greedy/beam decode. Adds Model::transcribe_pcm_ctc_logits (mirrors transcribe_pcm through mel + encoder, including the long-audio tiling path, then runs the CTC head only, skipping decode) and the matching C-API entry point + parakeet_capi_free_logits, following the existing validate/try-catch/ last_error conventions used by every other capi_* function. Bumps the ABI version to 6. Verified against the real parakeet-ctc-1.1b checkpoint (tests/test_capi_ctc_logits.cpp, gated on PARAKEET_TEST_GGUF_CTC): argmax-greedy decode over the exposed matrix reproduces transcribe_pcm(..., kCTC)'s own greedy text byte-for-byte, and matches the fixture's known NeMo reference.
Verifies the C-API boundary contract directly: a model with no CTC head (a pure RNNT/TDT streaming model) makes parakeet_capi_transcribe_pcm_logits fail cleanly (nonzero return, *out_logits left NULL, last_error set) rather than letting ctc_head_tensor's std::runtime_error cross the C boundary. Gated on PARAKEET_TEST_GGUF_NO_CTC, independent of the existing self-consistency block.
- parakeet_capi_transcribe_pcm_logits: check ctx before the output pointers (so an invalid out-param can still set ctx->last_error), and zero *out_T/*out_vocab_plus_1 alongside *out_logits up front so every failure path leaves defined state, matching the header's documented contract. - tests/test_capi_ctc_logits.cpp block 1: call the actual parakeet_capi_load / parakeet_capi_transcribe_pcm_logits / parakeet_capi_free_logits C-API instead of Model::transcribe_pcm_ctc_logits directly, so the test exercises the C boundary and malloc/free contract on the success path too (pk::Model is now used only for the reference text and tokenizer/blank_id).
- tests/test_capi_ctc_logits.cpp block 1: cache blank_id/tokenizer_pieces and release the pk::Model (model.reset()) before loading a second full model via parakeet_capi_load — the two were resident simultaneously, nearly doubling peak RAM for a 1.1B checkpoint and risking CI OOM. - src/model.cpp: transcribe_16k_ctc_logits's prompt-conditioning comment claimed CTC-only models are never prompt-conditioned "in practice", which overclaims a current-catalog observation as if it were a property the function depends on. Reworded to match transcribe_16k's existing, accurate wording (no-op when prompt.present == false, full stop).
localai-org-maint-bot
approved these changes
Jul 29, 2026
localai-org-maint-bot
left a comment
Collaborator
There was a problem hiding this comment.
@mudler This is good to merge from my review. The new C entry point preserves the existing mel/encoder/prompt-conditioning path, returns the CTCDecoder post-log-softmax [T, vocab+1] layout with a symmetric free function, and keeps failures inside the C boundary with cleared output ownership. The focused test covers C-API allocation/free self-consistency and the no-CTC error path. Source inspection and diff hygiene pass; I could not execute the local CPU build because this host currently has no cmake binary.
mudler
approved these changes
Jul 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The C-API currently only ever returns decoded text/timestamps/JSON, even
though
CTCDecoder::forwardalready computes a log-softmaxed[T, vocab+1]matrix internally (
src/ctc_decoder.cpp, viaggml_soft_max->ggml_log).Anyone doing external LM fusion on top of this library's CTC output (e.g.
pyctcdecode + a KenLM n-gram LM + hotword boosting) currently has no way to
get that matrix — only the greedy/beam text this library's own decode
produces.
This PR adds one new C-API entry point,
parakeet_capi_transcribe_pcm_logits,that runs mel + encoder + the CTC head on a PCM buffer and returns the
log-prob matrix (row-major
[T, vocab+1], already log-softmaxed) instead ofdecoded text, plus
parakeet_capi_free_logitsto release it. ABI bumped tov6; all existing entry points are unchanged.
What's included
Model::transcribe_pcm_ctc_logits/Model::transcribe_16k_ctc_logitsinsrc/model.hpp/src/model.cpp: mirrorstranscribe_pcm/transcribe_16kthrough mel + encoder (including the existing long-audio tiling path — no
new truncation limit), then runs only the CTC head, skipping greedy/beam
decode. Always forces the CTC head regardless of the model's preferred
decoder, and throws
std::runtime_error(via the existingctc_head_tensorlookup) if the model has no CTC head at all — e.g. apure TDT/RNNT-only streaming model.
parakeet_capi_transcribe_pcm_logits/parakeet_capi_free_logitsininclude/parakeet_capi.h/src/parakeet_capi.cpp, following the samevalidate -> try/catch ->
last_errorconventions as every othercapi_*function (no C++ exception ever crosses the C boundary; a NULL
ctxorany NULL out-param returns nonzero without writing through any pointer,
otherwise all outputs are zeroed up front so every failure path leaves
defined state).
tests/test_capi_ctc_logits.cpp(registered intests/CMakeLists.txt,LABELS "model"), two independent blocks:(
PARAKEET_TEST_GGUF_CTC), through the actual C-API (not just theunderlying C++ method): argmax-greedy decode over the exposed matrix
(via the same
ctc_greedy+detokenizepathdecode_enc_outusesinternally) reproduces
transcribe_pcm(..., kCTC)'s own greedytranscript byte-for-byte. Verified against the real
parakeet-ctc-1.1bq8_0 checkpoint ontests/fixtures/speech.wav.Only
blank_id/tokenizer_piecesare cached from the referencepk::Modelbefore it's released, so a second full model isn't keptresident alongside the C-API's own load.
(
PARAKEET_TEST_GGUF_NO_CTC): a model with no CTC head (verifiedagainst the 120M realtime-EOU streaming model) fails cleanly —
nonzero return,
*out_logitsleft NULL,last_errorset — never acrash.
Motivation / context
We're building a second
parakeet.cpp-backed compute backend for our ownCTC decode stack (pyctcdecode + KenLM + hotwords) that already runs against
another CTC engine, and this is the one piece of surface area this library
was missing to slot into it. Filing upstream since any consumer doing
external LM fusion on this library's CTC output hits the same gap.
This has already been through two rounds of review on our own fork
(happy-prime-inc#1) with fixes applied for validation
ordering, output-zeroing on error paths, test peak-RAM, and a couple of
comment-accuracy nits — landing here in what should be a reviewable state,
not a first draft.
Happy to adjust the entry point's shape (out-param ordering, naming, etc.)
if you'd prefer something different from what's here.