Skip to content

Add local Cohere Vulkan helper for Voxtype - #617

Draft
ryanrhughes wants to merge 2 commits into
masterfrom
voxtype-cohere-vulkan
Draft

ryanrhughes wants to merge 2 commits into
masterfrom
voxtype-cohere-vulkan

Conversation

@ryanrhughes

@ryanrhughes ryanrhughes commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

Voxtype's packaged Cohere backend uses ONNX and does not offer Vulkan. Add a small persistent transcribe-cpp helper that lets the existing Voxtype API client transcribe locally on Vulkan without a Voxtype fork or conflicting ggml libraries in one process.

The package includes reviewed Rust source, a locked dependency graph, pinned Vulkan/SPIR-V headers, and a synthetic speech probe. The portable x86_64 build rejects silent CPU fallback and software Vulkan renderers. It binds only loopback and logs timings without logging speech. Model weights are downloaded and verified by the paired Omarchy installer, not bundled here.

Cohere recordings are partitioned at quiet boundaries into segments of at most 35 seconds. This fixes the missing middle passages observed on roughly one-minute inputs. On nine excerpts totaling 302 seconds, three runs each, caption WER dropped from 18.30% to 1.86%; the three long clips took median 1.65–1.81 seconds on Intel Arc B390. Caption agreement is an approximate measure, not a general accuracy claim.

Validation:

  • Local makepkg --cleanbuild --nodeps build, including Rust segmentation tests; runtime shared libraries resolved.
  • Packaged binary transcribed the speech probe correctly on Intel Arc B390 using Vulkan.
  • cargo fmt --check, release Clippy with warnings denied, HTTP validation cases, and the actual Voxtype 1.0.1 client passed on the source build.
  • Repeated long-audio benchmark verified recovered passages and bounded segmentation.

Draft companion to omacom/omarchy#13098. This recipe is edge-only; review and package availability are required before rollout. No repository publication or release has been performed. Other GPU vendors and architectures still need validation.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant