Support Multiple AI Providers

[moha-b]

Description

Add support for running an AI model directly on the user’s phone without external API calls.

During setup, the user selects the language they want to learn and downloads the corresponding language model or language pack. After downloading it, they can chat and practise speaking entirely offline.

Expected features

Download a model or language pack for the selected language

Run conversations locally on the device

Work without an API key or internet connection after download

Keep conversations and learning data private

Allow users to delete or replace downloaded models

Show the model’s download size and device compatibility

[TheDevRo]

Came here to suggest this. another option would be for the user to be able to set up their own endpoints to use personal servers or things like "openrouter". I'd be willing to bet you still get a bunch of users on the hosted plan when that is launched, but this gives a more technical user the ability to use your app as well with their own infrastructure

[moha-b]

I like this direction. I think it makes sense to broaden the scope of this issue and split the implementation into separate sub-issues.

I’m thinking of renaming this issue to “Support Multiple AI Providers” and covering three options:

1. On-device models

• Download a language model or language pack
• Run conversations fully offline
• No API key required
• Keep user data private on the device

2. Bring Your Own AI

• Custom API endpoint
• Self-hosted servers
• OpenRouter or similar providers
• User-provided API keys and model configuration

3. Hosted AI

• The project’s own hosted AI service
• Simple setup for users who do not want to manage models, API keys, or infrastructure
• Could later support a hosted/paid plan

I think the parent issue should define the overall provider architecture, and each of these could then become its own sub-issue.

I’d also be happy to volunteer to work on this 😃 , @Chuloo give us your thoughts

[Chuloo]

This is an interesting issue, as a technical user, I understand it, then I think about non-technical users that are still unable to run this locally atp. How do you envision we keep this feature away from regular folks that just want to have conversations on the hosted version and call it a day? Put it in advanced settings? So it's discoverable but doesn't add friction to onboarding?

I'm currently planning how to add Gemini-3.8-flash-live with a toggle probably.

[moha-b]

Yes, I think Advanced Settings is the right place for alternative providers. Since Gemini Live is already being planned, it may be better to introduce a general provider selector rather than a Gemini-specific toggle.

Another possible revenue model is paid on-device language packs. Each language would have a dedicated local model with three learning tiers:

1. Beginner
2. Intermediate
3. Advanced

A purchase could unlock the next level, including an updated model or language pack with more advanced vocabulary, conversations, corrections, and learning goals. Conversations would work offline, user data would remain private, and Mural would avoid recurring inference costs for those users.
We could start with one language as a pilot and later apply the same structure to every supported language. Hosted AI could remain the simple default and a fallback for devices that cannot run local models.

This leaves two monetization options to consider: hosted conversation minutes or paid downloadable language levels—or potentially both. What direction do you think fits Mural best?

[KrOnAsK]

I would also love the opportunity to use my own provider (probably easiest to use the OpenAI compatible endpoints only).
Is there anyone already working on this, otherwise I would try and see if I can add it?

[KrOnAsK]

Following up on my question above: I went ahead and built the bring-your-own-endpoint part, see #125.

It adds a custom endpoint under Settings → Advanced on Android and iOS, next to the existing "use your own API key" option. You enter a base URL, an optional key, the API style (Chat Completions or Responses) and the models, and meanings, lookups, assessments and replies go to that server instead of OpenAI. Compatible servers don't offer the realtime voice API, so voice with a custom endpoint works turn by turn: the app records until you pause, sends the audio to /audio/transcriptions, gets a reply from the chat model and plays it back from /audio/speech. You can't interrupt Mural mid-reply, and web search isn't available.

I've been practising Spanish with it on Android against my university's vLLM gateway (qwen-3.5-397b, whisper-large, kokoro with the ef_dora voice). Two things came out of that: Qwen took about 36 s per reply until thinking was turned off (there's now a switch for it), and Whisper sometimes returned my Spanish as an English translation unless the app sends the learning language. There's also scripts/check_custom_endpoint.py, which runs the app's requests against a server before you set it up in the app.

The Swift core tests pass in CI, but I don't have a Mac, so the iOS app itself hasn't been built or tried on a phone yet.

On-device models and hosted AI aren't touched. @Chuloo, if you'd rather have a general provider selector (e.g. alongside Gemini Live), I'm happy to rework this to fit into that.