Surface + document the loopback Local API for on-device agents (Settings toggle + docs)#715
Conversation
The loopback-only Local API (POST /v1/transcribe, /v1/postprocess) already exists but was enable-only via a hidden "LocalAPIEnabled" default and was undocumented. This surfaces and documents it: - SettingsStore.localAPIEnabled (reuses the existing "LocalAPIEnabled" key) - LocalAPIServer.refresh() to start/stop the server live when toggled - a "Local API (on-device agents)" toggle in Settings (loopback-only, off by default) - README section + docs/local-api.md documenting endpoints & security Lets other apps/agents on the same Mac reuse FluidVoice's transcription instead of loading their own ASR model. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
github-actions
Bot
added
needs PR template
|
The PR Policy check is blocking this PR because required template information is missing. Please update the PR description with:
Visual files detected:
Screenshots or video are required for UI, UX, settings, onboarding, overlay, menu bar, or visual behavior changes. If this PR has no visual changes, check the no-visual-change box in the template. If this remains incomplete for 48 hours after opening, the PR may be closed. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 66dbd621be
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
|
||
| self.optionToggleRow( | ||
| title: "Local API (on-device agents)", | ||
| description: "Expose a loopback-only HTTP API on 127.0.0.1 (default port 47733) so local agents/apps on this Mac can reuse FluidVoice's transcription (POST /v1/transcribe). Off by default; never leaves this machine.", |
There was a problem hiding this comment.
When the configured post-processing provider is OpenAI, Groq, Anthropic, or another remote provider, a caller of the enabled /v1/postprocess endpoint has its text sent externally: InferenceAPIController.postprocess delegates to DictationPostProcessingService.process, which calls LLMClient with the selected provider URL. Therefore the unqualified “never leaves this machine” promise is false for one of the documented endpoints and may cause users to expose agent-supplied text under an incorrect privacy assumption; clarify that only the HTTP listener is local and that post-processing follows the configured provider.
Useful? React with 👍 / 👎.
|
|
||
| FluidVoice can expose a **loopback-only** HTTP API so other apps and AI agents running **on the same Mac** can reuse its transcription and post-processing — instead of bundling and loading their own ASR model. One model stays warm in FluidVoice; agents get FluidVoice-quality transcription with no extra memory footprint. | ||
|
|
||
| **Enable it:** `Settings → Local API (on-device agents)`. It is **off by default** and only ever binds to `127.0.0.1` (non-loopback connections are rejected at the listener). |
There was a problem hiding this comment.
The server currently creates NWListener(using:on:) with only a port and never sets a loopback requiredLocalEndpoint, so it listens on the wildcard local address; the newConnectionHandler merely rejects non-loopback peers after their connections have reached the listener. Consequently the documented guarantee that it “only ever binds to 127.0.0.1” is inaccurate and the port remains externally visible/reachable at the transport layer. Either bind the listener explicitly to loopback or describe the accept-time filtering accurately.
Useful? React with 👍 / 👎.
| func refresh() { | ||
| if LocalAPI.Configuration.current.enabled { | ||
| self.start() | ||
| } else { | ||
| self.stop() | ||
| } |
There was a problem hiding this comment.
If the toggle is switched off and then back on before the cancelled listener's asynchronous state callback reaches the main actor, start() installs a new listener but the old listener's .cancelled callback subsequently executes handleState and clears self.listener. The restarted listener is then orphaned, so a later off toggle cannot cancel it and another on toggle may attempt to open a duplicate port. Associate state callbacks with their originating listener and only clear the property when they still match.
Useful? React with 👍 / 👎.
Greptile SummaryThis PR surfaces the existing Local API for local integrations.
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
…o-cap docs) - LocalAPIServer: ignore state callbacks from a superseded listener (guard self.listener === listener, weak capture) so a rapid off->on toggle can no longer orphan/cancel the newly-installed listener (Codex P2 / Greptile P1). - Qualify the privacy claim: only the HTTP listener and /v1/transcribe are on-device; /v1/postprocess follows the configured provider (may be remote) (toggle text, README, docs). - Describe the accept-time loopback filtering accurately instead of claiming it binds 127.0.0.1. - Document the 300s (5 min) audio-duration cap alongside the 25 MB body cap. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
7d418cb to
0e68a5d
Compare
|
Thanks for the thorough review — all four points addressed in
Rebuilt clean ( |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0e68a5d532
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| } else { | ||
| self.stop() |
There was a problem hiding this comment.
When a connection has passed newConnectionHandler but its Task { @MainActor ... } has not executed yet, this call to stop() cannot see it in activeConnections. The queued task then creates and starts the handler without checking the enabled setting or its originating listener, allowing that connection to continue serving requests after the user switches the API off. Associate connection callbacks with the listener generation and cancel them when that listener is no longer current.
Useful? React with 👍 / 👎.
…ed section The toggle sat next to 'Copy to Clipboard', which lives inside the `if accessibilityEnabled` block, so it was hidden until the user granted Accessibility — unrelated to a loopback HTTP API. Moved it to the always-visible 'App Settings' section, and split the long description into description + footnote so it no longer truncates on one line. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Follow-up in |
settingsToggleRow renders description and footnote on single truncating lines; the previous footnote overflowed. Trimmed it to the essentials — the full contract (non-loopback filtering, limits, post-processing locality) lives in README + docs/local-api.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Adding a screenshot of the Toggle 
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e88f57419b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| You can also toggle it via `defaults`: | ||
|
|
||
| ```bash | ||
| defaults write com.FluidApp.app LocalAPIEnabled -bool true |
There was a problem hiding this comment.
When FluidVoice is already running, this command only changes the preference: LocalAPIServer.refresh() is invoked exclusively by the Settings toggle, and the lifecycle calls start() only at launch. Consequently, enabling via defaults does not start the listener, and disabling via defaults leaves the API serving until the app is relaunched. Either observe these preference changes or tell command-line users to restart FluidVoice.
Useful? React with 👍 / 👎.
| - Limits: request body **25 MB**; transcription audio **300 s (5 min)** per request (longer audio is | ||
| rejected). These caps are independent — compressed audio can satisfy the byte limit yet still hit the | ||
| duration limit. |
There was a problem hiding this comment.
For the documented audioBase64 input (and raw audio bodies), decodeAudioSamples reaches LocalAPIAudioDecoder.samples(from:), which reads min(file.length, maxFrames) and silently transcribes only the first five minutes. Only the path-based flow calls validateDurationWithinLimit and rejects overlong files. Clients relying on this statement can therefore receive a successful but incomplete transcript; either validate inline inputs too or document that they are truncated.
Useful? React with 👍 / 👎.
1 participant
Description
FluidVoice already ships a loopback-only Local API (
LocalAPIServer:POST /v1/transcribe,POST /v1/postprocess,GET /v1/health, plus history/dictionary), but it can only be turned on via the hiddenLocalAPIEnableduser default and it is undocumented. This PR surfaces and documents it so other apps and AI agents running on the same Mac can reuse FluidVoice's transcription instead of bundling and loading their own ASR model.SettingsStore.localAPIEnabled— accessor reusing the existing"LocalAPIEnabled"key (no change to how the server reads config).LocalAPIServer.refresh()— start/stop the listener live when the setting changes (no app restart).docs/local-api.md(endpoints, request/response, security, limits, examples).Type of Change
Related Issue or Discussion
None — opened directly. Motivation: reusing FluidVoice's on-device transcription from a local AI agent rather than running a second copy of the model. Happy to convert to a discussion first if you prefer.
Testing
xcodebuild -project Fluid.xcodeproj -scheme Fluid -configuration Debug build→ BUILD SUCCEEDED (before and after the review fixes).127.0.0.1:47733;GET /v1/health→200.POST /v1/transcribe {"path": ".../clip.ogg"}→ correct transcript, reusing the already-warm model (RSS unchanged — no second model process).refresh()path, including rapid off→on).Screenshots / Video
Adds a single Settings row built with the existing
optionToggleRowcomponent — visually identical to the neighboring "Copy to Clipboard" row, in the same section.(Settings → the row reads "Local API (on-device agents)".)
Review feedback addressed (Codex / Greptile)
stateUpdateHandlernow ignores callbacks from a superseded listener (guardsself.listener === listenerand captures the listener weakly), so a rapid off→on toggle can no longer orphan or cancel the newly-installed listener./v1/transcribeare on-device;/v1/postprocessruns the configured post-processing provider, which may be remote. Fixed the toggle text, README and docs.