Jarvis

Jarvis is a personal desktop meeting assistant adapted from an upstream desktop assistant codebase for private use.

The current goal is narrow: help me understand software engineering meetings faster by combining system audio transcription, screen/screenshot context, and concise AI suggestions. This repository is not a commercial product, has no subscription flow, and does not ship a hosted Jarvis API.

Current Milestone

• Tauri + React desktop shell is running.
• Screen-share-safe overlay behavior is inherited from the original app foundation.
• Meeting assistant modules have been added for transcript context, screen observations, prompt construction, and advisor suggestions.
• The meeting overlay opens in the desktop app, supports explicit Text+Screen privacy mode, and can generate screen-context suggestions from the meeting hotkey.
• Explicit screen captures now produce screen-anchored technical answers with Question, Answer, Approach, Code, Complexity, and Clarifying question sections.
• Debug Mode provides local screen/voice workflow traces, raw text model/STT I/O and capture metadata. Debug Mode alone no longer prints trace lines to the terminal: they are Debug and Trace entries of the diagnostic log.
• Meeting screen-context captures are downscaled and JPEG-encoded before model submission to keep payloads small enough for live use.
• Initial personal testing shows the screen-anchored flow is more useful than the earlier generic screen explanation path.

Personal-Use Scope

Jarvis is currently designed for local personal use only.

• Bring your own AI and speech-to-text providers through the existing custom curl provider flow.
• No bundled paid license system.
• No usage analytics or telemetry.
• No hosted update endpoint.
• No public download, marketing, referral, bounty, or support flow.

Architecture Notes

Start with architecture/README.md for the single tracked, privacy-safe runtime map and refactor-guardrail contract. In short, the app uses:

• src/ for the React frontend, hooks, provider configuration, and assistant UI.
• src-tauri/ for the native shell, global shortcuts, window behavior, capture, audio capture, and local database.
• src/lib/meeting/ for deterministic meeting policy, authority, and artifact contracts.
• src/lib/preparation/ for Interview Preparation Workspace composition and snapshot services.
• architecture/ for deletion governance, architecture drift baselines, and canonical replay evidence.

Detailed product working documents remain local and gitignored because they can contain private interview and memory context. A clean checkout does not depend on them for setup or verification.

Development

Install dependencies:

npm install

Run the browser preview:

npm run dev

Run the desktop app:

npm run tauri dev

If the default development port 1420 is already in use, set one alternate port for both Vite and Tauri:

JARVIS_DEV_PORT=1422 npm run tauri dev

The same variable works for the browser preview with JARVIS_DEV_PORT=1422 npm run dev. Invalid or out-of-range values fail before the development processes start.

Build the frontend:

npm run build

Check the Rust side:

cd src-tauri
cargo check

Run the fast architecture guardrails:

npm run verify:architecture

Run the full next-major verification suite (architecture, tests, production build, and Rust check):

npm run verify:next-major

Configuration

Jarvis expects you to configure local/custom providers before relying on meeting transcription or AI responses.

• AI providers are configured from curl commands with variables such as {{TEXT}}, {{SYSTEM_PROMPT}}, and optionally {{IMAGE}}.
• Speech providers are configured from curl commands that accept uploaded audio.
• System prompts and assistant behavior can be edited in the dashboard.

Azure MAI-Transcribe-2

Open Audio Settings, select Azure MAI-Transcribe-2 under Transcription Provider and enter the Speech resource endpoint (https://<resource>.cognitiveservices.azure.com) and its API key. A hostname without https:// is also accepted. Project and Azure OpenAI endpoints are not Speech endpoints. Selecting the provider is explicit; existing settings are not automatically migrated.

The built-in template uses the non-streaming Speech REST API with automatic language detection and verbatim output. Existing speech-bias terms are sent as phraseList.phrases; free-form prompts and continuation text are not supported by this model and are not sent. Audio format and cancellation remain on the shared STT path. Custom multipart templates can name the audio part using {{AUDIO}}, and {{STT_TERMS_JSON}} supplies the runtime terms as a JSON array. See Microsoft's MAI-Transcribe documentation for region, format, billing, and preview limitations.

Meeting Assistant Usage

• Open the floating Brain button to show the Meeting Assistant panel.
• Use the panel's Privacy selector to switch between Text and Text+Screen.
• Cmd+Shift+E captures the frontmost active window for meeting context, briefly hides the panel before capture, then reopens the panel with the generated suggestion.
• Screen capture is treated as the primary technical question source; recent meeting audio is used as clarification or follow-up context.
• The Meeting Assistant panel uses a fixed expanded width and wraps long answers vertically so responses do not push the window wider during a meeting.
• While the Meeting Assistant panel is open, Jarvis uses the native system cursor instead of its hidden/custom cursor mode to avoid cursor loss when moving across the transparent overlay area.
• Use Clear task when a screen question is no longer relevant. Screen tasks expire after the configured task memory window, which defaults to 30 minutes.
• Cmd+\ is the emergency hide/show shortcut. Hiding collapses the Meeting Assistant UI without stopping meeting audio capture.
• Enable Debug Mode in the Meeting Assistant panel to show recent screen and voice trace details and raw text model/STT I/O. Debug Mode defaults off; raw audio bytes and screenshots are not stored in traces by default.
• Log Level (Error, Warn, Info, Debug or Trace; default Info) sets the threshold of the diagnostic log: entries at that level and every more severe level go to the terminal and to local log files, which keep at most 50 MiB or 14 days. Debug Mode alone no longer prints trace lines to the terminal: trace and step changes are Debug and Trace entries of the diagnostic log, with names, statuses, durations and sizes and without trace metadata values. Log Level does not control Preparation output, the focus window and meeting window prints, shutdown messages, other console output, native prints, Session Recording or Native Stall Diagnostics files.
• Debug Mode also shows a compact Last capture row with a preview thumbnail, captured app/window, capture method, image size, bounds, and top window candidates.
• Screen-context debug traces include image media type, base64 payload size, capture sub-step timings, first-token timing, and model completion timing.
• When Screenshot settings are in Auto mode, Meeting Assistant screen analysis treats the configured screenshot auto prompt as user preference while preserving the screen-task answer format.
• Clarifying questions include quick Yes, No, Not sure, and Dismiss controls so you can steer follow-up suggestions without typing during a meeting.
• Cmd+Shift+S remains the separate manual screenshot/completion shortcut.

Privacy

Jarvis should only send data to providers you configure yourself. Be careful with meeting audio, screenshots, private code, and confidential company material. Review your provider terms and workplace policies before using it in real meetings.

Origin

This is a private fork of an upstream desktop assistant project. The fork is being reshaped into Jarvis for personal workflow support, not for redistribution or commercial use.