JARVIS — Personal Knowledge Interface

A local-first dashboard that turns selected Notion content into an interactive knowledge graph and combines it with grounded answers, focused tech news, weather, and private on-device voice input.

Why JARVIS

JARVIS is designed to reduce information friction without creating another system to maintain. Notion remains the source of truth. The application reads only explicitly shared content, derives relationships locally, and brings the most useful context into one calm interface. No paid AI API is required for its current capabilities.

Features

• animated, pointer-controlled neural core rendered with Canvas 2D
• concept map of the technical terms inside your selected Notion databases, not of page titles
• concrete database picker with stable Notion database IDs, parent context, icons, and persistent selection
• complete crawl of nested subpages, toggles, columns, lists, and database rows
• local SQLite index with full-text search, embeddings, and incremental synchronisation
• evidenced relations from Notion properties, page mentions, shared tags, co-occurrence, and optional semantic similarity
• every concept links to all of its sources, section paths, snippets, and Notion blocks
• hybrid retrieval over indexed passages with BM25, embeddings, and reciprocal rank fusion
• persistent force-directed graph with pan, zoom, focus, node dragging, and progressive snapshots, drawn in the same Canvas visual language as the core
• retrieval-augmented local Q&A with clickable citations, claim verification, and explicit uncertainty
• session-scoped follow-up context for natural multi-turn questions
• natural German answers generated by Qwen 3.5 4B through Ollama
• deterministic extractive fallback when the local model is unavailable
• relevance-ranked daily tech briefing from multiple public sources with a strict 72-hour freshness window
• daily five-term technology vocabulary carousel with professional and everyday examples
• explicit one-click saving of individual vocabulary terms to a configured Notion table
• local story bookmarks, relevance feedback, and daily browser cache
• current weather and four-day forecast from Open-Meteo
• manual-stop audio recording and local Whisper transcription
• spoken assistant with local function calling for dashboard data, Spotify, Google Calendar, Gmail that can read, summarize, archive, and trash but never send, Chrome searches, and allowlisted macOS actions
• spoken answers through the macOS system voices with configurable voice, speed, and volume, interruptible at any time
• explicit confirmation before irreversible actions, enforced again in the local action layer
• native microphone capture in the app, because the embedded WebView withholds browser recording on loopback
• native Apple Silicon app bundle powered by Tauri 2
• menu bar controls, launch at login, native notifications, and a global ⌘⇧J shortcut
• focused small-screen layout that prioritizes the core and knowledge map
• read-only Notion access with server-only credentials

Architecture

The application follows a feature-oriented structure. UI, domain types, server integrations, and pure algorithms are separated so each module has one clear responsibility.

app/                         Next.js routes and API boundaries
config/                      build integration
features/
  ai/                        local Ollama client, API bridge, and answer contracts
  assistant/                 voice-assistant tool catalogue, tool-calling loop, and speech output
  briefing/                  news aggregation, ranking, and types
  glossary/                  curated daily vocabulary rotation and explicit Notion export
  interface/                 components, hooks, map viewport and force layout, shared Canvas renderers, and styles
  knowledge/                 concept contracts, chunking, relation math, retrieval, and grounded answers
  speech/                    local whisper.cpp status and transcription bridge
  weather/                   Open-Meteo client and types
tests/
  unit/                      deterministic domain tests
  integration/               built-worker and security-boundary tests
scripts/                     local development and speech-model setup
desktop/
  actions/                   allowlisted macOS commands, Spotify and Google OAuth, and the local action API
  indexer/                   local SQLite knowledge index, Notion crawl, and concept extraction
  server/                    packaged loopback server adapter
src-tauri/                   native macOS shell, tray, shortcut, and app lifecycle
worker/                      Cloudflare/vinext runtime entry point

See Architecture for the data flows and design decisions.

Getting started

Requirements

• macOS on Apple Silicon is the primary supported local environment
• Node.js 22.13 or newer
• npm
• Homebrew
• Ollama for natural local answers
• whisper-cpp and ffmpeg for high-quality local speech-to-text
• an optional Notion internal integration for real workspace data

Installation

git clone git@github.com:Sissighn/jarvis-knowledge-interface.git
cd jarvis-knowledge-interface
npm install
cp .env.local.example .env.local
ollama pull qwen3.5:4b
brew install whisper-cpp ffmpeg
npm run setup:speech
npm run dev

Open the local URL printed by the development server. Without Notion credentials, JARVIS shows an honest setup state and does not invent sample knowledge.

Ollama runs the answer model on the same Mac, so no paid model API or API key is needed. Without Ollama, search and the extractive fallback remain available.

npm run dev starts the web interface, the local knowledge indexer, and the local Whisper service. Voice recordings continue until the user presses Stop. JARVIS then transcribes the complete recording locally, places the editable text in the command box, and sends nothing until Enter or the send button is used.

The downloaded model files use approximately 4 GB in total: about 3.4 GB for Qwen and 574 MB for Whisper, excluding the runtimes and package dependencies. Both models are excluded from Git.

Native macOS app

The Tauri app packages the existing server-rendered interface behind a loopback-only sidecar. Secrets and model excerpts stay behind the same server boundary as in local web development; native code owns the application lifecycle and macOS integrations.

Install the Rust toolchain once, then build the Apple Silicon app:

brew install rustup
$(brew --prefix rustup)/bin/rustup default stable
npm run desktop:build

The build creates:

src-tauri/target.noindex/release/bundle/macos/JARVIS.app
src-tauri/target.noindex/release/bundle/dmg/JARVIS_0.1.0_aarch64.dmg

For normal daily use, open the installed JARVIS.app from Spotlight, Launchpad, or the Applications folder. No terminal or npm command is required. A packaged app is a fixed snapshot: source changes are not copied into the installed app automatically.

Use this workflow while developing:

1. Quit the installed JARVIS app so it does not occupy the local desktop port.
2. Run npm run desktop:dev for a native development window with live frontend updates.
3. Edit and test the application.
4. Run npm run check before packaging a new version.
5. Run npm run desktop:build, then replace the older app in Applications with the newly generated JARVIS.app or install it from the DMG.

JARVIS does not currently include an automatic updater. That is intentional for the local development phase; signed update delivery can be added once versioning, Developer ID signing, and release publishing are established.

Local builds are ad-hoc signed and pass macOS code-signature verification. Public distribution to other Macs requires an Apple Developer ID certificate and notarization; those private credentials are intentionally not part of the repository.

desktop:build securely copies .env.local with owner-only permissions and prepares the ignored Whisper model under ~/Library/Application Support/com.sissighn.jarvis/. It does not embed credentials or personal Notion content in the app or DMG.

Native behavior:

• ⌘⇧J shows or hides JARVIS from anywhere on the Mac
• closing the window keeps JARVIS available from the menu bar
• the menu bar can open or fully quit the app and toggle Beim Anmelden öffnen
• release builds enable launch at login on first start
• macOS asks for notification permission before the first Morning Brief notification
• the packaged server binds only to 127.0.0.1; Ollama and Whisper remain local services

Use npm run desktop:dev while developing the native shell. The first native build compiles Rust dependencies and therefore takes longer than later builds.

Configure the local model

JARVIS defaults to Qwen 3.5 4B, a compact model suitable for the local setup. Keep Ollama running and configure another installed model only if needed:

OLLAMA_BASE_URL=http://127.0.0.1:11434
OLLAMA_MODEL=qwen3.5:4b

Concept extraction uses the same chat model. Embeddings use embeddinggemma, a roughly 622 MB model built for local devices (Ollama); the setup dialog shows its status and installs exactly that model on request.

OLLAMA_EMBEDDING_MODEL=embeddinggemma

For each question, the local index returns at most eight passages from at most five sources and sends only those to the Ollama process on the same device. Up to four recent turns help resolve references in follow-up questions, but previous answers are explicitly excluded as factual evidence. Every generated claim requires an inline citation and is checked again against the cited passage before it is displayed. If Ollama is offline, crawling and chunking still run and the AI steps continue at the next sync.

The local knowledge index

The index lives on this Mac only:

Environment	Location
Desktop app	~/Library/Application Support/com.sissighn.jarvis/knowledge-index.sqlite3
Development	.jarvis-dev/knowledge-index.sqlite3 (git-ignored)

npm run dev starts the indexer on 127.0.0.1:4318 next to the web interface; the desktop app answers /api/knowledge/* inside its own Node server. Hosted builds have no index and answer desktop_required.

First start after the update:

1. Open NOTION SETUP and choose the concrete databases JARVIS may index.
2. Save the selection. The current graph remains available while changed pages are read.
3. Concepts and evidenced relations appear after each completed database; no chat-model batch runs during indexing.
4. Optionally install the embedding model to add semantic retrieval. BM25 search and the concept map work without it.

Later starts synchronise automatically and read only pages whose last_edited_time changed. INDEX LÖSCHEN UND NEU AUFBAUEN in the setup dialog deletes the local index after a confirmation click and starts a complete rebuild. Deselecting a database removes its content from the map immediately and from the index after the next successful sync.

Configure local speech-to-text

JARVIS records audio with the browser MediaRecorder API and sends the finished in-memory recording only to the local whisper.cpp service. The default model is the multilingual large-v3-turbo-q5_0 variant, optimized for a practical balance of accuracy, speed, and memory usage on Apple Silicon.

WHISPER_BASE_URL=http://127.0.0.1:8178
WHISPER_PORT=8178
WHISPER_MODEL_PATH=models/whisper/ggml-large-v3-turbo-q5_0.bin
WHISPER_LANGUAGE=de

Recording works differently in the two runtimes, and the assistant panel shows which one is active. In the browser, MediaRecorder captures the audio. Inside the packaged app that path does not exist: WKWebView withholds navigator.mediaDevices from the loopback origin, because it only exposes microphone access in a secure context. The app therefore records natively in Rust, resamples to 16 kHz mono, and hands a finished WAV file to the same local transcription route. macOS asks once for microphone permission; the answer can be changed later under System Settings → Privacy & Security → Microphone.

The service runs with --convert, so whisper.cpp normalises every upload through ffmpeg. That scratch file is written relative to the working directory, which is why the app starts the service inside speech-scratch in its application-support directory: a bundled app inherits / from LaunchServices, and a failed scratch write surfaces as FFmpeg conversion failed. for every recording.

WHISPER_PORT controls the local service and must match the port in WHISPER_BASE_URL. The language defaults to German while retaining recognition of English technical terms through a domain-specific prompt. Set WHISPER_LANGUAGE=auto if the complete recording may use another primary language.

Voice workflow:

1. Press the microphone button to begin recording.
2. Speak for as long as needed; silence does not submit or intentionally stop the session.
3. Press Stop to end the recording and start local transcription.
4. The transcript goes to the voice assistant, which answers out loud (see below).

The text field next to the microphone remains the entry point for Notion knowledge questions. If transcription fails, the error remains visible and nothing is sent to a model.

Voice assistant

The microphone drives a spoken assistant: Whisper transcribes, a local tool-calling model decides which of the allowlisted functions to run, and the answer is spoken back through the macOS system voices. Everything runs on this Mac and costs nothing per request.

microphone → whisper.cpp → Ollama (tool calling) → dashboard data, Spotify, Google, macOS → spoken answer

What it can do

Area	Functions
Dashboard	current temperature, rain forecast, the five most important tech news, words of the day, personal dashboard summary
Spotify	search and play a track, artist, album, or playlist, pause, resume, next, previous, volume, name the current track
Google Calendar	read the agenda for today, tomorrow, or the next seven days, and create a new appointment after confirmation
Gmail	say how many unread mails are in the inbox, name sender and subject of the newest ones, search by term or by a single day, summarize one mail, archive mails, move mails to the trash — never send
Google Chrome	search for a term in Chrome, open a web address in Chrome
macOS	open an allowlisted program, open a file or folder inside your home directory, set or change the system volume, empty the trash after confirmation

Ask in plain German, for example „Wie warm ist es gerade?“, „Spiel Bohemian Rhapsody auf Spotify“, „Was steht morgen in meinem Kalender?“, „Trag am Freitag um 15 Uhr Zahnarzt ein“, „Habe ich neue Mails?“, „Habe ich am 5. August Mails bekommen?“, „Was steht in der Mail von der Bank?“, „Archivier die Newsletter“, „Such auf Chrome nach dem neuen MacBook“ or „Fass mir mein Dashboard zusammen“.

Mail is never sent

JARVIS can look into the mailbox and never into the outbox. Archiving and trashing need Google's gmail.modify scope — nothing narrower exists for either — and that scope would technically also permit sending. The boundary therefore does not rest on the scope alone:

• One door. Every Gmail request in desktop/actions/gmail.ts goes through gmailCall, which checks the path against a list of five allowed endpoints before a request is built. messages.send, drafts, untrash, settings, and DELETE are unreachable; a wrong call throws instead of reaching Google.
• No such function exists. The catalogue in features/assistant/tools.ts has no tool for writing, answering, or forwarding, and no tool may even be named after it — tests/unit/assistant-tools.test.ts fails if one appears.
• Nothing permanent. Trashing is messages.trash, recoverable for 30 days. Permanent deletion needs full mail.google.com access, which the login refuses to request.
• Nothing silent. Archiving and trashing stop the loop and ask first, naming exactly which mails are affected, and the action layer rejects the call again without confirmed: true. At most ten messages are touched by one command.

Creating an appointment writes only into the calendar: events are created without attendees and with sendUpdates=none, so nobody receives an invitation.

Summarizing a mail is the only place where text written by someone else enters the conversation. It is trimmed to 1500 characters, stripped of markup, and handed to the model as data; the system prompt forbids following instructions found in a tool result. The confirmation gate is what makes that safe in practice — a mail cannot talk you into clicking JA, FORTFAHREN.

Speech output

Answers use the macOS system voices through the browser speech synthesis API, so no additional service or download is needed. Voice, speed, and volume are configurable in the assistant panel and stay in browser storage. Pressing the microphone while JARVIS is talking interrupts it immediately and starts a new recording; UNTERBRECHEN only stops the speech.

Voice quality is worth two minutes of setup. A fresh macOS install ships only the compact German voice, which sounds like a decade-old navigation system. Download a Premium voice under System Settings → Accessibility → Spoken Content → System Voice → Manage Voices → German — Petra, Markus, and Viktor are the natural-sounding ones. The assistant picks the best installed German voice automatically, preferring premium over enhanced over compact, so a downloaded voice is used after the next restart without any configuration. Siri's own voices stay unavailable: Apple does not expose them to third-party apps through either speech synthesis or say.

Safety rules

• reading data and controlling media run without a prompt
• irreversible actions always ask first, for example „Der Papierkorb wird endgültig geleert. Fortfahren?“, and only run after a click or a spoken ja
• writing into the calendar asks the same way and names the appointment first, for example „Ich trage „Zahnarzt“ am 14.03.2026 um 15:00 Uhr in deinen Google-Kalender ein. Fortfahren?“
• an appointment is only created with a real date and time; a missing hour is asked for instead of guessed
• only web addresses with http or https are opened in Chrome, so no file: or javascript: address can be reached by voice
• the action layer independently rejects an unconfirmed irreversible action, regardless of what the model asked for
• only the functions in features/assistant/tools.ts exist; the model cannot invent new ones
• programs come from an allowlist, files stay inside your home directory, and symlinks that would leave it are rejected
• system commands run through execFile with argument lists, so no spoken text reaches a shell
• the action layer listens on loopback only and refuses requests carrying a foreign Origin

Model

Tool calling uses its own model so grounded knowledge answers stay untouched. The model must support tools in Ollama.

OLLAMA_ASSISTANT_MODEL=qwen3.5:4b

Without this variable the assistant falls back to OLLAMA_MODEL.

Connect Spotify

Playback control requires Spotify Premium. Login uses Authorization Code with PKCE, so no client secret is stored anywhere.

1. Open the Spotify Developer Dashboard and sign in.
2. Choose Create app. Any name and description work, for example JARVIS Local.
3. Add exactly http://127.0.0.1:4319/callback as Redirect URI. Spotify requires the explicit loopback IP; localhost is rejected.
4. Select Web API as the intended API and save.
5. Copy the Client ID from the app settings into .env.local:

SPOTIFY_CLIENT_ID=your_spotify_client_id
SPOTIFY_REDIRECT_PORT=4319

6. Restart npm run dev, open the assistant panel, and press VERBINDEN. The login opens in your default browser; JARVIS waits on the loopback port and stores the tokens in spotify-auth.json next to the knowledge index, readable only by your user account (chmod 600).

If no Spotify device is awake, JARVIS opens the Spotify app and transfers playback to it before starting. TRENNEN deletes the stored tokens.

Connect Google

Calendar and mail share one login. It uses Authorization Code with PKCE on a loopback port, exactly like Spotify.

1. Open the Google Cloud console, create a project, and open APIs & Services.
2. Under Enabled APIs & services → + ENABLE APIS AND SERVICES, enable the Google Calendar API and the Gmail API.
3. Configure the OAuth consent screen as External, and add your own Google address under Audience → Test users. A personal app stays in testing mode; the consent screen therefore shows an unverified app warning that you confirm through Advanced → Go to JARVIS.
4. Under Credentials → + CREATE CREDENTIALS → OAuth client ID, choose the type Desktop app. Google allows loopback redirects for that type, so no redirect URI has to be registered.
5. Copy the client ID and the client secret into .env.local. For a desktop client the secret is not a secret in the usual sense; it stays in the git-ignored file on this Mac.

GOOGLE_CLIENT_ID=your_google_client_id
GOOGLE_CLIENT_SECRET=your_google_client_secret
GOOGLE_REDIRECT_PORT=4320
JARVIS_TIME_ZONE=Europe/Berlin

6. Restart npm run dev, open the assistant panel, and press VERBINDEN next to GOOGLE. The consent screen opens in your default browser and asks for exactly two permissions: calendar events, and reading mail. JARVIS waits on the loopback port and stores the tokens in google-auth.json next to the knowledge index with owner-only permissions (chmod 600).

TRENNEN deletes the stored tokens and revokes them at Google. Access can also be withdrawn any time under third-party access in your Google account.

macOS permissions

Volume control and emptying the trash use AppleScript. macOS asks once for permission to control Finder and System Events; without it the assistant reports the missing permission instead of failing silently. Grant it under System Settings → Privacy & Security → Automation.

Additional programs can be allowed by voice with a comma-separated list:

JARVIS_ALLOWED_APPS=Figma,Slack

Not yet included: the Jarvis wake word and Notion knowledge as a tool. Knowledge questions stay in the text field for now.

Connect Notion safely

1. Create an internal Notion integration with read-content access. Add update content only if you want to save glossary terms from JARVIS.
2. Share only the pages or data sources JARVIS may read with that integration.
3. Add the token to the ignored .env.local file.
4. Restart the development server.
5. Choose the concrete Notion databases in NOTION SETUP. JARVIS indexes only rows and content from that persisted selection, regardless of how old a page is.

NOTION_ACCESS_TOKEN=secret_your_internal_notion_token
NOTION_GLOSSARY_PAGE_ID=your_notion_glossary_page_id
NOTION_GLOSSARY_TABLE_BLOCK_ID=your_notion_table_block_id

The token is consumed only by server routes. It is never embedded in the browser bundle or stored in the repository. Glossary export expects a normal Notion table with the eight columns shown in the example configuration. JARVIS writes only after an explicit ZU NOTION + action and rejects duplicate terms already present in the target table.

Configure weather

Weather uses Open-Meteo and requires no API key for this personal setup. Berlin is the default. Override it in .env.local:

WEATHER_LOCATION_NAME=Regensburg
WEATHER_LATITUDE=49.0134
WEATHER_LONGITUDE=12.1016

Advanced runtime overrides

These variables exist for packaging and local troubleshooting. None of them are required: every one has a working default, and the desktop app sets the ones it needs itself. They are omitted from .env.local.example on purpose so the normal setup stays minimal.

Variable	Default	Purpose
JARVIS_CONFIG_DIR	unset	directory holding the runtime .env.local and the knowledge index. The Tauri shell sets it to ~/Library/Application Support/com.sissighn.jarvis/ for the packaged sidecar.
JARVIS_ENV_FILE	unset	explicit path to the runtime environment file, checked first. The packaged server then falls back to $JARVIS_CONFIG_DIR/.env.local and the project's .env.local; the standalone indexer falls back to the project file only.
JARVIS_INDEX_DIR	unset	index location for the indexer only; used when JARVIS_CONFIG_DIR is unset. Without either, the index goes to ~/Library/Application Support/com.sissighn.jarvis/ when NODE_ENV=production and to .jarvis-dev/ otherwise.
JARVIS_SERVER_PORT	4317	loopback port of the packaged sidecar. The Tauri shell passes its own fixed port; values outside 1–65535 fall back to the default.
JARVIS_WEB_PORT	unset	binds the vinext dev server to 127.0.0.1 on this port. npm run desktop:web uses it to serve the native development window on 4317.
WHISPER_SERVER_BIN	/opt/homebrew/bin/whisper-server	path to the whisper-server executable, for a non-Homebrew or custom whisper.cpp build

Data sources and local processing

Capability	Source	Processing
Concept map	Notion API	local crawl, deterministic TF-IDF/phrase extraction, evidenced relations, optional embeddings, SQLite, and force layout
Knowledge Q&A	local index + local Ollama	BM25 and embedding retrieval fused with RRF, ≤ 8 passages from ≤ 5 sources, Qwen generation, claim verification, and citations
Tech briefing	OpenAI News, GitHub Changelog, Techpresso, Hacker News	server-side aggregation, scoring, and deduplication
Tech vocabulary	curated local catalogue + optional Notion table	deterministic five-term daily rotation; explicit single-row export
Weather	Open-Meteo	server-side fetch with a 30-minute memory cache
Voice input	microphone + local whisper.cpp	MediaRecorder capture and local large-v3-turbo transcription
Voice assistant	local Ollama + local action layer	local tool calling, allowlisted functions, confirmation before irreversible actions
Spotify control	Spotify Web API	PKCE login on loopback, tokens stored locally with owner-only permissions
Calendar and mail	Google Calendar API and Gmail API	PKCE login on loopback, two scopes only, endpoint allowlist that cannot reach sending, confirmation before every change
Speech output	macOS system voices	browser speech synthesis, no network request
Preferences	browser storage	stays on the current device

Techpresso is an optional source because its public archive endpoint is undocumented. If it changes or becomes unavailable, the other sources and the last daily browser cache continue to work.

Quality checks

npm run lint
npm run typecheck
npm run test:unit
npm run test:integration
npm run desktop:check
npm run check

npm run check executes the complete local web-quality pipeline. Integration tests build the application, verify server-rendered output, exercise the disconnected Notion state, and confirm that credentials remain behind the server boundary. GitHub Actions additionally builds the desktop sidecar and runs Rust formatting plus Tauri compilation checks on an Apple Silicon macOS runner.

Useful commands

Command	Purpose
npm run dev	start the web interface, the local knowledge indexer, and the Whisper service
npm run dev:web	start only the web interface
npm run dev:indexer	start only the local knowledge indexer
npm run dev:speech	start only the local Whisper service
npm run setup:speech	download or verify the ignored 574 MB speech model
npm run desktop:dev	run JARVIS inside the native Tauri development window
npm run desktop:setup	prepare private desktop configuration and the local speech model
npm run desktop:check	format-check and compile-check the native Rust shell
npm run desktop:build	create the Apple Silicon .app and .dmg bundles
npm run check	run linting, type checks, unit tests, integration tests, and the production build

Troubleshooting voice input

Symptom	Cause	Fix
Dieser Browser kann keine Audioaufnahme bereitstellen in the app	older build without native capture	rebuild with npm run desktop:build; the app records natively and the panel shows MIKROFON NATIV
FFmpeg conversion failed. for every recording	whisper.cpp cannot write its conversion scratch file in the working directory	make sure ffmpeg is installed and that the service runs in a writable directory; the app uses speech-scratch in its application-support directory
macOS never asks for the microphone	the app bundle lacks the usage description	rebuild; src-tauri/Info.plist carries NSMicrophoneUsageDescription, and permission can be re-granted under System Settings → Privacy & Security → Microphone
Recording works, but nothing is understood	the wrong input device is selected	pick the intended microphone under System Settings → Sound → Input; the capture always follows the system default input

Troubleshooting

• Microphone access denied: allow microphone access for the local JARVIS URL in the browser, then reload the page.
• Whisper is unavailable: run npm run setup:speech, then restart with npm run dev. If whisper-server is not installed under /opt/homebrew/bin/, point WHISPER_SERVER_BIN at your build.
• Port 8178 is already in use: stop the older Whisper process before restarting JARVIS, or configure a different local URL consistently.
• Technical terms are misheard: add important recurring vocabulary to the Whisper prompt in scripts/start-whisper-server.mjs and features/speech/server/whisper.ts.
• The first start takes longer: whisper.cpp loads the local model into unified memory before accepting transcription requests.
• The map stays empty: check the selected databases and the Notion connection; the setup dialog shows database and page progress.
• A large database takes time: Notion limits request throughput, but concepts are published after every database and indexing never waits for Qwen.
• Semantic results are missing: install EmbeddingGemma in the setup dialog. Lexical search and the map remain available without it.
• The index looks wrong: use INDEX LÖSCHEN UND NEU AUFBAUEN in the setup dialog for a complete local rebuild.

Privacy and cost

• no paid AI API or hosted database is required
• Qwen runs through Ollama on the same device; retrieved excerpts do not leave the Mac
• the last four Q&A turns are held only in session memory for follow-up questions and are not persisted
• voice recordings are sent only to the local whisper.cpp process and are not persisted by JARVIS
• the voice assistant decides locally which function to call; only Spotify, Google, and browser commands leave the Mac, and only after you connect the account in question
• Spotify tokens live in a local file with owner-only permissions, and PKCE means no client secret exists
• Google access covers calendar events plus reading, archiving, and trashing mail; sending is unreachable through an endpoint allowlist, permanent deletion is not granted, and new events carry no attendees and no invitations
• Google tokens live in a local file with owner-only permissions and are revoked at Google when you disconnect
• spoken answers use the macOS system voices without any network request
• the Notion token stays server-side and is excluded from Git
• Notion graph access is read-only; the optional glossary action appends only the selected row to its configured table
• saved and hidden stories remain on the current device
• individual feed failures do not break the entire briefing
• at most eight indexed passages from at most five sources are passed only to the locally running model
• indexed Notion content, concepts, and embeddings stay in a local SQLite file and are never written to Cloudflare, browser storage, or Git

The complete local data-flow and deletion guidance is documented in Privacy. The source code is available under the MIT License; separately installed runtimes, model weights, packages, and external services keep their own terms, summarized in Third-party notices.

Current scope

The command bar uses retrieval-augmented generation over the local index: BM25 and embedding rankings are fused, focused passages are extracted, and the local model formulates a concise answer. Inline citations are clickable, and a deterministic post-generation guard removes claims that lack sufficient lexical support in their cited passages. If context is missing or Ollama is offline, JARVIS reports uncertainty and falls back to the deterministic source view. The knowledge graph never modifies Notion; only the explicit glossary export appends one selected row to its separately configured table.

The microphone drives the spoken assistant, and the text field stays on Notion knowledge. Assistant functions are limited to the catalogue in features/assistant/tools.ts; the model selects from it and never extends it. Reading data and controlling media run immediately, while irreversible actions are confirmed twice, once by the user and once by the local action layer. The Jarvis wake word and Notion knowledge as an assistant function are deliberately not part of this stage.

The Qwen and Whisper capabilities are intentionally local runtime features. A hosted frontend or Worker cannot access models running on a personal Mac unless an explicit private connection is added; the supported workflow is currently npm run dev on the same device as Ollama and whisper.cpp.

Technology

Next.js 16, React 19, TypeScript, Canvas 2D, node:sqlite with FTS5, Ollama, Qwen 3.5, EmbeddingGemma, whisper.cpp, Whisper large-v3-turbo, vinext, Cloudflare Workers, the Notion API, Open-Meteo, BM25, reciprocal rank fusion, and cosine similarity.

License

Copyright © 2026 Setayesh (Sissighn). JARVIS source code is licensed under the MIT License.