Talk to it out loud. A little animated character lives on your desktop, listens, talks back, reacts, watches your screen when you ask — and can write its own tools to do new things.
Powered by Google Gemini 3.1 Flash Live — native, real-time speech-to-speech.
Quadius is a desktop voice companion. It runs as a frameless, transparent, always-on-top "desktop pet": an animated character that floats on your screen, lip-syncs while it speaks, and shows little expressions and speech bubbles. Under the hood it holds a live, full-duplex voice conversation with Gemini — you just talk, and it talks back, instantly.
What makes it different from a normal voice assistant: it can extend itself. Quadius ships with a pluggable tool system, and the model can author and run brand-new tools at runtime — real JavaScript with full access to your machine — so when you ask it to do something it can't yet, it can build the capability and (after a quick reconnect) use it.
| 🎙️ Real-time voice | Native speech-to-speech via the Gemini Live API. Always-listening (VAD) with barge-in (talk over it to interrupt), or push-to-talk with a global hotkey. |
| 🧑🎤 Animated character | A transparent, draggable, always-on-top desktop pet. Frame-based sprite states (idle / listening / thinking / speaking) with mouth lip-sync driven by the live audio amplitude, plus emotion expressions. |
| 🎨 Bring your own art | Upload your own PNG sprites per state right in the app (Settings → Appearance) with live preview — including a multi-frame "talking" set for lip-sync. Your art overrides the defaults; reset any time. |
| 🛠️ Self-writing tools | A pluggable tool system. The agent can create_tool / update_tool to author new JavaScript tools on the fly, with full system access (run programs, read/write files, manage windows, call APIs). |
| 🗣️ Proactive speech | Tools can make the character speak on their own — ctx.speak() / ctx.speakIn(ms, …) — so timers, reminders, and alerts actually talk to you, not just pop up. |
| 🖼️ On-screen output | Tools can pop a speech bubble (show_bubble) or render arbitrary HTML/CSS/JS in a window (show_visual) — charts, animations, mini-GUIs. |
| 👀 Screen vision | Ask it to look at your screen and it streams frames into the conversation (opt-in). |
| 🧠 Memory | Long-term facts it remembers across sessions, editable in Settings. |
| ⏰ Reminders & timers | Schedule reminders that announce themselves out loud when they fire. |
| 🎭 Personas | Saved presets of voice + personality + sprite set; switch in a click. |
| 📝 Full transcripts & logs | Every session is saved as a reviewable conversation transcript; structured JSONL logs with a live in-app viewer. |
| 📊 Usage metering | Session + lifetime talk time and an estimated cost readout. |
| 🔒 Local-first & key-safe | Your API key lives only in the main process / .env (gitignored, never sent to the renderer). The only network egress is to Google's Gemini API. |
- Node.js ≥ 18 (developed on Node 22)
- A Google Gemini API key — free from Google AI Studio
# 1. Clone
git clone https://github.com/<your-org>/quadius.git
cd quadius
# 2. Install dependencies (Electron + the Gemini SDK)
npm install
# 3. Add your API key — either:
cp .env.example .env # then edit .env and paste your key
# ...or just run the app and paste it in Settings → General → API key
# 4. Launch
npm startThe character appears on your desktop. Click Connect, allow microphone access, and start
talking. Hover the character for the control strip (Connect · Mic · Mode · Quit); right-click for
a context menu; open Settings from the tray icon or with Ctrl/Cmd + Shift + S.
First time? If you didn't create a
.env, the app opens Settings and walks you through pasting a key — it writes.envfor you and applies it without a restart.
The API key, the Gemini session, and all tools live in the main process. The character and
the settings UI are separate windows that talk to main only through a locked-down preload
bridge (window.quadius). Audio is captured in the renderer, streamed to main, relayed to
Gemini, and the response audio comes back the same way.
flowchart LR
subgraph Renderer["🖼️ Pet window (renderer)"]
Mic["🎙️ Mic → 16kHz PCM"]
Spk["🔊 Speakers ← 24kHz PCM"]
Char["🧑🎤 Sprite + lip-sync + captions"]
end
subgraph Main["⚙️ Main process (owns the API key)"]
Live["Gemini Live client"]
Tools["Tool registry (+ self-authored)"]
Feat["memory · reminders · vision · personas · history"]
end
Gemini["☁️ Gemini 3.1 Flash Live"]
Mic -->|IPC| Live -->|sendRealtimeInput| Gemini
Gemini -->|audio| Live -->|IPC| Spk
Gemini -->|toolCall| Tools -->|result| Gemini
Live --> Char
Tools --- Feat
style Main fill:#0e2a18,stroke:#39ff14,color:#eaeaea
style Renderer fill:#10131a,stroke:#5b8cff,color:#eaeaea
style Gemini fill:#1a2740,stroke:#4285f4,color:#eaeaea
A deep dive — process model, the audio round-trip, session resumption, and the tool-call sequence — is in docs/ARCHITECTURE.md.
The character is a set of frame-based sprite states. The speaking state is a mouth ramp — closed → wide — and Quadius picks the frame from the live output-audio amplitude, so the mouth tracks the actual voice. Captions are revealed paced to the audio (not dumped ahead of it).
Want your own look? Settings → Appearance lets you upload transparent PNGs for each state (face, talking frames, emotions) with a live thumbnail — see docs/CONFIGURATION.md.
Tools are small JavaScript modules the model can call. They're discovered from two places:
src/main/tools/builtin/ (shipped) and agent-tools/ (your own — and the ones the agent writes
for itself, hot-loaded). A tool looks like this:
export default {
name: 'get_weather',
description: 'Get the current weather for a city.',
parameters: {
type: 'object',
properties: { city: { type: 'string' } },
required: ['city'],
},
async execute(args, ctx) {
const r = await ctx.fetch(`https://wttr.in/${encodeURIComponent(args.city)}?format=j1`);
const data = await r.json();
return { tempC: data.current_condition[0].temp_C };
},
};Inside execute, the ctx object is the agent's hands on your machine:
| Capability | What it does |
|---|---|
ctx.speak(text) · ctx.speakIn(ms, text) |
Make the character talk out loud now / after a delay (timers, alerts). |
ctx.ui.showBubble(text) · ctx.ui.showVisual(html) |
Pop a speech bubble / render HTML in a window. |
ctx.ui.setExpression(emotion) |
Drive the character's emotion overlay. |
ctx.exec(cmd) · ctx.require(mod) |
Run a shell command / load any Node or Electron module. |
ctx.fs · ctx.path · ctx.os · ctx.fetch |
Files, paths, OS info, HTTP. |
ctx.callTool(name, args) |
Call another tool. |
ctx.memory · ctx.reminders · ctx.vision … |
The feature services. |
The built-in meta-tools (create_tool, update_tool, read_tool, delete_tool,
list_tools, reload_tools) are how the agent extends itself: it writes a new module, the
registry validates and hot-loads it, and it becomes callable on the next session connect.
📖 Full authoring guide, the complete ctx reference, and worked examples:
docs/TOOLS.md.
Tools run in the Electron main process with full, unsandboxed access to your computer — by design, so the agent can genuinely do things. A tool (including one the model writes) can run programs, read/write files, and reach the network. Mitigations are built in and configurable: turn tools off entirely (
toolsEnabled), or require per-call approval before anything runs (toolAutoApprove+ a tool'srequiresApproval), and an optional capability sandbox. Only run Quadius with a model and tools you trust. See docs/TOOLS.md#security.
Open Settings from the tray or Ctrl/Cmd + Shift + S. Tabs:
| Tab | What's there |
|---|---|
| General | Voice, model, system prompt, mic mode, push-to-talk hotkey, always-on-top / click-through, tool toggles, and API-key setup. |
| Appearance | Upload your own character PNG sprites per state, with live previews. |
| Personas | Saved voice + personality + sprite presets. |
| Memory | Long-term facts the model remembers; add / remove. |
| History | Past conversation transcripts — browse, search, export (txt / md / json). |
| Usage | Session and lifetime talk time + estimated cost. |
| Tools | Browse, edit, run, create, enable/disable tools; approve tool calls. |
| Logs | Live structured logs with level filters; open the log folder. |
| About | Build info. |
Every setting, default, voice, and data path is documented in docs/CONFIGURATION.md.
quadius/
├─ src/
│ ├─ main/ # Electron main process (owns the API key)
│ │ ├─ index.js # lifecycle + all IPC wiring
│ │ ├─ gemini/ # Live API client + session resumption
│ │ ├─ tools/ # tool registry, defineTool, builtin tools
│ │ ├─ characterAssets.js # quadius-char:// protocol for uploaded sprites
│ │ └─ memoryStore · reminders · personaManager · usageMeter · vision · …
│ ├─ preload/ # contextBridge: the window.quadius API
│ ├─ renderer/ # the transparent pet window
│ │ ├─ audio/ # mic capture + playback (AudioWorklets)
│ │ └─ character/ # sprite renderer, manifest, expressions
│ ├─ settings/ # the settings window (tabbed UI)
│ └─ shared/contract.js # single source of truth: IPC channels, states, defaults
├─ agent-tools/ # user/agent-authored tools (hot-loaded)
├─ docs/ # architecture, tools, configuration guides
└─ .env # your GEMINI_API_KEY (gitignored)
| Doc | What's inside |
|---|---|
| docs/ARCHITECTURE.md | Process model, audio pipeline, the Gemini Live session, the tool-call flow, feature services — with diagrams. |
| docs/TOOLS.md | Writing tools: module format, the full ctx reference, worked examples, runtime authoring, and the security model. |
| docs/CONFIGURATION.md | Every setting and default, the Settings window tab-by-tab, voices, mic modes, and where your data lives. |
- Electron (ESM) — transparent, frameless, always-on-top desktop window.
- @google/genai — the Gemini Live API (
gemini-3.1-flash-live-preview). - Web Audio + AudioWorklets — 16 kHz mic capture, gapless 24 kHz playback.
- Canvas sprite renderer — frame animation + amplitude-driven lip-sync.
- No bundler, no framework — plain ESM across main, preload, and renderer.
Issues and PRs are welcome. A good first contribution is a new tool in agent-tools/ — see the
authoring guide in docs/TOOLS.md. Please don't commit your .env or any
secrets (it's gitignored for you).
MIT © 2026 Quads Lab LLC.
Quadius talks to Google's Gemini API. Review Google's terms and pricing before heavy use. Tools run with full system access — only use models and tools you trust.