A minimal, privacy-focused Electron-based browser with an integrated AI Assistant side panel. The assistant supports local models (Ollama), Gemini, and OpenAI-compatible APIs, designed to feel like a native part of the browser UI.
GitHub: https://github.com/stingtao/ai-browser
⚠️ Note: This is a hobby project in active development. Not recommended for security-critical workflows.
- Chrome-like layout: tabs, address bar, content area, status bar
- Find in page (real-time highlight + match count + next/prev):
Cmd/Ctrl+F - Browser Settings:
- Theme: light / dark / system
- Home page, search engine, startup behavior
- History viewer + clear browsing data
- First-run "Import from Chrome" modal
- Multi-language support (English, Spanish, Traditional Chinese)
- Customizable user agent string
- Print:
Cmd/Ctrl+P - Page zoom:
Cmd/Ctrl +/Cmd/Ctrl -/Cmd/Ctrl 0(persists across restarts) - Download Manager: built-in download tracking with progress bars and file management
- AI Assistant:
- Integrated, resizable side panel
- Per-tab AI Assistant panel (open/close is independent per tab)
- Per-tab chat conversations (history list is shared)
- Chat-style conversation UI with conversation history
- Send button turns into Stop to cancel an in-flight chat/agent run (with an in-chat “Generating…” indicator)
- Real-time voice input with Gemini Live API (speech-to-text)
- Prompt shortcuts + prompt manager (template placeholders:
{{content}},{{title}},{{url}}) - Safe Markdown rendering (sanitized HTML)
- AI settings modal (provider/model/context mode)
- Support for local Ollama models, Gemini API, and OpenAI-compatible APIs
- Experimental Browser Agent mode (Playwright): operates the active tab via tools like
snapshot,findElements/readElement,collectLinks,paginateAndCollect,extractStructuredData/extractTables,readerExtract,click/clickByText/clickBySelector,hover/scroll,type/fillText,hotkey/press,navigate,waitFor/waitForLoad,screenshot,tabList/tabActivate/tabOpen/tabClose,networkGetResponses,cookieExport/cookieImport,downloadsWait,uploadFile,reportCreate,exportFile, and Google Docs/Sheets/Slides tools (reuses your login session) - Agent step trace UI (auto-collapses on completion; toggle to expand)
- Agent max steps limit (configurable in AI settings)
- Gemini API key manager with encryption support (save/update/clear) with validation
- AI panel font size (5 levels)
- Context modes: auto-selection, selection-only, or full page content
- App icon:
assets/app-icon.svg(used for the UI favicon + macOS Dock icon in dev)
- 🔎 Find in Page: Real-time highlight + match count refresh as you type
- 🛑 AI Stop UX: Send button becomes Stop (in place) + in-chat “Generating…” indicator
- 🎨 App Icon: New
assets/app-icon.svgused for favicon + macOS Dock icon in dev - 🧭 Browser Agent: Added
fillTextmacro (focus + type + retries), hover/scroll tools + double-click support, auto-repair for invalid JSON tool outputs, and improved Google Docs/Slides typing verification (includessnapshot.axText) - 🖥️ Browser Agent Click: More reliable click targeting via hit-testing (reduces “click didn’t land” on dynamic sites)
- 🤖 Gemini: Added
gemini-3-flash-previewmodel option
- 🎙️ Real-time Voice Input: Added Gemini Live API integration for real-time speech-to-text
- 📥 Download Manager: Complete download tracking with progress bars, file management, and folder access
- 🔎 Find in Page: Added a Find Bar UI (
Cmd/Ctrl+F) with match count and next/prev navigation - 🌐 Enhanced Multi-language Support: Improved internationalization (English, Spanish, Traditional Chinese)
- ⚙️ Advanced AI Settings: Multiple context modes (auto-selection, selection-only, full page)
- 🧠 AI Assistant Improvements: Per-tab conversations, per-tab panel open/close, and a Stop button for in-flight runs
- 🧭 Browser Agent UX: Collapsible step trace + configurable max steps
- 🔧 Browser Customization: System theme support, custom user agent strings, enhanced startup options
- 🎨 UI Improvements: Better responsive design, improved accessibility, enhanced visual feedback
- 🔒 Security Enhancements: Encrypted API key storage with fallback to safe storage
- 📝 Prompt Manager: Advanced prompt template system with page content integration
- 🗣️ Voice Models: Support for Gemini voice models with audio transcription capabilities
- 💾 Settings Persistence: Robust settings management with validation and sanitization
- Node.js: LTS version recommended (18+)
- Operating System: macOS 10.15+, Windows 10+, Linux (Ubuntu 18.04+)
- RAM: 4GB minimum, 8GB recommended
- Storage: 500MB free space
- Ollama: For local AI models (https://ollama.com)
- Gemini API Key: For Google's Gemini models
- OpenAI-compatible API: For alternative AI providers
npm install
npm startThe npm start script properly handles Electron startup and ensures ELECTRON_RUN_AS_NODE is not set.
npm start- Start the application in development modenpm run fix-electron- Fix Electron installation issues (useful for CI/CD)npm run postinstall- Automatically runs after npm install to fix Electron setup
- Install Ollama: https://ollama.com
- Make sure the Ollama service is running (default:
http://127.0.0.1:11434) - Open AI settings (
⚙) → set Provider toLocal (Ollama)→ refresh/pull models
Recommended: set it inside the app.
- Open AI settings (
⚙) - Set Provider to
Gemini API - Paste API key → Save/Update (you can Clear anytime)
- Pick a model from the predefined list (saved immediately)
Alternative: set an environment variable:
export GEMINI_API_KEY="YOUR_KEY"Use this if you have an OpenAI-compatible endpoint (for example: local servers, self-hosted gateways, or providers exposing /v1/chat/completions).
- Open AI settings (
⚙) - Set Provider to
OpenAI-compatible - Set:
- Base URL (example:
http://127.0.0.1:11434/v1) - Model (example:
gpt-4o-miniorllama3.1:8b) - Optional API key (Save/Update), or use
OPENAI_API_KEY
- Base URL (example:
export OPENAI_API_KEY="YOUR_KEY"Use this if you want the Browser Agent to write results into Google Docs / Sheets / Slides.
- Open AI settings (
⚙) → Integrations → Google OAuth - Create an OAuth client ID in Google Cloud Console (Desktop app), then paste:
- Client ID
- Client Secret
- Choose scopes, then click Connect (it opens your default browser for consent)
In AI settings → Agent → Mode, switch to Browser agent (Playwright).
- Runs a tool-using agent loop that can operate the current tab (so it can reuse your login state)
- Uses Chromium CDP on
127.0.0.1by default - Shows intermediate tool steps in a collapsible “Agent steps” trace (auto-collapses when finished)
- Uses trusted CDP mouse/key input (better compatibility with apps like Google Docs/Slides than
element.click()) - Tools:
- Observe/search:
snapshot,screenshot,findElements,readElement - Extract/research:
collectLinks,paginateAndCollect,extractStructuredData,extractTables,readerExtract,multiTabResearch,networkGetResponses - Targeting/helpers:
scrollIntoView,waitFor,waitForLoad - Input/actions:
click,clickByText,clickBySelector,hover,scroll,type,fillText,press,hotkey - Navigation/tabs:
navigate,tabList,tabActivate,tabOpen,tabClose - Session:
cookieExport,cookieImport - Google Workspace:
googleSheetsAppendRows,googleDocsCreateOrAppend,googleSlidesCreateDeck - Files/exports:
uploadFile,downloadsWait,reportCreate,exportFile
- Observe/search:
- Element ids (
data-sting-agent-id) persist for the duration of a single agent run (so the agent can reuse ids across steps). typesupports both element targeting and typing into the currently focused element (useful for canvas-style editors); ondocs.google.comit uses trusted key events + post-typing verification (click the editable canvas area first; sometimes double-click or press Enter)fillTextis a high-level macro for finicky editors (especially Google Docs/Slides): focuses via click/double-click (optionalenter+retries) and then types with verification- On
docs.google.com,snapshotalso includesaxText(Accessibility tree excerpt) to help verify text in canvas-style editors like Google Slides. - If the model returns non-JSON (or extra text), the agent will ask once for a JSON-only retry instead of immediately failing with “Agent returned invalid JSON”.
Max stepsis configurable in AI settings (prevents runaway loops)- Use the Stop button (Send → Stop) to cancel an in-flight agent run
Environment variables:
export STING_CDP_ENABLE=1
export STING_CDP_PORT=9222
export STING_CDP_ADDRESS=127.0.0.1Security note: CDP grants powerful control of the browser; keep it bound to localhost and avoid running untrusted local processes.
- Settings:
Cmd/Ctrl+, - History:
Cmd/Ctrl+Y - Toggle AI Assistant:
Cmd/Ctrl+Shift+A - Focus address bar:
Cmd/Ctrl+L - New tab:
Cmd/Ctrl+T - Close tab:
Cmd/Ctrl+W - Find in page:
Cmd/Ctrl+F - Print:
Cmd/Ctrl+P - Zoom:
Cmd/Ctrl +/Cmd/Ctrl -/Cmd/Ctrl 0
- Browser settings are stored in Electron
userDataas JSON. - Gemini API key is stored using Electron
safeStorageencryption when available; otherwise it falls back to plaintext storage on the device. - OpenAI-compatible API key is stored using Electron
safeStorageencryption when available; otherwise it falls back to plaintext storage on the device. - Google OAuth client secret + refresh token are stored using Electron
safeStorageencryption when available; otherwise it falls back to plaintext storage on the device. - AI Assistant options, prompts, chat history, and page zoom are stored in renderer
localStorage.
- Electron: Cross-platform desktop app framework
- Playwright: Browser automation for AI agent functionality
- WebSocket: Real-time voice communication with Gemini Live API
- Web APIs: Native browser APIs for web content interaction
- Main Process (
main.js): Window management, IPC communication, settings persistence, system integration - Preload Script (
preload.js): Secure IPC bridge between main and renderer processes - Renderer Process (
renderer/): Web-based UI with browser tabs, AI assistant panel, and user interactionsrenderer.js: Main application logicindex.html: UI structure and templatesstyles.css: Responsive design with light/dark theme support
- AI Integration: Multi-provider support (Ollama, Gemini, OpenAI-compatible)
- Browser Agent: CDP-based automation using Playwright Core
- Download Manager: Electron session-based download tracking
- Voice Input: WebSocket-based real-time audio streaming
- Settings System: JSON storage with encrypted API key management
- Internationalization: Built-in i18n system supporting 3 languages
- Context Isolation: Electron security best practices
- API Key Encryption: SafeStorage with fallback mechanisms
- CSP Headers: Content Security Policy implementation
- Input Sanitization: XSS protection for user-generated content
- Extension System: Browser extension API support
- Sync: Cross-device settings synchronization
- Performance: Better memory management and tab unloading
- Privacy: Enhanced tracking protection and cookie management
- Accessibility: Screen reader improvements and keyboard navigation
- Browser Agent mode is experimental and may have stability issues
- On Google Slides/Docs, entering text-edit mode can still be finicky; try
fillText(macro), precise clicks,clickwithcount=2, orpress→Enterbeforetype - Some models may output non-JSON in Agent mode; the app retries once, but persistent formatting issues can still interrupt runs
- Voice input requires stable internet connection
- Large page content may impact AI response performance
Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.
- Fork the repository
- Clone your fork:
git clone https://github.com/your-username/ai-browser.git - Install dependencies:
npm install - Start development:
npm start
main.js- Electron main process (window management, IPC, settings)preload.js- Secure IPC bridge between main and rendererrenderer/- Web-based UI (browser tabs, AI assistant, interactions)renderer.js- Main renderer logicindex.html- UI structurestyles.css- Styling
scripts/- Development and build scriptsassets/- Static assets (icons, logos)
- Electron: For the amazing cross-platform framework
- Google Gemini: For powerful AI models and Live API
- Ollama: For making local AI accessible
- Playwright: For browser automation capabilities
- OpenAI: For the API standards that enable interoperability
This project is licensed under the MIT License - see the LICENSE file for details.
This is an MVP hobby project. Do not use it as a hardened/secure browser for sensitive workflows. Always review AI output before acting on it. API keys are stored with encryption when available, but implement additional security measures for production use.