ScreenGuide AI

ScreenGuide AI turns screenshots into clear step-by-step guides. It uses the Google Gemini API to analyze screenshots, generate instructions, and export the result as DOCX, PDF, or a restorable JSON session.

This project was built iteratively with AI-assisted development. Review and test changes carefully before using it in production workflows.

---

Key Features

• AI guide generation: Generates titles and instructions from screenshot sequences using Gemini Flash/Flash-Lite models suitable for this workflow.
• Model selection: Loads available Gemini models from your API key and filters them to stable free-tier models that can process screenshots.
• Prompt/profile presets: Choose the guide style, such as standard SOP, beginner, technical SOP, training handout, quick checklist, or support article.
• Progress and cancellation: Full guide generation and merge-new-screenshots flows show progress and can be cancelled.
• Multi-language output: Generate guides in English, Dutch, Spanish, French, German, or Limburgish.
• Screenshot workflow:
  • Drag and drop screenshots, paste from the clipboard, or use the file selector.
  • Reorder screenshots with drag and drop.
  • Preview screenshots in a full-size viewer with previous/next navigation.
  • Detect possible duplicate screenshots.
• Electron automatic recording: In the Windows Electron shell, clicks can be captured as screenshots and reviewed before they are added to the workflow.
• Built-in image annotation:
  • Rectangle, circle, arrow, pencil, highlight, text, numbering, blur, crop, erase, undo, and redo.
  • Keyboard shortcuts and a visible shortcut legend.
  • Previous/next navigation between screenshots while keeping the editor open.
• Editable guide output:
  • Edit generated text directly.
  • Regenerate individual steps in different styles: shorter, longer, simpler, or professional.
  • Add, delete, merge, or reorder steps.
• Local session management: Save, load, duplicate, tag, import, and export sessions locally.
• Auto-save and recovery: Unsaved work is periodically saved and can be restored after a restart.
• Exports: Export to DOCX, PDF, or JSON. DOCX image export compresses large images to keep document size manageable.
• Light and dark mode.

---

Getting Started

You need a Google Gemini API key.

1. Visit Google AI Studio.
2. Sign in with your Google account.
3. Click Get API key.
4. Select Create API key in new project.
5. Copy the generated key.
6. Open ScreenGuide AI and save the key in Settings.

In the browser version, the key is stored in browser storage. In the Electron shell, the app uses secure Electron storage when available and migrates an existing browser-stored key.

---

Run Locally

Prerequisites:

• Node.js 22.12 or newer
• npm 10 or newer

Install dependencies:

npm install

Run the browser app:

npm run dev

Build the browser app:

npm run build

---

Electron Shell (Windows)

Use the Electron shell when you want the native desktop wrapper and Windows click capture.

Prerequisites:

• Node.js 22.12 or newer
• npm 10 or newer
• .NET 9 SDK

On Windows, the .NET SDK can be installed with:

winget install -e --id Microsoft.DotNet.SDK.9

Develop with Electron:

npm run electron:dev

Build the Windows installer:

npm run electron:build

This command publishes the native MouseHook helper, builds the Vite app, and packages the Windows installer into release_build_v2/.

For a clean installer build:

npm run build:full

The packaged app includes only the published native helper executable from:

native/MouseHook/bin/Release/net9.0-windows10.0.19041.0/win-x64/publish/MouseHook.exe

Build outputs are ignored by git:

• dist/
• release_build_v2/
• native/MouseHook/bin/
• native/MouseHook/obj/

---

Automatic Click Recording

Automatic recording is available in the Electron shell.

When enabled, clicks are captured as screenshots and placed in a review tray. You can open captures full-size, move previous/next, accept useful captures, discard unwanted captures, and skip possible duplicates before they are added to the main screenshot queue.

The Windows build includes the native MouseHook helper used for capturing clicks outside the ScreenGuide window.

---

Tech Stack

• Frontend: React, TypeScript, Tailwind CSS
• Desktop shell: Electron
• Native helper: .NET 9 Windows MouseHook worker
• AI: Google Gemini API via REST calls
• Local storage: IndexedDB for sessions, browser storage or secure Electron storage for the API key
• Exports: jspdf, html2canvas, docx, file-saver

ScreenGuide AI is a local-first single-page application with no backend server.