Skip to content

get-started.md

Latest commit

 

History

History
157 lines (110 loc) · 4.94 KB

get-started.md

File metadata and controls

157 lines (110 loc) · 4.94 KB
Raw

Language: English | 简体中文 | 日本語

OpenGUI Getting Started

This repository already contains the runnable backend and Android client.

Option 1: Bootstrap with Claude Code, Codex, or OpenCode

Open Claude Code, Codex, or OpenCode from the OpenGUI repository root, then ask it to read the bootstrap skill.

Start with the bootstrap skill:

Recommended prompt:

Read ./skills/open-gui-bootstrap/SKILL.md and help me run OpenGUI. Only ask me for phone-side actions.

The bootstrap skill guides the local backend and Android client setup: it starts the backend, builds or installs the Android client, runs adb setup, and checks whether the phone is visible to OpenGUI.

The same prompt works in OpenCode. Because this repository keeps the Skill in the top-level skills/ directory, specify the path explicitly as shown above. OpenCode's automatic discovery instead searches locations such as .opencode/skills/ and .agents/skills/; see its Agent Skills documentation. No OpenCode-specific OpenGUI configuration is required.

The skill should use the repository scripts directly:

  • server/start.sh
  • client/start.sh

It should only stop for phone-side actions or credentials:

  • connecting an Android device or starting an emulator
  • approving USB debugging
  • enabling Accessibility Service
  • granting overlay or screen-related permissions
  • providing model API credentials

After setup, run a basic smoke test:

cd server
pnpm opengui -- devices --json
pnpm opengui -- do "Observe the current Android screen and summarize what you see" --json

Option 2: Manual setup

1. Start the backend

cd server
./start.sh

What server/start.sh does:

  • checks Node.js 22+, pnpm, and Docker
  • starts PostgreSQL and Redis in Docker
  • creates server/apps/backend/.env from .env.example on first run
  • installs dependencies
  • generates Prisma client
  • pushes schema and seeds default backend data
  • starts the backend on port 7777

For the default first-run setup, add only your model API key:

  • VLM_API_KEY

The backend currently uses the VLM_* variables as the shared OpenAI-compatible model configuration for graph agents. They are used by planning, supervision, summarization, and the executor vision path.

VLM_BASE_URL and VLM_MODEL already have defaults in .env.example. Change them only when using a different OpenAI-compatible provider or model.

Example:

VLM_API_KEY=your_api_key
VLM_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1
VLM_MODEL=qwen3.6-plus

The backend can start without VLM_API_KEY, but real task execution will fail when the graph needs to call the model. LangSmith tracing and IM channel credentials are optional for the first run.

Useful endpoints after startup:

  • API: http://localhost:7777/api
  • Docs: http://localhost:7777/docs

2. Connect a device and install the Android client

Root access and an unlocked bootloader are not required. OpenGUI captures screenshots and performs gestures through Android's standard AccessibilityService APIs. ADB is used to install and launch the APK and to configure adb reverse for the local backend; it does not root or modify the Android system.

The current Android client requires Android 11 (API 30) or newer. Android 9 (API 28) and other older releases are not supported by the screenshot-based execution path. The client currently targets Android 15 (API 35).

cd client
./start.sh

What client/start.sh does:

  • checks adb and Java
  • requires a connected Android device
  • runs adb reverse tcp:7777 tcp:7777
  • builds the debug APK
  • installs the APK
  • launches com.coremate.opengui/.login.SplashActivity

The adb reverse mapping belongs to the current ADB device connection. It may be lost after the phone is disconnected and reconnected, the phone reboots, or the ADB server restarts. If the Android client can no longer reach the local backend, reconnect the device and run:

adb reverse tcp:7777 tcp:7777

Running client/start.sh again also recreates the mapping.

3. Complete phone-side permissions

Open the app and enable:

  • USB debugging approval
  • Accessibility Service
  • overlay permission
  • battery optimization exemption if needed

Current Source-Available Build Behavior

The Android app currently skips the old login gate in the source-available build and goes straight to HomeActivity.

For local runs, the backend task controllers also default to userId = 1, so first-run setup no longer depends on the older OTP flow.

More detail