AskEase - NVDA AI Screen Assistant

AskEase is a powerful NVDA add-on that integrates AI-driven screen reading assistance. It can capture screen content and analyze it through AI, providing detailed screen descriptions, operation guidance, and intelligent Q&A functionality for visually impaired users.

Features

• Intelligent Screen Analysis: Automatically captures screen content and provides detailed descriptions
• AI Chat Assistant: Ask questions about screen content and get intelligent responses
• Action Recording & Playback: Records user operation history and provides context-aware help
• Multi-language Support: Supports both Chinese and English interfaces
• Keyboard Shortcuts: Convenient keyboard shortcuts for various operations
• Customizable Settings: Configurable dialog behavior and search options

System Requirements

• NVDA 2019.3.0 or later
• Windows Operating System
• Valid OpenAI API key or Azure OpenAI service

User Guide

Installation

1. Download the AskEase-0.1.nvda-addon file
2. Run the file, and NVDA will automatically install the add-on
3. Restart NVDA to activate the add-on

Configuration

NVDA Settings

1. Open NVDA Settings dialog (NVDA menu > Preferences > Settings)
2. Select "AskEase" category from the left panel
3. Configure the following settings:
   • API Key: Enter your OpenAI API key
   • Model: Select the AI model to use (default: gpt-5-chat)
   • API Endpoint: Configure the API service address

Dialog Settings

Within the AI assistant dialog, you can access additional settings by expanding the "Settings" section. The available options include:

• Hide Dialog After Sending: When enabled, the dialog automatically hides after you send a message, returning focus to the previous application
• Use Help Documentation: When enabled, the system searches relevant help documentation to provide more accurate and context-aware responses
• Advanced Search: When enabled, provides enhanced search capabilities that may result in more comprehensive answers, though responses may be slightly slower

Note: All of these options are enabled by default to provide the best user experience. The advanced search feature may make responses slightly slower due to additional processing, but typically provides more thorough and helpful guidance.

Keyboard Shortcuts

Global Shortcuts (Available anywhere in Windows)

Shortcut	Function
NVDA+control+J	Describe current screen and focus
NVDA+control+O	Open AI assistant dialog
NVDA+control+D	Analyze current screen and refine earlier guidance
NVDA+control+Up Arrow	Go back to the previous guidance step
NVDA+control+Down Arrow	Go to the next guidance step

Dialog Shortcuts (Available within AI assistant dialog)

Shortcut	Function
Enter	Send message (from input field)
Shift+Enter	New line (from input field)
Escape	Close the dialog
Alt+C	Clear chat history
Alt+P	Go to previous conversation
Alt+N	Go to next conversation
Alt+S	Send message
F1	Get context-sensitive help

How to Use

1. Get Screen Description: Press NVDA+control+J to get detailed description of current screen

2. AI Chat: Press NVDA+control+O to open dialog and ask questions about screen content

   • Type your question and press Enter to send
   • Use Alt+C to clear chat history when needed
   • Use Alt+H to hide dialog and return focus to previous application
   • Navigate through conversation history with Alt+P (previous) and Alt+N (next)
   • Access the Settings section to customize dialog behavior

3. Smart Analysis: Press NVDA+control+D to let AI analyze current situation and provide operation suggestions

4. Navigate Guidance Steps: Use NVDA+control+Up/Down Arrow to navigate through operation steps

5. Get Help: Press F1 within the dialog for context-sensitive help information

6. Customize Settings:

   • Expand the Settings section within the dialog to access configuration options
   • Toggle "Hide Dialog After Sending" based on your workflow preference
   • Enable/disable help documentation search based on your needs
   • Adjust advanced search settings for optimal performance vs. thoroughness balance

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

Responsible AI

This project has been evaluated and certified to comply with the Microsoft Responsible AI Standard. The team will continue to monitor and maintain the repository, addressing any severe issues, including potential harms, if they arise.

License

This project is licensed under the GNU General Public License v2 or v3. See the LICENSE file for details.