- Takes a screenshot every 2 seconds (only of apps you choose)
- Reads and embeds the text from each screenshot with OCR
- Lets you chat with anything you've seen on your phone via a local LLM (my favorite: llama3.2 1B)
- Go to Settings > Apps > Special app access > Install unknown apps > give permission to Google Drive.
- Click on:
- For most Devices: https://drive.google.com/file/d/1Dugrs3gOF7K0YmruB51u33f3RrFzl8yh/view?usp=drive_link
- For all Devices (if the above doesn't work): https://drive.google.com/file/d/1Agc-64LTpT56GLNwmzkRsWn6usyl8NkB/view?usp=drive_link
- To give accessibility permissions you may need to click on the more info section on the top right of the HindsightMobile accessibility section and click "allow restricted settings"
git clone --recursive https://github.com/cparish312/HindsightMobile.git- Open the Project in Android Studio
- Connect your Android Device
- You need to do a release build for the LLM to run quickly:
- Go View -> Tool Windows -> Build Variants and then click the drop down for release
- Run the application using
Run>Run 'app'or the play button in Android Studio- If getting incompatible AGP version install the newest version of Android Studio
Setup an onboarding session or just chat about the project here
Ingest Screenshots: runs a manual ingestion of screenshots- Add to db
- OCR
- Embed
Manage Recordings: Takes you to manage recordings screen- If checked the app will be record
- Delete all content (screenshots, videos, embeddings, OCR Results) for a given app
Chat: go to chatScreen Recording: Start Screen recording Background Process (May have to click stop on Notification to stop)Auto Ingest: Auto Run Ingestion atHour to Auto Ingest (Military Time)- Default is 2 am
- IMPORTANT PLEASE READ THIS
Record New Apps By Default: when you enter an app that has not been recorded yet it will automatically start recording
- If you click on the Assistant's response you can see the exact prompt that went into the LLM