Repository navigation
feat(mosaico): add experimental camera gestures - #8
Merged
Merged
Conversation
Add swipe navigation, wake-only screensaver input, and held OK clock entry. Keep recognition local and disabled by default in display settings. Expand OTA slots to 7 MiB to embed both models with the application. Existing 4 MiB layouts require a first wired partition migration; segmented installation preserves NVS but replaces the old fallback slot. Validation: enabled/disabled firmware and test images build; 31 host Unity tests pass with ASan/UBSan; Bridge tests pass (620, 7 skipped). Camera behavior, performance, and rollback await hardware validation.
Every swipe direction and a horizontal greeting wave now advance to the next page; OK or thumbs-up held for about 500 ms enters the clock. - Recognize a wave as three alternating horizontal legs of at least 12% of the frame, for motion too small to be a swipe. - Issue one command per burst of motion: after a swipe or wave the hand must rest for 400 ms or leave the view. This replaces the 500 ms cooldown. - Anchor the stroke origin to a resting hand so the stroke window opens when the hand starts to move. - Accept a wave without first requiring the view to clear, so the clock can be woken by the hand that entered it. - Skip pose classification while the hand moves between frames, and log every observation with a hand for replay through GestureTracker. - Load the classifier at capture start and mark context changes caused by an accepted gesture explicitly, so other input always requires the view to clear again. Host tests: 13 gesture and image cases pass under ASan/UBSan. The wave thresholds are trial values without device data; recognition and false trigger rates are not validated on hardware.
A cold boot took about 12 s to the first bridge data because the main loop and BLE advertising waited for every peripheral, including about 4.4 s in the battery gauge driver. - Start BLE advertising right after the display so the bridge can reconnect while the remaining peripherals initialize. - Attach the BQ27220 on its own task; the battery stays hidden and the OTA power verdict unavailable until it is ready. - Bring the CO5300 up at zero brightness without the Display On delay and fade in after the first refresh, which removes the white flash of undefined frame memory. The board support change is recorded in UPSTREAM.md. Measured from UART timestamps on one cold boot each: advertising 8.5 s to 3.2 s, main loop 8.5 s to 3.8 s, first time sync 12.1 s to 5.1 s. See docs/validation/mosaico-boot.md.
- Drop the second bq27220_seal() in bsp_battery_init(). The driver already seals and confirms on every path that returns a handle, and each seal waits 2000 ms unconditionally. The board support change is recorded in UPSTREAM.md. - Colour the battery segments as unavailable when they are created. They used the default theme colour until the first render, which looked like a full battery before the first reading. Measured on one cold boot: gauge ready at 5.9 s instead of 7.9 s. Two other recorded boots found a mismatching stored gauge profile and rewrote it; the cause is not established. See docs/validation/mosaico-boot.md.
Set CONFIG_BQ27220_SEAL_SETTLE_MS to 200 instead of the driver default of 2000 so the first battery reading is not held back by the wait. The driver still confirms the sealed state after the wait and fails initialization otherwise. 200 ms is a trial value without datasheet support. On the one boot recorded with it the seal confirmation passed, but the stored gauge profile mismatched again and was rewritten, and later recordings lost the gauge output. Whether this value affects the stored profile is not established; revert this commit alone to restore the default.
Record espressif/dhara 1.1.0 to match dependencies.lock; the distributed license text is unchanged from 1.0.0. Describe the gesture context word layout, and state that the first completed refresh starts the brightness fade-in rather than begin(). Index the Mosaico boot flow document, list camera gestures in the verified device coverage, and describe camera start as up to three attempts, matching the code.
Without sdkconfig.defaults a clean configure selects the default esp32 target, whose DRAM cannot hold the 230400-byte gesture image buffer used by the conversion tests, so the test image fails to link. The other test applications already pin their target the same way.
Owner
Author
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7d83ae8461
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Earlier firmware stored brightness and the auto-clock timeout as separate keys in the shared "display" NVS namespace. When the combined mosaico/settings record is missing or invalid, load those keys before falling back to defaults, so an upgrade that preserves NVS keeps the user's preferences. Capture stays off, and the next save writes the combined record.
Owner
Author
|
@codex review |
|
Codex Review: Didn't find any major issues. Hooray! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add experimental on-device camera gestures to ESP-Mosaico and shorten its startup to first data.
Camera gestures (experimental, off by default)
CAMERAswitch, saved together with brightness and the auto-clock timeout as one NVS record.Storage and OTA
Startup
Documentation
CAMERA_UPSTREAM.md, local BSP changes inUPSTREAM.md, and new components inTHIRD_PARTY_LICENSES.md.Verification
firmware/targets/esp_mosaico(5,422,064-byte application),firmware/targets/esp_mosaico/test_apps/logic(also from a clean configure) andfirmware/test_apps/panel_state.git diff --checkis clean.