Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions docs/PROTOTYPE.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,22 +38,22 @@ The camera path can report tracking, an ARKit camera step, and an optional plane

The simulation path is a separate bench control on the phone. It can request the D2 hazard actuator, the D3 unavailable actuator, and a D4 left or right label. The screen says the input is simulated. Those controls do not read the camera and do not mean an obstacle was detected. The shared encoder rejects a camera-sourced hazard or a camera-sourced route cue, so a buggy packet of that shape does not move the actuators.

There is no random left or right choice. Nothing in the camera path picks a direction.
The Bluefy page can sound the left buzzer, the right buzzer, or both from boxes in the center half of the picture. That is image left and image right. It is not a turn command. The native camera packet still keeps both side bits off.

## Four channels
## Two buzzers

| Output | Pin | Meaning while the link is alive |
| --- | --- | --- |
| Hazard | D2 | Pulses only for an elevated claim. Off is not an all-clear. |
| System unavailable | D3 | Pulses when the selected source says the pipeline cannot support a claim. |
| Route demo | D4 | Pulses for a manual left or right label. The label is on the phone and in the claim. The pulse does not mean that side is safe. |
| Link | D5 | Board-owned. A short blip every 2 seconds means packets are arriving. A fast, mostly-on pattern means the watchdog expired. |
| Left buzzer | D2 Grove socket | Steady pulse when the claim has the left bit. |
| Status logic | D3 | On during the not-ready pattern. Do not put a buzzer here. |
| Right buzzer | D4 Grove socket | Steady pulse when the claim has the right bit. |
| Link indicator | D5 | Board-owned. A short blip every 2 seconds means packets are arriving. A fast pattern means the watchdog expired. Not a buzzer. |

The programmed watchdog is 500 ms after the last valid packet, using the board's `millis()`. At 500 ms the link is still a heartbeat. At 501 ms it is an outage. During an outage the board forces D2, D3, and D4 off and keeps the outage pattern on D5. A rejected packet does not refresh the watchdog. Before the first valid packet, the board is already in outage.
Packet version is 2. Byte 2 is a side bitmask: 1 left, 2 right, 3 both. Byte 4 is always 0. A version 1 packet is rejected and does not sound a side.

Silence on all four outputs means the board is unpowered, the firmware is not running, or the drivers are disconnected. That is a fault, not a clear path.
The not-ready pattern is a double blip on both buzzers once a second, with D3 held on. That pattern is not "objects on both sides." Objects on both sides use the steady pulse on both buzzers and leave D3 off.

The patterns are different periods in code so a supervisor can tell the channels apart. Whether a particular actuator feels distinct is a bench check, not a result recorded here.
The programmed watchdog is 500 ms after the last valid packet, using the board's `millis()`. At 500 ms the link is still a heartbeat. At 501 ms it is an outage. During an outage the board forces D2 and D4 off and keeps the fast pattern on D5. Quiet buzzers are not an all-clear. If D5 is also quiet, the board is unpowered or not running.

## Actuator wiring

Expand Down
65 changes: 35 additions & 30 deletions docs/WEB_DEMO.md
Original file line number Diff line number Diff line change
@@ -1,43 +1,48 @@
# Bluefy page
# Two blue buzzers

The page is [web/index.html](../web/index.html). It is the phone runtime for this increment. A standard iPhone 16, with Bluefy installed by you, opens the page. The page uses the rear camera, runs COCO-SSD in the browser, and writes the existing 18-byte Bluetooth packet to the UNO R4 WiFi. A laptop is not part of that runtime.
Use the Bluefy page at [web/index.html](../web/index.html) on a standard iPhone 16. Reflash the board with [prototype/firmware/uno_r4_wifi/uno_r4_wifi.ino](../prototype/firmware/uno_r4_wifi/uno_r4_wifi.ino) before opening the new page. An old version 1 sketch will reject the new packets, and an old page will be rejected by this sketch. Neither mismatch sounds a side buzzer.

This repository has not run that combination on a phone. A green test log is not a Bluefy result.
The page has not been run on a phone here, and the buzzers have not been heard here.

## What the packet means
## Pins

The native camera path still cannot send an elevated hazard. The web page uses a separate source, image heuristic, value 3.
| Cane side | Grove socket | What it does |
| --- | --- | --- |
| Left | D2 | Left buzzer. Image-left boxes, or the manual left switch. |
| None | D3 | Status output only. On during the not-ready double blip. No buzzer. |
| Right | D4 | Right buzzer. Image-right boxes, or the manual right switch. |
| None | D5 | Link indicator. Slow blip while packets arrive. Fast pattern when the 500 ms watchdog expires. No buzzer. |

- D2 pulses only for an elevated claim. From the web page that means a supported box is in the center of the picture, or you turned on the manual D2 switch. It is not a measured collision.
- D3 pulses when the camera, model, or page is not ready. Quiet D3 is not an all-clear.
- D4 pulses only for a manual left or right label. That label is not a clear direction.
- D5 is driven by the board. A short blip means packets are arriving. A fast pattern means the 500 ms watchdog expired. The page tries to send every 200 ms while it is connected.
Use the Grove sockets on D2 and D4. Do not use D3 as the right side. A Grove cable is SIG, VCC, and GND. The module in the photo is marked GND, VCC, NC, SIG. NC is unused. SIG goes to the digital pin through the Grove socket.

The image-heuristic packet is rejected if it contains a distance or a route. The model does not report meters, speed, or time to contact. If the model fails, the page shows the error and does not invent a warning from brightness.
Seeed's older Grove Buzzer v1.1 page describes an onboard transistor. That description is not proof for a module marked v1.2. Before you apply power, match your module to its own Seeed page. Connect it only if SIG is a logic input to the module's own driver. Do not connect a bare motor, or a piezo that expects the GPIO to supply its current, to D2 or D4. This repository does not add a new current rating for the module.

## Software already in the repo
## How a side is chosen

- Sketch: [prototype/firmware/uno_r4_wifi/uno_r4_wifi.ino](../prototype/firmware/uno_r4_wifi/uno_r4_wifi.ino)
- The compile job that passed on the previous pull request used Arduino CLI 1.5.1, board package `arduino:renesas_uno` 1.6.0, and ArduinoBLE 2.1.0. This page keeps those versions.
- Pick the board Arduino UNO R4 WiFi. The sketch calls `Serial.begin(115200)` and prints `Cane demo controller`, `Advertising CaneDemo as a BLE peripheral.`, `accepted source=`, and `rejected packet`.
- Pinned browser downloads, only when you tap Load object model: TensorFlow.js 4.22.0 and COCO-SSD 2.2.3 from jsDelivr, both Apache-2.0. The weights are the library's `lite_mobilenet_v2` files at `https://storage.googleapis.com/tfjs-models/savedmodel/ssdlite_mobilenet_v2/model.json`. The camera picture is not sent anywhere.
Section 5 still accepts only person, bicycle, car, motorcycle, bus, and truck, with a score of at least 0.5, and only when the box center is inside the middle half of the picture (from 25% to 75% of both width and height). Boxes farther out are ignored.

## What only you can do
For those boxes, image left and image right are the two sides of the unmirrored rear-camera frame. The page does not flip the video. A center x less than half the picture width sets the left bit. Greater than half sets the right bit. Exactly half sets both bits for that one box. Several boxes are combined, so a left box and a right box can sound together. That both-sides object pattern is a steady pulse. It is not the not-ready double blip.

1. Merge the pull request.
2. In the repository Settings, open Pages, and set the source to GitHub Actions. The workflow [pages.yml](../.github/workflows/pages.yml) does not run for a pull request, and it cannot switch that setting. Until that source is selected, there is no published page. `https://pschmeln.github.io/agent-workflow/` returned 404 when this branch was prepared.
3. After the Pages job on `main` succeeds, open the URL printed by that job in Bluefy on the iPhone 16. Do not type a guessed address.
4. Install Bluefy yourself. Flash the sketch. Wire four external drivers to D2, D3, D4, and D5. Do not power actuators from the header pins.
5. With a sighted spotter, use the page sections one at a time, then the combined switch. Background the app and confirm D5 changes to the fast pattern. Do not cover anyone's eyes.
The not-ready double blip plays on both buzzers when the camera, model, or page is not ready, and D3 stays on. It means the pipeline is down. It does not mean objects are on both sides. Quiet buzzers are not an all-clear: watch D5. A slow blip means the link is alive. A fast pattern means the link timed out.

## Actuator check
## Update steps

| What you do | What should move |
1. In Arduino IDE, select Arduino UNO R4 WiFi. Use the UNO R4 board package 1.6.0 and ArduinoBLE 2.1.0, the same versions the compile job uses.
2. Open the sketch above and upload it. Serial Monitor is 115200. The sketch prints `Cane demo controller`, `D2 left buzzer, D4 right buzzer, D3 status logic, D5 link logic.`, and `Advertising CaneDemo as a BLE peripheral.`
3. Plug the left buzzer into Grove D2 and the right buzzer into Grove D4.
4. Merge this pull request, set GitHub Pages to GitHub Actions, and open the Pages URL in Bluefy. Reload the page so Bluefy does not keep the previous file. The old page and the new sketch do not belong together.
5. On the page, try camera alone, model alone, then Bluetooth. Use the manual left and manual right switches one at a time. Then turn on section 5.

## What a spotter can check

| What you do | What should happen |
| --- | --- |
| Phone not connected | D5 fast, D2 D3 D4 off |
| Connected, no manual switch and no combined warning | D3 unavailable pattern, D5 slow blip |
| Manual D2 | D2 pulses, serial `source=simulation` |
| Manual D4 left or right label | D4 pulses. The label is not a safe turn |
| Combined warning on, supported object in the center box | D2 pulses, serial `source=image-heuristic` |
| Combined warning on, nothing in the center | D2 off. The page says this is not an all-clear |
| Leave Bluefy or disconnect | D5 fast within about a second |
| Power the board before the phone connects | D2 and D4 quiet, D5 fast |
| Connect, leave section 5 off and manual off | Both buzzers double-blip, D3 on, D5 slow. This is not-ready, not two objects |
| Manual left only | D2 steady pulse, D4 quiet, serial `source=simulation` |
| Manual right only | D4 steady pulse, D2 quiet |
| Section 5 on, one supported box left of center | D2 steady pulse, serial `source=image-heuristic` |
| Section 5 on, boxes on both image sides | D2 and D4 steady pulse, D3 off |
| Stop the camera or leave Bluefy | Double-blip or, if the link drops, D5 fast and both buzzers quiet |

Do this with eyes uncovered. The cane you already use stays the cane.
89 changes: 45 additions & 44 deletions prototype/firmware/uno_r4_wifi/cane_logic.c
Original file line number Diff line number Diff line change
Expand Up @@ -71,43 +71,42 @@ static void record_plane(const cane_input_t *in, cane_assessment_t *out) {
}
}

static int side_bits(int sides) {
return sides & (int)(CANE_SIDE_LEFT | CANE_SIDE_RIGHT);
}

void cane_assess(const cane_input_t *in, cane_assessment_t *out) {
out->source = in->source;
out->tracking = in->tracking;
out->scene_depth_supported = in->scene_depth_supported ? 1 : 0;
out->hazard = CANE_HAZARD_NO_CLAIM;
out->sides = 0;
out->system_unavailable = 1;
out->route_cue = CANE_ROUTE_NONE;
record_plane(in, out);

if (in->source == CANE_SOURCE_CAMERA) {
int pipeline_ready = in->camera_authorized && in->world_tracking_supported &&
in->tracking == CANE_TRACKING_NORMAL;
out->system_unavailable = pipeline_ready ? 0 : 1;
out->hazard = CANE_HAZARD_NO_CLAIM;
out->route_cue = CANE_ROUTE_NONE;
out->sides = 0;
return;
}

if (in->source == CANE_SOURCE_IMAGE_HEURISTIC) {
clear_plane(out);
out->route_cue = CANE_ROUTE_NONE;
if (!in->image_pipeline_ready) {
out->system_unavailable = 1;
out->hazard = CANE_HAZARD_NO_CLAIM;
out->sides = 0;
return;
}
out->system_unavailable = 0;
out->hazard = in->image_heuristic_elevated ? CANE_HAZARD_ELEVATED : CANE_HAZARD_NO_CLAIM;
out->sides = side_bits(in->image_sides);
return;
}

if (in->source == CANE_SOURCE_SIMULATION) {
out->hazard = in->simulation_elevated_hazard ? CANE_HAZARD_ELEVATED : CANE_HAZARD_NO_CLAIM;
clear_plane(out);
out->system_unavailable = in->simulation_unavailable ? 1 : 0;
if (in->route_cue == CANE_ROUTE_DEMO_LEFT || in->route_cue == CANE_ROUTE_DEMO_RIGHT) {
out->route_cue = in->route_cue;
}
out->sides = in->simulation_unavailable ? 0 : side_bits(in->simulation_sides);
}
}

Expand Down Expand Up @@ -150,9 +149,8 @@ void cane_command_from_assessment(
uint16_t time_ms_low,
cane_command_t *command) {
command->source = assessment->source;
command->hazard = assessment->hazard;
command->sides = side_bits(assessment->sides);
command->system_unavailable = assessment->system_unavailable ? 1 : 0;
command->route_cue = assessment->route_cue;
command->tracking = assessment->tracking;
command->scene_depth_supported = assessment->scene_depth_supported ? 1 : 0;
command->plane_hit_cm = assessment->plane_hit_valid ? assessment->plane_hit_cm : CANE_PLANE_CM_NONE;
Expand All @@ -165,26 +163,20 @@ int cane_command_admissible(const cane_command_t *command) {
command->source != CANE_SOURCE_IMAGE_HEURISTIC) {
return 0;
}
if (command->hazard != CANE_HAZARD_NO_CLAIM && command->hazard != CANE_HAZARD_ELEVATED) {
if ((command->sides & ~(int)(CANE_SIDE_LEFT | CANE_SIDE_RIGHT)) != 0) {
return 0;
}
if (command->system_unavailable != 0 && command->system_unavailable != 1) {
return 0;
}
if (command->route_cue != CANE_ROUTE_NONE && command->route_cue != CANE_ROUTE_DEMO_LEFT &&
command->route_cue != CANE_ROUTE_DEMO_RIGHT) {
return 0;
}
if (command->tracking < CANE_TRACKING_UNSUPPORTED || command->tracking > CANE_TRACKING_NORMAL) {
return 0;
}
if (command->source == CANE_SOURCE_CAMERA) {
if (command->hazard != CANE_HAZARD_NO_CLAIM || command->route_cue != CANE_ROUTE_NONE) {
return 0;
}
if (command->source == CANE_SOURCE_CAMERA && command->sides != 0) {
return 0;
}
if (command->source == CANE_SOURCE_IMAGE_HEURISTIC) {
if (command->route_cue != CANE_ROUTE_NONE || command->plane_hit_cm != CANE_PLANE_CM_NONE) {
if (command->source == CANE_SOURCE_IMAGE_HEURISTIC || command->source == CANE_SOURCE_SIMULATION) {
if (command->plane_hit_cm != CANE_PLANE_CM_NONE) {
return 0;
}
}
Expand All @@ -198,9 +190,9 @@ int cane_command_encode(const cane_command_t *command, uint8_t out[CANE_PACKET_B
}
out[0] = CANE_PACKET_VERSION;
out[1] = (uint8_t)command->source;
out[2] = (uint8_t)command->hazard;
out[2] = (uint8_t)command->sides;
out[3] = (uint8_t)command->system_unavailable;
out[4] = (uint8_t)command->route_cue;
out[4] = 0;
out[5] = (uint8_t)command->tracking;
out[6] = command->scene_depth_supported ? 1u : 0u;
out[7] = 0;
Expand All @@ -215,7 +207,7 @@ int cane_command_encode(const cane_command_t *command, uint8_t out[CANE_PACKET_B
int cane_command_decode(const uint8_t *bytes, size_t len, cane_command_t *command) {
uint16_t expected;
uint16_t actual;
if (len != CANE_PACKET_BYTES || bytes[0] != CANE_PACKET_VERSION || bytes[7] != 0) {
if (len != CANE_PACKET_BYTES || bytes[0] != CANE_PACKET_VERSION || bytes[4] != 0 || bytes[7] != 0) {
return 0;
}
expected = cane_crc16(bytes, 16);
Expand All @@ -224,9 +216,8 @@ int cane_command_decode(const uint8_t *bytes, size_t len, cane_command_t *comman
return 0;
}
command->source = bytes[1];
command->hazard = bytes[2];
command->sides = bytes[2];
command->system_unavailable = bytes[3];
command->route_cue = bytes[4];
command->tracking = bytes[5];
command->scene_depth_supported = bytes[6] ? 1 : 0;
command->sequence = read_le32(&bytes[8]);
Expand All @@ -239,9 +230,8 @@ void cane_link_init(cane_link_t *link) {
link->last_valid_ms = 0;
link->have_valid = 0;
link->command.source = CANE_SOURCE_CAMERA;
link->command.hazard = CANE_HAZARD_NO_CLAIM;
link->command.sides = 0;
link->command.system_unavailable = 1;
link->command.route_cue = CANE_ROUTE_NONE;
link->command.tracking = CANE_TRACKING_UNAVAILABLE;
link->command.scene_depth_supported = 0;
link->command.plane_hit_cm = CANE_PLANE_CM_NONE;
Expand All @@ -260,6 +250,11 @@ int cane_link_accept(cane_link_t *link, const uint8_t *bytes, size_t len, uint32
return 1;
}

int cane_fault_blip(uint32_t now_ms) {
uint32_t phase = now_ms % CANE_FAULT_PERIOD_MS;
return phase < 80u || (phase >= 160u && phase < 240u);
}

int cane_pulse(uint32_t now_ms, uint32_t period_ms, uint32_t on_ms) {
if (period_ms == 0 || on_ms == 0) {
return 0;
Expand All @@ -282,17 +277,23 @@ void cane_link_outputs(const cane_link_t *link, uint32_t now_ms, cane_outputs_t
outage ? CANE_LINK_OUT_PERIOD_MS : CANE_LINK_OK_PERIOD_MS,
outage ? CANE_LINK_OUT_ON_MS : CANE_LINK_OK_ON_MS);
if (outage) {
out->hazard = 0;
out->unavailable = 0;
out->route = 0;
out->left = 0;
out->right = 0;
out->status = 0;
return;
}
if (link->command.system_unavailable) {
int blip = cane_fault_blip(now_ms);
out->left = blip;
out->right = blip;
out->status = 1;
return;
}
out->hazard = link->command.hazard == CANE_HAZARD_ELEVATED &&
cane_pulse(now_ms, CANE_HAZARD_PERIOD_MS, CANE_HAZARD_ON_MS);
out->unavailable = link->command.system_unavailable &&
cane_pulse(now_ms, CANE_UNAVAILABLE_PERIOD_MS, CANE_UNAVAILABLE_ON_MS);
out->route = link->command.route_cue != CANE_ROUTE_NONE &&
cane_pulse(now_ms, CANE_ROUTE_PERIOD_MS, CANE_ROUTE_ON_MS);
out->status = 0;
out->left = (link->command.sides & CANE_SIDE_LEFT) &&
cane_pulse(now_ms, CANE_OBJECT_PERIOD_MS, CANE_OBJECT_ON_MS);
out->right = (link->command.sides & CANE_SIDE_RIGHT) &&
cane_pulse(now_ms, CANE_OBJECT_PERIOD_MS, CANE_OBJECT_ON_MS);
}

int cane_status_encode(
Expand All @@ -301,7 +302,7 @@ int cane_status_encode(
uint8_t bytes[CANE_STATUS_BYTES]) {
bytes[0] = CANE_PACKET_VERSION;
bytes[1] = (uint8_t)out->link_state;
bytes[2] = (uint8_t)((out->hazard ? 1u : 0u) | (out->unavailable ? 2u : 0u) | (out->route ? 4u : 0u) |
bytes[2] = (uint8_t)((out->left ? 1u : 0u) | (out->right ? 2u : 0u) | (out->status ? 4u : 0u) |
(out->link ? 8u : 0u));
bytes[3] = 0;
write_le32(&bytes[4], last_sequence);
Expand All @@ -316,9 +317,9 @@ int cane_status_decode(const uint8_t *bytes, size_t len, cane_status_t *status)
return 0;
}
status->link_state = bytes[1];
status->hazard_on = (bytes[2] & 1u) != 0;
status->unavailable_on = (bytes[2] & 2u) != 0;
status->route_on = (bytes[2] & 4u) != 0;
status->left_on = (bytes[2] & 1u) != 0;
status->right_on = (bytes[2] & 2u) != 0;
status->status_on = (bytes[2] & 4u) != 0;
status->link_on = (bytes[2] & 8u) != 0;
status->last_sequence = read_le32(&bytes[4]);
return 1;
Expand Down
Loading
Loading