Handle NACK from a Shimmer3/3R instead of dropping the connection

[marknolan]

The protocol has had a NACK_COMMAND_PROCESSED (0xFE) reply for a long time, but
the driver never learned about it. Grepping the whole driver for NACK — excluding
the generated instruction set and the unrelated Verisense protocol — returned two
comments and nothing else.

What was happening

Both protocol implementations tested only for ACK, with no else:

• ShimmerBluetooth.java:833
• LiteProtocol.java:569

So a NACK was read, silently discarded, and the transaction stalled with
mWaitForAck still true and the instruction stack still locked. Two seconds later
(ACK_TIMER_DURATION) the ACK timer expired and checkForAckOrRespTask called
connectionLost().

There was no retry to soften it. NUMBER_OF_TX_RETRIES_LIMIT is 0, and reading
the NACK byte itself resets mNumberofTXRetriesCount to 0, so 0 >= 0 always
holds, one of the two connectionLost() branches always won, and the soft-recovery
else was dead code. While streaming the same timeout instead cleared the entire
queued instruction stack.

A refused command was indistinguishable from a dead link, and cost the
connection.

This is not hypothetical and does not depend on any pending firmware change. The
shipping firmware NACKs on:

• any SET command while the device is sensing (ShimBt_isCmdBlockedWhileSensing,
  which covers most of the configuration surface)
• a sync-mode mismatch (sync commands while sync is disabled, or vice versa)
• SET_DAUGHTER_CARD_MEM when the EEPROM write fails
• GET_BMP180/GET_BMP280_CALIBRATION_COEFFICIENTS on a Shimmer3R, and
  GET_PRESSURE_CALIBRATION_COEFFICIENTS on a BMP581

The existing coping strategy was avoidance rather than handling, which is decent
evidence the failure mode is real — ShimmerBluetooth.java:3344 skips the BMP581
calibration read with the comment "its firmware NACKs … so don't send it."
Avoiding each NACK individually does not scale, and it makes every new firmware
NACK a latent connection-drop in every released copy of the driver.

What this changes

A refusal now unwinds the transaction and keeps the connection up.

ShimmerObject	adds NACK_COMMAND_PROCESSED = (byte) 0xFE
processNackFromCommand()	stops the ACK timer, clears mWaitForAck/mWaitForResponse, drops the refused instruction so the stack advances, completes the transaction, releases the stack lock
eventCommandRefused()	the application hook, called after the unwind

The refused instruction is dropped only when it is still queued, which
processNackFromCommand() determines from mWaitForAck on entry. A GET's
instruction is removed as soon as its ACK arrives, so an unconditional removal
would drop whatever was queued behind it and silently skip that command —
Copilot caught that in review, and it is fixed in 648897ba. The
ACK-wait and streaming paths both still have the instruction queued; the
response-wait path does not.

Handled in three places:

1. Waiting for an ACK, not streaming — the main configuration path.
2. Waiting for a response, not streaming — a NACK can land here instead if the
   firmware only discovers it cannot serve a GET while building the response.
   Checked before isKnownResponse(). LiteProtocol gained the matching branch
   in 648897ba after review; it only had the ACK-wait one at first.
3. A data packet followed by a NACK while streaming — this is the
   SET-while-sensing case, the most reachable one of the lot.

Two deliberate non-obvious choices

eventCommandRefused() is concrete with an empty default, not abstract — and
not a new ProtocolListener callback either. Every other notification hook here
(sendProgressReport, connectionLost, eventLogAndStreamStatusChanged) is
protected abstract, so adding one would break every downstream subclass and
listener implementation at compile time. This way nothing outside the driver has to
change, and anyone who wants the refusal surfaced overrides it.

A stray NACK with no command in flight is logged and ignored, not treated as a
refusal. processBytesAvailableAndInstreamSupported() runs only when
!mWaitForAck && !mWaitForResponse, so there is no transaction to fail there. The
TypeScript SDK gates its own NACK framing the same way (on a command genuinely
awaiting a reply) so that a leaked stream byte cannot fabricate a refusal; same
reasoning applied here.

Two supporting changes

• The resync helper accepted only 0x00 and 0xFF as a packet boundary, so a
  NACK sitting in the buffer could not be skipped past cleanly. It now accepts
  0xFE, and is renamed findOffsetOfNextZeroOrFF → findOffsetOfNextPacketBoundary
  to match what it actually does (private, one call site).
• NACK is registered in mBtCommandMapOther so btCommandToString names it in
  the logs. Deliberately not in mBtResponseMap — isKnownResponse() must stay
  false for it, or processResponseCommand() would try to parse it as a response
  body.

LiteProtocol gets the same treatment (it is live — BasicShimmerBluetoothManagerPc
instantiates it). Its NACK value is declared locally because the generated
LiteProtocolInstructionSet has no NACK entry; adding it to
LiteProtocolInstructionSet.proto and regenerating would be tidier, but that
rewrites a large generated file committed in two places for the sake of one
constant. Happy to do it that way instead if preferred.

Verification

Compile-verified only. ShimmerDriver compiles clean with no new warnings.

Note the checked-in ShimmerDriver Gradle wrapper (6.1) cannot build this project
at all in its current state, with or without this change: grpc-all:1.71.0 pulls a
guava whose module metadata has android/jre variants that Gradle 6.1 cannot
disambiguate, so :compileJava fails at dependency resolution. I confirmed that
against an untouched tree before assuming it was mine, and compiled by forcing
guava to the version the project itself declares. CI is unaffected because
gradle.yml builds ShimmerDriverPC with Gradle 8.10.2, which resolves it fine —
so CI here is a genuine check. The stale wrapper looks worth a separate ticket.

No hardware, and no automated test. There is no test harness for the BT receive
path — ShimmerBluetooth is abstract with a large number of abstract I/O methods
and no existing test instantiates it, so exercising a NACK end-to-end needs a fake
transport that does not exist yet. I did not want to half-build one inside this
change; it is worth its own piece of work, and it would pay for itself well beyond
this fix.

Worth a bench check: configure something while streaming (the firmware NACKs it) and
confirm the connection now survives and the queued commands continue, where
previously it dropped after ~2 s.

Not in scope

The related writeMem defect from the tracker ticket is untouched here: the
over-range branch at ShimmerBluetooth.java:4589 detects a buffer running past the
device's memory range and does nothing, its body being entirely commented out, and
MAX_CALIB_DUMP_MAX (4096) disagrees with the firmware's SHIMMER_CALIB_RAM_MAX
(1024 on Shimmer3) so the guard would be wrong even if enabled. Deciding what should
happen on an over-range write is a separate question from teaching the driver to
read a NACK, so I have left it for its own change.

🤖 Generated with Claude Code

[Copilot]

🟡 Changes recommended

There is at least one confirmed transaction-stack correctness bug (and an uncovered NACK path in LiteProtocol) that can cause queued commands to be dropped or time out incorrectly.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Updates the Shimmer driver to recognize and handle NACK_COMMAND_PROCESSED (0xFE) so a refused command unwinds the in-flight transaction and the connection is not dropped due to an ACK/response timeout.

Changes:

• Adds a protocol constant for NACK (0xFE) and registers it for logging/command-to-string mapping.
• Handles NACK in ShimmerBluetooth across ACK-wait, response-wait, and streaming (data packet followed by NACK) paths.
• Adds analogous NACK handling and an overridable eventCommandRefused(...) hook in LiteProtocol.

File summaries

File	Description
ShimmerDriver/src/main/java/com/shimmerresearch/driver/ShimmerObject.java	Adds NACK_COMMAND_PROCESSED constant (0xFE).
ShimmerDriver/src/main/java/com/shimmerresearch/comms/radioProtocol/LiteProtocol.java	Introduces NACK constant and transaction-unwind handling plus eventCommandRefused(...).
ShimmerDriver/src/main/java/com/shimmerresearch/bluetooth/ShimmerBluetooth.java	Adds NACK support in multiple receive paths, updates packet-boundary resync, and adds eventCommandRefused(...).

Review details

Suppressed comments (1)

ShimmerDriver/src/main/java/com/shimmerresearch/bluetooth/ShimmerBluetooth.java:1379

• findOffsetOfNextPacketBoundary() uses a raw 0 instead of the existing DATA_PACKET constant; now that this method was touched, using the named constant would avoid a magic number and match the surrounding code style (e.g., bufferTemp[0]==DATA_PACKET).

			byte b = buffer[i];
			if (b == 0 || b == ACK_COMMAND_PROCESSED || b == NACK_COMMAND_PROCESSED)
				return i;

• Files reviewed: 3/3 changed files
• Comments generated: 3
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[marknolan]

Confirmed and fixed in 648897b — this was a real bug, thanks. The GET path does removeInstruction(0) as soon as its ACK arrives, so a NACK in the response-wait state was dropping whatever was queued behind it and silently skipping that command.

Rather than pass a flag from each call site, processNackFromCommand() now captures mWaitForAck on entry: that is true only in the state that still has the refused instruction queued (the ACK block clears it before setting mWaitForResponse), and it is also true in the streaming data-packet-then-NACK path. The javadoc no longer claims the drop is unconditional. Same fix applied to LiteProtocol.processNackFromCommand().

[marknolan]

Fixed in 648897b. Added the response-wait NACK branch to LiteProtocol, checked before isKnownResponseByte() so it mirrors ShimmerBluetooth.processNotStreamingWaitForResp() exactly. It shares the same processNackFromCommand(), so it also picks up the conditional-drop fix from the other comment.

[marknolan]

All four review items are addressed in 648897ba — the three inline comments have individual replies, and this covers the suppressed one.

Suppressed comment (findOffsetOfNextPacketBoundary using a raw 0): agreed and fixed — it now reads b == DATA_PACKET || b == ACK_COMMAND_PROCESSED || b == NACK_COMMAND_PROCESSED, which is both self-documenting and consistent with bufferTemp[0]==DATA_PACKET in the surrounding code. Worth noting this was the most useful of the four after the transaction-stack bug, despite being the one held back.

Summary of the round:

Item	Outcome
processNackFromCommand() dropping the wrong instruction	Real bug, fixed. The drop is now conditional on mWaitForAck, which is set only in the states that still have the instruction queued. Fixed in LiteProtocol too, and the javadoc corrected.
LiteProtocol missing the response-wait NACK branch	Fixed — added, checked before isKnownResponseByte() to mirror ShimmerBluetooth.
Stale resync inline comment	Fixed.
Raw 0 instead of DATA_PACKET (suppressed)	Fixed.

The PR description has been updated rather than just corrected here, since the first version described the instruction drop as unconditional and a future reader could otherwise "fix" the code back to that.

Still compile-verified only, and still no automated coverage of the receive path — the transaction-stack bug is a good argument for the fake-transport harness noted in the description, since a test would have caught it and review only just did.

[Copilot]

🔵 Needs a closer look

It changes low-level transport state-machine behavior (including streaming parsing) without an automated/hardware-backed test to validate the end-to-end receive/unwind paths.

Review details

• Files reviewed: 3/3 changed files
• Comments generated: 0 new
• Review effort level: Lite