SIO32 mode response packet error handling misalignment

[Anders429]

When the GBA is receiving a response packet in SIO32 mode, the way libmobile interprets the returned error packet seems to be off by a few bytes.

For example, if my client code is receiving a packet and send a 0xF1 as a response in the expected way, the packet is never retried, whereas it is in SIO8 mode. When communicating using Wit-MKW's mGBA fork, I see the following happening while receiving:

sent from GBA	received from libmobile	comment
0x4b4b4b4b	0x99669a00	magic bytes + header command
0x4b4b4b4b	0x00020080	header length + data
0x4b4b4b4b	0x0000011c	checksum
0x81f10000	0x88000000	acknowledgement

After this, there is no retry of the response packet, which is unexpected.

I fiddled around with this quite a bit to see if I could figure out what was happening, and it turns out that if I both pad an extra 0x00 in front of the GBA's acknowledgement header, and send it when libmobile is sending me the checksum, it will then work. So this triggers the packet retry:

sent from GBA	received from libmobile	comment
0x4b4b4b4b	0x99669a00	magic bytes + header command
0x4b4b4b4b	0x00020080	header length + data
0x0081f100	0x0000011c	checksum, but GBA sends acknowledgement rotated by 1
0x00000000	0x88000000	acknowledgement

I believe the problem is in how this case differs from SIO8 mode to SIO32 mode:

libmobile/serial.c

Lines 240 to 270 in f126f3b

	case MOBILE_SERIAL_RESPONSE_ACKNOWLEDGE:
	if (b->current == 0) {
	b->current++;
	return s->device | 0x80;
	} else if (b->current == 1) {
	// There's nothing we can do with the received device ID.
	// In fact, the real adapter doesn't care for this value, either.
	b->current++;
	return 0;
	} else if (b->current == 2) {
	// Catch the error
	b->error = c;
	}
	if (s->mode_32bit && b->current < 4) {
	b->current++;
	return 0;
	}
	b->current = 0;
	// If an error happened, retry.
	if (b->error == MOBILE_SERIAL_ERROR_UNKNOWN_COMMAND ||
	b->error == MOBILE_SERIAL_ERROR_CHECKSUM ||
	b->error == MOBILE_SERIAL_ERROR_INTERNAL) {
	s->state = MOBILE_SERIAL_RESPONSE_START;
	break;
	}
	// Start over after this
	s->state = MOBILE_SERIAL_WAITING;
	break;
	}

In SIO8 mode, it uses the previously received byte, which means that when libmobile sends the device acknowledgement signal (0x88), it's doing so based on the previously received second checksum byte. That's why it works to not actually consider the GBA's device byte until b->current == 1. We then handle the error when b->current == 2. But in SIO32 mode, we aren't working with just the previous byte, we're working with the last 4 bytes that were received, so b->current == 0 handles the first byte sent for the checksum, b->current == 1 handles the second, and b->current == 2 attempts to read the error from the third byte.

I'm wondering if there needs to be some special casing for this, as there is for the acknowledgement when the GBA is sending a packet.