Fixed Find command to work with iPhone running ios 18.4.1

[mschering]

What does this implement/fix? Explain your changes.

This pull request replaces my old pull request implementing the Find command. I took the base code and tested it but ran into an issue:

The Find command implementation:

Z-Push/src/lib/request/find.php

Line 178 in af25a21

$message = self::$backend->Fetch(false, $u['longid'], $cpo);

Didn't pass the folderId to the backend Fetch() method:

Z-Push/src/lib/default/diffbackend/diffbackend.php

Line 129 in af25a21

public function Fetch($folderid, $id, $contentparameters) {

The message can't be found that way. I also pass the server ID and not the long ID as kopano is the only backend that splits the long ID:

Z-Push/src/backend/kopano/kopano.php

Line 750 in 07f60c7

list($fsk, $sk) = Utils::SplitMessageId($id);

The BackendDiff class doesn't do this.

Does this close any currently open issues?

Not sure. But it might close #141

Any relevant logs, error output, etc?

...

Where has this been tested?

Server (please complete the following information):

• OS: debian
• PHP Version: 8.3
• Backend for: Group-Office
• and Version: current development branch

Smartphone (please complete the following information):

• Device: iPhone 13
• OS: iOS 18
• Mail App Apple mail

[sebasi]

The backend should handle the long ids, as it generates these ids as it sees fit. ItemOperations doesn't have a folder id iirc so the handling of longids should be done exclusively in the backend IMHO.

[mschering]

OK thanks, I can add this in my backend. But is it ok to pass false to $folderId too?

[sebasi]

But is it ok to pass false to $folderId too?

I would say so, yes. That's by design. The longid is self contained. The backend should be able to uniquely identify a message of the user just with this id.
The spec gives some good ideas too: https://learn.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-ascmd/552f41c0-33ff-48ab-9fd0-92239e0577f0

[kour1er]

@mschering thanks for looking into this search issue - I've run into the same problem. I tried your patch (9575599) and I can now see search results coming from outside of the sync period but I only get Subjects, not the body of the emails returned in the search. Dovecot backend. I see things like BackendIMAP->StatMessage failed to retrieve overview. And StatusException: BackendDiff->Fetch('%s', '%s') Error, unable retrieve message from backend. I don't suppose you have had time to look into this further?

[mschering]

Do you know what folder id and uid it passes to the fetch method?

[kour1er]

@mschering so on the most recent Z-Push git version (659ecf8) I get things like this in the log:

z-push-dev  | 21/08/2025 12:52:04 [   39] [WARN] [test@example.com] BackendIMAP->StatMessage('','62ef5173:46656'): Failed to retrieve overview: UID sequence syntax error
z-push-dev  | 21/08/2025 12:52:04 [   39] [WARN] [test@example.com] BackendIMAP->StatMessage('','62ef5173:46656'): Failed to retrieve overview: UID sequence syntax error
z-push-dev  | 21/08/2025 12:52:04 [   39] [ INFO] [test@example.com] StatusException: BackendDiff->Fetch('%s','%s'): Error, unable retrieve message from backend - code: 8 - file: /usr/share/z-push/lib/default/diffbackend/diffbackend.php:134
z-push-dev  | 21/08/2025 12:52:04 [   39] [FATAL] [test@example.com] WBXML 10K debug data: AwFqAAAZRUYDNDJERDczMzYtNDE5Ri00RjY4LUJFQkMtOTU5ODJCRjY5ODk5AAFHSEkAAFADRW1haWwAAVIDZWY2YzEzYjAAAQAZSwN0bzoiVGVzdCIgT1IgY2M6IlRlc3QiIE9SIGZyb206IlRlc3QiIE9SIHN1YmplY3Q6IlRlc3QiIE9SICJUZXN0IiBPUiAiVGVzdCIAAQFMTQMwLTk5AAEOAQEBAQ==
z-push-dev  | 21/08/2025 12:52:04 [   39] [FATAL] [test@example.com] WBXML 10K debug data: AwFqAAAZRUYDNDJERDczMzYtNDE5Ri00RjY4LUJFQkMtOTU5ODJCRjY5ODk5AAFHSEkAAFADRW1haWwAAVIDZWY2YzEzYjAAAQAZSwN0bzoiVGVzdCIgT1IgY2M6IlRlc3QiIE9SIGZyb206IlRlc3QiIE9SIHN1YmplY3Q6IlRlc3QiIE9SICJUZXN0IiBPUiAiVGVzdCIAAQFMTQMwLTk5AAEOAQEBAQ==
z-push-dev  | 21/08/2025 12:52:04 [   39] [ INFO] [test@example.com] User-agent: 'Apple-iPhone17C1/2207.100'
z-push-dev  | 21/08/2025 12:52:04 [   39] [FATAL] [test@example.com] Exception: (StatusException) - BackendDiff->Fetch('%s','%s'): Error, unable retrieve message from backend
z-push-dev  | 21/08/2025 12:52:04 [   39] [FATAL] [test@example.com] Exception: (StatusException) - BackendDiff->Fetch('%s','%s'): Error, unable retrieve message from backend

and in your patched version (9575599) I see things like:

z-push-dev  | - - test@example.com 21/Aug/2025:12:55:02 +0100 "POST /Microsoft-Server-ActiveSync" 200
z-push-dev  | 21/08/2025 12:55:03 [   26] [WARN] [test@example.com] BackendIMAP->StatMessage('ef6c13b0','929'): Failed to retrieve overview: 
z-push-dev  | 21/08/2025 12:55:03 [   26] [WARN] [test@example.com] BackendIMAP->StatMessage('ef6c13b0','929'): Failed to retrieve overview: 
z-push-dev  | 21/08/2025 12:55:03 [   26] [ INFO] [test@example.com] StatusException: BackendDiff->Fetch('%s','%s'): Error, unable retrieve message from backend - code: 8 - file: /usr/share/z-push/lib/default/diffbackend/diffbackend.php:134
z-push-dev  | 21/08/2025 12:55:03 [   26] [ INFO] [test@example.com] cmd='ItemOperations' memory='1.64 MiB/2.00 MiB' time='0.14s' devType='iPhone' devId='5s4d7t6v050kju9s2rfdiep5ma' getUser='test@example.com' from='10.0.0.141' idle='0s' version='2.7.3
z-push-dev  | ' method='POST' httpcode='200'
z-push-dev  | - - test@example.com 21/Aug/2025:12:55:03 +0100 "POST /Microsoft-Server-ActiveSync" 200
z-push-dev  | 21/08/2025 12:55:11 [   38] [WARN] [test@example.com] BackendIMAP->StatMessage('ef6c13b0','940'): Failed to retrieve overview: 
z-push-dev  | 21/08/2025 12:55:11 [   38] [WARN] [test@example.com] BackendIMAP->StatMessage('ef6c13b0','940'): Failed to retrieve overview: 
z-push-dev  | 21/08/2025 12:55:11 [   38] [ INFO] [test@example.com] StatusException: BackendDiff->Fetch('%s','%s'): Error, unable retrieve message from backend - code: 8 - file: /usr/share/z-push/lib/default/diffbackend/diffbackend.php:134
z-push-dev  | 21/08/2025 12:55:11 [   38] [ INFO] [test@example.com] cmd='ItemOperations' memory='1.64 MiB/2.00 MiB' time='0.15s' devType='iPhone' devId='5s4d7t6v050kju9s2rfdiep5ma' getUser='test@example.com' from='10.0.0.141' idle='0s' version='2.7.3
z-push-dev  | ' method='POST' httpcode='200'

Also in 9575599 I get some odd search results where it looks like the same result is getting returned over and over (as in multiple copies of the same result). Please let me know if I can provide more useful diagnostic information. It's easy for me to switch between versions as I have this all in a Docker container. Test device is an iPhone 16 Pro on 18.6.2.

[mschering]

Is 'ef6c13b0' the correct folder name?

[kour1er]

so 'ef6c13b0' is the Inbox and that isn't where the search results are coming from