I think ctail misses a lot of certificates. It optimistically tries to download 1k certs, but doesn't check if server actually returned the requested number of certificates.
Here is example it requested 1k certs, but only got 8 results. The culprit is, the server doesn't have to honor the end position.
(debug message was inserted by me into the code, fmt.Printf("requested: %d downloaded: %d\n", stopIdx-index+1, len(entries.Entries)))
[*] Downloading JSON from https://ct2025-b.trustasia.com/log2025b/ct/v1/get-entries?start=255902840&end=255903839
requested: 1000 downloaded: 8
PS In my experience, no CTLogs server supports retrieving 1k certs in one request. It is usually in 32-256 range.
I think ctail misses a lot of certificates. It optimistically tries to download 1k certs, but doesn't check if server actually returned the requested number of certificates.
Here is example it requested 1k certs, but only got 8 results. The culprit is, the server doesn't have to honor the end position.
(debug message was inserted by me into the code,
fmt.Printf("requested: %d downloaded: %d\n", stopIdx-index+1, len(entries.Entries)))PS In my experience, no CTLogs server supports retrieving 1k certs in one request. It is usually in 32-256 range.