Problem
Date-scoped listing and search first select the configured date-partitioned S3 prefixes, but then reject objects whose S3 LastModified timestamp falls outside the requested window.
The affected checks are in:
S3EmailInbox._list_objects_for_rangesS3EmailInbox.view_page_by_date_ranges
This hides historically migrated messages. For example, an object stored at:
emails/2024/03/10/message-id
may have an S3 LastModified timestamp in 2026 because that is when it was copied into the bucket. A search for 2024-03-10 lists the correct prefix and then incorrectly filters the object out.
Desired behavior
Use the configured date partition/key as the authoritative date for membership in a requested date range. Do not discard a date-partitioned object merely because its S3 LastModified value reflects a later migration or restore.
Define an explicit fallback for objects that cannot be assigned a date from the configured key format. The fallback must preserve bounded scans and must not silently reintroduce LastModified as the received date for partitioned objects.
The displayed/sorted received_at value should also avoid presenting a migration timestamp as the email's received timestamp. Prefer a reliably parsed message Date header when available, with a documented fallback to the partition date.
Acceptance criteria
- An object under
emails/2024/03/10/ with LastModified in 2026 appears in a 2024-03-10 query.
- That object does not appear in a 2026 query solely because of its
LastModified timestamp.
- Paginated view, exact search, and fuzzy search behave consistently.
- Duplicate objects are still suppressed when prefixes overlap.
- Behavior for flat/non-date-partitioned keys is documented and tested.
- Regression tests cover migrated objects whose key date and
LastModified date differ.
Security/operations impact
This is a data-correctness issue for client archives: a successful search currently does not guarantee that all messages stored in the selected historical date partitions were considered.
Problem
Date-scoped listing and search first select the configured date-partitioned S3 prefixes, but then reject objects whose S3
LastModifiedtimestamp falls outside the requested window.The affected checks are in:
S3EmailInbox._list_objects_for_rangesS3EmailInbox.view_page_by_date_rangesThis hides historically migrated messages. For example, an object stored at:
may have an S3
LastModifiedtimestamp in 2026 because that is when it was copied into the bucket. A search for 2024-03-10 lists the correct prefix and then incorrectly filters the object out.Desired behavior
Use the configured date partition/key as the authoritative date for membership in a requested date range. Do not discard a date-partitioned object merely because its S3
LastModifiedvalue reflects a later migration or restore.Define an explicit fallback for objects that cannot be assigned a date from the configured key format. The fallback must preserve bounded scans and must not silently reintroduce
LastModifiedas the received date for partitioned objects.The displayed/sorted
received_atvalue should also avoid presenting a migration timestamp as the email's received timestamp. Prefer a reliably parsed messageDateheader when available, with a documented fallback to the partition date.Acceptance criteria
emails/2024/03/10/withLastModifiedin 2026 appears in a 2024-03-10 query.LastModifiedtimestamp.LastModifieddate differ.Security/operations impact
This is a data-correctness issue for client archives: a successful search currently does not guarantee that all messages stored in the selected historical date partitions were considered.