Skip to content

order_by treats a naive datetime object as local time, but a naive datetime string as UTC #1342

Description

@shashvat-singham

Describe the bug

order_by converts a datetime to microseconds differently depending on whether you pass it as a string or as a datetime object, when neither carries a timezone. The same wall-clock time gets two different order values, and the gap is whatever the client machine's UTC offset happens to be.

On a machine in IST (UTC+5:30):

>>> from datetime import datetime, timezone
>>> from qdrant_client.local.order_by import to_order_value

>>> to_order_value("2024-06-15 12:30:45")                          # naive string
1718454645000000
>>> to_order_value(datetime(2024, 6, 15, 12, 30, 45))              # naive object
1718434845000000
>>> to_order_value(datetime(2024, 6, 15, 12, 30, 45, tzinfo=timezone.utc))
1718454645000000

The string and the aware-UTC object agree. The naive object is 19800 seconds off — exactly the machine's offset. Run the same code in UTC and the discrepancy disappears; run it in UTC-8 and it flips sign. So local-mode ordering depends on where the client runs.

Cause

The two paths make opposite assumptions about what "no timezone" means.

datetime_utils.parse, used for strings, explicitly assumes UTC — matching qdrant core, whose accepted formats treat a missing offset as UTC:

if dt.tzinfo is None:
    # Assume UTC if no timezone is provided
    dt = dt.replace(tzinfo=timezone.utc)

to_order_value passes a datetime object straight to datetime_to_microseconds, which calls dt.timestamp(). For a naive datetime, timestamp() interprets it as local time, not UTC.

Expected behavior

A naive datetime object should be interpreted the same way as a naive datetime string — as UTC — so that ordering is reproducible regardless of the client's timezone and consistent with the server.

Concretely, datetime_to_microseconds attaching timezone.utc to a naive input (rather than letting timestamp() assume local time) would line the two paths up.

Note

#1310 is currently rewriting datetime_to_microseconds to fix a separate float-precision problem, and deliberately preserves the naive-input behaviour via dt.astimezone() to match what timestamp() did. I raised this there and said I'd file it separately so it's tracked on its own — the two are independent, and whichever way #1310 goes this question still needs an answer.

Environment

  • qdrant-client @ dev (f003e6c)
  • Python 3.11.9, Windows, machine timezone UTC+5:30

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions