Problem
A coordinate's fingerprint depends on how its scalars were spelled, not only on the values it holds. Two coordinates which compare equal, hold identical values and share a dtype can therefore carry different identities.
import numpy as np
from dascore.core.coords import get_coord
# 1. python type of the scalar
a = get_coord(start=0, stop=10, step=1.0) # int start, float64 dtype
b = get_coord(start=0.0, stop=10.0, step=1.0)
a == b # True
a.dtype == b.dtype # True
np.array_equal(a.values, b.values) # True
a.fingerprint() == b.fingerprint() # False <-- different identity
# 2. precision of a time step
t0 = np.datetime64("2020-01-01", "ns")
c = get_coord(start=t0, stop=t0 + np.timedelta64(400, "ms"),
step=np.timedelta64(4, "ms"))
d = get_coord(start=t0, stop=t0 + np.timedelta64(400_000_000, "ns"),
step=np.timedelta64(4_000_000, "ns"))
c == d # True
np.array_equal(c.values, d.values) # True
c.fingerprint() == d.fingerprint() # False <-- different identity
Why it matters
CoordRange.validate_start_stop_step_len (dascore/core/coords.py:1516) derives dtype from start + step but leaves start, stop and step as whatever objects the caller passed. _fingerprint_components (coords.py:1591) then hashes those scalars through _hash_scalar, which encodes the scalar's own type and precision. _get_fingerprintable_coord (coords.py:615) already normalizes units before hashing, precisely so metres and centimetres do not produce different identities — scalar type and time precision are the same class of difference and are not normalized.
Consequences:
CoordSummary conforms its values with ensure_consistent_dtype (coords.py:97), so a round-trip changes the identity: c.fingerprint() != c.to_summary().to_coord().fingerprint().
- The index stores that fingerprint as a coordinate's
def_key (dascore/io/index/ingest.py:109), so equal coordinates can be stored as two definitions rather than deduplicating, and an identity match can miss.
Real patch coordinates from the readers and get_example_patch are unaffected — they arrive as datetime64 ns and float arrays — so this is latent rather than actively breaking anything today. It surfaced while building the summary-level join for #972, where a prediction's identity must equal what re-ingesting the assembled patch records.
Fix, and why it is not a one-liner
Conforming the scalars in the validator, exactly as CoordSummary does, unifies both cases:
dtype = np.asarray(start + step).dtype
values["dtype"] = dtype
for name in ("start", "stop", "step"):
values[name] = ensure_consistent_dtype(values[name], name, dtype)
Tried on dev: it makes the fingerprints agree, but
- 4 tests fail —
tests/test_core/test_coords.py::TestCoordRange::test_len_one_array_like_start_no_deprecation and three in tests/test_io/test_prodml/test_prodml_write.py::TestProdMLWriteTimePrecision, which depend on sub-nanosecond and out-of-range datetimes surviving unconverted;
- it changes stored fingerprints, so
def_keys written by an older version no longer match the ones a new version computes for the same coordinate. Existing indexes would need rebuilding, or the change needs an index version bump.
An alternative worth weighing is normalizing inside _hash_scalar instead of at construction: the stored scalars keep their spelling, only the hash canonicalizes. That avoids the prodml fallout but still changes fingerprints, so the migration question is the same.
Either way this deserves its own decision about index compatibility rather than riding along with #972.
Problem
A coordinate's fingerprint depends on how its scalars were spelled, not only on the values it holds. Two coordinates which compare equal, hold identical values and share a dtype can therefore carry different identities.
Why it matters
CoordRange.validate_start_stop_step_len(dascore/core/coords.py:1516) derivesdtypefromstart + stepbut leavesstart,stopandstepas whatever objects the caller passed._fingerprint_components(coords.py:1591) then hashes those scalars through_hash_scalar, which encodes the scalar's own type and precision._get_fingerprintable_coord(coords.py:615) already normalizes units before hashing, precisely so metres and centimetres do not produce different identities — scalar type and time precision are the same class of difference and are not normalized.Consequences:
CoordSummaryconforms its values withensure_consistent_dtype(coords.py:97), so a round-trip changes the identity:c.fingerprint() != c.to_summary().to_coord().fingerprint().def_key(dascore/io/index/ingest.py:109), so equal coordinates can be stored as two definitions rather than deduplicating, and an identity match can miss.Real patch coordinates from the readers and
get_example_patchare unaffected — they arrive as datetime64 ns and float arrays — so this is latent rather than actively breaking anything today. It surfaced while building the summary-level join for #972, where a prediction's identity must equal what re-ingesting the assembled patch records.Fix, and why it is not a one-liner
Conforming the scalars in the validator, exactly as
CoordSummarydoes, unifies both cases:Tried on
dev: it makes the fingerprints agree, buttests/test_core/test_coords.py::TestCoordRange::test_len_one_array_like_start_no_deprecationand three intests/test_io/test_prodml/test_prodml_write.py::TestProdMLWriteTimePrecision, which depend on sub-nanosecond and out-of-range datetimes surviving unconverted;def_keys written by an older version no longer match the ones a new version computes for the same coordinate. Existing indexes would need rebuilding, or the change needs an index version bump.An alternative worth weighing is normalizing inside
_hash_scalarinstead of at construction: the stored scalars keep their spelling, only the hash canonicalizes. That avoids the prodml fallout but still changes fingerprints, so the migration question is the same.Either way this deserves its own decision about index compatibility rather than riding along with #972.