Preserve calendar intent in versioned time contracts

docs/contracts.md

@@ -5,6 +5,78 @@
 FeedSpec and MarketDataSpec define column mappings, frequency, timestamps, calendars, storage, and
 provenance. Consumers validate the contract before interpreting a table.
 
+### Calendar selection and temporal contract versions
+
+New time-aware consumers use `time_contract_version=2`. This temporal version is independent of
+the artifact's `version`. It preserves an omitted calendar separately from an explicit `None`:
+
+| Declaration | Effective calendar |
+|---|---|
+| Explicit function argument | That name, or no calendar when `None` |
+| Argument omitted, metadata declares a calendar | The metadata value, including explicit `None` in version 2 |
+| Neither declares a calendar | `NYSE` |
+
+Use the same `calendar` keyword in functions, configs and serialized metadata. Consumers use
+`UNSET_CALENDAR` as their omitted-argument default and call `FeedSpec.resolve_calendar`. Ordinary
+callers do not need the marker. `None` means observation order/row counts; a named `24/7` calendar
+still defines expected times and can detect missing observations.
+
+```python
+from ml4t.specs import FeedSpec, UNSET_CALENDAR
+
+metadata = FeedSpec(time_contract_version=2, calendar="24/7")
+assert metadata.resolve_calendar() == "24/7"
+assert metadata.resolve_calendar(calendar=None) is None
+assert FeedSpec(time_contract_version=2).resolve_calendar() == "NYSE"
+
+disabled = metadata.with_overrides(calendar=None)
+assert disabled.to_dict()["calendar"] is None
+inherited = disabled.with_overrides(calendar=UNSET_CALENDAR)
+assert "calendar" not in inherited.to_dict()
+```
+
+Serialize with `FeedSpec.to_dict()`, `MarketDataSemantics.to_dict()` or `MarketDataSpec.to_dict()`
+before using the shared JSON/YAML writer. These methods omit an undeclared calendar and retain
+explicit null. `dataclasses.asdict` is an in-memory projection, not the version-2 wire format.
+Copied/replaced version-2 objects and nested artifact-to-feed projections preserve declaration
+presence. Consumers must resolve the calendar before passing it to a provider; the unset marker is
+not a provider name. Specs validates names structurally; consumers validate provider availability
+and the requested schedule without substituting a different exchange.
+
+Constructors and payloads without a temporal version retain version 1. Its null metadata means
+unspecified and its `with_overrides(calendar=None)` retains the previous calendar. Explicit arguments
+to the new resolver still take precedence, including `None`. Migrating a legacy payload requires
+choosing whether an old null meant no declaration or an intentional opt-out: remove the key for
+inheritance, or retain null for disable, then set `time_contract_version=2`. Do not relabel legacy
+payloads automatically. Non-calendar null overrides retain the legacy no-override convention.
+
+### Fixed-bar grid declarations
+
+Version 2 adds explicit `bar_grid_anchor`, `bar_label` and `partial_bar_policy` fields. They are
+optional metadata, without inferred defaults. A consumer performing grid validation must require
+the fields it needs along with declared `data_frequency`, timestamp semantics and timezone for
+naive timestamps. Specs does not infer a frequency or compute market schedules.
+
+| Field | Supported declaration and meaning |
+|---|---|
+| `bar_grid_anchor` | `session_open`: start the grid at each authoritative tradable interval's open, including a reopen after a break. A continuous daily schedule uses its declared daily boundary. |
+| `bar_label` | `left` or `right`: which boundary the bar timestamp names. `bar_close` timestamp semantics requires `right`; `event_time` alone does not choose a side. |
+| `partial_bar_policy` | `include` or `exclude`: retain or omit a final interval shorter than the declared duration at a break or close. |
+
+Session labels are identifiers, not intraday bar timestamps; they cannot declare an intraday grid.
+Calendar closures and breaks come from the selected schedule, not missing observations. The metadata
+does not authorize filling prices, changing row counts, or treating event-count bars as fixed-time
+bars. Validation of invalid declarations occurs before a contract is returned or persisted.
+
+`WindowUnit` distinguishes `observation` (input rows), `expected_bar` (declared grid steps),
+`session` (authoritative trading sessions) and `elapsed` (an explicit duration). It performs no
+conversion. Consumers must retain the selected unit and must not reinterpret, for example, a
+session-count window as a count of observed dates when a scheduled session is absent. Annualization
+is a separately declared numerical convention; a calendar-free workflow does not imply NYSE scaling.
+
+The temporal contract does not change the defaults of already released consumers. Each library must
+adopt the resolver and serializer and qualify its public workflows before claiming conformance.
+
 ## Lifecycle and events
 
 LifecycleContract defines callback ordering, information availability, intent permissions, and

src/ml4t/specs/__init__.py

@@ -82,6 +82,14 @@
     MarketDataSpec,
     TimestampSemantics,
 )
+from .time import (
+    UNSET_CALENDAR,
+    BarGridAnchor,
+    BarLabel,
+    CalendarUnset,
+    PartialBarPolicy,
+    WindowUnit,
+)
 
 __version__ = "0.1.5"
 
@@ -92,9 +100,12 @@
     "ArtifactStorage",
     "AssetTarget",
     "BarPathPolicy",
+    "BarGridAnchor",
+    "BarLabel",
     "BarPayload",
     "CallbackCardinality",
     "CallbackExceptionSemantics",
+    "CalendarUnset",
     "CanonicalChildOrderIntent",
     "CanonicalTargetIntent",
     "EvaluationMode",
@@ -126,6 +137,7 @@
     "OrderParameters",
     "OrderSide",
     "OrderType",
+    "PartialBarPolicy",
     "PositionActionType",
     "PositionRuleDefinition",
     "PositionRulePolicy",
@@ -144,6 +156,8 @@
     "TimestampSemantics",
     "TradePayload",
     "UnsupportedLifecycleVersionError",
+    "UNSET_CALENDAR",
+    "WindowUnit",
     "canonical_intent_fixture",
     "compare_child_intents",
     "compare_target_intents",

src/ml4t/specs/market_data.py

@@ -3,11 +3,28 @@
 from __future__ import annotations
 
 from collections.abc import Callable, Mapping, Sequence
-from dataclasses import dataclass, field, replace
+from dataclasses import asdict, dataclass, field, replace
 from enum import StrEnum
 from typing import Any, ClassVar
 
-from .base import ArtifactKind, ArtifactProvenance, ArtifactSpec, ArtifactStorage, optional_str
+from .base import (
+    ArtifactKind,
+    ArtifactProvenance,
+    ArtifactSpec,
+    ArtifactStorage,
+    optional_str,
+    serialize_artifact_value,
+)
+from .time import (
+    UNSET_CALENDAR,
+    BarGridAnchor,
+    BarLabel,
+    CalendarUnset,
+    PartialBarPolicy,
+    _normalize_time_contract,
+    _resolve_calendar,
+    _serialize_time_contract,
+)
 
 
 class TimestampSemantics(StrEnum):
@@ -44,6 +61,10 @@ class FeedSpec:
         "bar_type": ("bar_type",),
         "timestamp_semantics": ("timestamp_semantics",),
         "session_start_time": ("session_start_time",),
+        "time_contract_version": ("time_contract_version",),
+        "bar_grid_anchor": ("bar_grid_anchor",),
+        "bar_label": ("bar_label",),
+        "partial_bar_policy": ("partial_bar_policy",),
     }
     _SEMANTICS_FIELDS: ClassVar[frozenset[str]] = frozenset(
         {
@@ -53,6 +74,10 @@ class FeedSpec:
             "bar_type",
             "timestamp_semantics",
             "session_start_time",
+            "time_contract_version",
+            "bar_grid_anchor",
+            "bar_label",
+            "partial_bar_policy",
         }
     )
 
@@ -74,19 +99,32 @@ class FeedSpec:
     mid_col: str | None = None
     bid_size_col: str | None = None
     ask_size_col: str | None = None
-    calendar: str | None = None
+    calendar: str | None | CalendarUnset = UNSET_CALENDAR
     timezone: str | None = None
     data_frequency: Any | None = None
     bar_type: str | None = None
     timestamp_semantics: TimestampSemantics | str | None = None
     session_start_time: str | None = None
+    time_contract_version: int = field(default=1, kw_only=True)
+    bar_grid_anchor: BarGridAnchor | str | None = field(default=None, kw_only=True)
+    bar_label: BarLabel | str | None = field(default=None, kw_only=True)
+    partial_bar_policy: PartialBarPolicy | str | None = field(default=None, kw_only=True)
 
     def __post_init__(self) -> None:
         semantics = self.timestamp_semantics
-        if semantics is None:
-            return
-        if not isinstance(semantics, TimestampSemantics):
+        if semantics is not None and not isinstance(semantics, TimestampSemantics):
             object.__setattr__(self, "timestamp_semantics", TimestampSemantics(str(semantics)))
+        _normalize_time_contract(self)
+
+    def resolve_calendar(
+        self, *, calendar: str | None | CalendarUnset = UNSET_CALENDAR
+    ) -> str | None:
+        """Resolve explicit selection, declared metadata, then NYSE, preserving None."""
+        return _resolve_calendar(self.calendar, self.time_contract_version, calendar)
+
+    def to_dict(self) -> dict[str, Any]:
+        """Serialize temporal declaration presence without leaking the unset marker."""
+        return serialize_artifact_value(_serialize_time_contract(asdict(self)))
 
     @classmethod
     def from_mapping(cls, mapping: Mapping[str, Any]) -> FeedSpec:
@@ -144,8 +182,15 @@ def from_any(cls, value: Any | None) -> FeedSpec:
         return cls.from_object(value)
 
     def with_overrides(self, **overrides: Any) -> FeedSpec:
-        """Return a new spec with explicit non-null overrides applied."""
-        updates = {key: value for key, value in overrides.items() if value is not None}
+        """Apply overrides; version 2 also accepts explicit calendar=None to disable it."""
+        version = overrides.get("time_contract_version")
+        if version is None:
+            version = self.time_contract_version
+        updates = {
+            key: value
+            for key, value in overrides.items()
+            if value is not None or (key == "calendar" and version == 2)
+        }
         return replace(self, **updates) if updates else self
 
     def resolve(self, columns: Sequence[str], entity_candidates: Sequence[str]) -> FeedSpec:
@@ -309,17 +354,31 @@ class MarketDataSemantics:
     """Temporal and execution semantics for tradable market data."""
 
     data_frequency: str | None = None
-    calendar: str | None = None
+    calendar: str | None | CalendarUnset = UNSET_CALENDAR
     timezone: str | None = None
     timestamp_semantics: TimestampSemantics | str | None = None
     session_start_time: str | None = None
     bar_type: str | None = None
+    time_contract_version: int = field(default=1, kw_only=True)
+    bar_grid_anchor: BarGridAnchor | str | None = field(default=None, kw_only=True)
+    bar_label: BarLabel | str | None = field(default=None, kw_only=True)
+    partial_bar_policy: PartialBarPolicy | str | None = field(default=None, kw_only=True)
 
     def __post_init__(self) -> None:
         semantics = self.timestamp_semantics
-        if semantics is None or isinstance(semantics, TimestampSemantics):
-            return
-        object.__setattr__(self, "timestamp_semantics", TimestampSemantics(str(semantics)))
+        if semantics is not None and not isinstance(semantics, TimestampSemantics):
+            object.__setattr__(self, "timestamp_semantics", TimestampSemantics(str(semantics)))
+        _normalize_time_contract(self)
+
+    def resolve_calendar(
+        self, *, calendar: str | None | CalendarUnset = UNSET_CALENDAR
+    ) -> str | None:
+        """Resolve explicit selection, declared metadata, then NYSE, preserving None."""
+        return _resolve_calendar(self.calendar, self.time_contract_version, calendar)
+
+    def to_dict(self) -> dict[str, Any]:
+        """Serialize temporal declaration presence without leaking the unset marker."""
+        return serialize_artifact_value(_serialize_time_contract(asdict(self)))
 
     @classmethod
     def from_mapping(cls, mapping: Mapping[str, Any] | None) -> MarketDataSemantics:
@@ -329,11 +388,19 @@ def from_mapping(cls, mapping: Mapping[str, Any] | None) -> MarketDataSemantics:
             raise TypeError("semantics must be a mapping or None")
         return cls(
             data_frequency=optional_str(mapping.get("data_frequency")),
-            calendar=optional_str(mapping.get("calendar")),
+            calendar=(
+                mapping.get("calendar", UNSET_CALENDAR)
+                if mapping.get("time_contract_version", 1) != 1
+                else optional_str(mapping.get("calendar"))
+            ),
             timezone=optional_str(mapping.get("timezone")),
             timestamp_semantics=mapping.get("timestamp_semantics"),
             session_start_time=optional_str(mapping.get("session_start_time")),
             bar_type=optional_str(mapping.get("bar_type")),
+            time_contract_version=mapping.get("time_contract_version", 1),
+            bar_grid_anchor=mapping.get("bar_grid_anchor"),
+            bar_label=mapping.get("bar_label"),
+            partial_bar_policy=mapping.get("partial_bar_policy"),
         )
 
 
@@ -345,6 +412,12 @@ class MarketDataSpec(ArtifactSpec):
     schema: MarketDataSchema = field(default_factory=MarketDataSchema)
     semantics: MarketDataSemantics = field(default_factory=MarketDataSemantics)
 
+    def to_dict(self) -> dict[str, Any]:
+        """Serialize the artifact with its temporal contract's omission semantics."""
+        payload = ArtifactSpec.to_dict(self)
+        payload["semantics"] = self.semantics.to_dict()
+        return payload
+
     @classmethod
     def from_mapping(cls, mapping: Mapping[str, Any]) -> MarketDataSpec:
         if not isinstance(mapping, Mapping):