When patches are placed side by side, an attribute the members disagree on is currently resolved by conflict: "raise" refuses, "keep_first" keeps one value, "drop" throws them all away. All three lose information that the output could carry, because concatenation preserves the distinction between its members — each member still occupies a known range of the concatenated dimension.
The proposal is a fourth policy, conflict="promote": an attribute whose members differ becomes a non-dimensional coordinate along the concatenated dimension, holding each member's value over that member's samples.
a = patch.update_attrs(quality="good")
b = other.update_attrs(quality="poor")
out = dc.spool([a, b]).concatenate(time=None, conflict="promote")
out[0].get_coord("quality") # "good" over a's samples, "poor" over b's
out[0].attrs.get("quality") # None — it is a coordinate now, not an attr
Why
keep_first fabricates provenance: the half of the array that came from b comes out claiming a's value. drop deletes the answer entirely. Promotion is the only resolution that keeps the output honest about all of its data, and it is exactly the information a concatenation has and a merge does not.
Round trip
The acceptance test, from the design note:
Collapse a constant coordinate to an attr on each of N patches, concatenate them, and the original coordinate comes back exactly — including empty values.
patches = [p.update_attrs(quality=q) for p, q in zip(patches, values)]
out = dc.spool(patches).concatenate(time=None, conflict="promote")
assert np.all(out[0].get_coord("quality").values == expected)
Empty values must survive as empty rather than being filled from a neighbour — see #983, which established that a missing value is a value where patches are partitioned.
Scope decisions
Implementation notes
The planner is the harder half. build_concat_plan/build_chunk_plan describe an output's attrs as row values in the catalog, and a promoted attribute is no longer an attr of the output at all — the catalog has to describe it as a coordinate whose values it knows only per member. derived_catalog already records per-member envelope columns, so the member rows carry what is needed; what is missing is a way for a plan row to say "this name is a coordinate here, not an attr" so that get_contents() and select() do not keep offering it as a scalar.
Assembly is the easier half: concatenate_planned knows each member's value and each member's extent along the dimension, so it can build the coordinate directly alongside the existing rider-joining logic in _concatenate_group.
Worth deciding during implementation: whether a promoted coordinate is worth the memory (one entry per sample, so a string array the length of the concatenated dimension), or whether it should use a run-length/segmented coordinate — CoordSegmented already exists for the non-uniform case.
Follow-up to #983 and #984; the design note is .scratch/kind-empty-rule.md.
When patches are placed side by side, an attribute the members disagree on is currently resolved by
conflict:"raise"refuses,"keep_first"keeps one value,"drop"throws them all away. All three lose information that the output could carry, because concatenation preserves the distinction between its members — each member still occupies a known range of the concatenated dimension.The proposal is a fourth policy,
conflict="promote": an attribute whose members differ becomes a non-dimensional coordinate along the concatenated dimension, holding each member's value over that member's samples.Why
keep_firstfabricates provenance: the half of the array that came frombcomes out claiminga's value.dropdeletes the answer entirely. Promotion is the only resolution that keeps the output honest about all of its data, and it is exactly the information a concatenation has and a merge does not.Round trip
The acceptance test, from the design note:
Empty values must survive as empty rather than being filled from a neighbour — see #983, which established that a missing value is a value where patches are partitioned.
Scope decisions
conflictpolices today.conflictvalue. The existing three keep their meanings.Implementation notes
The planner is the harder half.
build_concat_plan/build_chunk_plandescribe an output's attrs as row values in the catalog, and a promoted attribute is no longer an attr of the output at all — the catalog has to describe it as a coordinate whose values it knows only per member.derived_catalogalready records per-member envelope columns, so the member rows carry what is needed; what is missing is a way for a plan row to say "this name is a coordinate here, not an attr" so thatget_contents()andselect()do not keep offering it as a scalar.Assembly is the easier half:
concatenate_plannedknows each member's value and each member's extent along the dimension, so it can build the coordinate directly alongside the existing rider-joining logic in_concatenate_group.Worth deciding during implementation: whether a promoted coordinate is worth the memory (one entry per sample, so a string array the length of the concatenated dimension), or whether it should use a run-length/segmented coordinate —
CoordSegmentedalready exists for the non-uniform case.Follow-up to #983 and #984; the design note is
.scratch/kind-empty-rule.md.