do not mutate caller-supplied metadata in Parameter and Dimension - #64
do not mutate caller-supplied metadata in Parameter and Dimension#64jmccreight wants to merge 1 commit into
Conversation
Strict-mode construction aliased the per-entry sub-dict of the supplied metadata (self.meta = meta[name]), so creation-time bounded maximum resolution (Parameter) and the size setter (Dimension) wrote through to the caller's dict. A second Parameters instance built from the same metadata then read a resolved numeric 'maximum' where Parameters.add expects a dimension name, raising on bounded parameters. Copy the sub-dict at construction; instances keep their resolved values, shared metadata stays pristine. Adds a regression test with two instances sharing one MetaData dict.
|
Thanks for finding this — the root cause you identified is correct. I've pushed an alternative fix on branch The approach uses two separate
Since the caller's dict is never passed into the internal machinery, the shallow copies in This also prevents a subtler issue: All 302 tests pass. See: development...fix/deepcopy-metadata-isolation |
|
Agreed. See my comment about keeping the test and then you can close the PR. Thanks! |
…#65) Use two separate deep copies in Parameters.__init__: 1. __full_metadata: pristine copy, never mutated by the instance. Used by create_subset() to build child Parameters instances that need the original unresolved metadata (dimension names in bounded 'maximum' fields, original dimension sizes, etc.). 2. _working_metadata: separate working copy that self.metadata, self.__dimensions, and all Parameter instances are free to mutate (bounded resolution, dimension size changes, etc.). This supersedes the need for shallow copies in Parameter.__init__ and Dimension.__init__ since the caller's dict is never passed into the internal machinery. The approach also prevents create_subset() from crashing due to mutated __full_metadata, which would happen if __full_metadata and self.metadata shared sub-dicts. Addresses the same root cause as PR #64 (caller-supplied metadata being mutated across Parameters instances) but at a higher level, providing stronger guarantees.
Problem
Strict-mode
ParameterandDimensionconstruction alias theper-entry sub-dict of the caller-supplied metadata
(
self.meta = meta[name]). Two code paths then write through thatalias into the caller's dict:
Parameter.__init__: the creation-time bounded-maximum resolutionwrites
meta['bounded_dimension_name']and overwritesmeta['maximum'](dimension name -> numeric size).Dimension: the size setter writesmeta['size'].The first
Parametersinstance built from a metadata dict works andpoisons it; any later instance sharing that dict crashes in
Parameters.add()on bounded parameters, which readsmetadata[name]['maximum']expecting a dimension name:Repro (second iteration raises on current development):
Found integration-testing pywatershed against development ahead of the
next release (as discussed): pywatershed holds a module-level
MetaData().metadataand builds multipleParametersfrom it; 2 of 6DomainSubsettests failed this way.Fix
Copy the metadata sub-dict at strict-mode construction in both
classes (
self.meta = dict(meta[name])). Instances keep theirresolved values (
bounded_dimension_name, numericmaximum,size); the caller's dict is never modified. The non-strict pathsare untouched -- they take ad-hoc dicts and internal code deliberately
aliases
metabetween objects (e.g.ParamDimensionsfrom globaldimensions).
Testing
Parametersinstances sharing oneMetaDatadict, adding a bounded parameter to each; asserts theinstance sees the resolved bound and the shared dict is unchanged.
tests/funcsuite: 303 passed.test_domain_subset.pygoes 4/6 -> 6/6against this branch (packaging 26.3 environment).