Adopt Egeria 6.3 boxed properties; raise on failed bearer token requests

EGERIA_OPEN_ISSUES.md

@@ -10,7 +10,7 @@ It replaces the 2026-10-05 snapshot `docs/egeria-open-bugs-2026-10-05.md` (delet
 
 | Group | Issues |
 |---|---|
-| **Open on the Egeria side** (full entries below) | ISSUE-166, ISSUE-147, ISSUE-143, ISSUE-130, ISSUE-128, ISSUE-108 |
+| **Open on the Egeria side** (full entries below) | ISSUE-147, ISSUE-143, ISSUE-130, ISSUE-128, ISSUE-108 |
 | **Awaiting re-test:** a fix landed (mostly Egeria `main`, odpi/egeria#9363, merged 2026-10-06), not yet verified live | ISSUE-125, ISSUE-124, ISSUE-117, ISSUE-112, ISSUE-95, ISSUE-90, ISSUE-89, ISSUE-79, ISSUE-38 |
 | **Egeria-side, but tracked as pyegeria items** (a pyegeria change waits on Egeria) | ISSUE-146, ISSUE-132 |
 
@@ -37,7 +37,9 @@ It replaces the 2026-10-05 snapshot `docs/egeria-open-bugs-2026-10-05.md` (delet
 
 ### ISSUE-166: Security annotation properties are Java primitives, so an unset CVSS score, exploit/patch flag or posture count reads back as 0 / false
 
-**Layer:** Egeria Server (type model / properties classes) · **Status:** open, question for the Egeria team · **Found:** 2026-10-26, while adding the 6.3 security annotation types to pyegeria (odpi/egeria-workspaces#702 hand-off, step 3; PR #495).
+**Layer:** Egeria Server (type model / properties classes) · **Status:** fixed (Egeria 6.3, odpi/egeria#9389) · **Found:** 2026-10-26, while adding the 6.3 security annotation types to pyegeria (odpi/egeria-workspaces#702 hand-off, step 3; PR #495).
+
+**FIXED 2026-10-27: odpi/egeria#9389, merged as 12b1aaae.** Verified live the same day on egeria-platform c2ca054c (throwaway VulnerabilityAnnotation ed152edb, deleted, NotFound by GUID): unset cvssScore/exploitAvailable/patchAvailable were omitted; a merge update that left out 8.6 / true / false kept them; an explicit false came back. The pyegeria workarounds below are removed on branch `feature/boxed-properties-audit`. It boxes 217 number and flag fields across 86 property beans, not only the security ones. An unset value is no longer stored and is left out of the response, and a merge update keeps a value it does not mention. Defaults move to `OpenMetadataProperty.defaultValue`, published in the Core Content Pack as `additionalProperties.defaultValue` on each property's valid metadata value. `allowsDuplicateValues` now defaults to true everywhere, including SchemaAttribute (was false). The PR does not migrate stored data: elements written by earlier servers keep the 0 / false they were stored with. **pyegeria:** fixes that work with either server are on branch `feature/boxed-properties-audit`. The client-side workarounds stay until the PR merges and pyegeria targets that server.
 
 **What happens:** in `SecurityFindingAnnotationProperties`, `VulnerabilityAnnotationProperties` and `SecurityPostureAnnotationProperties` (`frameworks/open-metadata-framework/.../properties/surveyreports/`), these fields are Java primitives with a 0 / false default:
 

PYEGERIA_ISSUES.md

@@ -1835,7 +1835,9 @@ enough to track there too).
 
 ### ISSUE-166: Security annotation properties are Java primitives, so an unset CVSS score, exploit/patch flag or posture count reads back as 0 / false
 
-**Layer:** Egeria Server (type model / properties classes) · **Status:** open, question for the Egeria team · **Found:** 2026-10-26, while adding the 6.3 security annotation types to pyegeria (odpi/egeria-workspaces#702 hand-off, step 3; PR #495).
+**Layer:** Egeria Server (type model / properties classes) · **Status:** fixed (Egeria 6.3, odpi/egeria#9389) · **Found:** 2026-10-26, while adding the 6.3 security annotation types to pyegeria (odpi/egeria-workspaces#702 hand-off, step 3; PR #495).
+
+**FIXED 2026-10-27: odpi/egeria#9389, merged as 12b1aaae.** Verified live the same day on egeria-platform c2ca054c (throwaway VulnerabilityAnnotation ed152edb, deleted, NotFound by GUID): unset cvssScore/exploitAvailable/patchAvailable were omitted; a merge update that left out 8.6 / true / false kept them; an explicit false came back. The pyegeria workarounds below are removed on branch `feature/boxed-properties-audit`. It boxes 217 number and flag fields across 86 property beans, not only the security ones. An unset value is no longer stored and is left out of the response, and a merge update keeps a value it does not mention. Defaults move to `OpenMetadataProperty.defaultValue`, published in the Core Content Pack as `additionalProperties.defaultValue` on each property's valid metadata value. `allowsDuplicateValues` now defaults to true everywhere, including SchemaAttribute (was false). The PR does not migrate stored data: elements written by earlier servers keep the 0 / false they were stored with. **pyegeria:** fixes that work with either server are on branch `feature/boxed-properties-audit`. The client-side workarounds stay until the PR merges and pyegeria targets that server.
 
 **What happens:** in `SecurityFindingAnnotationProperties`, `VulnerabilityAnnotationProperties` and `SecurityPostureAnnotationProperties` (`frameworks/open-metadata-framework/.../properties/surveyreports/`), these fields are Java primitives with a 0 / false default:
 

docs/dr_egeria_manual.md

@@ -272,7 +272,7 @@ Dr.Egeria organizes its commands into "families," each corresponding to a specif
   - Security testing results (model 0665): `Create`/`Update Survey Report` catalogues one test run, through Asset Maker. It takes Category (e.g. Penetration Test Results), Purpose, Start Time and Completion Time; `Report Subjects` links the assets the run tested (ReportSubject). `Create`/`Update Security Finding`, `Vulnerability Finding` and `Penetration Test Finding` each create one finding under its `Survey Report` (ReportedAnnotation, required), with Severity (the CVSS ratings None/Low/Medium/High/Critical), CVSS Score and the type's own fields. `Affected Elements` links catalogued elements (AssociatedAnnotation), `Previous Finding` links the same finding in an earlier run (AnnotationExtension), and `Vulnerability Type` links a reference data value (ReferenceValueAssignment). `Create Security Posture Summary` records a run's counts: `Finding Counts By Severity` takes one `severity: count` pair per line. Egeria stores the CVSS score, exploit/patch flags and posture counts as primitives, so a merge update that leaves one out would reset it to 0 / false; these commands copy the current value into the update to prevent that (ISSUE-166). Processors in `md_processing/v2/security_officer.py`.
   - `Link`/`Unlink Control Applicability`: assess a control (any governance definition, typically a GovernanceControl) in a statement — `Is Applicable`, `Justification`, `Inclusion Reasons`, `Implementation Status` (NOT_STARTED, PLANNED, PARTIALLY_IMPLEMENTED, IMPLEMENTED, OTHER), `Implementation Description`, `Review Date`, and `Label` for the control's number in the standard. There is one relationship per control, and re-running Link updates it. Egeria's attach replaces the relationship's properties wholesale, so the command starts from the existing ones: re-linking with just a new status keeps the justification and the rest (`Merge Update: false` replaces them).
   - Reports: `Statement-of-Applicability-Controls` (one row per control — use `LIST` for a markdown table, `DICT` for a spreadsheet) and `Statement-of-Applicability-Implementation-Summary` (applicable controls counted by implementation status; `BAR`/`PIE` for a chart). Both take `soa_guid`, which accepts the statement's GUID or qualified name, e.g. `View Report` with `Report Parameters` `soa_guid: StatementOfApplicability::Coco Pharmaceuticals ISMS`. The summary is analytic-function-backed, which `View Report` cannot run yet; use it through `exec_report_spec` or `SecurityOfficer.get_control_implementation_summary`.
-  - Security testing results (Egeria 6.3, model 0665): each test run is a `SurveyReport`, with one `SecurityFindingAnnotation` (or `VulnerabilityAnnotation` / `PenetrationTestFindingAnnotation`) per finding and a `SecurityPostureAnnotation` summarizing the run. The report specs `Security-Findings` (most severe first; params `report_guid` or `element_guid`, each a GUID or qualified name, and optional `min_severity`), `Penetration-Test-Findings` and `Security-Posture` (one row per run, oldest first; optional `report_qualified_name_prefix` to follow one system's series) run through `View Report`. Egeria returns `cvssScore`, `exploitAvailable`, `patchAvailable` and the posture counts as 0 or false when they were never set, so these reports show such values blank rather than as an answer; new and resolved finding counts of 0 cannot be told apart from unrecorded. Dr.Egeria commands for creating survey reports and findings are not available yet.
+  - Security testing results (Egeria 6.3, model 0665): each test run is a `SurveyReport`, with one `SecurityFindingAnnotation` (or `VulnerabilityAnnotation` / `PenetrationTestFindingAnnotation`) per finding and a `SecurityPostureAnnotation` summarizing the run. The report specs `Security-Findings` (most severe first; params `report_guid` or `element_guid`, each a GUID or qualified name, and optional `min_severity`), `Penetration-Test-Findings` and `Security-Posture` (one row per run, oldest first; optional `report_qualified_name_prefix` to follow one system's series) run through `View Report`. From Egeria 6.3, a `cvssScore`, `exploitAvailable`, `patchAvailable` or posture count that was never set is left out, so it shows blank, and a 0 or false shown is a real answer. Annotations written by earlier servers may still show 0 or false for a value nobody set. Dr.Egeria commands for creating survey reports and findings are not available yet.
 - **Action Author**: Define governance action process flows — reusable single-step action types and multi-step processes — and wire them to the engines and elements that execute them, without writing code (e.g., `Create Governance Action Process`, `Create Governance Action Process Step`, `Link First Process Step`, `Link Next Process Step`, `Link Action to Action Executor`, `Link Action to Target`).
 - **Curation**: Apply classifications and relationships to *existing* Referenceable elements after the fact — this family creates no elements of its own, it curates ones created elsewhere. Every command names a `Target Element` (Reference Name) and either a classification level/status or a second element to relate to.
   - **Classifications** (`Classify`/`Reclassify`/`Declassify`, or `Classify`/`Update`/`Declassify` for the three multi-value ones): Impact, Confidence, Confidentiality, Criticality, Retention, Ownership, Digital Resource Origin, Zone Membership, Security Tags, Data Scope, Governance Expectations, Governance Measurements, Known Duplicate, Consolidated Duplicate, and the 0438 naming-standards markers Class Word, Modifier, Prime Word (added 2026-08-09 once Egeria PR #9166 shipped the backing `glossary-manager` endpoints — routed through `GlossaryManager`, not `ClassificationExplorer`, unlike every other classification here). Also, added 2026-08-21 once an `omvs_audit.py` gap-closure pass shipped their backing REST endpoints (all confirmed against a live 6.2-SNAPSHOT server): the 10 governance-point classifications (0435: Control/Verification/Enforcement/Execution Point, Policy Administration/Decision/Enforcement/Information/Management/Retrieval Point — routed through `GovernanceOfficer`), 6 metamodel markers (0463: Incomplete, ObjectIdentifier, ReferenceData, MobileResource, InstanceMetadata, MetamodelInstance — routed through `ClassificationExplorer`), ProjectKind (routed through `ProjectManager`), CollectionKind (routed through `CollectionManager`), and a Data Sharing Agreement retrofit pair for classifying/declassifying an already-existing Agreement (routed through `DigitalBusiness` — `Create Data Sharing Agreement`, Digital Products family, already sets this at creation time via `initialClassifications`). Also Promise (0010, added 2026-09-30 — `Classify`/`Declassify Promise`, routed through `ClassificationExplorer`): marks an element as a placeholder for a real-world resource that has not yet been delivered, with Deployment Status, Start/Due/Last Review/Completion Time and Additional Properties. A promised element is only visible to lineage requests, so `Declassify Promise` looks up its Target Element and sends the clear request with `forLineage=true` automatically.

md_processing/md_processing_utils/common_md_utils.py

@@ -507,16 +507,12 @@ def set_actor_manager_prop_body(object_type: str, qualified_name: str, attribute
     if "Role" in object_type:
         prop_bod["scope"] = attributes.get('Scope', {}).get('value', None)
         if "Person Role" in object_type or "Team Role" in object_type:
-            # headCount is an integer
-            headcount_entry = attributes.get('Headcount', {})
-            headcount = headcount_entry.get('value', 0)
+            # headCount is an integer; left unset when not given, so an update keeps the stored value
+            headcount = attributes.get('Headcount', {}).get('value')
             try:
-                if headcount is not None:
-                    prop_bod["headCount"] = int(headcount)
-                else:
-                    prop_bod["headCount"] = 0
+                prop_bod["headCount"] = int(headcount) if headcount not in (None, "") else None
             except (ValueError, TypeError):
-                prop_bod["headCount"] = 0
+                prop_bod["headCount"] = None
 
     return prop_bod
 
@@ -547,7 +543,7 @@ def set_solution_architect_body(object_type: str, qualified_name: str, attribute
         })
 
     if object_type in ("Solution Role", "SolutionRole", "SolutionActorRole"):
-        role_domain_id = attributes.get('Role Domain Identifier', {}).get('value', 0)
+        role_domain_id = attributes.get('Role Domain Identifier', {}).get('value')
         resolved_domain = resolve_enum(GovernanceDomains, role_domain_id)
         prop_bod["domainIdentifier"] = resolved_domain if resolved_domain is not None else role_domain_id
 

md_processing/v2/processors.py

@@ -777,6 +777,7 @@ async def _execute(self) -> Dict[str, Any]:
                 if self.command.verb in ["Create", "Define", "Register", "Add", "Upsert"]:
                     logger.info(f"Rewriting '{self.command.verb} {self.command.object_type}' to 'Update' as it already exists.")
                     self.command.verb = "Update"
+                    self._drop_create_defaults()
 
                 self.parsed_output["exists"] = True
                 header = self.as_is_element.get('elementHeader', {})
@@ -1954,6 +1955,22 @@ async def sync_members(self,
 
         return results
 
+    # Control flags rather than element properties: keep their defaults on an update.
+    _KEEP_DEFAULTS_ON_UPDATE = {"Merge Update"}
+
+    def _drop_create_defaults(self) -> None:
+        """Remove spec defaults the parser filled in for a Create that has become an Update.
+
+        The parser applies defaults only for create-like verbs, so a Create rewritten to Update
+        still carries them, and a merge update would send them over the stored values (e.g.
+        isNullable: true over a stored false). Egeria 6.3 keeps an omitted number or flag
+        (odpi/egeria#9389), so leaving them out is what preserves the current value.
+        """
+        attributes = self.parsed_output.get("attributes") or {}
+        for name in [n for n, d in attributes.items()
+                     if isinstance(d, dict) and d.get("is_default") and n not in self._KEEP_DEFAULTS_ON_UPDATE]:
+            del attributes[name]
+
     def filter_update_properties(self, properties: Dict[str, Any], merge_update: bool) -> Dict[str, Any]:
         """
         Filters properties for an update operation.

md_processing/v2/security_officer.py

@@ -300,10 +300,6 @@ async def _existing_applicability(self, soa_guid: str, control_guid: str) -> Dic
 }
 
 
-# Egeria declares these as Java primitives, so a merge update that leaves one out stores
-# its 0 / false default over the current value (confirmed live, ISSUE-166).
-_PRIMITIVE_FIELDS = ("cvssScore", "exploitAvailable", "patchAvailable", "resourcesInScopeCount",
-                     "resourcesTestedCount", "newFindingCount", "resolvedFindingCount")
 _FLOAT_FIELDS = {"cvssScore"}
 _INT_FIELDS = {"resourcesInScopeCount", "resourcesTestedCount", "newFindingCount", "resolvedFindingCount"}
 
@@ -403,12 +399,6 @@ async def apply_changes(self) -> str:
 
         if self.as_is_element:
             guid = self.as_is_element["elementHeader"]["guid"]
-            if merge_update:
-                # Without this, the merge would reset any primitive the command leaves out.
-                current = self.as_is_element.get("properties") or {}
-                for key in _PRIMITIVE_FIELDS:
-                    if key not in props and current.get(key) is not None:
-                        props[key] = current[key]
             self.last_body = body = {
                 "class": "UpdateElementRequestBody",
                 "properties": self.filter_update_properties(props, merge_update),