Say why an analytic result is absent instead of returning it as data (ISSUE-171)

PYEGERIA_ISSUES.md

@@ -172,6 +172,7 @@ re-test candidates moved into *Awaiting re-test* below the open sections.
   create-from-template) and 7 (`viewElementGUID`) are still open.
 
 **Open pyegeria items**
+- ISSUE-171: `exec_report_spec` returned an analytic result that was not measured as ordinary data or `empty`; it now returns the function's declared absence sentence
 - ISSUE-170: `AnalyticFunctionSpec` can't say what its results mean (unit, as-of, basis, absence sentences); added an optional `declaration`
 - ISSUE-169: `_resolve_analytic_function` imported any dotted path a report spec named; now registered names plus an allowlist of module prefixes
 - ISSUE-168: pyegeria can't run on Python 3.14: `nest_asyncio`, applied at import time, makes `asyncio.current_task()` return None
@@ -195,6 +196,25 @@ into the entry now, so it isn't rediscovered from scratch later.
 
 ---
 
+### ISSUE-171: `exec_report_spec` returned an analytic result that was not measured as ordinary data or `empty`; it now returns the function's declared absence sentence
+
+**Layer:** pyegeria · **Status:** in PR (branch `feat/state-aware-report-spec`, stacked on ISSUE-170's `feat/analytic-declaration`) · **Found:** 2026-10-30, Resource Explorer's portal-extensibility evaluation, §3 item 2.
+
+**What:** ISSUE-139/140 stopped charting a `not_established`/`not_applicable` result. The DICT path and the chart fallback still returned `{"kind": "json", "data": <result>}`, though, so a table or tile showed the routine's placeholder zeros as an answer. And `{"kind": "empty"}` looks the same to a reader whether nothing was found or nobody looked. Egeria Advisor, for example, renders `empty` as "No results returned".
+
+**Change** (`pyegeria/view/format_set_executor.py`):
+- **not_established / not_applicable** results now return `{"kind": "absent", "content": <sentence>, "absence": {...}, "envelope": {...}}` on every output format. There is no top-level `data`; a partial result rides under `detail`.
+- **nothing_found** results keep their data or chart (real zeros) and gain the same `absence` block. A chart's subtitle is the declared `measured_nothing` sentence.
+- **measured** results, and results with no envelope, are unchanged.
+
+`absence` gives the state (`not_measured`, `not_permitted`, `not_applicable` or `measured_nothing`), the sentence, the reason, and `declared`. The sentence comes from the function's registered `declaration.absence` (ISSUE-170), looked up by registry name or by dotted path. A function with no declaration gets words built from its own reason, with `declared: False`. `not_established` becomes `not_permitted` only when the result says so itself (`"absence": "not_permitted"`); the reason text is never parsed to guess.
+
+**Consumers to update:** Egeria Advisor's `report_pipeline`/`report_spec_agent` (trellis) treat an unknown kind generically. `content` is among the keys its fallback reads, so it shows the sentence, but it should handle `absent` explicitly. Any Portal tile that reads `data` for these states gets no `data` now.
+
+Tests: `tests/micro-tests/test_issue_171_state_aware_report_spec.py`; three ISSUE-140 tests updated to the new shape.
+
+---
+
 ### ISSUE-170: `AnalyticFunctionSpec` can't say what its results mean (unit, as-of, basis, absence sentences); added an optional `declaration`
 
 **Layer:** pyegeria · **Status:** in PR (branch `feat/analytic-declaration`) · **Found:** 2026-10-30, Resource Explorer's portal-extensibility evaluation, §3 item 1 (trellis `docs/design-notes/EVAL-PORTAL-EXTENSIBILITY-VS-RE-2026-10-30.md`).

pyegeria/view/format_set_executor.py

@@ -34,7 +34,8 @@
     get_report_registry,
 )
 from pyegeria.view.output_formatter import generate_output
-from pyegeria.view.analytic_registry import AnalyticActionSpec
+from pyegeria.view.analytic_registry import ABSENCE_STATES, AnalyticActionSpec
+from pyegeria.view.analytic_registry import get_analytic_registry as get_analytic_registry_functions
 from pyegeria.egeria_tech_client import EgeriaTech
 
 _CLIENT_CLASS_MAP = {
@@ -274,7 +275,10 @@ def _from_mapping(m: Dict[str, Any]) -> Optional[Dict[str, str]]:
             return None
         reason = str(m.get("state_reason", m.get("resultStateReason", m.get("result_state_reason", ""))) or "")
         subtitle = m.get("subtitle") or (state.replace("_", " ") + (f" \u00b7 {reason}" if reason else ""))
-        return {"state": state, "reason": reason, "subtitle": str(subtitle)}
+        found = {"state": state, "reason": reason, "subtitle": str(subtitle)}
+        if m.get("absence") in ABSENCE_STATES:  # the result says which absence (ISSUE-171)
+            found["absence"] = m["absence"]
+        return found
 
     if isinstance(result, dict):
         found = _from_mapping(result)
@@ -305,6 +309,62 @@ def _with_envelope(out: Dict[str, Any], envelope: Optional[Dict[str, str]]) -> D
     return out
 
 
+# Envelope state -> the declared absence it is (ISSUE-171). not_established reads as
+# not_measured unless the result itself claims not_permitted (an `absence` key);
+# the reason text is never parsed to guess which.
+_ABSENCE_FOR_STATE = {
+    "nothing_found": "measured_nothing",
+    "not_established": "not_measured",
+    "not_applicable": "not_applicable",
+}
+
+# Used only when the function declares no absence sentences; built from its own reason.
+_UNDECLARED_ABSENCE_WORDS = {
+    "not_measured": "Not measured",
+    "not_permitted": "Not measured: not permitted",
+    "not_applicable": "Does not apply",
+    "measured_nothing": "Measured; nothing found",
+}
+
+
+def _declaration_for(func_decl: str):
+    """The registered declaration for an analytic_function, by registry name or by the
+    dotted path a registered entry declares; None when the function declares none."""
+    registry = get_analytic_registry_functions()
+    spec = registry.get(func_decl) or next((s for s in registry.values() if s.function == func_decl), None)
+    return spec.declaration if spec is not None else None
+
+
+def _absence_for(envelope: Optional[Dict[str, str]], func_decl: str) -> Optional[Dict[str, Any]]:
+    """What a reader is told when a result is not a measured value (ISSUE-171): the absence
+    state, the function's declared sentence for it (or words built from its reason when it
+    declares none, marked declared=False), and the reason. None for a measured result or
+    one with no envelope, which keep their old shapes."""
+    if not envelope or envelope["state"] not in _ABSENCE_FOR_STATE:
+        return None
+    state = _ABSENCE_FOR_STATE[envelope["state"]]
+    if state == "not_measured" and envelope.get("absence") == "not_permitted":
+        state = "not_permitted"
+    reason = envelope.get("reason", "")
+    declaration = _declaration_for(func_decl)
+    if declaration is not None:
+        return {"state": state, "sentence": declaration.absence.sentence(state), "reason": reason,
+                "declared": True}
+    words = _UNDECLARED_ABSENCE_WORDS[state]
+    return {"state": state, "sentence": f"{words}: {reason}." if reason else f"{words}.", "reason": reason,
+            "declared": False}
+
+
+def _absent_output(result: Any, envelope: Dict[str, str], absence: Dict[str, Any]) -> Dict[str, Any]:
+    """Output for a result whose value is not established or does not apply: the sentence,
+    never top-level data a chart or table would draw as an answer. Any partial result rides
+    under `detail`."""
+    out: Dict[str, Any] = {"kind": "absent", "content": absence["sentence"], "absence": absence}
+    if result not in (None, [], {}):
+        out["detail"] = result
+    return _with_envelope(out, envelope)
+
+
 def _run_analytic_function(
     action: dict, *, params: Dict[str, Any],
     view_server: str, view_url: str, user: str, user_pass: str,
@@ -804,13 +864,14 @@ def _exec_analytic_chart(
     )
     envelope = _result_envelope(result)
     result = _jsonable(result)
-    if not result:
-        return _with_envelope({"kind": "empty"}, envelope)
+    absence = _absence_for(envelope, action["analytic_function"])
     # Chart only an established answer (ISSUE-140): measured, or nothing_found (a real
-    # zero). For not_established / not_applicable the numbers aren't known, so return
-    # the data and the state, never a chart whose zeros would read as an answer.
+    # zero). For not_established / not_applicable the numbers aren't known, so say so
+    # (ISSUE-171), never a chart whose zeros would read as an answer.
     if envelope and envelope["state"] not in CHARTABLE_STATES:
-        return _with_envelope({"kind": "json", "data": result}, envelope)
+        return _absent_output(result, envelope, absence)
+    if not result:
+        return _with_envelope({"kind": "empty", **({"absence": absence} if absence else {})}, envelope)
 
     try:
         heading = get_report_spec_heading(format_set_name) or str(format_set_name)
@@ -821,8 +882,10 @@ def _chart_out(chart):
         if not chart:
             return _with_envelope({"kind": "json", "data": result}, envelope)
         if envelope:  # the state rides on the chart itself (ISSUE-139)
-            chart["title"] = {"text": chart.get("title") or heading, "subtitle": envelope["subtitle"]}
-        return _with_envelope({"kind": "json", "data": chart}, envelope)
+            subtitle = absence["sentence"] if absence and absence["declared"] else envelope["subtitle"]
+            chart["title"] = {"text": chart.get("title") or heading, "subtitle": subtitle}
+        out = {"kind": "json", "data": chart, **({"absence": absence} if absence else {})}
+        return _with_envelope(out, envelope)
 
     if chart_kind == "line":
         if not isinstance(result, list) or not all(isinstance(pt, dict) for pt in result):
@@ -1000,9 +1063,13 @@ def exec_report_spec(
         )
         envelope = _result_envelope(result)
         result = _jsonable(result)
+        absence = _absence_for(envelope, action["analytic_function"])
+        if envelope and envelope["state"] not in CHARTABLE_STATES:
+            return _absent_output(result, envelope, absence)  # ISSUE-171
+        extra = {"absence": absence} if absence else {}
         if result is None or result == [] or result == {}:
-            return _with_envelope({"kind": "empty"}, envelope)
-        return _with_envelope({"kind": "json", "data": result}, envelope)
+            return _with_envelope({"kind": "empty", **extra}, envelope)
+        return _with_envelope({"kind": "json", "data": result, **extra}, envelope)
 
     required_params = action.get("required_params", action.get("user_params", [])) or []
     optional_params = action.get("optional_params", []) or []

tests/micro-tests/test_issue_138_140_149_analytic_envelope.py

@@ -118,7 +118,7 @@ def test_measured_bar_gets_subtitle_and_envelope(monkeypatch):
 def test_not_established_draws_no_chart(monkeypatch):
     _use_spec(monkeypatch, {"analytic_function": f"{MOD}.not_established"})
     out = fse.exec_report_spec("tile", output_format="BAR")
-    assert "$schema" not in out["data"]
+    assert out["kind"] == "absent" and "data" not in out  # ISSUE-171: the sentence, not a chart
     assert out["envelope"] == {"state": "not_established", "reason": "could not open the ODS",
                                "subtitle": "not established · could not open the ODS"}
 
@@ -133,13 +133,13 @@ def test_nothing_found_is_charted_as_real_zeros(monkeypatch):
 def test_not_applicable_draws_no_chart(monkeypatch):
     _use_spec(monkeypatch, {"analytic_function": f"{MOD}.not_applicable"})
     out = fse.exec_report_spec("tile", output_format="BAR")
-    assert "$schema" not in out["data"] and out["envelope"]["state"] == "not_applicable"
+    assert out["kind"] == "absent" and out["envelope"]["state"] == "not_applicable"
 
 
 def test_dict_output_carries_envelope(monkeypatch):
     _use_spec(monkeypatch, {"analytic_function": f"{MOD}.not_established"})
     out = fse.exec_report_spec("tile", output_format="DICT")
-    assert out["kind"] == "json" and out["envelope"]["state"] == "not_established"
+    assert out["kind"] == "absent" and out["envelope"]["state"] == "not_established"  # ISSUE-171
 
 
 def test_model_result_with_envelope_attribute(monkeypatch):

tests/micro-tests/test_issue_171_state_aware_report_spec.py

@@ -0,0 +1,132 @@
+"""ISSUE-171: when an analytic function's result is not a measured value, exec_report_spec
+says so -- with the function's declared absence sentence -- instead of returning empty
+or drawing numbers that read as an answer."""
+import sys
+import types
+
+import pytest
+
+import pyegeria.view.format_set_executor as fse
+from pyegeria.view.analytic_registry import clear_runtime_analytic_functions, register_analytic_functions
+
+MOD = "fake_issue_171_analytics"
+
+ABSENCE = {
+    "not_measured": "Coverage was not measured: the catalog could not be read.",
+    "not_permitted": "Coverage was not measured: reading the catalog is not permitted.",
+    "not_applicable": "Coverage does not apply to this resource.",
+    "measured_nothing": "The catalog was read and holds no columns.",
+}
+
+
+def not_established():
+    return {"state": "not_established", "state_reason": "could not open the ODS", "described": 0}
+
+
+def not_permitted():
+    return {"state": "not_established", "state_reason": "403 from the catalog", "absence": "not_permitted"}
+
+
+def not_applicable():
+    return {"state": "not_applicable", "state_reason": "not a database"}
+
+
+def nothing_found_empty():
+    return {"envelope": {"state": "nothing_found"}, "counts": {}}
+
+
+def nothing_found_counts():
+    return {"state": "nothing_found", "byType": {"Table": 0, "Column": 0}}
+
+
+def measured():
+    return {"state": "measured", "byType": {"Table": 3, "Column": 12}}
+
+
+@pytest.fixture(autouse=True)
+def fake_module(monkeypatch):
+    mod = types.ModuleType(MOD)
+    for f in (not_established, not_permitted, not_applicable, nothing_found_empty, nothing_found_counts, measured):
+        setattr(mod, f.__name__, f)
+    monkeypatch.setitem(sys.modules, MOD, mod)
+    monkeypatch.setenv("PYEGERIA_ANALYTIC_MODULE_PREFIXES", f"pyegeria,{MOD}")  # if ISSUE-169's resolver is in
+    monkeypatch.setattr(fse, "EgeriaTech", lambda *a, **k: pytest.fail("no Egeria call expected"))
+    monkeypatch.setattr(fse, "get_report_spec_heading", lambda name: "Coverage")
+    clear_runtime_analytic_functions()
+    yield
+    clear_runtime_analytic_functions()
+
+
+def _declare(func_name: str):
+    register_analytic_functions({f"cov-{func_name}": {
+        "name": f"cov-{func_name}", "function": f"{MOD}.{func_name}",
+        "declaration": {"result_kind": "measure", "absence": ABSENCE, "measures": [
+            {"name": "described", "unit": "columns", "as_of": "asOf", "value_field": "described"}]},
+    }})
+
+
+def _run(monkeypatch, func_name: str, output_format: str = "DICT"):
+    fmt = {"action": {"analytic_function": f"{MOD}.{func_name}"}, "target_type": "Referenceable"}
+    monkeypatch.setattr(fse, "select_report_spec", lambda name, fmt_type: fmt)
+    return fse.exec_report_spec("coverage", output_format=output_format)
+
+
+@pytest.mark.parametrize("output_format", ["DICT", "BAR"])
+def test_not_established_says_the_declared_sentence_and_draws_nothing(monkeypatch, output_format):
+    _declare("not_established")
+    out = _run(monkeypatch, "not_established", output_format)
+    assert out["kind"] == "absent"
+    assert out["content"] == ABSENCE["not_measured"]
+    assert out["absence"] == {"state": "not_measured", "sentence": ABSENCE["not_measured"],
+                              "reason": "could not open the ODS", "declared": True}
+    assert "data" not in out  # no top-level counts to draw as an answer
+    assert out["detail"]["described"] == 0  # the partial result is still there, labelled as detail
+
+
+def test_the_result_can_claim_not_permitted(monkeypatch):
+    _declare("not_permitted")
+    out = _run(monkeypatch, "not_permitted")
+    assert (out["kind"], out["absence"]["state"], out["content"]) == (
+        "absent", "not_permitted", ABSENCE["not_permitted"])
+
+
+def test_not_applicable_says_so(monkeypatch):
+    _declare("not_applicable")
+    out = _run(monkeypatch, "not_applicable", "BAR")
+    assert (out["kind"], out["content"]) == ("absent", ABSENCE["not_applicable"])
+
+
+def test_nothing_found_carries_the_measured_nothing_sentence(monkeypatch):
+    _declare("nothing_found_empty")
+    out = _run(monkeypatch, "nothing_found_empty")
+    assert out["kind"] == "json"  # a real answer: the dict, with its sentence
+    assert out["absence"]["state"] == "measured_nothing"
+    assert out["absence"]["sentence"] == ABSENCE["measured_nothing"]
+
+
+def test_nothing_found_counts_are_charted_with_the_declared_sentence(monkeypatch):
+    _declare("nothing_found_counts")
+    out = _run(monkeypatch, "nothing_found_counts", "BAR")
+    assert "$schema" in out["data"]
+    assert out["data"]["title"] == {"text": "Coverage", "subtitle": ABSENCE["measured_nothing"]}
+
+
+def test_undeclared_function_gets_words_from_its_own_reason(monkeypatch):
+    out = _run(monkeypatch, "not_established")
+    assert out["kind"] == "absent"
+    assert out["content"] == "Not measured: could not open the ODS."
+    assert out["absence"]["declared"] is False
+
+
+def test_measured_result_is_unchanged(monkeypatch):
+    _declare("measured")
+    out = _run(monkeypatch, "measured")
+    assert out["kind"] == "json" and "absence" not in out
+    assert out["data"]["byType"] == {"Table": 3, "Column": 12}
+
+
+def test_registered_name_finds_the_declaration(monkeypatch):
+    _declare("not_established")
+    assert fse._declaration_for("cov-not_established").absence.not_measured == ABSENCE["not_measured"]
+    assert fse._declaration_for(f"{MOD}.not_established") is not None
+    assert fse._declaration_for(f"{MOD}.unregistered") is None