Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
- `pyegeria/omvs/*`: service implementations (40+ modules); new API behavior should usually be added here first.
- `pyegeria/models/`: Pydantic models for Egeria API request/response bodies (`models.py`, `collection_models.py`).
- `pyegeria/view/_output_format_models.py`: Pydantic models (`Column`/`Format`/`FormatSet`/`ActionParameter`) backing the report spec registry.
- `pyegeria/view/output_formatter.py` + `pyegeria/view/base_report_formats.py`: materialization/rendering and report spec registry; generated specs live in `base_report_formats.py` under `generated_format_sets` (do not hand-edit), hand-maintained ones under `base_report_specs` (safe to edit). `get_report_registry()` auto-loads an additional CONFIG tier from `settings.Environment.pyegeria_report_spec_modules` on first call.
- `pyegeria/view/output_formatter.py` + `pyegeria/view/base_report_formats.py`: materialization/rendering and report spec registry; generated specs live in `base_report_formats.py` under `generated_format_sets` (do not hand-edit), hand-maintained ones under `base_report_specs` (safe to edit). `get_report_registry()` auto-loads an additional CONFIG tier on first call: `pyegeria.report_specs` entry points (alias `trellis.report_specs`; pyegeria's own analytic demos load this way) and the user report specs directory, plus the deprecated `PYEGERIA_REPORT_SPEC_MODULES`/config.json list (warns in 6.2.18, removed in 6.2.19; ISSUE-173). Report specs name analytic functions by registered name; a dotted path is deprecated.
- `pyegeria/view/format_set_executor.py`: side-effect-free FormatSet runner used by the MCP adapter and the `ViewProcessor` in Dr.Egeria; also runs `ActionParameter.analytic_function` (the `extra_find` path) and wraps its result as a `SERIES`/`BAR`/`PIE` Vega-Lite chart when requested.
- `pyegeria/view/analytic_registry.py` + `pyegeria/view/analytic_demo_specs.py`: catalog of analytic functions (`extra_find` targets) and one real demo `FormatSet` per function. A report spec may only name a registered function or a module under `PYEGERIA_ANALYTIC_MODULE_PREFIXES` (ISSUE-169); packages register through the `pyegeria.analytics` entry-point group (ISSUE-172).
- `pyegeria/view/_output_dashboard_sheet_models.py`: `DashboardSheet`/`Placement` — local (pre-Egeria-native) model for user-authored dashboards; built via `md_processing/v2/dashboard_sheet.py`'s processors (`Create Dashboard Sheet`, `Link Report to Dashboard Sheet`, `Add Text on Dashboard Sheet`).
Expand Down
4 changes: 2 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -268,12 +268,12 @@ Changes to `pyegeria/` (SDK API, config, report formatting) can break both.
- `create_egeria_bearer_token()` / `set_bearer_token()` propagate tokens to every subclient.

**`pyegeria/view/`** — output formatting and report spec registry
- `base_report_formats.py`: two separate dicts merged by `get_report_registry()` — `generated_format_sets` (auto-generated by `refresh_specs`; do not hand-edit) and `base_report_specs` (hand-maintained BUILTINS, e.g. `Referenceable`; safe to edit directly, `refresh_specs` never touches it). `get_report_registry()` also auto-loads a CONFIG tier on first call, from `settings.Environment.pyegeria_report_spec_modules` (config.json's `"Pyegeria Report Spec Modules"` list) or the `PYEGERIA_REPORT_SPEC_MODULES` env var — each entry a `.json` path or a `"pkg.mod:func"` loader — plus every `*.json` in the user report specs directory (`PYEGERIA_USER_REPORT_SPECS_DIR` > config.json `"Pyegeria User Report Specs Dir"` > `~/.pyegeria/report_specs`). Each entry loads independently: a failing or label-colliding entry is skipped with a warning (and returned by `refresh_report_specs()`), never blocking the others. **Gotcha:** `config/config.json` is only auto-discovered if `PYEGERIA_CONFIG_DIRECTORY` (or `PYEGERIA_ROOT_PATH`) is set — a bare shell with neither falls back to defaults silently, so a probe script that just imports `settings` may not see config.json values at all; use the `PYEGERIA_REPORT_SPEC_MODULES` env var directly when in doubt.
- `base_report_formats.py`: two separate dicts merged by `get_report_registry()` — `generated_format_sets` (auto-generated by `refresh_specs`; do not hand-edit) and `base_report_specs` (hand-maintained BUILTINS, e.g. `Referenceable`; safe to edit directly, `refresh_specs` never touches it). `get_report_registry()` also auto-loads a CONFIG tier on first call: every `pyegeria.report_specs` entry point (alias `trellis.report_specs`; a callable returning a FormatSetDict, or the dict itself — pyegeria's own analytic demo specs load this way, so after changing entry points run `uv sync`), then every `*.json` in the user report specs directory (`PYEGERIA_USER_REPORT_SPECS_DIR` > config.json `"Pyegeria User Report Specs Dir"` > `~/.pyegeria/report_specs`). config.json's `"Pyegeria Report Spec Modules"`, `PYEGERIA_REPORT_SPEC_MODULES` and `PYEGERIA_REPORT_FORMATS_JSON`/`_MODULES` still load but are deprecated: one DeprecationWarning per refresh in 6.2.18, removed in 6.2.19 (ISSUE-173). Each entry loads independently: a failing or label-colliding entry is skipped with a warning (and returned by `refresh_report_specs()`), never blocking the others. **Gotcha:** `config/config.json` is only auto-discovered if `PYEGERIA_CONFIG_DIRECTORY` (or `PYEGERIA_ROOT_PATH`) is set — a bare shell with neither falls back to defaults silently, so a probe script that just imports `settings` may not see config.json values at all; use the `PYEGERIA_USER_REPORT_SPECS_DIR` env var directly when in doubt.
- `output_formatter.py`: `generate_output()` — materializes elements into MD/LIST/DICT/REPORT formats.
- `_output_format_models.py`: Pydantic models `Column`/`Format`/`FormatSet`/`ActionParameter` — define new specs with these, not raw dicts. `ActionParameter` has two independent execution paths: `function`/`find_method` (element-query, formatted per-column) and `analytic_function`/`extra_find` (a plain function returning an already-aggregated result — see `output-formats-and-report-specs.md`'s "Analytic functions" section).
- `format_set_executor.py`: `exec_report_spec()` — runs either path; `SERIES`/`BAR`/`PIE` output formats wrap an analytic function's result as a Vega-Lite chart (dispatched before normal Format-row lookup, so no `FormatSet` needs to declare those as a `Format` row).
- `analytic_registry.py` / `analytic_demo_specs.py`: the catalog of analytic functions (`AnalyticFunctionSpec.generic` marks whether what it counts is a parameter or hardcoded) and one real, executable demo `FormatSet` per registered function. Installed packages add entries through the `pyegeria.analytics` entry-point group (`trellis.analytics` as an alias), read lazily on first registry use (ISSUE-172); a spec's optional `declaration` carries result kind, measures with unit/as-of/basis and the four absence sentences (ISSUE-170).
- **Gotcha — `analytic_function` can't name just any module** (ISSUE-169): it must be a registered name, a path a registered entry declares, or a path under `PYEGERIA_ANALYTIC_MODULE_PREFIXES` (default `pyegeria`, `trellis_analytics`); anything else raises `AnalyticFunctionRefused` before import. A test that injects a fake module must add it to that env var. And a `not_established`/`not_applicable` result comes back as `kind: "absent"` with a sentence in `content`, not as `data` (ISSUE-171).
- **Gotcha — `analytic_function` can't name just any module** (ISSUE-169): it must be a registered name, a path a registered entry declares, or a path under `PYEGERIA_ANALYTIC_MODULE_PREFIXES` (default `pyegeria`, `trellis_analytics`); anything else raises `AnalyticFunctionRefused` before import. Report specs (including `metric_trend`'s `metric_path`) should use the registered name: a dotted path there warns with a DeprecationWarning in 6.2.18 and is refused from 6.2.19 (ISSUE-173). A test that injects a fake module must add it to that env var. And a `not_established`/`not_applicable` result comes back as `kind: "absent"` with a sentence in `content`, not as `data` (ISSUE-171).
- `_output_dashboard_sheet_models.py`: `DashboardSheet`/`Placement` — a local (not yet Egeria-native) model for user-authored dashboards, built via Dr.Egeria's `Create Dashboard Sheet`/`Link Report to Dashboard Sheet`/`Add Text on Dashboard Sheet` (`md_processing/v2/dashboard_sheet.py`).
- Generated report specs follow `{Type}-DrE-{Basic|Advanced}` naming; carry an optional `family` string for discovery. Columns are the Create command's (input) attributes; display-only server-returned columns (e.g. the ISC mermaid graphs) go in `GENERATED_EXTRA_COLUMNS` in `commands/tech/gen_report_specs.py`, never in the compact spec.

Expand Down
23 changes: 22 additions & 1 deletion PYEGERIA_ISSUES.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ content was changed. The numbered section list above describes the

# Open Issues

**At a glance** — 2 open pyegeria items, 0 awaiting re-test, 0 open Egeria Server issues (2026-10-30: ISSUE-168, released in 6.2.16, and ISSUE-169..172, fixed for 6.2.17, moved to the Appendix. 2026-10-30: ISSUE-146's `remove_note` re-test passed, ISSUE-79's folder survey completes, ISSUE-108 did not reproduce at 200 links; all three moved to the Appendix. 2026-10-29: ISSUE-159, 132 and 85 live-verified; 12 Egeria-side fixes re-confirmed or closed on earlier passes and moved to the Appendix).
**At a glance** — 3 open pyegeria items, 0 awaiting re-test, 0 open Egeria Server issues (2026-10-31: ISSUE-173 added, report specs by registered name and the `pyegeria.report_specs` entry points. 2026-10-30: ISSUE-168, released in 6.2.16, and ISSUE-169..172, fixed for 6.2.17, moved to the Appendix. 2026-10-30: ISSUE-146's `remove_note` re-test passed, ISSUE-79's folder survey completes, ISSUE-108 did not reproduce at 200 links; all three moved to the Appendix. 2026-10-29: ISSUE-159, 132 and 85 live-verified; 12 Egeria-side fixes re-confirmed or closed on earlier passes and moved to the Appendix).

**Egeria team note, 2026-10-18 (Mandy Chessell).** Saved verbatim at
`~/.re-gate/EGERIA-TEAM-NOTE-2026-10-18.md`. Each entry below has a dated
Expand Down Expand Up @@ -192,6 +192,27 @@ into the entry now, so it isn't rediscovered from scratch later.

---

### ISSUE-173: Report specs named analytic functions by dotted path and loaded package specs through env vars; added the `pyegeria.report_specs` entry-point group and deprecated both

**Layer:** pyegeria · **Status:** built on branch `feat/analytic-registered-names`, no PR yet; targets 6.2.18, with removal in 6.2.19 · **Found:** 2026-10-31, the owner's ruling on Mandy's question 2 (portal analytics discovery), "yes to (b)", then "a" for report-spec registration (the Architect's R1, split note §11.3).

**What:** the Portal could only find a package's analytic tiles if something set `PYEGERIA_REPORT_SPEC_MODULES` (or config.json's `"Pyegeria Report Spec Modules"`) to that package's loader, and report specs named functions by dotted path. pyegeria's own analytic demo specs depended on config.json's key too. Separately, `metric_trend`'s `metric_path`, which a caller can override, was imported with no allowlist check, bypassing ISSUE-169.

**Change:**
- **Report-spec entry points** (`base_report_formats.py`): `refresh_report_specs()` reads the `pyegeria.report_specs` group, and `trellis.report_specs` as an alias, first. An entry point names a callable returning a `FormatSetDict` (or a dict of label to `FormatSet`), or such a dict itself. Each entry point loads in isolation: one that fails or clashes is skipped, logged, and returned by `refresh_report_specs()`, and the rest still load.
- **pyegeria's own demo specs** now load through its own entry point (`analytic_demos` in `pyproject.toml`), and the key is gone from `config/config.json`. A config entry naming the same loader as an installed entry point is treated as that entry point, not as a clash.
- **Deprecation:** config.json's list, `PYEGERIA_REPORT_SPEC_MODULES`, `PYEGERIA_REPORT_FORMATS_JSON` and `PYEGERIA_REPORT_FORMATS_MODULES` still load. When any is set, each refresh issues one `DeprecationWarning` and one WARNING log line naming them and the replacement.
- **Registered names** (`format_set_executor.resolve_report_spec_function`): a report spec's `analytic_function`, or `metric_trend`'s `metric_path`, given as a dotted path still resolves under the ISSUE-169 checks. It now raises a `DeprecationWarning` naming the registered name to use (or saying to register it), and logs once per path. A refused path raises with no deprecation warning. `metric_path` now goes through this resolver.
- **pyegeria's own specs** now use registered names: the SoA implementation summary (`control_implementation_summary`), every analytic demo spec, and the metric-trend demo's `metric_path` (`governed_coverage`). A test checks that every shipped spec names a registered function.

**Kept as dotted paths, on purpose:** `AnalyticFunctionSpec.function`, an action's `fetch`/`analytic` steps, and `analytic_binders` factories. They are code, not report-spec data, and binder factories have no registry to name them from.

**Downstream:**
- egeria-workspaces-fs `PyegeriaWebHandler/report_specs_handler.py` (quickstart and freshstart) names `pyegeria.view.overview_metrics.count_elements` and `.metric_trend`, with a dotted `metric_path`; it should use `count_elements` and `metric_trend`.
- trellis-analytics `portal.py`'s tile names `trellis_analytics.portal.curation_coverage_tile`, which needs a registered name, and its `report_specs` loader moves to a `pyegeria.report_specs` entry point.

**6.2.19:** refuse dotted paths in report specs, and stop reading the four deprecated sources.

### ISSUE-155: pyegeria gaps against Egeria `main` since 6.2 (omvs_audit + source diff, V6.2..05a5e982)

**Status 2026-10-29:** items 1 (Governance Engine moves, #460), 5 (undo/restore) and 6 (usage-activity and popularity classifications) are done and released, and item 2 is fixed under ISSUE-154. Still open: item 4 (`typeName`, which only `OpenMetadataTemplateRequestBody` carries) and item 7 (`GetOptions.viewElementGUID`, low priority).
Expand Down
5 changes: 1 addition & 4 deletions config/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,7 @@
"Pyegeria Root": "sample-data",
"Pyegeria Config Directory": "config",
"Pyegeria Publishing Root": "http://localhost:8085/outbox",
"Pyegeria User Report Specs Dir": "~/.pyegeria/report_specs",
"Pyegeria Report Spec Modules": [
"pyegeria.view.analytic_demo_specs:get_analytic_demo_specs"
]
"Pyegeria User Report Specs Dir": "~/.pyegeria/report_specs"
},
"feature_x_enabled": false,
"Logging": {
Expand Down
Loading
Loading