Detailed report, business revision, and scientifically-validated formula corrections - #1
Open
maxdolphin wants to merge 70 commits into
Open
maxdolphin wants to merge 70 commits into
maxdolphin wants to merge 70 commits into
Conversation
…e 1) Layered exec+analyst report adding benchmarking, risk & resilience analysis, prioritized action roadmap, and ESG framework (GRI/ESRS/TCFD) mapping. Builds only on existing metrics; adds no new scientific formulas. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
9 bite-sized TDD tasks: report_intelligence module (benchmark, risk, roadmap, ESG crosswalk, verdict, WoV chart), new PDF report sections behind a backward- compatible detailed flag, plus unit and smoke tests. No new scientific formulas. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…p, ESG, WoV chart) Pure-Python synthesis of existing OASIS profile + Ulanowicz metrics into structured report content. No new scientific formulas; only classification, sequencing, and reference lookups. 19 unit tests passing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… report New sections render behind a backward-compatible detailed flag (default on). Lean report reproducible via detailed=False. Smoke tests passing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Convert section/subsection/table/figure numbering to CSS counters so numbering is correct and mode-independent (no duplicate section numbers when detailed sections are inserted). Render the Window-of-Viability chart once in Benchmarking instead of also re-rendering it in Results. Add Appendix B metric glossary. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The app's '📕 PDF Report' button renders via src/pdf_generator.py (reportlab), not the WeasyPrint generator. Add Benchmarking, Risk & Resilience, Prioritized Action Roadmap, and ESG Framework Mapping sections there too, reusing the generator-agnostic report_intelligence module. Renumber Discussion/Conclusions and update the TOC. Integration test renders the real PDF and verifies sections via pypdf. 57 tests pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…idation (M2 core) Add src/network_ingestion.py: auto-detects adjacency-matrix vs edge-list CSV, builds a validated flow matrix, returns actionable warnings (isolated nodes, self-loops), and raises clear errors (non-square, non-numeric, negative, zero-flow). Edge lists (source/target/weight, with header synonyms) are the common org-data export shape. Wire into the app upload UI with format auto-detection, validation messages, and downloadable matrix/edge-list templates. 72 tests pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…dom placeholder Add build_flow_matrix_from_edges() as the provider-agnostic normalizer that CSV edge lists and all cloud connectors feed into. Refactor parse_edge_list to use it (DRY). Replace MicrosoftGraphConnector.get_flow_data's misleading np.random placeholder with a zero matrix + a flows_from_interactions() helper using the shared primitive, so analysis never runs on synthetic data. 76 tests pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…eport review design Design spec for a strategy-consultant-grade business revision: 7-dimension business-utility rubric, three-lens agent audit across two contrasting orgs, layered benchmarking model (theoretical/reference/peer), and Impact x Effort redesign roadmap. Presentation/IA/narrative only — formulas stay fixed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
9-task plan producing the Business Revision deliverable: surface inventory, dashboard + PDF evidence capture for two contrasting orgs, three-lens agent audit against the 7-dimension rubric, scored matrix + gap heatmap, layered benchmarking model, Impact x Effort roadmap, and final assembled document. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Un-ignore the business-revision evidence dashboards/reports so the audit task can reference committed screenshots. Captures Core Metrics, Network Analysis, Visualizations, OASIS Health and Detailed Report for both TechFlow Innovations (Combined Flows) and Balanced Test Organization.
…rt, value chain) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Overall product avg 2.85/5. Weakest: Visual effectiveness (PDF has zero embedded images) and Benchmark/context (pervasive across both surfaces). Convergent #1 gap: self-contradicting HEALTHY-vs-Non-Viable headline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ization Tier 1 theoretical (alpha 0.2-0.6, R optimum 1/e) ships now; Tier 2 promotes the already-coded Fath 2019 org anchor (alpha 0.30-0.45) over wetlands; Tier 3 peer cohort deferred. Includes gradient-not-pass/fail reframe. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
All 10 top gaps covered; formula guardrail verified (presentation-only, math roots handed to formula-validator). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add downstream handoff subsection (Immediate-horizon first specs + formula-validator track) and export the deliverable to PDF. Two-stage review passed: spec-compliant, all load-bearing figures fact-checked against evidence and source code. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Core Ulanowicz measures all correct; issues concentrated in derived/threshold/ composite layers (alpha-optimum constant, roll-up floor, FCI, connectivity inversion, directed-graph network metrics, benchmark provenance). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ked fix plan 27 defects (3 CRITICAL/14 MAJOR/10 MINOR). Track 1 = 15 paper-backed corrections; Track 2 = proprietary design decisions. Size-normalization constants reframed as size-scaling devices, not bugs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… ecology, ENA) Panel refutes the naive alpha-optimum swap: 0.4596 is empirically calibrated (not a theorem), Fath 2019 uses -a.ln(a) with 1/e peak for economics, and a global 0.37->0.4596 swap self-contradicts the robustness kernel. ENA-method fixes (connectivity F/N, Finn FCI, trophic level, Lindeman) confirmed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nel verdicts 27 defects with explanations + verification status. Core math confirmed correct; alpha-optimum swap reclassified after panel refuted it; ENA-method + network-science fixes expert-confirmed. Rendered to PDF. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The effective connectivity used the literal product-form C = exp(+½·Σ w·ln(Tij²/(Ti·Tj))), which carries a POSITIVE exponent. The canonical Zorach-Ulanowicz 2003 form (Appendix p.76) has a NEGATIVE exponent; the positive form equals N/F (the reciprocal, always < 1) and violates the connectivity >= 1 floor (Ulanowicz 2004 p.334). Compute C = F/N directly (Z-U 2003 p.72: "C ≡ F/N") in both the loop (ulanowicz_calculator) and vectorized (vectorized_metrics) paths, keeping them in exact agreement and guaranteeing the identity R = F/C² = N/C. Adds tests/test_ena_fixes.py (TDD) covering: C >= 1, C == F/N, R == F/C², and loop-vs-vectorized parity, all to 1e-9. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ack-1 FIX 2) ecosystem_flow_calculator.calculate_finn_cycling_index now implements the canonical Finn method (Finn 1976; Ulanowicz 2004 §5; Fath 2019 Principle 2): column-normalize by throughflow to form G, S = (I-G)^-1, then TSTc = Σ ((s_ii-1)/s_ii)·T_i and FCI = TSTc/TST. The prior code normalized by the scalar TST (G ≈ 0 so S ≈ I, cycling crushed) and summed the off-diagonal of S — a systematic ~2x underestimate. A pure 4-ring now yields FCI ≈ 1.0 (was 0), a 10% leak ring ≈ 0.932 (matches Ulanowicz 2004), and an acyclic chain ≈ 0. Singular (fully conservative) structures are handled as the vanishing-leak limit -> FCI -> 1. ulanowicz_calculator: the O(n²) "Finn Cycling Index" (self-loops + 2-cycles only, returns 0 on a pure ring) is renamed to calculate_short_cycle_proxy and honestly documented as a lower-bound proxy. calculate_finn_cycling_index is kept as a back-compat alias delegating to the proxy (so app.py, precompute_service, oasis_calculator, reports and the 'finn_cycling_index' metric key are unchanged). A new calculate_finn_cycling_index_full provides the canonical internal-only Finn index alongside. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…(Track-1 FIX 3) calculate_trophic_depth used nx.average_shortest_path_length, an unweighted topological hop count that ignores flow magnitudes and cannot produce the fractional effective trophic levels the ENA literature requires. Replaced with the flow-weighted effective trophic level (Levine 1980; Ulanowicz 2004 §4): column-normalize inflows to form G, S = (I-G)^-1, and the effective trophic level of each compartment is the column-sum of S. New helper calculate_effective_trophic_levels exposes the per-compartment levels; trophic depth is their maximum. Verified against Ulanowicz 2004 Fig.4 (a 60/30/10 mixed feeder yields effective level 2.5); a linear chain yields 1,2,3,4. The 'trophic_depth' metric key is unchanged for consumers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…o (Track-1 FIX 4) 1 - respiration/(TST+imports) is a system-wide respiratory-retention / dissipation ratio, not Lindeman between-level transfer efficiency (which requires the Lindeman spine [L]; Lindeman 1942; Ulanowicz 2004 §4). Renamed the method to calculate_respiratory_retention_ratio with an accurate docstring and a TODO for the true [L]-based per-level efficiency. calculate_lindeman_efficiency is kept as a back-compat alias and the 'lindeman_efficiency' metric key is retained (aliasing the new value) alongside the new 'respiratory_retention_ratio' key, so no consumer breaks. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… formula-fix pass Independent verification that the Track-1 fix pass is correct and regression-free: - full suite 175 passed - core Ulanowicz measures (TST/AMI/A/C/Phi/alpha, C=A+Phi) bitwise-unchanged vs pre-fix c137bf5 - loop vs vectorized parity <1e-9 on all shared metrics incl. effective_connectivity (F/N) - scientific validation: Everglades anchors SKIP (not ERROR), cone_spring/crystal_river identities pass, no new failures vs baseline - all 7 intended-behavior fixes verified (veto, Finn FCI, connectivity, betweenness, mutualism, gradient reframe, size normalization) - PDF smoke test: 3 sample orgs generate cleanly, gradient framing present No source code changed; adds only the evidence report and the fx_verify.py harness. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds precompute_full_profile() to compute EVERY displayed metric family once
(core Ulanowicz+vectorized, OASIS profile, network-analysis, report-intelligence)
by reusing the existing calculators/analyzers -- no formula reimplementation. A
single UlanowiczCalculator is shared with OASISCalculator so the core is computed
once. Each family is guarded independently so a degenerate graph yields per-family
_error markers instead of failing the whole profile.
Adds FORMULA_VERSION ("2026.07-fixes") stamped into every profile; bumping it
invalidates stored profiles from older formulas and forces recompute on read.
Schema: precomputed_metrics gains a formula_version column via an idempotent
ADD-COLUMN-IF-NOT-EXISTS migration (PRAGMA-guarded). save_precomputed_metrics
persists the version; get_precomputed_metrics accepts required_version and treats
a mismatch as a miss. _make_serializable now handles sets and non-str dict keys
so the richer tier=3 blob serializes cleanly. Existing tier=2 path unchanged.
PrecomputePipeline.get_full_profile(): hash -> tier=3 lookup; genuine HIT only
when version matches (no recompute); else compute + persist + return, with a
cache_hit flag. Not yet wired into app.py (Pass B).
TDD: tests/test_full_profile.py -- family coverage, cache-HIT-does-not-recompute
(spy asserts precompute_full_profile is not re-called), version-mismatch forces
recompute, degenerate graph resilience. Uses a throwaway SQLite DB. Full suite:
186 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e at provision, read at render Add provision_network(network_data) that computes the full-index profile once via pipeline.get_full_profile and stashes it in session_state['full_profile']. Wire it at all provision chokepoints (JSON/CSV upload, sample data, ecosystem samples, synthetic generation, direct analysis entry) plus a safety-net call at the top of show_analysis_page. Add get_active_profile() accessor: reads the stashed profile, compute+stores on miss (never crashes). Refactor render sites to READ the stored profile instead of recomputing: - display_oasis_health: reads full_profile['oasis']; live OASISCalculator only as fallback and for the interactive custom-weights widget. - display_network_analysis: reads full_profile['network_analysis']; summary report formatted from the read dict (no recompute); live analyzer as fallback. - show_analysis_page: SI/ELD/TD filled from full_profile['core'] instead of live calculate_* calls; live calculator only on miss. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PublicationReportGenerator gains an optional oasis_profile param; its OASIS section reads it instead of building OASISCalculator. pdf_generator's two OASIS recompute sites (health section + report-intelligence block) prefer report_generator.oasis_profile, recomputing only on a miss. display_detailed_report threads the active profile's oasis into both generators. Back-compat preserved (param optional -> falls back to live recompute when absent). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…sample orgs Asserts OASIS overall+status+5 dims, robustness, alpha, structural_information, and network-analysis sigma+density read from the stored profile are identical to a fresh independent computation. Also asserts second read is a cache HIT. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause: the reportlab PDF path only embedded charts when the caller passed a non-empty `charts` dict; the whole `if charts:` block was skipped for the charts=None path (tests, gen-report, and any missing wiring), so `pdfimages -list` reported 0 images. The OASIS radar/gauges and the Window-of-Viability curve were never wired in at all. Make the generator self-sufficient: build charts internally from the calculator and precomputed oasis_profile regardless of caller input. - Window-of-Viability / robustness curve via the native matplotlib builder (plot_sustainability_curve_matplotlib -> savefig buffer), with a Plotly kaleido fallback. Embedded in section 3.2 (sustainability). - Network flow Sankey diagram embedded in section 3.3 (Visualizations), which now always renders instead of only when charts were passed. - OASIS radar (create_oasis_radar_chart) + dimension gauges (create_all_dimension_gauges) embedded in the OASIS Health section. - Added _mpl_image() helper and _guarded_chart_block(): each chart embed is individually try/except-guarded so one failure logs a warning and is skipped rather than aborting the whole PDF; figure numbers don't burn on skipped charts. Caller-supplied plotly figures still embed on top. kaleido 0.2.1 verified working in-env (PNG export ~0.6s); WoV uses the matplotlib path per the credibility requirement. Verified per-org (charts=None): cone_spring_original 0 -> 4, balanced_org_test 0 -> 4, tech_company_combined_matrix 0 -> 4; degenerate 2/3-node networks render without crashing. tests/test_pdf_images.py added (>=3 images, WoV present, degenerate-safe). Full suite: 196 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… (R8, R9) PART A (R8): restructure the first content page into a self-contained exec one-pager with, in order: reconciled capped OASIS verdict (+capped_by "so what"), 4 KPI cards with reference anchors + α gradient position, the marquee "you are here" Window-of-Viability curve, top-3 risks (Evidence->Implication), and top-3 time-horizoned next steps; then a "— Detailed analysis follows —" divider gating analyst depth. All inputs read from the precomputed oasis_profile + report_intelligence views (no metric recompute); charts guarded. PART B (R9): lead the exec page with a "Why this applies to your organization" keystone — organizational evidence first (Fath 2019 α 0.30–0.45), network-lens framing that complements culture/engagement, and the indicative-reference caveat (never a bare Non-Viable/UNSUSTAINABLE verdict). PART C: add the same keystone as an in-app expander atop the OASIS Health view. Tests: tests/test_exec_onepager.py (12) covering capped verdict, α gradient, embedded WoV image, risk + next-step lines, keystone text, no-bare-fail guardrail, degenerate-safe, and 3-sample-org image builds. Full suite 208 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…aked ids, benchmark framing
Client-facing credibility fixes (TDD, tests/test_report_proofing.py):
1. Subsection headings renumbered to match parent sections:
Discussion 4.1/4.2/4.3 -> 9.1/9.2/9.3; Conclusions 5.1/5.2/5.3 -> 10.1/10.2/10.3
(src/publication_report.py).
2. Table of Contents now mirrors the real body headings via a single
build_toc_items() source of truth (+ BODY_HEADINGS). Removed phantom
entries ("Network Structure", "System Organization", "Information-Theoretic
Analysis", "Resilience Metrics"). Kept it entry-matched (no fake/blank
page-number column) rather than a multiBuild TOC.
3. Leaked metric identifiers humanized in user-facing text via
humanize_metric_name() (e.g. number_of_roles -> "number of functional
roles", relative_ascendency -> "relative ascendency (α)"). Dict keys/access
unchanged; applied at both roadmap render points.
4. Benchmark section leads with the organizational anchor (Fath et al. 2019,
α 0.30–0.45) as the PRIMARY comparator (Table 5). Ecological wetland anchors
demoted to "illustrative" calibration reference points (Table 5b).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add WEIGHTING_PROFILES to the OASIS composite so a consultant can select a named lens that MODESTLY re-weights the five dimensions, keeping equal 20% as the honest published default (expert-org-management.md §3). - WEIGHTING_PROFILES: Balanced (default, equal 20%), Scale-up / Growth (Open+Intelligent), Efficiency / Turnaround (Autonomous+Sustainable), Regulated / Resilience-first (Symbiotic+Sustainable). Modest tilts only; import-time guard asserts each covers the 5 dims and sums to 1.0. - OASISCalculator.apply_weighting_profile(): cheap recombination of the STORED dimension scores into a new overall + worst-dimension band-capped status (reuses compute_overall_status). Does NOT recompute any dimension metric. - app display_oasis_health: profile selectbox on the precomputed full_profile['oasis'] scores, instant re-weight + Δ vs balanced + capped_by; manual sliders retained as "Custom". - PDF report: one-line caption noting the active weighting lens (default "Balanced (equal weights)") for transparency. - tests/test_weighting_profiles.py (TDD): sum-to-1.0, balanced==equal parity, recombination==full-compute parity, tilt moves overall, cap survives. Full suite: 239 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Deepen the OASIS->ESG crosswalk from a one-to-one code lookup into a finding-specific structural-lens mapping (indicative, NOT compliance): - Per dimension: a list of GRI/ESRS/TCFD framework references at defensible series/pillar granularity, a disclosure-relevance sentence stating what the structural finding informs, and a materiality flag driven by the org's ACTUAL precomputed OASIS dimension status (CRITICAL->attention/material, WARNING->watch, HEALTHY->supporting evidence). - Fix the audited stretch (R17): SUSTAINABLE no longer maps to GRI 201-2 (climate financial implications); that code is explicitly dropped and the relevance sentence distinguishes structural resilience from climate risk. All analogue mappings (climate-scoped TCFD vs non-climate structure) are flagged `contextual` with an explicit caveat, never as direct disclosures. - INDICATIVE_ESG_CAVEAT as single source of truth; rendered in the PDF ESG section and carried on every crosswalk row. - Rewrite the PDF ESG section to render frameworks + relevance + materiality per dimension; keep backward-compatible gri_ref/esrs_ref/tcfd_ref strings. - TDD: tests/test_esg_crosswalk.py (14 tests) incl. status-driven materiality, the dropped/caveated climate stretch, and the caveat in the rendered PDF. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-cohort default) Adds a size/sector-matched peer-cohort benchmarking mechanism that scores alpha (and other key metrics) as a PERCENTILE within a cohort of other analyzed organizations — but only when a real cohort of >=10 eligible peers exists. HONESTY: no peer number is ever fabricated. A network is an eligible peer only if it carries a sector tag (deliberately registered via the ingestion path); untagged records (ecological samples / synthetic fixtures) are never counted as peers. With <10 matched peers the mechanism returns insufficient_cohort and NO percentile, and the benchmark section falls back to the existing indicative reference with an explicit note. This is the genuine DEFAULT state today. - db_manager: idempotent `sector` nullable column (ADD-COLUMN pattern); save_network accepts/preserves sector (COALESCE, never wiped to NULL). - peer_cohort.py (new): size-bucket derivation (micro/small/mid/large), compute_peer_percentile (+ median/quartiles), size-gated peer_benchmark with MIN_COHORT_SIZE=10, query_cohort over stored tier-3/tier-2 profiles (reuses precomputed metrics, recomputes nothing), peer_alpha_benchmark convenience, honest fallback note, and ingest_directory + CLI to grow a real cohort. - pdf_generator: wire peer benchmark into "5. Benchmarking & Position" (percentile when sufficient, else honest insufficient-cohort note); Fath anchor + ecological references unchanged. - tests/test_peer_cohort.py: percentile correctness, size-bucket boundaries, insufficient-cohort default, untagged-not-counted honesty guard, sufficient cohort, sector/size filters, ingestion path. Temp DB only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The published-value validation comparison layer compared engine magnitudes (Ascendency, Development Capacity, Overhead/Reserve, AMI, flow diversity) computed in nats against reference values quoted in bits for LOG2 networks. The prior nats->bits conversion was correct but wired via scattered `*_log2` name-mangled keys and did not cover statistical entropy (compared against a phantom 0.0 -> spurious 100% error). Replace it with a single, explicit, testable per-metric conversion: - `nats_to_bits(x) = x/ln2 = x*log2(e)` (nats->bits, right direction) - `BASE_DEPENDENT_METRICS` / `is_base_dependent()` classify which metrics scale with the log base; ratios/indices (alpha, robustness, TST) stay base-INVARIANT and are never converted - conversion applied ONLY for LOG2 base-dependent metrics; NATURAL and unknown/LOG10 bases pass through unchanged (no force-conversion) - map published `statistical_entropy` -> engine `flow_diversity` so H is compared against a real value instead of 0.0 Engine formulas and stored flow data are untouched. Note: the LOG2 networks still fail their magnitude checks because the stored internal-only flow matrices do not reproduce the papers' full-system values (this also breaks base-INVARIANT alpha, e.g. cone_spring 0.577 vs 0.505) -- a separate data-provenance issue, not a base mismatch. Adds tests/test_published_value_base.py (16 tests): conversion direction (hand-computed), base-dependence classification, LOG2 vs NATURAL vs unknown-base guards, base-invariant metrics untouched, and end-to-end wiring on cone_spring (bits) and prawns (nats). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Post-refactor (precompute, gradient reframe, metric sentinels) the analysis and
report paths crashed on several orgs. Reproduced each via a mock-Streamlit
harness + direct report-generator calls and fixed the root causes; added a
size-tiered guard so large networks no longer hang.
PART 1 — crash fixes (root causes):
- publication_report.py: report bracket-accessed keys absent on the cache-
reconstruction path (KeyError 'viability_lower_bound'/'viability_upper_bound',
trophic_depth, effective_link_density, is_viable). Added _ensure_metric_defaults()
in __init__ that backfills the indicative band edges (report_intelligence
VIABILITY_LOWER/UPPER = 0.2/0.6), derives is_viable, and coerces missing/None/
sentinel numeric metrics to safe floats. No formula changed; present values are
untouched. Also copies the incoming metrics dict so the caller's is not mutated.
- app.display_network_analysis: JSON-serialized profiles stringify int node-index
keys, so node_names[node_id] raised TypeError and centralities.get(i) silently
returned 0 for every node. Added _coerce_int_keys() / _node_label() helpers and
applied them to the centrality rankings, community membership and node table.
- Sentinel-safe formatting: added _safe_fmt() and applied it to every network
metric that PART 2 can turn into a sentinel string ('insufficient',
'skipped_large_graph', 'not_computed_large_graph') — small-world sigma/omega/
path length, rich-club, assortativity, robustness, path redundancy, flow — in
both display_network_analysis and _format_network_summary. network_analyzer
get_summary_report gained the same guard.
PART 2 — scale-aware guard (network_analyzer.py, thresholds APPROX=150, SKIP=600):
- betweenness: Brandes k-sample (k=min(n,100), seed=42) above threshold.
- closeness: deterministic k-node sample above threshold.
- small-world: skip average_shortest_path_length + sigma/omega with a
'not_computed_large_graph' sentinel (clustering stays exact).
- rich-club: skip the degree-preserving randomization with 'skipped_large_graph'.
- robustness: reduce random-failure simulations 10 -> 3.
- path redundancy: skip simple-path enumeration with 'skipped_large_graph'.
- get_all_metrics now reports computation_mode ('full'|'approximate') and
approximated_metrics.
True hang root cause: OASISCalculator.calculate_autocatalytic_index enumerated
ALL simple cycles then length-filtered post-hoc, so nx.simple_cycles walked an
exponential space (airport 100 nodes: ~119s; enzyme 336 nodes: effectively hung).
Now bounds cycle length at the generator via length_bound + a hard examine cap:
119s -> 0.02s. Enzyme full profile: hang -> ~1.5s.
Tests: tests/test_app_report_robustness.py — display functions + both report
generators clean on a small AND a large org, the viability-bounds KeyError
regression, get_all_metrics on a 300-node graph completes fast and reports
computation_mode='approximate' with the expected sentinels, and the formatting/
key-coercion helpers. Full suite: 301 passed.
Pressing Back then re-entering Use Sample Data snapped straight back to the previous analysis because selected_dataset_name (and the cached full_profile) persisted across the navigation and auto-re-selected the prior dataset. Clear both on back so the picker starts fresh. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Self-provisioning Gmail network source: admin OAuth (domain-wide delegation, metadata-only gmail.metadata scope), org-wide pull into a persisted gmail_interactions table (Stage 1), and a pure re-runnable weighting/build step (Stage 2) feeding the existing build_flow_matrix_from_edges -> provision_network -> get_full_profile path. Hybrid recency-decay x sustained-engagement weighting, both individual and department granularity, configurable window + half-life. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- §5.1 node resolution: individual=email, department=orgUnitPath leaf; filter external addresses to known org users (drop + report count) - BaseConnector conformance: explicit sync()/build() + get_flow_data wrapper - sync_run_id / now_utc minted in UI layer, never in reusable modules - add external-filtering test to the plan surface Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bite-sized TDD tasks for the two-stage Gmail connector: package skeleton, metadata-only SQLite store, pure hybrid weighting (Stage 2), GmailConnector auth+sync (Stage 1), legacy-stub retirement, Connect Gmail UI, full-suite regression. Each task ships tests + a commit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…standalone Eager imports in __init__ coupled all three submodules — importing gmail_store triggered imports of not-yet-created siblings, forcing throwaway stubs and making intermediate commits non-standalone. Resolve exports lazily via __getattr__ so each module imports independently and every commit stays green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…icit conn close, more tests Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…; doc fixes Guard half_life<=0 / beta<0 / window<0 (negative beta would inject negative flow weights, invalid in the Ulanowicz model). ImportError-only fallback. Correct 'ISO weeks' -> 7-day epoch windows in docstring. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rsing Store-level INSERT OR IGNORE on message-edge identity so re-syncing overlapping windows no longer double-counts flows (would inflate Ulanowicz weights). Atomic client assignment in authenticate. RFC 2822 address parsing via email.utils. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ector Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rash) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Full tests/ suite green: 321 passed (incl. ~22 new connector tests). Pre-existing validation/test_florida_bay.py collection error (relative-path script assuming CWD=validation/) is unrelated and out of scope. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ation, UI guard - gmail.metadata scope forbids the q search param (403); list SENT + filter internalDate client-side, preserving metadata-only. - per-user try/except so one bad mailbox doesn't abort the whole sync. - guard build_flow_matrix's NetworkIngestionError in the UI (empty/external-only window) with a friendly warning instead of a traceback. - replace deprecated datetime.utcnow() with datetime.now(timezone.utc). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR delivers three connected bodies of work on the OASIS engine and reporting, ending with a scientifically-validated correction pass:
docs/business-revision/), with a 7-dimension rubric, three-lens agent audit, scored gap matrix, benchmarking model, and prioritized roadmap.The headline: the core Ulanowicz information-theoretic mathematics was already correct (TST, AMI, Ascendency, Development Capacity, Overhead, α, and C=A+Φ are bitwise-unchanged by this PR). Every correction is in the derived / threshold / composite / presentation layers.
Scientific corrections (each expert-reviewed)
Track 1 — paper-backed / canonical (implemented):
d=1/flow, Brandes 2001) — this correctly feeds the OPEN dimension(n−1)²; small-world⟨k⟩=2m/n; ω lattice clustering; rich-club normalized; Katz adaptive α; log-base consistencyU=(I−D)⁻¹(direct + indirect), off-diagonal benefit:cost — Fath 2019 P8Track 2 — product/science decisions (expert-guided):
capped_byexplaining why.Not changed (deliberately)
No core Ulanowicz measure, no threshold constant (0.2/0.6, 1/e), and no α-optimum was blindly changed. An adversarial expert panel (mathematician, ecosystem-dynamics theorist, ENA methodologist) refuted a proposed global α-optimum swap (0.37→0.4596) as scientifically indefensible; it was reclassified to a documented calibration decision rather than applied.
Verification
docs/business-revision/evidence/FX-verification-report.mdEvidence & provenance
All validation reports, the 99-formula inventory, the six-family validation, the three expert-panel reports, the org-management PM guidance, and the identified-errors report (with PDF) are under
docs/business-revision/.🤖 Generated with Claude Code