Skip to content

refactor: remove the matplotlib plotting layer - #512

Merged
danielmarv merged 7 commits into
mainfrom
refactor/remove-plotting
Oct 4, 2026
Merged

danielmarv merged 7 commits into
mainfrom
refactor/remove-plotting

Conversation

@exploreriii

@exploreriii exploreriii commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Description:

edit: sorry for the spam

Since #490 the dashboard has drawn every chart from the JSON chart documents the data API emits from pipeline CSVs. The PNGs had no reader left, but the Python side still treated the PNG as the primary key for a chart. This PR removes that plumbing and the plotting layer with it.

  • Delete src/hiero_analytics/plotting/ and its tests, export/save.plot_and_save, the provenance PNG footer, and every styling constant only a rendered image needed. Drop matplotlib (and transitively pillow, contourpy, kiwisolver, fonttools, cycler, pyparsing). networkx stays: the network layout moves to analysis/network_layout.py and ships as data.
  • Chart specs are keyed by chart id (the former PNG stem, so chart document filenames and the dashboard's URL state are unchanged). Spec keys renamed: files → variants, interactive_sources → sources. FULL_ROW_CHARTS declares the full-row layout that was measured from image dimensions before.
  • The data API lists a variant when every CSV its source names exists, the sidecar does not record zero rows, and the chart document has something to draw. Variants lose file, width, height; interactive is always present.
  • The organisation palette moves out of pipelines/affiliation.py into config.charts + a pure analysis.affiliation.segment_colors, removing an export → pipelines import.
  • The two pipelines that only ever drew PNGs now persist their frames: onboarding writes onboarding_signal.csv and onboarding_efficiency.csv; contributor_churn writes funnel, retention, transitions and tenure tables (and contributor_progression.csv now goes through save_dataframe, author as a column). contributor_profiles adds max_difficulty_distribution.csv.
  • Web: ChartVariant.file removed, interactive required, the no-data placeholder removed, /charts dev proxy removed. CI stops copying outputs/charts into the Pages site. Docs describe provenance as sidecars + dashboard header/footer + snapshot.

Behaviour changes worth knowing:

  • A chart card whose dataset was never produced is no longer listed because a stale PNG happened to exist (the old gate could list an August image with no data behind it).

@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

📊 Dashboard preview

The dashboard was built for this PR.

➡️ Download dashboard-preview-512, unzip, then serve it with python3 -m http.server -d . and open the URL it prints. (The app fetches its data over HTTP, so opening index.html directly shows an empty page.)

The artifact is built from this PR — treat it as contributor-authored code and glance at the diff before opening it. It expires after 7 days and is replaced on each new push.

@exploreriii
exploreriii force-pushed the refactor/remove-plotting branch from 3fdf880 to 1d01dc0 Compare October 3, 2026 17:31
@exploreriii
exploreriii marked this pull request as draft October 3, 2026 17:35
@exploreriii
exploreriii force-pushed the refactor/remove-plotting branch 2 times, most recently from 262adc5 to 4dd56dd Compare October 3, 2026 17:47
Signed-off-by: exploreriii <133720349+exploreriii@users.noreply.github.com>
…erged HIP work

Restores the gate the retired renderer applied: zero rows stay in the
dataset, but a chart in which every value is zero is not listed. The rule
is now a declarative source option, hide_when_all_zero, so it lives in
the spec rather than in a pipeline.

Signed-off-by: exploreriii <133720349+exploreriii@users.noreply.github.com>
…c cannot reach

28 of 35 entries were already listed through chart sources or table
sections, which let the orphan check cover an output after the spec
stopped reading it. A guard test keeps the list from re-growing.

Signed-off-by: exploreriii <133720349+exploreriii@users.noreply.github.com>
Signed-off-by: exploreriii <133720349+exploreriii@users.noreply.github.com>
427 errors across 50 files, almost all pandas type-inference unions that
pyright cannot resolve without stubs; the report-only step produced
noise rather than signal. Ruff remains the Python gate.

Signed-off-by: exploreriii <133720349+exploreriii@users.noreply.github.com>
analysis is documented as I/O-free, but analysis.affiliation read
data/affiliations.yaml itself. AFFILIATIONS_PATH, load_affiliations and
load_manual_logins now live in data_sources.affiliations; the analysis
module takes the mapping as data, as every other transform does.

Signed-off-by: exploreriii <133720349+exploreriii@users.noreply.github.com>
The lint workflow checks .clusterfuzzlite/requirements.txt against uv.lock
(uv export); dropping matplotlib changed the lock without regenerating it.

Signed-off-by: exploreriii <133720349+exploreriii@users.noreply.github.com>
@exploreriii
exploreriii force-pushed the refactor/remove-plotting branch from 4dd56dd to f9a0e87 Compare October 3, 2026 17:51
@MonaaEid
MonaaEid marked this pull request as ready for review October 3, 2026 22:35
Copilot AI balanced review requested due to automatic review settings October 3, 2026 22:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@danielmarv
danielmarv merged commit 62a5652 into main Oct 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants