perf+feat: pipeline optimization, inspectr fast loading, plot UI improvements#441

Open

astafan8 wants to merge 66 commits into

perf/datadict-copy-optimization

Commits on Apr 15, 2026

perf: optimize DataDict copy, validate, and pipeline data flow

Mikhail Astafev
and
Copilot
committed

Commits on Apr 16, 2026

docs: add further optimization opportunities to performance plan

Mikhail Astafev
and
Copilot
committed
perf: round 2 optimizations — node overhead, numtype, invalid entries

Mikhail Astafev
and
Copilot
committed
fix: resolve mypy type errors in validate() monotonicity check

Mikhail Astafev
and
Copilot
committed
docs: add real dataset benchmark results (23 QCodes datasets)

Mikhail Astafev
and
Copilot
committed
docs: add large dataset benchmark (array paramtype, 15-61 MB each)

Mikhail Astafev
and
Copilot
committed

Commits on Apr 17, 2026

docs: replace benchmarks with improved methodology (v2)

Mikhail Astafev
and
Copilot
committed
docs: add interactive action benchmark with per-node profiling

Mikhail Astafev
and
Copilot
committed
perf: optimize _find_switches() — 2.5x faster gridding

Mikhail Astafev
and
Copilot
committed
docs: add real experimental data benchmark

Mikhail Astafev
and
Copilot
committed

Commits on Apr 20, 2026

perf: optimize inspectr — lazy snapshot, incremental refresh

Mikhail Astafev
and
Copilot
committed
perf: fast DB loading via load_by_id, bypass experiments enumeration

Mikhail Astafev
and
Copilot
committed
feat: add loading progress and status text to inspectr

Mikhail Astafev
and
Copilot
committed
fix: overlay text stuck after reloading same DB file

Mikhail Astafev
and
Copilot
committed
fix: collapse info pane by default, enable smooth scrolling

Mikhail Astafev
and
Copilot
committed
feat: add plot backend selector to inspectr toolbar

Mikhail Astafev
and
Copilot
committed
perf: add fast SQL-based DB overview, wider default window

Mikhail Astafev
and
Copilot
committed
feat: grid layout for pyqtgraph subplots (matching matplotlib)

Mikhail Astafev
and
Copilot
committed
feat: scrollable plot area toggle for both backends

Mikhail Astafev
and
Copilot
committed
fix: reduce matplotlib scrollable min height to 100px per row

Mikhail Astafev
and
Copilot
committed
feat: scrollable off by default, add min height spinbox

Mikhail Astafev
and
Copilot
committed
docs: reorganize PERFORMANCE_PLAN.md into implemented vs future

Mikhail Astafev
and
Copilot
committed
fix: add plotWidgetClass parameter to autoplotQcodesDataset

Mikhail Astafev
and
Copilot
committed
refactor: extract hint constant, clean up backend selector mapping

Mikhail Astafev
and
Copilot
committed
fix: address code review — timestamps, connections, dead code

Mikhail Astafev
and
Copilot
committed
feat: LaTeX-to-HTML conversion for pyqtgraph plot labels

Mikhail Astafev
and
Copilot
committed
fix: use HTML sub/sup instead of Unicode subscript/superscript

Mikhail Astafev
and
Copilot
committed
fix: only apply LaTeX conversion when actual LaTeX syntax is present

Mikhail Astafev
and
Copilot
committed

Commits on Apr 21, 2026

fix: resolve all mypy errors and add hypothesis to test deps

Mikhail Astafev
and
Copilot
committed
fix: mypy cross-stubs compat (PyQt5-stubs in CI vs PyQt6 locally)

Mikhail Astafev
and
Copilot
committed
fix: update qcodes import to current public API, remove stale ignore

Mikhail Astafev
and
Copilot
committed
fix: address review — rettype assert, scaleunits import, mypy config

Mikhail Astafev
and
Copilot
committed
fix: address PR review — 6 Copilot review comments

Mikhail Astafev
and
Copilot
committed

Commits on Apr 30, 2026

docs: add real-data profiling results (large complex 2D dataset)

Mikhail Astafev
and
Copilot
committed
perf: fix is_invalid() for numeric dtypes, default to metadataShape

Mikhail Astafev
and
Copilot
committed
perf: is_invalid 44x faster, fix mpl double-replot, label() skip validate

Mikhail Astafev
and
Copilot
committed
docs: update plan with implemented fixes and backend comparison

Mikhail Astafev
and
Copilot
committed

Commits on May 1, 2026

fix: records counter, is_invalid 44x, mpl double-replot, label skip validate

Mikhail Astafev
and
Copilot
committed
fix: reset imagData flag, equal grid stretch for aspect ratio

Mikhail Astafev
and
Copilot
committed
fix: regressions + feat: selection buttons

Mikhail Astafev
and
Copilot
committed
fix: recursion in setSelectedData, records from shapes, button layout

Mikhail Astafev
and
Copilot
committed
chore: remove unused imports from test_regressions

Mikhail Astafev
and
Copilot
committed
chore: remove data file references, use tmp_path in all tests

Mikhail Astafev
and
Copilot
committed
fix: mpl blank plot, pyqtgraph grid resize; reorganize tests

Mikhail Astafev
and
Copilot
committed
fix: deselect-all clears plots, pyqtgraph min size for font warning

Mikhail Astafev
and
Copilot
committed
feat: mpl colormap selector, pyqtgraph complex mode tests

Mikhail Astafev
and
Copilot
committed
fix: pyqtgraph image axis orientation, deselect-all UX

Mikhail Astafev
and
Copilot
committed
fix: rename deselect-all to 'Select first only' button

Mikhail Astafev
and
Copilot
committed
ci: trigger CI after history rewrite

Mikhail Astafev
and
Copilot
committed
Merge remote-tracking branch 'origin/master' into perf/datadict-copy-optimization
Mikhail Astafev
committed
chore: remove inspectr from mypy warn_unused_ignores override

Mikhail Astafev
and
Copilot
committed
fix: use plottr Qt imports instead of PyQt6 in tests

Mikhail Astafev
and
Copilot
committed
fix: ParamSpecBase import compat for older qcodes in CI

Mikhail Astafev
and
Copilot
committed

Commits on May 19, 2026

Merge remote-tracking branch 'origin/master' into perf/datadict-copy-optimization
Mikhail Astafev
committed

Commits on May 20, 2026

fix: refresh updates incomplete dataset records + persist plot backend

Mikhail Astafev
and
Copilot
committed
fix: handle metadata-only datasets gracefully (no KeyError)

Mikhail Astafev
and
Copilot
committed
fix: show missing-data-file message, clear stale run list on reload

Mikhail Astafev
and
Copilot
committed
fix: prominent no-data label in plot area, skip reload of same DB

Mikhail Astafev
and
Copilot
committed
fix: allow initial DB load when path is pre-set from cmdline

Mikhail Astafev
and
Copilot
committed

Commits on May 21, 2026

fix: always do full reload in loadFullDB, no skip optimization

Mikhail Astafev
and
Copilot
committed
fix: loadFullDB fully resets UI state (dates, runs, dbdf)

Mikhail Astafev
and
Copilot
committed
fix: move no-data banner to top of autoplot window

Mikhail Astafev
and
Copilot
committed
fix: make no-data banner text selectable and copy-pastable

Mikhail Astafev
and
Copilot
committed
fix: remove no-data banner once data arrives on refresh

Mikhail Astafev
and
Copilot
committed
refactor: generic showWarningBanner/removeWarningBanner on AutoPlotMainWindow

Mikhail Astafev
and
Copilot
committed
fix: address PR review — 14 comments

Mikhail Astafev
and
Copilot
committed