Skip to content

Split CI benchmarks into PR, scheduled, and manual tiers #47

Description

@parthdongre

Summary

Restructure performance CI so ordinary development gets fast regression feedback while very large physical stress tests run on an intentional cadence.

Why

FrameVitals has strong benchmark coverage, including billion-cell-scale workloads, but the largest workflows are too expensive to treat like normal per-push CI. Separating tiers will reduce noise and cost without weakening performance validation.

Proposed tiers

  1. PR guardrail: small deterministic performance regressions with short runtime.
  2. Scheduled benchmark: broader representative workloads on a daily/weekly cadence.
  3. Manual stress: extreme physical-scale runs through workflow_dispatch before major releases or architecture changes.

Acceptance criteria

  • PR benchmark jobs have a documented runtime budget.
  • Extreme multi-billion-cell workloads do not run on every normal development push.
  • Scheduled benchmarks retain representative native/fallback validation.
  • Manual stress workflows remain easy to run before releases.
  • Benchmark artifacts/results remain reproducible and attributable to a commit.
  • Documentation explains which benchmark tier should be used for each kind of change.

Priority: P0 — CI efficiency / performance
Suggested labels: performance, ci, infrastructure

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions