Skip to content

[Feature]: Run real polyglot toolchains on a schedule #41

Description

@Baziar

Problem statement

The runtime acceptance contract covers a broad command matrix, and official
generators have scheduled smoke coverage. That does not yet prove each advertised
runtime against a real pinned toolchain on every declared platform.

Proposed solution

Current foundation

  • runtime-acceptance-matrix.mjs contract/full modes;
  • cached pre-push runtime-contract gate;
  • cross-platform CLI, workspace E2E, Windows bridge, and generator workflows;
  • capability/support-tier contracts and explicit unsupported results;
  • an isolated real-world qualification harness that validates Live Board output
    alongside adoption, Model, Graph, Doctor, and consumer evidence.

Remaining work

Add scheduled jobs for Node, Python, Go, Rust, Java, .NET, PHP, Deno, and a weekly
C/C++ extension. Setup may use the network; product assertions run offline after
toolchains/fixtures are prepared.

Acceptance criteria

  • One versioned matrix maps support claims to pinned toolchain jobs.
  • Linux covers all primary runtimes; macOS/Windows cover declared subsets.
  • Each job runs adopt, status, strict intelligence, and runtime build/test or an explicit unsupported result.
  • Cache keys include OS, toolchain line, lock/fixture identity, and safe fallback.
  • A compact JSON artifact records runtime, OS, phase, verdict, and evidence.
  • Artifacts upload even after later failure and follow a retention policy.
  • Quarantine requires owner, reason, and expiry; retries cannot hide product failures.
  • Scheduled failures create a visible triage signal.

Completion evidence

Close after consecutive successful scheduled runs and a documented local
reproduction command for every runtime job.

Alternatives considered

The following alternatives or adjacent concerns are intentionally outside this issue:

  • running every toolchain on every pull request;
  • duplicating the official generator workflow;
  • treating third-party network failure as product success or failure.

Expected impact

Runtime support claims stay backed by current real-toolchain evidence, while PR
feedback remains bounded and scheduled failures remain actionable.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: ciCI, runtime, or platform matricesneeds-triageScope, owner, or scheduling is not confirmed

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions