Relay: a CI/CD engine and DAG scheduler in Rust

Relay is a from-scratch CI/CD engine in Rust you can actually read, built around a pure, deterministic DAG scheduler over an abstract executor. Most CI systems bury their scheduler behind YAML, network calls, and container runtimes, so the one interesting part, how jobs get ordered, parallelized, skipped, and cached, is impossible to study or unit-test in isolation. Relay pulls that part out so parallelism, failure propagation, and upstream-aware content-hash caching can all be tested without running a single real process. Zero external dependencies, pure Rust standard library.

Live demo · MIT licensed · pure Rust

Built from scratch by Pavan Nallamothu (LinkedIn, GitHub).

It pairs with Metronome, a from-scratch scheduler: Metronome decides when work runs over time, Relay decides in what order a graph of work runs and what can run at once.

The point

The core is one function:

schedule::run(&pipeline, &mut executor, limits) -> Report

The scheduler never spawns a process or a thread. It drives an Executor trait in virtual time. That single seam is what makes it testable: swap in a MockExecutor that returns scripted outcomes and records what ran concurrently, and you can assert every scheduling rule deterministically. Swap in the ShellExecutor and the same scheduler runs real shell commands.

What the scheduler guarantees:

• Dependency order. A job starts only after every job in its needs has finished successfully.
• Real parallelism. Up to N jobs run at once; the mock proves the peak concurrency is exactly N and never more.
• Failure propagation. A failed job marks its transitive dependents SKIPPED and they never run. Independent branches keep going. A continue-on-error job that fails does not skip its dependents.
• Content-hash caching. A job's cache key is a hand-rolled FNV-1a digest of its config plus the content of its declared input paths. The key is also upstream-aware. It folds in the resolved key of every dependency and the artifacts that dependency handed down, so a change in a non-cached upstream job busts the downstream key too. A stored match marks the job CACHED and skips execution, restoring its recorded outputs. Change an input and the key busts.
• Determinism. Same pipeline, same result and same order, every run. Ties break by config order.

Execution and isolation

The ShellExecutor gives every job its own working directory under base_dir/.relay-work/<job>. Before a job runs, the artifacts produced by its dependencies are materialized into that directory, so cross-job files travel through the documented produces to inputs mechanism rather than a shared current directory. Jobs cannot clobber each other, and a dependent reads its inputs even though it runs in a different directory from the producer.

Timeouts kill the whole process group, not just the direct sh child. A step that backgrounds a long-lived child (for example sleep 20 & echo x) can no longer hold a pipe open and outlive the deadline. Both the per-step timeout and the per-job timeout reap the entire group with SIGKILL, and the reader threads are bounded so they never block past the deadline.

The pipeline format

A readable text format with a parser and a matching printer that round-trip.

job build {
  needs fetch
  env RUST_LOG = debug
  step cargo build
  produces target/app
  timeout 300
  cache {
    key v1
    paths src, Cargo.toml
  }
  continue-on-error
}

• needs lists dependency job names.
• env sets one key/value pair per line.
• step is a shell command. Steps run in order and stop at the first failure.
• produces lists output files collected as artifacts for dependent jobs.
• timeout is a whole-job wall-clock cap in seconds. It is separate from the executor's per-step timeout and complements it. Whichever fires first stops the job.
• cache { key, paths } declares the inputs whose content decides the key.
• continue-on-error lets a failure not skip dependents.

Declared paths (produces and cache.paths) must stay inside the workspace. An absolute path or any .. component is rejected at parse time, so a pipeline cannot read or clobber files outside its base directory.

CLI

relay run   <pipeline> [--jobs N] [--fail-fast]   run it, print each status and a summary
relay plan  <pipeline> [--jobs N]                 print the DAG and wave order, run nothing
relay graph <pipeline>                            print the dependency graph

Example:

$ relay plan examples/diamond.relay --jobs 2
plan: examples/diamond.relay (4 jobs, --jobs 2)

dependency graph:
  fetch (root)
  build <- fetch
  lint <- fetch
  test <- build, lint

execution waves (dependency order, up to 2 at once):
  wave 0: fetch
  wave 1: build, lint
  wave 2: test

$ relay run examples/failure.relay
relay run: examples/failure.relay (--jobs 4)
  [ 0] setup                ok       success
  [ 1] unit                 xx       failed
  [ 2] docs                 ok       success
       package              --       skipped
       deploy               --       skipped
summary: 2 success, 0 cached, 1 failed, 2 skipped

Layout

• pipeline.rs config data model, parser, and printer
• dag.rs graph build, cycle detection, topological waves
• schedule.rs the pure deterministic scheduler
• executor.rs the Executor trait, ShellExecutor, and the Artifacts store
• cache.rs the content-hash cache key and store
• hash.rs hand-rolled FNV-1a
• report.rs per-job status, wave, timing, logs
• mock.rs the scripted executor used by tests
• main.rs the CLI

Tests

cargo test
cargo clippy --all-targets -- -D warnings

The suite covers the scheduler over the mock (dependency order, concurrency limit, skip propagation, continue-on-error, fail-fast, determinism, waves), graph validation (cycles, missing deps, duplicates), caching (hit, bust, determinism, upstream-aware busting), config round-trip, path confinement, job name validation, per-step and per-job timeouts that reap backgrounded children, and real integration tests that run actual shell commands in a temp dir and deliver an artifact from a producer into a dependent's separate working directory.

Non-goals

Relay is a readable reference engine, not a production CI platform. It does not do distributed or remote runners, containers or sandboxing, secrets management, a web UI or server, log streaming to a service, retries and backoff, artifact storage beyond the local cache, or a matrix expansion syntax. The ShellExecutor runs a scheduled parallel plan sequentially through the single-threaded executor seam; genuine multi-core execution of shell jobs is left out on purpose to keep the executor trait simple and the scheduler pure. See DESIGN.md.

Author

Pavan Nallamothu (pavanchow). MIT licensed.