Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,12 +33,8 @@ jobs:
# https://github.com/leynos/wireframe/actions/runs/39038420830). That is
# about 29% less per run for 3m40s more wall time.
#
# The coverage action supplies a 180-second per-test deadline. The
# aggregate tests/compile_error.rs test builds and executes twelve
# trybuild fixtures under coverage; its cold nested build exceeded that
# deadline even after the ordinary tests completed. .config/nextest.toml
# extends only this test to eight minutes, preserves the ordinary
# deadline, and retains the ten-minute whole-run ceiling.
# Compiler-driving UI fixtures have a separate bounded step. Runtime
# coverage does not start their nested trybuild compilation.
#
# Disk is the margin to watch: the minimum free space on two vCPU was
# 3,998 MiB, against 78,392 MiB on four.
Expand All @@ -62,14 +58,16 @@ jobs:
${{ (github.event.pull_request.head.repo.fork
|| github.event.pull_request.user.login == 'dependabot[bot]')
&& 'ubuntu-latest' || 'ubicloud-standard-2' }}
timeout-minutes: 30
timeout-minutes: 60
permissions:
contents: read
env:
CARGO_TERM_COLOR: always
BUILD_PROFILE: debug
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
with:
persist-credentials: false
- name: Setup Rust
uses: leynos/shared-actions/.github/actions/setup-rust@8e6a325765ba382691d1b1c65fcccf1fc1d1b00c
- name: Install development backend and linker
Expand All @@ -80,6 +78,10 @@ jobs:
sudo apt-get update
sudo apt-get install -y mold
mold --version
# This compiler-driven suite is bounded separately from runtime coverage.
- name: Validate compile-time APIs
timeout-minutes: 30
run: make test-ui
- name: Install mdtablefix
# `make check-fmt` runs `mdtablefix --check --git`; 0.6.0 is the
# first release with `--check` and `--git`.
Expand Down
9 changes: 4 additions & 5 deletions .github/workflows/coverage-main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,10 @@ jobs:
# arm: `push` carries no pull request, so the guard would be constant
# here and a constant guard reads as a decision nobody made.
#
# Four vCPU, and not measured at two: this lane runs the same
# `generate-coverage` over the same workspace as `ci.yml`, whose
# `build-test` now runs on two vCPU after a warm run passed the trybuild
# case that nextest had killed at 180 s there. This lane's own 240 s
# sample was taken on Namespace's shared profile, which is four vCPU and
# Four vCPU, and not measured at two: this lane runs the same runtime
# coverage as `ci.yml`; compile-time UI checks run in a separate CI step.
# This lane's own 240 s sample was taken on Namespace's shared profile,
# which is four vCPU and
# 16 GB, and it has not been sampled on two. It runs on merge pushes
# with nobody watching, so moving it needs its own measurement.
runs-on: ubicloud-standard-4
Expand Down
1 change: 0 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 7 additions & 7 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ tokio = { version = "1.47.1", default-features = false, features = [
"time",
"io-util",
] }
tokio-util = { version = "0.7.16", features = ["rt", "codec"] }
tokio-util = { version = "0.7.16", features = ["rt", "codec", "io"] }
tokio-stream = "0.1.17"
futures = "0.3.31"
async-trait = "0.1.89"
Expand All @@ -47,17 +47,17 @@ log = "0.4.28"
dashmap = "6.1.0"
leaky-bucket = "1.1.2"
tracing = { version = "0.1.41", features = ["log", "log-always"] }
tracing-subscriber = "0.3.18"
metrics = { version = "0.24.2", optional = true }
monotony = "1.0.0"
thiserror = "2.0.16"
itoa = "1.0"
static_assertions = "1.1.0"
derive_more = { version = "2.0.1", features = ["display", "from"] }
socket2 = "0.6.0"
bb8 = { version = "0.9.1", optional = true }

[dev-dependencies]
itoa = "1.0"
tracing-subscriber = { version = "0.3.18", features = ["env-filter"] }
rstest = "0.27.0"
rstest-bdd = "0.5.0"
rstest-bdd-macros = { version = "0.5.0", features = ["strict-compile-time-validation"] }
Expand Down Expand Up @@ -95,12 +95,12 @@ tokio = { version = "1.47.1", default-features = false, features = [
[target.'cfg(loom)'.dependencies]
loom = "0.7.2"

[target.'cfg(not(loom))'.dependencies]
metrics-exporter-prometheus = { version = "0.18.3", optional = true, features = ["http-listener"] }
[target.'cfg(not(loom))'.dev-dependencies]
metrics-exporter-prometheus = { version = "0.18.3", features = ["http-listener"] }

[features]
default = ["metrics", "serializer-bincode"]
metrics = ["dep:metrics", "dep:metrics-exporter-prometheus"]
metrics = ["dep:metrics"]
pool = ["dep:bb8"]
serializer-bincode = []
serializer-serde = []
Expand Down Expand Up @@ -145,7 +145,7 @@ big_endian_bytes = "deny"
unknown_lints = "deny"
renamed_and_removed_lints = "deny"
missing_docs = "deny"
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(loom)'] }
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(loom)', 'cfg(coverage)'] }

[workspace.lints.rustdoc]
missing_crate_level_docs = "deny"
Expand Down
1 change: 0 additions & 1 deletion crates/wireframe-verification/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ workspace = true

[dependencies]
stateright = "0.31.0"
wireframe = { path = "../..", features = ["test-support"] }

[dev-dependencies]
rstest = "0.27.0"
62 changes: 40 additions & 22 deletions docs/developers-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,19 @@ than all four repository-owned lanes sharing one profile.

| Workflow | Job | Trigger | Runner | Ceiling |
| ------------------------ | ------------------- | ------------------ | --------------------- | ------- |
| `ci.yml` | `build-test` | pull request, push | `ubicloud-standard-2` | 30 min |
| `ci.yml` | `build-test` | pull request, push | `ubicloud-standard-2` | 60 min |
| `coverage-main.yml` | `coverage-upload` | push | `ubicloud-standard-4` | 20 min |
| `advanced-tests.yml` | `advanced` | schedule | `ubuntu-latest` | 60 min |
| `delayed-pr-comment.yml` | `delay_and_comment` | dispatch | `ubuntu-latest` | none |

*Table 1: Where each repository-owned lane runs, and its ceiling.*

The required `build-test` job includes an unconditional, bounded 30-minute
`make test-ui` step after Rust and linker setup and before runtime coverage.
This keeps compile-time API validation within the existing required status,
without a job-level condition that could leave branch protection without a
failure result.

Pull-request, push and tag lanes move to Ubicloud, because those are the ones a
developer waits on: `build-test` waited 461 seconds for a GitHub-hosted runner
on 2026-09-16 to do 556 seconds of work. Scheduled and delayed-comment lanes
Expand Down Expand Up @@ -87,15 +93,21 @@ Two vCPU cost about 29% less per run for 3m40s more wall time, and memory is
far inside the 7.9 GiB that shape has.

The coverage action supplies a 180-second per-test deadline and a ten-minute
whole-run ceiling. `tests/compile_error.rs` runs fourteen trybuild fixtures
inside one test, including nested Cargo builds. A cold coverage run exceeded
that deadline on two vCPU after the other tests had completed
([PR #777's coverage run](https://github.com/leynos/wireframe/actions/runs/39203098537)).
The checked-in `.config/nextest.toml` preserves the action's deadlines for
ordinary tests and extends only `wireframe::compile_error::compile_tests` to
eight minutes. The ten-minute ceiling still bounds the coverage test run. The
four-vCPU `coverage-upload` lane remains on its measured shape; the smaller
runner has not been sampled with this per-test budget.
whole-run ceiling. `tests/compile_error.rs` runs trybuild fixtures inside one
test, including nested Cargo builds, which do not measure runtime behaviour and
can exceed that per-test deadline on a cold two-vCPU runner. An unconditional
`make test-ui` step runs in `build-test` for pull requests and pushes to
`main`, before runtime coverage, with a separate 30-minute step ceiling. This
keeps the compile-time API fixtures covered without making the instrumented
runtime- coverage run compile them.

The compile-time harnesses are excluded only when `cfg(coverage)` is active;
the fixtures remain in `tests/ui/`, and `make test-ui` runs both the root and
Loom API suites. The checked-in `.config/nextest.toml` retains its eight-minute
override for `wireframe::compile_error::compile_tests` during ordinary,
noncoverage nextest runs. The ten-minute ceiling still bounds the coverage test
run. The four-vCPU `coverage-upload` lane remains on its measured shape; the
smaller runner has not been sampled for coverage generation.

Free disk is the margin to watch: 3,998 MiB at its lowest on two vCPU.

Expand All @@ -112,12 +124,15 @@ default is the expensive failure mode. A ceiling close to the measured work is
the other one, because it cancels the run at the moment an overrun becomes
interesting and discards the log that would explain it.

Two ceilings are measured: `build-test` at 556 seconds on four vCPU (750
seconds on two, still well inside 30 minutes) and `coverage-upload` at 240
seconds on four vCPU. Two are judgements, and the guide says so rather than
implying otherwise. `advanced` has failed every night since at least 2026-09-09
and its last green run was 2025-10-04 at 32 seconds, so 60 minutes is generous
enough not to mask the repair when it lands.
The historical `build-test` sample was 556 seconds on four vCPU and 750 seconds
on two; the `coverage-upload` sample was 240 seconds on four vCPU. The 60-minute
`build-test` job ceiling is a judgement, not a measured limit: it allows the
existing lane and its new bounded UI step to finish, including cold compiler
work. The UI step's 30-minute ceiling is also a judgement. The four-vCPU
`coverage-upload` lane remains capped at 20 minutes. `advanced` has failed
every night since at least 2026-09-09 and its last green run was 2025-10-04 at
32 seconds, so 60 minutes is generous enough not to mask the repair when it
lands.

`delay_and_comment` declares no ceiling, deliberately. Its entire duration is a
`sleep` of the caller's `delay_minutes` input, so any fixed ceiling cancels a
Expand Down Expand Up @@ -1023,10 +1038,10 @@ The Loom, verification, and Whitaker commands keep their own `RUSTFLAGS`, as
the development and release recipes to the standard, and
`build_standard_ci_test.py` holds the CI install order. The coverage lane
(`coverage-main.yml:coverage-upload`) installs `mold` through `setup-rust`'s
`install-mold` input before `generate-coverage`, because the trybuild cases
compile scratch crates under `target/` that link with `mold` through the Cargo
configuration. The same file requires every Linux job that runs
`generate-coverage` to install `mold` first, by that input or by `apt`.
`install-mold` input before `generate-coverage`, because `.cargo/config.toml`
selects `mold` for Linux builds, including the instrumented runtime tests. The
same file requires every Linux job that runs `generate-coverage` to install
`mold` first, by that input or by `apt`.

The fragment configures the backend only and declares no `[target.<...>]`
table, so a direct `--config tools/dev-fast/config.toml` invocation behaves the
Expand Down Expand Up @@ -1199,6 +1214,9 @@ tracing and the current-thread Tokio runtime explicitly, then use
`WireframeServer::from_app` for server preparation, listener binding,
connection spawning, and shutdown-aware accept loops. Keep example-specific
address parsing, app construction, handlers, and middleware in the example file.
`tracing-subscriber` and the Prometheus exporter are development dependencies
for examples, tests, and doctests; applications supply their own tracing
subscriber and exporter.

Codec benchmark helpers live in `wireframe_testing::codec_benchmarks`. Bench
targets, direct unit tests, and BDD fixtures should import the workload matrix,
Expand Down Expand Up @@ -1459,8 +1477,8 @@ Place new snippets in `tests/ui/`, register them in `tests/compile_error.rs`,
and commit the generated `.stderr` file for compile-fail cases after verifying
that the diagnostics describe the intended contract. These files are exact
`rustc` diagnostic snapshots: generate and refresh them with the repository's
pinned `rust-toolchain.toml`. Normal validation is
`cargo test --test compile_error`; a deliberate developer-only refresh uses
pinned `rust-toolchain.toml`. Normal validation is `make test-ui`; a deliberate
developer-only refresh uses
`TRYBUILD=overwrite cargo test --test compile_error`. Review refreshed fixtures
as maintenance changes, and never run the overwrite command in CI.

Expand Down
26 changes: 26 additions & 0 deletions docs/multi-layered-testing-strategy.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,32 @@ generated `lcov.info` report to Codecov using a pinned version of the Codecov
GitHub Action (`18283e04ce6e62d37312384ff67231eb8fd56d24`, corresponding to
v5.4.3) to make coverage visible across pull requests.

## 1.1 Layer 0: Compile-Time API Boundary

**Objective:** To verify that the public API rejects the code it is supposed to
reject, and does so with the diagnostic the documentation promises. Runtime
tests cannot observe this: an API that should fail to compile is, by
construction, absent from any test binary that does compile.

Two harnesses cover the boundary. `tests/compile_error.rs` drives trybuild
fixtures in `tests/ui/` for the root crate, in both `pass` and `compile_fail`
modes. `wireframe_testing/tests/loom_api_surface.rs` is compiled twice, once
with `--cfg loom` and once without, to prove that Loom-only exports are absent
from ordinary builds and present under instrumentation.

`make test-ui` runs both harnesses. It runs in the required `build-test` job as
a bounded step *without* coverage instrumentation, because the nested `cargo`
invocations inside trybuild and the compile checks would otherwise report
coverage for code no test executed. Both harnesses therefore carry
`#![cfg(not(coverage))]`, and the coverage lane excludes only those two nested
compiler drivers. The runtime suite around them stays instrumented and
measured, so a green UI gate and a green coverage run answer different
questions rather than duplicating each other.

**Measurable Objective:** Every compile-fail fixture produces its committed
`.stderr` diagnostic, both Loom polarity checks build, and no runtime test is
excluded from coverage.

## 2. Layer 1: Foundational Correctness (Unit & Integration)

**Objective:** To verify that each component behaves correctly in isolation and
Expand Down
52 changes: 52 additions & 0 deletions docs/users-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -1568,6 +1568,58 @@ worker tasks.[^20][^37][^38] `ServerError` surfaces bind, factory-build, and
application-preparation failures as typed errors so callers can react
appropriately.[^21]

## Metrics

Wireframe's default `metrics` feature emits instrumentation through the
`metrics` facade but does not install a global recorder. The Prometheus
exporter is a development dependency of Wireframe, so applications that want
Prometheus metrics must add `metrics-exporter-prometheus` as a direct
dependency and install its recorder once during startup, before Wireframe
begins emitting metrics:

```rust,no_run
use std::time::Duration;

use metrics_exporter_prometheus::PrometheusBuilder;

fn main() -> Result<(), Box<dyn std::error::Error>> {
let prometheus = PrometheusBuilder::new().install_recorder()?;

// Keep the handle: the upkeep task holds a clone of it.
let upkeep_handle = prometheus.clone();
std::thread::spawn(move || {
loop {
std::thread::sleep(Duration::from_secs(5));
upkeep_handle.run_upkeep();
}
});

// Start the application after installing the recorder.
let _scrape_output = prometheus.render();
Ok(())
}
```

Keep the returned handle where the application can expose or render the
Prometheus output. Applications choose how to serve or otherwise collect it.

Wireframe depends on the `tracing` facade, which emits events only when the
application installs a subscriber; it does not ship one. `tracing-subscriber`
was previously a runtime dependency of Wireframe and is now a development
dependency, so an application that wants Wireframe's events must add
`tracing-subscriber` itself and install a subscriber once during start-up,
before the application begins emitting. Omit it and the instrumentation still
compiles, but every event is discarded.

`install_recorder` returns a `PrometheusHandle` but does **not** start the
exporter's background upkeep task; only `PrometheusBuilder::build` and
`PrometheusBuilder::install` spawn it. The task drains histogram buckets and
other self-maintaining structures, so without it that storage grows for as long
as no scrape triggers a render. Call `PrometheusHandle::run_upkeep` on a timer
(for example, every five seconds, matching the exporter's own default
interval), or serve metrics through `PrometheusBuilder::build`/`install` and
let the exporter own the task.

## Client runtime

`WireframeClient` provides a first-class client runtime that mirrors the
Expand Down
29 changes: 29 additions & 0 deletions docs/v0-3-0-to-v0-4-0-migration-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,35 @@ mutable drivers remain available as deprecated compatibility paths; migrate
tests to the prepared helpers when they need to prove one-time middleware
transformation or reuse prepared route services.

## Install your own tracing subscriber

`tracing-subscriber` is no longer a runtime dependency of Wireframe; it is a
development dependency used by the examples, benchmarks and tests. The
`tracing` facade Wireframe emits through stays a runtime dependency, but it has
no subscriber of its own, so a released application that relied on Wireframe
pulling in a subscriber must now declare one and install it during start-up.

Add the crate to the application's own manifest:

```toml
[dependencies]
tracing-subscriber = { version = "0.3.18", features = ["env-filter"] }
```

Then initialize it before any Wireframe work begins:

```rust
fn main() {
tracing_subscriber::fmt().init();
// Start the application here.
}
```

Applications that never configured a subscriber see no behaviour change: they
were discarding Wireframe's events before this release too. Applications that
did configure one keep working, because installing an application-owned
subscriber is now the supported arrangement rather than a duplicate.

## Migrate byte-handling APIs

The prepared-application transition is independent of the zero-copy byte
Expand Down
Loading
Loading