Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 118 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# Changelog

What each release changed for a consumer. Every release is tagged `v<VERSION>`
and `go/v<VERSION>`, and the pull requests named here carry the full evidence.
Within 0.x a minor release may break consumers
([versions and releases](docs/versions-and-releases.md)).

## 0.4.0 (2026-10-31)

The telemetry consumer is paced, so a busy host no longer wakes it once per
pump (#15).

- **Pacing.** The telemetry region publishes `drain_interval_ns` and
`idle_after_ns`, taken from reserved space. A consumer that found records
recently drains on the interval without asking to be woken; an idle one
sleeps on the doorbell. Defaults 10 µs and 1 ms, set with
`OURO_TELEMETRY_DRAIN_INTERVAL_US`, `OURO_TELEMETRY_IDLE_AFTER_US` or
`ouro_host_telemetry_pacing`. The C and Go readers follow them, and so does
the exporter, unchanged.
- **Cost.** On a Threadripper 3970X with pinned threads, telemetry now adds 5–6
ns per event instead of 17–30, and the host rings the doorbell 0–2 times in
2 million events instead of about 30,000.
- **Breaking, C ABI.** `ouro_telemetry_reader_t` gained a field. The wire,
layout and contract versions are unchanged, and a 0.3.0 host's zeroed fields
give 0.3.0 behaviour.
- **Gated.** `scripts/check_perf_budget.sh` holds the doorbell-write count of a
busy consumer under 400. The bench pins its threads and prints
`telemetry_wakes=`; the host counts `telemetry_wakes`.
- **Fixed.** A `Release` build defined `NDEBUG`, which compiled every `assert`
out of the C tests and harnesses. They now keep their assertions, and CI
builds and runs `Release` too.

## 0.3.0 (2026-10-30)

Every event the host stamps is described off the host the moment it is stamped
([D16](spec/decisions.md#d16-tap-telemetry-leaves-the-host-the-moment-an-event-is-stamped), #14).

- **The record and region.** A 128-byte, content-free record per stamped event,
written before the tap and routing into a ring in its own region. A full ring
is counted and reported by a gap record, never by blocking.
- **The grant.** `ouro_host_telemetry_enable`, `ouro_host_grant_telemetry`, and
`telemetry=yes` in a runtime grant file. Attach capability bit 2 hands one
slot two more descriptors.
- **Readers.** C `ouro_client_attach_telemetry` and `ouro_telemetry_*`; Go
`ring.AttachTelemetry`, `ConnectTelemetry` and `TelemetryReader`. Rust holds
the layout only.
- **The reference exporter.** `go/cmd/ouroboros-ring-otlp`, with its core in
`go/otlpexport`, sends OTLP/HTTP logs and releases records only once the
collector accepts them. `go/ring` still depends on nothing beyond the
standard library and `golang.org/x/sys`.
- **Compatibility.** `layout_version` stays 1. A grant file whose `telemetry`
value is neither `yes` nor `no` is now refused as malformed.

Also: the module-call docs say a service stop must interrupt the client (#13).

## 0.2.0 (2026-10-30)

An idle ring makes no wakeups (#12).

- **Doorbells ([D13](spec/decisions.md#d13-doorbells-idle-ends-sleep-in-the-kernel)).**
Each end of a queue pair sleeps on an `eventfd` and is signalled only while
asleep. `ouro_host_wait` (split into `_begin` and `_end` for a locked
embedding), `ouro_client_wait`, `Client.Wait` and `Client::wait`. The module
runtimes and callers in all three languages sleep on them, and the runtime's
accept loop blocks in `poll(2)`.
- **Notifications ([D14](spec/decisions.md#d14-notifications-reach-module-processes-and-callers)).**
Module processes and concurrent callers deliver subscribed notifications and
`overflow` to handlers, and can publish on their own slot. The Go concurrent
caller takes per-kind overflow handlers.
- **Liveness ([D15](spec/decisions.md#d15-liveness-is-the-process-not-a-timer)).**
A host holding a pidfd on a client releases its slot when the process exits
and grants it process liveness, so it need not heartbeat. Detach is announced.
A client watches the host's process too.
- **Compatibility.** The attach request grew to 20 bytes with a `capabilities`
word, negotiated so either side may be older. `layout_version` stays 1.

## 0.1.3 (2026-10-29)

- **Module client pool.** `ouro_module_client_pool_*` makes many module calls
at once over a pool of attached clients, waiting no longer than each call's
deadline (#10). A convenience of the C implementation, not part of the
contract.
- **Documentation.** `docs/`, with the architecture and a page for each part of
the ring (#9, #11).

## 0.1.2 (2026-10-28)

Machinery aimee had built on top of the ring moved into it (#8).

- **`ouro_host_attach_local`** attaches a client to a host in the same process.
- **`ouro_capture_flush_fd`** appends a capture stream to a file.
- **Flow-control tests** for block, shed, `control_lost` and `producer_reaped`,
which until then were tested only in aimee.
- **The dispatch budget:** `c/bench/dispatch.c`, `c/bench/baseline.json` and
`scripts/check_perf_budget.sh`, now run in CI.

## 0.1.1 (2026-10-27)

- **GCC workaround.** The ring's own build turns off GCC's IPA bit-CP pass,
which in GCC 15.2 was observed to pass a live count as 0 in a consumer built
`-Os -flto` (#7). A consumer using the ring's CMake build gets this without
knowing about it.

## 0.1.0 (2026-10-27)

The ring moved out of aimee as a contract with three reference implementations
held to it (#1).

- **The contract.** `spec/` (requirements R1 to R6, decisions D1 to D12) and
`vectors/`, generated from the spec and checked against every
implementation.
- **C.** The host, the client and the module runtime, as `libouroboros-ring`
and `libouroboros-ring-client`, with an installed CMake package and
pkg-config files. Owned in-process module services (#3).
- **Go and Rust.** Clients, module callers and module runtimes; pure Go with no
cgo, and Rust depending only on `libc`.
- **Releases** are cut from `VERSION` on `main` (#6). Nothing in the repository
builds, tests or runs against another project (#5).
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ and pins a release of it like any other.
- **The documentation** in [`docs/`](docs/README.md) explains how it works:
the [architecture](docs/architecture.md), attach and admission, routing and
flow control, module calls, arena payloads, capture, telemetry, conformance,
and versions.
versions, performance and configuration. What each release changed is in the
[changelog](CHANGELOG.md).

## Implementations

Expand Down
3 changes: 3 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ could disagree.
| [Telemetry](telemetry.md) | every stamped event described off the host at once: the record, the hand-off to one exporter process, loss, and the reference OTLP exporter |
| [Implementations and conformance](implementations-and-conformance.md) | what C, Go and Rust each provide, the evidence that holds them to the contract, and adding a language |
| [Versions and releases](versions-and-releases.md) | release tags, the contract's own version numbers, and how consumers pin a release |
| [Performance](performance.md) | what the benchmark measures, why a number belongs to its machine, and what the budget enforces |
| [Configuration](configuration.md) | every environment variable, flag and build option the implementations read |
| [Changelog](../CHANGELOG.md) | what each release changed for a consumer |

New to the ring: read the architecture page, then whichever of the others
matches what you are building. Implementing the ring in another language: read
Expand Down
31 changes: 23 additions & 8 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,14 @@ the kernel, not the client library, decides what a client can see
flowchart TB
subgraph host["Host address space"]
DIR["queue directory<br/>(ordinary private memory:<br/>slot → handle, principal, grants)"]
CTL["control region<br/>magic · versions · geometry ·<br/>host_epoch · host_heartbeat ·<br/>lease view (D12)"]
CTL["control region<br/>magic · versions · geometry ·<br/>host_epoch · host_heartbeat ·<br/>host_waiting · lease view (D12)"]
ARENA["arena region<br/>payload spans, by (offset, len)"]
QA["queue pair, slot 0<br/>inbound ring · outbound ring ·<br/>credits · heartbeat · control_lost"]
QA["queue pair, slot 0<br/>inbound ring · outbound ring ·<br/>credits · heartbeat · control_lost ·<br/>client_waiting"]
QB["queue pair, slot 1"]
TLM["telemetry region (D16, if enabled)<br/>pacing · consumer_waiting ·<br/>ring of records"]
end
subgraph exporter["Telemetry exporter"]
X1["telemetry region"]
end
subgraph ca["Client A"]
CA1["control (read-only)"]
Expand All @@ -76,19 +80,28 @@ flowchart TB
ARENA -.-> CB2
QA -.-> CA3
QB -.-> CB3
TLM -.-> X1
```

The exporter is an ordinary client too: it also maps control, arena and its own
queue pair, left out of the diagram.

| Region | Backing | Host | Each client | Holds |
|---|---|---|---|---|
| Control | one `memfd` | read/write | read-only | magic, `spec_version`, `layout_version`, flags, `slot_size`, `inline_budget`, `arena_size`, `host_epoch`, `host_heartbeat`, and the lease view |
| Queue pair | one `memfd` per admitted slot | read/write | read/write, its own slot only | inbound and outbound SPSC rings, credit counters, the client heartbeat, the sticky `control_lost` flag |
| Control | one `memfd` | read/write | read-only | magic, `spec_version`, `layout_version`, flags, `slot_size`, `inline_budget`, `arena_size`, `host_epoch`, `host_heartbeat`, `host_waiting`, and the lease view |
| Queue pair | one `memfd` per admitted slot | read/write | read/write, its own slot only | inbound and outbound SPSC rings, credit counters, the client heartbeat, the sticky `control_lost` flag, `client_waiting` |
| Arena | one `memfd` | read/write | read/write | payloads too large to travel inline |
| Telemetry | one `memfd`, only once enabled | read/write | read/write, the one granted consumer only | pacing values, `dropped`, `consumer_waiting`, the record ring ([telemetry](telemetry.md)) |
| Queue directory | not a region | private | none | which slot is whom, principals, grants, pending requests |

Because a client receives exactly three descriptors at attach (control, arena,
its own queue pair) it cannot map another client's rings, cannot enumerate other
slots, and cannot write the geometry its peers read. Those properties are
enforced. Arena bounds are not: every client maps the arena read-write for
The doorbells are `eventfd`s, not regions: one the host reads and every client
writes, and one per client that only the host writes.

A client receives only the descriptors for what it may map: control, arena and
its own queue pair, then the two doorbells if it was granted them, then the
telemetry region and its doorbell if it is the granted consumer. So it cannot
map another client's rings, cannot enumerate other slots, and cannot write the
geometry its peers read. Those properties are enforced. Arena bounds are not: every client maps the arena read-write for
zero-copy fan-out, so staying inside one's leases is a contract the client
libraries keep and conformance checks, not something hardware enforces
([D2](../spec/decisions.md#d2-threat-model)).
Expand Down Expand Up @@ -232,3 +245,5 @@ and Rust code creates no regions and accepts no attaches.
three implementations and how a new one is proven.
- [Versions and releases](versions-and-releases.md): what each version number
means and how a consumer pins the ring.
- [Performance](performance.md): measuring dispatch overhead, and the budget.
- [Configuration](configuration.md): every setting the implementations read.
40 changes: 27 additions & 13 deletions docs/attach-and-admission.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ sequenceDiagram
H->>H: allocate a directory slot, create its queue-pair memfd
H->>B: bind(ctx, host, slot, request)
B-->>H: OK (subscriptions, served kinds, outbound grants set)
H-->>C: reply (magic ABRP, OK, handle, negotiated version, granted capabilities, geometry, host_epoch)<br/>+ SCM_RIGHTS: control, arena, queue pair (+ host and client doorbells)
H-->>C: reply (magic ABRP, OK, handle, negotiated version, granted capabilities, geometry, host_epoch)<br/>+ SCM_RIGHTS: control, arena, queue pair (+ host and client doorbells) (+ telemetry region and doorbell)
C->>C: map control read-only, arena and queue pair read/write
else refused at either hook
H->>H: roll the slot back
Expand Down Expand Up @@ -63,6 +63,11 @@ sequenceDiagram
bound the slot to the client's process (`ouro_host_bind_process`), and tells
the client it need not heartbeat
([D15](../spec/decisions.md#d15-liveness-is-the-process-not-a-timer)).
`OURO_ATTACH_CAP_TELEMETRY` (bit 2), asked for together with bit 0, adds two
more descriptors, the telemetry region and its doorbell, for the one slot the
embedding granted it ([telemetry](telemetry.md#granting-it)). A reply's
descriptor count is therefore 3, 5 or 7, and always matches its
`capabilities`.

The client reads its geometry from the reply and the control region. It is
never compiled against those values, which is why the host can re-tune them
Expand Down Expand Up @@ -105,7 +110,7 @@ flowchart TB
dup -- "yes, and its process<br/>is still running" --> deny
dup -- "no, or its process has exited<br/>(pidfd)" --> bind["release the stale slot if any,<br/>enforce grants, subscribe,<br/>serve, grant outbound kinds"]
bind --> live["bind the slot to the peer's pidfd<br/>(D15: released when it exits)"]
live --> ok["OK: reply + three descriptors<br/>(five with a doorbell)"]
live --> ok["OK: reply + three descriptors<br/>(five with a doorbell,<br/>seven with telemetry)"]
```

A grant is one file per installed program in a policy directory
Expand Down Expand Up @@ -133,14 +138,18 @@ host hands its telemetry ring
([telemetry](telemetry.md), [D16](../spec/decisions.md#d16-tap-telemetry-leaves-the-host-the-moment-an-event-is-stamped)).
The host still grants it only while no other live slot holds it.

The listener also handles the case heartbeats alone handle badly. A process that
The listener also handles the case heartbeats alone handle badly: a process that
crashes cannot detach its slot, and its supervisor may restart it at once. When
the kernel supports `pidfd`, the runtime keeps a handle on each admitted
process: a new attach for a principal whose previous holder has exited releases
the stale slot immediately, while a live duplicate is still refused. Without
`pidfd` the stale slot waits for heartbeat reaping. `ouro_runtime_maintain`,
called periodically under the host lock, advances the host heartbeat and reaps
slots whose client heartbeat has stopped.
the kernel supports `pidfd`, the runtime keeps a handle on each admitted process
and watches it in the same `poll(2)` as the socket, so the slot is released the
moment the process exits
([D15](../spec/decisions.md#d15-liveness-is-the-process-not-a-timer)). A restarted
process therefore finds its principal free, while a live duplicate is still
refused. Without `pidfd` the stale slot waits for heartbeat reaping.
`ouro_runtime_maintain`, called under the host lock, advances the host
heartbeat and reaps slots whose client heartbeat has stopped;
`ouro_runtime_maintenance_due` says whether any slot still needs it, and when
none does the owner can sleep with no reap timer.

`refused_role_refs` lets a host that composes several ring instances refuse
role identities that belong to another instance, so a role cannot attach where
Expand All @@ -160,9 +169,14 @@ lock. This is a convenience of the C implementation, not part of the contract.
## Detach, reap and restart

- A client that is done calls `ouro_client_detach` (or its Go and Rust
equivalents) and closes its socket.
- A client whose heartbeat stops is reaped. Reaping drops every arena reference
the slot holds, discards its rings, and frees the slot. An event that was
blocked in flight for a reaped producer becomes a `producer_reaped` record.
equivalents). The detach is announced: the client marks its heartbeat
`OURO_HEARTBEAT_DETACHED` and rings the host, whose next pump drains what the
client queued and releases the slot.
- A process-bound client whose process exits is released at once, and is never
reaped for a stale heartbeat.
- Any other client whose heartbeat stops is reaped.
- Releasing a slot, by any of these, drops every arena reference it holds,
discards its rings and frees it. An event that was blocked in flight for a
released producer becomes a `producer_reaped` record.
- `ouro_host_bump_epoch` (or a host restart) changes `host_epoch`. Every
client sees `epoch_changed`, every mapping is stale, and clients re-attach.
Loading
Loading