Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@ All notable ModelPort changes are recorded here. The project follows

## [Unreleased]

### Runtime Adapter collection

- Collect validated Compute inventory immediately and periodically for every
enabled Runtime Adapter, with per-adapter isolation, a shared concurrency
bound, sanitized metrics and logs, bounded retry, and graceful shutdown.

## [0.1.3] - 2026-09-07

### Release correction
Expand Down
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ strip = "symbols"
async-stream = "0.3"
argon2 = "0.5"
axum = { version = "0.8", features = ["macros"] }
chrono = { version = "0.4", default-features = false, features = ["std"] }
futures-util = "0.3"
httpdate = "1"
jsonschema = { version = "0.48", default-features = false }
Expand Down
7 changes: 4 additions & 3 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,11 +101,11 @@ they are not eight top-level navigation entries:
| --- | --- | --- | --- |
| Models | Catalog identities, capabilities, limits, compatibility, and rate cards | Models | Partial: Provider-scoped inventory and logical catalog ship today |
| Providers | Hosted/local connectivity, credentials, account pools, health, and trust policy | Providers | Implemented under Settings and model views |
| Compute | Compute Nodes, GPUs, capacity observations, labels, freshness, and provenance | Compute | Target; no first-class inventory yet |
| Compute | Compute Nodes, GPUs, capacity observations, labels, freshness, and provenance | Compute | Partial: validated snapshot persistence and bounded collection ship; admin presentation remains a target |
| Deployments | Model/runtime/compute binding, endpoint, desired state, observed state, and reconciliation | Deployments | Target; local endpoints are currently configured as Providers |
| Routing | Logical models, aliases, eligibility, fallback, smart decisions, and evidence | Routing | Implemented, with some controls under Settings and Governance |
| Governance | Users, teams, keys, policies, quotas, budgets, and approvals | Governance | Implemented |
| Observability | Requests, attempts, usage, cost, latency, GPU/runtime telemetry, and retained evidence | Observability | Partial: request and Provider evidence ship; compute telemetry is target |
| Observability | Requests, attempts, usage, cost, latency, GPU/runtime telemetry, and retained evidence | Observability | Partial: request, Provider, and bounded Runtime Adapter collection metrics ship; compute dashboards remain a target |
| Operations | Readiness, incidents, backup, retention, upgrades, diagnostics, and reconciliation | Operations | Implemented for gateway operations; deployment operations are target |

This mapping is an information architecture contract, not a requirement to
Expand Down Expand Up @@ -290,7 +290,8 @@ boundary. Its adapter identities, discovery origins, credentials, and
collection/freshness policy do not participate in inference Provider routing
or inherit development-harness metadata. Enabled entries are validated and
their environment-backed credentials resolved at configuration load; polling
and inventory presentation are separate lifecycle slices.
is a bounded server lifecycle, while inventory presentation remains a separate
admin API slice.

Dashboard changes to control-plane records are persisted. They do not rewrite
`.env` or `config.toml`.
Expand Down
9 changes: 6 additions & 3 deletions docs/CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,11 @@ is never part of the TOML document or a serializable configuration type.
must cover at least one polling interval. Disabled declarations are inert:
their endpoint and credential are neither required nor resolved. Duplicate
TOML adapter tables, invalid enabled declarations, missing credentials, and a
registry over 64 entries fail configuration loading closed. This registry does
not start polling; background collection and admin inventory APIs remain
separate reviewed work.
registry over 64 entries fail configuration loading closed. At server startup,
each enabled entry receives a supervised, read-only collection loop with an
immediate capabilities-first attempt and bounded periodic retry. Registry,
credential, polling, and stale-after changes require a process restart; an
admin inventory API remains separate reviewed work.

## Required Minimum: DeepSeek-Only Example

Expand Down Expand Up @@ -992,6 +994,7 @@ lossless, but its configured output limits are still enforced.
| Trusted proxies, health exposure, private/insecure-URL policy, CSRF/origin policy | No | Yes |
| Admin bootstrap, session TTL, secure-cookie flag | No | Yes |
| Storage backend or state paths | No | Yes |
| Runtime Adapter registry, credentials, polling interval, or stale-after policy | No | Yes |

Reload from the dashboard Operations tab or restart the service. A successful
reload validates the new base snapshot but does not mutate `.env` or TOML.
Expand Down
23 changes: 19 additions & 4 deletions docs/OPERATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,19 @@ potentially paid request. Health semantics are documented below.

For a release or production trial, use [Production](PRODUCTION.md).

## Runtime Adapter Collection

Configured Runtime Adapters are polled by a provider-neutral, read-only
background collector. Each enabled adapter is attempted immediately at
startup, then on its configured interval. A shared concurrency bound prevents
an adapter fleet from exhausting the process; a failing adapter is isolated
with bounded backoff and sanitized error logging. Accepted Compute inventories
are immutable ledger evidence, while `fresh`, `stale`, and `unavailable` remain
server-owned projections. Shutdown stops new collection attempts and drains
active attempts for up to 10 seconds before emitting a sanitized warning. The
collector grants no Runtime Adapter mutation authority and does not expose an
admin API.

## Health Semantics

- `/livez` proves that the HTTP process can answer. It does not inspect storage
Expand Down Expand Up @@ -453,10 +466,12 @@ is process-local and requires restart/recreate after a configuration change.
## Configuration Reload

The dashboard Operations tab can reload the base configuration. Provider,
model, alias, and route values can update for new requests. Process layers,
security policies, transport settings, storage, sessions, and newly introduced
process environment variables require a restart. Use the full matrix in
[Configuration](CONFIGURATION.md#reload-versus-restart).
model, alias, and route values can update for new requests. Runtime Adapter
registry and collection-policy changes require a restart so the supervised
loops and their credentials stay aligned with the active process. Process
layers, security policies, transport settings, storage, sessions, and newly
introduced process environment variables also require a restart. Use the full
matrix in [Configuration](CONFIGURATION.md#reload-versus-restart).

Dashboard Settings exposes effective server/auth/rate values as read-only
runtime facts. Default provider and provider order remain runtime control-plane
Expand Down
26 changes: 22 additions & 4 deletions docs/RUNTIME_ADAPTER.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@
ModelPort publishes versioned, read-only discovery and Compute Node/GPU
observation contracts for external inference runtimes. The shipped v1alpha1
artifacts include wire contracts, offline validators, and a reusable
authenticated collection client. They do not expose a persisted inventory API,
reconciler, or mutation endpoint.
authenticated collection client. The external contract does not define an
admin API, reconciler, or mutation endpoint; ModelPort separately persists
validated observations and schedules bounded read-only collection.

## Contract Files

Expand Down Expand Up @@ -42,6 +43,23 @@ credential is not serializable and is redacted from debug output and upstream
errors. Configuration-file loading, scheduling, and storage are intentionally
outside this client boundary.

Accepted Compute observations can be persisted through the Enterprise Ledger.
PostgreSQL keeps immutable, identity-constrained snapshot history and the test
backend provides matching in-memory semantics. Exact retries are idempotent;
conflicting reuse of an adapter snapshot identity or observation timestamp is
rejected. The latest query derives `fresh`, `stale`, or `unavailable` from the
validated observation time, server time, and a bounded server-owned policy.
Derived state never changes the stored Runtime Adapter document.

When an enabled adapter is configured, ModelPort starts a bounded background
collector. It performs an immediate capabilities-first Compute read, persists
validated observations, then polls at the configured interval. Failed adapters
use bounded exponential backoff and do not stop healthy adapters; collection is
read-only and stops with the server during graceful shutdown. Collector metrics
contain only adapter IDs and bounded error classes. This collector does not add
an admin API, mutation operation, GPU control, or provider-specific runtime
management.

## Capability Rules

Every document has `apiVersion: runtime.modelport.io/v1alpha1`,
Expand Down Expand Up @@ -100,8 +118,8 @@ reinterpreted across versions. Additive experimental data belongs in
`local-inference-stack` checker remains an explicitly selected compatibility
mode, not the source of this contract.

Configuration integration, collection policy, persistence, derived freshness,
admin APIs, and all writes remain deferred to reviewed Issues. Offline
Retention, admin APIs, and all Runtime Adapter mutation operations remain
deferred to reviewed Issues. Offline
validation cannot start a process, download a model, access a GPU, or call a
network endpoint; the collection client performs only the two advertised safe
reads requested by its caller.
25 changes: 25 additions & 0 deletions migrations/0014_runtime_compute_snapshots.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
CREATE TABLE modelport_runtime_compute_snapshots (
adapter_id TEXT NOT NULL,
snapshot_id TEXT NOT NULL,
observed_at TIMESTAMPTZ NOT NULL,
observed_at_key TEXT NOT NULL,
accepted_at TIMESTAMPTZ NOT NULL DEFAULT now(),
document JSONB NOT NULL,
PRIMARY KEY (adapter_id, snapshot_id),
CONSTRAINT modelport_runtime_compute_snapshots_observation_unique
UNIQUE (adapter_id, observed_at_key),
CONSTRAINT modelport_runtime_compute_snapshots_identity_check CHECK (
length(adapter_id) BETWEEN 1 AND 63
AND length(snapshot_id) BETWEEN 1 AND 160
AND length(observed_at_key) = 30
AND observed_at_key ~ '^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}\.[0-9]{9}Z$'
AND jsonb_typeof(document) = 'object'
AND document #>> '{metadata,adapterId}' = adapter_id
AND document #>> '{metadata,snapshotId}' = snapshot_id
AND observed_at = (document #>> '{metadata,observedAt}')::timestamptz
AND observed_at = observed_at_key::timestamptz
)
);

CREATE INDEX modelport_runtime_compute_snapshots_latest_idx
ON modelport_runtime_compute_snapshots (adapter_id, observed_at_key DESC, accepted_at DESC);
5 changes: 4 additions & 1 deletion src/enterprise_ledger.rs
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@ use crate::{
usage::{current_period, quota_increment},
};

pub(crate) mod compute_inventory;

const DEFAULT_LEASE_TTL_SECS: u64 = 300;
const DEFAULT_RECONCILE_INTERVAL_SECS: u64 = 60;
const MIN_LEASE_TTL_SECS: u64 = 30;
Expand Down Expand Up @@ -79,6 +81,7 @@ struct MemoryLedger {
usage_reservations: HashMap<String, MemoryUsageReservation>,
budget_events: Vec<EnterpriseBudgetEvent>,
audit_events: Vec<EnterpriseAuditEvent>,
runtime_compute_snapshots: HashMap<(String, String), compute_inventory::MemoryComputeSnapshot>,
ops_incidents: BTreeMap<String, OpsIncidentDetail>,
ops_event_index: HashMap<String, String>,
ops_heartbeats: BTreeMap<String, OpsHeartbeat>,
Expand Down Expand Up @@ -878,7 +881,7 @@ impl EnterpriseLedger {
}

#[cfg(test)]
async fn postgres_for_tests(database_url: &str) -> Result<Self, AppError> {
pub(crate) async fn postgres_for_tests(database_url: &str) -> Result<Self, AppError> {
let pool = connect_pool(database_url, Some(4)).await?;
sqlx::migrate!("./migrations")
.run(&pool)
Expand Down
Loading