Toward an always-on AI agent that adapts to you, your computer, and its own tools.
Reference alpha: 0.4.0a1. Start the supported Windows/IQ3_S/Hermes
composition, choose a bounded file task, continue saved work and accept another
task in the same controller. Begin with the ordinary workflow
and setup helper. This is a prerelease toward the
always-on goal; application/reboot recovery and broad daily-use qualification
remain on the roadmap.
Validated development update: the ordinary launcher can now reopen a safely stopped task on the same Windows boot, preserving its completed read, conversation and original time limit. An installed IQ3_S trial completed the remaining work with one output write, then opened the answer read-only without another model call. See the workflow and reproduction steps and initial-task result. A separate installed trial now also qualifies a task created later through New task in the running controller: the saved read survived shutdown, two resumed model calls finished the correct 23-unit output, and reopening the completed result repeated no work. See the later-task result. This is a development-source increment beyond the released alpha above; computer-reboot and broad daily-use qualification remain on the roadmap.
Compiler development: an installed IQ3_S/Hermes task has now yielded to a
real foreground action, released the model's native resources, built and tested
C++, restored inference and completed its original task. The program produced
exactly 23 units. The fresh physical result
passed 44 primary checks and 34 independent review checks, including the timing
of foreground activity and normal owned cleanup. Its separate status observer
still failed during shutdown. The earlier attempt
retains its failed audit and missed foreground overlap. The separate
SDK evidence
qualifies compiler-error correction with scripted provider replies. Full
daily-use coexistence, real-model error correction and broad M4 remain open.
Try the compiler task and setup example
and read the compiler recipe and scope.
Video-tool development: a real IQ3_S/Hermes task now completes the same foreground handoff with a GPU video tool. It yields to Explorer activity, releases the model, encodes and checks a three-second clip, restores the model and finishes using the saved tool result. The fresh physical result passed all 47 primary checks; the separate observer still failed during shutdown. Local validation passed 1,541 core tests, the remaining workflow checks, packaging and fresh installation. The agent task and setup explain reproduction. The first attempt completed the video work but missed its planned foreground action. Both results are retained. This silent calibration clip does not qualify H3 generation, audio/video quality, sustained desktop performance or all M4 recovery cases.
Continuing this project with an AI or a team? Start with AGENTS.md, the dated continuation handoff, and the multi-agent workflow. They preserve the goal, current evidence, next work and coordination rules across tools and sessions.
You should be able to start your AI agent and keep using your computer normally: browse the internet, play a game, watch a video, or do your own work. The agent should adapt to your activity, make progress when resources are available, and back off when you need them. You should not have to manage competing model servers or repeatedly stop and restart the agent yourself.
The same coordination should happen inside the agent's workflow. If it needs to compile C++, run another model, or generate a video with ComfyUI or H3, Adaptive Inference should make room for that work: reduce background activity, adjust supported caches, or park and unload an idle model to release CPU, RAM, GPU and VRAM resources. After the tool finishes and capacity is available, the agent should restore what it needs and continue toward the original task.
Always on means continuity of the agent and its work. Its largest model does not have to remain fully loaded at every moment. Tasks, progress and tool results should survive resource changes, while the user's foreground experience takes priority. Resource transitions should be automatic, observable and recoverable.
That continuity must also survive closing the application or restarting the computer. On returning, the agent should recover its original objective and saved progress, verify any interrupted action, and continue without losing completed work or blindly repeating a tool action. The durable task foundation and a bounded installed-Hermes recovery adapter implement process continuation. Operating-system startup and physical reboot qualification remain part of the goal.
This is the long-term goal across engines such as Strata and llama.cpp, and agents such as Hermes and OpenClaw. Shared hardware has limits; the project aims to minimize interference through measured adaptation, rather than promise unlimited simultaneous work. See the always-on vision and acceptance criteria.
Accepted reference workflow: one complete Windows workflow passes independent acceptance using the existing IQ3_S model, an explicitly owned Strata backend and installed Hermes. It detects real foreground interaction, releases and restores its backend, makes room for a checked GPU video encode, and finishes the agent's saved task after a worker restart. The source-bound result includes the complete shutdown checks and retained unsuccessful attempts. This is a bounded qualification; persistent everyday operation, games and video playback, and computer-restart recovery are later milestones. The automatic handoff controller, resource waiting and durable demand build on the existing request queue, execution-time feedback and independently checked task results. Model weights and quality settings remain unchanged.
The task-control component saves Pause, Resume and Cancel requests, and a cooperating task owner acknowledges them at a safe checkpoint. It retains pending requests across owner replacement. The installed Windows profile connects these controls to the existing host, retained GUI and bounded Hermes report worker. It starts paused and uses explicit local interpreter, SDK and native-owner configuration. Its IQ3_S orders-report workflow has passed separate installed qualification. The retained-command interface retains one eligible user command during a background status check; its source-bound successor result includes actual GUI controls, task completion, worker replacement and normal shutdown.
The paused-budget expiry result adds one verified behavior: a safely paused task can reach its original time limit while preserving its report and conversation. In the installed IQ3_S trial, the model was parked, the task remained pending, and the same window displayed that saved work was preserved before one Stop produced normal shutdown. The later Continue saved task result verifies one explicit continuation in the same window: two requests before expiry, one afterward, the original report written once, task acceptance and normal shutdown after one Continue and one Stop. The independent terminal audit passed 70 checks. The feature guide explains the separate execution budgets and cumulative task allowance. Broader foreground use and reboot recovery remain development work.
The desktop control library provides the shared
authenticated transport, serialized listener, mailbox adapter and window-intent
and progress contracts for that integration. Importing the library starts no
GUI, agent or engine. The explicit desktop command composes those lifecycles
using the installed profile.
Accepted continuity result: the installed IQ3_S service completed a task after real foreground activity, safe resource-wait expiry and explicit continuation, then accepted and completed a separate task without restarting the controller. One normal Stop released its owned resources. The original 99-check handover audit and separate foreground audit both passed in case17. The inventory result was independently checked; the welcome task had format-only validation. The accepted result and replay and fresh qualification workflow are preserved in this repository. The fresh workflow still requires actual installation records and external model assets. The supported setup helper now creates the deployment profile and Start launcher from installed components. Its separate startup/Stop check now passes on the retained installation without loading the model. The task contracts, roadmap and handoff explain the supported scope and remaining work.
For the ordinary controls and one-time prerequisites, start with Use the Windows reference workflow.
This repository is the canonical source of truth for the implementation, contracts, documentation, tests and evidence. The portable package works independently of Strata or Hermes. Optional native integrations preserve pinned source baselines, patches and licenses locally; reproducing their source does not depend on an old PR or upstream repository remaining available. See the source preservation contract.
Upstream PRs are delivery and discussion records. New shared development happens here, then focused adapters and fixes can be offered to engine and agent projects.
The integration delivery plan tracks all eight engine and ten agent targets, the concrete integration for each, its test evidence and any upstream requirement still outstanding. The optional Hermes plugin and OpenClaw configuration composer use public host interfaces to classify background work while preserving model settings.
The standalone releases
provide the complete source archive, installable runtime, checksums and validation
receipts. The 0.4.0a1 reference alpha adds the supported desktop workflow and
task/setup helpers to the generalized integration foundation released in 0.3.0.
Use the archive and wheel from the same identified release. Packaging checks,
task-authoring checks and the original installed model/desktop results retain
their separate evidence scopes. Integrations follow the shared
agent/engine contract.
Python 3.11 or later; no GPU or model is required for the reproducible demo.
git clone https://github.com/midhatn/adaptive-inference.git
cd adaptive-inference
python -m venv .venv
# Linux/macOS:
source .venv/bin/activate
# Windows PowerShell instead: .venv\Scripts\Activate.ps1
python -m pip install -r requirements.lock
python -m pip install --no-deps -e .
adaptive-inference demoThe demo uses an explicitly simulated model server over real HTTP. It checks queue ordering, learned timing feedback and correction after a failed acceptance check. It is a reproducible software check, not an LLM benchmark.
To use an already running model server:
adaptive-inference doctor --config examples/engines/llamacpp.json
adaptive-inference serve --config examples/engines/llamacpp.jsonPoint a compatible client at http://127.0.0.1:18100/v1. For background agent work, use http://127.0.0.1:18100/background/v1. Both routes share the same queue and backend. Use a model name actually returned by that backend's /v1/models; examples deliberately do not select or download a model for you.
See reproduction instructions, engine and agent setup, and the development plan. For a complete preserved copy, use a Git clone or the full source distribution; the runtime wheel contains the installable portable package.
To verify the preserved references and reconstruct both optional integrations without network access:
python third_party/reference-sources/verify.py
python provenance/offline_acceptance.py --output ../adaptive-source-proof-01
The second command requires Git and a new output directory. It restores and verifies source only. The original bundled baselines occupy about 146 MB. Current development also includes a 48 MB source delta for the exact newer Hermes SDK used by the plugin and real task; its separate offline restoration command reconstructs all 20,897 tracked files. Model weights and compiler/Python prerequisites remain separate. See the native source guide for individual integration commands and build instructions.
For a new bounded file task, prepare-task takes an objective and explicitly selected files, copies the inputs and creates the task configuration automatically. The existing desktop and New task controls consume it. This authoring command does not start inference; the ordinary reference guide explains setup and use. The desktop setup helper calculates deployment hashes and provides a reusable launcher with a task picker and fresh run folder.
| Capability | Implementation |
|---|---|
| User/background scheduling | Bounded pending queue; stable arrival-offset ordering; one active upstream HTTP request |
| Online timing feedback | Context-separated Peak EWMA plus empirical p90, three-sample minimum, expiry and failure invalidation |
| OpenAI-compatible serving | Chat, text completion and embeddings where the selected backend advertises them; model listing; SSE forwarding |
| Conservative admission | Optional observed RAM/Windows-commit floor and sustained-recovery policy, bounded queue/response/time limits |
| Explicit context planning | Preserve the host's output ceiling; report whether prompt compaction is needed and give an exact target under the selected margin policy |
| Outcome-based correction | run_with_feedback: retain the original prompt, use caller-provided acceptance feedback, stop on success or attempt limit |
| Durable task continuation | SQLite checkpoints, one cooperating local owner, saved output/feedback and explicit reconciliation of interrupted actions; reproducible process-restart demo |
| Foreground and tool handoff | Optional Windows host with observed activity, acknowledged native release, fresh capacity checks and same-quality restoration; one complete IQ3_S workflow accepted |
| Installed-agent continuation | Pinned Hermes public middleware and native history; resource waiting outside generation and one verified worker restart without repeating the completed report write |
| Durable demand | Trusted demand survives worker replacement; the bounded installed composition includes verified safe paused expiry and one explicit same-window continuation without resetting the task allowance |
| Existing native integration | Pinned Strata memory/parking/lease patches and Hermes intent-tagging patches, with reconstruction verification |
Engine profiles cover Strata, llama.cpp, vLLM, SGLang, Ollama, LM Studio, MLX-LM and TGI. Client recipes cover Hermes, OpenClaw, Aider, OpenHands, Continue, Cline, Roo Code, LangGraph, AutoGen and CrewAI. The compatibility matrix distinguishes protocol contracts, documented configuration and actual application/native evidence. A profile is not a native GPU-memory plugin.
flowchart LR
U[User applications] --> G[Adaptive gateway]
A[Agent applications] --> G
G --> Q[Bounded queue and admission]
Q --> B[Backend adapter]
B --> E[Existing model server]
E --> L[Completed timing observations]
L --> Q
E --> V[Application acceptance check]
V -->|Correction feedback| A
The gateway never retries a generation on its own. Queued disconnected clients are removed; active disconnected requests are drained before another request is dispatched. If upstream completion becomes uncertain, the gateway stops dispatching and reports the state. It does not assume that closing an HTTP connection freed GPU memory.
Generic profiles do not resize a backend's cache, move KV state or preempt native generation. These require explicit backend capabilities and native verification. Strata's richer resource controls are preserved in its integration, with separate evidence. See architecture and evidence.
The latest complete handoff qualification used IQ3_S through the Windows Strata/Hermes composition, retaining context 65,536, output allowance 10,923 and high/8,192 reasoning settings. Both independently checked arithmetic tasks succeeded on their first attempts. Two 45-second foreground animation windows met the declared 25-ms p95 paint-interval budget, a GPU encode produced 90 checked frames, and Hermes resumed its original task after a safe worker restart with one report write. These are measured paint-submission intervals, not physical input latency. Earlier baseline runs use different runtime/native revisions, so this result makes no comparative performance-improvement claim.
An earlier IQ3_S parent/child file task separately completed six model requests and five native file operations through the released gateway and optional routing plugin. The host-created child independently checked the parent report. Its exact source, cleanup and reproduction limits remain preserved; it is a distinct composition from the handoff qualification.
Version 0.3.0 added general client routing and protected identity settings, with 110 core tests, 17 source-preservation checks and 12 task-boundary checks at that revision. An earlier handoff revision recorded 174 core, 64 native-owner/host, 20 harness, 11 foreground, 17 preservation and 12 task-boundary tests locally, plus the historical evidence-filter suites. These are historical counts, not an aggregate for the current revision. Installed SDK fixtures, native host-schema checks and subsequent validation are recorded separately in the integration evidence. These software tests run no inference model; the source-bound IQ3_S receipts supply the distinct hardware evidence.
The v0.2.0 durable-task foundation passed 95 software tests across Windows, Linux and macOS on Python 3.11 and 3.12. Its runnable demonstration kills and restarts real worker processes, retains the original task and corrective feedback, and refuses automatic replay of an uncertain external action. These use deterministic generation/tool fixtures; installed-agent boot recovery remains future integration work.
The earlier portable-gateway release also passed a fresh wheel installation and eight real Strata model requests. Both request classes engaged learned timing feedback, all eight integer answers matched exactly, and the separately operated backend shut down cleanly. This is functional evidence for that exact composition, not a speed claim or qualification of every listed application.
The evidence ledger links exact CI and model receipts, retains the failed idle-reload attempt, and separates predecessor results from the new gateway. The upstream PR index records earlier work, including closed and superseded PRs.
Research references and reuse decisions live in docs/research.md and REFERENCES.bib. NOTICE and third_party/ preserve source attribution and licenses. CITATION.cff describes how to cite this project.
The Papers with Code assessment records implementation audits and their practical fit: an optional offline timing-drift experiment, deferred native prefill interruption, and explicit limits on what catalog links or external benchmarks establish.
See portable policy setup for the optional memory gate and context-budget command, and task survival for durable continuation. Citations remain beside each reused mechanism, with licensed source references preserved locally. The changelog distinguishes release contents and acceptance boundaries.
Contributions should start here, pass the common contract tests, and then be offered upstream as narrowly scoped adapters or fixes. See CONTRIBUTING.md. The goal is useful task completion with feedback and recoverable mistakes; optimizations follow measured need.