From ba9920e4040a8460ce9ea1a1a62f83d27fe65802 Mon Sep 17 00:00:00 2001 From: Prasad Pamidi Date: Wed, 26 Aug 2026 09:22:32 -0700 Subject: [PATCH 01/10] Document Core AI benchmark design --- .../2026-08-26-core-ai-benchmark-design.md | 156 ++++++++++++++++++ 1 file changed, 156 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-26-core-ai-benchmark-design.md diff --git a/docs/superpowers/specs/2026-08-26-core-ai-benchmark-design.md b/docs/superpowers/specs/2026-08-26-core-ai-benchmark-design.md new file mode 100644 index 0000000..ad20843 --- /dev/null +++ b/docs/superpowers/specs/2026-08-26-core-ai-benchmark-design.md @@ -0,0 +1,156 @@ +# Core AI Benchmark Design + +**Date:** 2026-08-26 +**Status:** Approved + +## Purpose + +Determine whether Core AI should replace, complement, or remain behind Aria's existing MLX runtime for custom on-device language models used by Niora. + +The benchmark must measure the runtime decision on a physical iPhone. A successful compile or a Mac-only result is insufficient because Niora's constraints are device memory, latency, heat, and battery-sensitive execution. + +## Scope + +This increment adds an isolated Xcode 27 benchmark runner under Aria. It compares matched Qwen3-0.6B models through Core AI and MLX without changing Aria's public provider API or Niora's production code. + +The runner records: + +- model preparation and load duration; +- time to first generated token; +- output tokens per second; +- total generation duration and token counts; +- peak resident memory during each trial; +- thermal state before and after each trial; +- runtime errors and cancellations; and +- pass/fail results for a fixed behavioral corpus. + +## Non-goals + +- Shipping Core AI in Niora. +- Replacing `LLMProvider` or `FoundationModelsProvider`. +- Designing Dynamic Profiles or multimodal meal logging. +- Committing model weights, exported `.aimodel` assets, caches, or benchmark reports. +- Claiming energy impact from thermal state alone. Instruments remains the source for detailed CPU, GPU, Neural Engine, and energy analysis. + +## Repository Boundary + +The benchmark lives in Aria because it evaluates an execution runtime that multiple host apps can consume. Niora remains unchanged until the evidence supports a production integration. + +The benchmark is isolated from Aria's main `Package.swift` and normal Fastlane lanes. This preserves: + +- the iOS 18 and macOS 15 package floor; +- Xcode 26 package builds and tests; +- simulator builds, where Core AI is not currently available; and +- Linux builds of the `Aria` core target. + +The benchmark uses its own Xcode 27 project and an iOS 27 deployment target. It links the existing local Aria checkout for the MLX arm and Apple's `coreai-models` package for the Core AI arm. + +## Runner Architecture + +The runner is a minimal SwiftUI development app with four components: + +1. `BenchmarkConfiguration` defines the corpus, trial count, model identifiers, context length, and generation limits. +2. `BenchmarkRuntime` is a benchmark-local protocol that normalizes setup, unload, and streaming generation without changing Aria's production protocols. +3. `BenchmarkCoordinator` runs one runtime at a time, samples process memory and thermal state, and produces immutable trial records. +4. `BenchmarkReportWriter` encodes a versioned JSON report and exposes it through the system share sheet. + +Core AI uses `CoreAILanguageModel` and `LanguageModelSession` directly. MLX uses Aria's existing `MLXProvider`. The benchmark-local protocol prevents the spike from forcing the production `LanguageModel` adapter before its design is informed by measurements. + +## Model Parity + +The first comparison uses Qwen3-0.6B because both runtimes support it and its size is practical for repeated phone testing. + +Both arms use: + +- 4-bit weights using the closest supported runtime-specific representation; +- a 4,096-token context window; +- the same tokenizer family and instruction-tuned checkpoint lineage; +- identical prompts and maximum output-token limits; and +- deterministic sampling when both runtimes support it. + +Compression formats and exported model hashes are recorded in the report. Results are labeled "matched configuration," not "identical binary," because Core AI palettization and MLX quantization are runtime-specific. + +## Model Preparation + +Model assets live under a gitignored benchmark assets directory. + +Setup is explicit and separate from measured inference: + +1. Export Qwen3-0.6B for iOS using Apple's `coreai-models` tooling. +2. Place the exported resource bundle in the ignored benchmark assets directory. +3. Download and cache the matching MLX model before starting trials. +4. Record source revision, model identifiers, hashes, sizes, compression, and context settings. + +Network download and export time are not inference metrics. Core AI specialization time is recorded separately because it affects first-run product experience. + +## Benchmark Corpus + +The initial corpus has four categories: + +- **Short response:** concise wellness-oriented instruction following. +- **Long context:** a near-window transcript followed by a grounded question. +- **Structured output:** a small guided schema representative of Niora decision packets. +- **Tool use:** deterministic tools and expected calls derived from Aria's existing task-evaluation fixtures. + +Each case runs three measured trials per runtime after an unmeasured warm-up. Runtime order alternates by case to reduce thermal and cache-order bias. A cooldown gate pauses new trials while the device thermal state is serious or critical. + +## Measurement Semantics + +- **Load time:** start of runtime load through readiness for generation. +- **Time to first token:** generation request start through the first non-empty text delta. +- **Generation rate:** output tokens divided by time from first token through completion. +- **Total duration:** request start through terminal completion or error. +- **Peak resident memory:** highest sampled resident-memory value during the trial. +- **Thermal state:** `ProcessInfo` state captured before and after the trial. +- **Behavioral pass:** required content, forbidden content, expected structured decode, and expected tool call all succeed for the case. + +Cold-process measurements are run separately after app relaunch and are not mixed into warm-trial aggregates. + +## Report Format + +Reports are Codable JSON with a schema version. They include: + +- timestamp, device model, OS build, app build, and Xcode build; +- runtime and dependency revisions; +- model metadata and asset hashes; +- benchmark configuration; +- raw trial records, including failures; +- per-runtime aggregates using median and p95 where the sample count permits; and +- behavioral pass rates. + +Raw trials remain authoritative. Aggregate calculations are deterministic and unit tested. + +## Error Handling + +The coordinator records a failed trial instead of aborting the full run. Typed failure categories cover missing assets, unsupported runtime, model load or specialization failure, generation failure, timeout, cancellation, structured-output failure, and invalid measurement. + +The UI prevents a run when required assets are missing and shows the exact expected location. Cancellation stops the active generation, writes completed trial records, and marks the report incomplete. + +## Testing and Verification + +Deterministic unit tests cover: + +- benchmark sequencing and alternating runtime order; +- warm-up exclusion; +- duration and generation-rate calculations; +- peak-memory sampling aggregation; +- cooldown behavior; +- failure recording and cancellation; +- report encoding and schema versioning; and +- behavioral scoring. + +Runtime smoke tests require Xcode 27 and a physical iOS 27 device. They are opt-in and never run in standard Aria CI. Aria's existing Xcode 26 Fastlane build, test, and quality lanes must continue to pass unchanged. + +## Decision Rule + +The benchmark concludes with one of three recommendations: + +1. **Adopt Core AI:** Core AI materially improves latency, throughput, or memory without unacceptable behavioral regression. +2. **Retain MLX:** Core AI offers no material product advantage or introduces unacceptable integration/runtime constraints. +3. **Route by model or device:** each runtime wins for a distinct supported model, hardware class, or workload. + +No single metric decides the result. A runtime must meet the behavioral corpus first; performance improvements from a runtime that fails required structured output or tool use do not justify adoption. + +## Follow-on Work + +If Core AI is adopted or selectively routed, the next increment designs an iOS 27 `LanguageModel` adapter in Aria. That work will reuse `LanguageModelSession` while preserving Aria's context selection, memory provenance, workflow, observability, replay, and cross-platform `LLMProvider` boundary. From 58245d2fcb8a1bf04f12c2439bc0494272d820b9 Mon Sep 17 00:00:00 2001 From: Prasad Pamidi Date: Sun, 30 Aug 2026 01:05:10 -0700 Subject: [PATCH 02/10] Refocus Core AI design on LanguageModel integration --- .../2026-08-26-core-ai-benchmark-design.md | 156 --------------- ...08-30-language-model-integration-design.md | 183 ++++++++++++++++++ 2 files changed, 183 insertions(+), 156 deletions(-) delete mode 100644 docs/superpowers/specs/2026-08-26-core-ai-benchmark-design.md create mode 100644 docs/superpowers/specs/2026-08-30-language-model-integration-design.md diff --git a/docs/superpowers/specs/2026-08-26-core-ai-benchmark-design.md b/docs/superpowers/specs/2026-08-26-core-ai-benchmark-design.md deleted file mode 100644 index ad20843..0000000 --- a/docs/superpowers/specs/2026-08-26-core-ai-benchmark-design.md +++ /dev/null @@ -1,156 +0,0 @@ -# Core AI Benchmark Design - -**Date:** 2026-08-26 -**Status:** Approved - -## Purpose - -Determine whether Core AI should replace, complement, or remain behind Aria's existing MLX runtime for custom on-device language models used by Niora. - -The benchmark must measure the runtime decision on a physical iPhone. A successful compile or a Mac-only result is insufficient because Niora's constraints are device memory, latency, heat, and battery-sensitive execution. - -## Scope - -This increment adds an isolated Xcode 27 benchmark runner under Aria. It compares matched Qwen3-0.6B models through Core AI and MLX without changing Aria's public provider API or Niora's production code. - -The runner records: - -- model preparation and load duration; -- time to first generated token; -- output tokens per second; -- total generation duration and token counts; -- peak resident memory during each trial; -- thermal state before and after each trial; -- runtime errors and cancellations; and -- pass/fail results for a fixed behavioral corpus. - -## Non-goals - -- Shipping Core AI in Niora. -- Replacing `LLMProvider` or `FoundationModelsProvider`. -- Designing Dynamic Profiles or multimodal meal logging. -- Committing model weights, exported `.aimodel` assets, caches, or benchmark reports. -- Claiming energy impact from thermal state alone. Instruments remains the source for detailed CPU, GPU, Neural Engine, and energy analysis. - -## Repository Boundary - -The benchmark lives in Aria because it evaluates an execution runtime that multiple host apps can consume. Niora remains unchanged until the evidence supports a production integration. - -The benchmark is isolated from Aria's main `Package.swift` and normal Fastlane lanes. This preserves: - -- the iOS 18 and macOS 15 package floor; -- Xcode 26 package builds and tests; -- simulator builds, where Core AI is not currently available; and -- Linux builds of the `Aria` core target. - -The benchmark uses its own Xcode 27 project and an iOS 27 deployment target. It links the existing local Aria checkout for the MLX arm and Apple's `coreai-models` package for the Core AI arm. - -## Runner Architecture - -The runner is a minimal SwiftUI development app with four components: - -1. `BenchmarkConfiguration` defines the corpus, trial count, model identifiers, context length, and generation limits. -2. `BenchmarkRuntime` is a benchmark-local protocol that normalizes setup, unload, and streaming generation without changing Aria's production protocols. -3. `BenchmarkCoordinator` runs one runtime at a time, samples process memory and thermal state, and produces immutable trial records. -4. `BenchmarkReportWriter` encodes a versioned JSON report and exposes it through the system share sheet. - -Core AI uses `CoreAILanguageModel` and `LanguageModelSession` directly. MLX uses Aria's existing `MLXProvider`. The benchmark-local protocol prevents the spike from forcing the production `LanguageModel` adapter before its design is informed by measurements. - -## Model Parity - -The first comparison uses Qwen3-0.6B because both runtimes support it and its size is practical for repeated phone testing. - -Both arms use: - -- 4-bit weights using the closest supported runtime-specific representation; -- a 4,096-token context window; -- the same tokenizer family and instruction-tuned checkpoint lineage; -- identical prompts and maximum output-token limits; and -- deterministic sampling when both runtimes support it. - -Compression formats and exported model hashes are recorded in the report. Results are labeled "matched configuration," not "identical binary," because Core AI palettization and MLX quantization are runtime-specific. - -## Model Preparation - -Model assets live under a gitignored benchmark assets directory. - -Setup is explicit and separate from measured inference: - -1. Export Qwen3-0.6B for iOS using Apple's `coreai-models` tooling. -2. Place the exported resource bundle in the ignored benchmark assets directory. -3. Download and cache the matching MLX model before starting trials. -4. Record source revision, model identifiers, hashes, sizes, compression, and context settings. - -Network download and export time are not inference metrics. Core AI specialization time is recorded separately because it affects first-run product experience. - -## Benchmark Corpus - -The initial corpus has four categories: - -- **Short response:** concise wellness-oriented instruction following. -- **Long context:** a near-window transcript followed by a grounded question. -- **Structured output:** a small guided schema representative of Niora decision packets. -- **Tool use:** deterministic tools and expected calls derived from Aria's existing task-evaluation fixtures. - -Each case runs three measured trials per runtime after an unmeasured warm-up. Runtime order alternates by case to reduce thermal and cache-order bias. A cooldown gate pauses new trials while the device thermal state is serious or critical. - -## Measurement Semantics - -- **Load time:** start of runtime load through readiness for generation. -- **Time to first token:** generation request start through the first non-empty text delta. -- **Generation rate:** output tokens divided by time from first token through completion. -- **Total duration:** request start through terminal completion or error. -- **Peak resident memory:** highest sampled resident-memory value during the trial. -- **Thermal state:** `ProcessInfo` state captured before and after the trial. -- **Behavioral pass:** required content, forbidden content, expected structured decode, and expected tool call all succeed for the case. - -Cold-process measurements are run separately after app relaunch and are not mixed into warm-trial aggregates. - -## Report Format - -Reports are Codable JSON with a schema version. They include: - -- timestamp, device model, OS build, app build, and Xcode build; -- runtime and dependency revisions; -- model metadata and asset hashes; -- benchmark configuration; -- raw trial records, including failures; -- per-runtime aggregates using median and p95 where the sample count permits; and -- behavioral pass rates. - -Raw trials remain authoritative. Aggregate calculations are deterministic and unit tested. - -## Error Handling - -The coordinator records a failed trial instead of aborting the full run. Typed failure categories cover missing assets, unsupported runtime, model load or specialization failure, generation failure, timeout, cancellation, structured-output failure, and invalid measurement. - -The UI prevents a run when required assets are missing and shows the exact expected location. Cancellation stops the active generation, writes completed trial records, and marks the report incomplete. - -## Testing and Verification - -Deterministic unit tests cover: - -- benchmark sequencing and alternating runtime order; -- warm-up exclusion; -- duration and generation-rate calculations; -- peak-memory sampling aggregation; -- cooldown behavior; -- failure recording and cancellation; -- report encoding and schema versioning; and -- behavioral scoring. - -Runtime smoke tests require Xcode 27 and a physical iOS 27 device. They are opt-in and never run in standard Aria CI. Aria's existing Xcode 26 Fastlane build, test, and quality lanes must continue to pass unchanged. - -## Decision Rule - -The benchmark concludes with one of three recommendations: - -1. **Adopt Core AI:** Core AI materially improves latency, throughput, or memory without unacceptable behavioral regression. -2. **Retain MLX:** Core AI offers no material product advantage or introduces unacceptable integration/runtime constraints. -3. **Route by model or device:** each runtime wins for a distinct supported model, hardware class, or workload. - -No single metric decides the result. A runtime must meet the behavioral corpus first; performance improvements from a runtime that fails required structured output or tool use do not justify adoption. - -## Follow-on Work - -If Core AI is adopted or selectively routed, the next increment designs an iOS 27 `LanguageModel` adapter in Aria. That work will reuse `LanguageModelSession` while preserving Aria's context selection, memory provenance, workflow, observability, replay, and cross-platform `LLMProvider` boundary. diff --git a/docs/superpowers/specs/2026-08-30-language-model-integration-design.md b/docs/superpowers/specs/2026-08-30-language-model-integration-design.md new file mode 100644 index 0000000..0f6de96 --- /dev/null +++ b/docs/superpowers/specs/2026-08-30-language-model-integration-design.md @@ -0,0 +1,183 @@ +# Language Model Integration Design + +**Date:** 2026-08-30 +**Status:** Approved + +## Purpose + +Make Apple's iOS 27 `LanguageModel` protocol a first-class execution seam in Aria so Niora can use Apple's system model, a Core AI custom model, or another conforming model without changing its agent, memory, tool, or workflow layers. + +This increment is integration-first. It proves the production architecture with a lightweight Core AI device smoke test. Comparative benchmarking is deferred until Niora has a concrete runtime-selection decision. + +## Product Advantage + +Core AI controls how a custom model runs. Aria controls what makes that model useful inside Niora: + +- context budgeting and history selection; +- capability and model routing; +- relevant tool selection; +- memory provenance and retrieval; +- workflow and agent execution; +- fallback behavior; +- observability, evaluation, and replay. + +The integration must not imply that every Core AI call belongs behind Aria. Small one-shot features with no history, tools, routing, or memory may continue to use Foundation Models directly. + +## Scope + +This increment: + +1. Extends `FoundationModelsProvider` with an iOS 27 initializer accepting any concrete `LanguageModel`. +2. Preserves the existing iOS 26 system-model initializer and public type name. +3. Uses the injected model for text streaming, executable tools, and structured responses. +4. Adds deterministic tests around model-independent session construction. +5. Adds an opt-in Xcode 27 device smoke test using `CoreAILanguageModel` and Qwen3-0.6B. +6. Documents how a Niora provider route will inject a custom model after the smoke test passes. + +## Non-goals + +- Replacing Aria's cross-platform `LLMProvider` protocol with Apple's protocol. +- Adding Core AI or `coreai-models` to Aria's root package dependencies. +- Shipping a Core AI model in Niora in this increment. +- Removing MLX. +- Adopting Dynamic Profiles, multimodal prompts, or Private Cloud Compute. +- Building a benchmark application or selecting a default custom-model runtime. + +## Architecture + +The ownership boundary is: + +```text +Niora + domain behavior, permissions, feature routing + | +Aria + context, memory, tools, workflows, evaluation + | +FoundationModels.LanguageModel + | +Apple system model | Core AI model | other conforming model +``` + +Aria's portable core remains unchanged. The new API lives only in `AriaApple`, alongside the existing Foundation Models provider. + +## Provider Design + +`FoundationModelsProvider` remains a non-generic public struct. Making the type generic would ripple through Niora's `any LLMProvider` wiring and create an unnecessary source break. + +Instead, the provider stores an internal, sendable session factory. The existing initializer configures that factory with `SystemLanguageModel.default`. A new iOS 27 initializer is generic only at initialization: + +```swift +@available(iOS 27.0, macOS 27.0, *) +public init( + model: Model, + defaultInstructions: String? = nil, + capabilities: ProviderCapabilities, + typedTools: [FoundationModelsToolFactory] = [] +) +``` + +The initializer captures the concrete model in a closure that creates the non-generic `LanguageModelSession`. This preserves the provider's existing stored type and `LLMProvider` conformance while satisfying Foundation Models' concrete-model requirement. + +The session factory receives the accepted typed tools and transcript. Every path that currently constructs a `LanguageModelSession` must use the same factory, including: + +- ordinary text streaming; +- executable-tool streaming; and +- typed structured responses. + +This prevents the custom model from silently reverting to `SystemLanguageModel.default` on one response path. + +## Availability and Capabilities + +The existing provider checks `SystemLanguageModel.default.availability`. That check remains on the system-model initializer. + +An injected model does not inherit that check. Its load or generation failures propagate through the existing provider error path. Aria does not invent a second availability abstraction until two non-system model implementations demonstrate a shared need. + +Callers provide Aria's `ProviderCapabilities` for an injected model. The provider validates obvious contradictions against `LanguageModel.capabilities` when the corresponding Apple capability is representable, including vision, guided generation, reasoning, and tool calling. Unsupported requested behavior fails before generation with a typed configuration error. + +## Dependency Boundary + +Aria does not depend on Apple's `coreai-models` Swift package. It depends only on the system `FoundationModels` protocol that `CoreAILanguageModel` conforms to. + +This is required because: + +- `coreai-models` has an iOS 27 and macOS 27 package floor; +- Core AI is unavailable in the iOS Simulator SDK; and +- the upstream package currently fails simulator builds when `CoreAILM` is linked. + +The generic initializer lets a consumer that can safely own the dependency inject `CoreAILanguageModel` without pulling Core AI into every Aria consumer. + +## Core AI Proof + +The proof is an opt-in, device-only fixture outside Aria's normal product graph. It: + +1. Uses Xcode 27 and an iOS 27 physical device. +2. Loads an ignored Qwen3-0.6B Core AI resource bundle. +3. Constructs `CoreAILanguageModel` in the fixture. +4. Injects it into `FoundationModelsProvider`. +5. Runs one text case, one structured-output case, and one deterministic tool case through Aria. +6. Records the existing `TaskEval` result and basic timing for diagnostic context. + +The proof answers whether Core AI participates correctly in Aria. It does not claim runtime superiority over MLX. + +Model assets, caches, and reports are ignored and never committed. Model download, export, and specialization are explicit setup steps. + +## Niora Adoption Boundary + +Niora does not add `coreai-models` to its main app target while the upstream simulator issue remains open. + +After the device proof succeeds and the dependency can coexist with normal simulator builds, Niora adds a developer-only provider route in its existing AI capability routing layer. That route injects a Core AI model into the same Aria provider used by agent and workflow features. + +The first production candidate must be a flow that benefits from Aria's context, tools, memory, or fallback behavior. A simple one-shot transformation is not sufficient justification for migration. + +## Error Handling + +The provider preserves existing stream termination and typed Aria errors. New failure cases are handled as follows: + +- contradictory declared capabilities fail configuration before a request; +- model load, specialization, or executor failures propagate with their underlying error; +- unsupported transcript content is surfaced rather than silently removed; +- tool registration rejection remains visible through the current diagnostic event path; and +- cancellation terminates the stream without retrying on another model unless the caller explicitly configured tiered fallback. + +Fallback remains an Aria routing concern. The provider does not silently replace an injected model with the system model. + +## Testing + +Deterministic `AriaAppleTests` cover: + +- the existing initializer still builds system-model sessions; +- the iOS 27 initializer uses the injected session factory; +- text, tool, and structured paths all request sessions from that factory; +- capabilities map and contradictions fail early; +- injected-model errors are preserved; and +- existing iOS 26 behavior remains unchanged. + +The device proof is opt-in and excluded from standard CI. Normal verification remains: + +- Aria package tests and quality checks with stable Xcode; +- an Xcode 27 compile of the new availability-gated API; and +- the physical-device Core AI smoke test when model assets are present. + +## Success Criteria + +The increment is complete when: + +- existing Aria consumers require no source changes; +- stable Xcode package tests and Linux-safe boundaries still pass; +- Xcode 27 compiles the generic `LanguageModel` initializer; +- all session creation paths honor the injected model; +- the Qwen3-0.6B device proof completes text, structured, and tool cases through Aria; and +- Niora's future injection point is documented without changing its production dependency graph. + +## Deferred Benchmark Trigger + +A comparative Core AI versus MLX benchmark is created only if one of these decisions becomes real: + +- choosing the default runtime for the same model; +- dropping MLX support; +- routing by device class; +- meeting a documented latency, memory, thermal, or energy target; or +- comparing materially different Core AI and MLX model behavior for a Niora feature. + +Until then, Aria's existing task evaluation and smoke timing provide sufficient integration evidence. From a45e3d932b16c1423526952f1b5e1f66c708d91f Mon Sep 17 00:00:00 2001 From: Prasad Pamidi Date: Sun, 30 Aug 2026 01:19:08 -0700 Subject: [PATCH 03/10] Plan iOS 27 language model integration --- .../2026-08-30-language-model-integration.md | 492 ++++++++++++++++++ ...08-30-language-model-integration-design.md | 2 +- 2 files changed, 493 insertions(+), 1 deletion(-) create mode 100644 docs/superpowers/plans/2026-08-30-language-model-integration.md diff --git a/docs/superpowers/plans/2026-08-30-language-model-integration.md b/docs/superpowers/plans/2026-08-30-language-model-integration.md new file mode 100644 index 0000000..4dc8214 --- /dev/null +++ b/docs/superpowers/plans/2026-08-30-language-model-integration.md @@ -0,0 +1,492 @@ +# iOS 27 Language Model Integration Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to execute this plan task-by-task. + +**Goal:** Let `FoundationModelsProvider` run any iOS 27 `LanguageModel`, including `CoreAILanguageModel`, without changing Aria's public provider type, portable core, or existing iOS 26 call sites. + +**Architecture:** Keep `FoundationModelsProvider` non-generic and type-erase session construction behind an internal sendable factory. The existing initializer installs a system-model factory; an Xcode 27-only initializer captures a concrete `LanguageModel`. Text, selected-tool, and structured generation all use that factory. A nested opt-in package proves Core AI on a physical device without adding `coreai-models` to Aria's root graph. + +**Tech Stack:** Swift 6.3/6.4, Foundation Models, XCTest, Swift Package Manager, Fastlane, Core AI, Qwen3-0.6B, Apple's `coreai-models` revision `de31ba508895c7aa3bdcc57f8837a23f13316871`. + +**Spec:** `docs/superpowers/specs/2026-08-30-language-model-integration-design.md` + +## Global Constraints + +- Keep `Sources/Aria` free of Apple frameworks and the root `Package.swift` free of `coreai-models`. +- Preserve `FoundationModelsProvider()` and iOS 26 behavior. +- Guard iOS 27 symbols with `#if compiler(>=6.4)`; Xcode 26.6 uses Swift 6.3.3. +- Never silently replace an injected model with `SystemLanguageModel.default`. +- Validate the Aria capabilities that map to Apple capabilities: vision, structured/guided generation, and tools/tool calling. +- Keep the Core AI proof under `Examples/CoreAIProof`, outside the root package graph. +- Never commit model bundles, compiled caches, or proof reports. +- Use Fastlane for root-package build, test, and quality commands. + +--- + +### Task 1: Centralize session construction behind a deterministic seam + +**Files:** + +- Create: `Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift` +- Modify: `Sources/AriaApple/Providers/FoundationModelsProvider.swift:29-37,81-86,177-193,245-311` +- Modify: `Sources/AriaApple/Providers/FoundationModelsStructured.swift:64-94` +- Modify: `Tests/AriaAppleTests/Providers/FoundationModelsProviderTests.swift:144-220` +- Modify: `Tests/AriaAppleTests/Providers/FoundationModelsStructuredTests.swift:18-43` +- Create if shared helpers are needed: `Tests/AriaAppleTests/Providers/FoundationModelsSessionFactoryTestSupport.swift` + +- [ ] **Step 1: Add failing tests for all session paths** + +Add a lock-protected probe whose factory builder records requirements and throws before inference: + +```swift +private enum SessionProbeError: Error { case stop } + +final class SessionFactoryProbe: @unchecked Sendable { + private let lock = NSLock() + private var recorded: [FoundationModelsSessionRequirements] = [] + + var requirements: [FoundationModelsSessionRequirements] { + self.lock.withLock { self.recorded } + } + + func factory() -> FoundationModelsSessionFactory { + FoundationModelsSessionFactory( + validate: { _ in }, + build: { [weak self] _, _, requirements in + self?.lock.withLock { self?.recorded.append(requirements) } + throw SessionProbeError.stop + } + ) + } +} +``` + +Add tests named: + +- `testTextStreamRequestsSessionFromConfiguredFactory` expecting `[[]]`; +- `testExecutableToolStreamRequestsSessionFromConfiguredFactory` expecting `[[]]`; +- `testStructuredStreamRequestsGuidedSessionFromConfiguredFactory` expecting `[.guidedGeneration]`. + +Each consumes the returned stream, expects `AgentError.providerFailed`, and then checks the probe. No real model runs. + +Run: + +```bash +/Users/prasadmini/.rbenv/shims/bundle exec fastlane package_tests +``` + +Expected: compilation fails because the factory, requirements, and internal provider initializer do not exist. + +- [ ] **Step 2: Implement the factory** + +Create: + +```swift +#if canImport(FoundationModels) + import Aria + import FoundationModels + + @available(iOS 26.0, macOS 26.0, *) + struct FoundationModelsSessionRequirements: OptionSet, Sendable, Equatable { + let rawValue: UInt8 + static let vision = Self(rawValue: 1 << 0) + static let guidedGeneration = Self(rawValue: 1 << 1) + static let toolCalling = Self(rawValue: 1 << 2) + } + + @available(iOS 26.0, macOS 26.0, *) + struct FoundationModelsSessionFactory: Sendable { + typealias Validator = @Sendable (FoundationModelsSessionRequirements) throws -> Void + typealias Builder = @Sendable ( + [any FoundationModels.Tool], + Transcript, + FoundationModelsSessionRequirements + ) throws -> LanguageModelSession + + init(validate: @escaping Validator, build: @escaping Builder) { + self.validate = validate + self.build = build + } + + static let systemDefault = Self( + validate: { _ in try FoundationModelsProvider.checkSystemModelAvailability() }, + build: { tools, transcript, _ in + LanguageModelSession(tools: tools, transcript: transcript) + } + ) + + func makeSession( + tools: [any FoundationModels.Tool], + transcript: Transcript, + requirements: FoundationModelsSessionRequirements + ) throws -> LanguageModelSession { + try self.validate(requirements) + return try self.build(tools, transcript, requirements) + } + + private let validate: Validator + private let build: Builder + } +#endif +``` + +Make the existing public initializer delegate to an internal initializer with `.systemDefault`. Store `sessionFactory`, and rename `checkAvailability()` to `checkSystemModelAvailability()`. + +Replace the text-path constructor with: + +```swift +var requirements: FoundationModelsSessionRequirements = [] +if !registrableTools.isEmpty { requirements.insert(.toolCalling) } +let session = try self.sessionFactory.makeSession( + tools: registrableTools, + transcript: transcript, + requirements: requirements +) +``` + +Replace the structured-path constructor with the same call, starting from `[.guidedGeneration]` and adding `.toolCalling` when tools are present. + +- [ ] **Step 3: Verify and commit** + +```bash +/Users/prasadmini/.rbenv/shims/bundle exec fastlane package_tests +git add Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift Sources/AriaApple/Providers/FoundationModelsProvider.swift Sources/AriaApple/Providers/FoundationModelsStructured.swift Tests/AriaAppleTests/Providers +git commit -m "Centralize Foundation Models session construction" +``` + +Expected: all package tests pass; real-model tests may skip when unavailable. + +--- + +### Task 2: Add the iOS 27 model initializer and capability validation + +**Files:** + +- Modify: `Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift` +- Modify: `Sources/AriaApple/Providers/FoundationModelsProvider.swift:27-41` +- Create: `Tests/AriaAppleTests/Providers/FoundationModelsLanguageModelTests.swift` + +- [ ] **Step 1: Add Xcode 27-only failing tests** + +Under `#if canImport(FoundationModels) && compiler(>=6.4)` and `@available(iOS 27.0, macOS 27.0, *)`, add: + +- `testInjectedInitializerKeepsDeclaredCapabilities`, injecting `SystemLanguageModel.default` through the new generic API; +- `testValidationRejectsUnsupportedGuidedGeneration`; +- `testValidationRejectsRequestedToolCalling`; +- `testValidationAcceptsVisionGuidedGenerationAndToolCalling`. + +Construct Apple capability sets with `LanguageModelCapabilities([])` and `LanguageModelCapabilities([.vision, .guidedGeneration, .reasoning, .toolCalling])`. Require failures to be `AgentError.configurationInvalid`. + +Run: + +```bash +DEVELOPER_DIR=/Applications/Xcode-beta.app/Contents/Developer /Users/prasadmini/.rbenv/shims/bundle exec fastlane package_tests +``` + +Expected: compilation fails because the initializer and validator do not exist. + +- [ ] **Step 2: Implement the generic initializer** + +Add: + +```swift +#if compiler(>=6.4) + @available(iOS 27.0, macOS 27.0, *) + public init( + model: Model, + defaultInstructions: String? = nil, + capabilities: ProviderCapabilities, + typedTools: [FoundationModelsToolFactory] = [] + ) { + self.init( + defaultInstructions: defaultInstructions, + capabilities: capabilities, + typedTools: typedTools, + sessionFactory: .injected(model: model, declaredCapabilities: capabilities) + ) + } +#endif +``` + +Add an iOS 27 factory extension that captures `model`, captures `model.capabilities`, and builds: + +```swift +LanguageModelSession(model: model, tools: tools, transcript: transcript) +``` + +Implement: + +```swift +static func validate( + declared: ProviderCapabilities, + available: LanguageModelCapabilities, + requested: FoundationModelsSessionRequirements +) throws +``` + +Build the required set from requested requirements plus: + +- `declared.supportsVision -> .vision`; +- `declared.supportsStructuredOutput -> .guidedGeneration`; +- `declared.supportsToolUse -> .toolCalling`. + +Throw `AgentError.configurationInvalid("Language model does not support: ...")` listing every missing capability. Do not reject extra Apple capabilities and do not add reasoning to `ProviderCapabilities`. + +- [ ] **Step 3: Verify stable and beta toolchains** + +```bash +/Users/prasadmini/.rbenv/shims/bundle exec fastlane package_tests +DEVELOPER_DIR=/Applications/Xcode-beta.app/Contents/Developer /Users/prasadmini/.rbenv/shims/bundle exec fastlane package_tests +``` + +Expected: stable Xcode excludes Swift 6.4 declarations; Xcode 27 compiles them; all runnable tests pass. + +- [ ] **Step 4: Commit** + +```bash +git add Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift Sources/AriaApple/Providers/FoundationModelsProvider.swift Tests/AriaAppleTests/Providers/FoundationModelsLanguageModelTests.swift +git commit -m "Accept iOS 27 language models in AriaApple" +``` + +--- + +### Task 3: Document the model-neutral boundary and Niora insertion points + +**Files:** + +- Modify: `README.md:276-340` +- Modify: `docs/layers/03-providers.md:247-270` +- Modify: `docs/platform-boundary.md:80-126` + +- [ ] **Step 1: Add the public usage example** + +Show a consumer importing `CoreAILanguageModels`, loading `CoreAILanguageModel(resourcesAt:)`, and passing it to: + +```swift +let provider = FoundationModelsProvider( + model: model, + capabilities: ProviderCapabilities( + modelIdentifier: "coreai.qwen3-0.6b", + supportsToolUse: model.capabilities.contains(.toolCalling), + supportsStructuredOutput: model.capabilities.contains(.guidedGeneration) + ) +) +``` + +State that the consumer owns the Core AI dependency, model assets, device eligibility, and fallback. + +- [ ] **Step 2: Record provider behavior** + +Document that: + +- system availability applies only to the default initializer; +- injected load/executor failures use the existing `providerFailed(..., underlying:)` path; +- capability contradictions fail before session generation; +- no automatic fallback occurs; +- Core AI stays out of Aria's root package. + +- [ ] **Step 3: Name Niora's future seams** + +Document these exact insertion points: + +- `iOS/Niora/Services/AI/Runtime/AgentWiring.swift:63`; +- `iOS/Niora/LLMs/Aria/AriaContext.swift:557`; +- `iOS/Niora/Services/AICapabilityRouter.swift:96`. + +Specify a developer-only custom-local subtype beneath `.useLocal` after simulator compatibility is resolved. Do not add a cloud/server resolution. Note that `FoundationModelsWorkflowProvider` stays separate until migrated to Aria's `LLMProvider` surface. + +- [ ] **Step 4: Verify and commit** + +```bash +/Users/prasadmini/.rbenv/shims/bundle exec fastlane lint strict:true +/Users/prasadmini/.rbenv/shims/bundle exec fastlane package_tests +git add README.md docs/layers/03-providers.md docs/platform-boundary.md +git commit -m "Document custom LanguageModel injection" +``` + +--- + +### Task 4: Add an isolated Core AI device proof + +**Files:** + +- Create: `Examples/CoreAIProof/Package.swift` +- Create: `Examples/CoreAIProof/README.md` +- Create: `Examples/CoreAIProof/Tests/CoreAIProofTests/CoreAIProofTests.swift` +- Create: `Examples/CoreAIProof/Tests/CoreAIProofTests/ProofTool.swift` +- Create: `Examples/CoreAIProof/Tests/CoreAIProofTests/Resources/README.md` +- Modify: `.gitignore` + +- [ ] **Step 1: Create the nested package** + +Use a Swift 6.4 manifest with iOS 27 platform, local dependency `.package(path: "../..")`, and: + +```swift +.package( + url: "https://github.com/apple/coreai-models.git", + revision: "de31ba508895c7aa3bdcc57f8837a23f13316871" +) +``` + +Create one test target depending on Aria, AriaApple, AriaTesting, and `.product(name: "CoreAILM", package: "coreai-models")`, with `.copy("Resources")`. + +From the Aria root run: + +```bash +swift package show-dependencies +``` + +Expected: `coreai-models` is absent from the root graph. This read-only graph inspection is the narrow exception to the Fastlane rule. + +- [ ] **Step 2: Ignore proof assets and output** + +Add: + +```gitignore +Examples/CoreAIProof/Tests/CoreAIProofTests/Resources/Qwen3-0.6B/ +Examples/CoreAIProof/Reports/ +Examples/CoreAIProof/.build/ +Examples/CoreAIProof/.swiftpm/ +Examples/CoreAIProof/Package.resolved +``` + +The resource README must include Apple's export command: + +```bash +uv run coreai.llm.export Qwen/Qwen3-0.6B --platform iOS --output-dir ./exported-models +``` + +and direct the exported resource folder to `Tests/CoreAIProofTests/Resources/Qwen3-0.6B/`. + +- [ ] **Step 3: Add proof code** + +Add a deterministic `GenerableTool` named `coreai_aria_proof` whose JSON output always includes `COREAI_ARIA_TOOL_OK`. Confirm its method signature against `Sources/Aria/Providers/Tool.swift` before writing it. + +Load: + +```swift +let resources = try XCTUnwrap( + Bundle.module.url( + forResource: "Qwen3-0.6B", + withExtension: nil, + subdirectory: "Resources" + ) +) +let model = try await CoreAILanguageModel(resourcesAt: resources, mode: .eager) +``` + +Build declared capabilities from `model.capabilities`, register the proof tool, and inject the model into `FoundationModelsProvider`. + +- [ ] **Step 4: Add four opt-in cases** + +Gate the suite with `COREAI_ARIA_PROOF=1`. With the gate enabled, missing resources must fail clearly. + +Add: + +- `testTextStreamsThroughAria`: require start, text, and stop events; +- `testStructuredOutputThroughAria`: require partial output and a final two-field `@Generable` value; +- `testToolExecutionThroughAria`: require the tool event and marker; +- `testTaskEvalRecordsDiagnosticResult`: run one `TaskCase` for one trial, print the summary, and reject only infrastructure errors. + +Measure each with `ContinuousClock` and print timings. Add no performance thresholds and make no MLX comparison. + +- [ ] **Step 5: Document and compile for a physical target** + +The proof README must cover Xcode 27, an iOS 27 physical device, model export/copy, the environment gate, the upstream simulator limitation, and the fact that timings are diagnostics. + +Compile without running: + +```bash +cd Examples/CoreAIProof +DEVELOPER_DIR=/Applications/Xcode-beta.app/Contents/Developer xcodebuild build-for-testing -scheme CoreAIProof-Package -destination 'generic/platform=iOS' -skipPackagePluginValidation -skipMacroValidation +``` + +Expected: build-for-testing succeeds. Then open `Package.swift` in Xcode, select a connected physical iPhone, set `COREAI_ARIA_PROOF=1` on the test scheme, and run all four tests. Expected: all finish and the console prints timing plus a `TaskEval` summary. + +- [ ] **Step 6: Verify isolation and commit** + +```bash +cd ../.. +/Users/prasadmini/.rbenv/shims/bundle exec fastlane package_tests +git status --short +``` + +Expected: root tests pass and no model, report, nested build, or nested resolution file appears. + +```bash +git add .gitignore Examples/CoreAIProof +git commit -m "Add isolated Core AI device proof" +``` + +--- + +### Task 5: Final verification and review + +**Files:** + +- Update this checklist immediately as each step changes state. + +- [ ] **Step 1: Run regression gates** + +```bash +/Users/prasadmini/.rbenv/shims/bundle exec fastlane package_tests +/Users/prasadmini/.rbenv/shims/bundle exec fastlane lint strict:true +DEVELOPER_DIR=/Applications/Xcode-beta.app/Contents/Developer /Users/prasadmini/.rbenv/shims/bundle exec fastlane package_tests +``` + +Expected: stable and beta suites plus lint pass. + +- [ ] **Step 2: Verify isolation and patch hygiene** + +```bash +rg -n "coreai-models|CoreAILanguageModels" Package.swift Sources Tests +git diff --check +git status --short +``` + +Expected: no production/root references, a clean whitespace check, and only intended files. + +- [ ] **Step 3: Conduct focused review** + +Verify: + +- no direct session constructor remains in the provider text/structured paths; +- system availability is not applied to injected models; +- underlying model errors remain in `ErrorBox`; +- capability failures happen before the builder; +- cancellation still maps to `.cancelled` without retry; +- Swift 6.3 cannot see Swift 6.4 symbols; +- the proof is absent from the root graph; +- Niora source and dependencies remain unchanged. + +- [ ] **Step 4: Address every finding and rerun affected gates** + +Apply each fix test-first and mark its checklist item immediately. + +- [ ] **Step 5: Commit review adjustments if any** + +```bash +git add -A +git commit -m "Finish iOS 27 language model integration" +``` + +Skip when review produces no changes. + +## Success Criteria + +- [ ] Existing provider consumers compile unchanged on Xcode 26.6. +- [ ] Xcode 27 consumers can inject any concrete `LanguageModel`. +- [ ] Text, executable-tool, and structured paths use the injected factory. +- [ ] Unsupported declared/requested capabilities fail before generation. +- [ ] Injected models never silently fall back to the system model. +- [ ] Stable tests/lint and Xcode 27 compilation pass. +- [ ] Qwen3-0.6B completes all four physical-device proof cases. +- [ ] Niora's future routing seams are documented without changing its package graph. + +## Documentation Updates Required + +- [ ] `README.md` includes custom-model injection. +- [ ] `docs/layers/03-providers.md` explains behavior and Niora seams. +- [ ] `docs/platform-boundary.md` records dependency ownership. +- [ ] `Examples/CoreAIProof/README.md` contains complete device steps. + diff --git a/docs/superpowers/specs/2026-08-30-language-model-integration-design.md b/docs/superpowers/specs/2026-08-30-language-model-integration-design.md index 0f6de96..b89c904 100644 --- a/docs/superpowers/specs/2026-08-30-language-model-integration-design.md +++ b/docs/superpowers/specs/2026-08-30-language-model-integration-design.md @@ -93,7 +93,7 @@ The existing provider checks `SystemLanguageModel.default.availability`. That ch An injected model does not inherit that check. Its load or generation failures propagate through the existing provider error path. Aria does not invent a second availability abstraction until two non-system model implementations demonstrate a shared need. -Callers provide Aria's `ProviderCapabilities` for an injected model. The provider validates obvious contradictions against `LanguageModel.capabilities` when the corresponding Apple capability is representable, including vision, guided generation, reasoning, and tool calling. Unsupported requested behavior fails before generation with a typed configuration error. +Callers provide Aria's `ProviderCapabilities` for an injected model. The provider validates obvious contradictions against `LanguageModel.capabilities` where Aria has a corresponding declaration: vision, guided generation/structured output, and tool calling. Aria does not currently declare reasoning support, so the provider does not invent a new portable capability solely for this adapter. Unsupported requested behavior fails before generation with a typed configuration error. ## Dependency Boundary From f7a938b535557ae4863ca1212c4eaab82b152538 Mon Sep 17 00:00:00 2001 From: Prasad Pamidi Date: Sun, 30 Aug 2026 01:54:47 -0700 Subject: [PATCH 04/10] Centralize Foundation Models session construction --- .../Providers/FoundationModelsProvider.swift | 27 ++++++- .../FoundationModelsSessionFactory.swift | 54 +++++++++++++ .../FoundationModelsStructured.swift | 11 ++- .../FoundationModelsProviderTests.swift | 31 ++++++++ ...ationModelsSessionFactoryTestSupport.swift | 78 +++++++++++++++++++ .../FoundationModelsStructuredTests.swift | 13 ++++ .../2026-08-30-language-model-integration.md | 52 ++++++------- 7 files changed, 234 insertions(+), 32 deletions(-) create mode 100644 Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift create mode 100644 Tests/AriaAppleTests/Providers/FoundationModelsSessionFactoryTestSupport.swift diff --git a/Sources/AriaApple/Providers/FoundationModelsProvider.swift b/Sources/AriaApple/Providers/FoundationModelsProvider.swift index ec3fa73..63c9522 100644 --- a/Sources/AriaApple/Providers/FoundationModelsProvider.swift +++ b/Sources/AriaApple/Providers/FoundationModelsProvider.swift @@ -30,10 +30,25 @@ defaultInstructions: String? = nil, capabilities: ProviderCapabilities = .foundationModelsDefault, typedTools: [FoundationModelsToolFactory] = [] + ) { + self.init( + defaultInstructions: defaultInstructions, + capabilities: capabilities, + typedTools: typedTools, + sessionFactory: .systemDefault + ) + } + + init( + defaultInstructions: String? = nil, + capabilities: ProviderCapabilities = .foundationModelsDefault, + typedTools: [FoundationModelsToolFactory] = [], + sessionFactory: FoundationModelsSessionFactory ) { self.defaultInstructions = defaultInstructions self.capabilities = capabilities self.typedTools = typedTools + self.sessionFactory = sessionFactory } // MARK: Public @@ -84,6 +99,7 @@ // `FoundationModelsStructured.swift`). let defaultInstructions: String? let typedTools: [FoundationModelsToolFactory] + let sessionFactory: FoundationModelsSessionFactory /// Pull the new-turn prompt out of the message list. Returns the /// text that should be sent via `streamResponse(to:)` plus the @@ -248,8 +264,6 @@ honourSelection: Bool, continuation: AsyncThrowingStream.Continuation ) async throws { - try Self.checkAvailability() - let (prompt, history) = try Self.extractPrompt(from: messages) // Build the typed FM tools. Each factory is invoked with a // closure that yields events into this stream's @@ -305,9 +319,14 @@ defaultInstructions: self.defaultInstructions, toolDefinitions: toolDefinitions ) - let session = LanguageModelSession( + var requirements: FoundationModelsSessionRequirements = [] + if !registrableTools.isEmpty { + requirements.insert(.toolCalling) + } + let session = try self.sessionFactory.makeSession( tools: registrableTools, - transcript: transcript + transcript: transcript, + requirements: requirements ) let messageId = UUID().uuidString diff --git a/Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift b/Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift new file mode 100644 index 0000000..3c7e63c --- /dev/null +++ b/Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift @@ -0,0 +1,54 @@ +#if canImport(FoundationModels) + import Aria + import FoundationModels + + @available(iOS 26.0, macOS 26.0, *) + struct FoundationModelsSessionRequirements: OptionSet, Sendable, Equatable { + let rawValue: UInt8 + + static let vision = Self(rawValue: 1 << 0) + static let guidedGeneration = Self(rawValue: 1 << 1) + static let toolCalling = Self(rawValue: 1 << 2) + } + + @available(iOS 26.0, macOS 26.0, *) + struct FoundationModelsSessionFactory: Sendable { + typealias Validator = @Sendable ( + FoundationModelsSessionRequirements + ) throws -> Void + typealias Builder = @Sendable ( + [any FoundationModels.Tool], + Transcript, + FoundationModelsSessionRequirements + ) throws -> LanguageModelSession + + init( + validate: @escaping Validator, + build: @escaping Builder + ) { + self.validate = validate + self.build = build + } + + static let systemDefault = Self( + validate: { _ in + try FoundationModelsProvider.checkAvailability() + }, + build: { tools, transcript, _ in + LanguageModelSession(tools: tools, transcript: transcript) + } + ) + + func makeSession( + tools: [any FoundationModels.Tool], + transcript: Transcript, + requirements: FoundationModelsSessionRequirements + ) throws -> LanguageModelSession { + try self.validate(requirements) + return try self.build(tools, transcript, requirements) + } + + private let validate: Validator + private let build: Builder + } +#endif diff --git a/Sources/AriaApple/Providers/FoundationModelsStructured.swift b/Sources/AriaApple/Providers/FoundationModelsStructured.swift index ae8c1e3..8e86c4b 100644 --- a/Sources/AriaApple/Providers/FoundationModelsStructured.swift +++ b/Sources/AriaApple/Providers/FoundationModelsStructured.swift @@ -68,7 +68,6 @@ StructuredResponseEvent, any Error >.Continuation ) async throws where Content.PartiallyGenerated: Sendable { - try Self.checkAvailability() let (prompt, history) = try Self.extractPrompt(from: messages) // Forward each tool's `toolCallExecuted` ProviderEvent into @@ -89,7 +88,15 @@ defaultInstructions: self.defaultInstructions, toolDefinitions: toolDefinitions ) - let session = LanguageModelSession(tools: fmTools, transcript: transcript) + var requirements: FoundationModelsSessionRequirements = [.guidedGeneration] + if !fmTools.isEmpty { + requirements.insert(.toolCalling) + } + let session = try self.sessionFactory.makeSession( + tools: fmTools, + transcript: transcript, + requirements: requirements + ) let stream = session.streamResponse(to: prompt, generating: type) var lastRaw: GeneratedContent? diff --git a/Tests/AriaAppleTests/Providers/FoundationModelsProviderTests.swift b/Tests/AriaAppleTests/Providers/FoundationModelsProviderTests.swift index b143a91..94792a6 100644 --- a/Tests/AriaAppleTests/Providers/FoundationModelsProviderTests.swift +++ b/Tests/AriaAppleTests/Providers/FoundationModelsProviderTests.swift @@ -154,6 +154,37 @@ ) } + // MARK: - Session construction + + func testTextStreamUsesConfiguredFactory() async { + let provider = FoundationModelsProvider( + sessionFactory: testSessionFactory(expecting: []) + ) + + await assertExpectedSessionRequirements( + in: provider.stream( + messages: [.user("hello")], + tools: [], + options: .init() + ) + ) + } + + func testTextStreamRequestsToolCallingWhenTypedToolsAreOffered() async { + let provider = FoundationModelsProvider( + typedTools: [{ _ in SessionFactoryTestTool() }], + sessionFactory: testSessionFactory(expecting: [.toolCalling]) + ) + + await assertExpectedSessionRequirements( + in: provider.stream( + messages: [.user("use the tool")], + tools: [], + options: .init() + ) + ) + } + // MARK: - Smoke test (requires real model availability) func testStreamProducesTextDeltasWhenModelAvailable() async throws { diff --git a/Tests/AriaAppleTests/Providers/FoundationModelsSessionFactoryTestSupport.swift b/Tests/AriaAppleTests/Providers/FoundationModelsSessionFactoryTestSupport.swift new file mode 100644 index 0000000..7ac4228 --- /dev/null +++ b/Tests/AriaAppleTests/Providers/FoundationModelsSessionFactoryTestSupport.swift @@ -0,0 +1,78 @@ +#if canImport(FoundationModels) && (os(iOS) || os(macOS) || os(watchOS) || os(tvOS) || os(visionOS)) + import Aria + @testable import AriaApple + import FoundationModels + import XCTest + + @available(iOS 26.0, macOS 26.0, *) + enum SessionFactoryTestError: Error { + case expectedRequirementsReached + case unexpectedRequirements(FoundationModelsSessionRequirements) + case builderReached + } + + @available(iOS 26.0, macOS 26.0, *) + func testSessionFactory( + expecting expected: FoundationModelsSessionRequirements + ) -> FoundationModelsSessionFactory { + FoundationModelsSessionFactory( + validate: { actual in + guard actual == expected else { + throw SessionFactoryTestError.unexpectedRequirements(actual) + } + throw SessionFactoryTestError.expectedRequirementsReached + }, + build: { _, _, _ in + throw SessionFactoryTestError.builderReached + } + ) + } + + @available(iOS 26.0, macOS 26.0, *) + func assertExpectedSessionRequirements( + in stream: AsyncThrowingStream, + file: StaticString = #filePath, + line: UInt = #line + ) async { + do { + for try await _ in stream { } + XCTFail("Expected session factory validation to stop the stream", file: file, line: line) + } catch let error as AgentError { + guard case let .providerFailed(_, underlying) = error else { + return XCTFail("Expected providerFailed, got \(error)", file: file, line: line) + } + XCTAssertEqual( + underlying?.typeName, + "SessionFactoryTestError", + file: file, + line: line + ) + XCTAssertTrue( + underlying?.message.contains("expectedRequirementsReached") == true, + "Expected the requested requirements to reach validation, got \(String(describing: underlying))", + file: file, + line: line + ) + } catch { + XCTFail("Unexpected error: \(error)", file: file, line: line) + } + } + + @available(iOS 26.0, macOS 26.0, *) + struct SessionFactoryTestTool: FoundationModels.Tool { + @Generable + struct Arguments: Codable { + var value: String + } + + typealias Output = String + + let name = "session_factory_test" + let description = "Exercises session requirements." + + func call(arguments _: Arguments) async throws -> String { + "ok" + } + } +#endif + diff --git a/Tests/AriaAppleTests/Providers/FoundationModelsStructuredTests.swift b/Tests/AriaAppleTests/Providers/FoundationModelsStructuredTests.swift index a282957..fa5bb96 100644 --- a/Tests/AriaAppleTests/Providers/FoundationModelsStructuredTests.swift +++ b/Tests/AriaAppleTests/Providers/FoundationModelsStructuredTests.swift @@ -42,6 +42,19 @@ } } + func testStructuredStreamRequestsGuidedGeneration() async { + let provider = FoundationModelsProvider( + sessionFactory: testSessionFactory(expecting: [.guidedGeneration]) + ) + + await assertExpectedSessionRequirements( + in: provider.streamStructured( + messages: [.user("Give me one short quote.")], + as: TestQuote.self + ) + ) + } + // MARK: - Agent extension surface func testAgentRespondDecodesNonFMProviderTextDeltasAsJSON() async throws { diff --git a/docs/superpowers/plans/2026-08-30-language-model-integration.md b/docs/superpowers/plans/2026-08-30-language-model-integration.md index 4dc8214..921f35e 100644 --- a/docs/superpowers/plans/2026-08-30-language-model-integration.md +++ b/docs/superpowers/plans/2026-08-30-language-model-integration.md @@ -34,40 +34,41 @@ - Modify: `Tests/AriaAppleTests/Providers/FoundationModelsStructuredTests.swift:18-43` - Create if shared helpers are needed: `Tests/AriaAppleTests/Providers/FoundationModelsSessionFactoryTestSupport.swift` -- [ ] **Step 1: Add failing tests for all session paths** +- [x] **Step 1: Add failing tests for all session paths** -Add a lock-protected probe whose factory builder records requirements and throws before inference: +Add an outcome-based test factory. Its validator throws one error when the provider requests the expected requirements and a different error for the wrong requirements. Assert the error preserved by the public stream, not the test double's internal state: ```swift -private enum SessionProbeError: Error { case stop } - -final class SessionFactoryProbe: @unchecked Sendable { - private let lock = NSLock() - private var recorded: [FoundationModelsSessionRequirements] = [] - - var requirements: [FoundationModelsSessionRequirements] { - self.lock.withLock { self.recorded } - } +enum SessionFactoryTestError: Error { + case expectedRequirementsReached + case unexpectedRequirements(FoundationModelsSessionRequirements) + case builderReached +} - func factory() -> FoundationModelsSessionFactory { +func testFactory( + expecting expected: FoundationModelsSessionRequirements +) -> FoundationModelsSessionFactory { FoundationModelsSessionFactory( - validate: { _ in }, - build: { [weak self] _, _, requirements in - self?.lock.withLock { self?.recorded.append(requirements) } - throw SessionProbeError.stop + validate: { actual in + guard actual == expected else { + throw SessionFactoryTestError.unexpectedRequirements(actual) + } + throw SessionFactoryTestError.expectedRequirementsReached + }, + build: { _, _, _ in + throw SessionFactoryTestError.builderReached } ) - } } ``` Add tests named: -- `testTextStreamRequestsSessionFromConfiguredFactory` expecting `[[]]`; -- `testExecutableToolStreamRequestsSessionFromConfiguredFactory` expecting `[[]]`; -- `testStructuredStreamRequestsGuidedSessionFromConfiguredFactory` expecting `[.guidedGeneration]`. +- `testTextStreamUsesConfiguredFactory` expecting `[]`; +- `testTextStreamRequestsToolCallingWhenTypedToolsAreOffered` expecting `.toolCalling`; +- `testStructuredStreamRequestsGuidedGeneration` expecting `.guidedGeneration`. -Each consumes the returned stream, expects `AgentError.providerFailed`, and then checks the probe. No real model runs. +Each consumes the returned stream and requires `AgentError.providerFailed` whose underlying `ErrorBox.typeName` is `SessionFactoryTestError`. Also require the message to identify `expectedRequirementsReached`, so a wrong requirement or an accidentally reached builder cannot satisfy the assertion. No real model runs. Run: @@ -77,7 +78,7 @@ Run: Expected: compilation fails because the factory, requirements, and internal provider initializer do not exist. -- [ ] **Step 2: Implement the factory** +- [x] **Step 2: Implement the factory** Create: @@ -109,7 +110,7 @@ Create: } static let systemDefault = Self( - validate: { _ in try FoundationModelsProvider.checkSystemModelAvailability() }, + validate: { _ in try FoundationModelsProvider.checkAvailability() }, build: { tools, transcript, _ in LanguageModelSession(tools: tools, transcript: transcript) } @@ -130,7 +131,7 @@ Create: #endif ``` -Make the existing public initializer delegate to an internal initializer with `.systemDefault`. Store `sessionFactory`, and rename `checkAvailability()` to `checkSystemModelAvailability()`. +Make the existing public initializer delegate to an internal initializer with `.systemDefault`. Store `sessionFactory`, and keep `checkAvailability()` as the system factory's validator so existing prompt probes remain source-compatible. Replace the text-path constructor with: @@ -146,7 +147,7 @@ let session = try self.sessionFactory.makeSession( Replace the structured-path constructor with the same call, starting from `[.guidedGeneration]` and adding `.toolCalling` when tools are present. -- [ ] **Step 3: Verify and commit** +- [⚠️] **Step 3: Verify and commit** ```bash /Users/prasadmini/.rbenv/shims/bundle exec fastlane package_tests @@ -489,4 +490,3 @@ Skip when review produces no changes. - [ ] `docs/layers/03-providers.md` explains behavior and Niora seams. - [ ] `docs/platform-boundary.md` records dependency ownership. - [ ] `Examples/CoreAIProof/README.md` contains complete device steps. - From fa37142de9c87ab0be1bce877ce2895945260508 Mon Sep 17 00:00:00 2001 From: Prasad Pamidi Date: Sun, 30 Aug 2026 01:57:41 -0700 Subject: [PATCH 05/10] Accept iOS 27 language models in AriaApple --- .../Providers/FoundationModelsProvider.swift | 21 +++++ .../FoundationModelsSessionFactory.swift | 66 ++++++++++++++ .../FoundationModelsLanguageModelTests.swift | 85 +++++++++++++++++++ .../2026-08-30-language-model-integration.md | 10 +-- 4 files changed, 177 insertions(+), 5 deletions(-) create mode 100644 Tests/AriaAppleTests/Providers/FoundationModelsLanguageModelTests.swift diff --git a/Sources/AriaApple/Providers/FoundationModelsProvider.swift b/Sources/AriaApple/Providers/FoundationModelsProvider.swift index 63c9522..dc0aed4 100644 --- a/Sources/AriaApple/Providers/FoundationModelsProvider.swift +++ b/Sources/AriaApple/Providers/FoundationModelsProvider.swift @@ -39,6 +39,27 @@ ) } +#if compiler(>=6.4) + @available(iOS 27.0, macOS 27.0, visionOS 27.0, watchOS 27.0, *) + @available(tvOS, unavailable) + public init( + model: Model, + defaultInstructions: String? = nil, + capabilities: ProviderCapabilities, + typedTools: [FoundationModelsToolFactory] = [] + ) { + self.init( + defaultInstructions: defaultInstructions, + capabilities: capabilities, + typedTools: typedTools, + sessionFactory: .injected( + model: model, + declaredCapabilities: capabilities + ) + ) + } +#endif + init( defaultInstructions: String? = nil, capabilities: ProviderCapabilities = .foundationModelsDefault, diff --git a/Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift b/Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift index 3c7e63c..efd82fb 100644 --- a/Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift +++ b/Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift @@ -51,4 +51,70 @@ private let validate: Validator private let build: Builder } + +#if compiler(>=6.4) + @available(iOS 27.0, macOS 27.0, visionOS 27.0, watchOS 27.0, *) + @available(tvOS, unavailable) + extension FoundationModelsSessionFactory { + static func injected( + model: Model, + declaredCapabilities: ProviderCapabilities + ) -> Self { + let availableCapabilities = model.capabilities + return Self( + validate: { requested in + try Self.validate( + declared: declaredCapabilities, + available: availableCapabilities, + requested: requested + ) + }, + build: { tools, transcript, _ in + LanguageModelSession( + model: model, + tools: tools, + transcript: transcript + ) + } + ) + } + + static func validate( + declared: ProviderCapabilities, + available: LanguageModelCapabilities, + requested: FoundationModelsSessionRequirements + ) throws { + var required = requested + if declared.supportsVision { + required.insert(.vision) + } + if declared.supportsStructuredOutput { + required.insert(.guidedGeneration) + } + if declared.supportsToolUse { + required.insert(.toolCalling) + } + + let checks: [( + FoundationModelsSessionRequirements, + LanguageModelCapabilities.Capability, + String + )] = [ + (.vision, .vision, "vision"), + (.guidedGeneration, .guidedGeneration, "structured output"), + (.toolCalling, .toolCalling, "tool calling"), + ] + let missing = checks.compactMap { requirement, capability, name in + required.contains(requirement) && !available.contains(capability) + ? name + : nil + } + guard missing.isEmpty else { + throw AgentError.configurationInvalid( + "Language model does not support: \(missing.joined(separator: ", "))" + ) + } + } + } +#endif #endif diff --git a/Tests/AriaAppleTests/Providers/FoundationModelsLanguageModelTests.swift b/Tests/AriaAppleTests/Providers/FoundationModelsLanguageModelTests.swift new file mode 100644 index 0000000..38b4bd2 --- /dev/null +++ b/Tests/AriaAppleTests/Providers/FoundationModelsLanguageModelTests.swift @@ -0,0 +1,85 @@ +#if canImport(FoundationModels) && compiler(>=6.4) + import Aria + @testable import AriaApple + import FoundationModels + import XCTest + + @available(iOS 27.0, macOS 27.0, *) + final class FoundationModelsLanguageModelTests: XCTestCase { + override func setUpWithError() throws { + guard #available(iOS 27.0, macOS 27.0, *) else { + throw XCTSkip("Requires iOS 27 / macOS 27 runtime") + } + } + + func testInjectedInitializerKeepsDeclaredCapabilities() { + let declared = ProviderCapabilities( + modelIdentifier: "test.system-through-language-model", + supportsStructuredOutput: true + ) + + let provider = FoundationModelsProvider( + model: SystemLanguageModel.default, + capabilities: declared + ) + + XCTAssertEqual(provider.capabilities, declared) + } + + func testValidationRejectsUnsupportedGuidedGeneration() { + let declared = ProviderCapabilities( + modelIdentifier: "test.text-only", + supportsStructuredOutput: true + ) + + XCTAssertThrowsError( + try FoundationModelsSessionFactory.validate( + declared: declared, + available: LanguageModelCapabilities([]), + requested: [] + ) + ) { error in + guard case AgentError.configurationInvalid = error else { + return XCTFail("Expected configurationInvalid, got \(error)") + } + } + } + + func testValidationRejectsRequestedToolCalling() { + let declared = ProviderCapabilities(modelIdentifier: "test.text-only") + + XCTAssertThrowsError( + try FoundationModelsSessionFactory.validate( + declared: declared, + available: LanguageModelCapabilities([]), + requested: [.toolCalling] + ) + ) { error in + guard case AgentError.configurationInvalid = error else { + return XCTFail("Expected configurationInvalid, got \(error)") + } + } + } + + func testValidationAcceptsRepresentableCapabilities() throws { + let declared = ProviderCapabilities( + modelIdentifier: "test.capable", + supportsToolUse: true, + supportsVision: true, + supportsStructuredOutput: true + ) + + try FoundationModelsSessionFactory.validate( + declared: declared, + available: LanguageModelCapabilities([ + .vision, + .guidedGeneration, + .reasoning, + .toolCalling, + ]), + requested: [.toolCalling, .guidedGeneration] + ) + } + } +#endif + diff --git a/docs/superpowers/plans/2026-08-30-language-model-integration.md b/docs/superpowers/plans/2026-08-30-language-model-integration.md index 921f35e..d040d9a 100644 --- a/docs/superpowers/plans/2026-08-30-language-model-integration.md +++ b/docs/superpowers/plans/2026-08-30-language-model-integration.md @@ -147,7 +147,7 @@ let session = try self.sessionFactory.makeSession( Replace the structured-path constructor with the same call, starting from `[.guidedGeneration]` and adding `.toolCalling` when tools are present. -- [⚠️] **Step 3: Verify and commit** +- [x] **Step 3: Verify and commit** ```bash /Users/prasadmini/.rbenv/shims/bundle exec fastlane package_tests @@ -167,7 +167,7 @@ Expected: all package tests pass; real-model tests may skip when unavailable. - Modify: `Sources/AriaApple/Providers/FoundationModelsProvider.swift:27-41` - Create: `Tests/AriaAppleTests/Providers/FoundationModelsLanguageModelTests.swift` -- [ ] **Step 1: Add Xcode 27-only failing tests** +- [x] **Step 1: Add Xcode 27-only failing tests** Under `#if canImport(FoundationModels) && compiler(>=6.4)` and `@available(iOS 27.0, macOS 27.0, *)`, add: @@ -186,7 +186,7 @@ DEVELOPER_DIR=/Applications/Xcode-beta.app/Contents/Developer /Users/prasadmini/ Expected: compilation fails because the initializer and validator do not exist. -- [ ] **Step 2: Implement the generic initializer** +- [x] **Step 2: Implement the generic initializer** Add: @@ -233,7 +233,7 @@ Build the required set from requested requirements plus: Throw `AgentError.configurationInvalid("Language model does not support: ...")` listing every missing capability. Do not reject extra Apple capabilities and do not add reasoning to `ProviderCapabilities`. -- [ ] **Step 3: Verify stable and beta toolchains** +- [x] **Step 3: Verify stable and beta toolchains** ```bash /Users/prasadmini/.rbenv/shims/bundle exec fastlane package_tests @@ -242,7 +242,7 @@ DEVELOPER_DIR=/Applications/Xcode-beta.app/Contents/Developer /Users/prasadmini/ Expected: stable Xcode excludes Swift 6.4 declarations; Xcode 27 compiles them; all runnable tests pass. -- [ ] **Step 4: Commit** +- [⚠️] **Step 4: Commit** ```bash git add Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift Sources/AriaApple/Providers/FoundationModelsProvider.swift Tests/AriaAppleTests/Providers/FoundationModelsLanguageModelTests.swift From d3991592b8b5d2ea045d39361a54d65cd478c61b Mon Sep 17 00:00:00 2001 From: Prasad Pamidi Date: Sun, 30 Aug 2026 02:02:24 -0700 Subject: [PATCH 06/10] Satisfy capability validator lint --- .../FoundationModelsSessionFactory.swift | 32 ++++++++++++------- 1 file changed, 21 insertions(+), 11 deletions(-) diff --git a/Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift b/Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift index efd82fb..8f8ac0e 100644 --- a/Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift +++ b/Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift @@ -56,6 +56,12 @@ @available(iOS 27.0, macOS 27.0, visionOS 27.0, watchOS 27.0, *) @available(tvOS, unavailable) extension FoundationModelsSessionFactory { + private struct CapabilityCheck { + let requirement: FoundationModelsSessionRequirements + let capability: LanguageModelCapabilities.Capability + let name: String + } + static func injected( model: Model, declaredCapabilities: ProviderCapabilities @@ -95,18 +101,22 @@ required.insert(.toolCalling) } - let checks: [( - FoundationModelsSessionRequirements, - LanguageModelCapabilities.Capability, - String - )] = [ - (.vision, .vision, "vision"), - (.guidedGeneration, .guidedGeneration, "structured output"), - (.toolCalling, .toolCalling, "tool calling"), + let checks = [ + CapabilityCheck(requirement: .vision, capability: .vision, name: "vision"), + CapabilityCheck( + requirement: .guidedGeneration, + capability: .guidedGeneration, + name: "structured output" + ), + CapabilityCheck( + requirement: .toolCalling, + capability: .toolCalling, + name: "tool calling" + ), ] - let missing = checks.compactMap { requirement, capability, name in - required.contains(requirement) && !available.contains(capability) - ? name + let missing = checks.compactMap { check in + required.contains(check.requirement) && !available.contains(check.capability) + ? check.name : nil } guard missing.isEmpty else { From dfe392319ad65d3f3f4f0180a4efdc949782d13b Mon Sep 17 00:00:00 2001 From: Prasad Pamidi Date: Sun, 30 Aug 2026 02:02:41 -0700 Subject: [PATCH 07/10] Document custom LanguageModel injection --- README.md | 27 +++++++++++++ docs/layers/03-providers.md | 40 +++++++++++++++++++ docs/platform-boundary.md | 13 ++++++ .../2026-08-30-language-model-integration.md | 14 ++++--- 4 files changed, 89 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 3f33228..6a46367 100644 --- a/README.md +++ b/README.md @@ -304,6 +304,33 @@ for try await event in agent.stream(.message(.user("What's the time in Tokyo?")) } ``` +### Custom Apple language models (iOS 27+) + +`AriaApple` accepts any Foundation Models `LanguageModel`, so an app can run a +Core AI model through the same `LLMProvider`, agent, tool, memory, and middleware +surfaces as Apple's system model: + +```swift +import Aria +import AriaApple +import CoreAILanguageModels +import FoundationModels + +let model = try await CoreAILanguageModel(resourcesAt: modelResourcesURL) +let provider = FoundationModelsProvider( + model: model, + capabilities: ProviderCapabilities( + modelIdentifier: "coreai.qwen3-0.6b", + supportsToolUse: model.capabilities.contains(.toolCalling), + supportsStructuredOutput: model.capabilities.contains(.guidedGeneration) + ) +) +``` + +The app owns the Core AI package dependency, model resources, device-eligibility +checks, and fallback policy. Aria depends only on Foundation Models protocols and +does not silently replace an injected model when it cannot load or execute. + ### Production-grade middleware stack Long-running threads need bounded context, a summary of older turns, and a diff --git a/docs/layers/03-providers.md b/docs/layers/03-providers.md index 3f2d36e..fa2ee4d 100644 --- a/docs/layers/03-providers.md +++ b/docs/layers/03-providers.md @@ -256,6 +256,46 @@ Concrete implementations live in `AriaApple/Providers/` and conform to the proto The agent layer never sees the provider's native types. +### Foundation Models model injection + +On iOS 27 and related Apple platform releases, `FoundationModelsProvider` can +accept any concrete Foundation Models `LanguageModel`. Both Apple's default +system model and an injected model use the same transcript, tool, streaming, +and structured-output paths. + +The two initializers deliberately have different ownership rules: + +- `FoundationModelsProvider()` checks `SystemLanguageModel.default.availability`. +- `FoundationModelsProvider(model:capabilities:)` uses only the supplied model. + It does not check system-model availability or fall back to the system model. +- Declared capabilities that the injected model cannot provide fail with + `AgentError.configurationInvalid` before generation starts. +- Session-construction and model-execution errors continue through the existing + `AgentError.providerFailed(..., underlying:)` stream error path. + +Aria does not load custom model resources and does not choose a fallback. The +consumer owns the runtime package, assets, device eligibility, and routing +policy. In particular, the root `Aria` target does not depend on Core AI; +`AriaApple` integrates through Foundation Models protocols only. + +#### Niora integration map + +Once Core AI can coexist with Niora's normal simulator builds, Niora can add a +developer-only custom-local subtype beneath its existing `.useLocal` resolution. +It should not add a new cloud or server resolution for an on-device model. The +relevant seams in Niora are: + +- `iOS/Niora/Services/AI/Runtime/AgentWiring.swift:63` — construct the injected + provider for seeded agents. +- `iOS/Niora/LLMs/Aria/AriaContext.swift:557` — construct it for capability-based + agents while preserving the existing tool and middleware setup. +- `iOS/Niora/Services/AICapabilityRouter.swift:96` — select the developer-only + custom-local subtype without changing production routing. + +`FoundationModelsWorkflowProvider` remains a separate Niora adapter until its +workflow surface is migrated to Aria's `LLMProvider`; this integration must not +silently bypass that boundary. + ## What this layer does NOT include - Concrete provider implementations. Those live in `AriaApple`. diff --git a/docs/platform-boundary.md b/docs/platform-boundary.md index 3785555..93af499 100644 --- a/docs/platform-boundary.md +++ b/docs/platform-boundary.md @@ -84,6 +84,19 @@ Aria uses `#if canImport(...)` sparingly. The preferred pattern is **physical se If conditional compilation is unavoidable in core (very rare), use `#if canImport(...)` rather than `#if os(...)`. Reason: `canImport` is robust to future platforms; `os` requires updating every time a new target appears. +## Apple model adapters + +Foundation Models adapters belong in `AriaApple`. The portable `LLMProvider` +surface in `Aria` must not expose `LanguageModel`, `LanguageModelSession`, or +other Apple framework types. + +An application may inject an iOS 27 `LanguageModel` into +`FoundationModelsProvider`, including one supplied by Core AI. The application +owns that runtime dependency and its model resources. Neither the root package +graph nor `AriaApple` should add a direct Core AI dependency: integration occurs +through the Foundation Models protocol, keeping Core AI optional and leaving +device eligibility and fallback decisions with the application. + ## CI enforcement Aria's CI matrix: diff --git a/docs/superpowers/plans/2026-08-30-language-model-integration.md b/docs/superpowers/plans/2026-08-30-language-model-integration.md index d040d9a..c08dd8b 100644 --- a/docs/superpowers/plans/2026-08-30-language-model-integration.md +++ b/docs/superpowers/plans/2026-08-30-language-model-integration.md @@ -242,7 +242,7 @@ DEVELOPER_DIR=/Applications/Xcode-beta.app/Contents/Developer /Users/prasadmini/ Expected: stable Xcode excludes Swift 6.4 declarations; Xcode 27 compiles them; all runnable tests pass. -- [⚠️] **Step 4: Commit** +- [x] **Step 4: Commit** ```bash git add Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift Sources/AriaApple/Providers/FoundationModelsProvider.swift Tests/AriaAppleTests/Providers/FoundationModelsLanguageModelTests.swift @@ -259,7 +259,7 @@ git commit -m "Accept iOS 27 language models in AriaApple" - Modify: `docs/layers/03-providers.md:247-270` - Modify: `docs/platform-boundary.md:80-126` -- [ ] **Step 1: Add the public usage example** +- [x] **Step 1: Add the public usage example** Show a consumer importing `CoreAILanguageModels`, loading `CoreAILanguageModel(resourcesAt:)`, and passing it to: @@ -276,7 +276,7 @@ let provider = FoundationModelsProvider( State that the consumer owns the Core AI dependency, model assets, device eligibility, and fallback. -- [ ] **Step 2: Record provider behavior** +- [x] **Step 2: Record provider behavior** Document that: @@ -286,7 +286,7 @@ Document that: - no automatic fallback occurs; - Core AI stays out of Aria's root package. -- [ ] **Step 3: Name Niora's future seams** +- [x] **Step 3: Name Niora's future seams** Document these exact insertion points: @@ -296,7 +296,11 @@ Document these exact insertion points: Specify a developer-only custom-local subtype beneath `.useLocal` after simulator compatibility is resolved. Do not add a cloud/server resolution. Note that `FoundationModelsWorkflowProvider` stays separate until migrated to Aria's `LLMProvider` surface. -- [ ] **Step 4: Verify and commit** +- [x] **Step 4: Verify and commit** + +Verification: stable and Xcode 27 package suites pass. Focused strict lint is +clean for the changed Swift source; the repository-wide strict lane remains +blocked by 40 pre-existing violations outside this work. ```bash /Users/prasadmini/.rbenv/shims/bundle exec fastlane lint strict:true From 9ce0431f802e96295594591aa833ea29e26dfe97 Mon Sep 17 00:00:00 2001 From: Prasad Pamidi Date: Sun, 30 Aug 2026 02:10:31 -0700 Subject: [PATCH 08/10] Add isolated Core AI device proof --- .gitignore | 7 + Examples/CoreAIProof/Package.swift | 31 +++ Examples/CoreAIProof/README.md | 60 ++++++ .../CoreAIProofTests/CoreAIProofTests.swift | 186 ++++++++++++++++++ .../Tests/CoreAIProofTests/ProofTool.swift | 33 ++++ .../CoreAIProofTests/Resources/README.md | 16 ++ .../2026-08-30-language-model-integration.md | 16 +- 7 files changed, 343 insertions(+), 6 deletions(-) create mode 100644 Examples/CoreAIProof/Package.swift create mode 100644 Examples/CoreAIProof/README.md create mode 100644 Examples/CoreAIProof/Tests/CoreAIProofTests/CoreAIProofTests.swift create mode 100644 Examples/CoreAIProof/Tests/CoreAIProofTests/ProofTool.swift create mode 100644 Examples/CoreAIProof/Tests/CoreAIProofTests/Resources/README.md diff --git a/.gitignore b/.gitignore index 7b282f7..c138ee9 100644 --- a/.gitignore +++ b/.gitignore @@ -56,3 +56,10 @@ Icon # Local git worktrees (Claude / superpowers convention) .worktrees/ Vendor/ + +# Opt-in Core AI device proof +Examples/CoreAIProof/Tests/CoreAIProofTests/Resources/Qwen3-0.6B/ +Examples/CoreAIProof/Reports/ +Examples/CoreAIProof/.build/ +Examples/CoreAIProof/.swiftpm/ +Examples/CoreAIProof/Package.resolved diff --git a/Examples/CoreAIProof/Package.swift b/Examples/CoreAIProof/Package.swift new file mode 100644 index 0000000..2b2088a --- /dev/null +++ b/Examples/CoreAIProof/Package.swift @@ -0,0 +1,31 @@ +// swift-tools-version: 6.4 + +import PackageDescription + +let package = Package( + name: "CoreAIProof", + platforms: [ + .iOS(.v27), + ], + dependencies: [ + .package(name: "aria", path: "../.."), + .package( + url: "https://github.com/apple/coreai-models.git", + revision: "de31ba508895c7aa3bdcc57f8837a23f13316871" + ), + ], + targets: [ + .testTarget( + name: "CoreAIProofTests", + dependencies: [ + .product(name: "Aria", package: "aria"), + .product(name: "AriaApple", package: "aria"), + .product(name: "AriaTesting", package: "aria"), + .product(name: "CoreAILM", package: "coreai-models"), + ], + resources: [ + .copy("Resources"), + ] + ), + ] +) diff --git a/Examples/CoreAIProof/README.md b/Examples/CoreAIProof/README.md new file mode 100644 index 0000000..09f7420 --- /dev/null +++ b/Examples/CoreAIProof/README.md @@ -0,0 +1,60 @@ +# Core AI through Aria device proof + +This opt-in package verifies that a Core AI `CoreAILanguageModel` can use Aria's +text streaming, guided generation, typed tools, and task-evaluation surfaces. It +is deliberately outside Aria's root package graph and is not a benchmark. + +## Requirements + +- Xcode 27 or newer. +- A physical iPhone running iOS 27 or newer. +- `uv` and a checkout of Apple's + [`coreai-models`](https://github.com/apple/coreai-models) repository. + +Core AI is unavailable in the iOS Simulator SDK. The pinned upstream package +cannot currently compile when a Core AI product is linked for a simulator +destination, so use a generic iOS or connected-device destination only. + +## Prepare the model + +From the `coreai-models` checkout, export Qwen3-0.6B: + +```bash +uv run coreai.llm.export Qwen/Qwen3-0.6B --platform iOS --output-dir ./exported-models +``` + +Copy the exported model resource folder to: + +```text +Tests/CoreAIProofTests/Resources/Qwen3-0.6B/ +``` + +The copied directory is ignored by Git. It must contain the export's +`metadata.json`, model asset, and tokenizer resources. + +## Compile the proof + +From this directory: + +```bash +DEVELOPER_DIR=/Applications/Xcode-beta.app/Contents/Developer \ + xcodebuild build-for-testing \ + -scheme CoreAIProof-Package \ + -destination 'generic/platform=iOS' \ + -skipPackagePluginValidation \ + -skipMacroValidation +``` + +## Run on a device + +1. Open this directory's `Package.swift` in Xcode 27. +2. Select a connected iPhone running iOS 27 or newer. +3. Add `COREAI_ARIA_PROOF=1` to the test scheme's environment variables. +4. Run all `CoreAIProofTests` tests. + +Without the environment gate, the suite skips before looking for model assets. +With it enabled, missing resources fail with the expected destination path. + +Each case prints a `ContinuousClock` duration, and the task-evaluation case +prints its `TaskEval` summary. These values are diagnostic evidence only: there +are no performance thresholds and no Core AI versus MLX comparison. diff --git a/Examples/CoreAIProof/Tests/CoreAIProofTests/CoreAIProofTests.swift b/Examples/CoreAIProof/Tests/CoreAIProofTests/CoreAIProofTests.swift new file mode 100644 index 0000000..099d371 --- /dev/null +++ b/Examples/CoreAIProof/Tests/CoreAIProofTests/CoreAIProofTests.swift @@ -0,0 +1,186 @@ +import Aria +import AriaApple +import AriaTesting +import CoreAILanguageModels +import Foundation +import FoundationModels +import XCTest + +@available(iOS 27.0, *) +@Generable +private struct ProofStructuredResponse { + @Guide(description: "A short confirmation status") + var status: String + + @Guide(description: "A short diagnostic detail") + var detail: String +} + +@available(iOS 27.0, *) +final class CoreAIProofTests: XCTestCase { + override func setUp() async throws { + try await super.setUp() + try XCTSkipUnless( + ProcessInfo.processInfo.environment["COREAI_ARIA_PROOF"] == "1", + "Set COREAI_ARIA_PROOF=1 to run the physical-device proof" + ) + + let resources = try XCTUnwrap( + Bundle.module.url( + forResource: "Qwen3-0.6B", + withExtension: nil, + subdirectory: "Resources" + ), + "Export Qwen3-0.6B and copy it into the proof Resources directory" + ) + let model = try await CoreAILanguageModel(resourcesAt: resources, mode: .eager) + self.model = model + self.capabilities = ProviderCapabilities( + modelIdentifier: "coreai.qwen3-0.6b", + supportsToolUse: model.capabilities.contains(.toolCalling), + supportsStructuredOutput: model.capabilities.contains(.guidedGeneration) + ) + self.toolKit = registerFoundationModelsTool(ProofTool()) + } + + private var model: CoreAILanguageModel? + private var capabilities: ProviderCapabilities? + private var toolKit: FoundationModelsToolKit? + + private func provider(includeProofTool: Bool = false) throws -> FoundationModelsProvider { + let model = try XCTUnwrap(self.model) + let capabilities = try XCTUnwrap(self.capabilities) + let toolKit = try XCTUnwrap(self.toolKit) + return FoundationModelsProvider( + model: model, + defaultInstructions: "Follow the diagnostic request exactly and answer concisely.", + capabilities: capabilities, + typedTools: includeProofTool ? [toolKit.factory] : [] + ) + } + + private func printTiming(_ label: String, since start: ContinuousClock.Instant) { + print("CoreAIProof \(label): \(ContinuousClock.now - start)") + } +} + +@available(iOS 27.0, *) +extension CoreAIProofTests { + func testTextStreamsThroughAria() async throws { + let started = ContinuousClock.now + defer { self.printTiming("text", since: started) } + + var sawStart = false + var text = "" + var sawStop = false + for try await event in try self.provider().stream( + messages: [.user("Reply with one short sentence confirming this Core AI request.")], + tools: [], + options: GenerationOptions(maxTokens: 64) + ) { + switch event { + case .messageStart: + sawStart = true + case let .textDelta(delta): + text += delta + case .messageStop: + sawStop = true + default: + break + } + } + + XCTAssertTrue(sawStart, "Expected Aria's provider start event") + XCTAssertFalse(text.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty) + XCTAssertTrue(sawStop, "Expected Aria's provider stop event") + } + + func testStructuredOutputThroughAria() async throws { + let started = ContinuousClock.now + defer { self.printTiming("structured", since: started) } + + var sawPartial = false + var final: ProofStructuredResponse? + for try await event in try self.provider().streamStructured( + messages: [.user("Return a status and detail confirming the Core AI Aria proof.")], + as: ProofStructuredResponse.self + ) { + switch event { + case .partial: + sawPartial = true + case let .finish(content): + final = content + case .toolCallExecuted: + break + } + } + + XCTAssertTrue(sawPartial, "Expected at least one partial structured value") + let output = try XCTUnwrap(final) + XCTAssertFalse(output.status.isEmpty) + XCTAssertFalse(output.detail.isEmpty) + } + + func testToolExecutionThroughAria() async throws { + let started = ContinuousClock.now + defer { self.printTiming("tool", since: started) } + + var output: ProofToolOutput? + let toolKit = try XCTUnwrap(self.toolKit) + for try await event in try self.provider(includeProofTool: true).stream( + messages: [ + .user( + "Call coreai_aria_proof exactly once with request device_integration, " + + "then report the returned marker." + ), + ], + executableTools: [toolKit.anyTool], + options: GenerationOptions(maxTokens: 128) + ) { + guard case let .toolCallExecuted(call, result) = event, + call.name == ProofTool.name else { + continue + } + output = try result.output.decode(ProofToolOutput.self) + } + + XCTAssertEqual(output?.marker, "COREAI_ARIA_TOOL_OK") + } + + func testTaskEvalRecordsDiagnosticResult() async throws { + let started = ContinuousClock.now + defer { self.printTiming("task-eval", since: started) } + + let model = try XCTUnwrap(self.model) + let capabilities = try XCTUnwrap(self.capabilities) + let toolKit = try XCTUnwrap(self.toolKit) + let testCase = TaskCase( + query: "Call coreai_aria_proof once and report the returned marker.", + tools: [toolKit.anyTool], + expectedTool: ProofTool.name, + mustContain: ["COREAI_ARIA_TOOL_OK"], + note: "One-trial integration diagnostic; semantic misses are recorded, not gated." + ) + let report = await TaskEval(cases: [testCase], trials: 1).run( + label: "Core AI through Aria" + ) { testCase in + Agent(config: AgentConfig( + provider: FoundationModelsProvider( + model: model, + defaultInstructions: "Use the requested diagnostic tool and report its result.", + capabilities: capabilities, + typedTools: [toolKit.factory] + ), + tools: testCase.tools, + systemPrompt: "Report only results returned by the diagnostic tool." + )) + } + + print("\n\(report.summary())\n") + XCTAssertEqual(report.outcomes.count, 1) + XCTAssertNil( + report.outcomes.first?.error, + "The diagnostic records semantic misses but rejects infrastructure errors" + ) + } +} diff --git a/Examples/CoreAIProof/Tests/CoreAIProofTests/ProofTool.swift b/Examples/CoreAIProof/Tests/CoreAIProofTests/ProofTool.swift new file mode 100644 index 0000000..38a8235 --- /dev/null +++ b/Examples/CoreAIProof/Tests/CoreAIProofTests/ProofTool.swift @@ -0,0 +1,33 @@ +import Aria +import AriaApple +import FoundationModels + +@available(iOS 27.0, *) +@Generable +struct ProofToolInput: Codable { + @Guide(description: "The exact diagnostic request to acknowledge") + var request: String +} + +struct ProofToolOutput: Codable, Equatable, Sendable { + let marker: String +} + +@available(iOS 27.0, *) +struct ProofTool: GenerableTool { + typealias Input = ProofToolInput + typealias Output = ProofToolOutput + + static let name = "coreai_aria_proof" + static let description = "Returns the deterministic Core AI through Aria proof marker." + static let inputSchema = JSONSchema.object( + properties: [ + "request": .string(description: "The diagnostic request to acknowledge."), + ], + required: ["request"] + ) + + func call(_: ProofToolInput, context _: ToolContext) async throws -> ProofToolOutput { + ProofToolOutput(marker: "COREAI_ARIA_TOOL_OK") + } +} diff --git a/Examples/CoreAIProof/Tests/CoreAIProofTests/Resources/README.md b/Examples/CoreAIProof/Tests/CoreAIProofTests/Resources/README.md new file mode 100644 index 0000000..9b51fde --- /dev/null +++ b/Examples/CoreAIProof/Tests/CoreAIProofTests/Resources/README.md @@ -0,0 +1,16 @@ +# Core AI proof model resources + +Export Qwen3-0.6B from a checkout of Apple's `coreai-models` repository: + +```bash +uv run coreai.llm.export Qwen/Qwen3-0.6B --platform iOS --output-dir ./exported-models +``` + +Copy the exported resource folder itself to: + +```text +Tests/CoreAIProofTests/Resources/Qwen3-0.6B/ +``` + +The final directory must contain the export's `metadata.json`, model asset, and +tokenizer resources. Model files are intentionally ignored by Git. diff --git a/docs/superpowers/plans/2026-08-30-language-model-integration.md b/docs/superpowers/plans/2026-08-30-language-model-integration.md index c08dd8b..b2a0457 100644 --- a/docs/superpowers/plans/2026-08-30-language-model-integration.md +++ b/docs/superpowers/plans/2026-08-30-language-model-integration.md @@ -322,7 +322,7 @@ git commit -m "Document custom LanguageModel injection" - Create: `Examples/CoreAIProof/Tests/CoreAIProofTests/Resources/README.md` - Modify: `.gitignore` -- [ ] **Step 1: Create the nested package** +- [x] **Step 1: Create the nested package** Use a Swift 6.4 manifest with iOS 27 platform, local dependency `.package(path: "../..")`, and: @@ -343,7 +343,7 @@ swift package show-dependencies Expected: `coreai-models` is absent from the root graph. This read-only graph inspection is the narrow exception to the Fastlane rule. -- [ ] **Step 2: Ignore proof assets and output** +- [x] **Step 2: Ignore proof assets and output** Add: @@ -363,7 +363,7 @@ uv run coreai.llm.export Qwen/Qwen3-0.6B --platform iOS --output-dir ./exported- and direct the exported resource folder to `Tests/CoreAIProofTests/Resources/Qwen3-0.6B/`. -- [ ] **Step 3: Add proof code** +- [x] **Step 3: Add proof code** Add a deterministic `GenerableTool` named `coreai_aria_proof` whose JSON output always includes `COREAI_ARIA_TOOL_OK`. Confirm its method signature against `Sources/Aria/Providers/Tool.swift` before writing it. @@ -382,7 +382,7 @@ let model = try await CoreAILanguageModel(resourcesAt: resources, mode: .eager) Build declared capabilities from `model.capabilities`, register the proof tool, and inject the model into `FoundationModelsProvider`. -- [ ] **Step 4: Add four opt-in cases** +- [x] **Step 4: Add four opt-in cases** Gate the suite with `COREAI_ARIA_PROOF=1`. With the gate enabled, missing resources must fail clearly. @@ -395,7 +395,11 @@ Add: Measure each with `ContinuousClock` and print timings. Add no performance thresholds and make no MLX comparison. -- [ ] **Step 5: Document and compile for a physical target** +- [⚠️] **Step 5: Document and compile for a physical target** + +`build-for-testing` succeeds for generic iOS with Xcode 27. Running the four +cases remains pending because the ignored Qwen3-0.6B resources and a connected +iOS 27 device are not present in this worktree/session. The proof README must cover Xcode 27, an iOS 27 physical device, model export/copy, the environment gate, the upstream simulator limitation, and the fact that timings are diagnostics. @@ -408,7 +412,7 @@ DEVELOPER_DIR=/Applications/Xcode-beta.app/Contents/Developer xcodebuild build-f Expected: build-for-testing succeeds. Then open `Package.swift` in Xcode, select a connected physical iPhone, set `COREAI_ARIA_PROOF=1` on the test scheme, and run all four tests. Expected: all finish and the console prints timing plus a `TaskEval` summary. -- [ ] **Step 6: Verify isolation and commit** +- [x] **Step 6: Verify isolation and commit** ```bash cd ../.. From 19dfd2020df5554f964ac2a1ae86b4ecb309a913 Mon Sep 17 00:00:00 2001 From: Prasad Pamidi Date: Sun, 30 Aug 2026 02:17:13 -0700 Subject: [PATCH 09/10] Finish iOS 27 language model integration review --- .../FoundationModelsLanguageModelTests.swift | 1 - ...ationModelsSessionFactoryTestSupport.swift | 1 - .../2026-08-30-language-model-integration.md | 42 +++++++++++-------- 3 files changed, 25 insertions(+), 19 deletions(-) diff --git a/Tests/AriaAppleTests/Providers/FoundationModelsLanguageModelTests.swift b/Tests/AriaAppleTests/Providers/FoundationModelsLanguageModelTests.swift index 38b4bd2..f829a85 100644 --- a/Tests/AriaAppleTests/Providers/FoundationModelsLanguageModelTests.swift +++ b/Tests/AriaAppleTests/Providers/FoundationModelsLanguageModelTests.swift @@ -82,4 +82,3 @@ } } #endif - diff --git a/Tests/AriaAppleTests/Providers/FoundationModelsSessionFactoryTestSupport.swift b/Tests/AriaAppleTests/Providers/FoundationModelsSessionFactoryTestSupport.swift index 7ac4228..d65574e 100644 --- a/Tests/AriaAppleTests/Providers/FoundationModelsSessionFactoryTestSupport.swift +++ b/Tests/AriaAppleTests/Providers/FoundationModelsSessionFactoryTestSupport.swift @@ -75,4 +75,3 @@ } } #endif - diff --git a/docs/superpowers/plans/2026-08-30-language-model-integration.md b/docs/superpowers/plans/2026-08-30-language-model-integration.md index b2a0457..c30d567 100644 --- a/docs/superpowers/plans/2026-08-30-language-model-integration.md +++ b/docs/superpowers/plans/2026-08-30-language-model-integration.md @@ -435,7 +435,11 @@ git commit -m "Add isolated Core AI device proof" - Update this checklist immediately as each step changes state. -- [ ] **Step 1: Run regression gates** +- [⚠️] **Step 1: Run regression gates** + +Stable and Xcode 27 package suites pass. The strict lint lane still fails on 40 +pre-existing violations; focused strict lint passes for every changed Swift +source and the isolated proof sources. ```bash /Users/prasadmini/.rbenv/shims/bundle exec fastlane package_tests @@ -445,7 +449,7 @@ DEVELOPER_DIR=/Applications/Xcode-beta.app/Contents/Developer /Users/prasadmini/ Expected: stable and beta suites plus lint pass. -- [ ] **Step 2: Verify isolation and patch hygiene** +- [x] **Step 2: Verify isolation and patch hygiene** ```bash rg -n "coreai-models|CoreAILanguageModels" Package.swift Sources Tests @@ -455,7 +459,11 @@ git status --short Expected: no production/root references, a clean whitespace check, and only intended files. -- [ ] **Step 3: Conduct focused review** +- [x] **Step 3: Conduct focused review** + +Review found no Critical or Important issues. Two Minor full-range whitespace +findings are addressed in Step 4. Physical proof execution remains an external +runtime blocker, not an implementation defect. Verify: @@ -468,11 +476,11 @@ Verify: - the proof is absent from the root graph; - Niora source and dependencies remain unchanged. -- [ ] **Step 4: Address every finding and rerun affected gates** +- [x] **Step 4: Address every finding and rerun affected gates** Apply each fix test-first and mark its checklist item immediately. -- [ ] **Step 5: Commit review adjustments if any** +- [x] **Step 5: Commit review adjustments if any** ```bash git add -A @@ -483,18 +491,18 @@ Skip when review produces no changes. ## Success Criteria -- [ ] Existing provider consumers compile unchanged on Xcode 26.6. -- [ ] Xcode 27 consumers can inject any concrete `LanguageModel`. -- [ ] Text, executable-tool, and structured paths use the injected factory. -- [ ] Unsupported declared/requested capabilities fail before generation. -- [ ] Injected models never silently fall back to the system model. -- [ ] Stable tests/lint and Xcode 27 compilation pass. -- [ ] Qwen3-0.6B completes all four physical-device proof cases. -- [ ] Niora's future routing seams are documented without changing its package graph. +- [x] Existing provider consumers compile unchanged on Xcode 26.6. +- [x] Xcode 27 consumers can inject any concrete `LanguageModel`. +- [x] Text, executable-tool, and structured paths use the injected factory. +- [x] Unsupported declared/requested capabilities fail before generation. +- [x] Injected models never silently fall back to the system model. +- [⚠️] Stable tests and Xcode 27 compilation pass; strict lint remains blocked by 40 pre-existing violations. +- [⚠️] Qwen3-0.6B physical-device cases await model resources and a connected iOS 27 device. +- [x] Niora's future routing seams are documented without changing its package graph. ## Documentation Updates Required -- [ ] `README.md` includes custom-model injection. -- [ ] `docs/layers/03-providers.md` explains behavior and Niora seams. -- [ ] `docs/platform-boundary.md` records dependency ownership. -- [ ] `Examples/CoreAIProof/README.md` contains complete device steps. +- [x] `README.md` includes custom-model injection. +- [x] `docs/layers/03-providers.md` explains behavior and Niora seams. +- [x] `docs/platform-boundary.md` records dependency ownership. +- [x] `Examples/CoreAIProof/README.md` contains complete device steps. From 6cf233fe43a8d746f0765ea2bb78067d163dce13 Mon Sep 17 00:00:00 2001 From: Prasad Pamidi Date: Sun, 30 Aug 2026 03:54:09 -0700 Subject: [PATCH 10/10] Prepare 0.12.0 release Document custom Apple language model injection, update the runtime version, and apply the repository formatter. Co-Authored-By: Codex Opus 4.6 --- CHANGELOG.md | 19 ++ Sources/Aria/Aria.swift | 2 +- .../Providers/FoundationModelsProvider.swift | 38 ++-- .../FoundationModelsSessionFactory.swift | 170 +++++++++--------- 4 files changed, 127 insertions(+), 102 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9d6ff15..2cc1f25 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,25 @@ and Aria adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] +## [0.12.0] - 2026-08-30 + +### Added + +- **Custom Apple `LanguageModel` injection on iOS 27 and related platform releases.** + `FoundationModelsProvider` can now construct sessions from any Foundation + Models `LanguageModel`, including models backed by Core AI, while retaining + the existing transcript, tools, streaming, structured-output, and error + handling paths. Aria validates declared model capabilities before generation + and leaves model loading, routing, and fallback policy to the host app. +- **An isolated Core AI device proof.** The example package demonstrates the + integration boundary without adding Core AI or model assets to Aria's core + dependency graph. + +### Changed + +- Foundation Models session creation now flows through one internal factory so + the system model and injected models share the same execution behavior. + ## [0.1.4] - 2026-05-26 ### Added diff --git a/Sources/Aria/Aria.swift b/Sources/Aria/Aria.swift index 3cd773a..3abe056 100644 --- a/Sources/Aria/Aria.swift +++ b/Sources/Aria/Aria.swift @@ -21,7 +21,7 @@ public enum AriaInfo { /// /// Aria follows semantic versioning once it reaches `1.0.0`. Until then, /// breaking changes may occur on minor version bumps. - public static let version = "0.11.0" + public static let version = "0.12.0" } /// Internal logger used by core types. Backends are installed by the platform diff --git a/Sources/AriaApple/Providers/FoundationModelsProvider.swift b/Sources/AriaApple/Providers/FoundationModelsProvider.swift index dc0aed4..1da090c 100644 --- a/Sources/AriaApple/Providers/FoundationModelsProvider.swift +++ b/Sources/AriaApple/Providers/FoundationModelsProvider.swift @@ -39,26 +39,26 @@ ) } -#if compiler(>=6.4) - @available(iOS 27.0, macOS 27.0, visionOS 27.0, watchOS 27.0, *) - @available(tvOS, unavailable) - public init( - model: Model, - defaultInstructions: String? = nil, - capabilities: ProviderCapabilities, - typedTools: [FoundationModelsToolFactory] = [] - ) { - self.init( - defaultInstructions: defaultInstructions, - capabilities: capabilities, - typedTools: typedTools, - sessionFactory: .injected( - model: model, - declaredCapabilities: capabilities + #if compiler(>=6.4) + @available(iOS 27.0, macOS 27.0, visionOS 27.0, watchOS 27.0, *) + @available(tvOS, unavailable) + public init( + model: some LanguageModel, + defaultInstructions: String? = nil, + capabilities: ProviderCapabilities, + typedTools: [FoundationModelsToolFactory] = [] + ) { + self.init( + defaultInstructions: defaultInstructions, + capabilities: capabilities, + typedTools: typedTools, + sessionFactory: .injected( + model: model, + declaredCapabilities: capabilities + ) ) - ) - } -#endif + } + #endif init( defaultInstructions: String? = nil, diff --git a/Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift b/Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift index 8f8ac0e..9a46773 100644 --- a/Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift +++ b/Sources/AriaApple/Providers/FoundationModelsSessionFactory.swift @@ -3,24 +3,17 @@ import FoundationModels @available(iOS 26.0, macOS 26.0, *) - struct FoundationModelsSessionRequirements: OptionSet, Sendable, Equatable { - let rawValue: UInt8 - + struct FoundationModelsSessionRequirements: OptionSet, Equatable { static let vision = Self(rawValue: 1 << 0) static let guidedGeneration = Self(rawValue: 1 << 1) static let toolCalling = Self(rawValue: 1 << 2) + + let rawValue: UInt8 } @available(iOS 26.0, macOS 26.0, *) - struct FoundationModelsSessionFactory: Sendable { - typealias Validator = @Sendable ( - FoundationModelsSessionRequirements - ) throws -> Void - typealias Builder = @Sendable ( - [any FoundationModels.Tool], - Transcript, - FoundationModelsSessionRequirements - ) throws -> LanguageModelSession + struct FoundationModelsSessionFactory { + // MARK: Lifecycle init( validate: @escaping Validator, @@ -30,6 +23,17 @@ self.build = build } + // MARK: Internal + + typealias Validator = @Sendable ( + FoundationModelsSessionRequirements + ) throws -> Void + typealias Builder = @Sendable ( + [any FoundationModels.Tool], + Transcript, + FoundationModelsSessionRequirements + ) throws -> LanguageModelSession + static let systemDefault = Self( validate: { _ in try FoundationModelsProvider.checkAvailability() @@ -48,83 +52,85 @@ return try self.build(tools, transcript, requirements) } + // MARK: Private + private let validate: Validator private let build: Builder } -#if compiler(>=6.4) - @available(iOS 27.0, macOS 27.0, visionOS 27.0, watchOS 27.0, *) - @available(tvOS, unavailable) - extension FoundationModelsSessionFactory { - private struct CapabilityCheck { - let requirement: FoundationModelsSessionRequirements - let capability: LanguageModelCapabilities.Capability - let name: String - } - - static func injected( - model: Model, - declaredCapabilities: ProviderCapabilities - ) -> Self { - let availableCapabilities = model.capabilities - return Self( - validate: { requested in - try Self.validate( - declared: declaredCapabilities, - available: availableCapabilities, - requested: requested - ) - }, - build: { tools, transcript, _ in - LanguageModelSession( - model: model, - tools: tools, - transcript: transcript - ) - } - ) - } - - static func validate( - declared: ProviderCapabilities, - available: LanguageModelCapabilities, - requested: FoundationModelsSessionRequirements - ) throws { - var required = requested - if declared.supportsVision { - required.insert(.vision) - } - if declared.supportsStructuredOutput { - required.insert(.guidedGeneration) - } - if declared.supportsToolUse { - required.insert(.toolCalling) + #if compiler(>=6.4) + @available(iOS 27.0, macOS 27.0, visionOS 27.0, watchOS 27.0, *) + @available(tvOS, unavailable) + extension FoundationModelsSessionFactory { + private struct CapabilityCheck { + let requirement: FoundationModelsSessionRequirements + let capability: LanguageModelCapabilities.Capability + let name: String } - let checks = [ - CapabilityCheck(requirement: .vision, capability: .vision, name: "vision"), - CapabilityCheck( - requirement: .guidedGeneration, - capability: .guidedGeneration, - name: "structured output" - ), - CapabilityCheck( - requirement: .toolCalling, - capability: .toolCalling, - name: "tool calling" - ), - ] - let missing = checks.compactMap { check in - required.contains(check.requirement) && !available.contains(check.capability) - ? check.name - : nil - } - guard missing.isEmpty else { - throw AgentError.configurationInvalid( - "Language model does not support: \(missing.joined(separator: ", "))" + static func injected( + model: some LanguageModel, + declaredCapabilities: ProviderCapabilities + ) -> Self { + let availableCapabilities = model.capabilities + return Self( + validate: { requested in + try Self.validate( + declared: declaredCapabilities, + available: availableCapabilities, + requested: requested + ) + }, + build: { tools, transcript, _ in + LanguageModelSession( + model: model, + tools: tools, + transcript: transcript + ) + } ) } + + static func validate( + declared: ProviderCapabilities, + available: LanguageModelCapabilities, + requested: FoundationModelsSessionRequirements + ) throws { + var required = requested + if declared.supportsVision { + required.insert(.vision) + } + if declared.supportsStructuredOutput { + required.insert(.guidedGeneration) + } + if declared.supportsToolUse { + required.insert(.toolCalling) + } + + let checks = [ + CapabilityCheck(requirement: .vision, capability: .vision, name: "vision"), + CapabilityCheck( + requirement: .guidedGeneration, + capability: .guidedGeneration, + name: "structured output" + ), + CapabilityCheck( + requirement: .toolCalling, + capability: .toolCalling, + name: "tool calling" + ), + ] + let missing = checks.compactMap { check in + required.contains(check.requirement) && !available.contains(check.capability) + ? check.name + : nil + } + guard missing.isEmpty else { + throw AgentError.configurationInvalid( + "Language model does not support: \(missing.joined(separator: ", "))" + ) + } + } } - } -#endif + #endif #endif