Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
24bbbe9
feat(studio): Phase 1a foundation - mode gate, op registry, object store
meidad Jun 13, 2026
f917fa8
feat(studio): asset + edit-chain persistence (schema, types, bookkeep…
meidad Jun 13, 2026
dc91dba
feat(studio): engine, consent gate, identity gate
meidad Jun 13, 2026
7bf33ea
feat(studio): providers - local-sharp (deterministic) + gemini-image …
meidad Jun 13, 2026
c5c5b51
feat(studio): conversational MCP tools wired into the agent
meidad Jun 13, 2026
9d9d844
feat(studio): __studio_gc__ cron sentinel + cleanup
meidad Jun 13, 2026
1db2281
feat(studio): declare studio + studio-gc in the feature manifest
meidad Jun 13, 2026
52091fc
feat(studio): MobileApi RPCs - create/get-url/edit(stream)/history
meidad Jun 13, 2026
cec167f
test(studio): extend check:isolation to studio_assets + studio_edits
meidad Jun 13, 2026
e5b7710
fix(studio): address adversarial review (2 P0, 4 P1, 5 P2)
meidad Jun 13, 2026
381d443
chore(studio): keep the feature low-profile in the open-source repo
meidad Jun 13, 2026
9914266
feat(studio): daemon face embedder + on-device identity-report path
meidad Jun 13, 2026
d701e38
fix(studio): roll chain head back on a failed edit + GOOGLE_API_KEY s…
meidad Jun 13, 2026
0b7d909
test(studio): e2e uses a real photo + proves the generative path
meidad Jun 13, 2026
1bdd969
feat(studio): deviceRender op to commit on-device renders as edits
meidad Jun 15, 2026
ff1d87d
test(studio): real gRPC wire check for the deviceRender path
meidad Jun 15, 2026
eae18e5
feat(studio): Phase 3 beauty-ops sidecar — retouch op + provider + la…
meidad Jun 15, 2026
8cf59b8
feat(studio): Phase 3 generative depth ops (muscle/hair/beard/relight…
meidad Jun 15, 2026
e51b339
fix(studio): harden engine + sidecar per adversarial review
meidad Jun 15, 2026
2402796
test(studio): declare Phase 2+3 ops in the feature manifest
meidad Jun 15, 2026
c30f8b9
chore: gitignore locally-installed agent skills (.claude/skills, skil…
meidad Jun 15, 2026
eb48e92
fix(studio): register Studio (+ Loops) RPCs on the Connect server
meidad Jun 15, 2026
df429cd
fix(studio): serve local-fs blobs over HTTP so client uploads work
meidad Jun 15, 2026
dbf8b3e
fix(studio): dev override to enable cloud-AI consent (it had no write…
meidad Jun 15, 2026
304b227
feat(studio): expose cloud-AI consent via GetSettings + UpdatePermission
meidad Jun 15, 2026
0bb6ec6
test(studio): real HTTP e2e for local-fs blob serving (the upload path)
meidad Jun 16, 2026
01f15d5
fix(studio): mask edits resolve + relax generative safety filter (sur…
meidad Jun 16, 2026
4bf0406
feat(studio): StudioListAssets RPC — server-backed editing sessions f…
meidad Jun 16, 2026
358a710
feat(studio): AI-native edit suggestions — vision model proposes tap-…
meidad Jun 16, 2026
b7b7821
feat(studio): richer retouch suggestions + serve the original for com…
meidad Jun 16, 2026
4c2404b
feat(studio): add freckle softening as an optional skin suggestion
meidad Jun 16, 2026
64b9476
feat(studio): suggestions must improve quality (sharpness/detail), ne…
meidad Jun 16, 2026
214ecda
feat(studio): enforce quality on EVERY generative edit, not just auto…
meidad Jun 17, 2026
058179a
feat(studio): region edits via crop-inpaint — the brushed area actual…
meidad Jun 17, 2026
6c913cd
feat(studio): learn the user's photo-editing taste and personalize ed…
meidad Jun 17, 2026
c1f3363
test(studio): exercise + audit the learning feature; fix distill pars…
meidad Jun 17, 2026
7e7a074
chore(sdk): remove dead SDK-adaptation scaffolding
meidad Jun 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,14 @@ DATABASE_URL=postgresql://nomos:nomos@localhost:5432/nomos
# EMBEDDING_MODEL=gemini-embedding-001
# VERTEX_AI_LOCATION=global

# ─── Studio beauty-ops sidecar (optional, hosted Studio) ─────────
# The deterministic retouch/reshape sidecar (nomos-studio-sidecar). When absent,
# retouch falls back to the cloud (Gemini, consent-gated). Either point at an
# already-running instance, OR let the daemon spawn `uv run` from a sibling clone.
# NOMOS_STUDIO_SIDECAR_URL=http://127.0.0.1:8799
# NOMOS_STUDIO_SIDECAR_PATH=../nomos-studio-sidecar
# NOMOS_STUDIO_SIDECAR_PORT=8799

# ─── Permission Mode (optional) ──────────────────────────────────
# Controls how tool usage is handled: default, acceptEdits, plan, dontAsk
# NOMOS_PERMISSION_MODE=acceptEdits
Expand Down
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -63,3 +63,8 @@ skills/webapp-testing/
skills/xlsx/
skills/.anthropic-skills-fetched
.gstack/

# Locally-installed agent skills (e.g. `npx skills add Banuba/ai-skills`) — a tool
# for the assistant, not part of this repo.
.claude/skills/
skills-lock.json
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ See `.env.example` for the full set of optional variables (model, permissions, c
- **`sdk/`** -- Claude Agent SDK wrapper:
- `session.ts` -- wraps `query()`, supports V2 session API with feature detection. `RunSessionParams` accepts `systemPrompt` (full override), `anthropicBaseUrl` (custom API endpoint), and `systemPromptAppend` (append to preset). The `ANTHROPIC_BASE_URL` env var is propagated to child processes via the `env` option.
- `tools.ts` -- in-process MCP server exposing `memory_search` and `user_model_recall` tools
- `cost-tracker.ts` -- per-session and per-model token usage and USD cost tracking with `CostTracker` class, model pricing tiers, formatting utilities, and `getCostTracker()` singleton
- `cost-tracker.ts` -- per-session and per-model token usage and USD cost tracking with `CostTracker` class, model pricing tiers, and `getCostTracker()` singleton
- `token-estimation.ts` -- heuristic-based token counting (`roughTokenCount`, `bytesPerTokenForFileType`, `roughTokenCountForBlock/Content/Messages`, `formatTokenCount`)
- `retry.ts` -- `withRetry<T>()` async retry with exponential backoff + jitter, 429/529 handling, retry-after header parsing, persistent mode for daemon, abort signal support
- `cache-break-detection.ts` -- `PromptCacheTracker` class that detects cache-invalidating changes to system prompt, tool schemas, model, or betas across API calls
Expand Down
101 changes: 101 additions & 0 deletions eval/agent-eval.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1304,6 +1304,106 @@ async function runStyleProfiles(): Promise<void> {
if (!KEEP) await db.deleteFrom("style_profiles").where("user_id", "in", [A, B]).execute();
}

async function runStudioLearn(): Promise<void> {
// Studio learning: drive the REAL capture -> distill -> store path. Four committed
// edits fill recordEditSignal's buffer and trip flushPhotoStyle, which distills them
// (forked Haiku) into an editable photo-style.md vault note + photo_style user_model
// entries -- the exact rows suggestEdits + auto-enhance read back to personalize.
// Asserts both durable effects, the apply-side read, and per-user isolation.
const { recordEditSignal, readPhotoStyle, flushPhotoStyle } =
await import("../src/studio/learn.ts");
const db = getKysely();
const A = "eval-photo-a";
const B = "eval-photo-b";
const clear = async (): Promise<void> => {
await db
.deleteFrom("vault_notes")
.where("user_id", "in", [A, B])
.where("path", "=", "photo-style.md")
.execute();
await db
.deleteFrom("user_model")
.where("user_id", "in", [A, B])
.where("category", "=", "photo_style")
.execute();
};
const styleNote = (): Promise<{ content: string } | undefined> =>
db
.selectFrom("vault_notes")
.select(["content"])
.where("user_id", "=", A)
.where("path", "=", "photo-style.md")
.executeTakeFirst();
const photoPrefCount = async (userId: string): Promise<number> =>
Number(
(
await db
.selectFrom("user_model")
.select((eb) => eb.fn.countAll<number>().as("n"))
.where("user_id", "=", userId)
.where("category", "=", "photo_style")
.executeTakeFirst()
)?.n ?? 0,
);
await clear();

// Capture + read gate on NOMOS_ADAPTIVE_MEMORY (same flag as all other learning).
const priorAdaptive = process.env.NOMOS_ADAPTIVE_MEMORY;
process.env.NOMOS_ADAPTIVE_MEMORY = "true";
try {
if (!hasLLM) {
skip(
"[studio-learn] distills applied edits into a photo-style vault note + user_model",
"no LLM provider configured",
);
return;
}

const edits = [
"warm up the photo and add a soft golden-hour glow",
"smooth the skin but keep the pores and natural texture",
"deepen the contrast and make the colors pop",
"brighten the eyes and gently whiten the teeth",
];
const signals = edits.map((instruction) => ({ op: "editSemantic", instruction }));
// The real path: 4 edits fill the buffer and trip the flush (FLUSH_EVERY).
for (const s of signals) await recordEditSignal(A, s.op, s.instruction);

// recordEditSignal swallows flush errors by design (fire-and-forget); if nothing
// landed, drive the distiller directly so a genuine failure surfaces with a reason.
let note = await styleNote();
if (!note?.content) {
await flushPhotoStyle(A, signals);
note = await styleNote();
}

check(
"[studio-learn] writes an editable photo-style.md vault note",
!!note?.content && note.content.trim().length > 0,
note?.content?.slice(0, 80),
);
check(
"[studio-learn] accumulates photo_style preferences in the user model",
(await photoPrefCount(A)) >= 1,
`count=${await photoPrefCount(A)}`,
);
// Apply side: readPhotoStyle is what the engine (auto-enhance) + suggestEdits inject.
check(
"[studio-learn] readPhotoStyle surfaces the learned style for injection",
(await readPhotoStyle(A)).length > 0,
);
// Per-user scoped: B applied no edits, so it has neither the note nor any prefs.
check(
"[studio-learn] B (no edits) has no photo-style note or prefs (per-user scoped)",
(await readPhotoStyle(B)).length === 0 && (await photoPrefCount(B)) === 0,
);
} finally {
if (priorAdaptive === undefined) delete process.env.NOMOS_ADAPTIVE_MEMORY;
else process.env.NOMOS_ADAPTIVE_MEMORY = priorAdaptive;
if (!KEEP) await clear();
}
}

async function runWikiArticles(): Promise<void> {
// Derived store: wiki_articles. Deterministic write + per-user isolation, then
// the full LLM compile (2 Sonnet passes) pointed at a temp NOMOS_WIKI_DIR so it
Expand Down Expand Up @@ -2979,6 +3079,7 @@ async function runEval(): Promise<void> {
await runRelationshipStats();
await runManagedFiles();
await runStyleProfiles();
await runStudioLearn();
await runGraphMetadata();
await runBacklinks();
await runMetadataColumns();
Expand Down
121 changes: 121 additions & 0 deletions eval/feature-manifest.ts
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,127 @@ export const FEATURES: FeatureSpec[] = [
entry: ["registerDeltaSyncJobs"],
effects: [{ claim: "emits ingest:trigger for delta runs (behavioral)", notExercised: true }],
},
{
id: "studio-gc",
summary:
"Daily Studio object/row cleanup per owner: expire unconfirmed uploads (assets stuck pending past a TTL) and aged intermediate edit results no longer at the chain head, dropping their objects. Originals + the live head output are kept; the DB is the single clock (rows expired before object delete).",
trigger: { kind: "cron", sentinel: "__studio_gc__", schedule: "24h", fanOut: true },
entry: ["runStudioGc", "runStudioGcForUser"],
effects: [
{
claim: "GC marks expired Studio rows (status = 'expired')",
sql: {
query: "SELECT count(*) FROM studio_edits WHERE status = 'expired'",
expect: "nonzero",
},
notExercised: true, // the eval does not age rows past the TTL
},
],
invariants: [
"the original asset object is never deleted by GC",
"a row is marked expired before its object is deleted",
"every GC query is user_id-filtered",
],
},

// ── Studio (hosted-only feature) ──
{
id: "studio",
summary:
"Hosted-only media asset + edit pipeline (gated). Immutable original + a non-destructive op chain: validate op -> consent gate (cloud ops only) -> append (optimistic concurrency: parent must be a done+output edit) + idempotency -> provider (local-sharp deterministic / mediapipe-sidecar deterministic / GCP generative) -> identity gate (face-risk ops) -> persist output + preview. Manual on-device renders (adjust/makeup/reshape/hair/body) commit via the deviceRender op (the client uploads its own pixels, re-encoded server-side). retouch routes to the deterministic sidecar when up, else the generative cloud fallback. Phase-3 depth ops (muscle/hairstyle/beard/relight/expand/sky) are generative. Per-user scoped.",
trigger: { kind: "turn", gate: "studio" },
entry: [
"buildStudioMcpServer",
"buildStudioEngine",
"assertIdentityPreserved",
"ensureStudioSidecar",
"listAssets",
"suggestEdits",
],
effects: [
{
claim: "uploaded originals are recorded as studio_assets rows",
sql: { query: "SELECT count(*) FROM studio_assets", expect: "nonzero" },
notExercised: true,
},
{
claim: "each edit appends a completed studio_edits op row",
sql: {
query: "SELECT count(*) FROM studio_edits WHERE status = 'done'",
expect: "nonzero",
},
notExercised: true,
},
{
claim: "on-device renders commit as deviceRender edits (client-uploaded pixels)",
sql: {
query: "SELECT count(*) FROM studio_edits WHERE op = 'deviceRender' AND status = 'done'",
expect: "nonzero",
},
notExercised: true,
},
{
claim: "one-tap retouch records a done studio_edits row (sidecar or cloud fallback)",
sql: {
query: "SELECT count(*) FROM studio_edits WHERE op = 'retouch' AND status = 'done'",
expect: "nonzero",
},
notExercised: true,
},
{
claim: "Phase-3 generative depth ops record done studio_edits rows",
sql: {
query:
"SELECT count(*) FROM studio_edits WHERE op IN ('muscle','hairstyle','beard','relight','expand','sky') AND status = 'done'",
expect: "nonzero",
},
notExercised: true,
},
{
claim: "op params are stored as a jsonb object, never double-encoded",
noDoubleEncode: { table: "studio_edits", column: "params" },
notExercised: true,
},
],
invariants: [
"the original asset row is never mutated by an edit",
"every studio_assets / studio_edits query is user_id-filtered (zero-trust)",
"every generative (cloud) op is gated by the cloudAI consent toggle",
"every face-touching generative op passes the identity gate (assertIdentityPreserved)",
"a retried edit with a committed idempotency_key returns the existing row, never re-charges",
"an edit only chains onto a parent that is done with an output (no half-built chain)",
"deviceRender requires client bytes and is free + never consent/identity-gated (WYSIWYG)",
"a client-supplied mask must resolve to a studio asset owned by the same user",
],
},
{
id: "studio-learn",
summary:
"Studio learns the user's photo-editing taste from the edits they apply. Each committed editSemantic fires a signal (recordEditSignal); a background pass every few edits distills them (Haiku) into an editable photo-style.md vault note + photo_style user_model entries. It's injected back as personalized recommendations (suggestEdits style block) and a personalized auto-enhance (editSemantic personalize flag -> styleHint in the generative prompt), never overriding an explicit typed edit. Gated by NOMOS_ADAPTIVE_MEMORY; per-user scoped.",
trigger: { kind: "turn", gate: "studio" },
entry: ["recordEditSignal", "flushPhotoStyle", "readPhotoStyle"],
effects: [
{
// Exercised by runStudioLearn: 4 edits -> flushPhotoStyle distills the note.
claim: "learned editing taste is written as an editable photo-style.md vault note",
sql: {
query: "SELECT count(*) FROM vault_notes WHERE path = 'photo-style.md'",
expect: "nonzero",
},
},
{
claim: "structured photo_style preferences accumulate in the user model",
sql: {
query: "SELECT count(*) FROM user_model WHERE category = 'photo_style'",
expect: "nonzero",
},
},
],
invariants: [
"learning is gated by NOMOS_ADAPTIVE_MEMORY and is per-user scoped",
"personalization biases auto-enhance + suggestions, never an explicit typed edit",
],
},

// ── Per-turn (memory-indexer) ──
{
Expand Down
2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@
"@bufbuild/protobuf": "^2.12.0",
"@connectrpc/connect": "^2.1.1",
"@connectrpc/connect-node": "^2.1.1",
"@google/genai": "^2.8.0",
"@googleworkspace/cli": "^0.22.5",
"@grpc/grpc-js": "^1.14.3",
"@grpc/proto-loader": "^0.8.0",
Expand Down Expand Up @@ -99,6 +100,7 @@
"playwright": "^1.50.0",
"postgres": "^3.4.7",
"react": "^19.2.4",
"sharp": "^0.35.1",
"strip-ansi": "^7.2.0",
"ws": "^8.19.0",
"zod": "^4.3.6"
Expand Down
Loading
Loading