Follow-up to the Local Studio review on #186. Today there is no way to answer "did that change make it faster or slower?" for any mobile↔host path — every latency claim in recent PRs is a stopwatch observation on one machine.
The hooks to fix that mostly exist already and are unused.
What we can measure today but don't
_alleycat_seq is a free ordering and gap oracle. Every frame the daemon sends carries a monotonic sequence number, and Connect returns SessionInfo { attached, current_seq, floor_seq }. That is enough to compute, per reconnect: whether we resumed or drifted, how many frames were replayed, and how far the client had fallen behind. None of it is recorded.
kittylitter probe already reproduces the phone exactly, including --resume-from / --repeat-resume-from. tools/scripts/local-studio-e2e-proof.sh (added in #188) drives it for correctness; it emits no timings.
Local Studio's gateway carries per-section freshness. Freshness { observed_at, age_ms, max_age_ms, stale } and a requestId on every response. Unused, because the gateway itself is unused — but the same shape is the right target if the controller plane is ever revived.
refresh_models now fans out concurrently with a 20s per-runtime timeout (MODEL_LIST_RUNTIME_TIMEOUT) and records nothing about which runtime was slow. The timeout was presumably chosen from observation, not data.
Proposal
1. Emit structured timing spans from shared Rust. The crate already uses tracing. Add spans with a stable field set on the four paths that dominate perceived latency:
connect — dial → Connect response → first frame, plus attached and current_seq - resume_from.model_list — per runtime kind, so a slow Local Studio catalog is attributable rather than folded into one number.thread_start → first turn/start accepted. This is the "new session feels slow" metric that 6666098 and 1a2c5c1 were chasing without instrumentation.turn — turn/start → first assistant delta → turn/completed. Time-to-first-token matters far more than total here.
Fields, not messages, so they can be aggregated. These already surface in the simulator via stdout capture (see the daemon-debugging notes) with no extra plumbing.
2. Add a --timings flag to kittylitter probe. It already knows when it sent each request and when each frame arrived. Printing a small JSON summary (connect ms, response ms, time-to-first-notification, frame count, terminal method) turns the existing E2E harness into a benchmark for free.
3. Extend local-studio-e2e-proof.sh to record those timings per scenario and write them next to the existing evidence. Two immediate uses: a regression check against a committed baseline, and an actual data-driven answer for the MODEL_LIST_RUNTIME_TIMEOUT value.
4. Track three numbers over time, not a dashboard. Cold-launch reconnect to first usable thread list; new-session tap to first rendered user message; turn send to first assistant token. Everything else is diagnostic detail for when one of those three regresses.
Why this ordering
1–2 are small and unlock 3. 4 is the only part that needs a decision about where numbers get stored. None of it requires the controller/gateway plane, so it is unaffected by how the architecture question in #186 is resolved.
Non-goal
No telemetry leaving the device. Per the repo's existing stance, mobile logs stay local — this is about making local runs measurable and CI/manual runs comparable, not about collecting from users.
Follow-up to the Local Studio review on #186. Today there is no way to answer "did that change make it faster or slower?" for any mobile↔host path — every latency claim in recent PRs is a stopwatch observation on one machine.
The hooks to fix that mostly exist already and are unused.
What we can measure today but don't
_alleycat_seqis a free ordering and gap oracle. Every frame the daemon sends carries a monotonic sequence number, andConnectreturnsSessionInfo { attached, current_seq, floor_seq }. That is enough to compute, per reconnect: whether we resumed or drifted, how many frames were replayed, and how far the client had fallen behind. None of it is recorded.kittylitter probealready reproduces the phone exactly, including--resume-from/--repeat-resume-from.tools/scripts/local-studio-e2e-proof.sh(added in #188) drives it for correctness; it emits no timings.Local Studio's gateway carries per-section freshness.
Freshness { observed_at, age_ms, max_age_ms, stale }and arequestIdon every response. Unused, because the gateway itself is unused — but the same shape is the right target if the controller plane is ever revived.refresh_modelsnow fans out concurrently with a 20s per-runtime timeout (MODEL_LIST_RUNTIME_TIMEOUT) and records nothing about which runtime was slow. The timeout was presumably chosen from observation, not data.Proposal
1. Emit structured timing spans from shared Rust. The crate already uses
tracing. Add spans with a stable field set on the four paths that dominate perceived latency:connect— dial →Connectresponse → first frame, plusattachedandcurrent_seq - resume_from.model_list— per runtime kind, so a slow Local Studio catalog is attributable rather than folded into one number.thread_start→ firstturn/startaccepted. This is the "new session feels slow" metric that6666098and1a2c5c1were chasing without instrumentation.turn—turn/start→ first assistant delta →turn/completed. Time-to-first-token matters far more than total here.Fields, not messages, so they can be aggregated. These already surface in the simulator via stdout capture (see the daemon-debugging notes) with no extra plumbing.
2. Add a
--timingsflag tokittylitter probe. It already knows when it sent each request and when each frame arrived. Printing a small JSON summary (connect ms, response ms, time-to-first-notification, frame count, terminal method) turns the existing E2E harness into a benchmark for free.3. Extend
local-studio-e2e-proof.shto record those timings per scenario and write them next to the existing evidence. Two immediate uses: a regression check against a committed baseline, and an actual data-driven answer for theMODEL_LIST_RUNTIME_TIMEOUTvalue.4. Track three numbers over time, not a dashboard. Cold-launch reconnect to first usable thread list; new-session tap to first rendered user message; turn send to first assistant token. Everything else is diagnostic detail for when one of those three regresses.
Why this ordering
1–2 are small and unlock 3. 4 is the only part that needs a decision about where numbers get stored. None of it requires the controller/gateway plane, so it is unaffected by how the architecture question in #186 is resolved.
Non-goal
No telemetry leaving the device. Per the repo's existing stance, mobile logs stay local — this is about making local runs measurable and CI/manual runs comparable, not about collecting from users.