Skip to content

Add Timeline Origin Selector option - #4

Open
michaelwasserman wants to merge 1 commit into
mainfrom
feature-origin-selector
Open

michaelwasserman wants to merge 1 commit into
mainfrom
feature-origin-selector

Conversation

@michaelwasserman

@michaelwasserman michaelwasserman commented Oct 29, 2026 •

Copy link
Copy Markdown
Owner

DO NOT MERGE YET, THIS IS UNVERIFIED AND NEEDS MORE INVESTIGATION

Adds an Options dropdown allowing users to switch the timeline origin for event timestamps between session start (onstart) and audio start (audiostart), dynamically updating transcript markers across the timeline.

AFAICT, using onstart as the origin aligns speechStartTime and speechEndTime much closer to the acoustic spikes than audiostart, due to renderer dispatch lag.

Chromium should potentially also backdate audiostart.timeStamp (related to crbug.com/564526939)

  • The explainer instructs developers to use audiostart.timeStamp as the baseline, Under DOM Event rules, audiostart is created when AudioStarted() runs on the renderer main thread, after buffering and IPC, causing all stream-relative timestamps (speechStartTime) to be misplaced ~150–250ms to the right of actual audio spikes.
  • In crrev.com/c/8409366 Reilly Grant (reillyg@) noted:

    "I wish we carried through the value of base::TimeTicks::Now() from when the audio chunk was submitted to the recognizer so we could put that timestamp in the event rather than needing to derive it."*

  • Chrome should either
    • Provide the actual capture start timestamp of the first audio buffer on audiostart (e.g. event.captureStartTime), or
    • Expose a sessionAudioTimeOrigin attribute on SpeechRecognition in DOMHighResTimeStamp coordinates.

Perhaps the W3C Web Speech API Spec should clarify session start (onstart) and audio start (audiostart) relation to document time, or indicate that those event timestamps should be carried over from their backend initiation?

  • Spec PR #192 standardized speechStartTime and speechEndTime (in seconds) along the session audio timeline (t = 0.0s at the first audio frame consumed).
  • The spec does not specify or redefine audiostart.timeStamp. While the original explainer suggested adding offsets to audiostart.timeStamp to calculate absolute times, neither the explainer nor the spec accounts for the ~150–250ms capture & IPC pipeline latency.
  • The might need to define a standard mechanism to convert from t = 0.0s on the session audio timeline into document time (performance.timeOrigin / performance.now()), or align it with a MediaStreamTrack. e.g. maybe offering an explicit way to query the document-relative time origin of the session audio timeline (e.g. SpeechRecognition.sessionAudioTimeOrigin or clarifying audiostart semantics).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant