Skip to content

Windows: Mochi dances to the song's own beat (opt-in) - #412

Open
shakibbinkabir wants to merge 4 commits into
Louis-CFM:mainfrom
shakibbinkabir:spotify-beat
Open

shakibbinkabir wants to merge 4 commits into
Louis-CFM:mainfrom
shakibbinkabir:spotify-beat

Conversation

@shakibbinkabir

Copy link
Copy Markdown
Contributor

The Mac's Mochi bounces at 112 beats a minute whatever plays. On Windows the app can hear what the computer plays, so it can find the song's tempo and where its beats fall, and he lands on them. Off by default, one switch in Settings.

The compact island: the Mac's 112 while no beat is known, then a song at 92 and a song at 150

Recorded from the running app. So that three tempi sit side by side, the page was handed the three states by hand, in the shape the listener reports them; the live reading is further down.

Builds on #374 (the Spotify pill on Windows), which is not merged yet: the first two commits here are that PR's, unchanged. The two that are new are 595d993 and d142479. Once #374 is in, this is those two.

What the user sees

Nothing, until they ask: Settings → Integrations → Spotify → Dance to the song's beat. The row sits under "Announce new songs", built from the same row and hint as its neighbours, and is offered only where the song can be heard (Windows).

Before (as in #374) After
before after

With it on, a song starts on the Mac's 112, and about ten seconds in the bounce takes the song's tempo and lands on its beats. A pause or a seek keeps the tempo; where the beats fall is found again in a few seconds. Another song starts from scratch. Nothing else on screen changes: no new view, no new element in the island or the card.

How it works

Step Where
What the default output plays is read through a WASAPI loopback, a tenth of a second at a time beat.rs, tap
The sound is reduced, as it comes, to "how much just started": a hundred numbers a second. The sound itself is never held taktus.rs, Ear
A follower reads those until two readings agree, settles on sixteen seconds, then rests and only looks again now and then (10 s, 20 s, 40 s, 60 s apart). The tap is closed while it rests taktus.rs, Follower
The listener, the thread that already sleeps on Windows' media session in #374, tells the page a tempo and the time of one beat spotify.rs, Ears, tell_beat
The dance counts from that beat at that tempo. Given none, it is the 112 it was, bit for bit mochi/engine.ts, DanceClock

He dances between 80 and 160: a tempo read outside that is danced at half or double (dance_tempo).

Only while a Mochi is seen dancing. The page knows when one is (the compact island, Spotify's card in front, or Mochi on the desktop) and tells Rust when that changes (spotify_dancing). With the island hidden and nobody on the desktop, or the app paused, or the choice off, the tap is closed, no timer runs, and the thread sleeps on its channel exactly as in #374.

What it costs

Measured on the installed build (Windows 11, i7-1260P), on the listener's own thread (coucou-spotify, by thread times, 15.6 ms resolution):

Processor used by the thread
Two minutes of playback across a song change, the busiest stretch (a new song is listened to for its first sixteen seconds) 125 ms, 0.10% of one core
One minute paused, the choice still on 0 ms (what wakes it then is Spotify updating its timeline, as in #374)

The engine by itself (its bench, same machine): 0.26 ms to hear a second of sound, 0.27 ms for a reading of sixteen seconds, and about a quarter of a song listened to. It holds about 9 KB while it listens.

Privacy

  • Off by default. The row says what turning it on does: "Mochi listens to what the computer plays, a few seconds at a time, to find the song's tempo. Nothing is recorded or sent."
  • Nothing of the sound is stored, written or sent. Each tenth of a second is turned into onset values and dropped; the follower keeps at most sixteen seconds of those numbers, and none while it rests.
  • No network: the engine is std only and opens nothing, and nothing is added to the app.
  • It is the whole output that is heard, not Spotify alone: Windows' plain loopback does not separate apps. That is why it is a choice, and why the row says "what the computer plays".

The engine in the tree

The listening is taktus, a tempo and beat engine. I am its author; I published it as 0.1.0 while writing this, so judge it as you would any new code. It is here as one file, windows/src-tauri/src/taktus.rs, so that the app takes on no dependency:

  • One file, std only, no unsafe, no allocation while listening. 571 lines of code.
  • Exactly the published source. Left out: the page of documentation at its top (its examples would run as this crate's doctests) and its tests, which live in its repository. The header says so. To check that nothing else differs (prints nothing):
    curl -sL https://raw.githubusercontent.com/shakibbinkabir/taktus/v0.1.0/src/lib.rs \
      | sed -n '/^use std::sync::LazyLock;/,/^\/\/ ── Tests/p' | sed '$d' \
      | diff -B --strip-trailing-cr - <(sed -n '/^use std::sync::LazyLock;/,$p' windows/src-tauri/src/taktus.rs)
  • Licence. Apache-2.0, inside an MIT repository: the two go together, and what Apache asks is that its notice stays with the file. It is in the header, with the credit line. Nothing else is asked.
  • How good it is, measured on the five sets the field uses (Ballroom, GTZAN, Giantsteps, Hainsworth, SMC), with the tempo counted right within 4% at the beat, or at twice, three times, half or a third of it (the field's "Accuracy2"): 99.0, 95.1, 95.9, 88.7 and 48.4% (the last is a set chosen for being hard: rubato, no drums). When the follower says it has a beat, it is right 95 to 99 times in 100 on the first four. The README has every figure, ten other libraries on the same files, and what the numbers do not show; taktus.thisissbk.com reads a song dropped on the page.

If you would rather have it as a crate (taktus = "0.1", which has no dependencies of its own), or would rather not have the file at all, say so and I will reshape this.

Against the repo's rules

  • No third-party dependency: none added. Cargo.toml gains one feature of the windows crate already in use (Win32_Media_Audio_Endpoints).
  • No telemetry, no network call: none.
  • 0% CPU when the island is hidden: the listening is tied to a Mochi being seen dancing, not to music playing (measured above).
  • Never restyle what ships: one row and one hint added to Settings, with the existing classes (screenshots above). No other view is touched.
  • Pill IDs, Keychain, bundle identifier: untouched. settings.json gains danceToBeat, false when absent.
  • Swift / macOS / iPhone: untouched. Linux compiles and tests the same code and keeps the 112: it has no tap yet.
  • Not a release: no CHANGELOG section, no version change. windows/README.md gains a paragraph under what differs from the Mac.

Tested

  • npx tsc --noEmit, npm test (379 pass), cargo test --workspace --locked (253 pass), on Windows 11.
  • New tests, page side (tests/spotify.test.mjs): the bounce keeps any tempo it is given and is the Mac's when given none; the song's beat is danced to only when asked for, and no longer the moment it is not; Rust is told that somebody dances only when asked, once per change, through the island hiding, Mochi on the desktop, a pause, the app paused, and another pill's card in front.
  • New tests, Rust side: the tempo he dances at for a beat read at any level (beat.rs); a drum fed as the app feeds it is followed to its tempo, with a beat within 40 ms of a drum and rests in between (beat.rs); a settings file from before reads with the choice off (settings.rs); the player's state carries no tempo until one is heard (spotify.rs).
  • Live, on the installed build with a song playing in Spotify: the app was started mid-song, had no beat 8 seconds after launch and had one by 21 seconds, at 135.1 beats a minute; the next three looks said 134.86, 134.78 and 134.74, then it rested.
  • The tap opens on this machine and hears nothing when nothing plays (cargo test -p coucou the_tap_hears_what_plays -- --ignored --nocapture).

Not tested, and known limits

  • Bluetooth output. The loopback hears the sound before the device's own delay. On speakers and wired headphones that is a few tens of milliseconds; over Bluetooth he would land early by the device's latency. Not measured.
  • Another app playing sound while Spotify plays blurs the reading (see Privacy). Windows can tap one process instead; a note in beat.rs says what it would take.
  • Windows 10, a machine with no output device, and an output in exclusive mode: the tap fails to open and the song is left at 112 by construction, but none of the three was run.
  • Whether a tempo is the beat or its double is partly convention, and the engine reads some songs at twice or half what a listener would tap. The dance hides most of that by folding into 80 to 160.
  • Linux and macOS: nothing to test beyond the build and the tests, which are the same code.

Commits

  1. 595d993 taktus in the tree. The file, and nothing that uses it.
  2. d142479 Mochi dances to the song's own beat. Everything else. Needs the commit before it, and Windows: the Spotify pill through the system's media session, and an opt-in new-song announcement #374.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NBruuRH1qTWSvSpxoV6mKT

shakibbinkabir and others added 4 commits October 9, 2026 05:58
The Linux pill reads Spotify over MPRIS. Windows has System Media Transport
Controls: every player reports its track to the system and takes play, pause,
next… through it. The pill now reads Spotify's session there, behind the same
five functions the Linux client implements, so the page is the Linux one.

- No polling: one thread sleeps on a channel while the pill is declared, and
  Windows wakes it when a session comes or goes or Spotify's changes. Spotify
  updates its timeline as it plays; only a jump counts as news, so a playing
  track sends the island nothing.
- Spotify only: the session is picked by its app id (the Store build and the
  one from spotify.com). A browser playing a video is not shown.
- The cover comes with the session, so nothing is fetched.
- The session has no volume: the slider moves Spotify's level in the Windows
  volume mixer.
- Shuffle and repeat are left out of the card on Windows. Spotify answers that
  it changed them and changes nothing, so `modes` in the state tells the card.
- The session carries no Spotify URI: tracks get a `spotify:local:` id, so
  there is no page to open and an ad is not recognised as one.
- "Open Spotify" follows a `spotify:` link, which starts it or brings it
  forward; without Spotify, its download page.

The pill is now offered on both systems (`support: "yes"`), and the
Linux-only support kind this pill had introduced goes away with its last user.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When Spotify moves on to another song, the island opens on Spotify's card for
three seconds and folds back, and the pill that was in front gets its place
again. A light in Mochi's colour breathes behind him, the cover glows and a
sheen crosses it, so it reads as news and not as an island that opened by
accident. No sound: music is playing.

The idea and its look are Edurique's, from Louis-CFM#83. It is rebuilt here on the card
and the state machine as they are today, and nothing of the card changes
outside those three seconds.

- Off by default (Settings → Integrations → Spotify → Announce new songs).
  The island never opens on its own for news that asks nothing, and the Mac
  does not do this, so it is something to turn on.
- A new song is one that starts after another: not the first heard since
  Spotify started, not the same one going on after a pause or a seek, never
  an ad. Windows names a track before it says it plays; it is announced once
  it does.
- Never over an island that is open, under the mouse, pinned or waiting for
  an answer. The mouse coming, a key or an alert makes it an open island like
  any other: the auto-close delay applies again, the usual sounds come back,
  and the front stays where the user sees it.
- The state machine gains `glance(seconds)`: open, and fold after its own
  delay instead of the auto-close preference. It is the hover-open's short
  countdown with another length, and ends the same way (`userInteracted`).
- The light takes the colour of the pill's Mochi, so a colour picked in
  Settings → Active pills is the light's too.
- Songs skipped through carry one announcement on, and the front still goes
  back to the pill that had it before the first.

Co-authored-by: Edurique <225876024+Edurique@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
taktus reads the tempo of music and where its beats fall from the sound
itself (https://github.com/shakibbinkabir/taktus, Apache-2.0, by Shakib Bin
Kabir). This is its one source file as published in 0.1.0, put in the tree
whole so that the app takes on no dependency: no crate, nothing to fetch,
std only, no unsafe.

Left out of the file: the page of documentation at its top, with the
examples it runs, and its tests. Both live in its repository, with what it
was measured on. Nothing else is changed, and its header says so.

Nothing uses it yet: the next commit does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NBruuRH1qTWSvSpxoV6mKT
The Mac's Mochi bounces at 112 beats a minute whatever plays. On Windows the
app can hear what the computer plays (a loopback of the default output), so
it can find the song's tempo and where its beats fall, and he lands on them.

- Off by default: Settings → Integrations → Spotify → Dance to the song's
  beat. Listening to what the computer plays is for the user to ask for, and
  the row says what it does. It is shown only where the song can be heard.
- Nothing of the sound is kept or sent. As it comes it is reduced to "how
  much just started", a hundred numbers a second, and those are all that is
  looked at (taktus.rs, the commit before).
- Only while a Mochi is seen dancing: in the compact island, on Spotify's
  card, or on the desktop. The page says when (spotify_dancing), and with
  the island hidden and nobody on the desktop the tap is closed and the
  listener sleeps on its channel as before.
- A few seconds at a time. The follower listens until two readings agree
  (about ten seconds), settles on sixteen, then rests and only looks again
  now and then; the tap is closed while it rests. About a quarter of a song
  is heard.
- A pause or a seek keeps the tempo and forgets where the beats fall until
  they are found again; another song starts from scratch. Until a beat is
  known, and on Linux, he keeps the 112.
- He dances between 80 and 160: a tempo read outside that is danced at half
  or double.
- The whole output is heard, not Spotify alone, so another app's sound blurs
  the reading (a note in beat.rs says what tapping one process would take).

The player's state gains tempo and beatAt; the dance takes a tempo and a
beat to count from, and is unchanged when given none.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NBruuRH1qTWSvSpxoV6mKT
PythonTilk added a commit to PythonTilk/coucou that referenced this pull request Oct 9, 2026
…(opt-in)

Taken from shakibbinkabir's branch while it is still open upstream; its first
two commits are Louis-CFM#374, already here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PythonTilk added a commit to PythonTilk/coucou that referenced this pull request Oct 18, 2026
…s on top

From upstream: the Coucou window with Flow dictation, the Notetaker and
« OK Coucou », its Spotify pill, and settings as pages in that window.
Taken over this build's own versions: « OK Coucou » replaces the hotkey
voice (removed just before), upstream's Spotify replaces Louis-CFM#374 and Louis-CFM#412
(the beat dance goes with them), and the settings pages replace the
folding groups. Kept and moved onto the pages: Updates (General), the
chat provider picker (Chat), Phone alerts and Mail (Integrations).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant