Replace Spotify’s audio analysis with one that can be run on any song

[rigdern]

The major constraint with Spotify’s audio analysis web API is that the data is only available for Spotify’s audio files. Implications:

• Songs don’t work with Infinite Beat Saber when:
  • They aren’t on Spotify
  • They are on Spotify but the audio file we have is too different from Spotify’s (e.g. OneHope)
• Even when a song is on Spotify and is nearly identical to our audio file, it still takes 10 minutes or so to align the audio files (i.e. determine shiftTimestampsSeconds in RemixableSongs.cs)

For these reasons, it’d be great to have a replacement for Spotify’s audio analysis that we can use with any audio file we want. It would enable Infinite Beat Saber to work with any song and it’d remove some of the manual effort required to add a song to Infinite Beat Saber.

Resources

These resources may be helpful. I found them in spotify/web-api#209 (comment).

• Amen. A tool for algorithmic remixing.
• Lower level feature extractors:
  • librosa
  • essentia
  • meyda