Music Machinery

I’m putting the finishing touches on my Music Hack Day Denver hack.  My hack is called The MobiusTube. It’s the Infinite Jukebox for Youtube.   With The MobiusTube you can watch a never-ending and ever-changing version of your favorite music video.

This is an incremental hack. Standing directly on the shoulders of the infrastructure that I built for the Infinite Jukebox, The Bonhamizer and Girl Talk in the Box.

The hack takes advantage of the new HTM5 Video capabilities – in particular, the ability to quickly segue to any position in a video (as long as that position is already in the play buffer).  With that capability, it was pretty easy to repurpose the Infinite Jukebox to show synchronized video while playing an infinite song.   I flattened the visualization used in the Infinite Jukebox into a strip of connected tiles that shows the relationships of all the beats.

There were a few interesting technical challenges involved in the hack. The first being how to extract audio from Youtube videos. Luckily there’s a very active open source python library  called youtube-dl that provides a way to download most videos from youtube. The trusty swiss army knife of audio – ffmpeg – can then be used to extract the audio track so that it can be analyzed by The Echo Nest.

A significant challenge was the jitter in the HTML5 video slewing. Even under the best of circumstances, jumping to random points in a video would yield very unpleasant sounding audio artifacts. Skips and jitters made for a very unmusical experience.  To deal with this, I decided to simply mute the video soundtrack and play the audio separately using the Web Audio API. The Web Audio API does a fine job of giving jitter and artifact free audio no matter how often and how far we jump in the audio track. Even if the video falls behind (which sometimes happens due to buffering), there’s a seamless audio experience.

I wanted to be able to keep track of how long any particular video was played, so I could keep a leader board of sorts.  In previous hacks,  I would have just kept this data in a flat file (PBLML format), but I decided to try to get into this decade – so I gave redis a try. I’m glad I did, it seems perfect for this kind of task.

I borrowed some of the interactive control logic from Girl Talk In a Box, so you can interact with The MobiusTube letting you dynamically rework a music video.

I still have a few things to clean up, but I hope some folks can give it a try and give me some feedback. It is online now.  Check it out here:  The MobiusTube