Music Machinery

Maestro is a hack built at  Music Hack Day Toronto.  It allows you to ‘conduct’ your music by waving your iPhone around like a conductor waves their baton.  You can speed up and slow down your music at will.  Here’s the demo:

 

[youtube http://www.youtube.com/watch?v=Q8NYaKTJZR0#t=0m46s]

The hack was created by Wen-Hao Lue and Peter Sobot.

Brian McFee brought the heavy lifting to the Toronto Music Hack Day. His goal  – to see what it would sound like if Billie Holiday sang for Black Sabbath or if Kenny G and The Jesus Lizard formed a supergroup.  To do this he built the FRANKENMASHER 2000.   This program uses some heavy math to separate the vocals from one song and the instrumentation of another to combine them into what he calls a ‘horrible abomination of sonic torture’.  Here are some examples:

Brian has a blog post that describes a bit of the math involved and has more examples.  If you are into MIR, Brian’s the guy to keep an eye on. He’s doing interesting things.

This cool hack created at the Toronto Music Hack Day by Devin Sevilla from Rdio looks at what you are currently playing in Rdio (using the Rdio ‘now playing’ API) and then finds all the remixes of that song that have been posted to to Soundcloud.  It is a fantastic idea and works great. I really had no idea how many remixes are posted to to Soundcloud.  For instance, check out this orchestral version of Skrillex’s First of the Year.

A really cool hack.  Check out Remixes on Soundcloud

I didn’t make it to the Toronto Music Hack Day, but I’ve heard great things about the event. One hack, built by Ned Lovely is getting lots of attention. It is called Music Radiator.  It gives you spot-on genre playlists with a very slick user interface.  Pick one of hundreds of genres and just let the music flow. If you give a song a ‘thumbs up’ it will be added to your Rdio collection, give it a ‘thumbs down’ and you’ll never hear it again (well, at least never again on The Music Radiator).  Ned has a great sense of design, and the music and the music flows well. I may use this app as my primary way to listen to music when on the web.  Check it out.

The Way Out West Hack Battle 2, a 2 day music hackathon conducted in conjunction with the Way Out West music festival in Gothenburg Sweden shows that when in comes to music hacking, there’s no place like Sweden.  In 24 hours of hacking the 125+ hackers put together a total of 32 hacks.  These hacks were some of the most clever and innovative set of hacks put together at any hackathon.

The hackathon began with short API presentations by companies such as Spotify, Facebook, Sendgrid and The Echo Nest.   When I presented the Echo Nest API, I asked how many folks had  heard of the Echo Nest. There were quite a few who didn’t know anything about the Echo Nest. But at the end of the hackathon I was quite pleased to see that half of the demos used the Echo Nest’s API.  Some of my favorite Echo Nest hacks at the event were:

Music MAP – Music MAP is a Spotify app that lets you explore the world of music using a 3D globe. You can narrow your search down to country or region and filter the searches by genre and year. Looking for blues artists from the 1950s from Mississippi? The Music MAP can help you out.   Music MAP was one of the winners of the Echo Nest Prize.

Lego Rock Raiders  – This team built a Guitar Hero style game that you play by smashing on fruit (yes, real, juicy, sticky, messy fruit) that is connected to your computer (via makeymakey of course).  This team used the Echo Nest API to find the notes and beats used to drive the game. Quite fun and creative.  This hack also won an Echo Nest prize.

Other notable Echo Nest hacks were:

• intro ninja – similar artists and songs music guessing game – with a really good design
• beatiful –   an infinite runner game for IOS – it uses Echo Nest analysis data of a song to generate the level.
• Time Traveling music machine – a music discovery tool that helps you find artist influences. You  enter an artist and a year and go back in time to find that artist’s influences from that time.
• meshuggafy – a nifty web-based remixing tool
• SVT Playlist 2.0 – Uses the Echo Nest fingerprinter to identify music in TV shows shows. With this tool you can instantly find when your favorite music was played in your favorite TV show.
• My – Uses the Echo Nest data to help recommend music based upon your physical activity as determined by a fitbit.
• hackatune – A very nice looking and simple to use festival app that uses the Echo Nest to find the hotttest songs by festival artists.
• Autopilot – Get a customised festival schedule based on your social profile on Facebook, Spotify, Last-fm and Songkick. Then you can simply put on the AutoPilot and enjoy the ride!
• The Saddest Stylophonist – My hack – makes any song sadder and more pathetic by automatically adding a Stylophone accompaniment to the track.
• Spotify Screensaver 2 – Party Mode  – a Chrome extension that adds a “Party Mode” that shows cover art, song name and Fan Art.

I had a great time at the event. I got to meet some really interesting folks.  I sat next to Jens Nockert for much of the weekend.  Jens is famous for writing audio decoders in Javascript. I hope he works for the Echo Nest some day.  Jens was kind enough to let me tether through his iPhone so I could submit my hack when the local wifi connection to the Internet was getting overloaded.

I also got to chat with Per-Olov Jernberg.  He brought an Oculus Rift to the hackathon and built a really nifty 3D music explorer with it.  I got my first try with the 3D virtual reality goggles and I was really impressed with it. Quite immersive.  Per-Olov is quite the hacker – he’s been adding  Oculus Rift support to three.js.  I was excited to hear that he’s joined the Spotify team and may be applying some of his considerable hacking skills to improving their API.

I also met a number of students from Chalmers University – they were a pretty innovative and tech savvy bunch of folks.

#wowhack2 was a really great hackathon. Thanks to Paul Sonkamble and Andrew Mager for organizing such a cool event. Now for a day or two of exploring the Gothenburg city and countryside. It really is a beautiful part of the world.

I’m putting the finishing touches on my Music Hack Day Denver hack.  My hack is called The MobiusTube. It’s the Infinite Jukebox for Youtube.   With The MobiusTube you can watch a never-ending and ever-changing version of your favorite music video.

This is an incremental hack. Standing directly on the shoulders of the infrastructure that I built for the Infinite Jukebox, The Bonhamizer and Girl Talk in the Box.

The hack takes advantage of the new HTM5 Video capabilities – in particular, the ability to quickly segue to any position in a video (as long as that position is already in the play buffer).  With that capability, it was pretty easy to repurpose the Infinite Jukebox to show synchronized video while playing an infinite song.   I flattened the visualization used in the Infinite Jukebox into a strip of connected tiles that shows the relationships of all the beats.

There were a few interesting technical challenges involved in the hack. The first being how to extract audio from Youtube videos. Luckily there’s a very active open source python library  called youtube-dl that provides a way to download most videos from youtube. The trusty swiss army knife of audio – ffmpeg – can then be used to extract the audio track so that it can be analyzed by The Echo Nest.

A significant challenge was the jitter in the HTML5 video slewing. Even under the best of circumstances, jumping to random points in a video would yield very unpleasant sounding audio artifacts. Skips and jitters made for a very unmusical experience.  To deal with this, I decided to simply mute the video soundtrack and play the audio separately using the Web Audio API. The Web Audio API does a fine job of giving jitter and artifact free audio no matter how often and how far we jump in the audio track. Even if the video falls behind (which sometimes happens due to buffering), there’s a seamless audio experience.

I wanted to be able to keep track of how long any particular video was played, so I could keep a leader board of sorts.  In previous hacks,  I would have just kept this data in a flat file (PBLML format), but I decided to try to get into this decade – so I gave redis a try. I’m glad I did, it seems perfect for this kind of task.

I borrowed some of the interactive control logic from Girl Talk In a Box, so you can interact with The MobiusTube letting you dynamically rework a music video.

I still have a few things to clean up, but I hope some folks can give it a try and give me some feedback. It is online now.  Check it out here:  The MobiusTube

What’s nearly as anticipated as the re-release of Arrested Development? Why its the release of Infinite Jukebox for iPhone.  I’ve finally started coding it and have something to show. Here it is running in the simulator:

[youtube http://www.youtube.com/watch?v=C11r-qYWM2E]

And on my device:

[youtube http://www.youtube.com/watch?v=nW8FGR_ClpE&feature=youtube_gdata_player]

There still a long way to go, but progress is being made. No tricks here, Michael. Illusions.

Update: Here’s Monday’s coding update .. now with fancy animations (as suggested by Ajay)

[youtube http://youtu.be/oztFUho0Gu0]