Archive for category hacking
I work for Spotify now – so for my Sunday morning programming project I thought I’d write a simple Spotify App that uses The Echo Nest API to create playlists based upon a seed song. I’ve done this before, but the last time was a few years ago and the Spotify Apps API has changed quite a bit since then, so I thought I’d use this as an opportunity to freshen my understanding of the Spotify API as well as to demonstrate how to write a Spotify App that uses The Echo Nest API.
It did take me a bit of time to get the hang of the newer Spotify Apps API. It has changed quite a bit since I last used it and many of the examples that I relied on in the past, like Peter Watt’s excellent Kitchen Sink app, use an older version of the API. The new version has some significant changes including a nifty new module system along with a new approach to managing long-running queries that relies on promises. Once I got the hang of it, I decided that I like the new version – it makes for cleaner code and a much more efficient app since much less data needs to be moved around.
The app is on github – to use it you need to sign up for a developer account on Spotify and follow the rest of the Getting Started instructions (this means if you are not a developer, you’ll probably not be able to use the app).
The Spotify Apps API makes it super easy to be able to create apps that run inside Spotify. Its a very familiar programming environment for anyone who has done web programming. You can use all of your favorite libraries from jQuery to Lo-Dash to create really compelling apps that sit on top of the millions and millions of tracks in the Spotify catalog. However, unlike a web app where anyone can publish their app on the web, to publish a Spotify App you have to submit your app to the Spotify App approval process and only apps that Spotify approves are published. Spotify sets a high bar for what ultimately gets approved – which keeps the quality of the apps high, but also means that hacks and experiments built on the Spotify Apps platform will likely never be approved for release to the general public. It’s a difficult balancing act for Spotify. They’ve built the ultimate music hacking platform with this API, but if they open the doors to every music hack, they will ultimately dilute the listening experience of the user – like so other many App stores that are flooded with garbage apps, if they publish every app and hack then Spotify listeners would be inundated with the musical equivalent of flashlight and fart apps. With the approval process, Spotify essentially says ‘the listener comes first’ which is the right choice. Still, as a music hacker I do wish it was easier to publish rich music apps built on the Spotify platform. Luckily Spotify is committed to building an active and vibrant developer ecosystem so I don’t expect they we will be standing still.
Update 3/24/14: – I’ve added the ability to save these playlists back to Spotify, so you can take the Echo Nest radio playlists with you.
Second update 3/24/14 – note that Spotify’s recent announcement that they are closing app submissions means that you won’t be able to submit apps for publishing anymore, but you should be able to still create your own.
Last week at the SXSW Music Hack Championship hackathon I built The Autocanonizer. An app that tries to turn any song into a canon by playing it against a copy of itself. In this post, I explain how it works.
At the core of The Autocanonizer are three functions – (1) Build simultaneous audio streams for the two voices of the canon (2) Play them back simultaneously, (3) Supply a visualization that gives the listener an idea of what is happening under the hood. Let’s look at each of these 3 functions:
(1A) Build simultaneous audio streams – finding similar sounding beats
The goal of the Autocanonizer is to fold a song in on itself in such a way that the result still sounds musical. To do this, we use The Echo Nest analyzer and the jremix library to do much of the heavy lifting. First we use the analyzer to break the song down into beats. Each beat is associated with a timestamp, a duration, a confidence and a set of overlapping audio segments. An audio segment contains a detailed description of a single audio event in the song. It includes harmonic data (i.e. the pitch content), timbral data (the texture of the sound) and a loudness profile. Using this info we can create a Beat Distance Function (BDF) that will return a value that represents the relative distance between any two beats in the audio space. Beats that are close together in this space sound very similar, beats that are far apart sound very different. The BDF works by calculating the average distance between overlapping segments of the two beats where the distance between any two segments is a weighted combination of the euclidean distance between the pitch, timbral, loudness, duration and confidence vectors. The weights control which part of the sound takes more precedence in determining beat distance. For instance we can give more weight to the harmonic content of a beat, or the timbral quality of the beat. There’s no hard science for selecting the weights, I just picked some weights to start with and tweaked them a few times based on how well it worked. I started with the same weights that I used when creating the Infinite Jukebox (which also relies on beat similarity), but ultimately gave more weight to the harmonic component since good harmony is so important to The Autocanonizer.
(1B) Build simultaneous audio streams - building the canon
The next challenge, and perhaps biggest challenge of the whole app, is to build the canon – that is - given the Beat Distance Function, create two audio streams, one beat at a time, that sound good when played simultaneously. The first stream is easy, we’ll just play the beats in normal beat order. It’s the second stream, the canon stream that we have to worry about. The challenge: put the beats in the canon stream in an order such that (1) the beats are in a different order than the main stream, and (2) they sound good when played with the main stream.
The first thing we can try is to make each beat in the canon stream be the most similar sounding beat to the corresponding beat in the main stream. If we do that we end up with something that looks like this:
It’s a rat’s nest of connections, very little structure is evident. You can listen to what it sounds like by clicking here: Experimental Rat’s Nest version of Someone Like You (autocanonized). It’s worth a listen to get a sense of where we start from. So why does this bounce all over the place like this? There are lots of reasons: First, there’s lots of repetition in music – so if I’m in the first chorus, the most similar beat may be in the second or third chorus – both may sound very similar and it is practically a roll of the dice which one will win leading to much bouncing between the two choruses. Second – since we have to find a similar beat for every beat, even beats that have no near neighbors have to be forced into the graph which turns it into spaghetti. Finally, the underlying beat distance function relies on weights that are hard to generalize for all songs leading to more noise. The bottom line is that this simple approach leads to a chaotic and mostly non-musical canon with head-jarring transitions on the canon channel. We need to do better.
There are glimmers of musicality in this version though. Every once in a while, the canon channel will remaining on a single sequential set of beats for a little while. When this happens, it sounds much more musical. If we can make this happen more often, then we may end up with a better sounding canon. The challenge then is to find a way to identify long consecutive strands of beats that fit well with the main stream. One approach is to break down the main stream into a set of musically coherent phrases and align each of those phrases with a similar sounding coherent phrase. This will help us avoid the many head-jarring transitions that occur in the previous rat’s nest example. But how do we break a song down into coherent phrases? Luckily, it is already done for us. The Echo Nest analysis includes a breakdown of a song into sections – high level musically coherent phrases in the song – exactly what we are looking for. We can use the sections to drive the mapping. Note that breaking a song down into sections is still an open research problem – there’s no perfect algorithm for it yet, but The Echo Nest algorithm is a state-of-the-art algorithm that is probably as good as it gets. Luckily, for this task, we don’t need a perfect algorithm. In the above visualization you can see the sections. Here’s a blown up version – each of the horizontal colored rectangles represents one section:
You can see that this song has 11 sections. Our goal then is to get a sequence of beats for the canon stream that aligns well with the beats of each section. To make things at little easier to see, lets focus in on a single section. The following visualization shows the similar beat graph for a single section (section 3) in the song:
You can see bundles of edges leaving section 3 bound for section 5 and 6. We could use these bundles to find most similar sections and simply overlap these sections. However, given that sections are rarely the same length nor are they likely to be aligned to similar sounding musical events, it is unlikely that this would give a good listening experience. However, we can still use this bundling to our advantage. Remember, our goal is to find a good coherent sequence of beats for the canon stream. We can make a simplifying rule that we will select a single sequence of beats for the canon stream to align with each section. The challenge, then, is to simply pick the best sequence for each section. We can use these edge bundles to help us do this. For each beat in the main stream section we calculate the distance to its most similar sounding beat. We aggregate these counts and find the most commonly occurring distance. For example, there are 64 beats in Section 3. The most common occurring jump distance to a sibling beat is 184 beats away. There are ten beats in the section with a similar beat at this distance. We then use this most common distance of 184 to generate the canon stream for the entire section. For each beat of this section in the main stream, we add a beat in the canon stream that is 184 beats away from the main stream beat. Thus for each main stream section we find the most similar matching stream of beats for the canon stream. This visualizing shows the corresponding canon beat for each beat in the main stream.
This has a number of good properties. First, the segments don’t need to be perfectly aligned with each other. Note, in the above visualization that the similar beats to section 3 span across section 5 and 6. If there are two separate chorus segments that should overlap, it is no problem if they don’t start at the exactly the same point in the chorus. The inter-beat distance will sort it all out. Second, we get good behavior even for sections that have no strong complimentary section. For instance, the second section is mostly self-similar, so this approach aligns the section with a copy of itself offset by a few beats leading to a very nice call and answer.
That’s the core mechanism of the autocanonizer - for each section in the song, find the most commonly occurring distance to a sibling beat, and then generate the canon stream by assembling beats using that most commonly occurring distance. The algorithm is not perfect. It fails badly on some songs, but for many songs it generates a really good cannon affect. The gallery has 20 or so of my favorites.
(2) Play the streams back simultaneously
When I first released my hack, to actually render the two streams as audio, I played each beat of the two streams simultaneously using the web audio API. This was the easiest thing to do, but for many songs this results in some very noticeable audio artifacts at the beat boundaries. Any time there’s an interruption in the audio stream there’s likely to be a click or a pop. For this to be a viable hack that I want to show people I really needed to get rid of those artifacts. To do this I take advantage of the fact that for the most part we are playing longer sequences of continuous beats. So instead of playing a single beat at a time, I queue up the remaining beats in the song, as a single queued buffer. When it is time to play the next beat, I check to see if is the same that would naturally play if I let the currently playing queue continue. If it is I ‘let it ride’ so to speak. The next beat plays seamlessly without any audio artifacts. I can do this for both the main stream and the canon stream. This virtually elimianates all the pops and clicks. However, there’s a complicating factor. A song can vary in tempo throughout, so the canon stream and the main stream can easily get out of sync. To remedy this, at every beat we calculate the accumulated timing error between the two streams. If this error exceeds a certain threshold (currently 50ms), the canon stream is resync’d starting from the current beat. Thus, we can keep both streams in sync with each other while minimizing the need to explicitly queue beats that results in the audio artifacts. The result is an audio stream that is nearly click free.
(3) Supply a visualization that gives the listener an idea of how the app works
I’ve found with many of these remixing apps, giving the listener a visualization that helps them understand what is happening under the hood is a key part of the hack. The first visualization that accompanied my hack was rather lame:
It showed the beats lined up in a row, colored by the timbral data. The two playback streams were represented by two ‘tape heads’ – the red tape head playing the main stream and the green head showing the canon stream. You could click on beats to play different parts of the song, but it didn’t really give you an idea what was going on under the hood. In the few days since the hackathon, I’ve spent a few hours upgrading the visualization to be something better. I did four things: Reveal more about the song structure, show the song sections, show, the canon graph and animate the transitions.
Reveal more about the song
The colored bars don’t really tell you too much about the song. With a good song visualization you should be able to tell the difference between two songs that you know just by looking at the visualization. In addition to the timbral coloring, showing the loudness at each beat should reveal some of the song structure. Here’s a plot that shows the beat-by-beat loudness for the song stairway to heaven.
You can see the steady build in volume over the course of the song. But it is still less than an ideal plot. First of all, one would expect the build for a song like Stairway to Heaven to be more dramatic than this slight incline shows. This is because the loudness scale is a logarithmic scale. We can get back some of the dynamic range by converting to a linear scale like so:
This is much better, but the noise still dominates the plot. We can smooth out the noise by taking a windowed average of the loudness for each beat. Unfortunately, that also softens the sharp edges so that short events, like ‘the drop’ could get lost. We want to be able to preserve the edges for significant edges while still eliminating much of the noise. A good way to do this is to use a median filter instead of a mean filter. When we apply such a filter we get a plot that looks like this:
The noise is gone, but we still have all the nice sharp edges. Now there’s enough info to help us distinguish between two well known songs. See if you can tell which of the following songs is ‘A day in the life’ by The Beatles and which one is ‘Hey Jude’ by The Beatles.
Show the song sections
As part of the visualization upgrades I wanted to show the song sections to help show where the canon phrase boundaries are. To do this I created a the simple set of colored blocks along the baseline. Each one aligns with a section. The colors are assigned randomly.
Show the canon graph and animate the transitions.
To help the listener understand how the canon is structured, I show the canon transitions as arcs along the bottom of the graph. When the song is playing, the green cursor, representing the canon stream animates along the graph giving the listener a dynamic view of what is going on. The animations were fun to do. They weren’t built into Raphael, instead I got to do them manually. I’m pretty pleased with how they came out.
All in all I think the visualization is pretty neat now compared to where it was after the hack. It is colorful and eye catching. It tells you quite a bit about the structure and make up of a song (including detailed loudness, timbral and section info). It shows how the song will be represented as a canon, and when the song is playing it animates to show you exactly what parts of the song are being played against each other. You can interact with the vizualization by clicking on it to listen to any particular part of the canon.
Wrapping up – this was a fun hack and the results are pretty unique. I don’t know of any other auto-canonizing apps out there. It was pretty fun to demo that hack at the SXSW Music Hack Championships too. People really seemed to like it and even applauded spontaneously in the middle of my demo. The updates I’ve made since then – such as fixing the audio glitches and giving the visualization a face lift make it ready for the world to come and visit. Now I just need to wait for them to come.
I’ve spent the last 24 hours at the SXSW Music Hackathon championship. For my hack I’ve built something called The Autocanonizer. It takes any song and tries to make a canon out of it. A canon is a song that can be played against a copy of itself. The Autocanonizer does this by looking at the detailed audio in the song (via The Echo Nest analysis), and looks for parts of the song that might overlap well. It builds a map of all these parts and when it plays the song it plays the main audio, while overlapping it with the second audio stream. It doesn’t always work, but when it does, the results can be quite fun and sometimes quite pleasing.
To go along with the playback I created a visualization that shows the song and the two virtual tape heads that are playing the song. You can click on the visualization to hear particular bits.
There are some audio artifacts on a few songs still. I know how to fix it, but it requires some subtle math (subtraction) that I’m sure I’ll mess up right before the hackathon demo if I attempt it now, so it will have to wait for another day. Also, there’s a Firefox issue that I will fix in the coming days. Or you can go and fix all this yourself because the code is on github.
This past weekend was the fourth annual MIDEM Music Hack Day held in Cannes. During about 48 hours 2 dozen or so hackers collected in a beautiful hacking space at the top of the Palais Des Festivals to build something cool with music and technology.
The MIDEM Music Hack Day is no ordinary Music Hack Day. It has a very limited enrollment so only hackers that have demonstrated the ability to create music hacks are invited. Add to that, the fact that there is about 50% more time to build hacks and the result is a set of very high quality hacks.
Martyn Davies, master Music Hack day coordinator, kicked off the event with a talk to the MIDEM attendees about what Music Hack Day is all about. Martyn talked about the things that drive the hackers to spend their weekends hacking on code – in particular how the Music Hack Day is a chance to combined their love for music and technology, be creative and to build something new and cool during the weekend. Martyn demonstrated two representative hacks built at previous Music Hack Days. First he showed the demo given by master hacker @sydlawrence of a hack called Disco Disco Tech. The excitement in Syd’s voice is worth the price of admission alone.
Next he showed one of my favorite Music Hack Day hacks of all time, Johnny Cash is Everywhere by @iainmullen
One of the special features of the MIDEM Music Hack Day is that non-hackers get to pitch their hacking ideas to the hackers about what apps they’d really like to see created over the weekend. There were a number of pitches ranging from a proposal for an artist-centric tool for organizing a creative music production team to a whimsical request to show what the music on the Internet sounds like when it decays. (Here’s one answer). All of the idea pitches were interesting, but here’s the secret. The hackers are not going to build your idea. It’s not because they don’t like your idea, it is because they already have tons of good ideas. The hackers are a very creative bunch, each with a long list of ideas waiting to be built. What the hackers usually lack is a solid block of time to implement their own ideas and so a hackathon is the perfect time to take that best idea on the list and work for a solid 24 hour to get it done. It is rare for a hacker to get excited about building someone else’s idea, when they have so many of their own. As they say: “ideas are cheap, execution is everything“.
Once the opening talks concluded we hackers made their way up to the top of the Palais des Festivals (the heart of MIDEM) to our hacking space. It’s a great space with lots of natural light, a terrace that overlooks the French Riviera, and it is some distance away from the main conference so we were not bothered by stray walk-ons.
To kick things off, we went around the room introducing ourselves, briefly talking about our background skills, and ideas, and almost immediately got to hacking. Since all the hackers were experienced hackers there was no need for the typical API workshops or learning sessions. Everyone knew, for instance, that I was from The Echo Nest and was ready to answer any questions about the Echo Nest API that should arise.
The next 46 hours was a blur of coding, punctuated by food delivery, the sound of the espresso maker and the occasional wandering pigeon.
Hacker Self Portraits
There’s a math error lurking in there …
Early morning coding on the French Riveria
After 48 hours, we gathered in the Innovation Hall to demonstrate what we built. Each hacking team had about three minutes to show their stuff. Eighteen hacks were built. Here are some of my very favorites:
DJ Spotify – built by Yuli Levtov – This is a real hacker’s hack. Yuli had a problem. He wanted to use Spotify when he DJed, but Spotify won’t let you beat match and cross fade songs. In fact, Spotify won’t even let you play two streams at once. So Yuli got to work to make it happen. Along the way Yuli augmented his DJ playlists with BPM and key information from The Echo Nest (using a very clever growl hack). One of the highlights of my MIDEM week was listening to Yuli try to explain what OS virtualization is and how Soundflower works to a room full of Music Biz types. Yuli has a detailed blog post that describes how his hack works. Yuli’s hack was voted the best hack by the hackers. Well done Yuli!
That One Song - by Matt Ogle and Hannah Donovan – For this hack, Matt added a feature to his super popular This Is My Jam site. Type in any band and let the Jam community tell you the one song you should hear first. Plus: playback options, commentary, and an alternative “B-side” song recommendation for each artist.
Skrillex Invader 20 – by Vivien Barousse - Imagine Guitar Hero meets Space Invaders meets Skrillex meets a Piano Keyboard. Skrillex Invader 20 is a small game designed to help you improve your skills on a piano keyboard.
Scratchy Record - by Alastair Porter - Playing music from mp3s today has no soul. Scratchy Record reproduces the joy that can be had listening to music on vinyl. From the dirty needle causing extra noise, to the pops and skips that we all love, to the need to get up half way through the album and turn it over. Scratchy Record has it all.
HappyClappy - by Peck, ankit and mager – an IOS app that lets you queue up songs by clapping the rhythm. Uses The Echo Nest to search for songs by BPM.
PhatStats - Syd Lawerence – Syd tried to build a sustainable subscription business during the hack with PhatStats - A new tool to discover up and coming talent across the social web, and to monitor your videos and their engagement levels across the social web.
This is your tour - Sam Phippen - Going on tour is hard. You’ve got to find someone to tour with. You’ve got to pick cities and venues. You’ve got to book hotels, find places to eat and drink. All of this takes far too much time.
Nikantas – Sabrina Leandro – a clever app that helps you learn a new language through music lyrics. Fill in the blanks in lyrics of your favourite English, Spanish, Portuguese, French, German, or Italian artists. Can you recognise a word in a song? A word will be displayed on the screen, press the space key when the artist sings it
neoScores meets Deezer - Bob Hamblok – HTML5 sheet music score following inside Deezer.
Seevl hipster - by Alexandre Passant - Be a real hipster. Impress your friends with obscure music tastes. Do you want to impress your friend who’s into electro-folk, or that other one who only listens to avant-garde metal? Now you can! With seevl hipster, find obscure artists that match your friend tastes, and show-off on their Facebook wall.
Playlist Plus - Iain Mullan - Playlist Plus allows you to create a richer interactive version of a playlist. Add notes and comments to each track, to share with friends, or distribute an in-depth album review. Like a particular lyric? Bookmark it at the exact timestamp in the song. Think a track has a heavy Zeppelin influence? Link to the song/album it reminds you of. Cover version? Link to the original and let the reader stream it instantly from within your PlaylistPlus!
ScapeList - Mohammed Sordo - How does a landscape sound like? You take a picture of, let’s say, the Grand Canyon in Colorado, a la Instagram, but you also want to attach a song to it, a song that makes sense to you while you were taking that picture. Now imagine that other people went to the same place, took another picture of it but picked a different song. You end up with a playlist of songs related to that landscape, a ScapeList, curated by the users themselves, which you can listen to.
VideoFairy - by Suzie Blackman - A radio-style music discovery app designed for smart TV! It’s a bit like channel hopping, but for music videos. VideoFairy finds music videos from artists you’ll like with a simple interface that works with a remote control (use arrow keys and ‘enter’ on a keyboard).Designed for ‘lean back’ TV viewing with minimum interaction, you can sit back and watch new music recommended from your last.fm profile. Skip skip anything you don’t like with a simple tap of the remote.
Cannes Burn - my hack – a music visualizaton of Ellie Goulding’s Burn
I was unusually nervous and quite tired when I gave my demo, so I fell for a newbie demo mistake and had trouble getting my desktop to display properly. But when I finally did, my demo went off smoothly. I only had to say a few words and hit the play button, so despite the nerves, it was a pretty easy demo to give. Here’s my view of the audience while giving the demo:
After we presented the hacks the hackers themselves voted for the best hack which went to Yuli for his amazing DJ Spotify. Yuli is quite the gracious and humble winner, making sure everyone got a glass of his winning champagne.
After all the hacking the exhausted hackers took some time to kick back, have a good dinner, a few drinks and long conversations into the night about life as an international music hacker.
Halfway through the MIDEM Music Hack Day I paused to take stock. Here I was, on the other side of the world sitting at the top of the Palais des Festival, overlooking the French Riviera, surrounded by friends and writing code. It was a great place to be, and I felt very fortunate to be there. This was all possible because the music biz folks realize that we hackers have lots of ideas that will advance the state-of-the-art in music tech, and even more importantly we have the ability to actually turn those ideas into reality. And so, they treat us very well. It is good to be a music hacker.
My hack at the MIDEM Music Hack Day this year is what I’d call a Creative Hack. I built it, not because it answered any business use case or because it demonstrated some advanced capability of some platform or music tech ecosystem, I built it because I was feeling creative and I wanted to express my creativity in the best way that I can which is to write a computer program. The result is something I’m particularly proud of. It’s a dynamic visualization of the song Burn by Ellie Goulding. Here’s a short, low-res excerpt, but I strongly suggest that you go and watch the full version here: Cannes Burn
Unlike all of the other hacks that I’ve built, this one feels really personal to me. I wasn’t just trying to solve a technical problem. I was trying to capture the essence of the song in code, trying to tell its story and maybe even touch the viewer. The challenge wasn’t in the coding it was in the feeling.
After every hack day, I’m usually feeling a little depressed. I call it post-hacking depression. It is partially caused by being sleep deprived for 48 hours, but the biggest component is that I’ve put my all into something for 48 hours and then it is just over. The demo is done, the code is checked into github, the app is deployed online and people are visiting it (or not). The thing that just totally and completely took over my life for two days is completely gone. It is easy to reflect back on the weekend and wonder if all that time and energy was worth it.
Monday night after the MIDEM hack day was over I was in the midst of my post-hack depression sitting in a little pub called Le Crillon when a guy came up to me and said “I saw your hack. It made me feel something. Your hack moved me.”
Cannes Burn won’t be my post popular hack, nor is it my most challenging hack, but it may be my favorite hack because I was able to write some code and make somebody that I didn’t know feel something.
Back in 2001 when the first iPod was released, Shuffle Play was all the rage. Your iPod had your 1,000 favorite songs on it, so picking songs at random to play created a pretty good music listening experience. Today, however, we don’t have 1,000 songs in our pocket. With music services like Rdio, Rhapsody or Spotify, we are walking around with millions of songs in our pocket. I’ve often wondered what it would be like to use Shuffle Play when you have millions of songs to shuffle through. Would it be a totally horrible listening experience listening to artists that are so far down the long tail that they don’t even know that they are part of a dog? Would you suffer from terminal iPod whiplash as you are jerked between Japanese teen pop and a John Philip Sousa march?
To answer these questions, I built an app called Million Song Shuffle. This app will create a playlist by randomly selecting songs from a pool of many millions of songs. It draws from the Rdio collection and if you are an Rdio user you can hear listen to the full tracks.
The app also takes advantage of a nifty new set of data returned by the Echo Nest API. It shows you the absolute hotttnesss rank for the song and the artist, so you will always know how deep you are into the long tail (answer: almost always, very deep).
So how is listening to millions of songs at random? Surprisingly, it’s not too bad. The playlist certainly gets a high score for eclecticism and surprise, and most of the time the music is quite listenable. But give it a try, and form your own opinion.
Its fun, too, to see how long you can listen to the Million Song Shuffle before you encounter a song or even an artist that you’ve heard of before. If the artist is not in the top 5K artists, it is likely you’ve never heard of them. After listening to Million Song Shuffle for a little while you start to get an idea of how much music there is out there. There’s a lot.
For the ultimate eclectic music listening experience, try the Million Song Shuffle.
This weekend, I’ve been in London, attending the London Music Hack Day. For this weekend’s hack, I was inspired by daughter’s music listening behavior – when she listens to music, she is good for the first verse or two and the chorus, but after that, she’s on to the next song. She probably has never heard a bridge. So for my daughter, and folks like her with short attention spans, I’ve built Attention Deficit Radio. ADR creates a Pandora-like radio station based upon a seed artist, but doesn’t bother you by playing whole songs. Instead, after about 30 seconds or so, it is on to the next song. The nifty bit is that ADR will try to beat-match and crossfade between the songs giving you a (hopefully) seamless listening experience as you fly through the playlist. Of course those with short attention spans need something to look at while listening, so ADR has lots of gauges that show the radio status – it shows the current beat, the status of the cross-fade, tempo and song loading status.
There may be a few rough edges, and the paint is not yet dry, but give Attention Deficit Radio a try if you have a short listening attention span.
I’m writing this post from Espoo Finland which is home to three disruptive brands: Nokia, who revolutionized the mobile phone market in the 1990s with its GSM technology; Rovio, who brought casual gaming to the world with Angry Birds; and Children of Bodom perhaps one of the most well known melodic death metal bands. So it is not surprising that Espoo is a place where you will find a mix of high tech, playfulness and hard core music – which is exactly what I found this past weekend at the Helsinki Music Hack Day hosted at the Startup Sauna in Espoo Finland.
At the Helsinki Music Hack Day, dozens of developers gathered to combine their interest in tech and their passion for music in a 24 hour hacking session to build something that was music related. Representatives from tech companies such as SoundCloud, Spotify and The Echo Nest joined the hackers to provide information about their technologies and guidance in how to use their APIs.
After 24 hours, a dozen hacks were demoed in the hour-long demo session. There was a wide range of really interesting hacks. Some of my favorites are highlighted here:
Cacophony – A multi-user remote touch controlled beat data sequencer. This hack used the Echo Nest (via the nifty new SoundCloud/Echo Nest bridge that Erik and I built on the way to Espoo), to analyze music and then allow you to use the beats from the analyzed song to create a 16 step sequencer. The sequencer can be controlled remotely via a web interface that runs on an iPad. This was a really nice hack, the resulting sequences sounded great. The developer, Pekka Toiminen used music from his own band Different Toiminen which has just released their first album. You can see the band and Pekka in the video:
It was great getting to talk to Pekka, I hope he takes his hack further and makes an interactive album for his band.
Hackface & Hackscan - by hugovk - This is a pretty novel set of hacks. Hackface takes the the top 100 or 1000 artists from your listening history on Last.fm, finds photos of the artists (via the Echo Nest API), detects faces using a face detection algorithm, intelligently resizes them and composites them into a single image giving you an image of what your average music artist in your listening history looks like.
Hackscan - takes a video and summarizes it intelligently into a single image by extracting single columns of pixels from each frame. The result is a crazy looking image that captures the essence of the video.
Hugo was a neat guy with really creative ideas. I was happy to get to know him.
Stronger Harder Faster Jester – Tuomas Ahva and Valtteri Wikstrom built the first juggling music hack that I’ve seen in the many hundreds of hack demos I’ve witnessed over the years. Their hack used three bluetooth-enabled balls that when thrown triggered music samples.
The juggler juggles the balls in time with the music and the ball tossing triggers music samples that align with the music. The Echo Nest analysis is used to extract the salient pitch info for the aligment. It was a really original idea and great fun to watch and listen to. This hack won the Echo Nest prize.
µstify - This is the classic boy meets girl story. Young man at his first hackathon meets a young woman during the opening hours of the hackathon.
They decide to join forces and build a hack (It’s Instagram for Music!) and two days later they are winning the hackathon! Alexandra and Arian built a nifty hack that builds image filters (in the style of Instagram) based upon what the music sounds like. They use The Echo Nest to extract all sorts of music parameters and use these to select image filters. Check out their nifty presentation.
Gig Voter - this Spotify app provides a way for fans to get their favorite artists to come to their town. Fans from a town express an interest in an artist. Artists get a tool or helping them plan their tour based on information about where their most active fans actually are as well as helping them sell gigs to location owners by being able to prove that there is demand for them to perform at a certain location. Gig Voter uses Echo Nest data to help with the search and filtering.
Hit factory - Hit Factory is a generative music system that creates music based upon your SoundCloud tastes and adapts that music based upon your feedback . Unfortunately, no samples of the music are to be found online, but take my word, they were quite interesting – not your usual slightly structured noise.
Abelton Common Denominator – a minimal, mini-moog style interface to simplify the interaction with Abelton – by Spotify’s Rikard Jonsson.
Swap the Drop - this was my hack. You can read more about it here.
One unusual aspect of this Music Hack Day was that a couple of teams that encountered problems and were unable to finish their hacks still got up and talked about their failures. It was pretty neat to see hardcore developers get up in front of a room full of their peers and talk about why they couldn’t get Hadoop to work on their terrabyte dataset or get their party playlister based on Meteor to run inside Spotify.
I’ve enjoyed my time in Espoo and Helsinki. The Hack Day was really well run. It was held in a perfect hacking facility called the Startup Sauna.
There was plenty of comfortable hacking spots, great wifi, and a perfect A/V setup.
The organizers kept us fed with great food (Salmon for lunch!), great music, including a live performance by Anni.
There was plenty of Angry Birds Soda.
Many interesting folks to talk to …
Thanks to Lulit and the rest of the Aaltoes team for putting together such a great event.
I’ve been in Helsinki this weekend (which is not in Sweden btw) for the Helsinki Music Hack Day. I wanted to try my hand at a DJ app that will allow you to dynamically and interactively mix two songs. I started with Girl Talk in a Box, ripped out the innards and made a whole bunch of neat changes:
- You can load more than one song at a time. Each song will appear as its own block of music tiles.
- You can seamlessly play tiles from either song.
- You can setup branch points to let you jump from an point in one song to any point in another (or the same) song.
- And the killer feature – you can have two active play heads allowing you to dynamically interact with two separate audio streams. The two play heads are always beat matched (the first play head is the master that sets the tempo for everyone else). You can cross-fade between the two audio streams – letting you move different parts of the song into the foreground and the background.
All the regular features of Girl Talk in a Box are retained – bookmarks, arrow key control, w/a/s/d navigation and so on. See the help for more details on the controls.
You can try the app here: Swap the Drop
A Music Hack Day is unlike most other hackathons. There are no mega-prizes for the best hacks. There are no VCs wandering the hacker hallways trolling for the next startup. There are no briefs that describe the types of apps that you should build. Hackers don’t go to a Music Hack Day to win big prizes, or to launch their startup. Hackers go to Music Hack Days because they love music and they love to build stuff. At a Music Hack Day these passionate builders get to apply their talents to music, surrounded by like-minded peers and build their version of the future of music. The currency at a Music Hack Day is not money or VC attention, the currency is creativity. The Music Hack Day prize is knowing that you’ve built something cool enough to delight other music hackers.
So what happens at a Music Hack Day? How does it all work? What kind of hacks do people build? Read on to see exactly what happened at the Boston Music Hack Day 2013, held this last weekend.
Boston Music Hack Day
This weekend, hundreds of folks who are passionate about music and technology got together in Cambridge MA for Boston Music Hack Day 2013. The event was hosted at the Microsoft NERD - a wonderful facility that Microsoft makes available for all sorts of programmer events. Registration started at 9AM and by 10AM hackers were breakfasted and ready to go.
The event started off with some opening remarks by your truly, describing how a Music Hack Day works and how to have a successful event (meet other people, learn new stuff, build something, make sure you finish it, demo it and have fun).
Short technology presentations
Next up, organizations that had some sort of music technology such as an API or new gizmo that might be interesting to music hackers spent a few minutes talking about their technology. For many hackers, this was their first exposure to the music ecosystem – they don’t know what APIs are available for building apps so learning about music streaming APIs from companies like SoundCloud, Rdio and Spotify, and learning about all the music data available from APIs like The Echo Nest and the Free Music Archive is really important.
There were a few interesting devices available for hackers at the event. Techogym brought a high tech treadmill with its own API hoping that music hackers would build music-related exercise apps. Muzik brought a set of headphones that are instrumented with accelerometers and other sensors allowing for apps to adapt to the actions of the listener.
Sometimes hackers come to a Music Hack Day with ideas ready to go. Sometimes hackers come with their skills but no ideas. At the Project Pitch session, hackers had a minute to pitch their idea or to offer their skills. About 20 hackers braved the front of the room describing their idea or their skill set. One hacker described his project as help me with my homework (and yes, this hacker did find a teammate and they ultimately built a nifty hardware hack that satisfied the homework requirement too).
Tech Deep Dives
Next up on the schedule were the Tech Deep dives. Organizations had a half-hour to give a deeper view of what their technology is capable of. Some hackers want to know more about how to do particular things with an API or technology. This is their opportunity to find out all about the nuts and bolts, to ask questions from the experts. The Tech Deep Dives are strictly optional – many hackers skip them and instead start forming teams, sharing ideas, initializing their repos and writing code.
After all the preliminaries are over it was finally time to start hacking.
Hackers formed teams, large and small.
The competition for the best vest was fierce:
They staked out a comfortable workspace in chairs …
Or on the floor …
There were some very creative hardware hacks:
Plenty of good food
And lots of fun
The Microsoft NERD was only available until 9PM – after that we moved over to hack/reduce – a wonderful hacking space a five minute walk away. There we were greeted by a perfect hacking space with lots of great wifi, great hacker lighting, and lots of beer. Hacking continued all night. Some hackers did try to get some sleep (either at the hack, or back at home), but some hardcore hackers stayed the whole night.
By 9AM on Sunday morning, the hackers were back at the NERD, for lots of coffee, some breakfast and then more hacking.
At 2:30, hacking was officially over, and teams submitted their projects to Hacker League. Sixty hacks were submitted.
By 3PM the 200 hackers had all gathered back into the big room joined by a hundred folks who had come just to see the demo session. Hackers had two minutes to show their stuff. It is a hard demo to give. You are giving a demo of software that you’ve just finished building. It might have some bugs. The WiFi is a little flaky, you haven’t slept in 24 hours, your hands are shaking from too much coffee and too much nervousness, you have to type while holding a microphone and your laptop just won’t sync with the projector just right, and the audio isn’t coming out of the speakers, and the colors look all wrong on the screen. All in front of 300 people. I’ve done it dozens of times and it still is a really scary demo to give. But it is incredibly exhilarating too – to take nothing but an idea and turn it into something that can amaze or amuse a room full of tech elite in 24 hours. It is quite a rush.
There were two A/V setups so while one team was presenting, the next team was setting up. This allowed us to get through 60 demos in just over two hours. There was a very low incidence of demo fail. And only two inappropriate demos (one was a 2 minute powerpoint presentation with no tech built, the other was a 2 minute tech commercial for a product). I was worried that we might have a #titstare moment with one hack that seemed to contain questionable content but that hacker apparently decided not to present.
The 60 hacks represented a wide range of domains. There were games, music learning tools, programs designed to create, manipulate, remix and even destroy music. I’d love to cover them all, but there are just too many.
The full list is on hacker league. Here are some of my absolute favorites:
String Theory - A musical instrument and sound sculpture build from yearn and stretch sensors and powered by an Arduino.
The Lone Arranger - a terminal app that allows you to easily rearrange your audio. By a father and son hacking team.
The Secret History of Music - combs biographies, lyrics, and commentary from song meanings from two artists, combines them into one fictional artist, and uses Markov chain magic to generate a 50K novel about this new fictional band.
LED Soundsystem - this hack attempts to generate a light show synchronized to the music. It has a special place in my heart because the hacking team was working on the same problem I was working on for my abandoned hack – i.e. automatically finding the ‘drop’ in a song. Unfortunately, this team had a demo fail – but they are smart guys and I expect to see good stuff from them at the next hack day.
eHarmonica – an electronic harmonica!
Enter the dragon - In today’s world, everyone deserves a spectacular entrance. And we intend to give it to them. Enter the Dragon uses bluetooth technology to detect when a user enters the room and plays their personalized entrance music.
Dadabots - Dadabots are bot accounts on creative websites that make procedural creations or remixes of other creations
ios SoundPuzzle - A simple iOS ear training game built programmatically from the free music archive and the echonest remix api.
danceomatic – totally awesome automatic choreography from an mp3 and a web based stick figure performance.
Jotunnslayer – Never again listen to power metal without slaying ice giants. Die in battle. Earn your place in Valhalla.
Give Me Liberty or Give Me Death Metal - Political representation via musical exploration
Echos – Choose your favorite song and shoot to the beat! Fight enemies that respond to and are controlled by the music! Listen closely and experience unparalleled power as you get into the groove! Enjoy addictive arcade-style game-play in this twist on a classic formula.
TweetTones - a native iOS application that generates synthesized music from tweets in real-time.
Short-Attention Span Playlist Scanner - Glenn made a radio scanner that find and plays just the choruses.
ionian Eclipse - A web-based multiplayer top-down space shooter with procedurally generated enemies and interactions driven by music events.
ColorMe - Ever wondered what your music looked like? Now you can look at songs by your favorite artist with this super fun web app, powered by the Echo Nest.
How Repetitive - measures how often audio segments repeat themselves within in a given song.
Jason’s music visualizer - an html/css visualizer on steroids.
And last, but by no means least, Jonathan’s awesome MIDI Digester that converts audio to MIDI and back, over and over to generate some very strange sounds. The very essence of the music.
There are so many excellent hacks, I’m sure I’ve missed many notables. Luckily, Evolver.fm covered the event, so expect to see Eliot’s writups on all the best hacks on Evolver.fm.
At the end of the mega demo session, there’s a brief prize awarding ceremony where a half-dozen organizations give out modest prizes for hackers that made cool stuff using their tech.
Finally we adjourned to the local pub for some food, beer and hacking recaps.
Special thanks to the organizers of the event. The Music Hack Day would not happen without Elissa and Matt. They do all the hard work. Finding the venue, wrangling the sponsors and volunteers, making a mega Costco food run, dealing with the A/V, running the registration, selecting and hiring the caterers, designing t-shirts and so much more. There’s a huge amount of work that goes into planning the event, much more than meets the eye. Elissa and Matt are the unsung heroes of Music Hack Day. We should make a music hack to sing their song.
Thanks also to the event sponsors: Rdio, Spotify, Microsoft, hack/reduce, Free Music Archive, SoundCloud, Mailchimp and The Echo Nest, and the many volunteers who came and helped us run the whole show.
More Music Hack Days
Interested in going to a Music Hack Day? Check out the Music Hack Day calendar for upcoming events. There’s one in Helsinki this weekend, and there’s one in London in just a few weeks. More events are rumored to be in the planning stages for 2014.
(Photos mostly by Michelle Ackerman, a few by me)