Posts Tagged sweden
Last week, I ventured to Gothenburg Sweden to participate in the Way Out West Hack 2 – a music-oriented hackathon associated with the Way Out West Music Festival.
I was, of course, representing and supporting The Echo Nest API during the hack, but I also put together my own Echo Nest-based hack: The Saddest Stylophone. The hack creates an auto accompaniment for just about any song played on the Stylophone – an analog synthesizer toy created in the 60s that you play with a stylus.
How does it work? The Sad Stylophone takes advantage of the Echo Nest detailed analysis. The analysis provides detailed information about a song. It includes information about where all the bars and beats are, and includes a very detailed map of the segments of a song. Segments are typically small, somewhat homogenous audio snippets in a song, corresponding to musical events (like a strummed chord on the guitar or a brass hit from the band).
A single segment contains detailed information on the pitch, timbre, loudness. For pitch it contains a vector of 12 floating point values that correspond to the amount of energy at each of the notes in the 12-note western scale. Here’s a graphic representation of a single segment:
This graphic shows the pitch vector, the timbre vector, the loudness, confidence and duration of a segment.
The Saddest Stylophone only uses the pitch, duration and confidence data from each segment. First, it filters segments to combine short, low confidence segments with higher confidence segments. Next it filters out segments that don’t have a predominant frequency component in the pitch vector. Then for each surviving segment, it picks the strongest of the 12 pitch bins and maps that pitch to a note on the Stylophone. Since the Stylophone supports an octave and a half (20 notes), we need to map 12 notes onto 20 notes. We do this by unfolding the 12 bins by reducing inter-note jumps to less than half an octave when possible. For example, if between segment one and segment two we would jump 8 notes higher, we instead check to see if it would be possible to jump to 4 notes lower instead (which would be an octave lower than segment two) while still remaining within the Stylophone range. If so, we replace the upward long jump with the downward, shorter jump. The result of this a list of notes and timings mapped on to the 20 notes of the Stylophone. We then map the note onto the proper frequency and key position – the rest is just playing the note via timbre.js at the proper time in sync with the original audio track and animating the stylus using Raphael.
I’ve upgraded the app to include an Under the hood selection that, when clicked opens up a visualization that shows the detailed info for a segment, so you can follow along and see how each segment is mapped onto a note. You can interact with visualization, stepping through the segments, and auditioning and visualizing them.
That’t the story of the Saddest Stylophone – it was not the hack I thought I was going to make when I got to #wowhack – but I was pleased with the result, when The Sad Stylophone plays well, it really can make any song sound sadder and more pathetic. Its a win. I’m not the only one – wired.co.uk listed it as one of the five best hacks at the hackathon.
Give it a try at Saddest Stylophone.
In a lucky coincidence I happened to be in Stockholm yesterday which allowed me to give a short talk at the Stockholm Python user Group. About 80 or so Pythonistas gathered at Campanja to hear talks about Machine Learning and Python. The first two talks were deep dives into particular aspects of machine learning and Python. My talk was quite a bit lighter. I described the Million Song Data Set and suggested that it would be a good source of data for anyone looking for a machine learning research. I then went on to show a half a dozen or so demos that were (or could be) built on top of the Million Song Data Set. A few folks at the event asked for links, so here you go:
Core Data: Echo Nest analysis for a million songs
- Second Hand Songs – 20K cover songs
- MusixMatch – 237K bucket-of-words lyric sets
- Last.fm tags – song level tags for 500K tracks. plus 57 million sim. track pairs
- Echo Nest Taste profile subset – 1M users, 48M user/song/play count triples
Data Mining Listening Data: The Passion Index
Fun with Artist Similarity Graphs: Boil the Frog
Turning music into silly putty - Echo Nest Remix
I really enjoyed giving the talk. The audience was really into the topic and remained engaged through out. Afterwards I had lots of stimulating discussions about the music tech world. The Stockholm Pythonistas were a very welcoming bunch. Thanks for letting me talk. Here’s a picture I took at the very end of the talk: