boston area music tech meetup
Monday, March 30th I’ll be heading down to the Miracle of Science (a watering hole near MIT) to attend the first official zed equals zee Happy Hour. If you are interested in joining a mix of music techies, music bloggers and music makers for a few hours, to talk about music and technology feel free to join us. RSVP here at zed equals zee.
Put a DONK on it
Posted by Paul in code, fun, Music, The Echo Nest on March 25, 2009
rfwatson has just released a site called donkdj that will ‘remix your favourite song into a bangin’ hard dance anthem‘. You upload a track and donkdj turns it into a dance remix. The results are just brilliant. Here are a few examples:
The site uses The Echo Nest Remix API to do all of the heavy lifting – adding a kick, snap, claps and the infamous donk (I had to look it up … a donk is a a pipe/plank-sound, that is used in Bouncy/scouse house/NRG music). What is doubly cool is rfwatson has open sourced his remix code so you can look under the hood and see how it works and adapt it for your own use. The core of this remix is done in just 200 lines of python code.
donkdj is really cool – the results sound fantastic and the open sourcing of the code makes it easy for anyone else to make their own remixer. I can’t wait to see it when someone makes an automatic Stephen Colbert remixer.
Update: Ben showed me this post that points to this video about Donk:
The full series is available here.
The Loudness War Analyzed
Posted by Paul in code, data, fun, Music, research, The Echo Nest, visualization on March 23, 2009
Recorded music doesn’t sound as good as it used to. Recordings sound muddy, clipped and lack punch. This is due to the ‘loudness war’ that has been taking place in recording studios. To make a track stand out from the rest of the pack, recording engineers have been turning up the volume on recorded music. Louder tracks grab the listener’s attention, and in this crowded music market, attention is important. And thus the loudness war – engineers must turn up the volume on their tracks lest the track sound wimpy when compared to all of the other loud tracks. However, there’s a downside to all this volume. Our music is compressed. The louds are louds and the softs are loud, with little difference. The result is that our music seems strained, there is little emotional range, and listening to loud all the time becomes tedious and tiring.
I’m interested in looking at the loudness for the recordings of a number of artists to see how wide-spread this loudness war really is. To do this I used the Echo Nest remix API and a bit of Python to collect and plot loudness for a set of recordings. I did two experiments. First I looked at the loudness for music by some of my favorite or well known artists. Then I looked at loudness over a large collection of music.
First, lets start with a loudness plot of Dave Brubeck’s Take Five. There’s a loudness range of -33 to about -15 dBs – a range of about 18 dBs.
Now take a look at a track from the new Metallica album. Here we see a dB range of from about -3 dB to about -6 dB – for a range of about 3 dB. The difference is rather striking. You can see the lack of dynamic range in the plot quite easily.
Now you can’t really compare Dave Brubeck’s cool jazz with Metallica’s heavy metal – they are two very different kinds of music – so lets look at some others. (One caveat for all of these experiments – I don’t always know the provenance of all of my mp3s – some may be from remasters where the audio engineers may have adjusted the loudness, while some may be the original mix).
Here’s the venerable Stairway to Heaven – with a dB range of -40 dB to about -5dB for a range of 35 dB. That’s a whole lot of range.
Compare that to the track ‘supermassive black hole’ – by Muse – with a range of just 4dB. I like Muse, but I find their tracks to get boring quickly – perhaps this is because of the lack of dynamic range robs some of the emotional impact. There’s no emotional arc like you can see in a song like Stairway to Heaven.
Some more examples – The Clash – London Calling. Not a wide dynamic range – but still not at ear splitting volumes.
This track by Nickleback is pushing the loudness envelope, but does have a bit of dynamic range.
Compare the loudness level to the Sex Pistols. Less volume, and less dynamic range – but that’s how punk is – all one volume.
The Stooges – Raw Power is considered to be one of the loudest albums of all time. Indeed, the loudness curve is bursting through the margins of the plot.
Here in one plot are 4 tracks overlayed – Red is Dave Brubeck, Blue is the Sex Pistols, Green is Nickleback and purple is the Stooges.
There been quite a bit of writing about the loudness war. The wikipedia entry is quite comprehensive, with some excellent plots showing how some recordings have had a loudness makeover when remastered. The Rolling Stone’s article: The Death of High Fidelity gives reactions of musicians and record producers to the loudness war. Producer Butch Vig says “Compression is a necessary evil. The artists I know want to sound competitive. You don’t want your track to sound quieter or wimpier by comparison. We’ve raised the bar and you can’t really step back.”
The loudest artists
I have analyzed the loudness of about 15K tracks from the top 1,000 or so most popular artists. The average loudness across all 15K tracks is about -9.5 dB. The very loudest artists from this set – those with a loudness of -5 dB or greater are:
| Artist | dB |
|---|---|
| Venetian Snares | -1.25 |
| Soulja Boy | -2.38 |
| Slipknot | -2.65 |
| Dimmu Borgir | -2.73 |
| Andrew W.K. | -3.15 |
| Queens of the Stone Age | -3.23 |
| Black Kids | -3.45 |
| Dropkick Murphys | -3.50 |
| All That Remains | -3.56 |
| Disturbed | -3.64 |
| Rise Against | -3.73 |
| Kid Rock | -3.86 |
| Amon Amarth | -3.88 |
| The Offspring | -3.89 |
| Avril Lavigne | -3.93 |
| MGMT | -3.94 |
| Fall Out Boy | -3.97 |
| Dragonforce | -4.02 |
| 30 Seconds To Mars | -4.08 |
| Billy Talent | -4.13 |
| Bad Religion | -4.13 |
| Metallica | -4.14 |
| Avenged Sevenfold | -4.23 |
| The Killers | -4.27 |
| Nightwish | -4.37 |
| Arctic Monkeys | -4.40 |
| Chromeo | -4.42 |
| Green Day | -4.43 |
| Oasis | -4.45 |
| The Strokes | -4.49 |
| System of a Down | -4.51 |
| Blink 182 | -4.52 |
| Bloc Party | -4.53 |
| Katy Perry | -4.76 |
| Barenaked Ladies | -4.76 |
| Breaking Benjamin | -4.80 |
| My Chemical Romance | -4.81 |
| 2Pac | -4.94 |
| Megadeth | -4.97 |
It is interesting to see that Avril Lavigne is louder than Metallica and Katy Perry is louder than Megadeth.
The Quietest Artists
Here are the quietest artists:
| Artist | dB |
|---|---|
| Brian Eno | -17.52 |
| Leonard Cohen | -16.24 |
| Norah Jones | -15.75 |
| Tori Amos | -15.23 |
| Jeff Buckley | -15.21 |
| Neil Young | -14.51 |
| Damien Rice | -14.33 |
| Lou Reed | -14.33 |
| Cat Stevens | -14.22 |
| Bon Iver | -14.14 |
| Enya | -14.13 |
| The Velvet Underground | -14.05 |
| Simon & Garfunkel | -14.03 |
| Pink Floyd | -13.96 |
| Ben Harper | -13.94 |
| Aphex Twin | -13.93 |
| Grateful Dead | -13.85 |
| James Taylor | -13.81 |
| The Very Hush Hush | -13.73 |
| Phish | -13.71 |
| The National | -13.57 |
| Paul Simon | -13.53 |
| Sufjan Stevens | -13.41 |
| Tom Waits | -13.33 |
| Elvis Presley | -13.21 |
| Elliott Smith | -13.06 |
| Celine Dion | -12.97 |
| John Lennon | -12.92 |
| Bright Eyes | -12.92 |
| The Smashing Pumpkins | -12.83 |
| Fleetwood Mac | -12.82 |
| Tool | -12.62 |
| Frank Sinatra | -12.59 |
| A Tribe Called Quest | -12.52 |
| Phil Collins | -12.27 |
| 10,000 Maniacs | -12.04 |
| The Police | -12.02 |
| Bob Dylan | -12.00 |
(note that I’m not including classical artists that tend to dominate the quiet side of the spectrum)
Again, there are caveats with this analysis. Many of the recordings analyzed may be remastered versions that have have had their loudness changed from the original. A proper analysis would be to repeat using recordings where the provenance is well known. There’s an excellent graphic in the wikipedia that shows the effect that remastering has had on 4 releases of a Beatles track.
Loudness as a function of Year
Here’s a plot of the loudness as a function of the year of release of a recording (the provenance caveat applies here too). This shows how loudness has increased over the last 40 years
I suspect that re-releases and re-masterings are affecting the Loudness averages for years before 1995. Another experiment is needed to sort that all out.
Loudness Histogram:
This table shows the histogram of Loudness:
Average Loudness per genre
This table shows the average loudness as a function of genre. No surprise here, Hip Hop and Rock is loud, while Children’s and Classical is soft:
| Genre | dB |
|---|---|
| Hip Hop | -8.38 |
| Rock | -8.50 |
| Latin | -9.08 |
| Electronic | -9.33 |
| Pop | -9.60 |
| Reggae | -9.64 |
| Funk / Soul | -9.83 |
| Blues | -9.86 |
| Jazz | -11.20 |
| Folk, World, & Country | -11.32 |
| Stage & Screen | -14.29 |
| Classical | -16.63 |
| Children’s | -17.03 |
So, why do we care? Why shouldn’t our music be at maximum loudness? This Youtube video makes it clear:
Luckily, there are enough people that care about this to affect some change. The organization Turn Me Up! is devoted to bringing dynamic range back to music. Turn Me Up! is a non-profit music industry organization working together with a group of highly respected artists and recording professionals to give artists back the choice to release more dynamic records.
If I had a choice between a loud album and a dynamic one, I’d certainly go for the dynamic one.
Update: Andy exhorts me to make code samples available – which, of course, is a no-brainer – so here ya go: volume.py
Help! My iPod thinks I’m emo.
Posted by Paul in Music, recommendation, research, The Echo Nest on March 23, 2009
Here are the slides for the panel “Help! My iPod thinks I’m emo.” that Anthony Volodkin (from The Hype Machine) and I gave at SXSW on music recommendation:
Collaborative Filtering and Diversity
Posted by Paul in recommendation, research, Uncategorized on March 23, 2009
One of the things Anthony and I talked about at our “Help! My iPod thinks I’m emo.” SXSW panel last week is the ‘Harry Potter Effect’ – how popular items in a recommender can lead to (among other things) feedback loops that lead to a situation where the rich get richer. A popular item like that latest Coldplay or Metallica album get purchased often with other albums and therefore end up getting recommended more frequently – and because it gets recommended – it gets purchased more often until it is sitting on the top of the charts. The Harry Potter effect can result in a lowering of the diversity of items consumed.
In his post, Online Monculture and the End of the Niche, Tom Slee over at whimsley has run a simulation that shows how this drop in diversity occurs – and also explains the non-intuitive result that while the use of a recommender can lead to decreased diversity overall, it can lead to increased diversity for an individual. Tom explains this with a metaphor: In the Internet World the customers see further, but they are all looking out from the same tall hilltop. While without a recommender individual customers are standing on different, lower, hilltops. They may not see as far individually, but more of the ground is visible to someone.
As an example of this effect, here’s a recommendation from Amazon that shows how 8% of those that shopped for The Big Penis Book
went on to buy a Harry Potter book. A recommender that pushes those that are buying books about big penises toward Harry Potter may indeed increase the diversity of those individuals (they may never have considered harry potter before, because of all those penises), but does indeed lower the overall diversity of the community as a whole (everyone is buying harry potter).
It is an interesting post, with charts and graphs and a good comment thread. Worth a read. (Thanks for the tip Jeremy)
Yet another twitter music app – song.ly
song.ly is an app that lets you post songs on twitter. Here’s what you do:
- Go to song.ly
- Search for a song that you want to tweet. The catalog seems pretty big, so you’ll find most popular songs, and many unpopular ones – for instance, I had no problems finding a track by Winter Gloves, an Indie band from Montreal.

- When you find the song click on ‘tweet’ – you are brought to your twitter page with your status filled in with a tiny url pointing to your song.
- Add whatever words you want to your tweet – and hit update

- Anyone can listen to the song by clicking on your tweet

The really neat thing about song.ly is that you don’t have to register or login to do any of this. It just works. There’s also a firefox plugin that lets you share a track with two clicks: right click a music file, select ‘Song.ly: Share this song’ and your are done. Song.ly also has an API that lets you shorten/expand Mp3 urls to the song.ly short form.
I like the low impact model of Song.ly – I don’t have to register or login. If I want to tweet a song I just right click and go. Nice and easy.
Via The Echo Nest
Song Visualizations
Posted by Paul in Music, visualization on March 19, 2009
A couple of people sent me this link today: Song Visualizations with Echo Nest.These are song plots made using the Echo Nest Analyze API. They are quite aesthically pleasing. As the creator, Chris Mueller points out, the plots are similar to those that Anita created with the music box. In addition to the pitch plot, Chris includes a plot of the volume change over the course of the song. (It might be nice to low-pass filter the volume to make it smoother). These are very nice song visualizations. Well done.
After you check out the Song Visualizations, be sure to check out the rest of the blog. Chris has some great posts about subway maps (little known fact, I have a London Tube map with music artists superimposed on all the stations hanging in my kitchen).
(shhh, don’t tell anyone, but a secret spare time project of mine is to recreate this map properly, so that the artist relations reflect reality, and the artist popularity is proportional to a stations popularity … I have all of the artist relation data (thanks to the Echo Nest), I just need the tube connectivity graph and station data).
Overview of the Echo Nest Remix API
Posted by Paul in Uncategorized on March 19, 2009
Adam Lindsay has put together some really nice documentation for the remix API:
- An Overview of the Echo Nest Remix API – This is, as Adam puts it, a “high-level tourist’s guide” to the API. In this guide, Adam walks the reader through the hierarchy of information returned by the API (beats, bars, tatums, sections and segments), and then goes on to describe some of the ways all of this information can be retrieved and manipulated using the remix methods. Adam has put together some rather ingenious classes and patterns that make walking through the information in a song really easy. For instance, to find all of the beats in a song that fall on the first beat of the measure, one could code:
beats = song.analysis.beats ones = beats.that(fall_on_the(1))I like Adam’s way of thinking about remix: “Remix makes each song its own API: each song offers queries into its own features, and can return any number of transformed versions of itself, all of which are sensitive to the original song’s musical features”
- remix module documentation – Adam has generated some Javadoc style documentation for remix. This lays out all of the classes, building blocks, helper functions and variables that you need to know about to use remix. Until now, it has been necessary to look at code samples or delve into the remix code to see what methods and tools were available. This set of documentation lets you drive the car without having to open the hood to start it. Great stuff.
Thanks much, Adam for providing this documentation. The whole community is benefitting from your work. Awesome!
10 things I learned at SXSW
- I need to get a black tee-shirt

Black tees and Macs
- If I don’t have enough content to fill my timeslot, start showing pictures of puppies.
- If I start losing my hair, the best thing to do is to shave it all off and try to look like Clay Shirkey
- To be taken seriously, I must have the latest revision of Apple hardware
- Nuclear tacos can burn me a second time, 4 hours later.

Nuclear Tacos
- Amongst my SXSW peers, my attention span is actually way above average

Time to check twitter
- When I have a choice between form and content, always chose form.
- I need to turn off the key click on my iPhone lest I disturb my neighbors.
- No matter what my second-grade teacher taught me, proper and polite behavior during a talk is to be chatting a way to all of my neighbors (via twitter).

twitterholic
- Don’t try my DVI to VGA adapter for the first time 5 minutes before my talk (f*ck you apple!)
sxsw music discovery chaos?
Posted by Paul in Music, recommendation on March 18, 2009
The very last panel I attended at SXSW Interactive was a panel called “Music 2.0 = Music Discovery Chaos?” This was a roundtable discussion as opposed to a more traditional panel where ‘experts’ do most of the talking. Elliot and Sandy Hurst of Supernova.com guided a conversation about the state of music discovery.
To tell the truth, I had low expectations for this panel. These things often devolve into (a) discussion about business models, (b) people pimping their new site, (c) some self-proclaimed expert dominating the discussion. But instead of a trainwreck, this panel turned into one my highlights of SXSW.
There was a wide range of people with a wide range of views that participated in the discussion. There were music fans (of course) that touted their favorite discovery mechanisms (friends, last.fm, hype machine). There were music critics who reminded us of the role of the expert in filtering music, but who also admitted that there’s just too much music for them to deal with, so they need their own filters. There were music programmers who talked about the different levels of listening adventurousness based on demographics (us old people apparently are less adventurous). And there was the gadfly in the back of the room, who wondered why we cared so much about this – he had no problems finding music – and if people want to listen to American Idol, so what?
Early on in the discussion Elliot took a poll of the room that seemed to indicate that for many the primary way people found music was through friends. After this poll he ask me “why, since it seemed that most people found new music through their friends do we need machines to help us find music?”. I got to paraphrase the line from Mike McGuire: “music recommendation is for people that don’t have friends”. That got a bit of a laugh.
Of course, for a discussion like this, there’s never an ultimate agreement on anything. But it was fantastic to listen to the debate – especially by so many really smart people who are very passionate about music. Awesome panel! Good job Elliot and Sandy.


















