Paul
I'm the Director of Developer Community at The Echo Nest, a research-focused music intelligence startup that provides music information services to developers and partners through a data mining and machine listening platform. I am especially interested in hybrid music recommenders and using visualizations to aid music discovery.
The Echo Nest Cocoa Framework
Posted in code, Music, The Echo Nest, web services on July 1, 2009
Kamel Makhloufi (aka melka) has created a Cocoa Framework for the Echo Nest and has released it as open source. This framework makes it easy for Mac developers (and presumable iPhone and iTouch developers) to use the Echo Nest API services. Kamel’s goal is to build an application similar to Audiosurf (a music-adapting puzzle racer that uses your own music), but along the way Kamel realized his framework may be useful to others and so he has released it for all of us to use.
The Framework supports all of the Track/Analysis methods of the API including Track Upload, getting tempos, duration, bar, beat and tatum info as well as detailed segment information. On Melka’s TODO list is to add the Echo Nest artist methods.
Using the framework, Melka created a nifty track visualization tool that will render a colorful representation of the Echo Nest analysis for a track:
Kamel implemented this in about 300 lines of Objective-C code.
The Echo Nest Cocoa Framework is released under a GPL V3 license and is hosted on google code at: http://code.google.com/p/echonestcocoaframework/.
The release is just in time for Music Hackday – I’m hoping we see an iPhone app or two emerge from this event that use the Echo Nest APIs! Kamel’s framework is just the thing to make it happen
The Shill Machine
Posted in Music, recommendation on June 30, 2009
The very popular blog aggregator The Hype Machine has a ‘Popular Page‘ that shows the tracks that have been most favorited in the last 3 days. This is a great way to find out what the music zeitgeist is. However, Anthony (Mr. Hype Machine) recently discovered that a number of highly favorited artists seemed to have reached the popular page by nefarious means. According to Anthony, it appears that a number of artists became popular when many presumably fake accounts, created from the same IP address in a very short period of time all favorited a single artist in an apparent effort to get the artist to appear on the popular page. This type of hacking is not too surprising – whenever you have a chart or poll that relies on ‘the wisdom of crowds’ you are susceptible to the shill who will try to manipulate the chart in order to promote their interests. We see this in online polls, social news sites and popular music sites.
When Anthony became aware of how the Hype Machine was being manipulated, he and the rest of the Hype machine team fought back, instituting a Captcha mechanism to prevent automated account creation, ignoring favoriting activity for new accounts, and keeping a much closer eye on new account activity.
But Anthony didn’t stop there, he went one step further. He named names. He posted on his blog a list of all the artists that, according to Anthony have “attempted to manipulate the charts on the Hype Machine”. Anthony says he published the list to “let everyone make their own judgments about quality, integrity and marketing strategies:”. But really, I suspect that Anthony’s real motivation was to shame those that would attempt to try to enlist the Hype Machine to promote their band.
A commenter on that blog post that claims membership in one of the outed shilling bands protests that they absolutely did not create fake accounts and they had been unfairly defamed (literally) by the Hype Machine. But Anthony responds with a list 4 tracks by the band that had each been favorited from a single IP address by over 40 separate, newly created accounts. Anthony says “Given that this is a time-consuming activity that primarily benefits you, you can see how it appears likely that you or your team may have been involved”.
Should Anthony have outed these artists? Surely the excessive favoriting could have been an overzealous fan that decided to try out a new way to hype their favorite band (to put the ‘hype’ in Hype Machine, if you will), and the band is blameless. But from Anthony’s point of view it doesn’t really matter. Anthony is going to protect the integrity of the Hype Machine and he’s going to do it by pointing to any band that has benefited from ‘unnatural’ enthusiasm. Even if it means public humiliation for the blameless.
I suspect Anthony’s next problem will occur when some pranksters realize that they can get any band blacklisted at the Hype Machine by a bit of nefarious activity. By simply creating a set of sham accounts and favoriting tracks by the vicitim band from those sham acounts, the Hype Machine can be manipulated into blacklisting and humilating the band. Is your ex-girlfriend’s new boyfriend in a band? Get your dorm floor to create 50 Hype Machine accounts, favorite his tracks and watch the fun as he gets outed and shamed as a shill.
The lesson here is that charts that show popularity are hard to get right – they can be easily manipulated for fun or for profit. Anthony should be prepared to fight an escalating war against those that want to manipulate his charts. And the more popular the Hype Machine becomes, the bigger the target it will be for the hackers and the shills.
The Dissociated Mixes
Posted in fun, Music, remix, The Echo Nest, web services on June 30, 2009
Check out Adam Lindsay’s latest post on Dissociated Mixes. He’s got a pretty good collection of automatically shuffled songs that sound interesting and eerily different from the original. One example is this remixed audio/video of Beck’s Record Club cover of “Waiting for my Man” by The Velvet Underground and Nico:
Nothing but MJ …
Posted in Music, The Echo Nest on June 26, 2009
We’ve been listening nothing but Michael Jackson here today at the nest.
To help everyone remember how great MJ’s music is, Tristan put together a mix of MJ – 50 tracks in 25 minutes:
(== (+ “Clojure” “Echo Nest”) “woah!”)
Posted in code, Music, The Echo Nest on June 25, 2009
Here’s the first Echo Nest application (as far as I know) that is written in Clojure: Another reason I like Clojure
The Echo Nest Fanalytics
Posted in Music, recommendation, The Echo Nest on June 25, 2009
At the core of just about everything we do here at the Echo Nest is what we call “The Knowledge”. This is big pile of data that represents everything we know about music. To build ‘The Knowledge’ we crawl the web looking for every bit of info about music. We find music blogs, artist news, album reviews, biographies, audio, images, videos, fan activity and on and on. This gives us a huge set of raw data that represents the global conversation about music. Next, we apply a set of statistical and natural language processing algorithms to this raw data to give us a deeper understanding of what all this data means. For instance, one fundamental algorithm tells us whether a particular web document is about a particular artist. This might be easy for an artist with a distinctive name like Metallica, but may not be so easy for The Rolling Stones (is it the band or the magazine?), and can be hard for bands with ambiguous names like Air and Yes, and can be extremely difficult for artists such as Torsten Pröfrock who tragically has chosen the stage name ‘Various Artists‘ (what was he thinking?). Another algorithm that we apply to music reviews is sentiment analysis. This helps us decide whether or not a reviewer has a positive opinion about the music being reviewed. We can take a review like this one written by Jennie, my 14 year old daughter, and learn whether or not she likes the new album by Beyoncé and whether or not she tends to like R&B and pop music.
In addition to analyzing what people are writing about music, we also try to extract as much meaning as we can from the music itself. We apply digital signal processing and machine learning algorithms to audio allowing us to extract information such as tempo, key, song structure, loudness, energy, harmonic content and timbre from every song.
Traditionally, “The Knowledge” has helped us build tools to help music fans explore and discover music – using all this data helps us predict what type of music a listener might like. For the last year, we’ve offered artist similarity and music recommendation web services around this data. But now we are going to turn this all upside down. Instead of using this data to help listeners find new music, we are going to use this data to help artists find new fans. That is what Fanalytics is all about.
For example, music blogs and review sites are becoming increasingly important way for an artist to build buzz around a new release. However, there are thousands of music blogs – each with its own specialty. This becomes a problem for the artist. How can she decide which blogs she should target for promoting her new album? This is one of the problems that Fanalytics tries to solve. With ‘The Knowledge’ we know quite a bit about thousands of music blogs. We know the reputation and the reach of a blog. We know what types of a music a particular author tends to write about, and we know what kinds of music they tend to like. With this knowledge we can make what is essentially a recommendation engine for music promotion. For any artist we can recommend a set blogs and writers that would most likely be interested in writing about the artist.
In addition to this recommendation engine tailored to music promotion, Fanalytics also provides a set of analytics tools that use ‘The Knowledge’ to help artists better understand their audience. For instance, an artist can track everything that is being said online about them – every blog post, news item, music review, video, as well as their online ‘buzz’ – a quantitative measure of how much attention the artist is receiving from reviewers, bloggers, fans, etc.
We have just launched Fanalytics, but apparently we are already seeing strong interest from the labels. (According the press release Interscope, Independent Label Group (WMG), RCA Music Group (Sony) and The Orchard are already on board). That’s not too surprising, the labels are looking for new ways to reach out to fans. As we continue to grow “The Knowledge” here at the Echo Nest I’m sure we will be creating more interesting tools like Fanalytics that are built around the data .
New Echo Nest Java client released
Posted in code, java, Music, The Echo Nest, web services on June 24, 2009
We’ve just released version 1.1 of the Echo Nest Java Client. The Java Client makes it easy to access the Echo Nest APIs from a Java program. This release fixes some bugs and improves caching support. Here’s a snippet of Java code that shows how you can use the API to find similar artists for the band ‘Weezer’:
ArtistAPI artistAPI = new ArtistAPI(MY_ECHO_NEST_API_KEY);
List<Artist> artists = artistAPI.searchArtist("Weezer, false);
if (artists.size() > 0) {
for (Artist artist : artists) {
List<Scored<Artist>> similars =
artistAPI.getSimilarArtists(artist, 0, 10);
for (Scored<Artist> simArtist : similars) {
System.out.println(" " + simArtist.getItem());
}
}
}
Also included in the release is a command line shell that lets you interact with the Echo Nest API. You can start it up from the command line like so:
java -DDECHO_NEST_API_KEY=YOUR_API_KEY -jar EchoNestAPI.jar
Here’s an example session:
Welcome to The Echo Nest API Shell type 'help' nest% help 0) alias - adds a pseudonym or shorthand term for a command 1) chain - execute multiple commands on a single line 2) delay - pauses for a given number of seconds 3) echo - display a line of text 4) enid - gets the ENID for an arist 5) gc - performs garbage collection 6) getMaxCacheTime - gets the cache time 7) get_audio - gets audio for an artist 8) get_blogs - gets blogs for an artist 9) get_fam - gets familiarity for an artist 10) get_hot - gets hotttnesss for an artist 11) get_news - gets news for an artist 12) get_reviews - gets Reviews for an artist 13) get_similar - finds similar artists 14) get_similars - finds similar artists to a set of artists 15) get_urls - gets Reviews for an artist 16) get_video - gets video for an artist ( .. commands omitted ..) 53) trackTatums - gets the tatums of a track 54) trackTempo - gets the overall Tempo of a track 55) trackTimeSignature - gets the overall time signature of a track 56) trackUpload - uploads a track 57) trackUploadDir - uploads a directory of tracks 58) trackWait - waits for an analysis to be complete 59) version - displays version information nest% nest% get_similar weezer Similarity for Weezer 1.00 The Smashing Pumpkins 0.50 Ozma 0.33 Fountains of Wayne 0.25 Jimmy Eat World 0.20 Veruca Salt 0.17 The Breeders 0.14 Nerf Herder 0.13 The Flaming Lips 0.11 Death Cab for Cutie 0.10 Rivers Cuomo 0.09 The Rentals 0.08 Size 14 0.08 Nada Surf 0.07 Third Eye Blind 0.07 Chopper One nest% nest% get_fam Decemberists Familiarity for The Decemberists 0.8834854 nest% nest% trackUpload "09 When I'm Sixty-Four.MP3" ID: baad7cab21b853ea5ead4db0a12b1df8 nest% trackDuration Duration: 157.96104 nest% nest% trackTempo 140.571 (0.717) nest%
If you are interested in playing around with the Echo Nest API but don’t want to code up your own application, typing in webservice URLs by hand gets pretty old, pretty quickly. The Echo Nest shell gives you a simpler way to try things out.
Music HackDay is coming …
Posted in code, fun, Music, The Echo Nest, web services on June 23, 2009
If you live within a couple hundred miles of London, and you read this blog, then there’s no reason why you shouldn’t be planning on going to Music Hackday being held on July 11th and 12th at the Guardian offices in London. This is a great opportunity to connect with other developers that are creating next generation music applications, web sites, and gadgets. In addition to the developers, API providers will be showing off their wares (and some will even be unveiling new APIs). Companies include 7digital, Gigulate, Last.fm, People’s Music Store, Songkick, Soundcloud and The Echo Nest. Recently added to the agenda are workshops by Tinker.it and RjDj.
The Echo Nest will be there, represented by Adam Lindsay. He’ll guide you through using our various APIs including our artist recommendation APIs and our music analysis and remix APIs. Oh, and the developer that creates the coolest thing that uses the Echo Nest API will go home with a big, fat (i.e. 32gb) iPod touch.
Looking at the attendee list, the Music Hackday looks to be a who’s who in music tech – not only will it be a day of hacking, but it’s a great place to get to meet all of the folks that are creating the next generation of music apps. It looks like spaces are filling up quickly, so if you haven’t already registered, don’t dally, or you may miss out.
Where’s the Pow?
Posted in code, fun, Music, remix, The Echo Nest on June 21, 2009
This morning, while eating my Father’s day bagel, I got to play some more with the video aspects of the Echo Nest remix API. The video remix is pretty slick. You use all of the tools that you use in the audio remix, except that the object you are manipulating has a video component as well. This makes it easy to take an audio remix and turn it into a video remix. For instance, here’s the remix code to create a remix that includes the first beat of every bar:
audiofile = audio.LocalAudioFile(input_filename)
collect = audio.AudioQuantumList()
for bar in audiofile.analysis.bars:
collect.append(bar.children()[0])
out = audio.getpieces(audiofile, collect)
out.encode(output_filename)
To turn this into a video remix, just change the code to:
av = video.loadav(input_filename)
collect = audio.AudioQuantumList()
for bar in av.audio.analysis.bars:
collect.append(bar.children()[0])
out = video.getpieces(av, collect)
out.save(output_filename)
The code is nearly identical, differing in loading and saving, while the core remix logic stays the same.
To make a remix of a YouTube video, you need to save a local copy of the video. I’ve been using KeepVid to save local flv (flash video format) of any Youtube video.
Today I played with the track ‘Boom Boom Pow’ by the Black Eyed Peas. It’s a fun song for remix because it has a very strong beat, and already has a remix feel to it. And since the song is about digital transformation, it seems to be a good target for remix experiments. (and just maybe they won’t mind the liberties I’ve taken with their song).
Here’s the original (click through to YouTube to watch it since embedding is not allowed):
Just Boom
The first remix is to only include the first beat of every measure. The code is this:
for bar in av.audio.analysis.bars: collect.append(bar.children()[0])
Just Pow
Change the beat included from beat zero to beat three, and we get something that sounds very different:
Pow Boom Boom
Here’s a version with the beats reversed. The core logic for this transformation is one line of code:
av.audio.analysis.beats.reverse()
The 5/4 Version
Here’s a version that’s in 5/4 – to make this remix I duplicated the first beat and swapped beats 2 and 3. This is my favorite of the bunch.
These transformations are of the simplest variety, taking just a couple of minutes to code and try out. I’m sure some budding computational remixologist could do some really interesting things with this API.
Note that the latest video support is not in the main branch of remix. If you want to try some of this out you’ll need to check out the bl-video branch from the svn repository. But this is guaranteed to be rolled into the main branch before the upcoming Music Hackday. Update: the latest video support is now part of the main branch. If you want to try it out, check it out from the trunk of the SVN repository. So download the code, grab your API key and start remixing.
Update: As Brian pointed out in the comments there was some blocking on the remix renders. This has been fixed, so if you grab the latest code, the video output quality is as good as the input.
More confusing than Memento
Posted in code, fun, remix, The Echo Nest on June 20, 2009
Ben Lacker, one of our leading computational remixologists here at the Echo Nest has been improving the video remix capabilities of the Echo Nest remix API. On Friday, he remixed this mind blower. It’s Coldplay’s music video for ‘The Scientist’ – beat reversed, which means that song is played in reverse order beat by beat (but each beat is still played in forward order). Since Coldplay’s video is already shot in reverse order, the resulting video has a story that unfolds in proper chronological order, but where every second of video runs backwards, while the music unfolds in reverse chronological order while every beat runs forward. I get a little bit of a stomachache watching this video.
Ben has committed the code for this remix to the Echo Nest remix code samples so feel free to check it out and hack on it. I hope to see some more interesting music and video remixes coming out of the upcoming Music Hackday.
Music Machinery 