Posts Tagged Music

The Echo Nest Fanalytics

en_logo_250x200_lt1At the core of  just about everything we do here at the Echo Nest is what we call “The Knowledge”.  This is big pile of data that represents everything we know about music.   To build ‘The Knowledge’ we crawl the web looking for every bit of info about music. We find music blogs,  artist news, album reviews, biographies, audio,  images, videos,  fan activity and on and on.  This gives us a huge set of raw data that represents the global conversation about music.  Next,  we apply a set of statistical and natural language processing algorithms to this raw data to  give us a deeper understanding of what all this data means.  For instance,  one fundamental algorithm tells us whether a particular web document is about a particular artist. This might be easy for an artist with a distinctive name like Metallica, but may not be so easy for The Rolling Stones (is it the band or the magazine?), and can be hard for bands with ambiguous names like Air and Yes, and can be extremely difficult for artists such as Torsten Pröfrock  who tragically has chosen the stage name ‘Various Artists‘ (what was he thinking?).   Another algorithm that we apply to music reviews is sentiment analysis.  This helps us decide whether or not a reviewer has a positive opinion about the music being reviewed.  We can take a review like this one written by  Jennie, my 14 year old daughter, and learn whether or not she likes the new album by Beyoncé and whether or not she tends to like R&B and pop music.

In addition to analyzing what people are writing about music, we also try to extract as much meaning as we can from the music itself.  We apply digital signal processing and machine learning algorithms to audio allowing us to extract information such as tempo, key, song structure, loudness, energy, harmonic content and timbre from every song.

fanalyticsTraditionally, “The Knowledge” has helped us build tools to help music fans explore and discover music – using all this data helps us predict what type of music a listener might like.  For the last year, we’ve offered artist similarity and music recommendation web services around this data.  But now we are going to turn this all upside down.  Instead of using this data to help listeners find new music, we are going to use this data to help artists find new fans.  That is what Fanalytics is all about.

For example, music blogs and review sites are becoming increasingly important way for an artist to build buzz around a new release.  However, there are  thousands of music blogs – each with its own specialty.  This becomes a problem for the artist.  How can she decide which blogs she should target for promoting her new album?  This is one of the problems that Fanalytics tries to solve.  With ‘The Knowledge’ we know quite a bit about thousands of music blogs.  We know the reputation and the reach of a blog.  We know what types of a music a particular author tends to write about, and we know what kinds of music they tend to like.  With this knowledge we can make what is essentially a recommendation engine for music promotion.  For any artist we can recommend a set blogs and writers that would most likely be interested in writing about the artist.

In addition to this recommendation engine tailored to music promotion, Fanalytics also provides  a set of analytics tools that use ‘The Knowledge’ to help artists better understand their audience.   For instance,  an artist can track everything that is being said online about them – every blog post, news item, music review, video, as well as their online ‘buzz’ – a quantitative measure of how much attention the artist is receiving from reviewers, bloggers, fans, etc.

We have just launched Fanalytics, but apparently we are already seeing strong interest from the labels. (According the press release Interscope,  Independent Label Group (WMG), RCA Music Group (Sony) and The Orchard are already on board).  That’s not too surprising, the labels are looking for new ways to reach out to fans.  As we continue to grow “The Knowledge”  here at the Echo Nest I’m sure we will be creating more  interesting tools like Fanalytics that are built around the data .


, , ,

3 Comments

Music HackDay is coming …

If you live within a couple hundred miles of London, and you read this blog, then there’s no reason why you shouldn’t be planning on going to Music Hackday being held on July 11th and 12th at the Guardian offices in London.   This is a great opportunity to connect with other developers that are creating next generation music applications, web sites, and gadgets.  In addition to the developers,  API providers will be showing off their wares (and some will even be unveiling new APIs).  Companies include 7digital, Gigulate, Last.fm, People’s Music Store, Songkick, Soundcloud and The Echo Nest.    Recently added to the agenda are workshops by  Tinker.it and RjDj.

The Echo Nest will be there, represented by Adam Lindsay. He’ll guide you through using our various APIs including our artist recommendation APIs and our music analysis and remix APIs.  Oh, and the developer that creates the coolest thing that uses the Echo Nest API will go home with a big, fat (i.e. 32gb) iPod touch.

Looking at the attendee list,  the Music Hackday looks to be a who’s who in music tech –  not only will it be a day of hacking, but it’s a great place to  get to meet all of the folks that are creating the next generation of music apps.  It looks like spaces are filling up quickly, so if you haven’t already registered, don’t dally, or you may miss out.

, ,

2 Comments

Where’s the Pow?

This morning, while eating my Father’s day bagel, I got to play some more with the video aspects of the Echo Nest remix API.  The video remix is pretty slick.  You use all of the tools that you use in the audio remix, except that the object you are manipulating has a video component as well.    This makes it easy to take an audio remix and turn it into a video remix.  For instance, here’s the remix code to create a remix that includes the first beat of every bar:

 audiofile = audio.LocalAudioFile(input_filename)
 collect = audio.AudioQuantumList()
 for bar in audiofile.analysis.bars:
     collect.append(bar.children()[0])
 out = audio.getpieces(audiofile, collect)
 out.encode(output_filename)

To turn this into a video remix, just change the code to:

 av = video.loadav(input_filename)
 collect = audio.AudioQuantumList()
 for bar in av.audio.analysis.bars:
     collect.append(bar.children()[0])
 out = video.getpieces(av, collect)
 out.save(output_filename)

The code is nearly identical, differing in loading and saving, while the core remix logic stays the same.

To make a remix of a YouTube video, you need to save a local copy of the video.   I’ve been using KeepVid to save local flv (flash video format) of any Youtube video.

Today I played with the track ‘Boom Boom Pow’ by the Black Eyed Peas.  It’s a fun song for remix because it has a very strong beat, and already has a remix feel to it.  And since the song is about digital transformation, it seems to be a good target for remix experiments.  (and just maybe they won’t mind the liberties I’ve taken with their song).

Here’s the original (click through to YouTube to watch it since embedding is not allowed):

Just Boom

The first remix is to only include the first beat of every measure.   The code is this:

    for bar in av.audio.analysis.bars:
         collect.append(bar.children()[0])

Just Pow

Change the beat included from beat zero to beat three, and we get something that sounds very different:

Pow Boom Boom

Here’s a version with the beats reversed.  The core logic for this transformation is one line of code:

av.audio.analysis.beats.reverse()

The 5/4 Version

Here’s a version that’s in 5/4 – to make this remix I duplicated the first beat and swapped beats 2 and 3.  This is my favorite of the bunch.

These transformations are of the simplest variety, taking just a couple of minutes to code and try out.   I’m sure some budding computational remixologist could do some really interesting things with this API.

Note that the latest video support is not in the main branch of remix.  If you want to try some of this out you’ll need to check out the bl-video branch from the svn repository.     But this is guaranteed to be rolled into the main branch before the upcoming Music Hackday. Update: the latest video support is now part of the main branch.  If you want to try it out, check it out from the trunk of the SVN repository. So download the code, grab your API key and start remixing.

Update: As Brian pointed out in the comments there was some blocking on the remix renders. This has been fixed, so if you grab the latest code, the video output quality is as good as the input.

, , , ,

14 Comments

More confusing than Memento

Ben Lacker, one of our leading computational remixologists here at the Echo Nest has been improving the video remix capabilities of the Echo Nest remix API.   On Friday, he remixed this mind blower.  It’s Coldplay’s music video for ‘The Scientist’ – beat reversed, which means that song is played in reverse order beat by beat (but each  beat is still played in forward order).    Since Coldplay’s video is already shot in reverse order, the resulting video has a story that unfolds in proper chronological order, but where every second of video runs backwards, while the music unfolds in reverse chronological order while every beat runs forward.  I get a little bit of a stomachache watching this video.

Ben has committed the code for this remix to the Echo Nest remix code samples so feel free to check it out and hack on it.    I hope to see some more interesting music and video remixes coming out of the upcoming Music Hackday.

, , , ,

5 Comments

The Passion Index

One of the ways that Music 2.0 has changed how we think about music is that there is so much interesting data available about how people are listening to music.  Sites like Last.fm automatically track all sorts of interesting data that just was not available before.  Forty years ago, a music label like Capitol would know how many copies the album  Abbey Road sold in the U.S., but the label wouldn’t know how many times people actually listened to the album.  Today, however, our iPods and desktop music players keep careful track of how many times we play each song,  album and artist – giving us a whole new way to look at artist popularity.  beatles-countIt’s not just sales figures anymore, its how often are people actually listening to an artist.  If you go to Last.fm you can see that The Beatles have over  1.75 million listeners and 168 million plays.  It makes it easy for us to see how popular the Beatles are compared to another band (the monkees, for instance have 2.5m plays and 285K listeners).

With all of this new data available, there are some new ways we can look at artists.  Instead of just looking at artists in terms of popularity and sales rank,  I think it is interesting to see which artists generate the most passionate listeners.  These are artists that dominate the playlists of their fans.   I think this ‘passion index’ may be an interesting metric to use to help people explore for and discovery music.  Artists that attract passionate fans may be longer lived and worth  a listeners investment in time and money.

How can we calculate a passion index?   There are probably a number of indicators:  the number of edits to the bands wikipedia page,  the average distance a fan travels to attend a show by the artist, the number of fan sites for an artist.  All of these may be a bit difficult to collect, especially for a large set of artists.  One  simple passion metric is just  the average number of artist plays per listener.  Presumably if an artist’s listeners are playing an artist’s songs more than average they are more passionate about the artist.   One thing that I like about this approach to the passion index is that it is extremely easy to calculate – just divide the total artist plays by the total number of artist listeners and you have the passion index.   Yes, there are many confounding factors – for instance,  artists with longer songs are penalized – still I think it is a pretty good measure.

I calculated the passion index for a large collection of artists.  I started with about a million artists (it is really nice to have all this data at the Echo Nest;), and filtered these down to the 50K most popular artists.  I plotted the number of artist plays vs. the number of artist listeners for each of the 50 K listeners.    The plot shows that most artists fall into the central band (normal passion), but some (the green points) are high passion artists and some (the blue points) are low passion artists.

passion

For the 50K artists, the average track plays per artist/listener is just 11 plays (with a std deviation of about 11.5).  Considering that there are a substantial number of artists in my iTunes collection that I’ve played only once, this seems pretty resaonable.

So who are the artists with the highest passion index?   Here are the top ten:

Passion Listeners Plays Artist
332 4065 1352719 上海アリス幻樂団
292 10374 3032373 Belo
245 3147 773959 Petos
241 2829 683191 Reilukerho
208 4887 1020538 Sound Horizon
190 24422 4652968 동방신기
185 9133 1691866 岡崎律子
175 9171 1611106 Kollegah
173 17279 3004410 Super Junior
170 62592 10662940 Böhse Onkelz

I didn’t recognize any of these artists (and I’m not even sure if 上海アリス幻樂団 is really an artist – according to the Japanese wikipedia it is a fan club in Japan belo.1to produce a music game coterie – whatever that means).   Belo is a Brazilian pop artist that does indeed seem to have some rather passionate fans.

It is not surprising that it is hard for popular artists to rank at the very top of the  passion index.  Popular artists are exposed to many, many listeners which can easily reduce the passion index.    Here are the top passion-ranked artists drawn from the top-1000 most popular artists:

Passion Listeners Plays Artist
115 527653 60978053 In Flames
95 1748159 167765187 The Beatles
79 2140659 170106143 Radiohead
78 282308 22071498 Die Ärzte
75 269052 20293399 Mindless Self Indulgence
75 691100 52217023 Nightwish
74 332658 24645786 Porcupine Tree
74 1056834 79135038 Nine Inch Nails
72 384574 27901385 Opeth
70 601587 42563097 Rise Against
69 357317 24911669 Sonata Arctica
69 1364096 95399150 Metallica
66 460518 30625121 Children of Bodom
66 619396 41440369 Paramore
65 504464 33271871 Dream Theater
65 1391809 90888046 Pink Floyd
64 540184 34635084 Brand New
62 862468 54094977 Iron Maiden
62 1681914 105935202 Muse
61 381942 23478290 Beirut

I find it interesting to see all of the heavy metal bands in the top 20. Metal fans are indeed true fans.

Going to the other end of passion, we find the 20 popular artists that have the least passionate fans:

Passion Listeners Plays Artist
6 270692 1767977 Julie London
6 284087 1964292 Smoke City
6 294100 1784358 Dinah Washington
6 295200 1799303 The Bangles
6 295990 1832771 Donna Summer
6 306018 1905285 Bonnie Tyler
6 307407 2123599 Buffalo Springfield
6 311543 2085085 Franz Schubert
6 312078 1909769 The Hollies
6 313732 2190008 Tom Jones
6 325454 2025366 Eric Prydz
6 331837 2259892 Sarah Vaughan
6 332072 2016898 Soft Cell
6 407622 2622570 Steppenwolf
5 275770 1605268 Diana Ross
5 281037 1615125 Isaac Hayes
5 282095 1685959 The Isley Brothers
5 283467 1666824 Survivor
5 311867 1694947 Peggy Lee
5 333437 1925611 Wham!
5 388183 2244878 Kool & The Gang

I guess people are not too passionate about Soft Cell.

Here’s a passion chart for the top 100 most popular artists. Even the artists at the bottom of this chart are way above average on the passion index.

Passion Listeners Plays Artist
95 1748159 167765187 The Beatles
79 2140659 170106143 Radiohead
74 1056834 79135038 Nine Inch Nails
69 1364096 95399150 Metallica
65 1391809 90888046 Pink Floyd
62 1681914 105935202 Muse
61 1397442 85685015 System of a Down
61 1403951 86849524 Linkin Park
60 1346298 81762621 Death Cab for Cutie
57 1060269 61127025 Fall Out Boy
56 1155877 65324424 Arctic Monkeys
55 1897332 104932225 Red Hot Chili Peppers
54 950416 52019102 My Chemical Romance
50 1131952 56622835 blink-182
49 2313815 115653456 Coldplay
48 964970 47102550 Sigur Rós
48 1108397 53260614 Modest Mouse
48 1350931 65865988 Placebo
47 1129004 53771343 Jack Johnson
44 1297020 57111763 Led Zeppelin
43 1011131 43930085 Kings of Leon
42 947904 39970477 Marilyn Manson
42 1065375 45459226 Britney Spears
42 1246213 52656343 Incubus
42 1256717 53610102 Bob Dylan
41 1527721 62654675 Green Day
41 1881718 78473290 The Killers
40 1023666 41288978 Queens of the Stone Age
40 1057539 42472755 Kanye West
40 1108044 44845176 Interpol
40 1247838 49914554 Depeche Mode
40 1318140 53594021 Bloc Party
39 1266502 49492511 The White Stripes
38 1048025 40174997 Evanescence
38 1091324 42195854 Pearl Jam
38 1734180 67541885 Nirvana
37 978342 36561552 The Kooks
37 1097968 41046538 The Shins
37 1114190 42051787 The Offspring
37 1379096 51313607 The Cure
37 1566660 58923515 Foo Fighters
36 1326946 48738588 The Smashing Pumpkins
35 1091278 39194471 Björk
35 1271334 45619688 The Strokes
34 955876 33376744 Jimmy Eat World
34 1251461 42949597 Daft Punk
33 989230 33257150 Pixies
33 1012060 34225186 Eminem
33 1051836 35529878 Avril Lavigne
33 1110087 36785736 Johnny Cash
33 1121138 37645208 AC/DC
33 1161536 38615571 Air
32 961327 31286528 The Prodigy
32 1038491 33270172 Amy Winehouse
32 1410438 45614720 David Bowie
32 1641475 52612972 Oasis
32 1693023 54971351 U2
31 1258854 39598249 Madonna
31 1622198 51669720 Queen
30 1032223 31750683 Portishead
30 1178755 35600916 Rage Against the Machine
30 1249417 38284572 The Doors
30 1393406 42717325 Beck
29 1030982 30044419 Yeah Yeah Yeahs
29 1187160 34712193 Massive Attack
29 1348662 39131095 Weezer
29 1361510 39753640 Snow Patrol
28 985715 28485679 The Postal Service
28 1045205 30105531 The Clash
28 1305984 37807059 Guns N’ Roses
28 1532003 43998517 Franz Ferdinand
27 1000950 27262441 Nickelback
27 1395278 37856776 Gorillaz
26 1503035 40161219 The Rolling Stones
25 1345571 33741254 R.E.M.
24 1311410 32588864 Moby
23 973319 22962953 Audioslave
23 976745 22557111 3 Doors Down
23 1123549 26696878 Keane
22 998933 21995497 Justin Timberlake
22 1025990 23145062 Rihanna
22 1109529 24687603 Maroon 5
22 1120968 24796436 Jimi Hendrix
22 1160410 26641513 [unknown]
21 1151225 25081110 The Who
20 1057288 22084785 The Chemical Brothers
20 1105159 22925198 Kaiser Chiefs
20 1117306 22390847 Nelly Furtado
20 1201937 25019675 Aerosmith
20 1253613 25582503 Blur
19 968885 19219364 Simon & Garfunkel
19 974687 18528890 Christina Aguilera
19 1025305 20157209 The Cranberries
19 1144816 22252304 Michael Jackson
16 996649 16234996 Black Eyed Peas
16 1019886 16618386 Eric Clapton
15 980141 15317182 The Police
15 981451 15289554 Dido
14 973520 13781896 Elton John
13 949742 12624027 The Verve

I think it would be really interesting to incorporate the passion index into a recommender, so instead of just recommending artists that are similar to artists that a listener already likes, filter the similar artists with  a passion filter and offer up the artists that listeners are most passionate about. I think these recommendations would be more valuable to the listener.

, , ,

26 Comments

Remix 1.1 is released

Version 1.1 of the Echo Nest remix has been released.  Adam Lindsay, in his Remix Overview describes it thus:

Remix is a sophisticated tool to allow you to quickly, expressively, and intuitively chop up existing audio content and create new content based on the old. It allows you to reach inside the music, and let the music’s own musical qualities be your — or your computer’s — guide in finding something new in the old. By using Remix’s knowledge of a given song’s structure, you can render the familiar strange, or the strange slightly more familiar-sounding. You can create countless parameterized variations of a given song — or one of near-limitless length — that respect or desecrate the original, or land on any of countless steps in between.

This release as concentrated on making it easier to install. We now have install instructions for Linux, Mac and Windows.   We also now use the FFMpeg encoder/decoder instead of mad and lame.  This has a number of advantages; it makes it easier to install, it supports a larger number of file formats, and perhaps most importantly, it is the same decoder that the Echo nest Analyze uses. This ensures that audio segment boundaries fall exactly on zero-crossings.

Remix is really fun to play with, and the results are always interesting and sometimes even musical.  Here’s an example of a song released in the last year (can you guess it?) that has been remixed to include only the first beat of each measure.

, ,

3 Comments

Music Explorer FX

Sten has posted a link to his super nifty Music  Explorer FX.  Music Explorer FX is  a Java Fx application for exploring and discovering music.  In some ways, the application is like a much slicker version of  Music Plasma or Musicovery.  You can explore a particular neighborhood in the music world – looking at artist photos and videos, listening to music, reading reviews and blog posts, and following paths to similar artists.    It’s a very engaging application that makes it easy to learn about new bands.    I especially like the image gallery mode – when I find a band that I think might be interesting, I hit the play button to listen to their music, and then enter the image gallery to get a slide show of the band playing.  Here’s an example of ‘Pull Tiger Tail’ – a band that I just learned about today while exploring with MEFX.

mefx

Sten uses a number of APIs to make MEFX happen. He uses the Echo Nest for artist search and to get all sorts of info including artist familiarity, hotness, artist similarity, blogs, news, reviews and audio. He gets artist images from Flickr and Last.fm – and just to make sure he’s relevant in this Twitter-centric world, he uses the Twitter API to let you tweet about any interesting paths you’ve taken through the music space.

We are living in a remarkable world now – there’s such an incredible amount of music available. There are millions of artists creating music in all styles.  The challenge for today’s music listener is to find a way to navigate through this music space to find music that they will like.  Traditional music recommenders can help, but I really think that applications like the MEFX that enable exploration of the music space are going to be important tools for the adventurous music listener

, , , , ,

2 Comments

The end of an era at Last.fm

RJ just announced that he, along with the other two founders of Last.fm are leaving Last.fm. Details in this post:   Message from the Last.fm founders, Felix, RJ and Martin.  This is a big deal.  RJ, Felix and Martin laid out the roadmap that just about every music 2.0 company would follow.   They continuously  brought new innovations to music discovery that are now standard for music sites, innovations like  scrobbling of music taste,   web services to allow access to all of their music data, social tagging of music,  recommendation radio, to name just a few.

I hope that the folks remaining at Last.fm will keep the vision of Felix, RJ and Martin alive, and I wish Felix, RJ and Martin the very best in their next venture(s).  (I guess it is time to pay really close attention to playdar).

, ,

3 Comments

A glimpse of the Music Explorer

I noticed that Sten posted a photo from the JavaOne pavillion that happens to show the Music Explorer FX in the background.

Sten at JavaOne

Sten at JavaOne

Using my photo-forensic skills, I’ve extracted a detail that shows the Music Explorer FX:

mefx-detail.1

Here you can see Franz Ferdinand, surrounded by a set of similar artists, as well as a audio player,  along with graphics that show familiarity and hotness (all derived from the Echo Nest).   The Music Explorer FX is pretty neat. I hope Sten releases it soon.

, ,

3 Comments

The first music app in the Java App Store?

At the JavaOne keynote this morning, James Gosling and Jonathan Schwartz gave a demo of the new Java App store in front of 20,000 Java developers.    The Java App store is new online store for Java apps.  It’s just like the iPhone App store for Java.  Oh .. and it has about a billion potential shoppers.  One of the very first apps in the store  (and as far as I can tell, the only music app) is an application called Music Explorer FX.  This is a soon to be released Java FX application designed to help you explore the world of music.  I’ve had the opportunity to play with the Music Explorer – it is really quite cool. (and it makes heavy use of the Echo Nest API, which makes it doubly cool).  The developer of the application, Sten Anderson has written a teaser about the app on his blog.  As he says, stay tuned for to find out when the app will be available.

, , , , , ,

Leave a comment