Archive for category web services
In his spare time, Echo Nest developer Reid Draper built hotttnesss.com – a neat web app that shows the top 50 hotttest artists (according to the Echo Nest get_top_hottt_artists) along with sparklines showing the historical hotttnesss for the last week. Reid used the nifty jquery sparklines plugin to make it happen. Mouse over an artist name to get links to the Last.fm and Spotify pages for the artist so you can find out what the big deal is about Broken Bells or lyaz.
While watching the Olympics over the weekend, I wrote a little web-app game that uses the new Echo Nest get_images call. The game is dead simple. You have to identify the artists in a series of images. You get to chose a level of difficulty and the style of your favorite music, and if you get a high score, your name and score will appear on the Top Scores board. Instead of using a simple score of percent correct, the score gets adjusted by a number of factors. There’s a time bonus, so if you answer fast you get more points, there’s a difficulty bonus, so if you identify unfamiliar artists you get more points, and if you chose the ‘Hard’ level of difficulty you get also get more points for every correct answer. The absolute highest score possible is 600 but that any score above 200 is rather awesome.
The app is extremely ugly (I’m a horrible designer), but it is fun – and it is interesting to see how similar artists from a single genre appear. Give it a go, post some high scores and let me know how you like it.
Here at The Echo Nest we want to make the world easier for music app developers. We want to solve as many of the problems that developers face when writing music apps so that the developers can focus on building cool stuff instead of worrying about the basic plumbing . One of the problems faced by music application developers is the issue of ID translation. You may have a collection of music that is in one ID space (Musicbrainz for instance) but you want to use a music service (such as the Echo Nest’s Artist Similarity API) that uses a completely different ID space. Before you can use the service you have to translate your Musicbrainz IDs into Echo Nest IDs, make the similarity call and then, since the artist similarity call returns Echo Nest IDs, you have to then map the IDs back into the Musicbrainz space. The mapping from one id space to another takes time (perhaps even requiring another API method call to ‘search_artists’) and is a potential source of error — mapping artist names can be tricky – for example there are artists like Duran Duran Duran, Various Artists (the electronic musician), DJ Donna Summer, and Nirvana (the 60’s UK band) that will trip up even sophisticated name resolvers.
We hope to eliminate some of the trouble with mapping IDs with Project Rosetta Stone. Project Rosetta Stone is an update to the Echo Nest APIs to support non-Echo-Nest identifiers. The goal for Project Rosetta Stone is to allow a developer to use any music id from any music API with the Echo Nest web services. For instance, if you have a Musicbrainz ID for weezer, you can call any of the Echo Nest artist methods with the Musicbrainz ID and get results. Additionally, methods that return IDs can be told to return them in different ID spaces. So, for example, you can call artist.get_similar and specify that you want the similar artist results to include Musicbrainz artist IDs.
Dealing with the many different music ID formats One of the issues we have to deal with when trying to support many ID spaces is that the IDs come in many shapes and sizes. Some IDs like Echo Nest and Musicbrainz are self-identifying URLs, (self-identifying means that you can tell what the ID space is and the type of the item being identified (whether it is an artist track, release, playlist etc.)) and some IDs (like Spotify) use self-identifying URNs. However, many ID spaces are non-self identifying – for instance a Napster Artist ID is just a simple integer. Note also that many of the ID spaces have multiple renderings of IDs. Echo Nest has short form IDs (AR7BGWD1187FB59CCB and TR123412876434), Spotify has URL-form IDs (http://open.spotify.com/artist/6S58b0fr8TkWrEHOH4tRVu) and Musicbrainz IDs are often represented with just the UUID fragment (bd0303a-f026-416f-a2d2-1d6ad65ffd68) – and note that the use of Spotify and Napster in these examples are just to demonstrate the wide range of ID format.
We want to make the all of the ID types be self-identifying. IDs that are already self-identifying can be used without change. However, non-self-identifying ID types need to be transformed into a URN-style syntax of the form: vendor:type:vendor-specific-id. So for example, and a Napster track ID would be of the form: ‘napster:track:12345678’
What do we support now? In this first release of Rosetta Stone we are supporting URN-style Musicbrainz ids (probably one of the most requested enhancements to the Echo Nest APIs has been to include support for Musicbrainz). This means that any Echo Nest API method that accepts or returns an Echo Nest ID can also take a Musicbrainz ID. For example to get recent audio found on the web for Weezer, you could make the call with the URN form of the musicbrainz ID for weezer:
http://developer.echonest.com/api/get_audio ?api_key=5ZAOMB3BUR8QUN4PE &id=musicbrainz:artist:6fe07aa5-fec0-4eca-a456-f29bff451b04 &rows=2&version=3 - (try it)
For a call such as artist.get_similar, if we are using Musicbrainz IDs for input, it is likely that you’ll want your results in the form of Musicbrainz ids. To do this, just add the bucket=id:musicbrainz parameter to indicate that you want Musicbrainz IDs included in the results:
http://developer.echonest.com/api/get_similar ?api_key=5ZAOMB3BUR8QUN4PE &id=musicbrainz:artist:6fe07aa5-fec0-4eca-a456-f29bff451b04 &rows=10&version=3 &bucket=id:musicbrainz (try it) <similar> <artist> <name>Death Cab for Cutie</name> <id>music://id.echonest.com/~/AR/ARSPUJF1187B9A14B8</id> <id type="musicbrainz">musicbrainz:artist:0039c7ae-e1a7-4a7d-9b49-0cbc716821a6</id> <rank>1</rank> </artist>
<!– more omitted –>
Limiting results to a particular ID space – sometimes you are working within a particular ID space and you only want to include items that are in that space. To support this, Rosetta Stone adds an idlimit parameter to some of the calls. If this is set to ‘Y’ then results are constrained to be within the given ID space. This means that if you want to guarantee that only Musicbrainz artists are returned from the get_top_hottt_artists call you can do so like this:
http://developer.echonest.com/api/get_top_hottt_artists ?api_key=5ZAOMB3BUR8QUN4PE &rows=20 &version=3 &bucket=id:musicbrainz &idlimit=Y
What’s Next? In this initial release of Rosetta Stone we’ve built the infrastructure for fast ID mapping. We are currently supporting mapping between Echo Nest Artist IDs and Musicbrainz IDs. We will be adding support for mapping at the track level soon – and keep an eye out for the addition of commercial ID spaces that will allow easy mapping being Echo Nest IDs and those associated with commercial music service providers.
In the near future we’ll be rolling out support to the various clients (pyechonest and the Java client API) to support Rosetta Stone.
As always, we love any feedback and suggestions to make writing music apps easier. So email me (email@example.com) or leave a comment here.
- search_tracks: This is IMHO the most awesomest method in the Echo Nest API. This method lets you search through the millions of tracks that the Echo Nest knows about. You can search for tracks based on artist and track title of course, but you can also search based upon how people describe the artist or track (‘funky jazz’, ‘punk cabaret’, ‘screamo’. You can constrain the return results based upon musical attributes (range of tempo, range of loudness, the key/mode), you can even constrain the results based upon the geo-location of the artist. Finally, you can specify how you want the search results ordered. You can sort the results by tempo, loudness, key, mode, and even lat/long.This new method lets you fashion all sorts of interesting queries like:
- Find the slowest songs by Radiohead
- Find the loudest romantic songs
- Find the northernmost rendition of a reggae track
The index of tracks for this API is already quite large, and will continue to grow as we add more music to the Echo Nest. (but note, that this is an alpha version and thus it is subject to the whims of the alpha-god – even as I write this the index used to serve up these queries is being rebuilt so only a small fraction of our set of tracks are currently visible). And BTW if you are at the Stockholm Music Hack Day, look for Brian and ask him about the secret parameter that will give you some special search_tracks goodness!
One of the things you get back from the search_tracks method is a track ID. You can use this track ID to get the analysis for any track using the new get_analysis method. No longer do you need to upload a track to get the analysis for it. Just search for it and we are likely to have the analysis already. This search_tracks method has been the most frequently requested method by our developers, so I’m excited to see this method be released.
- get_analysis – this method will give you the full track analysis for any track, given its track ID. The method couldn’t be simpler, give it a track ID and you get back a big wad-o-json. All of the track analysis, with one call. (Note that for this alpha release, we have a separate track ID space from the main APIs, so IDs for tracks that you’ve analyzed with the released/supported APIs won’t necessarily be available with this method).
- capsule – this is an API that supports this-is-my-jam functionality. Give the API a URL to an XSPF playlist and you’ll get back some json that points you to both a flashplayer url and an mp3 url to a capsulized version of the playlist. In the capsulized version, the song transitions are aligned and beatmatched like an old style DJ would.
Brian also describes a new identify_track method that returns metadata for a track given the Echo Nest a set of musical fingerprint hashcodes. This is a method that you use in conjunction with the new Echo Nest audio fingerprinter (woah!). If you are at the Stockholm music hackday and you are interested in solving the track resolution problem talk to Brian about getting access to the new and nifty audio fingerprinter.
These new APIs are still in alpha – so lots of caveats surround them. To quote Brian: we may pull or throttle access to alpha APIs at a different rate from the supported ones. Please be warned that these are not production ready, we will be making enhancements and restarting servers, there will be guaranteed downtime.
The new APIs hint at the direction we are going here at the Echo Nest. We want to continue to open up our huge quantities of data for developers, making as much of it available as we can to anyone who wants to build music apps. These new APIs return JSON – XML is so old fashioned. All the cool developers are using JSON as the data transport mechanism nowadays: its easy to generate, easy to parse and makes for a very nimble way to work with web-services. We’ll be adding JSON support to all of our released APIs soon.
I’m also really excited about the new fingerprinting technology. Here at the Echo Nest we know how hard it is to deal with artist and track resolution – and we want to solve this problem once and for all, for everybody – so we will soon be releasing an audio fingerprinting system. We want to make this system as open as we can, so we’ll make all the FP data available to anyone. No secret hash-to-ID algorithms, and no private datasets. The Fingerprinter is fast, uses state-of-the-art audio analysis and will be backed by a dataset of fingerprint hashcodes for millions of tracks. I’ll be writing more about the new fingerprinter soon.
These new APIs should give those lucky enough to be in Stockholm this weekend something fun to play with. If you are at the Stockholm Hack Day and you build something cool with these new APIs you may find yourself going home with the much coveted Echo Nest sweatsedo:
Worth checking out: Normalisr
I’ll be giving a workshop on the Echo Nest API at the Boston Music Hack Day. Here are the slides – but you should really come to the workshop if you can – the slides don’t have all the music, video or presenter awesomeness that you’ll get at the live workshop. Hope to see you there.
We’ve been extremely busy this week at the Echo Nest getting ready for the Boston Music Hack Day. Not only have we been figuring out menus, panel room assignments, and dealing with a waitlist, we’ve also been releasing a set of new API features. Here’s a quick rundown of what we’ve done:
- get_images – a frequent request from developers – we now have an API method that will let you get images for an artist. Note that we are releasing this method as a sneak preview for the hack day – we have images for over 60 thousand artists, but we will be aggressively adding more images over the next few weeks (60 thousand artists is a lot of artists, but we’d like to have lots more). We’ll also be expanding our sources of images to include many more sources. The results of the get_images are already good. 95% of the time you’ll get images. Over the next few weeks, the results will get even better.
- get_biographies – another frequent request from developers – we now have a get_biographies API method that will return a set of artist biographies for any artist. We currently have biographies for about a quarter million artists – and just as with get_images – we are working hard to expand the breadth and depth of this coverage. Nevertheless, with coverage for a quarter million artists, 99.99% of the time when you ask for a biography we’ll have it.
- get_similar – we’ve expanded the number of similar artists you can get back from get_similar from 15 to 100. This gives you lots more info for building playlisting and music discovery apps.
- buckets – one issue that our developers have had was that to fill out info on an artist often took a number of calls to the Echo Nest – one to get similars, one to get audio, one for video, familiarity, hotttnesss etc. To fill out an artist page it could take half a dozen calls. To reduce the number of calls needed to get artist information we’ve added a ‘bucket’ parameter to the search_artist, the get_similar and the get_profile calls. The bucket parameter allows you to specify which additional artist info should be returned in the call. You can specify ‘audio,’ ‘biographies,’ ‘blogs,’ ‘familiarity,’ ‘hotttnesss,’ ‘news,’ ‘reviews,’ ‘urls,’, ‘images’ or ‘video’ and whenever you get artist data back you’ll get the specified info included. For example with the call:
http://developer.echonest.com/api/get_profile ?api_key=EHY4JJEGIOFA1RCJP &id=music://id.echonest.com/~/AR/ARH6W4X1187B99274F &version=3 &bucket=familiarity &bucket=hotttnesss
will return an artist block that looks like this:
<artist> <name>Radiohead</name> <id>music://id.echonest.com/~/AR/ARH6W4X1187B99274F</id> <familiarity>0.899230928024</familiarity> <hotttnesss>0.847409181874</hotttnesss> </artist>
There’s another new feature that we are starting to roll out. It’s called Echo Source – it allows the developer to get content (such as images, audio, video etc.) based upon license info. Echo Source is a big deal and deserves a whole post – but that’s going to have to wait until after Music Hack Day. Suffice it to say that with Echo Source you’ll have a new level of control over what content the Echo Nest API returns.