Posts Tagged api
Finding artist names in text
Let’s say you have a block of text – perhaps a tweet or a web page from a music review site. If you want to find out if the text mentions a particular artist such as Weezer, it is a pretty straightforward task: Just search through the text for the artist name and all the variants and aliases for that artist. It is pretty easy.
What is harder is trying to figure out if any artists are mentioned in a block of text, and if so, which ones. Since there are millions of artists, each with their own set of aliases and variants, the simple search that we use to find ‘Weezer’ in a tweet doesn’t work so well. The fact that many artist names are also common words adds to the difficulty.
Luckily I work with a bunch of really smart folks at The Echo Nest who’ve already had to solve this problem in order to make The Echo Nest work. Over on the Echo Nest blog, there’s a nifty description of the problem of artist name identification and extraction and an announcement of the release of a new (and very much beta) API called artist/extract that will expose some of this functionality to application developers that use our APIs.
This morning I spent a few minutes and created a little web app that lets you play with the artist/extract API. Here’s a screenshot:
In this example I’ve typed in the text:
I like Deerhoof, and Emerson, Lake and Palmer. I don’t like Coldplay, or Justin Bieber. GNR is OK. Go try it yourself!
You can see that it found Deerhoof and Coldplay, (easy enough), and a spelling variant of Emerson, Lake & Palmer. It also recognized GNR as two bands – GNR (a Portuguese rock band), and as a nickname for Guns N’ Roses. Also notice that it didn’t get confused by the mention of ‘ OK. Go’ that is embedded in there. The extractor is not always perfect – it tries hard to avoid confusing artists with regular English words (since just about every English word is a band name), so it will rely on letter case and other hints to try to separate real artist mentions from accidental ones.
The artist extractor is very much a beta api so it may be a bit unsteady on its feet and may sometimes not work as you’d expect it to. Nevertheless, it is a nifty bit of music data infrastructure that will help us understand better who is talking about what artists.
Read the API docs for Artist/Extract – or try out the little web demo.
What’s the TTKP?
Posted by Paul in data, fun, Music, web services on November 9, 2010
Whenever Jennie and I are in the car together, we will listen to the local Top-40 radio station (KISS 108). One top-40 artist that i can recognize reliably is Katy Perry. It seems like we can’t drive very far before we are listening to Teenage Dreams, Firework or California Gurls. That got me wondering what the average Time To Katy Perry (TTKP) was on the station and how it compared to other radio stations. So I fired up my Python interpreter, wrote some code to pull the data from the fabulous YES api and answer this very important question. With the YES API I can get the timestamped song plays for a station for the last 7 days. I gathered this data from WXKS (Kiss 108), did some calculations to come up with this data:
- Total songs played per week: 1,336
- Total unique songs: 184
- Total unique artists: 107
- Average songs per hour: 7
- Number of Katy Perry plays: 76
- Median Time between Katy Perry songs: 1hour 18 minutes
That means the average Time to Katy Perry is about 39 minutes.
Katy Perry is only the fourth most played artist on KISS 108. Here are the stats for the top 10:
| Artist | Plays | Median time between plays |
Average time to next play |
|---|---|---|---|
| Taio Cruz | 84 | 1:07 | 0:34 |
| Rihanna | 80 | 1:27 | 0:44 |
| Usher | 79 | 1:20 | 0:40 |
| Katy Perry | 76 | 1:18 | 0:39 |
| Bruno Mars | 73 | 1:30 | 0:45 |
| Nelly | 56 | 1:44 | 0:52 |
| Mike Posner | 56 | 1:57 | 0:59 |
| Pink | 47 | 2:20 | 1:10 |
| Lady Gaga | 47 | 1:59 | 1:00 |
| Taylor Swift | 41 | 2:17 | 1:09 |
I took a look at some of the other top-40 stations around the country to see which has the lowest TTKP:
| Station | Songs Per Hour | TTKP |
|---|---|---|
| KIIS – LA’s #1 hit music station | 8 | 39 mins |
| WHTZ- New York’s #1 hit music station | 9 | 48 mins |
| WXKS- Boston’s #1 hit music station | 7 | 39 mins |
| WSTR- Atlanta – Always #1 for Today’s Hit Music | 8 | 38 mins |
| KAMP- 97.1 Amp Radio – Los Angeles | 11 | 38 mins |
| KCHZ- 95.7 – The Beat of Kansas City | 11 | 32 mins |
| WFLZ- 93.3 – Tampa Bay’s Hit Music channe | 9 | 39 mins |
| KREV- 92.7 – The Revolution – San Francisco | 11 | 36 mins |
So, no matter where you are, if you have a radio, you can tune into the local top-40 radio station, and you’ll need to wait, on average, only about 40 minutes until a Katy Perry song comes on. Good to know.
Jennie’s ultimate road trip
Posted by Paul in code, fun, Music, The Echo Nest on October 20, 2010
Last weekend at Music Hack Day Boston, I teamed up with Jennie, my 15-year-old daughter, to build her idea for a music hack which we’ve called Jennie’s Ultimate Road Trip. The hack helps you plan a road trip so that you’ll maximize the number of great concerts you can attend along the way. You give the app your starting and ending city, your starting and ending dates, and the names of some of your favorite artists and Jennie’s Ultimate Road Trip will search through the many events to find the ones that fit your route schedule that you’d like to see and gives you an itinerary and map.
We used the wonderful SongKick API to grab events for all the nearby cities. I was quite surprised at the how many events SongKick would find. For just a single week, in the geographic area between Boston and New York City, SongKick found 1,161 events with 2,168 different artists. More events and more artists makes it easier to find a route that will give a satisfying set of concerts – but it can also make finding a route a bit more computationally challenging too (more on that later). Once we had the set of possible artists that we could visit, we needed to narrow down the list of artists to the ones would be of most interest to the user. To do this we used the new Personal Catalogs feature of the Echo Nest API. We created a personal catalog containing all of the potential artists (so for our trip to NYC from Boston, we’d create a catalog of 2,168 artists). We then used the Echo Nest artist similarity APIs to get recommendations for artists within this catalog. This yielded us a set of 200 artists that best match the user’s taste that would be playing in the area.
The next bit was the tricky bit – first, we subsetted the events to just include events for the recommended set of artists. Then we had to build the optimal route through the events, considering the date and time of the event, the preference the user has for the artist, whether or not we’ve already been to an event for this artist on the trip, how far out of our way the venue is from our ultimate destination and how far the event is from our previous destination. For anyone who saw me looking grouchy on Sunday morning during the hack day it was because it was hard trying to figure out a good cost function that would weigh all of these factors: artist preference, travel time and distance between shows, event history. The computer science folks who read this blog will recognize that this route finding is similar to the ‘travelling salesman problem‘ – but with a twist, instead of finding a route between cities, which don’t tend to move around too much, we have to find a path through a set of artist concerts where every night, the artists are in different places. I call this the ‘travelling rock star’ problem. Ultimately I was pretty happy with how the routing algorithm, it can find a decent route through a thousand events in less than 30 seconds.
Jennie joined me for a few hours at the Music Hack Day – she coded up the HTML for the webform and made the top banner – (it was pretty weird to look over on her computer and see her typing in raw HTML tags with attached CSS attributes – kids these days). We got the demo done in time – and with the power of caching it will generate routes and plot them on a map using the Google API. Unfortunately, if your route doesn’t happen to be in the cache, it can take quite a bit of time to get a route out of the app – gathering events from SongKick, getting the recommendations from the Echo Nest, and finding the optimal route all add up to an app that can take 5 minutes before you get your answer. When I get a bit of time, I’ll take another pass to speed things up. When it is fast enough, I’ll put it online.
It was a fun demo to write. I especially enjoyed working on it with my daughter. And we won the SongKick prize, which was pretty fantastic.
Jennie with the SongKick prize
The Echo Nest Song API
Posted by Paul in Music, The Echo Nest, web services on April 24, 2010
- Performance – api method calls run faster – on average API methods are running 3X faster than the older version.
- JSON Output – all of our methods now support JSON output in addition to XML. This greatly simplifies writing client libraries for the Echo Nest
- Nimble coding – with the new architecture it will be much easier for us to roll out new features – so expect to see new features added to the Echo Nest platform every month
- No cruft – we are revisiting our APIs to try to eliminate inconsistencies, redundancies and unnecessary features to make them as clean as we can.
The beta version of our next generation APIs are here: http://beta.developer.echonest.com/
The first significant new API we are adding is the Song API – this gives you all sorts of ways to search for and retrieve song level data. With the song API you can do the following:
- search for songs via artist name, song title, and description. You can affect the results with constraints and sorts:
- constrain the results by a number of factors including musical attributes like tempo, loudness, time signature and key, artist hotttnesss, location
- sort – the results by any of the attributes
- Find similar songs – find similar songs to a seed song
- Find profile – get all sorts of info about a song including audio, audio summary info, track data for different catalogs, song hottttnesss, artist_hotttnesss, artist_location, and detailed track analysis
- Identify songs – works in conjunction with the ENMFP
There are lots of things you can do with this API. Here’s just a quick sample of the types of queries you can make:
Find the loudest thrash songs
song/search?sort=loudness-desc&description=thrash
Find indie songs for jogging
song/search?min_tempo=120&description=indie&max_tempo=125
Fetch the tempo of Hey Jude
search?title=hey+jude&bucket=audio_summary&artist=the+beatles
Fetch the track audio and analysis of Bad Romance
search?title=bad+romance&bucket=tracks&bucket=id:paulify&artist=lady+gaga
Find songs similar to Bad Romance
song/similar?id=SOAOBBG127D9789749
- jen-api – a java client
- beta_pyechonest – a new branch of the venerable pyechonest library. Grab it from SVN with
svn checkout http://pyechonest.googlecode.com/svn/branches/ beta-pyechonest-read-only
I’ll be writing more about all of the new APIs real soon. Access the beta Echo Nest APIs here:
Flash API for the Echo Nest
Posted by Paul in code, The Echo Nest, web services on July 8, 2009
It’s a busy week for client APIs at the Echo Nest. Developer Ryan Berdeen has released a Flash API for the Echo Nest. Ryan’s API supports the track methods of the Echo Nest API, giving the flash programmer the ability to analyze a track and get detailed info about the track including track metadata, loudness, mode and key along with detailed information relating to the tracks rhythmic, timbrel, and harmonic content.
One of the sticky bits in using the Echo Nest from Flash has been the track uploader. People have had a hard time getting it to work – and since we don’t do very much Flash programming here at the nest it never made it to the top of the list of things to look into. However, Ryan dug in an wrote a MultipartFormDataEncoder that works with the track upload API method – solving the problem not just for him, but for everyone.
Ryan’s timing for this release is most excellent. This release comes just in time for Music Hackday – where hundreds of developers (presumably including a number of flash programmers) will be hacking away at the various music APIs including the Echo Nest. Special thanks goes to Ryan for developing this API and making it available to the world
(== (+ “Clojure” “Echo Nest”) “woah!”)
Posted by Paul in code, Music, The Echo Nest on June 25, 2009
Here’s the first Echo Nest application (as far as I know) that is written in Clojure: Another reason I like Clojure
New Echo Nest Java client released
Posted by Paul in code, java, Music, The Echo Nest, web services on June 24, 2009
We’ve just released version 1.1 of the Echo Nest Java Client. The Java Client makes it easy to access the Echo Nest APIs from a Java program. This release fixes some bugs and improves caching support. Here’s a snippet of Java code that shows how you can use the API to find similar artists for the band ‘Weezer’:
ArtistAPI artistAPI = new ArtistAPI(MY_ECHO_NEST_API_KEY);
List<Artist> artists = artistAPI.searchArtist("Weezer, false);
if (artists.size() > 0) {
for (Artist artist : artists) {
List<Scored<Artist>> similars =
artistAPI.getSimilarArtists(artist, 0, 10);
for (Scored<Artist> simArtist : similars) {
System.out.println(" " + simArtist.getItem());
}
}
}
Also included in the release is a command line shell that lets you interact with the Echo Nest API. You can start it up from the command line like so:
java -DDECHO_NEST_API_KEY=YOUR_API_KEY -jar EchoNestAPI.jar
Here’s an example session:
Welcome to The Echo Nest API Shell type 'help' nest% help 0) alias - adds a pseudonym or shorthand term for a command 1) chain - execute multiple commands on a single line 2) delay - pauses for a given number of seconds 3) echo - display a line of text 4) enid - gets the ENID for an arist 5) gc - performs garbage collection 6) getMaxCacheTime - gets the cache time 7) get_audio - gets audio for an artist 8) get_blogs - gets blogs for an artist 9) get_fam - gets familiarity for an artist 10) get_hot - gets hotttnesss for an artist 11) get_news - gets news for an artist 12) get_reviews - gets Reviews for an artist 13) get_similar - finds similar artists 14) get_similars - finds similar artists to a set of artists 15) get_urls - gets Reviews for an artist 16) get_video - gets video for an artist ( .. commands omitted ..) 53) trackTatums - gets the tatums of a track 54) trackTempo - gets the overall Tempo of a track 55) trackTimeSignature - gets the overall time signature of a track 56) trackUpload - uploads a track 57) trackUploadDir - uploads a directory of tracks 58) trackWait - waits for an analysis to be complete 59) version - displays version information nest% nest% get_similar weezer Similarity for Weezer 1.00 The Smashing Pumpkins 0.50 Ozma 0.33 Fountains of Wayne 0.25 Jimmy Eat World 0.20 Veruca Salt 0.17 The Breeders 0.14 Nerf Herder 0.13 The Flaming Lips 0.11 Death Cab for Cutie 0.10 Rivers Cuomo 0.09 The Rentals 0.08 Size 14 0.08 Nada Surf 0.07 Third Eye Blind 0.07 Chopper One nest% nest% get_fam Decemberists Familiarity for The Decemberists 0.8834854 nest% nest% trackUpload "09 When I'm Sixty-Four.MP3" ID: baad7cab21b853ea5ead4db0a12b1df8 nest% trackDuration Duration: 157.96104 nest% nest% trackTempo 140.571 (0.717) nest%
If you are interested in playing around with the Echo Nest API but don’t want to code up your own application, typing in webservice URLs by hand gets pretty old, pretty quickly. The Echo Nest shell gives you a simpler way to try things out.
New Java Client for the Echo Nest API
Posted by Paul in code, java, The Echo Nest, web services on April 7, 2009
Today we are releasing a Java client library for the Echo Nest developer API. This library gives the Java programmer full access to the Echo Nest developer API. The API includes artist-level methods such as getting artist news, reviews, blogs, audio, video, links, familiarity, hotttnesss, similar artists, and so on. The API also includes access to the renown track analysis API that will allow you to get a detailed musical analysis of any music track. This analysis includes loudness, mode, key, tempo, time signature, detailed beat structure, harmonic content, and timbre information for a track.
To use the API you need to get an Echo Nest developer key (it’s free) from developer.echonest.com. Here are some code samples:
// a quick and dirty audio search engine
ArtistAPI artistAPI = new ArtistAPI(MY_ECHO_NEST_API_KEY);
List<Artist> artists = artistAPI.searchArtist("The Decemberists", false);
for (Artist artist : artists) {
DocumentList<Audio> audioList = artistAPI.getAudio(artist, 0, 15);
for (Audio audio : audioList.getDocuments()) {
System.out.println(audio.toString())
}
}
// find similar artists for weezer
ArtistAPI artistAPI = new ArtistAPI(MY_ECHO_NEST_API_KEY);
List<Artist> artists = artistAPI.searchArtist("weezer", false);
for (Artist artist : artists) {
List<Scored<Artist>> similars = artistAPI.getSimilarArtists(artist, 0, 10);
for (Scored<Artist> simArtist : similars) {
System.out.println(" " + simArtist.getItem().getName());
}
}
// Find the tempo of a track
TrackAPI trackAPI = new TrackAPI(MY_ECHO_NEST_API_KEY);
String id = trackAPI.uploadTrack(new File("/path/to/music/track.mp3"), false);
AnalysisStatus status = trackAPI.waitForAnalysis(id, 60000);
if (status == AnalysisStatus.COMPLETE) {
System.out.println("Tempo in BPM: " + trackAPI.getTempo(id));
}
There are some nifty bits in the API. The API will cache data for you so frequently requested data (everyone wants the latest news about Cher) will be served up very quickly. The cache can be persisted, and the shelf-life for data in the cache can be set programmatically (the default age is one week). The API will (optionally) schedule requests to ensure that you don’t exceed your call limit. For those that like to look under the hood, you can turn on tracing to see what the method URL calls look like and see what the returned XML looks like.
If you are interested in kicking the tires of the Echo Nest API and you are a Java or Processing programmer, give the API a try.
If you have any questions / comments or problems abut the API post to the Echo Nest Forums.
The Blip.fm API
Posted by Paul in Music, web services on February 10, 2009
Blip.fm is often described as a twitter for music. Blip users post ‘blips’ to tracks – and as with Twitter, others can follow your Blips and listen to what you’ve posted. It’s micro-music-blogging. Now that Twitter has become so popular, there is a whole micro-economy built around Twitter – with multiple companies providing every different style of twitter client that you could possibly want, for just about any platform. Twitter has enabled this economy by providing a rich set of web services around their system that any client can tap into. Blip is hoping to do the same thing. They are providing a rich set of web services around their core that allows any third party to interact with the Blip service.
The current Blip web services are in private beta – and are likely to be extended and modified as the service matures. To use the web services you need to get an API key from blip.fm (via email). Despite the private beta nature of the API – there’s already quite a bit a functionality in the API. Here’s a quick rundown of what you can already do with the API:
- Post a blip
- Delete a blip
- Get a blip by ID
- Get all public blips that occurred over a range of time
- Search for by song or artist name
- User Related Blips
- Get blips for a user ordered by recency
- Get blips for users that a user is following
- Get a user’s playlist blips
- Get blips that have replies
- Get a user by name
- Get a user’s listeners
- Get a user’s preferences
- Get a user’s stats
- Give a user ‘props’
- Save a user’s preferences
- Sign up a new user
- Favorites
- Add a user as a ‘favorite’ dj
- add a blip to a user’s playlist
- remove a user as a a ‘favorite’ dj
- remove a blip from a user’s playlist
- Get a user’s favorite DJs
These services seem to be pretty all inclusive, covering every thing that any 3rd party client would want to do with the blip service.
The Blip services provide output in XML, JSON or serialized PHP. There’s a sample return for a getUserProfile request that returns my most recent blips at the end of this post.
Authentication - In general, any of the Blip web services that are related to a specific user require the call to be authenticated. Creating an authenticated call involves taking a hash of your blip secret key along with a few other fields (such as the timestamp) to create a signature that is appended to the request. (Does anyone else have problems trying to manage these secret keys in an open source project?, they really belong with the code, but if you check them into your open source code repository, they are not secret anymore!).
Terms-of-service – As far as I can tell, the Blip folks haven’t published a terms-of-service for the API. Not surprising since the API is still in private beta. Still, I like to know the rules of the road before I invest too much in an API. In particular, I’d like to know whether or not commercial use of the API is allowed. Blip does have rate limits – no more than one call every 30 seconds per API key for authenticated calls (there are some calls that are excluded from this rate limit).
Documentation – the documentation for the blip service is quite good for a private beta. I especially like the API Tool that lets you play with the API in the browser. They could improve the documentation a bit around what happens with failures – all they say for right now is Error message on failure – which is really not that helpful. In particular, it would be nice if they published a set of status codes that one could expect on error – so I can programmatically tell the difference between an authentication error (a user gave me the wrong password) and a rate limit exceeded error.
Summary – For a private beta, I’m quite impressed at how full featured the Blip.fm API is. They have a wide range of web services already built around their core system. They have figured out a good way to authenticate calls that manipulate user data. The documentation combined with the nifty API tool lets you easily explore the nooks and crannies of the API. They have API client libraries for PHP, Actionscript and Javascript (no Java or Python, sniff!). There’s lots of good stuff here.
Sample Blip return XML
<?xml version="1.0" encoding="UTF-8"?>
<BlipApiResponse>
<status>
<code>200</code>
<message>OK</message>
<requestTime>1234265571</requestTime>
<responseTime>1234265571</responseTime>
<rateLimit>0</rateLimit>
</status>
<result>
<total>2</total>
<offset>0</offset>
<limit>25</limit>
<count>2</count>
<collection>
<Blip>
<id>16946</id>
<url>http://centralvillage.blogs.com/cv/files/vampireweekend_oxford_comma.mp3</url>
<ownerId>37237</ownerId>
<artist>Vampire Weekend</artist>
<title>Oxford Comma</title>
<insTime>2008-06-17 12:12:38</insTime>
<message>vw</message>
<unixTime>1213704758</unixTime>
<toId />
<type>songUrl</type>
<status>active</status>
<reblipId />
<thumbplayLink />
<via />
<viaUrl />
<owner>
<id>37237</id>
<urlName>plamere</urlName>
<profilePic>http://blip.fm/_/images/nousericon.gif</profilePic>
<status>active</status>
<propsCount>0</propsCount>
<countryAbbr>us</countryAbbr>
<name />
<website />
<timeZone>US/Pacific</timeZone>
<lastBlipTime>0000-00-00 00:00:00</lastBlipTime>
<insTime>2008-06-17 09:18:28</insTime>
<updateTime>2009-02-05 12:40:39</updateTime>
</owner>
</Blip>
<Blip>
<id>16919</id>
<url>http://www.notontheguestlist.com/MynameIsjonas.mp3</url>
<ownerId>37237</ownerId>
<artist>Weezer</artist>
<title>My Name Is Jonas</title>
<insTime>2008-06-17 09:19:26</insTime>
<message>weezer in the morning</message>
<unixTime>1213694366</unixTime>
<toId />
<type>songUrl</type>
<status>active</status>
<reblipId />
<thumbplayLink />
<via />
<viaUrl />
<owner>
<id>37237</id>
<urlName>plamere</urlName>
<profilePic>http://blip.fm/_/images/nousericon.gif</profilePic>
<status>active</status>
<propsCount>0</propsCount>
<countryAbbr>us</countryAbbr>
<name />
<website />
<timeZone>US/Pacific</timeZone>
<lastBlipTime>0000-00-00 00:00:00</lastBlipTime>
<insTime>2008-06-17 09:18:28</insTime>
<updateTime>2009-02-05 12:40:39</updateTime>
</owner>
</Blip>
</collection>
</result>
</BlipApiResponse>
