I'm the Director of Developer Community at The Echo Nest, a research-focused music intelligence startup that provides music information services to developers and partners through a data mining and machine listening platform. I am especially interested in hybrid music recommenders and using visualizations to aid music discovery.
Posted in Music on February 13, 2016
I made a little update to SmarterPlaylists today to give you more info about the different attributes you can use to sort and filter your playlists. For instance, if you are adding a Range Filter to filter your playlists based upon a particular attribute, you will get a detailed description of the parameter you are filtering by along with info on the valid range of values for the attribute. Here’s an example:
I’ve also pushed out some stability updates to the backend server so programs that mysteriously failed to run in the past, will likely start working now. The slow and steady march to a Version 1.0 release is continuing. I’ve been getting lots of requests for new features. Currently on the top of the ‘new features’ to do list are:
- filtering playlist based on the date/time when tracks were added (i.e. get the 10 oldest tracks in the playlist, or get all the tracks that were added in the last 10 days)
- filtering by genre (i.e. get all of the tracks from my discovery weekly that are heavy metal)
If you have more suggestions, let me know with a comment or tweet at me at @plamere.
Posted in Music on January 19, 2016
“The making of a great compilation tape, like breaking up, is hard to do and takes ages longer than it might seem. You gotta kick off with a killer, to grab attention. Then you got to take it up a notch, but you don’t wanna blow your wad, so then you got to cool it off a notch. There are a lot of rules.” — Rob Gordon, High Fidelity
A common usage for Smarter Playlists is to create playlists that mimic popular radio formats like Contemporary Hit Radio or Adult Contemporary. For example, RadioILoveIt (a site for radio broadcasters) describes how a typical Contemporary Hit Radio station may organize its music in the following 6 buckets:
- A / Power currents: today’s most popular hits
- N / New currents: unfamiliar songs (even if they are from a familiar artist)
- B / Increasing currents: hits on the way up
- C / Decreasing currents: hits on the way down
- R / Recurrents: recent currents (e.g. less than 1 year old)
- G / Golds: format classics (e.g. more than 1 year old)
These six buckets are then scheduled into a repeating clock sequence like so:
- A – Power
- R – Recurrent
- C – Decreasing
- N – New
- A – Power
- R – Recurrent
- B – Increasing
- G – Gold
(repeat of sequence, starting from the top)
We can do this sort of thing pretty easily with Smarter Playlists. Create a playlist for each of the six buckets, and then create a Smarter Playlists program that draws from each of the buckets like so:
This will give us a playlist of hundreds of tracks that match the 8 song cycle of power / recurrent / decreasing / new / power / recurrent / increasing / gold
This is a good start, but there are a few problems with this playlist. First, we will get the same output every time we run the program, so we can mix things up a bit just by adding randomly selecting tracks from each of the inputs like so:
This will give us an 80 song place that repeats the 8 song cycle 10 times. There are still some problems with this however. Since we are using some of the same playlists multiple times as inputs (Power and Recurrents), it is possible that the same song could be sampled from each of these. Even if we weren’t reusing the same playlist as inputs, it is possible that a song may appear in more than one of the inputs anyway. The result is that we could end up with the same song appearing more than once in our output playlist. We can avoid this by adding a de-dup node:
The de-dup node will remove any song from the stream of tracks bound for our playlist that have already been added to the playlist. While we are at it, we may want to remove other offending songs. We could, of course, just make sure that we don’t add songs we don’t like to the input playlists, but often times these playlists are not under our control – (I could be using the Spotify playlist Today’s Top Hits as my source for Power Tracks for example). Likewise, there may be some artists that I don’t particularly like. By adding an artist filter and a track filter I can make sure my playlist is free from any songs that I don’t like:
There’s still another problem. A good playlist usually won’t have adjacent songs by the same artist. With this current program, there’s nothing to prevent that from happening. To fix this we can add a separate artists node. This component will reorder the tracks passing through it to minimize the number of adjacent songs with the same artist
Now we have a program that generates a playlist that meets some of the basic criteria of a good playlist – no duplicate tracks, no adjacent songs by the same artist, no songs that I don’t like, no artists that I don’t like. Unfortunately, all of this filtering and re-ordering has wrecked havoc with our repeating 8 song cycle of power / recurrent / decreasing / new / power / recurrent / increasing / gold. When we de-dupped a song, we may have filtered out a ‘power’ song, when we removed a song by an artist that I don’t like, it could have been a ‘new’ song. The resulting playlist after all of this post filtering may look very different from what we wanted, and the listening experience will suffer for it. Here’s the track output for a run of this program. If you look at the SRC column you can see that we are no longer respecting our repeating 8 song cycle:
We are starting with a ‘gold’ track, and as Rob Gordon says You gotta kick off with a killer, to grab attention. That should be a power track. The bottom line here is that it was not really possible in Alpha V2 of Smarter Playlists to create a playlist that has a defined order of songs like we see in commercial radio and still apply the usual filtering. In Smarter Playlists Alpha V3 we fix that with one new component.
Introducing the Mixer Component
So what do we do if we want to enforce a repeat song cycle, but still enforce a set of rules to ensure that we get a good playlist without repeating songs, artists and so on? We can use the newly added Mixer component! The Mixer is designed for this very situation. It is essentially an Alternator component but with all of the typical post-filtering built in, so you can generate a cycle of songs that respect all of the rules of a good playlist.
Let’s refactor our Simple Playlist using the Mixer:
The Mixer replaces the alternator, de-dup, artist filter, song filter and the artist separator, making the program quite a bit smaller. But more importantly, the mixer will maintain the proper song cycle while enforcing the most common ‘good playlist’ rules. Here’s the configuration editor for the Mixer:
As you can see, the Mixer has builtin support for de-duping tracks, maintaining a minimum artist separation and track filtering by banned artists and tracks. Inspecting the output of the program we can see that all the rules have been enforced (no dup tracks, no adjacent artists, no banned artists or tracks), and we are still respecting our repeating 8 song cycle of power / recurrent / decreasing / new / power / recurrent / increasing / gold tracks:
Posted in Music on January 18, 2016
Hey Smarter Playlist fans. I just pushed out an update for Smarter Playlists. Here’s what’s new:
Improved Component interaction – perhaps the biggest FAQ I’ve had for Smarter Playlists is how to interact with components – how to connect them, disconnect them, delete them and so on. I’ve made some improvements so hopefully this will be less of a confusing topic for new users. Now when you click on two components, you will get a message describing the types of connections you can make that includes buttons that you can click on to make the connections. No more ‘shift-space’ to connect messiness.
Added powerful Mixer component. – a common use case for Smarter Playlists is to build a playlist sequence from a number of different sources while enforcing a set of rules on the sequence (such as no duplicate songs, no adjacent artists, removing my banned songs and my banned artists). All of these functions have been a part of Smarter Playlists but using them in concert was difficult – any post filtering was bound to mess up the carefully sequenced set of tracks. The Mixer component is designed for just this use case. I’ll write up a full post on the Mixer in another blog post. Suffice it to say that the Mixer component gives you much more control about how tracks from different sources are ordered and combined.
Added the Date Filter Component – This component will filter the input stream and only pass through tracks that have a release date within the specified range.
Increased saved playlist maximum size – The maximum number of songs that can be added to a playlist has been increased from 200 to 1,000.
Added more refined component categories – There are now 7 different kinds of components:
- Sources – these components generate tracks
- Combiners – these components combine tracks from multiple sources
- Orderers – these components put tracks in order
- Selectors – these components select certain tracks from the stream (i.e. the first 10, the last 10)
- Filters – these components filter the stream of tracks based upon attributes of the tracks
- Conditionals – these components modify the stream of tracks based upon external signals such as the day of the week
- Miscs – components that don’t fit into any other category
That’s quite a bit of new stuff, but most of the work was behind the scenes getting the system robust enough to move out of alpha mode. This includes generating periodic snapshots of the program database, adding more admin tools to make it easier to support the app (such as a message of the day).
I’ve been getting lots of good suggestions for improvements to Smarter Playlists. Here’s what’s on the Coming Soon list.
- Filter playlists by time added
- Add better range documentation and checking component value parameters
- Add more docs on how to debug a Smarter Playlist
Thanks for the ideas and keep them coming.
Posted in Music on January 8, 2016
I’ve added a Date Filter component to Smarter Playlists. This component lets you filter tracks based upon the release date of the track’s album. Here’s an example program that uses the date filter to get all of the early Beatles tracks (those that were released before Revolver).
The Date Filter for this program is configured like so:
Thanks for all the suggestions, for improvements. We are rapidly heading toward an official release.
Posted in Music on January 6, 2016
Chris Kelly pointed out that he likes to preserve his Discover Weekly playlists like so:
I liked the idea so I added a new Smarter Playlists component to make this happen. The component is called Save to New Playlist.
With this new component you can save your Smarter Playlists to a new playlist that is tagged with a date or timestamp. It makes it easy to make the completist Discover Weekly Archiver that Chris Kelly likes:
Thanks Chris for the suggestion!
Posted in Music on January 5, 2016
Earlier this week I released the second alpha version of Smarter Playlists. This version of Smarter Playlists includes a scheduler that will let you schedule your programs to run periodically. In this post, I walk through creating and scheduling a Smarter Playlist that will automatically archive your discover weekly tracks.
Step 1 -Login to Smarter Playlists
Go to Smarter Playlists and login with your Spotify Credentials.
Step 2 -Import the My Discover Weekly Archiver
Go to the Imports page and import the My Discover Weekly Archiver by clicking the Import button.
Step 3 – Edit the program
Ideally, you’d be able to just import the program and run it, but unfortunately, there’s no API (yet) that allows Smarter Playlists to find your Discover Weekly playlist, so you’ll have to customize the program with your own URIs.
Go to the Smarter Playlists Programs page, find the program called “import of My Discovery Weekly Archiver” and edit it (by clicking the edit button).
Step 3 – add your Discovery Weekly URI to the program
You have to add your own URIs to the program. First, in the Spotify Client, browse to your Discover Weekly Playlist (you can get to it by clicking on ‘browse’ and then ‘discover’). Right click on the playlist and Copy Spotify URI.
Now in your Discover Weekly Archiver, double click on the box labeled ‘Discovery Weekly’. You’ll see the component editor page. In the URI field, paste the URI for your Discover Weekly.
Step 4 – add your Discovery Weekly Archive URI to the program
Go back to Spotify and create the playlist that will be your archive. Get its URI, head back to Smarter Playlists and add the URI to the My Discovery Weekly component, and the Save to Spotify component.
Step 5 – Test your program
Press the run button. If all goes well, you should see your Discovery Weekly Archive playlist populated with this week’s tracks. If you run it again, you should see the same tracks (since we filter out any dups).
Step 6 – Schedule your program to run once a week.
Once you are convinced that your program is running properly, you can schedule it to run once a week. To do this press the ‘schedule’ button next to your program to bring up the scheduler.
Click on the ‘How Often’ drop-down and select Every Week. Set ‘How Many Times’ to an appropriate value (50 will back you up for a year). Click on Update Schedule to make it happen. If you want to schedule the program to run on a particular day of the week, click on the calendar button in the Next Run field to pick a particular date.
Once you do this, your program will run once a week to archive your discoveries.
Step 7 – Check the status of your runs
You can check the status of your scheduled runs and see if there are any errors just be visiting the Scheduler page for your program.
Going Deeper – how does the program work?
Read on if you are interested in learning how the archiver works.
The archiver is rather simple program. It consists of 4 components – 2 playlist components, a filter and a playlist saver. The first playlist component reads in the current Discovery Weekly playlist. The second playlist component reads the contents of your current archive. These two playlists are input into the track filter, this only passes tracks through that are on the green stream that are not on the red stream. The result is that only tracks that are not currently in the archive are passed through. These tracks are then sent on to the save to Spotify component. This component will write the input tracks to the given playlist. This component has the append flag set so that it appends tracks to the archive instead of overwriting the archive.
With all that URI copying I think it is still a bit too complex to set up a simple program like the discovery archiver. But as new APIs are added things should get simpler.
Posted in Music on January 4, 2016
Today, I’m releasing the second alpha release of Smarter Playlists – the web app that helps you automate the creation of sophisticated playlists.
This big feature of this release is that you can now schedule your smarter playlist programs to run periodically. This lets you do all sorts of nifty things from automatically archiving your Discover Weekly recommendations to giving you a fresh mix of your favorites tracks with a few tracks from Fresh Finds sprinkled in.
If you are a professional curator managing dozens or hundreds of playlists, you can use the scheduling feature to keep your playlists fresh automatically.
The Scheduler lets you run your programs periodically. You can control how often your program runs, how many times it will run, and the exact date and time it will run.
This release also gives you better ways to manage your set of programs.
There are also a bunch of new components – such as personalized sources (my saved tracks, my saved albums, my followed artists, etc.), a nifty artist separation component that reorders tracks to maximize the distance between tracks by the same artist and a whole bunch more.
Note that the previous release is still online. Unfortunately there’s no automated way to move your programs from Alpha V1 to Alpha V2.
This release has been nearly ready to go since August, but life intervened, and my time for spare time projects evaporated. Over the holiday break, I finally had some time to devote to getting this out the door. Give it a go and send me feedback.
Check it out at: smarterplaylists.playlistmachinery.com
Posted in Music on August 17, 2015
I’m getting close to releasing the next version of Smarter Playlists. This will be a big update with all sorts of new features. Big tent features are:
Ability to schedule when your programs run. – You’ll be able to schedule your programs to run at a pre-determined time and period.
Improved system for sharing, publishing and importing Smarter Playlist Programs:
Improved program management UI:
- separate artists – minimizes the number of adjacent songs by the same artist
- mix in – flexibly mix two input streams
- personalized sources – my saved tracks produces a list of the current user’s saved tracks
Under the hood, there are huge changes – programs are no longer saved in the browser, but instead are saved on the server. The authentication model has been upgraded to use the 3-factor version. This will allow me to run programs on the server side and have them update your playlists on your behalf. An essential capability for scheduling.
All these changes mean that your Smarter Playlists V1 programs will no longer work with Smarter Playlists V2. You’ll have to re-create them in V2.
I’ve received lots of feedback from folks using Smarter Playlists V1 – and will work to incorporate all the suggested features, improvements and bug fixes too. Thanks, all, and keep the feedback coming.
Look for the alpha v2 release within the next week or so.
Posted in Music on July 25, 2015
I’ve just pushed out an early alpha version of Smarter Playlists, my summer spare-time project. Smarter Playlists is a playlist builder that lets you create new interesting playlists by combining sources of tracks (like albums, artists, other playlists) and filtering, sorting and re-arranging these tracks into a playlist. For example. Here’s a simple playlist that combines two Spotify playlists: Morning Commute and Your favorite Coffee house into a single new playlist:
Here’s a more complex example that starts with a Gothic Metal playlist and mixes in the top tracks from the band Ravenscry into the first songs in the playlist.
This is still very much an alpha version, so there are likely to be a bug or two – but give it a go if you are so inclined.
Check it out at Smarter Playlists
Over the past few weekends I’ve been working on a little side project called the Playlist Builder Library (or PBL for short). The Playlist Builder Library is a Python library for creating and manipulating playlists. It’s sort of like remix for playlists. With PBL you can take songs from playlists, albums, artists, genres and flexibly combined them, rearrange them, filter them and sort them into new playlists.
For example, here’s a PBL program that creates radio station of today’s top hits but guarantees that every 4th song is either by Sia or Katy Perry:[gist https://gist.github.com/plamere/2fa839150815f040450d]
Here’s the resulting playlist:[spotify spotify:user:plamere:playlist:6TIeQMve7pVBLCAY8WUX3L]
That’s 5 lines a code to create a non-trivial playlist.
PBL supports all sorts of sources for tracks such as Spotify playlists, top tracks from artists, albums, genres, the extremely flexible and powerful Echo Nest playlisting API. These sources can be manipulated in all sorts of interesting ways. Here are a couple more examples:
You can filter all the songs in ‘Your favorite coffeehouse’ to get just the lowest energy songs:
coffee = PlaylistSource('coffeehouse', ucoffee_house) low_energy_coffee = AttributeRangeFilter(coffee, 'echonest.energy', max_val=.5)
You an combine your favorite playlists in a single one:
playlist_names = ['Your Favorite Coffeehouse', 'Acoustic Summer','Acoustic Covers', 'Rainy Day'] all = DeDup(Alternate([Sample(PlaylistSource(n), 10) for n in playlist_names]))
Even sophisticated tasks are really easy. For instance, imagine dad is on a roadtrip with daughter. They agree to alternate between dad’s music and daughter’s music. Dad is selfish, so he makes a playlist that alternates the longest cool jazz tracks with the shortest teen party playlists with this 3 line script:
teen_party = First(Sorter(PlaylistSource('Teen Party'), 'duration'), 10) jazz_classics = Last(Sorter(PlaylistSource('Jazz Classics'), 'duration'), 10) both = Alternate([teen_party, Reverse(jazz_classics)])
Here’s the result[spotify spotify:user:plamere:playlist:0VKGTR6eCPe55bBjezi5z3]
Note that the average duration of Teen Party songs is much less than 3 minutes, while the average duration of Jazz Classics is above 6 minutes. Selfish dad gets to listen to his music twice as long with this jazz-skewing playlist.
There’s a whole lot of nifty things that can be done with PBL. If you are a Python programmer with an itch for creating new playlists check it out. The docs are online at http://pbl.readthedocs.org/ and the source is at https://github.com/plamere/pbl.
PBL is pretty modular so it is easy to add new sources and manipulators, so if you have an idea or two for changes let me know or just send me a pull request.