Music Machinery

This morning I added a ‘title filter’ component to SmarterPlaylists. This component lets you filter tracks based on regex string matching of the track title. This is useful for filtering out certain types of track – such as remixes, radio edits or songs with 9 in the title.

The title filter allows you to invert the sense of the filter, and makes it easy to ignore case in your matches as well:

Using this filter we can take our favorite dubstep playlist and eliminate all the radio edits.

To try this yourself, you can import the program.

Thanks to Drew Davidson for the component suggestion.

A common #smarterplaylists feature request has been to filter playlists by the relative date in which a track was added to the playlist.  Relative date filtering would allow you to do things like extract all the tracks in my “favs playlist” that have been added in the last six months.  Using a relative date like ‘six months ago’ versus an absolute date like ‘August 21, 2015’  makes it possible to write a #smarterplaylists program that will continue to surface up the tracks from the most recent six months, no matter how far in the future it is run.

This weekend, I’ve added support for a Playlist (Rel Date) component that will  filter playlists by the relative date added.  Here’s an example configuration for the component:

Relative dates are given as you would say them. Some examples of the type of relative dates supported are:

• 1 month
• 2 days 2 hours
• 3 years 2 months 2 days
• 3 years 2 months and 2 days ago
• 3 years, 2 months and 22 days ago!
• 1 h 3 w 2 d
• last week
• last month
• last year and three weeks ago
• six weeks
• a week ago
• twenty days ago
• 6 mnths, 2 wks

Keeping it Fresh
Here’s an example of how we can use the relative date filter to help us keep a periodically updating playlist filled with fresh tracks.

Here’s my Gothic Metal sampler.  It collects songs from a number of gothic metal playlists, selects songs at random and adds them to a playlist called Gothic Metal Sampler:

This program is scheduled to run once a day, giving me a fresh playlist of gothic metal to listen to on my morning commute.  Unfortunately, the playlist isn’t always so fresh. There’s nothing to prevent the same songs from appearing over and over. We are relying totally on the throw of the virtual dice to keep the playlist fresh.

With the new relative date support, this is very easy to fix.  The idea is to simultaneously update a separate archive playlist that keeps track of all the songs that have been ever added to my gothic metal sampler. I can then use the new relative date support to extract all the tracks that have been added to that playlist in the last week and use that set of tracks as input to the mixer track filter which will ensure that those tracks will not be included in the output.  The result is that the Gothic Metal Sampler is updated every day while no song will appear more frequently than once a week in the playlist.

Feel free to import the program for study and use.

Using the relative date filtering in this fashion is a pretty handy way for you to incorporate history into your programs.

Today I added a new SmarterPlaylists component. It has the catchy name playlist (abs date). It works similarly to the standard playlist component that sources tracks from a Spotify playlist, but it lets you order and filter the tracks based upon when they were added to the playlist.  With playlist (abs date) you can specify that you want the tracks returned in normal playlist order, or in the order in which tracks were added to the playlist.  You can also filter the tracks so that only tracks that were added between a range of dates are returned.

Here’s what the configuration looks like for the component:

Some possible use cases:

Let’s just listen to the most recent 20 tracks in my “Fav songs playlist”. To do this I add the Playlist (Abs Date)  component, and configure it to read songs from Fav Songs and return them in date added order:

I then connect it up to a component that extracts the last 20 songs from its source:

Here’s a Valentine’s Day use case – you want to generate a romantic list of songs that you and your SO were listening to when you first started dating last summer.  You can build a playlist of songs that you added to each of your fav songs playlist during that time like so:

where both Playlist (abs date) components are configured similarly:

This doesn’t handle all date related use cases yet.  For instance, it’d be great if we could extract out songs from a playlist based upon a relative time scale … such as all the tracks that were added in the last week.  Support for relative dates will be coming soon.

“The making of a great compilation tape, like breaking up, is hard to do and takes ages longer than it might seem. You gotta kick off with a killer, to grab attention. Then you got to take it up a notch, but you don’t wanna blow your wad, so then you got to cool it off a notch. There are a lot of rules.” — Rob Gordon, High Fidelity

A common usage for Smarter Playlists is to create playlists that mimic popular radio formats like Contemporary Hit Radio or Adult Contemporary. For example, RadioILoveIt (a site for radio broadcasters) describes how a typical Contemporary Hit Radio station may organize its music in the following 6 buckets:

• A / Power currents: today’s most popular hits
• N / New currents: unfamiliar songs (even if they are from a familiar artist)
• B / Increasing currents: hits on the way up
• C / Decreasing currents: hits on the way down
• R / Recurrents: recent currents (e.g. less than 1 year old)
• G / Golds: format classics (e.g. more than 1 year old)

These six buckets are then scheduled into a repeating clock sequence like so:

• A – Power
• R – Recurrent
• C – Decreasing
• N – New
• A – Power
• R – Recurrent
• B – Increasing
• G – Gold

(repeat of sequence, starting from the top)

We can do this sort of thing pretty easily with Smarter Playlists. Create a playlist for each of the six buckets, and then create a Smarter Playlists program that draws from each of the buckets like so:

This will give us a playlist of hundreds of tracks that match the 8 song cycle of power / recurrent / decreasing / new / power / recurrent / increasing / gold

This is a good start, but there are a few problems with this playlist. First,  we will get the same output every time we run the program, so we can mix things up a bit just by adding randomly selecting tracks from each of the inputs like so:

This will give us an 80 song place that repeats the 8 song cycle 10 times. There are still some problems with this however. Since we are using some of the same playlists multiple times as inputs (Power and Recurrents), it is possible that the same song could be sampled from each of these. Even if we weren’t reusing the same playlist as inputs, it is possible that a song may appear in more than one of the inputs anyway. The result is that we could end up with the same song appearing more than once in our output playlist. We can avoid this by adding a de-dup  node:

The de-dup node will remove any song from the stream of tracks bound for our playlist that have already been added to the playlist. While we are at it, we may want to remove other offending songs. We could, of course, just make sure that we don’t add songs we don’t like to the input playlists, but often times these playlists are not under our control – (I could be using the Spotify playlist Today’s Top Hits as my source for Power Tracks for example). Likewise, there may be some artists that I don’t particularly like. By adding an artist filter and a track filter I can make sure my playlist is free from any songs that I don’t like:

There’s still another problem.  A good playlist usually won’t have adjacent songs by the same artist. With this current program, there’s nothing to prevent that from happening. To fix this we can add  a separate artists node. This component will reorder the tracks passing through it to minimize the number of adjacent songs with the same artist

Now we have a program that generates a playlist that meets some of the basic criteria of a good playlist  – no duplicate tracks,  no adjacent songs by the same artist, no songs that I don’t like, no artists that I don’t like. Unfortunately, all of this filtering and re-ordering has wrecked havoc with our repeating 8 song cycle of power / recurrent / decreasing / new / power / recurrent / increasing / gold. When we de-dupped a song, we may have filtered out a ‘power’ song, when we removed a song by an artist that I don’t like, it could have been a ‘new’ song. The resulting playlist after all of this post filtering may look very different from what we wanted, and the listening experience will suffer for it. Here’s the track output for a run of this program.  If you look at the SRC column you can see that we are no longer respecting our repeating 8 song cycle:

We are starting with a ‘gold’ track, and as Rob Gordon says You gotta kick off with a killer, to grab attention. That should be a power track. The bottom line here is that it was not really possible in Alpha V2 of Smarter Playlists to create a playlist that has a defined order of songs like we see in commercial radio and still apply the usual filtering. In Smarter Playlists Alpha V3 we fix that with one new component.

Introducing the Mixer Component

So what do we do if we want to enforce a repeat song cycle, but still enforce a set of rules to ensure that we get a good playlist without repeating songs, artists and so on? We can use the newly added Mixer  component! The Mixer is designed for this very situation. It is essentially an Alternator component but with all of the typical post-filtering built in, so you can generate a cycle of songs that respect all of the rules of a good playlist.

Let’s refactor our Simple Playlist using the Mixer:

The Mixer replaces the alternator, de-dup, artist filter, song filter and the artist separator, making the program quite a bit smaller. But more importantly, the mixer will maintain the proper song cycle while enforcing the most common ‘good playlist’ rules.  Here’s the configuration editor for the Mixer:

As you can see, the Mixer has builtin support for de-duping tracks, maintaining a minimum artist separation and track filtering by banned artists and tracks. Inspecting the output of the program we can see that all the rules have been enforced (no dup tracks, no adjacent artists, no banned artists or tracks),  and we are still respecting our repeating 8 song cycle of power / recurrent / decreasing / new / power / recurrent / increasing / gold tracks:

Finishing Up

The Mixer component should make it easier to create broadcast-style playlists that follow a cycle, just like the pros do.  Give it a try at Smarter Playlists.

Hey Smarter Playlist fans. I just pushed out an update for Smarter Playlists.  Here’s what’s new:

Improved Component interaction – perhaps the biggest FAQ I’ve had for Smarter Playlists is how to interact with components – how to connect them, disconnect them, delete them and so on.   I’ve made some improvements so hopefully this will be less of a confusing topic for new users.  Now when you click on two components, you will get a message describing the types of connections you can make that includes buttons that you can click on to make the connections. No more ‘shift-space’ to connect messiness.

Added powerful Mixer component.  – a common use case for Smarter Playlists is to build a playlist sequence from a number of different sources while enforcing a set of rules on the sequence (such as no duplicate songs, no adjacent artists, removing my banned songs and my banned artists).  All of these functions have been a part of Smarter Playlists but using them in concert was difficult – any post filtering was bound to mess up the carefully sequenced set of tracks.  The Mixer component is designed for just this use case.  I’ll write up a full post on the Mixer in another blog post. Suffice it to say that the Mixer component gives you much more control about how tracks from different sources are ordered and combined.

Added the Date Filter Component – This component will filter the input stream and only pass through tracks that have a release date within the specified range.

Increased saved playlist maximum size – The maximum number of songs that can be added to a playlist has been increased from 200 to 1,000.

Added more refined component categories – There are now 7 different kinds of components:

• Sources –  these components generate tracks
• Combiners – these components combine tracks from multiple sources
• Orderers – these components put tracks in order
• Selectors – these components select certain tracks from the stream (i.e. the first 10, the last 10)
• Filters –  these components filter the stream of tracks based upon attributes of the tracks
• Conditionals – these components modify the stream of tracks based upon external signals such as the day of the week
• Miscs – components that don’t fit into any other category

That’s quite a bit of new stuff, but most of the work was behind the scenes getting the system robust enough to move out of alpha mode. This includes generating periodic snapshots of the program database, adding more admin tools to make it easier to support the app (such as a message of the day).

What’s next?

I’ve been getting lots of good suggestions for improvements to Smarter Playlists.  Here’s what’s on the Coming Soon list.

• Filter playlists by time added
• Add better range documentation and checking component value parameters
• Add more docs on how to debug a Smarter Playlist

Thanks for the ideas and keep them coming.

I’ve added a Date Filter component to Smarter Playlists. This component lets you filter tracks based upon the release date of the track’s album.  Here’s an example program that uses the date filter to get all of the early Beatles tracks (those that were released before Revolver).

The Date Filter for this program is configured like so:

Thanks for all the suggestions, for improvements. We are rapidly heading toward an official release.

Chris Kelly pointed out that he likes to preserve his Discover Weekly playlists like so:

I liked the idea so I added a new Smarter Playlists component to make this happen.   The component is called Save to New Playlist.

With this new component you can save your Smarter Playlists to a new playlist that is tagged with a date or timestamp.  It makes it easy to make the completist Discover Weekly Archiver that Chris Kelly likes:

Thanks Chris for the suggestion!

Earlier this week I released the second alpha version of Smarter Playlists. This version of Smarter Playlists includes a scheduler that will let you schedule your programs to run periodically.  In this post, I walk through creating and scheduling a Smarter Playlist that will automatically archive your discover weekly tracks.

Step 1 -Login to Smarter Playlists

Go to Smarter Playlists and login with your Spotify Credentials.

Step 2 -Import the My Discover Weekly Archiver

Go to the Imports page and import the My Discover Weekly Archiver by clicking the Import button.

Step 3 – Edit the program

Ideally, you’d be able to just import the program and run it, but unfortunately, there’s no API (yet) that allows Smarter Playlists to find your Discover Weekly playlist, so you’ll have to customize the program with your own URIs.

Go to the Smarter Playlists Programs page, find the program called “import of My Discovery Weekly Archiver” and edit it (by clicking the edit button).

Step 3 – add your Discovery Weekly URI to the program

You have to add your own URIs to the program.  First, in the Spotify Client, browse to your Discover Weekly Playlist (you can get to it by clicking on ‘browse’ and then ‘discover’).  Right click on the playlist and Copy Spotify URI.

Now in your Discover Weekly Archiver, double click on the box labeled ‘Discovery Weekly’.  You’ll see the component editor page.  In the URI field, paste the URI for your Discover Weekly.

Step 4 – add your Discovery Weekly Archive URI to the program

Go back to Spotify and create the playlist that will be your archive.  Get its URI, head back to Smarter Playlists and add the URI to the My Discovery Weekly component, and the Save to Spotify component.

Step 5 – Test your program

Press the run button. If all goes well, you should see your Discovery Weekly Archive playlist populated with this week’s tracks.  If you run it again, you should see the same tracks (since we filter out any dups).

Step 6 – Schedule your program to run once a week.

Once you are convinced that your program is running properly, you can schedule it to run once a week.  To do this press the ‘schedule’ button next to your program to bring up the scheduler.

Click on the ‘How Often’ drop-down and select Every Week.   Set ‘How Many Times’ to an appropriate value (50 will back you up for a year). Click on Update Schedule  to make it happen.  If you want to schedule the program to run on a particular day of the week, click on the calendar button in the Next Run field to pick a particular date.

Once you do this, your program will run once a week to archive your discoveries.

Step 7 – Check the status of your runs

You can check the status of your scheduled runs and see if there are any errors just be visiting the Scheduler page for your program.

Going Deeper – how does the program work?

Read on if you are interested in learning how the archiver works.

The archiver is rather simple program. It consists of 4 components – 2 playlist components, a filter and a playlist saver.  The first playlist component reads in the current Discovery Weekly playlist. The second playlist component reads the contents of your current archive. These two playlists are input into the track filter, this only passes tracks through that are on the green stream that are not on the red stream. The result is that only tracks that are not currently in the archive are passed through.  These tracks are then sent on to the save to Spotify  component.  This component will write the input tracks to the given playlist.  This component has the append flag set so that it appends tracks to the archive instead of overwriting the archive.

Wrapping up

With all that URI copying I think it is still a bit too complex to set up a simple program like the discovery archiver. But as new APIs are added things should get simpler.

Today, I’m releasing the second alpha release of Smarter Playlists – the web app that helps you automate the creation of sophisticated playlists.

This big feature of this release is that you can now schedule your smarter playlist programs to run periodically.  This lets you do all sorts of nifty things from automatically archiving your Discover Weekly recommendations to giving you a fresh mix of your favorites tracks with a few tracks from Fresh Finds sprinkled in.

If you are a professional curator managing dozens or hundreds of playlists, you can use the scheduling feature to keep your playlists fresh automatically.

The Scheduler lets you run your programs periodically. You can control how often your program runs, how many times it will run, and the exact date and time it will run.

This release also gives you better ways to manage your set of programs.

There are also a bunch of new components – such as personalized sources (my saved tracks, my saved albums, my followed artists, etc.),  a nifty artist separation component that reorders tracks to maximize the distance between tracks by the same artist and a whole bunch more.

Note that the previous release is still online. Unfortunately there’s no automated way to move your programs from Alpha V1 to Alpha V2.

This release has been nearly ready to go since August, but life intervened, and my time for spare time projects evaporated. Over the holiday break, I finally had some time to devote to getting this out the door.  Give it a go and send me feedback.

Check it out at: smarterplaylists.playlistmachinery.com