Tuning the Infinite Jukebox

Posted by Paul in Music on November 26, 2012

The Infinite Jukebox works hard to find the best way to make a song play forever.  This is a balancing act.  The jukebox needs to add branches to a song that will allow it to hyper-jump from one section to another.  If the destination of the branch sounds similar to the starting point of the branch, the listener won’t notice the jump (which is good), so we only want to pick branches that are between two very similar sounding beats. However,  if we restrict ourselves to only adding branches between very similar sounding beats, then, for some songs, there may not be enough branches too allow the Infinite Jukebox to really play the song forever.  So the Infinite Jukebox must weigh similarity against branch density. Most of the time it does a good job, but sometimes it gets it wrong.  Until now, if the Jukebox got it wrong, you had no option but to try another song, but today I’ve just released a new tuning feature that will allow you to optimize the balance between similarity and graph density, so when the Jukebox can’t get it exactly right, you can finish the job.

The new tuning feature allows you to delete offending branches, and to help determine the proper balance between similarity and branch density.  Here’s what you can do:

Delete branches – sometimes the Jukebox gets everything right except that a few of the branches give a jarring audio transition. This can especially occur with lyric heavy music.  If there’s a particular branch that you want to get rid of, you can now just delete it. To delete an edge, click on it. It will turn red. Then hit the [delete] key and the edge will be removed from the graph. (You can restore deleted edges by hitting the reset button in the Tuning dialog).

For more detailed tuning, press the new Tune button.  This button will bring up a dialog that will let you adjust a number of parameters.   You can drag the dialog around so you can see the song visualization, and you can play the song while you are tuning it, to get real-time feedback as to how your tuning is affecting the song.   The tuning options are:

Branch Similarity Threshold –  this is a slider that determines the similarity threshold between two beats.  A branch between two beats will be created only if the sim-distance between the beats is less than this  threshold. Lower values will yield higher quality (but fewer) branches.   So, if your song has lots of  jarring beat transitions, move this slider to the left.  In general, branches with a sim-distance less than around 55 sound pretty good, and below 40 are usually inaudible.

Branch Probability Range –  when the Jukebox is about to play a beat that has branches to other beats, there’s a random chance that the Jukebox will follow one of the branches to play the similar beat.  This slider, along with the Branch Probability Ramp-up Speed  slider control the probability that the Jukebox will take a branch.  The branch probability will start at the low end of the given range, and will gradually heat up until it reaches the high end of the range.  The speed of the heating is controlled by the Ramp-up speed.  Once a branch is taken the probability is reset to the low end of the range.  The effect is, that over time, the chance that the Jukebox will follow a branch increases at the ramp-up speed, until a branch is taken at which point the process starts over again.

Loop extension optimization – the Jukebox will try to add in some extra paths in the song that may go above the branch similarity threshold in order to increase the amount of the song that is will be included in the infinite play.  If you want total control, you can turn this feature off.

Allow only reverse branches:  To increase song variety the Jukebox normally allows for jumps that go forward and backwards. Click this checkbox if you only want backward branches.

Reset – Click the reset button to restore the song to the ‘factory’ settings.

The tuning dialog also shows some data that you can use to help you tune the song:

  • Branch chance:  this is a dynamic value that shows the current percentage chance of taking a branch.
  • Last Threshold  – this shows the sim-distance of the last branch that was taken.
  • Total Beats –  the total beats in the song
  • Total Branches –  the total branches in the song
  • Longest loop –  the longest loop in the song, this is given both in terms of beats and in total percentage of the song.
  • Deleted Branches –  the number of branches that have been deleted.

Sharing your tuned songs All of the tuning that you apply to a song is captured in the URL for the song, so if you bookmark, tweet or share the URL for the song, you are also sharing all of your tuning and edits.  For example, here’s a version of Skrillex’s Scary Monsters that has been tuned to have very frequent branching between only the best branches.

In the last two weeks, there have been a million visitors to the Infinite Jukebox, with many thousands of songs uploaded.  With the new tuning feature, I’m looking forward to seeing (and hearing) better sounding infinite songs.

 

 

5 Comments

4’33” Visualized

Posted by Paul in Music on November 23, 2012

Here’s a visualization of the live performance of John Cage’s 4’33 in the Infinite Jukebox performed live by the BBC Orchestra.

Fun to watch. Use the ‘bring it on home’ mode (press H) to listen to it without any loops.

1 Comment

Why does the Infinite Jukebox only work in Chrome and Safari?

Posted by Paul in Music on November 22, 2012

The most common complaint I’ve heard about the Infinite Jukebox is that it doesn’t work in Firefox or Internet Explorer.  I’ve  been accused of either being part of some vast conspiracy to infect everyone’s computer with the Chrome virus or being a lazy programmer that couldn’t go the extra mile and deal with the usual cross-browser differences.

Indeed, I may be lazy, but that’s not why you need Chrome or Safari to run the Infinite Jukebox. The real reason is that the Infinite Jukebox relies on the Web Audio API for rendering audio.  The Web Audio API  is a W3C draft standard that provides a high-level Javascript API for manipulating, processing and synthesizing audio in web applications. The Infinite Jukebox needs to be able to take an MP3 and  play subsections of that MP3 at very accurate time intervals.  This capability is provided by the Web Audio API. Without this capability, the Infinite Jukebox would be impossible.

As the following chart shows, only Webkit-based browsers like Safari and Chrome support the Web Audio API.

Can I use the Web Audio API? from caniuse.com

The exciting bit is that the Web Audio API is supported by  Safari on IOS 6.0 devices.  This means that the Infinite Jukebox will run on your iPhone or your iPad.  Surprisingly, it even runs on my two year old, very slow iPhone 4:

The good news is that Mozilla has announced that they will support the Web Audio API in an upcoming version of Firefox. It is on their roadmap, but it no date has been given.  Even when they do though, there still is the problem that Firefox does not support the playing of MP3 audio.  As for IE,  Microsoft hasn’t announced any plans to support Web Audio.

So no vast conspiracy, it’s just that the Infinite Jukebox needs an advanced browser feature that is only supported in 2 of the 4 major browsers.  Perhaps in a year, Firefox and IE will have caught up.  Until then, you’ll need to download Chrome or if you are on a Mac, fire up Safari to play an infinite version of your favorite song.

The Web Audio API  is an extremely powerful API and I suspect will be a big part of music on the web for the next decade. If you write music apps, you should pay attention to it.  Some good places to start are:  HTML5 Audio  and the multimedia section of HTML5 Rocks.

4 Comments

An Infinite Thanksgiving

Posted by Paul in fun, Music on November 21, 2012

For that long Thanksgiving drive, there’s nothing better than a handful of infinitely long songs to keep your ears occupied on the way to Grandma’s house:

Or you may want to head over to Evolver and check out the community built Thanksgiving playlist that will actually end before heat death of the universe.

1 Comment

Johnny Cash Has Been EVERYWHERE (Man)!

Posted by Paul in fun, Music on November 21, 2012

My favorite hack to come out of last week’s Music Hack Day London (and perhaps one of my favorite music hack of all time) is the hack by Iain Mullan called “Johnny Cash Has Been EVERYWHERE (Man)!“.  This hack has all the ingredients of a great music hack – great music, great execution, a bucket full of whimsy, and absolutely nothing about it that would make an MBA start writing a business plan.  It’s practically a perfect hack.  The only flaw is that Johnny Cash would never measure his distance travelled in kilometers. Check it out:    Johnny Cash Has Been EVERYWHERE (Man)!“.

, , ,

Leave a comment

Visualizing the Structure of Pop Music

Posted by Paul in code, data, ismir, Music, music information retrieval, The Echo Nest, visualization on November 19, 2012

The Infinite Jukebox generates plots of songs in which the most similar beats are connected by arcs. I call these plots cantograms. For instance, below is a labeled cantogram for the song Rolling in the Deep by Adele. The song starts at 3:00 on the circle and proceeds clockwise, beat by beat completely around the circle. I’ve labeled the plot so you can see how it aligns with the music. There’s an intro, a first verse, a chorus, a second verse, etc. until the outro and the end of the song.

Rolling in the Deep (labelled) by Adele

One thing that’s interesting is that most of the beat similarity connections occur between the beats in the three instances of the chorus. This certainly makes intuitive sense. The verses have different lyrics, so for the most part they won’t be too similar to each other, but the choruses have the same lyrics, the same harmony, the same instrumentation. They may even be, for all we know may even be exactly the same audio, that perfect performance, cut and pasted three times by the audio engineer to make the best sounding version of the song.

Now take a look at the cantogram for another popular song. The plot below shows the beat similarities for the song Tik Tok by Ke$ha. What strikes me the most about this plot is how similar it looks to the plot for Rolling in the Deep. It has the characteristic longer intro+first verse, some minor inter-verse similarities and the very strong similarities between the three choruses.

Tik Tok by Ke$ha

As we look at more plots for modern pop music we see the same pattern over and over again. In this plot for Lady Gag’s Paparazzi a cantogram we again see the same pattern.

Lady Gaga – Paparazzi

We see it in the plot for Justin Bieber’s Baby:

Justin Bieber – Baby

Taylor Swift’s Fearless has a two verses before the first chorus, shifting it further around the circle, but other than that the pattern holds:

Taylor Swift – Fearless

Now compare and contrast the pop cantograms with those from other styles of music. First up is Led Zeppelin’s Stairway to heaven. There’s no discernable repeating chorus, or global song repetition, the only real long-arc repetition occurs during the guitar solo for the last quarter of the song.

Led Zeppelin – Stairway to Heaven

Here’s another style of music. Deadmau5’s Raise your weapon. This is electronica (and maybe some dubstep). Clearly from the cantogram we can see that is is not a traditional pop song. Very little long arc repetition, with the densest cluster being the final dubstep break.

Deadmau5 – Raise your weapon

Dave Brubeck’s Take Five has a very different pattern, with lots of short term repetition during the first half of the song, while during the second half with Joe Morello’s drum solo there’s a very different pattern.

Dave Brubeck – Take Five

Green Grass and High Tides has yet a different pattern – no three choruses and out here. (By the way, the final guitar solo is well worth listening to in the Infinite Jukebox. It is the guitar solo that never ends).

Green Grass And High Tides by The Outlaws

The progressive rock anthem Roundabout doesn’t have the Pop Pattern

Yes – Roundabout

Nor does Yo-Yo Ma’s performance of the Cello suite No. 1.

01 Cello Suite No.1, 1. Prelude by Yo-Yo Ma

Looking at the pop plots one begins to understand that pop music really could be made in a factory. Each song is cut from the same mold. In fact, one of the most successful pop songs in recent years, was produced by a label with factory in its name. Looking at Rebecca Black’s Friday we can tell right away that it is a pop song:

Friday by Rebecca Black

Compare that plot to this years Youtube breakout, Thanksgiving by Nicole Westbrook, (another Ark Music Factory assembly):

Nicole Westbrook – It’s Thanksgiving (Official Video)

The plot has all the makings of the standard pop song for the 2010s.

In the music information retrieval research community there has been quite a bit of research into algorithmically extracting song structure, and visualizations are often part of this work. If you are interested in learning more about this research, I suggest looking at some of the publications by Meinard Müller and Craig Sapp.

Of course, not every pop song will follow the pattern that I’ve shown here. Nevertheless, I find it interesting that this very simple visualization is able to show us something about the structure of the modern pop song, and how similar this structure is across many of the top pop songs.

update: since publishing this post I’ve updated the layout algorithm in the Infinite Jukebox so that songs start and end at 12 Noon and not 3PM, so the plots you see in this post are rotated 90degrees clockwise from what you would see in the jukebox.

,

11 Comments

High Five Hero

Posted by Paul in code, events, fun on November 14, 2012

 

A guest post by Jennie Lamere.  This weekend, I had the pleasure of attending MIT’s Music Hack Day with my friend Barbie. We, along with about 250 others, worked all weekend to develop a music hack. In the twenty four hours we had, Barbie and I collectively slept a mere 6 hours. After endless cups of coffee and soda, we finally emerged with our hack, “High Five Hero.”

Our hack was a remixing tool driven by MaKey MaKey. MaKey Makey, created by Jay Silver and Eric Rosenbaum, is a device that plugs into computers, allowing virtually anything to be used in place of a keyboard. For our hack, we took four beats from various songs and made it so that each completed circuit yielded a different beat – the song could be remixed by completing the circuit in different orders. While neither Barbie nor I knew Javascript or html at the beginning of the weekend, we were well-versed in it by the end. A few hours in, we had the basic code written, so we decided to add more to the hack. We added a game mode, in which the two users must complete the circuit in specific patterns to play the song. We also added an “Expert’s Only” mode, in which combo moves must be done in order to play a beat.

While writing the code itself wasn’t exactly easy, we left the hack day around 8:00 on Saturday feeling very confident – all we had left to do was add the hardware component. Our idea was to have the MaKey Makey hooked up so that when we high fived, a circuit would be completed. However, we couldn’t get the MaKey MaKey set up in such a manner that the beats would all play at the same time. After gloves, tape, wire, more tape, aluminum and even more tape, we finally got the MaKey MaKey set up. This time, we were the problem – we could not get our timing to align in a way that would make the music sound good. We decided to abandon our original idea, and try to hook the MaKey MaKey up to something else besides our hands. After hours of brainstorming, we decided to go back to our original idea, but place the points for the MaKey MaKey in different places – one on each hand, and one on each knee. This seemed to work better, and the sound produced began to sound like music.

Barbie and I were extremely proud of our first music hack- High Five Hero. Despite being extremely nervous for our demo, showing off our hack seemed to go over well! We loved being able to talk to the other hackers, and seeing what they did. We are excited to attend our next Hack Day together!

[youtube http://www.youtube.com/watch?v=KozsJ-pIID4]

Editor: I took a video of the demo presentation of High Five Hero. Apologies for the unsteady hand:

[youtube http://www.youtube.com/watch?v=fFO46bGINzk]

, ,

Leave a comment

The Infinite Jukebox

Posted by Paul in Music on November 12, 2012

Another Music Hack Day weekend … this time in Boston hosted at MIT.  It was a pretty awesome event. The space at MIT was perfect for hacking, with the best network connectivity I’ve ever seen at a hacking event.   For my weekend hack, I took the idea from my Iceland hack (Infinite Gangnam Style), and made it work with any song.  The result is The Infinite Jukebox.

With The Infinite Jukebox, you can create a never-ending and ever changing version of any song.   The app works by sending your uploaded track over to The Echo Nest, where it is decomposed into individual beats.  Each beat is then analyzed and matched to other similar sounding beats in the song.  This information is used to create a detailed song graph of paths though similar sounding beats.  As the song is played,  when the next beat  has similar sounding beats there’s a chance that we will branch to a completely different part of the song. Since the branching is to a very similar sounding beat in the song, you (in theory) won’t notice the jump.  This process of branching to similar sounding beats can continue forever, giving you an infinitely long version of the song.

To accompany the playback,  I created a chord diagram that shows the beats of the song along the circumference of the circle along with with chords representing the possible paths from each beat to it’s similar neighbors.  When the song is not playing, you can mouse over any beat and see all of the possible paths for that beat.  When the song is playing, the visualization shows the single next potential beat.  I was quite pleased at how the visualization turned out. I think it does a good job of helping the listener understand what is going on under the hood, and different songs have very different looks and color palettes. They can be quite attractive.

I did have to adapt the Infinite Gangnam Style algorithm for the Infinite Jukebox. Not every song is as self-similar as Psy’s masterpiece, so I have to dynamically adjust the beat-similarity threshold until there are enough pathways in the song graph to make the song infinite. This means that the overall musical quality may vary from song to song depending on the amount of self-similarity in the song.

Overall, the results sound good for most songs.  I still may do a bit of tweaking on the algorithm to avoid some degenerate cases (you can get stuck in a strange attractor at the end of Karma Police for instance).  Give it a try, upload your favorite song and listen to it forever.  The Infinite Jukebox.

Some of my favorite listener contributed tracks:

117 Comments

Swinging Gangnam Style

Posted by Paul in Music on November 6, 2012

In preparation for the upcoming Music Hack Day Boston, I was running the latest build of The Echo Nest Remix through its paces.  As part of the Remix checkout I thought I’d try running Psy’s Gangnam Style through Tristan’s Swinger. The results are amusing.  Listen to Psywinger:

[audio http://static.echonest.com.s3.amazonaws.com/InfiniteGangnamStyle/GangnamStyle-Swung.mp3]

 

The code for this marvelous hack can be found here:  The Swinger on Github.

Leave a comment

Infinite Gangnam Style for iPhone

Posted by Paul in Music on November 3, 2012

One of the most frequent complaints I’ve heard about Infinite Gangnam Style is that it is hard to take it with you.  Due to the image swapping pseudo-video in the desktop version, it doesn’t run very well on iOS devices.  This means you can’t listen to Gangnam  Style 24/7.   Clearly, since Infinite Gangnam Style is supposed to let you listen to Gangnam Style forever, this would not do.  So, I’ve made an iPhone specific version called  Gangn∞m Style:

This version plays on your iPhone device through Mobile Safari, letting you listen to a never-ending, every changing version of Gangnam Style wherever you are and whenever you want.   So now there’s no excuse to ever stop listening to the greatest pop song of the millenium.

Some tech details: The hardest bit was figuring out how to prevent the iPhone from going to sleep after a minute or so of playing.  There’s no direct way to disable sleep mode in the browser, but there’s a little hack.  I created a 5 second long silent mp3, and then force that silence to loop forever with a bit of html:

        <audio src="silence.mp3" preload autoplay loop></audio>

This keeps Safari live and prevents the iPhone from deciding it wants to go to sleep. Of course this will wear down your battery, but that is one of the risks inherent in infinite listening.

I used jQTouch to give a nifty iOS look and feel in the browser, and of course I used the Echo Nest analyzer to figure out where all the beats were in the song, and to build the beat by beat similarity graph that I use to make the song play forever.

Go check out  Gangn∞m Style and let me know what you think.

7 Comments