Archive for category Music
Six Degrees of Black Sabbath (v2)
Posted by Paul in code, fun, Music, The Echo Nest on January 6, 2014
For my Christmas vacation programming project this year, I revisited an old hack: Six Degrees of Black Sabbath. I wrote the original, way back in 2010 at the very first San Francisco Music Hack Day. That version is still up and running, and getting regular visits, but it is getting a bit long in the tooth and so I’ve given it a total rewrite from the ground up. The result is the new Six Degrees of Black Sabbath:
Six Degrees of Black Sabbath is like the Oracle of Bacon but for music. It lets you find connections to just about any two artists based upon their collaborations. Type in the name of two artists, and 6dobs will give you a pathway showing the connections that will get you from one artist to another. For instance, if you enter ‘The Beatles’ and ‘Norah Jones’ you’ll get a path like:
- We start with The Beatles
- The Beatles had member George Harrison
- George Harrison performed with Ravi Shankar on the song Bangla Dhun and 26 others.
- Ravi Shankar was parent of Norah Jones
If you don’t like a particular connection, you can bypass it generating a new path. For instance, if we bypass Ravi Shankar, it will take us eight steps to get to Norah Jones from the Beatles:
The Beatles -> Paul McCartney -> The Fireman -> Youth -> Pigface
-> Mike Dillon ->Garage A Trois -> Charlie Hunter -> Norah Jones
Not all connections are created equal. Mick Jagger and Keith Richards have been playing together for over 50 years in the Rolling Stones. That’s a much stronger connection than the one between Mick Jagger and Fergie for performing a single song together at the Rock and Roll Hall of Fame. We take these connection strengths into account when finding paths between artists. Preference is given to stronger connections, even if those stronger connections will yield a longer path.
The new version of Six Degrees of Black Sabbath has a number of new features:
Video – Each step in a path is represented by a Youtube video – often with a video by the two artists that represent that step. I’m quite pleased at how well the video works for establishing the connection between two artists. Youtube seems to have it all.
Live stats – The app tracks and reports all sorts of things such as the longest path discovered so far, the most frequently occurring artists on paths, the most connected artists, most searched for artists and so on.
Larger database of connections – the database has about a quarter million artists and 2.5 million artist-to-artist connections.
Autocomplete for artist names – no need to try to remember how to spell ‘Britney Spears‘ – just start typing the parts you know and it will sort it out.
Spiffier looking UI – It still looks like it was designed by an engineer, but at least it looks like it was designed in this decade by an engineer.
Path finding improvements – faster and better paths throughout.
Revisiting this app after 4 years was a lot of fun. I got to dive deep into a bunch of tech that was new to me including Redis, Bootstrap 3, and the YouTube video search API. I spent many hours untangling the various connections in the new Musicbrainz schema. I took a tour through a number of Pythonic network graph libraries (Networkx, igraph and graph-tool), I learned a lot about Python garbage collection when you have a 2.5gb heap.
Give the app a try and let me know what you think.
Million Song Shuffle
Back in 2001 when the first iPod was released, Shuffle Play was all the rage. Your iPod had your 1,000 favorite songs on it, so picking songs at random to play created a pretty good music listening experience. Today, however, we don’t have 1,000 songs in our pocket. With music services like Rdio, Rhapsody or Spotify, we are walking around with millions of songs in our pocket. I’ve often wondered what it would be like to use Shuffle Play when you have millions of songs to shuffle through. Would it be a totally horrible listening experience listening to artists that are so far down the long tail that they don’t even know that they are part of a dog? Would you suffer from terminal iPod whiplash as you are jerked between Japanese teen pop and a John Philip Sousa march?
To answer these questions, I built an app called Million Song Shuffle. This app will create a playlist by randomly selecting songs from a pool of many millions of songs. It draws from the Rdio collection and if you are an Rdio user you can hear listen to the full tracks.
The app also takes advantage of a nifty new set of data returned by the Echo Nest API. It shows you the absolute hotttnesss rank for the song and the artist, so you will always know how deep you are into the long tail (answer: almost always, very deep).
So how is listening to millions of songs at random? Surprisingly, it’s not too bad. The playlist certainly gets a high score for eclecticism and surprise, and most of the time the music is quite listenable. But give it a try, and form your own opinion.
Its fun, too, to see how long you can listen to the Million Song Shuffle before you encounter a song or even an artist that you’ve heard of before. If the artist is not in the top 5K artists, it is likely you’ve never heard of them. After listening to Million Song Shuffle for a little while you start to get an idea of how much music there is out there. There’s a lot.
For the ultimate eclectic music listening experience, try the Million Song Shuffle.
Have a Very Nestive Christmas
Posted by Paul in code, Music, playlist, The Echo Nest on December 19, 2013
We are approaching peak Christmas music season. That means that many of us are getting really sick of hearing the same Christmas songs over and over. One can only hear Bing Crosby’s White Christmas so many times before measures must be taken. To remedy this situation, this morning I created a quick web app that let you chose from among a number of different Christmas genres (from classical to heavy metal) to let you add a little variety to your Christmas mix. If you are getting weary of the Christmas standards, but still want to listen to Christmas music, you may want to give it a try: The Christmas Playlister
Scary and Stretched
In the last few months I’ve found myself listening to Skrillex non-stop – usually because I’m working on some sort of Skrillexed-based hack. One thing about Skrillex – his music is quite layered, there’s lots of interesting sounds packed into every second of a song. I thought I’d explore this layering a little bit by applying Paul’s Stretch to Scary Monsters and Nice Sprites. The effect is quite pleasing.
Who are the shallowest artists?
Posted by Paul in data, Music, The Echo Nest on November 25, 2013
Yesterday, I wrote about who the Deepest Artists are. So naturally, today I’ll turn that on its head and take a look at who are the Shallowest Artists. I define a shallow artist as an artist that despite having a substantial number of released songs, has most listens concentrated in their top five tracks. These are the artists that are best known for just a small number of songs.
For each artist, I’ve calculated a Shallowness Score which is merely the percentage of an artist’s plays that occurs in an artist’s top 5 songs. A Shallowness Score of 71% means that 71% of all listens occur in the top 5 songs. Thus, 71% of all listens to Survivor (of Eye of the Tiger fame) are found in their top 5 songs.
Update: This post used to reference the Pitch Perfect Treblemakers, but Glenn points to an ambiguous artist issue with the Treblemakers where multiple artists were conflated. The Pitch Pefect Treblemakers only have 4 songs so they are no longer a candidate for this list.
Here are the top 15 Shallowest Artists. Click to see the full chart:
Click to see the full chart
As you’d expect, there are plenty of new artists on the list, artists like Icona Pop, Avicii and Zedd that have had a few charting songs. Being tagged as a shallow artist isn’t necessarily bad, it just means that your music is dominated by a handful of hits. That’s why we find Adele and Jeff Buckley on the same list as Paris Hilton and Smash Mouth.
Check out the full list of the Shallowest Artists as well as the full list of the Deepest Artists.
Who are the deepest artists?
Posted by Paul in data, fun, Music, The Echo Nest on November 24, 2013
Our playlists our filled with One Hit Wonders like My Sharona, Tainted Love and Final Countdown. One Hit wonders are the non-nutritious food of the music world – they are Twinkie’s, the Ho Hos and the Yodels of our musical diet. But what should we listen to when we want a full and nutritious musical meal? We should look for music by artists that have deeper catalogs – artists where the fans spend substantial time listening to the non-hits. These are the Deep Artists, the opposite of the One Hit Wonders – the artists that you can spend months or years listening to and exploring their collection.
Unfortunately, there’s no master list of Deep Artists – but I have lots of music listener data, so I figured I could build one. Here’s what I did. First I restricted my results to somewhat familiar artists with at least 100 songs in their catalog. I then scored each artist by the percentage of song plays that occur in the deep catalog versus the total plays for the artist – where deep catalog means a song that is not in the top ten for that artist. This gives each artist a Deepness Score that I could then use to sort artists to give us a list of the Deepest Artists. Here are the top ten:
Click for the full list
Not surprising to see Johann Sebastian Bach at number two. Bach has no real ‘hits’ – and indeed has an incredibly deep catalog. 90% of all Bach plays occur in Bach’s non-top 10. The number one deep artist is Vitamin String Quartet – they have 3500 covers of songs with no clear hits among them.
http://rd.io/x/QFqwK0FVUAYLooking at the full list we see jam bands like Phish and Grateful Dead, AOR staples like Pink Floyd and David Bowie.
I’ve built a list of a little over 500 of the Deepest Artists. These are artists that have a deepness score of 50% or greater – meaning that at least 50% of all listens for the artist is in the deeper cuts. This Thanksgiving if you are looking for some more nutritious music, stay away from Alice’s Restaurant and other One Hit Wonders and listen to music by artists on this Deep Artists list.
Update: Glenn looked at these results and felt that a nutritious music meal shouldn’t include Vitamin String Quartet (it’s the ‘artificially-fortified sugar-coated cereal of music’ according to Glenn), so Glenn took a different approach with different results. Glenn calls his results boring, but I think they are quite interesting. Read his post: Good Boring results
Children of the Hack (Angry Birds Edition)
Posted by Paul in data, events, hacking, Music, music hack day, The Echo Nest on November 18, 2013
I’m writing this post from Espoo Finland which is home to three disruptive brands: Nokia, who revolutionized the mobile phone market in the 1990s with its GSM technology; Rovio, who brought casual gaming to the world with Angry Birds; and Children of Bodom perhaps one of the most well known melodic death metal bands. So it is not surprising that Espoo is a place where you will find a mix of high tech, playfulness and hard core music – which is exactly what I found this past weekend at the Helsinki Music Hack Day hosted at the Startup Sauna in Espoo Finland.
At the Helsinki Music Hack Day, dozens of developers gathered to combine their interest in tech and their passion for music in a 24 hour hacking session to build something that was music related. Representatives from tech companies such as SoundCloud, Spotify and The Echo Nest joined the hackers to provide information about their technologies and guidance in how to use their APIs.
After 24 hours, a dozen hacks were demoed in the hour-long demo session. There was a wide range of really interesting hacks. Some of my favorites are highlighted here:
Cacophony – A multi-user remote touch controlled beat data sequencer. This hack used the Echo Nest (via the nifty new SoundCloud/Echo Nest bridge that Erik and I built on the way to Espoo), to analyze music and then allow you to use the beats from the analyzed song to create a 16 step sequencer. The sequencer can be controlled remotely via a web interface that runs on an iPad. This was a really nice hack, the resulting sequences sounded great. The developer, Pekka Toiminen used music from his own band Different Toiminen which has just released their first album. You can see the band and Pekka in the video:
[youtube http://www.youtube.com/watch?v=nLwrTf5JQ5U]It was great getting to talk to Pekka, I hope he takes his hack further and makes an interactive album for his band.
Hackface & Hackscan – by hugovk – This is a pretty novel set of hacks. Hackface takes the the top 100 or 1000 artists from your listening history on Last.fm, finds photos of the artists (via the Echo Nest API), detects faces using a face detection algorithm, intelligently resizes them and composites them into a single image giving you an image of what your average music artist in your listening history looks like.
Hackscan – takes a video and summarizes it intelligently into a single image by extracting single columns of pixels from each frame. The result is a crazy looking image that captures the essence of the video.
Hugo was a neat guy with really creative ideas. I was happy to get to know him.
Stronger Harder Faster Jester – Tuomas Ahva and Valtteri Wikstrom built the first juggling music hack that I’ve seen in the many hundreds of hack demos I’ve witnessed over the years. Their hack used three bluetooth-enabled balls that when thrown triggered music samples.
Photo via @tuomasahvra
The juggler juggles the balls in time with the music and the ball tossing triggers music samples that align with the music. The Echo Nest analysis is used to extract the salient pitch info for the aligment. It was a really original idea and great fun to watch and listen to. This hack won the Echo Nest prize.
µstify – This is the classic boy meets girl story. Young man at his first hackathon meets a young woman during the opening hours of the hackathon.
They decide to join forces and build a hack (It’s Instagram for Music!) and two days later they are winning the hackathon! Alexandra and Arian built a nifty hack that builds image filters (in the style of Instagram) based upon what the music sounds like. They use The Echo Nest to extract all sorts of music parameters and use these to select image filters. Check out their nifty presentation.
Gig Voter – this Spotify app provides a way for fans to get their favorite artists to come to their town. Fans from a town express an interest in an artist. Artists get a tool or helping them plan their tour based on information about where their most active fans actually are as well as helping them sell gigs to location owners by being able to prove that there is demand for them to perform at a certain location. Gig Voter uses Echo Nest data to help with the search and filtering.
Hit factory – Hit Factory is a generative music system that creates music based upon your SoundCloud tastes and adapts that music based upon your feedback . Unfortunately, no samples of the music are to be found online, but take my word, they were quite interesting – not your usual slightly structured noise.
Abelton Common Denominator – a minimal, mini-moog style interface to simplify the interaction with Abelton – by Spotify’s Rikard Jonsson.
Swap the Drop – this was my hack. You can read more about it here.
One unusual aspect of this Music Hack Day was that a couple of teams that encountered problems and were unable to finish their hacks still got up and talked about their failures. It was pretty neat to see hardcore developers get up in front of a room full of their peers and talk about why they couldn’t get Hadoop to work on their terrabyte dataset or get their party playlister based on Meteor to run inside Spotify.
I’ve enjoyed my time in Espoo and Helsinki. The Hack Day was really well run. It was held in a perfect hacking facility called the Startup Sauna.
There was plenty of comfortable hacking spots, great wifi, and a perfect A/V setup.
@sferik in a classic hacking position
The organizers kept us fed with great food (Salmon for lunch!), great music, including a live performance by Anni.
Anni performs. Photo by @sferik
There was plenty of Angry Birds Soda.
Many interesting folks to talk to …
Thanks to Lulit and the rest of the Aaltoes team for putting together such a great event.
Swap the Drop
Posted by Paul in code, data, events, fun, hacking, Music, music hack day, The Echo Nest on November 17, 2013
I’ve been in Helsinki this weekend (which is not in Sweden btw) for the Helsinki Music Hack Day. I wanted to try my hand at a DJ app that will allow you to dynamically and interactively mix two songs. I started with Girl Talk in a Box, ripped out the innards and made a whole bunch of neat changes:
- You can load more than one song at a time. Each song will appear as its own block of music tiles.
- You can seamlessly play tiles from either song.
- You can setup branch points to let you jump from an point in one song to any point in another (or the same) song.
- And the killer feature – you can have two active play heads allowing you to dynamically interact with two separate audio streams. The two play heads are always beat matched (the first play head is the master that sets the tempo for everyone else). You can cross-fade between the two audio streams – letting you move different parts of the song into the foreground and the background.
All the regular features of Girl Talk in a Box are retained – bookmarks, arrow key control, w/a/s/d navigation and so on. See the help for more details on the controls.
You can try the app here: Swap the Drop
SoundCloud => Echo Nest bridge service
On my way to Helsinki for the Music Hack Day I ran into Erik Michaels-Ober from SoundCloud. While sitting in the Frankfurt airport, we decided it would be pretty cool if it was super easy for developers to get the Echo Nest analysis for any SoundCloud track. So we put together a little prototype webservice that does just that. You give the web service a SoundCloud track ID, URN or URI and the service gives you back the corresponding Echo Nest Track ID that you can use to fetch the detailed Echo Nest analysis.
The service is deployed here: SCAnalyzer. There’s a quick, dirty and ugly demo of the service here: SCAnalyzer Demo. All the code is on github.
You have 24 hours to build the future of music
Posted by Paul in data, fun, hacking, Music, music hack day, The Echo Nest on November 12, 2013
A Music Hack Day is unlike most other hackathons. There are no mega-prizes for the best hacks. There are no VCs wandering the hacker hallways trolling for the next startup. There are no briefs that describe the types of apps that you should build. Hackers don’t go to a Music Hack Day to win big prizes, or to launch their startup. Hackers go to Music Hack Days because they love music and they love to build stuff. At a Music Hack Day these passionate builders get to apply their talents to music, surrounded by like-minded peers and build their version of the future of music. The currency at a Music Hack Day is not money or VC attention, the currency is creativity. The Music Hack Day prize is knowing that you’ve built something cool enough to delight other music hackers.
So what happens at a Music Hack Day? How does it all work? What kind of hacks do people build? Read on to see exactly what happened at the Boston Music Hack Day 2013, held this last weekend.
Boston Music Hack Day
This weekend, hundreds of folks who are passionate about music and technology got together in Cambridge MA for Boston Music Hack Day 2013. The event was hosted at the Microsoft NERD – a wonderful facility that Microsoft makes available for all sorts of programmer events. Registration started at 9AM and by 10AM hackers were breakfasted and ready to go.
The event started off with some opening remarks by your truly, describing how a Music Hack Day works and how to have a successful event (meet other people, learn new stuff, build something, make sure you finish it, demo it and have fun).
Build Something! – key advice for a Music Hack Day
Hackers riveted by Paul Lamere’s opening remarks
Short technology presentations
Next up, organizations that had some sort of music technology such as an API or new gizmo that might be interesting to music hackers spent a few minutes talking about their technology. For many hackers, this was their first exposure to the music ecosystem – they don’t know what APIs are available for building apps so learning about music streaming APIs from companies like SoundCloud, Rdio and Spotify, and learning about all the music data available from APIs like The Echo Nest and the Free Music Archive is really important.
Matt talks about the Reverb.com API
There were a few interesting devices available for hackers at the event. Techogym brought a high tech treadmill with its own API hoping that music hackers would build music-related exercise apps. Muzik brought a set of headphones that are instrumented with accelerometers and other sensors allowing for apps to adapt to the actions of the listener.
Hackers trying out the Technogym
Project Pitches
Sometimes hackers come to a Music Hack Day with ideas ready to go. Sometimes hackers come with their skills but no ideas. At the Project Pitch session, hackers had a minute to pitch their idea or to offer their skills. About 20 hackers braved the front of the room describing their idea or their skill set. One hacker described his project as help me with my homework (and yes, this hacker did find a teammate and they ultimately built a nifty hardware hack that satisfied the homework requirement too).
The homework as hack demo
Tech Deep Dives
Next up on the schedule were the Tech Deep dives. Organizations had a half-hour to give a deeper view of what their technology is capable of. Some hackers want to know more about how to do particular things with an API or technology. This is their opportunity to find out all about the nuts and bolts, to ask questions from the experts. The Tech Deep Dives are strictly optional – many hackers skip them and instead start forming teams, sharing ideas, initializing their repos and writing code.
Kevin does the Rdio API deep dive
Hack Time
After all the preliminaries are over it was finally time to start hacking.
Hackers formed teams, large and small.
The competition for the best vest was fierce:
They staked out a comfortable workspace in chairs …
Or on the floor …
There were some very creative hardware hacks:
Software hacks:
Plenty of good food
And lots of fun
Overnight Hacking
The Microsoft NERD was only available until 9PM – after that we moved over to hack/reduce – a wonderful hacking space a five minute walk away. There we were greeted by a perfect hacking space with lots of great wifi, great hacker lighting, and lots of beer. Hacking continued all night. Some hackers did try to get some sleep (either at the hack, or back at home), but some hardcore hackers stayed the whole night.
By 9AM on Sunday morning, the hackers were back at the NERD, for lots of coffee, some breakfast and then more hacking.
Hacking ends
At 2:30, hacking was officially over, and teams submitted their projects to Hacker League. Sixty hacks were submitted.
Demo Session
By 3PM the 200 hackers had all gathered back into the big room joined by a hundred folks who had come just to see the demo session. Hackers had two minutes to show their stuff. It is a hard demo to give. You are giving a demo of software that you’ve just finished building. It might have some bugs. The WiFi is a little flaky, you haven’t slept in 24 hours, your hands are shaking from too much coffee and too much nervousness, you have to type while holding a microphone and your laptop just won’t sync with the projector just right, and the audio isn’t coming out of the speakers, and the colors look all wrong on the screen. All in front of 300 people. I’ve done it dozens of times and it still is a really scary demo to give. But it is incredibly exhilarating too – to take nothing but an idea and turn it into something that can amaze or amuse a room full of tech elite in 24 hours. It is quite a rush.
Here I am giving my demo, heart pounding, blood racing …
There were two A/V setups so while one team was presenting, the next team was setting up. This allowed us to get through 60 demos in just over two hours. There was a very low incidence of demo fail. And only two inappropriate demos (one was a 2 minute powerpoint presentation with no tech built, the other was a 2 minute tech commercial for a product). I was worried that we might have a #titstare moment with one hack that seemed to contain questionable content but that hacker apparently decided not to present.
The 60 hacks represented a wide range of domains. There were games, music learning tools, programs designed to create, manipulate, remix and even destroy music. I’d love to cover them all, but there are just too many.
The full list is on hacker league. Here are some of my absolute favorites:
String Theory – A musical instrument and sound sculpture build from yearn and stretch sensors and powered by an Arduino.
String Theory
The Lone Arranger – a terminal app that allows you to easily rearrange your audio. By a father and son hacking team.
The Secret History of Music – combs biographies, lyrics, and commentary from song meanings from two artists, combines them into one fictional artist, and uses Markov chain magic to generate a 50K novel about this new fictional band.
LED Soundsystem – this hack attempts to generate a light show synchronized to the music. It has a special place in my heart because the hacking team was working on the same problem I was working on for my abandoned hack – i.e. automatically finding the ‘drop’ in a song. Unfortunately, this team had a demo fail – but they are smart guys and I expect to see good stuff from them at the next hack day.
eHarmonica – an electronic harmonica!
Enter the dragon – In today’s world, everyone deserves a spectacular entrance. And we intend to give it to them. Enter the Dragon uses bluetooth technology to detect when a user enters the room and plays their personalized entrance music.
Dadabots – Dadabots are bot accounts on creative websites that make procedural creations or remixes of other creations
ios SoundPuzzle – A simple iOS ear training game built programmatically from the free music archive and the echonest remix api.
danceomatic – totally awesome automatic choreography from an mp3 and a web based stick figure performance.
Jotunnslayer – Never again listen to power metal without slaying ice giants. Die in battle. Earn your place in Valhalla.
Give Me Liberty or Give Me Death Metal – Political representation via musical exploration
Echos – Choose your favorite song and shoot to the beat! Fight enemies that respond to and are controlled by the music! Listen closely and experience unparalleled power as you get into the groove! Enjoy addictive arcade-style game-play in this twist on a classic formula.
TweetTones – a native iOS application that generates synthesized music from tweets in real-time.
Short-Attention Span Playlist Scanner – Glenn made a radio scanner that find and plays just the choruses.
ionian Eclipse – A web-based multiplayer top-down space shooter with procedurally generated enemies and interactions driven by music events.
CFD is Colorful Fluid Dynamics: A HTML5 Music Visualizer and Spotify Plugin – A music visualizer written entirely in vanilla javascript/canvas that is based on computational fluid dynamics and a complex particle system.
ColorMe – Ever wondered what your music looked like? Now you can look at songs by your favorite artist with this super fun web app, powered by the Echo Nest.
How Repetitive – measures how often audio segments repeat themselves within in a given song.
Jason’s music visualizer – an html/css visualizer on steroids.
And last, but by no means least, Jonathan’s awesome MIDI Digester that converts audio to MIDI and back, over and over to generate some very strange sounds. The very essence of the music.
There are so many excellent hacks, I’m sure I’ve missed many notables. Luckily, Evolver.fm covered the event, so expect to see Eliot’s writups on all the best hacks on Evolver.fm.
Prizes
At the end of the mega demo session, there’s a brief prize awarding ceremony where a half-dozen organizations give out modest prizes for hackers that made cool stuff using their tech.
Finally we adjourned to the local pub for some food, beer and hacking recaps.
Special thanks to the organizers of the event. The Music Hack Day would not happen without Elissa and Matt. They do all the hard work. Finding the venue, wrangling the sponsors and volunteers, making a mega Costco food run, dealing with the A/V, running the registration, selecting and hiring the caterers, designing t-shirts and so much more. There’s a huge amount of work that goes into planning the event, much more than meets the eye. Elissa and Matt are the unsung heroes of Music Hack Day. We should make a music hack to sing their song.
Elissa and Matt. The Unsung Music Hack Day Heros
Thanks also to the event sponsors: Rdio, Spotify, Microsoft, hack/reduce, Free Music Archive, SoundCloud, Mailchimp and The Echo Nest, and the many volunteers who came and helped us run the whole show.
Some of the awesome and unusually attractive volunteers at Music Hack Day Boston
More Music Hack Days
Interested in going to a Music Hack Day? Check out the Music Hack Day calendar for upcoming events. There’s one in Helsinki this weekend, and there’s one in London in just a few weeks. More events are rumored to be in the planning stages for 2014.
(Photos mostly by Michelle Ackerman, a few by me)
Music Machinery 