Archive for category data
Yesterday, I wrote about who the Deepest Artists are. So naturally, today I’ll turn that on its head and take a look at who are the Shallowest Artists. I define a shallow artist as an artist that despite having a substantial number of released songs, has most listens concentrated in their top five tracks. These are the artists that are best known for just a small number of songs.
For each artist, I’ve calculated a Shallowness Score which is merely the percentage of an artist’s plays that occurs in an artist’s top 5 songs. A Shallowness Score of 71% means that 71% of all listens occur in the top 5 songs. Thus, 71% of all listens to Survivor (of Eye of the Tiger fame) are found in their top 5 songs.
Update: This post used to reference the Pitch Perfect Treblemakers, but Glenn points to an ambiguous artist issue with the Treblemakers where multiple artists were conflated. The Pitch Pefect Treblemakers only have 4 songs so they are no longer a candidate for this list.
Here are the top 15 Shallowest Artists. Click to see the full chart:
As you’d expect, there are plenty of new artists on the list, artists like Icona Pop, Avicii and Zedd that have had a few charting songs. Being tagged as a shallow artist isn’t necessarily bad, it just means that your music is dominated by a handful of hits. That’s why we find Adele and Jeff Buckley on the same list as Paris Hilton and Smash Mouth.
Our playlists our filled with One Hit Wonders like My Sharona, Tainted Love and Final Countdown. One Hit wonders are the non-nutritious food of the music world – they are Twinkie’s, the Ho Hos and the Yodels of our musical diet. But what should we listen to when we want a full and nutritious musical meal? We should look for music by artists that have deeper catalogs – artists where the fans spend substantial time listening to the non-hits. These are the Deep Artists, the opposite of the One Hit Wonders – the artists that you can spend months or years listening to and exploring their collection.
Unfortunately, there’s no master list of Deep Artists – but I have lots of music listener data, so I figured I could build one. Here’s what I did. First I restricted my results to somewhat familiar artists with at least 100 songs in their catalog. I then scored each artist by the percentage of song plays that occur in the deep catalog versus the total plays for the artist – where deep catalog means a song that is not in the top ten for that artist. This gives each artist a Deepness Score that I could then use to sort artists to give us a list of the Deepest Artists. Here are the top ten:
Not surprising to see Johann Sebastian Bach at number two. Bach has no real ‘hits’ – and indeed has an incredibly deep catalog. 90% of all Bach plays occur in Bach’s non-top 10. The number one deep artist is Vitamin String Quartet – they have 3500 covers of songs with no clear hits among them.
Looking at the full list we see jam bands like Phish and Grateful Dead, AOR staples like Pink Floyd and David Bowie.
I’ve built a list of a little over 500 of the Deepest Artists. These are artists that have a deepness score of 50% or greater – meaning that at least 50% of all listens for the artist is in the deeper cuts. This Thanksgiving if you are looking for some more nutritious music, stay away from Alice’s Restaurant and other One Hit Wonders and listen to music by artists on this Deep Artists list.
Update: Glenn looked at these results and felt that a nutritious music meal shouldn’t include Vitamin String Quartet (it’s the ‘artificially-fortified sugar-coated cereal of music’ according to Glenn), so Glenn took a different approach with different results. Glenn calls his results boring, but I think they are quite interesting. Read his post: Good Boring results
It has been nearly 10 years since Chris Anderson’s Wired article and subsequent book called The Long Tail. In the article, book, and subsequent blog posts Chris (and I can call him Chris because we once had a 3 minute conversation in Bilbao Spain, so we are friends) showed data about how music listening is changing as we move away from the physical constraints of CD shelves and replace them with the infinite virtual shelves of the online music store.
I thought it was time to look at the data again to see if the trends that my good friend Chris was seeing back in 2004 still persist today. In particular, in the blog post Latest Rhapsody data and more Chris showed how a substantial fraction of the music market is shifting away from the Walmart inventory of the top 50,000 tracks:
This chart shows that as Rhapsody’s collection size increased the amount of listener market share in the songs that were not in the top 50K grew from 26%, to 28% and then 30% over 3 years. Today’s music subscription services boast 10 million or more songs (but of course, those numbers start to get a bit meaningless beyond a certain point – it becomes filler). Let’s take a look to see if the fraction of listening that is not in the top 50K tracks has continued to grow. Here’s some pie:
This data shows that in 2013, with 10million+ tracks available, 42% of listens can be found in the long tail (i.e. beyond the top 50K tracks). So the trend has continued. More listening is taking place in less popular music.
In the same blog post, my pal Chris presents this Hitland vs. nicheland chart that shows what percentage of the music business is selling the top 100 artists. Back in 2006, 50% of Walmart’s music business was selling music by the top 100 artists, while for Rhapsody, about 25% of market share was in the top 100 artists.
Lets’s extend this chart using listening data from 2013:
As you can see, the trend continues, only 20% of listening is in the top 100 artists. It’s not a dramatic change, but it does show that the more music you make available, the deeper the catalog, the deeper the listening.
Here are a few more fun facts about today’s listening. 80% of all listening is concentrated in the top 5,000 artists. The top 1,000 songs account for about 13% of all listening, and 80% of all listening is spread over about 222,0000 songs.
My good buddy Chris’s Long Tail hypothesis has come under considerable scrutiny in the last few years, but by looking at the data we see that the trends Chris pointed out have continued. Our listening is less concentrated in the hits than ever before. Yes, hits are important and will alway be, but if you make more music available to listeners, they will indeed listen to it. So, remember my best friend’s three rules for the Long Tail:
- Make everything available
- Cut the price in half – now lower it
- Help me find it.
At The Echo Nest, we work hard to make rule 3 a reality – our mission is to help connect people with the best music, whether it be in the short head, or deep into that Long Tail.
I’m writing this post from Espoo Finland which is home to three disruptive brands: Nokia, who revolutionized the mobile phone market in the 1990s with its GSM technology; Rovio, who brought casual gaming to the world with Angry Birds; and Children of Bodom perhaps one of the most well known melodic death metal bands. So it is not surprising that Espoo is a place where you will find a mix of high tech, playfulness and hard core music – which is exactly what I found this past weekend at the Helsinki Music Hack Day hosted at the Startup Sauna in Espoo Finland.
At the Helsinki Music Hack Day, dozens of developers gathered to combine their interest in tech and their passion for music in a 24 hour hacking session to build something that was music related. Representatives from tech companies such as SoundCloud, Spotify and The Echo Nest joined the hackers to provide information about their technologies and guidance in how to use their APIs.
After 24 hours, a dozen hacks were demoed in the hour-long demo session. There was a wide range of really interesting hacks. Some of my favorites are highlighted here:
Cacophony – A multi-user remote touch controlled beat data sequencer. This hack used the Echo Nest (via the nifty new SoundCloud/Echo Nest bridge that Erik and I built on the way to Espoo), to analyze music and then allow you to use the beats from the analyzed song to create a 16 step sequencer. The sequencer can be controlled remotely via a web interface that runs on an iPad. This was a really nice hack, the resulting sequences sounded great. The developer, Pekka Toiminen used music from his own band Different Toiminen which has just released their first album. You can see the band and Pekka in the video:[youtube http://www.youtube.com/watch?v=nLwrTf5JQ5U]
It was great getting to talk to Pekka, I hope he takes his hack further and makes an interactive album for his band.
Hackface & Hackscan – by hugovk – This is a pretty novel set of hacks. Hackface takes the the top 100 or 1000 artists from your listening history on Last.fm, finds photos of the artists (via the Echo Nest API), detects faces using a face detection algorithm, intelligently resizes them and composites them into a single image giving you an image of what your average music artist in your listening history looks like.
Hackscan – takes a video and summarizes it intelligently into a single image by extracting single columns of pixels from each frame. The result is a crazy looking image that captures the essence of the video.
Hugo was a neat guy with really creative ideas. I was happy to get to know him.
Stronger Harder Faster Jester – Tuomas Ahva and Valtteri Wikstrom built the first juggling music hack that I’ve seen in the many hundreds of hack demos I’ve witnessed over the years. Their hack used three bluetooth-enabled balls that when thrown triggered music samples.
The juggler juggles the balls in time with the music and the ball tossing triggers music samples that align with the music. The Echo Nest analysis is used to extract the salient pitch info for the aligment. It was a really original idea and great fun to watch and listen to. This hack won the Echo Nest prize.
µstify – This is the classic boy meets girl story. Young man at his first hackathon meets a young woman during the opening hours of the hackathon.
They decide to join forces and build a hack (It’s Instagram for Music!) and two days later they are winning the hackathon! Alexandra and Arian built a nifty hack that builds image filters (in the style of Instagram) based upon what the music sounds like. They use The Echo Nest to extract all sorts of music parameters and use these to select image filters. Check out their nifty presentation.
Gig Voter – this Spotify app provides a way for fans to get their favorite artists to come to their town. Fans from a town express an interest in an artist. Artists get a tool or helping them plan their tour based on information about where their most active fans actually are as well as helping them sell gigs to location owners by being able to prove that there is demand for them to perform at a certain location. Gig Voter uses Echo Nest data to help with the search and filtering.
Hit factory – Hit Factory is a generative music system that creates music based upon your SoundCloud tastes and adapts that music based upon your feedback . Unfortunately, no samples of the music are to be found online, but take my word, they were quite interesting – not your usual slightly structured noise.
Abelton Common Denominator – a minimal, mini-moog style interface to simplify the interaction with Abelton – by Spotify’s Rikard Jonsson.
Swap the Drop – this was my hack. You can read more about it here.
One unusual aspect of this Music Hack Day was that a couple of teams that encountered problems and were unable to finish their hacks still got up and talked about their failures. It was pretty neat to see hardcore developers get up in front of a room full of their peers and talk about why they couldn’t get Hadoop to work on their terrabyte dataset or get their party playlister based on Meteor to run inside Spotify.
I’ve enjoyed my time in Espoo and Helsinki. The Hack Day was really well run. It was held in a perfect hacking facility called the Startup Sauna.
There was plenty of comfortable hacking spots, great wifi, and a perfect A/V setup.
The organizers kept us fed with great food (Salmon for lunch!), great music, including a live performance by Anni.
There was plenty of Angry Birds Soda.
Many interesting folks to talk to …
Thanks to Lulit and the rest of the Aaltoes team for putting together such a great event.
I’ve been in Helsinki this weekend (which is not in Sweden btw) for the Helsinki Music Hack Day. I wanted to try my hand at a DJ app that will allow you to dynamically and interactively mix two songs. I started with Girl Talk in a Box, ripped out the innards and made a whole bunch of neat changes:
- You can load more than one song at a time. Each song will appear as its own block of music tiles.
- You can seamlessly play tiles from either song.
- You can setup branch points to let you jump from an point in one song to any point in another (or the same) song.
- And the killer feature – you can have two active play heads allowing you to dynamically interact with two separate audio streams. The two play heads are always beat matched (the first play head is the master that sets the tempo for everyone else). You can cross-fade between the two audio streams – letting you move different parts of the song into the foreground and the background.
All the regular features of Girl Talk in a Box are retained – bookmarks, arrow key control, w/a/s/d navigation and so on. See the help for more details on the controls.
You can try the app here: Swap the Drop
A Music Hack Day is unlike most other hackathons. There are no mega-prizes for the best hacks. There are no VCs wandering the hacker hallways trolling for the next startup. There are no briefs that describe the types of apps that you should build. Hackers don’t go to a Music Hack Day to win big prizes, or to launch their startup. Hackers go to Music Hack Days because they love music and they love to build stuff. At a Music Hack Day these passionate builders get to apply their talents to music, surrounded by like-minded peers and build their version of the future of music. The currency at a Music Hack Day is not money or VC attention, the currency is creativity. The Music Hack Day prize is knowing that you’ve built something cool enough to delight other music hackers.
So what happens at a Music Hack Day? How does it all work? What kind of hacks do people build? Read on to see exactly what happened at the Boston Music Hack Day 2013, held this last weekend.
Boston Music Hack Day
This weekend, hundreds of folks who are passionate about music and technology got together in Cambridge MA for Boston Music Hack Day 2013. The event was hosted at the Microsoft NERD – a wonderful facility that Microsoft makes available for all sorts of programmer events. Registration started at 9AM and by 10AM hackers were breakfasted and ready to go.
The event started off with some opening remarks by your truly, describing how a Music Hack Day works and how to have a successful event (meet other people, learn new stuff, build something, make sure you finish it, demo it and have fun).
Short technology presentations
Next up, organizations that had some sort of music technology such as an API or new gizmo that might be interesting to music hackers spent a few minutes talking about their technology. For many hackers, this was their first exposure to the music ecosystem – they don’t know what APIs are available for building apps so learning about music streaming APIs from companies like SoundCloud, Rdio and Spotify, and learning about all the music data available from APIs like The Echo Nest and the Free Music Archive is really important.
There were a few interesting devices available for hackers at the event. Techogym brought a high tech treadmill with its own API hoping that music hackers would build music-related exercise apps. Muzik brought a set of headphones that are instrumented with accelerometers and other sensors allowing for apps to adapt to the actions of the listener.
Sometimes hackers come to a Music Hack Day with ideas ready to go. Sometimes hackers come with their skills but no ideas. At the Project Pitch session, hackers had a minute to pitch their idea or to offer their skills. About 20 hackers braved the front of the room describing their idea or their skill set. One hacker described his project as help me with my homework (and yes, this hacker did find a teammate and they ultimately built a nifty hardware hack that satisfied the homework requirement too).
Tech Deep Dives
Next up on the schedule were the Tech Deep dives. Organizations had a half-hour to give a deeper view of what their technology is capable of. Some hackers want to know more about how to do particular things with an API or technology. This is their opportunity to find out all about the nuts and bolts, to ask questions from the experts. The Tech Deep Dives are strictly optional – many hackers skip them and instead start forming teams, sharing ideas, initializing their repos and writing code.
After all the preliminaries are over it was finally time to start hacking.
Hackers formed teams, large and small.
The competition for the best vest was fierce:
They staked out a comfortable workspace in chairs …
Or on the floor …
There were some very creative hardware hacks:
Plenty of good food
And lots of fun
The Microsoft NERD was only available until 9PM – after that we moved over to hack/reduce – a wonderful hacking space a five minute walk away. There we were greeted by a perfect hacking space with lots of great wifi, great hacker lighting, and lots of beer. Hacking continued all night. Some hackers did try to get some sleep (either at the hack, or back at home), but some hardcore hackers stayed the whole night.
By 9AM on Sunday morning, the hackers were back at the NERD, for lots of coffee, some breakfast and then more hacking.
At 2:30, hacking was officially over, and teams submitted their projects to Hacker League. Sixty hacks were submitted.
By 3PM the 200 hackers had all gathered back into the big room joined by a hundred folks who had come just to see the demo session. Hackers had two minutes to show their stuff. It is a hard demo to give. You are giving a demo of software that you’ve just finished building. It might have some bugs. The WiFi is a little flaky, you haven’t slept in 24 hours, your hands are shaking from too much coffee and too much nervousness, you have to type while holding a microphone and your laptop just won’t sync with the projector just right, and the audio isn’t coming out of the speakers, and the colors look all wrong on the screen. All in front of 300 people. I’ve done it dozens of times and it still is a really scary demo to give. But it is incredibly exhilarating too – to take nothing but an idea and turn it into something that can amaze or amuse a room full of tech elite in 24 hours. It is quite a rush.
There were two A/V setups so while one team was presenting, the next team was setting up. This allowed us to get through 60 demos in just over two hours. There was a very low incidence of demo fail. And only two inappropriate demos (one was a 2 minute powerpoint presentation with no tech built, the other was a 2 minute tech commercial for a product). I was worried that we might have a #titstare moment with one hack that seemed to contain questionable content but that hacker apparently decided not to present.
The 60 hacks represented a wide range of domains. There were games, music learning tools, programs designed to create, manipulate, remix and even destroy music. I’d love to cover them all, but there are just too many.
The full list is on hacker league. Here are some of my absolute favorites:
String Theory – A musical instrument and sound sculpture build from yearn and stretch sensors and powered by an Arduino.
The Lone Arranger – a terminal app that allows you to easily rearrange your audio. By a father and son hacking team.
The Secret History of Music – combs biographies, lyrics, and commentary from song meanings from two artists, combines them into one fictional artist, and uses Markov chain magic to generate a 50K novel about this new fictional band.
LED Soundsystem – this hack attempts to generate a light show synchronized to the music. It has a special place in my heart because the hacking team was working on the same problem I was working on for my abandoned hack – i.e. automatically finding the ‘drop’ in a song. Unfortunately, this team had a demo fail – but they are smart guys and I expect to see good stuff from them at the next hack day.
eHarmonica – an electronic harmonica!
Enter the dragon – In today’s world, everyone deserves a spectacular entrance. And we intend to give it to them. Enter the Dragon uses bluetooth technology to detect when a user enters the room and plays their personalized entrance music.
Dadabots – Dadabots are bot accounts on creative websites that make procedural creations or remixes of other creations
ios SoundPuzzle – A simple iOS ear training game built programmatically from the free music archive and the echonest remix api.
danceomatic – totally awesome automatic choreography from an mp3 and a web based stick figure performance.
Jotunnslayer – Never again listen to power metal without slaying ice giants. Die in battle. Earn your place in Valhalla.
Give Me Liberty or Give Me Death Metal – Political representation via musical exploration
Echos – Choose your favorite song and shoot to the beat! Fight enemies that respond to and are controlled by the music! Listen closely and experience unparalleled power as you get into the groove! Enjoy addictive arcade-style game-play in this twist on a classic formula.
TweetTones – a native iOS application that generates synthesized music from tweets in real-time.
Short-Attention Span Playlist Scanner – Glenn made a radio scanner that find and plays just the choruses.
ionian Eclipse – A web-based multiplayer top-down space shooter with procedurally generated enemies and interactions driven by music events.
ColorMe – Ever wondered what your music looked like? Now you can look at songs by your favorite artist with this super fun web app, powered by the Echo Nest.
How Repetitive – measures how often audio segments repeat themselves within in a given song.
Jason’s music visualizer – an html/css visualizer on steroids.
And last, but by no means least, Jonathan’s awesome MIDI Digester that converts audio to MIDI and back, over and over to generate some very strange sounds. The very essence of the music.
There are so many excellent hacks, I’m sure I’ve missed many notables. Luckily, Evolver.fm covered the event, so expect to see Eliot’s writups on all the best hacks on Evolver.fm.
At the end of the mega demo session, there’s a brief prize awarding ceremony where a half-dozen organizations give out modest prizes for hackers that made cool stuff using their tech.
Finally we adjourned to the local pub for some food, beer and hacking recaps.
Special thanks to the organizers of the event. The Music Hack Day would not happen without Elissa and Matt. They do all the hard work. Finding the venue, wrangling the sponsors and volunteers, making a mega Costco food run, dealing with the A/V, running the registration, selecting and hiring the caterers, designing t-shirts and so much more. There’s a huge amount of work that goes into planning the event, much more than meets the eye. Elissa and Matt are the unsung heroes of Music Hack Day. We should make a music hack to sing their song.
Thanks also to the event sponsors: Rdio, Spotify, Microsoft, hack/reduce, Free Music Archive, SoundCloud, Mailchimp and The Echo Nest, and the many volunteers who came and helped us run the whole show.
More Music Hack Days
Interested in going to a Music Hack Day? Check out the Music Hack Day calendar for upcoming events. There’s one in Helsinki this weekend, and there’s one in London in just a few weeks. More events are rumored to be in the planning stages for 2014.
(Photos mostly by Michelle Ackerman, a few by me)
For my Boston Music Hack Day hack I built Yet Another Party Playlisting App (YAPPA), because the world needed another party playlister – but really, I built it because I needed another hack, because 15 hours into the 24 hour hackathon I realized that my first hack just wasn’t going to work (more on that in another post). And so, with 9 hours left in the hack day, I thought I would try my hand at the party playlisting app.
The YAPPA is a frequently built app. In some sense one can look at the act of building a YAPPA as a hacking exercise. Just as a still life painter will practice by painting a bowl of fruit, or a pianist will practice scales, a music hacker can build their hacking muscle by creating a YAPPA.
The essential features of a YAPPA are straightforward – create a listening experience for a party based upon the tastes of the guests. Allow guests to suggest music for the party, apply some rules to select music that satisfies all the guests, and keep the music flowing.
With those features in mind, I created my party playlisting app. The interface is dead simple – guests can add music to the party via the master web interface or text the artist and song from the mobile phones to the party phone number. Once the party has started, PAPPA will keep the music flowing.
The key technology of PAPPA is how it picks the music to play next. Most YAPPAs will try to schedule music based on fairness so that everyone’s music taste is considered. Some YAPPAs also use song attributes such as song hotttnesss, song energy and danceability to make sure that the music matches the vibe of the party. PAPPA takes a very different approach to scheduling music. That’s because PAPPA takes a very different approach to parties. PAPPA doesn’t like parties. PAPPA wants everyone to go home. So PAPPA takes all of these songs that have been carefully texted to the party phone number, along with all the artist and song suggestions submitted via the web and throws them away. It doesn’t care about the music taste of the guests at the party. In fact it despises their taste (and the guests as well). Instead, PAPPA selects and plays the absolute worst music it can find. It gives the listener an endless string of the most horrible (but popular) music. Here’s a sample (the first 3 songs are bait to lure in the unwitting party guests):
- Royals by Lorde
- Levels by Avicii
- Blurred Lines by Robin Thicke
- #Twerkit featuring Nicki Minaj by Busta Rhymes
- From The Bottom Of My Broken Heart by Britney Spears
- Amigas Cheetahs by The Cheetah Girls
- Do Ya Think I’m Sexy by Paris Hilton
- Incredible by Clique Girlz
- No Ordinary Love by Jennifer Love Hewitt
- Mexican Wrestler by Emma Roberts
- I Don’t Think About It by Emily Osment
- A La Nanita Nana by The Cheetah Girls
- Don”t Let Me Be The Last To Know by Britney Spears
- Wild featuring Big Sean by Jessie J
- Heartbeat (Album Version) by Paris Hilton
- Love The Way You Love Me by The Pussycat Dolls
- When You Told Me You Loved Me by Jessica Simpson
- Jericho by Hilary Duff
- Strip by Brooke Hogan
- Pero Me Acuerdo De Tí by Christina Aguilera
- Bang Bang by Joachim Garraud
- Right Now featuring David Guetta (Sick Individuals Dub) by Rihanna
- Wilde Piraten by The Cool Kids
- Friend Lover by Electrik Red
- Betcha Can’t Do It Like Me by D4L
- Who’s That Girl by Hilary Duff
- Get In There, Frank! by Fun
- Hold It Don”t Drop It by Jennifer Lopez
- Sweet Sixteen by Hilary Duff
- Live It Up featuring Pitbull by Jennifer Lopez
- Freckles by Natasha Bedingfield
- I Want You by Paris Hilton
- Hold It Close by Fun
- Magic by The Pussycat Dolls
- How To Lose A Girl by Mitchel Musso
- Fairy Tales by JoJo
- Slow It Down featuring Fabolous (Album Version (Explicit)) by The-Dream
- Mr. Hamudah by Charles Hamilton
- Promise by Vanessa Hudgens
- Metamorphosis by Hilary Duff
How does PAPPA find the worst music in the world? It looks through all the data that The Echo Nest is collecting about how people experience music online to find the songs that have been banned frequently. When a music listener says “ban this song” they are making a pretty strong statement about the song – essentially saying, “I do not ever want to hear that song again in my life”. PAPPA finds these songs that have the highest banned-to-play ratio (i.e. the songs that have been proportionally banned the most when play count is taken into consideration) and adds them to the playlist. The result being a playlist filled with the most reviled music – with songs by Paris Hilton, Jennifer Love Hewitt and the great Emma Roberts. The perfect playlist to send your guests home.
At this moment, lets pause and listen to the song Mexican Wrestler by Emma Roberts:
What happens to all those carefully crafted text messages of songs sent by the guests? No, there’s no Twilio app catching all those messages, parsing out songs and adding them to a play queue to be scheduled. They just go to my phone. That’s so if people are not leaving the party fast enough, I can use all the phone numbers of the guests to start to text them back and tell them they should go home.
By the way, if you look at the songs that were texted to me during my two minute demo you’d realize how fruitless a YAPPA really is. There’s no possible way to make a party playlist that is going to satisfy everyone in the room. Tastes are too varied, and there’s always that guy who thinks he is clever by adding some Rick Astly to the party queue. Here’s what was texted to me during my two minute demo:
- Gregory Porter – be good
- Rebecca Black – It’s Friday
- Weird Al Yankovic – Fat
- Lady Gaga – Applause
- Weird Al Yankovic – Amish Paradise (from a different phone number from the other weird Al fan)
- boss ass bitch
- Basement Jaxx raindrops
- John Mayer your body is a wonderland
- jay z holy grail
- Underworld spikee
- wake me up
- Britney Spears – Hit Me Baby One More Time
- Slayer War Ensemble
- Bieber baby
- Ra Ra riot
- Rick Astley
- Mikey Cyrus
- Hi paul
- Stevie wonder overjoyed
Imagine trying to build a party playlist based upon those 24 input songs. Admittedly, a hackathon demo session is not a real test case for a party playlister but I still think you’d end up with a terrible mix of songs that no smart algorithm, nor any smart human, could stitch together into a playlist that would be appropriate and pleasing for a party. My guess is that if you did an A/B test for two parties, where one party played music based upon suggestions texted to a YAPPA and the other party played the top hotttest songs, the YAPPA party would always lose. I’d run this test, but that would mean I’d have to go to two parties. I hate parties, so this test will never happen. Its one of the flaws in our scientific method.
Who are the worst artists?
Looking at the PAPPA playlists I see a number of recurring artists – Britney Spears and Paris Hilton seem to be well represented. I thought it would be interesting to create a histogram of the top recurring artists in the most banned songs list. Here’s the fascinating result:
One thing I find notable about this list is the predominance of female artists. Females outnumber males by a substantial amount. Here’s some pie:
80% of the most banned artists are female. A stunning result. There’s something going on here. Someone suggested that the act of banning a song is an aggressive act that may skew male, and many of these aggressively banning males don’t like to listen to female artists. More study is needed here. It may involve parties, so I’m out.
Wrapping it all up
I enjoyed creating my PAPPA YAPPA. Demoing it was really fun and the audience seemed to enjoy the twist ending. The patterns in the data underlying the app are pretty interesting too. Why are so many banned songs by female artists?
If you are having your own party and want to use PAPPA to help enhance the party you can go to:
Just replace the phone number in the URL with your own and you are good to go.
For my Malmö Music Hack Day hack I built an app called Dogstep. Dogstep takes any song and re-renders it such that a pack of dogs harmonizes along with the song. It was a lot of fun to build and I was rather pleased with the results. You can try the app out yourself: Dogstep.
I got to try a few new things on this hack. First, off I needed some good dog sounds. I found all I needed and more at Freesound.org. What a great resource that is! I then needed to process the barks (trim them, pitch shift them, volume-equalize ). For this I used Audacity. It was way easier to use than garageband and it has all the audio filters that I needed (including the awesome Paul’s stretch we can make any howl sound like a banshee from hell).
To create realistic and varying barking, I created a barking state machine, where each state in the machine represents the barking activity for a bar in the song and each state has a set of transitions to other states in the machine governed by a probability that that transition will be taken. When a song is playing, I use the state machine to pick the state for the currently playing bar and emit the barking audio at the right times within the bar. Here’s a visualization of the barking state machine:
In addition to these barks, I look for the loudest parts in the songs and add a bunch of extra howling at these peak moments. Finally, I use the Stylophone play-along algorithm to have one of the dogs try to sing along with the melody.
Creating this state machine was really fun. There’s still a few bits that I want to do – such as having separate state machines for different parts of the song – i.e. a state machine when the song is very quiet vs. one when the song is loud and energetic. A hack is never really done.
The source for the hack is on github.
This week, The Echo Nest and Getty Images announced that they were partnering to make thousands of high quality artist images available for developers through The Echo Nest API. Getty Images has spent years building an amazing library of artist images and now, as a result of this partnership, it is easy for developers to use these images in music apps.
I took the new Getty Images API for a spin and built a couple of apps that show how easy it is to build an Echo Nest app that uses the images. First, I built an app that shows images of the top hotttest artists:
Next, since Getty Images has some really awesome images going back to the classic rock era, I adapted my app to show some Getty Images for some of our top classic rock artists:
We’ve extended our image API to return additional information with the Getty images. There’s image attribution information, image size information, and some curated image tags that you can use to select the best image for your app. Tags include landscape, portrait, black-and-white, solo, award, performance, color and many more.
The Getty Images are really top notch. It’s a great addition to The Echo Nest API. I’m excited to see how developers will use this asset.
In yesterday’s post about the Hot Songs of Summer 2013, I noted that some songs were attracting a very passionate fan base. In particular, the song Miss Movin’ On by Fifth Harmony was an extreme outlier, attracting more than twice the number of plays per listener than any other song.
Based on this data I suggested that the Fifth Harmony was going places – such high passion among their listeners was surely indicative of future success. But now I am not so sure. Shortly after I made that post I learned that our crack data team here at The Echo Nest were already on to some Fifth Harmony shenanigans. Yes, Fifth Harmony is getting lots of plays, but many of these plays are due to an orchestrated campaign. Fifth Harmony fans are encouraged to go to music streaming sites such as Spotify and Rdio and stream Miss Movin’ On (aka MMO) 24/7. Here are some examples:
There are a number of twitter accounts that are prompting such MMO plays. The campaign seems to be working. 5H is moving up in the charts. Just take a look at the top songs on Rdio this week, Miss Movin’ On is number two on the list:
But what effect is this campaign really having on Fifth Harmony? Perhaps Fifth Harmony’s position on the charts is a natural outcome of their appeal, and is not a result of a small number of fans that stream MMO 24/7 with their computers and iPhones on mute. Can we see the effect that The Harmonizers are having? And if so, how substantial is this effect? The answer lies in the data, so that’s where we will go.
Can we see the effect of the Harmonizers?
The first thing to do is to take a look at the listener play data for MMO and compare it to other songs to see if there are any tell-tale signs of a shilling campaign. To do this, I selected 9 other songs with similar number of fans that appeal to a similar demographic as MMO. For each of these songs I ordered the listeners in descending play order (i.e. the first listener is the listener that has played the song the most) and plotted the number of plays per listener for the 10 songs.
As you can see, 9 out of 10 songs follow a similar pattern. The top listeners of a song have around a thousand plays. As we get deeper into the listener ranks, the number of plays per listener drops off at a very predictable rate. The one exception is Fifth Harmony’s Miss Movin’ On. The effect of the Harmonizers is clearly seen. The top plays are skewed to greatly inflate the total number of plays by two full orders of magnitude. We can also see that the number of listeners that are significantly skewing the data is relatively small. Beyond the top 200 most active listeners (less than 0.5 % of the Fifth Harmony listeners in the sample), the listening pattern for MMO falls in line with the rest of the songs. It is pretty clear that the Harmonizers are really having an effect on the number of plays. It is also clear that we can automate the detection of such shilling by looking for such non-standard listening patterns.
Update – a reader has asked that I include One Direction’s Best Song Ever on the plot. You can find it here.
How big of an impact do the Harmonizers have on the overall play count?
The Harmonizers are having a huge impact. 80% of all track plays of Miss Movin’ On are concentrated into just the top 1% of listeners. Compare that to the other 9 tracks in our sample:
Percentage of listeners that account for 80% of all plays
|Fifth Harmony – Miss Movin’ On||1.0|
|Lorde – Royals||14.0|
|Karmin – Acapella||16.0|
|Anna Kendrick – Cups||17.0|
|Taylor Swift – 22||14.0|
|Icona Pop – I love it||15.0|
|Birdy – Skinny Love||25.0|
|Lana Del Rey – Summertime Sadness||15.0|
|Christina Perri – A Thousand Years||21.0|
|Krewella – Alive||17.0|
A plot of this data makes the difference quite clear:
I estimate that at least 75% of all plays of Miss Movin’ On are overplays that are a direct result of the Harmonizer campaign.
What effect does the Fifth Harmony campaign have on chart position?
It is pretty easy to back out the overplays by finding another song that has a similarly-shaped plays vs listener rank curve once we get beyond past the first 1% of listeners (the ones that are overplaying the track). For instance, Karmin’s Acapella has a similar mid-tail and long-tail listener curve and has a similar audience size making it a good proxy. It’s Summer Time rank was 378. Based on this proxy, MMO’s real rank should be dropped from 45 to around 375. This means that a few hundred committed fans were able to move a song up more than 300 positions on the chart.
The bottom line here is that an organized campaign for very little cost has harnessed the most passionate fans to substantially bolster the apparent popularity of an artist, making the artist appear to be about 4 times more popular than it really is.
What does this all mean for music services?
Whenever there’s a high-stakes metric like chart position some people will try to find a way to game the system to get their stuff to the top of the chart. Twenty years ago, the only way to game the charts was either by spending lots of money buying copies of your record to boost the sales figures, or bribe radio DJs to play your songs to boost radio airplay. With today’s music subscription services, there’s a much easier way to game the system. Fans and shills need to simple play a song on autorepeat across a a few hundred accounts to boost the chart position of a song. Fifth Harmony proves that if you have a small, but committed fan base, you can radically boost your chart position for very little cost.
Obviously, a music service doesn’t like this. First, the music service has to pay for all those streams, even if no one is actually listening to them. Second, when a song gets to the top of a chart through shilling and promotion campaigns, it reduces the listening enjoyment for those who use the charts to find music. Instead of finding a new song that got to the top of the chart based solely (or at least mostly) on merit, they are listening to a song that is a product of a promotion machine. Finally, music services that rely on user play data to generate music recommendations via collaborative filtering have a significant problem trying to make sure that fake plays don’t improperly influence their recommendations.
So what can be done to limit the damage to music services? As we’ve seen, it is pretty easy to detect when a song is being overplayed via a campaign and these overplays can be removed. Perhaps even simpler though is to rely on metrics that are less easily gamed – such as the number of fans a song has instead of the total number of plays. For a music subscription service that has a credit card number associated with each user account, the number of fans a song has is a much harder metric to hack.
What does this say about Fifth Harmony fans ?
I am always happy when I see people getting excited about music. The Fifth Harmony fans are really excited about Miss Movin’ On, the tour and the upcoming album. Its great that the fans are so invested in the music that they want to help the band be successful. That’s what being a fan is all about. But I hope they’ll avoid trying to take their band to the top by a shortcut. As they say, it’s a long way to the top if you want to rock n’ roll. Let Fifth Harmony earn their position at the top of charts, don’t give them a free ride.
And finally, a special message to music labels or promoters: If you are trying to game the music charts by enlisting hundreds of pre-teens and teens to continuously stream your one song: screw you.
Update – I’ve received **lots** of feedback from Harmonizers – thanks. A common theme among this feedback is that the fan activities and organization really are a grassroots movement, and there really is no input from the labels. Many took umbrage with my suspicions that the label was pulling the strings. I remain suspicious, but less so than before. My parting ‘screw you’ comment was in no way directed at the 5H fans, it was reserved for the mythical music label marketeer who I imagined was pulling the strings. I’m hoping to dig in a bit deeper to understand the machinery behind the 5H fan movement. Expect a follow up article soon.