Something is fishy with this recommender

In my rather long winded post on the problems with current music recommenders, I pointed out the Harry Potter Effect. Collaborative Filtering recommenders tend to recommend things that are popular which makes those items even more popular, creating a feedback loop – (or as the podcomplex calls it – the similarity vortex) that results in certain items becoming extremely popular at the expense of overall diversity. (For an interesting demonstration of this effect, see ‘Online monoculture and the end of the niche‘).

Oscar sent me an example of this effect. At the popular British online music store HMV, a rather large fraction of artists recommendations point to the Kings of Leon. Some examples:

Oscar points out that even for albums that haven’t been released, HMV will tell you that ‘customers that bought the new unreleased album by Depeche Mode also bought the Kings of Leon’. Of course it is no surprise that if you look at the HMV bestsellers, The Kings Of Leon is way up there at position #3.

At first blush, this does indeed look like a classic example of the Harry Potter Effect, but I’m a bit suspicious that what we are seeing is not an example of a feedback loop, but is an example of shilling – using the recommender to explicitly promote a particular item. It may be that HMV has decided to hardwire a slot in their ‘customers who bought this also bought’ section to point to an item that they are trying to promote – perhaps due to a sweetheart deal with a music label. I don’t have any hard evidence of this, but when you look at the wide variety of artists that point to Kings of Leon – from Miley Cyrus, to Led Zeppelin and Nirvana it is hard to imagine that this is a result of natural collaborative filtering. Music promotion that disguises itself as music recommendation has been around for about as long as there have been people looking for new music. Payola schemes have dogged radio for decades. It is not hard to believe that this type of dishonest marketing will find its way into recommender systems. We’ve already seen the rise of ‘search engine optimization’ companies that will get your web site on the first page of google search results – it won’t be long before we see a recommender engine optimizer industry that will promote your items by manipulating recommenders. It may already be happening now, and we just don’t know about it. The next time you get a recommendation for The Kings of Leon because you like The Rolling Stones, ask yourself if this is a real and honest recommendation or are they just trying to sell you something.

This entry was posted on April 1, 2009, 8:39 am and is filed under Music, recommendation, research, The Echo Nest. You can follow any responses to this entry through RSS 2.0. Both comments and pings are currently closed.

#1 by LJ on April 1, 2009 - 9:37 am

Perhaps “customers who bought this also bought” should read “customers who per-ordered this also bought” in these kind of examples? Although I wouldn’t put it past most online retailers to do exactly what you suspect might be going on
#2 by Oscar Celma on April 1, 2009 - 10:06 am

Well, either is the best April’s fool joke, or their recommender is plain wrong! (and I think is the latter).

Also, my guess is they have a very spare matrix data, that promotes this silly effect when using CF.
#3 by Neal Lathia on April 1, 2009 - 10:28 am

Interesting- especially because shilling attacks have often been studied from the perspective of foreign (illegitimate) intruders to a recommender system, and not “a sweetheart deal” between the recommender system owner and a third party.

The latter obviously invalidates any kind of algorithmic defenses that can be set up: what can users do? (Nothing? Abandon the site?)
#4 by Garg Unzola on April 1, 2009 - 11:52 am

The Kings of Leon is one of the worst bands I’ve ever heard in my life. Any recommender system that pops up their name needs to be canned for malfunction.
#5 by The Schill on April 1, 2009 - 12:08 pm

As someone who has been in several talks with major retailers about recommendation engines, I can tell you that a great many of them ask to be able to “bump” certain titles up in the recommendation list…always titles where the retailer has a more favorable cut of the revenue.

In some cases, I think retailers see recommendation services as the “endcaps” of their store — a way to push the merchandise with the best revenue potential to the high-trafficked locations.