Collaborative Filtering at Amazon.com

The authors note:

Amazon.com has more than 29 million customers and several million catalog items. Other major retailers have comparably large data sources. While all this data offers opportunity, it's also a curse, breaking the backs of algorithms designed for data sets three orders of magnitude smaller.

The authors present the more traditional collaborative filtering and cluster models methods. These methods, unfortunately, tend to perform poorly on the scale that Amazon.com needs. Techniques available to speed up these methods degrade the results.

Amazon.com uses a technique they call item-to-item collaborative filtering. The key to their method is to perform the computationally expensive portion--a product-to-product matrix--offline. This allows high quality recommendations to be generated quickly, in real time.

The IEEE offers the article for purchase online, but it's a little pricey ($18 for non-members). If you want to read it, you may wish to visit the engineering library of your friendly neighborhood university. Or, if you are local, drop me a note and I'll share my copy.