Making Linear Autoencoders Work for Large Scale Recommendation Systems
Linear autoencoders for collaborative filtering in recommender systems are simple and surprisingly accurate as we explained in our blogpost on how linear methods work. The critical disadvantage of methods like EASE is that they are not applicable to real-world problems, where the number of items to be recommended is high. In this blogpost, we will explain how we made linear methods applicable even to the largest customers of Recombee with tens of millions of recommendable items.
We introduce ELSA, a scalable linear model that is basically a shallow autoencoder. What is interesting is that ELSA is not only more scalable, but it actually outperforms EASE and other linear methods in a number of tasks. We will first explain how ELSA is different and how we managed to make it scalable.
The problem of linear autoencoders (EASE, SLIM) is that the Item to Item weight matrix grows significantly with the number of items and using a sparse representation does not help. It makes these methods practically applicable to problems with tens of thousands of items. However many real-world recommendation problems are orders of magnitude bigger than that. In ELSA, we were able to replace this matrix by a trick allowing us to use a smaller matrix (items x d) instead, where d is a small constant (e.g. 512) you can choose. This trick makes both memory and time complexity of ELSA grow linearly with the number of items.
As you can see from our results on benchmarking data, the offline performance of ELSA is similar to EASE and it is beneficial to use ELSA when you have more than 20k recommendable items. What is surprising is that ELSA is consistently better than EASE or MF in online tests, probably thanks to a more robust training procedure.
We will make ELSA source codes available for everyone. Also, our paper was accepted for publication at the RecSyS 2022 conference in Seattle, so you can meet us there in case you are interested in our research.
Thanks to its API-first nature, it is quite straightforward to integrate your headless CMS with the most powerful AI-powered content recommendations available on the market. Luminary just did that with their own website, Kontent.ai and Recombee.
Like most of the world, the majority of 2021 was spent on home office or in isolation - which left us with all the time to be invested in work (and Netflix :) ) and improving UX for our clients. We are now happy to share new features we can offer to reach new levels of personalization.
At Recombee, we have always collaborated with academia — after all, five of our co-founders graduated from the Czech Technical University in Prague, one of the largest and oldest technical universities in Europe, and most of them hold a Ph.D. degree.