Jian Wang, Kaushik Rangadurai, David Hardtke, Krishnaram Kenthapadi
Real-time large-scale personalized recommendation systems power several user-facing products at many social media and web plat- forms. To meet business requirements, such applications must score millions of structured candidate documents associated with each query, offer a high degree of data freshness, and respond with low latency. To address these challenges, many such systems in practice make use of content based recommendation models based on logis- tic regression. A fundamental problem with content based models is that they are based primarily on the explicit user context in the form of user profile, but do not take into account implicit user con- text in the form of user interactions. We address the following problem: How do we incorporate user item interaction signals as part of the relevance model in a large-scale personalized recommendation system such that, (1) the ability to interpret the model and explain the recommendations is retained, and (2) the existing infrastructure designed for the (user profile) content-based model can be leveraged? We propose Dionysius, a hierarchical graphical model based framework for incorporating user interactions into recommender systems. We learn a hidden fields vector for each user by considering the hierarchy of inter- action signals, and replace the user profile based vector with this learned vector, thereby not expanding the feature space at all. Thus, our framework allows the use of existing recommendation infrastructure that supports content based features. We have implemented and deployed this system as part of the recommendation platform in a large professional social network. We validate the efficacy of our approach through extensive offline experiments with different model choices, as well as online A/B testing experiments. Our deployment of this system as part of the job recommendation engine has resulted in significant improvement in the quality of the retrieved results, thereby generating improved user experience and positive impact for millions of users.