Supervision : Patrick GALLINARI

Co-supervision : USUNIER Nicolas

Offline Evaluation of recommender systems

This thesis presents various experimental protocols leading to a better offline estimation of errors in recommender systems.
As a first contribution, results form a case study of a recommender system based on purchased data will be presented. Recommending items is a complex task that has been mainly studied considering solely ratings data. In this study, we put the stress on predicting the purchase a customer will make rather than the rating he will assign to an item. While ratings data are not available for many industries and purchases data widely used, very few studies considered purchases data. In that setting, we compare the performances of various collaborative filtering models from the litterature. We notably show that some changes the training and testing phases, and the introduction of contextual information lead to major changes of the relative perfomances of algorithms.
The following contributions will focus on the study of ratings data. A second contribution will present our participation to the Challenge on Context-Aware Movie Recommendation. This challenge provides two major changes in the standard ratings prediction protocol: models are evaluated conisdering ratings metrics and tested on two specifics period of the year: Christmas and Oscars. We provides personnalized recommendation modeling the short-term evolution of the popularities of movies.
Finally, we study the impact of the observation process of ratings on ranking evaluation metrics. Users choose the items they want to rate and, as a result, ratings on items are not observed at random. First, some items receive a lot more ratings than others and secondly, high ratings are more likely to be oberved than poor ones because users mainly rate the items they likes. We propose a formal analysis of these effects on evaluation metrics and experiments on the Yahoo!Music dataset, gathering standard and randomly collected ratings. We show that considering missing ratings as negative during training phase leads to good performances on the TopK task, but these performances can be misleading favoring methods modeling the popularities of items more than the real tastes of users.

2010-2013 Publications

• 2013
  • B. Pradel : “Évaluation des sytsèmes de recommandation à partir d’historiques de données”, thesis, phd defence 10/02/2013, supervision Gallinari, Patrick, co-supervision : Usunier, Nicolas (2013)
• 2011
  • B. Pradel, N. Usunier, F. Fogelman‑Soulié, J. Delporte, S. Sean, C. Rouveirol, S. Guerif, F. Dufau‑Joel : “A Case Study in a Recommender System Based on Purchase Data”, ACM International Conference on Knowledge Discovery and Data Mining, San Diego, CA, United States, pp. 377-385, (ACM) (2011)
• 2010
  • B. Pradel, A. Brenner, N. Usunier, P. Gallinari : “Predicting Most Rated Items in Weekly Recommendation with Temporal Regression”, Workshop on Context-Aware Movie Recommendation, CAMRa '10, Barcelona, Spain, pp. 24-27, (ACM) (2010)