Evaluating the Dynamic Properties of Recommendation Algorithms
R. Burke. Proceedings of the Fourth ACM Conference on Recommender Systems, page 225--228. New York, NY, USA, ACM, (2010)
Collaborative recommendation algorithms are typically evaluated on a static matrix of user rating data. However, when users experience a recommender system, it is dynamic, constantly evolving as new items and new users arrive. The dynamic properties of collaborative recommendation have become important as prediction algorithms based on the interactions of rating histories have been proposed, and as researchers seek to understand problems of robustness and maintenance in rating databases. This paper proposes a new evaluation method for the dynamic aspects of collaborative algorithms, the "temporal leave-one-out" approach, which can provide insight into both user-specific and system-level evolution of recommendation behavior. As a case study, the methodology is applied to the Influence Limiter algorithm 12, showing that its robustness to attack comes at a high accuracy cost.