Evaluation of recommender systems: A new approach
Expert Systems with Applications, 35 (3):
790--804 (October 2008)DOI: 10.1016/j.eswa.2007.07.047
Abstract
It is difficult to deny that comparison between recommender systems
requires a common way for evaluating them. Nevertheless, at present,
they have been evaluated in many, often incompatible, ways. We affirm
this problem is mainly due to the lack of a common framework for
recommender systems, a framework general enough so that we may include
the whole range of recommender systems to date, but specific enough
so that we can obtain solid results. In this paper, we propose such
a framework, attempting to extract the essential features of recommender
systems. In this framework, the most essential feature is the objective
of the recommender system. What is more, in this paper, recommender
systems are viewed as applications with the following essential objective.
Recommender systems must: (i) choose which (of the items) should
be shown to the user, (ii) decide when and how the recommendations
must be shown. Next, we will show that a new metric emerges naturally
from this framework. Finally, we will conclude by comparing the properties
of this new metric with the traditional ones. Among other things,
we will show that we may evaluate the whole range of recommender
systems with this single metric.