Year of Graduation
Methods for Evaluation of Collaborative Filtering Algorithms
Applied Mathematics and Information Science
In this paper we apply an axiomatic approach to study properties of evaluation methods for collaborative filtering algorithms. As a result we formulate five axioms where each axiom captures a property that is believed to be important for an evaluation metric used in a RS field. We study several existing evaluation metrics and show that some of them, such as Precision and NDCG, fail to comply with a number of axioms. Such evaluation metrics theoretically can produce a wrong ordering of RSs under certain circumstances. However, experimental part of this study has shown that popular evaluation metrics generally produce same rankings of well-known collaborative algorithms, such as SVD and k-NN.