Abstract
When reporting classifier accuracy, it's common to use hit ratio as a primary metric. However, hit ratio has a serious flaw. We examine the issues surrounding this flaw and explore its magnitude through an empirical experiment on three multivalued classification data sets, using two well-known machine learning models. The results demonstrate a real problem that we can't simply overlook, and we propose an alternative-Cohen's kappa. Like any other metric, it has its own shortcomings, but we believe it should be mandatory in any scientific report about classifier accuracy
Users
Please
log in to take part in the discussion (add own reviews or comments).