Computação por Humanos na Perspectiva do Engajamento e Credibilidade de Seres Humanos e da Replicação de Tarefas
Federal University of Campina Grande, PhD thesis, (2015)Abstract
Human computation is a computing approach that draws upon human cognitive abilities to solve computational tasks for which there are so far no satisfactory fully automated solutions. In human computation systems, the processors performing the computations are humans rather than machines. The effectiveness of this kind of system relies on its ability to optimize the use of the cognitive power provided by each human processor. However, little is known about how humans provide their cognitive power in these systems and how these systems can use such cognitive power properly. This study aims at advancing knowledge in this direction. To guide this study, we articulate a framework of theories and concepts about human computation, human engagement, human credibility, and the optimization of computational systems. Based on this theoretical-conceptual framework, we propose metrics to characterize the cognitive power available in a human computation system in terms of the engagement and the credibility of the participants. As case study of system optimization, we also propose a task replication algorithm that optimizes the use of the available cognitive power taking into account information about the credibility of participants. By using correlations, regressions, and clustering algorithms, we characterize the engagement and credibility of participants in data collected from six real systems. Several behavioral patterns are identified in such characterization. Participants can be divided into two broad classes of engagement: the transients, those who work in the system in just one day; and the regulars, those who exhibit a more lasting engagement. Regulars are the minority of participants, but they aggregate the larger amount of cognitive power to the system. They can be subdivided into five groups, labeled as: hardworking, spasmodic, persistent, lasting and moderate. The credibility of participants can be measured by using several different metrics based on the level of agreement among them. Regardless of the metric used, the credibility is negatively correlated with the degree of difficulty of the tasks. Results from simulation show that the proposed task replication algorithm can improve the ability of the system to properly use the cognitive power provided by participants. It also allows one to address trade-offs between different quality-of-service requirements.