Using Student Annotated Hashtags and Emojis to Collect Nuanced Affective States
Proceedings of the Fourth (2017) ACM Conference on Learning @ Scale, page 319--322. New York, NY, USA, ACM, (2017)DOI: 10.1145/3051457.3054014
Abstract
Determining affective states such as confusion from students' participation in online discussion forums can be useful for instructors of a large classroom. However, manual annotation of forum posts by instructors or paid crowd workers is both time-consuming and expensive. In this work, we harness affordances prevalent in social media to allow students to self-annotate their discussion posts with a set of hashtags and emojis, a process that is fast and cheap. For students, self-annotation with hashtags and emojis provides another channel for self-expression, as well as a way to signal to instructors and other students on the lookout for certain types of messages. This method also provides an easy way to acquire a labeled dataset of affective states, allowing us distinguish between more nuanced emotions such as confusion and curiosity. From a dataset of over 25,000 discussion posts from two courses containing self-annotated posts by students, we demonstrate how we can identify linguistic differences between posts expressing confusion versus curiosity, achieving 83\% accuracy at distinguishing between the two affective states.
Links and resources
Tags
community
@brusilovsky's tags highlighted