On Conducting Security Developer Studies with CS Students: Examining a Password-Storage Study with CS Students, Freelancers, and Company Developers.

A. Naiakshina, A. Danilova, E. Gerlitz, and M. Smith. CHI , page 1-13. ACM, (2020)


Ecological validity is a major concern in usable security stud­ies with developers. Many studies are conducted with com­puter science (CS) students out of convenience, since recruit­ing professional software developers in sufficient numbers is very challenging. In a password-storage study, Naiakshina et al. 28 showed that CS students behave similarly to free­lance developers recruited online. While this is a promising result for conducting developer studies with students, an open question remains: Do professional developers employed in companies behave similarly as well? To provide more insight into the ecological validity of recruiting students for secu­rity developer studies, we replicated the study of Naiakshina et al. with developers from diverse companies in Germany. We found that developers employed in companies performed better than students and freelancers in a direct comparison. However, treatment effects were found to be significant in all groups; the treatment effects on CS students also held for company developers.

Links and resources

BibTeX key:
search on:

Comments and Reviews  

There is no review or comment yet. You can write one!


Cite this publication