"Paint That Object Yellow": Multimodal Interaction to Enhance Creativity During Design Tasks in VR
2019 International Conference on Multimodal Interaction, page 195-204. (2019)Best Paper Runner-Up 🏆.DOI: 10.1145/3340555.3353724
Abstract
Virtual reality (VR) has always been considered a promising medium to support designers with alternative work environments. Still, graphical user interfaces are prone to induce attention shifts between the user interface and the manipulated target objects which hampers the creative process. This work proposes a speech-and-gesture-based interaction paradigm for creative tasks in VR. We developed a multimodal toolbox (MTB) for VR-based design applications and compared it to a typical unimodal menu-based toolbox (UTB). The comparison uses a design-oriented use-case and mea-sures flow, usability, and presence as relevant characteristicsfor a VR-based design process. The multimodal approach (1) led to a lower perceived task duration and a higher reported feeling of flow. It (2) provided a higher intuitive use and a lower mental workload while not being slower than an UTB. Finally, it (3) generated a higher feeling of presence. Overall, our results confirm significant advantages of the proposed multimodal interaction paradigm and the developed MTB for important characteristics of design processes in VR.
