Exploring the effect of visual and auditory information in haptic experience

Authors

Xinyi Gao, Hunan University, China, People's Republic of
Wei Wang, Hunan University, China, People's Republic of
Fangli Song, Hunan University, China, People's Republic of; School of Design, Georgia Institute of Technology
Fang Liu, Hunan University, China, People's Republic of

Abstract

Haptic experience always has a close relationship with visual and auditory modalities, but little research uses both visual and auditory modalities to map haptic perception in tangible interaction. It is essential to understand how to make haptic information harmonious with visual and auditory modalities, and how users describe haptic feedback in physical interface design. In this paper, participants are divided into three groups: auditory-haptic (AH), visual-haptic (VH), and haptic (H) groups to test a sensory integrated prototype in order to explore the effect of visual and auditory information on haptic experience and users’ description patterns. We find that users’ haptic descriptions range from abstract to concrete and are affected by modalities. Moreover, there is a massive difference in the function of visual and auditory influencing the sense of touch. The findings are beneficial to haptic experience design.

Keywords

haptic experience, multi-modal interaction, haptic prototyping, physical interfaces

DOI

https://doi.org/10.21606/drs.2022.496

Citation

Gao, X., Wang, W., Song, F., and Liu, F. (2022) Exploring the effect of visual and auditory information in haptic experience, in Lockton, D., Lenzi, S., Hekkert, P., Oak, A., Sádaba, J., Lloyd, P. (eds.), DRS2022: Bilbao, 25 June - 3 July, Bilbao, Spain. https://doi.org/10.21606/drs.2022.496

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License

Conference Track

Research Paper