Abstract
In the field of data sonification, the construction of meaning is hampered by the lack of shared perceptual codes derived from common modes of perception, as it happens for the visual register. In this paper, we re-organize knowledge from previous experimental projects to build the foundations of future work in data representation. This experimental investigation aims to identify patterns in the translation process from different sensory modalities. To this end, 80 audiovisual sonifications have been collected and analyzed through phenomenological analysis with the goal of recording sensory correspondences. The resulting cross-sensory design map is a visual synthesis of the analysis, and it has a dual function. In the research domain, it proposes testable hypotheses for a systematic approach to data sonification. In the practice, it offers a space that is based on shared conventions that aim to standardize the actions and the choices of both sonification experts and communication designers.
Keywords
data sonification, visualization, cross-sensory design, perceptual codes
DOI
https://doi.org/10.21606/drs.2022.380
Citation
Caiola, V., Lenzi, S., and Riccò, D. (2022) Audiovisual sonifications: A design map for multisensory integration in data representation, in Lockton, D., Lenzi, S., Hekkert, P., Oak, A., Sádaba, J., Lloyd, P. (eds.), DRS2022: Bilbao, 25 June - 3 July, Bilbao, Spain. https://doi.org/10.21606/drs.2022.380
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
Conference Track
Research Paper
Included in
Audiovisual sonifications: A design map for multisensory integration in data representation
In the field of data sonification, the construction of meaning is hampered by the lack of shared perceptual codes derived from common modes of perception, as it happens for the visual register. In this paper, we re-organize knowledge from previous experimental projects to build the foundations of future work in data representation. This experimental investigation aims to identify patterns in the translation process from different sensory modalities. To this end, 80 audiovisual sonifications have been collected and analyzed through phenomenological analysis with the goal of recording sensory correspondences. The resulting cross-sensory design map is a visual synthesis of the analysis, and it has a dual function. In the research domain, it proposes testable hypotheses for a systematic approach to data sonification. In the practice, it offers a space that is based on shared conventions that aim to standardize the actions and the choices of both sonification experts and communication designers.