Channels of multimodal communication: Relative contributions to discourse understanding

Abstract

When people communicate in normal environments, they use several communication channels, such as the verbal channel, prosody (non-segmental aspects of sound) and the visual channel. There is a great variety of views on the relative contribution of these channels to the overall understanding of messages. Linguists typically limit their interests to the verbal channel alone, while social psychologists often emphasize the importance of non-verbal components. This study offers an experimental methodology allowing to empirically evaluate the contributions of different channels. Two experimental studies are reported that used different stimulus material: movies and recorded conversation. According to the results of the experiments, all of the communication channels are highly significant, the verbal channel being the leading one. Experiment participants experienced difficulties in certain unusual conditions, such as the combination of the prosodic and visual channels without the verbal component. This paper is a contribution to the field of multimodal studies.


Back to Table of Contents