Visual cues to contrasts & audiovisual cue integration

We know from phenomena like the McGurk Effect that speech perception is not just what we hear, but what we experience with all of our senses; visual and tactile cues particularly seem to contribute to how a sound is perceived. What kind of effect (if any) might that have on a language’s phonology? Could visual information, such as lip rounding and lip protrusion, influence the direction of a sound change? Is a phonological feature like [±round] always realized in the same manner from contrast to contrast? Together with Baichen Du and Keith Johnson, we’ve been examining acoustic and visual phonetic cues in Mandarin. So far, we’ve found that variability in the visual cues of lip rounding and lip protrusion may be contributing to the current Mandarin sibilant merger. We’ve also found that [±round] is acoustically the same, but visually different, between round vowels to round consonants, which may have consequences for possible sound changes.