- Perceiving different idiolects and language varieties
Everyone speaks in a different way, influenced by many factors such as age, gender, social status, dialects and accents. We investigate how listeners represent and adapt to unfamiliar pronunciations in speech, and further, once adapted, how they generalize from prior experience to novel situations (e.g., novel words, talkers, accents). One interesting case to consider is when representations of the sound structure of language are misaligned between the listener and the talker, i.e. a native listener and a foreign-accented speaker. Continuing work along this line includes studies that examine 1) effects of expectations on perceptual adaptation, 2) the maintenance of adaptation over the long term, and 3) neural systems that supports adaptation to native-accented and foreign-accented speech.
- Speech adaptation across the life span
Effective spoken communication is vital to healthy aging. Speech recognition in challenging listening environments, including communicating with unfamiliar talkers, is more disruptive for older adults than for young adults, due to a compound of differences in auditory and cognitive abilities. Pioneering work shows that older adults with and without a hearing loss can rapidly adapt to speech with degraded intelligibility. The precise mechanisms by which older adults cope with challenging speech variants (e.g., nonnative-accented speech) remain elusive. In this project, we investigate whether older adults retain the perceptual flexibility to adapt to unfamiliar pronunciations and how their strategies change as a function of the specific acoustic patterns of speech categories. Furthermore, collaborating with experts on cognitive training, we study how individual variability in domain-general cognitive abilities (e.g., working memory) supports adaptive perception in this population.
- Modeling adaptive changes in speech perception
Speech perception is broadly believed to involve at least three mechanisms: (A) low-level ‘pre-linguistic’ cue normalization; (B) activation of linguistic category representations; and (C) post-perceptual decision-making. Adaptive changes like those observed for the perception of second language speech could originate from any (combination) of these three mechanisms, each implying fundamentally distinct perceptual, linguistic, and neural architectures. However, currently we know neither which mechanism(s) explains adaptivity, nor how they might interact depending on the type, amount, and/or time-course of exposure. We are currently developing an analytical framework of adaptive speech perception. This will be the first cognitive model of adaptive speech perception that integrates all three mechanisms, and specifies how each mechanism is affected by exposure.
Foreign-accented speech deviates from native norms of the target language and is generally harder to understand than native speech. However, non-native speech can be more easily understood than native speech when there is shared language background between the speaker and the listener. The weighting and integration of various acoustic cues is important for speech perception and language experience fundamentally shapes listeners’ cue-weighting strategies. We investigate the influences of both long-term (first language) and short-term (ambient language environment) language experience in modulating listeners’ speech perception in different language environments. Current work along this line involves further characterization of the phonetic variation in Mandarin-accented English, based on large samples of natural speech production. A special focus is on under-studied phonemes and production patterns in connected speech.
Talker identification can be easy. A single syllable reveals Mom’s voice over the phone. Talker identification can be hard. Speakers sound similar if you don’t know the language being spoken. Voice perception depends on language processing. Building upon an established “Language Familiarity Effect” (identifying talkers is easier in one’s native language than in unfamiliar languages) in talker identification, we take an individual differences approach and investigate the interplay of talker perception and language processing. We found that both linguistic (speaking a tone language) and relevant non-linguistic experience (musicianship) can enhance processing of talker information by sharpening general auditory skills (pitch processing). Meanwhile, individuals’ ability to encode phonetic detail strongly affects their ability to learn talker identity. Moreover, top-down lexical information may guide listeners’ interpretation of acoustic-phonetic variation and link it to talker identity. Continuing work along this line involves further testing of a direct link between fine-grained phonetic processing and talker identification. We aim to better understand how memory representation of sounds interfaces with talker processing systems (potentially by making use of speech variation) and guides online speech processing as well as voice perception.