ProsodAI - Neural Encoding of AI-Generated Speech Prosody by L1 and L2 Speakers

Team: Dr Julia Schwarz (Psychology), Linda Bakkouche, Stephanie Cooper, Xinbing Luo, Madeleine Rees, Prof. Brechtje Post (Theoretical and Applied Linguistics), Charles McGhee(Engineering), Dr Kai Alter (Medical Sciences, Newcastle University)

State-of-the-art artificial intelligence (AI)-powered voice clones closely resemble human voices and are becoming an indispensable tool for many educational and clinical applications such as second language learning, audiobooks, and assistive technology. While AI voices can closely imitate human voices, they are not yet identical, especially with respect to speech prosody. However, to what extent the prosodic differences of AI-voices impact speech processing has not yet been investigated. This project aims to unravel listeners’ neural responses to prosodic features in human vs. AI voices, and to evaluate whether any resulting differences in encoding may influence language comprehension, particularly among language learners.