Generative Reconstruction of Pediatric Disordered Speech for Automated Clinical Evaluation

Karen Rosero1 Eunjung Yeo2 David R. Mortensen1 Cortney Van’t Slot3 Rami R. Hallac3,4 Carlos Busso1
1Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA, USA · 2Department of Computer Science, University of Texas at Austin, Austin, TX, USA
3Department of Plastic Surgery, UT Southwestern Medical Center, Dallas, TX, USA · 4Analytical Imaging and Modeling Center, Children’s Health, Dallas, TX, USA
Code Model

Abstract

This paper presents ChiReSSD, a speech reconstruction framework for children with speech sound disorders (SSDs). ChiReSSD uses a target transcript, along with a short reference recording from a child with SSD, to synthesize a corrected version of the target utterance. The system improves intelligibility by suppressing mispronunciations while preserving the child’s speaker identity, speaking style, and prosodic characteristics. Standard style-preserving text-to-speech (TTS) reinforces existing mispronunciations rather than correcting them, since pathological articulation is encoded as part of the speaker style. ChiReSSD addresses this limitation by routing pronunciation through the text pathway, while speaker identity and prosody are controlled by a style pathway, without requiring paired typical–atypical data or a dedicated model per child. We evaluate ChiReSSD on the STAR dataset, achieving a 48% relative reduction in character error rate and a 19% gain in speaker similarity over style-preserving baselines. Consonant-level annotations from a certified speech-language therapist on 21 paired samples yield a Pearson correlation of ρ = 0.63 with the automatic estimates, indicating feasibility for assisting clinical metric computation and motivating larger-scale annotation studies. While specifically designed for children’s speech, the results in the TORGO dataset further indicate that ChiReSSD can generalize to adult dysarthric speech across severity levels.

Method Overview

ChiReSSD training pipeline: a style module extracts speaker identity and prosody from a reference SSD utterance, while a text module encodes the target phonemes; both condition the style diffusion denoiser and the decoder.

Audio Samples

Original SSD Target word: “Butterfly”
One-shot Reconstruction Target word: “Butterfly”
ChiReSSD Reconstruction Target word: “Butterfly”
Original SSD Target word: “Strawberries”
One-shot Reconstruction Target word: “Strawberries”
ChiReSSD Reconstruction Target word: “Strawberries”
Original SSD Target word: “Toothbrush”
One-shot Reconstruction Target word: “Toothbrush”
ChiReSSD Reconstruction Target word: “Toothbrush”

Original SSD samples were obtained from: J. Lawson and J. Cleland, “STAR: Speech Therapy Animation and Imaging Resource,” Proceedings of the International Clinical Phonetics and Linguistics Association, 2023.