Progress toward estimating the minimal clinically important difference of speech intelligibility: A crowdsourced perceptual experiment

Abstract

Purpose The purpose of this study was to estimate the minimal clinically important difference (MCID) of sentence intelligibility in neurologically healthy control speakers and speakers with dysarthria due to multiple sclerosis (MS) and Parkinson’s disease (PD). Methods Speakers were part of a larger study examining the acoustic and perceptual consequences of MS and PD. 48 speakers were included in the current study (16 healthy control speakers, 16 speakers with MS, and 16 speakers with PD). Speakers were recorded reading Harvard psychoacoustic sentences in speaking conditions purported to modulate intelligibility: habitual, clear, fast, loud, and slow. 240 listeners were recruited using the crowdsourcing website Prolific (prolific.co). A global ratings of change (GROC) scale developed by Jaeschke et al. (1989) was used as an ‘external anchor of meaningfulness’. Listeners heard a given speaker in one condition followed immediately by the same speaker in another condition. Listeners used Jaeschke’s GROC scale to indicate “how much more understandable?” one condition was over the other and were given response options on the following seven-point scale: (1) Almost the same, hardly any better at all; (2) A little better; (3) Somewhat better; (4) Moderately better; (5) A good deal better; (6) A great deal better; (7) A very great deal better. Previously published transcription intelligibility data for these speakers and stimuli were employed in calculations. Receiver operating characteristics (ROC) curves were used to determine how well the change in intelligibility scores between conditions differentiated between those speakers for whom listeners identified a change in understandability and those for whom listeners did not identify a change in understandability (Stipancic et al., 2018). The average difference in intelligibility for each response option on the GROC scale was defined as the MCID. Results & Discussion The MCID for a small change in intelligibility was determined to be 7% and for a moderate to large change in intelligibility was determined to be 15%. This work demonstrates feasibility of the novel experimental paradigm for collecting crowdsourced perceptual data for estimating MCIDs of speech outcomes. Findings are a critical step toward development of a universal language with which to evaluate changes in intelligibility as a result of speech-language therapy and disease progression.

Frits van Brenk

External consultant in the Motor Speech Disorders Laboratory

My research and teaching interests include motor speech disorders, clinical linguistics, experimental psycholinguistics, speech science, and research methods.