Risk of re-identification for shared clinical speech recordings

Abstract: Large, curated datasets are required to leverage speech-based tools in healthcare. These are costly to produce, resulting in increased interest in data sharing. As speech can potentially identify speakers (i.e., voiceprints), sharing recordings raises privacy concerns. We examine the re-identification risk for speech recordings, without reference to demographic or metadata, using a state-of-the-art speaker recognition system. We demonstrate that the risk is inversely related to the number of comparisons an adversary must consider, i.e., the search space. Risk is high for a small search space but drops as the search space grows ($precision >0.85$ for $<1*10^{6}$ comparisons, $precision <0.5$ for $>3*10^{6}$ comparisons). Next, we show that the nature of a speech recording influences re-identification risk, with non-connected speech (e.g., vowel prolongation) being harder to identify. Our findings suggest that speaker recognition systems can be used to re-identify participants in specific circumstances, but in practice, the re-identification risk appears low.

\section{Introduction}

Advances in machine learning and acoustic signal processing, along with widely available analysis software and computational resources, have resulted in an increase in voice and speech (hereafter ‘speech’ for simplicity) based diagnostic and prognostic tools in healthcare \cite{Fagherazzi21}. Applications of such technology range from early detection of cardiovascular \cite{Sara22}, respiratory \cite{Maor21}, and neurological \cite{Tracy20} diseases to prediction of disease severity \cite{Asgari10} and evaluation of response to treatment \cite{Pigoni19}. These advances harbor substantial potential to enhance patient care within neurology given the global burden of neurological diseases \cite{Daly19,Daly21}, poor global access to neurological expertise \cite{worldatlas04,Janca06}, and the established role of speech examination within the fields of neurology and speech-language pathology \cite{Duffy19}.

Large, curated datasets are needed to harness the advances in this area. However, these datasets are costly to assemble and require rare domain expertise to annotate. This has led to interest in data sharing among investigators and industry partners, yet sharing speech data is not without risk given the potential personally identifying nature of speech recordings \cite{RibaricPavesic17, Malin10}. This prevents the application of conventional de-identification approaches because removing identifying metadata (e.g., participant demographics, date and location of recording, etc.) may not be sufficient if the speech signal itself can act as an identifier \cite{ElEmam06}. In addition, de-identification approaches that involve modifying non-linguistic aspects of a sample through distortion or alteration of the signal may address the inherent identifiability of the speech signal (i.e., its potential as a voiceprint) \cite{RibaricPavesic17, Qian18}, but this is not an option when the goal is to use the acoustic signal to detect subtle non-linguistic abnormalities indicative of the presence of neurological disease, a central part of the existing use of speech examination in medicine (i.e., perceptual or acoustic analysis) \cite{RibaricPavesic17, Duffy19}.  In recognition of these potential problems, the HIPAA Safe Harbor implementation of de-identification, which is often invoked for the creation of research datasets, specifically mention voiceprints as an example of biometric identifiers. De-identification in compliance with HIPAA may still be possible under the Expert Determination implementation, whereby the risk of re-identification for the unmodified speech recordings is deemed very low according to generally accepted statistical and/or scientific principles. In this respect, various prior studies have investigated the risk of re-identification in research cohort datasets based on demographic or other metadata that may link a participant to their corresponding records \cite{Atreya13, BenitezMalin10, Cimino12}, but none have explicitly assessed the inherent risk of an acoustic signal itself. Determining the risk of re-identification for recordings in speech datasets and learning how to best mitigate such risk is thus necessary for healthcare institutions in order to protect patients, research participants, and themselves.

Unfortunately, the same machine learning advances that facilitate the potential use of speech in healthcare have also made de-anonymization or re-identification attacks more feasible. Attempting to re-identify from only a speech recording relies on the mature field of speaker recognition, of which there is an immense amount of research \cite{Hanifa21, Sztaho21}. These studies suggest the potential for identification from the acoustic signal alone may be high \cite{Nagrani19}, but there have been minimal studies using speaker recognition in the context of adversarial attacks that may result in potential harm to a speaker \cite{Lu21, Qian21}. Only one prior study relied upon a speaker recognition system to attempt re-identification, and the results suggested that the risk was high with a single unknown speaker and a reference set of 250 known speakers \cite{Qian21}. As such, the risk inherent in the acoustic signal, devoid of metadata, is non-zero but relatively unknown, and the feasibility for larger datasets remains unexplored.

In addition, these approaches are rarely applied to medical speech datasets \cite{Arasteh22}. Medical speech recordings generally differ in at least three systematic ways. First, the recordings typically contain abnormal speech, which may make re-identification harder since many speech disorders are the result of progressive neurological diseases, which cause changes in speech that evolve over months to years \cite{Duffy19}. Matching recordings from a time when a speaker was healthy or mildly affected to recordings where they have a more severe speech disorder may be more difficult \cite{Young10, Mustafa15, Derussis19}. Second, the premise of speaker verification systems are that there are recognizable between-speaker differences tied to identity. However, in a cohort enriched with abnormal speech, a substantial proportion of the variance would be tied to the underlying speech disorder as these cause recognizable deviations \cite{Duffy19}, resulting in speakers sounding less distinct \cite{Leveque22}. Finally, medical speech recordings typically contain responses to elicited speech tasks rather than the unstructured connected speech typically used in recognition experiments. Some speech task responses do contain connected speech (e.g., paragraph reading), but others are very dissimilar (e.g., vowel prolongation). The impact of speech tasks on identifiability remains unknown.

In this study we address the risk of re-identification for unknown speakers in a shared dataset in a series of experiments exploring the re-identifiability of speech recordings \textit{without} using any metadata. We accomplish this goal using a state-of-the-art speaker recognition deep learning architecture that an adversary might train on publicly available, identified recordings and apply it to a set of unidentified recordings. We first demonstrate that the risk is inversely related to the number of comparisons an adversary must consider, with this search space being a function of the size of a shared dataset of unknown speakers and the known-speaker dataset used by the adversary. The risk is high for a small search space but drops sharply as the search space grows. Next, we show that the nature of a speech recording substantially influences re-identification risk, with recordings containing non-connected speech (e.g., vowel prolongation) being harder to identify. Our findings suggest that in principle, speaker recognition systems can be used to re-identify unknown speakers, but in practice, the risk of re-identification appears low when the search space is large.

\section{Results}
For the purposes of our experiments, we assumed that a data recipient has decided to attempt re-identification, therefore becoming an adversary. We further assumed that this adversary would rely on an attack strategy known as a marketer attack, wherein they have access to a large dataset of known speakers (hereafter the ‘known’ set), perhaps obtained from an online source such as YouTube, to train a recognition system that is then used to re-identify as many unknown speakers in the shared dataset (hereafter the ‘unknown’ set) as possible (see \hyperref[met:exd]{"Methods"} for experimental design) \cite{BenitezMalin10, Dankar10}. Other attack scenarios are possible, but a marketer attack established a generally accepted baseline for risk. Further implications of these assumptions are addressed in the \hyperref[dis]{Discussion}.

To simulate this attack scenario, we built a text-independent speaker recognition system with a combination of x-vector extraction using the \hyperref[met:xvec]{ECAPA-TDNN model} \cite{Desplanques20} and a downstream \hyperref[met:plda]{Probabilistic Linear Discriminant Analysis (PLDA)-based classifier}\cite{Khosravani17, Borgstrom20} (see \hyperref[met]{"Methods"} for implementation). Figure \ref{Fig1} depicts the architecture of our system.

Since the adversary was comparing known and unknown speakers, our primary outcome of interest was the average number of false acceptances (FAs), where the model accepts a match for an unknown speaker without a true match, as compared to true acceptances (TAs) over several subsampled datasets generated from VoxCeleb\cite{Nagrani17, Chung18, Nagrani19} or our Mayo Clinic speech recordings dataset. This informed the reliability of re-identification. Using the counts, we also calculated the Pearson’s correlation coefficient between FAs and set size as well as the false acceptance rate (FAR) to extrapolate any existing trends for FAs.
\begin{figure}
\setcounter{figure}{0}
\centering
\includegraphics[height=6in]{figure1.png}
\caption{\textbf{Speaker Recognition System Architecture.} During training (top section) only the recordings from known speakers are used. These are fed into a pretrained speaker identification model, which outputs the predicted speaker out of a closed set. The prediction is not used, however, and instead the activations from the penultimate fully connected layer are extracted for each recording. These represent a low dimensional, latent representation for each recording that is enriched for speaker-identifying features (x-vectors). We used these x-vectors for known speakers to train a PLDA classifier, which scores the pairwise similarity for all recordings and proposes a threshold which minimizes the cost function (minDCF). This was done over several subsets and the threshold averaged to obtain the final training threshold. During testing (bottom section), x-vectors are extracted for recordings from unknown speakers in the same was as before. These are then fed into the trained PLDA but pairwise comparisons are done with the known speaker x-vectors. The training threshold is applied, and the net result is a set of matches (or no matches) for each recording. Those matches are used to calculate our metrics of interested (FAs, TAs, FAR, etc.)}
\label{Fig1}
\end{figure}

\subsection*{Effect of search space size: VoxCeleb Realistic Experiments}\label{res:search}
We relied upon VoxCeleb 1 and 2 to investigate the capability of a re-identification attack as a function of the size of the search space (i.e., the number of comparisons made to find matching speakers). We re-identified speakers by comparing each speaker in the known set to each speaker in the unknown set. The search space is thus the product of the size of the known and unknown set. As such, an increase in either set can increase the number of comparisons. We consider both cases separately, which allows us to consider one scenario that is dependent on the resources of the adversary (known set) and another that is under the control of the sharing organization (unknown set).

To construct a realistic scenario, we assumed that the known and unknown set would have low speaker overlap. To justify this assumption, one can consider what would be involved in constructing a set of known speakers. In the absence of metadata about the unknown speakers (e.g., the ages, location, etc.), there would be no way for an adversary to target a specific population to build out their known set. It is not often feasible for an adversary to manually collect and label speech recordings for a substantial proportion of the population. Instead, an adversary would likely need to rely on a programmatic approach using easily accessible identifiable audio, such as scraping audio from social media and video or audio sharing websites \cite{Noorden20}. This would still be difficult because of several confounding factors: (1) not all members of the population use these websites; (2) not all users have publicly accessible accounts; (3) users with publicly accessible accounts may not have identifiable information linked to it; (4) some accounts post audio/video from multiple speakers, including speakers who also have their own accounts; (5) many users do not post at all; and (6) the population of users is not representative of the general US population, let alone the subset with speech disorders – in terms of the distributions of both age and geographic area \cite{pew21}. As such, there is no reason to suspect that a speaker in a shared speech dataset would have a high likelihood of existing in an adversary’s set of identified audio recordings.

We also assumed that the adversary would not know which unknown speakers, if any, exist in the known speaker set. The adversary must therefore consider all potential matches rather than only focusing on the \textit{N} overall best matches ($ N = $ amount of overlap). This would reduce reliability since the likelihood of all potential matches being true is lower than the likelihood of the best \textit{N} matches being true.

We first trained the speaker recognition model with increasing numbers of speakers in the known set while maintaining a static unknown set size with low speaker overlap between sets (see \hyperref[met:vox]{“Methods”}). We found that increasing the number of speakers in the known set resulted in an increase in the mean number of FAs while TAs remained stable, with a linear correlation between FAs and number of known speakers ($r = 0.3047$, p-value $= 0.00015$, $t = 3.8914$, $df = 148$, Figure \ref{Fig2}a). We then trained a PLDA model with a known set of fixed size while increasing the number of speakers in the unknown set and maintaining low overlap (see \hyperref[met:vox]{“Methods”}). Increasing only the size of the unknown set had a similar yet more pronounced effect, with a higher linear correlation between FAs and number of unknown speakers ($r = 0.6037$, p-value $= 2.898*10^{-16}$, $t = 9.2129$, $df = 148$, Figure \ref{Fig2}b).

The difference in effect can be understood based on the geometry of the search space. While the unknown set remains significantly smaller than the known set, adding any additional speaker to the unknown set will result in a larger increase in the search space than adding a speaker to the known set. As such, we can better demonstrate the overall trend in FAs by considering the results in terms of total comparisons (i.e., search space size) rather than individual set size.

We observed that there was a high positive linear correlation between FAs and the number of comparisons ($r= 0.6934$, p-value $< 2.2*10{-16}$, $t = 13.541$, $df = 198$, Figure \ref{Fig2}c), with the mean number of FAs increasing from 0.04 to 2.84, while TAs remained stable. There was a corresponding drop in precision (Figure \ref{Fig3}a). It was notable that the FAR remained low and relatively stable, averaging at $4.152*10^{-07}$ (Figure \ref{Fig3}b), indicating that the demonstrated trend should hold in a real attack where we would expect even more comparisons. We further observed that using a stricter threshold for matches resulted in our model selecting only one match per speaker. This is functionally the same as limiting matches to only the best potential match for each speaker (rank 1 matches), which is an option for an adversary to increase reliability without knowledge of the amount of overlap.

\begin{figure}[!ht]
\centering
\includegraphics[width=\textwidth]{figure2.png}
\caption{\textbf{Number of true and false acceptances for the speaker recognition model in a realistic scenario.} \textbf{(a)} shows the counts when varying the number of known speakers while keeping the number of unknown speakers static, \textbf{(b)} shows the counts when varying the number of unknown speakers while keeping the number of known speakers static, and \textbf{(c)} shows the overall trend in terms of number of comparisons made (i.e., the search space size = known $*$ unknown speakers). All plots \textbf{(a-c)} include the Pearson's correlation coefficient and corresponding significance for false acceptances and number of speakers/comparisons. Each run is plotted as a single circle, with red horizontal lines indicating the mean number of false acceptances and green horizontal lines indicating the mean number of true acceptances. }
\label{Fig2}
\end{figure}

\begin{figure}[!h]
\centering
\includegraphics[width=\textwidth]{figure3.png}
\caption{\textbf{Precision and False Acceptance Rates for the speaker recognition model in a realistic scenario.} Precision \textbf{(a)} and false acceptance rates \textbf{(b)} are shown as a function of the number of comparisons. For both plots, each run is represented by a circle, and the mean is represented by a horizontal black line.}
\label{Fig3}
\end{figure}

\subsection*{Worst-case scenarios: VoxCeleb Known Overlap and Full Overlap Experiments}
There are two important initial assumptions in our construction of the VoxCeleb realistic experiments: (1) the adversary was unaware of the amount of overlap between known and unknown sets and (2) the amount of overlap was low. Thus, we considered how re-identification risk would be affected if either assumption was incorrect.

First, we considered a potential worst-case scenario in which the adversary did know the amount of overlapping speakers \textit{N} and was therefore able to limit potential matches to the top \textit{N} best matches. As previously mentioned, limiting the number of matches could theoretically improve model reliability, and further reducing the number of matches could produce more noticeable effects. We leveraged our base results from the realistic experiments and only considered the top \textit{N} best matches. We found that even with this knowledge, there was still a trend of increasing FAs, with a high linear correlation to number of comparisons ($r = 0.6981$, p-value $< 2.2*10{-16}$, $t = 13.72$, $df = 198$, Figure \ref{Fig4}a). This indicates that some FAs were seen as better matches than some TAs, as further supported by the associated drop in precision (Figure \ref{Fig4}b).

Next, we considered a less realistic worst-case scenario wherein all unknown speakers exist in the known speaker set. From an adversary’s perspective, a full overlap scenario would provide the best chance for the adversary to successfully re-identify speakers since most FAs occur when the model finds a match for unknown speakers who are not in the known speaker set. In fact, in our realistic scenarios, all FAs were associated with matches for non-overlapping speakers.

We assessed this scenario by replicating the realistic experiments with full overlap between known and unknown sets (see \hyperref[met:vox]{“Methods”}). As expected, the performance improved significantly, with most matches being correct (Figure \ref{Fig4}c). Even so, there was still a high positive linear trend for FAs, indicating that at high overlap, some FAs were ranked higher than TAs ($r = 0.6705$, p-value $= 1.019*10{-11}$, $t = 7.9821$, $df = 78$, Figure \ref{Fig4}d).

\begin{figure}[!h]
\centering
\includegraphics[width=\textwidth]{figure4.png}
\caption{\textbf{Results for our speaker recognition model in worst-case scenarios.} \textbf{(a)} shows the true and false acceptance counts for a known overlap scenario (limited to \textit{N} = 5 best matches), while \textbf{(b)} shows the corresponding precision as a function of the number of comparisons (search space size). \textbf{(c)} and \textbf{(d)} show the false and true acceptance counts for a full overlap scenario, where all unknown speakers are present in the known speaker set as a function of the number of comparisons (search space size). \textbf{(a)} and \textbf{(c)} also shows the Pearson's correlation coefficient and corresponding significance between false acceptances and number of comparisons. Each run is plotted as a single circle, with red horizontal lines indicating the mean number of false acceptances, green horizontal lines indicating the mean number of true acceptances, and black horizontal lines indicating the mean precision.}
\label{Fig4}
\end{figure}

\clearpage
\subsection*{Effect of tasks on risk of re-identification: Mayo Clinic speech dataset experiments}
Next, we shifted our focus from the public VoxCeleb dataset to a private dataset of Mayo Clinic medical speech recordings to look at factors specific to a clinical speech dataset, such as whether certain elicited tasks are easier for re-identification and whether being able to link recordings to the same speaker across tasks (pooling) increases risk.

We first compared the performance of the speaker recognition system across the various elicited speech tasks in the Mayo dataset based on the same adversarial attack scenario used with the VoxCeleb experiments (see \hyperref[met:mayo]{“Methods”}). In this scenario, the cross-task performance aligns with a real-world case where the known set contains connected speech recordings (i.e., recordings of continuous sequences of sounds like that of spoken language), but speakers are re-identified using a variety of elicited speech tasks (Table \ref{tab1}). Each task has a different degree of similarity to connected speech (left: most, right: least):

\begin{center}
    \textit{Reading Passage $>$ Sentence Repetition $>$ Word Repetition $>$ SMR $>$ AMR $>$ Vowel Prolongation}
\end{center}

The reading passage is essentially real-world connected speech in terms of content and duration, but sentence repetition is closer to the connected speech seen in most speech datasets \cite{Nagrani19}. As such, we selected sentence repetition recordings for speakers in the known set. The baseline was therefore when sentence repetitions were selected for both known and unknown speakers.

We observed that the total number of acceptances decreased as the unknown speaker tasks became less like the baseline known speaker task, but the proportion of TAs and FAs also varied across task. This made it more difficult to determine the performance through counts alone (Figure \ref{Fig5}a). When considering precision instead, we found that along with the reading task, the word repetition and sequential motion rate (SMR) tasks also had improved performance when compared to baseline (Figure \ref{Fig5}b). Notably, vowel prolongation had a precision of zero (no TAs) but a high number of FAs.

In addition to the cross-task performance, we also compared the within-task performance — where the same elicited task is used for both known and unknown speakers — to determine if anything about the nature of a given speech task affects re-identification. For example, the variance across recordings for the sentence repetition task reflects a combination of static speaker factors (e.g., identity, and age), dynamic speaker factors (prosody, e.g., the same speaker may emphasize different words in a sentence on repeated trials), and content factors (i.e., different words in different sentences). By contrast, a task like alternating motion rates involves repeating the same syllable as regularly and rapidly as possible, with most of the variance across speakers likely resulting from static speaker factors.  When considering all the elicited tasks, one would therefore expect the proportion of variance across speakers due to dynamic speaker factors to decrease following the same scale as similarity to natural speech. The reading passage would have the most variance due to dynamic speaker factors alone while vowel prolongation would have the least. By removing the confounding variable of different elicited tasks for known and unknown speakers (i.e., the model is both trained and tested on the same task), we can ascertain whether qualities of the speech task itself influence re-identification.

The within-task results did not exhibit the same effect as the cross-task. Instead, we observed a generally inverse effect, where performance improved as tasks became more dissimilar from connected speech and had less variance due to dynamic speaker factors (Figure \ref{Fig5}c-d).

\begin{figure}[!h]
\centering
\includegraphics[width=\textwidth]{figure5.png}
\caption{\textbf{Speaker recognition results for the clinical speech dataset.} \textbf{(a)} and \textbf{(b)} show cross-task results, where recordings for known speakers are always sentence repetition but the task for unknown speaker recordings varies. The baseline is when sentence repetitions are in both the known and unknown set. Pooling is when all recordings for an unknown speaker are linked together across all tasks. \textbf{(a)} show the breakdown of counts for this case while \textbf{(b)} is the corresponding precision. \textbf{(c)} and \textbf{(d)} show within-task results, where tasks for known and unknown speakers are always the same. \textbf{(c)} is the breakdown of counts for this case while \textbf{(d)} is the corresponding precision. Each run is plotted as a single circle, with red horizontal lines indicating the mean number of false acceptances, green horizontal lines indicating the mean number of true acceptances, and black horizontal lines indicating mean precision.}
\label{Fig5}
\end{figure}

\pagebreak

\section{Discussion}\label{dis}

In this study, we investigated the risk of re-identification for unidentified speech recordings without any other speaker or recording related metadata. To do so, we performed a series of experiments reflecting a marketer attack by an adversary with access to identified recordings from a large set of speakers and the capability to train a speaker recognition system, which would then be used to re-identify unknown speakers in a shared dataset. We systematically considered how changes in the size of the datasets and the nature of the speech recordings affect this risk. We found that it is feasible to use a speaker recognition system – a deep learning speaker embedding extractor (x-vectors) coupled with a PLDA backend – to re-identify speakers in an unknown set of recordings by matching them to recordings from a set from known speakers. Given the performance of current state of the art speaker recognition systems, this is not surprising. However, these systems have only rarely been applied in an adversarial attack scenario\cite{Lu21, Qian21} – i.e., their potential as an attack tool for an adversary who aims to re-identify speakers in a shared or publicly available dataset was largely unknown. Furthermore, the feasibility of such an attack has not been considered and may have been assumed to be low if speech recordings were stripped of all metadata (sometimes referred to as de-identified or anonymous in the literature) without considering the identifiability of the acoustic signal itself \cite{Zhang20, Sharma20, Turrisi21, Flechl22}.

Our findings suggest this is not true. Consistent with a previous study that found high re-identification risk for one unknown speaker with known sets of up to 250 speakers (search space $<=$ 250 comparisons) \cite{Qian21}, we observed that the risk was indeed high for small search spaces. For example, when attempting to re-identify 5 overlapping speakers between a small set of unknown speakers ($n=163$) and a moderate set of known speakers ($n=1000$) our system had nearly perfect precision (Figure \ref{Fig3}a) and identified 2 speakers on average (Figure \ref{Fig2}a). However, our experiments allowed us to extend this to more realistic search spaces, such as scenarios where an adversary uses a known speaker set of up to 7205 speakers and an unknown speaker set of up to 1000 speakers (search space $<= 6,000,000$ comparisons). We observed that the risk dropped sharply as the search space grew. The FAR was relatively stable at $\sim 4.152*10^{-7}$ (Figure \ref{Fig3}b), which translates to an average of increase of one FA for every $\sim 2.5$ million comparisons. This is a key take home message from these experiments – increasing the size of the search space, whether through growing the size of the adversary’s set of identified recordings or the shared dataset, resulted in a corresponding increase in the number of FAs. Given that the number of overlapping speakers remained constant, this suggests that the primary driver of FAs is the size of the non-overlapping known to unknown speaker comparison space, i.e., most FAs arise from non-overlapping unknown speakers being falsely matched to known speakers. In fact, all FAs in the realistic experiments correspond to non-overlapping unknown speakers. Here, it is worth noting that in the experiments where we only considered the top \textit{N} matches (where \textit{N} = amount of overlap), this trend remained true due to the fact that some of the FAs were scored higher than TAs by the model (Figure \ref{Fig4}). This suggests that for a sufficiently large search space, even considering only the best \textit{N} matches will result in many FAs. We pushed this line of reasoning to its limit by considering a worst-case scenario of full overlap, where all unknown speakers had a true match. Even in this scenario there were still many FAs, and the proportion of FAs grew with increasing search space size. Importantly, this scenario showed that overlapping speakers can still be falsely matched when the overlap is high.

Our experiments with the Mayo clinical speech recordings allowed us to assess the influence of speech task based on both cross-task and within-task performance. When the model was trained on sentence repetition (i.e., the known dataset consists of sentence recordings) and then applied to other tasks (i.e., the unknown set consists of elicited, non-sentence speech), the precision slightly improved for the reading passage, word repetition, and SMR tasks but deteriorated for the less connected speech-like tasks such as AMR and vowel. These results can be understood with reference to the threshold that was used for accepting a match. The threshold was chosen using sentence repetition task recordings, such that in most instances, all overlapping speakers were re-identified for unknown sets with similar connected speech tasks (sentence repetition, reading, word repetition, SMRs). The threshold for these similar tasks resulted in fewer overall acceptances (higher false rejection rate (FRR)), but as the tasks diverged from sentence repetition with respect to the degree of connectedness, there were also fewer FAs. This suggests that identifiable characteristics learned from training on the sentence repetition task translate relatively well to other connected speech tasks. It also demonstrates the difficulty of choosing a threshold when the tasks in the known set are very different from the unknown set – because of difference within a speaker across tasks, it becomes hard to balance the goal of some TAs with the flood of FAs as the search space increases. In this instance, a slightly stricter threshold which penalized FAs more than false rejections (FRs), unlike the one we used which penalized both equally, may have been better for the adversary. By contrast, the non-connected speech tasks (AMRs, vowel prolongation) had almost no TAs and a high number of FAs, suggesting that identifiable characteristics from connected speech tasks do not translate to non-connected speech tasks. This is not unexpected given that models perform worse when tested on data that is dissimilar from the training data \cite{Kenny07, Kinnunen10}. Following this, we also failed to find an effect for pooling across tasks. Generally, having more data for a speaker is expected to improve model performance, but it is possible that adding recordings of non-sentence tasks to the unknown set did not improve performance because the identifiable characteristics are different across tasks and the model is unable to accommodate all of them. In other words, any helpful characteristics from the connected speech tasks were cancelled out by competing characteristics from the non-connected speech tasks.

In the within-task scenarios, where the known and unknown set were made up of the same task, the performance was better than in the cross-task scenario. Importantly, this was true even for tasks that are different from connected speech. Vowel prolongation, which is non-connected and the most perceptually different from sentence repetition, exhibited the worst cross-task performance but the best within-task performance. This may be that due to fewer interfering dynamic speaker factors, this specific task isolated acoustic features that are tied to identity well, which allows for good within-task performance, but these features did not translate to connected speech tasks. Another important finding from the within-task results is that performance for sentence repetition was much weaker than expected based on the VoxCeleb experiments with a larger number of comparisons. We suspect this may be due to a combination of factors. First, it may be more difficult to differentiate speakers in an unknown set of elicited recordings where every speaker is uttering the same sentence. Second, the clinical recordings were all made by patients who were referred for a speech examination. Consequently, the resulting cohort contains mostly abnormal speech, which may impact the PLDA performance. Third, the Mayo clinical speech dataset is smaller than the VoxCeleb dataset both in terms of number of speakers and number of recordings per speaker, and the recordings are also shorter in duration. This likely had a negative impact on the training of the PLDA classification backend. It remains unknown if larger clinical datasets or datasets with more recordings per speaker may yield findings more similar to the VoxCeleb results.

Taken together, our findings suggest that the risk of re-identification for a set of shared clinical speech recordings devoid of any metadata in an attack scenario like the one we considered is influenced by (1) the number of comparisons an adversary must consider, which is a function of both the size of the unknown and the known datasets; (2) the similarity between the tasks and/or recordings in the sets; and (3) the characteristics of the recordings in the sets, such as duration of the recording, degree of speaker variance, and presence and type of speech disorders. These findings translate to actionable goals for both an adversary and the sharing organization.

While we assumed that the sharing organization already reduced risk by stripping recordings of demographic (e.g., age or gender) or recording metadata (e.g., date or location), we additionally suggest that re-identification risks could be further reduced by increasing the search space (i.e., larger shared dataset size) or decreasing the similarity between shared recordings and publicly available recordings (e.g., sharing vowel prolongation as long as a publicly available vowel prolongation dataset does not exist or sharing a larger variety of speech disorders instead of a single disorder). Even if the number of overlapping speakers increased with the size of the shared dataset, the results from the full-overlap scenario indicate that a model could still have reduced reliability due to an increasing FAR.

By contrast, the adversary can also use this knowledge to enhance their attacks. From their perspective, any additional information that can reduce the search space or increase similarity between recordings will increase the reliability of speaker matches. This could involve using demographics such as gender, be it shared or predicted by a separate model, to rapidly reduce the number of comparisons. For instance, when gender balance is 50:50, comparing unknown males to known males would reduce the number of comparisons by 75\%. The adversary may also seek out publicly available recordings of abnormal speech to refine their model(s) or reduce the search space. If social media groups exist where identified users with certain medical or speech disorders post video or audio, an adversary could restrict their known set to these users.

Similarly, research participants and support staff may also influence risk through disclosure of participation. By disclosing participation in a study that is known to share speech recordings, a participant would effectively reduce the size of the known set to one, increasing their individual risk of re-identification. In addition, having a confirmed match can increase risk overall as the adversary would have a baseline to determine the reliability of matches \cite{Liu19}.

Although the focus of this investigation has been on the change in relative risk with changes in dataset size and speech task, it is worth considering our findings in the context of other factors that impact risk in practice. The most obvious is the availability of additional data on the speakers or recording metadata. In this respect, it is worth noting that sufficient demographic data even in the absence of speech is well known to carry a significant risk of re-identification \cite{BenitezMalin10, Xia21}. If any aspect of the metadata makes a patient population unique (i.e., there is only one person in a given age range), the risk of re-identification increases \cite{ElEmam06, Malin10}. Furthermore, if any data about the speaker (e.g., sex) or recording (e.g., date) reduces the search space, re-identification risk would increase. Depending on the nature of the speech task there may also be content in the recording itself that is identifiable. During less structured speech tasks such as recordings of open-ended conversations, participants may disclose identifiable information about themselves (e.g., participant saying where they live). Removing these spoken identifiers is an active area of research \cite{Qian21}.

Still, it is important to acknowledge that simply because records are vulnerable to re-identification does not mean that they would be re-identified. Notably, when assessing privacy concerns, the probability of re-identification during an attack is ultimately conditional on the probability of an attack occurring in the first place \cite{Xia21}. In most instances where data is shared, the receiving organization or individual will not have any incentive to attempt re-identification. The sharing organization, and in some cases a receiving organization, may also take steps to discourage the risk of an attack. These may take the form of legal (e.g., data sharing agreements) or technical (e.g., limited, monitored access) deterrents to a re-identification attack \cite{Tucker16}. By contrast, the risk of an attack may be higher for publicly available datasets \cite{Culnane17}, but there may also be a greater risk for re-identification without a targeted attack. For example, in the field of facial recognition some companies have scraped billions of photos from publicly available websites to create massive databases with tens of millions of unique faces. These are then used to train a matching algorithm \cite{Noorden20} which an end user could query using a photo of an unknown face and obtain a ranked list of matching faces and the source (e.g., Facebook). The end user can visit the source website and instantly gain access to other data that may increase or decrease their confidence in a match as well as provide feedback on matches thereby gradually increasing the performance of the tool as well as the number of known faces. If similar databases are built for speech recordings, it will almost certainly include publicly available medical speech recordings. Every query to the system would then represent a threat to such a public sample being matched to a queried recording regardless of the intent of the user that queried the system. Such a scenario is difficult to simulate because of the continuously improving nature of the algorithm and the fact that users would incorporate various degrees of non-connected speech and non-speech data.

Refraining from publicly releasing datasets is an obvious mitigation strategy to some of these threats. However, the risk of re-identification must always be balanced with the benefit of data sharing, since larger, more representative datasets for development and testing of digital tools may benefit patients. It is critical that policymakers consider this balance in the context of the rapidly evolving field of artificial intelligence. Naïve approaches such as the ‘de-identification release-and-forget model’ are unlikely to provide sufficient protection \cite{Rocher19}. Similarly, informed consent for public release is problematic because the risk of re-identification will neither be static nor easily quantifiable over time. This has led to the development of potential alternative approaches such as data trusts, synthetic data, federated learning, and secure multiparty computation \cite{Young19, Rieke20, Kaissis20, Lan21}.

It should be recognized that there are several notable limitations to our investigation. First, while we relied upon state-of-the-art learning architectures, the risk may differ if other computational approaches are considered \cite{Hanifa21, Sztaho21}. Second, we did not consider multi-stage adversarial attacks where one model is used to predict a demographic, such as sex or age, that is then used to limit the search space, or a scenario where an adversary manually goes through all potential matches to attempt manual identity verification. However, such approaches would introduce additional uncertainty for the adversary and may still require many comparisons depending on the sizes of the datasets, in which case our results would apply \cite{Xia15, Venkatesaramani21}. Third, we did not directly consider the risk for normal versus abnormal speech. Nearly all recordings in the Mayo Clinic speech dataset contain abnormal speech, whereas all VoxCeleb recordings are all from normal speakers. Ideally, there would be a single dataset that contains both. Fourth, it should be noted that our results may not generalize well outside of the US due to the fact that the VoxCeleb data has a strong US bias, and all the Mayo recordings were captured in the US. As such, it will be important to perform future experiments that leverage alternative computational architectures, more complex adversarial attacks, conversational speech, and data from other geographic regions to assess the re-identification risk for medical speech data more comprehensively.

In summary, our findings suggest that while the acoustic signal alone can be used for re-identification, the practical risk of re-identification for speech recordings, including elicited recordings typically captured as part of a medical speech examination, is low with sufficiently large search spaces. Indeed, the size of the search space, as well as the similarity of the speech tasks in each set, may be the principal factors in determining the risk of re-identification if an attack occurs. This provides actionable recommendations to further increase participant privacy and considerations for policy around public release of speech recordings. Finally, we also provide ideas for future studies to extend this work, most notably the need to assess other model architectures and datasets as improvements in speaker recognition could substantially increase re-identification risk.

\section{Methods}\label{met}

\subsection*{Datasets}
An ideal dataset would consist of (1) a set of elicited speech recordings from tasks typically used in clinical or research speech evaluations, and (2) a set of unstructured speech recordings including the same speakers as (1) but acquired at a different time and place. This would allow us to directly assess the risk of re-identification of shared speech recordings by training a model on unstructured connected speech, similar to what an adversary may find online. Such a dataset does not exist. As such, we made use of two separate datasets. The first was a combination of the well-known VoxCeleb 1 and 2 datasets, which contain recordings from an online source for over 7000 speakers \cite{Nagrani17, Chung18, Nagrani19}. The second was a medical speech dataset from Mayo Clinic, which contains recordings of commonly used elicited speech tasks but with fewer speakers.

\subsubsection*{VoxCeleb}
The VoxCeleb 1 and 2 datasets are recent large-scale speaker recognition datasets containing speech clips extracted from celebrity interviews on YouTube
\cite{Nagrani17, Chung18, Nagrani19}. The utterances are examples of natural, real-world speech recorded under variable conditions for speakers with different ages, accents, and ethnicities. VoxCeleb 1 and 2 have a combined total of 1,281,762 recordings from 7,363 speakers.

\subsubsection*{Mayo Clinic speech recordings}
The Mayo Clinic clinical speech dataset consists of recordings from elicited speech tasks in previously recorded speech assessments. Each speaker has some combination of clips from various tasks commonly used in a clinical speech evaluation, including sentence repetition, word repetition, paragraph reading, alternating motion rates (AMRs), sequential motion rates (SMRs), and vowel prolongation \cite{Duffy19}. Clips from speakers vary in recording medium (cassette recording vs. DVD), microphone, degree of background noise, and presence and severity of motor speech disorder(s). There are a total of 20,228 recordings from 990 speakers (breakdown in Table \ref{tab1}).

\begin{table}[h]
\begin{center}
	    \small
		\begin{tabular}{ | c | c | c | c | c | c | c |}
		\hline
		\textbf{Vowel} & \textbf{Alternating} & \textbf{Sequential } & \textbf{Word} & \textbf{Sentence} & \textbf{Reading} \\
	     \textbf{Prolongation} & \textbf{Motion Rates} & \textbf{Motion Rates} & \textbf{Repetition} & \textbf{Repetition} & \textbf{Passage} \\
		 & \textbf{(AMR)} & \textbf{(SMR)} & & & \\
		\hline\hline
		 'aaaaaah' & 'puh', 'tuh', 'kuh' & 'puh-tuh-kuh' & 'catapult','catastrophe' & 'My physician...' & 'You wish to know...' \\
	     1846 recordings & 4205 recordings & 1119 recordings & 142 recordings & 278 recordings & 714 recordings \\
		 852 speakers & 826 speakers & 606 speakers & 72 speakers\textsuperscript{1} & 246 speakers\textsuperscript{2} & 542 speakers \\
		 & & & & & \\
		 & & & other words & other sentences & \\
		 & & & 4222 samples & 7702 samples & \\
		 &  & & 371 speakers\textsuperscript{1} & 586 speakers\textsuperscript{2} & \\
	  \hline
	  \end{tabular}

	  {\footnotesize \textsuperscript{1} 371 total unique speakers}
		{\footnotesize \textsuperscript{2} 586 total unique speakers}
		\end{center}
	\caption{\label{tab1}\textbf{Breakdown of number of recordings and speakers for each task in the Mayo Clinical Speech dataset.}}
		\end{table}

\subsection*{X-vector extraction with ECAPA-TDNN}\label{met:xvec}
We generated speaker embeddings using a Deep Neural Network to extract fixed-length embedding vectors (x-vectors) from speech recordings \cite{Borgstrom20, Desplanques20}. This technique has been shown to outperform previous embedding techniques, such as i-vectors \cite{IbrahimRamli18, Wilkinghoff20}, while offering competitive performance to newer end-to-end Deep Learning approaches \cite{Hanifa21, Sztaho21}.

Our network of choice was the state-of-the-art ECAPA-TDNN model, pre-trained on a speaker recognition task using VoxCeleb 1 and 2 \cite{Desplanques20}. This model extracts a 192-dim x-vector for each speech sample. The model is publicly available through SpeechBrain \cite{speechbrain}, an open-source AI speech toolkit, and is hosted at \url{https://huggingface.co/speechbrain/spkrec-ecapa-voxceleb}.

\subsection*{PLDA backend classifier}\label{met:plda}
Probabilistic linear discriminant analysis (PLDA) classifiers are a standard approach for speaker recognition due to their ability to reliably extract speaker-specific information from an embedding space using both within- and between-speaker variance \cite{PrinceElder07, Khosravani17}. PLDA is a dimensionality reduction technique that projects data to a lower dimensional space where different classes are maximally separated (i.e., maximal between-class-covariance). The advantage of PLDA over standard LDA is that it can generalize to unseen cases \cite{Mahajan06}. PLDA can then be used to decide if two datapoints belong to the same class by projecting two datapoints to the latent space and using the distance between them as a measure of similarity. This works well for speaker recognition as speaker embeddings are typically fed into a classifier in pairs, where the classifier’s role is to optimally reject or accept the hypothesis that the two recordings are from the same speaker. PLDA typically uses the log likelihood ratio (probability of recordings belonging to the same class vs different classes) to measure similarity, commonly referred to as PLDA scores. During training of a PLDA classifier, PLDA scores for each pairwise comparison in the training set are computed and then used to set a threshold for determining potential speaker matches \cite{PrinceElder07,Khosravani17}.

Our classifier was built and trained on a set of x-vectors extracted from either VoxCeleb speech recordings or Mayo Clinic speech recordings using functions from SpeechBrain \cite{speechbrain}. We aimed to maximize performance by giving the model multiple speech embeddings per speaker during training, each extracted from recordings under different degradation conditions (varying background noise, microphone distances, etc.), that were then averaged to create a single speaker embedding \cite{Khosravani17}.

\subsection*{Threshold calculation for acceptance/rejection}\label{met:th}

During training, an optimal threshold needs to be determined to classify whether a given PLDA score represents a match, which can then be applied to new, unseen recordings. Generally, the Equal Error Rate (EER) is used to select the threshold \cite{Khosravani17, Hanifa21, Sztaho21, Borgstrom20, Lu21}. The use of EER assumes that the ‘cost’ of a false acceptance (FA) is the same as a false rejection (FR), such that the optimal threshold is one where the False Acceptance Rate (FAR) equals the False Rejection Rate (FRR) \cite{Sztaho21}. While this may be feasible for smaller datasets, when the number of comparisons is on the order of several million, the EER often generates many potential matches per speaker. As such, this can overwhelm the system early on and make it difficult for an adversary to find reliable matches. In order to scale to large numbers of comparisons, the adversary must make decisions on how to calibrate the threshold calculation, such as penalizing FAs more heavily even if some true acceptances (TAs) are missed. From an adversary’s perspective, it is less costly to miss TAs if the identified accepted cases have a high likelihood of being true. In effect, precision is more important than recall. The Detection Cost Function (DCF, Equation (\ref{eq1}) \cite{LeeuwenBrummer07}) captures this well.

\begin{figure}[h]
\setcounter{figure}{0}
\renewcommand{\figurename}{Equation}
\large
$ DCF = C_{FR} * FR * prior_{target} + C_{FA} * FA * (1-prior_{target})$ \hfill (1)
\caption{\textbf{Detection Cost Function (DCF).} We take the cost of a false rejection ($C_{FR}$) multiplied by the total number of false rejections (FR) and the prior probability of the target and add it to the cost of a false acceptance ($C_{FA}$) multiplied by the total number of false acceptances (FA) and the complement of the prior probability.}
\label{eq1}
\end{figure}

Using this function, a threshold can be found by setting optimal cost and prior terms based on the adversary’s perspective (i.e., avoid FAs more aggressively) and then finding the FA and FR values that minimize the DCF (minDCF) \cite{LeeuwenBrummer07}. For example, as the prior probability of the target is lowered (i.e., if an adversary expects a small overlap), the calculation puts more emphasis on avoiding FAs (lower FAR) as compared to the EER. Increasing the cost of FAs and decreasing the cost of FRs further avoids FAs.

We used minDCF with two parameter configurations: (1) the default configuration for the SpeechBrain implementation of minDCF, where FAs and FRs are penalized equally ($C_{FA}=1$, $C_{FR}=1$, $prior=0.01$) \cite{speechbrain} and (2) a strict configuration with a higher penalty for FAs ($C_{FA}=10$, $C_{FR}=0.1$, $prior=0.001$).

Due to the large amount of training data in VoxCeleb, it was not computationally feasible to select a threshold for the entire identified set at once. As such, we calculated minDCF multiple times on subsets of speakers and averaged across runs to generate the optimal threshold for the entire known speaker set. For each run, latent representations from two random subsets of 100 speakers were selected from the known speaker set and fed to minDCF to select a threshold. If the two subsets had no overlapping speakers, the entire run was discarded as a threshold cannot be calculated. We ran this process between 100 to 500 times depending on the overall number of speakers in the known set. Smaller known sets required fewer runs to converge on an optimal threshold.

\subsection*{Experimental Design}\label{met:exd}
To examine the risk of re-identification for clinical speech recordings in a shared dataset, we constructed experiments based on a marketer attack, where the adversary has a large set of identified speech recordings that they use to re-identify unknown speakers in a shared dataset \cite{BenitezMalin10, Dankar10}.  We used the VoxCeleb datasets to explore the effect of scope on this re-identification risk, in terms of both the size of the known speaker set used by the adversary and the size of the shared unknown speaker dataset. We then looked at the effect of elicited speech tasks on re-identifying speakers using Mayo Clinic speech recordings.

The attack scenario is represented by separating speakers in the following manner:

\begin{itemize}
    \item \textbf{Known set:} This set represents speakers with identified audio data from an online source that the adversary has access to.
    \item \textbf{Unknown only set:} This set represents speakers in a shared dataset who do not have identifiable audio online. No unknown only speakers are present in the known set.
    \item \textbf{Overlap set:} This set is a proxy for speakers in a shared dataset who do have identifiable audio somewhere online. Some speakers from the known set are randomly selected to create this set.
    \item \textbf{Unknown set:} This represents the full shared dataset, consisting of both the unknown only set and overlap set.
\end{itemize}

In our experiments, we used all available speaker recordings for the known set but selected one sample per speaker for the unknown set. For overlapping speakers, the selected sample for the unknown set was withheld from the known set. The limit of one sample per speaker in the unknown set is based on the nature of a supposed real-world dataset where all speech is unlinked and partially de-identified, meaning the adversary needs to separately find potential matches for each sample regardless of whether they come from the same speaker.

\subsubsection*{VoxCeleb Experiments}\label{met:vox}

We constructed experiments to examine the effect of the search space on re-identification. Increasing either the known or unknown set size can increase the size of the search space. As such, we consider both cases.
We first trained our PLDA speaker recognition model with different sized known sets, varying from 1000 to 7205 speakers, while keeping the unknown set static at 163 speakers. The overlap set consisted of 5 speakers with an unknown only set of 158 speakers to represent a realistic scenario for the adversary where only a few speakers have identifiable audio online. Known and unknown speakers were selected randomly from all VoxCeleb speakers, leading to variation across runs. This provides an estimate of the variability in model performance across different datasets. We generated 50 subsampled datasets for each known set size of interest (1000, 4000, 7205).

We then trained our PLDA model with a static known set size of 6000 speakers while increasing the size of the unknown set from 150 to 1000 speakers. We maintained an overlap set of 5 speakers and selected speakers randomly from all VoxCeleb speakers to generate 50 subsampled datasets for each new anonymous set size (150, 500, 1000).

We also considered a full overlap scenario where all speakers in the unknown set are present in the known set. These experiments replicated Experiment 1 and 2 with maximum overlap and no unknown only set. All other parameters remained consistent with the corresponding realistic experiment. In these experiments, only 20 subsampled datasets were generated for each interest group as the larger overlap led to less variance across runs.
For all VoxCeleb experiments, the threshold was set using the strict minDCF configuration ($C_{FA}=10$, $C_{FR}=0.1$, $prior=0.001$, see \hyperref[met:th]{"Threshold calculation"}).

There is an important implication of the VoxCeleb experimental design. Because we were interested in a range of set sizes and wanted to complete multiple runs for each size, we combined the train and validation sets from VoxCeleb 1 and 2 and randomly selected a holdout set. However, the ECAPA-TDNN model used for extracting embeddings was pretrained on VoxCeleb, meaning it was exposed to most of the recordings (i.e., all but the validation cases) during the original training step \cite{Desplanques20}. The embeddings are almost certainly superior to what one may have obtained if the embedding model was retrained for each of our splits. Unfortunately, that is not a computationally feasible experimental design. Furthermore, superior embeddings mean we are likely to overestimate risk and draw more conservative conclusions. Given the stakes – re-identification of anonymous research patients – we feel this decision was justified. We also ran a set of experiments using the VoxCeleb validation sets as our unknown set (see \hyperref[sup]{“Supplementary Information”}). This only allowed for a small unknown set with fixed speakers across runs, so it may be overly optimistic regarding risk. In our opinion, the true risk lies in between our main results and the supplementary results.

\subsubsection*{Mayo Clinic speech recordings experiments}\label{met:mayo}
We used our Mayo Clinic speech recordings to assess the effects of elicited tasks on re-identification, considering cross-task and within-task performance.

To analyze cross-task performance, we kept the known set task consistent but changed the unknown set task. The sentence repetition task was selected for speakers in the known set as it is the closest to real-world free speech. The resulting known set consisted of 500 speakers and included all sentence repetition recordings, only excluding any repetitions of a specific sentence (‘My physician wrote out a prescription’) which was saved for the unknown set. We generated an unknown set for each elicited task with 50 unknown only speakers and 5 overlap speakers who had recordings available for both the unknown set task (e.g., ‘My physician…’ sentence, alternating motion rates, etc.) and the selected known set task (i.e., other sentence repetition recordings). We randomly selected one sample for each unknown speaker.

The known and unknown set sizes were bounded by the number of speakers with sentence repetition recordings (587 speakers), as the sentence-sentence configuration needed enough speakers to create a separate known and unknown only set. We also considered the sentence-sentence configuration (i.e., sentence repetitions in both sets) as the realistic baseline.

As a secondary part of this experiment, we pooled recordings from all elicited speech tasks in the unknown set to show a scenario where the adversary can link recordings from a given speaker, i.e., there would be more speech for each unknown speaker. The known set of sentence task recordings was maintained, but for the unknown set, all remaining speech embeddings for all tasks were averaged per speaker.

Our final experiment looked specifically at within-task performance to examine the nature of a speech task on re-identification. For our within-task experiment, we maintained the set sizes from our cross-task experiments but used the same task for both the known and unknown set. In addition, some tasks had fewer than 500 unique speakers (word repetition, reading passage), so not every known set had exactly 500 speakers.

For all Mayo Clinic clinical speech recordings experiments, we generated 20 subsampled datasets per task with default minDCF parameters ($C_{FA}=1$, $C_{FR}=1$, $prior=0.1$, see \hyperref[met:th]{"Threshold calculation"}) to account for the decrease in amount of data as compared to VoxCeleb experiments. We also did this because the ECAPA-TDNN model was pretrained to extract embeddings for VoxCeleb, so we expected the embeddings for Mayo clinical speech recordings to have slightly worse performance \cite{Desplanques20}.

\section*{Statistical Analyses}
We calculated the Pearson’s correlation coefficient to determine if a linear correlation existed between the number of FAs and number of speakers/comparisons for each \hyperref[met:vox]{VoxCeleb experiment}. A t-test was performed to find the significance of each correlation.

\section*{Data Availability}
VoxCeleb 1 and 2 are publicly available at \url{https://www.robots.ox.ac.uk/~vgg/data/voxceleb/}

Our Mayo clinical speech recordings are not publicly available because of the risk related to the release of the data. Interested researchers can contact the corresponding author for case-by-case data sharing.

\section*{Code Availability}
We used python to implement our code for pre-processing, extracting speaker embeddings, generating subsampled datasets, and running the PLDA. The source code will be available at \url{https://github.com/Neurology-AI-Program/Speech_risk}. The repository will also contains detailed documentation for using the scripts.

\section*{Author Contributions}
D.A.W., B.A.M., and H.B. conceived the ideas presented in this work and validated the results. J.R.D., R.L.U., and D.T.J. provided the resources necessary for this work. D.A.W., J.L.S., and H.B. developed the methodology for the experiments. D.A.W. curated the data for the Mayo Clinic speech recordings dataset. D.A.W. and H.B. developed the code for running experiments and visualizing the results. D.A.W. ran formal statistical analysis on the data. D.A.W. and H.B. wrote the original draft of the manuscript. All authors reviewed and edited the manuscript. D.T.J. and H.B. supervised.

\section*{Competing interests/disclosures}
D.A.W.: No disclosures

\noindent B.M.: Dr. Malin receives funding from the NIH

\noindent J.R.D.: Dr. Duffy receives funding from the NIH

\noindent R.L.U.: Dr. Utianski receives funding from the NIH

\noindent J.L.S.: Dr. Stricker receives funding from the NIH

\noindent D.T.J.: Dr. Jones receives funding from the NIH

\noindent H.B.: Dr. Botha receives funding from the NIH

\newpage

Audio and Speech Processing (eess.AS); Cryptography and Security (cs.CR); Machine Learning (cs.LG); Sound (cs.SD)