The efficiency of speech communication depends on fluency of speech delivery. Speech disorders are common communication impediments defined as any condition where a person has difficulty in forming speech sounds during communication [1, 2]. Among the various speech disorder, the cause for stuttering is unclear, and theories generally highlight factors such as genetic disposition, psychological factors, biological explanation, and family issues [3, 4]. For stuttered speech recognition systems using prolonged speech signals, it is required to further enhance the quality of the speech signals even after prolongation detection and correction [5–10]. In this work waveform similarity overlap-add (WSOLA) was applied before applying classification/clustering methods for stuttered speech. WSOLA could adjust the level of the audio segments for overlap and add them in order to maintain a consistent audio signal level. Then, the reconstructed signal was applied over Mel-frequency cepstral coefficients (MFCC) feature extraction and different decision-making algorithms such as K-means, fuzzy C-means (FCM), and support vector machine (SVM).

The rest of this paper is organized as follows: Section 2 describes the methodology adopted, Section 3 discusses the validation of the proposed method, Section 4 discusses the decision-making phase, and Section 5 discusses the dataset. The results and observations are discussed in Section 6, Section 7 discusses the challenges, and Section 8 concludes the work.

1.1 Waveform Similarity Overlap-Add (WSOLA)

Algorithms for high quality time-scale modification of speech are important for further smoothing of stuttered speech signals, where the potential of controlling the apparent speaking rate is a desirable feature. The WSOLA algorithm produces high quality speech output, is algorithmically and computationally efficient, and robust. The principle of WSOLA depends on wave similarity, and it conducts operations in the time domain [11].

WSOLA was designed based on the OLA waveform editing techniques [12–15]. If τ (n) represents the desired time warping function, the basic OLA strategy comprises excising segments at time instants τ⁻¹(Lk) from the input signal x(n), shifting them to time instants L_k, and adding them together to form the output signal y(n).

However, when constructing the output signal in this manner, the individual segments are added incoherently. This introduces irregularities and distortions in the time-scaled result.

In contrast, WSOLA introduces a tolerance Δk on the desired time-warping function to ensure signal continuity at segment joins [12].

Considering propagation in the left to right direction in the waveforms in Figure 1, we assume that the segment (S1) was last excised from the input signal and added to the output in position (a). We then need to find a segment (S2), located approximately at the time instant τ⁻¹(Lk) in the input signal, which will produce a natural continuation of the output signal when added in position (b). As (S1’) would add to (S1) (a) in a natural manner to reconstruct a portion of the original input signal, we select (b) such that it resembles (S1’) as closely as possible, and is located within the prescribed tolerance interval around τ⁻¹(Lk) in the input wave. The position of this best segment (S2) is identified by maximizing a similarity measure (such as the cross-correlation or the cross-AMDF) between the sample sequence underlying (1’) and the input signal. After excising (S2) and adding it in position (b), we can proceed to the next output segment, where (S2’) now plays the same role as (S1’) in the previous step.

1.2 Literature Work

Existing works related to time-scale modification (TSM) and its variants in speech recognition and their applications are tabulated in Table 1 [5, 14, 16–19].

The stuttered speech recognition process is shown in Figure 2. Stuttered speech signals are fed through pre-processing, then through a hybrid method for prolongation detection and correction, then reconstructed using WSOLA and processed through feature extraction and decision-making for stuttered speech recognition.

Stuttered speech recognition process with intermission of prolongation detection, correction, and WSOLA proceeds as follows

Step 1: Read stuttered speech signal with stop gap Stutt redness (8 kHz)

Step 2: Pre-emphasis is performed by filtering the speech signal with first-order finite impulse response (FIR) filter

Step 3: Prolongation detection and correction using spectral parameters

• • The signal is divided into frames of 200-ms duration.

• • Compute short-term energy, zero-crossing rate (ZCR), spectral entropy and centroid parameters for each frame for the complete signal strength.

• • Autocorrelation function is applied to compute the similarity between the adjacent frames for all the parameters.

• • Every parameter is identified and fixed with the threshold values.

• • Average autocorrelation values for the parameters are computed.

• • If the autocorrelation value between the adjacent frames is higher than the threshold, frames are retained by considering normal speech segments, else it is identified as a prolonged frame.

• • Frames obtained in previous steps are identified as un-prolonged frames and used for reconstructing the speech signal.

Step 4: Further reconstruction of the signal using WSOLA

• • Input audio signal x with analysis frame x_m. The output signal y is constructed iteratively.

• • Application of Hann window function, w, to the analysis frame x_m resulting in the synthesis frame y_m.

• • The next analysis frame x_m+1 having a specified distance of H_a of 0.04-ms samples from x_m and retrieve the adjusted frame x_m+1 as the frame in y_m+1 whose waveform is most similar to x_m.

• • The similarity is calculated using the autocorrelation function.

• • Overlap-add using the specified synthesis hop size (H_s) 0.01 ms and overlap of 0.5 ms.

Step 5: Feature extraction is performed to convert speech signals to parametric representations for further analysis using MFCC [20, 21].

• • The signal is divided into frames of 20 ms

• • Multi-tapering windowing (number of tapers: 5)

• • For each frame, apply DFT to convert signals from time domain to frequency domain

• • Mel-scale filter bank analysis

• • Log to obtain smaller components of a signal

• • DCT is applied to extract spectral features

Step 6: Clustering using K-means/FCM/SVM

• • Signal-to-noise ratio (SNR): A low-level of SNR will decrease how accurately the system can recognize speech. It is the ratio of the summed squared magnitude of the signal to the summed squared magnitude of the noise [22].

  (1)SNR=μ2σ2.

  Here, μ is the signal mean or expected value and σ is the standard deviation of the noise.

• • Total harmonic distortion (THD): The THD is a measure of the harmonic distortion present in a signal, and is defined as the ratio of the sum of the powers of all harmonic components to the power of the fundamental frequency [23].

  (2)THD=V22+V32+V42+⋯V1,

  where V_n is the root mean square (RMS) value of the nth harmonic voltage and V₁ is the RMS value of the fundamental component.

We considered two clustering methods and one classifier to calculate the recognition performance of the reconstructed signal obtained from the proposed hybrid method. The MFCC features were modelled using the K-means, FCM, and SVM classifiers. These are briefly described below and parameters considered are tabulated in Table 2.

K-means: It is a method of vector quantization that aims to partition ‘N’ observations into K clusters in which each observation belongs to the cluster with the nearest mean. The K-means algorithm is used to generate a vector quantization codebook for data compression. Each of clusters is defined by its central vector or centroid. According to the Euclidian distance function, the K-means algorithm clusters the data in to K groups that assigns objects to their closest cluster [10].

FCM: This method was introduced with the idea of a fuzzification parameter that determines the degree of fuzziness in clusters. This algorithm works by assigning membership to each data point corresponding to each cluster centre on the basis of the distance between the cluster centre and data point [24]. The performance of the system depends on different inputs and environments such as centroids, the fuzziness parameter-m, termination criteria, norm of the matrix, and partition matrix-U. The different ’m’ values considered are 1.5, 1.75, and 2.

SVM: SVM is a powerful machine learning tool that attempts to obtain a good separating hyper-plane between two classes in the higher dimensional space. It is a predominant technique to estimate the basic parameters of speech. Speech samples can be approximated as a linear combination of samples by minimizing the sum of squared difference between the actual speech samples and predicted values. Thus, a unique set of parameters or predictor coefficients can be determined. These coefficients form the basis for the linear prediction of speech samples. The importance of this method lies in its stability to provide extremely accurate estimates of speech parameters from data samples, and may have applications in understanding nonlinear phase characteristics of stuttered speech [25–28].

The experiments were conducted on the University College London Archive of Stuttered Speech (UCLASS) dataset. This repository comprises monologues, readings, and conversational recordings. For our simulation, 80 samples at word level were derived from the sentence recordings of the repository. It includes 22 words from female speakers and 58 words from male speakers with age ranging from 11 years to 20 years. The samples are selected to cover different ages and genders [27]. Through listening perception, vocalized, and nonvocalized prolongations are identified and derived manually. The dataset considered for analysis is shown in Table 3.

WSOLA was applied over a speech signal uttered as ‘mmmm/ooo/thers d/aaaaaaa/y’ (actual word mother’s day) and reconstructed using different scaling factors from 0.1 to 2.0. As per the observation shown in Figure 3, as we decrease the scaling factor, it increases the speech rate. Increasing the scaling factor will speed down the speech rate and analysis of waveforms, as shown in Figure 4. As per the observations shown in Table 4, by varying the window time, the overlap function serves as the reference for natural signal continuity and leads to better performance with reduced SNR and THD.

The WSOLA reconstruction efficiency was analysed for artifacts in three categories of speech: vowel-like, consonant like, and plosive sound, and the results are shown in Table 5. As per the observation for different age groups and sexes for vowel and consonant sounds, WSOLA results in better performance compared to plosive sound, because it increases the sharpness of harmonics and eliminates the discontinuous of the signal by tapering the beginning and end of the frame to zero. It also reduces the spectral distortion due to the overlap.

After further smoothening of the speech signal, it passes through the MFCC and decision-making algorithm using K-means, FCM, and SVM to analyse the effect of WSOLA on the performance of stuttered speech recognition. Improved recognition rate compared to that without WSOLA was observed. A higher recognition rate of 95% is observed using SVM, as shown in Table 6 and Figure 5.

Selecting the proper scaling rate for prolonged highly polyphonic speech signals is a challenge. Different types of speech signals require WSOLA to be processes with different scaling factors for better performance. Values fixed with the parameters considered for reconstruction also vary depending on the length and type of artifacts present in the speech signal.

This work proposes a method for smoothening speech signals after prolongation detection and correction using WSOLA. Results demonstrated 95% recognition rate using SVM applied after WSOLA signal smoothening. Additionally, this work can be extended to improve WSOLA utilization by identifying proper methods for selecting better scaling rates that apply to all prolongations with mixed phonemes.

Table 1. Existing work on TSM and its variants in speech recognition and their application.

Table 2. Parameter setting for decision-making and its variants.

Table 3. Dataset considered for experiment.

Table 4. Analysis of WSOLA for the signal ‘mmmm/ooo/thers d/aaaaaaa/y’ with varying window times and overlap ratios.

Table 5. Analysis of WSOLA for different types of speech.

Table 6. Effect of WSOLA on decision-making.