Driver distraction can be defined as a form of inattention by the driver toward a specific activity or event and may result in insufficient attention for safe driving. This preoccupation can occur either inside or outside the vehicle and may result in the delayed recognition of information required for safe driving. Driver distraction can be classified into four main categories: visual, auditory, biomechanical, and cognitive. Biomechanical distraction occurs when a driver performs a physical activity apart from driving, such as adjusting the radio volume, which requires the driver’s visual attention and the use of one of their hands to adjust the volume manually. The application of electromyography (EMG) signals in car-electronic equipment is intended for minimizing biomechanical distraction using certain finger movements to command the in-car electronic equipment without requiring drivers’ visual attention or their hands off the wheel.

Every nation recognizes fatality cases caused by road accidents as a global phenomenon [1]. On June 20th, a Bloomberg report cited the 2013 World Health Organization statistics, claiming that among the emerging countries, Malaysia has the most dangerous highways after Thailand and South Africa. Based on these statistics, 7,000 to 8,000 people among the estimated 30 million Malaysian population die each year on the road.

Research conducted by Addis and Tian [2] in 2019 showed that the main cause of car accidents is negligence during the operation of in-vehicle technologies. This form of negligence, which is caused by distraction to drivers and is primarily in the form of biomechanical distraction, can be solved using an operating system with easier controls. Finger movements can be used as an easier method to direct the required controls. This can be achieved using the EMG signal obtained from finger movements and applying it in real time.

Car crashes are primarily caused by distractions to drivers. The lack of attention on the road increases the risk of accidents to drivers and other road users. Hence, the EMG pattern recognition system was implemented for controlling in-car electronic equipment to resolve drivers’ inattention due to biomechanical distraction. It allows the driver to focus on the road while having their hands on the wheel.

A few factors must be considered when implementing an EMG signal to control in-car electronic equipment. Because EMG signals are nonstationary, a specific technique is required to extract the features of the EMG signal in terms of both time and frequency. In addition, the classifier used must demonstrate high accuracy and performance such that finger movements can be classified explicitly to control in-car electronic equipment.

Hence, this research was conducted to extract and analyze EMG signal features using a time–frequency distribution (TFD), as well as to determine the best classifier to accurately classify finger movements that can be used to control in-car electronic equipment.

Driving can be defined as an act of controlling a vehicle. It is an operation in which human beings are being transported from one location to another via certain mechanism. It is a complex task that requires different cognitive, physical, sensory, and psychomotor skills to be executed simultaneously [3]. Even though driving a vehicle can ease traveling, failure to drive appropriately or mistakes occurring while driving can cause accidents on the road. Driver distraction, which is a result of inadequate attention on the road, is a factor that causes car crashes. A driver’s inattention can be described as a lack of attention to safe driving when the driver is focusing on tasks other than driving. It delays the recognition of information as data for safe driving are interrupted. Driver distraction can be generally classified into four main categories: visual, auditory, biomechanical, and cognitive [4].

2.1 Biomechanical Distraction

A driver that is distracted from the principal driving task can threaten the safety of others as well as the driver himself. Biomechanical distraction occurs when a driver withdraws one or both hands from the steering wheel to grasp an object physically. A typical example of biomechanical distraction is when a driver attempts to adjust the volume of the radio. Another example is when a driver attempts to reply to a text message using a mobile phone while driving. According to statistics, one among six drivers will regularly send a text message while driving [5]. This type of activity distracts the driver from driving safely. Currently, drivers are more likely to encounter accidents owing to their increasing dependency on mobile networks and messaging applications [6].

2.2 EMG

EMG is a science associated with the sensing, interpretation, and utilization of an electrical signal emanating from muscle contraction [7]. Additionally, it is described as an electrodiagnostic medicine technique used to evaluate and record skeletal-muscle-generated electrical activity [8]. EMG is a technique that comprises several procedures for extracting electrical signals yielded by muscle contraction. The electrical signal produced is known as the EMG signal. The recording of an EMG signal includes three phases, i.e., the input, processor, and output phases [1]. The input phase involves the application of the electrodes.

By contrast, the processing phase occurs when the electrodes convert the bioelectric signal from the muscle into an electric potential; as such, they must be processed using an amplifier. The output phase occurs when the amplified signal is displayed using an analog or a digital system. The raw signal data extracted in the input phase from the human body using electrodes must undergo a few procedures before the signal can be applied in real time. The output of the EMG signal can be used in various applications, such as medical research, ergonomics, and biomedical engineering, to generate a device control command for rehabilitation equipment. Although EMG signals benefit humans in various types of applications, the different types of noise that occur because of the use of electronic equipment and a physiological factor result in undesirable effects that must be mitigated to achieve improved performance.

In EMG signals, noise is the main issue that must be overcome, as the distinguishing characteristics of the signal are often lost owing to the mixing of various types of noise. Various noise signals or artifacts can originate from different sources, such as the inside of the human body, the environment during sensing or data interpretation, or internal or external measurement devices. The characteristics of EMG signals depend on the participant’s internal structure, including the human skin shape, blood flow speed, measured skin temperatures, tissue composition, and measurement location [9]. Noise can be present in each of the above and may alter the results of feature extraction and signal diagnosis. As various forms of noise can contaminate the EMG signal, the latter becomes extremely difficult to analyze and interpret [1]. Hence, the signal-to-noise ratio, which indicates the ratio of the desired signal to the undesired signal, must be prioritized during data acquisition and interpretation, as noise does not constitute the desired EMG signal.

2.3 EMG Feature Extraction

Feature extraction reduces the original set of raw data into more manageable categories for processing. This is an important step in pattern recognition studies as it decreases the number of resources without causing the loss of important data. For feature extraction, the EMG signal can be classified into three technique categories: time-domain (TD), frequency-domain (FD), and time-frequency domain (TFD) techniques [10]. Each technique offers a different feature type. TD features are simple and efficient for EMG pattern recognition, whereas the frequency distribution estimates the EMG power spectrum in term of frequency [10]. Meanwhile, the time-frequency distribution provides information that is not available from the TD features or FD features [11].

Table 1 shows the advantages and disadvantages of feature extraction techniques for feature reduction and selection in EMG signal classification, as reported by researchers at the Prince of Songkla University, Thailand [9].

Even though the TD features exhibit lower computational complexity and are straightforward, the initial EMG signal data can be affected by interference, as this feature is calculated based on the signal’s amplitude [12]. The frequency distribution features may not be reliable for recognizing the pattern of finger movements in in-car electronic equipment because it requires stationary signals.

Time-frequency analysis in feature extraction involves techniques that simultaneously analyze a signal in the time and frequency domains using various time-frequency representations. As the EMG signal is nonstationary, the time-frequency technique can be used to analyze the signal [13]. Owing to the complex characteristics of EMG signals, the time-frequency distribution has been frequently used because it can provide temporal and spectral data [14].

The TD is associated with the variation in the amplitude of a signal with time. In addition, it pertains to the investigation of mathematical processes, physical signals, or time series of economic or environmental data in terms of time. In the TD, the signal or function value uses all real numbers for the case of continuous time, whereas they are used at various separate instants in the case of discrete time. The TD provides a record of the response of a dynamic system based on measured parameters as a function of time. In fact, the TD is the conventional domain used to observe the output of a dynamic system.

The TD is used for EMG signal feature extraction owing to its simplicity in implementation and computation [15]. It is typically used to detect muscle contraction, activity, and onset. It does not require any transformation to calculate the features based on raw EMG time series. The TD method can be used in low-noise environments and has a lower computational complexity compared with other methods.

However, TD analysis is efficient only when the signal is stationary. By contrast, an EMG signal comprises both stationary and nonstationary signals, depending on its application. Because its statistical properties change over time, the EMG signal is nonstationary, which render its application difficult in TD analysis. To obtain the features of TD analysis with regard to the EMG signal, calculation must be performed using the amplitude from the TD analysis. As the EMG signal is nonstationary, the recordings contain interference, causing the amplitude to change consistently, which renders the analysis of the recording difficult.

The FD method refers to the analysis of mathematical functions or signals based on frequency instead on time. The analysis of the FD shows the amount of signal over a spectrum of frequencies that is present within each frequency band. An FD representation can provide information regarding the phase shift that must be applied to each sinusoid. The frequency components can then be reconverted to restore the original time signal. In addition, a waveform identical to the desired signal can be generated by selecting the amplitudes, frequencies, and phases of the sinusoid waves.

In analyzing the real-time application of EMG frequency, the signals must first be converted into the FD such that the spectrum can be analyzed. This is because after the FD analysis, the EMG signal changes from the original time-varying amplitude to a frequency-varying power [16]. Even though feature extraction in the FD can directly show the distribution and variation of the signal frequency, it causes the loss of temporal information in the data.

In signal processing, time–frequency analysis includes techniques that simultaneously analyze a signal in the time and frequency domains using different time-frequency representations, i.e., the analysis is performed when the signal frequency characteristics vary with time. In most applications, the signal is primarily nonstationary, and the spectrum of the application changes over time. The time-varying signal in the time interval is sufficient short at the signal, rendering it stationary and divisible. When the short periods of a signal are segmented in the spectrum and estimated over the sliding windows, the time-frequency analysis of the signal can be performed. A TFD graph is shown in Figure 1.

The main properties in a time–frequency analysis include high resolutions in terms of both time and frequency, no cross-terms for eliminating bias components from undesired artifacts and noises, and a lower computational complexity for guaranteeing the time required to represent and process a signal on a time–frequency plane to enable real-time implementations.

EMG signal features are to be utilized as the input to the classifier for classifying movements. In EMG signal analysis, the instantaneous root mean square (RMS) voltage is used as the standard metric. The instantaneous RMS voltage is a function that measures the RMS voltage value at a specific time.

2.4 EMG Signal Classification

To map, identify, and estimate the different patterns of information acquired from the EMG signal, the information must be input to a classifier. This classifier is used to recognize different patterns of the extracted EMG signal features. The classifier provides observation and supervised learning data by supplying a training set of correctly measured data to the classifier. However, in the pattern-recognition-based EMG control method, performance can be degraded in the long term by various interfering factors [11]. Table 2 shows the advantages and disadvantages of a few classifiers, based on reports by researchers at Universiti Kebangsaan Malaysia, Malaysia, and Chung-Ang University, Korea [10].

The support vector machine (SVM) classifier is widely used to classify patterns recognized from EMG signals. This is because the disadvantage of this classifier can be easily overcome, unlike other classifiers. By filtering the raw EMG signal data using a bandpass filter, unnecessary noise can be discarded on the upper and lower cut-off frequencies. Additionally, a study conducted in 2018 revealed that using the noise remover feature on the extracted signal, the SVM classifier’s efficiency was higher than that of other classifiers based on machine learning [1]. In another study conducted in 2019, the SVM was preferred over other methods owing to its high calibration and classification speed, as well as the algorithm’s ability to perform well even when a small training set is used [16].

The current investigation is categorized into three stages: signal preprocessing, signal processing, and signal classification. In the signal preprocessing stage, a set of raw EMG signal data was used. These signals were filtered to discard contaminated noise before proceeding to the signal-processing stage. The EMG signal parameters and features were obtained using the time-frequency distribution in the signal processing stage. The results were derived in terms of both time and frequency. In the next stage, signal classification was performed to classify finger movements. Classifiers were compared to determine the classifier’s best accuracy and performance for the EMG signal parameters.

3.1 EMG Data Acquisition

Throughout the experiments, a Bagnoli desktop EMG device ((Delsys Inc., Natick, MA, USA) was used to acquire the EMG signals. Delsys EMG electrodes were used to record EMG datasets (DE 2.x series EMG sensors). The extensor and flexor muscles were the primary muscle groups addressed in this study, as they are the muscles that control grasping and finger movements. The extensor muscles located on the posterior side of the forearm stretch the fingers outward during the first stages of finger movement, whereas the flexor muscles flex the finger joints. In this regard, the extensor carpi ulnaris, extensor digitorum, extensor carpi radialis, and extensor digitorum mini are targeted. Meanwhile, the flexor carpi ulnaris, palmaris longus, and flexor carpi radialis were targeted in this study for specific flexor muscles. Owing to the configuration of the muscles in the human forearm, additional EMG electrodes were used to form the circumference of the forearm to obtain activity from the underlying muscles feeding the thumb and other fingers, including the flexor pollicis longus, extensor pollicis longus, and extensor pollicis brevis.

Eight EMG electrodes were placed over the circumference of the forearm. Each sensor was fitted with a two-slot adhesive skin interface to provide secure attachment to the skin. During the study, the wrist of each participant was fitted with a conductive adhesive reference electrode. The EMG signals obtained were amplified to a gain of 1,000 using a Delsys Bagnoli-8 amplifier. The signal was sampled at 4,000 Hz using a 12-bit analog-to-digital converter (BNC-2020 data acquisition board; National Instruments, Austin, TX, USA). Subsequently, the signal data were obtained using MATLAB.

For data acquisition, seven volunteers between the ages of 20 and 35 were recruited. All recruited participants were normally limbed and did not have any neurological or muscle problems. Each participant was required to perform 14 finger movements (12 classes of finger pressure and two classes of finger pointing). Table 3 lists the 14 movements involved in controlling an in-car electronic system.

These classes are abbreviated based on movement names. The aforementioned classes are the raw signals obtained from eight different electrodes. The movements investigated are illustrated in Figure 2.

Throughout the data acquisition period, both hands remained on the steering wheel to simulate a real-world driving scenario, in which participants were required to apply pressure to the individual or combined fingers instead of perform actual gestures, except for two-finger pointing classes conducted with both hands on the wheel. The participants were instructed to perform each of the 14 movements mentioned earlier and sustain it for 5 seconds in each session. Each movement was subjected to six trials.

3.2 Signal Preprocessing

EMG signal preprocessing is performed to improve the accuracy and response time of the data controller. In signal preprocessing, a filtering phase is required to discard undesired noise in the upper and lower cut-off frequencies. When the raw EMG signal is recorded, it is contaminated by various types of noise. The preprocessing step was implemented to minimize the effect of noise, which can affect data interpretation [17].

Information regarding the movement of the EMG signal data was loaded into MATLAB. This signal was a combination of eight different signals extracted from eight different electrodes. Before proceeding to the filtering phase, a combination of eight signals in a single movement was separated by each electrode. Hence, eight different signals were available for each movement. Each signal was filtered using a 20–450 Hz bandpass filter at a sampling rate of 4,000 Hz. All noises above 450 Hz and lower than 20 Hz in the signal were discarded. In 2018, Napolitano and Dogancay [18] reported that when digital estimates are performed, the continuous-time signal should be sampled for at least twice the maximum cutoff frequency to prevent both period and spectral frequency aliasing. Hence, a sampling rate of 4,000 Hz was selected as the sampling rate for these signals to interpret the data correctly. This process was repeated for each movement and participant.

3.3 Time-Frequency Distribution

The technique used in signal processing for analyzing EMG signals that can be implemented for controlling in-car electronic equipment is the TFD method. The functionality of the TFD can be described as varying frequency data at different time points and supplying the analyzed signals with abundant nonstationary information [4].

In this study, a TFD spectrogram was used for feature extraction. The spectrogram is a fundamental component of the TFD in signal analysis, particularly for noise and artifact reduction. The spectrogram is used to overcome the limitations of time and frequency representations for nonstationary EMG signals. It is defined as the squared magnitude of the short-time Fourier transform (STFT), as expressed in Eq. (1).

where S(t, f) is the time-frequency representation, x(τ) the EMG signal, and w(t) the observation window.

The TFD method is preferred to obtain time and frequency information simultaneously. The spectrogram reveals the nonstationary existence of EMG signals in the time-frequency analysis. In the TFD, the time and frequency resolutions can be adjusted to obtain valuable signal details.

Subsequently, the parameter of the EMG signal was estimated from the resulting time-frequency representation of the spectrogram. The RMS voltage (V_rms) was measured instantaneously over time, and the average values were obtained for hand movement prediction. The average RMS voltage can be expressed as follows:

where

Here, V_rms(t) is the instantaneous RMS voltage, S_x(t, f) the time-frequency representation, and f_max the maximum frequency of interest.

3.4 Machine Learning

After extracting the features from the signal, classification was implemented to differentiate between the feature vector and various categories. Each type of classifier offers different results in terms of accuracy and performance. The accuracy of the extracted features was compared with those of four different classifiers: the SVM, decision tree, linear discriminant, and K-nearest neighbor (KNN) classifiers.

The EMG signal features for all the participants were classified into two categories: training and testing. The training data comprised 80% of the total RMS voltage, and the remaining 20% was used as the testing data. The classification learning workflow is illustrated in Figure 3.

To examine the accuracy and performance of the classifier, the training data were uploaded as input to the classification learner toolbox in MATLAB. The RMS voltage is the predictor, and 14 movements were input to the classifier.

The EMG signal data comprises 10 classes of individual and combined finger movements. Every participant completed six tests for 10-finger movements, and the resting time between the tests was 3–5 seconds. Among the six tests, four and two tests were performed for training and testing, respectively. Raw data were obtained from 10 individual movements, five individual movement tests, and five combined movement tests using two electrodes. This section presents the results and discussions of all the methods used.

A single-movement signal is a combination of eight different electrode signals. This signal is separated before proceeding to the filtering phase. Figure 4 shows the EMG signal of the index class for Participant 1.

The acquired signals were recorded in terms of time and voltage. The raw EMG signal data were contaminated with undesired noise and must be eliminated before proceeding to the next stage. Using a 20–450 Hz bandpass filter, all frequencies above and below the bandpass cutoff frequency were discarded. The raw EMG signal was set at a sampling rate of 4 kHz in the filtering phase to avoid aliasing between the signals. The filtered signal for the I-I movement of Participant 1 is shown in Figure 5.

During signal processing, a feature was extracted from the filtered signal. The extracted feature is the parameter of the EMG signal, which is the RMS voltage. Using the TFD technique, the RMS voltage can be measured in terms of both time and frequency. Figure 6 shows the RMS voltage extracted from electrode 7 for the I-I movement from Participant 1.

Because the EMG signal is nonstationary, to apply the EMG signal to control in-car electronic equipment, the parameters of the EMG signal should be measured in terms of both time and frequency. This is because the EMG signal is recorded from the movement of body muscles. The body muscles of a driver changes based on the movement of his body. Therefore, the emitted signal obtained from the body muscles varies with frequency and time simultaneously. For example, if pressure is applied to a finger, then the acquired RMS voltage will exceed the typical RMS voltage.

After obtaining the RMS voltage for each movement from all the participants, an analysis based on the average RMS voltage was performed. Figure 7 shows the average RMS voltage acquired from all participants. The figure shows a comparison between the average RMS voltage for each electrode and the movement involved. Each movement involved eight different electrodes, and each electrode resulted in a different average RMS voltage. The highest average RMS voltage on a single electrode reflected the movement involved. The lowest value contributed the least for determining the movement. The highest average RMS voltages for each movement are presented in Table 4.

Electrode 1 exhibited the highest average RMS voltage. This was proven when six among eight movements involving Electrode 1 yielded the highest average RMS value. This was followed by Electrode 4, in which three of the movements were recorded as positive results. Subsequently, Electrodes 2 and 3, which involved two movements, recorded the next highest values. The highest average RMS voltage for Electrode 5 was recorded only for the M-M movement. Electrodes 6, 7, and 8 were not included in Table 4 because they did not provide significant values for determining the movements.

In the classification stage, four types of classifiers were used to classify the 14 movements. These classifiers included an SVM, a decision tree, a linear discriminant, and a KNN. The total feature extraction data acquired from the signal processing stage was 784 RMS voltage data points, of which 80% was used as input to the classifier as training data. The accuracy of the training data was compared among the classifiers, and the performance of the classifier with the highest accuracy was reviewed.

Figure 8 shows the results of the scatter plot diagram between Electrodes 1 and 2 in the classification learners using the MATLAB software. The accuracy of the EMG classification stage determines the percentage of RMS voltage that has been extracted. The scatter plot diagram shows the total training data for Electrodes 1 and 2. The relationship between the electrode data can be described as correct or incorrect. This relationship determines the accuracy of the training data; the more accurate the model predictions, the higher is the accuracy of the classifier in classifying the feature extraction. The accuracy of the training data trained based on each classifier is presented in Table 5.

The highest accuracy used to classify the 14 movements on the training data was indicated by the SVM (89.3%), followed by the decision tree (57.1%), linear discriminant (34.5%), and KNN (27.4%). Hence, it is proven that the accuracy of the SVM classifier is higher than those of the other classifiers. The performance of the SVM classifier is shown in the confusion matrix in Figure 9.

The confusion matrix above shows the performance between the true and predicted classes of the SVM classifier. The true class on the y-axis represents the actual input data supplied to the classifier, and the predicted class on the x-axis represents the predicted class of the SVM classifier. The performance of the class is described as either a true positive rate (indicated in green) or a false negative rate (indicated in red). The higher the true positive rate, the better is the performance of the classifier.

As an example of the I-I movement, the accuracy between the true and predicted classes shows a 100% true positive rate; therefore, the performance between the true and predicted classes is significant. For the I-M movement, the accuracy between the true and predicted classes shows an 83% true positive rate and a 17% false negative rate for the M-M movement. This implies that the performance of the I-M movement is not significant for the M-M movement of the predicted class.

The highest true positive rate exhibited by the SVM classifier in classifying the 14 movements was 100%, and the lowest true positive rate was 67%. In general, using the SVM classifier, the RMS voltage data exhibited high accuracy and performance, as the false negative rate for each true class did not exceed 50%.

In addition to accuracy, the temporal factor is an important indicator in vehicle applications. The computing time increases with the number of features. In this study, which involves eight channels, the total number of extracted features was eight (1 feature/channel × 8 channels = 8 features), which is relatively small; hence, the computing time is low. This was proven by calculating the computing time required to perform each classification technique using MATLAB on a PC equipped with a 3 GHz CPU and 8 GB of RAM. The results show that all the tested classifiers required a computing time of less than 0.05 seconds.

Previous studies indicated that although technology is evolving, humans remain cavalier and are easily distracted, particularly when attempting to multitask, e.g., driving while simultaneously adjusting the radio’s volume. Biomedical distraction is a primary form of distraction. The driver must withdraw his/her hands off the steering wheel physically to manually control the devices in the car. This form of distraction can be easily solved using a better operating system that offers easier control. In this regard, an electrodiagnostic medicine technique, otherwise known as EMG, can be used to achieve the goal using data obtained from muscle contraction. Although EMG has been proven to be a better alternative for achieving a better operating system, a few aspects must be considered, such as the feature extraction, which must be measured in terms of both time and frequency to be used in real-time applications. In addition, the most suitable classifier must be selected to determine the accuracy when classifying finger movements.

To measure and analyze EMG signals more effectively, noise must be eliminated. In this regard, a bandpass filter can be used to filter the noise by discarding all noise above the high cut-off frequency and below the low cut-off frequency. Using the TFD technique in the signal processing stage, EMG signal features can be analyzed in terms of both time and frequency. The results of this study showed the SVM classifier is the best classifier that can be used to classify as many as 14 types of finger movements for the EMG pattern recognition system used to control the in-car electronic system. This is because the SVM offers a higher accuracy level for classifying the feature of the EMG signal as compared with other classifiers.

Although the eight-channel EMG system is effective, the number of EMG channels utilized should be minimized without sacrificing classification performance. This will improve the system’s viability for real-time implementation while reducing the computing cost and signal noise. The correlation coefficients between each pair of channels must be computed to evaluate the possibility of eliminating a portion of the total channels utilized—this will be attempted in future studies.

Table 1. Advantages and disadvantages of various feature extraction techniques.

Table 2. Advantages and disadvantages of various classifiers.

Table 3. Classes for 14 finger movements.

Table 4. Highest average RMS voltage for each movement.

Table 5. Comparison of classification performance.