This paper is an extension of the work originally presented in the 2020 12th International Conference on Information Technology and Electrical Engineering (ICITEE) [1]. Since the inception of modern advertising, companies have been searching for methods to quantify consumers’ perceptions of their products. Often, companies rely solely on classical methods such as questionnaires and surveys. However, these methods typically lack concise insights into what the consumer truly feels because there is no way to ensure the correctness of a subject’s answers [2–4].

1.1 Neuromarketing

The aforementioned problem has given rise to a field of research known as neuromarketing, which blends two different concepts: neuroscience and marketing [5, 6]. Neuromarketing is a multifaceted field, and an important aspect of neuromarketing involves devices that are used to collect data from the brain of a subject. Neuromarketing techniques work by recording the brain activity of a subject when exposed to a certain stimulus. This is achieved by using reliable scientific data within the field of neuroscience, which are needed to attain a better understanding of how to alter a product to meet the demands of a specific client or how to advertise a product to reach the desired audience [5, 6]. These data are considered more reliable than the subject’s account because the brain’s response to seeing a product is rather instantaneous and accurate, which cannot be ensured by completing a simple questionnaire alone.

One of the first neuromarketing studies, published in 2004, asked participants to drink either Pepsi or Coca-Cola while being scanned with a functional magnetic resonance imaging (fMRI) machine [7]. Along with many other conclusions, this study deduced that different parts of the human brain became active when subjects knew the brand they were consuming. When subjects were informed that they were drinking Coca-Cola, they said they preferred Coca-Cola to Pepsi, and their frontal cortex was more active [8]. However, when they were not informed about the brand, the subjects reported that they preferred Pepsi, and a small area in the brain responsible for emotional and instinctual behavior became active [8]. The conventional method of questioning the subjects was not completely discarded; instead, fMRI was used to supplement the research and provide clear results. In the research, the subject’s account of their preferred brand helped the researchers understand where the decision came from, an emotional response or a planned response. This research was vital at the time to understand that brand preference can be manipulated and that advertising techniques can have a major influence on viewers and their brain than previously assumed.

Some of the collection methods used to acquire data in neuromarketing research include fMRI, electroencephalography (EEG), steady state topography (SST), eye tracking methods, voice analysis, galvanic skin response, heat rate monitoring, and respiratory monitoring [2]. The results from these methods can be used to understand the initial response of a subject to a product. This can be used subsequently to estimate their probability of responding positively or negatively to the product based on a quick reading, which can verify the response or even estimate the response of the consumer. In the long run, this process can save the time and money of companies by providing a quick and effective trial for their products because quantitative measuring techniques can be employed to supplement classic marketing techniques.

Neuromarketing aims to overcome the ambiguities resulting from conventional marketing methods. For example, conventional methods can lead participants into making choices that they would not make under normal circumstances. To attract more customers, major companies such as Delta, Coca-Cola, Google, ESPN, McDonalds, and Yahoo have developed advertising techniques through neuromarketing research [2].

1.2 Regions of the Brain

The distinct regions of the brain have been studied in depth. Khushaba et al. [6] observed significant changes in EEG power spectral activities in the frontal, temporal, and occipital regions when subjects chose their preferred item, Cherubino et al. [9] noted that decision-making processes, such as choosing the best product for each consumer based on cost and reward, occurred in the prefrontal cortex. The prefrontal cortex was also found to be responsible for connecting neurons in the occipital lobe to focus on and process visual stimuli [10]. Because aesthetics are a major component of advertising, they may be an indicator of visual alertness toward a brand, advertisement, or product. However, this does not mean that an increased activity in the occipital lobe directly indicates that subjects like the object they are looking at. In addition, considerable activity in the left frontal region of the brain may be associated with a positive emotional experience [9].

1.3 Data Processing

To interpret the EEG results, key features must be extracted from the signal and analyzed further for the desired application. Previous studies have used a multitude of features extracted from EEG data.

For example, a study [11] published in 2014 discussed brand preferences of automotive companies in Malaysia using an EEG machine. This study used the 14-channel Emotiv system, which is the same device used in other neuromarketing research [12]. The signal was processed using a Butterworth filter (0.5–60 Hz cut-off frequency). The alpha wave (8–13 Hz) was then extracted through fast Fourier transform to investigate three statistical features: power spectral density (PSD), spectral energy (SE), and spectral centroid (SC). These features were chosen to develop a feature vector that would help create an algorithm for pattern recognition. PSD was used because it provides insight into the frequency content of the signal. SC was important for determining the dominant spectral energy from the power spectrum. Therefore, it can be concluded that extracting power and energy from the EEG signal can provide adequate parameters for developing a pattern that can be subsequently recognized.

This conclusion is supported by another study published in 2015 by Bhardwaj et al. [13], who used PSD and energy to develop a machine learning algorithm. This algorithm could predict future responses to a product and, in theory, create a computer that can understand human emotions. To extract the PSD and energy, theta, beta, and alpha waves (4–30 Hz) were used instead of only alpha, which was used in the prior experiment. This created a matrix of all the relevant data free of redundant information. It can be concluded that power and energy are important factors for analyzing EEG data and predicting future responses using artificial intelligence in accordance with neuromarketing and brand preference.

These two examples were utilized in this study to compare the results from different features to obtain a better understanding of which feature yields the most accurate results. In addition, using various channels and band powers, several previous studies have used support vector machine (SVM) classifiers with accurate results [9, 14–16]. Specifically, Kirk et al. [15] used the 14-channel Emotiv EEG system and divided the signal into four band powers and obtained a 75.44% accuracy when classifying the image as whether preferred or unnoticed.

1.4 Classifiers

Different classification algorithms can be used to obtain the most accurate results from EEG signals acquired through preprocessing techniques. Thus, the optimization of the most accurate classification algorithm can be used to yield fairer results.

The k-nearest neighbor (KNN) classifier is based on the closest training examples in the feature space. It works by classifying based on the majority vote of its neighbors, that is, a preference is classified based on how similar it is to its k nearest neighbors [17].

The pseudo-quadratic discriminant analysis (pq-DA) classifier uses a quadratic decision surface to separate the dataset, where, unlike the linear discriminant analysis, it is not assumed that the covariance of the classes is identical. Therefore, it estimates one covariance matrix for each class. pq-DA uses the pseudo-inverse of quadratic covariance matrices when the covariance matrix is singular.

The naïve Bayes (NB) is based on the Bayes theorem, where classification is performed based on probability. NB works by calculating the membership probability of the sample to all classes in the dataset [18].

The SVM classifier was designed for binary classification and can be used to perform nonlinear classification. It relies on choosing the box constraint and kernel parameter or scaling factor; these two parameters together are known as the hyperplane parameters [17].

The random forest (RF) classifier comprises individual classification trees; each tree is constructed by selecting a random subset of input features and a different sample from the training data. The RF classifier classifies new samples based on the results of all classification trees. The trees then vote for the input data, and the forest selects the class with the most input data votes [19].

In the decision tree (DT) classifier, each attribute of the data is evaluated by dividing the data into smaller subsets and choosing the attribute that provides the highest information gain. The dataset is then split until the results become undesirable [19].

1.5 Objective/Scope

This study aims to investigate a set of 21 time- and frequency-domain features among six different classifiers (KNN, DA, NB, SVM, RF, and DT) to identify the best features, classifiers, and brain regions associated with decision-making and emotional thought. We believe that time-domain features are less investigated in the literature compared with frequency-domain features in EEG signal analysis, and such a comparison may highlight the important features for other studies, given the low sampling rate in the data under investigation. Furthermore, the comparison between different classifiers enables the investigation of whether some features are better than others under the used classifiers or are better than the rest, regardless of the classification algorithm, and indicates that the features are less classifier-dependent. This may help future investigations and researchers in selecting their features and classifiers for similar scenarios.

Furthermore, this study aims to determine the best brain regions associated with decision-making and emotional thought. This can help reduce the size of the data acquisition system and limit it to the region of interest, which means less cost and fewer computational needs.

The dataset used in this study was preprocessed using both independent component analysis (ICA) and principal component analysis (PCA) to enhance the classifier performance. This study differs from previous research in that it emphasizes EEG data processing, feature extraction, and optimization. Therefore, different optimization techniques and classifications of EEG channels can be compared.

2.1 Dataset

An EEG monitoring method was developed to record the brain’s signals over the scalp, where voltages vary as a result of neuronal activity in the brain [4]. Specifically, using an EEG, a clearer picture can be painted regarding the subject’s preference for a product because the areas of the brain can be linked to varying emotions and decision-making [5]. Because EEG is non-invasive, economic, and portable, it is suitable for examining a subject’s preference for certain products.

The EEG data for this research were obtained from an EEG-based neuromarketing study by Yadava et al. [16]. These data were posted online and made public for others to use and were ultimately chosen for this research because the methodology and device used in it achieved the goal of investigating neuromarketing through simplified systems.

The dataset was collected using the 14-channel (AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, and AF4) Emotiv EPOC+ system placed as in Figure 1 and sampled at 2, 048 Hz and down-sampled to 128 Hz. EEG is available with a varying number of electrodes and sampling rates, and increasing them may increase the accuracy of the results [4, 14]; however, this could increase the price of the system and the noise from neighboring channels. Therefore, it is paramount to determine a good trade-off between the price and accuracy of the system used. The 14-channel Emotiv system is an adequate and easy-to-use choice for neuromarketing applications.

The study sample consisted of 25 participants aged 18–38 years. The Emotiv EEG headset was placed on the heads of the participants, and the collected data were sent to a computer via Bluetooth connectivity. The participants were shown 42 images of shopping items, as shown in Figure 2, on a computer screen for 4 seconds. Thereafter, they were asked to classify the image into two categories, “like” and “dislike,” based on their preference [16].

2.2 Preprocessing

The preprocessing procedure involved five steps (Figure 3): subtracting the signal from the mean, bandpass filtering, ICA, PCA, and normalization.

First, the mean was subtracted from the original EEG signal to correct baseline signals. Thereafter, a fourth-order bandpass Butterworth filter was used. This was done with pass frequencies between 0.1 and 30 Hz; values higher than 30 Hz were filtered because this study considered frequencies only up to the beta waves. The Butterworth filter was selected because it is a maximally flat magnitude filter at the bandpass frequency borders.

ICA was used to remove artifacts from electrocardiogram, electromyogram, and electrooculogram signals and leave the most meaningful part of the EEG signal. ICA finds a linear representation of non-Gaussian data. This results in the components becoming statistically independent, which makes feature extraction efficacious because the ICA captures the essential structure of the data [20]. PCA was used to find the most dominant patterns in the signal of each channel in reference to the other channels, and outputs the signal in terms of a complementary set of plots [21]. In this study, PCA moves the data from a higher-dimensional state into a lower-dimensional space by maximizing the variance of each dimension.

Finally, normalization was performed, in which the signal was divided by its standard deviation to reduce redundancies and improve the integrity of the data.

2.3 Feature Extraction

To simplify the signals and extract important properties from the EEG data, various features were extracted from the signals. These features can be classified into time- and frequency-domain features. The time-domain features include slope sign change (SSC), zero crossing (ZC), Willison amplitude (WAMP), mean absolute value (MAV), root mean square (RMS), coherence, fractal geometry, waveform length (WL), Hjorth activity, Hjorth mobility (Mobility), Hjorth complexity (Complexity), skewness (Skw), and kurtosis (Kurt). The frequency-domain features include power, relative power, and band power, while other features include entropy.

The features were extracted with a window size of 128 samples (1 second), which overlapped with 64 samples (0.5 seconds). Therefore, seven values were extracted for each feature, with a recorded signal time of 4 seconds.

The band powers are the values of the power separated into different frequency bands. This study considered the following bands: delta (0.5–4 Hz), theta (4–8 Hz), alpha (8–12 Hz), beta low (12–18 Hz), and beta high (18–30 Hz). These were evaluated as absolute and relative powers using a reference band (0.5–30 Hz).

The time-domain features include the following parameters: The first feature is the MAV value, expressed in Eq. (1), which is the average of the absolute value of the EEG signal.

where x_n is the value of each data point and N is the number of data points.

The second feature is the RMS value, expressed in Eq. (2), which is the quadratic mean or square root of the mean squared value.

WL, expressed in Eq. (3), is the time required for the signal to repeat one full cycle or pattern.

WAMP, expressed in Eq. (4), is the number of times the difference between two amplitudes is greater than the threshold [22].

The SSC, expressed in Eq. (5), is the number of times the slope of the signal changes signs within a window given a threshold of variation [23].

ZC, expressed in Eq. (6), is the number of times a signal crosses zero within a window and a threshold for variation [23].

In Eqs. (4)–(6), the threshold values were determined by calculating the classification accuracy at different values, ending with the optimized threshold values.

The Higuchi’s fractal dimension is relevant to several applications in clinical neurophysiology [24]. Higuchi’s algorithm calculates the fractal dimension of a time series within the time domain. It is based on the measurement of the length of the curve that represents the considered time series while using a segment of samples as a unit.

Shannon entropy, expressed in Eq. (7), is another feature used in brain-computer interface systems. Entropy, a measure of uncertainty, can be used to measure the level of chaos in a system [25].

where p(x_n) is the probability of x_n and c is a positive constant acting as a measuring unit.

Three Hjorth parameters are suitable for analyzing nonstationary EEG signals: activity, mobility, and complexity [26].

Activity (8) returns information regarding the signal power [26].

where x(t) represents the EEG signal.

Mobility, expressed in Eq. (9), is an estimation of the mean frequency of the signal [26].

The Complexity, expressed in Eq. (10), represents the band-width and change in frequency [26].

Skw, expressed in Eq. (11), represents the asymmetry around the mean of the signal. A negative Skw value indicates that the data are spread more to the left of the mean than to the right, whereas a positive Skw means that the data are spread more to the right.

Kurt value, expressed in Eq. (12), measures the extent to which the distribution is an outlier.

2.4 Classification

The subjects in the dataset were asked to classify the images into two groups: “like” or “dislike.” The KNN classifier uses parameter k, which is the number of nearest neighbors. For this study, a k value of seven was chosen. Six different settings were used for the DA classifier: linear, quadratic, diagonal linear, diagonal quadratic, pseudo-linear, and pseudo-quadratic. In this study, the pseudo-quadratic DA was chosen because it does not fail.

The NB classifier has two options for the distribution type: normal or kernel. This study used kernel because it can be used when the distribution of a predictor may be skewed or have multiple peaks or modes. The SVM classifier was used because it has a good generalization ability and the capacity to deal with high-dimensional data [27]. SVM has many types of classification models, such as linear, radial, and polynomial models. A linear SVM classifier separates data into classes using a line, which is relatively fast when dealing with large amounts of data. The radial kernel SVM classifier uses a cylindrical shape to separate data into classes. This is a good approach when the data cannot be linearly separated and was used in this research. In addition, empirical error minimization was used over uniform error minimization because it was found to yield promising results with a significantly faster convergence [28]. The empirical technique creates a best-fit line from the data that is otherwise not linear to make the processing faster. A one-versus-one classification was used in the SVM coding design, and an empirical prior probability was used for each class based on its repetition instead of equal probabilities.

For the RF classifier, the number of bags can be manipulated, which in this study was increased from 10 to 100 in increments of 10. This procedure was performed to obtain optimal results. The number of splits was selected as 10. The maximum value from these 10 splits was chosen for each feature.

The K-fold for all classifiers in this study was 7. This means that for seven iterations, approximately 86% (36,900 samples) of the data were chosen as the training group and the remaining 14% (6,150 samples) were the testing group. The K-fold value was selected to ensure the inclusion of a higher number of trials in the testing set, given that the training and test were performed on all subjects together, across subjects and not within subjects.

The classifier was used with all the different features mentioned previously to determine the best features related to decision-making in the brain. In the first classification, all channels were grouped together. Furthermore, to understand the functions of different regions of the brain, various combinations of channels were created by dividing the brain into four regions of interest: frontal lobe (AF3, F7, F3, FC5, FC6, F4, F8, and AF4), temporal lobe (T7 and T8), parietal lobe (P7 and P8), and occipital lobe (O1 and O2). The frontal channels were further divided into two groups: the left (AF3, F7, F3, and FC5) and right (AF4, F8, F4, and FC6) frontal lobe.

Figures 4 and 5 depict the classification performance of the six classifiers across all 21 features. Figure 4 displays the classification accuracies for the 14 different time-domain features, whereas Figure 5 displays the classification accuracies for the seven frequency-domain features. It can be observed that the SVM and RF classifiers are the most accurate for most of the features across both the time- and frequency-domain features. In addition, for most of the 21 features, the KNN classifier yielded the least accurate results. The best classification accuracy was 66.9%, which was achieved using the SVM classifier and WAMP. Figure 5 shows that the RF classifier was more accurate than the SVM classifier and was more accurate than the other six classifiers for all frequency-domain features. Among the seven different frequency-domain features, the RP feature achieved the most accurate prediction performance, with an accuracy of 64.8% with the RF classifier. Notably, the differences in the accuracies of many features were relatively small, and the ranking of features based on their accuracy changed according to the experimental conditions (classifier used). This could be related to the low sampling rate of the system used to collect EEG signals. A larger sample with a higher resolution and higher sampling rate may enable the determination of the best features, specifically for the time-domain features.

To analyze the different regions of the brain, six of the seven frequency-domain features were considered with the RF classifier, which produced the highest accuracies. This is shown in Figure 6, where the two best performing features were the power of all bands and the beta low feature, which yielded 64.9% and 64.8% accuracy, respectively, in the temporal lobe. For all features except the delta band, the highest accuracy was observed in the temporal and frontal lobes. In the delta band, the highest performance was observed in the occipital and temporal lobes, where both reached a performance accuracy of 64.7%, while the frontal lobe had an accuracy of 64.4%.

Upon further analysis of the frontal lobe with the six frequency-domain features and RF classifier, the right frontal lobe performed better than the left frontal lobe for most features. In Figure 7, the highest classification accuracy was observed with the beta low feature, where a classification accuracy of 65.5% was achieved for the right frontal lobe. The alpha wave was the only feature for which the left frontal region outperformed the right frontal lobe, with an accuracy of 64.2% versus 63.3%.

Figure 8 shows the confusion matrix for WAMP obtained using the RF classifier. The “dislike” classification accuracy was 68.2%, the “like” decision was classified with 62.1% accuracy, and the overall accuracy was 65.76%.

This study aims to explore the functionality of six different classifiers and different regions of the brain using various different time- and frequency-domain features to understand decision-making and emotional response. Figures 4 and 5 display the six classifiers with time- and frequency-domain features, respectively, across all channels of the brain. The differences between the classification accuracies of the different features and within the same classifier were not considerable, which suggests further investigation on more data collected with higher sampling rates and resolutions to ensure meaningful differences between the features. A comparison between the results of the current work and the previous results in [1] reveals slight differences for the same feature used, which are ascribable to changes and enhancements in the filtration process and optimization of the classifier parameters. The best-performing feature was WAMP observed with the SVM classifier, which achieved an accuracy of 66.9%. This is supported by [29], where fall detection achieved the most accurate result of 97.35% while using the WAMP feature and SVM classifier. Another high-accuracy feature was Hjorth complexity, which was 66.3% for the SVM classifier. Although the Hjorth parameters are not commonly used in EEG emotion-based research, Li et al. [26] also achieved success when considering these parameters, which is why they were used in this research. In addition, the results of this study are comparable to those of previous studies conducted in the field. Chew et al. [14] achieved accuracies of 61% and 59% for alpha waves using SVM and KNN classifiers, respectively. Yadava et al. [16] achieved accuracies of 68.41% and 62.85% using the RF and SVM classifiers, respectively. Additionally, Doma and Pirouz [30] obtained accuracies of 67.71%, 66.28%, 66.58%, and 66.26% for the SVM, KNN, DA, and DT classifiers, respectively.

Figure 6 shows the results from the seven selected frequency-domain features when separating the channels by brain region for the RF classifier. The highest accuracy was observed in the temporal region of the power for all bands, with an accuracy of 64.9%. Additionally, it consistently achieved the highest accuracy when all features and regions were analyzed, which was supported by [26, 31, 32]. The temporal lobe is responsible for processing auditory information, memory, as well as the identification and categorization of objects. The frontal lobe was another high-accuracy lobe, and high emotion recognition in the frontal lobe was also observed in [33, 34]. Schmidt and Trainor [34] established that the frontal lobe can determine the intensity of emotions. By comparing Figures 5 and 6, it appears that the temporal lobe or frontal lobe alone led to more accurate results for the delta, beta high, and all power bands features; the temporal lobe for beta low achieved higher results than considering the whole brain.

In this study, the beta frequency was divided into two intervals: beta high (18–30 Hz) and beta low (12–18 Hz), which resulted in a higher classification accuracy in the temporal region than in the frontal region, whereas for the beta high feature, the better classification accuracy was in the frontal region. The difference in classification accuracy was slightly higher (0.3%) in the beta low temporal region than in the beta high frontal region; therefore, both were around the same percentage. The human skull is conductive, and the signals collected from one region include some of the signals in other regions, even after the application of PCA. Data collection using a system with a higher number of electrodes (e.g., 32 or 64 channels) can improve the separation of signals into principal components.

By analyzing Figure 7, we noted that the right frontal lobe was better at classifying “like” versus “dislike” decisions. This could be because the right frontal lobe is responsible for feelings of avoidance and negative emotions [35, 36], and as shown in Figure 8, the RF classifier was more accurate when classifying dislike decisions than like decisions. The left frontal cortex was more accurate than the right frontal cortex for the alpha wave only, where the left frontal cortex was associated with positive emotions, and alpha waves were high when relaxed. This correlation may explain why the RF classifier was more accurate in classifying alpha waves in the left frontal cortex. In Figure 7, it can also be observed that beta low had the highest classification accuracy in the right frontal cortex (65.5%), which was higher than considering any single lobe or the whole brain together. Researchers [26, 32, 33, 37, 38] have established that beta waves are effective for emotion recognition and contain important emotion-related neural information.

Considering the distributions of choices when using WAMP as the input feature, the detection rate of dislike was 79.86%, which was higher than the detection rate of 51.89% for the SVM classifier. For the RF classifier, the dislike detection rate had an accuracy of 68.17%, whereas the like detection rate was 62.07%. The like detection accuracy was significantly lower than the dislike rate for the SVM classifier compared to the RF classifier.

Future studies should explore the integration of these EEG features with other physiological measurements to enhance the estimation power of consumer preference classification models.

This study was based on preprocessing data using ICA, PCA, and classification using six different machine learning algorithms. The general conclusion was that the left frontal cortex, whole frontal lobe, and temporal lobe could yield good results in comparison to all channels together. Results such as beta low had the highest results when studied in the right frontal lobe of the brain, and the value exceeded the classification accuracy when all channels or each other lobe was considered. The same applies to the temporal lobe for the frequency-domain features, where the temporal lobe yields higher success than all channels for delta, beta low, beta high, and power for all bands together. Additionally, the frontal lobe outperformed all channels for delta, beta high, and power in all bands. This could limit the need for all 14 EEG channels in studies where two classification groups are used. This can also reduce the size and cost of future neuromarketing devices.

Time-domain features represent potential elements of the classifier, as indicated byWAMP and Hjorth complexity. Among the different frequency bands, the beta frequency band surpassed the other studied frequencies in terms of its accuracy. Further studies on larger datasets and different product categories are required to generalize the results.