For general pattern recognition systems, the patterns can be represented as certain features [1]. It is also well-known fact that the irrelevant or redundant features degrade the classification accuracy, so constructing an appropriate feature subset is essential to build a good classification algorithm. Therefore, developing recognition scheme of real world objects or concepts using computational methods, the selection of an appropriate set of features is of considerable importance [2]. In this paper, we propose a new feature extraction scheme combining locality preserving projection with global information. The proposed method uses principal component analysis (PCA) and linear discriminant analysis (LDA) for global property and locality preserving projections (LPP) for local property of the pattern to improve the classification ability. PCA and LDA are well known for preserving the most descriptive ones after projection while LPP is known for preserving the neighborhood structure of the data set [3]. Our combing method can integrate global and local descriptive information and find a richer set of alternatives beyond PCA, LDA and LPP. In order to find the hybrid features adaptively and find optimal parameters, the genetic algorithm (GA) is employed. GAs have been used for many scientific and engineering optimization and search problems [4]. GAs provide an adaptive and robust computational procedure modeled on the mechanics of natural genetic systems [5]. GAs can be considered as a good solution for finding a richer set of alternatives beyond PCA, LDA and LPP in search space, because they are complicated problem in a large search space [6].

We introduce some preliminaries including PCA, LDA, LPP and GA briefly in Section 2. The proposed global and local features combining method based on GA is explained in Section 3. In Section 4, we assess the experimental performance of our proposed system. Finally, the conclusion is drawn in Section 5.

2.1 Principal Component Analysis

PCA can choose a dimensionality reducing linear projection that maximizes the scatter of all projected samples. Let us assume that an object is represented by the n-dimensional features

and we have a set of objects

where each object belongs to one of C classes. If the total scatter matrix S_T is defined as

where μ ∈ Rⁿ is the mean vector of all objects and computed by

The objective of PCA is maximizing the determinant of the total scatter matrix of the projected samples as follows:

The new feature vectors Y_j ∈ R^m for j = 1, 2, …, N are defined by the following linear transformation

where W ∈ R^n×m is a matrix with orthonormal columns. W_opt that maximizes the objective function is given by the set of n-dimensional eigenvectors of S_T corresponding to the m-largest eigenvalues. A drawback of PCA is that the scatter being maximized is due not only to the between-class scatter that is useful for classification, but also to the within-class scatter that, for classification purposes, is unwanted information [7].

2.2 Linear Discriminant Analysis

The LDA is a linear projection method that maximizes the ratio of determinant of between-class covariance matrix to the determinant of within-class covariance matrix [8]. The within-class scatter matrix S_W and the between class scatter matrix S_b are defined by

and

where $X_j^c$ is the jth object of class c, N_c is the number of objects in class c, and μ_c ∈ Rⁿ is the mean of class c.

S_W and S_b represent the scatter of features around the mean of each class and scatter of features around the overall mean for all classes, respectively [9]. The objective of LDA is to maximize S_b while minimizing S_W as follows:

It has been proven that that this ratio is maximized when the column vectors of the projection matrix W_opt are the eigenvectors of $S_W^{-1}{S}_b$.

2.3 Locality Preserving Projections

Though the information of features is important, the local structure is also significant in many real-world applications [3, 10]. LPP is proposed for learning a locality preserving subspace and seeks to preserve the intrinsic geometry of the samples and local structure. The objective function of LPP is as follows:

where S is a similarity matrix and computed by

In (11), ɛ is a radius of the local neighborhood and defines the locality. Following some simple algebraic steps, we see that

where D is a diagonal matrix and its entries are column sums of S and L = D − S is the Laplacian matrix. Because the bigger value D is, the more important is Y_i, following constraint can be imposed

Finally, we can obtain reduced optimal problem as follows:

The W_opt that minimizes the objective function is given by the m-smallest eigenvalues to the generalized eigenvalue problem [11]

2.4 Genetic Algorithm

The GA typically maintains a fixed size population of individuals that represent the set of solution candidates for the optimization problem to be solved. The goodness of each candidate solution is evaluated using a fitness function. The population of the GA evolves with a set of genetic operators being applied. The basic genetic operators are selection, crossover and mutation. In the selection process, some individuals are selected to be copied into a tentative next population. An individual is selected for the next population with the number of copies proportional to the fitness value. The copied individuals are altered to search for a global optimal solution in the crossover and mutation processes. The GA is simple yet provides adaptive and robust optimization methodology. Given below is the basic configuration of the GA [6].

Table 6. Average, the best, and the worst accrucy of the proposed method.

Database	Average	Best	Worst
Sonar	86.27 (0.679)	92.16	76.47
Spambase	83.13 (0.009)	83.57	82.26
SPECTF	75.37 (0.084)	83.33	63.64
Heart	63.43 (0.026)	65.67	59.70

We propose a method which uses PCA and LDA for global property and LPP for local property of the pattern to improve the classification ability. It can integrate global and local descriptive information and find a richer set of alternatives beyond PCA, LDA and LPP. Based on them, we can obtain new features which preserve global and local structure, simultaneously as follows:

where ω₁, ω₂, and ω₃ are three weight parameters and $Y_i^1,Y_i^2$ and $Y_i^3$ are the features obtained by

where $W_{opt}^j$ for j = 1, 2, 3 is the optimal linear transformation matrixes of PCA, LDA and LPP, respectively.

Because with different ω₁, ω₂, and ω₃ values, (16) provides a rich set of alternatives to PCA, LDA and LPP, we have to find the best parameter setting. For these reason, we employ the GA to select weights parameters. GA can be considered as a good solution for finding a richer set of alternatives beyond PCA, LDA and LPP in a parametric space, because they are complicated combinational problem. Further, we select locality parameter ɛ using GA to find local features efficiently. The chromosome used for optimization is shown as Figure 1.

For experiments, four databases of the UCI repository [12] are used to test the proposed method. The databases used in the experiments are summarized in Table 1.

In this experiment, we use the genetic parameters given in Table 2. We divide each database into the four subsets. We use three subsets as training and the remaining one subset as test set. We made independent four runs to show the general performance of the algorithms. The training set is used to train classifier and find optimal parameter and the test set is used to evaluate the performance of the proposed method. The fitness function of a chromosome is computed by training accuracy using leave-one-out cross-validation of the training set with nearest neighbor (NN) classifier [13]. We denote the results of previous three methods using only PCA, LDA and LPP in Tables 3, 4, and 5, respectively. We also denote the accuracy of the proposed method in Table 6. In Tables 3–6, the numbers in the parenthesis denote the standard deviation of four runs.

Further, the result of the proposed method is compared with those of the conventional feature selection methods in Figure 2. It can be seen that, in all four databases, the proposed methods show better recognition accuracy than the previous methods. The reason for the better performance of the proposed method might be that proposed method considered global and local structures, simultaneously while conventional methods considered only one aspect structure. Further, optimal parameters can be obtained using GA to improve classification ability.

In this paper, we propose the improved the recognition techniques of real-world objects or concepts using computational methods. In classifier, the selection of an appropriate set of features is of considerable importance. Though PCA, LDA and LPP aim to preserve the global and local structure, respectively, it is more important that local and global structures are considered, simultaneously. Therefore, we have proposed a new feature extraction method which can preserve both extrinsic and intrinsic geometry of structures. To find optimal parameters, GA is used. For these reason, we can obtain feature extraction scheme that can have the more classification ability and use features which have global and local properties. The proposed method was applied to four databases selected in UCI machine learning repository and its effectiveness was demonstrated.

No potential conflict of interest relevant to this article was reported.

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP; Ministry of Science, ICT & Future Planning) (No. NRF-2017R1C1B5018408).