International Journal of Fuzzy Logic and Intelligent Systems 2020; 20(2): 145-155
Published online June 25, 2020
https://doi.org/10.5391/IJFIS.2020.20.2.145
© The Korean Institute of Intelligent Systems
Seok-Beom Roh^{1 }, Yong Soo Kim^{2} , and Tae-Chon Ahn^{3 }
^{1}Department of Electrical Engineering, University of Suwon, Hwaseongi, Korea
^{2}Department of Computer Engineering, Daejeon University, Dong-gu, Daejeon, Korea
^{3}Department of Electronics Convergence Engineering, Wonkwang University, Iksan, Korea
Correspondence to :
Tae-Chon Ahn (tcahn@wku.ac.kr)
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted noncommercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
In this study, a newly designed local model called locally weighted regression model is proposed for the regression problem. This model predicts the output for a newly submitted data point. In general, the local regression model focuses on an area of the input space specified by a certain kernel function (Gaussian function, in particular). The local area is defined as a region enclosed by a neighborhood of the given query point. The weights assigned to the local area are determined by the related entries of the partition matrix originating from the fuzzy C-means method. The local regression model related to the local area is constructed using a weighted estimation technique. The model exploits the concept of the nearest neighbor, and constructs the weighted least square estimation once a new query is provided given. We validate the modeling ability of the overall model based on several numeric experiments.
Keywords: k-nearest neighbors, Locally weighted regression, Weighted least square estimation, Lazy learning, Fuzzy C-means clustering
In the fields of machine learning and data analysis, the analysis of the relation between the input and output variables and the prediction of the output variables from newly given data have been some of the most relevant research topics [1]. In the studies dealing with the associated regression problem, two essential and conceptually diversified development strategies have been introduced: global and local learning techniques [2]. Global learning [3] assists in fitting a sophisticated model in terms of the structure to all the observation data patterns. Accordingly, the model that has been learned globally is valid throughout the input space. However, local learning [4, 5] focuses on and fits a relatively simple model to a subset of observations that are present only in a neighborhood of the query point [2]. In this scenario, the model identified by local learning pertains to some limited region of the entire input space, where the query point and its neighbors are positioned.
Local learning techniques, such as
A key issue in local learning techniques is the method to define a subset of all the data patterns to be used, to estimate the parameters of the local model at a related query point.
In this study, we are focused on developing a simple method to determine the neighbors of the given query point, without confining to any adaptive technique. Here, we develop a new nonparametric locally weighted regression (LWR), which is applicable within a region defined by the
The
Although it is very difficult to determine an appropriate structure for a global model, it is unnecessary to do so for a local model in LWR. Therefore, LWR is useful in the modeling of complex non-linear systems [15]. The proposed model, referred to as non-parametric LWR is a regression model, is available within an area in which the
This paper is organized as follows. First, in Section 2, we discuss the LWR based on the
The
The key assumption behind the
In the detailed description of the algorithm, we use the notations listed in Table 1.
The set of indices of the selected neighborhoods for a query is defined as follows:
Notation
After deciding the neighborhood of a query point
where
In case of function approximation, the
where
It is known that the output determined by locally weighted averages could be significantly biased at the boundaries of the local region [19]. To overcome this limitation of the conventional
The LWR is derived from the standard linear regression. The importance of a relevant instance is increased and that of an irrelevant one is decreased by weighting the data [19].
After determining the neighbors of a given query point,
We define the similarity (i.e., the kernel function) between two points, yielding the weight value of each neighbor as follows. The similarity is derived from the partition matrix defined in [23].
The kernel function is generally defined as any explicit functional form (e.g., Gaussian and ellipsoidal). However, the kernel function used in this study is a result of fuzzy clustering called as the fuzzy activation function. The kernel function called as the relative distance function is expressed as
where
The fuzzification coefficient,
The output value of the query point,
where
Here, we present the development of a methodology to design the local regression defined within an area in which several neighbors of a given query point are positioned.
The local models being used in this study involve a constant and a linear regression. A linear regression might exhibit a high modeling bias when dealing with nonlinear systems. A local linear regression focuses on the operating region only (i.e., the local area enclosed by the nearest neighbors); therefore, it can compensate for the high bias of the overall global linear regression.
First, when a query point is provided, the local area should be determined by the nearest neighbors. The “
The obtained nearest neighbors determine the local area where the local model becomes available (suitable). If a new query point is provided, then in return, a new local area is defined.
After determining the local area, the local model has to be identified. To estimate its parameters, the least square estimation (LSE) is used. In this case, WLSE is more aligned with the
To implement the local model within an area defined by the nearest neighbors of the query point, we substitute the kernel function (i.e., Gaussian function) with the similarity function defined in
When a new data point is newly provided, the already implemented local model related to the old query point is discarded and a new one is designed based on the information associated with the new given query point. Specifically, whenever a query point is provided newly, the coefficients of the local regression are re-estimated.
As noted, the local models could be either constant or linear. Both these options were subjects of this study.
A constant local model is defined as follows:
where
We modify the local objective function to be minimized,
where
This learning effect coefficient “
There are two extreme cases:
The optimized coefficients of the local model (6) can be calculated as follows:
where
In contrast with the global learning, in
For the constant local model, the WLSE is used to estimate this value, and the shape parameter is equal to 1; the resulting local model is equivalent to the radial basis function neural network (RBFNN), where the radial basis functions are defined by FCM.
The optimized coefficient has the form
The local model with a linear regression is defined as follows:
We modify the local objective function to be minimized, which is now expressed as
where
The optimal coefficients of the local model (
where
The pseudo code of the proposed model is as follows.
Step 1. The k nearest neighbors of the query point are determined.
Step 2. The weight values are calculated by
Step 3. The coefficient of a local model is calculated by
Step 4. The final output of the local model is calculated by
We conducted several experiments to validate the generalization performance of the proposed model and compared its behavior with those of other models already introduced in the literature. Synthetic low-dimensional data are dealt with in the first subsection. In the second subsection, some selected machine learning datasets from a machine learning repository (
We summarize the results of the experiments in terms of the mean and standard deviation of the performance index.
The important structural parameters are summarized in Table 2. In this study, the number of nearest neighbors (
The performance index of a model is defined as the root mean square error (RMSE) as follows:
A polynomial with two input variables is defined as follows:
The input variables,
We note that the number of nearest neighbors that defines the local area is irrelevant to the performance of both the constant local model and the linear local model.
Let us consider the relevance of some other design parameters, such as the fuzzification coefficient (
Figure 2 shows the performance index of the proposed models (constant local model and linear local model) according to the fuzzification coefficient (
As shown in Figure 2(a) and 2(b), the increase in the values of the shape parameter (
Table 4 summarizes and compares the performance of the proposed model with the already studied models, i.e., linear regression, second-order polynomial, fuzzy
The experiments conducted with several datasets obtained from a machine learning repository are discussed in this subsection. To validate the modeling performance of the proposed model, we performed experiments with nine machine learning datasets, whose detailed information is specified in Table 5.
Table 6 summarizes the comparison results of the proposed model and the already studied models that were implemented in WEKA [25] in terms of the performance index. In Table 6, EPI denotes the performance index of the test dataset.
From Table 6, we can see that the proposed model with a linear function is an appropriate model whose average rank is 2.67. The proposed model with a linear function has the best results in six of the nine datasets among all the models in terms of the performance index (i.e., RMSE).
In this paper, we introduce a new local model based on the LWR. The proposed model can be considered as the expanded version of the
No potential conflict of interest relevant to this article was reported.
Performance index (RMSE) of the proposed local models according to the fuzzification coefficient (
Table 1. Basic notations.
( | Query pattern (test pattern) and its target ( |
Predicted target of a query | |
( | Reference pattern (i.e., training pattern) and its target ( |
Weight matrix (a diagonal matrix with nonzero weights positioned on its diagonal) | |
Kernel function ( |
^{*}is
Table 2. Structural parameters of the proposed model.
Parameter | Value |
---|---|
Polynomial order ( | 0 (constant) or 1 (linear) |
Number of nearest neighbors ( | |
| 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100 |
| 10, 20, 30, 40, 50, 60, 70, 80, 90, 100 |
Fuzzification coefficient ( | 1.2, 1.4, ⋯, 3.8, 4.0 |
Shape parameter ( | 0.2, 0.4, ⋯, 1.8, 2.0 |
Table 3. Performance index (RMSE) of the proposed model versus the polynomial order of the basic model and the size of the neighborhood.
2 | 3.4 | 0.2 | 0.5503±0.0166 | - | - | - | - |
3 | 1.4 | 0.6 | 0.5163±0.0146 | - | - | - | - |
4 | 1.8 | 1.8 | 0.4436±0.0245 | - | - | - | - |
5 | 1.4 | 1.2 | 0.4505±0.0143 | - | - | - | - |
6 | 1.2 | 1.2 | 0.4533±0.0208 | - | - | - | - |
7 | 1.4 | 1.6 | 0.4580±0.0182 | - | - | - | - |
8 | 1.2 | 1 | 0.4573±0.0152 | - | - | - | - |
9 | 1.2 | 1.2 | 0.4585±0.0182 | - | - | - | - |
10 | 1.4 | 1.8 | 0.4464±0.0179 | 10 | 1.2 | 2 | 0.4282±0.0172 |
20 | 1.4 | 2 | 0.4624±0.0228 | 20 | 1.2 | 1.6 | 0.4344±0.0223 |
30 | 1.4 | 2 | 0.4596±0.0147 | 30 | 1.2 | 1.6 | 0.4342±0.0264 |
40 | 1.2 | 1 | 0.4604±0.0118 | 40 | 1.2 | 1.6 | 0.4390±0.0134 |
50 | 1.2 | 1.8 | 0.4616±0.0169 | 50 | 1.2 | 1.6 | 0.4323±0.0169 |
60 | 1.2 | 1 | 0.4660±0.0237 | 60 | 1.2 | 1.6 | 0.4288±0.0145 |
70 | 1.4 | 1.8 | 0.4617±0.0222 | 70 | 1.2 | 1.4 | 0.4367±0.0131 |
80 | 1.2 | 1 | 0.4556±0.0162 | 80 | 1.2 | 1.4 | 0.4251±0.0131 |
90 | 1.2 | 1.2 | 0.4559±0.0182 | 90 | 1.2 | 1.6 | 0.4310±0.0195 |
100 | 1.2 | 2 | 0.4602±0.0207 | 100 | 1.2 | 1.2 | 0.4291±0.0136 |
Values are presented as mean±standard deviation..
Table 4. Results of the comparative analysis.
Model | Testing data | |
---|---|---|
Linear regression | 2.9309±0.038 | |
Second order polynomial | 2.6674±0.0087 | |
Fuzzy | 0.4567 ±0.0234 | |
Local regression with LSE | ||
Constant local model | 0.5189±0.0334 | |
Linear local model | 0.7871±0.0155 | |
Proposed model | ||
Constant local model | 0.4436 ±0.0245 | |
Linear local model | 0.4251 ±0.0131 |
Values are presented as mean±standard deviation. Local regression with LSE is a type of local model that is available within an area defined by the nearest neighbors..
Table 5. Specification of the machine learning datasets.
Dataset | No. of data patterns | No. of features |
---|---|---|
Airfoil | 1,503 | 5 |
Autompg | 392 | 7 |
Boston housing | 506 | 13 |
Concrete | 1,030 | 8 |
CPU | 209 | 6 |
MIS | 390 | 10 |
NOx | 260 | 5 |
Wine quality (red) | 1,599 | 11 |
Yacht | 308 | 6 |
Table 6. Results of the comparative analysis.
Data | Proposed model | Multi-layer perceptron [25] | k-NN [25] | Linear regression [25] | Additive regression [25] | SVR [25] | |||
---|---|---|---|---|---|---|---|---|---|
Polynomial kernel | RBF kernel | ||||||||
Airfoil | |||||||||
EPI | 1.91 | 1.26 | 4.4 | 2.59 | 4.25 | 4.81 | 4.93 | 4.88 | 4.83 |
Rank | 2 | 1 | 5 | 3 | 4 | 6 | 9 | 8 | 7 |
Autompg | |||||||||
EPI | 2.75 | 2.51 | 3.19 | 2.88 | 2.99 | 3.37 | 3.53 | 3.44 | 3.55 |
Rank | 2 | 1 | 5 | 3 | 4 | 6 | 8 | 7 | 9 |
Boston housing | |||||||||
EPI | 3.47 | 3.09 | 4.32 | 4.41 | 5.22 | 4.8 | 4.79 | 4.95 | 5.49 |
Rank | 2 | 1 | 3 | 4 | 8 | 6 | 5 | 7 | 9 |
Concrete | |||||||||
EPI | 12.73 | 11.84 | 7.91 | 8.9 | 9.48 | 10.47 | 8.17 | 10.9 | 10.8 |
Rank | 9 | 8 | 1 | 3 | 4 | 5 | 2 | 7 | 6 |
CPU performance | |||||||||
EPI | 55.05 | 45.74 | 57.25 | 63.65 | 74.98 | 62.79 | 63.57 | 64.45 | 81.02 |
Rank | 2 | 1 | 3 | 6 | 8 | 4 | 5 | 7 | 9 |
MIS | |||||||||
EPI | 0.97 | 0.96 | 1.26 | 1.04 | 1.07 | 1.05 | 1.16 | 1.08 | 1.19 |
Rank | 2 | 1 | 9 | 3 | 5 | 4 | 7 | 6 | 8 |
NOx | |||||||||
EPI | 2.65 | 0.47 | 1.19 | 3.45 | 3.49 | 4.33 | 6.47 | 4.76 | 9.42 |
Rank | 3 | 1 | 2 | 4 | 5 | 6 | 8 | 7 | 9 |
Wine quality | |||||||||
EPI | 1.08 | 1.03 | 0.73 | 0.69 | 0.67 | 0.65 | 0.67 | 0.66 | 0.66 |
Rank | 9 | 8 | 7 | 6 | 4.5 | 1 | 4.5 | 2.5 | 2.5 |
Yacht | |||||||||
EPI | 4.91 | 2.05 | 1.12 | 8.59 | 9.42 | 8.91 | 2.79 | 10.66 | 11.94 |
Rank | 4 | 2 | 1 | 5 | 7 | 6 | 3 | 8 | 9 |
Average rank | 3.89 | 2.67 | 4 | 4.11 | 5.5 | 4.89 | 5.72 | 6.61 | 7.61 |
E-mail: nado9025@gmail.com
E-mail: kystj@dju.kr
E-mail: tcahn@wku.ac.kr
International Journal of Fuzzy Logic and Intelligent Systems 2020; 20(2): 145-155
Published online June 25, 2020 https://doi.org/10.5391/IJFIS.2020.20.2.145
Copyright © The Korean Institute of Intelligent Systems.
Seok-Beom Roh^{1 }, Yong Soo Kim^{2} , and Tae-Chon Ahn^{3 }
^{1}Department of Electrical Engineering, University of Suwon, Hwaseongi, Korea
^{2}Department of Computer Engineering, Daejeon University, Dong-gu, Daejeon, Korea
^{3}Department of Electronics Convergence Engineering, Wonkwang University, Iksan, Korea
Correspondence to:Tae-Chon Ahn (tcahn@wku.ac.kr)
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted noncommercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
In this study, a newly designed local model called locally weighted regression model is proposed for the regression problem. This model predicts the output for a newly submitted data point. In general, the local regression model focuses on an area of the input space specified by a certain kernel function (Gaussian function, in particular). The local area is defined as a region enclosed by a neighborhood of the given query point. The weights assigned to the local area are determined by the related entries of the partition matrix originating from the fuzzy C-means method. The local regression model related to the local area is constructed using a weighted estimation technique. The model exploits the concept of the nearest neighbor, and constructs the weighted least square estimation once a new query is provided given. We validate the modeling ability of the overall model based on several numeric experiments.
Keywords: k-nearest neighbors, Locally weighted regression, Weighted least square estimation, Lazy learning, Fuzzy C-means clustering
In the fields of machine learning and data analysis, the analysis of the relation between the input and output variables and the prediction of the output variables from newly given data have been some of the most relevant research topics [1]. In the studies dealing with the associated regression problem, two essential and conceptually diversified development strategies have been introduced: global and local learning techniques [2]. Global learning [3] assists in fitting a sophisticated model in terms of the structure to all the observation data patterns. Accordingly, the model that has been learned globally is valid throughout the input space. However, local learning [4, 5] focuses on and fits a relatively simple model to a subset of observations that are present only in a neighborhood of the query point [2]. In this scenario, the model identified by local learning pertains to some limited region of the entire input space, where the query point and its neighbors are positioned.
Local learning techniques, such as
A key issue in local learning techniques is the method to define a subset of all the data patterns to be used, to estimate the parameters of the local model at a related query point.
In this study, we are focused on developing a simple method to determine the neighbors of the given query point, without confining to any adaptive technique. Here, we develop a new nonparametric locally weighted regression (LWR), which is applicable within a region defined by the
The
Although it is very difficult to determine an appropriate structure for a global model, it is unnecessary to do so for a local model in LWR. Therefore, LWR is useful in the modeling of complex non-linear systems [15]. The proposed model, referred to as non-parametric LWR is a regression model, is available within an area in which the
This paper is organized as follows. First, in Section 2, we discuss the LWR based on the
The
The key assumption behind the
In the detailed description of the algorithm, we use the notations listed in Table 1.
The set of indices of the selected neighborhoods for a query is defined as follows:
Notation
After deciding the neighborhood of a query point
where
In case of function approximation, the
where
It is known that the output determined by locally weighted averages could be significantly biased at the boundaries of the local region [19]. To overcome this limitation of the conventional
The LWR is derived from the standard linear regression. The importance of a relevant instance is increased and that of an irrelevant one is decreased by weighting the data [19].
After determining the neighbors of a given query point,
We define the similarity (i.e., the kernel function) between two points, yielding the weight value of each neighbor as follows. The similarity is derived from the partition matrix defined in [23].
The kernel function is generally defined as any explicit functional form (e.g., Gaussian and ellipsoidal). However, the kernel function used in this study is a result of fuzzy clustering called as the fuzzy activation function. The kernel function called as the relative distance function is expressed as
where
The fuzzification coefficient,
The output value of the query point,
where
Here, we present the development of a methodology to design the local regression defined within an area in which several neighbors of a given query point are positioned.
The local models being used in this study involve a constant and a linear regression. A linear regression might exhibit a high modeling bias when dealing with nonlinear systems. A local linear regression focuses on the operating region only (i.e., the local area enclosed by the nearest neighbors); therefore, it can compensate for the high bias of the overall global linear regression.
First, when a query point is provided, the local area should be determined by the nearest neighbors. The “
The obtained nearest neighbors determine the local area where the local model becomes available (suitable). If a new query point is provided, then in return, a new local area is defined.
After determining the local area, the local model has to be identified. To estimate its parameters, the least square estimation (LSE) is used. In this case, WLSE is more aligned with the
To implement the local model within an area defined by the nearest neighbors of the query point, we substitute the kernel function (i.e., Gaussian function) with the similarity function defined in
When a new data point is newly provided, the already implemented local model related to the old query point is discarded and a new one is designed based on the information associated with the new given query point. Specifically, whenever a query point is provided newly, the coefficients of the local regression are re-estimated.
As noted, the local models could be either constant or linear. Both these options were subjects of this study.
A constant local model is defined as follows:
where
We modify the local objective function to be minimized,
where
This learning effect coefficient “
There are two extreme cases:
The optimized coefficients of the local model (6) can be calculated as follows:
where
In contrast with the global learning, in
For the constant local model, the WLSE is used to estimate this value, and the shape parameter is equal to 1; the resulting local model is equivalent to the radial basis function neural network (RBFNN), where the radial basis functions are defined by FCM.
The optimized coefficient has the form
The local model with a linear regression is defined as follows:
We modify the local objective function to be minimized, which is now expressed as
where
The optimal coefficients of the local model (
where
The pseudo code of the proposed model is as follows.
Step 1. The k nearest neighbors of the query point are determined.
Step 2. The weight values are calculated by
Step 3. The coefficient of a local model is calculated by
Step 4. The final output of the local model is calculated by
We conducted several experiments to validate the generalization performance of the proposed model and compared its behavior with those of other models already introduced in the literature. Synthetic low-dimensional data are dealt with in the first subsection. In the second subsection, some selected machine learning datasets from a machine learning repository (
We summarize the results of the experiments in terms of the mean and standard deviation of the performance index.
The important structural parameters are summarized in Table 2. In this study, the number of nearest neighbors (
The performance index of a model is defined as the root mean square error (RMSE) as follows:
A polynomial with two input variables is defined as follows:
The input variables,
We note that the number of nearest neighbors that defines the local area is irrelevant to the performance of both the constant local model and the linear local model.
Let us consider the relevance of some other design parameters, such as the fuzzification coefficient (
Figure 2 shows the performance index of the proposed models (constant local model and linear local model) according to the fuzzification coefficient (
As shown in Figure 2(a) and 2(b), the increase in the values of the shape parameter (
Table 4 summarizes and compares the performance of the proposed model with the already studied models, i.e., linear regression, second-order polynomial, fuzzy
The experiments conducted with several datasets obtained from a machine learning repository are discussed in this subsection. To validate the modeling performance of the proposed model, we performed experiments with nine machine learning datasets, whose detailed information is specified in Table 5.
Table 6 summarizes the comparison results of the proposed model and the already studied models that were implemented in WEKA [25] in terms of the performance index. In Table 6, EPI denotes the performance index of the test dataset.
From Table 6, we can see that the proposed model with a linear function is an appropriate model whose average rank is 2.67. The proposed model with a linear function has the best results in six of the nine datasets among all the models in terms of the performance index (i.e., RMSE).
In this paper, we introduce a new local model based on the LWR. The proposed model can be considered as the expanded version of the
Variation in shape of weight function.
Performance index (RMSE) of the proposed local models according to the fuzzification coefficient (
Table 1 . Basic notations.
( | Query pattern (test pattern) and its target ( |
Predicted target of a query | |
( | Reference pattern (i.e., training pattern) and its target ( |
Weight matrix (a diagonal matrix with nonzero weights positioned on its diagonal) | |
Kernel function ( |
^{*}is
Table 2 . Structural parameters of the proposed model.
Parameter | Value |
---|---|
Polynomial order ( | 0 (constant) or 1 (linear) |
Number of nearest neighbors ( | |
| 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100 |
| 10, 20, 30, 40, 50, 60, 70, 80, 90, 100 |
Fuzzification coefficient ( | 1.2, 1.4, ⋯, 3.8, 4.0 |
Shape parameter ( | 0.2, 0.4, ⋯, 1.8, 2.0 |
Table 3 . Performance index (RMSE) of the proposed model versus the polynomial order of the basic model and the size of the neighborhood.
2 | 3.4 | 0.2 | 0.5503±0.0166 | - | - | - | - |
3 | 1.4 | 0.6 | 0.5163±0.0146 | - | - | - | - |
4 | 1.8 | 1.8 | 0.4436±0.0245 | - | - | - | - |
5 | 1.4 | 1.2 | 0.4505±0.0143 | - | - | - | - |
6 | 1.2 | 1.2 | 0.4533±0.0208 | - | - | - | - |
7 | 1.4 | 1.6 | 0.4580±0.0182 | - | - | - | - |
8 | 1.2 | 1 | 0.4573±0.0152 | - | - | - | - |
9 | 1.2 | 1.2 | 0.4585±0.0182 | - | - | - | - |
10 | 1.4 | 1.8 | 0.4464±0.0179 | 10 | 1.2 | 2 | 0.4282±0.0172 |
20 | 1.4 | 2 | 0.4624±0.0228 | 20 | 1.2 | 1.6 | 0.4344±0.0223 |
30 | 1.4 | 2 | 0.4596±0.0147 | 30 | 1.2 | 1.6 | 0.4342±0.0264 |
40 | 1.2 | 1 | 0.4604±0.0118 | 40 | 1.2 | 1.6 | 0.4390±0.0134 |
50 | 1.2 | 1.8 | 0.4616±0.0169 | 50 | 1.2 | 1.6 | 0.4323±0.0169 |
60 | 1.2 | 1 | 0.4660±0.0237 | 60 | 1.2 | 1.6 | 0.4288±0.0145 |
70 | 1.4 | 1.8 | 0.4617±0.0222 | 70 | 1.2 | 1.4 | 0.4367±0.0131 |
80 | 1.2 | 1 | 0.4556±0.0162 | 80 | 1.2 | 1.4 | 0.4251±0.0131 |
90 | 1.2 | 1.2 | 0.4559±0.0182 | 90 | 1.2 | 1.6 | 0.4310±0.0195 |
100 | 1.2 | 2 | 0.4602±0.0207 | 100 | 1.2 | 1.2 | 0.4291±0.0136 |
Values are presented as mean±standard deviation..
Table 4 . Results of the comparative analysis.
Model | Testing data | |
---|---|---|
Linear regression | 2.9309±0.038 | |
Second order polynomial | 2.6674±0.0087 | |
Fuzzy | 0.4567 ±0.0234 | |
Local regression with LSE | ||
Constant local model | 0.5189±0.0334 | |
Linear local model | 0.7871±0.0155 | |
Proposed model | ||
Constant local model | 0.4436 ±0.0245 | |
Linear local model | 0.4251 ±0.0131 |
Values are presented as mean±standard deviation. Local regression with LSE is a type of local model that is available within an area defined by the nearest neighbors..
Table 5 . Specification of the machine learning datasets.
Dataset | No. of data patterns | No. of features |
---|---|---|
Airfoil | 1,503 | 5 |
Autompg | 392 | 7 |
Boston housing | 506 | 13 |
Concrete | 1,030 | 8 |
CPU | 209 | 6 |
MIS | 390 | 10 |
NOx | 260 | 5 |
Wine quality (red) | 1,599 | 11 |
Yacht | 308 | 6 |
Table 6 . Results of the comparative analysis.
Data | Proposed model | Multi-layer perceptron [25] | k-NN [25] | Linear regression [25] | Additive regression [25] | SVR [25] | |||
---|---|---|---|---|---|---|---|---|---|
Polynomial kernel | RBF kernel | ||||||||
Airfoil | |||||||||
EPI | 1.91 | 1.26 | 4.4 | 2.59 | 4.25 | 4.81 | 4.93 | 4.88 | 4.83 |
Rank | 2 | 1 | 5 | 3 | 4 | 6 | 9 | 8 | 7 |
Autompg | |||||||||
EPI | 2.75 | 2.51 | 3.19 | 2.88 | 2.99 | 3.37 | 3.53 | 3.44 | 3.55 |
Rank | 2 | 1 | 5 | 3 | 4 | 6 | 8 | 7 | 9 |
Boston housing | |||||||||
EPI | 3.47 | 3.09 | 4.32 | 4.41 | 5.22 | 4.8 | 4.79 | 4.95 | 5.49 |
Rank | 2 | 1 | 3 | 4 | 8 | 6 | 5 | 7 | 9 |
Concrete | |||||||||
EPI | 12.73 | 11.84 | 7.91 | 8.9 | 9.48 | 10.47 | 8.17 | 10.9 | 10.8 |
Rank | 9 | 8 | 1 | 3 | 4 | 5 | 2 | 7 | 6 |
CPU performance | |||||||||
EPI | 55.05 | 45.74 | 57.25 | 63.65 | 74.98 | 62.79 | 63.57 | 64.45 | 81.02 |
Rank | 2 | 1 | 3 | 6 | 8 | 4 | 5 | 7 | 9 |
MIS | |||||||||
EPI | 0.97 | 0.96 | 1.26 | 1.04 | 1.07 | 1.05 | 1.16 | 1.08 | 1.19 |
Rank | 2 | 1 | 9 | 3 | 5 | 4 | 7 | 6 | 8 |
NOx | |||||||||
EPI | 2.65 | 0.47 | 1.19 | 3.45 | 3.49 | 4.33 | 6.47 | 4.76 | 9.42 |
Rank | 3 | 1 | 2 | 4 | 5 | 6 | 8 | 7 | 9 |
Wine quality | |||||||||
EPI | 1.08 | 1.03 | 0.73 | 0.69 | 0.67 | 0.65 | 0.67 | 0.66 | 0.66 |
Rank | 9 | 8 | 7 | 6 | 4.5 | 1 | 4.5 | 2.5 | 2.5 |
Yacht | |||||||||
EPI | 4.91 | 2.05 | 1.12 | 8.59 | 9.42 | 8.91 | 2.79 | 10.66 | 11.94 |
Rank | 4 | 2 | 1 | 5 | 7 | 6 | 3 | 8 | 9 |
Average rank | 3.89 | 2.67 | 4 | 4.11 | 5.5 | 4.89 | 5.72 | 6.61 | 7.61 |
Variation in shape of weight function.
|@|~(^,^)~|@|Performance index (RMSE) of the proposed local models according to the fuzzification coefficient (