Classification, feature subset selection (FSS), and sequential problems are some of the critical problems in machine learning which has drawn substantial aspect from researchers in recent years. Due to the technological advances, data gets generated at an ever increasing space and the dimensionality or size of the datasets are increased by the day. Therefore, it is mandatory to design and develop the potent and adequate machine learning methods, that can be used to select, analyse and extract only the beneficial features. The existence of the dispensable and immaterial features in input may decreases the performance of systems. FSS is critical important pre-processing technique which selects only prominent (non-redundant and relevant) features. Pattern classification is a critical process which is utilize to classify the input dataset into one of the output classes.

As database and computer technologies advance speedily, various advancing challenges appear with respect to FSS and classification problems as follows:

• • Improvement in accuracy by using the same number of attributes;

• • Maintain accuracy by downsizing the attributes;

• • Improvement in accuracy by reducing the number of attributes.

FSS problem has drawn considerable attention from researchers in current years due to the existence of a huge attributes. Many real-time applications contain high-dimensional dataset. These thousands of features are usually provided to the learning algorithms for the classification task.

In many real-time applications, the attributes present in the dataset are ambiguous. Therefore, it becomes very challenging to classify the instance of input. Such kind of the problems in which the ambiguity presents in the attributes of the input dataset [1] are come under the uncertainty classification problem (UCP). Uncertainty classification is used for one class classification [2], binary classification problems [3,4] and imbalance classification problem via support vector machine (SVM) [5]. A gradient approach is developed in Naive Bayes (NB) for value uncertainty classification learning by Lee [6]. Zhang and Ji [1] developed fuzzy extreme learning machine (F-ELM) for uncertainty classification problem.

F-ELM is the latest classifier which associates the working of fuzzy inference system (FIS) and ELM. For uncertainty classification problem F-ELM algorithm is mostly used [1, 7]. F-ELM uses a membership function for required input which is represented by linguistic variables.

Many research papers show the different FSS techniques for various classification problems. However, FSS with incremental learning for uncertainty classification problem is absent in the vast literature survey.

The original F-ELM and ELM are the batch learning mode which considers that the data is available initially. So, F-ELM and ELM classifiers are unable to handle sequent or serial input. So that, online sequential extreme learning machine (OS-ELM) algorithm is presented by Nan-Ying et. al. for sequent input. Three layers are available for ELM and OS-ELM: input, hidden, and output. Randomly initialization of weights is carried out from the first (input) to second (hidden) layer and logically modulated the weights from the second (hidden) to third (output) layer. On account of random initialization, generalized performance of system may degrade. So that it is necessary to select the prominent features.

To distinguish said problem, in this paper incremental F-ELM (RFELM++) and incremental fuzzy online sequential ELM (RFOSELM++) for the UCP are proposed for batch mode and sequential input respectively. Thus, the key intent of this paper is to design RFELM++ and RFOSELM++ algorithm by using the activation function like radial basis function (RBF) and additive function which is missing in the vast literature survey. Another aspect of the paper is to design an IoT based expert system - Precision Agriculture by exploiting the proposed algorithms.

The overall structure of the paper is the following: in Section 2, the literature survey is detailed in the related work. The proposed incremental F-ELM and RFOSELM++ algorithms are depicted in Section 3. The experimental observations and results of clinical datasets are discussed and compared in Section 4. The validation and testing part of the proposed algorithm are discussed in detail in Section 5 as the results and discussions. Section 6 outlined the conclusion with further research direction.

2.1 Feature Subset Selection

In various growing areas FSS plays a very critical task. The key role of FSS is to choose prominent/optimal (relevant and non-redundant) feature subset which produces similar or improved (in some of the cases) results as compared to the prime or base data set in which all features are present. It is very hard to decide the significance of attributes without preparatory information [8]. Therefore, FSS is an prime step to select prominent features for efficient machine learning applications [9, 10]. The primary intention of using any FSS techniques is for [11]:

• • Improvement in classification accuracy,

• • Improvement in prediction performance,

• • Improvement in model interpretability,

• • Dimensionality reduction,

• • Storage management.

2.2 Pattern Classification

Pattern classification is specially used for decision making task. It classifies input entity to one of the predetermined target outputs. A supervised classification is specially used to categorised the given input data into one of the present target classes [18]. Having the data D = xⁱ, yⁱ, where i = 1, …, K and xⁱ ∈ Rⁿ is a single pattern or instance with n features, yⁱ ∈ {1, 2, …, m}. yⁱ is the total number of target class which is represented by m and K represents total number of patterns (D); a problem of pattern classification is the determination of a mapping model M(.), where M(xⁱ) = yⁱ [19]. Neural network (NN) is a robust computational tool which is specifically used for classification. Various NN based classifiers–backpropagation NN (BPNN), multilayer perceptron (MLP), radial basis function (RBF), decision tree, support vector machine (SVM), C4.5, fuzzy NN (FNN), convolutional neural network (CNN), ELM, etc.–are required for classification. Tao et al. [20] designed a deep CNN for detecting the defects in insulators. Wen et al. [21] developed a deep transfer learning (DTL) method for fault diagnosis. ELM is used in breast cancer detection with the help of multilayer fuzzy expert system which was developed by Mojrian et al. [22]. Basically, the classifier plays a very crucial act in system overall performance. The CNN classifier faces various problems like over fitting problem, local maxima and minima problem, etc. Therefore, classifier selection is a challenging task.

2.3 Incremental Learning

Incremental learning is used in various ways like incrementally add hidden nodes [34, 35], incrementally add streaming data [36], incremental FSS [37–40], etc. The exploitation of incremental learning in terms of incremental FSS for sequential input is also missing in the literature survey.

Incremental learning comes in various forms and definitions as

• • Incrementally add streaming data: Incremental learning is a learning for the data which appear sequentially or over time [36, 41]. Alexander Gepperth describes online learning, incremental learning and concept drift for the supervised learning paradigm in detail.

• • Incrementally add hidden nodes: Incremental learning defines as an incrementally insertion of the new neurons to the hidden layer. Error-minimized ELM (EM-ELM) [42], Convex incremental ELM (CI-ELM) [35] and uncremental extreme learning machine (RELM++) [34] are some of the examples of incremental learning. Yang et al. [43] designed a bidirectional extreme learning machines (B-ELM) algorithm for regression.

• • Incremental feature subset selection: Incremental learning in the form of incremental feature subset selection is used in many papers like [37–40]. Baluja [44] have developed a population-based incremental learning method which integrated learning and the random search based function optimization [44]. Tahir and Loo [15] used an incremental ELM is used for food recognition. Wang et al. [45] have designed incremental wrapper based gene selection with Markov blanket. Incremental wrapper subset selection (IWSS) uses a filter measure for feature ranking by using a sequential search [46]. Then, a SFS is used for incrementally insertion of the features.

Wang et al. [47] designed a framework for smart contracts based on a novel six-layer architecture.

This section is categorized into two subsections, RFELM++ and RFOSELM++ algorithms.

3.1 RFELM++ Algorithm

In this section, incremental learning is used to resolve the uncertainty classification problem. As already discussed in Section 2, F-ELM specifically uses for the uncertainty classification problem. Hence, RFELM++ algorithm is initiated. The paradigm of RFELM++ is broadly categorised into four subsystems: input, incremental learning for FSS, fuzzification, and classification.

3.2 RFOSELM++ Algorithm

ELM is a batch learning mode classifier which is not suitable for Sequential input. In this section, incremental learning for FSS is used to solve the UCP as well as sequential problem. Hence, incremental FSS techniques for the UCP is proposed for sequential input which contributes to RFOSELM++ algorithm.

In RFOSELM++, the ranked features are inserted sequentially i.e., one by one. The accuracy is calculated for all feature subsets which are equal to the number of attributes. The proposed RFOSELM++ learning algorithm is applied for every subset and predicted accuracy is calculated. The paradigm of RFOSELM++ is mostly same as RFELM++. Only the classification subsystem is different one. Through observations, the prominent feature subset is one which provides maximum accuracy.

For experimental observation, Waikato Environment for Knowledge Analysis (WEKA) and MATLAB R2014a are used. Precision, Recall and classification accuracy are used as an evaluation measure. The calculations of evaluation measures are depend upon the contingency table or confusion matrix of classifier. Confusion matrix contains both true and false positives (TP/FP) also true and false negatives (TN/FN) [52]. FP correspond to negative samples falsely identified as positive. FN correspond to positive samples falsely resembles as negative. TP are the samples perfectly identified as positive. TN refer to negative samples perfectly identified as negative. The perfectness of identified examples are measured by using precision. The measure of the all positive identified examples is called recall [53]. Accuracy, precision and recall (sensitivity) are measured by evaluating the respective values in Eqs. (1) to (3), respectively

In this section, the incremental learning capability for batch and sequential mode is exploited i.e., incremental learning for F-ELM (RFELM++) algorithm and incremental learning for fuzzy online sequential extreme learning machine, respectively.

4.1 RFELM++ (Prominent Feature Subset)

The experimental results of the proposed RFELM++ algorithm are enumerated. In order to prove the effectiveness of an initiated algorithm the same steps are implemented for ELM classifier (RELM++). The results of both RELM++ and RFELM++ are compared with sigmoidal activity function by considering PID and SHD datasets. The overall comparison of RELM++ and RFELM++ for PID dataset is as shown in Figure 2. The same results are evaluated for SHD dataset also. Through the results, it is observed that RFELM++ produces 6.266% and 0.855% increased accuracy in comparison of RELM++ for PID and SHD datasets consequently.

The overall performance of the FELM classifier is analysed by considering the precision, accuracy and recall. Figure 3 shows an overall comparison of performance for ELM and F-ELM with all features. The results are calculated by using all activity functions like sigmoid function (sig), radial basis function (rbf), sine function (sin), hardlim function (hardlim), triangular basis function (tribas) for SHD and PID datasets. By observations it is apprise that, F-ELM issues an improved performance as compared to ELM classifier.

Various FSS methods like half selection (HS), mean selection (MS), particle swarm optimization (PSO) based FS, neural network for threshold, self-regulated learning (SR-PSO) [54] are used to select prominent features. Table 1 indicates the output of all given FSS methods with ELM and FELM classifiers. For evaluation, the sigmoidal activation function is used for PID dataset. From experimented results, it is noticed that the F-ELM achieves 4.65% improved classification accuracy in comparison with ELM for the same number of feature. The result for RFELM++ algorithm for multiclass classification problem for CHD dataset is as shown in Figure 5.

In order to prove the effectiveness of the proposed RFELM++ algorithm, experiment results are derived and analysed by using the traditional classifiers–MLP, random forest (RF), Bayes net (BN), SVM, NB, RBF, and J4.8–by using prominent features. The prominent features are selected by using Relief algorithm. Five-fold cross-validation method is used for evaluation. Figure 4 shows the variation of the proposed RFELM++ algorithm with existing classifiers by using relief FSS algorithm for PID and SHD data set. It is evidently prove that the proposed RFELM++ algorithm enables maximum predictive accuracy with minimum number of features.

4.2 RFOSELM++ (Prominent Feature Subset)

PID dataset is used as an input dataset. Normalization is used in pre-process. The algorithm uses relief algorithm (distance measure) for ranking the features. IWSS [39, 40] is used which acquires a ranking of the features over to the target class. IWSS is used SFS search over the ranked attributes by incrementally inserting those features one by one. The wrapper way is used to measure the relevance of the new attribute. The classification accuracy is calculated by using OS-ELM classifier.

Experimental results of incremental attribute learning for sequential input is calculated for both ELM and F-ELM classifier. Table 2 shows the parameters required for the evaluation. Figures 6 and 7 show the comparison of RELM++ and RFELM++ for training and testing accuracy, respectively. Through observation, it is noticed that the proposed RFELM++ issues an improved as well as stable (constant) results as compared to RELM++.

In order to prove the validation of the proposed RFELM++ algorithm, experimental observations need to be validated and hypothetically tested [55]. F-ELM classifier is used after the feature selection by using statistical methods. The PID and SHD datasets are categorized into four samples like S4 (SHD ELM), S3 (PID ELM), S2 (SHD F-ELM) and S1 (PID F-ELM). The comparative analysis of before and after relief feature selection algorithm is as shown in Figure 8. Through the results it is noticed that, RFELM++ algorithm with feature subset produces improved performance as compared to with all features.

The correlation analysis technique is used to compute the robustness of the relationship between 2 or more variables. All the time the values of correlation are in between −1 to +1. The abbreviation of +1 is that the two variables are strongly related with positive linear, −1 is that two variables are resolutely related with negative linear and 0 is that no relationship in between the two variables. According to Evans classification, the correlation coefficient are broadly categorized into very strong, strong, moderate, weak, very weak [56].

The c (Pearson’s correlation coefficient) is given by

where ‘p’ is metric before feature selection and ‘q’ is metric after feature selection.

The coefficient (c = 0.6177) is calculated by using all features and Relief FS. Figure 8 illustrates the computation of evaluation metrics accuracy for ELM and F-ELM learning algorithms. Through this computation, it proves that the relationship in between the F-ELM (all features) and RFELM++ (feature subset) is strong as per Evans classification [56].

When the relationship (c) is strong, then the Student’s paired t-test is utilised for comparison in-between two discrete techniques. Paired t-test (pt-value) is calculated by using Eq. (5). To check the competence of the initiated RFELM++ algorithm, Student’s paired t-test is used. Based on the value of r = 0.6177, the t-value is calculated, i.e., −1.097.

where ‘q’ is the difference between ‘a’ (before applying FSS algorithm) and ‘b’ (after applying FSS algorithm) (q_i = a_i−b_i).

Hypothesis of testing [57] RFELM++:

• H₀: No variance between the generalization performance of F-ELM (with all features or before feature selection) and RFELM++ (after feature selection)

• H₁: Variance between the generalization performance of F-ELM (with all features or before feature selection) and RFELM++ (after feature selection)

The pt value i.e., Student’s t-distribution for the degree of freedom 4 is (2.776, ∝= 0.05) and (4.604, ∝= 0.01) [58]. By observation, it is notified that there is no variance between the generalization performance of F-ELM (with all features or before feature selection) and RFELM++ (after feature selection) for four samples, where ∝ is the level of significance. The main reasons behind the improved performances of the proposed algorithm are the removal of irrelevant and redundant features from the input dataset before weighted classification.

5.1 Benchmark Comparison

For performance comparison, accuracy of RFELM++ by using prominent feature subset is compared with F-ELM [59], modified fuzzy min-max neural network-FIS (MFMM-FIS) [60] and enhanced generalized adaptive resonance theory FIS (EGART-FIS) [61] for the PID dataset. Table 3 describes the comparative classification accuracy for the same. With the comparative analysis, it is observed that the testing and training accuracy is 83.47% to 86.80%, respectively by using only 62.5% features.

Agriculture is the backbone of Indian economy where approximate 60% of people are dependent directly or indirectly on agriculture. The expert advice is required for distinguishing the plant disease damage and nutrient imbalance. It is observed that, the conventional judgmental analysis is not enough while deciding the quantity of chemical or fertilizer to be used. The mis-proportional dose harms directly the health of the crop and hence the living beings. To overcome the said problem, this section is developed an IoT-based precision agriculture expert system by inferring proposed algorithms with improving the accuracy by reducing the features or by selecting the prominent features. Grape leaf diseases like downy mildew and nutrient deficiency which contains ambiguity is considered as a case study. Both are spread with yellow spot on leaf. The conceptual framework for the proposed IoT-based precision agriculture expert system is as shown in Figure 10.

In IoT-based precision agriculture expert system, the system is partitioned into three rough phases namely image processing, FSS by using incremental learning and classification. In image processing, the acquisition step is used to input the image. The image is acquired by using cell phone or any digital camera. The resolution of the image is 96 dpi (dots per inch). For interfacing, the ARDUINO hardware is used with various sensors like temperature, humidity etc. The leaf image is used to extract the feature points and are stored in the database.

Data normalization is used as the preprocessing method. In image analysis, image segmentation and feature extraction methods are considered. For image segmentation, region-based segmentation is used to separate the healthy and diseased region by using leaf color. The features extracted from an images are color, shape and texture [71]. Once features are extracted, the database is generated with integrated information from temperature and humidity sensors. On that created dataset the proposed algorithms are applied for feature selection and classification.

In order to increase the generalization performance of the system, the proposed RELM++ and RFOSELM++ algorithms are exploited. The performance of precision agriculture is evaluated and tested by using RELM++ and RFOSELM++ algorithms. The comparison in-between RELM++ and ELM for proposed system (plant disease and nutrient deficiency) is as shown in Figure 11 in terms of testing accuracy. Through this experimentation, it is intended that the RELM++ produces 5.71% increased accuracy and RFOSELM++ maintains (same accuracy) the accuracy with only 27.27% features. However, for Tribas activation function the classical method provides good results as compared to the proposed one. Tribas provided an optimal result only for certain number of hidden neurons for ELM [72].

The prime intention of this paper is to exploit the two proposed algorithms like RFELM++ and RFOSELM++ into the new developed IoT-based precision agriculture Expert system. The expert system helps farmers in decision making. The experimental results for ELM, OS-ELM, RELM++, RFOSELM++, RFELM++ and RFOSELM++ are calculated. RFELM++ produces 6.266% and 0.855% increased accuracy in comparison of RELM++ for PID and SHD datasets consequently. Additionally, various FSS methods like like HS, MS, PSO based FS, neural network for threshold, self-regulated learning (SR-PSO) [54] are used to select prominent features. Through the experimented results, it is noticed that the F-ELM achieves 4.65% improved classification accuracy in comparison with ELM for the same number of feature. The proposed RFOSELM++ maintain the accuracy by using 41.5% features as differentiate to OS-ELM for UCI Repository dataset. For benchmark problem, the results of initiated RFELM++ algorithm is compared with the similar existing approaches. From the comparative average analysis, it is noticed that the introduced RFELM++ issues 8.82% and 6.63% increased accuracy by reducing 6.41% and 10.71% for SHD and PID datasets, respectively. Table 3 describes the comparative classification accuracy for the same. With the comparative analysis, it is observed that the testing and training accuracy is 83.47% to 86.80%, respectively by using only 62.5% features. Hypothetically it is proved that there is no variance between the generalization performance of F-ELM (with all features or before feature selection) and RFELM++ (after feature selection) for four samples. A novel application, IoT-based precision agriculture expert system, is developed by exploiting the proposed algorithms RFELM++ and RFOSELM++. Currently, the algorithms are used for binary classification in future it can be used for multi-class classification. The precision system can be improved by considering the rain removal [73] and by using CNN [74].

Table 1. Comparative results of ELM and F-ELM with different FS methods for input PID dataset.

Table 2. Parameters for RFELM++.

Table 3. Comparison of RFELM++ with existing learning algorithms for input PID dataset.

Table 4. RFELM++ and existing learning algorithms comparison for input PID dataset.

Table 5. RFELM++ and existing learning algorithms comparison for Input SHD dataset.

Table 6. Evaluation comparative analysis for ELM and F-ELM with different FSS for input PID dataset.