As the possibility of hacking increases owing to the development of IT technology, security technologies for biometrics continue to develop because existing authentication methods cannot effectively protect the personal information of an individual. Biometrics technology is used to extract human characteristics and behavioral information to determine whether they belong to a particular person. There are two types of such technology. The first is used to authenticate a user through verification with a database using 1:1 matching. The second is used to identify the user through a search of their data in the database using 1:N matching. This type uses the individual’s biological characteristics such as iris, face, hand shape, ear shape, fingerprint, gate, and voice. Such biometric information differs from person to person and does not change over the years. There is also has an advantage in that there is no risk of memorization or exposure such as with other existing authentication methods. Of the various types of biometric authentication services, fingerprint recognition has been most widely commercialized.

Contemporary fingerprint comparison technology began in 1684 when Neemia Couture of England first discovered that human fingerprints differ from each other. Later, Henry first presented a taxonomy of fingerprints in 1900. The classification of fingerprints is based on features such as ridges, endings, bifurcations, deltas, and cores, and according to the flow, whorl, right loop, double loop, left loop, and arch. Korea has about 41 million fingerprints images in its database. To find a match out of such a large number of fingerprint images, an accurate classification of the images is required.

Kang et al. [1, 2] proposed an effective preprocessing method for determining a threshold value using a neural network or a neuro-fuzzy method for the histogram extracted from a fingerprint image. Jeong and Lee [3] classified fingerprint images with 88% accuracy while modifying the Markov model value using a genetic algorithm after extracting the ridge information in 16 directions using the Markov model for preprocessed fingerprint images.

The biggest problem with existing researches is the thinning required to obtain ridge direction information. Thinning causes noise such as twigs that were not in the original image. To remove such noise, additional image processing should be applied. In this study, we try to classify fingerprints without a thinning process using a convolution neural network (CNN).

In 2012, Krizhevsky et al. [4], proposed by the Supervision team led by Hinton, won the ImageNet Large Scale Visual Recognition Competition (ILSVRC) by a large gap of 16% over the second-place team. After that, deep learning began to attract attention, and has been used in various fields such as recognition and classification. Wang et al. [5] proposed a fingerprint classification method using the deep learning method. Bae et al. [6] classified fingerprints through an improved version of the neuro-fuzzy technique. Peralta et al. [7] conducted a study combining an ensemble technique using an improved model of Alexnet. Other various studies have also been conducted.

The proposed system is shown in Figure 1. Preprocessing is applied to an input fingerprint image to improve the quality [8]. Fingerprint classification is then conducted through the extraction and learning of image features using a CNN.

The rest of this paper is composed as follows: In Section 2, we describe existing fingerprint classification methods. In Section 3, we describe the learning method, the models used, and the techniques of VGGNet [9], which is a type of CNN. Section 4 describes experiments conducted on fingerprint recognition and an analysis of the results. Section 5 provides some concluding remarks.

The existing fingerprint classification method is as follows. To improve the quality of a fingerprint image, the noise is removed through normalization, and the fingerprint image area is isolated from the background using a segmentation technique. Then, binarization is applied based on the threshold value, and the image is enhanced using a Gabor or similar type of filter. Next, ridges 1 pixel thick are made using a thinning process. Finally, the types and positions of the minutiae are determined, and the fingerprints are classified by analyzing the directions of the entire ridges.

During the thinning process, unnecessary deformation of the ridges may occur, such as cut, twig, circular, or cross ridges, as shown in Figure 2. To remove these deformities, the end points are traced at the bifurcation point based on the angle between the end points and the average angle. Using certain criteria, cut ridges are then connected, twig ridges are removed, a part of the circular ridges is moved, and cross ridges are disconnected. Figure 3 shows the results of removing unnecessary ridges. Although these processes take a significant amount of time, human being can conduct this fingerprint classification without a thinning or post process. The purpose of this study is to implement the characteristics of human beings into the classification process.

3.1 Fingerprint Image Preprocessing

Preprocessing is also applied in the proposed method. Human beings can be said to perform image processing as determined by the illusion of people. An image preprocessing allows features to be accurately extracted using a CNN. An input image is normalized and segmented by calculating the directionality of the ridges. Binarization and Gabor filtering are then applied. The thinning process is deemed unnecessary based on the drawbacks described in Section 2 and is excluded. An input image and the processing results are shown in Figure 4.

3.2 Ensemble Learning

To improve the recognition rate of an image in a CNN, the more characteristic information that is available the better. One enhancement method used is the ensemble technique [10]. As shown in Figure 5, the ensemble technique is a method for predicting a result using differently generated features and combing them in a plurality of models, thereby achieving a better performance than using a single model. However, this technique has certain disadvantages such as an over fitting, and a slower learning speed owing to the increase in the number of parameters. To overcome these drawbacks, we use a batch normalization method [11].

3.3 Batch Normalization

When learning a neural network, the data are usually used in mini-batch units. The mean and standard deviation of each feature are obtained and normalized between zero and 1, and then converted into new values using a scale factor and a shift factor. This method is called batch normalization. When actually used, as shown in Figure 6, a batch normalization layer is added before entering a specific hidden layer, and the input values are then modified and put into the activation function.

Batch normalization plays a role similar to the existing regularization, and has the advantage of solving the over fitting phenomenon. The learning rate can also be set to a large value, and the learning speed can be improved.

3.4 Learning Using a CNN

The CNN model used in this study is VGGNet. The University of Oxford’s VGG team first discovered the concept of factorizing convolution, which is better at stacking 3 × 3 convolutions more deeply than 5 × 5 and 7 × 7 convolutions, and developed VGGNet based on this concept. Factorizing convolution has the advantage of reducing the number of parameters and extracting good features over existing large size filters.

Factorization convolution is shown in Figure 7. We compared three different models, VGGNet (model 1), VGGNet with preprocessing (model 2), and VGGNet with batch normalization and ensemble-based models (model 3).

Table 1 shows the CNN structure used for the models 1 and 2, when applying basic VGGNet. VGGNet has a simple structure and receives an input of 224 × 224 in size. After applying zero padding to the input image, convolution is calculated using a 3 × 3 filter with a stride of 1 twice, and the output is calculated in the same way using a filter size of 2 × 2 with a stride 2.

Between this calculations, the convolution is applied three times using a filter size of 3 × 3 with a stride of 1, and the pooling method is repeated three times. The output to the last pooling is 7 × 7 in size. The output is used as the input of the fully connected layer, and the probability of each class when outputting five outputs during the last output is calculated using Softmax.

As shown in Table 2, the disadvantages of the ensemble technique can be minimized when adding batch normalization between the convolution operation and the activation function operation using the structure shown in Table 1.

4.1 Experiment Environment

The data used in this paper were supplied by the Fingerprint Verification Competition (FVC) 2000, 2002, 2004 [12–14] (Figure 8). The types of fingerprint images are classified into five categories, namely, whorl, right loop, double loop, left loop, and arch. Details of the images are shown in Table 3 and Figure 9. The database consists of 788 learning images and 100 test images.

The experiment environment used for the learning and testing is as follows: Linux Ubuntu 16.02.2 was used for the operating system. The hardware consists of an Intel i7-6770k CPU, 32 GB memory, two NVIDIA Maxwell TITAN X GPUs, and the Tensorflow 1.1.0 deep running framework.

Three types of CNN models, basic VGGNet (model 1), VGGNet using preprocessed images (model 2), and VGGNet combining ensemble techniques (model 3), were learned and their performance compared. A fingerprint image with a size of 350 × 348 was reduced to 224 × 224. The hyper parameters used for the learning are shown in Table 4.

The number of epochs was set to 200, and the learning rate was set to 0.001; however, in case of the model 3, it was set to 0.01 because this model uses batch normalization. The batch size was set to 64; however, because model 3 uses both ensemble and batch normalization, we set its batch size to 16 to reduce memory usage. The optimizer uses SGD.

4.2 Experiment Results and Analysis

The performance evaluation results using the three models proposed in this paper are shown in Table 5. For model 1, the learning time was the shortest but the accuracy was the lowest. For model 2, we can see that the accuracy is higher than the learning time. Model 3 has a slightly different performance than model 2, but has a long learning time. For model 3, an accuracy level of 97.1% was obtained before the ensemble was applied, and an accuracy level of 98.3% was achieved after the ensemble was applied. Considering the speed of learning and the performance improvement, model 3, which showed a 98.3% recognition rate, was deemed the best model.

Table 6 shows the recognition results for each type when applying model 3. Twenty fingerprint images were randomly selected five times for each type of fingerprint image and tested, resulting in a total of 100 images. The test results show that the classification rates for left loop, double loop, and right loop are slightly lower. An example of an image misclassified during the test is shown in Figure 10. Because the center point cannot be found in Figure 10(a), it was misclassified as a whorl despite originally being a left and right loop type. Because of the contamination of the center, the image is difficult to classify by the human eye. The image shown in Figure 10(b) is a double loop, but was classified as a left loop because it has a pattern similar to this type of loop.

In this study, three models were created through a modification of the VGGNet structure, and fingerprint classification was applied using each model. In this study, we found that the preprocessing of a fingerprint is indispensable after comparing models 1 and 2. Model 3 has a small batch size but is fast and shows an excellent performance. Using model 3, the average classification rate was 97.2%.

The important aspect of fingerprint classification is a fast matching speed. In future research, we will study CNN structure optimization, and speed up the learning speed for fast matching. In addition, we will investigate methods to improve the classification performance of images with noise.

Table 1. Basic VGGNet structure (model 1 and 2).

Table 2. Improved VGGNet for ensemble (model 3).

Table 3. Type and number of fingerprints.

Table 4. Hyper parameter setting.

Table 5. Model performance evaluation.

Table 6. Confusion matrix of model 3.