Article Search
닫기

Original Article

Split Viewer

Int. J. Fuzzy Log. Intell. Syst. 2018; 18(2): 126-134

Published online June 25, 2018

https://doi.org/10.5391/IJFIS.2018.18.2.126

© The Korean Institute of Intelligent Systems

Comparisons of Deep Learning Algorithms for MNIST in Real-Time Environment

Akmaljon Palvanov, and Young Im Cho

Department of Computer Engineering, Gachon University, Seongnam, Korea

Correspondence to :
Young Im Cho (yicho@gachon.ac.kr)

Received: March 17, 2018; Revised: June 9, 2018; Accepted: June 12, 2018

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Recognizing handwritten digits was challenging task in a couple of years ago. Thanks to machine learning algorithms, today, the issue has solved but those algorithms require much time to train and to recognize digits. Thus, using one of those algorithms to an application that works in real-time, is complex. Notwithstanding use of a trained model, if the model uses deep neural networks it requires much more time to make a prediction and becomes more complicated as well as memory usage also increases. It leads real-time application to delay and to work slowly even using trained model. A memory usage is also essential as using smaller memory of trained models works considerable faster comparing to models with huge pre-processed memory. For this work, we implemented four models on the basis of unlike algorithms which are capsule network, deep residual learning model, convolutional neural network and multinomial logistic regression to recognize handwritten digits. These models have unlike structure and they have showed a great results on MNIST before so we aim to compare them in real-time environment. The dataset MNIST seems most suitable for this work since it is popular in the field and basically used in many state-of-the-art algorithms beyond those models mentioned above. We purpose revealing most suitable algorithm to recognize handwritten digits in real-time environment. Also, we give comparisons of train and evaluation time, memory usage and other essential indexes of all four models.

Keywords: Capsule networks, Dynamic routing, Residual learning, CNN, Logistic regression

No potential conflict of interest relevant to this article was reported.

Akmaljon Palvanov received his B.S. in telecommunication technologies from Tashkent University of Information Technologies named after Muhammad Al-Khwarizmi, Tashkent, Uzbekistan, in 2017. He is currently pursuing his M.S. in computer engineering at Gachon University. His current research interests include AI, data analysis and smart city.

E-mail: akmaljon.palvanov@gmail.com


Young Im Cho received her B.S., M.Sc., and Ph.D. from the Department of Computer Science, Korea University, Korea, in 1988, 1990, and 1994, respectively. She is a professor at Gachon University. Her research interest includes AI, big data, information retrieval, smart city, etc.

E-mail: yicho@gachon.ac.kr


Article

Original Article

Int. J. Fuzzy Log. Intell. Syst. 2018; 18(2): 126-134

Published online June 25, 2018 https://doi.org/10.5391/IJFIS.2018.18.2.126

Copyright © The Korean Institute of Intelligent Systems.

Comparisons of Deep Learning Algorithms for MNIST in Real-Time Environment

Akmaljon Palvanov, and Young Im Cho

Department of Computer Engineering, Gachon University, Seongnam, Korea

Correspondence to:Young Im Cho (yicho@gachon.ac.kr)

Received: March 17, 2018; Revised: June 9, 2018; Accepted: June 12, 2018

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Recognizing handwritten digits was challenging task in a couple of years ago. Thanks to machine learning algorithms, today, the issue has solved but those algorithms require much time to train and to recognize digits. Thus, using one of those algorithms to an application that works in real-time, is complex. Notwithstanding use of a trained model, if the model uses deep neural networks it requires much more time to make a prediction and becomes more complicated as well as memory usage also increases. It leads real-time application to delay and to work slowly even using trained model. A memory usage is also essential as using smaller memory of trained models works considerable faster comparing to models with huge pre-processed memory. For this work, we implemented four models on the basis of unlike algorithms which are capsule network, deep residual learning model, convolutional neural network and multinomial logistic regression to recognize handwritten digits. These models have unlike structure and they have showed a great results on MNIST before so we aim to compare them in real-time environment. The dataset MNIST seems most suitable for this work since it is popular in the field and basically used in many state-of-the-art algorithms beyond those models mentioned above. We purpose revealing most suitable algorithm to recognize handwritten digits in real-time environment. Also, we give comparisons of train and evaluation time, memory usage and other essential indexes of all four models.

Keywords: Capsule networks, Dynamic routing, Residual learning, CNN, Logistic regression

Fig 1.

Figure 1.

A bottleneck building block.

The International Journal of Fuzzy Logic and Intelligent Systems 2018; 18: 126-134https://doi.org/10.5391/IJFIS.2018.18.2.126

Fig 2.

Figure 2.

A network architecture for CapsNet, consists of three layers.

The International Journal of Fuzzy Logic and Intelligent Systems 2018; 18: 126-134https://doi.org/10.5391/IJFIS.2018.18.2.126

Fig 3.

Figure 3.

The Java based GUI application and the process recognizing handwritten digit. The same inputs are given and four tranined models recognize a written digit. Predicted digit, allocated time to make a prediciton and network accuracy of each model are shown.

The International Journal of Fuzzy Logic and Intelligent Systems 2018; 18: 126-134https://doi.org/10.5391/IJFIS.2018.18.2.126

Fig 4.

Figure 4.

Accuracy rate throughout epochs.

The International Journal of Fuzzy Logic and Intelligent Systems 2018; 18: 126-134https://doi.org/10.5391/IJFIS.2018.18.2.126

Fig 5.

Figure 5.

Loss function of all models.

The International Journal of Fuzzy Logic and Intelligent Systems 2018; 18: 126-134https://doi.org/10.5391/IJFIS.2018.18.2.126

Table 1 . Evaluation time and memory usage.

Regression modelCNNResNetCapsNet
Evaluation time (s)3–47–125–71–2
Memory (MB)3.1385749.4

Table 2 . Accuracy, training time and number of inputs.

Regression modelCNNResNetCapsNet
DatasetMNIST
Accuracy (%)92.198.197.399.4
Training time (min)2.5215347
Number of inputs50,00050,00050,00040,000

Share this article on :

Related articles in IJFIS

Most KeyWord