search for




 

Android-Based E-Board Smart Education Platform Using Digital Pen and Dot Pattern
International Journal of Fuzzy Logic and Intelligent Systems 2015;15(4):260-267
Published online December 30, 2015
© 2015 Korean Institute of Intelligent Systems.

Young Im Cho, and Aigerim Bakatkaliyevna Altayeva

Department of Computer Engineering, Gachon University, Seongnam, Korea
Correspondence to: Young Im Cho, (yicho@gachon.ac.kr)
Received December 8, 2015; Revised December 24, 2015; Accepted December 25, 2015.
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract

In the past, we implemented a web-based smart education platform, but this is not efficient in a smart or mobile education environment. Therefore, in this paper, we propose an Android-based e-board smart platform for a smart or mobile education system. Here, we use Anoto digital pen- and dot pattern-based technologies. This Android-based smart education platform is efficient for a smart education environment. Further, we implement the hardware and software parts of the technologies, an Anoto-based trajectory recognition algorithm, and a probabilistic neural network for handwritten digit and hand gesture recognition.

Keywords : Anoto digital pen, Dot pattern, E-board, M2M, Smart platform
1. Introduction

In the market, there are many types of interactive blackboards for a smart education system [1], ranging from study-purpose boards to commercial enterprise boards. Depending on the purpose changes their functionality sets, allowing the use of different types of high technologies and modern opportunities.

However, there are two main problems in interactive blackboards. One is that they require not only additional devices such as a personal computer and display equipment but also the knowledge of using the PC and software in order to launch and configure. The other problem is the launching time of these devices.

To overcome these two problems, we have developed an embedded Android-based e-board system with a short booting time. In this paper, we propose an Android-based e-board smart education platform. Here, we use an Anoto digital pen and a dot pattern for interactive blackboards [2, 3].

Our main idea is to use the Android device connected to a screen projector by using a digital pen-based dynamic UI/UX dot pattern. Instead of a number of running process sequences, the proposed method gives a chance to start working with one click. It considerably decreases the launching time and does not require the use of additional hardware equipment.

Technologies such as the Anoto digital pen and paper will allow users to access digital information through interaction with printed copies. This interpretation and transmission of handwritten text and images is based on the technology of the Anoto digital pen, which is a special digital pen, and paper with a printed pattern invisible to the eye.

Advanced image processing and complex infrastructure allow anything written with a pen on the paper, to be transformed into digital media via wireless communication. Such use of the pen and paper is a unique step forward in the field of electronic communication. In fact, it combines the digital world of the computer and the traditional world of paper and pen.

Various researchers have studied how best to incorporate the interaction with digital services through interactive paper interfaces [4]. The developed prototypes have shown the potential of the available technologies to overcome the digital divide and the use of paper as paper,.

However, most of the existing approaches focus on the enrichment of digital interactions through paper only, often forgetting to take into account the process of creating interactive paper documents and the effective management of information through two interfaces. The current lack of support for the generation of rich interactive paper documents leads to the separation of paper documents into two digital individuals who are unlikely to be matched again as the paper and the digital copies of the same document.

The rest of this paper is organized as follows: We present an overview of the smart education system in Section 2, reviewing Android based e-board platform in Section 3, and discuss about next generation e-board and its technologies in Section 4. Finally, we present a discussion of our research and exploring about web and Android based e-board system organization in Section 5 and the conclusions in Section 6.

2. Overview of Smart Education System

As ubiquitous learning contexts include context-sensitive interactions between students and the media, a smart education system can be used as a technology-supported learning environment considered to make adjustments and provide adequate support at the right places at the right time on the basis of the individual needs of students. Further, it analyzes the behavior and the performance of online learning, and the actual conditions [5].

It should be noted that the training in a smart education system does not mean that students have only a limited amount of free time. No specific training program is used for replacing their free time. An intelligent learning environment designed to track student learning, even when they are in the form of entertainment or game. Such type of learning plays the role of a wise friend looking for an opportunity to advise students in their daily lives, taking into account their needs and preferences.

As shown in Figure 1, a smart education system mainly consists of the following modules: learning status detecting module, learning performance evaluation module, adaptive learning task module, adaptive learning content module, personal learning support module, and a set of databases for storing learner data.

3. Android-Based E-Board Smart Platform

Our smart education system consists of several subsystems in the smart learning framework. To realize smart education, we create an Android-based next generation e-board by using dot patterns, M2M, machine technologies, and machine learning algorithms.

3.1 E-board

E-board is more than just a paperless board meeting solution. It is a suite of several integrated web-based governance modules. Effective governance requires strategic planning, sound policy, effective meetings, communications, and evaluations. E-board can help you with all of these.

An interactive whiteboard is a touch-sensitive screen that works in conjunction with a computer and a projector. Instead of using the mouse, you can control your computer through the interactive whiteboard screen just by touching it with a special pen. The first interactive whiteboard was manufactured by SMART Technologies Inc. in 1991. Educators were the first people to recognize the interactive whiteboard’s potential as a tool for collaboration, improving student learning outcomes, and streamlining lesson planning.

Interactive whiteboards are an effective way to interact with electronic content and multimedia in a multi-person learning environment. Learning activities with an interactive whiteboard may include the following [6]: manipulating text and images; taking notes in digital ink; saving notes for review via e-mail, the web, or print; viewing websites as a group; demonstrating or using software at the front of a room without being locked behind a computer; creating electronic lesson activities with templates and images; showing and writing notes over educational video clips; using presentation tools built into the interactive whiteboard software to enhance learning materials; and showcasing student presentations.

3.2 Digital Pen Subsystems

A digital pen is a battery-operated writing instrument that allows a user to digitally capture a handwritten note or drawing. Typically, a digital pen comes with a universal serial bus (USB) cradle to let the user upload the handwritten notes to a personal computer (PC). The pen looks like a regular ballpoint pen and can be used as such, but requires special digital paper if the user wishes to digitally capture what he has written. A page of the digital paper, which can be purchased in the traditional pad or sticky-note size, has small dots on it that allow the pen to “see” what has been written

Anoto Digital Pen and Paper Technology

This device looks and feels like a normal ballpoint pen. However, it contains an integrated digital camera, an advanced image microprocessor, and a Bluetooth transmitter. Any paper can be used with a digital pen, if the Anoto dot pattern is added to the layout before printing the paper. The Anoto dot pattern consists of numerous black dots that can be read by the digital pen but are almost invisible to the naked eye. The pen reads the pattern and registers what and where the user writes [7]. Digital images at a rate of 100 per second are taken from a template. It also contains an ink cartridge so that you can see what you have written or drawn. The ink from the pen is not visible to the camera. Therefore, the picture does not destroy your letter.

Image Processor

This calculates in real time the exact position of the entire Anoto own image. During image processing, the images are compared, and the information about how the pen is held is also gathered and stored.

Memory

All the data from the image processor are packaged and loaded into the memory, which can store written pages.

Bluetooth Transceiver

Information is transmitted by the Bluetooth transceiver directly to a computer or sent through a relay device for the Anoto lookup service. Hence, some of the information will be sent for further processing elsewhere.

Ink Cartridge and Sensor

The pen holds an ordinary ink cartridge to make visible all that you have written or drawn. A force-sensing resistor measures the stylus tip force.

Digital Paper

This is what makes this innovation so special. Printed on it is the pattern that consists of millions of tiny dots, which make it possible to determine the exact position of the pen, and it has a number of unique features to connect the model. The work consists of plain paper with a dot pattern invisible to the eye, that is, before anything is printed or the paper is placed on a laser printer.

Digital Pen

In addition to conventional ink, the digital pen has a digital camera with a special optical reader, advanced image processing system, and a communication unit, which makes possible the retrieval and storage of data during writing [2].

3.3 Digital Pen Technology

An evaluation of mainstream digital pens available in the market on the basis of economic, technical, and logistical criteria leads us to experiment with an Anoto Digital Scribe pen, which can write on any type of paper and offer the simplest possible functionality that satisfies the requirements of our study. The Anoto digital pen works with a scanner device clipped to the top of the paper sheet, which captures the handwriting and saves it in a persistent storage. The device can then be connected to a PC by using Bluetooth technology to download digital annotations.

Figure 2 depicts the different layers of our tracking surface. The tracking is realized by using an Anoto pattern printed on a special foil (d) combined with digital pens (a). Anoto-based digital pens are ballpoint pens with an embedded infrared (IR) camera (f) that tracks the pen movements simultaneously. The pen has to be used on a specially printed 600 dpi paper with a pattern of small dots having a nominal spacing of 0.3 mm (see Figure 3) [8].

The maximum size of the pattern that can be successfully printed is A0. However, stitching multiple A0-sized patterns together can result in larger tracking surfaces without any tracking penalties. Once the user touches the board with the pen, the camera tracks the underlying Anoto pattern. It can then derive its absolute coordinates on the pattern and send them to a computer over Bluetooth at a rate of 50 Hz.

The e-board application relies on a custom software solution designed to support the proposed pedagogical design, based on the unified teaching platform (UTP). ETP is designed to support CSCL activities based on 1:1 and 1:3 computing in the classroom, operating with wirelessly interconnected laptops, netbooks, or tablet PCs. The e-board system follows a client? server architecture that involves the following components [9]: server, IWB client, answer submission client, Android device, and Anoto digital pen.

The server provides the session management functionality, state management logic, and content repository managing the persistent storage for questions and answers generated using E-board. IWB Client provides answer visualization capabilities and a collaborative construction space for use with the IWB. Answer Submission Client provides the students with a means to submit their solutions digitally. Digital annotations are stored in the digital pens in a proprietary format. For the sake of interoperability, we have developed (using the SDK available for the DPs) our own library to convert these data to an open standards format (scalable vector graphics, SVG). The Android device is connected to a screen projector by using a digital pen-based dynamic UI/UX dot pattern. Thus, the Anoto digital pen transforms the pen and paper into a digital interface.

4. Next-Generation E-Board

Our goal is to create the next generation of e-boards with the following characteristics: easy to install in existing classrooms, cost effective to integrate in schools and colleges, an easy and intuitive interface, and permission for the organization staff to start using it as they used a standard whiteboard.

For this, we use Android-compatible mini-PC, digital projector, Anoto technology-based wireless digital pen, and adhesive dot pattern screen.

The smart system using next-generation e-boards involves the following steps: write on dot-patterned paper using a digital pen, save information in the server database, write on tablet PC using the touch pen, and check grading results.

4.1 Implementation Plan

The application is planned on an Android platform using Android SDK for the digital pen with the construction of the web server and the DB server for storing the data. To connect a digital pen and the smartphone, Bluetooth technology will be used, as shown in Figure 4.

Our solution instead uses a fully passive screen, which implies that there is no need to maintain or configure a device with big dimensions. Instead of sensors built into the screen, our project uses the “Dot Pattern” technology developed by Anoto in Figure 5 [3?5]. A dot pattern can be printed directly on any surface including paper. This implies that there is no need to ship a whole blackboard with the solution; instead, we can bundle a set of self-adhesive plastic reflective sheets that the customer can attach to his old blackboards in a manner similar to wall paper. These sheets already contain the dot pattern and can be easily replaced if necessary, as they are light, compact, and highly durable [10].

There are also additional benefits of using this approach: the tracking precision is independent of the board size as the dot pattern is always the same scale as compared to the digitizer and does not scale with the surface. Instead, we can just extend the pattern to cover a bigger surface. This solution can be used with any type of projector and projector image size; there is no need to adjust the projected image to precisely cover the full screen surface. Further, multiple digitizer pens can be connected at the same time, and a paper notebook with the dot pattern can be used as secondary input by teachers and students. Maintenance and shipping are easier and cost effective because all the devices and materials are small, inexpensive, and lightweight [11, 12].

The only significant drawbacks of this technology are that the pens need to be recharged periodically and some types of markers that make the pattern invisible to the digitizer pen can create small blind spots on the screen until wiped off. The first can be negated by having an additional pen if the battery life of one is not sufficient to last through the day, and the second is usually not an issue as the digital whiteboard promotes the use of the digital pen instead of classical whiteboard markers.

4.2 Dot Pattern Trajectory Recognition Algorithm

The trajectory recognition algorithm consists of acceleration acquisition, signal preprocessing, feature generation, feature selection, and feature extraction shown in Figure 6. The acceleration signals of hand motions are measured by the pen-type portable device [4].

Signal preprocessing involves calibration, a moving average filter, a high-pass filter, and normalization. First, the accelerations are calibrated to remove drift errors and offsets from the raw signals. These two filters are applied to remove high-frequency noise and gravitational acceleration from the raw data, respectively.

Feature generation signals of each axis (three axes) include mean, correlation among axes, interquartile range (IQR), mean absolute deviation (MAD), root mean square (RMS), VAR, standard deviation (STD), and energy.

Feature selection is carried out not only to ease the computational load but also to increase the classification accuracy. The reduced features are used as the inputs of the classifiers.

Feature extraction is an effective feature extraction method that uses a linear transformation to transform the original feature sets into a lower-dimensional feature space. PNN is a probabilistic neural network used as a classifier for handwritten digit and hand gesture recognition.

4.3 Probabilistic Neural Network

PNN is guaranteed to converge to a Bayesian classifier, and thus, it has a considerable potential for making classification decisions accurately and providing probability and reliability measures for each classification. The most important advantage of using PNN is its high speed of learning. Typically, the PNN consists of an input layer, a pattern layer, a summation layer, and a decision layer, as shown in Figure 7. The function of the neurons in each layer of the PNN can be defined as follows:

  1. Layer 1: The first layer is the input layer, and this layer performs no computation.

  2. Layer 2: The second layer is the pattern layer, and the number of neurons in this layer is equal to NL.

  3. Layer 3: The third layer is the summation layer. The contributions for each class of inputs are summed in this layer to produce the output as a vector of probabilities. Each neuron in the summation layer represents the active status of one class.

  4. Layer 4: The fourth layer is the decision layer.

4.4 Machine to Machine

To manage an interactive whiteboard using an Android gadget, we use themachine-to-machine (M2M) technology. M2M communications enable networked devices and services to exchange information and perform actions seamlessly without the need for human intervention. To better illustrate some entities, we have shown the M2M architecture of our solution in Figure 8. In an M2M ecosystem, a user connects his smartphone, acting as an M2M gateway, to collect information from sensors and M2M devices, over Bluetooth by using an M2M application. The M2M gateway sends the data by using a 3G or 4G network to a network SCL (NSCL), whose main function is to manage the data. In this case, the NSCL stores the data for backup storage purposes and sends the content to an Android-based interactive whiteboard manager program.

5. Web- and Android-Based E-Board System Organization

We will now discuss the user interface of our PenGraphy Board e-board system along with its tools and functions. After the connection to PenGraphy Board is established, a transparency board window will open, as shown in Figure 9. There a is Page Indicate, b is Short Cut Icon Area, c is Floating Menu, and d means additional board area.

To manage PenGraphy Board, the shortcut menu can be used, as shown in Figure 10. Table 1 explains the purpose of each label in the shortcut menu.

In the web-based PenGraphy Board, the shortcut menu can be moved to the left side, to the right side, or in front of an additional board, as shown in Figure 11. In the Android version, it will be opened as an external window that oversimplifies the process.

In Table 2, we compare the web-based and Android-based e-board systems from the point of view of mobility, execution time, dependence on other instruments, convenience, and the count of users. In the first experiment, we compare the mobility of the tools. It is obvious that smartphones are mobile because it is easy to use smartphones anywhere. However, PCs or laptops require a specific place. Next, we consider the average command execution time. Android-based commands execute faster than the corresponding web-based versions because of their mobility. Third, we compare the device dependence on other tools. A PC depends on a keyboard or a mouse, but smartphones have an LCD display that serves as a virtual keyboard or a mouse. Using a smartphone, we can give any command from any location in a room. Next, we compare the display size. It is obvious that PCs or laptops have a larger display than smartphones. In this experiment, the web-based e-board system has more advantages than the Android-based version. Next, we compare the user count of each device. There are twice as many smart phone users as PC and laptop users; this implies that the user count for the Android version will be relatively large.

From the experimental results, we conclude that the use of the Android-based e-board system has more advantages than that of the web-based system.

6. Conclusion

In this paper, we explore the characteristics of a next-generation e-board system using an Android platform and Anoto-based digital pens and technologies required for realizing a smart education system. Here, we considered the web-based and Android-based e-board systems and compared them on the basis of different parameters. During the realization of the project, we plan to use M2M technology to connect an Android gadget and an interactive whiteboard, along with a digital pen trajectory and neural network algorithms described in this paper.

Acknowledgments

This study is supported by Small & Medium Business Administration’s University?Industry Cooperation project called “Development of Electronic Board Writing Program based on Dot-Pattern Digital Pen” (Project number: C0298842).

Figures
Fig. 1.

Framework of a smart education system.


Fig. 2.

Users can interact with the projected image using digital pens from Anoto.


Fig. 3.

Rear projection screen has tiny dots printed on a special foil.


Fig. 4.

Connect a digital pen and smartphone.


Fig. 5.

Anoto technology-based digital pen and dot pattern.


Fig. 6.

Flow of trajectory recognition algorithm.


Fig. 7.

Probabilistic neural network classifier.


Fig. 8.

Machine-to-machine (M2M) communication architecture.


Fig. 9.

PenGraphy Board ?Window (Transparency board) mode layout.


Fig. 10.

Web-based PenGraphy Board shortcut menu.


Fig. 11.

Web-based PenGraphy Board extension menu.


TABLES

Table 1

PenGraphy Board shortcut menu

Label numberNamePurpose
1Convert modeConvert to Transparency or Board mode
2Mouse iconOnly T mode
3Page MoveInstrument used for moving a page
4EraserInstrument used for removing ink from a writing surface
5White penChoose a pen with red color
6Blue penChoose a pen with blue color
7Yellow penChoose a pen with yellow color
8Red penChoose a pen with red color
9Black penChoose a pen with black color
10ExtensionUse additional extension of PenGraphy Board

Table 2

Comparison of web-based and Android-based next-generation e-boards

Compare parameterWeb basedAndroid based
Mobility?+
Average command execution time11 s?3 s+
Dependence on additional instruments (mouse, keyboard)Dependent?Independent+
Display sizeLarge+Small?
Average count of smartphone users /Average count of computer users2 times less?2 times more+

References
  1. Gim, Y, Chung, M, and Kim, J (2013). A study on the actual condition and utilization plan of smart devices for educational purpose. Journal of Korean Society for Internet Information. 14, 47-55.
  2. Vinothkumar, B, Ragavendhira, S, and Vijiyalakshmi, R (2013). Anoto communicator: using digital image processing. International Journal of Emerging Technology & Research. 1, 103-108.
  3. Pen Generations, Digital pen overview. Available http://pengenerations.com
  4. Mahajan, L, and Kulkarni, GA (2015). Digital pen for handwritten digit and gesture recognition using trajectory recognition algorithm based on triaxial accelerometer. IOSR Journal of Electronics and Communication Engineering. 10, 24-31.
  5. Hwang, GJ, Tsai, CC, and Yang, SJH (2008). Criteria, strategies and research issues of context-aware ubiquitous learning. Educational Technology & Society. 11, 81-91.
  6. (2007) . Need funding? Grant providers and winners share tips for getting approved. Available http://downloads01.smarttech.com/media/education/pdf/iewinter06.pdf
  7. (2013) . Teaching and learning in the digital world: possibilities and challenges. Available http://www.learninglandscapes.ca/images/documents/llno12-vfinal-lr-links.pdf
  8. Haller, M, Brandl, P, Leithinger, D, Leitner, J, and Seifried, T 2007. Large interactive surfaces based on digital pens., Proceedings of the 10th International Conference on Humans and Computers (HC-2007), Japan, pp.172-177.
  9. Claudio, A, Marcelo, M, and Miguel, N 2010. CollBoard: supporting new media literacies and collaborative learning using digital pens and interactive whiteboards., Proceedings of the 18th International Conference on Computers in Education, Putrajaya, Malaysia, pp.368-375.
  10. Cho, YI, Andrey, G, Aigerim, A, Zharas, S, and Yergali, D 2015. Android-based dot-pattern recognition technology for e-board system., Proceedings of the 16th International Symposium on Advanced Intelligent Systems, Mokpo, Korea.
  11. Somekh, B, Underwood, J, Convery, A, Dillon, G, Lewin, C, Mavers, D, Saxon, D, and Woodrow, D (2004). Evaluation of the DfES ICT Test Bed Project: Annual Report 2004. Coventry: British Educational Communications and Technology Agency
  12. Kim, HH, Oh, DI, and Heo, G (2014). A study on the research trends of smart learning. Journal of Fisheries and Marine Sciences Education. 26, 156-165.
    CrossRef
Biographies

Aigerim Bakatkaliyevna Altayeva received her B.S., M.Sc., from the Department of Computer Science, IITU, Kazakhstan, in 2012, 2014. Now she is a Ph.D. student at Gachon University. Her interesting part is AI, Smart City, Big data, Information retrieval etc.

Tel: +82-31-750-5800, Fax: +82-31-750-5662,

E-mail: aikosha1703@gmail.com


Young Im Cho received her B.S., M.Sc., and Ph.D from the Department of Computer Science, Korea University, Korea, in 1988, 1990 and 1994, respectively. She is a professor at Gachon University. Her research interest includes AI, Big data, information retrieval, smart city etc.

Tel: +82-31-750-5800, Fax: +82-31-750-5662,

E-mail: yicho@gachon.ac.kr




July 2017, 17 (2)
Full Text(PDF) Free

Services

Funding Information