As competition in the mobile game market intensifies, the number of users accessing specific games decreases and marketing costs increase significantly. Accordingly, marketing methods for increasing the average revenue per user (ARPU), which is the average amount paid by game users over a certain period, are being developed [1]. Current marketing systems save user game behaviors and purchase patterns over long periods. They then learn the stored data and recommend effective items to the user. A long timeframe is needed to accumulate data and train an artificial intelligence (AI)-based model. However, games start to promote the purchase of items from the moment the user starts the game for the first time. Therefore, these methods have the disadvantage of providing marketing even to users who do not plan to purchase game items during the data accumulation and learning periods because time is required to accumulate the data; they also require a large amount of data storage space because they store both item purchaser and non-purchaser data [2]. To reduce these shortcomings, in this paper, we propose a system that analyzes the user’s initial game behavior and classifies whether the user is a future game item purchaser or non-purchaser within a short timeframe. At the beginning of the game, non-purchasers form a significantly higher proportion of total users than purchasers; thus, the data to be processed by the item recommendation system is highly skewed. In this study, we resampled the data to handle this imbalance, and customized function to train the data using neural networks. To effectively classify purchasers and non-purchasers, the performance is evaluated using various AI-based algorithms, such as multilayer perceptron (MLP) [3,4], convolutional neural networks (CNN) [5–7], and long short-term memory (LSTM) [8,9]. The remainder of this paper is organized as follows. In Section 2, we provide a new item purchaser identification system that includes data preprocessing, resampling, and a customized loss function. In Section 3, the proposed scheme is applied to actual game data and its effectiveness is demonstrated by comparing it with various neural networks. Finally, concluding remarks are presented in Section 4.

2.1 Preprocessing

This study used game data obtained from 95,761 users playing SuperPlanet’s mobile game [10], “Guardians of the Video Game,” as shown in Figure 1.

Immediately after first playing the game, it was observed that the number of item non-purchasers was much larger than the number of item purchasers; of the 95,761 players, 2,743 (2.9%) were item purchasers. Therefore, if all other users are classified as non-purchasers, the overall recognition rate becomes 97.1%. When an AI system learns using a database in which the amount of data belonging to each class is unbalanced, the system is highly likely to make biased predictions. To solve this problem, we have used resampling to make the number of training data entries equal in both the purchaser and non-purchaser classes, as shown in Figure 2.

Undersampling is a method of removing more visible majority class instances [2]. Through undersampling, non-purchaser data with a large weight are randomly extracted at the same number as the purchaser data with a small weight. The game item purchaser data were divided in a ratio of 8:2 for training and testing, respectively. For both the non-purchaser and purchaser data, 2,194 users were applied to the training data. The actual percentage of non-purchasers is 97.1%, approximately 34 times the percentage of purchasers at 2.9%. Therefore, the number of test data points is 18,666, which is 34 times the number of purchaser test data points.

The data used consisted of 25 features, such as goods, grades, and the number of wins achieved by each user, which were stored every day for 6 days after the user joined the game. User-purchased items and their purchase amounts were also added to the database for 30 days after registration. These data were obtained from 95,761 new users and saved each time they logged into the game. Missing values occurred when the user did not log in for a day or longer. In this study, we looked for dates on which we could predict a user’s purchasing behavior. Therefore, we created a 6 × 25 feature matrix using the average data from days 1 to N (1–6). We also applied the Pearson correlation coefficient (PCC) [11,12] to determine the correlation between variables. PCC is an analysis tool that can be used to understand the actual influence and characteristics of the variables applied. If the absolute value of the correlation coefficient is 1, the observed value appears exactly along a straight line, and as it approaches zero, it deviates from the straight line. Figure 3 shows the value of the PCC calculated using each feature and whether the item was purchased. In this study, features with coefficients greater than 0.1 are selected and used.

2.2 Custom Loss Functions

In a binary classification problem, the actual value and the value predicted by the model are represented by a 2 × 2 matrix, as shown in Table 1.

The terms “true” and “false” indicate whether the predicted result is correct [13], while the terms “positive” and “negative” indicate those cases in which the model classifies users as purchasers or non-purchasers. Therefore, we can interpret this as follows:

• • True Positive (TP): Number of purchasers correctly estimated as purchasers

• • True Negative (TN): Number of non-purchasers correctly estimated as non-purchasers

• • False Positive (FP): Number of non-purchasers estimated as purchasers

• • False Negative (FN): Number of purchasers estimated as non-purchasers

The classification error is the most widely used index for data learning in neural networks and is defined as

The goal of this study is to create a model with a high probability that the predicted purchaser is the actual purchaser. However, when learning using a loss function, as indicated in Eq. (1), there is a problem in that the predicted purchasers include many non-purchasers. When training the model, we focus more on predicting the purchaser class than on predicting all classes and propose four new loss functions and compare them with Eq. (1). The first loss function is customized to increase the precision, and is defined by

Precision is the percentage of actual purchasers that the model predicts as purchasers. The smaller the loss function is, the more likely the model is to correctly predict whether the user is a purchaser. The second loss function is defined as follows:

Because loss function (2) only considers the precision, it is likely to be biased toward non-purchasers. To solve this problem, the predicted value, which is the percentage of actual purchasers predicted by the model, is not biased when the recall is considered. The third loss function is defined as follows:

Eq. (4) is a simplified version of Eq. (3) which carries out simplification by penalizing incorrect answers instead of providing extra points for correct answers. This loss function is intended to simplify the unexpected effects. The last function is a generalized version of Eq. (4), and is represented as:

where n₁ and n₂ are coefficients and n₁+n₂ = 1. We weighted FP and FN to improve the classification performance; the optimal values were selected experimentally.

In this section, we evaluate the experimental results using the precision and recall values. We focused more on precision because we needed fewer non-purchasers in our predictive purchaser list. The test data consisted of 549 purchasers and 18,666 non-purchasers. The experimental results for determining the coefficients mentioned in (5) are listed in Table 2. In this experiment, data from the first day of game access were used.

We used the MLP [14] shown in Figure 4, whose layers and experimental parameters are listed in Table 3.

When n₁ is 0.1–0.3 and 0.9, all predictions are biased to non-purchasers and cannot be displayed. We obtained the highest precision value when n₁ was 0.75, and n₂ was 0.25. We then compared our loss functions (2), (3), (4), and (5) with the classification error (CE) and binary cross-entropy (BCE), defined by Eqs. (1) and (6), respectively.

where t_i is the ground truth and s_i is the output of the MLP. The data on the first day after connecting to game (1) and the average of the data on the first and second days (1–2) was used.

In addition, the average of the data from the first to the sixth day (1–6) was used. A comparison of the results is presented in Table 4.

The results in Eqs. (2) and (5) show that the best precision is 100. This means that the list of users who predict that the MLP model is a purchaser consists of all the actual purchasers. We can observe that the proportion of actual purchasers among the predicted purchasers is much higher than when the traditional loss function is used. The results of the deep neural networks, that is, CNN and LSTM, are compared with those of MLP and are listed in Table 5. The loss function for all neural networks uses Eq. (5), where n₁ is 0.75.

The comparison in Table 5 shows that MLP and LSTM have the highest precision. CNN has a convolution layer to extract meaningful features from data, but MLP and LSTM start classification without a feature extraction process. We believe that MLP and LSTM have excellent precision because meaningful features are selected from the game data using PCC. Because the number of parameters of MLP is much smaller than the number of parameters used by deep learning-based neural networks, MLP is an excellent choice for game item selection systems. Experiments show that MLP can predict the purchasing propensity over a month-long period using the 1-day data of new users. The layers and parameters of CNN and LSTM are listed in Tables 6 and 7, respectively.

In this study, we presented a method for predicting in-game item purchases by new users who have very recently started playing the game. The proposed system enables effective marketing based on a user’s purchasing propensity over a short period of time. In our game data, 2,743 out of 95,761 users were purchasers, and the proportion of non-purchasers compared to purchasers was extremely large. This is a general trend for all game data, because the number of purchasers is small when the game is first made available. Therefore, this study used an undersampling method to handle the unbalanced data. The loss functions used in neural networks are customized to create a model with a high probability of a predicted purchaser becoming an actual purchaser. In the future, we will proceed with research on applying the latest deep learning-based classifiers, as well as with studies to improve their performance. We also plan to include tasks in the game that can predict the user’s psychological or mental state as external factors. Subsequently, a study will be conducted to predict whether a game item is purchased by combining it with internal factors to increase ARPU.

Table 1. TP, FP, FN, and TN.

Table 2. Comparison of weights based on custom loss function.

Table 3. MLP parameters.

Table 4. Comparison of results based on the loss function.

Table 5. Comparison of the results when using a neural network.

Table 6. CNN parameters.

Table 7. LSTM parameters.