Probabilistic Modeling of Swarachakra Keyboard for ...

33 downloads 0 Views 327KB Size Report
reduce touch errors on an Indic script keyboard – Swarachakra. As of ... the finger leading to one of the most primary error called the 'fat finger problem' [4].
Probabilistic Modeling of Swarachakra Keyboard for Improved Touch Accuracy 1

2

Nikhil Wani , Adarsh Patodi and Sumit Singh Yadav 1

3

Vishwakarma Institute of Technology, Pune, India [email protected]

1

ITM University, Gwalior, Madhya Pradesh, India Indian Institute of Technology (IIT), Bombay, Mumbai, India 2

3

2

3

{adarshpatodi , ssysumitsingh }@gmail.com

Abstract. We present a probabilistic machine learning approach to reduce touch errors on an Indic script keyboard – Swarachakra. As of now the model is built purely based on the keyboard model, which extends to a probabilistic model, and is functionally independent of the language model. It is learned using 18,240 recorded touch inputs for which it uses a Naive Bayes classifier and assigns an adapted probability distribution to each of the 39 class labels, i.e. the keys on the keyboard. We show that a comparative reduction of error rate by 7.47% against the Non-Probabilistic model and 1.15%-3.15% against the baseline Swarachakra model was obtained when modeled using a probabilistic approach. Looking into the future, a hybrid model with incorporation of a language model will be designed to factor in with the keyboard model which may further meet user specific needs. Keywords: Touchscreen text input, Machine learning, Classification.

1

Introduction

In this paper, we propose a data-driven probabilistic approach to touch. We treat the problem at the intersection of a HCI approach and a Machine learning task where we are interested in assigning the correct probability value to the user input touch by mapping it to the intended touch input. Given that the text input in Indic scripts often involve typing a consonant and a vowel in combination and handling of the ‘chakra’, i.e. a circular input that appears on touch [2], is effective in context of the Indic keyboard, makes the first touch of the chakra naturally important. In recent analysis study, [6] reported an initial moderately high error rate of 13-18% on average which stabilizes to 68% as user session keep increasing and also suggest room for improvement in the corrected error rates. We identify that the challenge arises when the user

© Authors 2017, All Rights Reserved INTERACT 2017 Adjunct Proceedings

touches between two keys, which are adjacent to each other while being equidistant from the centers and when the keyboard needs to decide the selection of the key that the user wanted to type amongst those as shown in figure 1.

Fig. 1. User’s touch (red) being equidistant (d1 = d2) from the centers of keys (purple)

Hence, there is a clear need for techniques which handle this situation probabilistically and facilitate accurate input.

2

Related Work

While typing the intended key, the user touch gets displaced due to the size of the finger leading to one of the most primary error called the ‘fat finger problem’ [4]. This leads to ambiguous touches because of the size and softness of the touches by the fingertip of the users. Rashid, Daniel R et al [5] describes a ‘relative keyboard’, where keystrokes are treated as inputs in a continuous space relative to each other and uses the Keyboard model. Weir, Daryl, et al [3] proposed a machine learning approach using Gaussian regression to form a function, which predicts the touch coordinate of the intended touch. Here we address a similar form of problem where the task is considered to be a classification problem rather than a regression problem.

3 3.1

Methodology Touch as a Classification Problem

We overcome the problems discussed above by introducing a machine learning technique that looks at them as a classification problem and allows us to generate a probabilistic model based on the device’s reported touch locations. Hence, our task is to find the probability mapping between the two dimensional touch coordinates and the corresponding intended two dimensional touch coordinates on the 480 × 800 pixel device. After having analyzed user’s touches for each of the keys, as mentioned below in section 3.3, it was observed that most of the keys followed a Gaussian distribution, i.e. the probability of the center pixel of the key is the highest and reduces as we move away from the center towards the edges of the key. We therefore turn to a conditional probability based Naive Bayes Classifier [1] which would naturally learn this behavior better and eventually, assign probability score for each of the 39 class label during testing.

3.2 Data Generation and Collection considering User Touch Behavior In order to analyze the touch error on the Non-Probabilistic model, we identified factors that may account for introduction of errors in users’ touch behavior. 3 input hand postures – one thumb, two thumb and index figure input were considered. A corpus of 304 bi-grams was created which incorporated all the 38 character keys. Special symbols were grouped into a th single class, which was considered the 39 key.

Fig. 2. 8 angles of approach

Fig. 3. User text input with 3 hand postures – single

for the character key - द and क

thumb (left), double thumb (center) and index finger (right) using the Android tool.

8 angles of approach for each of the 38 keys were considered as shown in the

Figure 2. For e.g., - for key द following were the associated bigrams- जद, डद,

ढद, थद, धद, बद, भद and लद .The order in which these 304 bigrams were presented was randomized so that the reported error rates are accurate and also so that the users don’t get used to the word pattern being shown on the data collection tool. We used an Android user input entry test tool, with the interface as shown in Figure 3, to record user-typing behavior on Swarachakra’s Staggered layout with 10 novice users who participated for 33 sessions each, spread over 1-2 weeks. A mix of 6 right handed and 4 left handed users was considered. Experiments were conducted with a smartphone using the portrait screen layout with screen size 4.30-inch. The tool presented 304 bi-grams in random pattern and it hide user’s pressed character with a ‘*’ as shown in Figure 3(center and right) so that the typing speed of the user doesn’t consciously get affected upon encountering with an error. Total Number of Taps recorded (18,240 taps) = 38 character words * 8 angles of approach * 3 hand postures * 10 users *2 bi-grams. A corpus of 18,240 touch points was generated for modeling the probabilistic model using the Naive Bayes classifier. A heat map of these user touches is shown in Figure 5 (left).

3.3

Probabilistic Model

We first find the width and height of each key in the keyboard layout and generate the center pixel of each key. Initially, each pixel of the key would have equal probability of being pressed as shown in the Figure 4 (left). The training of the model has two phases, first phase deals with modeling each of our pixels in all of the keys to follow Gaussian Distribution over the key, i.e. the probability of the center pixel of the key would be 1(highest) as shown in Figure 4 (right) and would reduce as we move away from the center towards the edges of the key. Then we move towards the second phase where we retrain the classifier by adding to the obtained distribution, which allows the

Fig. 4. Prior probability (left) and posterior

Fig. 5. Left shows the plotted recorded

probability, training phase 1 (right) for key-क. points of the user taps. Right – shows the boundaries of the probabilistic model.

model to be influenced by the recorded 18,240 taps to reassign the distribution to meet and learn from the users’ touch. (x - xc)

2

+ (y - yc)

2

< r

2

Eq. 1

In the equation above, (xc, yc) represent the center point of the key. For training phase 1, we decide the radius (r1, r2, r3, r4..rn) to assign the right probability value for the pixel which the classification model will first learn. For e.g., if r1 = 2, then p( | क) = 1, if r2 = 4, then, p( | क) = 0.8, if r3 = 6 then p( | क) = 0.6 and if r4 = 8 then p( | क) = 0.4. This would allow the classifier to learn the Gaussian distribution for pixels, which is the desired functionality that the keyboard would incorporate. Any pixel beyond the r4 and at the corners of the circle would have low probability (i.e. < 0.4) value of belonging to that key and become the prime candidates post-training for being assigned a changed probability value since we expect the trained model to decide these values which are also influenced by the change in post-training boundary shape of the key. Figure 4(right) shows the

distribution using the shades of blue, darker shade at the center indicating highest probability of a touch being key - क. The retraining of current model, in training phase 2, with the recorded user touches introduces a shift in the initial circle shape of the probability and resulted in the non-circular boundaries shown below in Figure 5(right). Hence, this results in reduced error rates and builds a novel approach for recognizing key presses. The retrained model was also experimented using 10 folds cross validation to achieve results which are described in the next section.

4

Results

As shown in Table 1, first an error rate with the Non-Probabilistic model was obtained by simply comparing the reported touch coordinates with the intended key’s boundary pixels, i.e. the area that defines the key and checks if they lie inside the boundary of the key. Thus, here for three postures the Error rate is calculated as follows Error rate (%) = Total number of wrong touches × 100 Total number of touches

Eq.2

Total Error rate is the combined error rate, i.e. when we combine the data from all the 3 postures – Double thumb, Single thumb, Index finger and then calculate the error rate using the equation 2. The baseline Swarachakra model [6] reported an initial 13-18% error and then 6-8% stable error rate as the number of typing sessions increased. Following table summarizes the empirical results that were observed. There was a slight increase in errors when the model was training used SVM classifier [7] and when we looked at the experiment as a multi class classification task. Table 1. Comparative results of reduction in touch errors rate. Model Type Non-Probabilistic Model SVM Classifier Swarachakra Model(Baseline) Probabilistic Model + Naive Bayes Classifier

Double Thumb (%) 15.26 17.69 -

Single Thumb (%) 11.24 14.71 -

Index Finger (%) 10.5 13.47 -

Total Error Rate (%) 12.32 15.29 6–8

6.25

4.49

3.81

4.85

It was also observed that users committed more errors when they used double thumb. The Naive Bayes outperformed the SVM classifier in this case, since the training data followed a Gaussian distribution initially and then adjusted the distribution according to the user data.

5

Conclusion and Future Scope

In this paper we demonstrated the feasibility of looking at touch as classification problem using Naive Bayes approach and analyze 18,240 recorded touch inputs from three hand postures – double thumb, single thumb and index finger. We show that users are most prone to commit errors when using the double thumb. A comparative reduction of total error rate by 7.47% against the Non Probabilistic model and 1.15%-3.15% against the baseline Swarachakra model was obtained when modeled using a probabilistic approach. Looking into the future, an Indic language model will be incorporated with this probabilistic model to factor in with the keyboard model and may further meet user specific needs while increasing overall accuracy. Acknowledgements. We’d like to extend our thanks to our guide Prof. Anirudha Joshi and Manjiri Joshi, Prof. Girish Dalvi from Industrial Design Center, IITB for their endless support. We’d also like to thank Vivek Joseph Paul for the Android tool and Indradyumna Roy for considerable help.

References 1. 2.

3.

4.

5.

6.

7.

Rish, I.: An empirical study of the naive bayes classifier. In: IJCAI 2001 workshop on empirical methods in artificial intelligence. vol. 3, pp. 41–46. IBM (2001) Joshi, A., Dalvi, G., Joshi, M., Rashinkar, P., Sarangdhar, A.: Design and evaluation of devanagari virtual keyboards for touch screen mobile phones. In: Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services. pp. 323–332. ACM (2011) Weir, D., Rogers, S., Murray-Smith, R., L¨ochtefeld, M.: A user-specific machine learning approach for improving touch accuracy on mobile devices. In: Proceedings of the 25th annual ACM symposium on User interface software and technology. pp. 465–476. ACM (2012) Baudisch, P., Chu, G.: Back-of-device interaction allows creating very small touch devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 1923–1932. ACM (2009) Rashid, D.R., Smith, N.A.: Relative keyboard input system. In: Proceedings of the 13th international conference on Intelligent user interfaces. pp. 397–400. ACM (2008) Dalvi, G., Ahire, S., Emmadi, N., Joshi, M., Joshi, A., Ghosh, S., Ghone, P., Parmar, N.: Does prediction really help in marathi text input?: empirical analysis of a longitudinal study. In: Proceedings of the 18th International Conference on HumanComputer Interaction with Mobile Devices and Services. pp. 35–46. ACM (2016) Mayoraz, E., Alpaydin, E.: Support vector machines for multi-class classification. Engineering Applications of Bio-Inspired Artificial Neural Networks pp. 833–842 (1999)