1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51
Image Features based Affective Retrieval employing Improved Parameter and Structure Identification of Adaptive Neuro-fuzzy Inference System Dan Wang1, 2, Ting He 3, Qing Li3, Luying Cao3, Nilanjan Dey4, Amira S. Ashour5, Valentina E. Balas6, Pamela McCauley7, Jiang Xu8, Fuqian Shi3† 1
Tianjin Key Laboratory of Process Measurement and Control, School of Electrical Engineering and Automation, Tianjin University, 300072, P.R. China 2 Wenzhou Vocational & Technical College, Wenzhou, 325035, P.R. China 3 College of Information & Engineering, Wenzhou Medical University, Wenzhou, 325035, PR China 4 Dept. of IT, Techno India College of Technology, West Bengal, 740000, India 5 Department of Electronics and Electrical Communications Engineering, Faculty of Engineering, Tanta University, EGYPT 6 Department of Automation and Applied Informatics, Aurel Vlaicu University of Arad, Arad, 310130, Romania 7 Department of Industrial Engineering and Management Systems, University of Central Florida, Orlando, FL, 32825, USA 8 College of Design and Innovation, Tongji University, Shanghai, 210000, PR China †
Corresponding author, Fuqian Shi, email:
[email protected]
Abstract Affective computing has various challenges especially for features extraction. Semantic information and vocal messages contain much emotional information, while, extracting affective from features of images, and affective computing for image dataset are regarded as a promised research direction. This paper developed an improved adaptive neuro-fuzzy inference system (ANFIS) for images’ features extraction. Affective value of valence, arousal and dominance were the proposed system outputs, where the color, morphology and texture were the inputs. The least square and k-mean clustering were employed as learning algorithms of the system. This improved model for structure and parameter identification of ANFIS were trained and validated. The training errors of the system for the affective values were tested and compared. Data sources grouping and the ANFIS generating processes were included. In the network training process, the number of input variables and fuzzy subset membership function types has been relative to network model under different affective inputs. Finally, well-established training model was used for testing using International Affective Picture System (IAPS). The resulting network predicted those affective values, which compared to the expected outputs. The results demonstrated the effect of larger deviation of the individual data. In addition, the relationships between training errors, fuzzy sample set, training data set, function type and the number of membership functions were illustrated. The proposed model showed the effectiveness for image affective extraction modeling with maximum training errors of 14%. Keywords: Adaptive neuro-fuzzy inference system, structure identification, parameter identification, International Affective Picture System, k-mean clustering.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
1. Introduction Affective activity is intimated from the surrounding environment. It is valuable for human perception and decision making process. Image definitely contains rich emotions to viewers. Image emotion (affecting) analysis essentially focused on affective image classification to find features that can be for further emotions better classification. Generic features can be extracted as low-level features. Interpretable and attributes and state-of-art based features that can be considered as mid-level features, while semantic concepts described by facial expressions can be extracted as high-level features. The similarity of data in the affective retrieval depends on the similarity of impression. Obviously, image retrieval can considerably promote the research toward image understanding. Recently, researchers focused on how to use computer technology to automatically recognize the affective images based on their features. Image sentiment analysis is an important research direction combining with the fields of artificial intelligence, affective computing, image understanding, and pattern recognition. In order to reflect the image affective, the color; texture; and morphology features of image are considered to be feasible for image affective retrieval [1]. Strong reaction in the psychological aspects of the human response to various external stimuli is affection that includes joy, disgust, fear, sadness, surprise, and anger. Since, such emotions are subjective; vagueness; and complex, so complex mapping model between the low-level image features and high-level affective is required. Moreover, improving the accuracy of image affective recognition requires deep description of the image for automated recognition of the person's image to understand the nearest relationship mapping. A significant step before feature extraction is the dimensionality reduction for the images. Affective images were classified by extracting texture features of region of interest (ROI) using wavelet transform in [2] [3]. Janssen et al. [4] conducted several experiments to extract features from the heartbeats which were strongly related to emotions. Color scale-invariant feature transform (CSIFT) was proposed weighted classification algorithms to generate feature vectors and global integration of the HSV color histogram information. Thus, an image was formed in the form of multi-feature of the semantic [5]. Calculating the edges of an image (i.e. point in curvature direction) that led to a higher order affective semantic distinction between "static sense" and "dynamic sense" groups as was acquired [6]. Hayashi et al. [6] used color feature vector and back propagation neural network (BPNN) to form relationship between the image features and affective vocabularies. The output was an image affective recognition by dividing the image into several small pieces, and image affective features are extracted by color vector through calculating the average intensity value of each sub-block. In addition, linear maps strategies were applied to achieve affective recognition [7]. Continuously, Lin et al. [8] defined the images’ features as contrast, directivity, irregularities, uniformity, roughness and linear transform. Dail et al. [9] proposed HSV color components of the image in gray level and co-occurrence matrix as a texture feature quantity. This enabled intension-base image retrieval by analyzing the impact of the mentioned parameter description (soft-hard, warm-cold, grayish). Color, morphology, texture and other features of any image allow producing different affective response. However, the uncertainty and inherent ambiguity of the human affective lead to difficulty in the image affective recognition. Thus, many researchers focused on the image affective features using physiological signals including electromyogram (EMG), electroencephalograph (EEG), skin temperature, optical
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
pulse, skin conductance resistance, and the respiratory signals [10] [11] [12]. Tan et al. [13] collected brainwave signal and used signal feature extraction. The authors proposed an affective spectrum analytical method with BPNN to train the data pairs in order to achieve a good visual image. Fuzziness and uncertainty of have a significant role in the affective. Fuzzy Set is a non-classical set with a number from 0 to 1 to indicate the degree of membership for its elements, including fuzzy logic, fuzzy control and fuzzy reasoning. Combining the fuzzy set approach with the artificial neural networks technology can form a stronger learning and reasoning mechanisms that has a stronger ability to process information. The Artificial Neuro-Fuzzy Inference Systems (ANFIS) are kind of the artificial neural networks that functionally equivalent to fuzzy inference systems. They are employed in several applications. In ANFIS, the parameters of the membership function that designates the system behavior are extracted from a dataset. The ANFIS learns features in the data set and adapts the system parameters in consistent with a specified error criterion. Consequently, the ANFIS system is employed in the current proposed system. It extracts rules from the input and output data using neural network’s automatic learning mechanism, and initially constitutes an adaptive neuro fuzzy controller through the off-line training and on-line learning algorithm for fuzzy inference control under rules self-adjustment. Finally, it directed the system itself towards adaptive, self-organizing, and self-learning. A brief introduction to the theory of two widely used artificial intelligence-neural network and fuzzy reasoning basic theoretical knowledge is presented. Additionally, an integrated system comprised from both the neural networks and fuzzy inference mode system was proposed. The remaining sections are organized as follows. Section 2 includes the related work for the proposed system. In Sect.2, the methods and algorithms of the improved structure and parameter identification of the ANFIS are included. Section 3 introduces the proposed system. Afterward, Sect. 4 introduces case study of image features extraction using the proposed methods and algorithms. Finally, in Sect. 5 the conclusion is conducted. 2. Related Work The fuzzy neural network (neuro-fuzzy system) can be defined as a learning machine that applied to determine the fuzzy system's parameters, such as the fuzzy sets and fuzzy rules by manipulating the approximation procedures from the neural networks. Castillo et al. [14] formed simple universal approximation ability using standard type fuzzy neural network combined with fuzzy K-nearest neighbors and neural networks. Zhang et al. [15] proposed improved inference rules for the pessimistic optimism operation and the fuzzy neural network model compensation. Adeli et al. [16] improved the reliability and robustness of the neural networks system based on fuzzy theories. Furthermore, many applications using genetic algorithms, gradient descent, clustering algorithms, and artificial intelligence with fuzzy neural network were presented. Chow et al. [17] proposed an effective algorithm based on genetic algorithm fuzzy neural network for structure optimization and parameter optimization. Chen et al. [18] used adaptive neural network fuzzy inference system to establish the basic oxygen furnace (BOF) endpoint prediction model. In [19], an image de-noising model that protected the image details and filters the noise was proposed. Soft computing techniques including fuzzy c-means clustering [20], rough set (RS), and particle swarm optimization (PSO) [21], were applied in the ANFIS applications.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49
Turkmen et al. [22] introduced a detection method for the restoration of images corrupted by impulse noise. A novel approach was addressed for representing the EMG signals using the wavelet transform [23] [24]. Additionally, the Principal component analysis (PCA) [25], remote sensing [26], dimensional analysis, and features of the histogram, central tendency of image were developed combined with ANFIS for image process. Since, the semantic information and vocal messages contain much emotional information, so affective computing for image dataset is regarded as a promising research direction especially in medical field [27] [28]. Bozhkov et al. [29] introduced a relevant feature selection approach for neurophysiologic interpretation and validation. Ward et al. [30] detected the affective reactions to Human-computer interaction (HCI) situations and events. The authors illustrated that such reactions can reliably be detected in subtle and natural situations. Affective computing using neuro-network proved its effectiveness and robustness in some cases studies [31] [32] [33] [34]. Moreover, machine learning methods such as support vector machine (SVM) [35] and the unsupervised local deep feature [36], were widely used in affective computing for medical image dataset. Color, texture and morphology of images were the key features for revealing the transform from low level to high semantic [37]. Additionally, the preprocessing for the images was a significant step. In [38] [39], the authors filtered and transformed the data incorporating with Fast Wavelet Transform (FWT) for dimensionality reduction. From the preceding survey, the Adaptive Neuro Fuzzy Inference System (ANFIS), which is a Takagi-Sugeno model (T-S) based fuzzy inference system including fuzzification of fuzzy control, fuzzy reasoning and de-fuzzification can be used. 3. Methodology In the current work, the affective features extracting techniques, fuzzy logic based neuro-networks, and ANFIS based features extracting system using Matlab are proposed. Since, the most basic low-level visual content are the affective feature extraction and dimension reduction of image color, texture and morphology features. Hence, such visual features are used to produce a variety of different affective experience from the image. An image dataset that authorized from the International Affective Picture System (IAPS) is used. The neural network is used in fuzziness, fuzzy inference and de-fuzziness of the ANFIS. The structure and parameters identification is performed for generating the FIS rules automatically. The modeling process is based on the acquired input/output data sets through its online learning ability, such as the self-learning and information storage capabilities of the neural networks. The BPNN and the least square method hybrid learning algorithm are automatically adjusted the antecedent parameters (nonlinear parameter) system and the consequent parameters (linear parameters). Finally, the system achieves a stable and balance status. 3.1 Structure of adaptive neuro-fuzzy inference system The ANFIS is a Takagi-Sugeno based FIS including multi-input/single-output (MISO) and multi-input/multi-output (MIMO). In this work, MISO is considered, thus 2 input/1 output for the sample is shown in Figure 1, where two input variables are x , x and one output. 1
2
1
2
3 4 5 6 7
Figure 1. Two inputs/one output ANFIS The rules are as follows: Rule 1: if x is A and x is B , then f p x q x r 1 1 2 1 1 1 1 1 2 1 Rule 2: f x is A and x is B , then f p x q x r 1
8 9
2
2
2
2
2 1
2 2
2
Therefore, the output is given by: w1 f1 w2 f 2 w1 f1 w2 f 2 (1) w1 w2 Where, w1 and w2 represent the weights of these rules, w and w are the 1 2 proportion of total weights, the output f is the weighted average of f1 and f2. The ANFIS inference model is illustrated in Figure 2. f
10 11 12 13
14
15 16 17 18 19 20 21 22
Figure 2. ANFIS inference model There are five layers of the ANFIS as follows: Layer 1 is a fuzzy layer applying fuzzy process to the input data. The output layer nodes of the available functions are expressed as: O1,i Ai ( X 1 ), i 1, 2
O1, j B j ( X 2 ), j 1, 2
(2)
where, O denotes the outputs of the jth node of the ith layer. The output vale is the i, j
1 2
membership of fuzzy variables of the input signal. Let Ai and Bj are the fuzzy set of the input signals, ( X ) and Bj ( X 2 ) are the relagtive membership function. Ai 1
3
Using the Gaussian function as the membership function, thus:
4 5 6 7 8 9
( xi mi )2
where, {mi , bi } is the parameter set called antecedent parameters. th Layer 2: in this layer, refers to the node. The incentive intensity of the n output of the n th rule is defined as the multiple of all input signals’ memberships that given by: (4) O w ( X ) ( X ), i 1, 2 2, i
10 11 12 13
14 15 16 17 18 19 20 21 22 23
24 25 26
(3)
bi 2
A( x ) e
i
Ai
1
Bi
2
Layer 3: the node in this layer is labeled as N that has no parameters or unadjusted parameters. It is normalized operation for the fitness of the second layer rule, which calculated as follows: wi O3,i wi , i 1, 2 (5) w1 w2 Layer 4: all nodes in this layer use adaptive linear transfer function, and each node represents an output value corresponding to the output of the fuzzy rules as follows: (6) O w f w ( p x q x r ), i 1, 2
4,i
i
i
i
i 1
i
2
i
where, wi is from the third layer, where the parameter set pi , qi , ri is called descendants or conclusion parameters. Layer 5: this layer is defuzzification layer with unadjusted parameters node labeled for calculating total of all outputs from all layers. It is the total system outputs that calculated as follows: i wi f i (7) O5,i wi f i , i 1, 2 wi Although, the antecedent parameters are the static, total inputs of the system, which is the linear combination of the descendants. The final system output is given by: w1 w2 f f1 f2 w1 w2 w1 w2 w1 f1 w2 f 2 ( w1 x1 ) p1 ( w1 x2 ) q1 w1r1 ( w2 x1 ) p2 ( w2 x2 ) q2 w2 r2
27 28 29 30 31 32 33 34 35 36 37
(8) 3.2 Improved parameters and structure identification of the ANFIS 3.2.1 Parameter identification The ANFIS membership functions, types/frequency of training, membership function parameters, including antecedent nonlinear parameters, linear parameters and descendant linear parameters can be optimized. Fuzziness in a fuzzy set is defined by its Membership functions (MFs) that are the main blocks of fuzzy set theory. Since, the MFs affect the fuzzy inference system; hence, the MFs’ shapes are significant for each particular problem. The MFs have several shapes, such as Gaussian, triangular,
1 2 3 4 5 6 7 8 9 10 11 12
13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
trapezoidal, bell-shaped, and S-type functions. However, all the MFs vary between 0 and 1. The Gaussian and the bell MFs are widespread methods for postulating fuzzy sets due to their characteristics of being smooth and nonzero at all points. Thus, they are more appropriate to model the probability distributions derived from the natural phenomena and are used in several applications. Consequently, in the current work the function type is studied and mainly determined by the experimental method. This section, considers the parameter adjustment method for the membership function and develops a neural network based parameter adjustment process for the input and output data set. This process is called the learning algorithm of ANFIS that has the listed properties in Table 1. Table 1. Learning algorithms properties of the ANFIS Learning Process Transfer Signal Condition Conlusion Parameters Parameters Static Least Square Forward-propagation Output of Nodes Error Signals Gradient Descent Static Back-Propagation Typically, the ANFIS is an integrated system using adaptive networks hybrid learning procedure and the fuzzy inference system. It uses supervised learning on learning algorithm that has a function similar to the fuzzy inference system model. Thus, the used number of nodes and the signal error are considered to be the significant parameters in the ANFIS model. As depicted in table 1, the learning procedure employs the least squares estimate (LSE) and back-propagation-type gradient descent to estimate the model parameters. Typically, there are different types of the fuzzy inference system (FIS), namely Mamdani, Takagi–Sugeno, and Tsukamoto. However, the FIS of Takagi–Sugeno model is widely used in the application of ANFIS method, thus it is carried out in the current work. Improved least square estimation The least squares estimation (LSE) algorithm is a forward learning process. Initially, the condition parameters are fixed, and then adjust the conclusion parameters of the output signals. The basic principles of LSE method is calculating the square of the distance and reach the minimum actual output with the expected output data based on a large number of experimental dataset. Normally, the fitness function deviation ( x) in point ( xi , yi ) is i ( xi ) yi (i 1, 2., m) 0 . Thus, in order to find an approximation, reflecting the trend curve of the data points, and the sum of squared deviations usually are required to reach minimum vales. The formula is as follows: m
m
i 2 ( ( xi ) yi )2 i 1
35 36 37 38 39
(9)
i 1
The error cost function is defined as:
1 E = (t y )2 (10) 2 where, t and y are the expectation outputs and actual outputs. The improved estimation algorithms of { pk , qk , rk } is as follows: E E y Ok4 (t y )Ok4 xi pk y Ok4 pk E pk ( j 1) pk ( j ) pk ( j ) (t y )Ok4 xi pk
(11) (12)
1 2
The same operation on adjust equation of qk , rk can be repeated, where i 1, 2 is count of input signals, k 1, 2, r is count of rules.
3 4 5 6 7 8
Gradient descent For condition parameters adjustment, the Gradient descent algorithm is involved after the conclusion parameters optimization process. The system output is expressed by: r
y wk Ok4
(13)
k 1
9 10
The error is given by: 1 (t y ) 2 2 E E By continous calculating for the and , the following is obtained: m j b j E
11
12 13
(14)
E ( t y ) O5 E E O5 k4 5 wk (t y )wk Ok4 O5 Ok4
5=
14 15
r
k3
2j 18 19 20 21 22
23
24 25
26
27 28
k 1 r
( Ok3 ) 2 k 1
16
17
E E Ok4 Ok3 Ok4 Ok3
k4 [ Ok3 Ok3 ]
3 q 2 j
2 (O 2j m j ) 2 E E O 3 2 Oj 3 2 O e O e q O3 O q q p q q p O 2j b2j q
(15)
Here, k 1, 2, r . Assuming that the inputs x1 and x2 have 5 membership functions, then r is 25. When, j 1, 2, , 5 , then i 1, q 5 j 4,5 j 3, ,5 j , p q 5 j 10 . While, when j 6, 7, ,10 , then i 2, q j 5, j , j 5, j 15, p ( q j 10) / 5 . Thus, 2 E E O j 2( xi mi ) 2j 2 m j O j m j bj 2 (16) 2 2 E E O j 2 2( xi mi ) j b j O 2j b j bj3 Subsequently, E m j (k 1) m j (k ) m j (17) E b j (k 1) b j (k ) b j where, j 1, 2,10 and 0 is the learning rate.
1 2 3 4 5 6 7 8 9 10 11 12
3.2.2 K-mean based structure identification of the ANFIS In order to determine the optimum number of the fuzzy rules, and the selection of significant input variables from the input variables, structure identification is employed. Actually, the structure identification is a fuzzy subspace of the input space for ANFIS including fuzzy meshing, subtractive clustering method and fuzzy c-mean [40]. In the fuzzy meshing the distribution of the input data is independent to this method. Input space is initially divided into a number of uniform rectangular using one dimensional input value’s maximum/minimum. Assuming that the number of the fuzzy subset for N-dimensional input variables are pi (i 1, 2,, N ) . Therefore, the n
number of rules of ANFIS is R pi . From this formula it is observed that the rules i 1
13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
33 34 35 36 37
will be increased exponentially when the input data dimension increases. This will result in the "rule explosion", so this method does not apply to the case of high dimensions inputs. Since, the subtractive clustering method is an automatic fast estimation algorithm for a single set of data in the number of clusters and cluster center position coordinates [41]. Moreover, it is a density-based clustering algorithm, where each data point in the data set is potential cluster center, and close to the data size to calculate the density of the point data for judging whether the point is the center of the cluster basis or not. The maximum density of the data points are identified as the first cluster center, then the other points around the point were ruled out as a cluster center point. The end of the clustering process again from the remaining data is the same way to find the center of the next, until all the remaining points as the probability cluster center become less than a threshold set. The parameters of this method-clustering radius determine the number of input data into subspace, namely the number of fuzzy rules. Fuzzy c-mean clustering is based on the partition of a finite collection of n elements Xi(i=1,2,…,n) into a collection of c fuzzy clusters with respect to some given criterion, while the k-means clustering aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean serving as a prototype of the cluster. Typically, the K-means algorithm has several steps until convergence as follows: Algorithm: K- mean algorithm Pseudo-code Start Repeat until stable: Determine the coordinate of centroid Determine the distance of each object to the centroids Find the closest centroid Group the object based on minimum distance End The basic idea of the k-means clustering is to start with a data set that has randomly selected k data points as the initial cluster center, and then calculated for each data point to a distance k cluster centers. The pseudo-code for the special K-mean algorithm based ANFIS is as follows: Algorithm: K- mean algorithm for the ANFIS Pseudo-code Start
Input X and k If center of the cluster changed Select randomly the centers for the cluster Calculate the distance between the points and the cluster center Determine the distance of each object to the centroids Allocate points to the nearest cluster Re-calculate the mean of the cluster Else End if End 1 2 3
Figure 3 illustrated the K-mean algorithm for the ANFIS.
4
5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Figure 3. K-mean algorithm for the ANFIS The k-means clustering is the most popularly and commonly used clustering technique. It is robust, fast, relatively efficient and easy to understand. In accordance with the nearest neighbor rule, Figure 3 illustrated that all the data points are divided into its nearest cluster centers, where clustering is an unsupervised learning procedure collecting similar type of objects into a particular group. In the ANFIS, K points are selected randomly as the centers for the cluster. In order to allocate points to the nearest cluster, the distance between the points and the cluster center is calculated. Afterward, the mean of each cluster is calculated in the new generation of all the points to make it as a new data center point of each cluster, whether the change by comparing the new center and the center of the last resulting position. If the algorithm converges have not changed, this is the center of the resulting final cluster centers; while if a difference according to the new re-division of the center point of all the data points exist, thus this process is repeated until it meets the convergence condition. 4. Proposed System Connections between color, morphology, texture along with the characteristics of human affective using common extraction methods for these characteristics are
1 2 3 4 5 6 7 8 9 10 11 12
employed. High dimensional affective features space is reduced using the principal component analysis (PCA) based dimension reduction algorithm. Since, the artificial intelligence-neural network has high distributed memory parallel information performance and neural network computing. Therefore, it has good fault tolerance, highly nonlinear and self-learning, self-organization and self-adaptive. Furthermore, fuzzy logic has the advantage of dealing with non-accurate data. Fuzzy language information with expert knowledge can simulate the human intelligence judgment and decision making process. Consequently, in this work, a complementary using of both fuzzy logic and neural network for processing fuzzy and complex affective in formation has its unique advantages. Figure 4 demonstrated the image feature extraction proposed system framework using ANFIS.
13
14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42
Figure 4. Framework of the image feature extraction proposed system using ANFIS As illustrated in Figure 4, the International Affective Picture System (IAPS), which is a designed pictures’ database providing a standardized pictures set to study emotion is used in the current study. Color, texture, and morphology features of the image were extracted, where the PCA is employed for dimension reduction. The dimensionality reduction develops degrees of freedom set that can be used to imitate most of the data set variability. It produces a compact low-dimensional set of a specified high-dimensional data set. Afterward, the image feature vectors are obtained as the input vectors to the ANFIS. The ANFIS system is conducted to establish the mapping between the low-level image features and the high-level affective features. Three-dimensional characterization of human affective experience value (valence-arousal-dominance) was taken as the output vector in the ANFIS for training fuzzy neural network. In this process, fuzzy neural network input space is divided. The feature extraction methods of fuzzy parameters and membership functions, training times, training and other precision multiple tests are adopted to find the best accuracy of the proposed models. In the present case study, the affective adjectives were happiness, sadness, anger, and fear. The three-dimensional cluster analysis as k-means clustering method is applied to cluster the result into four classes and to obtain cluster center point of each class. More than 150 sample data is trained and tested. Finally, the test data is used to verify the proposed ANFIS model. 5. Results and Discussion In the current work, the experiment images and the three-dimensional (valence, arousal, dominance, VAD) emotion data obtained from the International Affective Picture Library System (IAPS). The system is a standardized set of emotions and
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
34
35
emotional note prepared by the Research Center Stimulation Photos System, which is composed by US NIMH (National Institute of Mental Health). It provides normative ratings of emotion for a set of color photographs that provide a set of normative emotional stimuli for experimental investigations of emotion and attention. The VAD dimensional value to human emotional experience is described, wherein the valence indicates the degree of excitement or calm wake, arousal represents degree of emotions positive or negative, and dominance represents individuals scenarios and others control state [42] [43] [44]. These three dimensions can not only describe the subjective experience of emotion, but also the better mapping of the performance of its external emotion and physiology [45]. In the IAPS experiment, they selected the 150 images and the corresponding three-dimensional VAD emotional value as the sample data. In this work, image extracted features are considered the input to the proposed ANFIS system, while the corresponding three-dimensional image’s VAD emotional value is the output of ANFIS system.
5.1 Data process 5.1.1 Image features extraction and dimensionality deduction Color, morphology and texture are the extracted visualization features relative to the human affective. Hence, in this experiment, the color histogram, Invariant Hu Moment (IHM), and Gray Level Co-occurrence Matrix (GLCM) are applied for the image extraction. The final extracted features were the 12 color features, 7 morphology features, and 8 texture features. However, it is not a feasible method for these 23 features as inputs to the ANFIS, where the system may generate rule explosion problem in the input/output space partition under such large features set. Furthermore, such large input features will have a great effect on the speed and accuracy of system modeling and training. Therefore, it is necessary to apply data dimensionality reduction method for processing such a 27 dimensional feature vector. Therefore, the PCA is used for dimensionality reduction in the current experiment. The relationship between the data dimension in the new feature space and the contribution rate of the original data to the original data is shown in Figure 5.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Figure 5. Dimensionality reduction for the inputs vector by using PCA In figure 5, the x-coordinate represents the dimensions correspond to the contribution rate of the original vector, and y-coordinate represents the feature vector of the original data and the cumulative contribution rate. The subsequent dimension will not be displayed when the cumulative contribution rate is more than 95%. Thus, as illustrated in Figure 5 that the contribution rate is 92% after four principal components dimension reduction. The first five principal component contribution rates reached more than 95%. Generally, the number of principal components selection can contain most information of the original data and test for the ANFIS modeling, while the cumulative contribution rate reached 90%. It is established that the PCA reduces the dimension of input vector, where the four principal components as inputs of ANFIS are allocated. 5.1.2 Data clustering and grouping With respect to the data clustering, the VAD values are divided in 6 catalogues by using the K-mean clustering. Figure 6 demonstrated the clustering results for the 150 sample data. In addition, the center points of each cluster are listed in Table 2.
20
21 22 23 24
25 26 27
Figure 6. Clustering results of 150 sample data by VAD Table 2. Centers of each cluster and points contained Clusters Centers Points (0.8287 0.3682 0.7496) 16 1 (0.0642 0.8575 0.2152) 37 2 (0.5330 0.2179 0.7520) 37 3 (0.9164 0.5732 0.8432) 18 4 (0.6866 0.7086 0.6492) 28 5 (0.3704 0.5159 0.4951) 14 6 Regarding the data grouping in the ANFIS system, it is required to divide the 150 sample data as training data and test data; respectively. Thus, it is necessary to train
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49
the ANFIS system and to adjust the parameters of the system as well as to validate the system. Randomly grouping may lead to a too small size sample of certain classes due to the differences in the proportion of the clustering results of the six categories while grouping the sample data. It will affect the ANFIS modeling of the effectiveness. Therefore, in this work a stratified sampling to method is used divide the sample data into two groups that labeled by 1-6 as emotion categories. These categories corresponding to the training data and test data number are (11, 32, 16, 24, 32, 12) and (2, 5, 2, 4, 5, 2); respectively. 5.2 ANFIS modeling The ANFIS model is built using R2012b MATLAB fuzzy logic toolbox. The fuzzy logic toolbox is an integrated functions set that can be designed for fuzzy inference system. Consequently, in this work the steps to create the ANFIS system are as follows: Step 1: Setting the parameters of the system environment according to the sample data to determine the training data and testing data. Step 2: The input variables of the experiment are four, while the number of fuzzy subsets of the input variables, membership function type, the number of fuzzy rules, and the generating method of the initial model are configured as follows. The number of fuzzy subsets of the input variables is set according to the data distribution of each variable. The number of the data distribution is more than that of the other; nevertheless the fuzzy subset number should be the fixable for to achieve good expression the features. The excessive ability of the ANFIS knowledge will cause an exponential growth in the rules. Membership function modeling is an important step that used to build a model for the effect of the relatively large impact. Thus, experience and experiments are used to choose the membership function, while Gaussian function was regarded as a feasible selection. The research shows that for the Sugeno model, the nonlinear membership function is more reasonable than that obtained by using the linear membership function. The usual methods for generating the initial model are the mesh generation method and the subtractive clustering method. Fuzzier subset is easy to cause the curse of dimensionality. The system is difficult to bear and the latter can effectively reduce the number of rules. However, the number of membership functions is greatly increased. Subtractive clustering is more suitable for input variables and more training samples. In the current experiment, the initial model is likely to result in structural redundancy by using subtractive clustering. Step 3: Setting up the desired error value of the network and the number of training steps by using the hybrid learning algorithm to generate the initial network training. Step 4: Introducing the test data to the trained adaptive neural fuzzy inference system for the validity of the test. 5.3 Simulation The “anfisedit” function in the Matlab toolbox is used for this Sugeno type fuzzy inference system. It requires 1-rank or 0-rank Sugeno system, single output, and weighted mean based defuzziness. All weighted vale is 1, and the outputs are VAD value, so three ANFIS for VAD are founded.
1 2 3 4 5 6 7 8
9 10 11 12 13 14 15 16 17 18 19 20 21 22
23
5.3 .1 Training Phase Input vector of the ANFIS is dimensionally deducted and labeled as T1, T2, T3, and T4 arousal’s vale A as output. The 130 sample data is used as a training set and 20 sample data is used as the test set. In order to speed the convergence, “mapminmax” is adopted and the data is also normalized between [–1, 1] as depicted in Table 3. Table 3 Training dataset after normalization T2 T3 T4 0.5264 0.4434 0.1083
1
T1 -0.1612
T5 -0.9341
2
-0.8634
0.4217
-0.5831
0.6597
0.0751
3
-0.6976
0.5833
-0.4102
0.4709
-0.0909
4
-0.9362
-0.2516
0.1657
0.4009
0.6153
5
-0.8016
0.6720
-0.7446
0.6850
0.0250
6
-0.6862
0.8390
-0.4952
-0.3601
0.9394
7
-0.7074
0.8285
0.1188
0.0873
-0.9051
8
-1.0000
-1.0000
-0.0929
-0.3306
0.0909
9
-0.7954
0.5589
-0.3875
-0.1685
0.7576
10
-0.8082
0.5385
0.1810
0.1539
0.1331
11
-0.8414
0.2797
0.0857
-0.6834
0.0593
12
-0.9430
-0.3502
-0.1672
-0.2593
-0.8630
13
-0.8688
0.0574
0.2345
-0.3346
0.0777
14
-0.7474
0.6786
0.8087
0.0133
0.5178
15
-0.7290
0.5912
0.2428
0.0099
-0.9473
The initialization model method includes grid division and subtraction clustering. Additionally, the learning algorithms include gradient descent method and hybrid algorithm based on gradient descent and least square. The grid division method is applied in the current work. Since, the simple gradient descent learning algorithm has slow converged of the training network, so the hybrid learning algorithm is selected. The choice of the number of fuzzy subsets and the type of membership function of each variable in the model training are determined by the experiment trials. Primarily, the number of four input variables of the fuzzy subset is 3, 3, 3, and 3. The type of the membership function is bell-type function, while the training time is 80, the expected error is set to 0.0001. The network training error diagram is shown in figure 7(a). However, if the function is changed from the bell-type to Gaussian type, the training error is illustrated in Figure 7(b).
1 2 3 4 5 6 7 8 9
(a) Bell type (b) Gaussian type Figure 7. Training error of Bell and Gaussian function based ANFIS Figure 7 (a) depicted that the training error is 0.0018632 after 80 epochs. Though, in Figure 7 (b), the training error is 0.0011029, which is better than the bell type function under the same training times. However, the system still requires improvement for the training error. The number of the initial input fuzzy is adjusted as 5 and the four subset members are 5, 3, 3, and 3. Thus, the results are illustrated in Figure 8(a) for the Bell type and Figure 8(b) for the Gaussian type function.
10
11 12 13 14 15 16 17 18 19
(a) Bell type (b) Gaussian type Figure 8. Training error of Bell and Gaussian function based ANFIS for 5 inputs Figure 8(a) demonstrated that the training error is 0.0016419, and from Figure 8-(b), it is depicted that the system converges after 50 epochs and the training error is 0.0086169. Since, the later training error is larger than that obtained with the bell type function based ANFIS. Accordingly, the number of fuzzy subset is finally allocated as 6, 4, 4, and 3. The best results for the system output and the training error value are demonstrated in Figure 9.
20
21 22 23 24 25 26 27 28 29 30 31 32
Figure 9. Training error of Bell type function based ANFIS Figure 9 establishes that the training error is 0.00013084 after 48 epochs. 5.3.2 Structure and fuzzy rules The first column is the input layer, including four input variables. The second layer is the fuzzification layer showing the number of each variable of the fuzzy subset. The third layer is the rule layer of 6×4×4×3=228 rules. The fourth layer is the defuzzification layer. Finally, the system outputs are at the fifth layer containing the arousal variable. Figure 10 illustrates the listed part of the rules in the model.
1
2 3 4 5 6 7 8 9 10
Figure 10. Rules for training (partly) 5.3.3 Membership training The ANFIS model adjusts the membership function parameters through the training data network. The following lists the network of four input variables T1, T2, T3, and T4 corresponding to the bell type membership function in the training before and after the change. The left sub-figures represent the training and right sub-figures represent after training as shown in Figure 11.
11
12
(a) T1 membership function
13
14
(b) T2 membership function
15
16
17
(c) T3 membership function
1 2 3 4 5 6 7 8 9
(d) T4 membership function Figure 11membership function change for variables T1-T4 5.3.4 Model Validation A-ANFIS Consider the image features to be the inputs, while the affective-arousal to be the output by using the trained ANFIS model in Section 4.2. The 20 test sample data is normalized into the system, and the obtained test results are shown in Figure 12.
10
11 12 13 14 15 16
17 18 19 20 21 22 23 24 25 26
Figure 12 Testing results for arousal The actual output of the system is compared to the expected output, and the results are shown in Table 4. Table 4 Expectation outputs and system prediction for arousal ID Expectation ANFIS Relative ID Expectation ANFIS Relative Prediction Error Prediction Error （%） （%） 6.1 6.133 0.54 11 5.82 5.816 -0.07 1 3.96 3.980 0.51 12 5.33 5.081 -4.67 2 5.21 5.215 0.1 13 6.59 6.584 -0.09 3 5.31 5.343 0.62 14 3.26 3.222 -1.17 4 6.5 6.554 0.83 15 5.48 5.528 0.88 5 1.87 1.911 2.19 16 4.02 4.15 3.23 6 5.54 5.838 5.38 17 7.34 7.349 0.12 7 5.05 5.083 0.65 18 4.44 4.496 1.26 8 9 6 5.97 -0.5 19 5.94 5.958 0.30 4.646 0.35 20 4.52 4.476 -0.97 10 4.63 Table 4 depicts that the existence of individual data deviation is slightly larger, which caused such errors relative to the training data of the sample set selection, the number of fuzzy subset membership function and type selection. However, the whole system has good prediction efforts. As the same process, the system network training error variation, prediction results and the comparison of expected output for valence and dominance are listed.
V-ANFIS
1 2 3 4 5
For the Valence-ANFIS network structure, there are four input variables of T1-T4, and one output variable, which represent the degree of Valence value. The resulting network training model effects better, while the number of input variables of fuzzy subset in turn takes 3, 3, 3, and 3, the membership function type is Gaussian function, the training error was shown in Figure 13.
6
7 8 9 10
Figure 13 Training errors of V-ANFIS Figure 13 illustrates the training error of 0.00034415 value at the 70 epochs, and the rule set contains 3×3×3×3=81 rules, where 20 of them are used as input to the system. Figure 14 demonstrates the testing results for the Valence-ANFIS.
11
12 13 14 15 16
Figure 14 Testing results for Valence -ANFIS The actual output of the system is compared to the expected output, and the results are shown in Table 5. Table 5. Expectation outputs and system prediction for valence ID Expectation ANFIS Relative ID Expectation ANFIS Relative Prediction Error Prediction Error （%） （%） 1.87 1.868 -0.11 11 5.18 5.178 -0.04 1 4.66 4.809 3.2 12 7.21 7.194 -0.22 2 3.9 3.897 -0.08 13 3.16 3.524 11.52 3 4 8.59 8.201 -4.53 14 4.88 5.161 5.76 1.47 1.676 14 15 7.04 7.099 0.84 5 4.95 5.039 1.8 16 8.59 8.753 1.9 6 4.69 4.528 -3.45 17 1.66 1.824 9.88 7 8.29 8.361 0.86 18 3.16 2.991 -5.35 8 2.06 2.093 1.60 19 5.71 5.762 0.91 9
1 2 3 4 5
10
3.46
3.413
-1.36
20
7.88
7.948
0.86
D-ANFIS Using the Dominance-ANFIS, the number of subset is 5, 3, 3, and 3 after several experiments. In the case of bell-type function, the obtained train error is shown in Figure 15.
6
7 8 9 10 11 12
Figure 15 Training errors of D-ANFIS Figure 15 establishes that the training error is 0.0002457, while the system runs after 85 epochs and system has 5×3×3×3=135 rules, 20 of them are included as input to the system. The test results are shown in Figure 16.
13
14 15 16 17 18 19
Figure 16 Testing results for Dominance–ANFIS The actual output of the system is compared to the expected output, and the results are shown in Table 6. Table 6 Expectation outputs and system prediction for dominance ID Expectation ANFIS Relative ID Expectation ANFIS Relative Prediction Error Prediction Error （%） （%） 2.83 2.61 -7.77 11 4.66 4.30 -7.70 1 4.24 4.31 1.70 12 6.77 6.75 -0.34 2 4.33 4.19 -3.23 13 3.79 4.04 6.52 3 6.5 6.78 4.25 14 6.49 6.49 0.02 4 2.53 2.538 0.32 15 5.73 5.66 -1.22 5
6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44
6.44 4.65 6.63 3.53 4.28
6.34 4.65 6.79 3.46 4.28
-1.55 0 2.44 -2.10 -0.02
16 17 18 19 20
6.28 2.88 4.30 4.97 5.99
6.28 2.87 4.21 4.98 5.98
-0.08 -0.52 -2.02 0.10 -0.12
The results obtained in Tables 4 through 6 establish that the training errors are all less than 14%, which proves the effectiveness of the proposed system. In the current work the FIS of Takagi–Sugeno model was used in the ANFIS method. However, there are different types of the FIS, namely Mamdani, Takagi–Sugeno, and Tsukamoto. Thus, it is recommended to compare the performance of these different methods with the proposed system as a future work. Moreover, another dimensionality reduction approaches can be involved instead of the PCA. In addition, as extension for the proposed system, training the ANFIS and collecting more input vectors as test is recommended as future study to improve the parameters and structure of the ANFIS for affective computing. Furthermore, more dimensional affective values can be prepared for model validation and also for improving the effectiveness of ANIFS system’s learning algorithms. Various studies were conducted on the emotion identification, features extraction, and classification through affective computing applications [46] [47] [48] [49] [50] [51] [52] [53] [54] [55] [56]. Consequently, the proposed system can be applied with different applications in the medical domain for example.
6. Conclusion Affective is an important part of the human intelligence, moreover the affective activity is inseparable from around the environment. Image contains a wealth of affective, human will understand a different image for different affective of products. With the development of computer technology, image analysis attracts more attention; it has a promising research direction combining with artificial intelligence, affective computing, image understanding, and pattern recognition. Due to the ambiguity and imprecision of the human affective, adaptive neuro-fuzzy inference system model can map low-level image features and high-level affective in an effectiveness way. Consequently, the current work introduced the background and significance of the affective image analysis, image feature extraction and affective research status by using fuzzy neural networks. Moreover, it represented the methods of the underlying visual feature extraction and image data dimensionality reduction method for multiple feature fusion. In addition, an improved artificial neural network model-ANFIS was developed along with its learning algorithm and performance. An integration of the neural networks and fuzzy inference mode system was introduced. The IAPS is used in the image experiment and preprocessing of images. The experimental results depicted that the training errors are all cases are less than 14%, which established the effectiveness of the proposed system. References [1] Ningning Liu, Emmanuel Dellandréa, Liming Chen, et al., Multimodal recognition of visual concepts using histograms of textual concepts and selective weighted late fusion scheme, Computer Vision and Image Understanding, 117(5): 493-512, 2013
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56
[2] Loris Nanni, Sheryl Brahnam, Alessandra Lumini, Random interest regions for object recognition based on texture descriptors and bag of features, Expert Systems with Applications, 39(1): 973-977, 2012 [3] Edward H.S. Lo, Mark R. Pickering, Michael R. Frater, John F. Arnold, Image segmentation from scale and rotation invariant texture features from the double dyadic dual-tree complex wavelet transform, Image and Vision Computing, 29(1): 15-28, 2011 [4] Joris H. Janssen, Wijnand A. Ijsselsteijn, Joyce HDM Westerink, Paul Tacken, Gert-Jan de Vries, The tell-tale heart: perceived emotional intensity of heartbeats, International Journal of Synthetic Emotions, 4(1): 65-91, 2013. [5] Yu Ying, Chen Yong, Based on curvature direction characteristic of image emotional semantic classification, Computer System, 18 (2): 121-124, 2009 [6] Hayashi T, Hagiwara M. An image retrieval system to estimate impression words form images using a neural network, .In: Proc of IEEE Int’l Conf. on Systems, Man and Cybernetics-Computational Cybernetics and Simulation, New York, 150-155, 1997 [7] Yoshida K, Kato T, Yanaru T. Image retrieval system using impression words system, // In Proc. of IEEE Int’l Conf. on Systems, Man and Cybernetics, San Diego, 2780-2784, 1998 [8] Lin HC, Chiu CY, Yang S N. Texture analysis and description in linguistic terms, // In Proc. of Asian Conf. on Computer Vision, Melbourne, Australia, 23-25, 2002. [9] Dai Y., Intention-based image retrieval with or without a query image, //In Proc of lnt’l Conf. on multimedia modeling, Australia, 26-32, 2004 [10] T. Abegaz, E. Dillon, J. E. Gilbert, Exploring Affective Reaction during User Interaction with Colors and Shapes , Procedia Manufacturing, 3, 5253 -5260, 2015 [11] P. Zhang, M. Gong, L. Su, J. Liu, Z. Li, Change detection based on deep feature representation and mapping transformation for multi-spatial-resolution remote sensing images, ISPRS Journal of Photogrammetry and Remote Sensing, 116, 24- 41, 2016 [12] Su Yun, Hu Bin, Xu Lixin, etc., EEG data for knowledge modeling and emotion recognition Chinese Science Bulletin, 11 (10): 1002-1009, 2015. [13] Prashant Lahane, Arun Kumar Sangaiah, An Approach to EEG Based Emotion Recognition and Classification Using Kernel Density Estimation, Procedia Computer Science, 48, 574-581, 2015 [14] Oscar Castillo, Patricia Melin, Eduardo Ramírez, José Soria, Hybrid intelligent system for cardiac arrhythmia classification with Fuzzy K-Nearest Neighbors and neural networks combined with a fuzzy system, Expert Systems with Applications, 39(3) : 2947-2955, 2012 [15] Zhang Y Q, Kandal A., Compensatory neuro-fuzzy systems with fast learn algorithms, 9 (1): 83-105, 1998 [16] Adli J and Karim A. Fuzzy-wavelet RBFNN model for freeway incident detection, Journal of transportation Engineering, 126 (6): 464-471, 2000 [17] Zhou Zhijian, Mao Zongyuan, fuzzy neural network structure and parameters optimization based on genetic algorithm, Hunan University of Technology (Natural Science ), 27 (1): 26-32, 1999 [18] Zhou Qin, Chen Jin, Dong Li, ANFIS system applications in the medical field, Chinese Journal of Clinical Thoracic and Cardiovascular housing, 22 (3): 252-256, 2015. [19] Fatma Latifoğlu, A novel approach to speckle noise filtering based on Artificial Bee Colony algorithm: An ultrasound image application, Computer Methods and Programs in Biomedicine, 111(3): 561-569, 2013 [20] A. M. Abdulshahed, A. P. Longstaff, S. Fletcher, A. Myers, Thermal error modelling of machine tools based on ANFIS with fuzzy c-means clustering using a thermal imaging camera , Applied Mathematical Modelling, 39, 1837 - 1852, 2015 [21] H. Jiang, C. Kwong, K. Siu, Y. Liu, Rough set and PSO-based ANFIS approaches to modeling customer satisfaction for affective product design , Advanced Engineering Informatics, 29, 727-738, 2015 [22] I. Turkmen, Efficient impulse noise detection method with ANFIS for accurate image restoration, International Journal of Electronics and Communications, 65, 132 -139, 2011 [23] S. Abbaspour, A. Fallah, M. Lindn, H. Gholamhosseini, A novel approach for removing ECG interferences from surface EMG signals using a combined ANFIS and wavelet , Journal of Electromyography and Kinesiology, 26, 52-59, 2016
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56
[24] R. Gupta, K. ur Rehman Laghari, T. H. Falk, Relevance vector classifier decision fusion and EEG graph-theoretic features for automatic affective state characterization , Neurocomputing, 174, Part B, 875 -884, 2016 [25] R. Sharma, M. Patterh, Optik , A new pose invariant face recognition system using PCA and ANFIS, International Journal for Light and Electron Optics, 126, 3483 - 3487, 2015. [26] K. A. Bhandari, R. M. Ramchandra, An Innovative Remote Sensing Image Retrieval Techniques Based on Haar Wavelet-LTRP and ANFIS, Procedia Computer Science, 79, 391-401, 2016 [27] W. Dai, D. Han, Y. Dai, D. Xu, Emotion recognition and affective computing on vocal social media , Information & Management, 52, 777 -788, 2015. [28] M. Kaminskas, F. Ricci, Contextual music information retrieval and recommendation: State of the art and challenges , Computer Science Review, 6, 89 -119, 2012 [29] L. Bozhkov, P. Georgieva, I. Santos, A. Pereira, C. Silva, EEG-based Subject Independent Affective Computing Models , Procedia Computer Science, 53, 375 - 382, 2015 [30] R. Ward, P. Marsden, Affective computing: problems, reactions and intentions , Interacting with Computers, 16, 707 - 713, 2004. [31] J. Vallverd, M. Talanov, S. Distefano, M. Mazzara, A. Tchitchigin, I. Nurgaliev, A cognitive architecture for the implementation of emotions in computing systems, Biologically Inspired Cognitive Architectures, 15, 34- 40, 2016 [32] A. Hussain, E. Cambria, B. Schuller, N. Howard, Affective neural networks and cognitive learning systems for big data analysis , Neural Networks, 58, 1- 3, 2014 [33] Y. Rao, Q. Li, L. Wenyin, Q. Wu, X. Quan, Affective topic model for social emotion detection , Neural Networks, 58, 29 - 37, 2014. [34] Y. Jiang, C. Liu, J. Ma, BYY harmony learning of t-mixtures with the application to image segmentation based on contourlet texture features, Neurocomputing, 188, 262 -274, 2016. [35] I. Sarafis, C. Diou, A. Delopoulos, Online training of concept detectors for image retrieval using streaming clickthrough data, Engineering Applications of Artificial Intelligence, 51, 150 -162, 2016 [36] Haralick, RM Statistical and structural approaches to texture, Proceedings of IEEE, 67 (5): 786-804, 1979 [37] A. Vlachostergiou, G. Caridakis, S. Kollias, Investigating Context Awareness of Affective Computing Systems: A Critical Approach , Procedia Computer Science, 39, 91 - 98, 2014. [38] A. ElAdel, M. Zaied, C. B. Amar, Fast beta wavelet network-based feature extraction for image copy detection, Neurocomputing, 173, Part 2, 306 -316, 2016. [39] M. Ponti, T. S. Nazar, G. S. Thum, Image quantization as a dimensionality reduction procedure in color and texture feature extraction, Neurocomputing 173, Part 2, 385 -396, 2016. [40] Hamid Taheri Shahraiyni, Sahar Sodoudi, Andreas Kerschbaumer, Ulrich Cubascha, A new structure identification scheme for ANFIS and its application for the simulation of virtual air pollution monitoring stations in urban areas, Engineering Applications of Artificial Intelligence, 41, 175-182, 2015. [41] Chiu S L. Fuzzy model identification based on cluster estimation, Journal of Intelligent and Fuzzy Systems, 2 (3):267-278, 1994. [42] Mebrabian, A., Framework for a comprehensive description and measurement of emotional states, Genetic, Social, and General Psychology Monographs, 121 (3): 3392-3611, 1995. [43] Mebrabian, A. Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in Temperament, Current Psychology: Developmental, Learning, Personality, Social, 14 (4):2612-2921, 1996. [44] Mebrabian, A., Wihardja C, Ljunggren E. Emotional correlates of preferences for situation activity combinations in everyday life, Genetic, Social, and General Psychology Monographs, 123 (4): 4612-4771, 1997. [45] Ting He, Luyin Cao, Valentina E. Balas, Pamela McCauley, Fuqian Shi, Curvature manipulation of the spectrum of Valence-Arousal-related fMRI dataset using Gaussian-shaped Fast Fourier Transform and its application to fuzzy KANSEI adjectives modeling, Neurocomputing, 174, Part B, 1049-1059, 2016. [46] Joost Broekens, Modeling the experience of emotion, International Journal of Synthetic Emotions, 1(1):1-17, 2010.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
[47]Hatice Gunes, Automatic, dimensional and continuous emotion recognition, International Journal of Synthetic Emotions, 1(1), 2010. [48] Roddy Cowie, Gary McKeown, Ellen Douglas-Cowie, Tracing emotion: an overview, International Journal of Synthetic Emotions, 3(1): 1-17, 2012. [49] Marcello Mortillaro, Ben Meuleman, Klaus R. Scherer, Advocating a componential appraisal model to guide emotion recognition, International Journal of Synthetic Emotions, 3(1): 18-32, 2012. [50] Raquel del Moral, Jorge Navarro, Rafael Lahoz-Beltra, P. C. Marijuán, Cognitive and Emotional Contents of Laughter: Framing a New Neurocomputational Approach, International Journal of Synthetic Emotions, 5(2): 31-54, 2014. [51] Joseph A. Mikels, Barbara L. Fredrickson, Gregory R. Larkin, Casey M. Lindberg, Sam J. Maglio, and Patricia A. Reuter-Lorenz, Emotional category data on images from the International Affective Picture System, Behavior research methods, 37(4): 626-630, 2005. [52] Jana Machajdik, and Allan Hanbury, Affective image classification using features inspired by psychology and art theory, In Proceedings of the 18th ACM international conference on Multimedia, pp. 83-92. ACM, 2010. [53] Joscha Bach, A framework for emergent emotions, based on motivation and cognitive modulators, International Journal of Synthetic Emotions (IJSE), 3(1): 43-63, 2012. [54] David K. Grunberg, Alyssa M. Batula, Erik M. Schmidt, and Youngmoo E. Kim, Synthetic emotions for humanoids: perceptual effects of size and number of robot platforms, International Journal of Synthetic Emotions (IJSE), 3(2): 68-83, 2012. [55] Noor Akhmad Setiawan, Fuzzy Decision Support System for Coronary Artery Disease Diagnosis Based on Rough Set Theory, International Journal of Rough Sets and Data Analysis (IJRSDA), 1(1): 65-80, 2014. [56] Hye-rin Kim, and In-kwon Lee, Combining Image Databases for Affective Image Classification, ACHI 2015: The Eighth International Conference on Advances in Computer-Human Interactions, pp.211-212, 2015.
