Internet addiction assessment tools - Wiley Online Library

bs_bs_banner

REVIEW

doi:10.1111/add.12202

Internet addiction assessment tools: dimensional structure and methodological status Catherine L. Lortie & Matthieu J. Guitton Institut Universitaire en Santé Mentale de Québec, Quebec City, QC, Canada and Faculty of Medicine, Laval University, Quebec City, QC, Canada

ABSTRACT Aims Excessive internet use is becoming a concern, and some have proposed that it may involve addiction. We evaluated the dimensions assessed by, and psychometric properties of, a range of questionnaires purporting to assess internet addiction. Methods Fourteen questionnaires were identified purporting to assess internet addiction among adolescents and adults published between January 1993 and October 2011. Their reported dimensional structure, construct, discriminant and convergent validity and reliability were assessed, as well as the methods used to derive these. Results Methods used to evaluate internet addiction questionnaires varied considerably. Three dimensions of addiction predominated: compulsive use (79%), negative outcomes (86%) and salience (71%). Less common were escapism (21%), withdrawal symptoms (36%) and other dimensions. Measures of validity and reliability were found to be within normally acceptable limits. Conclusions There is a broad convergence of questionnaires purporting to assess internet addiction suggesting that compulsive use, negative outcome and salience should be covered and the questionnaires show adequate psychometric properties. However, the methods used to evaluate the questionnaires vary widely and possible factors contributing to excessive use such as social motivation do not appear to be covered. Keywords Assessment, dimensional structure, DSM-V, internet addiction, problematic internet use, questionnaire. Correspondence to: Matthieu J. Guitton, CRIUSMQ, 2601 chemin de la Canardière F-6517, Québec, QC, Canada G1J 2G3. E-mail: [email protected] Submitted 16 March 2012; initial review completed 19 June 2012; final version accepted 21 March 2013

INTRODUCTION Due to technological progress and enhanced accessibility to information technologies, the last two decades have witnessed a considerable increase in internet use and related activities. However, this remarkable growth of internet consumption is paired inextricably with the rise of its excessive use. Dysfunctional internet use is associated with excessive computer use, involving losing a sense of time while on-line and developing a growing need to be connected more often and for longer periods of time, resulting in deleterious outcomes in real life and in increased offline social isolation [1,2]. People who are thought to experience internet ‘dependence’ lose control over their internet use and suffer from feelings of withdrawal, conflicts and negative life consequences in a way similar to that observed for chemical or behavioral addictions [3–5]. This compulsive behavior could become a © 2013 Society for the Study of Addiction

societal concern as it particularly strikes adolescents and children, who are more exposed to internet use than adults and consequently more vulnerable [6,7]. Despite heated debates on the nature of excessive internet use and whether or not it involves addiction [8,9], consensus is emerging regarding the existence of this problematic behavior [10]. What is considered internet addiction by some has been associated with physiological manifestations related commonly to dependences. For instance, excessive internet users show altered dopamine release in the nucleus accumbens [11] and reduced availability of dopamine D2 receptors in the striatum [12]. In addition, the difference between the scores of subjects addicted to the internet and normal controls to various assessment questionnaires were associated with structural and functional changes of the brain in previous neuroimaging studies. For instance, a decrease in gray matter density of adolescents addicted to Addiction, 108, 1207–1216

1208

Catherine L. Lortie & Matthieu J. Guitton

the internet was found using voxel-based morphometry [13]. In addition, multiple microstructure abnormalities were found in internet-dependent subjects [14]. These structural changes correlated significantly with the duration of the problematic behavior [14]. Furthermore, many functional magnetic resonance imaging (fMRI) studies reported enhanced activity in various brain regions of excessive internet video-game players compared to controls in response to internet video-game cues [15–18]. The patterns of activation, similar to those observed during cue-induced craving in patients with substance dependence, decreased following pharmacological treatment (i.e. 6 weeks of bupropion sustained release), as did craving symptoms and total game play time [15]. Subjects labeled as addicted to the internet have enhanced reward sensitivity and decreased loss sensitivity [19], and showed lower impulse control than the normal population during a Go/No-Go task [20]. In addition, a whole brain voxel-wise analysis showed lower fractional anisotropy throughout the brain of heavy internet users than controls, suggesting that white matter integrity could be a potential target to assess the effectiveness of interventions [21]. Taken together, these findings suggest that what is thought to be internet addiction might involve the same neurobiological mechanisms than those observed in ‘standard’ disorders of dependency [15,18]. As an abusive behavior, affecting multiple aspects of functioning (social, cognitive and physical), excessive internet use could eventually be recognized by the scientific community as an addiction [2,22]. A challenge for the coming years will be to address the need for reliable questionnaires to study and assess this little-known condition [2]. Our aim in this study was to develop a theoretical framework for optimizing an excessive internet use assessment questionnaire design. First, we conducted a systematic review of the questionnaires available in the literature in order to observe the present status of excessive internet use assessment in adolescents and adults, with a particular focus on their validation technique, differential power and other methodological aspects. We then measured the descriptive properties of the selected questionnaires and characterized their dimensional structure. A global dimensional structure emerged, reflecting the strategies used currently by research and clinical communities to understand and evaluate what may be internet addiction. Finally, an in-depth analysis of this dimensional structure allowed us to point out conceptual improvements that could be applied to questionnaires purporting to assess internet addiction and to suggest strategies aiming to optimize this disruptive behavior evaluation and clinical management. © 2013 Society for the Study of Addiction

INTERNET ADDICTION ASSESSMENT Questionnaires play a central role in the clinical screening of pathological behaviors and are also used widely in research contexts [23]. Indeed, researchers must distinguish between pathological subjects and normal controls before comparing the participants’ results on various measures. Excessive internet use is not an exception, and is assessed currently using self-report questionnaires. Therefore, we first conducted a systematic review of the questionnaires that supposedly evaluate internet addiction and analyzed their descriptive characteristics and methodological aspects. Questionnaire selection A systematic search of Medline and Google Scholar using the keywords ‘internet addiction questionnaire’, ‘internet addiction assessment’, ‘internet problematic use questionnaire’, ‘internet problematic use assessment’, ‘pathological internet use questionnaire’, ‘pathological internet use assessment’, ‘internet dependency questionnaire’, ‘internet dependency assessment’, ‘cyber addiction questionnaire’ and ‘cyber addiction assessment’ was performed to identify existing questionnaires assessing internet addiction. More than 360 papers were identified, published between January 1993 and October 2011. In order to detect the original papers in which the questionnaires were first described, used and/or validated, papers were omitted if they did not have a questionnaire name or acronym appearing in the title and/or the abstract. An initial sample of 30 questionnaires with a questionnaire name or acronym appearing in the title and/or the abstract was selected for further analyses. Within this initial sample, four questionnaires were not published in peer-reviewed papers, three papers were not published in English, five questionnaires did not assess excessive internet use per se, and used instead pathological gambling or substance dependence criteria, and four questionnaires had no dimensional structure or variables of addiction. Thus, the final sample was composed of 14 questionnaires (Table 1; for a list of the instruments excluded, see Table S1). Descriptive analysis The total number of questionnaire factors varied from two to nine [4.79 ⫾ 0.54; when applicable, results are presented as mean ⫾ standard error of the mean (SEM)], with the total number of items ranging from 10 to 44 (23.64 ⫾ 2.55, with 5.77 ⫾ 0.87 items per factor, Fig. S1). All questionnaires used Likert-type scales, ranging from 2 to 10 points (5.84 ⫾ 0.42 points). A standard questionnaire format did not emerge from our descriptive analysis. Addiction, 108, 1207–1216

© 2013 Society for the Study of Addiction

14

21

29

Compulsive Internet Use Scale (CIUS) [4]

Game Addiction Scale (GAS) [22]

Generalized Problematic Internet Use Scale (GPIUS) [25] Generalized Problematic Internet Use Scale 2 (GPIUS2) [26] Internet Addiction Test (IAT) [27,28] Internet Consequences Scales (ICONS) [29]

10

Inter-item: r = 0.13a–0.60

Inter-factor correlation: r = 0.83 to 0.88b, P < 0.05; Dc2 of factor covariance = 47.05, 4.95 and 8.74, all P < 0.05 Inter-factor correlation: r = 0.61; Dc2 of factor covariance = 42.90, P < 0.01

Time spent: r = 0.76, P < 0.01 and addiction potential: r = 0.30, P < 0.05 Rejection sensitivity: r = 0.35, loneliness: r = 0.33, procrastination: r = 0.33 and internet attitude: r = 0.56, all P < 0.001; time spent: r = 0.30 and depression: r = 0.20a, both P < 0.05 YDQ’s criteria: r = 0.72, on-line time per week: r = 0.46, time per session: r = 0.37, days per week: r = 0.21a, time on-line between 18 and 23 hours: r = 0.39, and time on-line between 23 and 8 hours: r = 0.41, all P < 0.01 OCS: r = 0.61, P < 0.0001, depression: r = 0.29a and loneliness: r = 0.27a Session duration: r = 0.35, total hours spent: r = 0.48, depression: r = 0.18a, anxiety: r = 0.29a, stress: r = 0.22a and DSM adapted pathological gambling criteria: r = 0.40, all P < 0.01 Academic performance: r = -0.25a, P < 0.05

Personal internet use: r = 0.30, overall internet use: r = 0.22a and duration of use: r = -0.18a, all P < 0.05

Time spent: r = 0.57, loneliness: r = 0.25a, life satisfaction: r = -0.24a, social competence: r = 0.19a and aggression: r = 0.23a, all P < 0.001 Depression: r = 0.26a, self-esteem: r = -0.22a, loneliness: r = 0.20a and shyness: r = 0.25a, all P < 0.05

Time spent: r = 0.42, experience of problematic internet use: r = 0.45, feeling of being addicted: r = 0.52 and OCS: r = 0.70, all P < 0.001

Convergent validity

Test–retest: r = 0.81; split-half a = 0.83

Split-half a > 0.57c

Test–retest: r = 0.83

Test–retest: r = 0.75, 0.65 and 0.75, all P < 0.001

Reliability

a = 0.85

a > 0.60c

a = 0.93

a = 0.94

a = 0.90

a = 0.94

a = 0.88

a > 0.73

a > 0.54c

a = 0.91

a > 0.78

a = 0.93

a = 0.90

a > 0.81

Internal consistency

This Table presents the questionnaire validity and reliability as measured by several statistical methods. Questionnaires that do not attain the criteria generally agreed upon by the scientific community are indicated. ar is small from 0.1 to 0.3, medium from 0.3 to 0.5 and strong from 0.5 to 1.0. bInter-factor correlation over r = 0.71 is non-discriminant, meaning that more than 50% of the variance is shared between the factors. cCronbach’s a is unacceptable under 0.5, poor from 0.5 to 0.7, acceptable from 0.7 to 0.8, good from 0.8 to 0.9 and excellent from 0.9 to 1.0.

Revised Online Cognition Scale (R-OCS) [33]

18

Item to total score: r > 0.51

20

Revised Internet Addiction Test (R-IAT) [32]

Item to total score: r = 0.31–0.70

33

Inter-factor correlation: r = 0.47–0.63, P < 0.01


Problematic Internet Usage Scale (PIUS) [8] Problem Video Game Playing Test (PVGT) [31]

Inter-factor correlation: r = 0.58–0.76b, P < 0.001




Discriminant validity


Content validity: established by a panel of content experts followed by a panel of 12 nursing students

20

36

20

44

20

Inter-item: r = 0.22a–0.81

Criteria validity: comparisons between diagnostic group and normal group on the subscales, F = 42.39, 33.07 and 22.18, all P < 0.001 Criteria validity: factors loading and factors variances are invariant between heavy players and normal players, both P > 0.05

Construct validity

Problematic Internet Use Questionnaire (PIUQ) [30]

Internet Related Problem Scale (IRPS) [3] Online Cognition Scale (OCS) [9]

31

Chinese Internet Addiction Inventory (CIAI) [24]

15

Items

Questionnaire

Table 1 Attributes of the selected questionnaires.

Internet addiction assessment tools 1209

Addiction, 108, 1207–1216

1210


Methodological approach to internet addiction assessment All questionnaires analyzed had their validity and reliability evaluated at least once (Table 1). However, due to the disputed nature of excessive internet use, there are no existing standards in terms of methodological procedures for assessment. Thus, the methodological approaches reflected in the selected questionnaires will be presented in detail here. Construct validity evaluates to what extent a questionnaire assesses the theoretical concept it is supposed to. The construct validity of the questionnaires was evaluated in most cases using inter-item and item to total score Pearson’s product correlations (six of 14, 43%; Table 1). The majority of them were of sufficient significance (r > 0.30). Criteria validity was established by comparing supposedly addicted individuals to normal groups in two cases, and content validity was judged as sufficient by a panel of experts in one questionnaire. The discriminant validity of the questionnaires, i.e. their ability to produce comparable results under constant conditions, was mainly determined using interfactor Pearson’s product correlations (seven of 14, 50%; Table 1). However, two questionnaires had inter-factor correlations over the accepted threshold (r > 0.71), meaning that the factors had more than 50% of shared variance. In such cases, the authors applied a secondorder model to their factors to establish sufficient discriminant validity. Two studies also compared models to establish the discriminant validity of their questionnaire, one allowing the factors to covary freely (unconstrained) and the other constraining the factors’ covariance to one (constrained). Most instrument scores (10 of 14, 71%) were correlated with various other constructs in order to establish the convergent validity of the questionnaires (for example: time spent on-line, depression, loneliness and other problematic internet use scales, Table 1). Note that many Pearson’s products were somewhat weak (r < 0.30; 18 of 40, 45%). The reliability of some questionnaires was measured using test–retest correlations (three of 14, 21%), which were of satisfactory strength (r > 0.60; Table 1). Two studies also used split-half Cronbach’s alphas to establish the questionnaire reliability. Also, and for the entire sample, questionnaire reliability was measured using Cronbach’s alphas to evaluate their internal consistency, which was acceptable for the vast majority of questionnaires (a > 0.70; 12 of 14, 86%). The predictive potential of questionnaires regarding attempts to control or cease the excessive behavior in the future was not assessed in any of the selected studies. © 2013 Society for the Study of Addiction

Discriminatory needs have not been taken into consideration in the majority of the evaluated questionnaires. In fact, few of them (four of 14, 29%) provided a cut-off point on their scale (GAS, IAT, PVGT, R-IAT). The cut-off point is a value over which the subject score is considered as deviant from the ‘normal’ population. This cut-off point has to be validated in order to increase the differential power of the questionnaire, i.e. its ability to distinguish between ‘normal’, ‘at risk’ and ‘pathological’ populations. Not having a validated cut-off point impairs the usefulness of these questionnaires in clinical and research context, particularly in the case of subjective behaviors such as excessive internet use. Furthermore, the present questionnaires’ external validity sometimes reflects the relative novelty of this mental health sphere. For instance, the samples sometimes consisted of high school or undergraduate students (CIAI, GAS, GPIUS, OCS, PIUS, R-IAT and R-OCS) or were relatively small (IAT, IRPS). Conversely, a double validation on two distinct populations was used for some questionnaires, merging the general population, heavy internet users, undergraduate students and the elderly (CIUS, GPIUS2, ICONS, IRPS and PVGT). The remaining questionnaire was validated in the general population (PIUQ). Factorial analyses are often used in questionnaire validation processes to evaluate the construct validity of the model proposed and its factors. Two methods are used mainly to identify the factors of a questionnaire, which are the exploratory factorial analysis (EFA) and the confirmatory factorial analysis (CFA). Some questionnaires used two factorial analysis methods (EFA and CFA: CIAI, PVGT, R-IAT) or an exploratory method only (GPIUS, IAT, PIUQ, PIUS). One questionnaire did not perform a factorial analysis (IRPS), but most instruments’ factors were validated using a confirmatory factor analysis, which has been recommended by psychometricians (nine of 14, 64%: CIAI, CIUS, GAS, GPIUS2, ICONS, OCS, PVGT, R-IAT, R-OCS) [34]. Additional attributes of selected questionnaires can be found in Table S2.

DIMENSIONAL STRUCTURE The dimensional structure of an assessment questionnaire is linked inherently to the definition of the disorder agreed upon by the scientific community. From the controversial ‘addictive’ nature of excessive internet use to its disputed existence, topped by the lack of agreement on its theoretical basis among its supporters, the evaluation of internet dependency presents a major challenge. In this context, we investigated the dimensional structures of the selected questionnaires. Addiction, 108, 1207–1216

Internet addiction assessment tools

Dimensional structure The dimensional structure of the questionnaires was assessed as follows: all factors of the selected questionnaires were first listed and then pooled into broad dimensions of addiction according to their conceptual similarity. Sixty-seven individual factors were identified within the sample and pooled into seven dimensions of addiction (examples are provided in Table 2, the complete list of factors and dimensions is provided in Table S3). The relative weight of each dimension of addiction was calculated as a percentage based on its occurrence across our sample of questionnaires. For instance, the dimension ‘compulsive use’ was evaluated in 11 of 14 questionnaires; it was thus present in 79% of the questionnaires. The dimension ‘negative outcomes’ was evaluated in 86% of the questionnaires, ‘salience’ in 71%, ‘social comfort’ in 50%, ‘withdrawal symptoms’ in 36%, ‘mood regulation’ in 43% and ‘escapism’ in 21%. Of note, each of the selected questionnaires evaluated several dimensions simultaneously (3.86 ⫾ 0.38 dimensions per questionnaire, median: 4, minimum: 1, maximum: 6). In order to examine if associations between different dimensions of addiction existed, i.e. if

1211

some dimensions were more likely to co-occur in the same questionnaire, the percentage of co-occurrence of the dimensions was calculated (Fig. 1). The percentage of co-occurrence represented the proportion of questionnaires in which two given dimensions were both assessed over the total number of questionnaires. For instance, the dimensions ‘compulsive use’ and ‘negative outcomes’ were both assessed in 10 of the 14 questionnaires; thus, they had a co-occurrence percentage of 71%. In other words, 71% of the time ‘compulsive use’ and ‘negative outcomes’ were assessed in the same questionnaire. Our analysis demonstrated that the dimensions ‘compulsive use’ and ‘negative outcomes’ are central to questionnaires purporting to assess internet addiction, being present in more than half of them, usually both assessed in questionnaires (co-occurrence of 71%) and also co-occurring with many other dimensions of addiction. The dimension ‘salience’ was also present in more than half the questionnaires and co-occurred frequently with ‘negative outcomes’ and ‘compulsive use’. Therefore, those three dimensions seem to form the core of existing questionnaires. Surprisingly, the dimensions ‘escapism’ and ‘withdrawal symptoms’ were not related strongly to any other dimension and were also less covered by the questionnaires in general.

Table 2 Example of factors and corresponding items pooled into the seven dimensions. Dimensions of addiction

Example of factor

Example of item

Compulsive use

Excessive use Loss of control Negative side effects

I find it difficult to control my internet use; I have attempted to spend less time on-line but have not been able to How often do others in your life complain to you about the amount of time you spend on-line?; I have gotten into trouble with my employer or school because of being on-line I think obsessively about going on-line when I am off-line; over time, have you been spending much more time thinking about playing video games, learning about video-game playing, or planning the next opportunity to play? Do you feel restless, frustrated, or irritated when you cannot use the internet?; I feel anxious if I have not read my e-mail or connected to the internet for some time Do you use the internet to escape from your sorrows or get relief from negative feelings?; I have gone on-line to make myself feel better when I was down or anxious When I am on-line, I don’t need to think about off-line problems; have you played video games as a way of escaping from problems or bad feelings? I can control how others perceive me when on-line; I feel safer relating to people on-line rather than face-to-face

Negative outcomes

Conflict Salience

Anticipation

Cognitive preoccupation

Withdrawal symptoms

Withdrawal Withdrawal symptoms

Mood regulation

Mood modification Mood alteration

Escapism

Distraction Escapism due to other problems

Social comfort

Perceived social benefits available on-line Preference for on-line social interaction

The dimensional structure of the questionnaires was established through pooling the factors into the seven dimensions of addiction according to their conceptual similarity. Compulsive use is synonym for tolerance and the inability to control, reduce or stop on-line behavior. Negative outcomes are deleterious consequences of excessive internet use. Salience stands for salience and cognitive preoccupation. Withdrawal symptoms are felt when not on-line. Mood regulation means that the internet is used to regulate the mood. Escapism means that the internet is used to escape from other problems. Social comfort refers to a preference for on-line social interaction.


Addiction, 108, 1207–1216

1212


Figure 1 Relative weight and co-occurrence of dimensions among the questionnaires. The size of the ovals showing dimensions of addiction corresponds to their relative weight, which was calculated as a percentage reflecting their occurrence across the selected questionnaires.The percentages of co-occurrence (gray arrows) represent the number of questionnaires in which the two dimensions were both assessed across the total number of questionnaires. For the purposes of intelligibility, only percentages of co-occurrence over 40% are represented

Dimensions distribution The earliest questionnaires purporting to assess internet addiction, built in the mid-1990s, referred to diagnostic criteria for substance abuse from various versions of the DSM to define and set criteria for excessive internet use, by substituting ‘substance abuse’ with ‘internet abuse’ [35]. Later on, researchers also adapted six to seven of the pathological gambling diagnostic criteria to develop different scales to measure problematic internet use and video-game-playing addiction (e.g. [36,37]). Other questionnaires specialized for assessing on-line video-gaming addiction (e.g. [38–40]) have referred to the pioneering Young’s Internet Addiction Test (IAT) [5], which was also based on the clinical DSM definition of pathological gambling. Similarly, several questionnaires have been derived from the diagnostic criteria for pathological gambling of the ICD-10 [41] to measure problematic internet use (e.g. [42,43]). Despite the widespread adaptation of DSM and ICD diagnostic criteria for substance dependence and pathological gambling to define and measure what is thought to be internet addiction, little is known about the validity of such a strategy. Therefore, we superimposed the seven dimensions of the questionnaires identified in this study onto the two most used diagnostic criteria for substance dependence according to their conceptual similarity by three independent evaluators (Fig. 2). © 2013 Society for the Study of Addiction

A kappa coefficient demonstrated a good inter-judge reliability (k = 0.47). The distribution of dimensions of addiction over DSMIV-TR and ICD-10 diagnostic criteria for substance dependence was calculated as the proportion of criteria used to measure one particular dimension over the total number of criteria in each diagnostic manual. For example, the dimension ‘compulsive use’ was related to three different DSM-IV-TR criteria over the total of seven criteria it has for substance dependence; the dimension ‘compulsive use’ is present in 42.86% of the DSM-IV-TR [DSM-IV-TR: compulsive use (43%), negative outcomes (7%), salience (14%), withdrawal symptoms (14%), social comfort (0%), escapism (14%) and mood regulation (7%); ICD-10: compulsive use (33%), negative outcomes (17%), salience (17%), withdrawal symptoms (17%), social comfort (0%), escapism (17%) and mood regulation (0%)]. Similarly, the proportions of dimensions over the questionnaires were considered (see relative weights above). The proportions of dimensions across the questionnaires were then compared to the proportions of dimensions across DSM-IV-TR and ICD-10 diagnostic criteria for substance dependence using a c2 test. As expected, the proportions did not differ from each other (DSM-IV-TR: c2 = 21, d.f. = 18, 1-b = 0.82, P = 0.28; ICD-10: c2 = 14, d.f. = 12, 1-b = 0.68, P = 0.30).

DISCUSSION What is next for researchers and clinicians? All questionnaires included in the sample had their validity and reliability evaluated at least once using various statistical methods. Most results regarding questionnaire validity (i.e. construct, discriminant and convergent validity) were satisfactory, meaning that questionnaires assessing excessive internet use and its consequences are doing so. However, very few authors confirmed the criteria validity of their questionnaires, leading to the conclusion that existing questionnaires do not emphasize the need to distinguish between ‘normal’ everyday users and ‘pathological’ heavy users of the internet. For a questionnaire to have sufficient criteria validity, and thus to differentiate between functional internet users and dysfunctional internet users, the on-line activities of normal controls should also be monitored [18] and strict selection criteria must be applied to detect excessive internet users (for example, a score of 70 or higher on Young’s IAT [5,20]). With regard to reliability, the Cronbach’s a measure of internal consistency is used widely and it is reflected in the selected studies. Additional evaluations of questionnaire reliability are, however, recommended as the Addiction, 108, 1207–1216


1213

Figure 2 Superimposition of dimensions of addiction to DSM-IV-TR and ICD-10 diagnostic criteria for substance dependence. The dimensions of addiction were superimposed on the diagnostic criteria for substance dependence according to their conceptual similarity by three independent judges (k = 0.47)

frequency of leisure activities could be influenced by age and periods of the year, which could, in turn, have an effect on the test–retest reliability of the questionnaires. A suggestion to optimize the assessment of excessive internet use would be to use two or more instruments and to verify the reliability of the self-reports by asking peers and guardians to validate the amount of hours per day the subject is on-line, and to what extent this disrupting behavior is interfering with their lives [14,19]. All models underlying existing questionnaires were evaluated by various statistical methods including factor loading of items and factor credit and usefulness. The vast majority of studies succeeded in identifying significant factors for what may be internet addiction and relied upon validated items to assess this problematic behavior. At present, however, we unfortunately could © 2013 Society for the Study of Addiction

not recommend one assessment questionnaire over others based on our analyses. Nevertheless, various techniques can be employed by clinicians and researchers to improve excessive internet use assessment. For instance, some additional elements could be evaluated in questionnaires to ensure that immoderate behaviors are occurring, such as minimum time spent on-line (e.g. 6 hours on-line everyday aside from work, more than 30 hours per week), symptom duration (e.g. for more than 3 months), and severity (e.g. social function significantly impaired, including decline in academic or professional performance). Results of a questionnaire purporting to assess internet addiction must acknowledge a persistent and recurrent maladaptive behavior that disrupts personal, familial or professional activities and must reflect the distress Addiction, 108, 1207–1216

1214


experienced. To do so, assessment must not be confused by professional internet use or occasional internet use. An evaluation tool should also predict the ability of the excessive user to control or cease the problematic behavior. Unfortunately, this aspect was not assessed in any of the selected questionnaires. When compared with other ‘classical’ forms of addiction such as tobacco addiction, the lack of predictive properties of current assessment tools may be perceived as a major weakness of this emerging field. Thus, one of the milestones of future research on excessive internet use evaluation should be to include this aspect.

Social comfort and the internet As mentioned previously, the majority of existing questionnaires refer to criteria for substance dependence and pathological gambling in diagnostic manuals to define what may be internet addiction and build assessment questionnaires. It is therefore not surprising that no significant difference was identified in the distribution of dimensions. However, there is a considerable theoretical and conceptual gap between pathological gambling, substance dependence and excessive internet use. Our results demonstrate the absence of criteria assessing the dimension of social motivation of addictions in both DSM-IV-TR and ICD-10 (Fig. 2). Questionnaires identified in the present study are also directed predominantly towards the assessment of salience, compulsive use and negative outcomes, whereas the social comfort associated with on-line interactions compared to face-to-face interactions is sometimes under-represented. Similarly, social aspects were omitted in previous definitions of this condition [1,2]. This situation could be problematic with regard to dysfunctional internet use and its outcomes. Indeed, the internet is an appealing medium for engagement in social interactions and for increasing the size of one’s social network [44–46]. It is also used by socially anxious individuals to reduce the stress of rejection by substituting face-to-face interactions with virtual communication [38,47,48]. The under-consideration of the social motivation to be on-line in the assessment-oriented literature is worrying, as the social use of the internet (e.g. e-mail, newsgroups, chatrooms, on-line games, etc.) is thought to be more addictive, seems to cause more psychological problems than other uses and may be responsible for the tremendous amount of time users spend on-line [5,30,49]. We thus recommend that social motivation should be taken into account when developing assessment questionnaires. For instance, questionnaires could include items that measure the preference for on-line social interactions (e.g. ‘Are most of your on-line friends your real-life friends too, or only Web buddies?’) and the © 2013 Society for the Study of Addiction

social usage of the internet (e.g. ‘Do you generally use the internet to perform solitary activities, such as information research or one-player games?’).

CONCLUSION The rise of excessive internet use could be associated with social and economic costs in the coming decades, and poses implications for users, clinical and research communities alike. The present status of the assessment of what may be recognized as internet addiction indicates some heterogeneity in study methodology and differences in the descriptive aspects of the questionnaires. The analyses performed also point to some conceptual improvements that could be made to the instruments, namely that the social aspects of this condition are currently slightly disregarded, despite their possible involvement in the dependency mechanism. While there is an ongoing debate on the nature of excessive internet use and whether it may involve addiction mechanisms, exposure to internet-related activities is constantly growing and the outcomes of such a condition could be deleterious in relation to various aspects of an individual’s life [1,3–5]. The development of effective solutions to the evaluation and clinical management of excessive internet use awaits greater involvement of the psychiatric and research communities before questionnaires can be disseminated as reliable assessment tools. The dimensional framework we identified and discussed in this study will further our understanding of what could be internet addiction, inform future debates on this issue and, ultimately, aid in developing effective ways to assess and manage this little-known condition. Declaration of interests None. Acknowledgements The authors thank Catherine Gaudreau for her help with questionnaire analysis, and Dr Anna Lomanowska and Christine Marquilly for their valuable feedback on the manuscript. This work was supported by the Canadian Institutes of Health Research (CIHR—grant no. 89699) and the Natural Sciences and Engineering Research Council of Canada (NSERC—grant 371644). M.J.G. holds a Career Grant from the ‘Fonds de la Recherche en Santé du Québec’ (FRSQ). C.L.L. is supported by a Graduate Scholarship from the Natural Sciences and Engineering Research Council of Canada (NSERC). All authors report no connection with tobacco, alcohol, pharmaceutical or gaming industry and no constraints on publishing. Addiction, 108, 1207–1216


References 1. Beard K. W., Wolf E. M. Modification in the proposed diagnostic criteria for internet addiction. CyberPsychol Behav 2001; 4: 377–83. 2. Block J. J. Issues for DSM-V: internet addiction. Am J Psychiatry 2008; 165: 306–7. 3. Armstrong L., Phillips J. G., Saling L. L. Potential determinants of heavier internet usage. Int J Hum Comput Stud 2000; 53: 537–50. 4. Meerkerk G. J., Van Den Eijnden R. J. J. M., Vermulst A. A., Garretsen H. F. L. The Compulsive Internet Use Scale (CIUS): some psychometric properties. CyberPsychol Behav 2009; 12: 1–6. 5. Young K. S. Internet addiction: the emergence of a new clinical disorder. CyberPsychol Behav 1998; 1: 237–44. 6. Anderson C. A., Gentile D. A., Buckley K. Violent Video Game Effects on Children and Adolescents: Theory, Research, and Public Policy. New York: Oxford University Press; 2007. 7. Griffiths M. D., Davies M. N. O., Chappell D. Online computer gaming: a comparison of adolescent and adult gamers. J Adolesc 2004; 27: 87–96. 8. Ceyhan E., Ceyhan A. A., Gürcan A. The validity and reliability of the Problematic Internet Usage Scale. Educ Sci Theory Pract 2007; 7: 411–6. 9. Davis R. A., Flett G. L., Besser A. Validation of a new scale for measuring problematic internet use: implications for preemployment screening. CyberPsychol Behav 2002; 5: 331– 45. 10. Gentile D. A. Pathological video game use among youth 8 to 18: a national study. Psychol Sci 2009; 20: 594–602. 11. Mitchell P. Internet addiction: genuine diagnosis or not? Lancet 2000; 355: 632. 12. Kim S. H., Baik S.-H., Park C. S., Kim S. J., Choi S. W., Kim S. E. Reduced striatal dopamine D2 receptors in people with internet addiction. NeuroReport 2011; 22: 407–11. 13. Zhou Y., Lin F. C., Du Y. S., Qin L. D., Zhao Z. M., Xu J. R. et al. Gray matter abnormalities in internet addiction: a voxelbased morphometry study. Eur J Radiol 2011; 79: 92–5. 14. Yuan K., Qin W., Wang G., Zeng F., Zhao L., Yang X. et al. Microstructure abnormalities in adolescents with internet addiction disorder. PLoS ONE 2011; 6: e20708. 15. Han D. H., Hwang J. W., Renshaw P. F. Bupropion sustained release treatment decreases craving for video games and cue-induced brain activity in patients with internet video game addiction. Exp Clin Psychopharmacol 2010; 18: 297– 304. 16. Han D. H., Kim Y. S., Lee Y. S., Min K. J., Renshaw P. F. Changes in cue-induced, prefrontal cortex activity with video-game play. CyberPsychol Behav Soc Netw 2010; 13: 655–61. 17. Kim Y.-R., Son J.-W., Lee S.-I., Shin C.-J., Kim S.-K., Ju G. et al. Abnormal brain activation of adolescent internet addicts in a ball-throwing animation task: possible neural correlates of disembodiment revealed by fMRI. Prog NeuroPsychopharmacol Biol Psychiatry 2012; 39: 88–95. 18. Ko C.-H., Liu G.-C., Hsiao S., Yen J.-Y., Yang M.-J., Lin W.-C. et al. Brain activities associated with gaming urge of online gaming addiction. J Psychiatr Res 2009; 43: 739–47. 19. Dong G., Huang J., Du X. Enhanced reward sensitivity and decreased loss sensitivity in internet addicts: an fMRI study during a guessing task. J Psychiatr Res 2011; 45: 1525–9. 20. Dong G., Lu Q., Zhou H., Zhao X. Impulse inhibition in people with internet addiction disorder: electrophysiological © 2013 Society for the Study of Addiction

21.

22.

23.

24.

25.

26.

27.

28. 29.

30.

31.

32.

33. 34.

35.

36.

37.

38.

39.

1215

evidence from a Go/NoGo study. Neurosci Lett 2010; 485: 138–42. Lin F., Zhou Y., Du Y., Qin L., Zhao Z., Xu J. et al. Abnormal white matter integrity in adolescents with internet addiction disorder: a tract-based spatial statistics study. PLoS ONE 2012; 7: e. Lemmens J. S., Valkenburg P. M., Peter J. Development and validation of a game addiction scale for adolescents. Media Psychol 2009; 12: 77–95. Pincus H. A., Page A. E. K., Druss B., Appelbaum P. S., Gottlieb G., England M. J. Can psychiatry cross the quality chasm? Improving the quality of health care for mental and substance use conditions. Am J Psychiatry 2007; 164: 712–9. Huang Z., Wang M., Qian M., Zhong J., Tao R. Chinese internet addiction inventory: developing a measure of problematic internet use for Chinese college students. CyberPsychol Behav 2007; 10: 805–11. Caplan S. E. Problematic internet use and psychosocial wellbeing: development of a theory-based cognitive behavioral measurement instrument. Comput Human Behav 2002; 18: 553–75. Caplan S. E. Theory and measurement of generalized problematic internet use: a two-step approach. Comput Human Behav 2010; 26: 1089–97. Widyanto L., McMurran M. The psychometric properties of the Internet Addiction Test. CyberPsychol Behav 2004; 7: 443–50. Young K. S. Caught in the Net. New York: John Wiley & Sons, Inc.; 1998. Clark D. J., Frith K. H. The development and initial testing of the Internet Consequences Scales (ICONS). Comput Inform Nurs 2005; 23: 285–91. Thatcher A., Goolam S. Development and psychometric properties of the Problematic Internet Use Questionnaire. S Afr J Psychol 2005; 35: 793–809. Luke King D., Delfabbro P. H., Zajac I. T. Preliminary validation of a new clinical tool for identifying problem video game playing. Int J Ment Health Addict 2011; 9: 72–87. Chang M. K., Law S. P. M. Factor structure for Young’s Internet Addiction Test: a confirmatory study. Comput Human Behav 2008; 24: 2597–619. Jia R., Jia H. H. Factorial validity of problematic internet use scales. Comput Human Behav 2009; 25: 1335–42. Frazier T. W., Youngstrom E. A. Historical increase in the number of factors measured by commercial tests of cognitive ability: are we overfactoring? Intelligence 2007; 35: 169–82. Griffiths M. D. Internet Addiction: Does It Really Exist?– Psychology and the Internet: Intrapersonal, Interpersonal, and Transpersonal Implications. New York: Academic Press; 1998. Charlton J. P., Danforth I. D. W. Distinguishing addiction and high engagement in the context of online game playing. Comput Human Behav 2007; 23: 1531–48. Salguero R. A. T., Moran R. M. B. Measuring problem video game playing in adolescents. Addiction 2002; 97: 1601–6. Chak K., Leung L. Shyness and locus of control as predictors of internet addiction and Internet use. CyberPsychol Behav 2004; 7: 559–70. Ko C., Yen J., Chen C., Chen S., Yen C. Gender differences and related factors affecting online gaming addiction among Taiwanese adolescents. J Nerv Ment Dis 2005; 193: 273–7. Addiction, 108, 1207–1216

1216


40. Yee N. Motivations for play in online games. CyberPsychol Behav 2006; 9: 772–5. 41. World Health Organization. International Classification of Diseases (10th rev.). Geneva: World Health Organization; 1993. 42. Grüsser S. M., Thalemann R., Griffiths M. D. Excessive computer game playing: evidence for addiction and aggression? CyberPsychol Behav 2007; 10: 290–2. 43. Thalemann R., Wolfing K., Grüsser S. M. Specific cue reactivity on computer game-related cues in excessive gamers. Behav Neurosci 2007; 21: 614–8. 44. Ellison N. B., Steinfield C., Lampe C. The benefits of Facebook ‘friends’: social capital and college students’ use of online social network sites. J Comput Mediat Commun 2007; 12: 1143–68. 45. Guitton M. J. The immersive impact of meta-media in a virtual world. Comput Human Behav 2012; 28: 450–5. 46. Morahan-Martin J., Schumacher P. Loneliness and social uses of the internet. Comput Human Behav 2003; 19: 659– 71. 47. Caplan S. E. Relations among loneliness, social anxiety, and problematic internet use. CyberPsychol Behav 2007; 10: 234–42. 48. Erwin B. A., Turk C. L., Heimberg R. G., Fresco D. M., Hantula D. A. The internet: home to a severe population of individuals with social anxiety disorder? J Anxiety Disord 2003; 18: 629–46. 49. Douglas A. C., Mills J. E., Niang M., Stepchenkova S., Byun S., Ruffini C. et al. Internet addiction: meta-synthesis of qualitative research for the decade 1996–2006. Comput Human Behav 2008; 24: 3027–44.

Supporting information Additional Supporting Information may be found in the on-line version of this article at the publisher’s web-site: Figure S1 Trends in the descriptive properties of assessment questionnaires throughout the years. The descriptive properties of assessment questionnaires followed non-linear Gaussian curves throughout the years. (A) Number of items: reduced c2 = 2.44, adjusted r2 = 0.32, two-tailed Fisher statistic F(4,10) = 919.01, P < 0.0001. (B) Number of factors (inversed): reduced c2 = 0.54, two-tailed Fisher statistic adjusted r2 = 0.33, F(4,10) = 171.20, P < 0.0001. (C) Number of items per factor: reduced c2 = 0.69, adjusted r2 = 0.55, two-tailed Fisher statistic F(4,10) = 216.48, P < 0.0001


Table S1 Instruments excluded from the sample and corresponding exclusion criteria. More than 360 papers were identified, the vast majority of which were ignored for not having a questionnaire name or acronym appearing in the title and/or the abstract. Of the 30 questionnaires remaining after the first selection, 16 instruments were excluded according to exclusion criteria presented above. The final sample was composed of 14 questionnaires. Specifically, questionnaires which used DSM criteria for substance abuse or pathological gambling, but substituted ‘substance abuse’ with ‘internet abuse’ and ‘gambling’ with ‘internet’, were excluded. Questionnaires assessing disorders (e.g. pathological gambling) not perpetrated on the internet were also excluded. Questionnaires assessing video-gaming were not excluded from the sample because video-gaming is conceptually very close to internet addiction and are generally defined similarly as ‘technological addictions’. Table S2 Additional attributes of the selected questionnaires. This Table presents the instruments’ validation populations, descriptive properties, model fit to the data and sampling adequacy. Additional measures of the construct validity of the questionnaires, including the item factor loadings, the variance explained by the model and the factor eigenvalues, are included. Correlations are shown in italics. Table S3 Dimensional structure of the questionnaires. All 67 individual factors of the questionnaires pooled into the seven dimensions of addiction according to their conceptual similarity. The questionnaires included here are: the Chinese Internet Addiction Inventory (CIAI), the Compulsive Internet Use Scale (CIUS), the Game Addiction Scale (GAS), the Generalized Problematic Internet Use Scale (GPIUS), the Generalized Problematic Internet Use Scale 2 (GPIUS2), the Internet Addiction Test (IAT), the Internet Consequences Scales (ICONS), the Internet Related Problem Scale (IRPS), the Online Cognition Scale (OCS), the Problematic Internet Use Questionnaire (PIUQ), the Problematic Internet Usage Scale (PIUS), the Problem Video Game Playing Test (PVGT), the Revised Internet Addiction Test (R-IAT) and the Revised Online Cognition Scale (R-OCS).

Addiction, 108, 1207–1216