Psychological Assessment

3 downloads 0 Views 278KB Size Report
Mar 10, 2014 - (2002) focused on the different conceptual characteristics and ... Marylène Gagné, School of Psychology, University of Western Aus- tralia ..... satisfaction, but examination of correlations reveals that only .... probabilistic answer to a determined element of the scale. ..... 50%, eigenvalue .... pioneering study.
Psychological Assessment Evidence Against the Continuum Structure Underlying Motivation Measures Derived From Self-Determination Theory Emanuela Chemolli and Marylène Gagné Online First Publication, March 10, 2014. http://dx.doi.org/10.1037/a0036212

CITATION Chemolli, E., & Gagné, M. (2014, March 10). Evidence Against the Continuum Structure Underlying Motivation Measures Derived From Self-Determination Theory. Psychological Assessment. Advance online publication. http://dx.doi.org/10.1037/a0036212

Psychological Assessment 2014, Vol. 26, No. 2, 000

© 2014 American Psychological Association 1040-3590/14/$12.00 DOI: 10.1037/a0036212

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Evidence Against the Continuum Structure Underlying Motivation Measures Derived From Self-Determination Theory Emanuela Chemolli

Marylène Gagné

Milan, Italy

University of Western Australia

Self-determination theory (SDT) proposes a multidimensional conceptualization of motivation in which the different regulations are said to fall along a continuum of self-determination. The continuum has been used as a basis for using a relative autonomy index as a means to create motivational scores. Rasch analysis was used to verify the continuum structure of the Multidimensional Work Motivation Scale and of the Academic Motivation Scale. We discuss the concept of continuum against SDT’s conceptualization of motivation and argue against the use of the relative autonomy index on the grounds that evidence for a continuum structure underlying the regulations is weak and because the index is statistically problematic. We suggest exploiting the full richness of SDT’s multidimensional conceptualization of motivation through the use of alternative scoring methods when investigating motivational dynamics across life domains. Keywords: self-determination theory, motivation, Rasch analysis, quasi-simplex, relative autonomy index Supplemental materials: http://dx.doi.org/10.1037/a0036212.supp

have long theorized about this process (e.g., Grusec & Goodnow, 1994), which has been key to understanding how children assimilate values and regulations for socially acceptable behavior. Although deemed a natural tendency, internalization is determined by personal history, activity characteristics, and context. Because these can vary, the regulation of a particular behavior can be internalized in different ways, ranging from remaining completely external (therefore not internalized), to being regulated by internal pressures, to being completely self-regulated. To put it differently, quality of motivation is determined by the type of internalization that has taken place. Internalization can be of an “intrinsic” nature, such that the person develops an interest for doing the activity itself and consequently finds it enjoyable (e.g., I have fun cleaning my room). It can also be of an “identified” nature, such that the person’s regulation is transformed into a value for the outcome of the activity (e.g., I value having a clean room). Internalization can also be of an “introjected” nature, such that the person’s regulation is internalized in its original external form (essentially rewarding and punishing oneself, e.g., I clean my room because I don’t want to be a dirty person). SDT proposes that these types of internalization differ in the degree to which they are autonomously regulated. However, descriptions for each regulation differ across the SDT literature. Deci and Ryan (1985) initially stated that introjection, though internalized, is a nonself-determined type of regulation. Ryan and Connell (1989) referred to two different “kinds” of internalization, namely, introjection and identification, whereas Koestner and Losier (2002) focused on the different conceptual characteristics and correlates of each. Moreover, Ryan and Deci (2002, p. 17) have stated that each regulation “describes a theoretically, experientially, and functionally distinct type of regulation,” whereas Ryan (1993) added that introjection represents internal but heterono-

Self-determination theory (SDT; Ryan & Deci, 2000) suggests that there are different types of motivation, such that people vary not only in level of motivation but also in the source or quality of that motivation. At the same time, SDT postulates a continuum of autonomy to order those types of motivation. We show that evidence for the continuum is weak and that it has led researchers to adopt a scoring method that hinders the advancement of motivational knowledge. By showing that the construct of motivation is multidimensional, we argue for alternative scoring methods that are likely to yield more accurate and useful knowledge.

SDT SDT postulates an intrinsic tendency of organisms to develop by integrating their experiences into a coherent inner structure called the self. This natural tendency consists in internalizing the regulation of behaviors that were originally alien to the self but that come to be valued (Ryan, 1995). Developmental psychologists

Emanuela Chemolli, Milan, Italy; Marylène Gagné, School of Psychology, University of Western Australia. Both authors contributed equally to this work. This research was supported in part by a Society for Human Resource Management grant and a grant from the Fonds Québécois de Recherche sur la Société et la Culture, awarded to Marylène Gagné. We thank Maureen Ambrose for asking the question that sparked this work and David Andrich for feedback on our Rasch analyses. Correspondence concerning this article should be addressed to Marylène Gagné, School of Psychology, University of Western Australia, 35 Stirling Highway, Crawley, 6009, WA, Australia. E-mail: [email protected] 1

CHEMOLLI AND GAGNÉ

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

2

mous (i.e., nonautonomous) pressures that disrupt autonomy and is external to the self (i.e., with an external locus of causality). Despite these seemingly qualitatively different types of regulation, SDT hypothesizes that they align along a continuum of relative autonomy. Ryan and Deci (2002) do not suggest that this continuum is developmental, so a person does not have to progress through stages of autonomy. A person can adopt a behavioral regulation anywhere on this continuum dependent upon prior experiences and situational factors (Ryan & Deci, 2002). For example, a person might work on an activity for a reward (external regulation), and later, if the external pressure is not too high, the person might develop an interest for the activity (intrinsic motivation). Alternatively, a person who has identified with the value of an activity may lose that sense of value under a controlling mentor and become externally regulated. Thus, although there are predictable reasons for movement between orientations, there is no necessary sequence. We argue that the continuum argument has muddled the description of the different regulations, such that they are alternatively described as differing in kind or as varying in terms of their level of self-determination. For example, the continuum has led to describing introjected regulation as being less self-determined (instead of nonself-determined) than identified regulation, and as more self-determined than external regulation. Deci and Ryan (2012, p. 89) described introjection as the “least autonomous form of internal regulation,” and as less “fully” internalized than identification, which is less “fully” internalized than intrinsic motivation (Deci & Ryan, 2002, 2012; Ryan & Deci, 2002). Ryan (1993) also described it as partially internalized because it is not fully organized and assimilated. We show that empirical evidence supports the idea that motivation differs in kind more than in degree and that it should consequently not be described as falling along a continuum of autonomy. We also show that Guttman’s (1954) theory has not been used as it was intended to support the continuum structure of motivation and that the use of the relative autonomy index (RAI) has generated problems for the advancement of knowledge in the domain of human motivation. We do so by analyzing data from the Multidimensional Work Motivation Scale (MWMS; Gagné et al., 2013) and from the Academic Motivation Scale (AMS; Vallerand et al., 1992) to demonstrate lack of support for a continuum structure using the more sophisticated and appropriate Rasch analytical technique.

that because the different regulatory styles were intercorrelated according to a quasi-simplex (ordered correlation) pattern, they fell along a continuum of autonomy. They used Guttman’s (1954) quasi-simplex structure model to explain this pattern, whereby subscales that are theoretically closer are more positively related than subscales that are not theoretically close. For example, their External Regulation subscale was more positively related to the Introjection subscale than to the Intrinsic Motivation subscale.

Guttman’s Radex Theory The quasi-simplex model used by Ryan and Connell (1989) was drawn from radex theory, which stands for “a radial expansion of complexity” and describes “functional interdependence between mental abilities, whether they differ in kind or in degree” (Guttman, 1954, p. 339). A radex is a bidimensional space organized as a circumplex composed of a series of simplexes (see Figure 1). Difference in kind implies contiguity among components around a circumplex, whereas difference in degree implies continuity (a simplex) within each elementary component of the circumplex. A simplex is a variable set of the same kind (e.g., numerical ability) that possesses a “simple order of complexity” (Guttman, 1954, p.

b1

b2

b3

b4

b5

c2 c3 3 c4 c5

Defining a Continuum of Motivation Aristotle (350 B.C., Book V, part 3; 384 B.C.–322 B.C.) in Physic defined something as continuous when the touching limits of units are joined to become one, and something as contiguous when units form a succession that can but do not need to be in contact. As such, a continuum is defined as a continuous whole, and as an unbroken series, sequence, or progression (Manser, 2008). Terms that have been used to describe the motivation continuum have included conceptual similarity and contiguity, which do not imply that motivation types should form a continuous whole. Nonetheless, the continuum hypothesis was first tested by Ryan and Connell (1989), who investigated elementary schoolchildren’s external, introjected, identified, and intrinsic motivation for engaging in achievement and prosocial behaviors. They argued

Figure 1. A schematic diagram of a Radex of motivation with five simplexes and 25 discernible elementary components. a1 ⫽ intrinsic motivation – item 1; a2 ⫽ intrinsic motivation – item 2; a3 ⫽ intrinsic motivation – item 3; a4 ⫽ intrinsic motivation – item 4; a5 ⫽ intrinsic motivation – item 5; .b1 ⫽ integrated regulation – item 1; b2 ⫽ integrated regulation – item 2; b3 ⫽ integrated regulation – item 3; b4 ⫽ integrated regulation – item 4; b5 ⫽ integrated regulation – item 5; c1 ⫽ identified regulation – item 1; c2 ⫽ identified regulation – item 2; c3 ⫽ identified regulation – item 3; c4 ⫽ identified regulation – item 4; c5 ⫽ identified regulation – item 5; .d1 ⫽ introjected regulation – item 1; d2 ⫽ introjected regulation – item 2; d3 ⫽ introjected regulation – item 3; d4 ⫽ introjected regulation – item 4; d5 ⫽ introjected regulation – item 5; e1 ⫽ external regulation – item 1; e2 ⫽ external regulation – item 2; e3 ⫽ external regulation – item 3; e4 ⫽ external regulation – item 4; e5 ⫽ external regulation – item 5. After Guttman (1954).

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

EVIDENCE AGAINST THE CONTINUUM

260). In the field of psychology, people who want to test such an assumption typically rely on the use of factor analysis, and one could argue for a “simplex pattern” if one found that all items on a scale loaded onto the same factor or dimension (i.e., a unidimensional solution; see also Marsh, 1993). A radex represents an ordering of multiple simplexes that represent constructs that differ in kind (e.g., different types of abilities, or motivations) but that are functionally related. In the case of mental abilities, Guttman argued that the statistical ordering of mental abilities along the “radex” represents some functionality between them. If we argue that motivational regulations differ in kind, then we would need to describe them as a contiguous ordering of “kinds” of internalization along a circumplex (much like Guttman described relations among different types of mental abilities, such as verbal, numerical, reasoning, and the like). If, however, we argue that motivational regulations differ in degree of internalization (what Guttman would describe as differences in complexity on a particular mental ability), then we would describe it as lying along a continuum of autonomy or as a simplex. Ryan and Connell (1989) described a simplex as a configuration of variables that are ordered in terms of conceptual similarity and that vary in terms of kind and degree. According to Guttman, such an ordering would actually be described as a full radex, which would mean that motivation would be represented as a series of simplexes (one for each regulation) ordered around a circumplex. It thus appears that Ryan and Connell may have merged the concepts of kind and degree as being one and the same thing. But to conform to Guttman’s radex theory, SDT would need to specify some form of functional relationship between the regulations, similar to what Guttman had demonstrated with mental abilities. To our knowledge, SDT does not argue for functionality between the different regulations and has even explicitly stated that “progression” among the different types of regulation does not happen in a stagelike process (Deci & Ryan, 1985).

The Nature of Motivation If motivational regulations were to differ in degree instead of kind, as the continuum hypothesis implies, empirical evidence would need to demonstrate the following features. First, the motivational regulations would yield a single dimension when using factor analytic techniques, at the very least when trying to obtain second-order factors. Looking at the factor structure obtained from past research, we find that they all obtain multidimensional solutions, where each latent factor represents one of the regulations (Brière et al., 1995; Fernet et al., 2008; Gagné et al., 2010; Guay et al., 2000; Millette & Gagné, 2008; Pelletier et al., 1995, 1998; Tremblay et al., 2009; Vallerand et al., 1992). These are sometimes compared with alternative models specifying one-factor structures that have yielded poor fit (Mallett et al., 2007), sometimes to alternative models specifying separate higher order factors for autonomous versus controlled motivation, which yield adequate fit (Gagné et al., 2010). Li and Harmer (1996) argued for a structural equation modeling test of the continuum structure by specifying that each motivational construct along the continuum of relative autonomy is influenced directly by the prior adjacent construct and indirectly by other constructs that precede that adjacent one. Results demonstrated that the Sport Motivation Scale (Brière et al., 1995) fitted

3

a simplex structure. The authors concluded that regulatory styles “can be organized meaningfully along a single dimension representing individual differences in the degree of self-determination of behavior regulation in activity” (Li & Harmer, 1996, p. 403). Besides the fact that this method does not produce evidence for single dimension, Rogosa and Willett (1985) demonstrated that such a test is insensitive to even extreme violations of simplex structures. In addition, if regulations differed in degree or complexity, empirical evidence would demonstrate that different degrees of autonomy (represented by the different regulations) yield different levels of outcomes. However, empirical evidence, including neuropsychological evidence, shows that the different regulations actually produce different outcomes, not just different levels of the same outcomes. For example, Ryan and Connell (1989) found that each regulation was associated with different patterns of coping strategies and that only external and introjected regulations were related to anxiety toward school, whereas only identified and intrinsic regulations were related to enjoyment. Koestner and Losier (2002) provided a good review on the different conceptual features of each type of regulation, including those relating to the emotional experience associated with each regulation, the motivating force and goal orientation behind each, and their differential affective, cognitive, and behavioral consequences. Ryan, Kuhl, and Deci (1997) further argued that the neural networks involved in autonomous regulation are different from those involved in controlled regulation. They explained that autonomous regulation would require more experience-dependent functions, such as higher order reflective and emotional processes (i.e., right prefrontal cortex), whereas controlled regulation would require inhibiting these functions. Recent evidence concurs with this idea. Lee, Reeve, Xue, and Xiong (2012) demonstrated that there is more insular cortex activity (involved in emotional processing) when people are intrinsically motivated, whereas there is more posterior cingulate cortex activity (involved in cognitive processing) when they are extrinsically motivated. Lee and Reeve (2013) additionally showed that the anterior cingular cortex, known to be active when people feel agentic, is indeed more active when people report doing things out of autonomous motivation and that the angular gyrux, known to be active when people feel a loss of agency (i.e., pressure), is indeed more active when people report doing things out of controlled motivation (introjected and/or external). Such evidence on the “wiring” of motivation in the brain suggests that there are indeed qualitative differences in internalization processes, rather than quantitative differences. Finally, if motivation were to differ in degree instead of kind, a person could only “be” on one location on the continuum at any given time (just like a thermometer will only show one temperature at any given moment). However, if we offer people several reasons for engaging in an activity that reflect the different types of motivation, people endorse more than one reason. For example, a teacher may work not only to make money but also because she finds her work to be important for students, and she enjoys doing it. We could thus say that the teachers’ motivation is external, identified, and intrinsic at the same time. It is therefore obvious that researchers, by allowing people to rate each type of motivation, assume that each regulatory style is a continuum on its own (each a thermometer), not that the types align along a continuum.

CHEMOLLI AND GAGNÉ

4

And this assumption is supported by factorial evidence showing that the subscales fall onto different dimensions.

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Operationalizing Motivation Why does all this matter? Because the way motivation is described in SDT affects how researchers compute motivational scores. Scholars often use the RAI (Grolnick & Ryan, 1987), where regulations are weighted according to their place on the continuum and where controlled forms of motivation are subtracted from autonomous forms of motivation [the most common formula is “2(intrinsic) ⫹ 1(identified)⫺1(introjection)⫺2(external)”]. Researchers have justified the use of the RAI by arguing that it reflects the continuum structure of motivation. However, this requires one to believe that a person is situated on one location on the continuum even though this “position” is derived from scores on multiple locations on this continuum. Another issue is that the RAI is a difference score. Problems with difference scores are well described in the literature, and we mention only a few here. First, difference scores are less reliable, statistically speaking, than their components (Edwards, 2001; Johns, 1981). Second, difference scores may mask important results, such that there is great information loss (Edwards, 2001). For example, a person who scores 4 out of 7 on external regulation and 7 out of 7 on intrinsic motivation would have an RAI of 3. A person who scores 1 on external regulation and 4 on intrinsic motivation would also have an RAI of 3. Yet, these people have different motivational profiles that are likely to yield different patterns of behavior. Moreover, in the case of “simple difference scores” such as the RAI, one can completely predict the score from one of its components (Zuckerman, Gagné, Nafshi, Knee, & Kieffer, 2002). One of the criteria required to validate difference scores is the demonstration that “the construct measured by a difference score is discriminable from the variables measured by its components” (Johns, 1981, p. 453). In the present case, we cannot ascertain whether relations between RAI scores and other variables (e.g., performance) are driven by the “autonomous” or the “controlled” portion of the equation. For example, Bono and Judge (2003) found the RAI to be related to job satisfaction, but examination of correlations reveals that only the “autonomous” part of the equation was related to it and that this correlation was stronger than the one with the RAI. Conclusions drawn from results obtained with the RAI can therefore mask important information that could serve to make important decisions about interventions, which need not only be efficacious but also cost-effective. It would be important to know whether an intervention should focus on increasing autonomous motivation or on decreasing controlled motivation. Given these arguments, we believe that the multidimensionality of motivation, which is one of SDT’s strengths relative to other motivation theories, is sacrificed with the use of the RAI. This has been pointed out by Judge and colleagues (2005, p. 267), who did use the index, but underlined that it would be more “productive for future research to investigate the motives separately.” Koestner and Losier (2002) also recommended that researchers use the regulations as separate variables. Though these arguments do not completely invalidate research in which the index has been used, its use may have led to loss of information about the impact of the

different types of motivation on other variables (Koestner & Losier, 2002) and, as Bono and Judge (2003) have shown, underestimated the effect size of motivation on outcomes. Finally, the weighing of each regulation in the RAI is also problematic as the assumed “distance” between the regulations may not reflect the true distance between the different “levels” of self-determination. Essentially, no one to our knowledge has ever presented validation evidence for the weights given to each regulation. In the formula given earlier, the distance between external regulation and introjection is smaller than the distance between introjection and identification. Indeed, there is a difference of 1 unit between external and introjected regulation (as weights are ⫺2 and ⫺1), and a difference of 2 units between introjection and identified regulation (as weights are ⫺1 and ⫹ 1). This difference has never been discussed or justified, and does not fit most empirical results that show that introjected regulation is more highly positively related to identified than to external regulation (Brière et al., 1995; Fernet et al., 2008; Gagné et al., 2010, 2013; Guay et al., 2000; Mallett et al., 2007; Millette & Gagné, 2008; Pelletier et al., 1995, 1998; Tremblay et al., 2009; Vallerand et al., 1992). In short, the RAI does not take into account a person’s multiple motives, but hides them in a construct that unfortunately loses much of its richness. Motivation scores have been computed in alternative ways, for example, by computing indices for controlled motivation (i.e., averaging external and introjected regulation) and for autonomous motivation (i.e., averaging identified and intrinsic motivation; e.g., Bono & Judge, 2003; Sheldon & Elliot, 1999; Williams, Grow, Freedman, Ryan, & Deci, 1996). This approach is supported by evidence for a second-order factor structure (e.g., Gagné et al., 2010, 2013) and by evidence showing that it is more meaningful to look at autonomous versus controlled motivation than to look at intrinsic versus extrinsic motivation (Deci & Ryan, 2008). Other research shows that autonomous and controlled motivation sometimes interact to affect performance, which shows the importance of treating these two overarching forms of motivation as separate variables (Grant, Nurmohamed, Ashford, & Dekas, 2011). Other research has used each regulation as a separate variable. In some domains and for specific behaviors, the different regulations yield different outcomes. For example, Koestner, Losier, Vallerand, and Carducci (1996) found that identified political regulation was only related to voting behavior, whereas intrinsic motivation was only related to involvement in political movements. More recently, researchers have begun to construct personbased clusters or profiles (e.g., Vansteenkiste, Sierens, Soenens, Luyckx, & Lens, 2009), which assumes that we must look at a person’s total motivational profile when relating motivation to antecedents and outcomes. In this case, the pattern of covariation between the motivation subscales is thought to influence or be influenced by other variables. This approach not only assumes a multidimensional structure for motivation, but exploits it more fully. We conclude that researchers use different scoring systems that are more or less in line with the theoretical proposition advanced in SDT that motivation is multidimensional and that it is best represented by a continuum structure. We consequently tested the simplex assumption, which has been tested in various ways by previous researchers, with a better suited statistical approach,

EVIDENCE AGAINST THE CONTINUUM

Rasch analysis, to convince researchers to more carefully choose their scoring methods, which would help the development of integrative knowledge in the field of motivation.

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Rasch Analysis The probabilistic counterpart to Guttman’s radex theory is item response theory (Lord, 1980), of which Rasch is a special case. Given that most data in the social sciences are probabilistic (i.e., measured with error), Rasch analysis is better suited to examine whether motivation falls along a continuum of autonomy, because motivation is measured using categorical variables that contain error. Simply put, “Guttman created a nonparametric approach that yields ordinal level measures, whereas Rasch created a parametric model that yields interval level measures” (Engelhard, 2008, p. 173). Rasch analysis (Rasch, 1960) has been applied to measurement tools in several disciplines, including health studies, education, psychology, and economics (Bond & Fox, 2007) and is now established as the standard for modern psychometric evaluation of scales (Miceli, Settanni, & Vidotto, 2008). Through the Rasch method, researchers examine the behavior of persons (e.g., employees) in relation to a set of items (e.g., attitude surveys, test questions; Callingham & Bond, 2006) presupposing a model of relationship between a particular position on a continuum and a probabilistic answer to a determined element of the scale. The Rasch model builds a hypothetical unidimensional line along which items and persons are located according to (in our case) their quality and quantity of motivation (Baghaei, 2008). The items that fall close enough to the hypothetical line contribute to the measurement of the single dimension defined by the theory. Those that fall far from it are measuring another dimension that is irrelevant to the main Rasch dimension. Unidimensionality is best represented as a single latent variable being tested through all the items that form a hierarchical continuum—that is, only one latent trait explains the correlations among the test items (van Alphen, Halfens, Hasman, & Imbos, 1994). This assumption is at the heart of the Rasch model. Both the Rasch and the Guttman models recognize that person measurement and item calibration can be viewed as simultaneous processes (something that is not taken into account with factor analytic techniques), such that both allow for the simultaneous estimation of item and person parameters, which are represented through row and column effects that can be used to reproduce the full set of values in the data (Engelhard, 2008). But they also differ in important ways (Andrich, 1985; Engelhard, 2008). Guttman’s model is deterministic. It is intended to be used with binary data with items that are “rankable” in terms of their complexity. It also assumes that there is a functional relation between items on a scale, such that endorsement of less complex items is required for the endorsement of more complex ones (e.g., one must understand the principles of addition before understanding the principles of multiplication). This is very different from Likert scales, where endorsement of some items is unnecessary for the endorsement of others. In the case of motivation, it is not assumed that people must have external regulation before having any other more “autonomous” forms of motivation (Deci & Ryan, 2002). This is why Rasch analysis is better suited to the task at hand.

5 Study Overview

We tested the tenability of the continuum structure across two different motivation scales developed using SDT, using Ryan and Connell’s (1989) adjacency index and Rasch analysis. Within the conceptual framework of SDT, many instruments have been developed to assess the various motivational regulations. Among these, we chose the MWMS (Gagné et al., 2013) and the AMS (Vallerand et al., 1992). The MWMS and the AMS are composed of five subscales assessing intrinsic motivation, three types of extrinsic motivation (external, introjected, and identified regulation), and amotivation.

Method Samples Data for the MWMS were collected from employees of various organizations in the province of Québec (Canada) and in Italy. The final sample consisted of 2,063 people. Out of this, 960 were workers from a high-tech Canadian company (n ⫽ 62) and business students working part time (n ⫽ 898), who completed the English version of the MWMS. We could not collect age and gender data from the high-tech company. Business students had an age range from 18 to 43 years (M ⫽ 21.75, SD ⫽ 3.63), and 56% were women. Another 417 were Canadian workers from a hightech company (n ⫽ 32), a chemical plant (n ⫽ 185), and a government organization (n ⫽ 200) who completed the French version of the scale. We again could not obtain age and gender data in the high-tech organization. Workers in the chemical plant had an age range from 21 to 62 years (M ⫽ 42.48, SD ⫽ 8.97), and 71.9% were men, 16.2% women, and 11.9% did not report their gender. The age of government employees ranged from 18 to 65 (M ⫽ 46.60, SD ⫽ 9.31), but they did not report their gender. Finally, 686 Italian hospital workers completed an Italian version of the scale. Health workers (including nurses, laboratory and radiology technicians, physiotherapists, and nursing assistants) of hospitals located in districts of Brescia and Verona (Northern Italy) completed a paper survey at work. Sixty-six percent were nurses, 16.9% nursing assistants, 3.6% technicians and physiotherapists, and 13.4% did not report their occupation. Eighty-one percent were women, and their age ranged from 25 to 52 years old, with tenure ranging from 2 to over 30 years. Data for the English version of the AMS were collected from business undergraduate students in a Canadian business school in exchange for extra credit. The final sample consisted of 504 students (57.7% women), and the average age was 22.61 years (SD ⫽ 5.67).

Measures The MWMS is a self-report measure that covers the types of regulations previously described for the work domain (Gagné et al., 2013). For analyses, we left out amotivation to be consistent with previous continuum analyses (Ryan & Connell, 1989). Items were answered using either a 5-point scale, ranging from 1 (Not at all for this reason) to 5 (Exactly for this reason), or a 7-point scale, ranging from 1 (Not at all for this reason) to 7 (Exactly for this reason), depending on language. Validation evidence for the

CHEMOLLI AND GAGNÉ

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

6

MWMS shows that the scale is best represented through six first-order factors for amotivation (three items), social external regulation (three items), material external regulation (three items), introjected regulation (four items), identified regulation (three items), and intrinsic motivation (three items); two second-order factors, one on which both external regulation subscales load, and one on which both identified and intrinsic motivation load; and a third-order factor on which the second-order external regulation and the first-order introjected regulation factors load. This factor structure was found to be invariant across seven languages (Gagné et al., 2013). The AMS is a self-report measure that covers the types of regulations previously described for the academic domain (Vallerand et al., 1992). Again we left out amotivation. Items were answered using a 7-point scale ranging from 1 (Not at all for this reason) to 7 (Exactly for this reason). Validation evidence for the AMS shows that the scale is best represented through seven first-order factors (composed of four items each) for amotivation, external regulation, introjected regulation, identified regulation, and three types of intrinsic motivation. Higher order structures were not tested in the original validation.

Analytical Strategy In order to verify whether the items of the MWMS and the AMS fall along a continuum of motivation, we tested both the Guttman and the Rasch models. The Guttman model was tested using the procedure proposed by Ryan and Connell (1989), who conceived a statistical tool to test the simplexlike or ordered correlation pattern; they attributed an adjacency index to the correlations between different types of motivation according to how close the different regulations are along a continuum. They then computed the amount of variance accounted for by this adjacency index in the obtained squared correlations among the reason categories (squared correlations were used in order to restore interval scale properties to this data so as to meet the assumptions of a correlational test). (Ryan & Connell, 1989, p. 753)

We then tested whether the MWMS and the AMS items fall along a continuum structure using the Rasch model. We used a partial credit model (Masters, 1982), which is an extension of the Rasch model (Rasch, 1980; Wright & Masters, 1982; Wright & Stone, 1979), because it incorporates the possibility of having differing number of response opportunities for different items on the same test (please see the supplemental material for details about the procedure used; Bond & Fox, 2007).

Results The MWMS At first glance, correlations among the four subscales show a simplex pattern of relations (see Table 1). The largest correlations appear off the diagonal of the matrix, whereas the ones below that diagonal are slightly lower, and the one between external regulation and intrinsic motivation is the smallest, showing a seemingly perfect simplex matrix. In order to test the Guttman model, we followed Ryan and Connell’s (1989) procedure. We first squared each correlation. We then assigned an adjacency index to each correlation as follows: r (intrinsic motivation, identified regula-

Table 1 Means, Standard Deviations, and Correlations for the MWMS Subscales Regulation 1. 2. 3. 4.

External regulation Introjected regulation Identified regulation Intrinsic motivation

M

SD

1

2

3

3.12 4.01 4.16 4.16

1.12 1.16 1.26 1.33

.47ⴱⴱⴱ .25ⴱⴱⴱ .11ⴱⴱⴱ

.55ⴱⴱⴱ .35ⴱⴱⴱ

.65ⴱⴱⴱ

Note. MWMS ⫽ Multidimensional Work Motivation Scale. ⴱⴱⴱ p ⬍ .001.

tion) ⫽ 3, r (intrinsic motivation, introjected regulation) ⫽ 2, r (intrinsic motivation, external regulation) ⫽ 1, r (identified regulation, introjected regulation) ⫽ 3, r (identified regulation, external regulation) ⫽ 2, and r (introjected regulation, external regulation) ⫽ 3. We then computed the amount of variance explained in the squared correlations by the adjacency indices using a regression where the squared correlations are regressed onto the adjacency indices (n ⫽ 6 correlations). The congruency coefficient is the R obtained from this regression. Results of this regression revealed a congruency coefficient of .87, F(1, 4) ⫽ 12.84, p ⬍ .01, demonstrating that more than 75% of the variance in squared correlation is accounted for by the adjacency indices. This congruency coefficient is higher than the one found by Ryan and Connell (1989), which was .79 (p ⬍ .01). Following this procedure, we would conclude that the subscales of the MWMS fall along a continuum of relative autonomy. We then used the Winsteps software (Linacre, 2003) to conduct a Rasch analysis. The value of Rp was 0.86, indicating a high likelihood of replicability. The person separation index, unlike the person reliability estimate, which has a maximum value of 1.00, is not constrained by an upper boundary, but has a range from zero to infinity. The recommendation is that the separation ratio should exceed 2 (Miceli et al., 2008), and it was 2.46 for this sample; therefore, we found an acceptable separation ratio. Principal components analysis of residuals decomposes the matrix of items or person correlations based on residuals to identify other possible factors (dimensions) that might be affecting response patterns (Linacre, 1998; Smith, 2000). Its aim is to attempt to extract one common factor that explains the most residual variance—and this variance should be random noise, such that there is no meaningful structure in the residuals— under the hypothesis that there is such a dimension (Linacre, 1998). If this is found, we have evidence of a unidimensional continuum underlying the data. In other words, if the dimension in the residuals is at “noise” level, then there is no shared second dimension. If the dimension is above the noise level (i.e., eigenvalue larger than 2.0), then there is a “second” dimension in the data; similarly, a third dimension is investigated, and so on. We performed an unrotated principal components analysis so as to not mask the actual variance structure and dimensionality of the data. Figure 2a presents the scree plot showing the relative size of the resulting variance components (logarithmically scaled). Results showed multidimensionality “problems”: Most of the variance in the data was unexplained (U ⫽ 63.4%). The variance explained by the measure (M) was 36.6% (eigenvalue 9.2; a good value should be above 50%). This was composed of the item difficulty variance (I),

EVIDENCE AGAINST THE CONTINUUM

a 100 63.4 50

36.6

29

8

7.6

5.4 5

0.5

T

M

P

I

U

1

2

3

4.4

4.1

4

5

Variance Components

b 100

2.0 eigenvalue if we are dealing with a unidimensional measure (Linacre, 2010; Raîche, 2005). In our data, it was 3.8, which means that there is weak evidence that the scale falls along a single dimension. If the scale was unidimensional, the unexplained variance would be random. However, our results indicate that the unexplained variance is actually composed of five main contrasts (first contrast ⫽ 15.1% of the unexplained variance; second contrast ⫽ 8%; third contrast ⫽ 5.4%; fourth contrast ⫽ 4.4%; and fifth contrast ⫽ 4.1%; see Figure 2). The variance explained by the first contrast, 15.1%, was higher than the 5% agreed-upon cutoff. In other words, when the variance explained by the first contrast in the residual is higher than 5%, there is at least one second dimension at work, so that the data may hide additional “dimensions” (please see the supplemental material for more details). To conclude, the statistical analyses derived from Rasch did not yield support for the SDT-based premise that the MWMS is structured along a single continuum, as there is evidence for additional dimensions underlying the data. This is consistent with the fact that previous confirmatory factor analysis (CFA) results yielded a multidimensional factor structure representing each motivation regulation for the MWMS (Gagné et al., 2013).

57.1 42.9

50

Variance Logiscaled

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Variance Logiscaled

15.1

7

The AMS

34.3

13 1 13.1 8.6

7.3 4.9

5

0.5

T

M

P

I

U

1

2

3

4.4

4

3.7

5

Variance Components

Scree plot of the variance component percentage sizes, logarithmically scaled: On plot On x-axis Meaning T TV Total variance in the observations (always 100%) M MV Variance explained by the Rasch measures (difference between TV and UV) P PV Variance explained by the person motivation or the fraction of MV attributable to person variance I IV Variance explained by the item difficulties or the fraction of MV attributable to item variance U UV Unexplained variance or raw residuals 1 U1 First contrast (component) in the residuals6 2 U2 Second contrast (component) in the residuals7 3 U3 Third contrast (component) in the residuals7 4 U4 Fourth contrast (component) in the residuals7 5 U5 Fifth contrast (component) in the residuals7

Figure 2. Standardized residual variance scree plot for (a) the Multidimensional Work Motivation Scale (MWMS) and (b) the Academic Motivation Scale (AMS), logarithmically scaled.

At first glance, correlations among the four subscales do not show a simplex pattern of relations (see Table 2). The largest correlations are between nonadjacent subscales. In order to test the Guttman model, we again followed Ryan and Connell’s (1989) procedure. Results of the regression revealed a congruency coefficient of .88, F(1, 4) ⫽ 13.77, p ⬍ .05, demonstrating that more than 77% of the variance in squared correlation is accounted for by the adjacency indices. This congruency coefficient is higher than the one found by Ryan and Connell (1989), which was .79 (p ⬍ .01). Following this procedure, we would conclude that the subscales of the AMS fall along a continuum of relative autonomy even though it is obvious that they do not. We again used the Winsteps software (Linacre, 2003) to conduct a Rasch analysis. The value of Rp was 0.88, indicating good replicability, and the person separation index was 2.67, indicating acceptable person separation. A principal components analysis of the residuals showed multidimensionality problems (see Figure 2b for the scree plot); most of the variance in the data was unexplained (U ⫽ 57.1%). The variance explained by the measure (M) was 42.9% (less than the recommended ⬎ 50%, eigenvalue ⫽ 18). This was composed of the item variance (I ⫽ 34.3%, eigenvalue 14.4) and the person variance (P ⫽ 8.6%, eigenvalue 3.6). As

Table 2 Means, Standard Deviations, and Correlations for the AMS Subscales Regulation

which constituted 29% of the variance (eigenvalue 7.3), and of the person motivation variance (P), which constituted 7.6% of the variance (eigenvalue 1.9). According to Rasch model simulations, the first contrast (here, contrast means “component” or “dimension”) in the “unexplained variance” (residual variance) should have an eigenvalue lower than

1. 2. 3. 4.

External regulation Introjected regulation Identified regulation Intrinsic motivation

M

SD

1

2

3

4.79 3.01 5.74 4.35

1.25 1.51 1.16 1.05

.26ⴱⴱⴱ .23ⴱⴱⴱ .49ⴱⴱⴱ

⫺.09 .32ⴱⴱⴱ

.18ⴱⴱⴱ

Note. AMS ⫽ Academic Motivation Scale. ⴱⴱⴱ p ⬍ .001.

CHEMOLLI AND GAGNÉ

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

8

mentioned for the MWMS, the eigenvalue of the first contrast in the unexplained variance should be below 2.0, but in these data, it was 5.5. This provides weak evidence for a unidimensional continuum. Again, the variance explained by this first contrast was 13.1% (and then second contrast ⫽ 7.3%; third contrast ⫽ 4.9%; fourth contrast ⫽ 4.4%; and fifth contrast ⫽ 3.7%), which is higher than the recommended cutoff of 5%. There is therefore at least one noticeable secondary dimension in the interitem correlation matrix. These results give indications of multidimensionality issues and therefore do not support a unidimensional structure for the scale (please see the supplemental material for more details). In short, our results show there was substantial structure within the items that is not explained by the primary factor, and we therefore did not find support for the SDT-based premise that the AMS is structured along a single continuum. This is in line with previous CFA evidence that the AMS is also best represented by a multidimensional factor structure (Vallerand et al., 1992).

Discussion The use of analytical methods to verify factorial structures, such as CFA, and of analytical methods that dig deeper into the “behavior” of a scale, such as Rasch analysis, challenge the conceptualization and operationalization of motivation from SDT that motivation can be represented using a continuum of selfdetermination. We believe that this cannot be ignored and that it can serve to refine SDT in a way that will make it more precise when it is being used to predict how antecedents act on motivation, and how motivational regulations act on outcomes that researchers deem important to their field of study. We evaluated the postulate that, according to SDT, motivation is best represented along a continuum of relative autonomy. We showed some contradictions in how motivational regulations have been described in the literature, and argue for a conceptualization that describes regulations as varying in kind rather than in degree, using a clear definition of the concept of continuum. We pointed to empirical evidence that supports a multidimensional over a unidimensional conceptualization of motivation. We also pointed out serious issues with the use of the RAI on both conceptual and statistical grounds and suggested alternative scoring procedures to best represent the true nature of motivation. We then demonstrated, using Rasch analysis, that motivation cannot be described as falling along a continuum of autonomy and explained why this analytical method is better adapted to the type of data we use in psychology than Guttman’s method. Results did show a particular pattern of correlations among the MWMS subscales, such that there appears to be some contiguous ordering that has also been observed in other studies (e.g., Li & Harmer, 1996; Ryan & Connell, 1989). This pattern, however, was not evident in data from the AMS. Though we tested whether these correlation patterns fit a simplex structure using Ryan and Connell’s (1989) adjacency index procedure, we show that this test appears to be insensitive. As Ryan and Connell found support for a simplex structure through the adjacency index, so have we, even in the case in which it was obvious that the pattern of correlation did not demonstrate the simplex structure. We also observed in the literature that people who have used this index find supportive results even though an eyeball analysis of their correlation matrix far from conforms to a

simplex structure (e.g., Fernet et al., 2008; Guay et al., 2000). For this reason, it was important to use a more stringent technique that is better suited for the type of data we typically use in our fields, the Rasch model, to test for a continuum structure. We found no evidence for the postulate that motivation is represented along a single continuum. We found instead that the MWMS and the AMS are best represented through a multidimensional structure, which concurs with previous empirical evidence demonstrating that all of the motivation scales based on SDT yield multidimensional structures (e.g., Gagné et al., 2013; Vallerand et al., 1992). We conclude that the pattern of correlation that is often, though not always, found between the different types of motivation cannot be described using a continuum. Since Aristotle specified that “a thing that is in succession and touches is ‘contiguous’” (Aristotle, 350 B. C.), it may be more accurate to describe adjacent constructs, such as these motivational styles, as contiguous instead of continuous. Statistically and philosophically speaking, the concept of a contiguum allows for the possibility to be at more than one location at a time, as it allows for multidimensionality. In this sense, it better reflects the reality that people do report having more than one type of motivation for a particular activity. Interestingly, Ryan and Connell (1989) referred to “adjacent categories” when reporting results for each type of regulation in their pioneering study. Given the repeated observation of a specific pattern of correlations between the different types of motivation, though variable across studies, we may need to find another way to describe this pattern to fit with empirical evidence. Because previous researchers have mostly tested for a continuum or simplex structure underlying SDT-based scales, we concentrated our tests on demonstrating that two of these scales did not conform to this structure. We did not, however, test for the radex assumption. Testing for a radex structure would require following Gurtman and Pincus’ (2003) circumplex assumptions using Browne’s (1992) circular stochastic modeling approach, specifically CIRCUM (Browne, 1995). This technique has been applied to test for a circumplex model of values (Grouzet et al., 2005) and could be the next step in evaluating the structure of SDT’s conceptualization of motivation. We observed, from reviewing the literature on the concept of internalization, that the definitional issue we identified mostly had to do with how introjection is described. It has been alternatively described as non-self-determined, and sometimes as more autonomous than external regulation, but less autonomous than identified regulation. In the RAI, it is weighted negatively, like external regulation. This weight does not reflect the factor structure found in most research, which shows that it is indeed more related to identified than to external regulation. This issue could either reflect a problem with the way introjection is conceptualized and/or reflect a problem with how it is being operationalized (i.e., the items measuring it). Following the definition and description of introjection from the literature, our inclination would be to argue that introjection is a type of internalization that is not autonomous at all and that is qualitatively different from the internalization that takes place when people identify with an activity. Neuropsychological evidence could possibly shed light on whether they are indeed different psychological processes. What we have demonstrated, through an evaluation of motivation concepts derived from SDT and through Rasch analysis, should caution researchers against using aggregation methods such

EVIDENCE AGAINST THE CONTINUUM

as the RAI to examine the relation of motivation to various outcomes and antecedents. Our conclusions concur with those of Judge, Bono, Erez, and Locke (2005), who advised against the exclusive reliance on such composite scores, and with those of Edwards (2001), who argues against the use of difference scores in general. We similarly advocate for the use of separate scores for each type of regulation, for the use of polynomials (Edwards, 2001), or for the use of person-based profiles (Vermunt & Magidson, 2005).

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Limitations and Future Research One limitation of our statistical demonstration is that we tested the Guttman and Rasch models on data from only two SDT-based scales, the MWMS and the AMS. Future research should therefore attempt to replicate our tests on other SDT-derived scales to verify the generalizability of our findings. We feel confident that our findings are meaningful because we duplicated the quasi-simplex results obtained by Ryan and Connell (1989) in the original study supporting the continuum structure of the SDT motivation scales, but we failed to confirm such a structure with more precise analyses inspired by the Rasch model. We also only used the subscales used by Ryan and Connell and left out amotivation. Other scales also have an “integrated regulation” subscale. Future analyses could include these two subscales to see whether different results would emerge, though this is improbable given that adding subscales is likely to increase the number of dimensions found, not decrease it. From a psychometric point of view, future research could verify whether each motivational regulation is unidimensional, using the Rasch method. If we found evidence for unidimensionality within each of the subscale, we could conclude that motivation is best represented by using many “thermometers”— one for each type of motivation. It is currently not possible to do this with only three or four items per subscale.

Conclusion Does SDT need a continuum structure? We do not think so. SDT offers an amazingly rich and useful conceptualization of motivation based on different internalization processes that is quite unique in psychology. Using the concept of a continuum and the practice of using the RAI dilutes this richness and also dilutes the richness of the results that are drawn from research. We hope that this article will encourage a reflection that we feel is essential to the development of valid theory-driven measures and corresponding scoring systems. We hope that other researchers will try the same specialized analyses with SDT-based scales in other life domains, and perhaps also with other psychological constructs. The same reality may apply to other motivational concepts, such as resultant achievement motivation, conceptualized as the difference between need for achievement and fear of failure (Atkinson & Litwin, 1960), to emotions with the now widely accepted practice of considering positive and negative affect as separate dimensions (Watson, Clark, & Tellegen, 1988), to organizational behavior concepts, such as the tripartite model of commitment (Meyer, Parfyonova, & Gagné, 2010), and the multifactor leadership questionnaire (Bass & Riggio, 2006), both of which are sometimes described by using a theoretical continuum structure.

9

Theories, and in this case SDT, are meant to help us make sense of our world, but as they are being developed and used, they sometimes need to be refined and transformed. Two important qualities of good theories are their falsifiability and their utility (Bacharach, 1989). With our increasingly sophisticated research and analytical methods, we can more precisely test theories, which increases the likelihood that we will falsify them (Meehl, 1967). And with better instruments, we can increase the utility of a theory. We believe that the refinements we propose for SDT achieve both. By using more sophisticated analytical methods, we propose a more precise conceptualization of motivation that better fits the empirical evidence we have for it. And by doing this, we show which operationalizations are better, and we show that they are likely to yield more precise results that will lead to more useful applications of the theory.

References Andrich, D. A. (1985). An elaboration of Guttman scaling with Rasch models for measurement. In N. B. Tuma (Ed.), Sociological methodology (pp. 33– 80). San Francisco, CA: Jossey-Bass. doi:10.2307/270846 Aristotle. (350 B. C.). Physics. Retrieved from http://classics.mit.edu/ Aristotle/physics.6.vi.html Atkinson, J. W., & Litwin, G. H. (1960). Achievement motive and test anxiety conceived as motive to approach success and motive to avoid failure. Journal of Abnormal and Social Psychology, 60, 52– 63. doi: 10.1037/h0041119 Bacharach, S. B. (1989). Organizational theories: Some criteria for evaluation. Academy of Management Review, 14, 496 –515. Baghaei, P. (2008). The Rasch model as a construct validation tool. Rasch Measurement Transactions, 22, 1145–1146. Retrieved from www.rasch .org/rmt/rmt221a.htm Bass, B. M., & Riggio, R. E. (2006). Transformational leadership (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates. Bond, T. G., & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). Mahwah, NJ: LEA. Bono, J. E., & Judge, T. A. (2003). Self-concordance at work: Understanding the motivational effects of transformational leaders. Academy of Management Journal, 46, 554 –571. doi:10.2307/30040649 Brière, N. M., Vallerand, R. J., Blais, M. R., & Pelletier, L. G. (1995). Développement et validation d’une mesure de motivation intrinsèque, extrinsèque et d’amotivation en contexte sportif: L’Échelle de Motivation dans les Sports (ÉMS). [Development and validation of a measure of intrinsic and extrinsic motivation, and amotivation in the sports context: The Sport Motivation Scale]. International Journal of Sport Psychology, 26, 465– 489. Browne, M. W. (1992). Circumplex models for correlation matrices. Psychometrika, 57, 469 – 497. doi:10.1007/BF02294416 Browne, M. W. (1995). CIRCUM: Notes on usage. Columbus, OH: Browne. Callingham, R., & Bond, T. (2006). Research in mathematics education and Rasch measurement. Mathematics Education Research Journal, 18, 1–10. doi:10.1007/BF03217432 Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self determination in human behavior. New York, NY: Plenum Press. doi:10.1007/ 978-1-4899-2271-7 Deci, E. L., & Ryan, R. M. (2002). Self-determination research: Reflections and future directions. In E. L. Deci & R. M. Ryan (Eds.), Handbook of self-determination research (pp. 431– 441). Rochester, NY: University of Rochester Press. Deci, E., & Ryan, R. (2008). Facilitating optimal motivation and psychological well-being across life’s domains. Canadian Psychology, 49, 14 –23. doi:10.1037/0708-5591.49.1.14

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

10

CHEMOLLI AND GAGNÉ

Deci, E. L., & Ryan, R. M. (2012). Motivation, personality, and development within embedded social contexts: An overview of selfdetermination theory. In R. M. Ryan (Ed.), Oxford handbook of human motivation (pp. 85–107). Oxford, United Kingdom: Oxford University Press. doi:10.1093/oxfordhb/9780195399820.013.0006 Edwards, J. R. (2001). Ten difference score myths. Organizational Research Methods, 4, 265–287. doi:10.1177/109442810143005 Engelhard, G., Jr. (2008). Historical perspectives on invariant measurement: Guttman, Rasch, and Mokken. Measurement: Interdisciplinary Research and Perspectives, 6, 155–189. doi:10.1080/ 15366360802197792 Fernet, C., Senécal, C., Guay, F., Marsh, H., & Dowson, M. (2008). The Work Tasks Motivation Scale for Teachers (WTMST). Journal of Career Assessment, 16, 256 –279. doi:10.1177/1069072707305764 Gagné, M., Forest, J., Gilbert, M.-H., Aube, C., Morin, E., & Malorni, A. (2010). The Motivation at Work Scale: Validation evidence in two languages. Educational and Psychological Measurement, 70, 628 – 646. http://dx.doi.org/ doi:10.1177/0013164409355698 Gagné, M., Forest, J., Vansteenkiste, M., Crevier-Braud, L., Van den Broeck, A., Aspeli, A. K., . . . Westbye, C. (2013). The Multidimensional Work Motivation Scale: Validation evidence in seven languages and nine countries. European Journal of Work and Organizational Psychology. Advance online publication. doi:10.1080/1359432X.2013 .877892 Grant, A. M., Nurmohamed, S., Ashford, S. J., & Dekas, K. (2011). The performance implications of ambivalent initiative: The interplay of autonomous and controlled motivations. Organizational Behavior and Human Decision Processes, 116, 241–251. doi:10.1016/j.obhdp.2011.03 .004 Grolnick, W. S., & Ryan, R. M. (1987). Autonomy in children’s learning: An experimental and individual difference investigation. Journal of Personality and Social Psychology, 52, 890 – 898. doi:10.1037/00223514.52.5.890 Grouzet, F., Kasser, T., Ahuvia, A., Fernández Dols, J. M., Kim, Y., Lau, S., . . . Sheldon, K. M. (2005). The structure of goal contents across 15 cultures. Journal of Personality and Social Psychology, 89, 800 – 816. doi:10.1037/0022-3514.89.5.800 Grusec, J. E., & Goodnow, J. J. (1994). Impact of parental discipline methods on the child’s internalization of values: A reconceptualization of current points of view. Developmental Psychology, 30, 4 –19. doi: 10.1037/0012-1649.30.1.4 Guay, F., Vallerand, R. J., & Blanchard, C. (2000). On the assessment of situational intrinsic and extrinsic motivation: The Situational Motivation Scale (SIMS). Motivation and Emotion, 24, 175–213. doi:10.1023/A: 1005614228250 Gurtman, M. B., & Pincus, A. L. (2003). The circumplex model: Methods and research applications. In J. A. Schinka & W. F. Velicer (Eds.), Handbook of psychology: Research methods in psychology (Vol. 2, pp. 407– 428). Hoboken, NJ: Wiley. Guttman, L. (1954). A new approach to factor analysis: The radex. In P. Lazarfeld (Ed.), Mathematical thinking in the social siences (pp. 258 – 348). New York, NY: Free Press of Glencoe. Johns, G. (1981). Difference score measures of organizational behavior variables: A critique. Organization Behavior and Human Performance, 27, 443– 463. doi:10.1016/0030-5073(81)90033-7 Judge, T. A., Bono, J. E., Erez, A., & Locke, E. A. (2005). Core selfevaluations and job and life satisfaction: The role of self-concordance and goal attainment. Journal of Applied Psychology, 90, 257–268. doi:10.1037/0021-9010.90.2.257 Koestner, R., & Losier, G. F. (2002). Distinguishing three ways of being internally motivated: A closer look at introjection, identification, and intrinsic motivation. In E. L. Deci & R. M. Ryan (Eds.), Handbook of self-determination research (pp. 101–121). Rochester, NY: University of Rochester Press.

Koestner, R., Losier, G. F., Vallerand, R. J., & Carducci, D. (1996). Identified and introjected forms of political internalization: Extending self-determination theory. Journal of Personality and Social Psychology, 70, 1025–1036. doi:10.1037/0022-3514.70.5.1025 Lee, W., & Reeve, J. (2013). Self-determined, but not non-self-determined, motivation predicts activations in the anterior insular cortex: An fMRI study of personal agency. Social, Cognitive, and Affective Neuroscience, 8, 538 –545. doi:10.1093/scan/nss029 Lee, W., Reeve, J., Xue, Y., & Xiong, J. (2012). Neural differences between intrinsic reasons for doing versus extrinsic reasons for doing: An fMRI study. Neuroscience Research, 73, 68 –72. doi:10.1016/j .neures.2012.02.010 Li, F., & Harmer, P. (1996). Testing the simplex assumption underlying the Sport Motivation Scale: A structural equation modeling analysis. Research Quarterly for Exercise and Sport, 67, 396 – 405. doi:10.1080/ 02701367.1996.10607971 Linacre, J. M. (1998). Structure in Rasch residuals: Why principal components analysis (PCA)? Rasch Measurement Transactions, 12, 636. Retrieved from http://www.rasch.org/rmt/rmt122m.htm Linacre, J. M. (2003). A user’s guide to WINSTEPS: Rasch-model computer programs. Chicago, IL: MESA Press. Linacre, J. M. (2010). A user’s guide to Winsteps, Ministep: Rasch-Model Computer Programs. Chicago, IL: MESA Press. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Mahwah, NJ: Erlbaum. Mallett, C., Kawabata, M., Newcombe, P., Otero-Forero, A., & Jackson, S. (2007). Sport motivation scale-6 (SMS-6): A revised six-factor sport motivation scale. Psychology of Sport and Exercise, 8, 600 – 614. doi: 10.1016/j.psychsport.2006.12.005 Manser, M. H. (2008). The facts on file dictionary of foreign words and phrases (2nd ed.). New York, NY: FactsOnFile. Marsh, H. W. (1993). Stability of individual differences in multiwave panel studies: Comparison of simplex models and one-factor models. Journal of Educational Statistics, 30, 157–183. doi:10.1111/j.1745-3984.1993 .tb01072.x Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149 –174. doi:10.1007/BF02296272 Meehl, P. E. (1967). Theory-testing in psychology and physics: A methodological paradox. Philosophy of Science, 34, 103–115. doi:10.1086/ 288135 Meyer, J. P., Parfyonova, N., & Gagné, M. (2010). Motivation and commitment. In S. Albrecht (Ed.), Handbook of employee engagement (pp. 62–73). Cheltenham, United Kingdom: Edward Elgar. Miceli, R., Settanni, M., & Vidotto, G. (2008). Measuring change in training programs: An empirical illustration. Psychology Science Quarterly, 50, 433– 447. Millette, V., & Gagné, M. (2008). Designing volunteers’ tasks to maximize motivation, satisfaction and performance: The impact of job characteristics on volunteer engagement. Motivation and Emotion, 32, 11–22. doi:10.1007/s11031-007-9079-4 Pelletier, L. G., Fortier, M. S., Vallerand, R. J., Tuson, K. M., Brière, N. M., & Blais, N. M. (1995). Toward a new measure of intrinsic motivation, extrinsic motivation, and amotivation in sports: The Sport Motivation Scale (SMS). Journal of Sport and Exercise Psychology, 17, 35–53. Pelletier, L. G., Tuson, K. M., Green-Demers, I., Noels, K., & Beaton, A. M. (1998). Why are you doing things for the environment? The Motivation Toward the Environment Scale (MTES). Journal of Applied Social Psychology, 28, 437– 468. doi:10.1111/j.1559-1816.1998 .tb01714.x Raîche, G. (2005). Critical eigenvalue sizes in standardized residual principal components analysis. Rasch Measurement Transactions, 19, 1012. Retrieved from http://www.rasch.org/rmt/rmt191h.htm

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

EVIDENCE AGAINST THE CONTINUUM Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenahagen, Denmark: Danmarks Paedagogiske Institut. Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests. Chicago, IL: University of Chicago Press. (Original work published 1960) Rogosa, D. R., & Willett, J. B. (1985). Satisfying a simplex structure is simpler than it should be. Journal of Educational Statistics, 10, 99 –107. doi:10.2307/1164837 Ryan, R. M. (1993). Agency and organization: Intrinsic motivation, autonomy and the self in psychological development. In J. Jacobs (Ed.), Nebraska symposium on motivation: Developmental perspectives on motivation (Vol. 40, pp. 1–56). Lincoln: University of Nebraska Press. Ryan, R. M. (1995). Psychological needs and the facilitation of integrative processes. Journal of Personality, 63, 397– 427. doi:10.1111/j.14676494.1995.tb00501.x Ryan, R. M., & Connell, J. P. (1989). Perceived locus of causality and internalization. Journal of Personality and Social Psychology, 57, 749 – 761. doi:10.1037/0022-3514.57.5.749 Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55, 68 –78. doi:10.1037/0003-066X.55.1.68 Ryan, R. M., & Deci, E. L. (2002). Overview of self-determination theory: An organismic dialectical perspective. In E. L. Deci & R. M. Ryan (Eds.), Handbook of self-determination research (pp. 3–33). Rochester, NY: University of Rochester Press. Ryan, R. M., Kuhl, J., & Deci, E. L. (1997). Nature and autonomy: An organizational view on the social and neurobiological aspects of selfregulation in behavior and development. Development and Psychopathology, 9, 701–728. doi:10.1017/S0954579497001405 Sheldon, K. M., & Elliot, A. J. (1999). Goal striving, need satisfaction and longitudinal well-being: The self-concordance model. Journal of Personality and Social Psychology, 76, 482– 497. doi:10.1037/0022-3514 .76.3.482 Smith, R. M. (2000). Fit analysis in latent trait measurement models. Journal of Applied Measurement, 1, 199 –218. Tremblay, M. A., Blanchard, C. M., Taylor, S., Pelletier, L. G., & Villeneuve, M. (2009). Work Extrinsic and Intrinsic Motivation Scale: Its

11

value for organizational research. Canadian Journal of Behavioural Science, 41, 213–226. doi:10.1037/a0015167 Vallerand, R. J., Pelletier, L. G., Blais, M. R., Brière, N. M., Senécal, C., & Vallières, E. F. (1992). The Academic Motivation Scale: A measure of intrinsic, extrinsic, and amotivation in education. Educational and Psychological Measurement, 52, 1003–1017. doi:10.1177/ 0013164492052004025 van Alphen, A., Halfens, R., Hasman, A., & Imbos, T. (1994). Likert or Rasch? Nothing is more applicable than good theory. Journal of Advanced Nursing, 20, 196 –201. doi:10.1046/j.1365-2648.1994 .20010196.x Vansteenkiste, M., Sierens, E., Soenens, B., Luyckx, K., & Lens, W. (2009). Motivational profiles from a self-determination perspective: The quality of motivation matters. Journal of Educational Psychology, 101, 671– 688. doi:10.1037/a0015083 Vermunt, J. K., & Magidson, J. (2005). Latent GOLD 4.0 user’s guide. Belmont, MA: Statistical Innovations. Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54, 1063–1070. doi:10.1037/0022-3514.54.6.1063 Williams, G. C., Grow, V. M., Freedman, Z., Ryan, R. M., & Deci, E. L. (1996). Motivational predictors of weight-loss and weight-loss maintenance. Journal of Personality and Social Psychology, 70, 115–126. doi:10.1037/0022-3514.70.1.115 Wright, B. D., & Masters, G. N. (1982). Rating scale analysis: Rasch measurement. Chicago, IL: MESA Press. Wright, B. D., & Stone, M. H. (1979). Best test design: Rasch measurement. Chicago, IL: MESA Press. Zuckerman, M., Gagné, M., Nafshi, I., Knee, C. R., & Kieffer, S. C. (2002). Testing discrepancy effects: A critique, a suggestion, and an illustration. Behavior Research Methods, Instruments, and Computers, 34, 291–303. doi:10.3758/BF03195457

Received May 16, 2013 Revision received December 13, 2013 Accepted January 22, 2014 䡲