Missing:
DOGMATISM AND INNOVATIVENESS: A PSYCHOMETRIC EVALUATION OF A NEW MEASURE OF DOGMATISM FOR MARKETING Dr. Ronald Goldsmith, Florida State University, USA Dr. Ronald Clark, East Carolina University, USA Ms. Kelly Cowart, Florida State University, USA ABSTRACT Although the personality trait of dogmatism has been studied in consumer behavior, its reliable and valid measurement has often been controversial. A new measure of dogmatism has been proposed (Altemeyer 2002), but its psychometric characteristics have not been independently evaluated. The present study evaluates the DOG scale for dimensionality, internal consistency, and nomological validity using data from a sample of 742 US students and adults. The results show that DOG scale is internally consistent (α = .90), but not unidimensional, with three factors emerging from a factor analysis. Although correlations of the three factors as well as the whole scale were consistently negative with two measures of innovativeness, indicating nomological validity for the scale, direction-of-item wording issues should be further examined. INTRODUCTION Dogmatism was originally conceived as a component of the broader concept of authoritarianism (Adorno et al. 1950; Rokeach 1960). It specified a continuum representing the “open” versus the “closed” mind. Highly dogmatic individuals cling to their beliefs, are less open to change, and resist efforts to change their minds. Unfortunately, the scientific study of dogmatism has been hindered owing to the lack of a universally accepted, reliable, and valid measure (Altemeyer 1996, pp. 191-192). The popular Rokeach (1960) scale lacks content validity, is focused on right wing authoritarianism, suffers from poor internal consistency, and all of its items are worded in the portrait direction, possibly leading to yea-saying response bias (Altemeyer 1996, pp. 192-193). A large number of studies have used the scale, but the results are marred by a lack of findings with other variables of interest. To overcome this shortcoming, Altemeyer reconceptualized dogmatism to refer to relatively unchangeable, “unjustified certainty,” meaning that “you will seldom if ever accept that you are wrong” (Altemeyer 1996, p. 201). He subsequently developed a new, balanced dogmatism scale containing ten positively worded (i.e., worded in the dogmatic direction so that agreement indicates high dogmatism) and ten negatively worded (i.e., worded so that the dogmatic response is to disagree) items. Altemeyer (1996, Ch. 8) presents the results of four studies demonstrating the validity and usefulness of the scale. The topics of these studies were themed to pertain to religious beliefs and attitudes. Another test of the scale appears in Altemeyer (2002). This study found that students who claimed to believe every word of the Bible is true and came from God scored higher on the dogmatism scale than those who acknowledged that contradictions and inconsistencies in the Bible exist. Consumer researchers have used the dogmatism trait to try to understand consumer reactions to new products. Jacoby (1971) found that low dogmatics were more likely to choose new brands than were high dogmatics. Durand et al. (1974) found that consumers of a less dogmatic nature perceived lower levels of risk inherent within unfamiliar purchase situations than did more dogmatic individuals. Goldsmith and Goldsmith (1980) found that high dogmatics were more confident in their evaluation of new brands than were low dogmatics. Shimp and Sharma (1987) found a positive relationship between dogmatism and ethnocentrism, implying a resistance to the new and different. Although Reisenwitz and Cutler (1998) failed to find a relationship between dogmatism and Internet usage by university students, the majority of findings show that dogmatism is negatively related to innovative consumer behavior. Thus, we chose this relationship for our test of the nomological validity of the new dogmatism scale. To encourage and promote the further use of the new scale to study dogmatism, the present study was conceived as a psychometric evaluation. The goals were to assess first of all the dimensionality of the scale, a topic not discussed in Altemeyer’s published studies. If dogmatism newly conceptualized is a unitary rather than a multidimensional construct, its measure should be unidimensional. The second goal was to reevaluate the scale’s internal consistency. Altemeyer (1996, p. 204) reports a coefficient alpha for the scale of .90, an excellent value, but additional evidence for internal consistency is lacking in the other reports. The third goal was to test the scale for nomological validity. This is discussed in more detail below. Finally, we sought to gather data from a slightly different population of than those used in other studies to enhance the scope of the scale’s application.
Our test of nomological validity employed the concept of innovativeness. This stable personality trait describes an openness to and willingness to try new things. In the field of consumer research, it is argued that innovative consumers, those who are most likely to buy new products, are likely to be less dogmatic, that is resistant to change, than their less innovative counterparts (e.g., Schiffman and Kanuk 2007, pp. 123-124). Evidence for this belief is mixed. While some studies support the idea that innovators might be less dogmatic (Jacoby 1971) other studies do not (Reisenwitz and Cutler 1998). These inconsistent findings are likely due to many reasons, including the lack of a reliable and valid measure or the problem of matching dogmatism with a criterion at its same conceptual level; there is little reason to believe that right wing authoritarianism is inherently related to acceptance of new things. Thus, we propose a fair test of the hypothesized negative relationship between dogmatism and innovativeness can be made by using two independent measures of innovativeness at the global or general personality level to match the conceptualization of dogmatism as “unjustified certainty.” METHOD Participants Data were collected using 746 paper and pencil questionnaires distributed at a large state university in the southeastern United States. The data collection was part of an undergraduate extra credit exercise in marketing research. Students were asked to complete one survey themselves and then were trained to obtain a non-student quota sample following restrictions. Specifically, each student was instructed to acquire two completed surveys from non-students aged 25 to 34, two completed surveys from non-student consumers aged 35 to 45, and two surveys from individuals 45 and older. Other restrictions placed on the quota sample were: 1) students were instructed to strive for an approximately equal distribution of gender; (2) respondents could not be students or employees of the university; and (3) each questionnaire had to have a valid phone number and first name for the respondent for validation purposes. Four of the questionnaires were deemed unusable, leaving a base sample of 742. Validation calls randomly made to 10% of the respondents to verify that they had indeed completed the questionnaire did not reveal cheating on the part of the data gatherers. The final sample consisted of 360 men (49%) and 380 women (51%), with 2 missing values for sex. Participant ages ranged from 18 to 90, with a mean age of 34.51 years (SD = 13.91). Fifty percent of the participants were 30 years or older. The mean ages of the men and women were not significantly (p < .05) different. Seventy percent of the participants were white, 10.9% were African American, 11.5% were Hispanic American, 3.2% were Asian American, and 4.6% were others or missing. Levels of self-reported education ranged from “Less than a High School Graduate” (.9%) to “A Graduate Degree” (11.6%). Annual household income ranged from $19,000 or less (16%), to $100,000 or more (25%). Thus, although participants were drawn from a variety of socioeconomic levels, the majority came from the upper end of the SES scale. Measures The scales measuring the constructs of interest were combined with other measures not related to this study and demographic items into a paper and pencil questionnaire. Dogmatism was measured using Altemeyer’s (2002) new DOG scale. This consists of 20 items. We chose not to include the first two distractor items so as to make a long questionnaire shorter. We measured global innovativeness using two scales. The first was nine items from the Jackson Personality Inventory (Jackson 1976) and the second was eight items from the Hurt, Joseph, and Cook (1977) scale. A seven-point Likert response format was used (1 = Strongly Disagree, 7 = Strongly Agree) for all the scales. RESULTS We performed preliminary analyses of the two measures of Innovativeness to evaluate them for dimensionality and reliability. A principal axis factor analysis followed by an oblique rotation of the nine Jackson (1976) items showed it to have two factors. This was apparent from examination of the scree plot as well as the two eigenvalues greater than 1.0 (3.6 & 1.3) with a third eigenvalue of .83. The six positively-worded items (i.e., worded in the direction of innovativeness such as “I am always seeking new ways to look at things.”) formed the first factor, and the three negatively-worded items (e.g., “I seldom bother to think of original ways of doing a task.”) formed the second factor. Thus, there was a clear direction-of-item wording confound with the scale that will be discussed later. Removing none of the items contributed meaningfully to increasing internal consistency, so after appropriate item reversals, we summed responses to all nine items to form the measure. Summed scores ranged from 9 to 63 with a mean of 41.88 (SD = 7.89). Higher scores indicated higher levels of innovativeness. Coefficient alpha was .80.
A similar factor analysis of the eight items taken from the Hurt et al. (1977) scale showed that three of the items did not load in a factor analysis with the remaining items and did not contribute to internal consistency, and so they were removed, leaving five items. Two of the included items were worded negatively (e.g., “I am generally cautions about accepting new ideas.”) and three were worded positively (e.g., “I feel that I am an innovative person.”). Removing none of these items improved internal consistency, so after reversing the negative item scores the five items were summed so that higher scores indicated higher innovativeness. The summed scale scores had a mean of 23.01 (SD = 4.51) and ranged from 7 to 35. Because coefficient alpha was only .66, conclusions involving this scale might need to be viewed with caution. The Pearson correlation between the two innovativeness scales was .54 (p < .001), indicating convergent validity. To assess the possible influence of demographics on the focal relationships, we examined their influence on the innovativeness measures. None of the non-parametric correlations (Kendall’s Tau-B) between the two innovativeness scales and the age, education, or income of the participants was statistically significant. Likewise, there were no significant differences in mean scores on these scales across gender or ethnic categories. Thus, the scores on the two innovativeness measures were independent of participant demographics. The first step in the psychometric evaluation of the DOG scale was to reverse code the negatively worded [undogmatic] items so that the higher the score, the more dogmatic the response. Next, we factor analyzed the 20 items using a principle axis extraction followed by an oblique rotation. Using an eigenvalue greater than 1.0 criterion and the scree plot, three factors were extracted. Following a guideline that loadings greater than 3.0 indicated membership with a factor. The first two factors are easy to interpret. Factor 1 contains nine negatively worded or undogmatic items with loadings on the factor greater than 4.0. These nine items appear to share the common idea that truth is not absolute and that one should remain open to new information that might lead one to change their mind. Only one item, number 9, has a sizable cross-loading on another factor. Factor 2 contains seven items worded in the positive direction, that is, they all express the confident opinion that the participant knows the truth about things. This is the core idea of the dogmatism concept. Factor 3 is somewhat problematical. Three of these four items are positively worded, but one item, number 19, is negatively worded. Of particular interest are two concerns with these items. First, with the exception of item 17, their loadings on their primary factor are low, and second, two of the items have substantial cross-loadings on at least one other factor, suggesting that their meaning is not as “pure” as the other items, that participants may be faced with some ambiguity in interpreting them. Thus, our interpretation of the structure of the DOG scale is that direction-of-item wording might lead respondents to respond in slightly different ways, leading to a multi-dimensional scale. The third factor may simply be poorly written items to which respondents have difficulty relating to their own feelings about themselves. Alternatively, these four items appeared at the end of the scale, in positions 17, 15, 19, and 20 respectively, so that fatigue with the length of the scale asking similar questions may have lead to their poor psychometrics compared with the other items. The third step in the analysis was to compute coefficient alpha for the scale and to examine item statistics (i.e., interitem correlations, item-to-total correlations, and item means). The diagnostic statistics showed that although item 1 had the lowest squared multiple correlation (.26) followed by item 20 (.28), all the items contributed to alpha, and dropping none of them would substantially improve it beyond the .90 level. Thus, with a mean interitem correlation of .30 and an alpha of .90, these results compare favorably with those reported by Altemeyer (1996, p. 204). Item-to-total correlations for the scale ranged from .38 to .63 (M = .52). The mean item-to-total correlations exceed the recommended threshold of .50 (Hair et al. 2006; Robinson, Shaver, and Wrightsman 1991). Item means for the scale ranged from 2.3 to 4.2 (M = 3.3). Item means closer to the midpoint of the response choices (in this case 4) are viewed as exhibiting better measurement properties than item means at or near the anchor points of the scale (DeVellis 1991). For the final step in the scale’s evaluation we added the item responses to form four scales: a nine-item scale consisting of the first factor items (DOG1), a seven-item scale consisting of the second factor items (DOG2), a four-item scale representing the third factor (DOG3), and finally, the 20 item scale (DOG). Dogmatism scores were correlated with the scores on the two innovativeness scales. As hypothesized, all correlations were negative, supporting the hypothesis that dogmatism, an unwillingness to change ones mind, and innovativeness, a willingness to try new things, are negatively related (r’s = -23 & .33). This finding supports the nomological validity of the DOG scale. However, the substantial differences in the size of the correlations between DOG1 and DOG2 with the innovativeness items suggest that the direction in which these items are worded might influence statistical findings. We conclude with two issues. First, we assessed the relationships between demographic data and the dogmatism scale. Scores on the DOG scale were uncorrelated with the measures of age, education, and income. There was no statistically significant
difference in the mean scores for men and women, nor were there any mean differences for race. Finally, we constructed a ten-item short form for the DOG scale consisting of the five positively and five negatively worded items with the highest factor loadings on the first two factors. This scale had a coefficient alpha of .83, and was negatively correlated with the innovativeness scales (Jackson r = -.22, p < .01 and Hurt r = -.29, p < .01). Item statistics in the abbreviated scale are improved over the psychometric properties of the original scale. For example, item-to-total correlations for the abbreviated scale ranged from .44 to .61 (M = .52). The mean item-to-total correlations exceed the recommended threshold of .50, and the minimum item-to-total correlation was higher than 4 of the original items. Item means for the abbreviated scale ranged from 2.8 to 3.9 (M = 3.5). Item means for the abbreviated scale are closer to the midpoint (4) of the response choices exhibiting better measurement properties than the original version of the scale. Thus, a minor sacrifice in internal consistency yields a scale that can be used instead of the long form, provided one is concerned with using a balanced scale. DISCUSSION The purpose of the present study was to conduct a psychometric evaluation of Altemeyer’s (2002) newly proposed dogmatism scale. Using data from a large heterogeneous sample, we assessed its dimensionality, internal consistency, and nomological validity. The results showed that although the scale was not unidimensional, this was most likely due to direction-of-item wording (Goldsmith 2000; Spector et al. 1997). It had satisfactory internal consistency, and its negative correlations with two measures of innovativeness supported its nomological validity. Moreover, scores were unrelated to several demographic characteristics. Overall, the findings suggest that the scale might be a useful tool in the study of dogmatism conceptualized as unjustified certainty. It has good internal consistency, overcoming many of the shortcomings apparent in earlier scales. It seems to be appropriate as a general measure of dogmatism free from any political overtones and useful for any adult population. In cases were the full twenty-items are too long for a researcher’s study, the short form proposed here might work nearly as well, with only some loss of internal consistency. The new scale seems to be more content valid than older scales, capturing the key ideas of the conceptual definition and not limited to a right-wing authoritarianism bias. Moreover, the balanced wording of the new scale overcomes the problems associated with Rokeach’s scale where all the items were worded in the portrait direction (Altemeyer 1996, pp. 192-193). Unfortunately, the correction might have introduced another problem: direction-of-item wording (Herche and Engelland 1996; Miller and Cleary 1993). Note that the items worded in the undogmatic direction were somewhat more highly related to the innovativeness scales, so that there may be a yeasaying component to responses to these items. Using the total, balanced scale might be advisable to help mitigate the effects of yeasaying, but it obviates a unidimensional scale and might introduce its own interpretational problem (Schriesheim and Eisenbach 1995). This might be a purely methodological artifact or it raises the possibility that (and this might hold true for both the innovation scales as well) these factor analytic findings are revealing an aspect of these constructs themselves. It might be the case that what these scale score distributions are showing us is that only some people are unambiguously high or low on dogmatism (or innovativeness). Many people might be honestly consistent in agreeing with items that reflect both predispositions, openness to new ideas and resistance to change, simultaneously! They might be telling us that sometimes they feel one way and sometimes the other. Theories of individual differences assume a consistency across situations that might not describe the psychological reality for most people. If further studies replicate the finding that some of the items do not load on one of the two main factors, additional item analysis might need to be performed, and new items added to the scale to correct this problem. The short form of the scale might be useful where brevity is a concern. Dogmatism has not received a great deal of research attention by consumer researchers, perhaps owing to shortcomings in its measurement. This new measurement instrument should permit consumer researchers to investigate the potential influence of dogmatism on consumer reaction to marketing stimuli. The purpose of advertising and promotion is often to change consumers’ minds in favor of a promoted brand. Consumers who hold fast to their beliefs represent a particularly difficult challenge. Study of how dogmatism influences consumer reaction to persuasion can lead to appeals that are more effective. Our study has several limitations. Use of a convenience sample does not permit generalizations of point and interval estimates to larger populations. People at the lower end of the socio-economic scale were underrepresented, as were people from other geographic areas of the country. Finally, we were able to evaluate nomological validity of the DOG scale with only one other construct among many possible members of its nomological network.
Future studies to evaluate the psychometrics of the scale should use different samples of participants to extend the scope of the findings. Broadening the scope of the concepts used to evaluate the nomological validity of the scale would enable researchers to better understand the role dogmatism plays in belief formation and in behavior. Confirmatory factor analysis techniques would permit more fine-grained examination of trait, method, and error elements of the scales. Finally, a reliable and valid instrument permits researchers to focus on understanding the etiology of dogmatism as well as its consequence. Better understanding the construct thus can be gained, assuring its place in the larger study of authoritarianism. REFERENCES Altemeyer, Bob. 1996. The Authoritarian Specter. Cambridge, MA: Harvard University Press. Altemeyer, Bob. 2002. “Dogmatic Behavior among Students: Testing a New Measure of Dogmatism.” Journal of Social Psychology, 142(6), 713-717. DeVellis, Robert F. 1991. Scale development: Theory and applications. Newbury Park, CA: Sage Publications. Durand, Richard M., Duane L. Davis, and William O. Bearden. 1977. "Dogmatism as a Mediating Influence on the Perception of Risk in Consumer Choice Decisions," The Journal of Psychology 95: 131-138. Goldsmith, Ronald E. 2000. "Effects of Direction-of-Item Wording on Scale Psychometrics," in Society for Marketing Advances, Dawn R. Deeter-Schmelz and Timothy P. Hartman (Ed.), pp. 134-139. Goldsmith, Elizabeth B., and Goldsmith, Ronald E. 1980. Dogmatism and Confidence as Related Factors in Evaluation of New Products. Psychological Reports 47: 1068-1070. Hair, Joseph E., Black, William C., Babin, Barry J., Anderson, Rolph E., and Tatham, Ronald L. 2006. Multivariate Data Analysis (6th ed.). Upper Saddle River, NJ: Pearson Prentice Hall. Herche, Joel and Brian Engelland. 1996. "Reversed-Polarity Items and Scale Unidimensionality," Journal of the Academy of Marketing Science 24 (4): 366-374. Hurt, H. T., Joseph, Katherine, and Cook, Chester D. 1977. Scales for the Measurement of Innovativeness. Human Communications Research 4(1): 58-65. Jackson, Douglas N. 1976. Jackson Personality Inventory Manual. Goshen, NY: Research Psychologists Press. Jacoby, Jacob. 1971. "Personality and Innovation Proneness," Journal of Marketing Research, 8 (May), 244-247. Miller, Timothy R. and T. Anne Cleary. 1993. "Direction of Wording Effects in Balanced Scales," Educational and Psychological Measurement 53: 51-60. Reisenwitz, T. H., and Cutler, B. D. 1998. Dogmatism and Internet Usage by University Students: Are Dogmatics Late Adopters? Journal of Marketing Theory and Practice 6(3): 43-50. Robinson, John P., Shaver, Phillip R., and Wrightsman, L S. 1991. “Criteria for Scale Selection and Evaluation.” In Measures of Personality and Social Psychological Attitudes. Eds. J. P. Robinson, P. R. Shaver, and L. S. Wrightsman, San Diego: Academic Press, pp. 1-15. Schiffman, Leon G., and Kanuk, Leslie L. 2007. Consumer Behavior (9th ed.). Upper Saddle River, NJ: Prentice Hall. Schriesheim, Chester A. and Regina J. Eisenbach. 1995. "An Exploratory and Confirmatory Factor-Analytic Investigation of Item Wording Effects on the Obtained Factor Structures of Survey Questionnaire Measures," Journal of Management 21 (6): 1177-1193. Spector, Paul E., Paul T Van Katwyk, Michael T. Brannick, and Peter Y. Chen. 1997. "When Two Factors Don't Reflect Two Constructs: How Item Characteristics Can Produce Artifiactual Factors," Journal of Management 23 (5): 659-677.
