Effects of Online Comments on Smokers ... - Wiley Online Library

Journal of Computer-Mediated Communication

Effects of Online Comments on Smokers’ Perception of Antismoking Public Service Announcements∗ Rui Shi Annenberg School for Communication, University of Pennsylvania, 3620 Walnut Street, Philadelphia, PA 19104

Paul Messaris Annenberg School for Communication, University of Pennsylvania, 3620 Walnut Street, Philadelphia, PA 19104

Joseph N. Cappella Annenberg School for Communication, University of Pennsylvania, 3620 Walnut Street, Philadelphia, PA 19104

On YouTube antismoking PSAs are widely viewed and uploaded; they also receive extensive commentary by viewers. This study examined whether such evaluative comments with or without uncivil expressions influence evaluations by subsequent viewers. Results showed PSAs with positive (i.e. antismoking) comments were perceived by smokers as more effective than PSAs with negative (prosmoking) comments. Smokers in the no-comment condition gave the highest perceived effectiveness score to PSAs. Smokers’ readiness to quit smoking moderated the effect of comments on PSA evaluation. Smokers reading negative uncivil comments reported more negative attitude toward quitting and a lower level of perceived risk of smoking than those reading negative civil comments but positive civil and positive uncivil comments did not elicit different responses. Key words: Antismoking PSA, Recommendation Systems, Social Influence. doi:10.1111/jcc4.12057

Information in the age of the Internet no longer flows one way from media to audience. Audiences create media as well as comment on it. Various news websites allow comments in response to virtually every news story. Amazon not only allows potential buyers to read previous buyers’ reviews of the product, but also recommends books that they might find interesting based on what ‘‘people like you’’ have bought. These are all examples of online recommendation systems, which allow individuals to express and share their opinions. It is important to understand online recommendation systems because whatever media product it accompanies – a piece of news, a music video, or an antismoking public service announcement (PSA) – the information carried by the recommendation system synchronizes with the media product and thus becomes a part of the message sent to the audience. Among various forms of online recommendation systems, commentary is of particular interest because of the amount of information it carries. While rating scores only describe the valence of the social norm, comments can simultaneously tell viewers how good or bad other people feel about ∗ Accepted

by previous editor Maria Bakardjieva

Journal of Computer-Mediated Communication 19 (2014) 975–990 © 2014 International Communication Association

975

something and why they feel this way. Incivility, sometimes referred to as ‘‘flaming,’’ has been a problem common to online comments and reviews. This study explores one form of online recommendation system, i.e. commentary, and its potential impact on audiences’ responses to the core message that is the object of the commentary. The objects selected here are health messages, specifically antismoking ads.

Impact of Online Recommendation System Recommendation systems can affect people by creating the sense that a social consensus exists and in the process of presenting that consensus increasing pressures for conformity. In a study investigating why experts often failed to predict the market performance of cultural products such as songs, books, and movies, website users were given the chance to download unknown songs from unknown bands for free. Those in the control condition were only given a list of the song names while those in the social-influence condition were given the name of each song as well as how many times each song had been downloaded by previous listeners. Researchers found that unless a song was of very high quality, its success was unpredictable because of the social influence provided through the online recommendations (Salganik, Dodds, & Watts, 2006). If people’s evaluations can be swayed by the preferences of previous users, it would not be surprising that comments left by previous audience could also influence subsequent audience responses. Walther, DeAndrea, Kim, and Anthony (2010) showed that antimarijuana PSAs on YouTube were evaluated by college students as more effective when accompanied by positive (that is antidrug and proad) comments compared with the same PSAs accompanied by negative (that is prodrug and antiad) comments. The effect of comments on PSAs’ perceived effectiveness was particularly strong for those who identified with the prior commenters. Walther and colleagues (2010) directly addressed the effects of online comments, but only in a relative sense: They assessed the PSA in a positive-comment condition relative to that in the negativecomment condition. The results indicate that positive comments reinforced the PSA better than negative comments, but without a control group it is impossible to know whether comments of either valence actually improve or harm the effectiveness of the PSA than when it stands alone. The current study aims to address this question by including a control group. In addition to comment valence, Walther et al. (2010) manipulated message intensity and profanity in the original design but this factor yielded no effects. Since their population was of college age, the absence of effects from incivility may not generalize to a more representative population. The Walther et al. study is a significant step in understanding the impact of recommendation systems. We will add to their work by examining a different content domain (tobacco control), employing a representative sample of smokers, including a no-comment control to assess the impact of commentary at all, and distinguishing factorially positive and negative substantive comments from civil and uncivil ones. In addition, our study presents comments in a more ecologically valid way, mimicking more closely the distribution of positive and negative comments on YouTube in response to a PSA rather than providing an unrealistic distribution of uniformly positive or negative comments. Although the research on recommendation systems is relatively new and limited in its breadth and depth, other research that predates recommendation systems but focuses on social influence under certain and uncertain conditions is relevant to the possible effects of recommendations. These studies have potential implications for online recommendation systems with civil and uncivil commentary. If online comments are an indicator of social norms, then research on conformity effects as one form of social influence may help us to understand the potential power of online commentary. The autokinetic experiments conducted by Muzafer Sherif (1935, 1936) were among the earliest 976


investigations into people’s compliance with social norms and associated perceptions. They showed people’s perceptions of the physical world could be shaped by others’ expressed opinions. Social influence can also shape people’s attitudes and beliefs. The Asch paradigm examined conformity effects of a minority to a majority (e.g. Asch, 1956) demonstrating conformity even when stimuli were clear and majority opinion erroneous. Personal attitudes can also be affected by perceived public opinion (Sonck & Loosveld, 2010). In both laboratory and field experiments, attitudes were found to be swayed by reports of public opinion polls, especially when initial attitudes are neutral or when the issues of little personal relevance (Giner-Sorolila & Chaiken, 1997; Kang, 1998).

Profanity and Persuasion Incivility, sometimes referred to as ‘‘flaming,’’ has been a problem common to online comments and reviews. The use of uncivil language has been found to be more prevalent and obvious in computer-mediated communication than in face-to-face situations (Orenga, Zornoza, Prieto, & Peiro, 2000). Cautions have been raised on the uncivil atmosphere surrounding political discussion on social network sites (Kushin & Kitchener, 2009) as well as on the prevalence of profanity among online health service sites and forums from which women seek help (Finn & Banach, 2000). A survey of YouTube users finds flaming to be common among commenters who used offensive language because they perceive flaming as normative (Moor, Heuvelman, & Verleur, 2010).The impact of uncivil postings on the internet has been explored but findings have been inconclusive. People report they sometimes refrain from uploading videos to avoid flaming comments (Moor, Heuvelman, & Verleur, 2010). In an experimental test in the political arena, participants’ intention to participate in discussion was not influenced by the impolite language involved. Discussants in the uncivil condition were perceived as more dominant and less credible, but incivility did not decrease the persuasiveness of the content (Ng & Detenber, 2005). Although the general research on profanity and incivility in traditional settings is extensive, general conclusions are difficult to infer as contexts play a significant role in the direction of effects. On the one hand, incivility has been shown to decrease trust (Mutz & Reeves, 2005), credibility (Bostrom, Baseheart, & Rossiter, 1973), liking (Mutz, 2007), perceived sociointellectual status and aesthetic quality (Mulac, 1976) of the speaker. These factors can contribute to a communicator’s persuasiveness in some circumstances (e.g. Eagly & Chaiken, 1975). On the other hand, incivility increases people’s perception of the speakers’ intensity (i.e. dynamism — how passionate, strong, and enthusiastic the speaker is) and thus increases their persuasiveness (Scherer & Sagarin, 2006). The relationship between incivility and persuasion becomes more complicated when taking into account people’s pre-existing attitudes. In some cases people tend to forgive uncivil comments if they agree with the content, but they punish the speaker for incivility if the speaker is on the opposing side (Mutz, 2007). In other cases people reward uncivil speakers in proattitudinal condition but do not differentiate civil and uncivil speakers in counterattitudinal conditions (Scherer, 2007). These previous studies on social influence and incivility in traditional settings help explain the possible underlying mechanisms of possible effects and thus postulate potential impact of online recommendation systems.

Effectiveness of Antismoking PSAs Many health messages are posted on YouTube including antismoking PSAs. There are various motives for people or organizations to upload the PSAs online, sometimes to attempt to reach out, sometimes Journal of Computer-Mediated Communication 19 (2014) 975–990 © 2014 International Communication Association

977

to poke fun or ridicule. Regardless of the motive, those PSAs are there and their presence is almost a cottage industry. A great deal of research has been directed at the elements that help to create effective PSAs especially in the tobacco control arena. Researchers have evaluated the role of argument strength (Strasser et al., 2009), content themes (Sutfin, Szykman, & Moore, 2008), and various format features like smoking cues (Kang, Cappella, Strasser, & Lerman, 2009), and sensation value (Strasser et al., 2009). However, less emphasis has been given to role of the social context within which PSAs operate and the impact of contextual factors on overall ad effectiveness. Durkin and Wakefield (2006) for example found that daily smokers who discussed the antismoking PSAs after watching them were more motivated to quit. More relevant to the current study is an experiment that compared the effects of an antismoking PSA on college students when it was accompanied by discussion of different sources. Findings showed that the PSAs followed by friends’ discussion about the harm of smoking had a larger impact on students’ attitude toward smoking compared with the PSAs followed by strangers’ discussion (Samu & Bhatnagar, 2008). The result further illustrates that social influence has the potential to alter – positively or negatively — the effectiveness of antismoking PSAs.

Theoretical Background Recommendation systems present information that conveys opinions from members of the public. The information does offer what others think and in that sense offers normative considerations. Past research on social influence and conformity has showed a perceived social norm can lead people to follow the opinion or behavior of others because people are motivated by the desire to be accurate, to fit in, and to maintain a positive self-concept (see Cialdini & Goldstein, 2004 for a review). Two types of norms were theorized to have an impact on human actions, injunctive norms (what the majority approve) and descriptive norms (what the majority do) (Cialdini, Reno, & Kallgren, 1990). Theory of Normative Social Behavior (Rimal & Real, 2005) later argued the descriptive norms have a direct effect on behavior and the injunctive norms (together with outcome expectations and group identity) serve as moderator of such association, i.e., perceived prevalence of the behavior would only promote the behavior if the behavior is socially approved. The Integrative Model of Behavior Prediction (IM, Fishbein, 2008) depicted a slightly different path for normative pressures to affect behavior. Descriptive norms and injunctive norms are both considered direct determinants of behavioral intention. The IM posits that people’s intention to take a specific action relies on three factors: their own attitude towards the target action (i.e. how favorable they feel towards performing the behavior), the injunctive and descriptive normative pressure they perceive concerning this action (i.e. what they perceive others may approve or do), and the self-efficacy perceived to perform such action (i.e. how capable they think they are to perform the behavior if they want to). The relative weight of each of the three determinants of intention varies for different populations and behaviors. Applying IM to the scenario of smokers watching antismoking PSAs, IM would predict that smokers generate their own judgment about the PSA, but at the same time they form injunctive normative beliefs as well as descriptive normative beliefs by reading comments posted by fellow viewers. Such social influence, however, may not be strong or even pertinent because the injunctive norms in IM requires motivation to comply with significant others (e.g. what would my mom want me to do) and the descriptive norms in the IM specifies perceived prevalence among peers (e.g. what would smokers like me say of this video). The distribution of positive and negative comments gives viewers a sense of general social approval or disapproval as commenters are members of the public at large, but these commenters are anonymous, their smoking status mostly unstated, and the readers have no interpersonal ties to the individual commentators. 978


The current study was built in a YouTube-like visual context with anti-smoking PSAs as the media product under investigation. On YouTube, antismoking PSAs are widely viewed and uploaded; they also receive extensive commentary by viewers. Viewers of these health messages are also exposed to the comments accompanying them. This study examined whether such evaluative comments influence judgments about the PSAs when viewed subsequently by others. In line with the Integrative Model and previous studies on online recommendation system and social influence it was hypothesized that: H1. Comment valence influences smoker’s perceived effectiveness (PE) of the PSA such that positive (i.e. proad or antismoking) comments make the PSA more persuasive and negative (i.e. antiad or prosmoking) comments make the PSA less persuasive. As IM suggests the weight of normative pressure relative to attitude and self-efficacy may vary under different circumstances for different population, it was further hypothesized: H2. PSA quality moderates the influence of comments on people’s PE of the PSA such that the effect of comments will be particularly strong when people are unsure of the PSA. H3. Smokers’ readiness to quit smoking moderates the influence of comments on their PE of the PSA such that the effect of comments will be particularly strong when smokers are more (versus less) ready to quit smoking. Since findings from the previous studies on incivility are mixed and lead to conflicting predictions we only raise a research question: RQ. How will uncivil comments influence people’s PE of the PSA, attitudes, perceived risk, and intention to quit smoking?

Method Participants and Study Design A total of 592 adult smokers were recruited through Knowledge Networks, a survey research company which has developed a representative panel of adults in the United States. To be eligible for the study, a subject must be a regular smoker who (a) smokes cigarettes currently, (b) smokes an average of five or more cigarettes a day in the past week, and (c) smoked more than 100 cigarettes in their lifetime. Panelists who failed to meet all three criteria were thanked and excluded from the study sample. The final working sample has a mean age of 49.47 years old. Most of the participants had finished high school (93.8%), approximately half (50.8%) were males, 82.1% were White, 7.1% were African-American, 5.6% were Hispanic, 2.7% mixed, and 2.5% marked ‘‘other.’’ Subjects have smoked 33.83 years on average (SD = 12.91), and they have previously quit smoking 4.62 times (SD = 8.45) on purpose for more than one complete day. Fagerstrom Test for Nicotine Dependence (FTND) was used to measure participants’ addiction level (Heatherton, Kozlowski, Frecker, & Fagerstrom, 1991). Participants for the current study scored 4.17 on average (SD = 2.21) which indicates moderate nicotine dependence. This study adopted a 2 comment valence (positive vs. negative) x 2 comment tone (civil vs. uncivil) x 2 PSA quality (strong vs. weak) mixed experiment design. Comment valence and comment tone were manipulated between subjects and PSA quality within subjects. There were two control conditions as well, one with no comment and the other with mixed comments. Journal of Computer-Mediated Communication 19 (2014) 975–990 © 2014 International Communication Association

979

Antismoking PSAs and Comments Four strong and four weak PSAs were selected from a pool of 99 previously rated antismoking PSAs. Strong PSAs were those rated as top ten in perceived effectiveness by a separate sample of adult smokers; weak PSAs were those rated in the bottom 10. Every PSA had 10 comments appended to it. Comments were defined as positive if they supported the PSA or were against smoking; comments were defined as negative if they criticized the PSA or supported smoking. Comments are categorized as uncivil if they a) include swear words (e.g. fuck; shit; damn, etc.); or b) involve personal attack (e.g. smokers should all die); or c) are insulting (e.g. stupid, retarded); or d) use extremist references (e.g. Nazi, Hitler). To avoid case-category confound (Jackson, 1992), a pool of 292 comments was created. These were evenly distributed across four comment treatment conditions, which means a quarter of the comments in the pool were positive civil (PC), a quarter positive uncivil (PU) a quarter negative civil (NC), and the last quarter negative uncivil (NU). All the comments were selected from YouTube comment page and some were modified slightly to fit the treatment conditions. Procedure When entering the online survey participants were told the purpose of the study was to ‘‘test a new website where people can share health related video clips, public service announcements and advertisements.’’ They answered questions about their smoking history and current level of readiness to quit smoking before the PSA viewing. Subjects were randomly assigned to one of the six comment conditions (PC, PU, NC, NU, Control - mixed comments, and Control - no comment). One strong PSA and one weak PSA were randomly selected out of the eight PSAs for each participant and the selected two PSAs were shown in random order. For treatment conditions, 10 comments were randomly generated from the comment pool for each PSA according to the condition. It is not common to see 10 comments in a row on YouTube with the same valence and tone. To increase the ecological validity of the study, comments were selected in the ratio of 8/2 for valence and 8/2 for tone for the treatment groups. When combined, every Valence x Tone condition keeps a ratio of 7:1:1:1. So for example a PSA in the PU condition would be accompanied by 7 PU comments, 1 PC, 1 NC and 1NU yielding 8 positive and 2 negative comments; 8 uncivil and 2 civil comments. For the mixed comments control group, comments were selected in a ratio of 5/5 for valence and 5/5 for tone. The no-comment control group received only the PSA. The order of the 10 selected comments was randomized before they were presented to the participants. PE of the PSA was surveyed right after each exposure to a PSA and its accompanying comments. The design strategy we use gives every subject a unique set of 10 comments for each PSA. In other words, the chance of two participants watching the same PSA accompanied by the same 10 comments in the same order was virtually zero. Therefore, any observed effects cannot be attributed to a specific set of comments attached to each condition. The usual case-category confound was effectively eliminated. Measures Dependent Variables Perceived Effectiveness of the PSA was assessed with three components — convincingness, confidence, and smoking-related thought (Zhao, Strasser, Cappella, Lerman, & Fishbein, 2011). Convincingness was measured with an item that read (a) ‘‘This ad was convincing’’; confidence was measured with an item saying (b) ‘‘Watching this ad helped me feel confident about how to best deal with smoking’’; smoking-related thought was measured using the difference of two items: (c) ‘‘The ad put thoughts 980


in my mind about quitting smoking,’’ and (d) ‘‘The ad put thoughts in my mind about wanting to continue smoking.’’ All items were measured on a 5-point scale ranging from ‘‘1- strongly disagree’’ to ‘‘5- strongly agree.’’ PE score of the PSA was calculated as PE = [a + b + (c – d)/2 + 3] / 3 (Cronbach’s α = .74 for the three components). Attitude, perceived risk, and intention to quit were asked once after exposure to the PSAs and comments. Attitude was measured with a 5-point scale composed of eight items describing the possible benefits of quitting and harms of smoking (Cronbach’s α = .91). Examples are ‘‘I would add years to my life if I quit smoking completely and permanently in the next 3 months,’’ and ‘‘If I continue to smoke cigarettes at my current pace, it is likely that I will get heart disease.’’ Subjects indicated their perceived risk level by locating how much they ‘‘worry about your health risks because of your smoking’’, ‘‘think about the serious health effects of smoking,’’ and ‘‘feel afraid or anxious about your smoking’’ on a four point scale ranging from ‘‘not at all’’ to ‘‘very much’’ (Cronbach’s α = .88). Finally participants’ intention to quit smoking in the next three months was assessed on a 4-point scale (definitely will not – definitely will) that included three items: ‘‘quit smoking completely and permanently,’’ ‘‘reduce the number of cigarettes you smoke in a day,’’ and ‘‘talk to someone (friend, family member, spouse) about quitting smoking’’ (Cronbach’s α = .84).

Moderators and Covariates A modified version of the Ladder of Contemplation (Biener & Abrams, 1991) was used to measure participants’ level of readiness to quit smoking. Participants were asked to choose a number between 0 to 10 to indicate where they were now at in thinking about quitting smoking. Statement at 0 read ‘‘I have no thoughts about quitting smoking,’’ and 10 meant ‘‘I am taking action to quit smoking.’’ Study sample averaged 5.52 on the 11-point ladder (SD = 2.87). Subject’s readiness to quit was recoded into three categories for the analysis: a score of 0 to 2 was considered of low readiness to quit (20.17%), 3 to 7 were coded as moderate readiness (49.28%), and subjects scoring 8 to 10 were of high readiness (30.55%). People’s familiarity with online comments in general was measured with the question ‘‘How often do you read comments left by previous viewers?’’ using a 4-point scale ranging from ‘‘1- never’’ to ‘‘4always’’ (M = 2.38, SD = .73). Subjects in all but the no-comment conditions were asked the number of comments read. 55.3% of the respondents reported reading all the comments, 28.3% read some, 10% read a few, and 6.4% none of the comments. Subjects across conditions did not differ on the number of comments read. This variable served as a covariate in the model and no one was dropped from the study as a result of their response to this question. Mediators Transportation describes how much people felt ‘‘transported’’ into the ad and became involved with it (Green & Brock, 2000). It was measured on a 5-point scale ranging from ‘‘strongly disagree’’ to ‘‘strongly agree’’ with five items (Cronbach’s α = .86 and .92 for the first and second exposure respectively). Sample items are: ‘‘I could picture myself in the scene of the events shown in the ad,’’ and ‘‘The events in the ad are relevant to my everyday life.’’ Subjects’ emotional reaction to the PSA was assessed with five items asking how much they agree or disagree that they felt afraid, guilty, angry, hopeful, and proud. The first three emotions were averaged to indicate Negative Emotion, and the last two Positive Emotion. Journal of Computer-Mediated Communication 19 (2014) 975–990 © 2014 International Communication Association

981

Two scales were created to measure subjects’ reactance to the PSA. The Exaggeration scale asked to what extent did people think the information presented about smoking was exaggerated / dishonest / fake / insulting / stupid (Cronbach’s α = .92 and .93 for the first and second exposure). The Manipulation scale has four items measuring how much people felt they were manipulated by the ad (Cronbach’s α = .90 and .92 for the first and second exposure). Items included ‘‘this ad tried to make a decision for me,’’ and ‘‘this ad tried to pressure me.’’

Results Manipulation Check The comment manipulation was successful. Among the comment conditions subjects in the positive condition considered the comments to be most favorable to the PSA, and subjects in the negative condition perceived the comments to be most critical of the PSA (M positive = 3.27, SD = .78; M mixed = 2.71, SD = .81; M negative = 2.25, SD = .76), F (2, 457) = 78.59, p < .001. Those in the civil condition thought the comments were respectful, and those in the uncivil condition believed the comments were impolite (M civil = 2.65, SD = .85; M mixed = 2.37, SD = .83; M uncivil = 2.06, SD = .87), F(2, 459) = 22.19, p < .001. All six single degree of freedom contrasts (favorable – critical: positive vs. negative, positive vs. mix, negative vs. mix; respectful – impolite: civil vs. uncivil, civil vs. mix, uncivil vs. mix) were significant after applying Holm Bonferroni to adjust for family-wise error with the weakest contrast having a mean difference of .28 on a 5-point scale, t (276) = 2.60, p = .01.

Table 1 Analysis of Covariance of Perceived Effectiveness Across PSA Quality by Comment Valence, Comment Tone, and Smokers’ Readiness to Quit Source

df

Number of Comments Read (N) Comment Valence (V) Comment Tone (T) Readiness to Quit (R) V×T V×R V×T×R Subjects within-group PSA Quality (Q) Q×V Q×R Q × Subjects within group

Between subjects 1 1 1 2 1 2 2 357 Within subjects 1 1 2 357

F

η2

p

16.14∗ 6.20∗ 0.52 15.31∗ 0.85 3.16∗ 1.74 (.69)

.04 .02 .00 .08 .00 .02 .01

.00 .01 .47 .00 .36 .04 .18

15.27∗ 0.75 6.31∗ (.34)

.04 .00 .03

.00 .39 .00

Note: Values enclosed in parentheses represent mean square errors. Interaction terms T × R, Q × N, Q × T, Q × V × T, Q × V × R, Q × T × R, Q × V × T × R were included in the model but were not reported in the table since they are of little theoretical value and their effects were not significant. ∗ p < .05 982


Hypothesis Testing A four-way mixed design ANCOVA was first conducted, with two levels of comment valence (positive vs. negative), two levels of comment tone (civil vs. uncivil), the third factor (within subject) for PSA quality (strong vs. weak), and the fourth factor included three levels on smokers’ readiness to quit (low vs. moderate vs. high). Number of comments read served as the covariate in the model. Table 1 presents the results of the hypothesis tests and Table 2 the means by condition for PE. A significant main effect was found for comment valence, F(1, 357) = 6.20, p = .01, indicating positive comments generated a higher PE than negative comments. Strong PSAs remained relatively strong and weak PSAs remained relatively weak no matter what comments accompanied them, F(1, 357) = 15.27, p < .001, but PSA quality did not interact with comment valence (Q X V in Table 1), F(1, 357) = .75, p = .39. Thus, the hypothesis that the effect of comments would be particularly strong with weak PSA was not supported. The interaction term between comment valence and smokers’ readiness to quit was significant (V X R in Table 1), F(2,357) = 3.16, p = .04, indicating the effect of comments’ valence on PE depends on smokers’ readiness to quit. Planned contrasts showed that positive and negative comment conditions do not differ on PE for smokers who are not ready to quit, t(357) = −.94, p = .35, but for those who are moderately or highly ready to quit smoking positive comments make them evaluate the PSA as more effective than the negative comments, t moderate readiness (357) = 3.07, p = .002; t high readiness (357) = 2.51, p = .01. Deleterious Effect of Comments The two control conditions (the mixed-comments condition and the no-comment condition) were not included in the analysis above because we sought to replicate Walther et al.’s findings with a different sample and context. The effects of any directional comments (pro or con) on the ads require comparison to two types of controls – no comment and balanced comments. So further analyses were conducted to compare the average PE across four valence conditions (positive, negative, mixed, none). One-way ANOVA shows significant differences among the four comment-valence conditions, F(3, 546) = 7.31, p < .001. Planned contrasts indicate that positive comments fail to improve PE compared with the no-comment condition. On the contrary, positive comments had a lower PE than the no-comment condition, though the difference was only marginally significant, t(549) = 1.72, p = .08. As can be seen from Table 2, PSAs with no comments received the highest PE of all conditions. Planned contrasts comparing the no-comment group with all the other groups combined showed the no comment condition had a significantly higher PE, t(549) = 3.21, p = .002, indicating overall the existence of comments decreases the PE of the antismoking PSAs. Table 2 Sample Size and Mean (standard deviation) of Perceived Effectiveness Across Conditions

Positive Civil (PC) Positive Uncivil (PU) Negative Civil (NC) Negative Uncivil (NU) Control - Mixed Control – None

n

Strong PSA

Weak PSA

Two PSAs Averaged

108 85 98 113 98 90

3.25 (.74) 3.27 (.75) 3.12 (.75) 2.98 (.73) 3.13 (.83) 3.47 (.60)

2.56 (.78) 2.52 (.68) 2.39 (.78) 2.35 (.76) 2.52 (.87) 2.61 (.78)

2.90 (.63) 2.89 (.59) 2.73 (.63) 2.64 (.64) 2.82 (.73) 3.04 (.58)

Note: N = 567 for strong PSA; N = 574 for weak PSA; N = 550 for PSA averaged. Journal of Computer-Mediated Communication 19 (2014) 975–990 © 2014 International Communication Association

983

Figure 1 Perceived effectiveness for strong and weak PSAs across four comment conditions. Error bars showed the 95% CI. For strong PSAs the none condition differed significantly from each of the three other conditions. For weak PSAs only the negative comment condition differed significantly from the none condition.

Since the no-comment condition achieves the highest PE, further analysis was conducted to examine how PSA quality moderates the influence the three types of comments’ have on PE compared with the no comment condition. As shown in Figure 1, compared with the no-comment condition, strong PSAs suffered significantly in all three comment conditions while weak PSAs were affected only by negative comments. The number of comments people read was related to judgments of perceived effectiveness of the ads (See Table 1). For both positive and negative conditions those who read a few or some comments (M positive = 3.02, SD = .60; M negative = 2.84, SD = .59) perceived the PSA to be significantly more effective than those who read all the comments (M positive = 2.79, SD = .62; M negative = 2.57, SD = .66), t positive (173) = 2.42, p = .017; t negative (171) = 2.70, p = .003. One interpretation of this result is that the more comments read directly affected PE of the ads in a deleterious way regardless of the valence of the comments. However, it is also possible that viewers’ judgments of the ad’s effectiveness decreased the likelihood that they would read the comments that followed.

Comments as Distractors To unravel the counterintuitive results about comments and explore the causal direction a mediation analysis was conducted. Mediation analysis (see Figure 2 for the path model) using joint-significance tests showed negative comments decreased PE because they lowered subjects’ transportation level, made them believe the PSA was exaggerated, and elicited their negative emotions like fear, guilt, and anger. Positive comments decreased PE only because they lowered subjects’ transportation level, indicating they distracted people from the PSA and lowered their level of engagement. 984


n.s. p < .05

Figure 2 Mediation analysis for strong PSAs. Standardized path coefficients with S.E. in parentheses. Model fit: X2 = 10.00 (p = .02); RMSEA = .06, p = .25; CFI = .99; TLI = .93; SRMR = .009. Exogenous variables in the model are dummy variables with the no-comment condition as the base category. The same model was run for weak PSAs and the result pattern was the same except the positive to transportation path was only marginally significant.

To confirm the distraction effect, we coded participants’ responses to the free recall task, which asked subjects to write down everything they could remember from the two PSAs they just watched. Recall of the first and the second PSA were coded separately on whether it was blank (yes/no), inaccurate (yes/no), or irrelevant (yes/no) by two independent coders who were blind to the hypotheses and conditions (Cohen’s Kappa > .78 for all six categories). To compute accuracy score, blanks were treated as missing data, an accurate recall was given a value of +1, an inaccurate recall a −1, and an irrelevant comment a 0. An overall accuracy score was then calculated for each individual by adding their scores for both PSAs. The overall accuracy score ranges from −2 (recall both PSAs incorrectly) to +2 (recall both correctly). ANOVA showed positive, negative, mixed, and no comment conditions differ significantly on their mean accuracy score, F(3, 366) = 2.61, p = .05. Planned contrast showed that people in the no comment condition indeed remember the PSAs better than those in the comment conditions, t(366) = −2.30, p = .02. The no-comment (M = .67, SD = .87) condition scored higher on recall accuracy than the positive condition (M = .29, SD = .96), t(366) = −2.46, p = .01, and the negative condition (M = .30, SD = .96), t(366) = −2.46, p = .01, but it did not surpass the mixed-comment condition (M = .46, SD = .90), t(366) = −1.19, p = .23. These results suggest that both positive and negative comments can have deleterious effects on smokers’ evaluation of the antismoking PSA. While the negative comments created an antiad or prosmoking norm that lowered smokers’ PE of the ad, the positive comments harmed the PSA mainly because they were a distraction.

Incivility and Attitude, Perceived Risk, and Intention to Quit Neither the main effect of comment tone nor its interaction with comment valence had a significant effect on PE in the four-way mixed design ANCOVA. To explore the possible effects incivility has on the persuasiveness of the comments, smokers’ attitude, perceived risk, and intention to quit were compared across the four comment-treatment conditions (PC, PU, NC, NU). A one-way MANOVA was conducted with comment-treatment condition as fixed factor and attitude, perceived risk, and intention as dependent variables. Result revealed a significant multivariate main effect for comment Journal of Computer-Mediated Communication 19 (2014) 975–990 © 2014 International Communication Association

985

Figure 3 Smokers’ Attitude Towards Quitting, Perceived Risk, and Intention to Quit Across CommentTreatment Conditions. Error bar showed the 95% CI. Table 3 Summary of Simple Regression Analyses for Variables Predicting Attitude and Perceived Risk Attitude Model Condition (NC vs. NU) Age Familiarity Condition × Age Condition × Familiarity

Perceived Risk

B

SE B

β

B

SE B

β

−1.34 −.04 .23 .03 −.15

.77 .02 .36 .01 .21

−.82 −.74∗ .20 1.04∗ −.28

−.89 −.03 .47 .02 −.20

.77 .02 .37 .01 .21

−.53∗ −.40 .40 .82∗ −.40

Note. ∗ p < .05. condition, Wilks’ λ = .93, F(9, 915.236) = 3.04, p = . 001. Given the significance of the overall test, the univariate main effects were examined. Significant differences among treatment conditions were obtained for attitude, F(3, 378) = 6.04, p = .001, and perceived risk F(3, 378) = 4.62, p = .003, but not for intention, F(3, 378) = 1.14, p = .33. As shown in Figure 3, orthogonal contrasts reveal that the positive civil and the positive uncivil groups do not differ on attitudes, t(382) = 1.03, p = .30) or perceived risk, t(393) = −.24, p = .81, but those reading the negative uncivil comments score significantly lower than those reading the negative civil comments on attitude toward quitting, t(382) = 3.31, p = .001, and perceived risk of smoking, t(393) = 2.97, p = .003. Since all subjects in the study were smokers, positive comments could be considered as counterattitudinal and negative comments proattitudinal to the subjects. Therefore this set of findings supports Scherer’s (2007) argument that incivility improves the persuasiveness of proattitudinal messages. In other words, smokers reward incivility on their own side but ignore incivility on the opposing side. Further analysis showed that the strengthening effect of incivility on negative comments was not moderated by people’s familiarity with online comments, but it was moderated by age (See Table 3). Unlike the majority of the sample, senior smokers (age 60 and above, 19.4% of the sample) do not appreciate the anti-PSA / prosmoking comments that are uncivil.

Discussion This study found smokers’ perceived effectiveness of the antismoking PSAs was influenced by comments following the PSAs. PSAs with positive comments were perceived by smokers as more effective than 986


PSAs with negative comments, which replicated the general findings of Walther et al. (2010). However, smokers in the no-comment condition gave the highest PE score to PSAs. In addition, smokers’ prior readiness to quit smoking moderated the effect of comments. The most hardcore smokers considered the PSAs ineffective no matter what comments they saw. For smokers in the middle or at the top of the contemplation ladder, however, positive comments made PSA more effective compared with negative comments. The most surprising finding from the study is that positive comments failed to improve PSA evaluation over the no-comment exposure to ads. Quite opposite to the hypothesis, even positive comments significantly decreased smokers’ perceived effectiveness if the ad was strong. The key question then is: Why did the positive condition fail to generate a higher PE than the no comment condition? Three possible explanations are offered: 1 Comments distracted audience from the PSA, decreased their level of engagement and thus decreased PE. As demonstrated in the result section, mediation analysis and the test of recall indicated the presence of comments was a distraction. 2 The stimuli were not pure. As mentioned in the method section, comments in the positive condition consisted of eight positive comments and two negative comments for each PSA. It is thus possible PE was dragged down by the two negative comments mixed in the positive condition. The current study adopted an 8–2 ratio for comments for ecological validity. A pure positive condition could still elicit higher PE than the current stimuli did. What remains unclear, however, is whether it will be able to elicit significantly higher PE than the no comment control condition. Of course, the reality of online recommendations is that they are not pure conditions but are very likely to represent a range of points of view, sometimes refutational in a substantive sense but with each side refuting the other. In this sense, our findings are realistic and representative. 3 Positive comments were resources for smokers to counterargue. Positive comments are of the same valence as antismoking PSA, but the creator of the comments has less credibility and authority than creators of the PSAs. When subjects, who were all smokers, tried to counter argue, those in the positive condition may ignore the PSA and focus on the comments instead because they are an easier target to attack. Studies using rating scores or ranking information to create social influence succeeded in promoting positive group’s evaluation from no information control (Cohen & Golden, 1972; Salganik, Dodds, & Watts, 2006), which indicates comments may function differently from rating or ranking. As mentioned before, comments can describe both valence of the norm and the arguments and rationales supporting the norm. On the one hand, the arguments and rationales embedded in the comments may strengthen the influence of online recommendation system by making social norm more salient; on the other hand, they may weaken the influence by providing viewers with resources to counterargue or by overwhelming viewers with too much information. To test the third explanation, future studies can compare the effects of different kinds of online recommendation system, positive rating scores versus positive comments for example.

Implications The detrimental effect of comments on PE, attitude, and perceived risk found by the current study seems to suggest antismoking PSAs would be better off without comments, especially if the PSAs are strong or if the target audience is somewhat ready to quit smoking. Social media have received a great deal of attention from public health officials in part because of the personalness of these media and in part because of the power they give to audiences to contribute. This power is double-edged as the present study shows. Contributors can offer extensive comments and these in turn can undermine the impact of health messages when the comments are not moderated Journal of Computer-Mediated Communication 19 (2014) 975–990 © 2014 International Communication Association

987

in some way. Our data suggest that under some conditions the intended message is lost or reduced in effectiveness during the diffusion process. At the first glance findings of the current study seem to be inconsistent with predictions made by social influence theories, as positive comments failed to improve PSA evaluation, but a closer read of the theories like the Integrative Model and the Theory of Normative Social Behavior would reveal that in the smoking domain social normative pressure works best when it’s from smokers’ significant others or from fellow smokers. Social influence theories may not be quite applicable to virtual interactions involving anonymous communicators, but it doesn’t mean they are obsolete for the research on new media, as not all the online interactions are anonymous. These theories would still provide guidance in scenarios where smokers see an antismoking PSA on Facebook accompanied by a positive comment posted by their mom, or an ad tweeted to them by their smoking friend. Understanding the influence of online commentary for a variety of topics, under different conditions of trustworthiness and bias of the commentators, and with various degrees of control by the web sites involved will require concerted effort by researchers as we seek to understand this version of social influence as manifested in the emerging social media environment. One thing is very clear: It is no longer possible to consider the influence of news or other messages in the public information environment apart from the comments which follow them.

Acknowledgments This publication was made possible by Grant Number 5P20CA095856 and R01CA160226 from the National Cancer Institute (NCI). We acknowledge that the NCI does not bear any responsibility for the content reported in this article.

References Asch, S. (1956). Studies of independence and conformity: A minority of one against a unanimous majority. Psychological Monographs: General and Applied, 70(9), 1–70. Bostrom, R. N., Baseheart, J. R., & Rossiter, C. M., Jr. (1973). The effects of three types of profane language in persuasive messages. Journal of Communication, 23(4), 461–475. Biener, L., & Abrams, D. B. (1991). The contemplation ladder: Validation of a measure of readiness to consider smoking cessation. Health Psychology, 10, 360–365. Cialdini, R. & Golstein, N. (2004). Social influence: Compliance and conformity. Annual Review of Psychology, 55, 591–621. Cialdini, R. B., Reno, R. R., & Kallgren, C. A. (1990). A focus theory of normative conduct: Recycling the concept of norms to reduce littering in public places. Journal of Personality and Social Psychology, 58(6), 1015. Cohen, J. & Golden, E. (1972). Informational social influence and product evaluation. Journal of Applied Psychology, 56(1), 54–59. Durkin, S., & Wakefield, M. (2006). Maximizing the impact of emotive antitobacco advertising: Effects of interpersonal discussion and program placement. Social Marketing Quarterly, 12, 3–14. Eagly, A. H., & Chaiken, S. (1975). An attribution analysis of the effect of communicator characteristics on opinion change: The case of communicator attractiveness. Journal of Personality and Social Psychology, 32(1), 136–144. Finn, J., & Banach, M. (2000). Victimization online: The downside of seeking human services for women on the internet. CyberPsychology & Behavior, 3(5), 785–796. 988


Fishbein, M. (2008). A reasoned action approach to health promotion. Medical Decision Making, 28, 834–844. Giner-Sorolila, R., & Chaiken, S. (1997). Selective use of heunrstic and systematic processing under defense motivation. Personality and Social Psychology Bulletin, 23(1), 84–97. Green, M. C., & Brock, T. C. (2000). The role of transportation in the persuasiveness of public narratives. Journal of Personality and Social Psychology, 79(5), 701–721. Heatherton, T. F., Kozlowski, L. T., Frecker, R. C., & Fagerstrom, K. O. (1991). The Fagerstrom test for nicotine dependence: A revision of the Fagerstrom tolerance questionnaire. British Journal of Addiction, 86, 1119–1127. Jackson, S. (1992). Message effects research. New York: Guilford. Kang, Y., Cappella, J. N., Strasser, A. A., & Lerman, C. (2009). The effect of smoking cues in antismoking advertisements on smoking urge and psychophysiological reactions. Nicotine & Tobacco Research, 11(3), 254–261. Kang, M. (1998). The influence of public opinion polls on public opinion: The role of motivation and ability in the elaboration likelihood model. Dissertation Abstracts International, 59 (2A), 359. (UMI No. 9825262) Kushin, M. J., & Kitchener, K. (2009). Getting political on social network sites: Exploring online political discourse on Facebook. First Monday, 14(11). Moor, P. J., Heuvelman, A., & Verleur, R. (2010). Flaming on YouTube. Computers in Human Behavior, 26(6), 1536–1546. Mulac, A. (1976). Effects of obscene language upon three dimensions of listener attitude. Communication Monographs, 43(4), 300–307. Mutz, D. C. (2007). Effects of ‘in-your-face’ television discourse on perceptions of a legitimate opposition. American Political Science Review, 101(4), 621–635. Mutz, D. C., & Byron, R. (2005). The new videomalaise: Effects of televised incivility on political trust. American Political Science Review, 99(1), 1–16. Ng, E. W. J., & Detenber, B. H. (2005). The impact of synchronicity and civility in online political discussions on perceptions and intentions to participate. Journal of Computer-Mediated Communication, 10(3). ´ J. M. (2000). The influence of familiarity among group Orenga, V., Zornoza, A. M., Prieto, F., & Peiro, members, group atmosphere and assertiveness on uninhibited behavior through three different communication media. Computers in Human Behavior, 16, 141–159. Rimal, R. N., & Real, K. (2005). How behaviors are influenced by perceived norms: A test of the theory of normative social behavior. Communication Research, 32(3), 389–414. Salganik, M. J., Dodds, P. S., & Watts, D. J. (2006). Experimental study of inequality and unpredictability in an artificial cultural market. Science, 311(5762), 854–856. Samu, S., & Bhatnagar, N. (2008). The efficacy of anti-smoking advertisements: The role of source, message, and individual characteristics. International Journal of Nonprofit and Voluntary Sector Marketing. Special Issue: Social Marketing, 13(3), 237–250. Scherer, C. R. (2007). Indecent influence: What mediates the relationship between swearing and persuasion. Dissertation Abstracts International, 67 (9B), 5471. (UMI No. 3235678) Scherer, C. R., & Sagarin, B. J. (2006). Indecent influence: The positive effects of obscenity on persuasion. Social Influence, 1(2), 138–146. Sherif, M. (1935). A study of some social factors in perception. Archives of Psychology, 27(187). Sherif, M. (1936). The psychology of social norms. Harper & Brothers. Sonck, N. & Loosveld, G. (2010) Impact of poll results on personal opinions and perceptions of collective opinion. International Journal of Public Opinion Research, 22, 230–255 Journal of Computer-Mediated Communication 19 (2014) 975–990 © 2014 International Communication Association

989

Strasser, A. A., Cappella, J. N., Jepson, C., Fishbein, M., Tang, K. Z., Han, E., et al. (2009). Experimental evaluation of antitobacco PSAs: Effects of message content and format on physiological and behavioral outcomes. Nicotine & Tobacco Research, 11(3), 293–302. Sutfin, E. L., Szykman, L. R., & Moore, M. C. (2008). Adolescents’ responses to anti-tobacco advertising: Exploring the role of adolescent smoking status and advertisement theme. Journal of Health Communication, 13(5), 480–500. Walther, J. B., DeAndrea, D., Kim, J., & Anthony, J. C. (2010). The influence of online comments on perceptions of anti-marijuana public service announcements on YouTube. Human Communication Research, 36(4), 469–492. Zhao X., Strasser, A., Cappella, J. N., Lerman, C., & Fishbein, M. (2011). A measure of perceived argument strength: Reliability and validity. Communication Methods & Measures, 5, 48–75.

About the Authors Rui Shi ([email protected]) is a Doctoral Student at the Annenberg School for Communication at the University of Pennsylvania. Her research focuses on health campaigns and their interplay with new media. Paul Messaris ([email protected]) is Lev Kuleshov Professor of Communication at the University of Pennsylvania. His research centers on visual communication and digital media. Joseph N. Cappella ([email protected]) is Gerald R. Miller Professor of Communication at the University of Pennsylvania. His research interests include cognitive processing of verbal and visual materials, organization of social interaction, and message effects.

990
