www.faculty.jacobsuniversity.de/dkuester/ICPS2015/ SWIC_ICPS2015.pdf
Methods: Design
Abstract We show how the perceived pleasantness of basic sine waves as well as slightly more complex sounds may be modulated by contextual factors, such as whether sounds are presented at home vs. the laboratory. Implications are discussed in respect to conducting emotion perception studies on the Internet.
Supporting Summary In recent years, the study of emotions has been moving away from the perception of static to dynamic stimuli (e.g., Krumhuber, Kappas, & Manstead, 2013; Korb et al., 2014). At the same time, possibilities of conducting psychological research online have greatly increased (Küster & Kappas, 2013; Reips, 2007), eliciting interest in context effects for traditional survey materials (Smyth, Dillman, & Christian, 2007). However, as opportunities for online research expand to include multimodal content, questions that have been raised for surveys may need to be rephrased. Thus, while crowdsourcing platforms such as Amazon’s Mechanical Turk (MTurk) have gained popularity as sources of data with acceptable reliability (e.g., Mason & Suri, 2012; Crump, McDonnell, & Gureckis, 2013; Goodman, Cryder, & Cheema, 2013), differences between online research and the traditional psychological laboratory might be more pronounced for more complex multimodal designs. E.g., in the present research, we aimed to investigate if basic sine waves are perceived differently in a potentially less structured but more relaxed context at home. We conducted 2 studies with a total of 136 participants who were exposed to 20 simple sine waves that were differing in duration, base frequency, frequency transition range, and pitch (ascending vs. descending). In the first study, 27 participants evaluated valence and arousal of these sounds in the laboratory, while 25 unique workers on MTurk participated at home. Our results suggest that, while there were no differences in perceived arousal, the sounds tended to be perceived as slightly more pleasant when evaluated at home, in particular for intrinsically more pleasant upward-sloped sounds (cf. Komatsu & Yamada, 2011). In the second study (84 subjects, 26 in the laboratory; 90 sounds, 42 saw tooth-waves, Read & Belpaeme, 2012), we improved the experimental design to focus on the potential effects of having a headphone available vs. not. This study replicated findings of the first experiment, suggesting a more pleasant experience at home compared to the laboratory. No significant differences emerged for the use of headphones. Additionally, participants responded with significantly higher arousal ratings in the laboratory, supporting our assumption of an overall more relaxed home-environment that may have a subtle influence on emotional sound perception. Overall, both studies are consistent with the proposition that even the interpretation of short synthetic sounds may be dependent on contextual factors. However, our data are still consistent with the notion that online studies may be at least as suitable as the laboratory for a subjective evaluation of simple synthetic sounds in terms of valence and arousal.
Study 1
Study 2
Replication + extension
Participants N = 52; 27 in the Laboratory (Mage = 20.00, SD = 1.46; 18 female), 25 from Home (Mage = 34.95, SD = 8.23; 14 female).
Methods: Materials & Procedure
Participants N = 84 (Mage = 29.02, SD = 10.25; 49 female; 5 unanswered*) • Headphones at Home (N = 30; Mage = 34.80, SD = 10.03; 14 female) • Speakers at Home (N = 28; Mage = 32.65, SD = 9.10; 19 female) • Headphones in the Lab (N = 26; Mage = 19.84, SD = 2.39; 19 female)
Stimuli 30 sine waves. 15 were designed to be perceived as moderately positive, while 15 were designed to be moderately negative. The combination of “duration” and “frequency transition-range” were derived from Komatsu and Yamada (2011).
* All participants who did not indicate their gender took part in the online version of the study.
Stimuli 90 sounds: 42 saw-tooth waves, 48 sine waves from Read and Belpaeme (2012).
Measures Subjective evaluations of valence and arousal using a modified online version of the Affect Grid (Russell, Weiss, & Mendelsohn, 1989).
Measures Subjective evaluations of valence and arousal using a modified online version of the Affect Grid (Russell, Weiss, & Mendelsohn, 1989). Figure 1. Web interface for the Affect Grid (Russell, Weiss, & Mendelsohn, 1988)
Results
p < .0001
p < .0001 p < .001
References Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013). Evaluating Amazon’s Mechanical Turk as a tool for experimental behavioral research. PLoS ONE, 8, e57410. doi:10.1371/journal.pone.0057410 Goodman, J. K., Cryder, C. E., & Cheema, A. (2013). Data collection in a flat world: The strengths and weaknesses of Mechanical Turk samples: data collection in a flat world. Journal of Behavioral Decision Making, 26, 213–224. doi:10.1002/bdm.1753 Komatsu, T., & Yamada, S. (2011). How does the agents’ appearance affect users’ interpretation of the agents’ attitudes: experimental investigation on expressing the same artificial sounds from agents with different appearances. International Journal of Human-Computer Interaction, 27, 260–279. doi:10.1080/10447318.2011.537209 Korb, S., With, S., Niedenthal, P., Kaiser, S., & Grandjean, D. (2014). The perception and mimicry of facial movements predict judgments of smile authenticity. PLoS ONE, 9, e99194. doi:10.1371/journal.pone.0099194 Krumhuber, E., Kappas, A. & Manstead, A. S. R. (2013). Effects of dynamic aspects of facial expressions: A review. Emotion Review, 5, 41-45. doi:10.1177/1754073912451349 Küster, D., & Kappas, A., (2013). Measuring emotions in individuals and Internet communities. In T. Benski, & E. Fisher (Eds.), Internet and emotions (pp.48-61). Routledge. Mason, W., & Suri, S. (2012). Conducting behavioral research on Amazon’s Mechanical Turk. Behavior Research Methods, 44, 1–23. doi:10.3758/s13428-011-0124-6 R Core Team (2014). R: A Language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL http://www.R-project.org/ Read, R., & Belpaeme, T. (2012). How to use non-linguistic utterances to convey emotion in child-robot interaction. In Proceedings of the 7th Annual ACM/IEEE International Conference on Human-Robot Interaction, 219-220. doi: 10.1145/2157689.2157764 Read, R., & Belpaeme, T. (2014). Situational context directs how people affectively interpret robotic non-linguistic utterances (pp. 41–48). In Proceedings of the 2014 ACM/IEEE International Conference on Human-robot Interaction, 41-48. doi:10.1145/2559636.2559680 Reips, U.-D. (2007). The methodology of Internet-based experiments. In A. Joinson, K. McKenna, T. Postmes, & U.-D. Reips (Eds.), The Oxford Handbook of Internet Psychology (pp. 373-390). Oxford University Press. Russell, J. A., Weiss, A., & Mendelsohn, G. A. (1989). Affect Grid: A single-item scale of pleasure and arousal. Journal of Personality and Social Psychology, 57, 493502. doi:10.1037/0022-3514.57.3.493 Wilcox, R. R. (2012). Introduction to robust estimation and hypothesis testing (3rd ed.). Boston: Academic Press. Wilcox, R.R., & Schönbrodt, F.D. (2015). The WRS package for robust statistics in R (version 0.27.5). Retrieved from https://github.com/nicebread/WRS. Zillmann, D. (1971). Excitation transfer in communication-mediated aggressive behavior. Journal of Experimental Social Psychology, 7, 419-434.
Acknowledgments This research was funded by the EU FP7 EMOTE (ICT-2011-8 317923).
Contact Pasquale Dente
[email protected] Jacobs University Bremen Campus Ring 1 28759 Bremen, Germany
Figure 2. Bar plot of valence ratings with confidence intervals. Ratings were not normally distributed (ShapiroWilk test); thus, the Wilcoxon rank-sum test has been used (R Core Team, 2014).
Figure 4. Distribution of arousal ratings. Arousal ratings in Study 2 were not normally distributed (Shapiro-Wilk test). Differences were therefore tested by a boostraping Robust ANOVA (Wilcox, 2012) using the WRS package (Wilcox & Schönbrodt, 2015) in R. Contrasts show (p < .0001, number of bootstrap samples = 2000) that both “Internet” conditions differed significantly from the “Laboratory” condition. No differences were found for comparisons on headphones vs. speakers at home.
Figure 5. Distribution of valence ratings. Valence ratings in Study 2 were not normally distributed (Shapiro-Wilk test). Differences were therefore tested by a boostraping Robust ANOVA (Wilcox, 2012). Contrasts show that both “Internet” conditions differed significantly (p < .0001, number of bootstrap samples = 2000) from the “Laboratory” condition. No differences were found for comparisons on headphones vs. speakers at home.
Figure 3. Bar plot of arousal ratings with confidence intervals. Ratings were not normally distributed (ShapiroWilk test); thus, the Wilcoxon rank-sum test has been used (R Core Team, 2014).
Results: Summary Findings from both studies point towards a more positive overall evaluation of simple sounds at home vs. in the laboratory. Furthermore our results suggest that the use of headphones vs. speakers results in no systematic differences – despite their evident differences at the technical level. This suggests that this aspect of the mode of presentation might not be relevant for most of the types of designs involving the presentation of emotional sounds in social psychological research. Importantly, participants in the laboratory reported significantly higher arousal in comparison to our sample of online workers, supporting our assumption of an overall more relaxed home-environment. This finding is relevant because it provides evidence for some likely limitations in respect to using different types of emotion-eliciting materials across online –and offline contexts. The social context of participating in a laboratory study may therefore influence perception of even very minimal materials - such as the simple sine waves used in the present research.
Conclusions & Future Work While most of the observed valence effects were relatively subtle, differences concerning arousal appeared to be more pronounced and reminiscent, for example, of some of the classical works on excitation transfer theory (e.g., Zillmann, 1971). Here, the question arises, to what extent social psychology in general might still underestimate the excitement that participating in a psychological experiment may have for young undergraduate students. Today, this may be a more stressful situation for most people than it is to participate in a (paid) online experiment. However, the potential impact of differences in mean age and experience with psychological surveys between online workers and our student sample may suggest an alternative account that would still need to be addressed in further online vs. offline comparisons. In addition, we propose further tests of synthetic affective stimuli using our online version of the Affect Grid to assess to what extent our findings can be generalized to other materials. Overall, both studies suggest that even the interpretation of short synthetic sounds may be dependent on contextual factors. Further, our data generally support the notion that, with the proper tools, online studies may be at least as suitable as the laboratory for a subjective evaluation of simple synthetic sounds in terms of valence and arousal.
