Immersive 3D, Live-Action, Social Skills Training for Children with ASD Laura Boccanfuso Technology and Innovation Lab Yale Child Study Center Yale University
Lauren DiNicola
Abstract
Gabriella Greco
Deficits in social functioning are a core impairment for children with autism spectrum disorder (ASD). Social stories, role play and video modeling have been shown to promote some proficiency in social scenarios [12], but these techniques are limited, and generalizing learned skills remains difficult [11]. Lack of realism, interactivity, and immersion may contribute to the diminished effectiveness of these approaches.
Marilena Mademtzi Yale Child Study Center
[email protected]
Yale University
Megan Lyons
[email protected] [email protected]
Elizabeth Schoen Simmons Yale Child Study Center Yale University
[email protected] [email protected] Frederick Shic Technology and Innovation Lab Yale Child Study Center Yale University
[email protected]
[email protected]
Katarzyna Chawarska Yale Child Study Center Yale University
[email protected] Brian Scassellati Department of Computer Science Yale University
[email protected]
Combining three-dimensional (3D), live-action video within an immersive, virtual environment delivers a potentially powerful platform for improving socials skills in school-aged children with ASD. In this approach, the child receiving intervention will not only be able to observe social exchanges between two children but also to virtually engage with an age-matched peer during a customized interaction. This study leverages features from two clinically-validated approaches to further improve social skills intervention and more effectively generalize social functioning in children with ASD.
Author Keywords Virtual reality; Interaction design; Autism spectrum disorder; Video Modeling; Social skills training
ACM Classification Keywords D.2.2 Design Tools and Techniques: User interfaces;
Sample Social Skills Vignettes Menu A. Conversation 1. Initiating 2. Sharing ideas 3. Listening 4. Maintaining dialogue B. Social Etiquette 1. Turn-taking 2. Politeness 3. Self-control C.
Nonverbal Communication 1. Monitoring facial expressions 2. Reading body language 3. Identifying emotions 4. Using gestures
D. Interaction Strategies 1. Giving, receiving compliments 2. Joining group activities 3. Dealing with teasing, rejection
Figure 1
Introduction Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by deficits in social communication and repetitive, restricted patterns of behaviors, interests or activities [15]. Currently, there is growing interest in the use of innovative technologies to facilitate learning for these individuals, especially concerning social communication and emotion development. While previous studies have addressed these therapeutic targets with varying success, generalized benefits outside of the technology environment remain a challenge. Video modeling (VM) is an educational technique through which students learn by viewing and imitating behaviors of an agent demonstrating social skills via video. This technique has been widely used for autism spectrum disorder (ASD) and has been shown to be an effective, evidence-based instructional strategy for individuals with ASD [6]. However, one of the major obstacles in using traditional VM strategies for practicing social exchanges is the inability to interact directly with the VM agent and receive responses contingently based on performance. Traditional virtual reality (VR) techniques provide opportunities for direct interaction, but generating virtual content is expensive. VR environments featuring first-person, live-action video, where contingent feedback can be provided to students, may support more effective learning in individuals with ASD [13]. In this study, a menu of response choices enables a facilitator to select responses appropriate for each distinct social interaction. In this way, we present a customized and realistic social exchange for each individual and for each potential social response.
This research leverages expertise from specialists in speech and language pathology, psychology and computer science to build on existing methodologies and present a new interaction design for advancing social skills training (SST) for children with ASD.
Related Work Video modeling. Recent evidence supports VM, which provides visual examples of social scenarios, as a viable tool for teaching social skills to children with ASD [9]. VM may be even more effective than real-life peer modeling concerning the development of certain skills [10]. Nonetheless, even when combining VM with other SST techniques, generalized social improvements can be difficult to attain [11]. Virtual Reality. Virtual reality (VR) is a threedimensional (3D), immersive platform wherein individuals can virtually interact with an environment. The significant potential of VR for achieving a number of therapeutic objectives is well established [3,4,5]. By video recording scenarios similar to real-world social encounters, individuals can rehearse social situations that appear to be threatening within the safety of a virtual environment. Collectively, recent studies assert that VR simulations may elicit the same emotions as real-world events, and experiences within a virtual world may lead to modified social behavior [1]. Merging Video Modeling and Virtual Reality. The current project aims to enhance recent intervention protocols for children with ASD by combining VM of social stories with a naturalistic, interactive VR procedure, allowing children to test social strategies with unfamiliar others in a realistic but safe virtual
Sample Social Vignette 1. A participant observes a social interaction that goes poorly. 2. A virtual coach appears and guides the participant through what went wrong. 3. Each of the actors in the vignette separately describes his/her feelings and thoughts during the social encounter. 4. The actors re-enact the social scenario using a better approach. 5. The virtual coach reappears and asks the participant to try. 6. One of the actors turns, faces the participant and provides him/her the opportunity to apply the social skill lesson. 7. The participant responds the facilitator selects the appropriate video segment response. 8. At the end of the session, similar social scenarios are revisited in one-onone role-play with the facilitator. Figure 2
environment and to receive feedback in real time without risk of social penalty.
classes of social scenarios (Fig. 1) and an SST vignette modeled after [14] (Fig. 2) are included for reference.
Technology
The progression of each virtual interaction will be contingent upon the child’s social choices, with different choices eliciting different pre-recorded responses. A trained experimenter will control the video responses following pre-determined guidelines of behavior. Participants will be allowed to attempt multiple approaches to each scenario in order to engage in selfdirected social exploration and learning.
This study will employ three, relatively low-cost and commercially available technologies. Time-synchronized video streams will be collected using two GoPro cameras and a side-by-side dual-camera housing for recording 3D video. Recorded video will be displayed with wide-angle video playback provided by the Oculus Rift, an immersive, VR head-mounted display. Software designed to control the playback of specific video segments will feature an interface with a menu of appropriate contingent-based social feedback for each social vignette. Trained personnel will select each social-response video segment. The use of live-action VR is particularly beneficial for delivering stimuli recorded from a wide range of realworld social contexts and for use within a variety of classroom, clinical and home environments. Video and accompanying audio can be used to simulate settings with various levels of distraction and even feature locales with which the participant is already familiar. Moreover, because the technology is highly portable, integrating this protocol into an existing therapeutic program would not require significant modifications.
Study Design We will initially recruit 5 children, aged 6-10 years, to participate in pilot tests scheduled to begin in summer 2015. Preliminary testing will take place at the Yale Child Study Center over a period of 3 months. A larger population of 25 age-matched children is expected to be recruited to participate in longer-term testing over a period of 9 months. All children will receive a 30 total hours of the described intervention. Initial pilot testing will assess the feasibility of conducting future studies for a longer study period and with a greater number of participants. Future work will explore increasing system autonomy by tracking user position and conducting speech recognition to autonomously adapt video responses.
Video content Social Skills Vignettes. Each video will first display children modeling a targeted social skill through a scripted vignette. Participants will then be able to interact directly with unfamiliar, age-matched peers in a situation that mirrors the modeled scenario. Vignette content will reflect social skills targeted by previous SST protocols [8]. A sample SST menu illustrating four
Conclusion Despite research that has explored the utility of VR for achieving therapeutic objectives, the majority of previous paradigms feature expensive, computergenerated simulations rather than live-action video. Additionally, previous SST protocols for children with ASD involve hypothetical stories and/or familiar peers
[8]. Our study will evaluate the potential of employing 3D live-action, first-person virtual reality and real-time adaptive social feedback for advancing social skill proficiencies in young children with ASD. This procedure will improve upon previous social skills interventions by allowing children with ASD to interactively develop social strategies, requiring generalization of acquired skills concurrently with the intervention itself.
References 1.
2.
Riva, G. ed. Virtual reality in neuro-psychophysiology: Cognitive, clinical and methodological issues in assessment and rehabilitation. Vol. 44. IOS press, 1997. Mitchell, P., Parsons, S. & Leonard, A. (2007). Using virtual environments for teaching social understanding to 6 adolescents with autistic spectrum disorders. Journal of Autism and Developmental Disorders, 37(3): 589-600.
3.
Hoffman, H. G. (2004). Virtual-reality therapy. Scientific American, 291: 58-65.
4.
North, M. M., North, S. M. & Coble, J. R. Virtual reality therapy. (1996). IPI Press.
5.
Parsons, T. D. & Rizzo, A. A. (2008). Affective outcomes of virtual reality exposure therapy for anxiety and specific phobias: A meta-analysis. Journal of Behavior Therapy and Experimental Psychiatry, 39(3): 250-261.
6.
National Professional Development Center on Autism Spectrum Disorders (2013). Evidence-based practices for children and youth with ASD Retrieved from http:// autismpdc.fpg.unc.edu/content/briefs
7.
Dautenhahn, K. (2000, September). Design issues on interactive environments for children with autism. Proceedings International Conference on Disability, Virtual Reality and Associated Technologies (ICDVRAT). Alghero, Sardinia, Italy.
8.
Rao, P. A., Beidel, D. C. & Murray, M. J. (2008). Social skills interventions for children with asperger’s syndrome or high-functioning autism: A review and recommendations. Journal of Autism and Developmental Disorders, 38: 353-361.
9.
Delano, M. (2007). Video modeling interventions for individuals with autism. Remedial and Special Education, 28: 33-42.
10. Charlon-Christy, M. H., Le, L., & Freeman, K. A. (2000). A comparison of video modeling with in vivo modeling for teaching children with autism. Journal of Autism and Developmental Disorders, 30(6): 537-552. 11. Sansoti, F. J. & Powell-Smith, K. A. (2008). Using computer-presented social stories and video models to increase the social communication skills of children with high-functioning autism spectrum disorders. Journal of Positive Behavior Interventions, 10: 162-178. 12. Thiemann, K.S. and Goldstein, H. (2001). Social stories, written text cues, and video feedback: Effects on social communication of children with autism. Journal of Applied Behavior Analysis, 34(4): 425–446. 13. Bellini, S. and Akullian, J. (2007). A meta-analysis of video modeling and video self-modeling interventions for children and adolescents with autism spectrum disorders. Exceptional Children, 73(3): 264–287. 14. TD Social Skills Channel. “Social Skills Training: Taking Turns Speaking.” https://www.youtube.com/user/TDSocialSkills/feat ured. Web. Mar 2015 Accessed. 15. American Psychiatric Association (APA) (2013) Diagnostic and Statistical Manual of Mental Disorders (DSM-IV. 5th ed.). Washington, DC: APA.
