C 2010 Wiley Periodicals, Inc.
Psychology in the Schools, Vol. 48(1), 2011 View this article online at wileyonlinelibrary.com
DOI: 10.1002/pits.20541
IMPROVING READING COMPREHENSION OF AT-RISK HIGH-SCHOOL STUDENTS: THE ART OF READING PROGRAM R. STEVE MCCALLUM, KATHERINE R. KROHN, CHRISTOPHER H. SKINNER, ANGELA HILTON-PRILLHART, MICHAEL HOPKINS, STEVEN WALLER, AND FRITZ POLITE
University of Tennessee Participants (115 low-socioeconomic-status [SES], inner-city, high-school students) were exposed to three reading conditions: (1) a control condition in which students silently read brief selected passages; (2) an experimental condition in which students were prompted to perform a three-part (Ask, Read, and Tell [ART]) comprehension enhancement exercise before, during, and after reading the selected passages; and (3) another experimental condition in which ART was followed by peer discussion (PD) of the reading. Students answered 10 comprehension questions after reading passages under each condition, which served as the dependent variable. Results of a repeatedmeasures analysis of variance revealed a significant main effect, and follow-up analysis showed significantly higher levels of comprehension on ART + PD passages relative to either the control or ART passages. Discussion focuses on using PD to enhance comprehension and future research to determine which combination of intervention components are necessary to occasion enhanced C 2010 Wiley Periodicals, Inc. comprehension.
Proficient reading is thought to develop via a hierarchical process of skill development that includes phonemic awareness, phonics, fluency, vocabulary, and comprehension (Adams, 1990; Denton & West, 2002; National Institute of Child Health and Human Development, 2000; Shanahan, 2005). The final stage, comprehension, is critical as in almost all instances the purpose or function of reading is comprehension (Skinner, Neddenriep, Bradley-Klug, & Ziemann, 2002). Although various prereading skills are required for comprehension development, they are not necessarily sufficient. Research by the RAND Reading Study Group (RAND, 2002) indicates that many third-grade students who are reading at grade level will not automatically develop reading comprehension strategies. Thus, in many cases students require explicit instruction designed to enhance their comprehension skills (Bishop, Reyes, & Pflaum, 2006). Unfortunately, comprehension strategies are not taught as frequently as some students require for growth (Pressley, 2002; RAND, 2002). When reading for comprehension, effective readers actively apply strategies designed to monitor and enhance their comprehension (Collins & Stevens, 1982; Golinkoff, 1976; Hodge, 1993; Jackson, 1989; Lundenberg, 1987; McNeil, 1987; Mier, 1984; Palincsar & Brown, 1984). Some of the research-based comprehension enhancement strategies include teaching or prompting readers to (a) make connections to text based on their background knowledge, (b) make predictions about text, (c) visualize text content, (d) ask questions when confused or uncertain about content, (e) use strategies to summarize text, and (f) problem solve and clarify (Anderson, 1992; Brown, 2008; Daly, Chafouleas, & Skinner, 2005). Explicit comprehension instruction often focuses on teaching these strategies (Almasi, 2003; Brown, 2008; Clark & Graves, 2005; Pressley & Afflerbach, 1995) with two primary goals: (1) creating immediate success in understanding reading material and (2) providing students with a range of comprehension strategies that they can use with a variety of texts (Flynt & Cooter, 2005; RAND, 2002). Ask, Read, Tell Many empirically supported programs and curricula designed to teach comprehension enhancement strategies are composed of a variety of empirically supported components (Anderson,
Correspondence to: Christopher H. Skinner, The University of Tennessee-Knoxville, 1122 Volunteer Boulevard, 518 Bailey Education Addition, Knoxville, TN 37996-3452. E-mail:
[email protected]
78
The ART of Reading Program
79
1992; Brown, 2008). The current study was initiated by educators who charged the researchers with developing a simple comprehension enhancement intervention that could be used to enhance comprehension in at-risk high-school students who attended a brief (2-week) summer enrichment activity. Applying previous research, experimenters developed Ask, Read, Tell (ART), a simple procedure that prompts students to conduct pre-, during-, and postreading activities designed to enhance comprehension. During the first step, Ask, students were prompted to read the title and ask themselves questions regarding the material (e.g., What does this material cover? How familiar is this material? What do I already know about the topic?). These prereading procedures were designed to activate appropriate schema and increase students’ motivation to read for comprehension (Daly et al., 2005; Idol-Maestas, 1985). During the second step, Read, students were taught to stop at the end of each paragraph and determine if the passage continued to make sense to them. They also underlined unknown words and tried to figure out their meanings by using context clues. The goal of this stage was to enhance students’ comprehension monitoring while reading. In the third stage, Tell, students were taught to tell themselves what they read (Flynt & Cooter, 2005). This process was made concrete by prompting students to answer the questions that they had posed to themselves in the prereading (Ask) step. In addition to evaluating ART, we investigated if supplementing pre- and during-reading activities with postreading peer discussion (PD) would enhance comprehension. There is evidence to support the utility of postreading peer and/or teacher-led discussion (Applegate, 2007; Berne & Clark, 2008; Blanton, Pilonieta, & Wood, 2007; Boyd, 1997; Brock, 1997; D’Alessio, 1996; Goatley, 1997; Zemelman, Daniels, & Hyde, 1993). For instance, Donahue, Voekl, Campbell, and Mazzeo (1999) concluded that students earned higher comprehension scores after participating in peer-led discussion groups at least once or twice per week. Additionally, Fox and Wilkinson (1997) found that participation in peer-led discussion groups not only led to student engagement, but also increased their enjoyment of reading. Because many of the behaviors that ART was designed to occasion were unobservable, we could not provide feedback or consequences based on students’ adherence to the ART procedures. We added a PD component to motivate the students to apply the strategies outlined in ART. Thus, one goal of the current study was to determine if student comprehension was enhanced when they were prompted to apply ART. A second goal was to determine if adding a PD component to ART enhanced comprehension. M ETHOD Participants and Materials Students from two high schools involved in a summer program at a large southeastern university participated in the study. The program, Project GRAD, is designed to assist at-risk students by providing services that will equip and encourage them to graduate from high school and succeed in postsecondary education. In total, 115 students participated in the reading comprehension intervention. The majority of participants were in 10th (65%) and 11th grades (27%), although freshman (1%) and senior (7%) high-school students were also represented. Participants ranged in age from 15 to 18 years, with more female (62%) than male (38%) participants. Black (72%) and White (23%) students far outnumbered students who self-identified as Other (3%), Asian (1%), or Hispanic (1%). Approximately 10% of the participants reported that they were receiving additional educational services beyond general education instruction. Students read 400-word, fourth-grade-level reading passages from the Timed Readings (3rd ed.) series (Spargo, 1989) over the 6 days of intervention. One or two passages were selected for each day. The fourth-grade-level passage was select by Project GRAD staff who wanted to minimize the probability of some student becoming frustrated over reading material that was too difficult. Each Psychology in the Schools
DOI: 10.1002/pits
80
McCallum et al.
passage included 10 multiple-choice questions (5 inference and 5 fact questions) that were used to assess student comprehension after the intervention. Materials constructed by the researchers included the ART of Reading Student Worksheet form. This form contained the steps of the reading comprehension intervention. Students following the ART Worksheet engaged in the three major steps—Ask, Read, Tell—and related activities. Each student received a copy of the Student Worksheet to help them adhere to the steps (see Appendix A). The original directions were used to instruct students in the first experimental condition of the study. A second version of the ART Worksheet was created to accommodate a second treatment condition, and results from both conditions were compared to a control (non-ART) condition. The first experimental condition included the ART of Reading steps as described earlier in text, whereas the second condition required the additional step of a 2-minute discussion over the reading with one peer (with time held constant across all conditions). We refer to the first worksheet as the original ART form and the second as the ART + PD form. Design and Procedure This study was conducted over 2 weeks at a large southeastern university. Participants congregated in a large university auditorium during 8 consecutive weekdays for 1 hour each day during the implementation phase of this study. Three conditions were counterbalanced and included the independent condition (regular ART procedures), the ART + PD condition (ART procedures plus a 2-minute discussion with a peer), and the control condition (non-ART). Two passages were used most days, with the exception of only one passage on Days 4 and 8. Participants were initially instructed to take 10 minutes to read the passage and were given 3 minutes to answer the questions. After one session using the 10-minute reading period, the time allocated for reading passages was shortened to 6 minutes to conserve time and to maintain interest. After the first session, it was obvious that most participants finished well before the end of the 10-minute period. For consistency, each condition was administered one time with the 10-minute reading time before switching to 6 minutes. In the control condition, participants were instructed simply to read the passage for the designated time and were immediately given 10 multiple-choice questions to answer in a 3-minute time period. Participants were not prohibited from rereading, but they were not instructed to do so. Specifically, they were told, “Take your time and read the passage carefully when instructed to begin. You will have 6 minutes. When you finish reading, please remain seated and quietly study what you have read. At the end of the 6 minutes, you will pass the reading passages to your left and we will collect them. Then you will be given a handout with 10 questions based on the reading passage, and will have 3 minutes to complete the questions.” Instructions were read to participants before the first implementation of ART to orient them to the ART procedures and the form, and to explain the intent of the procedures, as follows: “Our goal is to help you develop critical thinking skills that help you comprehend better what you read. We use a 3-step process called ART. A stands for ask, R stands for read with alertness, and T for tell. First, you will learn to ask yourself some questions before you read, then learn to focus on the reading content, and finally to tell yourself or someone else what you just read. Look at the ART worksheet in front of you.” The proctor then led the students through each step on the ART form one time. One proctor read the intervention instructions each day, and three or more additional proctors assisted with the distribution and collection of materials. In the ART condition, the Student Worksheet form was distributed to participants at the same time as the reading passage. Participants were still given 6 minutes to read, but, in this condition, Psychology in the Schools
DOI: 10.1002/pits
The ART of Reading Program
81
they also completed the form to guide their reading. In the ART + PD condition, participants were given 4 minutes to read the story and complete the ART form before they were instructed to discuss the reading for 2 minutes with one peer. Time was held constant for the three conditions. The 10 questions were distributed to participants in all conditions, with a standard time of 3 minutes to complete them. In total, each condition was administered three times over the intervention days. Procedural Integrity and Inter-Scorer Agreement One proctor also completed a procedural integrity checklist during each administration session (see Appendix B). Procedural integrity was 100%. Additionally, although one researcher was primarily responsible for scoring comprehension responses, a second researcher scored 20% of the worksheets across students. For each worksheet, inter-scorer agreement was calculated by dividing the number of agreements by the number of agreements plus disagreements and then multiplying by 100. Average inter-scorer agreement was 99.9% (range = 90% –100%). R ESULTS Prior to analysis, participants’ reading comprehension scores were aggregated by summing scores from the three comprehension measures within each condition to form a composite reading comprehension (CRC) score for the ART condition (mean [M] = 24.83, standard deviation [SD] = 3.88), ART + PD condition (M = 25.80, SD = 3.00), and control condition (M = 25.02, SD = 3.87). Descriptive statistics, paired t tests, and effect sizes are presented in Table 1. Participants’ reading comprehension scores were analyzed using a within-subject, repeatedmeasures analysis of variance (ANOVA) to determine if significant differences existed among the three conditions (ART, ART + PD, and Control). Results revealed a significant main effect, F (2, 218) = 7.637, p < .01, η2 = .065. Paired sample t tests were conducted, and the Bonferroni correction was applied to control experimenter-wise error rate (i.e., to maintain an alpha of 0.05, the critical value was set to .017). Post-hoc analysis show the ART + PD intervention produced significantly higher scores than did both the ART intervention, t(109) = 3.671, d = .27, and the control condition readings, t(109) = 3.07, d = .25. Differences between the control and ART conditions were not statistically significant, t(109) = 0.808, d = .06.
Table 1 Means, Standard Deviations, Paired t Tests, and Effect Size Experimental Group
M
SD
Control
25.02
3.87
ART
24.83
3.88
ART
24.83
3.88
ART + PD
25.80
3.00
ART + PD
25.80
3.00
Control
25.02
3.87
t
p
d
.81
>.05
.06
−3.67
