Misunderstanding âReading Comprehensionâ: Implications for Instruction and Testing. Dick Schutz. 3RsPlus, Inc. The belief that text has intrinsic meaning, and ...
Misunderstanding “Reading Comprehension”: Implications for Instruction and Testing Dick Schutz 3RsPlus, Inc. The belief that text has intrinsic meaning, and that reading involves extracting this meaning from
the text can be traced back to E. L. Thorndike’s influential article of 1917, Reading as Reasoning. www.archive.org/stream/readingasreasoni00thor/readingasreasoni00thor_djvu.txt It seems to be a common opinion that reading (understanding the meaning of printed words) is a rather simple compounding of habits. Each word or phrase is supposed, if known to the reader, to call up its sound and meaning and the series of word or phrase meanings is supposed to be, or be easily transmuted into, the total thought. It is perhaps more exact to say that little attention has been paid to the dynamics whereby a series of words whose meanings are known singly produces knowledge of the meaning of a sentence or paragraph. It will be the aim of this article to show that reading is a very elaborate procedure, involving a weighing of each of many elements in a sentence, their organization in the proper relations one to another, the selection of certain of their connotations and the rejection of others, and the cooperation of many forces to determine final response. In fact we shall find that the act of answering simple questions about a simple paragraph. . . includes all the features characteristic of typical reasonings. (p. 323)[Emphasis added In correct reading (1) each word produces a correct meaning, (2) each such element of meaning is given a correct weight in comparison with the others, and (3) the resulting ideas are examined and validated to make sure that they satisfy the mental set or adjustment or purpose for whose sake the reading was done. (p. 327) In educational theory, then, we should not consider the reading of a text-book or reference as a mechanical, passive, undiscriminating task, on a totally different level from the task of evaluating or using what is read. While the work of judging and applying doubtless demands a more elaborate and inventive organization and control of mental connections, the demands of mere reading are also for the active selection which is typical of thought. It is not a small or unworthy task to learn "what the book says." (p. 332) [Emphasis added}
Given this belief, Thorndike viewed oral reading as counterproductive: In school practice it appears likely that exercises in silent Reading to find the answers to given questions, or to give a summary of the matter read, or to list the questions which it answers, should in large measure replace oral reading. The vice of the poor reader is to say the words to himself without actively making judgments concerning what they reveal. [Emphasis added] Reading aloud or listening to one reading aloud may leave this vice unaltered or even encouraged.
Electronic copy available at: http://ssrn.com/abstract=1531314
Perhaps it is in their outside reading of stories and in their study of geography, history, and the like, that many school children really learn to read. (p. 332) [Emphasis added]
Since reading=reasoning=higher order thinking, reading and intelligence tests measure one and the same: It thus appears that reading an explanatory or argumentative paragraph in his text-books on geography or history or civics, and though to a less degree) reading a narrative or description, involves the same sort of organization and analytic action of ideas as occur in thinking of supposedly higher sorts. [Emphasis added] This view is supported by the high correlations between such reading and verbal completion tests, Binet-Simon tests, analogies tests and the like. These correlations, when corrected for attenuation, are probably, for children of the same age, as high as +.80. (p. 331)
Fast forward to today. Reading tests are still based on Thorndike’s conception. From the NEAP 2009 Reading Framework: www.nagb.org/publications/frameworks/reading09.pdf The recommended 2009 NAEP Reading Assessment is guided by a definition of reading that reflects scientific research, draws on multiple sources, and conceptualizes reading as a dynamic cognitive process. This definition applies to the assessment of reading achievement on NAEP and states that reading is an active and complex process that involves:
• • •
Understanding written text. Developing and interpreting meaning. Using meaning as appropriate to type of text, purpose, and situation.
Terms used in the definition can be further explained as follows: Understanding written text: Readers attend to ideas and content in a text by locating and recalling information and by making inferences needed for literal comprehension of the text. In doing so, readers draw on their fundamental skills for decoding printed words and accessing their vocabulary knowledge. Developing and interpreting meaning: Readers integrate the sense they have made of the text with their knowledge of other texts and with their outside experience. They use increasingly more complex inferencing skills to comprehend information implied by a text. As appropriate, readers revise their sense of the text as they encounter additional information or ideas. Using meaning: Readers draw on the ideas and information they have acquired from text to meet a particular purpose or situational need. The use of text may be as straightforward as knowing the time when a train will leave a particular station or may involve more complex behaviors such as analyzing how an author developed a character’s motivation or evaluating the quality of evidence presented in an argument. (PDF p. 18-19) NAEP explicitly and deliberately incorporates differential student backgrounds into the test: 2
Electronic copy available at: http://ssrn.com/abstract=1531314
Factors related to the text being read and to readers’ backgrounds and experiences influence reading performance. For example, understanding the vocabulary, concepts, and structural elements of the text contributes to readers’ successful comprehension. Comprehension is also affected by readers’ background knowledge and by the context of the reading experience. The background knowledge that students bring to the NAEP Reading Assessment differs widely. To accommodate these differences, passages will span diverse areas and topics and will be as engaging as possible to the full range of students in the grades assessed. (PDF p. 19) In short, the current NAEP view is the 1917 Thorndike view. Ignorant of the history of the English language and the structure of the Alphabetic Code, the NAEP Framework speaks of “phonics knowledge” and “the alphabetic principle.” These terms are convoluted and confounded with the “five essentials of NCLB (since NAEP has to be “in compliance” with NCLB). The rationale for the focus on Comprehension?
By grade 4, when the NAEP Reading Assessment is first administered, students should have [Emphasis added] a well-developed understanding of how sounds are represented alphabetically and should have had sufficient practice in reading to achieve fluency with different kinds of texts (National Research Council 1998). Because NAEP tests at grades 4, 8, and 12, the assessment focuses on students’ reading comprehension, not their foundational skills related to alphabetic knowledge. That is, in NAEP’s view “alphabetic knowledge” is foundational to reading, but it isn’t reading.
Fatal Flaws in the NAEP View Here’s the big problem: The NAEP Reading Tests and the test results bear no relationship to the reading instruction that is going on in schools. Teaching kids how to read begins to wind down toward the end of grade 2 and is over by the end of grade 3. NAEP testing doesn’t begin until Grade 4. There are other problems: •
•
•
If one is changing the definition of “reading” from the initial definition in 1992 and again in 2007, how can the test be measuring the same “ability” over the years? If one takes the NAEP sample items and removes them from any one or all of the Framework matrices, it’s impossible to classify them per the Framework. If one tries to map the items individually or collectively to texts used in reading instruction in schools, the effort will be in vain. 3
•
If one scrambles items from NAEP, the “Armed Services Vocational Aptitude Battery” the current military test that has a continuous history extending back to the Yerkes intelligence Test Alpha used in World War I, and the SAT (formerly the Scholastic Aptitude Test), the item forms are indistinguishable. Differences in “difficulty” are largely manipulated with the foils of the multiple choice items. As with “reasoning” and “intelligence,” “reading comprehension” can’t be taught directly –any more than “spoken language comprehension” can be taught directly (Willingham, 2009; Hirsch, 2006; Burkard, 2007). “Meaning” in both spoken and written communication is a function of background information. Background information can certainly be taught, but there is no basis for confounding “reading” with background information that a student lacks. D. B. Elkonin explains succinctly: Understanding, which is often considered as the basic consideration in the process of reading, arises as a result of the correct recreation of the sound form of words. He who, independently of the level of understanding of words, can correctly recreate their sound forms is able to read. (1963) Unfortunately, the 2009 NAEP Framework introduces the construct of “Meaning Vocabulary.” These items are carefully contrived to tap what Thorndike refers to as the “over and under‐potency of the words or phrases in the paragraph”. (p. 322). Thorndike uses the example: In Franklin, attendance upon school is required of every child between the ages of seven and fourteen on every day when school is in session unless the child is so ill as to be unable to go to school, or some person in his house is ill with a contagious disease, or the roads are impassable. (p.323)
That’s very complicated syntax and semantics. Franklin is a place, not the person, Ben Franklin. And it’s easy to get tangled up in the conditional clauses. The NAEP Framework echoes Thorndike: . . .the association between vocabulary knowledge and comprehension is strong; students who know the meanings of many words and who also can use the context of what they read to figure out the meanings of unfamiliar words are better comprehenders than those who lack these attributes (National Reading Panel 2000a). In the 2009 NAEP Reading Assessment, vocabulary will be assessed systematically through carefully developed items that measure students’ ability to derive the meanings of words within the context of the passages they read. (PDF. p 20) But these are the same item forms used to measure “general ability” and “scholastic aptitude.” 4
Contaminated by both background information and specific vocabulary, the NAEP Reading Test taps something other than reading expertise. Most psychologists would call the “something other” general intelligence, a combination of fluid intelligence and crystallized intelligence. From the Thorndike/NAEP perspective, “reading” is an ability that is a life‐long endeavor. From the Elkonin perspective, reading expertise can be taught to all children, with few exceptions by the end of Grade 2 or by the end of Grade 3 at the latest. The children still have a lot to learn, but they have been taught to read and can now read to learn. NAEP is regarded as the best of standardized achievement tests—the Nation’s Report Card. What does this say about lesser reading tests that conform to the same template? The answer doesn’t require much reasoning ability. The background information provided here should suffice. If you have correctly recreated the sound forms of the foregoing words, you’ve comprehended the communication.
References Burkard, T. Inside the Secret Garden: The Progressive Decay of Liberal Education. UK: U. of Buckingham Press, 2007. Elkonin, D. B. The Psychology of Mastering the Elements of Reading. Chap. In: Brian & Joan Simon (Eds.) Educational Psychology in the U.S.S.R. Stanford: Stanford U. Press, 1963. (pp. 165179). Hirsch, E. D. The Knowledge Deficit. Boston: Houghton Mifflin Harcourt, 2006 Thorndike, E. L. Reading as Reasoning: A Study of Mistakes in Paragraph Reading, Journal of Educational Psychology June 1917, 323-332. www.archive.org/stream/readingasreasoni00thor/readingasreasoni00thor_djvu.txt . Willingham, D. Reading Is Not a Skill--And Why This Is a Problem for the Draft National Standards. http://voices.washingtonpost.com/answer-sheet/daniel-willingham/willingham-reading-is-not-ask.html 5
