Beyond Pictures And Words: Alternative Information ...

12 downloads 0 Views 2MB Size Report
Thanks are due to Marcel Just, John R. Hayes, it. This article avoids a ..... a stern-faced John Sirica sentencing a group cept FISH. Note that CAT and FISH haveĀ ...
Psychological Bulletin 1978, Vol. 85, No. 3, 532-554

Beyond Pictures and Words: Alternative Information-Processing Models for Imagery Effects in Verbal Memory David Kieras

vt

r PsycholoIcal Bulletin 1978, Vo. 85, No. 3, 532-554

Beyond Pictures and Words: Alternative Information-Processing Models for Imagery Effects in Verbal Memory David Kieras University of Arizona This article presents an information-processing approach to imagery effects in verbal memory tasks. A general model of the process of forming images from verbal input is developed, based on propositional memory representations like those used in computer simulations of sentence comprehension, visual scene analysis, and image processing. Data in support of the general model are reviewed. The general model is then refined in several classes of alternative models that attempt to account for imagery effects, with emphasis on sentence memory results, by using different mechanisms in the general model. The major division in the alternative models is whether the facilitation produced by imagery is due to the actual storage of image information or is just a by-product of image formation or the use of high-imagery materials. Some of the models are rejected on the basis of published data. Two of the remaining models would require substantial progress'in the study of sentence memory and comprehension in a way not directly related to imagery. The models most likely to be successful are those that assume that the use of imagery results in the storage of redundant information that provides alternate retrieval routes. Additional data are needed to decide whether the redundant information is contributed directly by the image representation itself or supplied by additional purely semantic information made available during image formation. Imagery is one of the most powerful factors influencing memory for verbal materials. The facilitation typically produced by the use of imagery strategies or high-imagery materials is so pronounced that a satisfactory explanation of imagery effects will be essential to an adequate theory of verbal memory. A major obstacle facing a satisfactory explanation is that the nature of imagery itself is unclear and currently under active debate. Three major positions have developed: the mental picture position, the dual-code position, and the propositionalrepresentationposition. The goal

of this article is to provide a set of possible explanations for imagery effects, based on the propositional representation position. A preliminary step is to clarify these three positions. The mental picture position (cf. Anderson & Bower, 1973, p. 230; Pylyshyn, 1973) holds that word and sentence meanings are themselves images, described as subjectively experienced "mental pictures." Paivio (1976) pointed out that this position is basically a straw man, without serious adherents. However, some approaches to the role of imagery in comprehension appear to be very

close to this position (e.g., Bugelski, 1970). The problem presented by the mental picture

Work on this paper was supported by an R. K Mellon Postdoctoral Fellowship and National Institutes of Health Grant NIH 16-MH 07722 while the

position is that although few people take it

author was at Carnegie-Mellon University. Thanks are due to Marcel Just, John R. Hayes, Peter Polson, Terry Daniel, and Allan Paivio for helpful comments. Requests for reprints should be sent to David Kieras, Department of Psychology, University of Arizona, Tucson, Arizona 85721. Copyright 1978 by the American Psychological

seriously, much effort has been spent refuting it. This article avoids a similar waste and does not consider this position any further. The dual-code position developed by Paivio (1971a, 1971b) accounts for imagery effects in terms of the differential availability of Association, Inc. 0033-2909/78/8S03.0532$00.75

532

16

IMAGERY EFFECTS IN VERBAL MEMORY

visual and verbal codes, which are taken to be implicit associations or rearousals of past sensory events. This position has the merit of accounting for a great many imagery-related phenomena in verbal learning. However, there are a number of processes involved in experiments on imagery that exist at a lower level of analysis than the gross imagery effects accounted for by the dual-code model. These are the processes that are required for performing the basic tasks on which imagery manipulations are performed, such as visual perception, sentence and word comprehension, and the manipulation of images in problem solving. For example, in a typical experiment on the effects of imagery on sentence memory, the subject must visually perceive the words in the presented sentence, access their meanings, perform a syntactic analysis of the sentence, and construct a representation of the content of the sentence. Such processes must be performed any time that a sentence is to be comprehended, regardless of whether imagery is manipulated or not. The dual-code position has little to say about these complex processes, since the only mechanism available is the simple arousal of associated visual and verbal codes. Current empirical evidence and computer simulations of complex mental proceses indicate that perception and comprehension processes must operate in terms of deep units, such as concepts, relations, propuse dere erties, features, and meanings, rather than surface units, such as word representations or copies of sensory input. Hence the ideas that sentence meaning can be coded as arousals of word representations and that visual scenes are remembered in the form of rearousals of sensory input are no longer plausible. Hence the dual-code theory can offer explanations only at the level of gross imagery effects; it s nt c Is not capable of providing a complete account of performance in verbal and perceptual tasks, since it lacks the machinery to operate at the deeper levels. A more vague form of the dual-code theory appears in the literature as the position that images and perceptual information have fundamentally different representational and processing properties from those of verbal information. For example, images are described as "holistic," "holographic," or "ana-

533

logical," whereas verbal representations are assigned the opposite properties. This approach has little explanatory value, since these properties themselves must be explained; otherwise, the superior memory performance with imagery can be simplyand uninterestingly-attributed to the mere presence or indivisible superiority of images. This vague form of the dual-code theory suffers even more than Paivio's rigorous theory from a lack of comprehensive explanatory power. The propositional representation position on the nature of imagery has been proposed by Anderson and Bower (1973, pp. 451461),1 Pylyshyn (1973), and Simon (1972) and is related to proposals made by Schank (1972), Palmer (1975), and others. This position is based on the argument that all knowledge, regardless of its source modality, can be expressed in a single, uniform, abstract type of representation, the proposition. Unlike the dual-code position, there is thus no 1 This paper owes much to the insightful discussion of imagery by Anderson and Bower (1973, pp. 230-233; 449-461). However, their position on imagery was not definitely stated and so requires some comment to avoid confusion. Anderson and Bower postulated propositional representations that include perceptual information in the same format as semantic information. However, perceptual information was given no special role in explaining imagery effects. They rejected the modality-specific interference results as being artifactual and made a strong suggestion that confounded word attributes account for the bulk of imagery effects. In their in paired-associate discussion of imagery effects learning, they argued that both imagery and verbal set of mediresult-a same the produce mnemonics ating relations. The subject can "unpack" verbal concepts into elaborative details, eventually adding primitive perceptual concepts to the memory reprethese the sheer quantity ofcontent, sentation, but it is not their perceptual elaborative details, that is important. That mediating relations might not be as important in sentence memory was not considered. Thus, Anderson and Bower appear to have argued against perceptual information playing a key role in imagery effects; their position is perhaps best approximated overall by the semantic elaboration model in this article. However, a more accurate summary of their wide-ranging discussion is that Anderson and Bower did not commit themselves to

a single, definite explanation for imagery effects but had as their main goal the refutation of the mental picture position.

534

DAVID KIERAS

fundamental difference in how perceptually based and verbally based information is represented in memory. This approach and its terminology needs some explanation because much of the debate on the nature of imagery has resulted from misinterpretation and misapplication of the concept of a proposition. Since propositions have been heavily used in linguistic theory and in psychological theories of sentence comprehension, the mistaken idea has arisen that propositions are themselves verbal entities and are capable of expressing only verbal information. Rather, the concept of proposition used in these theories is descended from formal logic, in particular, predicate logic. Predicate logic is a formal system in which propositions are represented with notation such as P(x) to symbolize that x has property P, and R(x, y) to indicate that x stands in relation R to y. Formal logic has no commitment to the source of propositions; a formula of the form P(x) could equally well express the propositions that x is red, x is normally distributed, x is horizontal, or x is a cognitive theory. Some propositional theories of memory use notation very similar to this logical notation, for example, (CONSTRUCT, BABYLONIANS, GARDEN) in Kintsch (1974). A useful method of expressing propositions is as a network. A network consists of nodes interconnected by links: The nodes represent concepts, and the connecting links represent the relations between concepts. A proposition consists of a pair of nodes connected by a link. For example, the proposition (NEAR, CAT, FISH) would appear as a node for the concept CAT connected by a NEAR link to a node for the concept FISH, as shown in Figure E

2 Ceral.

WT