Conceptual and Methodological Issues in Social Problem-Solving ...

8 downloads 56 Views 2MB Size Report
who require a reliable and valid measure of social problem-solving abilitms. However, ... The term social problem solving refers to problem solving as it occurs in.
BEHAVIOR THERAPY 26, 409-432, 1995

Conceptual and Methodological Issues in Social Problem-Solving Assessment THOMAS J . D ' Z u R I L L A

State Umverstty o f New York at Stony Brook ALBERT MAYDEU-OLIVARES

Umverstty o f lllmols at Urbana-Champatgn Several promising instruments are currently available to researchers and climclans who require a reliable and valid measure of social problem-solving abilitms. However, all of these measures have shortcomings and none has definitive, unequivocal support for ~ts construct validity at the present time. The conceptual and methodological ~ssues that are most directly related to the validity of social problem-solving measures were &scussed. The strengths and weaknesses of the major current instruments were examined with respect to these issues. Empirical evidence related to convergent and &scriminant validity was also reviewed. RecommendaUons were made for the improvement of these measures as well as the future development of new and better measures of social problem-solving processes and outcomes.

The term social problem solving refers to problem solving as it occurs in the real world. Within the past decade, research interest in this concept has expanded rapidly across several different areas of psychology (D'Zurilla, 1986; Nezu & D'Zurilla, 1989; Poon, Rubin, & Wilson, 1989; Sinnott, 1989; Sternberg & Wagner, 1986; Tisdelle & St. Lawrence, 1986). In the clinical and counseling area, studies have focused on the relations between problem-solving ability and psychological and behavioral adjustment, as well as the efficacy of problem-solving training as a clinical intervention method (see D'Zurilla, 1986, 1990; Heppner, 1990; Heppner & Hillerbrand, 1991; Nezu, Nezu, D'Zurilla, & Rothenberg, in press; Nezu, Nezu, & Perri, 1989). In general, the results of these studies have supported the view that problem solving is an !mportant factor in adjustment, and that problem-solving training is a promising method for improving a person's adaptive functioning and, consequently, reducing and preventing psychological and behavioral disorders.

Requests for reprints should be sent to Thomas J. D'Zurilla, Department of Psychology, State University of New York at Stony Brook, Stony Brook, N.Y. 11794-2500, fax number: 518-6327876. The authors would like to thank two anonymous revmwers for their helpful comments and suggesuons on an earlier draft of this manuscript. 409 0005-7894/95/0409-043251 00/0 Copyraght 1995 by Assooatlon for Advancement of Behavior Therapy All rights of reproduction m any form reserved

410

D'ZURILLA & MAYDEU-OLIVARES

However, many of these studies have been viewed with skepticism because o f the use of inadequate or questionable problem-solving measures (Butler & Meichenbaum, 1981; D'Zurilla, 1986; Tisdelle & St. Lawrence, 1986; Yoman & Edelstein, 1993). The most serious concern has been the lack of convincing evidence for the construct validity of the measures. Hence, current researchers have no assurance that their instruments are actually measuring problemsolving ability. The major purpose of this article is to discuss the conceptual and methodological issues in the design and development of social problem-solving measures that bear most directly on their construct validity. In the second half of the article, we will review the strengths and weaknesses of the major existing problem-solving measures for adults, including relevant data on convergent and discriminant validity. We hope that this discussion will help researchers and clinicians select the most appropriate and valid measures for their purposes, as well as provide useful standards and guidelines for the development of new and better measures.

The Concept Of Social Problem Solving Although different conceptualizations of social problem solving have appeared in the literature (e.g., Heppner & Krauskopf, 1987; Spivack, Platt, & Shure, 1976), we have consistently espoused a concept that is based in part on views expressed by Skinner (1953) and Davis (1966). In this approach, social problem solving is defined as the self-directed cognitive behavioral process by which a person attempts to identify or discover effective or adaptive ways of coping with problematic situations encountered in everyday living (D'Zurilla, 1986; D'Zurilla & Goldfried, 1971; D'Zurilla & Nezu, 1982). As this definition implies, problem solving is conceived here as a conscious, rational, effortful, and purposeful activity. Given this definition, it follows that a problem (or problematic situation) is a life situation in which no effective or adaptive coping response is immediately apparent or available to the individual, thus requiring problem-solving behavior (D'Zurilla & Goldfried, 1971). In addition, a solution is the product or outcome of the problem-solving process when it is applied to a specific problematic situation-i.e., a situation-specific coping response or response pattern. Within this problem-solving framework, an important distinction is made between problem solving and solution implementation (D'Zurilla & Goldfried, 1971; Skinner, 1953). Whereas problem solving is the process of finding solutions to specific problems, solution implementation refers to the process of carrying out those solutions in the actual problematic situations. These two sets of skills should not be confused. Depending on the particular problem domain, the solution implementation skills that might be required for adaptive functioning would cover the full range of possible coping performance skills. Some of these coping skills might be correlated with problem-solving skills, but others might be completely independent of these skills. The impor-

PROBLEM-SOLVING ASSESSMENT

411

tant implications of this distinction for social problem-solving assessment will be discussed later. It would be helpful at this point to clarify further the relationship between problem solving and the concept of coping. According to Lazarus and Folkman (1984), coping refers to the cognitive and behavioral activities by which a person attempts to manage specific stressful situations, as well as the emotions that they generate. Given this general definition of coping, problem solving is clearly a coping process, but not all coping is problem-solving. Most attempts to distinguish problem solving from other forms of coping have been based on a functional analysis of coping. Thus, it has been common in coping research to identify problem solving as a problem-focused coping strategy, which is distinguished from emotion-focused coping (Folkman & Lazarus, 1980). The aim of problem-focused coping is to change the stress situation for the better, while the function of emotion-focused coping is to manage the emotions generated by the situation. Recently, however, D'Zurilla and Nezu (D'Zurilla, 1986, 1990; Nezu & D'Zurilla, 1989) have argued that problem solving cannot be distinguished from other coping activities on the basis of function because problem solving can serve a variety of different coping functions, depending on the specific goals that are set (e.g., change the situation, reduce anxiety), and the particular solutions that are generated (e.g., instrumental actions, relaxation activities). In recent years, investigators working independently in the areas of information processing (Logan, 1988, 1989) and social cognition and personality (Epstein, 1990, 1994; Epstein & Meier, 1989; Epstein, Lipson, Holstein, & Huh, 1992) have pointed to a potentially important coping distinction that is based on aprocess analysis of coping rather than a functional analysis. These investigators have identified two qualitatively different information-processing systems that play a role in coping: (a) an automatic or experiential system and (b) a nonautomatic or rational system, which includes rational problem solving. The automatic/experiential system operates primarily at the preconscious level, is rapid (oriented toward immediate action), effortless, unintentional, emotional, and is intuitively validated ("It feels right"). In contrast, the nonautomatic/rational system is conscious, slower (oriented toward delayed action), deliberate, purposeful, effortful, analytic, logical, and objectively validated (i.e., requires validation through logic and evidence). According to Logan (1988), these two systems are based on different underlying mechanisms; the automatic/experiential system is based on obligatory (i.e., unintended) memory retrieval, whereas the nonautomatic/rational system involves the deliberate, intentional use of algorithms (e.g., problem-solving techniques). Epstein (Epstein & Meier, 1989; Epstein et al., 1992) maintains that most everyday coping behavior, adaptive as well as dysfunctional, is directed by the experiential system. According to D'Zurilla and Goldfried (1971), rational problem solving is most likely to occur in critical problematic situations, where much is at stake and automatic memory retrieval has failed to produce an adequate coping response. Rational problem solving does not replace the experiential system at this point but, rather, the problem solver attempts to con-

412

D'ZURILLA & MAYDEU-OLIVARES

trol automatic processing (e.g., inhibiting automatic action, screening out poor quality responses) while conducting a rational search for a more effective or adaptive coping response (see also Logan, 1989). A number of coping inventories are available that assess a broad range of cognitive and behavioral coping activities (for example, Epstein & Meier, 1989; Folkman & Lazarus, 1980; Lazarus & Folkman, 1984; Stone & Neale, 1984; Tobin, Holroyd, Reynolds, & Wigal, 1989). Although most o f these inventories contain subscales that assess some problem-solving processes (e.g., information gathering, generation of alternative coping responses), none of them provide a specific, comprehensive assessment of problem-solving abilities. We will now turn to a discussion of the instruments that were specifically designed to assess these abilities.

Process Versus Outcome Measures It is useful in social problem-solving research to distinguish between two major types of problem-solving measures: (a) process measures and (b) outcome measures. Process measures assess the general cognitive and behavioral activities that facilitate or inhibit the discovery of effective or adaptive problemsolving solutions, whereas outcome measures assess the quality of specific solutions to specific problems. Process measures are most useful for assessing specific strengths and deficits in problem-solving attitudes and skills, whereas outcome measures are most useful for evaluating problem-solving performance, or the ability of individuals to apply their problem-solving skills effectively to specific problems. Although process and outcome measures are expected to be related, their correlations are not always expected to be high because variables other than problem-solving attitudes and skills are likely to influence the quality of problem-solving outcomes in specific situations (for example, emotional inhibitions, solution-implementation skill deficits, low self-efficacy expectancies). Process measures can also be divided into inventories and performance tests. As the term is used here, an inventory is a broad survey of problem-solving attitudes, strategies, and techniques, both positive (facilitative) and negative (inhibitive). Some inventories also estimate the extent to which the person actually uses the problem-solving skills that person possesses, as well as the manner in which these techniques are typically applied (e.g., efficiently, systematically, impulsively, carelessly, etc.). Most problem-solving inventories are pencil-and-paper questionnaires that employ Likert-type items, but other formats can be used for this purpose as well, such as structured interviews, audiotape procedures, and computer-assisted implementations. One disadvantage of inventories is that they do not actually test the person's skills and abilities. A performance test, on the other hand, directly assesses a person's ability to apply skills effectively to a particular problem-solving task. The performance test format presents a subject with a task that requires use of a specific ability or skill (e.g., problem recognition, problem definition and formulation, generation of alternative solutions, decision making). The person's performance is then judged or evaluated and this measure is viewed as an in-

PROBLEM-SOLVINGASSESSMENT

413

dicator of level of ability in that particular area (for examples, see D'Zurilla & Nezu, 1980; Nezu & D'Zurilla, 1979, 1981a, 1981b; Platt & Spivack, 1976; Spivack, Platt, & Shure, 1976). Because performance tests more closely approximate real life problem-solving behavior than inventories, these measures are generally expected to have greater external validity, although this is an issue that has not yet been adequately researched. All outcome measures are performance tests. However, instead of testing a specific ability or one particular component of the problem-solving process, these tests assess overall problem-solving performance by presenting the person with a problem and asking that person to solve it, after which the quality of the solution is evaluated. Most outcome measures employ hypothetical test problems, but subjects' solutions for their real current problems have also been assessed (see Camp, Doherty, Moody-Thomas, & Denney, 1989; D'Zurilla, 1990; Marx, Williams, & Claridge, 1992; Schotte & Clum, 1987). When real problems are used, the subjects may be asked to report their solutions verbally, either before or after the fact, or their actual solution performance may be observed, either in the natural environment or in some contrived, quasinaturalistic problem solving situation. Instead o f assessing process and outcome separately, some investigators have assessed these variables conjointly, vis h vis the same problem-solving task (Getter & Nowinski, 1981; Goddard & McFall, 1992; Plienis et al., 1987). In this approach, process skills and abilities are directly assessed during or following a particular problem-solving task, and then these process measures are later related to solution quality. A useful method for assessing covert processes during a problem-solving task is the think aloud method described by Meichenbaum, Henshaw, and Himmel (1982), or the Articulated Thoughts during Simulated Situations (ATSS) procedure developed by Davison, Robins, and Johnson (1983).

Issues in Test Design and Development With the above conceptual framework as background, we will now turn to a discussion of the conceptual and methodological issues in test construction that are most directly related to the construct validity o f social problemsolving measures. 1

Content Sampling and Validation The first condition that a problem-solving measure must satisfy in order to ensure its construct validity is that its items be a representative sample of the domain being measured. For process measures, the target domain would be some theoretical model of the social problem-solving process or some specific component o f that process (e.g., decision making). With regard to outcome measures, the relevant domain would be the range of everyday problems that i Parts of this discussion are based on Goldfried and D'Zurilla's (1969)guldehnes for the empirical developmentof situation-specifictests of behavioral competence.

414

D'ZURILLA & MAYDEU-OLIVARES

are likely to be personally relevant and significant for well-being for most of the individuals within the particular target population. Ideally, a test developer identifies a target population, defines the domain to be measured, specifies the goal of the measuring instrument, and then constructs items so as to adequately represent the content of the target domain. Sampling representativeness of the items can be ensured by the following process: (1) having the items generated by individuals who are very familiar with the particular theoretical construct or problem domain (e.g., scholars, members of the target population, people who interact regularly with members of the target population), (2) devising procedures to screen the items and choose the most representative ones, and (3) validating the content sampling procedure by logically examining the items and checking them against the relevant domain. In the development of outcome tests, it is important to sample from all major types of problems within the targeted domain. Because research has not yet determined if social problem-solving ability is the same or different across different kinds of problems, it would be hazardous at this time to draw conclusions about a person's general problem-solving ability on the basis of test items that were too narrowly sampled or too few in number. If it is not practical to sample broadly because the problem domain is too large, then the test developer should define the problem domain and goal of measurement more narrowly and focus only on some specific class of problems within the relevant domain (e.g., study problems, dating problems, etc.).

Distinguishing Rational Problem Solving From Experiential Processing In order to establish construct validity, procedures are also needed to ensure that the problem-solving measure will distinguish between rational problem solving and experiential coping. In the development of process measures, adequate content sampling from some theoretical model of rational problem solving should be sufficient to accomplish this goal. However, the task is more difficult with outcome tests because they focus only on the end products of the problem solving process-i.e., situation-specific coping responses. It is not possible to determine from the form or quality of these coping responses alone whether they are actually problem-solving solutions, mediated by a rational problem-solving process, or automatic coping responses directly retrieved from memory. Hence, the test developer must take specific steps during test construction to ensure that the test items will be problematic (D'Zurilla & Goldfried, 1971) and, thus, likely to generate rational problem-solving activities and solutions. One important procedure is to provide a clear definition of the term problem to individuals from the target population who are asked to identify representative problems. Otherwise, these individuals might report familiar and routine hassles or pressures that are likely to generate only experiential coping responses. In addition to emphasizing the lack of an obvious or ready coping response, it might also be helpful to point out that a person in a true problematic situation is likely to feel puzzled, perplexed, stumped, confused, or uncertain, which are important cues for problem solving.

PROBLEM-SOLVINGASSESSMENT

415

Another possible procedure is to administer the test problems to a sample of subjects from the target population using think aloud methods (e.g., Davison et al., 1983; Meichenbaum et al., 1982), self-monitoring procedures, interviews, or questionnaires designed to obtain a report of their problem-solving processes during or immediately following the problem-solving task. These reports could then be analyzed by the investigators to determine the extent to which rational problem-solving activities occurred. Test items would be retained only if they generated significant problem-solving thinking in all or some high percentage of subjects. A third possible approach is based on the assumption that perceived problemsolving difficulty, defined in terms of the expenditure of time and mental effort during the problem-solving task, would accurately reflect the amount of rational problem-solving activity. Subjects are asked to solve each test problem and then rate how difficult the problem-solving task was immediately after reporting each solution. Test items would be considered problematic and retained if the difficulty rating for all or some high percentage of subjects is above some arbitrarily chosen high number. In another approach, item difficulty is assessed on the basis of independent solution effectiveness ratings. Test items are considered too easy if the effectiveness rating for all or some high percentage of subjects is above some arbitrarily chosen high number, and too difficult if the rating for all or some high percentage of subjects is below some arbitrarily chosen low number. Although this procedure was originally designed to screen out items that are not likely to discriminate between different levels of problem-solving competence (Goldfried & D'Zurilla, 1969), the removal of extremely easy items is also expected to eliminate many ordinary and routine stressors that are likely to generate only experiential coping responses. The use of this procedure alone, however, is not sufficient because it does not guarantee that all of the remaining items will generate rational problem solving. A final possible method is based on the assumption that a ready, effective coping response is associated with a low likelihood of rational problem solving. In this procedure, subjects are asked to report coping responses to each test problem after being given experiential coping instructions (e.g., "Report the first response that comes to mind which 'feels' right"). After the effectiveness of each coping response is assessed by qualified judges, a test item would be considered nonproblematic and discarded if the effectiveness rating for all or some high percentage of subjects was above some arbitrarily chosen high number. Test Instructions and Format

The instructions that are used to present a problem-solving test to subjects can also influence the validity of test responses. Instructions for problemsolving inventories should provide a clear and specific definition of the target problem domain so that subjects will respond appropriately. With regard to outcome tests, the instructions should describe the instrument as a problemsolving test so that subjects will be set to find the most appropriate, practical, and effective solutions.

416

D~ZURILLA & MAYDEU-OLIVARES

The items in a problem-solving outcome test usually consist of vignettes in which a protagonist is confronted with a specific problem or issue to be resolved. In some tests the (e.g., the MEPS), the protagonist is some hypothetical other person (other person instructions), while in other tests (e.g., Freedman, Rosenthal, Donahoe, Schlundt, & McFall, 1978; Getter & Nowinski, 1981) the pronoun "you" is used to instruct the subjects to place themselves in the situation (self instructions). In addition to these instructions, some tests ask for the subjects' best solutions (ideal solutton instructions), while others ask them to report the solutions that they would (or did) actually carry out (actual solutton instructions). Several studies have found that the quality of test responses varies with different test instructions (Freedman et al., 1978; Marx et al., 1992; Penn, Spaulding, & Hope, 1993). However, it is not clear from the results of these studies what responses are most valid, for what purpose, and under what conditions. In general, however, it could be argued that the responses produced by self and actual solution instructions might have greater ecological validity because they increase the likelihood that the test problems will be perceived as personally relevant (Camp et al., 1989). The test format can also affect the validity of a problem-solving measure. Most tests use a pencil-and-paper format (D'Zurilla & Nezu, 1990; Getter & Nowinski, 1981; Heppner & Petersen, 1982; Platt & Spivack, 1975a). However, some investigators have used an audiotape procedure (Freedman et al., 1978; Goddard & McFall, 1992) and a structured interview (Heppner, Hibel, Neal, Weinstein, & Rabinowitz, 1982; Plienis et al., 1987). All three of these formats rely on verbal communication, thus a potential threat to validity is poor comprehension of test items by individuals with lower educational and intellectual levels. Because the popular pencil and paper format is particularly susceptible to this threat, it is recommended that test authors check the readability level of these measures, using such indices as the Gunning Fog Index (Gunning, 1952). Another relevant format distinction is a free-response format versus a forcedchoice format. In the former, no restriction is made on the type of response that the subject can report to a test item, whereas in the latter format, the subject is forced to choose between two or more given alternatives (e.g., truefalse, yes-no, or multiple choice). Freedman et al. (1978) found that a multiplechoice version of their problem-solving test produced more effective responses than a free-response version. Most importantly, the free-response version distinguished between delinquents and nondelinquents, whereas the multiplechoice version did not. One possible explanation for these results is that a free-response format taps a broader range of problem-solving skills and abilities than a multiple-choice format, which primarily measures the ability to recognize an effective solution.

Outcome Test Scoring Methods Two general types of scoring procedures have been used in problem-solving outcome assessment: quantitative scoring and qualitative scoring. In quan-

PROBLEM-SOLVING ASSESSMENT

417

titative scoring, some relevant solution variable is identified and a frequency count is taken. For example, in the scoring system for the MEPS (Platt & Spivack, 1975a; Spivack, Shure, & Platt, 1985), quantitative scores are computed for relevant means (discrete steps that enable the protagonist to move closer to the goal), obstacles (interferences with goal attainment), and time (references to the fact that problem solving takes time or that appropriate timing is an important component of an effective solution). Qualitative scoring, on the other hand, usually involves a rating of the subject's overall solution on some dimension of solution quality, such as effectiveness, appropriateness, passivity, or avoidance (Fischler & Kendall, 1988; Freedman et al., 1978; Getter & Nowinski, 1981; Marx et al., 1992). Several investigators have suggested that quantitative scores may not provide sufficient relevant information about a person's solutions to distinguish between different levels of problem-solving competence and adjustment. In one study, Fischler and Kendall (1988) found that qualitative scores (i.e., social appropriateness ratings), but not quantitative scores, were related to adjustment in school children. In another study, Marx et al. (1992) found that only qualitative scores (i.e., effectiveness ratings) distinguished between depressed psychiatric patients and nondepressed patients. However, until more studies are done on this issue, the best recommendation to researchers at the present time is to analyze their data using both types of scoring procedures. When solution effectiveness ratings are used, they are typically assessed for interrater reliability, but they are generally assumed to have external validity. However, adequate reliability does not guarantee validity. A major threat to the validity of these ratings occurs when they are performed by unqualified raters who are not sufficiently familiar with the criteria that are generally used to judge coping performance in the particular target environment. Qualified raters would include authority figures and experts who routinely evaluate coping behavior in a particular setting (e.g., supervisors, instructors, counselors), as well as members of the target population who have previously been judged as competent on the basis of such criteria as grades, peer ratings and nominations, letters of recommendation, and/or scores on a valid measure of social problem-solving ability. In a recent study on interpersonal problem solving in college students, Yoman and Edelstein (1993) evaluated the external validity of solution effectiveness ratings by relating them to actual solution outcomes in a role-play situation. The raters were undergraduates who were judged to be socially competent on the basis of a score < 25 on the Social Introversion scale of the MMPI. No significant relationship was found between solution effectiveness ratings and actual problem-solving goal accomplishment, but they were significantly related to social impact (i.e., incidental social consequences). Although the negative results for goal accomplishment in this study raise questions about the validity of solution effectiveness ratings in general, it is possible that the Social Introversion scale of the MMPI might not be an adequate criterion for selecting socially competent solution raters.

418

D'ZURILLA & MAYDEU-OLIVARES

Verbal Versus Observational Measures In this context, a verbal or self-report problem-solving measure is one that focuses on the subject's oral or written problem-solving responses in a test or interview situation, whereas an observational measure focuses on the subject's actual problem-solving behavior in the natural environment or in some quasi-naturalistic problem-solving situation (e.g., a role-play situation). Both approaches can be used to assess problem-solving processes as well as outcomes; however, each approach has its own advantages and limitations. Verbal problem-solving assessment has two major advantages: (a) it is the most practical, efficient, and cost-effective way to obtain a measure of problemsolving ability from a large number o f subjects across a wide range of problematic situations, and (b) it is the only way to obtain a direct, comprehensive assessment of covert problem-solving processes, which constitute the major part o f the overall problem-solving process. The limitations of verbal problem-solving assessment include the same threats to validity that are associated with any other self-report measure, including faking or deliberate distortion, expectancy effects, response sets, forgetting, recall biases, and comprehension problems. The external validity of verbal problem-solving measures may also be limited because such variables as emotionality, motivation, and self-efficacy expectancies are less likely to influence the person's problem-solving performance in a test situation than in a real life problem-solving situation. It should be noted that some of the above threats to validity apply only to inventories, and not to verbal performance tests (process as well as outcome measures), which require subjects to use their skills and abilities at the moment. Because the latter tests focus on a person's present ability to solve problems (or make decisions, generate alternative solutions, etc.) rather than on past problem-solving performance, forgetting or recall biases are not an issue with these tests. Due to the limitations of verbal assessment, some investigators have argued that social problem-solving assessment should focus more on observational methods (Butler & Meichenbaum, 1981; Krasnor & Rubin, 1981; Tisdelle & St. Lawrence, 1986). However, although an observational approach would avoid most of the limitations of verbal assessment, it has its own unique disadvantages. In addition to being costly, inefficient, and time-consuming, this approach also has some serious threats to validity. As previously noted, it is not possible to determine from the form or quality of situation-specific coping responses alone whether they are problem-solving solutions or experiential coping responses. Therefore, when an observer records and judges overt coping responses in real life or role-play stress situations, one could not conclude that problem-solving ability is being measured, unless it had been determined in advance that these situations were problematic for the particular subjects, or that they prompted rational problem-solving activities. Another potential threat to validity with an observational approach is that actual solutions outcomes might be confounded by the quality of the person's solution-implementation skills (e.g., assertiveness skills). In other words, a

PROBLEM-SOLVING ASSESSMENT

419

potentially good solution might have a poor outcome because the person has deficient solution-implementation skills, which would reduce the validity of the outcome measure. One possible way to control for solution-implementation skills would be to assess these skills independently and then adjust the subjects' problem-solving scores for skill deficits similar to a handicap in athletic competition. Because verbal and observational methods are not mutually exclusive, the most useful and valid assessment strategy might be one that integrates these two approaches. For example, verbal methods such as interviews and questionnaires might be employed first to identify critical problematic situations for a particular target population, and then observational methods could be used to assess the subjects' overt problem-solving performance in these situations. During or following these problem-solving events, other verbal methods (e.g., audiotape think-aloud recording devices, questionnaires) could be used to assess covert problem-solving processes, which would then be related to the observational measures. Another possible integrative strategy that might be more cost-effective than the above approach is self-observation or problem-solving self-monitoring (Barlow, Hayes, & Nelson, 1984; D'Zurilla, 1986). In this method, the subject identifies, observes, and records significant problematic situations, mediating cognitive and behavioral problem-solving activities, and solution-implementation activities as they occur in the real life setting. The major appeal of this approach is that it capitalizes on some of the important advantages of both verbal and observational assessment (e.g., efficiency, specificity, cost-effectiveness, immediate recording of behavior), while avoiding some of their major limitations (e.g., reliance on memory, abstraction, high cost, inconvenience).

Evaluation Of Major Current Problem-Solving Measures Because it is beyond the scope of this paper to review all of the problemsolving measures that have been used in social problem-solving research, we will focus on seven major instruments: (a) the Problem-Solving Inventory (PSI; Heppner & Petersen, 1982), (b) the Social Problem-Solving Inventory (SPSI; D'Zurilla & Nezu, 1990), (c) the Social Problem-Solving Inventory-Revised (SPSI-R; D'Zurilla, Nezu, & Maydeu-Olivares, 1995), (d) the Means-Ends Problem-Solving Procedure (MEPS; Platt & Spivack, 1975a), (e) the Interpersonal Problem-Solving Assessment Technique (IPSA~, Getter & Nowinski, 1981), (f) the Adolescent Problems Inventory (API; Freedman et al., 1978), and (g) the Inventory of Decisions, Evaluations, and Actions (IDEA; Goddard & McFall, 1992). The first three instruments are process measures, whereas the last four are outcome tests. The process measures are all self-report inventories, and the outcome tests are all verbal performance tests. These seven instruments were chosen on the basis of the following three criteria, listed in the order of importance: (a) frequency of use, (b) psychometric properties, and (c) test design and development. By far, the most frequently used measures over the past 10 years have been the PSI and the MEPS. All seven measures have adequate to good reliability and at least some im-

420

D'ZURILLA & MAYDEU-0LIVARES

pressive evidence for their validity (see references cited below). As expected, considering the time that they have been in existence, the PSI and the MEPS have the most support in terms of validity. However, most of this support focuses on their relations with various criterion measures of adjustment and psychological well-being. Unfortunately, these data alone do not constitute definitive evidence for construct validity because the problem-solving measure could be tapping some other construct that is related to adjustment (for example, some personality dimension). Thus, studies on convergent and discriminant validity are also needed, which examine the concurrent relations between the particular problem-solving measure and other measures of social problem solving, as well as measures of overlapping and distinctive constructs. In the sections below, we will first describe each measure, and then we will evaluate them with respect to the test design and development standards discussed earlier. Following this discussion, we will present some empirical evidence related to the convergent and discriminant validity of some of these instruments.

Description of the Measures Problem-Solving Inventory (PSI). The PSI is a 35-item, Likert-type inventory that is described by its authors as a measure of"problem-solving appraisal," or an individual's perceptions of the individual's own problem-solving behavior and attitudes (Heppner, 1988)0 The PSI is derived from an initial pool of 50 items that were generated to fit D'Zurilla and Goldfried's (1971) original social problem-solving model. On the basis of a principal components factor analysis of this item pool, Heppner and Petersen (1982) identified three factors, which they labeled: Problem-Solving Confidence (11 items), ApproachAvoidance Style (16 items), and Personal Control (5 items). According to Heppner (1988), Problem-Solving Confidence is defined as "self-assurance while engaging in problem-solving activities" (p. 1); Approach-Avoidance Style refers to "a general tendency of individuals to approach or avoid problem-solving activities" (p. 2); and Personal Control measures "the extent to which individuals believe that they are in control of their emotions and behavior while solving problems" (p. 2). A total PSI score has been the most popular measure used in research, but three scale scores corresponding to the three factors can also be computed. Empirical findings on reliability and validity of the PSI have been reported in Dixon, Heppner, and Anderson (1991), Elliott, Godshall, Herrick, Witty, and Spruell (1991), Elliott, GodshaU, Shrout, and Witty (1990), Heppner (1988), Heppner and Petersen (1982), Heppner, Kampa, and Brunning (1987), Larson, Piersel, Imao, and Allen (1990), and MacNair and Elliott (1992). Soctal Problem-Solving Inventory (SPS1). The SPSI is a 70-item, Likerttype inventory that is linked to the social problem-solving model introduced by D'Zurilla and Goldfried (1971) and later expanded and refined by D'Zurilla and Nezu (D'Zurilla, 1986; D'Zurilla & Nezu, 1982, 1990). According to this model, problem-solving outcomes in the real world are largely determined by two major processes: (a) problem orientation and (b) problem-solving proper. Problem orientation is a motivational process involving the operation of a

PROBLEM-SOLVING ASSESSMENT

421

set of relatively stable cognitive schemas (constructive as well as dysfunctional) that reflect a person's general awareness and perceptions o f everyday problems, as well as that person's own problem-solving ability. Together with the emotions and behavioral approach-avoidance tendencies that are assumed to accompany them, these cognitive schemas are believed to have a generalized influence on problem-solving performance. Problem-solving proper, on the other hand, involves the search for a solution through the rational application of problem-solving skills and techniques that are designed to maximize the probability of finding the best or most adaptive solution for a particular problem. In this model, four major problemsolving skills are identified: (a) problem definition and formulation, (b) generation of alternative solutions (c) decision making, and (d) solution implementation and verification. 2 On the basis of this general model, D'Zurilla and Nezu (1990) constructed a 30-item Problem Orientation Scale and a 40-item Problem-Solving Skills Scale, as well as seven 10-item subscales that assess the cognitive, emotional, and behavioral aspects of problem orientation, and the four major problemsolving skills. A global problem-solving score can be calculated, as well as separate scores for each of the two major scales and the seven subscales. The readability level of the SPSI has been estimated to be at the 12.4 grade level (Frauenknecht, 1990). 3 Data on the reliability and validity of the SPSI can be found in D'Zurilla and Nezu (1990), D'Zurilla and Sheedy (1991, 1992), and Sadowski and Kelley (1993). Social Problem-Solving Inventory-Revised (SPS1-R). The SPSI-R is a 52item, Likert-type inventory that is linked to a five-dimensional model of social problem solving which, in turn, is based on a factor analysis (Maydeu-Olivares & D'Zurilla, in press) of the original theory-driven SPSI. The inventory consists of five major scales that measure five different but related problem-solving dimensions: (1) Positive Problem Orientation (five items), (2) Negative Problem Orientation (10 items), (3) Rational Problem Solving (20 items), (4) Impulsivity/Carelessness Style (10 items), and (5) Avoidance Style (seven items).' The Positive Problem Orientation scale taps a constructive problem-solving cognitive "set" (e.g., perceived challenge, self-efficacy, positive outcome expec-

2 The term "solution tmplementation" is included in the name of the fourth problem-solving skill only to highlight the fact that solutzon ver~catlon- i. e., the momtoring and evaluation of solution outcomes-can only occur after the solution has been implemented. As noted previously, solution-implementationskills are distingmshed fromthe conceptof problem-solvingskills m our social problem-solvingmodel and, thus, are not assessed by the SPSI. Recently, Frauenknecht (1990; Frauenknecht & Black, 1995)developed the Social ProblemSolving Inventoryfor Adolescents (SPSI-A)by modifyingthe wording of the SPSI items to reduce their readability level to grade 6.8. This instrument is likely to be parucularly useful with younger adolescents and individuals with lower intellectual and educational levels. 4 This five-factorstructure has been cross-validatedin independent samples of collegestudents (Maydeu-Ohvares& D'Zurilla,in press)and high-schoolstudents (Sadowski,Moore,& Kelley, 1994).

422

D'ZURILLA & MAYDEU-OLIVARES

tancies), whereas the Negative Problem Orientation scale reflects an inhibitive or disruptive cognitive emotional orientation toward problems in living (e.g., perceived threat, self-inefficacy, negative outcome expectancies, low frustration tolerance). The Rational Problem Solving scale measures a constructive problem-solving dimension that may be defined as the rational, deliberate, and systematic application of effective problem-solving strategies and techniques. The Impulsivity/Carelessness Style scale reflects a deficient problem-solving pattern that may be described as narrow, impulsive, careless, hurried, and incomplete. The Avoidance Style scale taps another defective problem-solving dimension characterized by procrastination (putting off problem solving), passivity or inaction (waiting for problems to resolve themselves), and dependency (shifting the responsibility for problem solving to others). The Rational Problem Solving scale is broken down into four subscales (each with five items) corresponding to the four problem-solving skills assessed by the SPSI. Although the factor analysis failed to distinguish among these four skills, continued research on these subscales is recommended to determine if any of these skills might have discriminant validity in some clinical populations. Support for the reliability and validity of the SPSI-R can be found in D'Zurilla et al. (1995), Burns and D'Zurilla (1995), Chang and D'Zurilla (in press), D'Zurilla and Chang (in press), and Sadowski, Moore, and Kelley (1994). Means-Ends Problem-Solving Procedure (MEPS). The MEPS represents the operationalization of the following three hypothetical components of means-ends thinking (Platt & Spivack, 1975a, Spivack et al., 1985): (1) the ability to conceptualize the sequenced steps or means that are necessary to achieve a particular problem-solving goal, (2) the ability to anticipate possible obstacles that may interfere with goal attainment, and (3) the ability to appreciate that successful problem solving takes time or the fact that appropriate timing may be essential for effective solution implementation. According to Spivack et al. (1976), means-ends thinking is one o f the major cognitive abilities underlying the real life interpersonal problem-solving process. From this viewpoint, the MEPS would be classified as a process measure. However, because the components of means-ends thinking represent a problem solution rather than the antecedent process that enables a person to find a solution, the MEPS is classified as an outcome test within our problemsolving framework. The MEPS can be administered using either an interview or a pencil-andpaper format. Subjects are presented with a series of 10 hypothetical interpersonal problems or conflict situations consisting of incomplete stories that have only a beginning and an ending. In the beginning, the need or goal of the protagonist is specified, and at the end, the protagonist successfully satisfies the need or achieves the goal. The instructions present the instrument as a test of imagination. Subjects are asked to make up the middle part of the story

PROBLEM-SOLVING ASSESSMENT

423

that connects the beginning with the ending, s As previously noted, the M E P S uses a quantitative scoring system that computes separate frequency scores for relevant means, obstacles, and time. Although the number o f relevant means has been the most c o m m o n M E P S score used in research, Spivack et al. (1985) have more recently r e c o m m e n d e d a total score that sums the n u m b e r o f relevant means, obstacles, and time. 6 Platt and Spivack (1975b) performed a factor analysis on M E P S scores using three different samples. Each time, they obtained a single underlying factor, which provides support for the unidimensionality o f the M E P S . In addition to these results, data on the reliability and validity o f the M E P S can be f o u n d in Butler and Meichenbaum (1981), D'Zurilla (1986), Nezu and D'Zurilla (1989), Marx et al. (1992), Platt and Spivack (1975a), Schotte and C l u m (1982, 1987), and Spivack et al. (1976). Interpersonal Problem-Solving Assessment Technique (IPSAT). The IPSAT is a 46-item, theory-driven o u t c o m e test that was designed to assess interpersonal problem-solving competence in college students. The items and scoring system for the I P S A T are based on Rotter's social learning t h e o r y (Rotter, Chance, & Phares, 1972) and research o n assertive behavior. The test uses a semistructured, pencil-and-paper format. Subjects are asked to (a) imagine being in each problematic situation at the present m o m e n t , (b) write down alternative ways o f handling the situation, and (c) indicate which solution they would actually implement. Thus, in addition to assessing problem-solving outcomes, the IPSAT also assesses two process v a r i a b l e s - t h e generation o f alternative solutions and decision making. Chosen solutions are coded using the following qualitative categories: effective, avoidant, inappropriate, dependent, and unscorable, based o n criteria provided in a detailed scoring manual. Data supporting the reliability and validity o f the IPSAT are reported in Getter and Nowinski (1981). Adolescent Problems Inventory (API). The A P I is a 44-item, empirically derived o u t c o m e test that was designed to assess personal and interpersonal problem-solving competence in adolescent boys. Test construction was based on Goldfried and D'Zurilla's (1969) behavioral analytic model for assessing competence. Test items are presented via audiotape in individual testing sess The MEPS was originally a four-item test developed to measure means-ends thinking in impulsive adolescents. However, the test was later expanded to 10 items and the target population has become the general population of adolescents and adults. In addition, there is also a children's version (see Platt & Spivack, 1975a). In the research hterature on the MEPS, means have often been confused with alternative solutions. Thus, the number of relevant means has often been interpreted incorrectly as a measure of the ability to generate appropriate alternative soluuons. In fact, means are the specific steps that comprise a solution. Although MEPS stones may contam alternative solutions, the MEPS scoring system does not provide a frequency count of these different solutions. Instead, relevant means are summed across the different solutions and provide a measure of the abihty to conceptualize appropriate soluuons.

424

D'ZURILLA & MAYDEU-OLIVARES

sions. Subjects are asked to imagine being in the situation and then to report into a second audiotape machine what they would say and do if they were really there. Test scores are based on independent competence ratings using criteria outlined in a rater's manual. Empirical support for the reliability and validity of the A P I can be found in Freedman et al. (1978), Hains and H e r m a n (1989) and Simonian, Tarnowski, and Gibbs (1991). Inventory of Decistons, Evaluations, and Actions (IDEA). The I D E A is a 40-item, empirically-derived outcome test that was designed to assess heterosexual problem-solving competence in college women. Test construction was based on a modified version of Goldfried and D'Zurilla's (1969) behavioral analytic model. The targeted problem domain was the range of heterosexual problematic situations commonly faced by college women. The test instructions, testing format, and scoring system for the I D E A are basically the same as those used in the A P I . Data on the reliability and validity of the I D E A are presented in Goddard and McFall (1992).

Evaluation of Test Design and Development Because all three process measures are self-report instruments, they all have the same threats to validity that are associated with any other self-report measure. And because they are all inventories, and not performance tests, they can only estimate the quality of a person's problem-solving skills; they do not actually test the person's skills. In addition, another c o m m o n limitation of all these instruments is their instructions lack a specific definition of the term "problem." As a result, some subjects might define their problem domains either more broadly or more narrowly than the test author intended, which would tend to reduce the validity o f the test responses. 7 Except for these particular limitations, the SPSI and SPSI-R seem to adequately meet all of the other recommended standards for the construction of problem-solving inventories. However, the status of the PSI in the area of content sampling and validation is questionable. The 35-item PSI is derived from an initial item pool of only 50 items, and no procedures were reported for ensuring that this item pool would be adequately representative of D'Zurilla and Goldfried's (1971) social problem-solving model. In addition, the PSI also has a conceptual weakness in that its empirically derived three-factor structure has not been linked to any specific theory or model of social problem solving. According to the scale definitions presented by Heppner (1988), the PSI assesses problem-solving behavior only at a global level--i.e., selfconfidence, approach-avoidance tendencies, and personal control beliefs. Thus, the PSI cannot be used to assess specific problem-solving skills or test theoretical hypotheses regarding the nature or quality of specific problem-solving processes. Because all four outcome measures are verbal tests, and not observational measures, they also have some of the c o m m o n threats to validity that are associated with self-report assessment in general. Another shortcoming of all these measures is that insufficient procedures were used during test develop7 This deficiency has been corrected in the S P S I - R with the use o f new a n d improved instructions ( D ' Z u r l l l a et al., 1995).

PROBLEM-SOLVING ASSESSMENT

425

ment to ensure that the test items would generate rational problem-solving processes and solutions rather than experiential coping responses. In addition, a c o m m o n practical limitation of these tests is that their administration and scoring tend to be costly and time-consuming. Because o f the need for adequate content sampling, outcome tests that are designed to assess a broad problem domain must necessarily contain a large number of items (e.g., 46, 44 and 40 for the IPSAT, API and the IDEA, respectively). For many researchers and clinicians, these long tests are impractical and cost-ineffective. Except for the above shortcomings, the IDEA and the API appear to adequately meet all o f the other recommended standards for the development o f problem-solving outcome tests. However, the adequacy o f the MEPS and the IPSAT is uncertain because no specific information was provided about the content sampling and validation procedures, nor the procedures for developing the scoring criteria. In addition to these deficiencies, three MEPS items are questionable because they have antisocial problem-solving goals (i.e., killing a former SS trooper, stealing a diamond, getting revenge), which is likely to reduce the personal relevance o f this test for many individuals. Moreover, the MEPS scoring system focuses only on quantitative scores. As previously noted, qualitative scores may have more validity for some purposes. Finally, the instructions that present the MEPS as a test of imagination rather than a problem-solving test could reduce the ecological validity o f this instrument.

Convergent Validity In order to establish construct validity, it is important to demonstrate that the problem-solving measure (a) has relatively high correlations with other measures of social problem solving and (b) is significantly correlated with measures of similar or overlapping constructs, but not highly enough to be redundant or indistinguishable from these measures. Ideally, a test author would like to show that the problem-solving instrument is significantly related to a measure of actual problem-solving performance in the natural environment. Unfortunately, such definitive validational studies have yet to be done. One of the major obstacles to this research is the lack o f reliable and valid criterion measures of actual problem-solving performance in the real life setting. However, although not as convincing as the above results would be, four of the present measures have been reported to be significantly related to independent global judgments of problem-solving competence in the real w o r l d - t h e PSI (Heppner et al., 1982), the SPSI-R (Burns & D'Zurilla, 1995), the IDEA (Goddard & McFall, 1992), and the MEPS (Platt & Spivack, 1975a). Of the seven problem-solving measures reviewed here, correlations have been examined between the PSI and the SPSI, SPSI-R, and MEPS; between the SPSI and the MEPS; and between the SPSI-R and the MEPS, all in college student samples. The correlations between the PSI, SPSI, and SPSI-R represent process-process relations (i.e., relations between different measures of problem-solving activities), whereas the correlations between these inventories and the MEPS represent process-outcome relations (i.e., relations between problem-solving activities and specific problem solutions).

426

D'ZURILLA & MAYDEU-OLIVARES

As expected, the correlations between the PSI and the SPSI and SPSI-R were all found to be significant, and most of them are moderately high in magnitude, suggesting that the constructs measured by these inventories have a large amount of variance in common (D'Zurilla & Nezu, 1990; D'Zurilla et al., 1994). However, the studies on the relations between these inventories and the MEPS have produced mixed results; In two studies, MEPS scores were found to be significantly related to the PSI (Nezu & Ronan, 1988) and the SPSI (D'Zurilla & Nezu, 1990). On the other hand, two other studies found only nonsignificant correlations between the MEPS and the PSI (Heppner & Petersen, 1982) and the SPSI-R (Francis & D'Zurilla, 1993). Thus, the issue o f the empirical and theoretical relations between these particular problemsolving process measures and the MEPS remains cloudy, and thus, requires further investigation. In addition to the MEPS, the IDEA is the only other problem-solving outcome test that has been found to be significantly related to a measure of problem-solving processes. Goddard and McFall (1992) found that IDEA scores were related to several questionnaire measures of decision-making processes derived from McFall's (1982) social information-processing model. However, these process measures were developed for this particular study, and no data were reported on their psychometric properties. Nevertheless, however, these results strongly suggest that IDEA scores might be tapping rational problemsolving processes. Constructs that overlap with or are similar to social problem-solving processes include problem-focused coping, problem-engagement coping (i.e., problem-focused coping & cognitive restructuring), rational decision-making style, internal locus of control, optimism/pessimism, rational/irrational beliefs, positive affectivlty/negative affectivity, and self-esteem (D'Zurilla, 1986; D'Zurilla & Chang, in press; Heppner, 1988). The SPSI has been found to be correlated with internal locus of control (D'Zurilla & Nezu, 1990), and SPSI-R scores have been found to be related to problem-engagement coping (D'Zurilla & Chang, in press), internal locus of control, self-esteem (D'Zurilla et al., 1995), optimism, pessimism, positive affectivity, and negative affectivity (Chang & D'Zurilla, in press). PSI scores have been found to be associated with problem-focused coping (MacNair & Elliott, 1992), problem-engagement coping (Larson et al., 1990), rational decision making (Chartrand, Rose, E1liott, Marmarosh, & Caldwell, 1993; Phillips, Pazienza, & Ferrin, 1984), internal locus of control (Heppner & Petersen, 1982), self-esteem, irrational beliefs (Heppner, Reeder, & Larson, 1983), and negative affectivity (Chartrand et al.). All of the correlations reported in these studies are in the expected direction and none is high enough to suggest that the particular problem-solving measure is redundant with any of these other constructs. In conclusion, five problem-solving measures have empirical support for their construct validity in the sense of convergent v a l i d i t y - t h e PSI, SPSI, SPSI-R, IDEA, and MEPS. Of these measures, the PSI and SPSI-R appear to have the strongest and most consistent support. Although there are fewer

PROBLEM-SOLVING ASSESSMENT

427

data on the SPSI and the IDEA, the results are promising. On the other hand, the evidence for the MEPS is very limited.

Discrirninant Validity In order to establish the discriminant validity of a particular problem-solving measure, it is important to show that the measure (a) adds significant incremental validity to the prediction of adjustment or psychological well-being above and beyond what measures of similar or overlapping constructs contribute, and (b) has only nonsignificant or low correlations with measures of divergent constructs that are assumed to be largely independent of social problem-solving processes. In the only study in the first category, Chang and D'Zurilla (in press) found that SPSI-R scores were significantly related to measures of psychological stress and symptomatology even after partialing out the variance associated with pessimism and negative trait affectivity. In addition, SPSI-R scores were also found to significantly predict problem-engagement coping even after controlling for optimism and positive trait affectivity. Constructs that are expected to be mostly independent of social problemsolving processes include general intelligence, as measured by traditional IQ tests and academic aptitude tests (D'Zurilla, 1986; Spivack et al., 1976), and experiential coping (Epstein, 1990; D'Zurilla & Chang, in press). The problemsolving measures that have been found to be at least partially independent of general intelligence or academic aptitude include the PSI (Elliott et al., 1990; Heppner, 1988), the SPSI (D'Zurilla & Nezu, 1990; D'Zurilla & Sheedy, 1992), the SPSI-R (D'Zurilla et al., 1994), the MEPS (Spivack et al.), and the IPSAT (Getter & Nowinski, 1981). In addition, the Rational Problem Solving scale of the SPSI-R was also found to have little or no variance in common with several measures of experiential coping (D'Zurilla & Chang). In addition to the overlapping and divergent constructs identified above, it is also important for construct validation to show that problem-solving measures have discriminant validity over specific personality dimensions that are found to be significantly related to social problem-solving ability. SPSI-R scores have been found to be correlated with extroversion, neuroticism, and psychoticism, as measured by the Eysenck Personality Questionnaire (Burns & D'Zurilla, 1995), and IPSAT scores have been found to be related to deference, order, abasement, social extroversion, dominance, and aggression, as measured by the Edwards Personal Preference Schedule (Getter & Nowinski, 1981). To date, however, no studies have tested whether any of the current problem-solving measures are significantly related to external criteria of adjustment or psychological well-being after partialing out the variance associated with these particular personality measures. In conclusion, the findings reported in this section indicate that four problemsolving measures have support for their discriminant validity in terms of their relative independence from the constructs of general intelligence or academic aptitude: the PSI, SPSI, SPSI-R, MEPS, and IPSAT. In addition, the SPSI-R

428

D'ZURILLA & MAYDEU-OLIVARES

is the only instrument that has also been found to be distinguishable from measures of experiential coping, optimism, pessimism, and positive and negative trait affectivity.

General Conclusions and Recommendations Several promising instruments are currently available to researchers and clinicians who require a reliable and valid measure of social problem-solving abilities and performance. However, all of these measures have shortcomings, and none has definitive, unequivocal evidence for its construct validity at the present time. Among the process measures, the SPSI-R appears to have the strongest support in the sense of test design and development, theoretical foundation, convergent validity, and discriminant validity, and the IDEA seems to have the most impressive support among the outcome measures. Considering the limitations of all these measures, we make the following recommendations for future research: 1. More definitive studies are needed on the relations between verbal problem-solving measures and the actual problem-solving processes and solutions used by individuals in the real world. 2. More research is needed to examine and clarify the relations between process measures and outcome measures (e.g., the SPSI-R and the IDEA). 3. Reliable, valid, and cost-effective observational and self-monitoring measures are badly needed that focus on actual problem-solving processes and outcomes in the natural environment. In addition to providing a more objective and valid assessment of problem-solving performance, these measures are needed as criterion measures for the research recommended in #1. 4. A battery of reliable and valid performance measures is needed that test a person's ability to apply specific process skills and techniques effectively to specific problems (e.g., problem definition and formulation, generation o f alternative solutions, evaluation of solutions). 5. Considering the practical limitations of tests that are lengthy, there is a need for shorter tests that focus on more specific problem types (e.g., academic problems, drug-related problems, eating problems, dating problems), so that researchers and clinicians can select the test that is most useful, practical, and cost-effective for their particular purposes. 6. More definitive studies on discriminant validity are needed that test whether problem-solving measures are distinguishable from more global, overlapping constructs, such as generalized self-efficacy or perceived control, self-esteem, rational/irrational beliefs, and personality. With regard to the latter, studies focusing on the popular five-factor personality m o d e l - i.e., Neuroticism, Extroversion, Openness to Experience, Agreeableness, and Conscientiousness (Costa & McCrae, 1985)- are particularly recommended to ensure that problem-solving measures are not tapping any of these dimensions. In closing, although the current problem-solving measures all have certain shortcomings, we have attempted to provide good models for problem-solving

PROBLEM-SOLVING ASSESSMENT

429

process and outcome assessment. It is hoped that these models will not only help to improve the current measures but also provide stimulation and guidance for the future development of new and better problem-solving measures.

References Barlow, D. H., Hayes, S. C., & Nelson, R. O. (1984). The scwnttst practttioner: Research and accountabthty m chmcal and educational settmgs. Elmsford, NY: Pergamon. Burns, L. R., & D'Zurilla, T. J. (1995). Is soctalproblem-solvmg a general abdity? Personal vs. mterpersonalproblems. Manuscript submitted for publication, State Unwersity of New York at Stony Brook. Butler, L., & Meichenbaum, D. (1981). The assessment of interpersonal problem-solving skills In P. C. Kendall & S. D. Hollon (Eds.), Assessment strategwsfor cognmve-behavtoral mterventions (pp. 197-225). New York: Academic Press. Camp, C. J., Doherty, K., Moody-Thomas, S., & Denney, N. W. (1989). Practical problem solving types and sconng methods. In J. D. Sinnott (Ed.), Everyday Problem solvmg: Theory and apphcattons (pp. 211-228). New York: Praeger. Chang, E. C., & D'Zurilla, T. J. (in press). Relations between problem orientation and optimism, pessimism, and trait affectivity: A construct validation study. Behavtour Research and Therapy. Chartrand, J. M., Rose, M., Elliott, T. R., Marmarosh, C., & Caldwell, S. (1993). Peeling back the onion: Personality, problem solving, and career decision-making style correlates of career indecision. Journal o f Career Assessment, 1, 66-82. Costa, P. T., Jr., & McCrae, R. R. (1985). The NEO Personality Inventory Manual. Odessa, FL: Psychometric Assessment Resources. Davis, G. A. (1966). Current status of research and theory in human problem solving. Psychologwal Bulletin, 66, 36-54. Davison, G. C., Robins, C., & Johnson, M. K. (1983). Articulated thoughts during simulated situations: A paradigm for studying cognition in emotion and behavior Cogntttve Therapy and Research, 2, 17-40. Dixon, W. A., Heppner, P. P., & Anderson, W. P. (1991). Problem-solving appraisal, stress, hopelessness, and suicide ideation in a college population. Journal o f Counseling Psychology, 38, 51-56. D'Zurilla, T. J. (1986) Problem-solving therapy: Asoctal competence approach to chmcal mterventton. New York: Springer. D'Zurdla, T. J. (1990). Problem-solving training for effective stress management and prevention. Journal o f Cogmttve Psychotherapy: A n Internattonal Quarterly, 4, 327-355. D'Zurilla, T. J., & Chang, E. C (m press). The relations between social problem solving and coping. Cognitive Therapy and Research. D'Zurilla, T. J., & Goldfried, M. (1971). Problem-solving and behavior modificauon. Journal o f Abnormal Psychology, 78, 104-126. D'Zurdla, T. J., & Nezu, A. (1980). A study of the generation-of-alternatives process m social problem solving. Cogmttve Therapy and Research, 4, 67-72. D'Zurilla, T. J., & Nezu, A. (1982). Social problem-solving in adults. In P. C. Kendall (Ed.), Advances m cogmtive-behavioral research and therapy (Vol. 1, pp. 201-274). New York: Academic Press. D'Zurilla, T. J., & Nezu, A. M. (1990). Development and preliminary evaluauon of the Social Problem-Solving Inventory. Psychologwal Assessment: A Journal o f Consulting and Chnwal Psychology, 2, 156-163. D'Zurilla, T. J., Nezu, A. M., & Maydeu-Olivares, A. (1994). Manual f o r the Social ProblemSolving lnventory-Rewsed. Unpublished manuscript, State University of New York at Stony Brook.

430

D'ZURILLA & MAYDEU-OLIVARES

D'Zunlla, T. J , & Sheedy, C. F (1991) Relation between social problem solving abdlty and subsequent level of psychological stress m college students Journal o f Personahty and Social Psychology, 61, 841-846 D'Zurdla, T J , & Sheedy, C F (1992) The relation between social problem-solvmg abdity and subsequent level of academic competence m college students. Cognmve Therapy and Research, 16, 589-599. Elhott, T., Godshall, E, Hernck, S., Witty, T, & Spruell, M. (1991) Problem-solving appraisal and psychological adjustment followmg spmal cord injury Cognmve Therapy and Research, 15, 387-398 Elhott, T., Godshall, F., Shrout, J R, & Witty, T. E (1990) Problem-solving appraisal, selfreported study habits, and, performance of academically at-risk college students. Journal o f Counsehng Psychology, 37, 203-207 Epstein, S (1990) Cognitive-experiential self-theory. In L A. Pervm (Ed), Handbook ofpersonahty Theory and Research (pp 165-192). New York: Gmlford Press Epstein, S. (1994). Integration of the cognitive and the psychodynamic unconscious Amerwan Psychologtst, 49, 709-724 Epstein, S, Lipson, A , Holstein, C , & Huh, E (1992) Irrational reactions to negative outcomes. Evidence for two conceptual systems. Journal o f Personahty and Soctal Psychology, 62, 328-339 Epstein, S, & Meier, P (1989). Constructive thinking. A broad coping variable with specific components Journal o f Personahty and Soctal Psychology, 57, 332-350. F~schler, G L , & Kendall, P C (1988). Social cognmve problem solving and chddhood adjustment. Qualitative and topological analyses Cogmttve Therapy and Research, 12, 133-153 Folkman, S., & Lazarus, R. S (1980) An analysis of copmg in a mtddle-aged community sample Journal o f Health and Social Behavtor, 21, 219-239 Francis, D., & D'Zurllla, T J (1993) [Correlations among SPSI-R scores, the Problem-Solving Inventory, and the Means-Ends Problem-Solving Procedure] Unpublished data, State Umvers~ty of New York at Stony Brook. Frauenknecht, M (1990) Psychometric evaluation of the Personal Problem-Solving Inventory for adolescents (PPSI) Dlssertatton Abstracts Internatmnal, 52, 141A. Frauenknecht, M , & Black, D R (1995) Social Problem-Solwng Inventory for Adolescents (SPSI-A) Development and psychometric evaluation Journal o f Personahty Assessment, 64, 522-539 Freedman, B 1., Rosenthal, L., Donahoe, C P Jr, Schlundt, D G., & McFall, R M (1978). A social-behavioral analysis of skill deficits m dehnquent and non-dehnquent boys. Journal o f Consultmg and Chmcal Psychology, 46, 1448-1462 Getter, H , & NowmskL J K (1981). A free response test of interpersonal effectiveness Journal o f Personahty Assessment, 45, 301-308 Goddard, P, & McFall, R M (1992) Decision-making skills and heterosexual competence in college women. An reformation-processing analysis Journal o f Soctal and Chntcal Psychology, 11, 401-425. Goldfried, M R , & D'Zurdla, T J (1969). A behavlor-analyuc model for assessing competence In C D Splelberger (Ed), Current toptcs tn chnical and commumty psychology (Vol 1, pp 151-196). New York Academic Press Gunning, R. (1952) The techmque o f clear wrltmg New York' McGraw-Hill Hams, A A., & Herman, L P. (1989). Social cognmve skills and behavioral adjustment of dehnquent adolescents m treatment. Journal o f Adolescence, 12, 323-328 Heppner, P. P. (1988). The Problem Solvmg Inventory Palo Alto, CA: Consulting Psychologist Press. Heppner, P. P (1990) Problem solvmg and cogmtive therapy. [Special issue] Journal o f Cogmrive Psychotherapy" An Internattonal Quarterly, 4(3) Heppner, P P, Hlbel, J , Neal, G. W, Wemsteln, C L, & Rabinowltz, F. E (1982) Personal

PROBLEM-SOLVING ASSESSMENT

431

problem solving: A descnptwe study of individual &fferences. Journal of Counsehng Psychology, 29, 580-590. Heppner, P. P., & Hillerbrand, E. T. (1991). Problem-solving training: ImplicaUons for remedial and preventive training. In C R. Snyder & D. R. Forsyth (Eds), Handbook ofsoctal and chmcal psychology (pp. 681-698). Elmsford, NY: Pergamon. Heppner, P. P., Kampa, M., & Brunnmg, L (1987). The relationship between problem-solving self-appraisal and indices of physical and psychological health Cogmttve therapy and research, 11, 155-168. Heppner, P. P., & Krauskopf, C J. (1987). An information-processing approach to personal problem solving. The Counsehng Psychologtst, 15, 371-447 Heppner, P. P., & Petersen, C. H. (1982). The development and lmphcauons of a personal problemsolving inventory. Journal of Counsehng Psychology, 29, 166-175. Heppner, P. P., Reeder, B. L., & Larson, L. M. (1983). Cogmtive variables associated with personal apprmsal: Implications for counsehng. Journal of Counsehng Psychology, 30, 537-545. Krasnor, L. R., & Rubln, K. H (1981). Assessment of social problem-solving skdls m young children. In T. V. Merluzzl, C. R. Glass, & M Genest (Eds.), Cogmttve assessment (pp. 452-476). New York: Gmlford. Larson, L., Piersel, W. C., Imao, R., & Allen, S. (1990). Sigmficant predictors of problem-solving appraisal. Journal of Counsehng Psychology, 37, 482-490. Lazarus, R S, & Folkman, S. (1984). Stress, appratsal, and coping. New York: Springer. Logan, G. D. (1988). Toward an instance theory of automatlzauon. Psychologwal Revww, 95, 492-527. Logan, G D. (1989) AutomaUclty and cognmve control. In 3. A. Bargh & J. S. Uleman (Eds.), Unmtended thought (pp. 52-74) New York: Gmlford. MacNair, R. R., & Elliott, T. R. (1992). Self-perceived problem-solvmg abdity, stress appraisal, and coping over time Journal of Research m Personahty, 26, 150-164. Marx, E. M., Williams, J. M. G., & Claridge, G C. (1992). Depression and social problem solving Journal of Abnormal Psychology, 101, 78-86. Maydeu-Ohvares, A., & D'Zunlla, T. J. (in press). A factor-analytic study of the Social ProblemSolving Inventory: An integraUon of theory and data. Cogmttve Therapy and Research. McFall, R. M. (1982). A review and reformulation of the concept of social skills. Behavtoral Assessment, 4, 1-33. Meichenbaum, D., Henshaw, D., & Himel, N. (1982). Coping with stress as a problem-solving process. In H W. Krohne & L. Laux (Eds.), Achwvement, stress and anxwty (pp. 127-142). New York: Hemisphere. Nezu, A., & D'Zurflla, T. J. (1979). An experimental evaluauon of the decision-making process m social problem solwng. Cogmttve Therapy and Research, 3, 269-277 Nezu, A., & D'Zurilla, T. J (1981a); Effects of problem defimtion and formulation on decision making m the social problem solving process. Behavtor Therapy, 12, 100-106. Nezu, A., & D'Zurilla, T. J. (1981b). Effects of problem definition and formulation on the generation of alternaUves in the socml problem-solving process Cogmttve Therapy and Research, 5, 265-271. Nezu, A. M., & D'Zurilla, T. J. (1989). Socml problem solwng and negative affectwe condiUons In P. C. Kendall, & D. Watson (Eds.), Anxwty and depresston: Distmctlve and overlappmg features (pp. 285-315). New York: Academic Press. Nezu, A. M., Nezu, C. M., D'Zurdla, T. J., & Rothenberg, J. L. (m press). Problem-solvingtherapy In J. S. Kantor (Ed.), Chnical depresston during addwtton recovery: Processes, dtagnos~s, and treatment. New York: Marcel Dekker. Nezu, A. M., Nezu, C. M., & Perri, M. G. (1989). Problem-solwng therapyfordepresston: Theory, research, and clinwal guldehnes. New York: Wiley. Nezu, A M., & Ronan, G. E (1988). Social problem solving as a moderator of stress-related depressive symptoms: A prospective analysis. Journal of Counseling Psychology, 35, 134-138.

432

D'ZURILLA & MAYDEU-OLIVARES

Penn, D L., Spaulding, W, & Hope, D (1993). Problem solving from different perspecuves: An investigation of instructional context on social problem-solving abdlty. Journal o f Cogmttve Psychotherapy. A n Internattonal Quarterly, 7, 49-61. Phillips, S D., Pazienza, N., & Ferrln, H (1984). Deoslon-making styles and problem solving appraisal. Journal o f Counsehng Psychology, 31, 497-502. Platt, J J., ~z Spwack, G. (1975a). Manualfor the Means-Ends Problem Solvtng Procedure (MEPS) A measure ofmterpersonal cogntttveproblem-solvtng skdl Unpublished document. Philadelphm Hahnemann Community Mental Health/Mental Retardation Center. Platt, J J , & Spwack, G. (1975b) Umdimenslonallty of the Means-Ends Problem-Solving Procedure Journal o f Chnwal Psychology, 31, 15-16. Platt, J J., & Splvack, G. (1976). Measures ofmterpersonalcognittveproblemsolvingforadults and adolescents Unpublished document, Umverslty of Medicine and Dentistry of New Jersey. Phems, A. J , Hansen, D. J., Ford, E, Smith, S. Jr, Stark, L J , & Kelly, J. A. (1987). Behavioral small group traimng to improve the social skills of emotionally-disordered adolescents Behavtor Therapy, 18, 17-32. Poon, L. W., Rubm, D. C., & Wilson, B A (Eds). (1989) Everyday cognmon in adulthood and late hfc New York" Cambridge Umvers~ty Press. Rotter, J B., Chance, J. E , & Phares, E J (1972). Apphcattons o f a soctal learning theory o f personahty. New York: Holt, Rinehart, & Winston Sadowskl, C., & Kelley, M L. (1993) Social problem-solving in suicidal adolescents. Journal o f Consultmg and Chmcal Psychology, 61, 121-127. Sadowskl, C, Moore, L A., & Kelley,M L (1994) Psychometric properties of the Social ProblemSolving Inventory (SPS1) w~th normal and emotionally-disturbed adolescents. Journal o f Abnormal Chdd Psychology, 22, 487-500. Schotte, D E., & Clum, G A (1982). Smclde ideation m a college population. A test of a model Journal o f Consuttmg and Chmcal Psychology, 50, 690-696 Schotte, D E , & Clum, G A. (1987) Problem-solving skills m suicidal psychiatric patients. Journal o f Consulting and Chntcal Psychology, 55, 49-54 Slmoman, S J., Tarnowskl, K. J , & Gibbs, J C (1991) Socml skills and antisocial conduct of delinquents. Child Psychtatry and Human Development, 22, 17-27. Smnott, J D (Ed.) (1989). Everyday problem solving" Theory andappheattons. New York Praeger Skinner, B E (1953). Sctence and human behavtor. New York: Macmillan. Spwack, G, Platt, J J , & Shure, M. B (1976) The problem-solvmg approach to adjustment San Francisco. Jossey-Bass Spwack, G., Shure, M B, & Platt, J. J (1985). Means-Ends Problem Solvtng (MEPS) Sttmuh and scoring procedures supplement. Unpublished document, Hahnemann University Sternberg, R. J , & Wagner, R. K (Eds). (1986) Practwal mtelhgence. Nature and ortgms o f competence in the everyday world. New York: Cambridge University Press. Stone, A A., & Neale, J H. (1984). A new measure of daily coping" Development and preliminary results. Journal o f Personahty and Soctal Psychology, 46, 892-906. Tlsdelle, D. A., & St Lawrence, J. S. (1986). Interpersonal problem solving competence' Review and critique of the literature Chmcal Psychology Review, 6, 337-356. Tobm, D L., Holroyd, K A., Reynolds, R. V, & Wigal, J. K. (1989) The hierarchical factor structure of the Coping Strategies Inventory Cognitive Therapy and Research, 13, 343-361 Yoman, J., & Edelstein, B. A. (1993). Relationship between solution effectiveness ratings and actual solution Impact in social problem solving Behavior Therapy, 24, 409-430 RECEIVED: September 3, 1993

ACCEPTED: June 10, 1994