From Empirical Studies of Programmers, Sixth Workshop, Norwood NJ: Ablex Publishing Company.
WHEN, WHY AND HOW DO NOVICE PROGRAMMERS REUSE CODE? Christopher M. Hoadley Graduate Group in Science and Math Education (SESAME) 4533 Tolman Hall #1670 University of California at Berkeley Berkeley CA 94720-1670 USA tophe @ cs.berkeley.edu Marcia C. Linn Education in Math, Science, and Technology 4533 Tolman Hall #1670 University of California at Berkeley Berkeley CA 94720-1670 USA
[email protected] Lydia M. Mann Holy Names College 3500 Mountain Blvd. Oakland CA 94619-1627 USA Michael J. Clancy 779 Soda Hall #1776 Computer Science Division University of California at Berkeley Berkeley CA 94720-1776
[email protected] KEYWORDS: Computer-human interaction, Computer science education, LISP, Problem solving, Programming, Software engineering--Software reusability
TABLE OF CONTENTS
ABSTRACT
1. INTRODUCTION o 1.1 How do expert programmers reuse code? o 1.2 How do courses prepare students for reusing code? 2. STUDY 1: CODE SUMMARIES AND REUSE o 2.1 METHODS o 2.2 Analysis and Results o 2.3 Discussion and Conclusions 3. STUDY 2: CODE REUSE, CODE SUMMARIES, AND BELIEFS o 3.1 Methods o 3.2 Results 4. DISCUSSION 5. REFERENCES ACKNOWLEDGMENTS APPENDIX A: THE APPLICATIVE OPERATOR INTERVIEW o Section 1 o Section 2 o Interviewer Script APPENDIX B: THE RECURSIVE PROBLEMS INTERVIEW o Section 1 o Section 2
ABSTRACT Two studies of when, why, and how novice LISP programmers reuse code[1] suggest ways to improve programming instruction. Undergraduates in an introductory computer science course were given opportunities to reuse previously defined functions in solving problems. To reuse code students could invoke a function or tailor an example to a new situation. Both abstract understanding of a function and belief in the benefits of reusing code contributed to code reuse. Abstract understanding combined with positive views of reuse resulted in reuse 85% of the time. When neither positive beliefs nor abstract understanding were present reuse dropped to 10%. Thus, to increase reuse, we advocate instruction that emphasizes abstract understanding and stresses the benefits of reuse.
1. INTRODUCTION Large software engineering projects involve frequent code reuse. Software design teams reuse library routines and create reusable program components to simplify
programming. How can we prepare students for jobs in software engineering? We report on two studies of code reuse in introductory programming courses. We distinguish three types of code reuse. Software engineers both create and reuse software libraries. We refer to reuse of previously written code with procedure or function calls as code invocation. Students often copy lines of program examples, modifying the code to meet problem specifications. We refer to duplication of code as code cloning. Research on programming has described application of learned, generalized patterns, called plans (Soloway), templates (Linn & Clancy), or idioms (Anderson). Throughout this paper, we examine both code invocation and code cloning but not template use. Effective code reuse permits programmers to solve large problems efficiently. Yet many software engineers complain that good design practices such as reuse are neglected in modern programming projects (Wayt Gibbs, 1994; Yourdon, 1992) Often reuse is thwarted by problems in project coordination (Brooks, 1982). For example, team members may lack knowledge of relevant program components, components may not fit project needs, or component development may be unsynchronized. Preparing students to participate in large projects in the software domain requires better understanding of code reuse, the focus of this research. Do novices reuse code? Linn and Clancy (Linn & Clancy, 1992b; Linn, Katz, Clancy, & Recker, 1992) identify "recycling" of code as an important goal for programming courses. Yet informal observation suggests that spontaneous recycling of code is rare. For example, while serving as a teaching assistant in an introductory Computer Science course (taught in the Scheme dialect of LISP), the first author observed the following phenomenon. The first test featured a two-part problem labeled "Part 1" and "Part 2" on successive pages. The test was administered shortly after the introduction of lambda expressions, in which functions are first-class objects. By using the function from part 1, the student could solve part 2 using a one-line function. Otherwise the student needed to write a syntactically complex doubly-nested lambda expression. Few students realized that they could or, indeed, should reuse part 1 of the question in solving part 2. In another course, students defined a recursive function given an recurrence relation. Students used recursion as the main control structure from the very beginning of this course, studying the textbook Structure and Interpretation of Computer Programs by Abelson and Sussman (Abelson & Sussman, 1985). They wrote recursive functions in class, on homework, and on tests for over a month. In this problem, students were given this recurrence relation and were asked to write a recursive function that implemented the function and to give a brief English summary of what the function and recurrence relation did:
Although most students could implement the function using the recurrence relation, very few recognized that the function implemented multiplication. Students' difficulties in comprehending code suggested a potential barrier to code reuse. These examples suggested two factors in code reuse. First, students may lack a disposition to reuse code. Second, students may not comprehend code, even when they reuse it, thus limiting future reuse. We investigate these questions in two studies reported here. 1.1 How do expert programmers reuse code? Expert programmers reuse code in all three ways defined above: invocation, cloning, and pattern implementation. For familiar problems, experts rely on abstract patterns whereas novices tend to have fewer abstracted ideas to draw on. Templates abstract a pattern and link it to information such as basic code structures, conditions of applicability, common bugs, and test cases (Linn & Clancy, 1992b; Rist, 1991; Soloway, 1986). While experts sometimes hold similar templates, often they do not. In Pascal, experts tend to develop similar templates, possibly due to similarities in instruction and to the structure of the language (Linn & Clancy, 1992a; Linn & Clancy, 1992b). However, when Linn, Katz, Clancy, and Recker (Linn et al., 1992) replicated their Pascal work with expert LISP programmers, LISP experts viewed application operations as templates but used recursion idiosyncratically, describing a wide variety of recursive templates. Templates may rest upon different models of the programming language. For example, diSessa draws a distinction between functional and structural models of programming (diSessa, 1986). Structural models are complete, predictive models of the programming language based on a machine model. By simulating the computer running a particular procedure, one gains a structural model of how the programming language works and of the effects of each command. Soloway has shown that experts use such models in maintaining and modifying programs. Programming instruction that teaches students the effects of individual commands or that encourages tracing (stepping through programs) targets structural knowledge (Mann, Linn, & Clancy, 1994). Functional models abstract the actions necessary to implement a solution. For example, templates or plans can provide direct links between program goals and code abstractions that match these goals. Programmers can use these models without understanding how the computer would carry out the model at the machine level
(Mann, 1991). While functional knowledge can be more directly used to achieve the programmers goals, a functional model of the programming language may not cover all situations. Experts may directly reuse code, either by cloning or invoking it. Experts may also learn patterns that they then may reuse. They may use their functional and structural knowledge of the language to extend their repertoire of patterns. While template usage appears to be a natural, commonly occuring form of recycling for experts, code reuse does not occur as frequently. Since novices have few template-like patterns in the beginning, one would expect them to recycle through code cloning or invocation. 1.2 How do courses prepare students for reusing code? Programming instruction can help students learn to abstract their programming knowledge. A focus on structural knowledge helps students predict and understand a machine model. Focus on functional knowledge helps students map goals to programs. Various instructional strategies emphasize different types of information to help students reuse patterns. One model of instruction is example-based, with practice and feedback to help abstract from the examples. Anderson, Farrell, and Sauers and Anderson, Pirolli, and Farrell (Anderson, Farrell, & Sauers, 1984; Anderson, Pirolli, & Farrell, 1988) propose that students have heuristics (called weak methods) that help them generalize from examples to patterns called idioms. Problem solutions are learned and reused when like situations arise. Anderson's theory predicts that students will learn idioms and emphasizes the need for a complete repertoire of idioms. Others have attempted to teach template-like knowledge directly. Linn and Clancy (Anderson et al., 1984; Anderson et al., 1988; Linn & Clancy, 1992b) have attempted to emphasize templates as abstractions that include links to applications, common bugs, and test cases. In one course at the University of California, Berkeley, two languages were taught, Pascal and Logo, in order to get students to generalize language-independent templates (Linn & Clancy, 1995). Clancy and Linn (Clancy & Linn, 1992; Linn, 1995) presented templates in the context of case studies of complex problems (Linn et al., 1992). Schank et al. (Schank, 1989; Schank, Linn, & Clancy, 1993) implemented an online template library for students to draw upon in a Pascal course, but found that students needed practice and concrete examples to internalize the templates. Students referenced templates from the library in solving problems, but mostly relied on knowledge gained from case studies in solving complex problems. Rich and Waters (Rich & Waters, 1988) have programmers define their own templates (called clichés, in their terminology) that then can be reused automatically.
These findings have suggested that students need to construct their templates from examples and to practice them broadly to gain abstract understanding. Research indicates that programming instruction should help students develop a variety of effective models. Functional knowledge, or "how-to" knowledge, easily maps from goals to code structure but may be too narrow. Structural or machinelinked knowledge is important because it is comprehensive. Good programmers draw on both structural and functional models to gain generality. Mann (Mann, 1991) found that students developed robust functional knowledge when they had first learned a structural model. Thus, instruction should include both structural and functional information to help students develop reuseable templates. Since code reuse is one of the earliest forms of recycling available to students, it may be a precursor to development of linked structural and functional models. On the other hand, perhaps these models are needed for direct reuse of code through invocation or cloning. We undertook two studies to investigate code reuse. The first study examined code invocation on a course examination. The second examined code reuse in more detail through an extended interview session. Participants in both studies were students from introductory LISP programming courses.
2. STUDY 1: CODE SUMMARIES AND REUSE In Study 1 we investigated code comprehension and reuse. To study how students reuse code we examined performance on a midterm examination where students were asked to summarize a function and were given an opportunity to reuse the function. To determine the role of summarizing in students' reuse, we had half the students complete the summary before solving the problem and half complete it after solving the problem. Since these were written exams, we could not distinguish code cloning from template instantiation, so we only measured invocation. Our multiplication example in the introduction suggests that students learn to interpret each step, but do not comprehend the purpose of a function. Hence, we asked students to both complete sample calls and summarize the function. 2.1 METHODS Participants were 69 students in the course "Structure and Interpretation of Computer Programs." This course is the first required course for computer science majors at the University of California, Berkeley. We studied performance on a problem from the second midterm examination (see Figure 1). Students were given a function, asked to trace the function on various inputs, predict outputs, summarize the function in brief
English sentences, and write a new function, potentially by reusing the given function. Half of the students received a version of the test in which they were asked to summarize before being given a chance to reuse, and half received a version in which the chance to reuse preceded the prompt for a summary. The problem was one of several on an hour-long exam. Although reuse was not specifically requested, it shortened and simplified the solution. 2.2 Analysis and Results We investigated two questions. First, does the act of summarizing increase the likelihood of reuse? Second, is quality of summary associated with reuse? Abstract and algorithmic summaries. We scored summaries as algorithmic, abstract, or incorrect. Algorithmic summaries correctly restated the actions of the function, without specifying the relationship between the input and output of the function. These resemble glass-box models from software testing, since they rely on the details of how the function carries out its business. For the function mystery, an algorithmic summary might be, "The function checks to see if the sentence is empty; if so, it returns nil. Otherwise, if the pred of the first element is true, apply fnc to it and recurse on the rest; if the pred was false, recurse on the rest of the sentence." Abstract summaries correctly stated the relationship between the input and output of the function, regardless of how this was going to be carried out. These resemble black-box testing models, because they treat the function as a machine whose inner workings are unknown. An abstract summary for the mystery function might state, "The function produces a list of all the items in the sentence that satisfy pred, with fnc applied to them." There were a few summaries that included both types of information, explaining the overall relationship between input and output, plus the manner in which this relationship was maintained or produced. In this study, we kept the mixed responses as a separate category. The second study collapsed mixed and abstract summaries. The observed frequencies of these summaries appear in Figure 2.
Figure 1: Study 1
Problem Code invocation. In this study, correct code invocation on the exam was scored as reuse. Thirty-one of the sixty-nine students, or 45% did reuse code in their solutions. Effects of summarization on reuse. Although results were not statistically significant, students who first summarized the function and then had the opportunity to reuse it, did invoke the function more often (eighteen of thirty-four, or 53%) than students who first had a chance to invoke and then were asked to summarize the function (thirteen of thirty-five, or 37%). In addition, the type of summary was associated with code reuse, although not statistically significant. Excluding those who gave incorrect answers to the sample calls (seven of the sixty-nine students), the frequency of reuse by summary type is displayed in Figure 3. Only a minority of those with algorithmic summaries reused but over half of those giving abstract summaries reused. Figure 2: Frequencies of Function Summary Types (Study 1)
Figure 3: Summary Type vs. Reuse in Study 1 Reuse no invocation invocation Totals:
High-level 17
Summary Type Algorithmic Mixed 10 2
Incorrect 4
Totals: 33
22 39
4 14
0 4
29 62
3 5
2.3 Discussion and Conclusions Results of this study suggested a relationship between summary type and code reuse. The literature on self-explanation suggests that prompts to explain may increase understanding (Chi & Bassok, 1989; Pirolli, 1991). Students, when prompted to summarize, may have integrated functional and structural models of the code to be reused, increasing their likelihood of reuse. Furthermore, abstract understanding of a function, either with or without prompting, may also contribute to reuse. To explore these trends we conducted a second study.
3. STUDY 2: CODE REUSE, CODE SUMMARIES, AND BELIEFS Study 1 offered intriguing hints about code reuse and construction of abstract understanding among novice programmers. In Study 2 we broadened our hypotheses and increased the validity of our methods. We added an hypothesis about student disposition, examining whether student beliefs about reuse influenced performance. To increase validity, we looked at two forms of reuse across nine diverse problems. Study 1 examined one form of reuse: code invocation. Study 2 investigated both cloning and invocation. To study cloning we observed students as they solved problems and debriefed them after problem solving. To increase problem diversity we examined both recursive and non-recursive functions. 3.1 Methods
To gain insight into student decision-making this study featured individual interviews. Two sets of interviews were conducted with two different sets of students. The first interview focused on applicative operators, non-recursive functions that apply operations to items in lists. The second featured recursive functions. Participants. Thirty-one students in an introductory LISP course participated in the study. Eighteen students responded to the applicative operators questions near the middle of the fourteen week semester. Thirteen more students responded to the questions on recursive problems during the last month of the course. Recursion was introduced about two-thirds of the way through the semester. Twenty-five of the students were male and six were female (the class as a whole was 35% percent female and 65% male). Two researchers (Hoadley and Mann) performed the actual interviews. Unlike the course used in Study 1, this course attracts a general audience. Many students were taking the course to satisfy computing requirements in economics, cognitive science, or other majors. Some prospective computer science majors were also enrolled. All 130 students in the course were required to participate in one 60- to 90-minute research interviews for course credit. The course includes labs, homework, and activities with computer-based interactive programming case studies, as well as lectures and discussion sections. Figure 4: Interview procedure (Study 2) APPLICATIVE OPERATORS INTERVIEW Read two functions from the course Summarize functions Solve problem (opportunities for reuse) Solve problem (opportunities for reuse, including reuse of last problem) Read two novel functions Trace functions and provide results to sample calls Summarize novel functions Rename novel functions
Solve problem (opportunities for reuse) Solve red herring problem (no opportunities for reuse) Solve problem (opportunities for
RECURSION INTERVIEW Read three functions from the course Summarize functions Solve problem (opportunities for reuse) Solve problem (opportunities for reuse, including reuse of last problem) Read two novel functions Trace functions and provide results to sample calls Summarize novel functions Rename novel functions Discuss function from class (Students were asked to summarize and give sample calls, but were helped if they couldn't ) Solve problem (opportunities for reuse) Solve red herring problem (no opportunities for reuse) Solve problem (opportunities for
reuse, including reuse of an earlier problem) Answer debriefing questions
reuse, including reuse of an earlier problem) Answer debriefing questions
Interviews. The interview procedure is summarized below in Figure 4, and the full text of the interview problems and debriefing are included in Appendices A and B. In each interview, students summarized several functions and then solved programming problems that provided opportunities for reuse of code from some of the functions. Some distractor problems did not provide opportunities for effective reuse. Each interview consisted of two sections. In section one, the functions were drawn from the course (the textbook, the lectures, and the case study materials). Section two featured functions that the students had not previously seen. For new functions students were asked to predict output from sample calls, and to create meaningful names. Students were also asked if the functions were familiar. After completing both sections, students reflected on their problem solving. They were asked about (a) alternative solution strategies and (b) whether they had copied code from other portions of the interview. Students who did not make use of prior code were asked about opportunities for reuse. Students were asked to go back and try to reuse the appropriate prior code if time allowed. Finally, students were asked to explain when reuse was a good idea. Our data sources were students' worksheets, audio tapes of their problem-solving sessions, and in some cases interviewers' notes taken during the sessions. Invocation or template usage? To examine both cloning and invocation, this study combined student-generated code with audiotaped explanations and interviewer notes. As in the previous study, we identified those who invoked (called) previously defined functions. We then examined student work for evidence of cloning. Recall that cloning is copying code from a previously defined function to solve a new problem, while template usage is writing code from an internalized general pattern. When code in a student solution resembled a previously defined function, we searched for confirming evidence that the similarity came from copying code, rather than similar internal templates. Evidence of cloning included a student stating an earlier solution was used as a guide, or the interviewer noticing that the student was copying code from another page. When cloning could not be distinguished from chance similarity or the effects of template usage, the function was excluded from the analyses. Also, when code was reused incorrectly, the function was scored as non-reuse. Scoring. Each problem was scored for reuse of each function that could plausibly be included in a solution of the problem. For example, in the recursion interview, students were presented with three previously defined functions; nth, countdown, and correct-p-matches. One problem on the interview could be solved
using countdown and/or nth, and so two "opportunities for reuse" were scored as instances of cloning, invocation, or non-reuse. In some cases a student invoked a previously defined function that included another previously defined function, or obviated the need for a previously defined function. For example, in the applicative operator interview, the function list-max calls the max function internally, and so if a student calls the list-max function, they no longer need to call max, since he or she has implicitly invoked it. In this case, since there was really no opportunity to explicitly reuse the smaller, included function, the data point for the smaller function was excluded; it was not counted as an "opportunity for reuse." (The function max was actually a special case. Max was reused by all students but omitted from the analysis because it was viewed as a part of the LISP language rather than as a function written by a programmer.) Ambiguous instances of cloning (where it could not be determined if the student cloned or had merely written code similar to another function) were omitted from the analyses, which probably lowered the overall percentages of reuse. 3.2 Results Overall, students reused in about 40% of their opportunities for reuse. No spontaneous reuse was recorded for the distractor problems. Of the thirty-one students, five reused at every opportunity, twelve never reused, and fifteen reused on some problems. Further categorization of reuse into code invocation and code cloning revealed similar diversity. Although a few students consistently invoked or cloned prior functions, the majority of students used both cloning and invocation, often even on the same problem. Recursive problems and applicative operator problems had similar rates of reuse. Recursion problems led to more code invocation and applicative operator problems led to more code cloning (see Figure 5). Figure 5: Reuse rates of previously defined functions for Recursion and Applicative Operator Problems
Beliefs about Reuse. Beliefs about problem-solving can influence performance (Chi & Bassok, 1989; Pirolli, 1991; Schoenfeld, 1983; Songer & Linn, 1991). Beliefs about code reuse explain many of the failures to reuse observed in this study. Beginning students of programming often believe complexity should be avoided, not managed (Fleury, 1993), so we predicted student beliefs about reuse would affect their behavior. To assess beliefs we asked students the following question at the end of the interview: "When do you think it's a good idea to reuse functions from one program in another, if ever?" We also probed students' reasons for using or not reusing codes at many points in the interview. Seven students believed reuse was a good idea whenever possible and an equal number believed reusing code was generally a bad idea. The remaining students either were ambivalent towards reuse (i.e. viewed it as a matter of style or preference) or expressed conflicting views in different parts of the interview. Reasons given for reusing were ease and simplicity. One student described reuse as "how to not reinvent the wheel." Some students also described how using prior solutions made programming problems easier to think about; one student claims reuse makes problems "conceptually a lot easier." Other explanations were: "[Interviewer: Why do you reuse functions?] It's just easier that way, instead of writing them out... With the index-flat function, the one that I was going to rewrite for index, would have taken me a while to think of a new way to write it. So it just clicked in my mind that it would be easier to do this way." "[Interviewer: Do you reuse code frequently?] Yeah, I like to have the function there to look at it later on if I need to, it just, it just helps to have it like in the editor window just to be able to see it... if it needs to be adapted then, ... I adapt it, but if I can use it the way it is, then I use it the way it is. But, um, I find that making auxiliary functions helps a lot, because you can--your final function is a lot more simple... There's a problem we did, uh, I think it was the difference between dates, where we had our final functions all written out before we even started working on it really. Just because we knew what each of the auxiliaries had to do, and then, later on we figured out the auxiliaries. So, you know, if you can figure out what you want your function to do first, and find the auxiliaries, or name the auxiliaries, then..." "[Interviewer: What made you decide to use the function you defined?] It makes it conceptually a lot easier. As soon as I saw how the two were connected, I didn't have to worry about thinking through that step, assuming the first one was correct. [Later in the same interview, the interviewer asks: Why did you solve this problem using the functions provided?] Again, because I don't have to rethink through everything, as soon as I see... it's the same answer as the previous one as soon as I see how it operates."
Those opposed to reuse emphasized that functions in this course were short, that they had poor memories, and that comprehending the code of others is troublesome. For example: [Interviewer: could you use list-max if you had to?] "... list-max is such a simple function, you might as well just do it within the function. It's just a waste. Otherwise, the computer has to call a different function. There's really no point." [Interviewer: do you ever use other people's code?] "...it seems like instead of trying to figure out somebody else's---see like, when you asked me, could any of these functions work, you gotta go back, you've gotta figure out what do they do, and especially if it's... some big thing that whips around and everything and goes all over the place you're going to, figure out what this does, and if it's going to return a number or a list, and you know what you're looking for, but if you look--if you kind of remember how it works you can write it yourself, and uh, so everything, you know, I mean you know what it's going to do." [Interviewer: is it best to reuse or write from scratch?] "..it'd be best to have your, it's kind of the same concept of understanding of what's going on, it's best to have your own program, and you know how it works. But I could see how you could just plug em together. It's just kind of a taste." Some students described explicitly abstracting ideas from prior examples. For instance: [Interviewer: Do you ever reuse functions?] "I think I tend to reuse, like I was doing here, reuse more the, the concepts, rather than keeping them [the functions]... I don't necessarily call on...other people's functions, it's more like...use the stuff, incorporate." [Interviewer: Do you ever reuse functions?] "...I wouldn't go back to previous homework assignments to do my functions. I don't think I've ever used one that [the instructor] has said in class..." [Experimenter: How about from case studies?] "Um, I use the ideas from the case study. Like on one homework assignment I remember I used the idea of grouping elements. I forget what the exact assignment is, but I needed to group a list of names, until ones that are similar to each other. But it was from the statistics case study, and I used that idea, but I totally had to change the code because the code was working for numbers and they broke it down into um numbers higher than a certain value and then they added that value and then the numbers lower than that value. But for this thing that I had to do, we were working with names I think. Had to use that idea somehow. But I didn't reuse the code." [Experimenter: So why don't you use code?] Well, when I do my homework, I just go to the lab. I just do it. I don't really like to...I don't know. I like to think things out again, to make sure I understand it. And if I reuse code, I introduce all different kinds of complications."
Some students avoided explicit reuse because they viewed it as "copying the work of others." Many students asked the experimenter if reusing code was allowed. Code reuse may seem to be plagiarism for some students. Students who expressed value judgments about reuse were scored as having positive or negative beliefs about reuse. The reuse rates for each belief category is shown in Figure 6. For each group of students (pro-reuse, opposed to reuse, and neutral/other) all opportunities for reuse were tabulated. Instances of code invocation, cloning, and non-reuse were counted and summed for the entire group. The graph below shows the percentages of reuse instances out of total opportunities for reuse by group. Notably, of the seven who expressed beliefs that reuse was undesirable and their roughly 50 opportunities to invoke or clone functions, none invoked a previously defined function. Figure 6: Beliefs and Reuse (Study 2)
Comprehension and Reuse. Studies have suggested a relationship between comprehension and reuse. We examined the relationship between the type of summary the students gave of a previously defined function and whether the students then went on to reuse that particular function. Summaries were categorized as abstract, algorithmic, or incorrect. Those classified as mixed (i.e., containing both algorithmic and abstract information) in Study 1 were classified as abstract in this study. The incidence of each type of summary is similar to that found in the first study (Figure 7). Figure 7: Frequencies of Function Summary Types (Study 2)
To assess the relationship between summary type and invocation or cloning, we analyzed responses only for students who were neutral or positive towards reuse. All students who were opposed to reuse were excluded. As shown in Figure 8, for functions with abstract summaries, reuse was more frequent than for functions with algorithmic summaries. Note that these data are ennumerating each "opportunity for reuse". Included for comparison in Figure 8 are reuse rates for functions that the students themselves had created. Even students' own functions were not reused as frequently as those that were correctly summarized at the abstract level. Figure 8: Reuse and Summary Types for Students Who Believe in Reuse or are Neutral (Study 2)
Patterns of performance by student. Some students reuse more frequently than others. We examined whether reuse rates for each person were related to summary type and belief. Figure 9 shows the relationship between reuse, abstract summaries, and belief. Frequency of reuse is computed as percentage of total opportunities for reuse. Reuse rates between 0% and 33% are listed as Low, reuse rates between 33% and 66% are scored as Medium and those above 66% are scored High. Sixteen students fell into the Low reuse category (with twelve never reusing at all), eight fell into the Medium reuse category, and seven fell into the High reuse category (with five reusing 100% of the time). As can be seen, those who gave three abstract summaries were most likely to be pro-reuse and to reuse frequently.
Figure 9: Students' Beliefs, Abstract Summaries, and Reuse Rates by Person (Study 2)
4. DISCUSSION To reuse code in solving new problems, students must believe that recycling code is possible and desirable. Approximately 20% of students in an introductory course for non-computer science majors rejected code reuse. Some saw reuse as plagiarism and others distrusted code written by others. These students reused code less than 5% of the time on average. Even when reuse was an efficient and straightforward option, these students preferred to create their own code. The remaining 80% of the students who are either neutral towards or who favor reuse, in contrast, do so regularly. We refer to students who believe in reuse as recyclers. Recyclers point out that reuse is efficient, reduces complexity, and streamlines debugging. For the recyclers, code comprehension influenced both frequency and form of reuse. Code understood abstractly was reused more than code understood algorithmically. As expected, code understood at an abstract level was frequently reused. Surprisingly, code understood at an algorithmic level was reused even less than code that was misunderstood. This leads to interesting questions about the relationship between the transition from structural models of programming languages to functional knowledge of code segments. Recyclers produced abstract summaries two-thirds of the time.
Recyclers reused code they had summarized abstractly 65% of the time. Recyclers who consistently gave abstract summaries reused code 80% of the time. Thus the combination of belief in recycling of code and ability to give an abstract summary led to frequent code reuse. When students reuse code they can either clone available solutions or directly invoke code. As expected, students who produced abstract code summaries were more likely to invoke than to clone code. In addition, those giving abstract summaries often seemed to view the code they were recycling as a pattern (see quotes in 3.2 section). Recall that in this course students learned from case studies that emphasized recycling of abstract patterns called templates (Bell, Linn, & Clancy, 1994; Clancy & Linn, 1992; Mann, Linn, & Clancy, 1994). As students became adept at summarizing and reusing code, this study suggests, they start to organize their programming knowledge in template-like patterns. The transition from reuse of code to reuse of patterns deserves additional study in the context of larger programming problems. To improve student performance, this study suggests that instructors emphasize recycling of code. Even though students learned about recycling in case studies, 20% contended that they should not reuse code. In some cases, students concluded that reuse was tedious and deficient. For some students, code comprehension may stand in the way of reuse. When students had difficulty in summarizing code they also had difficulty in reusing code. Other students made the legitimate point that, for short programs, inventing new code is just as easy as recycling. These students may recycle more complex code. Still others however, believed that recycling was plagiarism, perhaps because they see it as copying work of others. This topic should be addressed in programming courses. To improve recycling, courses can also place more emphasis on code comprehension. Some students may view algorithmic code comprehension as sufficient rather than looking for abstract representations of code. To form more abstract views of code, instructors might provide examples of abstract summaries and also include reading and summarizing code among the mix of assignments in the course (e.g. (Dalbey & Linn, 1986; Mandinach & Linn, 1987). Recycling can also be fostered by encouraging students to create abstract, generalized functions in the first place. If students create functions with recycling in mind, then they will reuse code more often. To prepare programmers for the workplace, courses should develop a disposition towards recycling. Students may attempt to emulate programmers like Richard Stallman who wrote the EMACS text editor by himself (Kidder, 1981). Yet countless accounts of successful software projects stress the importance of working in teams
and using code written by others (Agogino & Wood, 1994; Lammers, 1986). If students develop a lifelong habit of reusing code of others, they will succeed in most work settings. The differences between algorithmic and abstract understandings of functions suggest that students may have a hard time moving from structural to functional understanding of their programming environment. If indeed students need both functional and structural models of the programming language in order to develop and use libraries then we must attempt to foster integration of these two types of knowledge. This topic merits further research.
5. REFERENCES Abelson, H. & Sussman, G. (1985). Structure and interpretation of computer programs. Cambridge, MA: MIT Press. Agogino, A. M. & Wood, W. H. (1994). The SYNTHESIS Coalition: Information technologies enabling a paradigm shift in engineering education. In M. Linna & P. Ruotsala (Eds.), Proceedings of the conference on computers and hypermedia in engineering eduation (pp. 3-10). Anderson, J. R., Farrell, R., & Sauers, R. (1984). Learning to program in LISP. Cognitive Science, 8(2), 87-129. Anderson, J. R., Pirolli, P. L., & Farrell, R. (1988). Learning to program recursive functions. In M. T. H. Chi, R. Glaser, & M. Farr (Eds.), The nature of expertise (pp. 153-183). Hillsdale, NJ: Lawrence Erlbaum Associates. Bell, J. E., Linn, M. C., & Clancy, M. J. (1994). Knowledge integration in introductory programming: CodeProbe and interactive case studies. Interactive learning environments, 4(1), 75-95. Brooks, F. P. (1982). The mythical man-month : essays on software engineering. Reading, Mass.: Addison-Wesley Pub. Co. Chi, M. T. H. & Bassok, M. (1989). Learning from examples via self-explanations. In L. B. Resnick (Ed.), Knowing, learning, and instruction: Essays in honor of Robert Glaser (pp. 251-282). Hillsdale, NJ: Lawrence Erlbaum Associates. Clancy, M. J. & Linn, M. C. (1992). Designing Pascal solutions: A case study approach (1st ed.), (Principles of Computer Science) (A. V. Aho & J. D. Ullman, Series Eds.). New York, NY: W. H. Freeman and Company.
Dalbey, J. & Linn, M. C. (1986). Cognitive consequences of programming: Augmentations to BASIC instruction. Journal of Educational Computing Research, 2(1), 75-93. diSessa, A. (1986). Models of computation. In D. Norman & S. Draper (Ed.), User centered system design: New perspectives on human-computer interaction (pp. 201218). Hillsdale, NJ: Lawrence Erlbaum Associates. Fleury, A. E. (1993). Student beliefs about pascal programming. Journal of Educational Computing Research, 9(3), 355-371. Kidder, T. (1981). The soul of a new machine. New York: Avon Books. Lammers, S. (1986). Programmers at work: Interviews. Redmond, WA: Microsoft Press. Linn, M. C. (1995). Designing computer learning environments for engineering and computer science: The scaffolded knowledge integration framework. Journal of Science Education and Technology, 4(2), 103-126. Clancy, M. J. & Linn, M. C. (1992). Case studies in the classroom. Special Interest Group on Computer Science Education (SIGCSE) Bulletin, 24(1), 220-224. Linn, M. C. & Clancy, M. J. (1992a). Can experts' explanations help students develop program design skills? International Journal of Man-Machine Studies, 36(4), 511551. Linn, M. C. & Clancy, M. J. (1992b). The case for case studies of programming problems. Communications of the ACM, 35(3), 121-132. Linn, M. C. & Clancy, M. J. (personal communication, May, 1995). Linn, M. C., Katz, M., Clancy, M. J., & Recker, M. (1992). How do LISP programmers draw on previous experience to solve novel problems? In E. De Corte, M. C. Linn, H. Mandl, & L. Verschaffel (Ed.), Computer-based learning environments and problem solving. Berlin: Springer-Verlag. Mandinach, E. B. & Linn, M. C. (1987). Cognitive consequences of programming: Achievements of experienced and talented programmers. Journal of Educational Computing Research, 3(1), 53-72.
Mann, L. (1991). The implications of functional and structural knowledge representations for novice programmers. Unpublished doctoral dissertation, University of California, Graduate Group in Science and Mathematics Education (SESAME). Mann, L. M., Linn, M. C., & Clancy, M. J. (1994). Can tracing tools contribute to programming proficiency? The LISP Evaluation Modeler. Interactive Learning Environments, 4(1), 96-113. Pirolli, P. L. (1991). Effects of examples and their explanations in a lesson on recursion: A production system analysis. Cognition and Instruction, 8, 207-259. Rich, C. & Waters, R. C. (1988). The Programmer's Apprentice: a research overview. Computer, vol.21(no.11), 10-25. Rist, R. (1991). Knowledge creation and retrieval in program design: A comparison of novice and intermediate student programmers. Human-Computer Interaction, 6, 1-46. Schank, P. (1989). A Pascal template library. Unpublished master's thesis, University of California, Computer Science Division. Schank, P. K., Linn, M. C., & Clancy, M. J. (1993). Supporting Pascal programming with an on-line template library and case studies. International Journal of ManMachine Studies, 38, 1031-1048. Schoenfeld, A. H. (1983). Beyond the purely cognitive: Belief systems, social cognitions, and metacognitions as driving forces in intellectual performance. Cognitive Science, 7(4), 329-363. Soloway, E. (1986). Learning to program = learning to construct mechanisms and explanations. Communications of the ACM, 29(9), 850-958. Songer, N. B. & Linn, M. C. (1991). How do students' views of science influence knowledge integration? Journal of Research in Science Teaching, 28(9), 761-784. Wayt Gibbs, W. (1994). Software's chronic crisis. Scientific American, 271(3), 72-81. Yourdon, E. (1992). Decline and fall of the American programmer, (Yourdon Press computing series) (E. Yourdon, Series Ed.). Englewood Cliffs, New Jersey: Yourdon Press.
ACKNOWLEDGMENTS
We appreciate helpful discussions with Betsy Davis, Christina Schwarz, and Sherry Hsi. Cooperation from instructors Brian Harvey and Joshua Paley is gratefully acknowledged. Ondine Shugart, Patricia Kim, and Elizabeth Hoadley helped edit the manuscript. This work was supported by a grant from the National Science Foundation as part of the Hypermedia Case Studies in Computer Science project. Christopher Hoadley was also supported by a University of California Regents' Fellowship and the Evelyn Lois Corey Fellowship during portions of this work.
APPENDIX A: THE APPLICATIVE OPERATOR INTERVIEW Section 1 Remember the following functions from the statistics case study. (defun length-max (L1 L2) (if (> (length L1) (length L2)) L1 L2)) (defun max (x y) (if (> x y) x y)) (defun list-max (L) (reduce #'max L))
1. Summarize each of these functions (what it's for) by giving a short description of each one in English. 2. Write a function max-age which takes as its argument a list of names and ages and returns the age of the oldest person. For example: (max-age '((peter 8) (latisha 7) (joanne 8) (jose 9)))
would return 9. 3. Write a function max-female to return the oldest female in a list of names, ages, and sexes. For instance: (max-female '((peter 8 m) (latisha 7 f) (joanne 8 f) (jose 9 m)))
would return joanne. Section 2
Here are two new functions: (defun keep-only (x L) (remove-if-not #'(lambda (e) (equal x e)) L)) (defun mystery (x L) (length (keep-only x (mapcar #'first L))))
1. a. What would LISP return for each of the following? (keep-only 5 '(1 2 3 4 5 6 7)) (keep-only 'the '(the bear went over the mountain)
1. b. Summarize keep-only by giving a short description in English. 2. a. What would LISP return for each of the following? (mystery 1 '((0 1) (0 0) (5 0) (1 0 1))) (mystery 'dog '((hot dog) (dog tired) (dog eat dog world) (three dog night)))
2. b. Summarize the mystery function by giving a short description in English. 3. Do these functions remind you of any functions you've encountered before in the course; are they similar to ones you've worked with or with those you've written yourself? 4. Change the name of the mystery function in the definition above to something more appropriate. 5. You are given a houseplan list. A houseplan list contains rooms, which floor they are on, and how many square feet each contains. Write a function first-floorrooms which takes a houseplan list as its argument and returns the number of rooms on the first floor. For example: (first-floor-rooms '((cellar 0 600) (living-room 1 350) (kitchen 1 400) (small-bath 1 70) (bath 2 100) (den 1 250) (bedroom 2 400) (bedroom 2 380))
would return 4. 6. Again, you are given a houseplan list. Write a function square-feet which returns the total square footage of the building. For example: (square-feet '((cellar 0 600) (living-room 1 350) (kitchen 1 400) (small-bath 1 70) (bath 2 100) (den 1 250) (bedroom 2 400)
(bedroom 2 380))
would return 2550. 7. Write a function bedrooms which takes a houseplan list and returns the number of bedrooms. (rooms named "bedroom") Interviewer Script I am doing a study of how people in CS3 learn about functions. I'll be asking you to solve some problems, and then later I'll be asking you questions about how you solved them. These problems won't affect your grade. If you'd like, I can answer questions about the problems at the very end of the interview. I'd like to tape record the interview in case I miss something you said--the tapes and your results will be confidential. Is that OK? And let me make sure I've got your name down right for telling Mike you have credit for interviewing. SECTION 1 (debriefing) 1. Describe, if you can, how you solved the problem (ask them this particularly if they didn't say much) 2. Can you think of any other ways of doing the problem? What are they?Did you consider them while working on the problem? What made you choose this way? 3. a. (if they did use functions) Why did you solve this problem using the functions provided? In general, when do you use previously defined functions? (When is it good to use previously defined functions, and when is it good to write new ones or rewrite the old ones?) 3. b. (if they seem to have cloned) Were you using any code or examples as a guide for the problem? Did you use the code from the statistics case study here? (If not) Were you using another example from the course or were just coming up with it? 3. c. (if they did not use functions, i.e., cloned or built from scratch) Do you think this problem could be solved using the functions here? Could you use them as is or would you have to change them?
SECTION 2 (debriefing) 1. Do the functions mystery or keep-only remind you of other functions you've encountered in the course? 2. Describe, if you can, how you solved the problem (ask them this particularly if they didn't say much) 3. Can you think of any other ways of doing the problem? What are they? Did you consider them while working on the problem? What made you choose this way? 4. a. (if they did use functions) Why did you solve this problem using the functions provided? In general, when do you use previously defined functions? (When is it good to use previously defined functions, and when is it good to write new ones or rewrite the old ones?) 4. b. (if they seem to have cloned) Were you using any code or examples as a guide for the problem? Did you use the code from the statistics case study here? (If not) Were you using another example from the course or were just coming up with it? 4. c. (if they did not use functions, i.e., cloned or built from scratch) Do you think this problem could be solved using the functions here? Could you use them as is or would you have to change them? 5. Do you ever reuse functions in CS3? Only ones you've written, or only ones other people have written (like from the book, or from lecture)? When do think it's a good idea to reuse functions from one program in another, if ever?
APPENDIX B: THE RECURSIVE PROBLEMS INTERVIEW Section 1 Remember the following functions from class. (defun nth (n L) (cond ((null L) nil)
((= n 0) (first L)) (t (nth (- n 1) (rest L)))) ) (defun countdown (n) (cond ((= n 0) nil) (t (cons n (countdown (- n 1))))) ) (defun correct-p-matches (perm guess) (cond ((null perm) 0) ((equal (first perm) (first guess)) (+ 1 (correct-p-matches (rest perm) (rest guess)))) (t (correct-p-matches (rest perm) (rest guess)))) )
1. Summarize each of these functions (what it's for) by giving a short description of each one in English. 2. Write a function zip that takes as its argument two lists and returns a list of corresponding elements from the two lists. For example: (zip '(red green yellow) '(apple kiwi banana))
would return ((red apple) (green kiwi) (yellow banana)).
3. Write a function number-elements that takes a list as an argument, and "numbers" each element of the list. For example: (number-elements '(partridge-in-a-pear-tree turtle-doves french-hens))
would return ((1 partridge-in-a-pear-tree)(2 turtle-doves)(3 french-hens)).
Section 2 Here are two new functions: (defun mystery1 (flat) (cond
((null flat) nil) (T (cons (list (first flat) (second flat)) (mystery1 (rest (rest flat))) )) ) ) (defun mystery2 (table key new-value) (cond ((null table) nil) ((equal (first (first table)) key) (cons (list key new-value) (rest table))) (T (cons (first table) (mystery2 (rest table) key new-value))))))
1. a. What would LISP return for each of the following? (mystery1 '(a b c d e f)) (mystery1 '(1 (a b) 2 x 3 ((z))))
1. b. Summarize mystery1 by giving a short description in English. 2. a. What would LISP return for each of the following? (mystery2 '((a b) (x y)) 'x 27) (mystery2 '((a b) (x y)) 'm '(3 2 1))
2. b. Summarize the mystery2 function by giving a short description in English. 3. Do these functions remind you of any functions you've encountered before in the course; are they similar to ones you've worked with or with those you've written yourself? 4. Change the name of the mystery1 function in the definition above to something more appropriate. 5. Change the name of the mystery2 function in the definition above to something more appropriate. 6. Here is the function flatten that you saw in the recursion lab exercises.
(defun flatten (LoL) (cond ((null LoL) nil) ((atom LoL) (list LoL)) (T (append (flatten (first LoL)) (flatten (rest LoL)))) ) )
What will LISP return for each of the following? (flatten '(((a b) c) d)) (flatten '((a b) (c d)))
7. You are given a list of student names with associated exam scores. Write a function adjust-score that, given the list and a student name and a number, returns the list that results from adding the number to the score associated with the given name. For example: (adjust-score '((john 13) (pekyew 17) (robin 5)) 'pekyew 3)
should return ((john 13) (pekyew 20) (robin 5))
8. Again, you are given a list of student names with associated exam scores. Write a function average-score that returns the average score of all the students in the list. (You may assume that there is at least one student in the list.) For example, (average-score '((kay 13) (themba 18) (wai-yee 5)))
should return 12, that is, (13 + 18 + 5)/3. 9. Now consider the same data represented in a flat list, in which names and scores appear in the same order as in a regular list but without the pesky parentheses. For example, the flat version of ((kay 13) (themba 18) (wai-yee 5))
is (kay 13 themba 18 wai-yee 5)
Write a function change-assoc-val that, given a flat list, a name, and a new score, changes the score associated with the name to the new value. For example, (change-assoc-val '(kay 13 themba 18 wai-yee 5) 'wai-yee '20)
should return (kay 13 themba 18 wai-yee 20)
Notes [1] To avoid confusion, we will refer to programs or text meant to be used by a computer as code, and will refer to classifications used in data analysis as scores or scorings. Copyright © 1995 by the authors. All rights reserved. The copyright will be transferred sometime in the near future to Ablex Publishing Company, Inc. without notice and this version may no longer be accessible. This document is included in this server by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. This work may not be reposted without the explicit permission of the copyright holder. Christopher Hoadley tophe @ cs.berkeley.edu
