Identifying didactic knowledge: An empirical study of ... - Springer Link

Higher Education (2005) 49: 345–372 DOI 10.1007/s10734-004-6677-x

Ó Springer 2005

Identifying didactic knowledge: An empirical study of the educationally critical aspects of learning about information systems CHRIS COPE1 & MIKE PROSSER2 1

Department of Information Technology, La Trobe University, PO Box 199, Bendigo, 3552, Victoria, Australia (E-mail: [email protected]); 2Higher Education Academy, U.K. Abstract. This paper backgrounds and illustrates an approach to researching didactic knowledge. Underlying the approach are some of the frameworks developed from the student learning research. Based on these frameworks the experience of learning about a particular phenomenon is theorised as having educationally critical aspects. Without addressing these aspects in learning experiences, students are highly unlikely to achieve the target understanding desired by the teacher. The approach developed to research educationally critical aspects compares scholarly and student experiences of learning about a phenomenon using the analytical framework of a structure of awareness. An application of the approach is reported in the form of a phenomenographic study which researched didactic knowledge about the concept of an information system (IS). Multiple, previously unreported, educationally critical aspects of learning about the concept of an IS were identified. Some implications of this finding for the design of IS undergraduate curricula, textbooks, teaching strategies and learning tasks are proposed. Finally, the research approach is promoted as a means to investigate didactic knowledge about other phenomena. Keywords: didactic knowledge, educationally critical aspects, information systems, phenomenographic research, student learning research.

Introduction An on-going issue for universities is external pressure for quality assurance. While a common perspective on quality assurance involves a focus on rating departments and teachers, an alternative perspective puts the focus on improving students’ learning outcomes. From this alternative perspective a precursor to improvement is research into didactic knowledge – how a phenomenon is best taught and learnt to encourage high quality, conceptual development learning outcomes (Bowden and Marton 1998; Entwistle 1998; Ramsden 2003). A powerful contribution to identifying didactic knowledge has been made by research into students’ and teachers’ perceptions of their own experiences of learning environments. This research has become known

346

CHRIS COPE AND MIKE PROSSER

as the student learning research (Biggs 1999). One of the vital contributions to the student learning research has been the work of Noel Entwistle. Through empirical studies, collation of research findings and theorising about teaching and learning, Entwistle has developed a conceptual framework which describes the teaching–learning process in higher education in terms of the major influences on the quality of students’ learning outcomes (Entwistle 1998, 2000, 2003; Entwistle and Smith 2002; Entwistle et al. 2002). An important part of Entwistle’s framework concerns target and personal understanding (Entwistle and Smith 2002). Target understanding involves a teacher interpreting a curriculum to describe the level of understanding of a phenomenon a specific group of students should attain. The teacher decides on the aspects of the phenomenon which need to be covered and the relative emphasis to be given to each aspect. Personal understanding is the level of understanding attained by a student in response to their perception of the target understanding, the learning environment and the expectations of the teacher. The empirical study reported in this paper is underlain by, extends and practically implements the part of Entwistle’s conceptual framework which deals with target and personal understanding. The study extends the notion of target understanding to suggest that teachers should be describing a scholarly learning experience to which students should aspire. The scholarly experience incorporates both the target understanding and the specifics of an optimal approach to learning which is likely to lead to the development of the target understanding. Importantly the study also proposes and demonstrates an approach to identifying the aspects of the scholarly learning experience which need full emphasis if the target understanding is to be achieved. These aspects are termed educationally critical aspects and form vital didactic knowledge about a phenomenon. As an example of educationally critical aspects, Marton and Booth (1997) have analysed some of the research into the experience of developing arithmetic skills. To develop these skills requires learning to experience certain aspects of numbers simultaneously. An ordinal aspect refers to the position of a number in a sequence. A cardinal aspect refers to numerosity, the sevenness of seven, for example. A part-whole aspect concerns the ability to experience a number as ‘a sum of smaller numbers and as a part of larger numbers’ (p. 60). Each of these aspects is educationally critical to the development of arithmetic skills, as is the ability to experience each aspect simultaneously. Unless a student comes

IDENTIFYING DIDACTIC KNOWLEDGE

347

to terms with each of these educationally critical aspects in learning experiences, the development of arithmetic skills is unlikely. The phenomenon investigated in the empirical study reported in this paper was the concept of an information system (IS). The problem addressed was lack of didactic knowledge about the concept of an IS in the literature. Without this knowledge, the achievement of an appropriate target understanding by undergraduate students, as prospective IS professionals, cannot be assured. Yet, a logical argument has been proposed for a link between an inadequate personal level of understanding of the concept of an IS by practitioners and the reported failure of a significant proportion of IS development projects (Weber 1996; Cope et al. 1997; Cope and Horan 1998; Cope 2000a;). Researching didactic knowledge about IS, in the form of the educationally critical aspects of the experience of learning about the concept of an IS, could provide an impetus for improved teaching and learning, and a contribution toward quality assurance of IS courses. The paper should be of value to researchers and teachers interested in assuring the quality of students’ learning in higher education in general. Although the results of the study are specific to one phenomenon, the research approach is not.

Theoretical background Underlying the notion that the experience of learning about a phenomenon has educationally critical aspects which form important didactic knowledge is a phenomenographic perspective on learning. This perspective has been derived from an interpretation, in terms of the analytical framework of a structure of awareness, of three consistent findings of phenomenographic research into student learning (Booth 1994, 1997; Marton and Booth 1996, 1997; Marton 1998). This section of the paper considers the nature of phenomenographic research, the three consistent findings, the interpretation, a model for analysing and understanding learning experiences developed from the interpretation, and the notion of educationally critical aspects.

Phenomenographic research approaches Phenomenographic research approaches were developed in the early 1970s to qualitatively investigate the different ways in which groups of

348


individuals conceptualised, perceived or understood phenomena in the world (Marton 1981). Phenomenographic research takes a second-order research perspective – the focus is on analysing other peoples’ accounts of their experiences of phenomena. Phenomenographic studies have been widely used to investigate aspects of students’ learning experiences in higher education (Entwistle 1997).

Three key insights into student learning The first of the major insights to which phenomenographic studies have substantially contributed concerns students’ levels of understanding of a phenomenon. The results of many studies have shown that a phenomenon can be understood in a limited number of qualitatively different ways (for example: Johansson et al. 1985; Lybeck et al. 1988; Prosser and Millar 1989). Of considerable importance, the research has found that the different ways of understanding a phenomenon are related in a hierarchy of levels of understanding based on logical inclusiveness. Levels of understanding higher in the hierarchy represent deeper levels of understanding and are inclusive of levels of understanding lower in the hierarchy. The second of the major insights concerns students’ approaches to learning. Two qualitatively different approaches to learning have been consistently described – surface and deep learning approaches (for example, Marton and Sa¨ljo¨ 1976; Entwistle and Ramsden 1983; Biggs 1988; Entwistle 1997; Svensson 1997). A surface approach is characterised by a student completing a learning task as perceived without attributing meaning to the task. The learning task is focused on in isolation. A typical intention of a student adopting a surface approach is rote-learning of the content associated with a learning task for later reproduction in assessment situations. A deep approach, in contrast, seeks meaning in a learning task through trying to relate the task to other tasks and/or existing understanding and/or personal experience. A deep approach focuses on developing the cohesive whole which is considered to represent understanding (Entwistle 1998). Two other findings about students’ learning approaches are important to the study reported here. First, students cannot be categorised as surface or deep learners. In different learning contexts the same student has been found to take different approaches to learning about the same phenomenon (Entwistle 1979; Entwistle and Ramsden 1983; Ramsden 1997). Second, although the essence of surface and deep approaches


349

remains the same, specific important detail may change from context to context (Booth 1997; McCune and Entwistle 2000). The finding of a significant relationship between students’ learning approaches and learning outcomes is the third of the major insights. Qualitative and quantitative studies at both task and topic level have found that surface learning approaches are associated with lower levels of understanding of a phenomenon and deep learning approaches, deeper levels of understanding (for example, Marton and Sa¨ljo¨ 1976; Entwistle et al. 1979; Prosser and Millar 1989; Crawford et al. 1994, 1998; Hazel et al. 1996, 2002).

The analytical framework of a structure of awareness A structure of awareness, based on the field of consciousness (Gurwitsch 1964), has been used as a framework by Booth (1997) and Marton and Booth (1997) to analyse the three consistent findings of phenomenographic research. The analysis led to the development of the particular phenomenographic perspective on learning on which this paper is based. A description of a structure of awareness based on Booth (1992, 1997), Bowden and Marton (1998), Marton (1998) and Marton and Booth (1997) and a diagrammatic representation follows (Figure 1). Theory suggests that awareness is made up of three overlapping areas: the margin, the thematic field and the theme (Figure 1). When

Margin External horizon

Thematic field

Theme

Internal horizon

Figure 1. A structure of awareness.

350


contemplating some phenomenon in the world at a particular time and in a particular context, an individual’s awareness is likely to consist of aspects of the phenomenon evoked by the context. These aspects will be simultaneously present in awareness and are known collectively as the thematic field. The individual will also be aware in a less focussed sense of other aspects of the world not considered to be related to the phenomenon, for instance, the noise of a car driving past. These non-related aspects of the world make up the margin of awareness. Out of all the aspects making up the thematic field, a number of related aspects of the phenomenon will emerge and become the focus of awareness. These related aspects are known as the theme of awareness. A structure of awareness has also been described by Marton and Booth (1997) in terms of an internal and an external horizon (Figure 1). The internal horizon consists of the aspects of the phenomenon simultaneously present in the theme of awareness, and the relationships between these aspects and between the aspects and the phenomenon as a whole. The external horizon consists of the thematic field and the margin, that is, all aspects that are part of awareness at a particular instant but which are not thematic. The external horizon as an area of awareness forms the context in which the internal horizon sits. The boundary between the external and internal horizons delimits the phenomenon from its context. As an individual’s awareness takes on a particular structure when a phenomenon is contemplated, a level of understanding of a phenomenon can be described in terms of a structure of awareness. A description should incorporate a structural component (the nature of the internal and external horizons), and a referential component (the meaning inherent in the structure). The detail of the structural component should include the aspects of the phenomenon simultaneously present in the internal horizon, the way each aspect is conceptualised, the existence and nature of relationships between aspects, and the nature of the boundary between the internal and external horizons.

Interpretation of the three consistent findings of phenomenographic studies The finding of inclusive hierarchies of levels of understanding of a phenomenon have been interpreted logically in terms of a structure of awareness by Marton and Booth (1997). In comparing different levels of understanding in a hierarchy, the deeper levels involve an internal horizon containing more aspects of the phenomenon and/or an individual


351

aspect of the phenomenon conceptualised in a deeper way and/or more and better defined relationships between the aspects. Being aware of more aspects of a phenomenon means that aspects of the phenomenon which may have been part of the external horizon have become part of the internal horizon. In this circumstance the phenomenon is being understood as a broader entity in its context. The differences between surface and deep learning approaches have been considered logically in terms of the framework of a structure of awareness (Booth 1997). One major difference lies in whether or not the overall meaning of the content associated with a learning task is part of the internal horizon of awareness. In a deep approach, the developing meaning is part of the internal horizon, in a surface approach it is not. Another difference lies in the complexity of the internal horizon of awareness. In a surface approach, the internal horizon is likely to consist of the current learning task, the up-coming assessment task and the final exam. In a deep approach different perspectives on a particular learning task and related tasks in the same or different subject areas and previous experiences are likely to be aspects of the internal horizon of awareness. The empirical findings of significant relationships between level of understanding of a phenomenon and approach to learning have also been explained logically using the framework of a structure of awareness (Booth 1997). A deep understanding infers a structure of awareness with many strongly related aspects of the phenomenon simultaneously present in the internal horizon. Such a structure of awareness requires an active learner, one who seeks to bring new perspectives to bear on a phenomenon, one who looks for relationships between different aspects of a phenomenon. A student using a surface learning approach only considers isolated aspects of the content they are learning. There is no intention to seek any relationships between aspects of the content. The development of a more complex structure of awareness (a deeper understanding) for such a learner is unlikely. A student using a deep learning approach, in comparison, opens themselves up to the possibility of changing their structure of awareness through trying to see how the parts form a whole, through trying out their understandings in different situations, through relating what they are learning to other parts of the subject, other subjects and their personal experiences. In this situation, awareness is dynamic. There is the possibility of the development of a more complex structure of awareness and, hence, a deeper understanding of a phenomenon.

352


A model of an experience of learning about a phenomenon The three insights into student learning and their interpretation using a structure of awareness have been used by Marton and Booth (1997) to develop a model for analysing and understanding experiences of learning about a phenomenon. A learning experience can be analysed and described in terms of related ‘what’ and ‘how’ components. The ‘what’ component concerns the level of understanding of the phenomenon developed as a consequence of the learning experience. The ‘how’ component concerns the approach used to learn about the phenomenon. While the ‘what’ and ‘how’ components of a learning experience are inseparable in practice, the different components can be analysed separately and the relationship between the components investigated to describe and understand the experience as a whole.

The notion of educationally critical aspects The significance of Marton and Booth’s (1997) model of a learning experience is that students learning about a phenomenon will be undergoing a learning experience consisting of related ‘what’ and ‘how’ components. The findings of many phenomenographic studies suggest the existence of distinct qualitative variation in the ‘what’ and ‘how’ components. Given that the ‘what’ and ‘how’ components of a learning experience have been found to be related, it follows that distinct variation in learning experience is likely. Some of the various learning experiences will be more successful than others. Successful learning experiences are more likely to use a deep learning approach leading to a deeper level of understanding of a phenomenon. The notion then arises of aspects of the experience of learning about a phenomenon in which there are critical differences between less and more successful learning experiences. These aspects are educationally critical to the development of a deep understanding of the phenomenon. Some educationally critical aspects might lie in the ‘what’ component of a learning experience – level of understanding. In a teaching situation it seems important that a teacher interpret the curriculum to determine a target understanding of a phenomenon desired of the students. Students will develop their own personal understanding of the phenomenon in response to the learning experiences designed and facilitated by the teacher (Entwistle and Smith 2002). The first insight into student learning described previously indicates that the various personal


353

understandings of a group of students could be described in a hierarchy of a limited number of different levels. These levels would, at least initially, not be as deep as the level of the target understanding. There are likely to be educationally critical differences between the various student levels of understanding of the phenomenon and the target understanding (Booth 1997; Marton and Booth 1997; Bowden and Marton 1998; Marton 1998). If a structure of awareness (Figure 1) is used as an analytical framework to consider different levels of understanding of a phenomenon then the source of these critical differences can be threefold. First, aspects of the phenomenon in the internal horizon of the target understanding may not be part of some or any of the students’ levels of understanding. Second, individual aspects in the internal horizon of the students’ levels of understanding may not be conceptualised in the same way as in the target understanding. Third, the number and strength of relationships between the different aspects of the phenomenon in the internal horizon may vary between the students’ and the target understanding. The aspects, and relationships between aspects, of a phenomenon in which there are critical differences between the target and the students’ levels of understanding are likely to be educationally critical to the development of the target understanding. The term ‘threshold concepts’, used by Entwistle (2003) and Meyer and Land (2003, in press) to describe certain concepts which are keys to developing deeper ways of understanding a phenomenon, seems equivalent to the educationally critical aspects of a target understanding of a phenomenon. Some educationally critical aspects might lie in the ‘how’ component of a learning experience – approach to learning. Logically there are likely to be aspects of approach to learning about a specific phenomenon in which there are critical differences between an optimal deep approach desired by a teacher and the less effective learning approaches likely to be used by students. A learning approach which does not incorporate these educationally critical aspects is unlikely to lead to the development of the teachers’ target understanding. Other educationally critical aspects might lie in the relationship between the ‘what’ and ‘how’ components of a learning experience. The details of this relationship are likely to vary from phenomenon to phenomenon. As qualitatively different ‘what’ and ‘how’ components of learning about a particular phenomenon can be expected, so can qualitatively different relationships between the ‘what’ and ‘how’ components. There are likely to be aspects of the relationship between the teachers’ target level of understanding and an optimal deep approach to

354


learning in which there are educationally critical differences with the various relationships found in students’ learning experiences. The logical, likely existence of educationally critical aspects in the ‘what’, ‘how’ and relationship between ‘what’ and ‘how’ aspects of a learning experience suggests that teachers should be interpreting curricula to design a scholarly learning experience to which students should aspire, not just a target understanding. Knowledge about all educationally critical aspects of the experience of learning about a phenomenon allows learning tasks to be designed which are more likely to encourage the development of the target understanding. A review of the IS education research literature found that the educationally critical aspects of learning about the concept of an IS at the undergraduate level are not known (Cope 2000a). The aim of the empirical study reported next was to identify the educationally critical aspects as a contribution to didactic knowledge.

Method A research approach for investigating the educationally critical aspects of the experience of learning about the concept of an IS was devised on the basis of the theoretical background outlined in the previous section. A scholarly experience of learning about the concept of an IS was identified from the literature and compared with a group of undergraduate students’ learning experiences identified empirically. The scholarly learning experience represented current research knowledge about the target understanding of the concept of an IS appropriate for new graduates and specific detail of an optimal deep approach to learning about IS, one likely to be effective in developing the target level of understanding. Identifying educationally critical aspects required comparing the scholarly and students’ learning experiences to determine aspects of the scholarly learning experience that were not addressed at all or were addressed inadequately in the students’ learning experiences. The method involved comparing the ‘what’ and ‘how’ and relationship between ‘what’ and ‘how’ components of the different learning experiences and then collating the findings to describe the educationally critical aspects of the learning experience as a whole. To provide data for the comparison, a phenomenographic study of undergraduate students’ experiences of learning about the concept of an IS was undertaken at La Trobe University, Bendigo, Australia.


355

Data were collected from 112 students undertaking a year of study about IS early in a Bachelor of Computing. Data collection was both intensive and extensive. The intensive data collection consisted of semi-structured interviews at the beginning (n ¼ 23) and end (n ¼ 15) of the year of study. The extensive data collection consisted of questionnaires at the beginning (n ¼ 81), middle (n ¼ 62, 58 retained from the original sample) and end (n ¼ 59, 32 retained from the original sample) of the year. Participants were a convenience sample of volunteers. Analysis of the data occurred in two stages. First, the interview transcripts were combined to form a large pool of decontextualised statements about the experience of learning about the concept of an IS. The pool of statements was analysed using a phenomenographic approach to investigate the ‘what’ and ‘how’ components of the learning experiences. Hierarchies of distinctly different levels of understanding of the concept of an IS and approaches to learning about IS were described. The second stage of data analysis investigated the relationship between the ‘what’ and ‘how’ components. Each interview transcript and questionnaire was classified against the two hierarchies. Multiple researchers, some external to the study, were used to classify the transcripts and questionnaires as part of establishing the reliability of the data analysis. Exact statistical techniques (Babinec and Mehta 1999) were then used to investigate the relationship (see Cope (2000a) for a full description of the method).

Results A scholarly experience of learning about the concept of an IS The nature of the experience of learning about the concept of an IS of an exemplary undergraduate scholar was described, based on a review of the literature (Cope 2000a). This scholarly learning experience is described in terms of ‘what’ and ‘how’ components and the relationship between them. ‘What’ component This component concerns the nature of a target understanding of the concept of an IS appropriate for new graduates as prospective IS professionals. To contribute productively to the successful development of

356


an IS in a modern organisational context new graduates need to view an IS as a social system in which decision making is supported by embedded IT. The IT captures, stores, maintains, organises and outputs data in a formal, organised way and assists with the communication of informal data within and from outside the organization. The people and the processes they perform are part of the IS. In particular, people attribute meaning to the formal and informal data they receive from the IT and make decisions based on the meaning. The scope of an IS needs to be conceptualised as incorporating multiple, interrelated organisational functions (Clarke 1992; Land 1992; Wand and Weber 1995; Winter et al. 1995; Falkenberg et al. 1996; Beynon-Davies 1998; Alter 1999).

‘How’ component Significant contributions to the IS education (Lu 1994; Thomson 1994; Godfrey 1995; Cope and Horan 1997; Wagner 1997) and phenomenographic research literature (Booth 1997; Marton and Booth 1997; Bowden and Marton 1998) were used to describe the nature of an optimal deep approach to learning about the concept of an IS for undergraduate students. Such a learning approach would incorporate an explicit intention to develop the related set of aspects of an IS which constitute the target understanding. The approach would involve students actively seeking different perspectives on aspects of the concept of an IS in a variety of cooperative, experiential learning tasks; in other students’ levels of understanding and approaches to learning about IS; and in personal experiences. Through reflection, the meaning inherent in each perspective would be sought and related to current understanding and the meaning inherent in other perspectives. Underlying this approach would be an awareness on the part of the student of the learning approach they are using and their developing understanding of the concept of an IS.

Relationship between the ‘what’ and ‘how’ components Based on the results of many phenomenographic studies of students’ learning experiences, it seems logical that, in an IS education context, the ‘what’ and ‘how’ components of the scholarly learning experience are related. Only through approaching learning about IS in the optimal manner described is the development of the target understanding likely.


357

Variation in a group of undergraduate students’ experiences of learning about the concept of an IS ‘What’ component Six distinctly different but hierarchically related levels of understanding of the concept of an IS were described in the phenomenographic study of the interview transcripts (Table 1). Only the referential aspect (meaning) is given for each level of understanding. Detail of the structural aspects and quotes illustrating each level of understanding have been published elsewhere (Cope 2000a, 2000b, 2002). Levels of understanding higher in the hierarchy (Levels 5 and 6) are inclusive of and represent a deeper understanding of the concept of an IS than those lower in the hierarchy. Levels of understanding higher in the hierarchy involve an awareness of more related aspects of an IS and delimit an IS as a broader entity within an organisation. Levels of understanding 1 through 4 represent IT-only views of an IS. Levels of understanding 5 and 6 incorporate people in the IS, one aspect of an IS as a social system. ‘How’ component Two distinctly different approaches to learning were identified in the interview transcripts. A detailed description and quotes to illustrate the various aspects of the learning approaches can be found in Cope (2000a, 2003).

Table 1. Levels of understanding of the concept of an IS Level

Meaning

6

A number of communicating information systems within a single organisation A computerised data manipulation system and people gathering data, disseminating information and communicating to support a single organisational function A computerised data manipulation system supporting many people within a single organisational function A data manipulation system supporting an individual within a single organisational function A simple information retrieval system. A personal search of a static information source

5

4 3 2 1

358


1. Student Surface Approach – Acquiring knowledge associated with isolated aspects of the content with the intention of being able to recall or apply that knowledge in assessment situations. In this approach, learning about IS was experienced as a series of isolated and generally unrelated learning and assessment tasks which had to be completed successfully to satisfy academic achievement requirements. Each learning task was experienced as a potential source of knowledge which needed to be acquired. Knowledge was variously experienced as facts contained in lecture notes and summaries, formulae, how and when to apply formulae, tactics to solve problems, skills, and how to use software. There was variation in the experience of acquiring this knowledge. Acquiring knowledge was construed as either an active, intentional process of memorising the content or a less active process of gaining knowledge through absorbing the content, having it sink in, taking it in. There was also variation in how the acquired knowledge was to be recalled or applied in assessment situations. In some instances the intention was to be able to directly recall the knowledge. In other instances, the intention was to be able to apply memorised procedures, problem solving approaches and software instructions. 2. Student Deep Approach – Seeking the meaning of the content with the intent of developing or deepening understanding. In this approach, learning about IS involved a process of seeking the meaning of the content encountered in learning and assignments tasks. These tasks were perceived as involving some content which had a meaning beyond the immediate task. The learning approach included a motivation to seek this meaning as a part of learning in the academic setting and for personal reasons beyond the academic setting. Meaning was sought in the learning and assignment tasks by relating them to other aspects of the same subject’s content or the content of other IS and computing subjects. The intention of the learning approach was to develop or deepen understanding. There was variation in the experience of having an understanding. In some instances, an understanding meant having a clear picture, a completed puzzle, a scenario, one’s own version of the content. In other variations, an understanding formed a background which could be used to link new material or allowed a concept to be explained to others off the top of the head. In yet other variations an understanding meant knowing when and how to apply concepts in new situations with competency.


359

Relationship between the ‘what’ and ‘how’ components A significant association was identified between level of understanding and approach to learning about IS (Interviews: n ¼ 15, Kendall’s taub ¼ .634, p ¼ .007; Questionnaires: n ¼ 59, Kendall’s tau-b ¼ .295, p ¼ .015). The student deep approach was associated with the development of levels of understanding higher in the hierarchy (Levels 5 and 6). The student surface learning approach was associated with the lower levels of understanding (Levels 1–4).

Variation in experience of learning about the concept of an IS The investigation of scholarly and undergraduate students’ experiences of learning about the concept of an IS identified a hierarchy of three distinctly different learning experiences (Table 2). The scholarly learning experience is at the top of the hierarchy. An optimal deep learning approach for IS undergraduate students leads to the development of the target understanding appropriate for new graduates. Next in the hierarchy is student experience A, which involved the student deep approach to learning and related understanding of the concept of an IS involving limited aspects of an IS as a social system. At the bottom of the hierarchy is student experience B, which involved the student surface learning approach and an understanding of an IS in which only IT components form the internal horizon of awareness.

Discussion This section of the paper reports an analysis of the educationally critical differences between the three experiences of learning about the concept of an IS. The aspects of the learning experience in which critical differences were identified are then collated to address the aim of the study. The significance of the findings for teaching and learning about the concept of an IS is considered.

Comparison of learning experiences The significance of identifying three distinctly different experiences of learning about the concept of an IS at the undergraduate level is that

Student Experience B

Understanding of an IS as an IT (data manipulation) system of varying complexity and scope

The target understanding of an IS as an organization-wide social system of people, supported by embedded IT, attributing meaning to varied sources of data as a precursor to meaningful decision making Understanding of an IS as IT and people (in a limited way) supporting a number of interrelated organisational functions

Scholarly Experience

Student Experience A

‘What’ component

Experience

Logically significant

Empirically and logically significant

Optimal deep approach – intention to develop a deep understanding through intentionally seeking many varied perspectives on the concept of an IS. Includes perspectives beyond the academic setting

Student deep approach – relating of learning tasks to other learning tasks in the same subject and other computing subjects. Intention to seek meaning in learning tasks as a means of deepening understanding Student surface approach – focus on learning tasks in isolation. Intention to commit content to memory for later reproduction

Empirically and logically significant

Relationship between components

‘How’ component

Table 2. Distinctly different experiences of learning about the concept of an IS

360 CHRIS COPE AND MIKE PROSSER


361

some of the differences may be educationally critical to the development of the target understanding. The three learning experiences were compared in terms of their ‘what’, ‘how’ and relationship between ‘what’ and ‘how’ components. A structure of awareness was used as an analytical framework for the comparison. All differences were determined in the light of the scholarly learning experience. ‘What’ component Multiple aspects of the concept of an IS were identified in which there were educationally critical differences between the various levels of understanding. The detail of these aspects has been focussed on elsewhere (Cope 2000b, 2002). This paper focuses more on the ‘how’ component of the learning experience. ‘How’ component The critical differences between the three qualitatively different approaches to learning about IS identified in the study are summarised in Table 3. The rows in Table 3 represent a hierarchy of approaches to learning about IS. The columns represent aspects of approach to learning about IS in which educationally critical differences were identified. Each cell, comprising an intersection of a row and a column describes the presence or not of the aspect (column) for the particular learning approach (row). The line between two cells in a column represents the location of an educationally critical difference. For instance, in the column for the aspect ‘Acquiring of isolated facts and procedures related to the concept of an IS’, there is an educationally critical difference between the student surface approach and the student deep approach (but not between the student deep and optimal deep learning approaches). An educationally critical difference between the two student learning approaches lies in the process of seeking meaning relevant to the concept of an IS through relating learning tasks to other learning tasks, both in the same subject and in other IS and computing subjects. This process was not an aspect of the student surface approach but is an educationally critical aspect of the student deep approach and of any approach to learning likely to lead to the development of the target understanding of the concept of an IS. The optimal deep learning approach is logically inclusive of the student deep approach but has a broader focus. Both approaches involve an intention to seek and relate the meaning associated with various perspectives on the concept of an IS but the optimal deep approach

No

No Yes

Student deep

Student surface

Acquiring of isolated facts and procedures related to the concept of an IS

No

In learning tasks only

In personal experience as well

Seeking the meaning of an IS through relating different perspectives

Aspects in which there were educationally critical differences

Optimal deep

Learning approach

Table 3. Educationally critical differences between the approaches to learning about IS

No

Limited

Yes

Awareness of own related ‘‘what’’ and ‘‘how’’ components of learning about the concept of an IS

362 CHRIS COPE AND MIKE PROSSER


363

incorporates perspectives beyond the learning tasks presented by teachers, that is, perspectives in personal experiences of information systems. In addition, the optimal deep approach involves a student being aware of their own understanding of the concept of an IS and intentionally relating the meaning inherent in the different perspectives to current understanding. Associated with this awareness of the concept of an IS is an awareness of the learning process and the need to seek varied perspectives on the concept of an IS. The educationally critical nature of the differences between the learning approaches is justified using a phenomenographic perspective on the mechanism through which a deeper level of understanding of a phenomenon is developed. The mechanism, described previously in the paper, involves the dynamic restructuring of awareness in response to different perspectives on the phenomenon of interest (Booth 1997). The student surface approach only considered one perspective on the concept of an IS, the current learning task. The student deep approach involved seeking and relating the meaning in a limited number of different perspectives on the concept of an IS. The optimal deep approach incorporates personal experiences of an IS as a range of additional perspectives which can provide practical relevance to the learning tasks. With a broader range of perspectives it seems logical that a meaningful restructuring of awareness is more likely. Relationship between the ‘what’ and ‘how’ components The experience of learning about the concept of an IS lies in the relationship between the ‘what’ and ‘how’ components. Comparing the three learning experiences in terms of this relationship identified further educationally critical aspects. The educationally critical difference between the relationships lay in an increasing awareness on the part of the student of the ‘what’ and ‘how’ components of the learning experience and, importantly, the relationship between the components. In other words, increasing awareness of the experience of learning about the concept of an IS. Student experience B (Table 2) involved a lack of awareness of the experience of learning about the concept of an IS. Learning about IS involved acquiring isolated pieces of knowledge for reproduction in assessment situations. The notion of developing a deep understanding of the concept of an IS and the learning approaches necessary to achieve this understanding were not considered as part of this learning experience. Student experience A incorporated a limited awareness of developing a deeper understanding of an IS. The process of seeking and relating

364


meaning required effort on behalf of the learner. Students were able to articulate the seeking meaning process and what constituted an understanding of an IS. This implies some awareness of developing an understanding of an IS, in contrast to the surface learning approach where the process and nature of understanding were not considered. The scholarly learning experience, in contrast, incorporates an advanced awareness of the learning experience. This awareness is evident in the description of the optimal deep learning approach – seeking and reflecting on different perspectives on the concept of an IS and different approaches to learning about IS as a means of developing a deeper understanding. Different perspectives are intentionally sought. This implies that a student is aware of limitations with their current understanding and approaches to learning and seeks to address these limitations. How can these limitations be perceived by the student? First, a student needs knowledge about the target level of understanding and the optimal deep approach to learning about IS. Second, a student needs to be aware of their current level of understanding and approach to learning about IS. This was found previously to be an educationally critical aspect of the optimal deep learning approach. Third, a student needs to reflect on the differences between their current learning experiences and their knowledge of the nature of a scholarly learning experience. This reflection needs to occur separately for the ‘what’ and ‘how’ components of the learning experience. An additional educationally critical aspect of the scholarly learning experience has been brought to light. Students need the ability to reflect on the differences between the ‘what’ and ‘how’ components of their current learning experiences and a scholarly learning experience. Without this reflection, perception of limitations in current understanding and approach to learning about IS is unlikely. Without this perception, the educationally critical intention to seek different perspectives on an IS has less motivation.

Summary of the educationally critical aspects of the experience of learning about the concept of an IS The scholarly experience of learning about the concept of an IS featured an optimal deep learning approach and a logically related target understanding for new IS graduates. The logical relationship between these two components of the learning experience means that it is


365

educationally critical that undergraduate students adopt the optimal deep approach. A number of aspects of this learning approach were found to be critical to its effectiveness. These aspects were an intention to seek a deeper understanding; seeking different perspectives on the concept of an IS and learning about IS through learning tasks of varying nature, personal experiences, and current understanding; and seeking and relating the meanings inherent in the different perspectives. Without any one of these educationally critical aspects the learning approach is unlikely to be effective. An awareness, on the part of the student, of their own and the scholarly learning experience is also critical. Only through comparing learning experiences in terms of their ‘what’ and ‘how’ components is a student likely to become aware of limitations in their understanding of an IS and approach to learning about IS. The recognition of limitations instigates the educationally critical intention to seek a deeper understanding of the concept of an IS. In response to this intention students need to seek and relate different perspectives on the various educationally critical aspects of the ‘what’ component of the learning experience – the target level of understanding an IS as a social system supported by embedded IT (described in Cope 2002).

Implications for teaching and learning about the concept of an IS The study reported in this paper identified considerable didactic knowledge about the concept of an IS. The knowledge is in the form of multiple, educationally critical aspects of the experience of learning about the concept of an IS at the undergraduate level. There is evidence to suggest that these educationally critical aspects are unlikely to be part of the learning experiences of current IS undergraduate students. First, these educationally critical aspects have not been reported previously and may be taken for granted by teachers. Secondly, a problem recognised with current IS curricula and textbooks is the lack of emphasis given to the concept of an IS (Weber 1996). To improve the quality of IS students’ learning outcomes the educationally critical aspects identified in this study need to become explicit, thematised parts of undergraduate curricula, textbooks, teaching strategies and learning activities. Importantly, students need to be aware, from the beginning of their studies, of the nature of the target level of understanding of the concept of an IS and how that understanding can be achieved. As a consequence and progressively

366


through the curriculum, learning tasks need to be designed which thematise for students the nature of a scholarly learning experience and the students’ developing understanding of the concept of an IS and the learning approaches being used to develop understanding. Learning tasks need to encourage reflection on the differences between the students’ and the scholarly learning experience. Learning tasks need to provide perspectives for students on the differences in the ‘what’ and ‘how’ component of the learning experience, encouraging reflection on the different perspectives, seeking the meaning in each perspective and interrelating the meanings to deepen understanding. The point of the argument here is that learning tasks need to be designed which make students aware of the experience of learning about the concept of an IS. An example of such a learning task involves students drawing rich pictures of their experiences of learning about information systems. Rich pictures (Checkland 1981) were originally a drawing tool used to describe and understand IS problem domains in organisational contexts. Recently, rich pictures have been used as a graphic organiser in

Figure 2. Student’s rich picture of the experience of learning in an undergraduate IS subject.


367

educational settings (Horan 2002). Figure 2 is a students’ rich picture of the experience of learning in an undergraduate IS subject. The rich picture consists of icons which represent the major aspects of the students’ learning experience. Relationships between aspects are recorded with arrows and the relationships are named. As well as requiring a student to thematise their learning experiences in terms of understanding of the content and approach to learning, rich pictures provide insights for a teacher into the structure of a students’ awareness when they contemplate their learning experiences (see Figure 1). The icons represent the existence and nature of the aspects of the learning experience that are part of the internal horizon of a students’ awareness. The named arrows represent the relationships between the thematic aspects. The picture as a whole provides insights into the external horizon of the students’ awareness, that is the context in which the student views learning as taking place. Even superficial analysis of rich pictures can provide a teacher with valuable didactic knowledge.

Conclusion The research approach illustrated in this paper has extended Entwistle and Smith’s (2002) notions of target and personal understanding to highlight the importance of teachers considering and comparing target and personal learning experiences as a means of identifying didactic knowledge. In extending these notions this paper has been underpinned by and made a contribution to Entwistle’s conceptual model of the teaching–learning process in higher education (Entwistle 2003). The research approach used successfully in this study can be of value to other researchers of student learning and to teachers in a number of ways The structure inherent in the research approach and ensuing description of didactic knowledge provides a framework for justifying on-going quality assurance. The outcomes of the research approach are streamlined in that critical didactic knowledge is identified. The changes needed to address the new didactic knowledge are minimised. Through investigating a scholarly learning experience, current knowledge and research about a phenomenon are revealed to researchers and teachers. Through investigating students’ learning experiences, learning situations are able to be seen through students’ eyes. Research from

368


this perspective over the past 30+ years has provided great insights into student learning and teaching. Through comparing learning experiences, aspects of a phenomenon and learning about the phenomenon which may have been taken for granted and not focused on in teaching become apparent. The hierarchy of levels of understanding of a phenomenon identified as part of the approach to researching didactic knowledge forms a framework for assessing improvements in students’ learning outcomes.

Future research Given the limited context in which the empirical study reported in this paper was conducted, it is likely that only some of the possible variation in the experience of learning about the concept of an IS was identified. Hence there is no guarantee that all of the educationally critical aspects of the learning experience became evident. Investigating the learning experiences of a broad spectrum of IS students of different age groups, at various educational levels, geographical locations and learning under different teaching strategies would be of value. Action research is also required to plan, implement and evaluate learning activities based on the educationally critical aspects that were identified. The research approach used in the study reported in this paper has not been used extensively. The approach needs verification and refinement gained through use in identifying the educationally critical aspects of the experience of learning about other phenomena. Broader use of the research approach and the development of other approaches to researching didactic knowledge are essential to improving students’ learning outcomes and satisfying quality assurance demands on universities.

Acknowledgement We wish to acknowledge Noel Entwistle’s examination of our PhD theses. His insightful and constructive reports provided great impetus for our research careers.


369

References Alter, S. (1999). ‘A general, yet useful theory of information systems’, Communications of the Association for Information Systems 1(13). Online cais.isworld.org/articles/ default.asp?vol=1&art=13 Babinec, T. and Mehta, C. (1999). Thinking about exact statistics. Online www.spss.com/downloads/Papers.cfm?prod_familyID=00005&Name=SPSS Beynon-Davies, P. (1998). Information Systems Development (3rd edition). London: MacMillan Press. Biggs, J.B. (1988). ‘Assessing student approaches to learning’, Australian Psychologist 23, 197–206. Biggs, J.B. (1999). Teaching for Quality Learning at University. Buckingham: SRHE and Open University Press. Booth, S.A. (1992). Learning to program: A phenomenographic perspective. (Go¨teborg Studies in Educational Sciences, 89). Go¨teborg: Acta Universitatis Gothoburgensis. Booth, S.A. (1994). ‘On phenomenography, learning and teaching’, in Ballantyne, R. and Bruce, C. (eds.), Proceedings of Phenomenography: Philosophy and Practice. Brisbane, Queensland: QUT Publications and Printing, pp. 3–6. Booth, S.A. (1997). ‘On phenomenography, learning and teaching’, Higher Education Research and Development 16(2), 135–157. Bowden, J.A. and Marton, F. (1998). The University of Learning: Beyond Quality and Competence in Higher Education. London: Kogan Page. Checkland, P. (1981). Systems Thinking, Systems Practice. Chichester, UK: Wiley. Clarke, R. (1992). Extra-organisational systems: A challenge to the software engineering paradigm. Presented at the IFIP World Congress, Madrid. Online www.anu.edu.au/ people/Roger.Clarke/SOS/PaperExtraOrgSys.html Cope, C.J. (2000a). Educationally Critical Aspects of the Experience of Learning About the Concept of an Information System. Doctoral dissertation, La Trobe University, Australia. Online ironbark.bendigo.latrobe.edu.au/cope/cope-thesis.pdf Cope, C.J. (2000b). ‘Educationally critical aspects of a deep understanding of the concept of an information system’. Presented at the Fourth Australasian Computing Education Conference, Monash University, Australia. Cope, C.J. (2002). ‘Educationally critical aspects of the concept of an information system’, Informing Science Journal 5, 67–78. Cope, C.J. (2003). ‘Educationally Critical characteristics of deep approaches to learning about the concept of an information system’, Journal of IT Education 2, 415–427. Cope, C.J. and Horan, P. (1997). ‘The role played case: An experiential approach to teaching introductory information systems development’, Journal of Information Systems Education 8(2–3), 33–39. Cope C.J., Horan, P. and Garner, M. (1997). ‘Conceptions of an information system and their use in teaching about IS’, Informing Science Journal 1(1), 7–19. Crawford, K., Gordon, S., Nicholas, J. and Prosser, M. (1994). ‘Conceptions of mathematics and how it is learned: The perspectives of students entering university’, Learning and Instruction 4, 331–345. Crawford, K., Gordon, S., Nicholas, J. and Prosser, M. (1998). ‘Qualitatively different experiences of learning mathematics at university’, Learning and Instruction 8, 455– 468.

370


Entwistle, N.J. (1979). ‘Stages, levels, styles or strategies: Dilemmas in the description of thinking’, Educational Review 31, 123–132. Entwistle, N.J. (1997). ‘Introduction: Phenomenography in higher education’, Higher Education Research and Development 16(2), 127–134. Entwistle, N.J. (1998). ‘Improving teaching through research on student learning’, in Forest, J.F.F. (ed.), University Teaching: International Perspectives. New York: Garland Publishing. Entwistle, N.J. (2000). ‘Promoting deep learning through teaching and assessment: conceptual frameworks and educational contexts’. Presented at TLRP Conference, Leicester. Online www.ed.ac.uk/etl/publications.html Entwistle, N.J. (2003). ‘Concepts and conceptual frameworks underpinning the ETL project’, Occasional paper 3, ETL project, School of Education, University of Edinburgh. Online www.ed.ac.uk/etl Entwistle, N.J., Hanley, M. and Ratcliffe, G. (1979). ‘Approaches to learning and levels of understanding’, British Educational Research Journal 5, 99–114. Entwistle, N.J., McCune, V. and Hounsell, J. (2002). ‘Approaches to studying and perceptions of University teaching–learning environments: Concepts, measures and preliminary findings’, Occasional Report 1, ETL Project, School of Education, University of Edinburgh. Online www.ed.ac.uk/etl Entwistle, N.J. and Ramsden, P. (1983). Understanding Student Learning. London: Croom Helm. Entwistle, N.J. and Smith, C. (2002). ‘Personal understanding and target understanding: Mapping influences on the outcomes of learning’, British Journal of Educational Psychology 71, 321–342. Falkenberg, E.D., Hesse, W., Lindgreen, P., Nilsson, B.E., Han Oei, J.L., Rolland, C., Stamper, R.K., van Assche, F.J.M., Verrijn-Stuart, A.A. and Voss, K. (1996). A Framework of Information Systems Concepts. Report of the IFIP WG 8.1 Task Group FRISCO. Godfrey, R.M. (1995). ‘Students as end-users: Participative design of the I/S learning experience’, Journal of Computer Information Systems 36(1), 17–22. Gurwitsch, A. (1964). The Field of Consciousness. Pittsburgh: Duquense University Press. Hazel, E., Prosser, M. and Trigwell, K. (1996). ‘Student learning of biology concepts in different university contexts’, Research and Development in Higher Education 19, 323–326. Hazel, E., Prosser, M. and Trigwell, K. (2002). ‘Variation in learning orchestration in university biology courses’, International Journal of Science Education 24, 737– 751. Horan, P. (2002) A new and flexible graphic organiser for IS learning: The Rich Picture. Proceedings of Informing Science conference, Cork, Ireland. Online ecommerce.lebow.drexel.edu/eli/2002Proceedings/papers/Horan133Newan Johansson, B., Marton, F. and Svensson, L. (1985). ‘An approach to describing learning as change between qualitatively different conceptions’, in Pines, A.L. and West, L.H.T. (eds.), Cognitive Structure and Conceptual Change . New York: Academic Press, pp. 233–258. Land, FF. (1992). ‘The information systems domain’, in Galliers, R. (ed.), Information Systems Research: Issues, Methods and Practical Guidelines. Oxford, UK: Blackwell Scientific Publications, pp. 6–13.


371

Lu, H. (1994). ‘A preliminary study of student responses to different CIS course teaching strategies’, Journal of Computer Information Systems 34(4), 31–36. Lybeck, L., Marton, F., Stro¨mdahl, H. and Tullberg, A. (1988). ‘The phenomenography of the ‘Mole Concept’ in chemistry’, in Ramsden, P. (ed.), Improving Learning. New Perspectives. London: Kogan Page, pp. 81–108. Marton, F. (1981). ‘Phenomenography – describing conceptions of the world around us’, Instructional Science 10, 177–200. Marton, F. (1998). ‘Towards a theory of quality in higher education’, in Dart, B. and Boulton-Lewis, G. (eds.), Teaching and Learning in Higher Education. Camberwell, Vic., Australia: Australian Council for Educational Research, pp. 177–200. Marton, F. and Booth, S. (1996). ‘The learner’s experience of learning’, in Olson, D.R. and Torrance, N. (eds.), The Handbook of Education and Human Development: New Models of Learning, Teaching and Schooling. Oxford: Blackwell, pp. 534–564. Marton, F. and Booth, S. (1997). Learning and Awareness. Mahwah, NJ: Erlbaum. Marton, F. and Sa¨ljo¨, R. (1976). ‘On qualitative differences in learning. I. Outcome and process’, British Journal of Educational Psychology 46, 4–11. McCune, V. and Entwistle, N.J. (2000). ‘The deep approach to learning: Analytic abstraction and idiosyncratic development’, Paper presented at the Innovations in Higher Education Conference, Helsinki. Online www.ed.ac.uk/etl/publications.html Meyer, J.H.F. and Land, R. (2003). ‘Threshold concepts and troublesome knowledge (1): linkages to thinking and practising within the disciplines’ in Rust, C. (ed), Improving Student Learning: Improving Student Learning Theory and Practice\ten years on Oxford: OCSLD, pp. 412–424. Meyer, J.H.F. and Land, R. (2005). ‘Threshold concepts and troublesome knowledge (2): Epistemological considerations and a conceptual framework for teaching and learning’, Higher Education 49(3), 373–388 (this issue). Prosser, M. and Millar, R. (1989). ‘The how and what of learning physics’, European Journal of Psychology in Education 4, 513–528. Ramsden, P. (1997). ‘The context of learning’, in Marton, F. Hounsell, D. and Entwistle, N.J. (eds.), The Experience of Learning (2nd edition) Edinburgh: Scottish Academic Press, pp. 198–216. Ramsden, P. (2003). Learning to Teach in Higher Education (2nd edition). London: RoutledgeFalmer. Svensson, L. (1997). ‘Skill in learning’, in Marton, F. Hounsell, D. and Entwistle, N.J. (eds.), The Experience of Learning (2nd edition). Edinburgh: Scottish Academic Press, pp. 59–71. Thomson, N.S. (1994). ‘Using TQM principles to teach current topics in information systems’, Journal of Information Systems Education 6(2), 65–72. Wagner, W. P. (1997). ‘Teaching information systems concepts using World-Wide Web-Based content’, Journal of Computer Information Systems 38(2), 76–81. Wand, Y. and Weber, R. (1995). ‘On the deep structure of information systems’, Information Systems Journal 5, 203–223. Weber, R. (1996). ‘A neglected aspect of the information systems curriculum: What is an information system?’, in Arnott, D. Dampney, K. and Scollary, A. (eds.), The Australian Debate on Information Systems Curriculum. Proceedings of the Australian

372


Information Systems Curriculum Working Conference. Melbourne: Monash University, Department of Information Systems, pp.71–75. Winter, M.C., Brown, D.H. and Checkland, P.B. (1995). ‘A role for soft systems methodology in information systems development’, European Journal of Information Systems 4, 130–142.