Word Conference on Educational Multimedia and Hypermedia June 20-25, 1998 Freiburg, Germany
Poster Presentation - Session # 459
ASSESSING LOW-ATTAINING PUPILS IN ARITHMETIC WORD PROBLEM SOLVING
Katerina Georgouli T.E.I of Athens, Department of Informatics E-mail:
[email protected]
Maria Samarakou, Maria Gregoriadou University of Athens, Department of Informatics E-mail:
[email protected]
ABSTRACT This paper describes a standing alone computer based assessing system called ASSA (Adaptive System for Student Assessment), able also to be a part of a larger intelligent tutoring system. It is designed to support the assessment of pupils of age 8-12 who show a very low performance in solving simple arithmetic word problems. The main aim of the system is to tackle the lowattaining student’s ability to state what calculation should be used in order to solve a practical problem, involving the operation of addition or subtraction. The system uses any of the models of the operation and in any appropriate context and is able to interpret the answer. The main purpose of the underlying research is to study the difficulty a computer system faces to adapt to the student’s individualities concerning his learning preferences, his real learning strengths but also his current motivational state.
INTRODUCTION One of the challenges for the application of artificial intelligence (AI) to education is the introduction of AI tools in everyday classroom school life. Two questions are included in the concept of didactics, according to the classical theory of Bildung (Klafski 1963), namely what to teach and how to teach. One of the requirements for a successful AI educational tool is that to be adaptive it must have a wide range of instructional strategies (Ohlsson, 1987). ITSs need to represent different teaching strategies so that most suitable one can be applied to a particular student, or in a particular situation. (Ohlsson, 1987). A teaching strategy is responsible for the selection of the appropriate presentation and assessing methods, and for sending them to the interface administrator, where they become concrete interactions with the student. Today’s ITSs have very limited pedagogy and underpinning models of learning and teaching. Most AI researchers just put together different pedagogies in order to take decisions about teaching styles. However there is an increased emphasis on student-centred learning, learning through inquiry, as opposed to a more didactic or drill-and-practice approach that many ITSs favour. If ITSs don’t have a foundation for didactic tutoring they have an even less principled approach to these styles.
They lack the ability to tutor flexibly, to adopt different teaching styles and strategies when appropriate, and to permit students to use different learning styles. Such systems moreover cannot assist the low-attaining students to advance in their learning.Some researchers are still producing ITSs, trying to push into more complex words more complex pedagogies within the ITS framework, really a very tough objective to succeed. In this paper an intelligent assessing system, called ASSA is presented, having the general structure of an ITS. ASSA is designed to assess students of elementary school who show low performance in arithmetic word problem solving. In order to teach low-attainers in mathematics there is the need of the existence of an individualised curriculum, in other words the use of a teaching strategy in accordance with the student’s strengths (Haylock, 1991). A first aim of the system is to be able to adapt its performance to the student’s knowledge state and mainly to his individual characteristics and strengths, using different appropriate dynamically generated assessing strategies. A second aim of the system is to show motivation-based tactics as defined by Malone and Lepper (1987) and Keller (1983) and become a motivational competent tutor as suggested in (Lepper, Woolverton, Mumme, & Gurtner, 1993). In the next paragraphs we will consider ASSA’s knowledge representation and architectural ideas which permit to the system to adapt low-attaining student’s behaviour in order to find the best performance with her. THE DOMAIN ASSA assesses the student’s understanding of a major objective of numeracy in today’s word, namely whether he knows what calculation to choose, even if she has to use a calculator, for the whole range of numerical situations which might encountered in everyday life. The system asks the student to solve simple problems of addition and subtraction, expressed verbally, keeping the quantities of the problem ‘situated’, according to what can be called the objectives-and-assessment approach: specify a sequence of objectives to be attained by the pupils, assess the pupils’ current attainment against these objectives, and then design a learning program to move them forward. (Duncan, 1978; Ainscow and Tweddle, 1979). The objectives-and-assessment approach for mathematical low-attaining pupils can only be endorsed if the target objectives are realistic and relevant. They must be reasonable targets for the pupils concerned to aim for, given the teacher’s judgements about their abilities and intellectual deficiencies, and they must be relevant to the real needs of the pupils in the world in which they live and in which they are growing up (Haylock, 1991),. Greeno and his colleagues have considered a new schema to categorise basic problems of addition and subtraction, based to their semantic relations. According to this they distinguished three main categories of verbal arithmetic problems, problems of change, composition and comparison. Each of these three categories is further subdivided in different types of problems according to which is the quantity in demand and whether the problem is a problem of augmentation or reduction (. (Duncan, 1978; Ainscow and Tweddle, 1979) In ASSA, a similar analysis has been followed, proposed by Haylock (1991), considered as more relevant to the real needs of the low-attainers, where six different categories of contexts (sets of things, money, length and distance, weight, capacity and liquid volume, time) are used. According to the above analysis six models have been
considered for addition and subtraction (aggregation, augmentation, partitioning, reduction, comparison, inverse of addition) and have been identified thirty-four classes of problems, that relate contexts with models in an appropriate way. Following this analysis the system is concerned only with assessing the appropriate operation for a calculation which might arise in the chosen contexts.
CONTEXTS
Aggregation
Sets
Money
Lenght
Weight
Capacity
Time
+
+
+
+
+
+
+
+
+
+
+
_
_
_
_
_
_
_
_
_
_
Augmentation MODELS Partitioning
_
Reduction Comparison
_
_
_
_
_
_
Inverse of Addition
_
_
_
_
_
_
Figure 1: Classes of problems Furthermore the final types of problems derive from the level of difficulty in syntax each model may have (Malone and Lepper, 1987 and Keller, 1983) and from the chosen size of quantities (numbers from 1-10 or 10-100). In ASSA an assessing strategy is responsible for what part of the domain is currently under consideration, which type of problem has to be presenting for solving, the appropriate style and level of difficulty, and how to handle the student’s responses. Wrong responses are surely an inevitable part of the learning process and can be an excellent source of information about the student’s cognitive strengths and weaknesses (Dockkrell, J. & McShane, J. 1993), information which can guide the system to adapt its assessing strategy to the student’s above mentioned characteristic. ASSA’s main aim is to track the student’s performance, to detect his current motivational state and to change dynamically, if appropriate, the assessing strategy after each interaction, adapting it to the student’s real needs.
MOTIVATIONAL ASPECTS Motivation is concerned with an individual’s willingness to persist and contribute effort to the task in which he or she is engaged (Snell, 1992, p.32). In ASSA we have focused on three motivational aspects as proposed by del Soldato et al.(1995), namely effort (or persistence), confidence and independence. Effort refers to how the results of the process of solving a task was achieved, confidence relies mostly on the students’ beliefs on their efficiency to perform the instructional task (Shunk, 1989) and independence relates to the perceived feeling of needing or not needing the tutor’s help to complete the instructional task. ASSA’s ARCHITECTURAL IDEAS and USER MODELLING ASSA has followed in general the structure of MORE, a system designed as a testbed for the feasibility of implementing specific motivational tactics (del Soldato, du Bouley, 1995). In MORE student modelling is separated into two parts, one based on motivational issues and the other based on domain-based knowledge issues. In ASSA a third part is considered based on students’ general aptitudes issues. The three modelling modules have been kept separately for pragmatic reasons. In fact Lepper et al. (1993) make such a separation a “central tenet” for the model they are building of skilled teacher performance. Output from these three modules is reconciled by a third module, the educational planner. The interface module is independent from the rest of the system. It receives orders from the educational planner (the type of problem to be presented, the supply of help and presence or not of a calculator’s simulation on the desktop) and reacts accordingly. The interface offers the teacher the possibility of maintaining the default scenario at the start of each session, changing the attributes of the LUs, the semantic links between them and even extracting LUs from the list. It also offers a friendly environment for problems manipulation. The problems’ database contains all the problems that could be presented during the assessment. A set of standard problems is offered from the system in which the teacher could add his own. The non-standard problems stay in the database for any following assessment, except if they are changed or deleted from the teacher. The problems have a form of standard sentences incorporating the different types in a question, leaving blank boxes for the values of quantities involved. Values of quantities are generated in a random way when the problem is presented. The interface is also responsible for response’s time estimation, provision of help about the use of the system when required by the user and provision of specific help about the solution of the problem when the educational planner decides so If desirable the teacher can provide to the system a first student’s performance rating for the student, different from the default at the beginning of the assessing session. KNOWLEDGE REPRESENTATION IDEAS The architecture of ASSA is based on a knowledge representation conceptually divided into three levels. The lower level contains the domain knowledge, concerning knowledge about the characteristics of the different types of problems, the student’s performance to a specific type, how much he insisted to find the right solution, his overall performance, etc.
The domain-based modeller is responsible to detect the current state of the student’s achievement in problem solving (domain-based student modelling) and react appropriately. ASSA includes a domain-based planner for retrieval of domain knowledge during assessing. It’s a typical domain-based planner which selects actions according to whether the learner knows a topic or has mastered a skill. In all domain representation alternatives, declarative knowledge is partitioned into units. In ASSA declarative domain knowledge consists of Learning Units (LU) which are also part of a network of semantic associations. A Learning Unit is an elementary block of knowledge described with a set of attributes (like type of unit, previous type to be assessed, next type to be assessed, whether the current or next type is obligatory to be assessed etc.). Each LU can be used independently of any assessing strategy and it can take part in more complex LU constructs.
APTITUDE LAYER
CURRICULUM LAYER
EDUCATIONAL KNOWLEDGE LAYER
MOTIVATIONAL KNOWLEDGE LAYER
DOMAIN KNOWLEDGE LAYER
Figure 2: Layers of Knowledge in ASSA In the middle level there are three layers, the ‘curriculum’ layer the ‘educational knowledge’ layer and the ‘motivational knowledge’ layer. The curriculum layer contains knowledge about the current assessing strategy. The assessing strategy is expressed as a compound structure that results from the associations of LUs through their existing semantic links. This structure is stored in a dynamic data list structure called Scenario The adaptability of the system is based on its ability to change the scenario links, when they are characterized as ‘loose’. When assessing starts a first basic scenario is provided to the domain-based planner by the system. As assessing progresses the scenario’s links can change dynamically by the educational planner in order to adapt the assessing strategy to the student’s observed strengths and motivation state. A curriculum modeler, part of the domain based planner, is included to the system in order to carry out these changes. To take account of motivational factors, the twin activities of “detecting the state” and “reacting appropriately” are extending by adding a motivational state and motivational planning to the traditional ITS architecture. The motivational knowledge layer consists of the motivational modeler, an expert module responsible to diagnose the current motivational state of the student (motivational based student modeling) and react in order
to maintain his/her motivation. Motivational planning takes into account other variables than domain-based planning in the student model and widens the educational planner’s space of possible reactions. The educational knowledge layer consists of the Educational planner, an expert module responsible to use the knowledge contained in the units of the rest of layers in order to decide about the appropriateness of the used assessing strategy in each step of the interaction. The decisions are about the selection of the right problem to be presented according to the motivational-based planner the domain-based planner and the knowledge in aptitude layer. However, motivational tactics do not always simply complete the traditional domain-based planning. Sometimes they compete with it as well. In this case a conflicts’ solver planner, part of the Educational planner takes over in order to settle the differences. Decisions are related to the provided help, the permitted number of trials in order to find the right answer, the possible extraction of a specific class of problems from assessing because of the student’s inability to solve them etc. The upper level is called the ‘aptitude’ layer. This layer contains knowledge about the student’s preferences and individual characteristics like learning preferences, general aptitudes like learning strengths, behavioral characteristics etc. One more modeler, the aptitude modeler, is added to traditional ITS architecture responsible to detect the above characteristics and react appropriately (aptitude based student modeling). In all layers all kind of declarative knowledge is represented as slots in a frame’s structure. Slots of the same frame can contain knowledge from different layers. Procedural knowledge is representing in the form of production rules. The new knowledge produced by the rules concerns the aptitude layer and the domain knowledge layer and has as result the chained firing of rules that change the links between the LUs in curriculum layer. CONCLUSION This paper presented ASSA, an intelligent adaptive system for assessing low-attaining students in arithmetic verbal problem solving. ASSA is able to 1. detect the current state of the student’s achievement in problem solving 2. detect aptitude characteristics 3. detect the student’s motivational state, and 4. react with the purpose of adapt its performance to the student’s individual behavioral and motivational characteristics. During assessing the system tackles the objective (the choice the right operation, addition or subtraction) in a systematic way according to the current scenario. The scenario can be changed dynamically after each interaction with the student. At the end of session the different kinds of new knowledge which derive from the firing of rules of different planners are formatted and provided to the teacher as the student’s profile This kind of information is believed to be very useful to the teachers, especially to those who have not enough time or experience to follow a similar assessment procedure. We have already implemented a prototype of our system which is not been yet validated in a classroom environment. First results showed that the system is able to adapt its assessing strategy to the student cognitive strengths and to implement motivational tactics in a satisfactory level. Although the implementation of a domain-based student model was
not of first priority in our work the need of a more efficient student modeling method emerged and the need of the enrichment of motivational planner in rules was obvious too. Next step is to put ASSA in use by expert teachers of elementary school in order to gain more accurate results which will guide to a more efficient system. We believe also that the existence of an embedded pre-assessment test would help the system to design a first more accurate domain-based student model, thus helping the student not to get bored engaged in solving of problems he already knows. REFERENCES 1.
Ainscow, M. and Tweddle, D. A. (1979). Preventing Classroom Failure: an Objectives Approach. John Wiley &Sons.
2.
Brian D. J. & Larkin J. H. (1984). An integrated model of skill in solving elementary word problems. Cognition and Instruction. 1, 245-296.
3.
Carpenter, T.P. & Moser, J.M. (1982). The development of addition and subtraction problem solving skills. Addition and subtraction. A cognitive perspective (p. 2-24). Hillsdale, NJ. Erlbaum.
4.
Duncan, A., (1978) Teaching Mathematics to Slow Learners. Ward Lock Educational.
5.
De Corte, E., & Verschaffel, I. (1995) The effect of semantic structure on first graders’ solutions strategies of elementary addition and subtraction word problems. Journal of Research in Mathematics Education.
6.
De Corte, E., & Verschaffel, I., De Win L. (1985) The influence of rewording verbal problems on children’s problem representations and solutions. Journal of Educational Psychology, 77, pp 460-470.
7.
Del Soldato Teresa, Benedict Du Boulay (1995) Implementation of Motivational Tactics in Tutoring Systems, Journal of Artificial Intelligence in Education 6(4) 337-338.
8.
Haylock Derek (1991) Teaching Mathematics to Low Attainers, Paul Chapman.
9.
Heller, J.I., & Greeno J.G. (1978) Semantic processing of arithmetic word problem solving. Paper presented at the Meeting of the Midwestern Psychological Association, Chicago.
10. Keller, J. M. (1983).Motivational design of instruction. In C.M. Reigeluth (ed.), InstructionalDesign Theories and Models: An overview of their Current Status (pp 386-434). Hillsdate, NJ: Lawrence Erlbaum Associates. 11. Lepper, M.R., Woolverton, M., Mumme, D., & Gurtner, J. (1993) Motivational techniques of expert human tutors: Lessons for the design of computer-based tutors. In S.P. Lajoie & S.J. Derry (eds), Computers as cognitive tools (pp 75-105). Hillsdate, NJ: Erlbaum Associates 12. Major Nigel (1993a) Teachers and Intelligent Tutoring Systems, in Proceedings of the Seventh International PEG Conference 93 Herriot-Watt University, Edinburg. 13. Major Nigel (1995) Modelling Teaching Strategies, Journal of Artificial Intelligence in Education 6(2/3) 117-152. 14. Malone, T., & Lepper, M.R. (1987) Making Learning Fun. In R. Snow & M. Faee (Eds.), Aptitude, Learning and Instruction: Cognitive and affective Process Analyses (pp. 223-253).
Hillsdate, NJ: Lawrence Erlbaum Associates. 15. Ohlsson, S. (1987a) Some principles of intelligent tutoring. In Lawler, R. and Yazdani, M. (Eds) AI and Education: Learning Envirnments and Intelligent Tutoring Systems. Ablex Publishing, Norwood, New Jersey.. 16. Riley M.S., Greeno J.G. & Heller J.I. (1983) Development of children’s problem solving ability in arithmetic. H.P. In Ginsburg (Ed.). The development of mathematical thinking (pp. 153-196) San Diego, CA: Academic Press. 17. Robbins B. (1988) Mathsteps - Assessment & Teaching Strategies for Special Needs in Maths. 18. Sleeman D. H. (1984d) A user-modelling front-end subsystem., in Jrnl Man-Machine Studies, vol 23, no. 1, pp 71-88. 19. Self John (1988) Artificial Intelligence and Human Learning - Intelligent-Aided Instruction. Chapman and Hall Computing. 20. Wenger E.(1987) Artificial Intelligence and Tutoring Systems, Morgan Kaufman.
