1.3.2 Incoming Maturita Exam in the English Language. ...... special course books
and textbooks preparing students for the Maturita Exam in the English.
MASARYK UNIVERSITY BRNO FACULTY OF EDUCATION Department of English Language and Literature
Oral Part of the Maturita Exam Bachelor Thesis
Brno 2008
Thesis Author: Michaela Tvrdoňová
Thesis Supervisor: Mgr. Naděžda Vojtková
Declaration Hereby I declare that I worked on this thesis on my own and used only the sources listed in the bibliography. I agree that the thesis be placed in the library of the Faculty of Education of Masaryk University in Brno and made accessible for study purposes. Brno 18th April 2008
………...…………...................... Michaela Tvrdoňová 2
Acknowledgement I would like to thank Mgr. Naděžda Vojtková for her exceptional helpfulness and precious advice. I would also like to thank my husband and family for all the admirable support.
3
Contents
Contents ..................................................................................................................................... 4 Introduction ............................................................................................................................... 5 1. Theoretical Part ..................................................................................................................... 7 1.1 Definition of the Maturita Exam ............................................................................................... 7 1.2 Basic Structure and Organization ............................................................................................. 8 1.2.1 Present Maturita Exam in General ........................................................................................................ 8 1.2.1 Incoming Maturita Exam in General..................................................................................................... 9
1.3 Maturita Exam in the English Language.................................................................................10 1.3.1 Present Maturita Exam in the English Language ................................................................................ 10 1.3.2 Incoming Maturita Exam in the English Language............................................................................. 10
1.4 Oral Part of the Maturita Exam in the English Language.....................................................12 1.4.1 Present Oral Part of the Maturita Exam in the English Language ...................................................... 12 1.4.2 Incoming Oral Part of the Maturita Exam in the English Language ................................................... 13
1.5 Comparison with the CEFR and UCLES ................................................................................14 1.5.1 The Common European Framework of Reference for Languages (CEFR) ........................................ 14 1.5.2 University of Cambridge Local Examinations Syndicate (UCLES) ................................................... 16
1.6 Assessment ..................................................................................................................................17 1.6.1 Assessment in the Present and Incoming Form of the Oral Part of the Maturita Exam ...................... 18
2. Practical Part ....................................................................................................................... 20 2.1 Maturita Exam at a Private Secondary School .......................................................................20 2.1.1 Preparing Students for the Exam......................................................................................................... 21 2.1.2 Current Form of the Exam .................................................................................................................. 22 2.1.3 Assessment.......................................................................................................................................... 23
2.2 Maturita Exam at a Public Grammar School .........................................................................24 2.2.1 Preparing Students for the Exam......................................................................................................... 25 2.2.2 Current Form of the Exam .................................................................................................................. 26 2.2.3 Assessment.......................................................................................................................................... 27
2.3 Maturita Exam According to CERMAT .................................................................................28 2.3.1 Preparing Students for the Exam......................................................................................................... 28 2.3.2 Form of the Exam ............................................................................................................................... 29 2.3.3 Comparison with PET ......................................................................................................................... 30 2.3.4 Assessment.......................................................................................................................................... 31
2.4 Questionnaires ............................................................................................................................31 2.4.1 Analysis .............................................................................................................................................. 33
2.5 Conclusion of the Practical Part ...............................................................................................34
Conclusion ............................................................................................................................... 37 Works cited ............................................................................................................................... 39 Resumé ..................................................................................................................................... 42 Appendices ............................................................................................................................... 43
4
Introduction
The aim of this thesis is to focus on the differences between the present form and the forthcoming form of the Maturita Exam in the English language, particularly those that are connected with the oral part of the exam. The reason for choosing the oral part is simple; according to the law the present exam is not actually allowed any other form. As a secondary school teacher preparing students for the Maturita Exam I was interested in probing into legal roots of the Czech educational system. I wanted to know more about what the organization of this exam was and what served as the base for all secondary schools of the Czech Republic providing this type of exam. I was keen to know how the oral performance of students should be assessed appropriately, because testing spoken language in general belongs to most problematic areas in testing and none of colleagues could answer my questions connected with the topic. And since schools and their teachers are currently preparing students for the new form of the exam, I also wondered what the actual reason for changing the whole Maturita Exam was. The questions I want to answer ask whether the current form of the Maturita Exam corresponds to the needs of our present students graduating from secondary schools, what the reasons
for
changing
the format of the exam
were,
what makes
the exam
standardized and what its assessment is based on. The beginning of the theoretical part of this thesis deals mainly with the general definition of the Maturita Exam and its oral form in the English language. Following chapters describe the basic structure and organization of the exam and give a detailed account of the main differences between the present form and future changes in the upcoming form. These are to happen in compliance with the new Czech legislation described here as well. Next chapter engages in contextualizing the Maturita Exam within the Common European Framework of Reference for Languages (hereinafter referred to as the CEFR) and offers a comparison between the exam and another language exam. Last chapter of the theoretical part tries to focus on different approaches used for the assessment of oral exams, especially in relation to the assessment of this particular exam. The practical part focuses on actual examples taken from my own teaching experience, together with the experience of my colleagues. I try to look at the present form of the exam from different angles, having chosen two different schools; the expected version 5
of the upcoming exam is also reviewed. I seek to discover both the advantages and disadvantages of the old and new form. However, this part aims to prove that the present form of the exam is not sufficient and suitable, as it does not fulfil current demands for a unified and objective examination in a foreign language. These demands are further dealt with, and opinions on how to improve the situation of the Maturita Exams from the position of a language teacher are offered as well.
6
1. Theoretical Part
The following the definition
chapters
of the term,
of the theoretical
basic
structure
and
part
focus
on
organization
the Maturita Exam, of the exam.
Later,
the thesis moves on to differences between the present and incoming form of the exam, particularly in relation to the oral part. The comparison with the CEFR and University of Cambridge Local Examinations Syndicate (hereinafter referred to as UCLES) is offered as well. The theoretical part ends with assessment in both forms of the exam.
1.1 Definition of the Maturita Exam Students who finish studying at a secondary school in the Czech Republic can take different types of final exams depending on the type of school they study at. Having passed the exam students obtain a relevant certificate proving that they have successfully finished their studies. The Maturita Exam is a prevalent type of exam certifying the climax of secondary education in the Czech Republic. The exam itself is not compulsory. Yet, it is taken by the majority of students since it is an essential prerequisite for admission to universities and other institutions of post-secondary or tertiary education. It is also considered a necessity by many employers, and thus it serves as a sign of social prestige. The Czech word “maturita” comes originally from the Latin word “maturitas”, which in English means “maturity” (“Global Glossary”). Because of the graphic similarity of these three words it is easily recognisable by English native speakers. This fact might have been the possible reason for a growing tendency to use this term of the Maturita Exam instead of trying to translate or adapt the whole term into English.1 It is possible to come across other terms, for instance school-leaving examination or final examination, but this thesis will hereinafter employ the term of the “Maturita Exam” with regard to the more frequent usage of this term. Due
to the meaning
of the word
the exam
is usually
assumed
to be
a kind
of examination that certifies the maturity of an individual. It is often connected with the age of 18 or 19, which is the customary age of students who take the exam. It is also associated 1
See the title of New Headway Talking Points: Speaking Preparation for the Maturita Exam.
7
with other events accompanying this stage of life, such as suffrage or acquiring a driving licence. On the whole, the exam might be taken as a representation of abandoning childhood and entering the world of adults with all its duties and responsibilites. This term can be traced in other European countries where the above mentioned meaning
of the word
remains
the same,
e.g.
in Slovakia,
Poland,
Italy
and
Switzerland. It is not only the term, but even a similar type of such an exam that is to be found throughout Europe. The Czech Maturita Exam can be thus compared to the “Advanced Level” (so-called “A-level”) in the United Kingdom or “Abitur” in Germany. In general the term of the Maturita Exam defines an exam, which is taken by students after 13 years of schooling. It should summarize their overall knowledge of the subjects chosen, the knowledge they have gained during their studies at both primary and secondary school.
1.2 Basic Structure and Organization The Maturita Exam is a state exam, which means that it is managed by the Ministry of Education, Youth and Sports of the Czech Republic (hereinafter referred to as the “Ministry of Education”). The basic structure and organization is common for all the secondary schools providing this type of exam,2 varying furthermore in detail according to the specific type of secondary school and its curriculum.
1.2.1 Present Maturita Exam in General The exam as such presently consists of four individual exams in four different subjects. These four exams form the compulsory part of the whole exam, in addition with the possibility of one extra optional subject (however, it is more of an exception if a student chooses the fifth subject). The choice of the subjects depends on students and on the type of secondary school they study at. There is only one general rule - the necessity to take the Czech language as the key subject. At vocational schools the Maturita Exam comprises the compulsory exam in the Czech language, the compulsory-optional exam, the theoretical exam in specialized subjects and the practical exam in specialized training (Vyhláška č. 442/1991 Sb. §9; par. 1).
2
For more information and differences between Czech secondary schools and American high schools see .
8
At grammar schools the Maturita Exam comprises the compulsory exam in the Czech and a foreign language, with two compulsory-optional subjects. If the field of study is further specialized, e.g. there are students orientated at Mathematics, one more compulsory subject is added - in this case it is Mathematics (ibid §10; par. 1). At specialized secondary schools the Maturita Exam comprises the compulsory exam in the Czech language, the compulsory-optional exam and the theoretical and practical exam in specialized subjects, according to the school’s specialization (ibid §11; par .1). At secondary schools of art the Maturita Exam comprises the compulsory exam in the Czech and a foreign language, the exam in History of Art and the exam in specialized subjects (ibid §12; par. 1). In accordance with the subject chosen there are different forms of the exam to be applied. The exam can either have a written form, oral form, practical form or it can be a combination of any of these. In computer science the practical part is to be expected, while the exam in the Czech language comprises a written part preceding the oral one. However, most subjects tend to have the oral form. Unlike the exam in the Czech language, the current structure for the exam in the English language omits the written part completely, thus – without any distinction – belonging to the majority of subjects which have solely the oral part.
1.2.1 Incoming Maturita Exam in General The incoming form of the Maturita Exam will consist of two parts: the common part and the profile part. The common part of the Maturita Exam will consist of three exams, namely the exam in the Czech language, the exam in a foreign language, and the exam in a compulsory-optional subject (Zákon č. 561/2004 Sb. §78). The profile part will consist of three compulsory exams, and the “head teacher shall specify the selection of subjects of compulsory examinations in compliance with the Framework Educational Programme and he/she shall further set out the form, topics, and dates when such examinations are to be taken“ (Zákon č. 561/2004 Sb. §79; par. 2). The common part represents the part which is common for all schools; the profile part enables schools to incorporate requirements according to their school’s orientation. The whole system is to alter, and the main points dealing with the English langage are discussed in the chapter below.
9
1.3 Maturita Exam in the English Language Many fundamental differences can be found between the present and incoming form of the Maturita Exam in the English language. The exam in this language (or any other foreign language) is going to be changed completely, and the main differences are depicted below.
1.3.1 Present Maturita Exam in the English Language There are two significant notices of the Ministry of Education, where the present form of the Maturita Exam in the English language is embedded, namely Vyhláška ministerstva školství,
mládeže
a tělovýchovy
České
republiky
č.
442/1991
Sb.
o ukončování
studia ve středních školách a učilištích [Notice of the Ministry of Education, Youth and Sports of the Czech Republic No. 442/1991 Coll. on accomplishment of studies in secondary schools and training centres] and its further amendment in Vyhláška č. 672/2004 Sb. [Notice No. 672/2004 Coll]. Not much information can be found on investigating these documents in more detail. This gives schools and their head teachers a possibility to manage the exam mostly themselves. As has been noted, if students choose the English language to be one of the subjecs of their Maturita Exam, they can be certain of the exam having the oral part, which is stated clearly in the law (Vyhláška č. 442/1991 Sb. §9-§12). Nevertheless, this rule has been violated (and is still being violated) in many schools. Teachers usually lack the presence of the written part, considering it indispensable when assessing the overall knowledge of the language. They point out that the written part does exist while assessing the Czech language and they enquire about the reasons for the dissimilarity. The practice of incorporating written part has, however, remained illicit. Only recent changes in the Czech legislation dealing with the Maturita Exam have started to alter the rules, which have been valid in the Czech educational system for many decades.
1.3.2 Incoming Maturita Exam in the English Language The incoming form of the Maturita Exam in the English language is managed by the Centre for the Reform of the Maturita Exam (hereinafter referred to as CERMAT)3 in accordance with the catalogues of requirements published by the Ministry of Education. 3
For more information see .
10
These are Katalog požadavků k maturitní zkoušce: Anglický jazyk [The Catalogue of Requirements for the Maturita Exam: English Language] from 2004 and Katalog požadavků zkoušek společné části maturitní zkoušky: Anglický jazyk [The Catalogue of Requirements for Exams of the Common Part of the Maturita Exam: English Language]4 from 2008. They refer to the latest detailed outline of the new Maturita Exam. The new form will be put into practice in order to achieve a centralized system for all schools in the Czech Republic, a system that could be easily comparable and that would meet the growing demands for a unified and complex final exam in foreign languages. Therefore, in the case of the Maturita Exam in the English language (or any other foreign language), the exam has a centrally set form. There are three parts of the exam: the didactic test (consisting of listening comprehension, reading comprehension and use of English), the written assignment and the oral part (Katalog požadavků zkoušek 17). The didactic test and the written assignment will be executed by all schools at the beginning of April, the oral part will take place in May or June according to the headmaster‘s decision (“Reforma v otázkách”). As is mentioned above, the new Maturita Exam in the English language aims to become a complex exam by examining receptive, productive and interactive skills. Receptive skills include listening and reading, productive skills include writing and speaking, and interactive skills include oral interaction. There are two levels that students can choose from: basic and higher. The risk of choosing the higher might be counterbalanced by an advantage: if they choose the higher level and pass, they will not have to sit the exam in the Eglish language at the university entrance exams (or at least this is the desired state, the question remains if the universities are willing to accept it). This thesis deals with the basic level, which is connected to the B1 level of the CEFR (in terms of content and methodology). The B1 level is later summarized in the following chapter. Important features characterizing the present and future form of the oral part of the Maturita Exam are introduced below. It is necessary to note that the following chapters deal solely with the Maturita Exam in the English language.
4
The newest version of the catalogues can be found at .
11
1.4 Oral Part of the Maturita Exam in the English Language Not only the exam in general, but also the oral part is going to be changed significantly, especially when comparing it with the traditional method of the present form, which was employed in the Czech educational system for many decades. The differences are stated below.
1.4.1 Present Oral Part of the Maturita Exam in the English Language The notices
mentioned
above
state
the following
about
the oral
part
of the Maturita Exam in the English Language. Regardless of the type of secondary school there are 25 to 30 topics. These are specified by the head teacher and students choose one of them 15 minutes prior to the exam. Then the notices describe the duration of preparation for the exam and the exam itself, which is 15 minutes for the preparation and 15 minutes for the exam. The Examination Board (hereinafter referred to as the “Board”) is named as the authority assessing performance of each student and finally deciding on the final assessment. The assessment is decided on quite vaguely, it is not possible to talk about a standardized exam. The oral part with the assessment is not recorded anywhere and it is left completely up to the teachers examining the student. It is also not specified in any way and no further description upon that subject can be found. There are no existing criteria for teachers to follow. That means that the results differ from school to school, according to the strictness of the teachers and the requirements the teachers have. Furthermore, the whole exam is supposed to prove thorough knowledge of the English language. However, the exam quite often has a form of a monologue, which the student has learnt by heart (as is the case with many secondary schools in the Czech Republic). Or the student simply paraphrases learnt facts about various topics. This practice cannot prove the student’s knowledge of the language and the teachers cannot assess the performance according to the complex system of the language. All the above stated facts may lead to conclusions claiming that the exam is not objective, and they might have served as the basis for remodelling the system and creating a new one, which is to come into force in the school year of 2009/2010.
12
1.4.2 Incoming Oral Part of the Maturita Exam in the English Language The requirements for the basic level in the oral part of the new Maturita Exam are divided according to: productive skills and strategies, interactive skills and strategies, language competence, topics, communication situations and language management (Katalog požadavků zkoušek 8-16). Since the CEFR served as the basis for this exam, the previous is described in more detail in the chapter connected with the CEFR itself. The catalogue of requirements gives a detailed description of the form of the oral part. It lasts 15 minutes, with preceding preparation that lasts 15 minutes as well. The preparation can be done with the help of a dictionary. There are four parts of the oral exam, yet the actual timing for each part has not beed stated so far (which should be if we are to talk about a standardized exam). The first part is a motivational introduction and it requires the student to react to questions related to general topics. The second part is an independent oral performance on a general topic using an outline and stimulus/stimuli (e.g. a photograph, a map). The third part is an independent oral performance or interaction with regard to specific/specialized topic, as set by the school. The fourth part is an oral interaction between the student and the examiner within the scope of ordinary communicative situations and general topics (Katalog požadavků zkoušek 18). The student starts the exam by drawing one number and getting the relevant worksheet. These worksheets can be provided by CERMAT or prepared by the school in accordance with the catalogue of requirements. The Board is appointed to assess the performance; so far the model of the Board has not changed. What has changed is the role of the assessor. While the assessor just listens and helps during the present form of the Maturita Exam, during the new form he/she is required to fill in the assessment grid, according to the centrally set criteria (“Reforma v otázkách”). This fact distinguishes the old and the new form considerably. It is evident that the new form of the exam checks the overall language competence of the students (unlike the old form) through a variety of tasks. They cannot learn anything by heart to later present without much effort. The exam consists of many tasks that give the students the possibility to prove their strengths, from short interactions to long turn taking, for all of these are common and important in everyday life. It requires constant attention both from the student and the teacher. Underhill argues that “[this] is a human approach; we want to encourage people to talk to each other as naturally as possible” (4). The new form of the exam demonstrates the student’s thorough knowledge and correct usage of the language 13
at the given level. Thus, it rsembles other international exams in the English language, at this level e.g Preliminary English Test (hereinafter referred to as PET) by UCLES. This exam together with the CEFR is discussed in the following chapter.
1.5 Comparison with the CEFR and UCLES The new form of the Maturita Exam was prepared in accordance with the CEFR. UCLES also acknowledges its alignment to the CEFR and claims that it offers “a valuable frame of reference for its work” (“Cambridge ESOL Exams”). The relationship between Cambridge ESOL (English for Speakers of Other Languages) Examinations and the CEFR is from the conceptual perspective in the aim to make exams in the English language “comprehensive, transparent and coherent” (ibid). This endeavour marks off the present form of the Maturita Exam.
1.5.1 The Common European Framework of Reference for Languages (CEFR) The Common European Framework of Reference for Languages was put together by the Council of Europe. It is “a document which describes in a comprehensive manner i) the competences necessary for communication, ii) the related knowledge and skills and iii) the situations and domains of communication” (“Common European Framework”). Its main aim is to promote “a clear definition of teaching and learning objectives and methods and [provide] the necessary tools for assessment of proficiency” (ibid). The new exemplified
Maturita Exam
in the CEFR.
is set
There
are
according six
to the Common
levels
altogether.
Reference The basic
Levels level
of the Maturita Exam refers to the level of B1, which is named Threshold and characterizes an independent user (together with the Vantage level of B2; in our context this would be the higher level suggested for students of Czech grammar school). A basic user precedes and a proficient user follows. It is important to note that this division does not correspond to the classic division into basic, elementary and advanced (ibid; ch. 3). Looking at B1 at the global scale of the Common Reference Levels, following depiction of the level is given:
14
B1 Can understand the main points of clear standard input on familiar matters regularly encountered in work, school, leisure etc. Can deal with most situations likely to arise whilst travelling in an area where the language is spoken. Can produce simple connected text on topics which are familiar or of personal interest. Can describe experiences and events, dreams, hopes and ambitions and briefly give reasons and explanations for opinions and plans. (ibid; ch. 3.3)
Later a comprehensive analysis of the functions, notions, grammar and vocabulary essential for performing communicative tasks is dealt with in the content coherence part of the Common Reference Levels:
Level B1 reflects the Threshold Level specification for a visitor to a foreign country and is perhaps most categorised by two features. The first feature is the ability to maintain interaction and get across what you want to, in a range of contexts, for example: generally follow the main points of extended discussion around him/her, provided speech is clearly articulated in standard dialect; give or seek personal views and opinions in an informal discussion with friends; express the main point he/she wants to make comprehensibly; exploit a wide range of simple language flexibly to express much of what he or she wants to; maintain a conversation or discussion but may sometimes be difficult to follow when trying to say exactly what he/she would like to; keep going comprehensibly, even though pausing for grammatical and lexical planning and repair is very evident, especially in longer stretches of free production. The second feature is the ability to cope flexibly with problems in everyday life, for example cope with less routine situations on public transport; deal with most situations likely to arise when making travel arrangements through an agent or when actually travelling; enter unprepared into conversations on familiar topics; make a complaint; take some initiatives in an interview/consultation (e.g. to bring up a new subject) but is very dependent on interviewer in the interaction; ask someone to clarify or elaborate what they have just said. (ibid; ch. 3.6)
Most important for the organization and further assessment of the oral part of the New Maturita Exam are undermentioned qualiative aspects of spoken language use of the B1 level, divided into five categories (ibid; ch. 3.4; Table 3): 15
B1 Range: Has enough language to get by, with sufficient vocabulary to express him/herself with some hesitation and circumlocutions on topics such as family, hobbies and interests, work, travel, and current events. Accuracy: Uses reasonably accurately a repertoire of frequently used ‘routines’ and patterns associated with more predictable situations. Fluency: Can keep going comprehensibly, even though pausing for grammatical and lexical planning and repair is very evident, especially in longer stretches of free production. Interaction: Can initiate, maintain and close simple face-to-face conversation on topics that are familiar or of personal interest. Can repeat back part of what someone has said to confirm mutual undestanding. Coherence: Can link a series of shorter, discrete simple elements into a connected, linear sequence of points.
These
requirements
are
essentially
comprehensive
and
their
incorporation
into assessment of the new Maturita Exam is desirable. By way of contrast, there is nothing of this kind to be found in connection with the present form of the Maturita Exam. By implementing a respective form of the exam and constructing a marking system based on the above mentioned aspects, the assessment of the exam will definitely fulfil the demands for the comprehensive,
transparent
and
coherent
exam as
mentioned
at the beginning
of this comparison with the CEFR and UCLES.
1.5.2 University of Cambridge Local Examinations Syndicate (UCLES) Since Cambridge ESOL Examinations, provided by UCLES, are well-known to many teachers and students in the Czech Republic, the Maturita Exam is often compared to these exams. The reasons are obvious. Firstly, PET as one of these ESOL exams is generally known to be at the B1 level, as the basic level of the Maturita Exam. Secondly, it contains all language skills – reading, writing, listening and speaking, “[providing] practical language practice in a variety of everyday work, study and leisure situations” (“Preliminary English Test”). Even the length of the speaking part is similar – 10 to 12 minutes, with the exception that there are two candidates present at the oral part – usually interacting, whereas there is only one student at the oral part of Maturita Exam.
16
Finally, PET is characterized by the relevant “Can Do” statements, which comply with the Common Reference Level of B1. These statements were developed by Association of Language Testers in Europe (ALTE) to describe general language ability in individual CEFR levels. The “Can Do” statements for PET state that the person taking the exam “CAN express opinions on abstract/cultural matters in a limited way or offer advice within a known area, and understand instructions or public announcements” (“CEFR and ‘Can Do’”). PET is by many regarded a suitable exam which should even substitute for the new Maturita Exam. The arguments include the long tradition of UCLES, the recognition of their certificates (they are recognized worldwide by universities, employers etc) and already existing and proved marking systems (for there is none for the Maturita Exam to give to teachers at the moment). These arguments cannot, however, change the current situation and the only hope that can be given to such people is that the Ministry of Education and everybody connected with the new Maturita Exam will do their best to make the exams comparable in the future.
1.6 Assessment Assessment is an important part of the learning and teaching process. Since there are many types of assessment and various interpretations of the term, it is important to note that this thesis deals with assessment “in the sense of the assessment of the proficiency of the language user” (“Common European Framework” ch. 9.1). On defining assessment, it is easy to come across notions that assessment can be problematic. Underhill claims that objective tests with e.g. multiple-choice tests are easier than tests requiring subjective judgement of the person assessing (e.g. spoken tests) and that the problem of a teacher executing the assessment is mainly connected with reliability (88 and 27). There are different people with different assessing techniques, moods, approaches, finally providing different assessment on different occasions. The more people assessing, the more varying the assesment is. Underhill argues that “this poor reliability makes it difficult to be confident that the scores awarded in an oral test are accurate and trustworthy” (88). To develop an oral test with general validity and great reliability, it is necessary to consider the questions in the following validity categories (as quoted in Underhill 105108): 17
1. Face validity: On the face of it, does it look like a reasonable test? Do the people who use the test think it’s a good test? 2. Content validity: Is it relevant? Do the items or tasks in the test match what the test as a whole is supposed to assess? 3. Construct validity: Does the test match the theory behind it? 4. Reliability: Are markers consistent with themselves in the scores they give, or do their standards vary from day to day? Are there big differences in the marks awarded by different markers? Does the test procedure itself seem to produce consistent scores? 5. Concurrent validity: How do learners’ scores on the test compare with their scores on other language tests? 6. Predictive validity: Can the test predict how successful the learners will be at using the language in the future?
These are most practical questions to answer and they should be kept in mind by all the teachers concerned with oral testing, especially those who construct and carry out large exams of the Maturita Exam format. It is possible to find similar concepts connected to assessment as mentioned by the CEFR; these are validity, reliability and feasibility (“Common European Framework” ch. 9.1). Unfortunately this was not considered earlier, leaving thus the inapt present form of the Maturita Exam in practice for so long. The new form is to change accordingly. It is important to note that the new system of assessment will be criterion-based, with the aim to avoid subjective assessment as much as possible.
1.6.1
Assessment
in the Present
and
Incoming
Form
of the Oral
Part
of the Maturita Exam To start with the present form of the Maturita Exam, the assessment is not embedded by any criteria or instruction manual. As was demonstrated above, the assessment is rather subjective, attitudinal and arbitrary. No two marks can be explicitly compared. The new form of the exam has not provided much detailed information upon the subject of the oral part yet, but it is already part of the law, showed in the following quotation. “The Ministry shall lay down in an implementing legal regulation: terms, forms, rules of the course and the manner of evaluation of examinations in the common part of school-leaving examinations” (Zákon č. 561/2004 Sb. §81; par. 10). 18
Weighting will be used for the new form of the exam, the system of 2:1:1 (the didactic test, written assignment, oral part respectively), where the didactic test has two marks due to its
two parts:
listening
comprehension
and
reading
with
use
of English
(“Reforma v otázkách”). The model used for oral part of the exam is as presented by Underhill (3):
SPEAKER/LISTENER=====MESSAGE======LISTENER/SPEAKER || EXAMINER
The model has not changed in this point of view, the difference lies in the other assessor, who does not intervene and records the exam. Especially the recording is something completely new, characterized by its analytical nature according to centrally set criteria. Yet, the examiner is to assess as well, which “is economical, but it does require somebody to carry out two roles at the same time, and this can be difficult to do” (Underhill 2). The examiner’s global assessment is based on the general impression of the student, which is used by UCLES as well. Since there is not any marking system officially published so far, the teachers are left with expectations of what will come. Some claim that the assessment will be identical to the assessment used for the writtten assignment. In 2007 there were training courses for English teachers on how to assess the written assignment, and the criteria did resemble those set by the CEFR or used by UCLES. However, the training course composed of two seminars only, which was not sufficient at all, not even for a proper introduction to the matter. It is expectable that the teachers do not remember much one year later. In comparison with the training of examiners provided by UCLES, where they all have to meet yearly at standardization
sessions,
this training
is insufficient
and
not
guaranteeing
the standardization of the Maturita Exam at all. And since the oral part of this exam will be criterion-based, it should follow the example of Cambridge ESOL Examinations. The training courses for assessing the oral part have not even started. It is advisable to start as soon as possible, because even Underhill suggests that “it may be difficult to arrange thorough training for all the teachers involved - [g]ood teachers do not necessarily make good assessors” (28). And making Czech teachers be proper assessors will take a lot time.
19
2. Practical Part
The first, theoretical part deals mainly with the concept of the Maturita Exam as such, with the differences and similarities between the old and new form and the types of assessment used for these. The practical part deals with three actual examples of the oral part of the Maturita Exam and one speaking test according to UCLES. At the beginning of this practical part I introduce the present Maturita Exam of a private secondary business school in Prague. Then a different form of the exam is offered through the example of a public grammar school located in Prague as well. To get the complete view I considered necessary to include the potential new form of the exam, as described by CERMAT (the organization attending to the Maturita Exam, managed by the Ministry of Education). Next chapter of the practical part presents a short research. The research was undertaken with the help of questionnaires; these questionnaires were given to people who had already passed the Maturita Exam and thus were able to compare and analyze. The main aim was to show how they evaluate the present form of the exam with its preparation, organization and outcome. Last questions of the questionnaire also include certain inquiries into the people’s knowledge of the new version of the exam and ask for their opinion. The conclusion of the practical part sums up the overall information shown in the previous parts. It contains my personal critical view of both forms of the exam and it endeavours to suggest the best way for preparing secondary-school students for the exam, which is based on this information, mentioned requirements from the CEFR and CERMAT, then on given questionnaires and finally on my practical experience.
2.1 Maturita Exam at a Private Secondary School The first example of the Maturita Exam in the English language is taken from a private secondary business school called PB-Vyšší odborná škola a Střední škola managementu,5 s.r.o. (hereinafter referred to as “PB-VOŠ”, which is the abbreviation used by the school
5
See .
20
itself, even though it does not include the title of the secondary school in it). The school is located in Prague (to be more exact in Prague 8 - Kobylisy, Nad Rokoskou 111/7) and it has been preparing students for the Maturita Exam and the Absolutorium Exam6 for more than sixteen years. I have chosen this school because this is the place where I have gathered all my knowledge concerning the Maturita Exams, since I have been employed there as a full-time English teacher for nearly four years. As was mentioned above, it is a “business school”, which might be an inappropriate translation into English provided by Czechs, but it is frequently used by various institutions and that is why it is used in this thesis as well. This type of school is sometimes also referred to as a “business academy”, “academy of commerce” or simply a “trade school”. Since it is a smaller private school, it has only one class of students sitting the Maturita Exam, i.e. each year there are approximately 34 students taking it. Not all of them choose the English language as one of the compulsory-optional subjects for the exam, but a vast majority does. This might be due to the fact that the choice is limited. Out of the four compulsory subjects (as stated by the law, mentioned in the theoretical part) the Czech language is compulsory for all of them. Further Economics and Accounting are compulsory (as given by our school; being a specialized school it has these two subjects to represent the most important areas that the school specializes in and differs from other secondary schools7). Thus it leaves students with the possibility of only one compulsoryoptional subject, which can be Mathematics, the English language and Information Technology. Leaving out Mathematics, which is generally considered unpopular with students, it is usually English that is chosen by most of them. And that is the reason why students can be prepared for the exam as a whole class and not only as a few individuals.
2.1.1 Preparing Students for the Exam Our students are prepared for the exam in a similar way but each time with a different attitude depending on the teacher (which is only natural since no teacher can be identical to any other). We have tried hard to unify the system of the Maturita Exam and integrate all subjects into it (or at least most of them), because this meets the requirements of the Ministry of Education and the future Maturita Exams. 6
An examination certifying a type of accomplished degree of tertiary education in the Czech Republic. Lower degree than the Bachelor Exam. 7 Other specialized schools choose other subjects to be compulsory for their students, e.g. Psychology in secondary medical schools etc.
21
The subject of the English language is a compulsory subject for all students. There are four lessons of English a week in a ten-month school year. There is one optional extra lesson in the fourth year in case the students choose English Conversation out of three other subjects. The primary material used in our classes is New Headway English Course, published by Oxford University Press (it is considered a common course book for secondary schools). We usually start from the Elementary level and finish with the Pre-Intermediate level, rarely with the Intermediate one. This course book is the essential core of the preparation itself, leaving the students somewhere around the B1 level according to the CEFR. Throughout their studies, but especially during the last two years, students are motivated to work independently at home too. They are encouraged to read original English texts and books and they work out different projects in English (eTwinning8 etc). There are also other materials used, e.g. grammar exercises, listening and speaking activities from special course books and textbooks preparing students for the Maturita Exam in the English language (Angličtina – Maturitní témata [English – Topics for the Maturita Exam]). Students attend theatre perfomances in English,9 visit cinemas to watch different films in their original English versions. PB-VOŠ participates in various competitions in the English language (e.g. Olympiáda v anglickém jazyce [The Olympics in the English Language], Literární soutěž SPUSA [SPUSA Writing Contest]10) and organizes excursions to Great Britain. Computers with the Internet also serve as a very important part of the preparation. Another significant area is Business English terminology, which is taught via materials provided by the teachers and acquired from different sources (newspaper articles, TV news etc). It expects students to incorporate their knowledge from other subjects into English and it is later examined during the Maturita Exam, as each topic is to be somehow connected to business, and thus show the linking of subjects taught at our school.
2.1.2 Current Form of the Exam The similarity between PB-VOŠ and other secondary schools is in the number of topics (twenty-five) and the areas they cover (e.g. sports and games, education in English speaking countries and the Czech Republic, food etc). The difference lies in the form of the exam. Unlike other schools there is no written part, neither a preliminary grammar test, nor an essay. The exam is oral only,
8
See . The Bear Educational Theatre, see . 10 See .
9
22
and the important thing is that we do not require a fifteen-minute monologue on a chosen topic (which is unfortunately the case with many of the secondary schools). A student chooses one out of the twenty-five topics, which reflects requirements according to Vyhláška č. 442/1991 Sb. [Notice No. 442/1991 Coll.], and he/she11 gets two sheets of paper including various tasks.12 He gets the fifteen-minute preparation time as required by the law. Then he sits in front of the Board and the fifteen-minute oral examination starts. At first he introduces himself and answers general questions that the Board asks him. Then he focuses on the paper he got and prepared during the preparation time. The first side includes a text from the area of the topic that he has chosen. He is supposed to read and translate a certain part of the text (there is an English-Czech/Czech-English dictionary available at the preparation desk). Below the text there are questions which can be divided into two categories. The first category includes questions checking the reading comprehension of the text (reading for gist), while the second category asks about general issues, plus it quite often asks for the student’s opinion (e.g. “What currency is used in the Czech Republic and abroad? Do you like shopping?”). The second side of the paper concentrates on grammar. There is one to three grammatical phenomena covered and each is to be presented through one or two grammar exercises (e.g. an exercise which wants students to choose “some” or “any” for different sentences according to the rules acquired and practised in the lessons). The grammar part never demands an explanation of the grammatical rule from the side of the students, just a passive usage of the phenomena. The exam finishes with eventual questions from the Board.
2.1.3 Assessment Assessment varies according to the teacher. Since there is a wide range of activities to be done in the fifteen-minute examination time, some teachers do not hesitate to help the student if he seems weak in any of the parts. In such a case the teacher simply moves on to a different part, a part the students is better at (the examining teacher is at the same time the student’s English teacher, so s/he should know the student and his strengths and weaknesses). Unfortunately some teachers tend to overlook these weaknesses and either in the name of fair play or revenge they do not give the student any possibility to show his strengths. 11 12
Due to simplification, both sexes are hereinafter referred to as “he”. For further illustration see the worksheets in appendix 1.
23
Therefore this makes the assessment biased and subjective, because teachers can choose different attitudes to different students, based on arbitrariness. There are also no objective criteria that could be used for the assessment (as mentioned in the theoretical part). The exam is not standardized in any way.
2.2 Maturita Exam at a Public Grammar School The second example used for the description and comparison is the Maturita Exam at a public grammar school, to be more specific at Gymnázium Na Pražačce in Prague 3,13 Nad Ohradou 2825/23 (hereinafter referred to as “Gymnázium”). I chose this school to illustrate
the difference
between
PB-VOŠ
as
a private
specialized
school
and
this Gymnázium as a public grammar school, because the Maturita Exam in any subject is supposed to predicate an equal level of acquired knowledge and it should be comparable within any school in the Czech Republic; even though it is not, as is shown hereafter. Before I start with further description, I would like to mention one interesting fact. When I was in the process of choosing a suitable school to contrast with PB-VOŠ, I encountered some problems in the area of making teachers from other schools talk about the Maturita Exam at their schools. I got the impression that they are in a way abashed at revealing the actual truth about the manner the exam is executed in (getting materials and information was made possible only due to the kindness of a colleague of mine, who was willing to help me; I do not suppose I would get any help from a teacher who did not know me). This does not correspond to the general idea that the exam should look alike and be transparent. And it is a proof that can be used in favour of the new form of the exam. This school differs from PB-VOŠ in many featues, e.g. as was mentioned the founder is the state, not a private enterpreneur, there is a much higher number of students, nevertheless there is one important general aspect: it is a type of comprehensive school which covers a wide range of subjects. On the other hand PB-VOŠ specializes mainly in subjects connected to business, thus leaving other subjects out (e.g. Chemistry, Biology etc). The English language at Gymnázium is also a compulsory subject. Talking about the Maturita Exam, apart from the Czech language there is another compulsory subject, which can be any foreign language that is taught at this school and fulfiling requirements by the Ministry of Education (e.g. number of years taught, allocation). Hence there are 13
See .
24
two subjects left for the students to choose and this time they are given a wider number of compulsory-optional subjects – about ten (unlike PB-VOŠ, which offers only three). The English language is one of the most chosen subjects for the exam and this fact shows the general
fondness
of students
in learning
this particular
foreign
language.
And
again it leaves space for teachers to prepare all the students in the class for the exam, and not just a few of them.
2.2.1 Preparing Students for the Exam Due to the higher number of students preparing for the Maturita Exam there are more English teachers preparing them, i.e. the variety in different methods and approaches is larger. Yet the tendency to keep the standard universal can be noticed even at this school (which should not be anything extraordinary but a commonplace; unfortunately not always, as I have experienced myself many times). The allocation is similar to PB-VOŠ with three to four lessons of English a week. There is also the possibility to choose English Conversation as an optional subject, even though not only for one year, but for two, and with the allocation of two extra lessons, not just one. Altogether this shows more lessons of English than at PB-VOŠ. Owing to this, and the fact that students with generally better marks are accepted at this type of comprehensive public schools, the final level of English is much higher than at PB-VOŠ. Comparing it with the CEFR the level can be from B2 to C1, which agrees with the suggested level of CERMAT for the new Maturita Exam (according to CERMAT it is B2 for grammar schools). When comparing with UCLES, it is the level of First Certficate of English (hereinafter referred to as FCE), which is actually the exam many students choose to sit in the last year of their studies, together with their Maturita Exam. Higher level of English is reflected in different levels of the course books used by Gymnázium. These course books include New Headway English Course (the same course book used at PB-VOŠ), Blueprint and New Opportunities; when comparing with PB-VOŠ they are all quite similar in format and content, the only difference being in the language level. While PB-VOŠ finishes with Pre-Intermediate level, Gymnázium reaches UpperIntermediate level and sometimes even Advanced. Further preparation includes similar materials like the ones at PB-VOŠ, i.e. extra texts and miscellaneous exercises to practise all language skills and functions, special textbooks with the Maturita topics and independent work at home. Students visit cinemas, participate in competitions, take part in excursions to Europe and work on computers. There is only thing 25
lacking in comparison with PB-VOŠ and that is no Business English or any other English for Special Purposes. To sum up, the form of the preparation is rather alike, but there are greater demands on students in the area of home self-study and preparation, more time devoted to the subject itself and much higher level of the acquired language. Even stricter discipline plays a big part.
2.2.2 Current Form of the Exam To start with the similarities in the form of the exam, it is possible to mention that the exam is oral only as well, the number of topics is the same, i.e. twenty-five, and the areas
they
cover
are
also nearly
the same
(so are
the ones
suggested
by
CERMAT: e.g. my family, housing etc). The exam has the same rules according to the law, i.e. the student chooses one of the topics and has fifteen minutes for preparation with a dictionary and fifteen minutes for the oral exam. The introductory part, where he introduces himself and the topic, are the same too. The dissimilarity starts with the worksheet he gets on having chosen one of the topics.14 The worksheet is covered on one side only and it is divided into two parts, part A and B (the student can choose which part he wants to start with). The first part contains a text from the area of the topic. The student is supposed to read the text during the fifteen-minute preparation to get acquainted with the topic and to answer a few questions, which ask for explanation of some words used in the text (e.g. “Explain the term ‘consumer society’.”) and require the student’s reading comprehension. The student does not read the text out loud. The second part, part B is said to take about 12 minutes and it contains three to five questions inquiring into the student’s knowledge of the given topic and asking him for his personal opinion on the matter (e.g. “How has shopping in the Czech Republic changed in past 10 years? What do you like wearing on different occassions?”). At the end the Board sometimes asks supplementary questions too. Generally the student spends most of the time talking, with the help of the layout of the worksheet. He may be interrupted if necessary, but on the other hand he is hardly ever helped. If this is the case with a weaker student and he does not know how to start or continue during the exam, he is helped only by repeating the question from the worksheet.
14
For further illustration see the worksheet in appendix 2.
26
2.2.3 Assessment Assessment differs greatly in the sphere of strictness. Where our students are helped, students of Gymnázium are left to cope with the situation. The standard is generally higher, which is due to the type of the school, and so are the demands. The various level examined also means that if a student gets an excellent mark at PB-VOŠ, it will definitely not correspond to a student that also gets an excellent mark at Gymnázium. Another difference can be found in no stand-alone grammar part, i.e. the grammar is assessed throughout the student‘s performance. Thus a student of PB-VOŠ, who has learnt irregular verbs by heart and subsequently manages the grammar exercise checking this phenomenon, will get an excellent mark when assessing the grammar part of his performance; no continuous monitoring for overall grammar accuracy. A student of Gymnázium, who is checked continuously in the whole area of grammar, will get a worse mark, even though he knows the irregular verbs by heart as well and uses them in his speech correctly. The similarities are unfortunately the grave ones. There is no system for the teachers to follow while assessing, so the assessment is done on the principle of arbitrariness again. It depends on the teacher, his methods of assessing, experience, his relationship to the student and other factors, many of which should be irrelevant in the process of assessment. There is nobody to record what has been said and done during the examination, so if the teacher decides to give the student a worse mark and he can convince the Board about it, there will be no proof for the student that he could use further on while defending his performance and pleading for a better mark. The greatest problem common for both schools is the exam not being complex at all. It does not consider the complexity of language as a whole; it simply takes out only a small piece out of a larger unit. It does not allow students to show what they are good at; some are better at productive skills, some shine at the receptive. Even if the student is good at speaking, he still might lack the confidence in speaking out in front of the Board, especially if this is done without any interaction from the side of the teacher. Or he might not know much about the topic, while being excellent at others. This fact influences the whole result, not taking into consideration the possibility of the student performing better with another topic or with another skill.
27
2.3 Maturita Exam According to CERMAT The upcoming form of the Maturita Exam in the English language is managed by CERMAT in accordance with the catalogues of requirements published by the Ministry of Education (already mentioned in the theoretical part). Since most of the information is covered in the theoretical part and since the form of the exam is expected to be universal for all schools, there is no need for lengthy descriptions. I will just briefly summarize the most important pillars and later focus just on the example of the oral part of the exam for the basic level. The basic level corresponds with the B1 level of the CEFR, and with PET exam in UCLES. The new Maturita Exam in the English language is a complex exam examining receptive, productive and interactive skills. There are two levels that a student can choose from: basic and higher. Each level has three parts: a didactic test, a written assignment and an oral exam. The oral part is assigned, administrated and assessed by individual schools; however, in compliance with the centrally set methodology.
2.3.1 Preparing Students for the Exam It might seem that there is no major change in the field of preparation. The number of lessons taught in English does not change. The course books remain the same at most schools, with a few new ones appearing on the market (e.g. Maturita Solutions by Oxford University Press). The emphasis is placed on acquainting students with the form of the whole exam, the content of its individual parts and typology of exercises and taks used. In the catalogues of requirements there are set skills and strategies that can be followed. Another common feature joining the old and new form is the existence of the topics, but here it is not just choosing one and dealing with it only. One worksheet covers four topics (e.g. school, travelling, Prague and free time), each part covering one topic. The school has the right to change one part according to the school’s specialization. Worksheets can be acquired from CERMAT or the school can work out its own ones, especially if it considers necessary to include one specific topic characteristic for the school orientation (e.g. PB-VOŠ would probably like to include some of the Business English), nevertheless it has to keep the centrally set rules and methodology.
28
What was added were communication areas (e.g. public, educational, work area etc), which run through the language skills and the topics. These may be helpful during the preparation. What is important to stress is that the Maturita Exam in a foreign language will be compulsory for everyone, not just one of the compulsory-optional subjects. Even though teachers tend to prepare all students for the Maturita Exam even nowadays, this time it will not be voluntary but a necessity.
2.3.2 Form of the Exam The oral part lasts 15 minutes, which is the same for the old form. The student draws one number and gets a worksheet. He has 15 minutes for preparation with a dictionary. He can write down notes that can be used during the examination (so far the form is the same). The exam consists of four parts.15 I would like to introduce the worksheets, which were part of the catalogue of requirements.16 The first part is common for the old form, even though it is not always present at all schools. It is an introduction, where the student can expect general questions from the examiner. The second part consists of three subtasks. The first wants the student to choose one picture out of two and describe it; there are some points that can help him to speak (e.g. “What kind of place is it? What can you see? What do the people look like?). The second subtask demands comparison of the two pictures, again giving some hints (e.g. place, people, season, atmosphere, your preferences). The third subtask wants the students to talk about his experience from his best holiday. The third part consists of two subtasks. The first one resembles the monologue present at many schools nowadays. The student is to speak on his own about Prague. Hints are present to help him (e.g. location, parts, people, places of interest, transport) and he should use a map to illustrate his speech. In the second subtask the examiner talks together with the student about an imaginary situation [e.g. “Your English speaking friend is showing you some photographs from his/her stay in Prague (pictures 3A-3D). He/She would like you to help him/her identify the places in the photographs.”]. The examiner starts the conversation. The fourth, last part is interactional as the second subtask of the third part. An imaginary situation is suggested, where the examiner and the student are two friends 15 16
For further illustration see the worksheets in appendix 3. The latest catalogue published in 2008.
29
planning a day out. According to some pictures of places to visit or things to do they should decide together which three places they are going to go to. The examiner starts the conversation again. This is the end of the oral exam. Timing for the individual parts has not been published yet, but I believe it will be, as it is necessary for any standardized exam.
2.3.3 Comparison with PET It is evident that the exam is different from the present form. It resembles the form of the Cambridge PET exam in many ways. When comparing the two together with the oral examiner’s worksheets in the PET exam,17 we can recognize many similarities. There are four parts. The introductory part is present at both, but with the PET exam giving numbers and spelling is always present. Describing pictures remains, but with PET one does not have the chance of choosing one photograph out of two, he gets one from the examiner. There are also two imaginary situations, where the people are supposed to interact (part two and four). What I believe will be different is interruption from the side of the examiner. While Czech teachers tend to interrupt, correct mistakes and lead the conversation elsewhere (quite often inhibiting the student from speaking), the examiner at PET prompts only when necessary (usually not at all), since this is required by UCLES, as it secures the standardization of the exam – to provide identical conditions for all candidates. With PET exam it takes 12 minutes, with the difference that there are two candidates talking to each other and the examiner fulfils just the role of an examiner. On the other hand with the Maturita Exam there is just the student and the examiner, so the examiner fulfils the role of listener/speaker and the examiner. I consider the PET model more convenient, both from the point of time (more people examined in less time) and from the side of the situation, where the people examined interact between each other, i.e. two people of more or less the same level (that might serve as a factor limiting stress). The examiner also has time to listen and pay proper attention to what is being said (unlike the old and new Maturita Exam). The role of the assessor stays the same; he is present at the exam without interrupting it and putting down the evaluative judgement (with the old form of the Maturita Exam he is there just to listen, help, ask additional questions and discuss the final mark with the examiner). Generally I see the oral part of the Maturita Exam as needlessly complicated, with two many subtasks. Due to that I expect problems arising with keeping the time limit. But 17
See appendix 4 for the sample papers. The sample papers were acquired in free resources at .
30
on the other hand when comparing it with the old form, it is definitely a better exam, being complex and leaving the student to show his overall knowledge of the language in many areas, not just within one topic (where if he does not know it, he fails).
2.3.4 Assessment The oral part has a 25% share in the final mark a student gets from the subject. It is assessed by the school but in accordance with the centrally set criteria. As was mentioned the assessor writes down what has been said and done, with all eventual mistakes. According to this record, plus the examiner’s suggestion, the final mark is worked out. This makes the assessment more transparent and impartial. If a problem arises with the given mark, this record can be shown to demonstrate and retrace the course of the exam. It is possible to judge even years later what the student demonstrated and deduce the mark again. The problem is with the fact that there are no criteria available yet. There are no training
seminars
for
teachers
to participate
in.
The radical
change
in the format of the exam definitely requires extensive training of all English teachers, because nothing is more prone to subjective assessment than the oral exam. Teachers are not used to assessing in accordance with some methodology manual, neither are they aware of the right way of recording the oral exam on a paper. In general the assessment is based on fairer and more universal principles than now. The final mark could be easily comparable throughout the Czech Republic, but also according to the CEFR and with other language exams. I would suggest considering the possibility of two students being examined at the same time. Not only would it save time, it would also leave the examiner with more space for observing the examination.
2.4 Questionnaires I
decided
to use
questionnaires
to find
out
opinions
on
the old
form
of the Maturita Exam. The questionnaires were given to people who had already taken the exam. I was curious about what they thought about the form, the result and the recognition of the exam, wondering if their answers would agree with the protests against the old form of the exam.
31
There were twenty questionnaires18 distributed in 2008 to twenty people, mostly students of Vyšší odborná škola [Higher Proffesional School] at PB-VOŠ. Most of these students took their Maturita Exam in 2006 or 2007; some of them took the exam before 2000. Some of the respondents are our students, studying at the secondary business school. The questionnaires were anonymous, nevertheless some felt the need to sign them. I was pleased with their unbiased attitude and willigness to cooperate. There were twenty questions, but I chose just eight that I consider important for this thesis and the answers are included in the table of frequency below. The left out questions cover e.g. the type of the school where the students took the exam (16 people studied at secondary specialized schools), the place (17 of them in Prague), the year when they started studying English (16 of them were younger than 15 years old), the course books used at primary and secondary schools (14 of them used Headway at some point of their learning process) and the fact if they were satisfied with the result (16 of them were satisfied with the result). It is important to note that if the numbers in each category do not equal the total number of respondents (i.e. twenty), it is because some of them did not fill in their answers or wrote “I don’t know”. A Table of Frequency YES
NO
12
6
Were you satisfied with the preparation for the exam?
13
6
Were you satisfied with the form of the exam?
18
2
Did you attend any special courses preparing you for the exam?
5
15
Do you think that these courses are necessary?
7
5
Do you think that the exam prepared you sufficiently for the future
4
13
20
0
Did the mark you got actually correspond to your level of English at that time?
(i.e. using English at university entrance exams, future occupation, and personal use)? Should
there
be
a compulsory
Maturita Exam
in a foreign
language? Should this foreign language be English? 18
5
The form of the questionnaire is included in appendix 5.
32
-
Some students also mentioned that outside-school courses were not necessary but useful, and the rest thought it depended on indivual people. For some of them the applicability of English in their future lives was enough as far as university entrance exams were concerned, but not for their future occupation. There
were
also answers
to other
questions
that I
will
try
to summarize
in the following paragraph. 2 people mentioned that they had not liked stress that had been connected with the exam, 1 person wrote he had not liked the Board. Another person wrote that he had appreciated the presence of his English teacher, the contact with the teacher that he had known. There were 5 people who had also had a written part of the exam, not just an oral one. 4 people wrote that they had not liked the fact they had had to learn the topics by heart, 4 would have preferred more conversation in the exam, and 1 student would have liked the exam to have a similar format like FCE by UCLES. All respondents do not generally know much about the new Maturita Exam, they are happy to have passed theirs. However, 3 wrote that they considered the new form difficult, and 1 thought it easier.
2.4.1 Analysis If I look at the results of the questionnaires, there are some interesting conclusions to be drawn. 18 people were satisfied with their results, but evidently not fully, because there were more than 2 people wanting to include more conversation into the exam and offering other suggestions. Only 4 of them considered the exam sufficient for their future use of English, which is striking when comparing it to the other result of 16 of them being satisfied with the result. This might show that the students were happy to get a good mark, but generally consider the achieved level of English insufficient in comparison with their future needs. All of them believe that the Maturita Exam in a foreign language should be compulsory, which is surprising for me, since many of my current students are afraid of taking the Maturita Exam in English and say that it should not be compulsory. It is also interesting that the answer of the respondents was unanimous with all of them. Some people mentioned the stress, which I consider important as well; I am sure it can influence many people’s performance. By giving them the chance to prove their skills and abilities in more areas than in just the oral part (which is the case of the new Maturita Exam with the added didactic test and a written essay), they can achieve a better mark and relieve the stress from the oral part only. 33
Some respondents did not consider the preparation sufficient, which should also be kept in mind when preparing the students for their exam. The teacher should ask his students for feedback, to see if there is anything they miss, anything he could improve. To sum up, it is shocking (but expected) to have found out that the respondents mostly do not regard the exam sufficient for their future use. This agrees with the protests again the old form of the Maturita Exam and the call for the reform. The new form should be of more use to the students, be more practical and more impartial in the assessment. There should be no “washback effect”19, when students learn the language just in order to pass the exam, not to actually learn something they could use in the future.
2.5 Conclusion of the Practical Part At the beginning of the practical part there are two different types of the current Maturita Exam presented. These two schools share certain common features, but they also differ in certain aspects. Most importantly, it is necessary to realize that one school is private and the other is public. The difference also lies in the former being a specialized business school and the latter being a general grammar school. This is naturally reflected in the preparation and requirements for the exam, where the private business school uses specialized Business English vocabulary and lower level of General English (usually Pre-Intermediate) and the grammar school does not include any specialized vocabulary, but demands higher level of General English (Upper-Intermediate). This is to remain the same in the case of the new form of the exam, where students of specialized and vocational schools should choose the lower B1 level, while grammar school students should choose the higher B2 level. Generally the current preparation for the exam (as carried out throughout the four years of studying at a secondary school) is rather similar. Both schools use similar course books and similar methods for the preparation; the allocation of English lessons a week is nearly the same. Even the organization of the exam is alike. There are twenty-five topics for the student to choose from and then it slightly varies within the exam itself. The public grammar school does not require any reading or translation and the correct usage of grammar is shown throughout the student’s oral performance, not on an individual exercise. 19
Described as the impact testing has on teaching and learning (Brown).
34
The preparation according to CERMAT is not totally distinct from the present form. The course books more or less remain the same, so does the allocation of the lessons. Nevertheless more emphasis is put on the division into listening, reading, use of English, writing and speaking, as they are all integral parts of the exam. Each of these parts then employs a variety of exercises, methodology of which is unified in Katalog požadavků k maturitní zkoušce: Anglický jazyk [The Catalogue of Requirements for the Maturita Exam: English Language] from 2004 and the latest Katalog požadavků zkoušek společné části maturitní zkoušky: Anglický jazyk [The Catalogue of Requirements for Exams of the Common Part of the Maturita Exam: English Language] from 2008. Furthermore, each of these parts is assessed and has its own fixed share in the overall mark. Due to the complexity of the exam there is a wider possibility that the student will be assessed correspondingly to his actual knowledge of the languge (unlike the present form, where a student can get an excellent mark for a fifteen-minute monologue on the topic “My family”; a topic he prepared beforehand and learnt by heart, and otherwise is not capable of e.g. making himself understood in an English speaking country). The present exam itself does not in any way cover all the required areas of the language in comparison not only with the CEFR, but also with e.g. UCLES. It neither meets the demands for an objective assessment. The unification of the requirements, preparation, form, organization and final transparent assessment of students make the upcoming Maturita Exam by far a better exam than the current one. It can be easily compared on a large scale (i.e. results from all secondary schools can be comparable) and it should show the actual level of English of each student (again like with Cambridge Exams, plus according to the CEFR), because it runs through all the student’s skills. The stress of the exam for the student stays; there are both advantages and disadvantages connected with the new form. The advantage has already been mentioned, and it says that by dividing the exam into more parts it can divide the stress into smaller parts, so that the student does not have to be afraid of having just one part in front of him, which he can pass, but also fail. With more parts he can rely on showing his strengths somewhere else, e.g. during the listening test or the written essay. The disadvantage is connected with the more parts as well. More parts mean a longer period of the exam linked with prolonged waiting. The stress can be minimized but acquainting students with the format and organization of the exam, the typology of individual exercises (which will be common for all schools) and more practice, which will be made easier by knowing what to expect. 35
It is also a good idea to let students be examined by their English teacher, somebody they know (which is impossible with UCLES). The oral part generally employs more tasks, which are more interactive. The problem I see here is only with the teachers, since their approach to teaching English is quite often oldfashioned. They use more of the grammar exercises and learning-by-heart methods rather than activites developing speaking skills. Another problem might arise with the assessment, because there have not been any criteria provided yet from the Ministry of Education. Schooling of English teachers should begin as soon as possible as well. Requirements for the new Maturita Exam also need time to get complete and detailed, there are many questions emerging each day. In neither forms there is any consideration for students with specific learning disabilities, just a vague argument that the student’s disabilities will be kept in mind. I cannot see any respect for students of multiple intelligences either, nor with Attention Deficit/Hyper Activity Disorder. All in all, the exam should become fully standardized, which cannot be said about the present form of the Maturita Exam in any way, as shown above. The act of making the exam standardized, which is the intention of the whole change happening these days, includes unifying the requirements and presenting them to all students equally, hence making the preparation similar and easier to compare. Secondly, the content and the course of the exam are to be rendered homogeneously, with a slight change in certain parts to present the indivual orientation of the school. Thirdly, the assessment needs to get cohesive, in order for the final marks to be comparable. This will require a great deal of training of English teachers, because they are mostly not used to assessing in accordance with any criteria. Time will be needed for a slow start, with a lot of practice even from the side of the teachers. The main aim should be actually reaching the level of international language exams (which are tried-and-true, especially their system of assessment), and, most importantly, are applicable for all users of English. This was not the fact with the answers in the questionnaires of people who had taken their Maturita Exam some years ago; they felt the exam did not prepare them sufficiently, thus making it unnecessary. Once this is managed, the exam will prove that it was worth considering the change in its form.
36
Conclusion This bachelor thesis focused on the oral part of the Maturita Exam, particularly on the differences between the present and incoming form, together with its assessment. It sought to find the reason for changing the present system and inquired whether it corresponded to the needs of students leaving secondary schools. In the theoretical part I started with the basic structure and organization of the present and incoming Maturita Exam in general, moved on to depicting the exam in the English language, and finally described its oral part in both forms. I based my research on existing legal
documents
securing
the exam
in the past,
present
and
future.
Furthemore,
the requirements for the oral part of the exam, as set by the Common European Framework of Reference for Languages, were offered, together with presenting PET as a relevant exam with nearly the same requirements. Finally, various types of assessment used for both forms were included in the last chapter. The practical part showed actual examples of different types of the oral part of the exam. At the beginning I chose two different schools to display the heterogeneity of the present exam – a private secondary business school and a public grammar school, both based in Prague. To contrast these I continued with the example given by the Ministry of Education for the new form of the exam, and another example of the speaking test in PET in Cambridge ESOL Examinations. The last chapter of the practical part analyzes questionnaires with the topic of the Maturita Exam, which I gave to twenty people to see what their Maturita Exam in the English language had looked like, and whether they were satisfied with its form, result and future recognition. All in all, I found out that the present form of the exam does not in any way compare to the requirements set by the CEFR, as it does not fulfil the demands for a complex objective exam. Even the respondents of the questionnaires wrote that they had not considered themselves being sufficiently prepared for the future in terms of their knowledge of English. Hence, the reason for changing the present form of the Maturita Exam can be interpreted as answering the call for a standardized exam, which would correspond to given requirements. I found out that the present form does not secure any uniformity in the requirements, format, course and outcome of the exam. There are no criteria used for the assessment, thus not making it possible to compare one same mark between schools, sometimes not even within the school itself, because each teacher uses different methods 37
(even formats) for the exam and its assessment. The new form should ensure the comparability of the exam not only in the Czech Republic, but also worldwide. The problem I see in no available training for English teachers to make them properly prepared in the area of assessing the oral performance of their students. Czech teachers are not used to this new criterion-based type of assessment, and I consider it essential for them to know all about the procedure before preparing students for the new form of the exam. Since this is to set off in 2010, I cannot see how the Ministry of Education intends to train all the teachers in the whole country. Even if it does, it will be done after they have prepared their students for the exam, which I assume to be inappropriate.
38
Works cited
Abbs, Brian, and Ingrid Freebairn. Blueprint. Harlow: Longman, 1995.
Brown, James D. Extraneous variables and the washback effect. Shiken: JALT Testing & Evaluation SIG Newsletter. 16 Apr. 2008 < http://www.jalt.org/test/bro_14.htm>.
“Cambridge ESOL Exams and the CEFR.” University of Cambridge: ESOL Examinations. 17 Oct. 2007 < http://www.cambridgeesol.org/what-we-do/research/cefr/index.html>.
“CEFR and ‘Can Do’.” University of Cambridge: ESOL Examinations. 12 Feb. 2008 .
“Common European Framework of Reference for Languages: Learning, Teaching, Assessment.” Council of Europe. 9 Jan. 2008 .
Council of Europe. Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge: CUP, 2001.
Council of Europe. Společný evropský referenční rámec pro jazyky: Jak se učíme jazykům, jak je vyučujeme a jak v jazycích hodnotíme. Překl. Ivanová, Lenochová, Línková, Šimáčková. Olomouc: Univerzita Palackého, 2002.
El-Hmoudová, Dagmar. Angličtina – Maturitní témata [English – Topics for the Maturita Exam]. Petra Velanová: Třebíč, 2007.
39
Falla, Tim, and Paul A. Davies. Maturita Solutions. Czech ed. Oxford: OUP, 2007.
Gault, James. New Headway Talking Points: Speaking Preparation for the Maturita Exam. Czech ed. Oxford: OUP, 2005.
“Global Glossary.” Classics Technology Center. 19 Nov. 2007 .
Harris, Michael. New Opportunities. Harlow: Longman, 2006.
Katalog požadavků k maturitní zkoušce: Anglický jazyk [The Catalogue of Requirements for the Maturita Exam: English Language]. Ministerstvo školství, mládeže a tělovýchovy [Ministry of Education, Youth and Sports]. 2004.
Katalog požadavků zkoušek společné části maturitní zkoušky: Anglický jazyk [The Catalogue of Requirements for Exams of the Common Part of the Maturita Exam: English Language]. Ministerstvo školství, mládeže a tělovýchovy [Ministry of Education, Youth and Sports]. 2008.
“Katalogy požadavků [Catalogues of Requirements].” CERMAT. 18 Apr. 2008 .
“Preliminary English Test.” University of Cambridge: ESOL Examinations. 12 Feb. 2008 < http://www.cambridgeesol.org/exams/general-english/pet.html>.
“Reforma v otázkách [The Reform in Questions].” CERMAT. 7 Apr. 2008 < http://www.cermat.cz/lstDoc.aspx?nid=10772>.
Soars, John, and Liz Soars. New Headway English Course. Oxford: OUP, 2004.
Underhill, Nic. Testing Spoken Language: A Handbook of Oral Testing Techniques. Cambridge: CUP, 1987. 40
Vyhláška ministerstva školství, mládeže a tělovýchovy České republiky č. 442/1991 Sb., o ukončování studia ve středních školách a učilištích [Notice of the Ministry of Education, Youth and Sports of the Czech Republic No. 442/1991 Coll., on Accomplishment of Studies in Secondary Schools and Training Centres]. Ministerstvo školství, mládeže a tělovýchovy [Ministry of Education, Youth and Sports]. 1991.
Vyhláška č. 672/2004 Sb. [Notice No. 672/2004 Coll]. Ministerstvo školství, mládeže a tělovýchovy [Ministry of Education, Youth and Sports]. 2004.
Zákon č. 561/2004 Sb., o předškolním, základním, středním, vyšším odborném a jiném vzdělávání (školský zákon) [Act No. 561/2004 Coll., on Pre-school, Basic, Secondary, Tertiary Professional and Other Education (the Education Act)]. Ministerstvo školství, mládeže a tělovýchovy [Ministry of Education, Youth and Sports]. 2004.
41
Resumé Tato bakalářská práce se zabývá ústní částí maturitní zkoušky v anglickém jazyce, zejména rozdíly mezi její současnou a budoucí podobou. Uvedeny jsou výhody i nevýhody obou podob zkoušky, spolu s různými typy jejího hodnocení. Následuje porovnání s cambridgeskou zkouškou PET jako zkouškou, která odpovídá jedné ze dvou úrovní nové maturity a zároveň odpovídá úrovni B1 dle Společného evropského referenčního rámce pro jazyky, který sloužil jako podklad pro zpracování katalogů požadavků k nové maturitní zkoušce z anglického jazyka. Práce hledá důvod, který vyvolal radikální změnu ve formě této zkoušky, a odpověď na otázku, zda její současná podoba odpovídá potřebám absolventů středních škol. Skutečné příklady různých typů ústních částí maturitní zkoušky jsou analyzovány v praktické části.
42
Appendices
Appendix 1a and 1b: Worksheets for the oral part of the present Maturita Exam at PB-VOŠ
Appendix 2: Worksheet for the oral part of the present Maturita Exam at Gymnázium
Appendix 3a, 3b, 3c and 3d: Worksheets for the oral part of the incoming Maturita Exam according to the catalogue of requirements by CERMAT
Appendix 4a, 4b and 4c: Sample papers for the PET speaking test
Appendix 5: Exemplary questionnaire
43
App. 1a: Worksheets for the oral part of the present Maturita Exam at PB-VOŠ
44
App. 1b: Worksheets for the oral part of the present Maturita Exam at PB-VOŠ
45
App. 2: Worksheet for the oral part of the present Maturita Exam at Gymnázium
46
App. 3a: Worksheets for the oral part of the incoming Maturita Exam according to the catalogue of requirements by CERMAT
47
App. 3b: Worksheets for the oral part of the incoming Maturita Exam according to the catalogue of requirements by CERMAT
48
App. 3c: Worksheets for the oral part of the incoming Maturita Exam according to the catalogue of requirements by CERMAT
49
App. 3d: Worksheets for the oral part of the incoming Maturita Exam according to the catalogue of requirements by CERMAT
50
App. 4a: Sample papers for the PET speaking test
51
App. 4b: Sample papers for the PET speaking test
52
App. 4c: Sample papers for the PET speaking test
53
App. 5: Exemplary questionnaire
54
