The present study aimed to evaluate Top Notch series, which is widely used in EFL contexts. ... 3 regard to ACTFL standards, the series suffer from some shortcomings that ... HamiloÄlu and Karlıova (2009) conducted a comparative study and evaluated five English ... Each Top Notch unit is made up of six two-page lessons.
The Iranian EFL Journal February 2013 Volume 9 Issue 1
ISSN On-line: 1836-8751
The Iranian EFL Journal February 2013 Volume 9 Issue 1
Chief Editors
Dr. Paul Robertson Dr. Rajabali Askarzadeh Torghabeh
Iranian EFL Journal 1
The Iranian EFL Journal February 2013 Volume 9 Issue 1 (pp.162-171)
Title Textbook Evaluation based on the ACTFL standards: The case of Top Notch series Authors Minoo Alemi Sharif University of Technology
Zahra Mesbah Sharif University of Technology
Biodata Minoo Alemi is a faculty member of Languages and Linguistics Department at Sharif University of Technology. Her main areas of interest are inter-language pragmatics, SLA, ESP, and syllabus design. She has published 20 textbooks in general English and ESP, 34 papers in different areas in international journals, and given presentations on TEFL at 57 national and international conferences. Zahra Mesbah is an M.A student in TEFL at Sharif University of Technology. Her main areas of interest are second language learning and teaching, educational technology, and curriculum instruction. She has some publications and attended both national and international conferences on TEFL.
Abstract The present study aimed to evaluate Top Notch series, which is widely used in EFL contexts. Fifty EFL teachers evaluated the series based on ACTFL standards by Cisar’s (2000) checklist. The findings indicate that the series enjoy some benefits for language learners such as encouraging the students to communicate successfully by offering lots of opportunities for interaction, and demonstrating culture-based aspects through lively and authentic unbiased visual images. The series also let students connect their foreign language learning with other disciplines such as Medical Science, Art, Geography, etc. Moreover, the series offer vast opportunities for EFL learners to compare their own language with English as a foreign language. However, with 2
regard to ACTFL standards, the series suffer from some shortcomings that require EFL teachers’ consideration. Keywords: Textbook/Course book evaluation, Top Notch, EFL context, ACTFL standard
1. Introduction ELT materials (textbooks) have always played a very crucial role in the realm of language teaching and learning. Since textbook evaluation subsumes production, evaluation, and adaptation of materials, it is of paramount importance to conduct evaluation procedures that assure their pedagogical contribution to the learning and teaching process. In this sense, the use of textbook has prominence in ELT classroom (Hutchinson & Torres, 1994). It is also noteworthy that there is no single book that can satisfy all learners’ needs (Lamb & Nunan, 1996). It is of utmost importance for teachers to choose an appropriate textbook that can best fit a particular EFL/ESL teaching and learning objectives and context. Teachers make necessary adjustments by adapting the materials before applying them in ELT classroom. Therefore, there is a necessity for assessing textbooks in different contexts to discover the benefits of one over the others, which in turn will lead to selection of an appropriate textbook. In an EFL context like Iran, EFL learners have rarely access to the native speakers, thus the teachers mediate between the learner and the text. Accordingly, a textbook plays a central role in this context and textbook evaluation is, therefore, a necessity. It seems that only few studies have dealt primarily with the evaluation of Top Notch series as one of the most widely used course books in Iranian institutes, in this context. This study investigates the strengths and weaknesses of this series.
2. Review of the Related Literature In the literature on textbook evaluation, a large number of researchers have focused their attention on evaluating textbooks which have been used in an EFL context. Ranally (2002), for instance, evaluated New Headway which is used at the foreign language institutes in South Korea. Having analyzed the data, the researcher found that there is a good balance of work on both accuracy and fluency while the book suffers from some shortcomings particularly in the methodology. It was also revealed that there is not any consideration of speaking skills in the book. Hamiloğlu and Karlıova (2009) conducted a comparative study and evaluated five English course books from the view points of vocabulary selection and teaching techniques. The books were Countdown to First Certificate, Advanced Master Class, Grammar In Context 2, New Headway Advanced, and Top Notch 2. The results of evaluation indicated that all the selected course books integrated lexis into their syllabus, giving emphasis to vocabulary building .The results also showed that all the intended course books employed colorful layout to facilitate vocabulary acquisition and comprehension with the use of illustrative/pictorial modes except for Grammar in context 2 which has pictures and drawing, though not colorful. 3
Elsewhere, Litz (2000) evaluated First Hand 2 which reflects multi- syllabi, and integrates four skills in a well organized design. He also elaborated on some of its weak points such as its repetitive activities and its failure to engage learners in meaningful use of language. In a similar vein, Eslami, Esmaeli, Ghavaminia, and Rajabi (2010) set out to evaluate the four mostly instructed courses in Iran English language institutes. The books were Top Notch, Interchange, Headway, and On Your Mark. The researchers evaluated the course books under study in 2 stages based on Mcdounough and Shaw’s (2003) division of course book evaluation into internal and external evaluation. After a thorough examination of introduction and table of contents, and in-depth examination of the cover page and introductory sections of the books, they concluded that Top Notch best meets Mcdounough and Shaw’s (2003) evaluation criteria. Sahrahgard, Rahimi, and Zaremoaeyeddi (2009) also carried out an in-depth examination of the third edition of Interchange. They found that Interchange 3rd edition suffers from some weak points such as a lack of reference to teachers and learners. The next shortages were the overreliance on input enhancement techniques, ignoring the significant role of self-directed activities in task completion. However, there were also some positive sides in the series, including its great emphasis on pair work and meaningful interaction. Similarly, Riazi and Mosallanejad (2010) investigated the types of learning objectives in four textbooks taught in Iranian high schools. The findings of this study indicated that a sequential ordering was noticeable in terms of level of complexity and text length. In fact, it was found that the most prevalent learning objectives in the textbooks were lower order cognitive skills, i.e., knowledge, application, and comprehension. In line with the abovementioned research, the present study endeavors to evaluate the intended course book using the questionnaire developed by Cisar (2000) based on ACTFL standards to shed more light upon the strengths and weaknesses of the series.
3. Methodology 3.1 Participants The participants of the study were 50 Iranian EFL teachers who were asked to complete the checklist. The teachers had at least 5 years experience of teaching Top Notch series. Their age ranged from 25 to 30. 3.2 Instrument The current study employed two kinds of instruments including Top Notch series and Cisar's (2000) checklist. A brief description of each is provided below. 4
3.2.1 Top Notch series Top Notch (Saslow & Ascher, 2006) was selected to be evaluated in this study. Top Notch, a dynamic 6-level course for international communication, as the authors claim, aims to prepare students to understand spoken and written English and to express themselves appropriately. The series is based on new standards and uses the natural language that people really speak with a rock-solid learner-centered approach. Each Top Notch unit is made up of six two-page lessons. The sections include the introduction, controlled practice, controlled practice, free practice, free practice, and review. Top Notch series provides learners with an opportunity to see their own progress at the end of every lesson. Each student’s book consists of 10 units accompanied by the workbook, CD, teacher’s manual and guide. 3.2.2 The Checklist To conduct the evaluation, Cisar’s (2000) checklist which is based on the American Council on Teaching of Foreign languages (ACTFL) standard was employed. A 4-point Likert scale ranging from Not at all to Completely was used to elicit the teachers' viewpoints regarding the content of the books. The standards provide an opportunity to monitor all the phases of instruction. Overall, the standards used in the checklist could be categorized into five main components. These categories are communication, culture, connection, comparison, and community. Communication refers to the ability to communicate in language with specific reference to the student’s participation in communicative acts. Culture implies an understanding of other cultures in view of the commonly held practices and products. The third component is Connection which sheds light on the degree to which learners acquire information and accordingly expand their knowledge of other discipline via foreign language learning. The Comparison element determines the strength of the book in terms of making an association between linguistic and cultural aspects of the textbooks under the investigation. The last component is Community which focuses on learner’s participation in multilingual communities at home and around the world. Furthermore, it suggests how much the textbook encourages using language beyond the classroom setting.
4. Results and Discussion With specific reference to the research questions, the following analytical procedures were employed. The inter-rater reliability which was calculated via intraclass for the questionnaire was 0.8. To address the first research question, “How Top Notch series are in terms of ACTFL standards”, descriptive statistics including the mean and standard deviation for each subcomponent were calculated. The results are reported in Table 1.
As displayed in the table, regarding the component of communication, the total mean is 3.7 and the standard deviation is .53. This could in effect mean that the series is successful at 5
preparing students to interact and communicate effectively. It convincingly encourages the learners to get involved in communicative acts. Employing an integrated multi-skills syllabus, the series provides the students with opportunities to relate topics, themes, communication functions and grammatical points. The activities are in real-world context and represent realworld tasks and thus allow students to take advantage of different strategies. Furthermore, the topics of discussion are all around concrete ideas. This part confirms the findings of Eslami et al.’s (2010) study which emphasized rich and authentic topics, subject and social content of Top Notch.
With regard to the cultural aspect, a total mean of 3.57 was obtained with the standard deviation being .59. It worth mentioning that the series is not restricted to a particular country or region, nor is it culturally biased or specific. It tries to cater for appropriate social environment in terms of topics and situation to develop cultural awareness. It also aims to demonstrate a balanced picture of different societies and nationalities without being culturally-bound. Thus, it is rich in terms of cultural content.
Given the third component of the checklist, i.e. that of connection, as the results demonstrate, the total mean obtained is 3.25 and the standard deviation is .80. Although the standard deviation shows a little inconsistency among raters, most of the teachers deemed it as a merit of the series to allow students to further their knowledge of other disciplines such as computer science, art, health, etc. Another point which is worth noticing is that the series enables the students to use previously learnt materials to do real life tasks. In fact, students are given opportunities to build on their prior personal experiences and existing background knowledge.
With respect to the comparison part, the mean score was 2.16 and the standard deviation was 1.73. The results also showed that the teachers participating in the study tended to rate the items of this part rather differently. The variation can be attributed to the fact that the series includes a few examples such as Ramadan and praying costume in Top Notch3 and sometimes no activities which represent Muslim culture. Some raters mentioned that, though it represents few aspects of Iranian culture, it cannot be considered as a sign of inadequacy of the course book. Despite this fact, it is worth mentioning that the series is rich in terms of providing learners with a variety of situations to compare and contrast their own culture with foreign language culture. As for the community part, the total mean score was 2.15 and a value of 1.58 was obtained for the standard deviation. The results also revealed that most of the teachers agreed that the series through different activities can somehow enable learners to participate in multilingual communities in their country and around the world. The researchers believe that the variation in the teachers' ratings is associated with Iranian EFL context rather than the book. Iranian EFL learners have rare opportunities 6
to interact with native speakers. One of the unique features of the series is that it caters for students a lot of on-line activities and even songs with the lyrics to encourage them to communicate beyond the scope of classroom. The unit wrap-up which is presented in top notch activities motivates the learners to be more critical and creative to use pictures, instructed vocabularies and grammar to draw on their own story and to use language outside the classroom as well. Table 1. Descriptive statistics for ACTFL standards Standards
Total Mean
Std. Deviation
Communication Culture Connection Comparison
3.7 3.57 3.25 2.16
.53 .59 .80 1.7
Community
2.15
1.58
Top Notch series was evaluated based on ACTFL standards and then some general merits and demerits of the intended series for EFL context were determined. As one of its merits, Top Notch series starts with a preview section which introduces the unit topic and objectives. It also has a well-organized presentation of the materials which allow students to preview the vocabulary, grammar and social language from the unit. This is in contrast with what Riasati and Zare's (2010) concluded about Interchange series. Their findings showed that the Interchange series did not include a detailed overview of the function, structure, vocabulary, any review section or any sort of evaluation quizzes at the end of each unit. While Interchange series suffers from a lack of supplementary teaching materials, Top Notch series compensates for these shortcomings by offering “teacher’s resource disk” which consists of extensive learning activities. Moreover, Top Notch interaction and check point parts tend to provide the students with an appropriate model and then prompt them to produce their own piece of language. EFL course books, especially those implemented in an EFL context, are definitely the primary source of learning how to pronounce or utter a word. As Hamiloğlu and Karlıova (2009) pointed out, Top Notch like headway offers lots of opportunities to improve pronunciation through a specific pronunciation section, where there is ample practice of rhythm, intonation, etc. Unlike Sahragard et al.'s (2009) study which showed that Interchange series caters for more involvement of the learners in the classroom event, the observations made in the present study revealed that Top Notch series through different on-line activities and projects encourages learners to use language beyond the classroom as well. As also confirmed by Eslami et al.'s (2010) study, it was observed that Top Notch series integrates lexis into its syllabus effectively and gives great emphasis to world knowledge in 7
different parts of each lesson. The in-depth analysis of the series also sheds light on the fact that new words can be consolidated if they are being repeated in all language skills. With regard to grammar and form of language, Top Notch series offers grammar booster for each grammatical point and employs both deductive and inductive approach toward teaching grammar. As one of the demerits, the proportion of the tasks that involve students in pair work or group work is a little bit more than those task involving learners individually. Although, it would foster the opportunities for more interaction and communication, it can also reduce the autonomy of language learners because they can no more rely on themselves to follow the course book. The results of the study also indicated that the series imposes the content; in fact, it gives little opportunities to either teachers or learners to determine the source of the input and the topic content. In line with what Eslami et al. (2010) conclude about the series, the present study also provided evidence for the claim that Top Notch attempts to present all the four skills in an integrated way. However, there are some parts where one of the skills has been thoroughly discarded. Although the series includes both conversation models and conversation pair works in each lesson, the dialogues sometimes seem to be too short and artificial, lacking the features of natural speech as well. Given the cultural aspects, it appears that there is sometimes no correspondence between Iranian cultures and western cultures; for example, Iranian learners may not know much about Mariachi band, harvest moon, thanksgiving, etc. In such cases, the burden is placed on the shoulders of the language teachers to help learners get acquainted with such cultural issues through providing them with the necessary information and feedback. Besides, taking the teacher's guide of the series into account, it was found that the time suggested to be allocated to some aspects of language such as listening, discussion, and interaction is not sufficient. As a matter of fact, is believed that the students need more time to accomplish the activities and to consolidate them to their memories. It is common that authors always make claims about their books. Top Notch authors are not an exception in this respect and they mention seven claims at the back of the book. What is of paramount importance, however, is the extent to which such claims are met in real classroom setting. The first claim is that the series has employed “essential model conversations that make key social language unforgettable and easy to personalize”. As also indicated by Eslami et al. (2010), the researchers sometimes found the conversations artificial and distant from real life or
8
everyday situations, because they tend to be so short and in some instances free from authentic interactional features such as pauses, repetition, etc. With regard to the second claim implying that "Intensive vocabulary development with active recycling" has been utilized throughout the book, the researchers were unanimous that the series, in fact, did employ a rich syllabus in terms of vocabulary. The new vocabularies entered to the picture not only through vocabulary sub-headings but they were also reinforced by repetition in all four skills. The series also offers extensive opportunities for learners to utilize new vocabularies in Top Notch interaction part and unit Wrap up, where they are asked to tell the story. Yet another claim made by the authors is that the book is equipped with a “Complete grammar support-extended by a bound in grammar booster”. The researchers believe the series substantiated the claim appropriately. Through the grammar booster, learners can benefit from more explanations, practices, and grasping additional grammar points both inductively and deductively. Pertaining to the claim that there are "Top Notch interaction unique step-by- step discussion builders that guarantee success for all learners", the researchers believe that the discussion builders encourage learners to engage in free discussions and help them get nearer to the ultimate goal of communicating their thoughts effectively by drawing on the previously learnt materials. Although massive opportunities for recycling the language occur in this part, sometimes the topics seem to be boring and neither teachers nor learners have the option to choose between the topics. The authors of the series also claim that thorough attention has been paid to pronunciation throughout the book. The analyses showed that ample support is provided in the series for this claim. Each unit includes specific pronunciation point and another benefit is that pronunciation activities in most of the units are based on the unit’s content. After listening once, students can enjoy the opportunity to read the text to a partner with correct rhythm and intonation and receive feedback on their pronunciation. The other claim posited by authors is that "a wide array of learning strategies and activities that promote critical thinking” has been focused on in the series. The researchers found that the types of activities are somehow in accordance with this statement. The first two lessons in each unit focus on controlled activities and the last two ones on free practice. Given the reading section, the learning strategies start with inference followed by other strategies, namely predicting, guessing, and critical thinking, while in the listening part, they move from auditory or sound discrimination, general idea, and inference to developing critical thinking. Finally the authors claim that the series enjoys "authentic and refreshing content that connects students to the real world”. Although sometimes the topics fall out learners’ favor, overall the 9
series through employing lots of authentic pictures and natural language not only shows cosmic underlying values but also connects students to the real world.
5. Conclusion The findings of the present study can yield the general conclusion that the Top Notch series meets ACTFL standards in the most areas to a great extent and it is successful at immersing learners in an English speaking environment, and making the classroom itself a microcosm of English-speaking world. Furthermore, the series seems to be appealing to both teachers and students because of its physical characteristics including a colorful design and layout, wellorganized and a high impact syllabus and the presentation of real-world and natural language in most parts. Although it should be admitted that this series has its own shortcomings as well.
The results of the present study can help enriching the literature on textbook evaluation in the EFL context. Besides, the findings can also be to the benefit of EFL teachers. The evaluation results can also be applicable in terms of enhancing the effective use of the series in that they may assist material developers to understand what areas of the series need further modifications. In the same vein, the recommendation is that teachers be encouraged to adapt their existing practices and bring the supplementary materials to the classroom to enhance their quality of teaching, which could in turn result in better learning outcomes on the part of language learners.
There are a number of limitations which have to be considered by the future researchers. In fact, in order to yield more conclusive and comprehensive results, it is recommended to use a larger number of raters with more teaching experience in terms of Top Notch series. Moreover, with a larger sample size, the generalizability of the results would be more probable. Indeed a possible future research direction is to try to administer the checklist not only to a large number of teachers but also to a larger sample of learners who study the series to have their own perspectives regarding the pros and cons of the series.
10
References Cisar, S. H. (2000). Standard textbook evaluation guide: Foreign language standards implementation guide, Indiana University. Eslami Rasekh, A., Esmaeli, S., Ghavamnia, M. and Rajabi, S. (2010). ‘Don’t judge a book by its cover: Textbook evaluation in the EFL setting’. The Journal of International Social Research, 3(14), 448-461. Hamiloğlu, K. and Karlıova, H. (2009). ‘A content analysis on the vocabulary presentation in EFL course books’. Ozean Journal of Social Science, 2(1), 43-54. Hutchison, T. and Torres, E. (1994). ‘The textbook as agent of change’. ELT Journal, 48(4), 315-328. Litz, D. R. A. (2000). ‘Textbook evaluation and ELT management: A South Korean case study’. Asian EFL Journal, no volume given, 1-53. Retrieved from http://www.asian-efljournal.com/Litz_thesis.pdf Mcdounough, J. and Shaw, C. (2003). Materials and method in ELT. Oxford: Blackwell. Ranalli, J. 2002. An Evaluation of New Headway upper-intermediate. Birmingham University. Riasati, M. J. and Zare, P. (2010). ‘Textbook evaluation: EFL teacher’s perspectives on new interchange’. Studies in Literature and Language, 1(8), 54-60. Riazi, A. M. and Mosalanejad, N. (2010). ‘Evaluation of learning objective in Iranian high school and pre-university English textbook using Bloom’s taxonomy’. TESL-EJ, 13(4), 1-14. Sahragard, R. and Rahimi, A.and Zaremoayeddi, I. (2008). ‘An in-depth evaluation of interchange series’ (Third edition). Porta Linguarum, 37-54. Saslow, J. and Ascher, A. (2006). Top Notch series: English for today’s world. White Plains, New York: Pearson Education.
11
