Do Exam Aims and Content Reflect those of the Curriculum? An Evaluative Study
DOI:
https://doi.org/10.37134/ajatel.vol14.1.6.2024Keywords:
EFL high-stakes tests, Curriculum, Aims, Objectives, Content Validity, EvaluationAbstract
Language tests, particularly high-stakes language tests, are a powerful tool for evaluating educational outcomes, but their effectiveness hinges on how they are constructed. Failure to construct valid sound tests can negatively impact both teaching and learning. Therefore, continuous research is needed to investigate and evaluate such types of tests. This study examines the congruency between the aims and content of a high-stakes public EFL examination and the prescribed curriculum in Libyan schools. Document analysis was conducted on a sample of the studied test, focusing on its intended objectives and their reflection on the curriculum's goals and content. While the exam offered some practical advantages, the findings revealed a mismatch between the administered exam and the curriculum, and between the stated aims and the actual content of the exam itself. The assessment focused solely on grammar points and reading comprehension based on information cloned verbatim from the prescribed textbook. Writing was assessed indirectly through true-false responses to prompts involving structured or unstructured sentences. Notably, the exam entirely neglected listening and speaking skills. These findings suggest that the current examination system could hinder curriculum implementation and EFL education in Libyan schools. Therefore, the study calls for examination reform to ensure alignment between the tests and the English curriculum, ultimately promoting effective second language learning.
Downloads
References
Abdulhamid, N. (2019). What Is the Relationship Between Alignment and Washback? A Mixed-Methods Study of the Libyan EFL Context (Doctoral dissertation, Carleton University).
Ahmed, J. U. (2010). Documentary research method: New dimensions. Indus Journal of Management & Social Sciences, 4(1), 1-14.
Ali, M. A. A. (2008). The Oral Error Correction Techniques Used by Libyan Secondary School Teachers of English (Doctoral dissertation, University of Sunderland).
Amin, R. (2021). Students’ perception on high-stakes English language testing in Bangladesh: expectations vs reality (Doctoral dissertation, Brac University).
Alderson, J. C., & Kremmel, B. (2013). Re-examining the content validation of a grammar test: The (im) possibility of distinguishing vocabulary and structural knowledge. Language Testing, 30(4), 535-556.
Alderson, J. C., & Wall, D. (1993). Does washback exist? Applied linguistics, 14(2), 115-129.
Beglar, D., & Hunt, A. (1999). Revising and validating the 2000-word level and university word-level vocabulary tests. Language testing, 16(2), 131-162.
Bejar, I. I., Mislevy, R. J., & Zhang, M. (2016). Automated scoring with validity in mind. The Wiley handbook of cognition and assessment: Frameworks, methodologies, and applications, 226-246.
Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative research journal, 9(2), 27-40.
Bridglall, B. L., Caines, J., & Chatterji, M. (2014). Understanding validity issues in test-based models of school and teacher evaluation. Quality Assurance in Education, 22(1), 19-30.
Chapelle, C. A., & Voss, E. (Eds.). (2021). Validity argument in language testing: Case studies of validation research. Cambridge University Press.
Cohen, L. Manion, L. and Morrison, K. (2011) Research Methods in Education, 7th ed. London: Routledge.
Cohen, L. Manion, L. and Morrison, K. (2017) Research Methods in Education, 8th ed. London: Routledge.
Creswell, J. W. (2015). Educational research: Planning, conducting, and evaluating quantitative and qualitative research. Pearson.
Denscombe, M. (2021). Research methods for small-scale social research projects, 7th ed. McGraw-Hill Education.
Desalegn, G., Disassa, R., & Kitila, T. (2023). The Influence of High-Stakes English Examinations on Students’ Out-of-Classroom English Learning Practices: A Comparative Study. Education Research International, 2023.
Dong, M., & Liu, X. (2022). Impact of learners' perceptions of a high-stakes test on their learning motivation and learning time allotment: A study on the washback mechanism. Heliyon, 8(12).
Dong, M., Fan, J., & Xu, J. (2021). Differential washback effects of a high-stakes test on students’ English learning process: Evidence from a large-scale stratified survey in China. Asia Pacific Journal of Education, 43(1), 252-269.
Elabbar, A. A. (2021). Reforming the Libyan education system: Seven articulated years via a strategic planning pyramid. London Journal of Research in Humanities and Social Sciences, 22(14), 11-25.
French, S., Dickerson, A., & Mulder, R. A. (2023). A review of the benefits and drawbacks of high-stakes final examinations in higher education. Higher Education, 1-26.
Frino, L. Mhochain, R.N. O'Neil, H. and McGarry, F. (2021) English for Libya, Preparatory 3: Teacher's Book, Reading-UK: Garnet Publishing Ltd.
Fulcher, G. (2011) 'Cheating gives lies to our test dependence: Policymakers are using language tests to carry a larger social burden than they can reasonably bear'. the Guardian Weekly, Tuesday 11th October 2011, Learning English (4), http://www.guardian.co.uk/education/2011/oct/11/why-more-language-testcheating?INTCMP=SRCH. Online accessed on 12 November 2023.
Fulcher, G. (2015). Re-examining language testing: A philosophical and social inquiry. Routledge.
Haladyna, T. M., Nolen, S. B., & Haas, N. S. (1991). Raising standardized achievement test scores and the origins of test score pollution. Educational Researcher, 20(5), 2-7.
Han, B., Dai, M., & Yang, L. (2004). Problems with College English Tests as emerged from a survey. Foreign languages and their teaching, 179(2), 17-23.
Henning, G. (1987) A Guide to Language Testing: Development, Evaluation, Research. Cambridge: Newbury House.
Hughes, A. & Hughes, J. (2020). Testing for language teachers. Cambridge University Press.
Gandini, E. A., & Horák, T. (2020). Promoting positive washforward through personalised test feedback and other benefits: Piloting a computer-based testing system. Language Learning in Higher Education, 10(1), 235-244.
Gashaye, S., & Degwale, Y. (2019). The Content Validity of High School English Language Teacher Made Tests: The Case of Debre Work Preparatory School, East Gojjam, Ethiopia. International Journal of Research in Engineering, IT and Social Sciences, 9(11), 41-50.
Gorgodze, S., & Chakhaia, L. (2021). The uses and misuses of centralised high stakes examinations-Assessment Policy and Practice in Georgia. Assessment in Education: Principles, Policy & Practice, 28(3), 322-342.
Green, A. (2013). Washback in language assessment. International Journal of English Studies, 13(2), 39-51.
Ghuma, M.A. (2011) The Transferability of Reading Strategies between L1 (Arabic) and L2 (English), unpublished PhD thesis. University of Durham.
Ghuma, M. A. (2021). Content Validity of National Exams of English Reading in Libya. Libya Bulletin for Studies 7th issue. Dar Azzawyah Lelketab, (7). 59-78.
Haggie, D. (2008). The strategic use of marking technologies to support innovation and diversity in assessment. International Association of Educational Assessment, September, in Singapore. http://www. iaea2008. Cambridge assessment. org. uk/ca/digital Assets/180436_Haggie. pdf.
Hoque, M. E. (2016). Teaching to the EFL curriculum or teaching to the test: An investigation. The EDRC Journal of Learning and Teaching, 1(1), 1-25.
Kane, M. T., & Wools, S. (2019). Perspectives on the validity of classroom assessments. In Classroom assessment and educational measurement (pp. 11-26). Routledge.
Kang, M. K., & Chang, H. J. (2014). Ensuring Validity of Practical English Certification Test of Local Office of Education in Korea. Journal of Pan-Pacific Association of Applied Linguistics, 18(2), 111-122.
Katiso, A. (2022). Analyzing Content and Face Validity of English Final Examination Prepared by Damboya District Education Office: Grade Seven in Focus (MA Dissertation, Haramaya University).
Klingebiel, S., Hartmann, F. L., Madani, E., Paintner, J., Rohe, R. A., Trebs, L., & Wolk, T. (2024). Methods for Data Collection and Analysis. In Exploring the Effectiveness of International Knowledge Cooperation: An Analysis of Selected Development Knowledge Actors (pp. 43-50). Cham: Springer Nature Switzerland.
Mayring, P. (2021). Qualitative Content Analysis: A Step-by-Step Guide. United Kingdom: SAGE Publications.
ME (Ministry of Education) (2008) 'The Development of Education, the national report of Libya presented to the international conference on education, session (48), Geneva, November,' 2008, Libya Authority of Education 25 - 28.
Messick, S. (1984). The psychology of educational measurement. ETS Research Report Series, 1984(1), i-55.
Moritoshi, T. P. (2002). Validation of the test of English conversation proficiency. Unpublished master's dissertation. University of Birmingham, UK. (Available online: http://www. bhamlive1. bham. ac. uk/Documents/collegeartslaw/cels/essays/matefltesldissertations/MoritoshiDiss. pdf).
Onaiba, A. M. E. M., & Mustafa, A. (2014). Investigating the washback effect of a revised EFL public examination on teachers' instructional practices, materials, and curriculum (Doctoral dissertation, University of Leicester).
Orafi, S. M. S. (2008). Investigating teachers' practices and beliefs in relation to curriculum innovation in English language teaching in Libya (Doctoral dissertation, University of Leeds).
Orafi, S. M. S., & Borg, S. (2009). Intentions and realities in implementing communicative curriculum reform. System, 37(2), 243-253.
Otman, W., & Karlberg, E. (2007). The Libyan economy: economic diversification and international repositioning. Springer Science & Business Media.
Ozer, I., Fitzgerald, S. M., Sulbaran, E., & Garvey, D. (2014). Reliability and content validity of an English as a Foreign Language (EFL) grade-level test for Turkish primary grade students. Procedia-Social and Behavioral Sciences, 112, 924-929.
Özkan, U. B. (2023). Validity and Reliability in Document Analysis Method: A Theoretical Review in the Context of Educational Science Research. Education, (56), 823-848.
Schmitt, N., Schmitt, D., & Clapham, C. (2001). Developing and exploring the behaviour of two new versions of the Vocabulary Levels Test. Language testing, 18(1), 55-88.
Shaw, S., Crisp, V., & Johnson, N. (2012). A framework for evidencing assessment validity in large-scale, high-stakes international examinations. Assessment in Education: Principles, Policy & Practice, 19(2), 159-176.
Shihiba, S. E. S. (2011). An investigation of Libyan EFL teachers’ conceptions of the communicative learner-centered approach in relation to their implementation of an English language curriculum innovation in secondary schools (Doctoral dissertation, Durham University).
Siddiek, A. G. (2010). The impact of translation on language acquisition and knowledge transfer in the Arab world. European Journal of Social Sciences, 16(4).
Shohamy, E. (2020). The power of tests: A critical perspective on the uses of language tests. Routledge.
Sun, M. (2022). Validity and Fairness of TOEFL iBT Reading Test. Learning & Education, 10(8), 141-142.
Takeno, J., & Moritoshi, P. (2018). Re-examining the English Proficiency Level of Japanese EFL Learners. Chugokugakuen Journal, 17, 35-39.
Wisdom, S. C. (2018). Teachers' Perceptions About the Influence of High-Stakes Testing on Students (Doctoral dissertation, Walden University).
Ziebell, N. (2018). Curriculum alignment: Performance types in the intended, enacted, and assessed curriculum in primary mathematics and science classrooms. Studia paedagogica, 23(2), 175-203.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Abdulhamid Onaiba
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
