A CEFR-Informed English Test: How Content Validity Index and Content Validity Ratio are used for Content Validation

Kamarul Ariffin Ahmad; Izazol Idris

doi:10.37134/ejoss.vol11.1.7.2025

Authors

Kamarul Ariffin Ahmad Fakulti Bahasa dan Komunikasi, Universiti Pendidikan Sultan Idris, 35900 Tanjong Malim, Malaysia
Izazol Idris Fakulti Pembangunan Manusia, Universiti Pendidikan Sultan Idris, 35900 Tanjong Malim, Malaysia

DOI:

https://doi.org/10.37134/ejoss.vol11.1.7.2025

Keywords:

CEFR, Content Validity Index (CVI), Content Validity Ratio (CVR), English Entrance Examiantion

Abstract

Current education roadmap in Malaysia has seen enormous initiatives taken by the government to ensure the education system is at par with the international standard where the English syllabus and assessments are tailored according to the Common European Framework of Reference (CEFR). This includes higher learning institutions, thus this research aimed to develop the English entrance test that is CEFR-informed. 11 CEFR experts were involved in the study where they reviewed 60 multiple choice questions for reading, grammar and reading comprehension. The results showed that all items were accepted and valid based on the computation of Content Validity Index (CVI) and Content Validity Ratio (CVR). The lowest CVI recorded was .82 with S-CVI of .95 and the lowest CVR was .64 which were above the suggested cut-off value by experts and research. The study concluded that all the items were carefully developed based on the framework and the needs of the test.

Downloads

Download data is not yet available.

References

Adom, D., Adu-Mensah, J., & Dake, D. A. (2020). Test, measurement, and evaluation: Understanding and use of the concepts in education. International Journal of Evaluation and Research in Education (IJERE), 9(1), 109. https://doi.org/10.11591/ijere.v9i1.20457

Ahmad Tamizi Abu, Rabiatul Adawiah Ahmad Rashid, & Salmiza Saleh. (2021). Al-Alaq Teaching Module: The Process for Evaluating Content Validity and Its Feasibility. Journal of Contemporary Social Science and Education Studies (JOCSSES), 1(2), 175–184. Retrieved from https://jocss.com/index.php/multidiscipline/article/view/7

Ahorsu, D. K., Lin, C.-Y., Imani, V., Saffari, M., Griffiths, M. D., & Pakpour, A. H. (2020). The fear of covid-19 scale: Development and initial validation. International Journal of Mental Health and Addiction, 20(3), 1537–1545. https://doi.org/10.1007/s11469-020-00270-8 Almanasreh, E., Moles, R., & Chen, T. F. (2019). Evaluation of methods used for estimating content validity. Research in Social and Administrative Pharmacy, 15(2), 214–221. https://doi.org/10.1016/j.sapharm.2018.03.066

Aryadoust, V., Ng, L. Y., & Sayama, H. (2020). A comprehensive review of rasch measurement in Language assessment: Recommendations and guidelines for research. Language Testing, 38(1), 6–40. https://doi.org/10.1177/0265532220927487

Babo, R., V. Babo, L., T Suhonen, J., & Tukiainen, M. (2020). E- assessment with multiple-choice questions: A 5 year study of students’ opinions and experience. Journal of Information Technology Education: Innovations in Practice, 19, 001–029. https://doi.org/10.28945/4491

Cohen-Swerdlik (2009). Psychological Testing and Assessment: An Introduction to Tests and Measurement (7th ed.). The United States of America: McGraw-Hill.

Haladyna, T.M., (2004). Developing and Validating Multiple-Choice Test Items (3rd ed.). London: Lawrence Erlbaum.

Hasimah Ja’afar. (2020, February). Discussion on the Current Instrument for Entrance Examination. Personal Interview.

Kamarul Ariffin Ahmad, (2018). Item Analysis and Score Dependability of English Proficiency 1 Examination in a Public University (Master’s Dissertation). University of Malaya, Kuala Lumpur.

Kamarul Ariffin Ahmad, & Zahari Suppian. (2022). International and National Teacher Education Conference (INTEC 2022). In Teacher Education in The Next Normal (pp. 117–126). Hat Yai; Prince of Songkla University.

Kementerian Pendidikan Malaysia, Malaysia Education Blueprint, 2013-2025: Preschool to post-secondary education (2013). Putrajaya.

Lynn, M. R. (1986). Determination and quantification of content validity. Nursing research, 35(6), 382-386.

Massitah Kipli, & Ahmad Zamri Khairani. (2020). Content Validity Index: An application of validating CIPP instrument for programme evaluation. Int Multidiscip Res J, 2(4), 31-40.

Mohd Effendi, Ahmad Zamri, Hisyamsani Idris & Normawati Rahman. (2017). Kesahan Kandungan Pakar Instrumen IKBAR Bagi Pengukuran AQ Menggunakan Nisbah Kesahan Kandungan.

Nor Ashiquin C. Alih, Masdinah Alauyah Md Yusoff, & Abdul Halim Abdul Raof. (2020). Teachers’ Knowledge and Belief on the CEFR Implementation in Malaysian ESL Classroom. International Journal of Multidisciplinary and Current Educational Research (IJMCER), 2(5), 126–134.

Norshahira Osman, Norashida, S. R., & Lukman, Z. M. (2021). Content validity determination of the suicidal ideation behaviour assessment instruments using content validity ratio (CVR) formula. Journal of Drug Delivery and Therapeutics, 11(4-S), 81–85. https://doi.org/10.22270/jddt.v11i4-s.4968

Rachmat, N. A., & Arfiandhani, P. (2019). “I use multiple-choice question in most assessment I prepared”: EFL teachers’ voice on summative assessment. ETERNAL (English, Teaching, Learning, and Research Journal), 5(1), 163. https://doi.org/10.24252/eternal.v51.2019.a13

Shrotryia, V. K., & Dhanda, U. (2019). Content validity of assessment instrument for employee engagement. SAGE Open, 9(1), 215824401882175. https://doi.org/10.1177/2158244018821751

Zeraati, M., & Alavi, N. M. (2014). Designing and validity evaluation of quality of nursing care scale in Intensive Care Units. Journal of Nursing Measurement, 22(3), 461–471. https://doi.org/10.1891/1061-3749.22.3.461

Zuraidah Mohd Don, & Mardziah Hayati Abdullah. (2019). What the CEFR is and isn’t. Kementerian Pendidikan Malaysia. https://www.moe.gov.my/en/menumedia/printed-media/newspaper-clippings/what-the-cefr-is-and-isn-t-free-malaysia-today-27-mei-2019