A Systematic Review on Instruments to Assess Critical Thinking & Problem-Solving Skills

  • Alysuliana Alias Department of Physics, Faculty of Science and Mathematics, Universiti Pendidikan Sultan Idris, 35900 Tanjong Malim, Perak, MALAYSIA
  • Lilia Ellany Mohtar Department of Physics, Faculty of Science and Mathematics, Universiti Pendidikan Sultan Idris, 35900 Tanjong Malim, Perak, MALAYSIA
  • Shahrul Kadri Ayop Department of Physics, Faculty of Science and Mathematics, Universiti Pendidikan Sultan Idris, 35900 Tanjong Malim, Perak, MALAYSIA
  • Fanny Rahmatina Rahim Department of Physics, Faculty of Mathematics and Natural Sciences, Universitas Negeri Padang, Jl. Prof Hamka, Padang 25131, INDONESIA
Keywords: instrument, CTPS skills, assessment, critical thinking and problem-solving

Abstract

Critical Thinking and Problem Solving (CTPS) are soft skills essential to be equipped among students according to 21st-century learning. Several instruments have been developed to measure CTPS skills among students in various subjects. This review aimed to identify the type of instrument and the context they are used. The research methodology was based on the systematic literature search in online databases, mainly in Scopus and ERIC, complemented by Google Scholar and MyCite. The keywords used during the searching process included “instrument”, “test”, “assessment”, “critical thinking”, and “problem-solving”. The screening process took into consideration the type of publication in which only the articles written in Malay or English language within the year 2017 to 2021 were considered. The result of this review revealed that the test, rubric, observation sheet, and questionnaire were the most used types of instruments. Meanwhile, the contexts of measurement were by group discussion, experimental report, quiz, and answer sheet.  This review can help educators and researchers make informed decisions about choosing the appropriate CTPS instrument and context for their assessments.

Downloads

Download data is not yet available.

References

World Economic Forum. (2020). The future of jobs report. https://www.weforum.org/reports/the-future-of-jobs-report-2020

Reynders, G., Lantz, J., Ruder, S. M., Stanford, C. L., & Cole, R. S. (2020). Rubrics to assess critical thinking and information processing in undergraduate STEM courses. International Journal of STEM Education, 7(9), 1–15. https://doi.org/10.1186/s40594-020-00208-5

Chan, C. K. Y., & Luk, L. Y. Y. (2020). Development and validation of an instrument measuring undergraduate students ’ perceived holistic competencies. Assessment & Evaluation in Higher Education, 46(3), 467–482. https://doi.org/10.1080/02602938.2020.1784392

Dwyer, C. P., Hogan, M. J., & Stewart, I. (2014). An integrated critical thinking framework for the 21st century. Thinking Skills and Creativity, 12(June 2014), 43–52. https://doi.org/10.1016/j.tsc.2013.12.004

Kementerian Pengajian Tinggi Malaysia. (2006). Modul pembangunan kemahiran insaniah (soft skills) untuk IPT Malaysia. Universiti Putra Malaysia.

Docktor, J. L., Strand, N. E., Mestre, J. P., & Ross, B. H. (2015). Conceptual problem solving in high school physics. Physical Review Special Topics - Physics Education Research, 11(2), 1–13. https://doi.org/10.1103/PhysRevSTPER.11.020106

Rear, D. (2019). One size fits all? The limitations of standardised assessment in critical thinking. Assessment & Evaluation in Higher Education, 44(5), 664–675. https://doi.org/10.1080/02602938.2018.1526255

Wren, D., & Cashwell, A. (2018). Mission Possible: Measuring Critical Thinking and Problem Solving. Educational Leadership, 75(5), 70–75.

Mueller, J. F., Taylor, H. K., Brakke, K., Drysdale, M., Kelly, K., Levine, G. M., & Ronquillo-Adachi, J. (2020). Assessment of Scientific Inquiry and Critical Thinking: Measuring APA Goal 2 Student Learning Outcomes. Teaching of Psychology, 47(4), 274–284. https://doi.org/10.1177/0098628320945114

Mabruroh, F., & Suhandi, A. (2017). Construction of Critical Thinking Skills Test Instrument Related The Concept On Sound Wave. Journal of Physics: Conference Series, 812(012056). https://doi.org/10.1088/1742-6596/755/1/011001

Alsaleh, N. J. (2020). Teaching Critical Thinking Skills: Literature Review. TOJET: The Turkish Online Journal of Educational Technology, 19(1), 21–39. http://www.tojet.net/articles/v19i1/1913.pdf

Larsson, K. (2021). Using Essay Responses as a Basis for Teaching Critical Thinking-a Variation Theory Approach. Scandinavian Journal of Educational Research, 65(1), 21–35. https://doi.org/10.1080/00313831.2019.1650824

Istiyono, E., Dwandaru, W. S. B., Lede, Y. A., Rahayu, F., & Nadapdap, A. (2019). Developing IRT-based physics critical thinking skill test: A CAT to answer 21st century challenge. International Journal of Instruction, 12(4), 267–280. https://doi.org/10.29333/iji.2019.12417a

Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Guidelines and guidance preferred reporting items for systematic reviews and meta-analyses:The PRISMA statement. PLOS Medicine, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097

Gusenbauer, M., & Haddaway, N. R. (2020). Which academic search systems are suitable for systematic reviews or meta-analyses ? Evaluating retrieval qualities of Google Scholar , PubMed , and 26 other resources. Research Syntheses Methods, 11(2), 181–217. https://doi.org/10.1002/jrsm.1378

Kurniawan, W., Pathoni, H., Muliawati, L., Kurniawan, D. A., Romadona, D. D., Ningsi, A. P., & Dari, R. W. (2020). Relationship of science process skills and critical thinking of students in physics Subject. Universal Journal of Educational Research, 8(11), 5581–5588. https://doi.org/10.13189/ujer.2020.081162

Priyaadharshini, M., & Sundaram, B. V. (2018). Evaluation of higher-order thinking skills using learning style in an undergraduate engineering in flipped classroom. Computer Applications in Engineering Education, 26(6), 2237–2254. https://doi.org/10.1002/cae.22035

Nonis, S. A., & Hudson, G. I. (2018). Developing and assessing critical thinking skills in marketing students : The power of making explicit problem-solving processes. Journal of Education for Business, 94(3), 195–203. https://doi.org/10.1080/08832323.2018.1504737

Dolapcioglu, S., & Doganay, A. (2020). Development of critical thinking in mathematics classes via authentic learning: An action research. International Journal of Mathematical Education in Science and Technology. https://doi.org/10.1080/0020739X.2020.1819573

Re, M. R., Amenduni, F., Medio, C. De, & Valente, M. (2019). How to use assessment data collected through writing activities to identify participants’ critical thinking levels. Journal of E-Learning and Knowledge Society, 15(3), 117–132. https://doi.org/10.20368/1971-8829/1135051

Kuang, C. Y., Kuen, Y. L., & Shu, F. C. (2017). The development and validation of a mechanical critical thinking scale for high school students. EURASIA Journal of Mathematics Science and Technology Education, 13(5), 1361–1376. https://doi.org/10.12973/eurasia.2017.00675a

Uvathi Mariappan, Arsaythamby Veloo, & Kanageswari Suppiah Shanmugam. (2018). Pembinaan dan pengesahan instrumen kemahiran insaniah bagi subjek perakaunan di matrikulasi. Jurnal Kurikulum & Pengajaran Asia Pasifik, 6(4), 29–39.

Aris Hanafi Abu Jalil, Gunathevan Elumalai, & Mohd Izwan Shahri. (2019). Pembinaan rubrik kemahiran insaniah bagi matapelajaran Kokurikulum pelajar program matrikulasi Kementerian Pendidikan Malaysia. Jurnal Penyelidikan Dedikasi, 17, 126–145.

Zainab Ab Rahman, Hoon, T. S., & Sidhu, G. K. (2019). Using the peer socratic questioning (PSQ) technique to develop critical thinking skills in group discussion. Jurnal Penyelidikan Tempawan, XXXVI, 91–103.

Tiruneh, D. T., Cock, M. De, Weldeslassie, A. G., Elen, J., & Janssen, R. (2017). Measuring critical thinking in physics : Development and validation of a critical thinking test in electricity and magnetism. International Journal of Science and Mathematics Education, 15(4), 663–682. https://doi.org/10.1007/s10763-016-9723-0

Lertyosbordin, C., Maneewan, S., Yampinij, S., & Thamwipat, K. (2019). Scoring rubric of problem-solving on computing science learning. International Education Studies, 12(8), 26–32. https://doi.org/10.5539/ies.v12n8p26

Bernstein, A. G., & Isaac, C. (2018). Critical thinking criteria for evaluating online discussion. The International Journal for the Scholarship of Teaching and Learning, 12(2), 1–8.

Sadhu, S., & Laksono, E. W. (2018). Development and validation of an integrated assessment for measuring critical thinking and chemical literacy in chemical equilibrium. International Journal of Instruction, 11(3), 557–572. https://doi.org/10.12973/iji.2018.11338a

Bandyopadhyay, S., & Szostek, J. (2019). Thinking critically about critical thinking: Assessing critical thinking of business students using multiple measures. Journal of Education for Business, 94(4), 259–270. https://doi.org/10.1080/08832323.2018.1524355

Azizi Alias, & Kamisah Osman. (2018). Pentaksiran alternatif: Pembinaan dan pelaksanaan rubrik dalam pendidikan Sains. Penerbit Universiti Kebangsaan Malaysia.

Huang, B., & Jong, M. S. Y. (2020). Developing a generic rubric for evaluating students’ work in STEM education. 2020 International Symposium on Educational Technology (ISET), 210–213. https://doi.org/10.1109/ISET49818.2020.00053

Boughey, S. (2019). Developing rubric to assess 3rd - 5th grade students understanding of science concepts via screencast models. Indiana University, Bloomington.

Tadele, K., & Sitotaw, B. (2018). Perception and trends in assessment of students’ learning in Physics courses. Latin-American Journal of Physics Education, 12(1), 1307–1315. https://doi.org/10.17576/jpen-2018-43.02-09

Fukuzawa, S., & DeBraga, M. (2019). Graded response method: does question type influence the assessment of critical thinking? Journal of Curriculum and Teaching, 8(1), 1–10. https://doi.org/10.5430/jct.v8n1p1

Poce, A., Medio, C. De, Amenduni, F., & Re, M. R. (2019). Critical thinking assessment : A first approach to the automatic evaluation. 18th International Conference on Information Technology Based Higher Education and Training (ITHET), 331–338.

Harahap, D. E., Festiyed, & Djamas, D. (2019). Preliminary study on development of assessment performance instruments on physics learning to improve students’ critical thinking ability. Journal of Physics: Conference Series, 1185(012128), 1–7. https://doi.org/10.1088/1742-6596/1185/1/012128

Tangkin, W. P. (2019). Pemanfaatan rubrik sebagai instrumen penilaian alternatif. Scholaria: Jurnal Pendidikan Dan Kebudayaan, 9(1), 29–39. https://doi.org/https://doi.org/10.24246/j.js.2019.v9.i1.p29-39

Brookhart, S. M. (2018). Appropriate criteria : Key to effective rubrics. Frontiers in Education, 3(22), 1–12. https://doi.org/10.3389/feduc.2018.00022

Afifah Fadhlullah, & Nurbarirah Ahmad. (2017). Thinking outside of the box: Determining students’ level of critical thinking skills in teaching and learning. Asian Journal of University Education, 13(2), 51–70.

Rathakrishnan, M., Ahmad, R., & Choo, L. S. (2017). Online discussion : Enhancing students’ critical thinking skills. AIP Conference Proceedings, 1891(1), 020120.

Beatty, P. C., Collins, D., Kaye, L., Padilla, J. L., Willis, G. B., & Wilmot, A. (2019). Advances in questionnaire design, development, evaluation and testing. John Wiley & Sons.

Batsell Jr, W. R., Perry, J. L., Hanley, E., & Hostetter, A. B. (2016). Ecological validity of the testing effect : The use of daily quizzes in introductory psychology. Teaching of Psychology, 1(6), 18–23. https://doi.org/10.1177/0098628316677492

Ahmad, A., & Awang, M. I. (2016). Pengukuran dan penilaian pendidikan. Dewan Bahasa dan Pustaka.

Tenam-Zemach, M., & Flynn, J. E. (2015). Rubric nation: Critical inquiries on the impact of rubrics in education. Information Age Publishing Inc

Published
2022-06-03
How to Cite
Alias, A., Mohtar, L. E., Ayop, S. K., & Rahim, F. R. (2022). A Systematic Review on Instruments to Assess Critical Thinking & Problem-Solving Skills. EDUCATUM Journal of Science, Mathematics and Technology, 9, 38-47. https://doi.org/10.37134/ejsmt.vol9.sp.5.2022