Creating a software application to help university educators to reflect on the cognitive complexity of their exam questions, using Bloom’s Taxonomy and automated classification

Abstract

Previous research has shown that many university educators struggle to accurately evaluate the cognitive complexity of exam questions (and overall exams) which they write, based on Bloom's Taxonomy. This can lead to concerns about the design of exams. Software tools could possibly assist educators via automated classification methods. This paper reports a work-in-progress project that is creating a software application (tool) to assist university educators with writing exams. We evaluate 3 methods of automated classification including keywords-based approach, OpenAI evaluation, and an existing algorithm. The tool is designed to be able to help educators to reflect on their exam, by providing educators with meaningful feedback on question complexity and the overall exam, assisting in exam design. The software tool developed in this study is expected to benefit educators by providing objective feedback and serving as a professional development resource.