ISSN 2360-7963
Abstract
This study examines the use of generative artificial intelligence (AI) to assist in grading and feedback on undergraduate academic writing. While AI technologies in education have shown considerable potential, their adoption remains controversial due to contrasting evidence about their effectiveness. At a university in Hong Kong, an initiative was launched to use generative AI for providing formative feedback on students’ written drafts prior to final submission. This mixed-methods study evaluates the initiative by comparing AI-generated grades with those assigned by human instructors and analysing stakeholders' perceptions of the AI feedback. The research instruments used may partly explain the differences between the quantitative and qualitative research results. Quantitative analysis revealed a high inter-rater reliability between AI-generated scores and human-assigned grades, demonstrating the technical accuracy of the system. However, qualitative data from student interviews and instructor reflections revealed significant concerns about the AI’s ability to deliver contextually relevant, personalised, and pedagogically meaningful guidance. Stakeholders expressed scepticism and mistrust, emphasising the lack of depth, specificity, and adaptability in AI feedback as compared to human feedback. By exploring these discrepancies, this study underscores the tension between technical reliability and perceived pedagogical value in AI-assisted assessment. It advocates for stakeholder-informed, hybrid approaches that integrate AI with human oversight to ensure AI tools genuinely support student learning outcomes. To address these challenges effectively, future initiatives should prioritise collaborative design, transparent communication, and stakeholder education.
Keywords: artificial intelligence/English Language Teaching/feedback/grading/higher education, revision