Can AI Replace Human Insight? Evaluating the Quality of Feedback from Human Tutors Verus ChatGPT on EFL Students' Argumentative Writing

Since the emergence of AI models, concerns have grown about their potential capabilities and associated risks. This study investigates the role of a feedback-rich environment in improving learning outcomes and examines the impact of generative AI (ChatGPT) in providing formative feedback on argumentative writing by EFL students. Research has consistently shown that timely and constructive feedback is crucial for learning (Mackey, 2020). However, educators often face challenges in managing large classes and delivering personalized feedback, which is key to supporting inclusive instruction. This study addresses the question, "Can AI replace human insights?" by comparing the quality of feedback provided by human tutors and ChatGPT across 200 essays written by 50 EFL students in Malaysia. The research focuses on first- and second-year learners whose first language is not English. The study follows a mixed-methods approach, conducted in two phases: interviews with learners and an assessment of their writing progress at the end of a 15-session argumentative writing course. Participants were assessed before and after the course, with one group receiving AI-generated feedback and the other receiving feedback exclusively from human tutors. Besides, the thematic analysis of interviews was run qualitatively using NVivo to explore student attitudes, preferences, barriers, and challenges, and identify key patterns. Additionally, pre- and post-test writing scores were evaluated using the IELTS writing rubric by two independent raters, with an inter-rater reliability of 0.9. The findings suggest that feedback from human tutors is consistently supportive, sensitive to students’ backgrounds and experiences, and effective in fostering engagement. However, there were instances where human feedback exhibited bias, either contextual or by subtly pressuring students to conform to tutors' preferences. In contrast, AI-generated feedback adhered more closely to educational standards, offering clearer, more specific, and criteria-based suggestions for improvement. In terms of student progress, no significant difference was observed in the overall post-test results between the two groups; both groups demonstrated improvement. However, the human feedback group showed greater gains in idea presentation, organization, and exemplification, whereas the AI feedback group saw more progress in vocabulary use and sentence structure. One notable concern is that human raters assessed the post-test essays. Despite the essays being anonymized and randomly distributed, the essays from the human feedback group consistently received higher ratings, suggesting that human feedback may have contributed to a more personalized and humanized writing style, which caught the attention of the raters. Looking forward, the integration of AI in educational practices has the potential to transform both instruction and assessment. The ability of AI to provide formative feedback reflects a shift in instructional methods, making AI a valuable supplement in diverse learning environments. Furthermore, the way feedback is delivered can influence learners’ understanding and performance, extending the implications into assessment practices. Future research should explore the broader applications of AI in education, examining its role not only in instructional support but also in assessment to optimize its benefits while addressing its limitations.