The Effect of Artificial Intelligence Tools on EFL Students’ Writing Skills: A Mixed-Methods Study at Universitas Panca Sakti Bekasi

Although artificial intelligence (AI) has increasingly been incorporated into EFL writing instruction, previous research has largely focused on overall writing performance, with less attention to changes across specific writing dimensions and students’ actual experiences of using AI during the writing process. This study examined the effect of AI-assisted instruction on EFL students’ writing performance, with particular attention to five writing dimensions—content, organization, vocabulary, language use, and mechanics and explored how students used and experienced AI during writing. Using a quasi-experimental explanatory sequential mixed-methods design, the study involved 60 English Language Education students at Universitas Panca Sakti Bekasi, equally divided into experimental and control groups. The experimental group engaged in AI-assisted writing activities integrated into regular writing tasks, while the control group received conventional instruction. Pre- and post-test writing scores were analyzed using descriptive statistics and ANCOVA, complemented by semi-structured interviews and AI Usage Logs. The experimental group demonstrated significantly greater improvement in overall writing performance than the control group (F = 381.24, p < .001, partial η² = .870), with significant differences across all five dimensions. Language use showed the largest raw gain, whereas mechanics yielded the largest effect size. AI Usage Logs further showed that students primarily used AI for language feedback, idea generation, vocabulary enrichment, and revision. The findings indicate a substantial effect within the specific instructional context and demonstrate the value of examining writing outcomes alongside students’ actual AI-use practices.