Generative AI versus human conversational agent for reducing procrastination : a randomized pilot trial

Abstract

Objectives The use of generative artificial intelligence (AI) based chatbots in mental health interventions is rapidly evolving, but evidence of their efficacy and safety is scarce. This randomized pilot trial investigated the feasibility, efficacy, mechanisms, and unwanted effects of a generative AI chatbot (ChatGPT-4) vs. human conversational agent as a cognitive behavioral micro-intervention for procrastination.

Methods 62 university students (Mage 23.60 years, 93.5% female) were analyzed and assigned (concealed) to a three-session chat-based intervention with either the chatbot or human conversional agent. Outcomes were assessed using questionnaires at baseline, post-intervention, and 6-month follow-up. The primary outcome was self-reported procrastination. Secondary outcomes included depression, anxiety, worry, and rumination. Additionally, unwanted negative effects and common factors of therapeutic change were assessed.

Results Intention-to-treat analyses revealed a non-significant group × time interaction effect (p = .117, η2p = 0.05). Unwanted negative effects at post-assessment were reported by 43.1% of participants, with no significant group difference. Common factors were rated similarly across groups and did not mediate change in procrastination.

Conclusions Our findings do not indicate superiority of either conversational agent and both interventions had comparable, rather high rates of unwanted effects, indicating that more research in larger, clinically meaningful samples is needed to confirm the efficacy and safety of AI-based (micro-)interventions.