Can an artificial intelligence chatbot help people achieve goals in the same way as a human coach? Can AI coaching also improve wellbeing, resilience, and perceived stress? And what are the strengths and limitations of digital coaching compared with human coaching?
Research Purpose
To examine the effectiveness of an AI coaching chatbot called Vici. The researchers sought to determine whether using the chatbot could improve goal attainment, psychological wellbeing, and resilience, and reduce perceived stress compared with a control group that received no coaching.
Methodology: Randomised Controlled Longitudinal Trial (RCT)
The study included 169 undergraduate students in the United Kingdom. Participants were randomly assigned to either an experimental group that used the AI coaching chatbot for six months or a control group that did not use the chatbot. Measurements were collected at baseline, monthly over six months, and again three months after the intervention ended.
The chatbot was designed to help users define goals, create action plans, monitor progress, and receive feedback based on the GROW model and goal-setting theory.
Key Findings
Participants who used Vici showed greater improvement in goal attainment than the control group. The difference emerged after several months of use and remained evident three months after the study ended. In addition, more frequent use of the chatbot was associated with greater improvements in goal attainment.
However, no significant differences were found between the groups in psychological wellbeing, resilience, or perceived stress.
Conclusions and Recommendations
The researchers conclude that AI coaching can be effective when focused on a clearly defined objective such as goal attainment and progress monitoring. However, this form of coaching does not appear to produce the broader outcomes often associated with human coaching, such as improvements in wellbeing, emotional resilience, or stress reduction.
The researchers recommend that future studies examine hybrid models that combine human coaches and AI systems, as well as AI coaches designed to target outcomes beyond goal attainment.
Importance for Coaches
This is one of the first experimental studies to rigorously examine the effectiveness of an AI coach. The findings suggest that AI may become a useful tool for expanding access to coaching and helping people achieve goals at scale. At the same time, the study highlights the importance of human elements in coaching, including relationship, empathy, and contextual understanding, which were not reflected in the AI outcomes.
Terblanche, N., Molyn, J., De Haan, E., & Nilsson, V. O. (2022). Coaching at Scale: Investigating the Efficacy of Artificial Intelligence Coaching. International Journal of Evidence Based Coaching and Mentoring, 20(2), 20–36.