YW.

AI Chatbot & Wellness

wb-img

Overview

Mental health concerns are rising, particularly among post-secondary students, who may lack access to traditional therapeutic resources due to barriers like long wait times and high costs. To help address these challenges, we explored the potential of large language model-based chatbots for supporting mental health and wellbeing in student populations.

  • Study 1 uses an A/B testing approach to examine the benefits of providing personalized training. It assesses whether a chatbot platform (Group B) outperforms the traditional survey platform, Qualtrics (Group A), in enhancing mental health outcomes.
  • Study 2 involves comparing two LLM-powered chatbot conditions to a control group. Within the two LLM conditions, we conducted an A/B test by varying the training materials to determine which approach was more effective in guiding participants to apply mindfulness techniques to improve wellbeing and coping abilities.

    • Study 1 Results

    Participants randomly assigned to the Customized Chatbot condition (Group B) experienced improved positive reappraisal (β=.03, 95% CI[.01, .05], p=.001) and wellbeing (β=.02, 95% CI[.00, .03], p=.005) compared to the other conditions (control condition and Group A).

    mindfuladventure-img

    Positive feedback was also received from participants in the Customized Chatbot condition. I conducted sentiment analysis using R and included a few examples of feedback along with their respective sentiment scores.

    User Feedback Sentiment
    The chatbot was a very fun touch and it was very easy to use. 0.841872912
    I loved the chatbot it was very entertaining and it allowed me to focus more 0.756522747
    I really liked how the chatbot would say "I'm glad you felt happy" whenever I ranked my happiness high. It was really nice :) 0.701037648
    I like the chatbot with the cute little robot; it made me me smile and make me feel like someone is actually texting me. It adds a personal touch, and navigating through the chatbot was simple, which is a plus. 0.454786194
    I liked the design of the chatbot, and I also liked the messages it sent. The messages sounded like they actually care. 0.4125728
    It felt very supportive and simple to use. I enjoyed the personal connection I felt with it even though it's only AI 0.37227916
    I liked how it responded to what we choose and from our answers, choose the best option for us to do. 0.305505046
    I liked that it feels like you are chatting with someone 0.301511345

    Study 2 Results

    Participants randomly assigned to the two LLM-powered chatbot conditions experienced increase positive mood (β=.16, 95% CI[.03, .28], p=.018) and reduced stress (β=-.18, 95% CI[-.30, -.05], p=.005) compared to the control condition. The mindfulness chatbot condition outperformed the value chatbot condition for improving participants' perspective taking skills (β=.02, 95% CI[.00, .03], p=.010).

    domain_chatbot-img domain_chatbot-img