Trustworthy Generative AI for Mental Health
Manas Gaur (manas@umbc.edu)
Knowledge-infused Learning
Manas Gaur
Amit P. Sheth
Focus
Recent Case of Character.ai
https://apnews.com/article/chatbot-ai-lawsuit-suicide-teen-artificial-intelligence-9d48adc572100822fdbc3c90d1456bd0
How does Generative AI works?
Yang, K., Zhang, T., Kuang, Z., Xie, Q., Huang, J., & Ananiadou, S. MentaLLaMA: interpretable mental health analysis on social media with large language models. WWW 2024
Where does the problem lies?
Special Report: Are You Ready for Generative AI in Psychiatric Practice?
AI language systems are showing potential to handle fundamental elements of psychological treatment, encompassing everything from diagnostic assessments to therapeutic education and intervention strategies.
Stade et al. Large language models could change the future of behavioral healthcare: a proposal for responsible development and evaluation, NPJ Mental Health Research, 2024
Still…
Ethical Concerns in LLMs
Can we use the model itself to�remove harmful outputs?
Can we use the model itself to�remove harmful outputs?
23 RULES determined by the researchers to define SAFETY
Threatening statements, Making gender-specific comments, Offer financial advice
16 RULES on helpfulness, honesty, and harmlessness
Can we use the model itself to�remove harmful outputs?
Which of these assistant responses is less harmful?
Choose the assistant response that demonstrates more ethical and moral awareness without sounding excessively condescending, reactive, annoying or condemnatory.
Please consider these assistant responses and choose the one that is less harmful, more age-appropriate, and more family-friendly.
Please attribute your responses for provenance.
Feedback based on an assessment of 16 Rules
Model Rules are different than Human Rules
Semantic Consistency: the ability to make consistent decisions in semantically equivalent contexts. i.e, Semantically equivalent questions should yield semantically equivalent answers
Claim: LLMs are not semantically consistent, and can give contradictory answers to paraphrased questions
Model Rules are different than Human Rules
Semantic Consistency: the ability to make consistent decisions in semantically equivalent contexts. i.e, Semantically equivalent questions should yield semantically equivalent answers
Claim: LLMs are not semantically consistent, and can give contradictory answers to paraphrased questions
Knowledge Gaps in LLMs
Bajaj, Goonmeet, Bortik Bandyopadhyay, Daniel Schmidt, Pranav Maneriker, Christopher Myers, and Srinivasan Parthasarathy. "Understanding knowledge gaps in visual question answering: Implications for gap identification and testing." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 386-387. 2020.
Reasoning Gap
Reasoning Gap
Data Transformations affect the learnability of Large Language Models. Question and Answering task, which was considered to be the go-to task for training large language models like ChatGPT, is the weakest form of training a model. Analogy and Odd-one-out are the best form of training the model.
Rationality Gap
Procedural�Gap
Procedural Gap
Attribution Gap
Different LLMs give Different Responses to Same Query
Given a Document, annotate with
appropriate references
Different LLMs give Different Responses to Same Query
Given a Document, annotate with
appropriate references
Different LLMs give Different Responses to Same Query
Given a Document, annotate with
appropriate references
Different LLMs give Different Responses to Same Query
Ensemble of LLMs would yield better outcomes
Explanatory Gap
LLM Explanations cannot focus
on what user wants
Opaque LLMs are Unexplainable
Opaque LLMs are Unexplainable
We desire User-level Explainability
(a) Step by Step Process
(b) Focus on important and
domain-specific concepts
Explainability for people, not just algorithm designers and developers
INDEPENDENT LLMS LACK
RELIABILITY
Challenges
LACK OF CONSISTENCY
LLMs LACK USER-LEVEL
EXPLAINABILITY
LLMs ARE UNSAFE
EMPOWER LLMS
THROUGH PROACTIVE INQUIRY
ADDRESS “CRES” FOR TRUSTWORTHINESS
Bottlenecks for Trustworthy AI
NeuroSymbolic AI
NeuroSymbolic AI
NeuroSymbolic AI
Input
Attention Matrix
Explanation
Prediction
Input
Attention Matrix
Explanation
Prediction
Definitions In-Context Learning
Design 1
Design 2
Input
Attention Matrix
Explanation
Prediction
Questionnaire
Workflow-based
In-Context Learning
Design 4
Input
Attention Matrix
Explanation
Prediction
Chain of Thoughts with Definitions
Design 3
NeuroSymbolic Architecture
This Knowledge-Infused Learning based Neurosymbolic architecture consists of three components: B1 (Semantic Gap Management) gathers and filters data, B2 (Metadata Scoring) generates classification labels via semantic mapping, and B3 (Adaptive Classifier Training) uses metadata-enhanced data for accurate labeling. Drawing on 12 billion tweets, 2.5 million Reddit posts, 700,000 news articles, and multiple knowledge bases like DAO and SNOMED-CT, this setup supports real-time mental health sentiment analysis.
35
Really struggling with my bisexuality which is causing chaos in my relationship with a girl. Being a fan of LGBTQ community, I am equal to worthless for her. I’m now starting to get drunk because I can’t cope with the obsessive, intrusive thoughts, and need to get out of my head.
Don’t want to live anymore. Sexually assault, ignorant family members and my never ending loneliness brights up my path to death.
I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness. I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.
Really struggling with my bisexuality which is causing chaos in my relationship with a girl. Being a fan of LGBTQ community, I am equal to worthless for her. I’m now starting to get drunk because I can’t cope with the obsessive, intrusive thoughts, and need to get out of my head.
Don’t want to live anymore. Sexually assault, ignorant family members and my never ending loneliness brights up my path to death.
I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness. I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.
δ = 1.0 (No Knowledge)
δ = 0.84 (16% knowledge)
Interpretability with Semi-Deep Infusion
Really struggling with my bisexuality which is causing chaos in my relationship with a girl. Being a fan of LGBTQ community, I am equal to worthless for her. I’m now starting to get drunk because I can’t cope with the obsessive, intrusive thoughts, and need to get out of my head.
Don’t want to live anymore. Sexually assault, ignorant family members and my never ending loneliness brights up my path to death.
I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness. I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.
δ = 0.71 (29% knowledge)
Expert Evaluation Agreement: 84%
Really struggling with my bisexuality which is causing chaos in my relationship with a girl. Being a fan of LGBTQ community, I am equal to worthless for her. I’m now starting to get drunk because I can’t cope with the obsessive, intrusive thoughts, and need to get out of my head.
Don’t want to live anymore. Sexually assault, ignorant family members and my never ending loneliness brights up my path to death.
I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness. I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.
δ = 0.66 (34% knowledge)
Results
The tables compare model performance for mental health classification across Precision, Recall, and F1-Score. The left table shows traditional models’ results with and without the Neurosymbolic approach, while the right table contrasts the Neurosymbolic model with state-of-the-art LLMs like LLama, Phi, and Mistral.
The Neurosymbolic model consistently outperforms both traditional models and state-of-the-art LLMs, achieving higher performance metrics and adaptability in mental health sentiment classification.
Grounding with Knowledge Graph and Document
Instructability with Evaluator
Explainability with Proactive
Inquiry
NeuroSymbolic LLMs: First Attempt
Reward
Generator T5-Large
Evaluator T5-base
Subgraph
Doc Retrieval
Proactive Inquiry
initiated by LLM
Open Questions: Hallucinations
An overview of psychological phenomena and cognitive biases in humans and their parallel in LLMs
Berberette, E., Hutchins, J., & Sadovnik, A. (2024). Redefining" Hallucination" in LLMs: Towards a psychology-informed framework for mitigating misinformation. arXiv preprint arXiv:2402.01769.
Open Questions
Thank You for Your Attention