1

​

Ph.D. Dissertation Defense

Knowledge-infused Learning

​

Manas Gaur

mgaur@email.sc.edu

Artificial Intelligence Institute

Department of Computer Science and Engineering

University of South Carolina

Advisor:

Dr. Amit P. Sheth

Committee Members

Dr. Biplav Srivastava

Dr. Krishnaprasad Thirunarayan

Dr. Valerie L. Shalin

Dr. Jyotishman Pathak

Dr. Pooyan Jamshidi

Dr. Vignesh Narayanan

Dr. Lorne Hofseth

​

March 25, 2022

2

A bit about me and my Ph.D. journey

M. Gaur (2022)

Netaji Subhas University of Technology

2013

B.S, Computer Science

Advisor: Dr. Ram Shringar Raw

Delhi Technological University

2015

M.S, Software Engineering

Advisor: Dr. Vinod Kumar Panchal

Dr. Talel Abdessalem

Dr. Petko Bogdanov

UofSC CS and AI Institute

Advisor: Dr. Amit P. Sheth

​

“You have to work harder than I do”

3

​

​

Knowledge-infused Learning

​

Manas Gaur

Ph.D. Defense

4

Goal:

Use Human Knowledge to make AI Explainable and Interpretable.

Vision:

We want to make next generation neuro-symbolic AI approach

inspired by human’s ability to combine data and knowledge.

M. Gaur (2022)

5

M. Gaur (2022)

A DARPA Perspective on Artificial Intelligence

6

M. Gaur (2022)

Low-level Data

Sensors, Text, Image, and Collection

Neural Network and Deep Learning

Decisions/Actions

System 1

Statistical AI is a Black Box

Knowledge Graph (Labeled Nodes and Edges)

Symbolic Reasoning

System 2

Neural Network and Deep Learning

Decisions/Actions

System 1

Low-level Data

Sensors, Text, Image, and Collection

7

M. Gaur (2022)

Can a model capture context and handle uncertainty in input?

​

Can user-level explanations be obtained from the success or failure of an AI model?

​

Can we control an AI model by making it interpretable?

​

How can we make an AI model self-explainable?

​

Statistical AI alone is not enough!!

8

M. Gaur (2022)

Sheth and Thirunarayan, Duality of Data and Knowledge, IEEE Computer Society 2021

Low-level Data

Sensors, Text, Image, and Collection

Neural Network and Deep Learning

Knowledge Graph (Labeled Nodes and Edges)

Symbolic Reasoning

Decisions/Actions

System 1

System 2

Neural Network and Deep Learning

Decisions/Actions

System 1

Low-level Data

Sensors, Text, Image, and Collection

9

M. Gaur (2022)

History

McCarthy and Hayes - 1968

Some Philosophical Problems from the Standpoint of Artificial Intelligence

Douglas Hofstadter - 1979

Gödel, Escher, Bach

Amit Sheth -

2001: World Model and its utility in enhancing search, personalization, and profiling

2002: Semantic Enhancement Engine to improve information retrieval using ontologies

2005: Semantics for the semantic web: The implicit, the formal and the powerful

2010: Computing for Human Experience

​

Leslie Valiant - 2006

Robust logics

How machines can acquire and manipulate commonsense knowledge ?

Daniel Kahneman - 2011

Thinking Fast and Slow

Stitching System 1 and System2 : NeuroSymbolic AI

Amit Sheth - 2017

Knowledge will propel machine understanding of content; Semantic-Cognitive-Perceptual Computing

Gary Marcus - 2019

Rebooting AI

AI need a hybrid “Knowledge-driven” approach

10

Knowledge Graphs (KG)

M. Gaur (2022)

1. Machine readable structured representation of knowledge

​

• Consisting of entities, entity types, and relationships in various forms (e.g., labeled property graphs and RDFs).

​

Speer et al. AAAI’17

Vrandečić et al. ACM Comm’14

Gaur et al. ICSC’19

Miller, ACM Comm’95

ConceptNet

World War I fought_with Poisonous Gas

Subject

Predicate

Object

11

Knowledge Graphs (KG)

M. Gaur (2022)

Question

Response

With KG you can answer any open domain question.

12

M. Gaur (2022)

Statistical AI alone is not enough!!

Can a model capture context and handle uncertainty in input?

​

Can user-level explanations be obtained from the success or failure of an AI model?

​

Can we control an AI model by making it interpretable?

​

How can we make an AI model self-explainable?

​

13

M. Gaur (2022)

Questions are Assessors

14

M. Gaur (2022)

Scope (Data and Method)

Reddit

Mental Health

(2005-2018)

>13 Million posts

>2 Million Users

Twitter

(Event-Specific)

>80 Million tweets

> 15 Million Users

(Aggregated)

Clinical Diagnostic Interviews

180 Patients

60 minutes interview

(manually transcribed)

>13.8 Million clinical notes on >124K patients with mental health conditions

Heterogenous Dialog datasets with >100K dialogs on Politics, Travel, News, Mental Health, and Geography

Classification Models

Generative Models

15

M. Gaur (2022)

Attention Neural Network

(Vaswani et al. NIPS’17)

Large Amount of Textual Data

Objective Function:

Word co-occurrences

Standard Training Process of Language Model

AI in Natural Language Understanding (NLU)

16

M. Gaur (2022)

Attention Neural Network

(Vaswani et al. NIPS’17)

Large Amount of Textual Data

AI in NLU

Foundational Models:

​

1. Transformers

(Vaswani et al. NIPS’17)

• Autoencoders

(Kramer AIChE’91 and Hinton & Salakhutdinov. Science’04)

• Recurrent Neural Networks (Williams et al. Nature’86)
• Long Short Term Memory (Hochreiter & Schmidhuber Neural Computation’97)

17

{AI + NLU} for Classification Task

What would have happened if Facebook was present in World War I ?

What would have happened if Facebook was present in World War II ?

What

Happened

Facebook

World

War

I

SIMILAR

What

Happened

Facebook

World

War

II

M. Gaur (2022)

https://gluebenchmark.com/

(Example from Quora Question Pairs Dataset)

Similarity computing function

Probability distribution of a sentence

18

What would have happened if Facebook was present in World War I ?

World War I <fought_with> Trenches

World War I <fought_with> Poisonous Gas

World War I <fought_with> Guns

Trenches

Poisonous

Gas

Guns

What

Happened

Facebook

World

War

I

M. Gaur (2022)

{AI + Knowledge + NLU} for Classification Task

External knowledge in the form of

<Subject><Predicate><Object>

19

What would have happened if Facebook was present in World War II ?

World War II <fought_with> Ships

World War II <fought_with> Fighter Planes

World War II <fought_with> Tanks

Ships

Fighter

Planes

Tanks

What

Happened

Facebook

World

War

I

M. Gaur (2022)

{AI + Knowledge + NLU} for Classification Task

20

Comparison with & without Knowledge

SIMILAR

Without generic world knowledge

DIFFERENT

With generic world knowledge

M. Gaur (2022)

(BERT) Neural Language Model without Knowledge on Quora Question Pairs

Neural Language Model with Knowledge on Quora Question Pairs (KI-BERT)

​

[Faldu et al. 2021]

70%

72%

%age correct classification

21

M. Gaur (2022)

Context Sensitivity:

​

Two sentences are different because of the concepts World War I and World War II

Bottom Line

Bottom Line: With Knowledge, AI can be made sensitive towards a context.

SIMILAR

Without generic world knowledge

DIFFERENT

With generic world knowledge

22

M. Gaur (2022)

Why is it Important: Complex Environment

Kursuncu and Gaur et al. CSCW’19

“#MyJihad is to my prayer for mother, then father, then god, then other relatives in …”

“I asked about the paths to Paradise It was said that there is no path shorter than Jihad; killing of apostates”

Non-Extremist

Extremist

Extremist

Extremist

23

M. Gaur (2022)

Why is it Important

Kursuncu and Gaur et al. CSCW’19

“#MyJihad is to my prayer for mother, then father, then god, then other relatives in …”

“I asked about the paths to Paradise It was said that there is no path shorter than Jihad; killing of apostates”

24

M. Gaur (2022)

Why is it Important

Kursuncu and Gaur et al. CSCW’19

paradise

killing

Hate attacks

violence

Context understanding

Deeper Semantics

jihad

paradise

god

prayer

jihad

Recall

Baseline

Religion

+

Ideology

Religion

+

Hate

Hate

+

Ideology

All Three

82%

82%

85%

88%

94%

25

“#MyJihad is to my prayer for mother, then father, then god, then other relatives in …”

“I asked about the paths to Paradise It was said that there is no path shorter than Jihad; killing of apostates”

paradise

killing

Hate attacks

violence

Context understanding

Deeper Semantics

jihad

paradise

god

prayer

jihad

26

AI for NLU in a Generative Task

Attention Neural Network

(Vaswani et al. NIPS’17)

Large Amount of Textual Conversational Data in General Domain

M. Gaur (2022)

Trained Generative Model

27

{Statistical AI + NLU} for Sentence Generation

Do you feel nervous?

More than half the days

Do you feel irritated or self destructive?

Do you feel something extreme might happen to you?

Are you able to relax?

The model can generate and ask either of these questions

They are either bad questions or irrelevant

( A clinician won’t ask either of these)

M. Gaur (2022)

A model trained to asked questions

Risky

28

{Statistical AI + NLU} for Sentence Generation

Do you feel nervous?

More than half the days

Do you feel irritated or self destructive?

Do you feel something extreme might happen to you?

Are you able to relax?

The model can generate and ask either of these questions

M. Gaur (2022)

A model trained to asked questions

29

{Statistical AI + Knowledge + NLU} for Generation

Do you feel nervous?

More than half the days

Do you feel irritated or self destructive?

Do you feel something extreme might happen to you?

Are you able to relax?

M. Gaur (2022)

Do you feel nervous?

More than half the days

Do you feel Irritated?

Are you bothered by becoming easily annoyed or irritable?

Are you bothered by any relaxation troubles?

Knowledge Infusion using Medical Questionnaire (Med_Q)

These questions are medically valid and safe.

Roy and Gaur et al. ACL’22

Safety Checks

30

M. Gaur (2022)

Do you feel nervous?

More than half the days

Attention Model

(Vaswani et. al. NIPS’18)

T5

(Raffel et. al. ACL’20)

KI LSTM

(Ours)

KI Attention

Model

(Ours)

30.6%

17.1%

10.6%

13.3%

KI: Knowledge Infusion

Percentage of unsafe generated questions

{Statistical AI + Knowledge + NLU} for Generation

Do you feel Irritated?

Are you bothered by becoming easily annoyed or irritable?

Are you bothered by any relaxation troubles?

Roy and Gaur et al. ACL’22

31

M. Gaur (2022)

Knowledge

If is cause then symptom

​

If is symptom then medication

​

If is medication then treatment

Probability next question generation is

Process

Safety Checks

Do you feel nervous?

More than half the days

Do you feel Irritated?

Are you bothered by becoming easily annoyed or irritable?

Are you bothered by any relaxation troubles?

32

Bottom Line

M. Gaur (2022)

Knowledge

Uncertainty and Risk:

​

Model can sense:

1. When the generated question is unsafe
2. When the generated question is safe

Do you feel nervous?

More than half the days

Do you feel Irritated?

Are you bothered by becoming easily annoyed or irritable?

Are you bothered by any relaxation troubles?

33

Stitching the Classification and Generation Tasks

M. Gaur (2022)

Context Sensitivity

Classification Task

Generation Task

Handling Uncertainty or Risk

Human Annotation

Experience

(e.g. History)

Web Search

Corpus

Expert Guidelines

Generative Output

Classification Output

Labeled Dataset

34

Stitching the Classification and Generation Tasks

M. Gaur (2022)

Context Sensitivity

Classification Task

Generation Task

Handling Uncertainty or Risk

Generative Output

Classification Output

Labeled Dataset

Experience

(e.g. History)

Web Search Corpus

Expert Guidelines

Interpretability

Experience

(e.g. History)

Web Search Corpus

Expert Guidelines

User-level Explainability

matching

35

Stitching the Classification and Generation Tasks

M. Gaur (2022)

Context Sensitivity

Classification Task

Generation Task

Handling Uncertainty or Risk

Generative Output

Classification Output

Labeled Dataset

Interpretability

User-level Explainability

Information graph (KG, Lexicons, etc.)

Kashyap and Sheth CIKM’94

36

What do we learn from these Examples?

M. Gaur (2022)

Context Sensitive Capture:

Statistical AI is opinionated based on the text it sees and input is partial representation of the world.

Uncertainty and Risk:

Statistical AI, fail to establish the connection between input and output

User-level Explainable:

Statistical AI’s explanations are system-oriented and not rich enough for user-level understanding.

Interpretable:

A Statistical AI model that you can understand and control

Transferable:

Statistical AI learns the data and not the task

Knowledge can highlight the context in input.

Knowledge can assess risky prediction

Knowledge can mend the focus of statistical AI.

Knowledge can enable User-level explanations

Knowledge can help in generalize across tasks

37

Assessment

M. Gaur (2022)

38

Thesis Statement : Knowledge-infused Learning (KiL)

M. Gaur (2022)

Knowledge-infused Learning is a class of Neuro-Symbolic AI techniques that incorporate broader forms of knowledge (lexical, domain-specific, common-sense, and constraint-based) into addressing limitations of either symbolic or statistical AI approaches, such as model interpretations and user-level explanations. Compared to powerful statistical AI that exploit data, KiL benefit from data as well as knowledge.

39

Types of Knowledge-infused Learning

M. Gaur (2022)

Shallow Infusion : Navigating the black box from Outside

1. Capture Context : 45-89% better over State-of-the-art [Gaur et al. AAAI’20, ICHI’21]
2. Minimize uncertainty in model by 61-83% [Gaur et al. WWW’19, PLoS’21, ACL’22]
3. Post-Hoc Explainability [Kursuncu and Gaur et al. CSCW’19, Gaur et al. AMIA’21]

Semi-Deep Infusion : Navigating the black box from Inside

• Capture Context : 61-94% better over State-of-the-art [Gaur et al. CIKM’18, JMIR’21]
• Minimize uncertainty in model by 47-84% [Gaur et al. DSAA’18, Roy and Gaur et al. IJCAI’22*, ACL’22]
• Explainability [Gaur et al. AAAI’22, Roy and Gaur et al. IJCAI’22, ACL’22]
• Interpretability [Kursuncu and Gaur et al. CSCW’19, Roy and Gaur et al. IJCAI’22, ACL’22]

* submitted

40

Characteristics (X) - Contributions (Y) Graph

M. Gaur (2022)

Entity Normalization

Retrieval and Ranking

Optimization

Loss Function

Evaluation Metrics

Context Sensitive

Uncertainty and Risk

Interpretability

User-level Explainability

Transferability

ICHI’21

PLoS’21

ICHI’21

CIKM’18

WWW’19

PLoS’21

ACL’22

WWW’19

PLoS’21

CIKM’18

AAAI’22

DSAA’18

JMIR’21

AAAI’20

AAAI’20

ACL’22

JMIR’21

AAAI’22

CSCW’19

AAAI’20

AMIA’21

AAAI’22

AAAI’22

Ontology/KG

ISWC’18

ICSC’19

ISWC’18

ICSC’19

ISWC’18

ICSC’19

41

From GLUE to KILU: Application Contribution

M. Gaur (2022)

Corpus of Linguistic Acceptability

Summarizing Clinical Interviews

DSM-5 and PHQ-9

Flesch Reading, Divergence, Theme Overlap

Matthew Correlation

Rich Evaluation metrics

Stanford Sentiment Treebank

Assessing Severity in User-generated Content

Ordinal Error, Perceived Risk Measure, Ranked Precision/Recall

DSM-5 and Drug Abuse Ontology

Accuracy

Question NLI

ConceptNet and WordNet

Concept Mover Distance, BLEURT

F1-Score and Accuracy

User-language Paraphrase Corpus

Microsoft Paraphrase Corpus

Recognizing Textual Entailment

Conversational Information Seeking

Process Knowledge NLG

Gaur et al. JMIR’21

Gaur et al. Pone’21, WWW’19, ACL’ 22

Gaur et al. AAAI’22

Roy & Gaur et al. ACL

ConceptNet, WikiNews, Wikipedia , MS-MARCO

Logical Coherence, Semantic Relevance, BLEURT

Accuracy

PHQ-9, GAD-7, C-SSRS

Accuracy

Avg. # Unsafe Matches, Avg. #KG concept Matches,

Avg. Sq. Rank Error

Reagle & Gaur FirstMonday’22

Sellam et al. BLEURT ACL’20, Nguyen et al. MS MARCO, NIPS’16,

Kusner et al. Word Mover ICML’15, Cameron et al. PREDOSE JBI’13

General Language Understanding Evaluation (GLUE) (Wang et al. ICLR’19)

Knowledge-intensive Language Understanding (KILU)

(Sheth and Gaur IEEE IC’21)

Modified and Adapted from McCarthy et al. IGI’12

42

M. Gaur (2022)

1. Knowledge-infusion for Suicide Risk Classification

II. Knowledge-infusion for Language Generation

43

M. Gaur (2022)

• Knowledge-infusion for Suicide Risk Classification

II. Knowledge-infusion for Language Generation

44

M. Gaur (2022)

No Risk

Suicidal

Ideation

Suicidal

Behaviors

Suicidal Attempt

7%

12%

25%

37%

Probability of Admission to Hospital

​

• 80% of the patients suffering from Borderline Personality Disorder have suicidal behavior.

​

• 5-10% of whom commit suicide.

​

• Individuals may attempt to conceal suicidal thoughts in Clinical Settings

​

• They express freely on Social Media
• r/SuicideWatch

341, 000 Subscribers

​

• Current models that predict Suicide Risk are not clinically grounded and explainable

Why Suicide Risk Classification?

Individuals with Borderline Personality Disorder (2019)

Veen et al. BMC’19

45

M. Gaur (2022)

How does annotator look at a user’s post in r/SuicideWatch

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

Label: Moderate Risk

​

( UMD Suicidality Dataset

Shing et al. CLPsych’18 )

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

Annotator Perspective for Moderate Risk

46

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

Input Raw Text

Annotated by Expert

(Label: Moderate Risk)

Annotated by SVM

(Label: Low Risk)

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

47

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

Annotated by Expert

(Label: Moderate Risk)

Annotated by SVM

(Label: Low Risk)

Annotated by CNN

(Label: Low Risk)

48

M. Gaur (2022)

How does model look at a user’s post?

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

Annotator Perspective for Moderate Risk

​

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

Model’s Perspective for Low Risk

​

​

49

M. Gaur (2022)

Starting simple

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

Model’s Perspective for Low Risk

​

53%

SVM-L

Recall

50

M. Gaur (2022)

A complex model

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

Model’s Perspective for Low Risk

​

53%

SVM-L

Recall

Shing et al. CLPsych’18

57%

CNN

Kim EMNLP’14

51

M. Gaur (2022)

Story always starts with a POST

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

Model’s Perspective for Low Risk

53%

SVM-L

Recall

Shing et al. CLPsych’18

57%

CNN

Kim EMNLP’14

CNN + GL

Sawhney et al. ACL’22

62%

Gambler Loss

Ziyin et al. NIPS’19

52

M. Gaur (2022)

Something is wrong with the labels

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

Model’s Perspective for Low Risk

​

No Risk

Low Risk

Moderate Risk

Severe Risk

Without Definitions these labels are subject to different interpretations/thresholds

Suicide Indication

Suicide Ideation

Suicide Behavior

Suicide Attempt

Columbia Suicide Severity Rating Scale

Posner et al. Columbia Univ’08

Concept Classes

53

M. Gaur (2022)

Shallow Infusion: Locate and Embed Concept Phrases

Suicide Indication

Suicide Ideation

Suicide Behavior

Suicide Attempt

I do have a potential to live a decent life but not with people who [abandon me]. [Hopelessness] and [feelings of betrayal] have turned my [nights to days]. I am developing [insomnia] because of my [restlessness].

I just [can’t take it anymore]. Been [abandoned] yet again by someone I cared about. I've been [diagnosed with borderline] for a while, and I’m just going to [isolate myself] and [sleep forever].

Gaur et al. WWW’19

Concept Phrases

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

Annotator Perspective for Moderate Risk

​

( UMD Suicidality Dataset

Shing et al. CLPsych’18 )

54

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

Annotated by Expert

(Label: Moderate Risk)

I do have a potential to live a decent life but not with people who [abandon me]. [Hopelessness] and [feelings of betrayal] have turned my [nights to days]. I am developing [insomnia] because of my [restlessness].

I just [can’t take it anymore]. Been [abandoned] yet again by someone I cared about. I've been [diagnosed with borderline] for a while, and I’m just going to [isolate myself] and [sleep forever].

Concept Phrases

Suicide Indication

Suicide Ideation

Suicide Behavior

Suicide Attempt

55

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

CNN Layers

I do have a potential to live a decent life but not with people who [abandon me]. [Hopelessness] and [feelings of betrayal] have turned my [nights to days]. I am developing [insomnia] because of my [restlessness].

I just [can’t take it anymore]. Been [abandoned] yet again by someone I cared about. I've been [diagnosed with borderline] for a while, and I’m just going to [isolate myself] and [sleep forever].

Transformed Input Text

Output

CNN with Semantic Embedding Loss ( )

56

M. Gaur (2022)

Gaur et al. WWW’19

CNN Layers

Suicide Indication

Suicide Ideation

Shallow Infusion: Locate and Embed Concept Phrases

I do have a potential to live a decent life but not with people who [abandon me]. [Hopelessness] and [feelings of betrayal] have turned my [nights to days]. I am developing [insomnia] because of my [restlessness].

I just [can’t take it anymore]. Been [abandoned] yet again by someone I cared about. I've been [diagnosed with borderline] for a while, and I’m just going to [isolate myself] and [sleep forever].

57

M. Gaur (2022)

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

Model’s Perspective for Moderate Risk

Shallow Infusion: Locate and Embed Concept Phrases

Gaur et al. WWW’19

58

M. Gaur (2022)

Shallow Infusion improves Recall

53%

SVM-L

Recall

Shing et al. CLPsych’18

57%

CNN

Gaur et al. WWW’19

CNN + GL

Sawhney

et al.

ACL’22

62%

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

Model’s Perspective for Moderate Risk

74%

CNN with Semantic Embedding Loss

Gaur et al. WWW’19

Gaur et al. WWW’19

CNN +

Concept

Phrase

Gaur et al. WWW’19

84%

SOTA

59

M. Gaur (2022)

Gaur et al. WWW’19

Gaur et al. PLoS’21

Numbers are SNOMED-CT / ICD-10 ids

Isolate myself

Feelings of Betrayal

Lack of Trust

704373005

Social Isolation

77096008

Hopelessness

Restlessness

Just Can't...

Abandoned

Sleep Forever

Feeling hopeless

30707703

Feeling agitated

24199005

Feeling abandoned

225015008

Feeling suicidal

225457007

Emotional State

106126000

Mental State Finding

36456004

Disturbance in thinking

26628009

Suicidal thoughts

6471006

Multi-hop

User-level Explainability

​

Why Moderate Risk?

60

M. Gaur (2022)

Isolate myself

Feelings of Betrayal

Lack of Trust

704373005

Social Isolation

77096008

Hopelessness

Restlessness

Just Can't...

Abandoned

Sleep Forever

Feeling hopeless

30707703

Feeling agitated

24199005

Feeling abandoned

225015008

Feeling suicidal

225457007

Emotional State

106126000

Mental State Finding

36456004

Disturbance in thinking

26628009

Suicidal thoughts

6471006

Multi-hop

Stopping Criteria: When the traversed node MATCHES Predicted Label

61

62

63

I know I could have a good life ahead of me, but I can't achieve it with people who keep leaving me. My feelings of despair and being betrayed have completely disrupted my sleep schedule. My constant state of agitation is causing me to develop sleep problems. I've reached my breaking point. Once again, someone important to me has left me behind. I've had a borderline personality disorder diagnosis for some time now, and I've decided to completely withdraw from everyone and everything.

64

65

M. Gaur (2022)

Explainability as a metric

Gaur et al. WWW’19

Gaur et al. PLoS’21

Perceived Risk Measure

37%

54%

16%

14%

61%

47%

(I)

(II)

: Perceived Risk Measure (PRM)

SVM-L

Shing et al. CLPsych’18

SVM-L

+ Concept Phrases

WWW’19

+Supportive Label

PLoS’21

+Supportive Label

PLoS’21

CNN with Semantic Embedding Loss

. WWW’19

CNN + Concept Phrase

WWW’19

Annotator Disagreements

Annotators who agree with the predictions

mis-classification

66

Where are we?

M. Gaur (2022)

Gaur et al. WWW’19, PLoS’21, AMIA’21

Kursuncu et al. CSCW’19

• Concept Classes are Explainable

• Explainability as a metric prevent uncertain predictions

Shallow Infusion

67

M. Gaur (2022)

Gaur et al. CIKM’18

Words

Words

Sentences

Sentences

Self-correlation

Or Self-Attention

Self-correlation

Or Self-Attention

Concept Classes in KG

Input (words or sentences)

Cross-correlation

Or

Cross-Attention

AI Models learn by Similarities

(A)

(B)

(C)

68

Modulating between Embedding Spaces

M. Gaur (2022)

Gaur et al. CIKM’18

75%

C_N

P_k

Decoder

Encoder

posts

Concept Classes

Autoencoder

• Representation Learners
• Representation Modulators

69

Semantic Encoding and Decoding

M. Gaur (2022)

Gaur et al. CIKM’18

Posts-by-Posts Self-Attention Matrix

Concept-by-Concept Self-Attention Matrix

Linear Projection between two embedding spaces

Sylvester Equation

Simoncini et al. SIAM’16

70

Semantic Encoding and Decoding

M. Gaur (2022)

Gaur et al. CIKM’18

Machine Learning Model

Sylvester Equation

Simoncini et al. SIAM’16

​

​

​

​

​

​

Input Layer

Feed Forward Neural Network

δ : Tunable Parameter for Knowledge Infusion

(1-δ) : Forces Knowledge Infusion

Allow model interpretability

71

Really struggling with my bisexuality which is causing chaos in my relationship with a girl. Being a fan of LGBTQ community, I am equal to worthless for her. I’m now starting to get drunk because I can’t cope with the obsessive, intrusive thoughts, and need to get out of my head.

Don’t want to live anymore. Sexually assault, ignorant family members and my never ending loneliness brights up my path to death.

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness. I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

Really struggling with my bisexuality which is causing chaos in my relationship with a girl. Being a fan of LGBTQ community, I am equal to worthless for her. I’m now starting to get drunk because I can’t cope with the obsessive, intrusive thoughts, and need to get out of my head.

Don’t want to live anymore. Sexually assault, ignorant family members and my never ending loneliness brights up my path to death.

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness. I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

δ = 1.0 (No Knowledge)

δ = 0.84 (16% knowledge)

Interpretability with Semi-Deep Infusion

Really struggling with my bisexuality which is causing chaos in my relationship with a girl. Being a fan of LGBTQ community, I am equal to worthless for her. I’m now starting to get drunk because I can’t cope with the obsessive, intrusive thoughts, and need to get out of my head.

Don’t want to live anymore. Sexually assault, ignorant family members and my never ending loneliness brights up my path to death.

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness. I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

​

δ = 0.71 (29% knowledge)

Expert Evaluation Agreement: 84%

Really struggling with my bisexuality which is causing chaos in my relationship with a girl. Being a fan of LGBTQ community, I am equal to worthless for her. I’m now starting to get drunk because I can’t cope with the obsessive, intrusive thoughts, and need to get out of my head.

Don’t want to live anymore. Sexually assault, ignorant family members and my never ending loneliness brights up my path to death.

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness. I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

δ = 0.66 (34% knowledge)

72

73

Reduction in Uncertainty / Risky Predictions

M. Gaur (2022)

Gaur et al. CIKM’18

Large Feature Set

[Gkotsis, Nature’17]

False

Alarms

Description of Concept Classes

​

DSM-5: Knowledge Source for Mental health Conditions

domain-specific ontology

30%

CNN

4%

3%

2.5%

1.12%

CNN

Random Forest

Adding various forms of knowledge

74

Intermediate Conclusions

M. Gaur (2022)

Gaur et al. CIKM’18

• Posts and Concept Classes bring Context Sensitivity inside the model
• Uncertainty in controlled due to Weight Matrix
• Explainability is achieved by analyzing Weight Matrix
• Interpretation can be gauged by tuning δ

75

Where are we?

M. Gaur (2022)

Gaur et al. CIKM’18, ICHI’21

76

M. Gaur (2022)

• Knowledge-infusion for Suicide Risk Classification

II. Knowledge-infusion for Language Generation

77

Process Knowledge

M. Gaur (2022)

Has subject wished he was dead or wished he could go to sleep and not wake up?

YES / NO

Has subject had any thoughts of killing himself?

YES / NO

Has subject been thinking about how he might do this?

YES / NO

Has subject has these thoughts and some intentions of acting on them?

YES / NO

Process Knowledge for Suicide Risk Classifcation

Roy and Gaur et al. IJCAI’22*

Suicide Indication

Suicide Ideation

Suicide Behavior

Suicide Attempt

Concept Classes

78

Semi-Deep Infusion: Process Knowledge

M. Gaur (2022)

Really struggling with my bisexuality which is causing chaos in my relationship with a girl. Being a fan of LGBTQ community, I am equal to worthless for her. I’m now starting to get drunk because I can’t cope with the obsessive, intrusive thoughts, and need to get out of my head.

Don’t want to live anymore. Sexually assault, ignorant family members and my never ending loneliness brights up my path to death.

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

Has subject wished he was dead or wished he could go to sleep and not wake up?

YES

Has subject had any thoughts of killing himself?

YES

Has subject been thinking about how he might do this?

NO

Has subject has these thoughts and some intentions of acting on them?

NO

Y = Suicide Ideation

Simple Text Classification

Process Knowledge-based Classification

Roy and Gaur et al. IJCAI’22

79

Semi-Deep Infusion: Process Knowledge

M. Gaur (2022)

Really struggling with my bisexuality which is causing chaos in my relationship with a girl. Being a fan of LGBTQ community, I am equal to worthless for her. I’m now starting to get drunk because I can’t cope with the obsessive, intrusive thoughts, and need to get out of my head.

Don’t want to live anymore. Sexually assault, ignorant family members and my never ending loneliness brights up my path to death.

I do have a potential to live a decent life but not with people who abandon me. Hopelessness and feelings of betrayal have turned my nights to days. I am developing insomnia because of my restlessness.

I just can’t take it anymore. Been abandoned yet again by someone I cared about. I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

Simple Text Classification

Process Knowledge-based Classification

Roy and Gaur et al. IJCAI’22

80

Semi-Deep Infusion: Process Knowledge

M. Gaur (2022)

Process Knowledge-based Classification

Roy and Gaur et al. IJCAI’22

81

Semi-Deep Infusion: Process Knowledge

M. Gaur (2022)

Roy and Gaur et al. IJCAI’22

Rudin Nature’19

Process Knowledge Structure in C-SSRS

C-SSRS: Columbia Suicide Severity Rating Scale

Decision Tree:

Optimize by Bernoulli loss

Don’t want to live anymore. Sexually assault, ignorant family members and my never ending loneliness brights up my path to death. [...] I've been diagnosed with borderline for a while, and I’m just going to isolate myself and sleep forever.

Has subject had any thoughts of killing himself? YES

p

p_sub

82

Semi-Deep Infusion: Process Knowledge

M. Gaur (2022)

Roy and Gaur et al. IJCAI’22

ERNIE: Zhang et al. ACL’19

W2V: Mikolov et al. NIPS’13

Process Knowledge Structure in C-SSRS

C-SSRS: Columbia Suicide Severity Rating Scale

ERNIE

W2V

ERNIE

W2V

ERNIE

W2V

simF = Gaussian

simF = Cosine

Vanilla Baseline

62%

69%

69%

78%

71%

72%

AUC-ROC

W2V: Word2Vec

83

Semi-Deep Infusion: Process Knowledge Explanations

M. Gaur (2022)

Roy and Gaur et al. IJCAI’22, ACL’22

Process Knowledge Structure in C-SSRS

C-SSRS: Columbia Suicide Severity Rating Scale

I wish I could give a shit about what would make it to the front page. I have been there and got nothing. Same as my life. I do have a gun.’, ’I thought I was talking about it. I am not on a ledge or something, but I do

have my gun in my lap.’, ’No. I made sure she got an education and she knows how to get a job. I also have recently bought her clothes to make her more attractive. She has told me she only loves me because I buy her things.

1. Wish to be dead - Yes

2. Non-specific Active Suicidal Thoughts - Yes

3. Active Suicidal Ideation with Some Intent to Act - Yes

4. Label: Suicide Behavior or Attempt

Self Explainable for an end-user

(1,2,3 verify adherence to the clinical guideline on diagnosis which a clinician understands)

47%

70%

XLNet

Yang et al. NIPS’19

Process Knowledge (Ours)

Agreement with Experts

84

Generative Models: Matching with Process Knowledge

M. Gaur (2022)

Gaur et al. AAAI’22

• Are the generated questions contextual and diverse? (Minimize Redundancy)

​

• Are the generated questions semantically related with each other?

​

• Are the generated questions in logical order? (Process Knowledge)

​

​

85

Generative Models: Matching with Process Knowledge

M. Gaur (2022)

Generator Network

Generator Network

Passages related to posts (P)

Capture Context:

1. TF-IDF / BM-25
2. Hyperlinks ( Asai et al. ICLR’20 )
3. Maximize Inner Product Search (MIPS) (Bachrach et al. RecSys’14) (Lewis et al. NIPS’20)

​

Reward

Process Knowledge

Semantically related Questions

Logical Order

Retriever

86

Generative Models: Matching with Process Knowledge

M. Gaur (2022)

Generator Network

Passage Context Controlled by KG

Reward

Maximize Inner Product Search (MIPS) Between KG and Passages

​

Capture Context

Semantic Relations

Logical Order

Passage Context Controlled by KG

Reward

Generator Network

Evaluator Network

Constraint on Logical Order

Retriever

Retriever

Generator-Evaluator Pairing forces Order

​

Capture Context

Semantic Relations

Logical Order

87

Ordering the Generated Questions

M. Gaur (2022)

Gaur et al. AAAI’22

Cross entropy loss

Reverse Cross entropy loss

88

Semi-Deep Infusion in Generative Models

M. Gaur (2022)

Gaur et al. AAAI’22

More Examples:

https://github.com/manasgaur/AAAI-22

Query:

Bothered by trouble concentrating while reading newspaper or watching television

• Do you have a hard time falling asleep and staying asleep?
• Do you feel like you sleep a lot but are still tired?
• Would you like to know about some major sleep disorders?
• Would you like to know about the 5 major sleep disorder types?

1. Do you feel like you sleep a lot but are still tired?
2. How many hours of sleep do you get on average each night?
3. How long have you struggled with sleep difficulties
4. Have you been diagnosed with any sleep disorder?

Baseline Generator (T5 1.2B)

ISEEQ (405 M)

Symptoms

Symptoms

Clarification

Symptoms

Cause

Cause and Symptoms

Diagnosis

Sequence

89

M. Gaur (2022)

​

Recently, I've been really struggling to stay focused while trying to do simple things like reading the newspaper or watching TV. It’s like I can’t keep my mind on it for more than a few minutes. Could it be depression?

• Do you have a hard time falling asleep and staying asleep?
• Do you feel like you sleep a lot but are still tired?
• Would you like to know about some major sleep disorders?
• Would you like to know about the 5 major sleep disorder types?

• Do you feel like you sleep a lot but are still tired?
• How many hours of sleep do you get on average each night?
• How long have you struggled with sleep difficulties
• Have you been diagnosed with any sleep disorder?

Baseline Generator (T5 1.2B)

ISEEQ (405 M)

Symptoms

Symptoms

Clarification

Symptoms

Cause

Cause and Symptoms

Diagnosis

Sequence

90

Where are we?

M. Gaur (2022)

Gaur et al. CIKM’18

Roy and Gaur et al. ACL’22

Roy and Gaur et al. IJCAI’22

91

M. Gaur (2022)

Summarizing Technical Contributions (KiL can do, statistical AI cannot)

Concept Classes

Knowledge Attention

Shallow Infusion

Shallow and

Semi-Deep Infusion

• Capture

Context

• Handle Uncertainty and Risk
• Model Interpretability

• User-level Explainability
• Self Explainable Model

​

​

​

Knowledge graph Context Controller

Logical Order Constraints

Semi-Deep Infusion

• Process-Controlled Question Generation

Process Knowledge

Semi-Deep Infusion

92

Datasets for AI+NLU Research Community

M. Gaur (2022)

Corpus of Linguistic Acceptability

Summarizing Clinical Interviews

DSM-5 and PHQ-9

Flesch Reading, Divergence, Theme Overlap

Matthew Correlation

Rich Evaluation metrics

Stanford Sentiment Treebank

Assessing Severity in User-generated Content

Ordinal Error, Perceived Risk Measure, Ranked Precision/Recall

DSM-5 and Drug Abuse Ontology

Accuracy

Question NLI

ConceptNet and WordNet

Concept Mover Distance, BLEURT

F1-Score and Accuracy

User-language Paraphrase Corpus

Microsoft Paraphrase Corpus

Recognizing Textual Entailment

Conversational Information Seeking

Process Knowledge NLG

Gaur et al. JMIR’21

Gaur et al. Pone’21, WWW’19, ACL’ 22

Gaur et al. AAAI’22

Roy & Gaur et al. ACL

ConceptNet, WikiNews, Wikipedia , MS-MARCO

Logical Coherence, Semantic Relevance, BLEURT

Accuracy

PHQ-9, GAD-7, C-SSRS

Accuracy

Avg. # Unsafe Matches, Avg. #KG concept Matches,

Avg. Sq. Rank Error

Reagle & Gaur FirstMonday’22

General Language Understanding Evaluation (GLUE) (Wang et al. ICLR’19)

Knowledge-intensive Language Understanding (KILU)

(Sheth and Gaur IEEE IC’21)

Modified and Adapted from McCarthy et al. IGI’12

93

M. Gaur (2022)

Concerns regarding Knowledge Infusion

Knowledge-infusion is a necessary bias for controlling AI, but there are concerns

​

1. Extrapolation or Overgeneralization of the Model:
1. Sentence: Your mom and dad are toxic.
2. Paraphrased with Statistical AI: Toxicity is in your mom and dad
3. Paraphrased with Knowledge-infused Learning: Your parents are radioactive

Reagle and Gaur FirstMonday’22

2. Disparity:

• Sentence: She has her boundaries for a reason.
• Paraphrased with Statistical AI: She has her borders for a factor.
• Paraphrased with Knowledge-infused Learning: She has her bound/limits for a purpose/cause.

​

94

M. Gaur (2022)

Further Research Questions : Deep Infusion

Gaur et al. AAAI’19

• We know Neural Models learn by performing abstraction at each layer
• Which layer require Knowledge?
• How much knowledge infusion needs to happen to orient the layer’s representation towards the task?

​

​

• Overgeneralization or Extrapolation of the Neural Models due to Knowledge Infusion can be controlled:
• By understanding the concept a neuron has learnt
• Removing unwanted neurons (Deterministic Dropout)

Deep Infusion a.k.a Layerwise Knowledge Infusion

95

Backpropagation

Connector acting like a toggle switch

96

Thesis Publications

[AAAI 2022] MG, KG, VS, HJ. ISEEQ: Information Seeking Question Generation using Dynamic Meta-Information Retrieval and Knowledge Graphs

​

[ACL 2022] RS, AN, MG, A Risk-Averse Mechanism for Suicidality Assessment on Social Media

​

[IEEE IC 2021] MG, KF, AS. "Semantics of the black-box: Can knowledge graphs help make deep learning systems more interpretable and explainable?."

​

[IEEE IC 2021] AS, MG, KR, KF. "Knowledge-intensive language understanding for explainable ai."

​

[PLoS 2021] MG, VA, AA, UK, TK, JB, JP, AS. "Characterization of time-variant and time-invariant assessment of suicidality on Reddit using C-SSRS."

​

[JMIR 2021] MG, VA, UK, AA, VS, TK, JB, MN, AS. "Knowledge-infused abstractive summarization of clinical diagnostic interviews: Framework development study."

​

[ICHI 2021] MG, KR, AS, BS, AS. "“Who can help me?”: Knowledge Infused Matching of Support Seekers and Support Providers during COVID-19 on Reddit."

​

[AMIA 2021] MG, RT, TK, AS, JP. "Comparing Suicide Risk Insights derived from Clinical and Social Media data."

​

97

Thesis Publications

[AAAI 2020] MG, CA, SA, WG, KZ, AJ. "Unsupervised detection of sub-events in large scale disasters."

​

[HT 2020] MG, UK, AS, RW, SY. "Knowledge-infused deep learning."

​

[AAAI 2019] MG, UK, AS, "Knowledge infused learning (k-il): Towards deep incorporation of knowledge in deep learning."

​

[IEEE IC 2019] AS, MG, UK, RW. "Shades of knowledge-infused learning for enhancing deep learning."

​

[WWW 2019] MG, AA, JS, UK, TK, RK, AS, RW, JP. "Knowledge-aware assessment of severity of suicide risk for early intervention."

​

[ICSC 2019] MG, AA, UL, UK, TK, AG, AS, RW, JP. "Question answering for suicide risk assessment using reddit."

​

[ICSC 2019] MG, SS, AG, AS. "empathi: An ontology for emergency managing and planning about hazard crisis."

​

[CIKM 2018] MG, UK, AA, AS, RD, TK, JP, "" Let Me Tell You About Your Mental Health!" Contextualized Classification of Reddit Posts to DSM-5 for Web-based Intervention."

​

[DSAA 2018] QH*, MJ*, IMD*, MG*, LZ, "A hybrid recommender system for patient-doctor matchmaking in primary care."

98

Other Publications

[ECML 2021] KR, QZ, MG, AS, "Knowledge infused policy gradients with upper confidence bound for relational bandits."

​

[IJID 2021] SB., MS, MG, AM. "Impact of reproduction number on the multiwave spreading dynamics of COVID-19 with temporary immunity: A mathematical model."

​

[SocInfo 2020] TW, HI, UK, MG, VL, TK, AS, IBA, "Alone: A dataset for toxic behavior among adolescents on twitter."

​

[KDD 2020] NS, MG, SB, CR, SB, AS "EXO-SIR: An epidemiological model to analyze the impact of exogenous infection of covid-19 in india."

​

[CSCW 2019] UK, MG, CC, AA, TK, VS, DA, IBA, AS. "Modeling islamist extremist communications on social media using contextual dimensions: religion, ideology, and hate."

​

​

​

[ISWC 2018] AG, MG, SS, TK, AS, Personalized Health Knowledge Graph.

​

[ICHI 2018] QH, IMD, MJ, MG, LZ. "A collaborative filtering recommender system in primary care: Towards a trusting patient-doctor relationship."

​

[WI 2018] SB, MG, BB, VS, AS, BM. Enhancing crowd wisdom using explainable diversity inferred from social media.

99

Ph.D. @ Kno.e.sis & AIISC

M. Gaur (2022)

100

M. Gaur (2022)

Publications and Contribution to Research Community

Publications → 38

Venues:

AAAI, ECML-PKDD, ACL, CIKM, WWW, CSCW, ISWC, DSAA, ICHI, AMIA, JMIR, PLoS One, etc.

18 (Conferences), 9 (Journals), 3 (AAAI Symposiums), 3 (Book Chapters), 3 (IEEE Magazine), and 2 (Workshops)

Resources (>1K downloads)

Suicide Risk Severity Lexicon

[WWW 2019]

Services

DSM-5 and DAO KG [CIKM 2018]

Reddit CSSRS Dataset v1.0 [WWW 2019]

Reddit CSSRS Dataset v2.0

[PLoS One 2021]

Moments of Change Dataset [NAACL CLPsych 2022]

PRIMATE (submitted to CLPsych)

PC Member: ACL-RR, NeurIPS, CIKM, Web Conference, AAAI, EMNLP, NAACL, Big Data, ICWSM, Web Science, etc.

Application Track Chair: ISIC

Co-Editor: Knowledge-infused Learning (IEEE) &

Personal Knowledge Graphs: Methodology, Tools, and Applications (IET-UK)

Session Chair: ICSC & WI

Nominated for UofSC Breakthrough Research Awards

101

M. Gaur (2022)

Technical Outreach

Tutorials

Workshops

Invited Talks

Knowledge-infused Deep Learning [ ACM HT 2020]

Knowledge-infused Reinforcement Learning [KGC 2022]

Explainable AI using KG

[ ACM CoDS-COMAD 2021]

Explainable Data for AI in Cyber Social Threats & Public Health

[ AAAI ICWSM 2021]

Knowledge-infused Mining & Learning [ACM SIGKDD 2020]

Knowledge-infused Learning [Knowledge Graph Conf. 2021]

Knowledge-infused Learning [ACM CIKM 2022]

PODCASTS

Chaos

Orchestra

102

Grant Proposals (DoD, AFoRL, NSF, NIH, etc.)

M. Gaur (2022)

Partners

& Collaborators

Maria Liakata

Adam Tsakalidis

Time-Sensitive Sensing of Language in User Generated Content ($24K)

Development of an Instrumented, Intelligent Infant Interaction Laboratory for the Prediction of ASD ($39K)

Jessica Bradshaw

Ugur Kurşuncu

Valerie Shalin

Advancing Neuro-symbolic AI with Deep Knowledge-infused Learning ( $140K )

Amit

Sheth

Valerie Shalin

Amit Almor

Amitava Das

KREDIT: Knowledge infoRmEd NLU for Deception IdenTification

Sriram Natarajan

Qi

Zhang

Meera Narasimhan

A Knowledge-Enhanced Approach to Contextual and Personalized Virtual Health Assistant ($1M)

Nitin Agarwal

UALR

Jonathan Beich

WSU

Raminta

Daniulaityte

ASU

Srinivasan Parthasarathy

OSU

Carlos Castillo

UPF Spain

EPSRC-UKRI

NSF

AFoRL

NSF

UofSC

Lead Contributor

Technical Contributor

Lead Contributor

Technical Contributor

Lead Contributor

Ponnurangam Kumaraguru

IIIT-H, India

103

Research and Industry

Internships

Fellowships

104

Fortunate to Mentor 20 students

2 High School

8 Masters

5 Early Ph.Ds.

5 Undergraduate

105

Dr. Krishnaprasad Thirunarayan

Dr. Jyotishman Pathak

Dr. Amit P. Sheth

(Advisor)

Dr. Biplav Srivastava

Dr. Lorne Hofseth

Dr. Valerie L. Shalin

Thanks to my Amazing Committee Members

Dr. Vignesh Narayanan

Dr. Pooyan Jamshidi

106

Thanks to Mentors

Dr. Sanjaya Wijeratne

(Holler.io)

Dr. Kalpa Gunaratna

(Samsung Research)

Dr. Shreyansh Bhatt

(Amazon Research)

Dr. Saeedeh Shekarpour

(Univ. of Dayton)

Dr. Sarasi Lalithsena

(IBM Research)

Dr. Hemant Purohit

(George Mason Univ.)

Dr. Sujan

Perera

(Amazon Research)

Dr. Ugur Kurşuncu

(Georgia State Univ.)

Dr. Ke Zhang

(Dataminr)

Dr. William Goves

(Dataminr)

Dr. Sam

Anzaroot

(Verneek)

Dr. Qiwei Han

(NOVA SBE)

Dr. Alejandro Jaimes

(Dataminr)

Dr. Ramakanth

Kavuluru

(Univ. of Kentucky)

Dr. Vijay Srinivasan

(Samsung Research)

Dr. Petko Bogdanov

(State University of New York, Albany)

107

Thanks to Friends

A

I

I

S

C

​

&

​

K

N

O

E

S

I

S

​

108

Acknowledgement

109

Ph.D. and The Ultramarathons

• Mountaineer Rumble 50K (~33 miles)
• Overall: 10 out of 44
• Age Group: 6
• Carolina Reaper 50K (~33 miles)
• Overall: 13
• Age Group: 10
• Hilton Head Half Marathon (~13.11 miles)
• Overall: 16 out of 656
• Age group: 4 out of 59
• Uptil now
• Distance Covered: 2,210 miles
• Ran all the highest peaks in and around South Carolina

110

THANK YOU!

​

Questions?

111