1 of 18

NLP & Depression

Michelle Renee Morales - mmorales@gradcenter.cuny.edu

Corpus Analysis 4/5/17

2 of 18

DEPRESSION

2

3 of 18

3

In the United States, depression affects approximately 14.8 million adults (~6.7% of the population age 18 and older (ADAA, 2015)
The World Health Organization forecasts that if current demographics continue, depression will be the greatest source of disability worldwide by the year 2030 (Mathers et al. 2008)
For ⅓ of the world there exists 1 clinician for every 2 million people (Tom Insel, 2016)
Key findings in the November 2015 NYC Health report found the average wait time for the MHA-NYC hotline was 1 minute 39 seconds

4 of 18

Definition

4

5 of 18

Diagnostics

Very difficult to diagnose → standard measurements of depression include:

Structured clinical interview (DIA-X, DIPS, SCID)
Clinical rating scales (Hamilton Rating Scale for Depression)
Self report measures (Beck Depression Inventory and Patient Health Questionnaire)

Treatable illness

medication, e.g. antidepressants
therapies

5

6 of 18

Biological & Physiological Markers

*This search has had limited success

lower level of serotonin
low functioning of the neurotransmitter GABA
cortical and limbic systems (neuroimaging studies)
small hippocampal volumes
galvanic skin responses
saccadic eye movements
changes in REM sleep parameters

*(Cummins et al., 2015)

6

7 of 18

Nonverbal Behavior

7

Cummins et al. (2015)

8 of 18

Voice & Speech

8

Signal	Effect	References
Pitch range	↓	Darby and Hollien, 1977
Pitch variation	↓	Blanken et al., 1993
Speaking rate	↓	Blanken et al., 1993; Cannizaro et al. 2004
Speech pause duration	↑	Stassen et al., 1998; Alpert et al., 2001; Mundt et al. 2012
Voice tenseness	↑	Scherer et al., 2013
Average syllable duration	↑	Honig et al., 2014
Average phone duration	↑	Morales and Levitan, 2016

9 of 18

Language

9

Signal	Effect	References
First-person singular pronouns	↑	Stirman and Pennebaker, 2001; Rude et al., 2004
First-person plural pronouns	↓	Stirman and Pennebaker, 2001; Rude et al., 2004
Social references	↓	Rude et al., 2004
Negatively-valenced words	↑	Rude et al., 2004
Positively-valenced words	↑	Rude et al., 2004
Increased grammatical complexity	↓	Zinken et al., 2010
Sleep/fatigue related words	↑	Morales and Levitan, 2016

10 of 18

Types of Data Collection

Social media

Speech task

Read task or spontaneous speech task (Valstar et al. 2014)

Interview

With Clinician
With virtual human (Gratch et al. 2014)

Writing task

Essays (Rude et al. 2004)
Blog posts (Milne et al. 2016)
Description of symptoms (Zinken et al. 2010)

10

11 of 18

Example:

Simsensei

11

12 of 18

Open-source Data

Audio/Visual Emotion and Depression Recognition dataset (AVEC):

Videos labelled for depression and emotion

Distress Analysis Interview Corpus (DAIC-WOZ):

Interview videos between Simsensei (virtual human) and human, labelled for anxiety, depression, and PTSD

RO Shared Task dataset:

Reachout.com forum blog posts labelled for green/yellow/red

12

13 of 18

Types of Data Annotation

Tweets → “I was just diagnosed with depression…”
Manual annotation with various judges

By researchers
By Amazon Mechanical Turkers

Self-report
Surveys
Clinical assessment

13

14 of 18

Useful Feature Extraction Tools

14

Covarep
OpenSMILE

OpenFace

Linguistic Inquiry and Word Count (LIWC)
Google’s Syntaxnet (Parsey McParseface)
Automatic speech recognition:

IBM Watson Speech-to-text
Google’s Speech API

15 of 18

My Dissertation Project

Goals:

Build a multimodal system for depression assessment
Release tool for multimodal feature extraction
Improve depression detection performance
Explore methods to combining modalities

15

Multimodal features

Machine

Learning

+

16 of 18

OpenMM: Open-source Multimodal Feature Extraction

https://github.com/michellemorales/OpenMM

16

76%

Humans = 47.3%

17 of 18

Combining Modalities?

EARLY

17

Visual Features

Audio Features

Ling. Features

Classifier 1

Classifier 2

Classifier 3

Fusion

Visual Features

Audio Features

Ling. Features

Fusion

Classifier

LATE

18 of 18

Societal Impacts

Things to think about:

Safety
Privacy
AI Ethical guidelines

Current applications →

18

Coming soon...Health Support Responses

Health & Wellness Exhibit

mHealth

Affectiva Social Robot

Bellabeat

Affectiva Social Robot as Healthcare companion
Bellabit - necklace that tracks physio data to tell when you’re stressed and suggest meditation/breaks
Virtual assistants include health support responses
Mental Health research & drug efficacy
mHealth - The two-year project, funded by a $1.8 million grant from the National Institute of Mental Health, will measure a person’s mental state by analyzing his or her voice, call patterns, location and text messages. It’s being developed by Cogito, an MIT-based company that’s working with the U.S. Department of Veterans Affairs to develop mHealth-based behavioral health programs for service members.
The primary goal of the platform in healthcare, the panel said, will be to improve collaboration between the patient and the doctor by improving communication.