Using Sentiment Analysis to Differentiate Types of Loans

​

Data from: Consumer Financial Protection Bureau Complaints

Team 7: Xi Du, Alan Hunt, Leah Gall, Jason Jimenez

What we Looked for in Data

• Relatable

​

• Emotional

​

• Well-structured

Data Collection

Data Collection

Data Cleaning

​

On XX/XX/2022 I submitted a request over the telephone to get a refund on my payments made during Covid. I was told the total amount I would get and it would take six weeks for a refund. \n\nOn XX/XX/2022 I called backend and they said it was still in processing and it would take 10-12 weeks from the request. \n\nOn XX/XX/2022 I called back and they said it hasnt been processed and that I need to continue to wait and that it will be another 8 weeks from the the time it was approved. They can not say why it hasnt been process or when it would be processed even though I requested back on XX/XX/XXXX. \n\nDoing research it appears there have been plenty of people to get their refund who have applied on or after XX/XX/XXXX through the same servicer. \n\nThis servicer is acting in a deceptive manner. We have given them ample time to resolve this issue.

Research Questions

• Do customers have different proportions of negative words in complaints about two types of products: student loan and vehicle loan or lease?

​

• Can we differentiate student loan and vehicle loan/lease complaints using classification models?

​

​

Research Questions

• Do customers have different proportions of negative words in complaints about two types of products: student loan and vehicle loan or lease?

​

• Can we differentiate student loan and vehicle loan/lease complaints using classification models?

​

​

Methodology

• Calculate the proportions of negative words
• Perform Hypothesis T-Test
• Build and Compare Classification Models

First Research Question

Do customers have different proportions of negative words in complaints about two types of products: student loan and vehicle loan or lease?

Methodology

• Calculate the proportions of negative words
• Perform Hypothesis T-Test
• Build and Compare Classification Models

Second Research Question

Can we differentiate student loan and vehicle loan/lease complaints using classification models?

Methodology (Calculate the proportions of negative words)

• Define the proportion
• Use a Negative Word Dictionary
• Stem the words using Snowball algorithm
• Use the English Snowball stopword list

Methodology (Calculate the proportions of negative words)

• Define the proportion

​

Methodology (Calculate the proportions of negative words)

• Define the proportion
• Use a Negative Word Dictionary
• Stem the words using Snowball algorithm
• Use the English Snowball stopword list

Methodology (Calculate the proportions of negative words)

• Use a Negative Word Dictionary
• Find on GitHub

​

​

Credit:

Minqing Hu and Bing Liu. "Mining and Summarizing Customer Reviews." Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2004), Aug 22-25, 2004, Seattle, Washington, USA.

Bing Liu, Minqing Hu and Junsheng Cheng. "Opinion Observer: Analyzing and Comparing Opinions on the Web." Proceedings of the 14th International World Wide Web conference (WWW-2005), May 10-14, 2005, Chiba, Japan.

​

Methodology (Calculate the proportions of negative words)

• Use a Negative Word Dictionary
• Find on GitHub

​

​

Methodology (Calculate the proportions of negative words)

• Define the proportion
• Use a Negative Word Dictionary
• Stem the words using Snowball algorithm
• Use the English Snowball stopword list

Methodology (Calculate the proportions of negative words)

• Stem the words using Snowball algorithm
• Use wordStem( ) in SnowballC package

​

Methodology (Calculate the proportions of negative words)

• Stem the words using Snowball algorithm
• Use wordStem( ) in SnowballC package
• Compare stemmed words with Negative Word List

​

Methodology (Calculate the proportions of negative words)

• Stem the words using Snowball algorithm
• Use wordStem( ) in SnowballC package
• Compare stemmed words with Negative Word List

​

The number of negative words in each narrative

Methodology (Calculate the proportions of negative words)

• Define the proportion
• Use a Negative Word Dictionary
• Stem the words using Snowball algorithm
• Use the English Snowball stopword list

Methodology (Calculate the proportions of negative words)

• Use the English Snowball stopword list
• Remove the stopwords

Methodology (Calculate the proportions of negative words)

• Use the English Snowball stopword list
• Remove the stopwords

The number of non-stop words in each narrative

Calculate the Proportion!

Boxplot of the Proportions of Negative Words Distribution for Two Products

Boxplot of the Proportions of Negative Words Distribution for Two Products

Boxplot of the Proportions of Negative Words Distribution for Two Products

Hypothesis testing: T-test

H₀: There is no difference in the proportions of negative words used between Vehicle and Student loans.

H_a: There is a difference in the proportions of negative words used between Vehicle and Student loans.

Results (Hypothesis testing: T-test)

Results (Hypothesis testing: T-test)

< 0.05

​

Statistically Significant!

Results (Hypothesis testing: T-test)

Classification Model Building

• Create/add new factors
• Split into train and test
• Run multiple versions

​

Results (Classification Models)

Conclusion

• From two sample T-test
• From classification models

​

​

​

​

Next Steps

• Use Tree-based models
• Add more predictors

Thank You for Listening!