1 of 9

Automatic Detection of Moral Values in Music Lyrics

Vjosa Preniqi, Iacopo Ghinassi, Julia Ive, Kyriaki Kalimeri, Charalampos Saitis

2 of 9

Overview of This Work

Objective: Detecting morality rapidly from lyrics for deeper understanding of our music-listening behaviours.

Methods: Use an LLM to generate synthetic lyrics for fine-tuning a moral classifier.

​

Evaluation: Use a small human-annotated lyrics dataset as ground truth.

Impact: Supports annotation-free morality learning in lyrics, relevant for creative industries.

3 of 9

Moral Foundations Theory (Haidt and Graham, 2004)

Care/Harm:

Involves concerns of others suffering and includes virtues like empathy and compassion.

Fairness/Cheating

Focuses on issues of unfair treatment, inequality, and justice.

Loyalty/Betrayal

Pertains to group obligations such as loyalty and the vigilance against betrayal.

Authority/Subversion:

Centres on social order and hierarchical responsibilities, obedience and respect.

Purity/Degradation

Relates to physical and spiritual sanctity, including virtues like chastity and self-control.

4 of 9

Can We Predict MFT Values in Lyrics?

    • Limited research exists on morality and music in MIR.
    • Lack of lyrics datasets with moral annotations.
    • Moral inference in text is domain dependent task.
    • Lyrics complex structures and figurative language.

Main Challenges:

    • Applying MoralBERT model (Preniqi et al., 2024)
    • Implementing domain adversarial (Guo et al., 2023).
    • Fine-Tuning with moral annotated corpora.
    • Twitter/X (Hoover et al., 2020), Reddit (Tragger et al., 2022), and Facebook (Beiro et al., 2023 ).

Insights from social media:

5 of 9

Proposed Solution

  • Generating a morally nuanced synthetic lyrics dataset, for fine-tuning a BERT model with Domain Adversarial.
  • Creating a dataset of 200 human-annotated song lyrics for evaluation.
  • Benchmarking our approach against a zero-shot GPT-4 classification model.

​

​

​

6 of 9

MFT Prediction Results: F1 Scores

 

F1 Weighted

F1 Binary

 

MoralBERT

GPT-4

BERT SL

MoralBERT SL

MoralBERT

GPT-4

BERT SL

MoralBERT SL

Care

.80 ± .03

.68 ± .03

.81 ± .03

.83 ± .03

.68 ± .05

.64 ± .04

.68 ± .05

.75 ± .04

Harm

.68 ± .03

.75 ± .03

.71 ± .03

.70 ± .03

.62 ± .05

.71 ± .04

.63 ± .05

.69 ± .04

Fairness

.55 ± .03

.73 ± .03

.73 ± .03

.74 ± .03

.30 ± .05

.39 ± .06

.41 ± .06

.38 ± .06

Cheating

.84 ± .03

.80 ± .03

.86 ± .02

.69 ± .03

.27 ± .09

.16 ± .07

.52 ± .08

.32 ± .06

Loyalty

.69 ± .03

.67 ± .03

.77 ± .04

.79 ± .04

.38 ± .06

.34 ± .06

.21 ± .08

.27 ± .09

Betrayal

.81 ± .02

.72 ± .03

.89 ± .02

.84 ± .02

.34 ± .07

.31 ± .06

.40 ± .11

.37 ± .08

Authority

.77 ± .03

.75 ± .03

.77 ± .03

.84 ± .03

.45 ± .06

.42 ± .06

.35 ± .07

.39 ± .09

Subversion

.80 ± .03

.72 ± .03

.80 ± .03

.71 ± .03

.44 ± .07

.39 ± .06

.40 ± .07

.43 ± .06

Purity

.77 ± .03

.86 ± .02

.89 ± .02

.90 ± .02

.41 ± .06

.56 ± .07

.55 ± .08

.63 ± .08

Degradation

.74 ± .03

.81 ± .03

.81 ± .03

.86 ± .03

.34 ± .06

.40 ± .07

.30 ± .07

.32 ± .10

Average

.75 ± .03

.75 ± .03

.80 ± .03

.80 ± .03

.42 ± .06

.43 ± .06

.45 ± .07

.46 ± .07

7 of 9

MFT Prediction Results: Precision Scores

8 of 9

Summary

Main Takeaways:

    • This work provides valuable insights into the automatic detection of moral values in music lyrics.
    • The use of synthetic data generated by GPT-4 presents a promising approach for training models in data-scarce domains.

Implications:

    • This research paves the way for deeper understanding of moral expression in music and its influence on listeners.
    • Enhancing music tagging, recommendation systems, and personalisation of music streaming services.
    • Understanding the evolution of societal values and cultural norms reflected in music over time.

Future Research

    • Analysing moral expressions within specific lyrical structures (e.g., verses and choruses).
    • Exploring moral values in non-English lyrics to understand cross-cultural moral expressions in music.

9 of 9

Thank You!

Preprint

Web App

v.preniqi@qmul.ac.uk