A Language Model-Based Playlist Generation�Recommender System
Enzo Charolois–Pasqua · Eléa Vellard · Youssra Rebboud · Pasquale Lisena · Raphael Troncy
Love
Christmas
Hype
Country
2
Artist/Genre/Style (25%)
Cunningham, Sally Jo; Bainbridge, David; Falconer, Annette�"More of an art than a science": Supporting the creation of playlists and mixes. ISMIR 2006
Event or Activity (25%)
Romance (19%)
Mood (16%)
Other (2.6%)
Playlist names are not necessarily bound to a category
“Housewarming Party”
3
“Country summer”
“Spring awakening”
(real playlist names from the Million Playlist Dataset)
“Marina and the diamonds”
“Dancing in the kitchen”
4
What are the best tracks for this playlist title? 🍋
Problem statement
5
Ching-Wei Chen, Paul Lamere, Markus Schedl, and Hamed Zamani. �Recsys Challenge 2018: Automatic Music Playlist Continuation. RecSys 2018
Related work
Text & NLP Approaches
Large Language Models in Music Recs
6
Gap: Limited focus on playlist titles as input
Playlist Titles in Recommender Systems
RecSys Challenge 2018: cold-start with titles
Approach
#clustering #fine-tuning #language-models #semantic-similarity
7
Phase 1
Fine-Tune a Language Model
Phase 2
Predict with Semantic Similarity
Main idea
8
words occurring often together in sentences
close in the�embedding space
titles occurring often together in playlists
close in the�embedding space
Phase 1: Fine-tune a Language Model
9
MPD
Pretrained�SBERT
Playlist embeddings
Clustering
Fine-tuning
Fine-tuned language
model
Phase 2: Predict with Semantic Similarity
10
Fine-tuned language
model
New�Playlist Title
MPD
Embedding representation
Embedding representation
K most relevant playlists
Ranking based on cosine similarity
N recommended tracks
Voting mechanism
Evaluation
#quantitative #qualitative
11
Quantitative evaluation
12
Method | R-Precision | NDCG |
Monti et al. (Only title) | 0.0837 | 0.1260 |
Faggioli et al. (Only title) | 0.1093 | 0.2451 |
Kim et al. (Only title) | 0.0760 | 0.1866 |
Pre-trained | 0.1570 | 0.2731 |
Fine-tuned (cross-entropy) | 0.1556 | 0.2825 |
Fine-tuned (triplet loss) | 0.1285 | 0.2297 |
from RecSys Challenge 2018
our method
Qualitative evaluation
Rock Classics
13
▶ My Heart Will Go On� Céline Dion
▶ Highway to Hell� AC/DC
▶ Smells Like Teen Spirit� Nirvana
▶ It’s My Life� Bon Jovi
Qualitative evaluation
total recommended tracks
14
Method | Quality@10 | Quality@66 |
Pre-trained | 0.7376 | 0.7231 |
Fine-tuned (cross-entropy) | 0.7789 | 0.7719 |
Fine-tuned (triplet loss) | 0.7533 | 0.7461 |
Predict with LLMs
#prompt #zero-shot #few-shot
15
LLM Generation
You are an expert in music playlist generation.
Your task is to generate the continuation of a playlist given only its title and five example songs with their artists.
Important:
• You have to select only songs released before October 2017.
• Propose a COMPLETE playlist consisting of exactly 10 songs.
…
• All recommended songs must be UNIQUE and must not repeat any of the five example songs provided.
Playlist Title: "{playlist_title}"
Examples:
(1) {"song": "{song1}", "artist": "{artist1}"}
(2) . . .
Output format (strict):
[
{"song": "<title>", "artist": "<artist>"},
...
]
Answer ONLY with the JSON list exactly as specified above. Do not output anything else.
16
Setting a persona
Give precise instructions
Include N samples
Ask a specific output
approximation to match MPD content
Evaluation of the LLM
17
Metric | GPT-4o�(0-shot) | GPT-4o�(5-shot) | Our Method (FT-C) |
Precision@10 | 0.0636 | 0.1227 | 0.1793 |
Recall@10 | 0.0073 | 0.0197 | 0.0382 |
MRR@10 | 0.1636 | 0.2505 | 0.3254 |
R-Precision@10 | 0.0157 | 0.0338 | 0.0496 |
NDCG@10 | 0.1900 | 0.3249 | 0.3740 |
Qualitative Score | 0.7175 | 0.7953 | 0.7789 |
Conclusion
Contributions
Future Work
18
Thank you!
This presentation