Machine Learning for Music

Artist Similarity Mapping via Dimensionality Reduction

​

​

Michael Rossetti

https://github.com/s2t2/ml-music-2023/

Background

Inspiration

Spotify AI provides customized song recommendations

Let's use machine learning to determine which artists are most similar to each other, based on patterns in their song data.

Research Plan

Objectives and Constraints

Goals / Objectives:

1. Produce a two-dimensional mapping of artists, using audio data from Youtube
2. Visually observe which artists are most similar (i.e. closest on the map)
3. Provide recommendation list of similar artists

Assumptions / Constraints:

• Use unsupervised machine learning methods only
• Focus on a small number of songs and artists to start (because we can only download so much data from YouTube, and because we can only see so many points on a plot)

Methods

Step 1: Downloading Audio from YouTube

Collect music videos for a variety of artists (7 to 10 songs per artist)

Using Pytube package in Python: https://pytube.io/

Youtube Audio Dataset v1

Contains 206 videos, by 24 artists, spanning over 15 hours

Step 2: Cutting Tracks

Choose uniform track length of 3 or 30 seconds

Final remainder discarded (variable length)

Using Librosa package in Python: https://librosa.org/

Track 1 (length s seconds)

Audio File

(variable full length)

Track 2 (length s seconds)

Track 3 (length s seconds)

Step 3: Encoding Audio Features

Choose number of MFCCs between 8 and 20 (e.g. 13)

Using Librosa package in Python: https://librosa.org/

feature encodings

Track 1 (length s seconds)

Track 2 (length s seconds)

Track 3 (length s seconds)

Track n (length s seconds)

. . .

Step 4: Dimensionality Reduction

Conduct PCA using 2 or 3 components

reduced "embeddings"

feature encodings

Using Scikit-learn package in Python: https://scikit-learn.org

Dimensionality Reduction Techniques

• Principal Component Analysis (PCA)
• Benefits: interpretable / explainable
• Drawbacks: affected by outliers
• T-distributed Stochastic Neighbor Embedding (T-SNE)
• Benefits: more robust to outliers
• Drawbacks: less interpretable / explainable
• Uniform Manifold Approximation & Projection (UMAP)
• Benefits: fast, scalable, may perform better on complex data
• Drawbacks: less interpretable / explainable
• etc.

Step 5: Plotting the Embeddings

Inspect the results to see which tracks and artists are most similar

Using Plotly package in Python: https://plotly.com/python

reduced "embeddings"

Step 6: Plotting the Centroids

Characterize the center point for each track or artist

Using Plotly package in Python: https://plotly.com/python

embedding centroids

(mean X and Y for all tracks)

Results / Demo!

1. Methods Demo Notebook
2. Results Repository:
1. length_3_mfcc_13/pca_2.html
2. length_30_mfcc_13/pca_2_centroids.html
3. length_3_mfcc_13/pca_3_centroids.html

Results

30 second tracks: PCA

Results

30 second tracks: T-SNE

Results

30 second tracks: UMAP

Results

3 second tracks: PCA

Results

3 second tracks: T-SNE

Results

3 second tracks: UMAP

Conclusions

• These methods can be used by music platforms to provide artist recommendations
• Dimensionality reduction, when applied to audio features such as MFCCs, can detect which artists are related to each other
• These methods, due to their unsupervised nature, can provide recommendations for new artists that may emerge over time, and may reduce manual effort and costs associated with human-provided recommendations

Thank you!

Extra Slides / Appendix

Contact

Presentation by: Michael Rossetti

Contact: Email | LinkedIn | GitHub

Interests:

• Machine Learning (AI/ML)
• Unsupervised Learning
• Deep Learning
• Music Information Retrieval

​

PCA

Tuning Number of Components