Notewise Evaluation of Source Separation: �A Case Study For Separated Piano Tracks

Yigitcan Özer, Hans-Ulrich Berendes, Vlora Arifi-Müller,

Fabian-Robert Stöter, and Meinard Müller

Notewise Evaluation of Source Separation

Music source separation: Split recordings into individual parts, e.g. instruments

​

Typical evaluation: Use entire tracks to calculate metrics, e.g.,� Signal-to-Distortion Ratio (SDR)

Objective: More nuanced understanding of model performance

​

Proposed: SDR of single note events

Enables evaluation of single notes or specific note groups

Case study: Separating piano in Piano Concerto Dataset (PCD)� (orchestra + piano)

​

​

Özer, Schwär, Arifi-Müller, Lawrence, Sen, Müller. Piano Concerto Dataset (PCD): A Multitrack Dataset of Piano Concertos. TISMIR, 2023.

© AudioLabs, 2024

Hans-Ulrich Berendes

Notewise Evaluation of Source Separation: �A Case Study For Separated Piano Trackss

How to obtain the single note events?

Score-Informed Audio Decomposition with NMF

• What we need: splitting audio signal into note events (time-frequency split)

​

• Nonnegative Matrix Factorization (NMF)

🡪 Split spectrogram into two nonnegative matrices

​

• Synchronized score of Piano Concerto Dataset to guide factorization

​

• For source separation evaluation:

Decompose both, reference and separated track

​

​

​

​

​

​

​

Lee, Seung. Algorithms for non-negative matrix factorization. NIPS, 2000.

Ewert, Müller. Using Score-Informed Constraints for NMF-based Source Separation, ICASSP, 2012.

© AudioLabs, 2024

Hans-Ulrich Berendes

Notewise Evaluation of Source Separation: �A Case Study For Separated Piano Trackss

Notewise Evaluation

UMX

SPL

DMC

HDMC

AudioShake

Trackwise

SDR

Left Hand

Right Hand

All

Notes

SDR (dB)

Özer, Müller. Source Separation of Piano Concertos Using Musically Motivated Augmentation Techniques. IEEE TASLP, 2024.

Stöter, Uhlich, Liutkus, Mitsufuji. Open-Unmix–A reference implementation for music source separation. JOSS, 2019.

Hennequin, Khlif, Voituret, Moussallam, Spleeter: A fast and efficient music source separation tool with pre-trained models. JOSS, 2020.

Défossez, Usunier, Bottou, Bach. Music source separation in the waveform domain. arXiv, 2021.

Défossez. Hybrid spectrogram and waveform source separation. ISMIR Workshop Music Source Separation, 2021.

To find out what insights we got into the five tested source separation models, please come to our poster!

© AudioLabs, 2024

Hans-Ulrich Berendes

Notewise Evaluation of Source Separation: �A Case Study For Separated Piano Trackss