1 of 11

Exploring the Inner Workings of Large Generative Music Models

Marcel A. Vélez Vásquez, Charlotte Pouw,

John Ashley Burgoyne, and Willem Zuidema

​

2 of 11

Our motivation

  • Generative models are everywhere
  • Gap in music interpretability:

3 of 11

Our motivation

  • Generative models are everywhere
  • Gap in music interpretability:

4 of 11

Our motivation

  • Generative models are everywhere
  • Gap in music interpretability:

5 of 11

Our motivation

  • Generative models are everywhere
  • Gap in music interpretability

6 of 11

Objective

  • Investigate interpretability techniques on MusicGen
  • Two experiments:
    • DecoderLens
    • Interchange interventions

​

7 of 11

DecoderLens - Method

8 of 11

DecoderLens - Key findings

  • Challenging to decode intermediate layers except for the last couple

​

9 of 11

Interchange intervention - Method

10 of 11

Interchange interventions - Key findings

​

  • Removing original was more successful
  • Introducing new concept is difficult

​

11 of 11

And we did much more!

​

​

…comparison of model sizes

…different intervention techniques

…more extensive evaluation