1 of 12

MINGUS - Melodic Improvisation Neural Generator Using Seq2seq

Vincenzo Madaghiele

Pasquale Lisena

Raphaël Troncy

1

2 of 12

Experiment

  • Ablation analysis with multiple musical features
  • Training on Weimar Jazz DB and Nottingham DB
  • Multi-level evaluation metrics
  • Comparison of LSTM-based models and Transformers on music generation task
  • Highly adaptable architecture

2

Vincenzo Madaghiele, Pasquale Lisena, Raphaël Troncy

3 of 12

State of the art

  • Modified LSTM-based model with Sequential Attention
  • Conditioning on chords
  • Evaluation with common ML metrics

3

SeqAttn

BebopNet

  • Transformer-based model
  • Conditioning on multiple musical features
  • Evaluation with music-related metrics

Vincenzo Madaghiele, Pasquale Lisena, Raphaël Troncy

4 of 12

Datasets

4

Weimar Jazz DB

Nottingham DB

Vincenzo Madaghiele, Pasquale Lisena, Raphaël Troncy

5 of 12

Data representation

5

Example melody, from Charlie Parker improvisation on Billie’s Bounce, Weimar Jazz DB

Vincenzo Madaghiele, Pasquale Lisena, Raphaël Troncy

6 of 12

MINGUS architecture

6

Vincenzo Madaghiele, Pasquale Lisena, Raphaël Troncy

7 of 12

Accuracy and Perplexity

7

Vincenzo Madaghiele, Pasquale Lisena, Raphaël Troncy

8 of 12

MGEval and harmonic coherence

8

Vincenzo Madaghiele, Pasquale Lisena, Raphaël Troncy

9 of 12

User evaluation

9

Vincenzo Madaghiele, Pasquale Lisena, Raphaël Troncy

10 of 12

Discussion

  • Good generation performance across multiple metrics
  • Especially good at harmonic modeling on complex harmony
  • Motif-level coherence
  • Highly adaptable architecture

10

Conclusions

Limitations

  • Long-term coherence
  • Performance are very dependent on dataset

Vincenzo Madaghiele, Pasquale Lisena, Raphaël Troncy

11 of 12

Future work

  • Automatic harmonization
  • Bass line generation
  • Score music classification
  • Long-term musical coherence

11

Vincenzo Madaghiele, Pasquale Lisena, Raphaël Troncy

12 of 12

Thank you!

Vincenzo Madaghiele, Pasquale Lisena, Raphaël Troncy

[vincenzo.madaghiele, pasquale.lisena, raphael.troncy]@eurecom.fr

​

​

Code, datasets and generated music examples are available at:

git.io/mingus

Web application for evaluation:

mingus.tools.eurecom.fr

12

Vincenzo Madaghiele, Pasquale Lisena, Raphaël Troncy