1 of 85

Why

Research

Music?

Doug Turnbull

Ithaca College, USA

IndabaX Nigeria

September 2021

Thanks for having me.�I wish that I could be there in person.

Why Research Music?

Talk Abstract:

Music is important. It entertains us, is part of our culture, and is a powerful way to communicate with one another. Music is also a big industry with over $60B USD (₦24T NGN) in global revenues per year. In this talk, I’ll motivate why music is a rich domain for machine learning research. I also describe how one can get started using resources that are being developed by a growing community of music information retrieval researchers.

Speaker Biography:�Doug Turnbull is an Associate Professor in the Department of Computer Science at Ithaca College in New York, USA. His research focuses on music information retrieval, computer audition, and machine learning. He is currently working on Localify.org which explores using music event recommendations and playlist generation on Spotify to support local music communities. This project is funded by the National Science Foundation and being developed by a large team of undergraduate students at Ithaca College and Cornell University. More information about his research can be found at https://dougturnbull.org.

2 of 85

Music is ...

3 of 85

Def’n:

vocal and/or instrumental sounds combined in such a way as to produce beauty of form, harmony, and expression of emotion.

- Oxford Languages Dictionary

4 of 85

Music is important

5 of 85

Entertainment

Culture

Religion

Political

Communication

6 of 85

7 of 85

Entertainment

Culture

Religion

Political

Communication

8 of 85

9 of 85

10 of 85

Entertainment

Culture

Religion

Political

Communication

11 of 85

12 of 85

13 of 85

14 of 85

Entertainment

Culture

Religion

Political

Communication

15 of 85

16 of 85

Entertainment

Culture

Religion

Political

Communication

17 of 85

18 of 85

Entertainment

Culture

Religion

Political

Communication

19 of 85

Music is important

20 of 85

Music is big business

21 of 85

22 of 85

23 of 85

Music is always changing

24 of 85

25 of 85

Created

Produced

Distributed

Shared

Consumed

26 of 85

How do you listen to

music?

27 of 85

How did you listen to music 10 years ago?

28 of 85

How did you listen to music 20 years ago?

29 of 85

30 of 85

31 of 85

Music is beautiful

32 of 85

33 of 85

Music is important

Music is big business

Music is always changing

Music is beautiful

34 of 85

But I’m a Machine Learning Researcher...

35 of 85

Data

36 of 85

Data

Music Information

37 of 85

Audio Content

Listening Histories

Music Scores

Lyrics

Tags

Images

Videos

38 of 85

Audio Content

Listening Histories

Music Scores

Lyrics

Tags

Images

Videos

39 of 85

Representation

40 of 85

2D Representation

41 of 85

Low-level Representation

1D Time Series

2D Image

Hand-Designed Features

RMS

ZCR

MFCC

Chromagram

Rhythmic Patterns

Tempogram

...

Supervised Learning

Nearest Neighbors

Support Vector Machine

Decision Trees

Ensemble Methods

...

Machine Learning

42 of 85

Low-level Representation

1D Time Series

2D Image

Convolutional Neural Networks

...

Recurrent Neural Networks

Deep Learning

43 of 85

Tasks

Classification

44 of 85

45 of 85

Tasks

Classification

Structural Segmentation

Transcription

Beat Tracking

46 of 85

Application

Content-based Recommendation

Music Education

Intelligent Digital Audio Workstation

Fingerprinting

47 of 85

Tutorial Later Today!

48 of 85

Audio Content

Listening Histories

Music Scores

Lyrics

Tags

Images

Videos

49 of 85

Audio Content

Listening Histories

Music Scores

Lyrics

Tags

Images

Videos

50 of 85

51 of 85

How do I

recommend music?

52 of 85

53 of 85

User-Song Matrix Representation

r_i,u = how much user u likes (or dislikes) song s

song s

user u

|U|x|I|

r_u,s

54 of 85

Recommender System Algorithms

Neighborhood Approaches

User-User
Item-Item

Matrix Factorization Approaches

Alternating Least Squares (ALS)
Sparse Linear Method (SLIM)

(Deep) Network Approaches

Embarrassingly Shallow Autoencoders (EASE)
Multiple Variational Autoencoder (Multi-VAE)

55 of 85

What makes music recommendation special:

Music can be consumed quickly
Many different listening contexts
A user can form an opinion quickly
Repeat recommendations
Implicit feedback
Extreme Long Tail distribution

Schedl, M., Knees, P., McFee, B., Bogdanov, D., & Kaminskas, M. (2015). Music recommender systems. In Recommender systems handbook (pp. 453-492). Springer, Boston, MA.

56 of 85

57 of 85

Access to Music ≠ Fair Exposure to Music

58 of 85

59 of 85

Can we use

music recommendation for

social & economic good?

60 of 85

61 of 85

Sound techs

Front office

printer

Local builders

Catering

Local restaurants

hotels

62 of 85

63 of 85

64 of 85

65 of 85

66 of 85

67 of 85

68 of 85

69 of 85

70 of 85

71 of 85

72 of 85

73 of 85

74 of 85

75 of 85

How can you use

music recommendation

to support your local

Nigerian music communities?

76 of 85

77 of 85

Audio Content

Listening Histories

Music Scores

Lyrics

Tags

Images

Videos

78 of 85

Audio Content

Listening Histories

Music Scores

Lyrics

Tags

Images

Videos

79 of 85

Music Information�Retrieval

80 of 85

Tutorial Later Saturday at 11:40 am!

81 of 85

82 of 85

83 of 85

84 of 85

Women in

Music IR

Mentoring Program

Johanna Devaney

Saturday at 2:30

85 of 85

Why

Research

Music?

Doug Turnbull

Ithaca College, USA

IndabaX Nigeria

September 2021

Thanks for having me.�I wish that I could be there in person.

Why Research Music?

Talk Abstract:

Music is important. It entertains us, is part of our culture, and is a powerful way to communicate with one another. Music is also a big industry with over $60B USD (₦24T NGN) in global revenues per year. In this talk, I’ll motivate why music is a rich domain for machine learning research. I also describe how one can get started using resources that are being developed by a growing community of music information retrieval researchers.

Speaker Biography:�Doug Turnbull is an Associate Professor in the Department of Computer Science at Ithaca College in New York, USA. His research focuses on music information retrieval, computer audition, and machine learning. He is currently working on Localify.org which explores using music event recommendations and playlist generation on Spotify to support local music communities. This project is funded by the National Science Foundation and being developed by a large team of undergraduate students at Ithaca College and Cornell University. More information about his research can be found at https://dougturnbull.org.