1 of 33

Collaborations in ML:

why do we even care

Rosanne Liu

Google Brain, ML Collective

https://rosanneliu.com/

​

New in ML Workshop @ ICML 2022

July 18, 2022

2 of 33

What is “collaborate”

(Wrong answers first)

  • ❌ Working with your advisor / manager / assigned mentor
  • ❌ Working with your students / reports
  • ❌ Working with the same group of people for the >1 time
  • ❌ Working with your family

​

(So what is?)

  • ✅ Working with people that you would not normally work with

3 of 33

What is “collaborate”

(Wrong answers first)

  • ❌ Working with your advisor / manager / assigned mentor
  • ❌ Working with your students / reports
  • ❌ Working with the same group of people for the >1 time
  • ❌ Working with your family

​

(So what is?)

  • ✅ Working with people that you would not normally work with

4 of 33

What is “collaborate”

(Wrong answers first)

  • ❌ Working with your advisor / manager / assigned mentor
  • ❌ Working with your students / reports
  • ❌ Working with the same group of people for the >1 time
  • ❌ Working with your family

​

(So what is?)

  • ✅ Working with people that you would not normally work with

5 of 33

What is “collaborate”

(Wrong answers first)

  • ❌ Working with your advisor / manager / assigned mentor
  • ❌ Working with your students / reports
  • ❌ Working with the same group of people for the >1 time
  • ❌ Working with your family

​

(So what is?)

  • ✅ Working with people that you would not normally work with

6 of 33

What is “collaborate”

(Wrong answers first)

  • ❌ Working with your advisor / manager / assigned mentor
  • ❌ Working with your students / reports
  • ❌ Working with the same group of people for the >1 time
  • ❌ Working with your family

​

(So what is?)

  • ✅ Working with people that you would not normally work with

7 of 33

What is “collaborate”

(Wrong answers first)

  • ❌ Working with your advisor / manager / assigned mentor
  • ❌ Working with your students / reports
  • ❌ Working with the same group of people for the >1 time
  • ❌ Working with your family
  • ❌ Working with Geoff Hinton

​

(So what is?)

  • ✅ Working with people that you would not normally work with

8 of 33

What is “collaborate”

(Wrong answers first)

  • ❌ Working with your advisor / manager / assigned mentor
  • ❌ Working with your students / reports
  • ❌ Working with the same group of people for the >1 time
  • ❌ Working with your family
  • ❌ Working with Geoff Hinton

​

(So what is?)

  • ✅ Working with people that you would not normally work with

9 of 33

What is “collaborate”

(Wrong answers first)

  • ❌ Working with your advisor / manager / assigned mentor
  • ❌ Working with your students / reports
  • ❌ Working with the same group of people for the >1 time
  • ❌ Working with your family
  • ❌ Working with Geoff Hinton

​

(So what is?)

  • ✅ Working with people that you would not normally work with

10 of 33

Why should we care about this 👇?

✅ Working with people that you would not normally work with

​

Escaping local optima!

Think SGD, Dropout, data augmentation… all the “randomness” we incorporated in NN training.

​

But there are also more reasons than improving your own optimization.

That is to improve the optimization of society as a whole.

11 of 33

Why We Collaborate

Why We DON’T Collaborate

  • It’s not the fastest route to success.
  • It doesn’t have any short-term benefit on any quantifiable metrics.
  • It incurs lots of peripheral costs.
  • It may result in people getting scooped, taken advantage of, embarrassed, hurt or harmed.
  • …

12 of 33

Why We Collaborate

Why We DON’T Collaborate

  • It’s not the fastest route to success.
  • It doesn’t have any short-term benefit on any quantifiable metrics.
  • It incurs lots of peripheral costs.
  • It may result in people getting scooped, taken advantage of, embarrassed, hurt or harmed.
  • …

13 of 33

Why We Collaborate

Why We DON’T Collaborate

  • It’s not the fastest route to success.
  • It takes a long warm-up period before it starts producing reward.
  • It incurs lots of peripheral costs.
  • It may result in people getting scooped, taken advantage of, embarrassed, hurt or harmed.
  • …

14 of 33

Why We Collaborate

Why We DON’T Collaborate

  • It’s not the fastest route to success.
  • It takes a long warm-up period before it starts gaining reward.
  • It incurs lots of peripheral costs.
  • It may result in people getting scooped, taken advantage of, embarrassed, hurt or harmed.
  • …

🇺🇸: It’s just unnatural

15 of 33

Why We Collaborate

Why We DON’T Collaborate

I get it. You don’t have to, if you think you’re not at a position to make the best use of it yet.

But, for those with:

  • some level of autonomy;
  • certain freedom to choose;
  • more than one terms in their objective for success;

I think You should at least consider it.

16 of 33

Why We Collaborate

Why We DON’T Collaborate

I get it. You don’t have to, if you think you’re not at a position to make the best use of it yet.

But, for those with:

  • some level of autonomy
  • certain freedom to choose
  • more than one terms in their objective for success

I think you should at least consider it.

17 of 33

Now, for those with a choice, why should they choose to collaborate widely?

18 of 33

Now, for those with a choice, why should they choose to collaborate widely?

  1. Individually

19 of 33

Now, for those with a choice, why should they choose to collaborate widely?

powerless

powerful

  • Individually

20 of 33

Now, for those with a choice, why should they choose to collaborate widely?

powerless

powerful

  • Individually

seemingly “safe”

“scary”

21 of 33

Now, for those with a choice, why should they choose to collaborate widely?

powerless

powerful

  • Individually

seemingly “safe”

“scary”

22 of 33

Now, for those with a choice, why should they choose to collaborate widely?

“Science is a team effort, often straddling disciplines and regions. Research collaborations can foster greater understanding, respect trust and knowledge.”

​

https://www.nature.com/collections/dcgahiejgg

2. Collectively

23 of 33

My collaboration story

24 of 33

My collaboration story

My life followed a ladder

25 of 33

My collaboration story

My life followed a ladder

Until it suddenly turned into a jungle gym

26 of 33

The “Jungle Gym” experience has led me to

  • Co-founding and leading ML Collective (the most *nontraditional* research lab)
    • all about fostering collaborations among people with or without affiliations
    • focuses on long-term growth only
    • serving human before serving science

​

  • Joining Google Brain: the most natively collaborative environment in a “traditional” research setting
    • While most places for research are governed with a mix of top-down and bottom-up
    • Brain is more bottom-up than most

27 of 33

vs.

Ladder

Jungle gym

28 of 33

Could it have been any other way?

year

29 of 33

Could it have been any other way?

year

30 of 33

So, why should we care about this 👇?

✅ Working with people that you would not normally work with

​

  1. Escaping local optima?

Think SGD, Dropout, data augmentation… all the “randomness” we incorporated in NN training.

  • Just because it’s hard?
  • Optimize individually, optimize collectively

31 of 33

So, why should we care about this 👇?

✅ Working with people that you would not normally work with

​

  • Escaping local optima?

Think SGD, Dropout, data augmentation… all the “randomness” we incorporated in NN training.

  • Just because it’s hard?
  • Optimize individually, optimize collectively

32 of 33

So, why should we care about this 👇?

✅ Working with people that you would not normally work with

​

  • Escaping local optima?

Think SGD, Dropout, data augmentation… all the “randomness” we incorporated in NN training.

  • Just because it’s hard?
  • Optimize individually, optimize collectively

33 of 33

The eventual goal: a vibrant playground where any kind of players can get a chance.

https://mlcollective.org/

​