1 of 7

MLCommons and �Science Data WG

2 of 7

We are forming a non-profit called “MLCommons”

The MLCommons mission is to accelerate ML innovation and increase its positive impact on society.

​

3 of 7

What will ML Commons do?

Phase 1:� Administer MLPerf benchmarks: dedicated and neutral staff will help us scale, make submission easier, and ensure longevity� Create large, public, more diverse datasets: datasets drive ML innovation and more than anything else, and public datasets are 10-100x smaller than industry

​

Phase 2, partial list:� Best practices: use benchmarks, datasets to develop best practices that increase portability and reduce friction → faster adoption, more customers, and bigger market� Outreach activities: challenges to nurture academic research and highlight positive impact of ML, curated resource lists e.g. responsible ML best practices

​

4 of 7

MLCommons Five Principles

  1. Grow ML markets and make the world a better place�
  2. Get everyone involved
    1. Be global, inclusive, and fair
    2. Serve academia, small companies, and large companies
    3. Make it easy for “prospective members” to get involved
    4. Be as open with our IP as possible while still paying the rent�
  3. Act through collective engineering
    • Keep leadership mostly technical, hands-on-involved folks with smattering of experienced org folks for wisdom
    • Favor data-driven decisions, design simplicity, and focus on real user value�
  4. Make fast but consensus-supported decisions
    • Very low barrier for “experimental” working groups with well reviewed path to full endorsement
    • Favor grudging consensus over 51/49 votes, especially for big decisions
    • Minimize participation paperwork
    • Make communication electronic and rapid
    • Make technical contributions easy

​

​

​

  1. Build an org people want to be part of
    1. Be welcoming, informal, and friendly
    2. Recognize and reward contributions
    3. Celebrate with cake

5 of 7

Why support MLCommons?

Organization

Primary benefits

Academic or small software company

Better public data sets enable enable you to do research, create new ML-driven products or services

Large software company

Research and small companies provide innovation to improve products / introduce new ones

​

Benchmarks accelerate hardware development, improving perf/$ and lowering opex; ensure benchmarks match your use case

Chip, system integration, or hardware design company

More use of ML by rapidly growing small companies and rapidly innovating big companies gives you a bigger market

​

Data sets enable better designs and value add offerings

Everyone

Learn from industry peers and leaders in different areas

6 of 7

Transition Plan includes

​

  1. Working Groups will be required to follow the MLCommons Bylaws, Working Group Policy, and Code of Conduct. We crafted this policy to be as lightweight as possible, and the biggest change is to require high-level minutes using a specific template with a legal disclaimer at the top. We will have a short guide for working group chairs and more support, e.g. informational meeting, soon.

​

  • We will consolidate existing working groups and chairs into this organization. Key changes:
    1. Merge Submitters/Special Topics/Results trios into Inference and Training groups to simplify things (i.e. only one meeting invite and mailing list) and because this change has already happened in practice.
    2. Decommission working groups that are no longer active such as Pre-Silicon.
    3. Ensure each working group has two (2) co-chairs as required by the MLCommons WG policy.
    4. Potentially introduce a few cross-group chairs (to be proposed in separate email) for things like process and systems.

​

  • Working Groups and Working Group mailing lists will become MLCommons Members Only.
    • We know some working groups like HPC, Tiny, and Research Sub-groups may be too early stage for many of the participants to join and that we have one non-member WG chair. We’re working on creatively handling these cases. :-)�
  • Submission to an MLPerf Benchmark will require membership.
    • Membership is free for academics.

7 of 7

Board & �Executive Director

Inference

Training

HPC

Power

Mobile

Datasets

Best Practices

Research

Scientific

Medical

Tiny

System Description & Logging

Advisory

Boards

Cross-group??

(e.g. Process, Systems, Community <locale>)

Algorithms