PyTorch Foundation

Technical Advisory Committee Meeting

March 13, 2025

​

Reminder for meeting participants

• This meeting is being recorded
• Please remember to be on mute upon entry unless you would like to speak
• To unmute a dial-in phone line use *6
• All zoom chat messages will become public
• Attendance is taken purely upon #info in Zoom Chat

2

2

Antitrust Policy Notice

Linux Foundation meetings involve participation by industry competitors, and it is the intention of the Linux Foundation to conduct all of its activities in accordance with applicable antitrust and competition laws. It is therefore extremely important that attendees adhere to meeting agendas, and be aware of, and not participate in, any activities that are prohibited under applicable US state, federal or foreign antitrust and competition laws.

​

Examples of types of actions that are prohibited at Linux Foundation meetings and in connection with Linux Foundation activities are described in the Linux Foundation Antitrust Policy available at http://www.linuxfoundation.org/antitrust-policy. If you have questions about these matters, please contact your company counsel, or if you are a member of the Linux Foundation, feel free to contact Andrew Updegrove of the firm of Gesmer Updegrove LLP, which provides legal counsel to the Linux Foundation.

3

3

Agenda

• Roll Call
• Foundation updates
• Working Group Updates
• Roadmap Discussion
• Open Discussion and call for topics for future meetings

​

Post ideas to the public tac slack channel: https://pytorch.slack.com/archives/C0808K2MN95 #pytorch-tac-public

4

Updated TAC Representatives - Premier

5

5

Foundation Updates

​

PyTorch Foundation has Grown and Matured

​

• The Foundation has changed since its inception
• Membership has grown now to 29 members
• TAC has been formed, chairs elected and structure and process instituted
• Working groups and exploratory groups have been launched
• Conference has grown 6x in attendance in 2 years
• New PyTorch Day events and community-focused meetups
• New programs including Ambassador, OSPO outreach, speaker’s bureau and training and certs
• AI industry is fast growing and more projects released in open source and leveraging PyTorch
• Opportunity to create a stronger community
• Opportunity to host best of breed tools with less integration friction

​

7

7

PyTorch Foundation has Grown and Matured

​

• Proposing changes to the charter for two core objectives:
• Capture and codify the role of the TAC
• Facilitate the ability to adopt new Foundation-hosted projects

​

8

8

An Umbrella Foundation supports Foundation Hosted Projects

What does it mean to be a Foundation hosted Project?

• Project transitions to vendor neutral governance
• Hosted under the PyTorch umbrella
• Trademarks and domains are held by the Foundation as a neutral party
• Project adopts a governance framework consistent with the PyTorch principles

​

9

9

PyTorch Foundation as an Umbrella Foundation

10

10

PyTorch Landscape

​

Self Hosted

PyTorch Ecosystem

Foundation Hosted

PyTorch Core

PyTorch Platform

PyTorch Verticals

future

now

Comparing Technical Project Types

now

future

Benefits to Projects

Project Benefits:

• Gain access to enterprise-grade infrastructure and development resources, including CI/CD pipelines, security scanning tools, and dedicated build environments supported by the Linux Foundation
• Benefit from neutral governance and transparent decision-making processes that protect your project's long-term sustainability while maintaining technical independence and innovation
• Join a thriving technical community with direct access to PyTorch core developers, working groups, and technical steering committees for collaboration and knowledge sharing

Operational Support:

• Leverage professional project management, marketing, and community development support to grow your project's visibility and adoption within the AI/ML ecosystem
• Receive guidance on technical best practices, including security protocols, release management, and documentation standards from experienced maintainers
• Access legal support for intellectual property management, licensing compliance, and trademark protection through the Linux Foundation's legal team

Growth Opportunities:

• Accelerate adoption by becoming part of the official PyTorch ecosystem, reaching a vast community of developers, researchers, and enterprises
• Participate in PyTorch Foundation events, working groups, and technical initiatives to shape the future of AI infrastructure and tooling
• Build credibility through association with one of the world's leading open source AI platforms, backed by major technology companies and research institutions

12

12

How to Host a Project

• Sign contribution agreement
• Requires transfer of marks (word and logo marks) to neutral 3rd party (LF)
• Adopt a governance framework consistent with PyTorch values (LF MVG)
• Setup a Technical Steering Committee (TSC) and join the TAC
• Project maintainers continue to manage project

13

13

Foundation-Project Structure

14

14

Technical Advisory Committee

Governing Board

Technical Project - Technical Steering Committee

• Lead & Core Maintainers
• Contributors
• Community

People

Process

Policies

• Terms of Use
• Privacy Policy
• Code of Conduct
• Antitrust
• Trademarks

• Voting
• Decision Making
• Meetings
• Contributions

Foundation Level

Project Level

• TAC provides support resources
• Manages project lifecycle process
• Approves and retires projects

vLLM

15

15

• vLLM is already a Linux Foundation project in incubation status with LF AI & Data.
• The project relies on PyTorch and is a valuable ecosystem project that could benefit from tighter integration with the PyTorch Foundation.

DeepSpeed

16

16

• DeepSpeed is already a Linux Foundation project in incubation status with LF AI & Data.
• The project relies on PyTorch and is a valuable ecosystem project that could benefit from tighter integration with the PyTorch Foundation.

Proposed Motion

Proposal: PyTorch TAC approves vLLM as a hosted project of the PyTorch Foundation

17

17

Proposed Motion

Proposal: PyTorch TAC approves DeepSpeed as a hosted project of the PyTorch Foundation

18

18

What’s Next?

​

• Accepted projects will need to be onboarded to the Foundation
• Project maintainers should join the TAC meetings
• TAC must define the life cycle process for Platform projects
• TAC must define the requirements for each stage of the life cycle
• TAC Chairs must approve the process and requirements for Platform projects

​

19

19

Working Group Updates

TAC Updates: Projects and Groups

PyTorch Technical Advisory Committee (TAC)

CI Working Group

Multi-cloud Working Group

Ecosystem Working Group

Exploratory Group - Out-of-tree accelerators

Community-led effort, Slack channel #tac-out-of-tree-eg

Exploratory Group Process

Exploratory Groups: TAC-level initiatives that have potential to become full working groups

Process:

• exploratory group is proposed to the TAC via a presentation in a TAC meeting, TAC approves
• requirements: general interest, more than one org in the TAC willing to participate
• a quantifiable goal is set up (can be a working PoC, an analysis, an architectural proposal), and a timeline (e.g. 3 months)
• a Slack channel is set up: all conversations are kept on Slack to allow tracking progress and encouraging participation
• the group reports progress during TAC meetings (just like with working groups)
• after the set time, the group presents at the TAC and TAC votes on whether to convert to working group, give more time, or wind down

22

22

Ecosystem Working Group

​

Ecosystem Landscape is Released!

24

https://landscape.pytorch.org

Ecosystem Process - How

• Users submit by opening an issue in GH using the provided template.
• Discussion, approvals are open, transparent, and visible in GitHub

​

​

• Review and approvals will be managed from a new Ecosystem working group.
• Thank you to all who have volunteered for this team!

https://github.com/pytorch-fdn/ecosystem

CI Working Group Update

​

Cloud Cost Management: Update

Current focus areas

​

1. Cloud Cost Distribution
• Research areas - Community member distribution for specific jobs and use cases, Multi-Cloud CI support
• Who: LF, Meta, NVIDIA, AMD, Microsoft, Multi-cloud working group
2. Migration of GH to the Foundation
• Collecting data to Identify the projects and their status
3. Migration to Cloudflare for CDN
4. Security scanning methodology

​

​

​

​

​

CI Updates

• Credits Budget: 500K monthly
• AWS Costs in January: 488K,
• Credits remaining: 1009K
• LF runners running at 50% load
• (excluding inductor test jobs)

​

• Current projects
• Monitoring improvements and strategy
• Pursuing OSS credits for Datadog or Honeycomb
• Ongoing maintenance

​

See the public project plan: https://lf-pytorch.atlassian.net/jira/software/c/projects/CI/issues/

​

​

Multi-cloud Working Group Update

Andrea Frittoli, IBM

Goal

​

• Develop a sustainable, equitable, community managed approach to the CI/CD pipeline for PyTorch in a multi-cloud environment.

​

• Community lead work, with contributions from AMD, Huawei, IBM, Intel, Meta, NVIDIA, the LF and more

​

30

CONFIDENTIAL | SHARED UNDER NDA

Glossary

​

• Runner: a server, virtual machine or container that can run a GitHub Action job
• Self-hosted runner: any runner that is not managed by GitHub
• PyTorch runner: runners managed by the PyTorch community
• Ecosystem runner: runners managed by a member/vendor company, used in PyTorch CI

31

CONFIDENTIAL | SHARED UNDER NDA

Barriers to Multi-Cloud Today

​

• CI Jobs assume AWS infrastructure
• They cannot be shifted as is to non-AWS runners
• Direct access to storage in AWS is slow and expensive
• The autoscaler efficiently manages a fleet of runners
• Limited to runners in AWS
• Ecosystem runners exist today, but:
• Require custom CI jobs
• High bar of implementation for companies

​

​

32

CONFIDENTIAL | SHARED UNDER NDA

Requirements

• Multi-cloud
• Runners
• Ecosystem Pools
• Scheduling
• Metrics & Monitoring

​

• Budget & Billing
• Security
• Infrastructure as Code & Testing
• Software Supply Chain Security

​

33

CONFIDENTIAL | SHARED UNDER NDA

Proposal: Three workstreams

Stream 1: CI Jobs and Supporting Infrastructure

​

• Make CI jobs portable across clouds
• Make data and credentials required by jobs available across clouds

​

Stream 2: Monitoring and Observability

​

• Extend monitoring and observability to runners across clouds
• Re-use existing components (Hub, Ternary), focus on open standards (OpenTelemetry)

​

Stream 3: Autoscaler & Scheduling

​

• Autoscale PyTorch runners outside of AWS
• Autoscale for ecosystem runners
• Autoscale for containers-based runners

34

CONFIDENTIAL | SHARED UNDER NDA

Stream 1 - CI Jobs and Supporting Infrastructure

• Focus first on following CI Jobs:
• Contributing more significantly to the cost of the CI service
• More likely to run into stockout because of quota limits
• New ones being defined
• Work on Stream 1 can begin now!
• Looking for contributors to this stream
• One person to drive the implementation
• Enables running existing jobs on static runners
• Both PyTorch runners and ecosystem runners
• Focus on the most immediate opportunities
• Oracle Cloud Credits available from CNCF

​

​

​

​

​

​

35

Next Steps

• Stream 1 staffing and implementation
• Experimentation and implementation can start now
• Some design work is still ongoing
• Stream 2 and Stream 3 design to continue
• Present plan from each stream to the TAC
• Staffing and implementation can start in parallel to stream 1
• No silver bullet for cloud-agnostic CI:
• Remove barriers to multi-cloud
• Best practices and open standards to pave the way
• Each runner pool provider will require extra work to integrate
• Focus on the most immediate opportunities first

​

36

PyTorch Out-of-tree Exploratory Group Updates�

2025/03/13

On Mar 7, having first group meeting:

• Discuss goal of out-of-tree CI
• Actionable next steps

​

Meeting Record & Slide: https://drive.google.com/drive/folders/1dD_71aZTHfQpkcSKl595ZcadWwagjWkQ

​

Group Events

Building an evolving quality validations and standards for

PyTorch out-of-tree accelerators.

• Integration Tests
• E2E Model Validations
• Accelerator Quality Standards (in Future, RFC)

​

​

More details in Validation of Out-of-Tree Accelerator Integration for PyTorch

Proposal

Expect CI workflow for Out-of-tree Accelerators

1. PR trigger workflow capability verification (Non-blocking)
1. Test E2E workflow on self-created repos (org cosdt/gaudi)
2. Migrate implementation to upstream
2. Gather more Out-of-tree relative concern topics
• [RFC] Test Cases Enabling for Accelerators
• Enhancement(Graphcore): General testing/CI/Device abstraction/Docs
3. Setup a periodic meeting
• Vote for time/period will post in channel #tac-out-of-tree-eg

Next Steps

​

​

Join the Discussion on the Slack Channel

https://pytorch.slack.com/archives/C084MU6PS7N

42

Questions and discussion

Roadmap discussion

​

​

https://dev-discuss.pytorch.org/t/meta-pytorch-team-2025-h1-roadmaps/2794

PyTorch Conference

​

Call for Reviewers!

Join the PyTorch Conference Program Committee to review and score submissions, providing valuable feedback to craft an engaging and impactful event.

💡 Time Commitment: Contribute as little as 1 hour or up to 5 hours, based on your availability and expertise.

📅 Submit Interest by April 13

👉 Learn More & Apply

​

46

Open Discussion

Any team member

Next meeting

Next Meeting

​

• Tuesday, April 8, 2025 at 10am EST
• Please ensure you are registered for the meeting series and that meetings are reflected in your calendar via LFX Meetings

​

​

49

49

Call for Agenda Items

• Deep dive topics?
• Community presentations?
• Tech talk topics?

​

Send ideas to nrausch@linuxfoundation.org

​

​

50

50

Addendum

General Information

Links

Mailing List: �https://lists.pytorch.org/g/tac

Past Meeting Slide Decks: �https://lists.pytorch.org/g/tac/messages

General PyTorch Governance: https://pytorch.org/docs/master/community/governance.html

PyTorch Labs:�https://github.com/pytorch-labs

​

Submit an Ecosystem Tool:

https://github.com/pytorch-fdn/ecosystem �

Submit a Blog:�https://form.asana.com/?k=mPrsX9EcZ0BoGnqz543C1A&d=9283783873717 �

Points of Contact

Matt White (Executive Director)

Nancy Rausch (Program Manager)

Jennifer Bly (Marketing Director)

Regina Nkenchor (Program Manager)��

52

Legal Notices

The Linux Foundation, The Linux Foundation logos, and other marks that may be used herein are owned by The Linux Foundation or its affiliated entities, and are subject to The Linux Foundation’s Trademark Usage Policy at https://www.linuxfoundation.org/trademark-usage, as may be modified from time to time.

Linux is a registered trademark of Linus Torvalds. Please see the Linux Mark Institute’s trademark usage page at https://lmi.linuxfoundation.org for details regarding use of this trademark.

Some marks that may be used herein are owned by projects operating as separately incorporated entities managed by The Linux Foundation, and have their own trademarks, policies and usage guidelines.

TWITTER, TWEET, RETWEET and the Twitter logo are trademarks of Twitter, Inc. or its affiliates.

Facebook and the “f” logo are trademarks of Facebook or its affiliates.

​

LinkedIn, the LinkedIn logo, the IN logo and InMail are registered trademarks or trademarks of LinkedIn Corporation and its affiliates in the United States and/or other countries.

YouTube and the YouTube icon are trademarks of YouTube or its affiliates.

All other trademarks are the property of their respective owners. Use of such marks herein does not represent affiliation with or authorization, sponsorship or approval by such owners unless otherwise expressly specified.

The Linux Foundation is subject to other policies, including without limitation its Privacy Policy at https://www.linuxfoundation.org/privacy and its Antitrust Policy at https://www.linuxfoundation.org/antitrust-policy. each as may be modified from time to time. More information about The Linux Foundation’s policies is available at https://www.linuxfoundation.org.

Please email legal@linuxfoundation.org with any questions about The Linux Foundation’s policies or the notices set forth on this slide.

​

​