Scalable-I/O Virtualization (SIOV) R2 �Overview�

OCP Workstream

Server Project

​

Last Update: October 2023

​

Contact Point: siovr2.admin@opencompute.org

What is Scalable I/O Virtualization (SIOV)?

• SIOV is hardware-assisted I/O virtualization designed for the hyperscale era, with the potential to support thousands of virtualized workloads per server.
• SIOV moves the non-performance-critical virtualization and management logic off the device and into the virtualization stack. It uses a scalable identifier on the device to address the workloads’ memory.

​

• SIOV delivers key benefits vs. prior art:
• Reduces the per-VM virtualization cost on the devices
• More efficiently supports large numbers of VMs and containers
• Provides more flexibility to the virtualization stack for provisioning & composability.

​

• Value Proposition: Virtualized I/O devices become much more configurable and scalable while delivering near-native performance to each VM/container/microservice.

Why Scalable I/O Virtualization (SIOV)?

• SIOV enables virtualization modernization of I/O devices (PCIe, CXL, etc.) for today’s massive-scale clouds
• SIOV is much more scalable and flexible than previous technologies, providing efficient I/O access to potentially thousands of containers or virtual machines
• SIOV is simpler and more efficient to implement and deploy for I/O device manufacturers, software providers and cloud operators

​

• Driving as a standard enables an open ecosystem that fuels adoption, growth, and innovation
• Host Devices
• Target Devices
• Software & Operating Systems

SIOV R1 Contribution & R2 Workstream Plan

• Intel & Microsoft contributed SIOV R1 to OCP to encourage an open ecosystem hyperscale I/O virtualization
• Scalable I/O Virtualization
• https://www.opencompute.org/documents/ocp-scalable-io-virtualization-technical-specification-revision-1-v1-2-pdf
• Arm, AMD, Ampere, Broadcom, Intel, Microsoft, Nvidia, & Red Hat formed an OCP SIOV R2 workstream to drive a broader ecosystem & further enhance data center I/O virtualization
• High level Scope:
• Ensure definition is compatible across multiple host and device architectures to provide a consistent usability experience.
• Ensure definition meets new, emerging, and complementary usages.
• Ensure a viable ecosystem transition path to SIOV Revision 2.0.

SIOV R2 Workgroup Scope

• Ensure Specification is compatible across multiple host and device architectures to provide a consistent usability experience. Key focus areas:
• Standard method for Assignable Device Interface (ADI) identification and enumeration, and ADI resource management (e.g. address spaces and interrupts)
• Standard methods for scaling ADI usage to future virtualization deployments (e.g. efficient/scalable interrupts or notifications, performance scaling, host/device communication efficiencies)
• Reference software implementations of the Specification
• If necessary, propose revisions to the generic mechanisms provided by other standards to ensure ecosystem interoperability with SIOV Revision 2.0 virtualization model
• Quality of Service (QoS) associations and management for ADIs
• Reliability, Availability, and Serviceability (RAS) capabilities and expectations
• Privilege/execution levels of virtualization software to enable optimal device resource management and isolation properties

​

• Ensure Specification meets new, emerging, and complementary usages. Key focus areas:
• Extensions to support confidential computing (trusted execution) with ADIs
• Extensions to optimize live migration of virtual devices backed by ADIs
• Enhancements to Address Translation Services (ATS) for ADIs
• Extending support and interoperability to Peer-2-Peer (P2P) topologies

​

• Ensure a viable ecosystem transition path to SIOV Revision 2.0. Key focus areas:
• Backward compatibility to SIOV Revision 1.0
• Heterogeneous deployment and coexistence of devices within a system, including SR-IOV, SIOV Revision 1.0, and SIOV Revision 2.0 target devices
• Ensure that key properties of prior virtualization models have reasonable corollary properties in SIOV Revision 2
• Specific transition guidance (e.g. SR-IOV to SIOV Revision 2.0 device transition guidance)

SIOV R2 Workgroup Participation

• Workgroup members have aligned to:
• Scope focus (prior slide and incorporate into CLA)
• Operating governance model
• Public distribution of specification at version 0.7 (and beyond)
• Expanding to new membership after releasing 2 public versions (0.7, 0.8)
• New membership requirements:
• Written details of intended contribution (actual IP, participation, deliverables) from new member
• 75% majority existing member approval required to accept
• Signature of existing CLA, unmodified (includes/aligns to scope)

​

BACKUP

SR-IOV (Single-Root I/O V) 🡪 Scalable I/O Virtualization (Scalable-IOV)

9

VM 1

Container 1

VM x

Container x

VM x

Container x

VM p

Container p

• While SR-IOV maps virtual devices as Virtual Functions (VFs) in HW, S-IOV composes virtual devices through light-weight and scalable Assignable Device Interfaces (ADIs) for fast-path operations and software mediated slow-path operations

I/O Interface Agnostic (PCIe, CXL, etc.)

PCIe

Scalable IOV Benefits

10

Scalable IOV: High Level Software Architecture

14

Device specific VDCM to decide what to intercept vs direct map

Emulating infrequent operations in VDCM simplifies device without compromising performance

Host Driver

Guest Driver

Virtual Device Composition Module

(VDCM)

IOMMU

(w/ Scalable IOV

extensions)

VMM

Hosting

Device

Function

………………….....

PASID

PASID

Frequent

Direct Mapped Operations

Host

Guest

VDEV

Infrequent

Host Intercepted Operations

Device-specific software components

………

Backend Resources

AI Resource Mapping

VM Manager