Linux on RISC-V

Drew Fustini <dfustini@baylibre.com>

Slides: tinyurl.com/riscv-kr-22

June 1-3, 2022

$ whoami

• Linux kernel developer, BayLibre
• embedded software consultancy based in Nice, France, with ~50 engineers around the world contributing to open source projects like Linux, U-Boot, Android and Zephyr
• Board of Directors, BeagleBoard.org Foundation
• Board of Directors, Open Source Hardware Association (OSHWA)
• OSHW Certification Program
• Ambassador, RISC-V International

RISC-V: a Free and Open ISA

• Started by a computer architecture research group at University of California Berkeley in 2010 led by Krste Asanovic
• V as in the roman numeral five, because it is the 5th RISC instruction set to come out of UC Berkeley
• Free and Open because the specifications are published under an open source license: Creative Commons Attribution 4.0 International
• Volume 1, Unprivileged Spec v. 20191213 [PDF]
• Volume 2, Privileged Spec v. 20211203 [PDF]

What is different about RISC-V?

• Simple clean-slate design
• Avoids any dependencies on microarchitecture style (in-order, out-of-order, etc)
• Modular design
• Suitable for everything from microcontrollers to supercomputers
• Stable base
• Base integer ISAs and standard extensions are frozen
• Additions via optional extensions, not new versions

(source: Instruction Sets Want to be Free, Krste Asanovic)

RISC-V base integer ISAs

• RV32I: 32-bit
• less than 50 instructions needed!
• RV64I: 64-bit
• Most important for Linux
• RV128I: 128-bit
• Future-proof address space

(source: RISC-V Summit 2019: State of the Union, Krste Asanovic)

RISC-V base integer registers

• XLEN defines the register width
• XLEN=32 for RV32I
• XLEN=64 for RV64I
• 32 registers named x0 to x31
• Dedicated PC register
• Base ISA talk by Andrew Waterman explains the instruction encoding scheme

(source: Figure 2.1: RISC-V base unprivileged integer register state)

RISC-V ABI

• x1 to x31 are all equally general-use registers as far as the processor is concerned
• RISC-V psABI defines standard functions for these registers [PDF]
• s0 to s11 are preserved across function calls
• argument registers a0 to a7 and the temporary registers t0 to t6 are not

(source: RISC-V Assembly Programmer's Manual)

RISC-V Standard Extensions

• M: integer multiply/divide
• A: atomic memory operations
• F, D, Q: floating point, double-precision, quad-precision
• G: “general purpose” ISA, equivalent to IMAFD
• C: compressed instructions conserve memory and cache
• Most Linux distributions target RV64GC

(source: RISC-V Summit 2019: State of the Union, Krste Asanovic)

Ratified in 2021

• 15 new specifications representing more than 40 extensions
• Vector
• Hypervisor
• Scalar Cryptography
• Bit Manipulation

RISC-V Profiles

• RISC-V is a highly modular and extensible architecture
• Flexibility to pick and choose what is right for your processor design, but that flexibility creates a large number of possible combinations
• RISC-V Profiles specify a much smaller set of ISA choices that represent the most common use-cases
• RVM for microcontrollers intended to run bare-metal code or an RTOS
• RVA for application processors designed to run full operating system like Linux
• RISC-V Summit talk by Greg Favor [slides]

Learn more about RISC-V

• Get up-to-speed quick with the RISC-V Reader

Learn more about RISC-V

• Textbook: Computer Organization and Design, RISC-V Edition

RISC-V and Industry

• RISC-V International now controls the specifications: riscv.org
• Non-profit with 2,700+ members including companies & universities from 70 countries
• Become a member - free of cost to individuals and non-profits!
• RISC-V International YouTube channel has hundreds of talks
• Companies have already shipped billions of RISC-V cores
• Nvidia GPUs have RISC-V cores for system management tasks
• Seagate and Western Digital are using RISC-V cores in storage controllers

(source: State of the Union, Krste Asanovic)

RISC-V and Industry

• No ISA licensing fees or royalties
• Avoid legal costs and delays caused by complex licensing agreements
• Freedom to choose microarchitecture implementation
• An open ISA means that everyone has an architecture license
• Freedom to leverage existing open source implementations
• Broad range of open source cores already available from small embedded cores to high-performance out-of-order superscalar designs

“Is RISC-V an Open Source processor?”

• RISC-V is a set of specifications under an open source license
• RISC-V implementations can be open source or proprietary
• Open specifications make open source implementations possible
• An open ISA makes it possible to design an open source processor

RISC-V open source cores

• Academia
• Rocket and BOOM from Berkeley, PULP family of cores from ETH Zurich
• Industry
• SweRV created by Western Digital and now developed by CHIPS Alliance
• OpenHW Group creating proven verified IP like their Core-V designs
• Google OpenTitan silicon root of trust project uses LowRISC Ibex core

RISC-V open source cores

• FPGA soft-cores
• PicoRV32, RVfpga, SERV, and VexRiscV
• FOSSi Foundation
• El Correo Libre monthly newsletter for the latest on open source cores
• Build your own open source SoC with an open source silicon toolchain
• Google worked with Skywater to open source their 130 nm PDK (process development kit). Google offers free-of-cost MPW (multi-project wafer) runs to open source projects. Learn to design your own using open source design tools with Zero to ASIC.

RISC-V software ecosystem

• RISC-V already has a well supported software ecosystem
• RISC-V International software committee coordinates efforts of member organizations
• RISC-V extension and feature support
• PLCT Lab led by Wei Wu at ISCAS does a lot of compiler and runtime work
• Operating systems: Linux, BSDs, FreeRTOS, Zephyr
• Toolchains & libraries: gcc, glibc, gdb, binutils, clang/llvm, newlib
• Languages and Runtimes: V8, Node.js, Rust, Go, OpenJDK, Python

RISC-V Privileged Architecture

• Three privilege modes
• User (U-mode): application
• Supervisor (S-mode): OS kernel
• Machine (M-mode): firmware
• Environment Call (ECALL) instruction
• Transfer control to a higher privileged mode
• Userspace program (u-mode) uses ECALL to make system call into OS kernel (s-mode)

Control and Status Registers (CSRs)

• CSR have their own dedicated instructions to read and write
• CSR are specific to a mode (e.g. m-mode and s-mode)
• Machine Status (mstatus) is an important CSR

(source: Introduction to the RISC-V Architecture [PDF],Drew Barbier)

RISC-V Virtual Memory

• satp CSR (Supervisor Address Translation and Protection) controls supervisor-mode address translation and protection
• Sv32: 3 level page table
• Sv39: 3 level page table
• Sv48: 4 level page table
• Sv57: 5 level page table

(source: Demystifying the RISC-V Linux software stack, Nick Kossifidis)

RISC-V Trap Handling

• Exceptions occur synchronously
• Interrupts occur asynchronously
• <x>cause CSR indicates which interrupt or exception occurred
• mcause for m-mode / scause for s-mode
• Corresponding bit is set in <x>E/IP CSR

(source: RISC-V Privileged Architecture [PDF], Allen Baum)

What is a Hart?

• Hart is a hardware thread
• Each RISC-V core contains an independent instruction fetch unit
• A RISC-V core with multi-threading (SMT) would contain multiple harts
• Each hart is a processor from the perspective of Linux
• Imagine a RISC-V laptop which has 2 cores with 2 harts per core
• Linux would see 4 processors

(source: Section 1.1 in RISC-V Unprivileged spec)

RISC-V Interrupts

• Local per-hart interrupts
• CLINT (Core Local Interruptor)
• CLIC (Core Local Interrupt Controller)
• Global interrupts
• PLIC (Platform Level Interrupt Controller)

(source: RISC-V Fast Interrupts [PDF], Krste Asanovic)

Advanced Interrupt Architecture (AIA)

• Developed on the AIA SIG mailing list: tech-aia
• APLIC (Advanced Platform-Level Interrupt Controller) replaces PLIC
• Adds IMSIC (Incoming Message-Signaled Interrupt Controller) for PCIe
• AIA is complimented by ACLINT (Advanced Core Local Interruptor)
• Developed on the tech-unixplatformspec mailing list
• Backwards compatible with the SiFive CLINT but restructured to be more efficient
• RISC-V Summit talk by Anup Patel and John Hauser [slides]

RISC-V Boot Flow

Boot ROM

M-mode

First stage bootloader

(U-Boot SPL or vendor firmware)

U-Boot

S-mode

Linux

kernel

RISC-V Boot Flow

Boot ROM

M-mode

U-Boot

S-mode

Linux

kernel

SBI

First stage bootloader

(U-Boot SPL or vendor firmware)

Supervisor Binary Interface (SBI)

• Non-ISA RISC-V specification
• This means it does not add or modify and RISC-V instructions
• The calling convention between S-mode and M-mode
• Allows supervisor-mode (s-mode) software like the Linux to be portable across RISC-V implementations by abstracting platform specific functionality

Supervisor Binary Interface (SBI)

• Required by the UNIX-Class Platform Specification
• Mailing list: tech-unixplatformspec
• This will be replaced by the upcoming RISC-V Platform Specification
• Small core along with a set of optional modular extensions
• Base extension - query basic information about the machine
• Timer extension - program the clock for the next event
• IPI extension - send an inter-processor interrupt to harts defined in mask
• RFENCE extension - instructs remote harts to execute FENCE.I instruction

SBI Extensions

• Hart State Management (HSM)
• S-mode can request to stop, start or suspend a hart
• System Reset
• Supervisor-mode software can request system-level reboot or shutdown
• Performance Monitoring Unit
• Interface for supervisor-mode to configure and use the RISC-V hardware performance counters with assistance from the machine-mode
• ”Performance Monitoring in RISC-V using perf” by Atish Patra

Hypervisor extension

• Hypervisor Supervisor mode (HS-mode) where host kernel runs
• Virtualized Supervisor mode (VS-mode) where the guest kernel runs

OpenSBI

• Open source implementation of SBI
• Core library
• Platform specific libraries
• Full reference firmware for some platforms
• Provides runtime services to S-mode software
• SBI extensions present on a platform define the available runtime services
• Unimplemented instructions will trap and OpenSBI can emulate

(source: OpenSBI Deep Dive, Anup Patel)

OpenSBI Generic Platform

• No need to add code to OpenSBI for each new platform
• First-stage bootloader, like U-Boot SPL, is expected to pass a Device Tree�to OpenSBI which describes all the platform specific functionality
• The same OpenSBI binary can be used across platforms
• Many RISC-V boards and emulators now use Generic Platform
• Linux distros do not need to ship a different OpenSBI build for each board

OpenSBI Domain Support

• An OpenSBI domain is a system-level partition of underlying hardware having its own memory regions and HARTs
• Talk by Anup Patel

​

UEFI Support

• UEFI is a standard interface between firmware and operating systems, and it is used on most x86 and arm64 platforms
• U-Boot and TianoCore EDK2 both have UEFI implementations on RISC-V
• Grub2 can be used as an UEFI payload on RISC-V
• UEFI support for RISC-V added in Linux 5.10

UEFI Support

• Boot hart ID is known only at boot and it is needed before ACPI tables or DT properties can be parsed
• Hart ID is passed in the a0 register on non-UEFI systems, but the UEFI application calling conventions do not allow this
• RISC-V EFI Boot Protocol allows the OS to discover the boot hart ID
• The public review process has completed, and Sunil V L has added support to the Linux kernel for RISCV_EFI_BOOT_PROTOCOL

RISC-V Platform Specification

• Goal is to support “off-the-shelf” software by standardizing the interface between hardware platforms and operating systems
• Created by the Platforms Horizontal Subcommittee (HSC)
• Bi-weekly meetings chaired by Kumar Sankaran
• Mailing list: tech-unixplatformspec
• Platforms talk at RISC-V Summit 2021
• Philipp Tomsich, Chair of Software HSC, and Mark Himelstein CTO RISC-V International

RISC-V Platform Specification

• OS-A Platform
• “A” as in application, this is a category of platforms that support full OS like Linux
• OS-A Common Requirements
• OS-A Embedded Platform
• OS-A Server Platform
• RVM-CSI Platform
• Bare-metal applications or RTOS running on RISC-V microcontrollers
• CSI is common software interface; goal is to ease porting, not binary compatibility

RISC-V Platform Specification

• OS-A common requirements for Embedded and Server
• Must comply with the RVA22U and RVA22S ISA profiles as defined in RISC-V ISA Profiles
• Common requirements for Debug, Timers, Interrupt Controllers
• Requires serial console with UART 8250 or UART 16550
• Requirements for runtime services such as SBI extensions
• Software components must comply with the RISC-V Calling Convention specification and the RISC-V ELF specification

RISC-V Platform Specification

• OS-A Embedded Platform
• Target might be a single board computer or mobile device
• PMU counters and events for performance monitoring
• Boot process must comply with Embedded Base Boot Requirements (EBBR) spec
• EBBR requires a subset of the UEFI spec which U-Boot has implemented
• Device Tree (DT) is the required mechanism for the hardware discovery and config
• GPT partitioning required for shared storage

RISC-V Platform Specification

• OS-A Server Platform
• Goal is for an enterprise Linux distro like RHEL to “just work” on server-class hardware that complies with this
• System peripheral requirements like PCIe, watchdog timer, system date/time
• RAS (Reliability, Availability, and Serviceability) requirements like ECC RAM
• ACPI is the required mechanism for the hardware discovery and configuration
• Must comply with the RISC-V ACPI Platform Requirements Specification

​

RISC-V ACPI Platform Specification

• Defines mandatory ACPI tables and objects for RISC-V server platforms
• New tables are needed for RISC-V
• RISC-V Hart Capabilities Table (RHCT)
• RISC-V Timer Description Table (RTDT)
• More details in ‘ACPI for RISC-V: Enabling Server Class Platforms’
• Sunil V L from Ventana Microsystems at RISC-V Summit [slides]

RISC-V emulation in QEMU

• Support for RISC-V in mainline QEMU
• Boots 32-bit and 64-bit mainline Linux kernel
• Machine configs to boot same binaries as some RISC-V dev boards
• Supports the new Hypervisor and Vector extensions

RISC-V in the Linux kernel

• Initial port by Palmer Dabbelt was merged into Linux 4.15 back in 2018
• “It’s a fun, friendly, and still pretty small community” - Björn Töpel [1]
• Palmer continues to maintain the riscv tree
• Development happens on the linux-riscv mailing list
• View the archive on lore.kernel.org
• IRC: #riscv on libera.chat

Recently added to Linux

• KVM RISC-V support by Anup Patel added in Linux 5.16
• Add KVM support for the Hypervisor specification
• SBI SRST extension support by Anup Patel added in Linux 5.17
• Support for the SBI SRST (system reset) extension which allows systems that do not have an explicit driver in Linux to reboot

New in Linux 5.18

• Add Sv57 page table support by Qinglin Pan
• use 5-level page table to support Sv57 which expands the virtual address space to �57 bits (128 petabytes)
• Improve RISC-V Perf support by Atish Patra
• Recent talk: Perf on RISC-V: The Past, the Present and the Future (slides)

New in Linux 5.18

• RISC-V CPU Idle support by Anup Patel
• cpuidle and suspend drivers now support the SBI HSM extension
• Provide framework for RISC-V ISA extensions by Atish Patra
• Linux was no longer correctly parsing the RISC-V ISA string as the number of RISC-V extensions has grown and extension names are no longer a single character
• This series implements a generic framework to parse multi-letter ISA extensions.
• Based on initial work by Tsukasa OI

Coming in Linux 5.19…

• RISC-V Patches for the 5.19 Merge Window, Part 1 - Palmer (2022-05-31)
• Support for page-based memory attributes <we’ll dive into that topic in a few slides>
• Support for running rv32 binaries on rv64 systems via the compat subsystem
• Support for kexec_file()
• Support new generic ticket-based spinlocks, which allows us to also move to qrwlock
• A handful of cleanups and fixes, include some larger ones around atomics and XIP
• Part 2? Follow linux-riscv and look at Palmer’s riscv/for-next branch

Work in progress

• [PATCH v10 00/16] riscv: Add vector ISA support by Greentime Hu
• Vector ISA support based on the ratified Vector 1.0 extension
• Defines new structure __riscv_v_state in struct thread_struct to save/restore the vector related registers. It is used for both kernel space and user space.
• [PATCH v6 0/7] RISC-V IPI Improvements by Anup Patel
• Traditionally, RISC-V S-mode software like the Linux kernel calls to into M-mode runtime firmware like OpenSBI to issue IPIs (inter-processor interrupts)
• AIA (advanced interrupt architecture) provides the ability for S-mode to issue IPIs without any assistance from M-mode. This improves efficiency.

Work in progress

• [PATCH v4 0/4] Add Sstc extension support by Atish Patra
• Traditionally, an SBI call is necessary to generate timer interrupts as S-mode does not have access to the M-mode time compare registers.
• This results in significant latency for the kernel to generate timer interrupts at kernel
• For virtualized environments, it’s even worse as the KVM handles the SBI call and uses a software timer to emulate the time compare register.
• Sstc extension allows kernel to program a timer and receive interrupt without supervisor execution environment (M-mode/HS mode) intervention.

Linux distro: Fedora

• Aims to provide a complete Fedora experience on RISC-V
• Talk by Wei Fu, RISC-V Ambassador and Red Hat engineer [slides]
• Installation instructions for QEMU and RISC-V dev boards

Linux distro: Debian

​

• riscv64 is port of Debian to RISC-V
• “a port in Debian terminology means to provide the software normally available in the Debian archive (over 20,000 source packages) ready to install and run”
• 95% of packages are built for RISC-V
• The Debian port uses RV64GC as the hardware baseline and the lp64d ABI

Linux distro: Ubuntu

• riscv64 supported since the release of Ubuntu 20.04 LTS.
• Ubuntu 22.04 pre-installed SD-card image for SiFive boards and QEMU
• Starting with Ubuntu 22.04, a server install image is made available to install Ubuntu on NVMe drive of the SiFive Unmatched board

Linux distros

• OpenSuSE
• RISC-V support is still under development with Tumbleweed images for some boards
• Arch Linux
• Community effort has 95% of core packages building for RISC-V
• Gentoo
• riscv64 stages are available on the Gentoo download page

OpenEmbedded and Yocto

• meta-riscv: general hardware-specific BSP overlay for RISC-V devices
• works with different OpenEmbedded/Yocto distributions and layer stacks
• Supports both QEMU and RISC-V dev boards

​

BuildRoot

• RISC-V port is now supported in the upstream BuildRoot project
• “Embedded Linux from scratch in 45 minutes (on RISC-V)”
• Tutorial by Michael Opdenacker at FOSDEM 2021
• Use Buildroot to compile OpenSBI, U-Boot, Linux and BusyBox
• Boot the system in QEMU

SiFive Freedom Unleashed

• Launched in 2018 as first Linux-capable RISC-V dev board
• Exciting to see Fedora GNOME desktop on RISC-V
• $999 was too expensive for widespread adoption
• FU540 SoC chip was never sold separately

​

Microchip PolarFire SoC

​

• Same RISC-V cores as the SiFive FU540 SoC but adds a FPGA fabric
• FPGA with 25k to 460k logic elements (LEs)
• Supports DDR4 and PCIe Gen2
• Full commercial product family
• Parts will be available from distributors
• Microchip Icicle dev board ($499)

Kendryte K210

• 400 MHz dual core RV64GC
• 8MB SRAM but no DRAM
• Dev boards starting at $14
• Upstream support in Linux and u-boot
• Buildroot support by Damien Le Moal can create a busybox-based rootfs

SiFive Unmatched

​

• SiFive Freedom FU740 SoC
• 4x U74 RV64GC application cores
• Mini-ITX PC form factor
• 16GB DDR4, 4x USB 3.2, one x16 PCIe slot
• M.2 for NVMe SSD and WiFi/Bluetooth
• Shipped in 2021 for $665
• Discontinued in 2022 to focus on next-gen

T-Head XuanTie C910

​

• T-Head (“píng tóu gē”) is part of Alibaba
• High performance RV64GC with up 16 cores
• 12-stage pipeline, out-of-order, multi-issue architecture
• comparable to Arm Cortex-A73
• 2 core ‘ICE’ SoC made in low qty for evaluation
• T-Head ported Android 10 (AOSP) to RISC-V and showed a demo on the ICE in early 2021

Porting Android to RISC-V

• Mao Han presented why and how Alibaba T-Head ported Android to RISC-V (jump to 4:32)
• PDF of slides
• Update on Android 12 from April (jump to 13:23)
• PDF of slides
• How Alibaba is Porting RISC-V to the Android OS blog post with more technical details

T-Head RVB-ICE dev board

• ‘ICE’ SoC featuring dual C910 core at 1.2GHz
• 4GB LPDDR4, 16GB eMMC, 7 inch touchscreen, WiFi, Gigabit Ethernet
• Produced in limited quantity and available on AliExpress for $399

RISC-V Android SIG (Special Interest Group)

• GitHub organization riscv-android-src “contains all the modified AOSP(Android open source project) repositories with RISC-V support”
• Instruction to build and run Android 12 on RISC-V

T-Head XuanTie C906

• single-core RV64GC, only up to 1GHz, simpler 5-stage in-order pipeline

​

Allwinner D1 SoC

• mass production low cost SoC with a single T-Head C906 core at 1 GHz

Allwinner Nezha D1 dev board

• Official D1 board made by Allwinner Online, $115 bundle on AliExpress

​

RISC-V Developer Boards

• Initiative from RISC-V International to get Linux-capable boards into the hands of open source developers
• Launched in 2021 with the Allwinner D1 Nezha and SiFive Unmatched
• Fill out this form to apply
• Need to be RISC-V International member (or part of a member organization),�but remember that individuals can join RISC-V International free of cost
• Explain why you are interested in RISC-V and what you plan to do with dev board
• To improve your chances, don’t overestimate your hardware requirements like RAM

Allwinner D1 open source community

• linux-sunxi: strong open source community for Allwinner SoCs
• D1 wiki page and Allwinner Nezha board wiki page
• Telegram group: Mainline Linux for D1 (190 members)
• Samuel Holland has been working on getting mainline to run
• U-Boot SPL: https://github.com/smaeul/sun20i_d1_spl
• OpenSBI: https://github.com/smaeul/opensbi
• U-Boot: https://github.com/smaeul/u-boot/tree/d1-wip
• Linux: https://github.com/smaeul/linux/tree/riscv/d1-wip

​

Fedora on Allwinner D1

• Wei Fu has created a Fedora “Rawhide” image for the Allwinner D1 Nezha dev board that includes the XFCE desktop environment

Lichee RV-Nezha CM

• Allwinner D1 with DDR3 RAM of either 512MB ($22) or 1GB ($32)
• 1.14” SPI LCD, USB Type-C OTG, uSD
• Lichee RV Dock: HDMI out, USB-A host port, WiFi+BT, mic, audio out
• Lichee RV 86 Panel: 8 inch screen
• More details on sunxi wiki

Xassette Asterisk

​

• Allwinner F133 combines the Allwinner D1 with 64MB DDR2 in a single package
• Board design published as Open Source Hardware under the CERN OHL-w v2 licence
• Designed with KiCad (open source CAD sw)
• Not available commercially but possible to be hand assembled by hobbyists

MangoPi-Nezha MQ

• Allwinner F133 (also known as D1s)
• WiFi, USB Type C, mic, audio out
• DSI and RGB display connectors
• Open source hardware: KiCad files on GitHub
• $39 on Crowd Supply

Allwinner D1 mainline Linux support

• Allwinner reused peripheral IP from their existing ARM SoCs and the Linux kernel already has drivers for most of them
• T-Head cores like C910 and C906 do have some non-standard functionality for performance but it’s not needed to boot
• Instructions to accelerate I-cache and TLB synchronization
• T-Head MMU has a non-standard ‘enhanced’ mode that is needed to support DMA with devices on non-coherent interconnects
• Linux needs to enable that ‘enhanced’ MMU mode to function properly

How to handle non-coherent interconnects?

• The original RISC-V Privileged spec stated that “the use of hardware-incoherent regions is discouraged due to software complexity, performance, and energy impacts”
• However, non-coherent interconnects are important for low cost SoCs
• T-Head designed the C9xx cores in 2019, and there were no RISC-V extensions that provided ability to handle non-coherent devices

T-Head PTE format

• T-Head used bits in the PTE (Page Table Entry) to specify memory type

​

​

​

​

​

​

� … but those bits were already marked reserved in RISC-V priv spec

| 63 | 62 | 61 | 60 | 59-8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0

SO C B SH RSW D A G U X W R V

^ ^ ^ ^

​

BIT(63): SO - Strong Order

BIT(62): C - Cacheable

BIT(61): B - Bufferable

BIT(60): SH - Shareable��0000 - NC Weakly-ordered, Non-cacheable, Non-bufferable, Non-shareable

0111 - PMA Weakly-ordered, Cacheable, Bufferable, Shareable

1000 - IO Strongly-ordered, Non-cacheable, Non-bufferable, Non-shareable

Page-Based Memory Types extension

• Svpbmt proposed by Virtual Memory TG and ratified at the end of 2021
• “S” = supervisor-mode (privileged architecture), “v” for virtual memory

Here is the svpbmt PTE format:

| 63 | 62-61 | 60-8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0

N MT RSW D A G U X W R V

^

RISC-V

Encoding &

MemType RISC-V Description

---------- ------------------------------------------------

00 - PMA Normal Cacheable, No change to implied PMA memory type

01 - NC Non-cacheable, idempotent, weakly-ordered Main Memory

10 - IO Non-cacheable, non-idempotent, strongly-ordered I/O memory

11 - Rsvd Reserved for future standard use

Svpbmt support in Linux

• [PATCH v10 00/12] riscv: support for Svpbmt and D1 memory types
• by Heiko Stuebner based on initial work by Alibaba kernel engineer Guo Ren
• Implements the official RISC-V Svpbmt extension
• The standard Svpbmt and custom T-Head PTE formats both use the highest bits to determine memory type but the encoding and semantics are different
• The custom T-Head PTE format is supported through boot-time code patching using the Linux Alternatives Framework
• Expected to land in Linux 5.19 as it is in Palmer’s pull request

Cache Management Operations

• Instructions to manage cache are important for SoCs which that lack cache coherent interconnects
• Zicbom extension (“Z” prefix means Unpriv spec) was ratified at the end of 2021, and it defines cache-block management (CBO) instructions:
• CBO.CLEAN guarantee store by hart can be read from mem by non-coherent device
• CBO.INVAL guarantee hart can load data written to memory by non-coherent device
• CBO.FLUSH guarantees both
• Support for Non-Coherent I/O Devices in RISC-V from RV Summit [slides]

CMO support in Linux

• riscv: implement Zicbom-based CMO instructions + the t-head variant by Heiko Stuebner
• Implements Zicbom-extension to handle cache clean, invalidate, flush

* cbo.clean rs1

* | 31 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 |

* 0...01 rs1 010 00000 0001111

*

* cbo.flush rs1

* | 31 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 |

* 0...10 rs1 010 00000 0001111

*

* cbo.inval rs1

* | 31 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 |

* 0...00 rs1 010 00000 0001111

​

#define CBO_INVAL_A0 ".long 0x15200F"

#define CBO_CLEAN_A0 ".long 0x25200F"

#define CBO_FLUSH_A0 ".long 0x05200F"

CMO support in Linux

• T-Head implemented custom cache instructions before Zicbom existed

* dcache.ipa rs1 (invalidate, physical address)

* | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 |

* 0000001 01010 rs1 000 00000 0001011

* dache.iva rs1 (invalida, virtual address)

* 0000001 00110 rs1 000 00000 0001011

*

* dcache.cpa rs1 (clean, physical address)

* | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 |

* 0000001 01001 rs1 000 00000 0001011

* dcache.cva rs1 (clean, virtual address)

* 0000001 00100 rs1 000 00000 0001011

*

* dcache.cipa rs1 (clean then invalidate, physical address)

* | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 |

* 0000001 01011 rs1 000 00000 0001011

* dcache.civa rs1 (... virtual address)

* 0000001 00111 rs1 000 00000 0001011

​

#define THEAD_INVAL_A0 ".long 0x0265000b"

#define THEAD_CLEAN_A0 ".long 0x0245000b"

#define THEAD_FLUSH_A0 ".long 0x0275000b"

CMO support in Linux

• While the Zicbom and T-Head instructions are different, they provide the same functionality, so the T-Head variant handled with the existing alternatives mechanism
• Allwinner D1 needs these cache instructions for peripherals like�MMC (SD card), USB, and Ethernet to work
• Unfortunately, these patches use pre-coded CMO instructions and Palmer would prefer that Linux support wait until the instructions are added to gcc and binutils

Allwinner D1 IOMMU support

• [PATCH 0/5] iommu/sun50i: Allwinner D1 support by Samuel Holland
• IOMMU is not needed for boot
• Optional feature for the display engine and video decoder
• Without IOMMU support, video/frame buffers have to be contiguous in physical memory, and that requires the user to know how much memory to reserve for them at boot

T-Head released RISC-V cores as open source!

• OpenE902, OpenE906, OpenC906, and OpenC910 cores on GitHub under permissive Apache 2.0 licence

XiangShan (香山)

• open source high-performance RISC-V processor project from the Chinese Academy of Science
• RISC-V Summit 2021 talk by Professor Yungang Bao (slides)
• “Contribute to XiangShan and realize your ideas on real chips! The open-source XiangShan will be taped-out every ~6 months”
• Nanhu is the 2nd generation microarchitecture
• Target: 2GHz@14nm, SPEC CPU2006 20 marks; 7.81 CoreMark/MHz

RISC-V Lab

• PLCT Lab at Chinese Academy of Sciences
• Status report from lab director Wei Wu
• Continuous Integration (CI) for open source software projects on RISC-V hardware
• 70+ SiFive Unmatched boards
• 100+ Allwinner Nezha D1 boards
• Open source devs can request access

No hardware? Try Renode!

​

• Emulate physical hardware systems including CPU, peripherals, sensors, and networking
• Run the same binaries as the real hardware �for over 30 supported dev boards
• Microchip PolarFire SoC Icicle Kit
• Kendryte K210
• SiFive HiFive Unleashed

How to get involved with RISC-V International?

• Become a member
• Individuals and non-profits can join free of cost
• RISC-V Technical wiki landing page is the single best place to visit
• Technical Working Groups
• Recently Ratified Extensions

How to get involved with RISC-V International?

• RISC-V mailing list server
• Only RISC-V members can participate
• Archives of all the lists are public
• Technical Meetings Calendar
• Many groups have bi-weekly or monthly meetings
• ICS File / Google Calendar

RISC-V Summit 2021

• YouTube playlist with 93 talks

RISC-V Spring Week 2022

• Videos on the RISC-V YouTube channel
• State of the Union & the Road Ahead
• Maturing the RISC-V Ecosystem
• Evolving the Role of Software in the RISC-V Ecosystem
• RISC-V IOMMU Architecture Overview

Embedded Linux Conf 2021

• Initializing RISC-V A Guided Tour for ARM Developers
• Ahmad Fatoum & Rouven Czerwinski, Pengutronix
• Building a Low-key XIP-enabled RISC-V Linux System
• Vitaly Vul, Konsulko AB
• Perf on RISC-V: The Past, the Present and the Future
• Atish Patra & Anup Patel, Western Digital
• "A New user(space): Adding RISC-V Support to Zephyr RTOS" [slides]
• Kevin Hilman & Alexandre Mergnat, BayLibre

RISC-V meetups around the world

​

Find more at: community.riscv.org

RISC-V Open Hours

• Bi-weekly virtual meetup for the community to interact in real-time
• Primary focus on RISC-V support in open source software and RISC-V dev boards
• Call for participation is open! No prepared talk or slides required!
• Schedule
• Wednesday, June 8, 7:00 PM (US PDT) which is Thursday morning in Asia
• Wednesday, June 29, 9:00 AM (US PDT) which is early evening in Europe

Linux on RISC-V

Drew Fustini <dfustini@baylibre.com>

Slides: tinyurl.com/riscv-kr-22

June 1-3, 2022

BONUS:�What about RISC-V on FPGAs?

Introduction

• “RISC-V and FPGAs: Open Source Hardware Hacking” keynote at Hackaday Supercon 2019 by Dr. Megan Wachs

Open source FPGA toolchains

• Project IceStorm for Lattice iCE40 FPGA
• “A Free and Open Source Verilog-to-Bitstream Flow for iCE40 FPGAs” by Claire Wolf
• Project Trellis for the more capable Lattice ECP5 FPGA
• “Project Trellis and nextpnr FOSS FPGA flow for the Lattice ECP5” by Myrtle Shah
• Project X-Ray and SymbiFlow for much more capable Xilinix Series 7
• “Xilinx Series 7 FPGAs Now Have a Fully Open Source Toolchain!” by Tim Ansell
• “Open Source Verilog-to-Bitstream FPGA synthesis flow, currently targeting Xilinx 7-Series, Lattice iCE40 and Lattice ECP5 FPGAs. Think of it as the GCC of FPGAs”

Hackaday Supercon badge

• RISC-V “soft” core on ECP5 FPGA
• Gigantic FPGA In Game Boy Form Factor

​

Why design an SoC in Python?

• Python has advantages over traditional HDL like VHDL and Verilog
• Many people are familiar with Python than HDL (hardware description languages)
• There are currently more software developers than hardware designers
• Migen is a Python framework that can automate chip design
• Leverages the object-oriented, modular nature of Python
• Produces Verilog code so it can be used with existing chip design workflows
• “Using Python for creating hardware to record FOSS conferences!”

​

source: http://goo.gl/mZJvFQ

​

​

LiteX

• Based on Migen, builds full SoC that can be loaded into an FPGA

LiteX

• “LiteX vs. Vivado: First Impressions”
• Collection of open cores for DRAM, Ethernet, PCIe, SATA and more...

Linux on LiteX-VexRiscv

​

• VexRiscv: 32-bit Linux-capable RISC-V core
• Designed to be FPGA friendly
• Written in Spinal HDL (based on Scala)
• Builds an SoC using VexRiscv core and LiteX modules
• Such as LiteDRAM, LiteEth, LiteSDCard, LitePCIe
• “This project demonstrates how high level HDLs (Spinal HDL, Migen) enable new possibilities and complement each other. Results shown here are the results of a productive collaboration between open-source communities”
• Supports large number of FPGA dev boards including Digilent Arty A7

https://twitter.com/GregDavill/status/1231082623633543168

Open Source ECP5 FPGA boards

• Radiona.org ULX3S
• 32MB SDRAM; ESP32 on-board for WiFi and Bluetooth; $115 on CrowdSupply or Mouser

Open Source ECP5 FPGA boards

• OrangeCrab by Greg Davill
• 128MB DDR RAM; Adafruit Feather form factor; available for $129

Want to learn FPGAs? Try Fomu!

• Online workshop from Tim Ansell and Sean Cross
• $50 on CrowdSupply
• Fits inside USB port!
• Learn how to use:
• MicroPython
• Verilog
• LiteX

​