Published using Google Docs
Verified Boot for Embedded Systems (VBE) Proposal (public)
Updated automatically every 5 minutes

Verified Boot for Embedded Systems

Authors: sjg
Self Link: go/cros-vbe

Visibility: Public
Originally Proposed: 2021-03-26 / Last Updated: 2023-09-20

Objective

Define a full-featured, verified-boot flow for Embedded Systems that uses and encourages Open Source firmware.

Requirements

Out of scope

Background

For some years ARM-based embedded systems have happily used U-Boot to initialise hardware, load a FIT (Flat Image Tree) containing Linux with a devicetree and boot it. Many of these are custom-built and fully integrated (e.g. using Buildroot or Yocto) with their own update mechanisms.

But for several reasons this era is drawing to an end:

ARM's Embedded Base Boot Requirements (EBBR) aims to address these issues using the UEFI specification, with various U-Boot extensions to handle the various verification and update requirements. This document suggests an alternative approach. A discussion of the benefits is at the end.

Note: This document is based almost entirely on existing work, e.g. U-Boot, FIT, Chromium OS verified boot.

Sel also

Design ideas

The diagram below shows the VBE flow, starting with the boot ROM and ending with Fwupd (link).

[1]

Briefly, the non-obvious components are as follows:

Firmware structure

The overall firmware image is built by Binman, which includes a directory of available pieces for each flow, permits updates, supports compression and hashing, allows insertion of signatures, etc. This is used by Fwupd and can be used by developers to modify and replace firmware components. Firmware updates may replace parts of the firmware. The directory can be placed in the devicetree for use by fwupd.

Updatable parts of the firmware are signed with signatures embedded in the firmware image. These are checked by VPL when deciding which flow to take. If no valid SPL is available, recovery mode is selected.

Firmware may be in its own storage , such as SPI flash, with a read-only part containing the recovery firmware to prevent bricking.  If stored on the disk, unbricking may require external action.

FIT

Flat Image Tree (spec) is an ideal format for storing images. It supports many useful features: multiple images along with 'configurations' that collect them together, hashing, compression, signing (including adding new signatures to an existing FIT) and is easily extensible. It is shipped as a single file, simplifying management of the contents. Adding FIT support to an existing codebase is fairly simple: perhaps 800 LOC when libfdt is used.

The extensibility of FIT is key to VBE, since we cannot imagine all possible future requirements for firmware and changing to a different format later would be very disruptive.

Firmware updates

Every platform potentially has its own method of updating the firmware. The method and the required parameters are encoded in the devicetree in /chosen/fwupd, for example:

        chosen {
                fwupd {
                        firmware {
                                compatible = "fwupd,a-b-rec";
                                storage = <&spiflash>;
                        };
                };
        };

The compatible string directly maps to a driver in fwupd which knows how to deal with that method. Vendor-specific methods or options can be implemented as needed, using a different compatible string. In an A/B update, only one side is updated, with the other being written after a successful boot. Signalling for this is handled via NV data.

If a firmware update fails, or does the wrong thing, or writes unsigned firmware, then the device may end up in recovery mode on the next boot. Nothing fwupd can do can compromise the system, however, since it cannot itself sign the firmware, only install pre-signed images[3].

See VBE Firmware Update for more information.

OS update

Updates to the operation system are installed by fwupd in the form of new FITs. The place to install these is typically controlled by the 'bootflow' file, although the devicetree may also provide this.

Bootflow

The available boot flows are described in a bootflow file, accessible to U-Boot and fwupd. A boot flow record includes the FIT configuration to load (which in turn indicates kernel, devicetree, initrd, FPGA), base kernel command-line arguments and any other pieces defined by VBE. Each record in the file can be individually signed (according to firmware policy) and only records that can be verified will be accessible at boot time.

Firmware selection (in VPL) may adjust the use of bootflow in several ways, for example:

Typically the bootflow file is stored in a particular directory / file or block. In this example, the partition UUID is fixed in the factory:

        chosen {
                fwupd {
                        bootflow {
                                compatible = "fwupd,a-b-rec";
                                storage = <&ufs>;
                                filesystem = "ext4";
                                partition-uuid = "cd8a3a34-e2d0-4d0c-97d0-e2b1f14220a8";
                                directory = "/boot";
                        };
                };
        };

The format of the bootflow file is described in VBE Bootflow'. The menu is described in VBE UI.

Adding keys

The user can add new keys to the system, if permitted, using fwupd or an OS tool that uses it. This allows support for self-signed FITs. New keys can be placed in firmware like any other payload and are signature-checked by VPL on the next boot, before being installed for future use, perhaps after user-presence check. A U-Boot UI could provide a way to install keys also, although the security aspects of this need some thought.

Fixups

Since U-Boot loads the FIT it can perform any devicetree fix-ups that are needed for the memory layout, console selection, etc. This is a fairly simple process and already implemented. Should the fix-up algorithm itself need to change, it is possible to update U-Boot.

For the kernel command-line, U-Boot provides a way to replace ${var} expressions in the template provided by the bootflow record.

OS Requests

With VBE, Operating Systems publish their booting requirements as part of the FIT, where they can be easily read and acted on. Requests include random numbers and seeds and address-space layout randomisation (ASLR). U-Boot processes the requests and adds the resulting information to the /chosen node for use by the OS, using the 'fixup' mechanism. In the case of Linux, this renders the EFI stub (which on ARM reads information from tables and writes it into the DT) unnecessary.

Communication through phases

Each phase of boot needs to pass information to the next. VBE uses bloblist, a simple data structure which can be implemented with little code and does not require UUIDs. WIthin the bloblist, each blob can have a particular purpose, including one for the vboot state. Note that bloblist does not replace devicetree which is much more suitable for more comprehensive and extensible transfer of information to later boot stages or Linux.

Communication to firmware from Linux user space is through the NVdata. If larger chunks of information are needed (e.g. a new key), they are written to the read/write firmware space. Since the firmware is not required to be running once linux is fully booted, such information is considered on the next boot.

Migration path

Many of the pieces needed by VBE are already available. Some more design work is obviously needed, to flesh out the ideas presented here. At minimum, the following need to be implemented:

Avoiding U-Boot and Linux

While U-Boot is used here and is intended to be the reference implementation, the interfaces do not mandate it, and other bootloaders could provide similar features. The TPL/VPL/SPL/U-Boot pieces can be replaced entirely with a different bootloader which provides the same functionality.

Similarly, while Linux is envisaged, VBE is capable of booting other operating systems, providing its full boot support (including verified boot, for example) regardless of which is chosen, since it does not require the OS to communicate with firmware in any way, although some features are lost. Since fwupd supports Windows, it would in fact be possible to perform firmware updates from Windows as well as Linux, if desired.

More complex firmware

U-Boot provides support for loading arbitrary binaries from a FIT ("loadables"), while still providing verified-boot support ("signed configurations"). As firmware becomes more complex, it should be possible to add these images and stitch them into the architecture without undue effort.

Devicetree location

At present it is common to package the devicetree with Linux for ease of update. VBE should enable easier and more pervasive firmware update so that it will become common for Linux to rely on the devicetree provided by the firmware, with an override if necessary, for particular situations.

SoC Boot ROMs

Many SoCs support signing of images using a key within the SoC, along and verification during boot. This feature can be used to load the initial TPL / VPL binaries along with any settings needed for the device. It provides a guarantee that unauthorised code cannot be run at boot. Once the system boots to VPL, the VBE process takes over to provide a standard boot flow across all SoCs, regardless of their particular implementation of SoC-signed images.

If the SoC does not support signing, then read-only flash can be used instead, programmed once at the factory (and perhaps later during refurbishment/ RMA) and protected against in-the-field updates.

ACPI

It is not expected that ARM systems would need to use ACPI with VBE, although it is possible. For Intel systems, U-Boot supports ACPI, including comprehensive runtime code-generation as needed. ACPI updates require a firmware update since these tables are not easily packaged with the OS.

Testing

VBE lends itself to robust testing due to its simplicity. Much of the flow exists within U-Boot and can be tested in sandbox, U-Boot's native test environment. This avoids any need for real hardware or qemu and can run in the CI environment. Test cases with different bootflow files can be implemented similarly to U-Boot's existing vboot tests, by updating the bootflow file, starting TPM/NVdata conditions and running the boot in sandbox. For firmware update, unit tests in fwupd can check that the correct data ends up in the firmware space. The loose contract between firmware and fwupd lends itself to separate testing, with perhaps just some hardware- or qemu-based 'happy-path' tests to cover the overall flow.

Development

VBE should be a very friendly architecture to develop within. U-Boot is already familiar to most embedded engineers and VBE makes use of existing U-Boot technologies which have been around for years. It uses the Linux coding style, Kconfig and has a powerful driver model / devicetree architecture. Commands are available to view, debug and adjust settings during development. While fwupd will be new to some, it is well-established in the area of peripheral-firmware update and is already in several distributions. It has wide OEM adoption.

Example boot and update

This describes the steps that a system goes through through to boot and update itself:

  1. SoC boot-ROM loads TPL into SRAM, either from write-protected flash or using a signature check
  2. TPL performs basic init such as clocks, cache, TPM and the like, then loads VPL into SRAM. It sets up an empty bloblist region for storing console output, vboot data, etc.
  3. VPL checks the Sec. data and NV data, and decides which SPL to boot (e.g. A, B, recovery). Let's say it selects A. It adds vboot information to the bloblist and then loads SPL-A (say version 6) into SRAM.
  4. SPL-A sets up SDRAM and boots into U-Boot. It could also load secure-firmware pieces if needed.
  5. U-Boot checks the bootflow and the vboot information in the bloblist, then decides which configuration to boot. Let's say it chooses a configuration with kernel v5.10. It verifies the configuration, reads and verifies the images, sets up the command line, performs fix-ups on the devicetree and command line, then boots Linux
  6. Linux starts up from the assigned root device using dm-verity and operates normally
  7. Sometime later fwupd checks the firmware-update configuration, runs the appropriate driver which sees an update to firmware v7 available on the Internet. It downloads it, performs some checks and stores into the firmware area, partition B (since A is in use). The update is not yet applied but the NVdata is updated to indicate that firmware should try booting partition B.
  8. A little later fwupd does the same for the kernel-update configuration, downloading and storing a new FIT with kernel v5.12 in it, along with a devicetree overlay. It adds a new record to the bootflow and marks a previous one for archiving if the new one works out OK.
  9. Sometime later the user reboots the machine. This time, VPL sees that it should try firmware B. It does so and the signature checks out, so it loads SPL B (i.e. version 7), which loads U-Boot B and the 'B' version of any secure firmware, etc.
  10. U-Boot B loads the bootflow records and sees the new one to try. It verifies the new configuration, then boots it.
  11. Linux 5.12 starts up as normal
  12. Sometime later fwupd checks the firmware-update configuration, runs the appropriate driver which sees that the B firmware booted OK, so it writes it to the 'A' partition as well, then marks the firmware update as completed. VPL will later update the rollback counters in Sec. data on a subsequent reboot.
  13. A little later fwupd does the same for the kernel-update configuration, moving the old v5.10 configuration to the archive and making v5.12 the default configuration.

Alternatives considered

We could continue with the existing EBBR with a mandated UEFI approach. This has the benefit of seemingly unstoppable momentum. But:

Benefits

Here with a brief summary of the benefits of VBE:

More reading

Page  /

[1] Diagram here

[2] In fact recent changes in U-Boot allow skipping grub by setting up Boot#### and BootOrder variables (see docs)

[3] As an aside, it is even possible to update UEFI firmware using fwupd.

[4] Although it does work on Works on a 32-bit Arm v7: STM32MP1

[5] This would seem to be inevitable, if firmware is on secure storage. But with VBE it is possible to update the firmware from user space, perhaps under control of the secure firmware through a kernel API.

[6] Also see 'FFA Update' at https://developer.arm.com/documentation/den0077/latest/ and https://developer.arm.com/-/media/Files/pdf/FWU-PSA-A_DEN0118_1.0ALP3.pdf