Welcome to the Yocto Project Summit 2023.11

The beginner class will start at 12:30 UTC

​

The slides for this presentation → http://bit.ly/3sS8Jf6

​

​

The URL for this presentation

Intro to Yocto Project

​

https://bit.ly/3sS8Jf6

bit.ly/3sS8Jf6

Creating a Custom Embedded Linux Distribution for Any Embedded Device Using the Yocto Project

Behan Webster

Tom King

The Linux Foundation

November 29, 2023

(CC BY-SA 4.0)

3

Yocto Project Summit 2023.11

Intro to Yocto Project

bit.ly/3sS8Jf6

Yocto Project Overview

• Collection of tools and methods enabling
• Rapid evaluation of embedded Linux on many popular off-the-shelf boards
• Easy customization of distribution characteristics
• Supports x86, ARM, MIPS, Power, RISC-V
• Based on technology from the OpenEmbedded Project
• Layer architecture allows for�easy re-use of code

4

bit.ly/3sS8Jf6

What is the Yocto Project?

• Umbrella organization under Linux Foundation
• Backed by many companies interested in making Embedded Linux easier for the industry
• Co-maintains OpenEmbedded Core and other tools (including opkg)

5

bit.ly/3sS8Jf6

Yocto Project Governance

• Organized under the Linux Foundation
• Split governance model
• Technical Leadership Team
• Advisory Board made up of participating organizations

6

bit.ly/3sS8Jf6

Yocto Project Member Organizations

PLATINUM MEMBERS

GOLD MEMBERS

SILVER MEMBERS

bit.ly/3sS8Jf6

Yocto Project Overview

• YP builds packages - then uses these packages to build bootable images
• Supports use of popular package formats including:
• rpm, deb, ipk
• Releases on a 6-month cadence
• Latest (stable) kernel, toolchain and packages, documentation
• eSDK, devtool, toaster

8

bit.ly/3sS8Jf6

Yocto Project Release Versions

9

bit.ly/3sS8Jf6

Yocto is based on OpenEmbedded-core

10

Metadata describing approximately 800 "core" recipes used for building boot images. Includes support for graphics, networking, kernel recipes, tools, much more.

bit.ly/3sS8Jf6

Intro to OpenEmbedded

• The OpenEmbedded Project co-maintains OE-core build system:
• bitbake build tool and scripts
• Metadata and configuration
• Provides a central point for new metadata
• (see the OE Layer index)

11

bit.ly/3sS8Jf6

What is Bitbake?

• Bitbake
• Powerful and flexible build engine (Python)
• Reads metadata
• Determines dependencies
• Schedules tasks

12

Metadata – a structured collection of "recipes" which tell BitBake what to build, organized in layers

bit.ly/3sS8Jf6

• Poky is a reference distribution
• Poky has its own git repo
• git clone git://git.yoctoproject.org/poky
• Primary Poky layers
• oe-core (poky/meta)
• meta-poky (poky/meta-poky)
• meta-yocto-bsp
• Poky is the starting point for building things with the Yocto Project

​

meta (oe-core)

meta-poky

meta-yocto-bsp

Other layers

OK, so what is Poky?

13

bit.ly/3sS8Jf6

Poky in Detail

• Contains core components
• Bitbake tool: A python-based build engine
• Build scripts (infrastructure)
• Foundation package recipes (oe-core)
• meta-poky (Contains distribution policy)
• Reference BSPs
• Yocto Project documentation

​

​

​

bit.ly/3sS8Jf6

Putting It All Together

• Yocto Project is a large collaboration project
• OpenEmbedded is providing most metadata
• Bitbake is the build tool
• Poky is the Yocto Project reference distribution
• Poky contains a version of bitbake and oe-core from which you can start your project

15

bit.ly/3sS8Jf6

Build System Workflow

16

http://bit.ly/YPS202105Intro

bit.ly/3sS8Jf6

BITBAKE

This section will introduce the concept of the bitbake build tool and how it can be used to build recipes

17

bit.ly/3sS8Jf6

Metadata and bitbake

• Most common form of metadata: The Recipe
• A Recipe provides a “list of ingredients” and “cooking instructions”
• Defines settings and a set of tasks used by bitbake to build binary packages

busybox

glibc

sysvinit

coreutils

libgtk

Metadata

BitBake

bit.ly/3sS8Jf6

• Metadata exists in four general categories:
• Recipes (*.bb)
• Usually describe build instructions for a single package
• PackageGroups (special *.bb)
• Often used to group packages together for a FS image
• Classes (*.bbclass)
• Inheritance mechanism for common functionality
• Configuration (*.conf)
• Drives the overall behavior of the build process

What is Metadata?

bit.ly/3sS8Jf6

• Append files (*.bbappend)
• Define additional metadata for a similarly named .bb file
• Can add or override previously set values
• Include files (*.inc)
• Files which are used with the include directive
• Also can be included with require (mandatory include)
• Include files are typical found via the BBPATH variable

Other Metadata

bit.ly/3sS8Jf6

OE-CORE Breakdown

21

*.bb: 839

packagegroup*: 25

*.bbclass: 210

*.conf: 72

*.inc: 300

core-image-*: 20

bit.ly/3sS8Jf6

Introduction to Bitbake

• Bitbake is a task executor and scheduler
• By default the build task for the specified recipe is executed

$ bitbake myrecipe

• You can indicate which task you want run

$ bitbake -c clean myrecipe�$ bitbake -c cleanall myrecipe

• You can get a list of tasks with

$ bitbake -c listtasks myrecipe

bit.ly/3sS8Jf6

Building Recipes

• By default the highest version of a recipe is built (can be overridden with DEFAULT_PREFERENCE or PREFERRED_VERSION metadata)

$ bitbake myrecipe

• You can specify the version of the package you want built (version of upstream source)

$ bitbake myrecipe-1.0

• You can also build a particular revision of the package metadata

$ bitbake myrecipe-1.0-r0

• Or you can provide a recipe file to build

$ bitbake -b mydir/myrecipe.bb

bit.ly/3sS8Jf6

Running bitbake for the First Time

• When you do a really big build, running with --continue (-k) means bitbake will proceed as far as possible after finding an error

$ bitbake -k core-image-minimal

• When running a long build (e.g. overnight) you want as much of the build done as possible before debugging issues
• Running bitbake normally will stop on the first error found

$ bitbake core-image-minimal

• We'll look at debugging recipe issue later...

bit.ly/3sS8Jf6

Bitbake is a Task Scheduler

• Bitbake builds recipes by scheduling build tasks in parallel

$ bitbake recipe

• This looks for recipe.bb in BBFILES
• Each recipe defines build tasks, each which can depend on other tasks
• Recipes can also depend on other recipes, meaning more than one recipe may be built
• Tasks from more than one recipe are often executed in parallel at once on multi-cpu build machines

25

bit.ly/3sS8Jf6

Recipe Basics – Default Tasks*

Create binary package(s)

do_fetch

do_unpack

do_patch

do_configure

do_install

do_compile

*Simplified for illustration

Note: to see the list of all possible tasks for a recipe, do this:

$ bitbake -c listtasks <recipe_name>

do_populate_sysroot

do_package_*

Locate and download source code

​

Unpack source into working directory

Apply any patches

Perform any necessary pre-build configuration

​

Compile the source code

Installation of resulting build artifacts in WORKDIR

​

Copy artifacts to sysroot

bit.ly/3sS8Jf6

Simple recipe task list*

27

*Simplified for illustration

$ bitbake hello

NOTE: Running task 337 of 379 (ID: 4, hello_1.0.0.bb, do_fetch)

NOTE: Running task 368 of 379 (ID: 0, hello_1.0.0.bb, do_unpack)

NOTE: Running task 369 of 379 (ID: 1, hello_1.0.0.bb, do_patch)

NOTE: Running task 370 of 379 (ID: 5, hello_1.0.0.bb, do_configure)

NOTE: Running task 371 of 379 (ID: 7, hello_1.0.0.bb, do_populate_lic)

NOTE: Running task 372 of 379 (ID: 6, hello_1.0.0.bb, do_compile)

NOTE: Running task 373 of 379 (ID: 2, hello_1.0.0.bb, do_install)

NOTE: Running task 374 of 379 (ID: 11, hello_1.0.0.bb, do_package)

NOTE: Running task 375 of 379 (ID: 3, hello_1.0.0.bb, do_populate_sysroot)

NOTE: Running task 376 of 379 (ID: 8, hello_1.0.0.bb, do_packagedata)

NOTE: Running task 377 of 379 (ID: 12, hello_1.0.0.bb, do_package_write_ipk)

NOTE: Running task 378 of 379 (ID: 9, hello_1.0.0.bb, do_package_qa)

*Output has been formatted to fit this slide.

bit.ly/3sS8Jf6

SSTATE CACHE

• Several bitbake tasks can use past versions of build artefacts if there have been no changes since the last time you built them

28

bit.ly/3sS8Jf6

Simple recipe build from sstate cache*

29

*Simplified for illustration

$ bitbake -c clean hello

$ bitbake hello

NOTE: Running setscene task 69 of 74 (hello_1.0.0.bb, do_populate_sysroot_setscene)

NOTE: Running setscene task 70 of 74 (hello_1.0.0.bb, do_populate_lic_setscene)

NOTE: Running setscene task 71 of 74 (hello_1.0.0.bb, do_package_qa_setscene)

NOTE: Running setscene task 72 of 74 (hello_1.0.0.bb, do_package_write_ipk_setscene)

NOTE: Running setscene task 73 of 74 (hello_1.0.0.bb, do_packagedata_setscene)

*Output has been formatted to fit this slide.

bit.ly/3sS8Jf6

RECIPES

This section will introduce the concept of metadata and recipes and how they can be used to automate the building of packages

30

bit.ly/3sS8Jf6

What is a Recipe?

• A recipe is a set of instructions for building packages, including:
• Where to obtain the upstream sources and which patches to apply (this is called “fetching”)
• SRC_URI
• Dependencies (on libraries or other recipes)
• DEPENDS, RDEPENDS
• Configuration/compilation options
• EXTRA_OECONF, EXTRA_OEMAKE
• Define which files go into what output packages
• FILES_*

31

bit.ly/3sS8Jf6

Example Recipe – ethtool_3.15.bb

32

bit.ly/3sS8Jf6

What can a Recipe Do?

• Build one or more packages from source code
• Host tools, compiler, utilities
• Bootloader, Kernel, etc
• Libraries, interpreters, etc
• Userspace applications
• Package Groups
• Full System Images

33

bit.ly/3sS8Jf6

Recipe Operators

A = “foo” (late assignment)

B ?= “0t” (default value)

C ??= “abc” (late default)

D := “xyz” (Immediate assignment)

​

A .= “bar” “foobar” (append)

B =. “W0” “W00t” (prepend)

C += “def” “abc def” (append)

D =+ “uvw” “uvw xyz” (prepend)

34

bit.ly/3sS8Jf6

More Recipe Operators

A = “foo”

A:append = “bar” “foobar”

B = “0t”

B:prepend = “W0” “W00t”

​

A:remove = “oob” “far”

​

B:remove = “00” “Wt”

​

35

bit.ly/3sS8Jf6

Recipe Override Operators

OVERRIDES = “os:arch:machine”

A = “abc”

​

A:os = “ABC” (Override)

​

A:append:arch = “def” (Conditional append)

​

A:prepend:os = “XYZ” (Conditional prepend)

36

bit.ly/3sS8Jf6

Unsetting Variables

​

unset DATE (unset Variable)

​

unset do_fetch[noexec] (unset task attribute)

37

bit.ly/3sS8Jf6

Bitbake Variables/Metadata

• These are set automatically by bitbake
• TOPDIR – The build directory
• LAYERDIR – Current layer directory
• FILE – Path and filename of file being processed
• Policy variables control the build
• BUILD_ARCH – Host machine architecture
• TARGET_ARCH – Target architecture
• And many others...

38

bit.ly/3sS8Jf6

Build Time Metadata

• PN – Package name (“myrecipe”)
• PV – Package version (1.0)
• PR – Package Release (r0)
• P = “${PN}-${PV}”
• PF = “${PN}-${PV}-${PR}”
• FILE_DIRNAME – Directory for FILE
• FILESPATH = "${FILE_DIRNAME}/${PF}:\
• ${FILE_DIRNAME}/${P}:\
• ${FILE_DIRNAME}/${PN}:\
• ${FILE_DIRNAME}/files:${FILE_DIRNAME}

39

bit.ly/3sS8Jf6

Build Time Metadata

• TOPDIR – The build directory
• TMPDIR = “${TOPDIR}/tmp”
• WORKDIR = ${TMPDIR}/work/${PF}”
• S = “${WORKDIR}/${P}” (Source dir)
• B = “${S}” (Build dir)
• D = “${WORKDIR}/${image}” (Destination dir)
• DEPLOY_DIR = “${TMPDIR}/deploy”
• DEPLOY_DIR_IMAGE = “${DEPLOY_DIR}/images”

40

bit.ly/3sS8Jf6

Dependency Metadata

• Build time package variables
• DEPENDS – Build time package dependencies
• PROVIDES = “${P} ${PF} ${PN}”
• Runtime package variables
• RDEPENDS – Runtime package dependencies
• RRECOMMENDS – Runtime recommended packages
• RSUGGESTS – Runtime suggested packages
• RPROVIDES – Runtime provides
• RCONFLICTS – Runtime package conflicts
• RREPLACES – Runtime package replaces

41

bit.ly/3sS8Jf6

Common Metadata

• Variables you commonly set
• SUMMARY – Short description of package/recipe
• HOMEPAGE – Upstream web page
• LICENSE – Licenses of included source code
• LIC_FILES_CHKSUM – Checksums of license files at time of packaging (checked for change by build)
• SRC_URI – URI of source code, patches and extra files to be used to build packages. Uses different fetchers based on the URI.
• FILES – Files to be included in binary packages

42

bit.ly/3sS8Jf6

Examining Recipes: bc

• Look at 'bc' recipe:
• Found in

poky/meta/recipes-extended/bc/bc_1.06.bb

• Uses LIC_FILES_CHKSUM and SRC_URI checksums
• Note the DEPENDS build dependency declaration indicating that this package depends on flex to build

43

bit.ly/3sS8Jf6

Examining Recipes: bc_1.6.bb

SUMMARY = "Arbitrary precision calculator language"

HOMEPAGE = "http://www.gnu.org/software/bc/bc.html"

​

LICENSE = "GPLv2+ & LGPLv2.1"

LIC_FILES_CHKSUM = "file://COPYING;md5=94d55d512a9ba36caa9b7df079bae19f \

file://COPYING.LIB;md5=d8045f3b8f929c1cb29a1e3fd737b499 \

file://bc/bcdefs.h;endline=31;md5=46dffdaf10a99728dd8ce358e45d46d8 \

file://dc/dc.h;endline=25;md5=2f9c558cdd80e31b4d904e48c2374328 \

file://lib/number.c;endline=31;md5=99434a0898abca7784acfd36b8191199"

SECTION = "base"

DEPENDS = "flex"

​

SRC_URI = " ${GNU_MIRROR}/bc/bc-${PV}.tar.gz \

file://fix-segment-fault.patch "

SRC_URI[sha256sum] = "4ef6d9f17c3c0d92d8798e35666175ecd3d8efac4009d6457b5c99cea72c0e33"

​

inherit autotools texinfo update-alternatives

​

ALTERNATIVE_${PN} = "dc"

ALTERNATIVE_PRIORITY = "100"

BBCLASSEXTEND = "native"

44

bit.ly/3sS8Jf6

Building upon bbclass

• Use inheritance for common design patterns
• Provide a class file (.bbclass) which is then inherited by other recipes (.bb files)

inherit autotools

• Bitbake will include the autotools.bbclass file
• Found in a classes directory via the BBPATH

45

bit.ly/3sS8Jf6

Examining Recipes: flac

• Look at 'flac' recipe
• Found in

poky/meta/recipes-multimedia/flac/flac_1.3.2.bb

• Inherits from both autotools and gettext
• Customizes autoconf configure options (EXTRA_OECONF) based on "TUNE" features
• Breaks up output into multiple binary packages
• See PACKAGES var. This recipe produces additional packages with those names, while the FILES_* vars specify which files go into these additional packages

46

bit.ly/3sS8Jf6

Examining Recipes: flac.bb

SUMMARY = "Free Lossless Audio Codec"

DESCRIPTION = "FLAC stands for Free Lossless Audio Codec, a lossless audio compression format."

HOMEPAGE = "https://xiph.org/flac/"

BUGTRACKER = "http://sourceforge.net/p/flac/bugs/"

SECTION = "libs"

LICENSE = "GFDL-1.2 & GPLv2+ & LGPLv2.1+ & BSD"

LIC_FILES_CHKSUM = "file://COPYING.FDL;md5=ad1419ecc56e060eccf8184a87c4285f \

file://src/Makefile.am;beginline=1;endline=17;md5=09501c864f89dfc7ead65553129817ca \

file://COPYING.GPL;md5=b234ee4d69f5fce4486a80fdaf4a4263 \

file://src/flac/main.c;beginline=1;endline=18;md5=09777e2934947a36f13568d0beb81199 \

file://COPYING.LGPL;md5=fbc093901857fcd118f065f900982c24 \

file://src/plugin_common/all.h;beginline=1;endline=18;md5=f56cb4ba9a3bc9ec6102e8df03215271 \

file://COPYING.Xiph;md5=b59c1b6d7fc0fb7965f821a3d36505e3 \

file://include/FLAC/all.h;beginline=65;endline=70;md5=64474f2b22e9e77b28d8b8b25c983a48"

DEPENDS = "libogg"

SRC_URI = "http://downloads.xiph.org/releases/flac/${BP}.tar.xz"

SRC_URI[sha256sum] = "91cfc3ed61dc40f47f050a109b08610667d73477af6ef36dcad31c31a4a8d53f"

​

(con't next page)

47

bit.ly/3sS8Jf6

Examining Recipes: flac.bb (con't)

(con't from previous page)

CVE_PRODUCT = "libflac"

inherit autotools gettext

EXTRA_OECONF = "--disable-oggtest \

--with-ogg-libraries=${STAGING_LIBDIR} \

--with-ogg-includes=${STAGING_INCDIR} \

--disable-xmms-plugin \

--without-libiconv-prefix \

ac_cv_prog_NASM=’’ \

"

EXTRA_OECONF += "${@bb.utils.contains("TUNE_FEATURES", "altivec", " --enable-altivec", " --disable-altivec", d)}"

EXTRA_OECONF += "${@bb.utils.contains("TUNE_FEATURES", "core2", " --enable-sse", "", d)}"

EXTRA_OECONF += "${@bb.utils.contains("TUNE_FEATURES", "corei7", " --enable-sse", "", d)}"

PACKAGES += "libflac libflac++ liboggflac liboggflac++"

FILES_${PN} = "${bindir}/*"

FILES_libflac = "${libdir}/libFLAC.so.*"

FILES_libflac++ = "${libdir}/libFLAC++.so.*"

FILES_liboggflac = "${libdir}/libOggFLAC.so.*"

FILES_liboggflac++ = "${libdir}/libOggFLAC++.so.*"

​

​

48

bit.ly/3sS8Jf6

Grouping Local Metadata

• Sometimes sharing metadata between recipes is easier via an include file

include file.inc

• Will include .inc file if found via BBPATH
• Can also specify an absolute path
• If not found, will continue without an error

require file.inc

• Same as an include
• Fails with an error if not found

49

bit.ly/3sS8Jf6

Examining Recipes: ofono

• Look at 'ofono' recipe(s):
• Found in

poky/meta/recipes-connectivity/ofono/ofono_1.19.bb

• Splits recipe into common .inc file to share common metadata between multiple recipes
• Sets a conditional build configuration options through the PACKAGECONFIG var based on a DISTRO_FEATURE (in the .inc file)
• Sets up an init service via do_install:append()
• Has a _git version of the recipe (not shown)

50

bit.ly/3sS8Jf6

Examining Recipes: ofono.bb

require ofono.inc

​

SRC_URI = "\

${KERNELORG_MIRROR}/linux/network/${BPN}/${BP}.tar.xz \

file://ofono \

"

SRC_URI[md5sum] = "a5f8803ace110511b6ff5a2b39782e8b"

SRC_URI[sha256sum] = "a0e09bdd8b53b8d2e4b54f1863ecd9aebe4786477a6cbf8f655496e8edb31c81"

​

CFLAGS:append:libc-uclibc = " -D_GNU_SOURCE"

​

​

51

bit.ly/3sS8Jf6

Examining Recipes: ofono.inc

HOMEPAGE = "http://www.ofono.org"

SUMMARY = "open source telephony"

DESCRIPTION = "oFono is a stack for mobile telephony devices on Linux. oFono supports speaking to telephony devices through specific drivers, or with generic AT commands."

LICENSE = "GPLv2"

LIC_FILES_CHKSUM = "file://COPYING;md5=eb723b61539feef013de476e68b5c50a \

file://src/ofono.h;beginline=1;endline=20;md5=3ce17d5978ef3445def265b98899c2ee"

​

inherit autotools pkgconfig update-rc.d systemd bluetooth

​

DEPENDS = "dbus glib-2.0 udev mobile-broadband-provider-info"

​

INITSCRIPT_NAME = "ofono"

INITSCRIPT_PARAMS = "defaults 22"

​

PACKAGECONFIG ??= "\

${@bb.utils.filter('DISTRO_FEATURES', 'systemd', d)} \

${@bb.utils.contains('DISTRO_FEATURES', 'bluetooth', 'bluez', '', d)} \

"

PACKAGECONFIG[systemd] = "--with-systemdunitdir=${systemd_unitdir}/system/,--with-systemdunitdir="

PACKAGECONFIG[bluez] = "--enable-bluetooth, --disable-bluetooth, ${BLUEZ}"

​

(con't next page)

52

bit.ly/3sS8Jf6

Examining Recipes: ofono.inc

(con't from previous page)

EXTRA_OECONF += "--enable-test"

​

SYSTEMD_SERVICE_${PN} = "ofono.service"

​

do_install:append() {

install -d ${D}${sysconfdir}/init.d/

install -m 0755 ${WORKDIR}/ofono ${D}${sysconfdir}/init.d/ofono

​

# Ofono still has one test tool that refers to Python 2 in the shebang

sed -i -e '1s,#!.*python.*,#!${bindir}/python3,' ${D}${libdir}/ofono/test/set-ddr

​

}

​

PACKAGES =+ "${PN}-tests"

RDEPENDS_${PN} += "dbus"

RRECOMMENDS_${PN} += "kernel-module-tun mobile-broadband-provider-info"

​

FILES_${PN} += "${systemd_unitdir}"

FILES_${PN}-tests = "${libdir}/${BPN}/test"

RDEPENDS_${PN}-tests = "python3 python3-pygobject python3-dbus"

​

​

53

bit.ly/3sS8Jf6

WHEN THINGS GO WRONG

Some useful tools to help guide you when something goes wrong

54

bit.ly/3sS8Jf6

Bitbake Environment

• Each recipe has its own environment which contains all the variables and methods required to build that recipe
• You've seen some of the variables already
• DESCRIPTION, SRC_URI, LICENSE, S, LIC_FILES_CHKSUM, do_compile(), do_install()
• Example
• S = "${WORKDIR}"
• What does this mean?

bit.ly/3sS8Jf6

Examine a Recipe's Environment

• To view a recipe's environment

$ bitbake -e myrecipe

• Where is the source code for this recipe"

$ bitbake -e virtual/kernel | grep “^S=”

S="${HOME}/yocto/build/tmp/work-shared/qemuarm/kernel-source"

• What file was used in building this recipe?

$ bitbake -e netbase | grep “^FILE=”

FILE="${HOME}/yocto/poky/meta/recipes-core/netbase/netbase_5.3.bb"

56

bit.ly/3sS8Jf6

Examine a Recipe's Environment (cont'd)

• What is this recipe's full version string?

$ bitbake -e netbase | grep “^PF=”

PF="netbase-1_5.3-r0"

• Where is this recipe's BUILD directory?

$ bitbake -e virtual/kernel | grep “^B=”

B="${HOME}/yocto/build/tmp/work/qemuarm-poky-linux-\

gnueabi/linux-yocto/3.19.2+gitAUTOINC+9e70b482d3\

_473e2f3788-r0/linux-qemuarm-standard-build"

• What packages were produced by this recipe?

$ bitbake -e virtual/kernel | grep “^PACKAGES=”

PACKAGES="kernel kernel-base kernel-vmlinux kernel-image \ kernel-dev kernel-modules kernel-devicetree"

57

bit.ly/3sS8Jf6

BitBake Log Files

• Every build produces lots of log output for diagnostics and error chasing
• Verbose log of bitbake console output:
• Look in …/tmp/log/cooker/<machine>

$ cat tmp/log/cooker/qemuarm/20160119073325.log | grep 'NOTE:.*task.*Started'

NOTE: recipe hello-1.0.0-r0: task do_fetch: Started

NOTE: recipe hello-1.0.0-r0: task do_unpack: Started

NOTE: recipe hello-1.0.0-r0: task do_patch: Started

NOTE: recipe hello-1.0.0-r0: task do_configure: Started

NOTE: recipe hello-1.0.0-r0: task do_populate_lic: Started

NOTE: recipe hello-1.0.0-r0: task do_compile: Started

NOTE: recipe hello-1.0.0-r0: task do_install: Started

NOTE: recipe hello-1.0.0-r0: task do_populate_sysroot: Started

NOTE: recipe hello-1.0.0-r0: task do_package: Started

NOTE: recipe hello-1.0.0-r0: task do_packagedata: Started

NOTE: recipe hello-1.0.0-r0: task do_package_write_rpm: Started

NOTE: recipe hello-1.0.0-r0: task do_package_qa: Started

NOTE: recipe ypdd-image-1.0-r0: task do_rootfs: Started

58

bit.ly/3sS8Jf6

BitBake Per-Recipe Log Files

• Every recipe produces lots of log output for diagnostics and debugging
• Use the Environment to find the log files for a given recipe:

$ bitbake -e hello | grep “^T=”

T="${HOME}yocto/build/tmp/work/armv5e-poky-linux-gnueabi/hello/1.0.0-r0/temp"

• Each task that runs for a recipe produces "log" and "run" files in

${WORKDIR}/temp

​

​

59

bit.ly/3sS8Jf6

BitBake Per-Recipe Log Files

$ cd ${T} (See definition of T in previous slide)

$ find . -type l -name 'log.*'

./log.do_package_qa

./log.do_package_write_rpm

./log.do_package

./log.do_fetch

./log.do_populate_lic

./log.do_install

./log.do_configure

./log.do_unpack

./log.do_populate_sysroot

./log.do_compile

./log.do_packagedata

./log.do_patch

60

These files contain the output of the respective tasks for each recipe

bit.ly/3sS8Jf6

BitBake Per-Recipe Log Files

$ cd ${T} (See definition of T in previous slide)

$ find . -type l -name 'run.*'

./run.do_fetch

./run.do_patch

./run.do_configure

./run.do_populate_sysroot

./run.do_package_qa

./run.do_unpack

./run.do_compile

./run.do_install

./run.do_packagedata

./run.do_populate_lic

./run.do_package

./run.do_package_write_rpm

61

These files contain the commands executed which produce the build results

bit.ly/3sS8Jf6

BUILDING A FULL EMBEDDED IMAGE WITH YOCTO

This section will introduce the concept of building an initial system image

62

bit.ly/3sS8Jf6

Quick Start Guide in one Slide

1. Download Yocto Project sources:
• Use git and checkout a known branch

$ git clone -b kirkstone git://git.yoctoproject.org/poky.git

• Build one of the reference Linux distributions:

$ source poky/oe-init-build-env mybuild

• Check/Edit local.conf for sanity (e.g. modify MACHINE = "qemux86" or MACHINE = "qemuarm")

mybuild$ bitbake -k core-image-minimal

• Run the image under emulation:

mybuild$ runqemu qemux86

• Profit!!! (well… actually there is more work to do...)

63

bit.ly/3sS8Jf6

Host System Layout

$HOME/yocto/

|---build (or whatever name you choose)

Project build directory

|---downloads (DL_DIR)

Downloaded source cache

|---poky (Do Not Modify anything in here*)

Poky, bitbake, scripts, oe-core, metadata

|---sstate-cache (SSTATE_DIR)

Binary build cache

64

* We will cover how to use layers to make changes later

bit.ly/3sS8Jf6

oe-core (meta)

meta-poky

meta-yocto-bsp

Poky Layout

$HOME/yocto/poky/

|---LICENSE

|---README

|---README.hardware

|---bitbake/ (The build tool)

|---documentation/

|---meta/ (oe-core)

|---meta-poky/ (Yocto distro metadata)

|---meta-yocto-bsp/ (Yocto Reference BSPs)

|---oe-init-build-env (Project setup script)

|---scripts/ (Scripts and utilities)

65

Note: A few files have been items omitted to facility the presentation on this slide

bit.ly/3sS8Jf6

Setting up a Build Directory

• Start by setting up a build directory
• Local configuration
• Temporary build artifacts

$ cd $HOME/yocto/

$ source ./poky/oe-init-build-env build

​

• Replace build with whatever directory name you want to use for your project
• You need to re-run this script in any new terminal you start (and don’t forget the project directory)

66

bit.ly/3sS8Jf6

Build directory Layout

$HOME/yocto/build/

|---bitbake.lock

|---cache/ (bitbake cache files)

|---conf/

| |--bblayers.conf (bitbake layers)

| |--local.conf (local configuration)

| `--site.conf (optional site conf)

`---tmp/ (Build artifacts)

67

Note: A few files have been items omitted to facility the presentation on this slide

bit.ly/3sS8Jf6

Building a Linux Image

• General Procedure:
• Create a project directory using
• source oe-init-build-env [prj-dir]
• Configure build by editing local.conf
• $HOME/yocto/build/conf/local.conf
• Select appropriate MACHINE type
• Set shared downloads directory (DL_DIR)
• Set shared state directory (SSTATE_DIR)
• Build your selected Image
• $ bitbake -k core-image-minimal
• (Detailed steps follow…)

68

bit.ly/3sS8Jf6

Update Build Configuration

• Configure build by editing local.conf

$HOME/yocto/build/conf/local.conf

• Set appropriate MACHINE, DL_DIR and SSTATE_DIR
• Add the following to the bottom of local.conf

​

MACHINE = "qemuarm"

DL_DIR = "${TOPDIR}/../downloads"

SSTATE_DIR = "${TOPDIR}/../sstate-cache/${MACHINE}"

​

• Notice how you can use variables in setting these values

69

bit.ly/3sS8Jf6

Building an Embedded Image

• This builds an entire embedded Linux distribution
• Choose from one of the available Images
• The following builds a minimal embedded target

$ bitbake -k core-image-minimal

• On a fast computer the first build may take the better part of an hour on a slow machine multiple ...
• The next time you build it (with no changes) it may take as little as 5 mins (due to the shared state cache)

70

bit.ly/3sS8Jf6

Booting Your Image with QEMU

• The runqemu script is used to boot the image with QEMU
• It auto-detects settings as much as possible, enabling the following command to boot our reference images:

$ runqemu qemuarm [nographic]

• Use nographic if using a non-graphical session (ssh), do not type the square brackets
• Replace qemuarm with your value of MACHINE
• Your QEMU instance should boot
• Quit by closing the qemu window
• If using “nographic”, kill it from another terminal:

$ killall qemu-system-arm

71

bit.ly/3sS8Jf6

LAYERS

This section will introduce the concept of layers and how important they are in the overall build architecture

72

bit.ly/3sS8Jf6

Layers

• Metadata is provided in a series of layers which allow you to override any value without editing the originally provided files
• A layer is a logical collection of metadata in the form of recipes
• A layer is used to represent oe-core, a Board Support Package (BSP), an application stack, and your new code
• All layers have a priority and can override policy, metadata and config settings of layers with a lesser priority

73

bit.ly/3sS8Jf6

Layer Hierarchy

74

meta (oe-core)

meta-poky

BSP layer

UI/GUI layer

Commercial layers (OSV or middleware)

Developer layer(s)

bit.ly/3sS8Jf6

Using Layers

• Layers are added to your build by inserting them into the BBLAYERS variable within your bblayers file

$HOME/yocto/build/conf/bblayers.conf

​

BBLAYERS ?= " \

${HOME}/yocto/poky/meta \

${HOME}/yocto/poky/meta-poky \

${HOME}/yocto/poky/meta-yocto-bsp \

"

75

bit.ly/3sS8Jf6

Board Support Packages

• BSPs are layers to enable support for specific hardware platforms
• Defines machine configuration variables for the board (MACHINE)
• Adds machine-specific recipes and customizations
• Boot loader
• Kernel config
• Graphics drivers (e.g, Xorg)
• Additional recipes to support hardware features

76

bit.ly/3sS8Jf6

Notes on using Layers

• When doing development with Yocto, do not edit files within the Poky source tree
• Use a new custom layer for modularity and maintainability
• Layers also allow you to easily port from one version of Yocto/Poky to the next version

77

bit.ly/3sS8Jf6

Creating a Custom Layer

• Layers can be created manually
• They all start with “meta-” by convention
• However using the bitbake-layers tool is easier

$ bitbake-layers create-layer meta-ypdd

• This will create meta-ypdd in the current dir

78

bit.ly/3sS8Jf6

Create a Custom Layer

$ cd yocto

yocto$ source poky/oe-init-build-env build

yocto/build$ bitbake-layers create ypdd

Please enter the layer priority you'd like to use for the layer: [default: 6] 6

Would you like to have an example recipe created? (y/n) [default: n] y

Please enter the name you'd like to use for your example recipe: [default: example] example

Would you like to have an example bbappend file created? (y/n) [default: n] n

​

New layer created in meta-ypdd.

​

Don't forget to add it to your BBLAYERS (for details see meta-ypdd\README).

yocto/build$

79

bit.ly/3sS8Jf6

The new Custom Layer

yocto/build$ tree meta-ypdd

meta-ypdd/

|--COPYING.MIT (The license file)

|--README (Starting point for README)

|--conf

| `--layer.conf (Layer configuration file)

`--recipes-example (A grouping of recipes)

`--example (The example package)

|--example-0.1 (files for v0.1 of example)

| |--example.patch

| `--helloworld.c

`--example_0.1.bb (The example recipe)

80

bit.ly/3sS8Jf6

Layer.conf

# We have a conf and classes directory, add to BBPATH

BBPATH .= ":${LAYERDIR}"

​

# We have recipes-* directories, add to BBFILES

BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \

${LAYERDIR}/recipes-*/*/*.bbappend"

​

BBFILE_COLLECTIONS += "ypdd"

BBFILE_PATTERN_ypdd = "^${LAYERDIR}/"

BBFILE_PRIORITY_ypdd = "6"

81

bit.ly/3sS8Jf6

Adding Layers to Your Build

• Add your layer to bblayers.conf
• $HOME/yocto/build/conf/bblayers.conf

​

BBLAYERS ?= " \

${HOME}/yocto/poky/meta \

${HOME}/yocto/poky/meta-poky \

${HOME}/yocto/poky/meta-yocto-bsp \

${HOME}/yocto/build/meta-ypdd \

“

82

bit.ly/3sS8Jf6

Adding Layers to Your Build

> bitbake-layers --help

usage: bitbake-layers [-d] [-q] [--color COLOR] [-h] <subcommand> ...

BitBake layers utility

optional arguments:

-d, --debug Enable debug output

-q, --quiet Print only errors

--color COLOR Colorize output (where COLOR is auto, always, never)

-h, --help show this help message and exit

subcommands:

<subcommand>

layerindex-fetch Fetches a layer from a layer index along with its

dependent layers, and adds them to conf/bblayers.conf.

layerindex-show-depends

Find layer dependencies from layer index.

add-layer Add a layer to bblayers.conf.

remove-layer Remove a layer from bblayers.conf.

flatten flatten layer configuration into a separate output

directory.

show-layers show current configured layers.

show-overlayed list overlayed recipes (where the same recipe exists

in another layer)

show-recipes list available recipes, showing the layer they are

provided by

show-appends list bbappend files and recipe files they apply to

show-cross-depends Show dependencies between recipes that cross layer

boundaries.

​

83

bit.ly/3sS8Jf6

layers.openembedded.org

​

​

​

​

​

84

bit.ly/3sS8Jf6

Adding Layers to Your Build

• Add your layer to bblayers.conf

​

bitbake-layers add-layer \

${HOME}/yocto/build/meta-ypdd

85

bit.ly/3sS8Jf6

Build Your New Recipe

• You can now build the new recipe

$ bitbake example

• This will now build the example_0.1.bb recipe which is found in

meta-ypdd/recipes-example/example/example_0.1.bb

​

Note: Build fails w/o ${CFLAGS} and ${LDFLAGS} meanwhile (QA-error) in the recipe.

86

bit.ly/3sS8Jf6

IMAGES

This section will introduce the concept of images; recipes which build embedded system images

87

bit.ly/3sS8Jf6

What is an Image?

• Building an image creates an entire Linux distribution from source
• Compiler, tools, libraries
• BSP: Bootloader, Kernel
• Root filesystem:
• Base OS
• services
• Applications
• etc

88

bit.ly/3sS8Jf6

Extending an Image

• You often need to create your own Image recipe in order to add new packages or functionality
• With Yocto/OpenEmbedded it is always preferable to extend an existing recipe or inherit a class
• The simplest way is to inherit the core-image bbclass
• You add packages to the image by adding them to IMAGE_INSTALL

89

bit.ly/3sS8Jf6

A Simple Image Recipe

• Create an images directory

$ mkdir -p ${HOME}/yocto/build/meta-ypdd/recipes-core/images

​

• Create the image recipe

$ vi ${HOME}/yocto/build/meta-ypdd/recipes-core/images/ypdd-image.bb

​

DESCRIPTION = "A core image for YPDD"

LICENSE = "MIT"

​

# Core files for basic console boot

IMAGE_INSTALL = "packagegroup-core-boot"

​

# Add our desired packages

IMAGE_INSTALL += "psplash dropbear"

​

inherit core-image

​

IMAGE_ROOTFS_SIZE ?= "8192"

90

bit.ly/3sS8Jf6

Build and Boot Your Custom Image

• Enable the meta-ypdd layer in your build
• Edit conf/bblayers.conf and add the path to meta-ypdd to the BBLAYERS variable declaration

​

(example in the next slide)

​

91

bit.ly/3sS8Jf6

Add Your Layer

• Make sure your layer is added to BBLAYERS in bblayers.conf

$HOME/yocto/build/conf/bblayers.conf

BBLAYERS ?= " \

${HOME}/yocto/poky/meta \

${HOME}/yocto/poky/meta-poky \

${HOME}/yocto/poky/meta-yocto-bsp \

${HOME}/yocto/build/meta-ypdd \

"

• (We already did this step in a previous section manually and with bitbake-layers add-layer.)

92

bit.ly/3sS8Jf6

Build and Boot Your Custom Image

• Build your custom image:

$ bitbake -k ypdd-image

(If your SSTATE_DIR is configured correctly from a previous build this should take less than 5 minutes)

​

• Boot the image with QEMU:

$ runqemu qemuarm \

tmp/deploy/images/qemuarm/ypdd-image-qemuarm.ext4 \

[nographic]

93

Use nographic if using ssh environment

bit.ly/3sS8Jf6

Build and Boot Your Custom Image

• Verify that dropbear ssh server is present

$ which dropbear

​

• If you used the graphical invocation of QEMU using VNC viewer, you will see the splash screen on boot.

94

bit.ly/3sS8Jf6

Toaster

The following section introduces toaster

95

bit.ly/3sS8Jf6

bit.ly/3sS8Jf6

toaster in one slide

• Download clone poky git repo:

$ mkdir -p ${HOME}/toaster ; cd ${HOME}/toaster �$ git clone -b kirkstone git://git.yoctoproject.org/poky

# Note: git checkout is required by toaster - do not use the release tarball

• Download/install dependencies:

$ sudo apt-get install python-virtualenv�$ virtualenv venv�$ source venv/bin/activate # you’ll have to activate it every time to enter the environment�(venv)$ pip3 install -r poky/bitbake/toaster-requirements.txt

• Start toaster:

(venv)$ source poky/oe-init-build-env toasterprjdir

(venv)$ source toaster start # wait 2 minutes …

• Profit!!! (well… actually there is more work to do...)

$ firefox http://localhost:8000

97

bit.ly/3sS8Jf6

TODO Toaster

98

bit.ly/3sS8Jf6

Create new build ...

1)

​

2)

​

​

​

3)

99

bit.ly/3sS8Jf6

Toaster demo/walkthrough

• Main page
• Create new project
• Select Machine
• Add custom layers or recipes
• Build an image
• Image
• Image manifest

​

​

100

bit.ly/3sS8Jf6

BUILD AN APPLICATION

Adding a "hello world" application to our custom image

101

bit.ly/3sS8Jf6

Building an Application

• General procedure:
• Write hello world application (hello.c)
• Create recipe for hello world application
• Modify image recipe to add hello world application to your image
• What follows is the example of a simple one C file application
• (Building a more complicated recipe from a tarball would specify how to find the upstream source with the SRC_URI)

102

bit.ly/3sS8Jf6

Add Application Code

• For a simple one C file package, you can add the hello application source to a directory called files in the hello package directory

​

$ mkdir -p ${HOME}/yocto/build/meta-ypdd/\

recipes-core/hello/files

​

$ vi /scratch/sandbox/meta-ypdd/recipes-core/\

hello/files/hello.c

103

bit.ly/3sS8Jf6

Application Code

104

#include <stdio.h>

​

int main(int argc, char **argv) {

printf("Hello World\n");

return 0;

}

$ vi /scratch/sandbox/meta-ypdd/recipes-core/hello/files/hello.c

bit.ly/3sS8Jf6

Add Application Recipe

• Write hello world recipe
• Create directory to hold the recipe and associated files

$ mkdir -p ${HOME}/yocto/build/meta-ypdd/\

recipes-core/hello

• (We actually did this already in the previous step)

​

• Create hello_1.0.bb (next slide)

$ vi ${HOME}/yocto/build/meta-ypdd/\

recipes-core/hello/hello_1.0.bb

105

bit.ly/3sS8Jf6

Application Recipe

106

DESCRIPTION = "Hello World example"

LICENSE = "MIT"

​

LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"

​

S = "${WORKDIR}"

​

SRC_URI = "file://hello.c"

​

do_compile() {

${CC} ${CFLAGS} ${LDFLAGS} hello.c -o hello

}

​

do_install() {

install -d -m 0755 ${D}/${bindir}

install -m 0755 hello ${D}/${bindir}/hello

}

$ vi ${HOME}/yocto/build/meta-ypdd/recipes-core/hello/hello_1.0.bb

bit.ly/3sS8Jf6

Add Application to the Image

• Modify image recipe to add hello world application to your image
• See example on next slide

​

107

bit.ly/3sS8Jf6

Add hello to Image

108

DESCRIPTION = "A core image for YPDD"

LICENSE = "MIT"

​

# Core files for basic console boot

IMAGE_INSTALL = "packagegroup-core-boot"

​

# Add our desired extra files

IMAGE_INSTALL += "psplash dropbear hello"

​

inherit core-image

​

IMAGE_ROOTFS_SIZE ?= "8192"

$ vi ${HOME}/yocto/build/meta-ypdd/recipes-core/images/ypdd-image.bb

Add the package 'hello'�to your image recipe

bit.ly/3sS8Jf6

Build and Test Application

• Now (re)build your image recipe

$ bitbake -k ypdd-image

• hello_1.0.bb will be processed because it is in your custom layer, and referenced in your image recipe.
• Boot your image using runqemu, as before:

$ runqemu qemuarm tmp/deploy/images/\

qemuarm/ypdd-image-qemuarm.ext4 nographic

• You should be able to type "hello" at the command line and see "Hello World"

​

109

bit.ly/3sS8Jf6

Embedded Linux Development with Yocto Project

Training from The Linux Foundation

Want to learn how to use Yocto Project like a Pro?

https://training.linuxfoundation.org/

​

Embedded Linux Platform Development with Yocto Project

http://bit.ly/eldyocto

​

bit.ly/3sS8Jf6

111

It’s not an Embedded

Linux Distribution

​

​

It Creates a

Custom One For You

bit.ly/3sS8Jf6

TIPS HINTS AND OTHER RESOURCES

The following slides contain reference material that will help you climb the Yocto Project learning curve

112

bit.ly/3sS8Jf6

Common Gotchas When Getting Started

• Working behind a network proxy? Please follow this guide:
• https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy
• Do not try to re-use the same shell environment when moving between copies of the build system
• oe-init-build-env script appends to your $PATH, it's results are cumulative and can cause unpredictable build errors

113

bit.ly/3sS8Jf6

Project Resources

• The Yocto Project is an open source project, and aims to deliver an open standard for the embedded Linux community and industry
• Development is done in the open through public mailing lists: openembedded-core@lists.openembedded.org, poky@yoctoproject.org, and yocto@yoctoproject.org
• And public code repositories:
• http://git.yoctoproject.org and
• http://git.openembedded.org
• Bug reports and feature requests
• http://bugzilla.yoctoproject.org

114

bit.ly/3sS8Jf6

Tip: ack-grep

• Much faster than grep for the relevant use cases
• Designed for code search
• Searches only relevant files
• Knows about many types: C, asm, perl
• By default, skips .git, .svn, etc.
• Can be taught arbitrary types
• Perfect for searching metadata

​

115

bit.ly/3sS8Jf6

Tip: ack-grep

116

alias bback='ack-grep --type bitbake'

bit.ly/3sS8Jf6

TIP: VIM Syntax Highlighting

• https://github.com/openembedded/bitbake/tree/master/contrib/vim
• Install files from the above repo in ~/.vim/
• Add "syntax on" in ~/.vimrc

​

$ tree ~/.vim/

/Users/chris/.vim/

├── ftdetect

│   └── bitbake.vim

├── ftplugin

│   └── bitbake.vim

├── plugin

│   └── newbb.vim

└── syntax

└── bitbake.vim

117

bit.ly/3sS8Jf6

TIP: VIM Syntax Highlighting

118

bit.ly/3sS8Jf6

End

​

​

​

HAVE FUN and thank you for joining !

119

bit.ly/3sS8Jf6