1 of 60

Bazel Bootcamp

Danna Kelmer (dannark@)

Structure

Intro + Demo (3 slides, 15 mins)
Project structure (15 slides, 30 mins)
Break (5 mins)
Backend Exercise (45 mins)
Break (5 mins)
Tests (4 slides, 10 mins)
Java Client and Tests (45 mins)
Break (5 mins)
Rules (4 slides, 15 mins)
TypeScript (30 mins)
Integration Test (20 mins)
Phases of the Build (4 slides, 10 mins)
Query (4 slides, 5 mins)

Introduce yourself! Danna, work at google, will be instructor. We also have TAs who will be helping out. They’ll be swapping out over course of bootcamp but please introduce yourselves

Main goal today is to learn about bazel and how it works. Instruction + create configuration files for a project that enable bazel to run. A goal of mine in creating the boot camp was to have it not be like those code labs where you copy paste code, so expect to get stuck occasionally, but it will be fun. You will start with none of the configuration files and by the end will have the completed version.

I’ll demo the project shortly.

2 of 60

Before we get started

Connect to wifi: Google Guest
Which version of bazel?
Download repository:�git clone -b 0.23.2 https://github.com/bazelbuild/codelabs.git
Intros!

Bazel

bazel.build

3 of 60

Bazel

bazel.build

4 of 60

History + Intro to Bazel Basics
Bazel Project Structure
Exercise: Backend
Tests
Exercise: Tests
Rules Deep Dive
Exercise: Frontend
Exercise: Integration Tests
Query

Bazel

bazel.build

5 of 60

Intro + Bazel Project Structure

6 of 60

What is Bazel?

.java

.h

.cpp

.h

.plist

…

(etc.)

.scala

javac

gcc

.m

tsc

…

(etc.)

packages

executables

transformed source

Bazel

bazel.build

7 of 60

Features of Bazel?

Correct: no clean, incremental builds, reproducible commands
Fast: parallelizable, incremental
Portable: caching
Cross platform
Universal: multi-language build system
Extensible: starlark

Bazel

bazel.build

8 of 60

Where did Bazel Come From?

pre-2003: single Makefile
2003: gconfig (BUILD file → Makefile)
2006: BUILD file parser introduced (Beginning of modern Bazel)
2007: Remote execution and caching support added
2015: Blaze is open sourced as Bazel

Bazel

bazel.build

9 of 60

WORKSPACE files

BUILD files

Bazel

bazel.build

10 of 60

WORKSPACE

Text file called WORKSPACE
At the root of your project
Only one per (separate) project
Has a globally unique name

Bazel

bazel.build

11 of 60

Multiple Bazel Workspaces

$ tree

.

├── churros_project

│ └── WORKSPACE

├── donut_project

│ └── WORKSPACE

└── ice_cream_project

└── WORKSPACE

Bazel

bazel.build

12 of 60

BUILD files

Multiple BUILD files in each project
Each directory with a BUILD file is called a package

Excluding directories that contain BUILD files themselves

Packages can have multiple targets (declared in its BUILD files)

Rules and Files

Targets are the smallest units that can be built

bazel build //path/to/mypackage:mytarget

Written in language called Starlark

Bazel

bazel.build

13 of 60

├── WORKSPACE�├── BUILD.bazel

├── README.md

├── animations

│ ├── BUILD.bazel

│ ├── PACKAGE.md

│ ├── browser

│ │ ├── BUILD.bazel

│ │ ├── index.ts

│ │ ├── package.json

│ │ ├── public_api.ts

│ │ └── tsconfig-build.json

│ ├── index.ts

│ ├── rollup.config.js

│ ├── src

│ │ ├── animation_player_spec.ts

│ │ ├── players

│ │ ├── private_export.ts

│ │ ├── util.ts

│ │ ├── util_spec.ts

│ │ └── version.ts

│ ├── utils

│ │ ├── BUILD.bazel

│ │ └── util.ts

│ └── tsconfig-build.json

...

13

# How many packages do we have�# in this workspace?

14 of 60

├── WORKSPACE�├── BUILD.bazel

├── README.md

├── animations

│ ├── BUILD.bazel

│ ├── PACKAGE.md

│ ├── browser

│ │ ├── BUILD.bazel

│ │ ├── index.ts

│ │ ├── package.json

│ │ ├── public_api.ts

│ │ └── tsconfig-build.json

│ ├── index.ts

│ ├── rollup.config.js

│ ├── src

│ │ ├── animation_player_spec.ts

│ │ ├── players

│ │ ├── private_export.ts

│ │ ├── util.ts

│ │ ├── util_spec.ts

│ │ └── version.ts

│ ├── utils

│ │ ├── BUILD.bazel

│ │ └── util.ts

│ └── tsconfig-build.json

...

14

# How many packages do we have�# in this workspace?

top level package

15 of 60

├── WORKSPACE�├── BUILD.bazel

├── README.md

├── animations

│ ├── BUILD.bazel

│ ├── PACKAGE.md

│ ├── browser

│ │ ├── BUILD.bazel

│ │ ├── index.ts

│ │ ├── package.json

│ │ ├── public_api.ts

│ │ └── tsconfig-build.json

│ ├── index.ts

│ ├── rollup.config.js

│ ├── src

│ │ ├── animation_player_spec.ts

│ │ ├── players

│ │ ├── private_export.ts

│ │ ├── util.ts

│ │ ├── util_spec.ts

│ │ └── version.ts

│ ├── utils

│ │ ├── BUILD.bazel

│ │ └── util.ts

│ └── tsconfig-build.json

...

15

# How many packages do we have�# in this workspace?

top level package
animations package

16 of 60

├── WORKSPACE�├── BUILD.bazel

├── README.md

├── animations

│ ├── BUILD.bazel

│ ├── PACKAGE.md

│ ├── browser

│ │ ├── BUILD.bazel

│ │ ├── index.ts

│ │ ├── package.json

│ │ ├── public_api.ts

│ │ └── tsconfig-build.json

│ ├── index.ts

│ ├── rollup.config.js

│ ├── src

│ │ ├── animation_player_spec.ts

│ │ ├── players

│ │ ├── private_export.ts

│ │ ├── util.ts

│ │ ├── util_spec.ts

│ │ └── version.ts

│ ├── utils

│ │ ├── BUILD.bazel

│ │ └── util.ts

│ └── tsconfig-build.json

...

16

# How many packages do we have�# in this workspace?

top level package
animations package
animations/browser package

17 of 60

├── WORKSPACE�├── BUILD.bazel

├── README.md

├── animations

│ ├── BUILD.bazel

│ ├── PACKAGE.md

│ ├── browser

│ │ ├── BUILD.bazel

│ │ ├── index.ts

│ │ ├── package.json

│ │ ├── public_api.ts

│ │ └── tsconfig-build.json

│ ├── index.ts

│ ├── rollup.config.js

│ ├── src

│ │ ├── animation_player_spec.ts

│ │ ├── players

│ │ ├── private_export.ts

│ │ ├── util.ts

│ │ ├── util_spec.ts

│ │ └── version.ts

│ ├── utils

│ │ ├── BUILD.bazel

│ │ └── util.ts

│ └── tsconfig-build.json

...

17

# How many packages do we have�# in this workspace?

top level package
animations package
animations/browser package
animations/utils package

18 of 60

Anatomy of a Label

@angular//animations/utils:draw_circle

optional repository name

root of repository

package

target

Bazel

bazel.build

19 of 60

Anatomy of a Label - Shorthand

//animations/util:draw_circle

package

target

root of workspace

containing $PWD

Bazel

bazel.build

20 of 60

27

package(default_visibility = ["//visibility:private"])

load("@build_bazel_rules_typescript//:defs.bzl", "ts_library")

ts_library(

name = "lib",

srcs = glob(["*.ts"]),

tsconfig = ":tsconfig.json",

deps = [

"@ngdeps//types/node",

],

)

j

28 of 60

A workspace may have many packages
A package may have many target definitions
A target could be a file or a rule

Workspace

Package

Target

Bazel

bazel.build

29 of 60

Anatomy of an Invocation

bazel --max_idle_secs build --c opt //animations/util:draw_circle

startup options

label

command options

command

bazel

Bazel

bazel.build

30 of 60

.bazelrc

Examples� startup --output_base=/tmp� build --logging=5� build --compiler=/path/to/c++/compiler
.bazelrc can be set in

Workspace root directory
Per user in $HOME
Globally in /etc/.bazelrc

Bazel

bazel.build

31 of 60

Exercise #1: Backend

32 of 60

Tests

33 of 60

Testing With Bazel

Run tests with bazel test command
Like bazel run where Bazel checks the exit code at the end and enforces timeouts
Tests should be hermetic, e.g. don’t access internet
Tests access inputs through runfiles
Testing benefits from caching

Bazel

bazel.build

34 of 60

Testing With Bazel

Testing attributes

flaky
shard_count
size/timeout
tags

local
remote
exclusive

Bazel

bazel.build

35 of 60

Data dependencies

Dependencies available at run time
Help parallelize the build

Bazel

bazel.build

36 of 60

Testing With Bazel

Testing flags

--test_output=<summary, error, streamed, ...>
--test_summary=<short,detailed, none, testcase>
--test_filter=<...>
--runs_per_test

Bazel

bazel.build

37 of 60

Testing With Bazel

Test Environment Variables

TEST_TMPDIR
TEST_SRCDIR

Bazel

bazel.build

38 of 60

Exercise #2: Java Client and Unit Tests

39 of 60

Testing Demo

40 of 60

Rules Deep Dive

41 of 60

Rules

Establish relation between inputs and outputs
Language specific set of instructions for creating outputs
Common types: *_library, *_binary, *_test
All rules in build files have names, sometimes affects output artifacts

Bazel

bazel.build

Inputs:

Source files
tools

Types of outputs:

Compiled executables
Libraries
Executable tests
Other outputs

Library:

rules specify separately-compiled modules in the given programming language. Libraries can depend on other libraries, and binaries and tests can depend on libraries, with the expected separate-compilation behavior.

Binary:

*_binary rules build executable programs in a given language. After a build, the executable will reside in the build tool's binary output tree at the corresponding name for the rule's label, so //my:program would appear at (e.g.) $(BINDIR)/my/program.

Such rules also create a runfiles directory containing all the files mentioned in a data attribute belonging to the rule, or any rule in its transitive closure of dependencies; this set of files is gathered together in one place for ease of deployment to production.

Test:

rules are a specialization of a *_binary rule, used for automated testing. Tests are simply programs that return zero on success. (check exit code of program)

Rule names affecting output: e.g. java_library creates libname

42 of 60

Attributes

All rules have attributes that allow the user to specify the actual inputs, outputs, tools etc
Some attributes for customization/configuration
Have names and types
Most rules have srcs, deps, data attributes

Bazel

bazel.build

43 of 60

Outputs

Rules have certain predeclared outputs

jar file
binaries
apk

Outputs of one rule become input to other rule
Have default and extra outputs

Bazel

bazel.build

44 of 60

Rule Implementation Function

What the rule actually does
Language specific steps for how to create the final output, called Actions
May create various intermediary artifacts

Bazel

bazel.build

45 of 60

Target Graph

//packages/core:core

//packages/bazel/src/ngc-wrapped:xi18n

//packages/bazel/src/ngc-wrapped:xi18n_bin

//packages/bazel/src/ngc-wrapped:ngc_lib

//packages/compiler-cli:compiler-cli

//packages/compiler:compiler

Bazel

bazel.build

46 of 60

Action Graph

core.umd.js

rollup

di.js, linker.js, render.js, ...

tsc

di.ts, linker.ts, render.ts, ...

uglify-js

Created by rules
A spawned process -- calls exec
Can examine the actions by passing --subcommands to the build

Can be executed remotely

Bazel

bazel.build

47 of 60

Look at bootcamp WORKSPACE file

Bazel

bazel.build

48 of 60

Exercise #3 + #4: Frontend + Integration Tests

49 of 60

Exploring Bazel

50 of 60

Exploring Bazel's output tree

bazel info

INFO: Invocation ID: 80e22a8f-43c4-40f8-8c61-a75ce680b7cd

bazel-bin: /private/.../execroot/angular/bazel-out/darwin-fastbuild/bin

output_base: /private/var/tmp/_bazel_$USR/7828fda17547c0ca9f5e48bf8194e356

50

51 of 60

What is Query?

A bazel command
Allows you to ask questions about your BUILD graph

Bazel

bazel.build

52 of 60

List all targets in //proto/logger

$ bazel query //proto/logger:all

$ bazel query //proto/logger:*

Bazel

bazel.build

53 of 60

List all targets in the project

$ bazel query //...

Bazel

bazel.build

54 of 60

Find all dependencies of //go/cmd/server:go-server

$ bazel query “deps(//go/cmd/server:go-server)”

Bazel

bazel.build

55 of 60

Why does �//java/src/main/java/bazel/bootcamp:JavaLoggingClient depend on�//proto/logger:logger_proto

$ bazel query ‘somepath(//java/src/main/java/bazel/bootcamp:JavaLoggingClient, //proto/logger:logger_proto)’

$ bazel query ‘allpaths(//java/src/main/java/bazel/bootcamp:JavaLoggingClient, //proto/logger:logger_proto)’

Bazel

bazel.build

56 of 60

Why does the //foo tree depend on //bar/baz?

$ bazel query ‘somepath(foo/…, //bar/baz:all)’

Bazel

bazel.build

57 of 60

What C++ libraries do all the foo tests depend on that the foo_bin target does not?

$ bazel query ‘kind("cc_library", deps(kind(".*test rule", foo/...)) except deps(//foo:foo_bin))’

Bazel

bazel.build

58 of 60

cquery
aquery

Bazel

bazel.build

59 of 60

Genquery

chocolate/BUILD

genquery(

name = “chocolate-deps”,

expression = “deps(//chocolate:choclate_lib”,

scope = [“//chocolate:chocolate_lib”],

)

java_binary(

name = “chocolate”,

deps = [“:chocolate_lib”],

main_class = “chocolate.chocolateMain”,

)

Bazel

bazel.build

60 of 60

Thank you!