Udgivet via Google Docs
Bazel's dependency on the shell and stuff
Opdateres automatisk hvert 5. minut

Bazel's dependency on the shell and stuff*

Author: laszlocsomor@google.com

Tracking bug: https://github.com/bazelbuild/bazel/issues/4319

Status: under review

Last update: 2018/04/09

Reviewers: lberki@google.com [waiting], jcater@google.com [LGTM]

Guiding philosophy: https://lamport.azurewebsites.net/pubs/state-the-problem.pdf

*: including, but not limited to: coreutils, Perl

Problem

Bazel can't work without a shell and bintools[1]. Bazel assumes that /bin/sh and /bin/bash always exist and that bintools are installed and on the PATH. This assumption is wrong on most platforms. Notably on Windows there's no default POSIX shell and the Bazel docs recommend installing MSYS2[2] (a Bash port with some coreutils).

This document shows how to formalize Bazel's dependency on the shell. Doing so allows Bazel to handle a missing shell and work without one. The same formalization is useful to express the dependency on the bintools.

[1] In this document by "bintools" we mean the coreutils and additional command line tools such as Perl.

[2] Along with some of the bintools.

Current system

Bazel uses the shell or bintools in the following places:

[1] Examples: pkg.bzl, java_rules_skylark.bzl, java_grpc_library.bzl, protobuf.bzl

Requirements

The solution ensures that:

In addition, a refined solution ensures that:

Proposed approach

Instead of storing the shell's path in the ShellConfiguration, create a toolchain rule that stores the path. Every rule that needs a shell interpreter should require this toolchain, and ask it for the interpreter's path.

Initial solution

  1. Remove Bash- and shell-related code from ShellConfiguration, including the hardcoded Bash paths and the checking of the BAZEL_SH environment variable (wrongly so in the environment of the Bazel server, not of the Bazel client).
  2. Implement a repository rule that discovers the shell interpreter on the local machine, and creates a toolchain rule that stores its path (or the empty string if no interpreter is found). Change Bazel to automatically create this repository and register the toolchain. Users can manually register additional toolchains for remote builds or alternative shell paths.
  3. Change all native rules that need a shell, including all test rules, to require the toolchain. To look up the shell interpreter's path, the rule asks the toolchain rule and reports an analysis-time error if the path is empty. The toolchain returns either the --shell_executable flag's value (in the ShellConfiguration fragment) or the selected toolchain's path attribute's value.
  4. Change Skylark rules to implicitly depend on the shell toolchain.[1] If the rule uses ctx.action.run_shell but the shell path is empty, the Skylark evaluator reports an evaluation error. Since the toolchain rule itself is written in Skylark, it would depend on itself -- to avoid this we can use a boolean attribute that's only ever true on this rule.
  5. Change java_binary and py_binary rules (not on Windows) to depend on the toolchain and insert the local Bash or Env path into the launcher script's shebang line.
  6. Change every binary rule to implicitly depend on the toolchain, so that the RunCommand can retrieve the shell path for it. Later maybe change the RunCommand to not use a shell at all.

[1] Reason: Bazel will require an explicit dependency eventually, see the "refined solutions", 3rd bullet point. Starting to require this dependency from one Bazel release to another without a transitional period would break existing Skylark rules, in user code and in released libraries.

Refined solution