Starlark

A Lightweight, Deterministic, and Secure Scripting Language

CSE 645 - Spring 2025�Seminar in Languages

By

Siva Sathwik Kommi

References

1. Bazel Build Information�2. GitHub Repository

3. Laurent Le Brun blog

​

Large Code Bases ? Configuration and Scaling?

What is Starlark?

• Starlark, formerly Skylark, is an Embedded Scripting Language developed by Google specifically for Bazel build and test system for managing configurations for their large scale code bases.
• Starlark is syntactically very similar to Python but, in terms of functionality it is vastly different.
• It is used in Bazel build system, which is owned by Google.

​

Motivation

• Requirement for faster reproduction of system configuration in real time scenarios.
• Need for a deterministic scripting language for the Bazel build system.
• Imminent requirement for a scripting language that will support parallelism and handle large code bases.

​

Evolution of Starlark

​

​

• Skylark, was introduced to deal with the scalability aspect of Makefiles for larger projects for Google Bazel.�
• The initial attempt was to use Python Script (BUILD) to reproduce the functionality which was slow and still had the bottleneck being Make.�
• Skylark was introduced to use the BUILD functionality while also safely accommodating macros and extensions that users need by using certain mechanisms…

​

Novelty of Starlark

• Sandboxed Execution: No direct access to system libraries.
• Deterministic Evaluation: Output remains the same for given inputs.
• No Concurrency Issues: Execution is thread-safe.
• Portable: Works across different platforms.

Design, Development and Implementations of Starlark

Design Principles

Deterministic evaluation

Hermetic execution

Simplicity

Parallel Evaluation

Focus on tooling

High Level Abstraction

1

2

3

4

5

6

1. Deterministic Execution

​

• As Starlark is majorly used for system configuration, it is important to obtain identical configurations whenever a system is built.
• Deterministic execution in general means that the order of execution is fixed.
• The thread safe feature of Starlark ensures deterministic execution.

​

2. Hermetic Execution

• The Starlark execution system will not handle Input Outputs, system states or any dependencies that are a part of the system or an external component.
• Executions are local to the module/file that is being executed , all necessary dependencies and external resources are deterministically loaded into the system if mentioned as a part of the code by specific constructs.

3. Parallelism

• Starlark allows parallel execution of the modules.
• In Order to preserve determinism of execution in this parallel execution environment, all variables and function returns are made immutable.
• Parallel execution is enforced by loading multiple blocks/modules at a time.

4. Simplicity

• Skylark is set up to be very simple and syntactically understandable.
• The dialect is very similar to python and it allows dynamic typing when using inbuilt constructs.
• The english like representation allows users to understand the functionality of a configuration script better than its existing alternatives.
• End users, irrespective of their programming knowledge can easily access and understand this language.

5. Focus on Tooling

• Starlark is developed in such a way that it is readable by both human and automated tools.
• Several implementation components are used in Starlark which requires it to be strongly declared and safe for execution.

6. High level Abstraction

​

• Starlark has a high level of abstraction as it has a dialect that is very similar to Python.
• This significantly sizes the learning curve down and promotes ease of use.
• It also allows users to understand the semantics of Starlark better due its python similarity.

Starlark Module System

​

Starlark uses the concept of modularity, where functionalities are implemented as code blocks/files.

• Read and Parse : Read source code and parse it as a syntax tree.
• Load Dependencies : Load statements happens in bounded recursions or parallel.
• Static Checks : Name resolution, validity of identifiers and conflict handling.
• Evaluate Code : Code is executed, variables and functions are defined.
• Freeze Variables : All globals are set as immutable to ensure thread safety and eliminate unexpected behavior.
• Expose Bindings : The result variables, functions are made available to other dependent modules.

Step 1: Read and Parse

What Happens?

• Starlark reads the module and converts it into an Abstract Syntax Tree (AST).

Why?

• Necessary for code analysis and execution.
• Prepares the code for further steps.

​

Abstract Syntax Tree for a Simple Addition Program

Step 2: Load Dependencies

• Using load Statements:
• Similar to import in Python.
• Example:�

​

• How Does It Work?
• Dependencies are loaded in parallel for efficiency.
• Loaded only once and cached for future use.

​

Step 3: Static Checks

​

• What Are Static Checks?
• Verifying that all variables and functions are defined.
• Preventing conflicts between local variables and imported symbols.
• Disallowing reassignment of global variables or function redefinition.
• Benefits:
• Makes code easier to reason about.
• Catches errors early during code analysis.

​

Step 4: Code Evaluation

​

​

• How Does Evaluation Work?
• Code is executed in order after static checks pass.
• load statements only bring symbols into the environment.
• Function definitions (def) are assigned to names.
• Key Point: Order of code matters; calls to functions must be after their definition.

​

Step 5: Freezing Variables

​

• What is Freezing?
• After evaluation, global variables and functions become immutable.
• Why?
• Ensures thread safety.
• Prevents race conditions or unpredictable behavior.
• Example:
• Once _cache is defined, it cannot be modified by any thread.

​

Step 6: Expose Bindings

• The variables, files and results of running functions are accessible by parent/host modules for further executions and access.
• This is highly useful in the modular implementation of Starlark by keeping modules isolated yet using results from other modules for different executions.

​

Starlark - Comparison and Implementations

Starlark vs other Scripting Languages

​

​

Why can’t we just use Python? Why is Starlark needed?

Similarities with Python

As mentioned before, Starlark is very similar to python syntactically.

• Print function
• Function definitions
• Lists, Dictionaries and other Data Types.

​

​

Differences from Python

In terms of functionality, Starlark has some restrictions unlike python including :

• No recursive calls or while loops.
• No classes, objects.
• No access to external libraries or I/O.
• No multithreading functionality
• Parallelizable
• Int type is limited to 32-bit signed integers. Overflows will throw an error.
• Dictionary literals cannot have duplicate keys. For example, this is an error: {"a": 4, "b": 7, "a": 1}.

​

​

Why is thread safety crucial for Starlark ?

1. Organizing and Reusing Code

• The module system helps in structuring code and breaking it down into smaller, manageable pieces, allowing for easier maintenance and reusability.

2. Ensuring Deterministic, Thread-Safe Execution

• The Starlark module system ensures that code runs in a predictable, deterministic manner and that all global variables and functions are immutable after evaluation, ensuring thread safety across executions.

​

Code Evaluation and Different Implementations

Go Implementation of Starlark

• Go is efficient, simple, and has strong concurrency support.
• Allows embedding Starlark in Go applications.
• Provides a safe and deterministic execution environment.
• Import path for the Go package is "go.starlark.net/starlark"

Design Choices for Go Implementation

• Scanner : Token Processing
• Parser : Precedence Climbing to execute code. Routes Errors to Resolver.
• Resolver : Detects structural errors and enforces Starlark Scoping rules.
• Evaluator : Uses Recursive Tree walk approach instead of bytecode execution to ensure consistency with Go.
• Data types : Integers, Float and Dictionary to maintain predictable order of execution.
• Freeing : Immutability of variables to allow safe concurrent execution using “Frozen” flag.
• Fail - Fast Iterators : Iterating over mutable structure temporarily freezes it preventing changes.

Starlark interpreter in Go

Java Implementation of Starlark

• Java implementation of Starlark allows Java Projects to leverage Starlark for configurations using a .jar extension.
• It uses Java inbuilt data types to reproduce functionalities of dictionary and lists.
• It is specific to Bazel and might not operate smoothly with other applications.
• Google does not currently offer a public staple API for accessing Java implementation of bazel.

Design Choices for Java Implementation of Starlark

• Binary Implementation : Set as java_binaries in order to support Java BUILDS using .jar.
• final keyword : Making variables/objects and other constructs immutable.
• Comprehension : The functionality is mimicked in Java to help run functions.
• Similar to Go Implementation, It has a Scanner, Parser, Resolver to handle tokens and resolve structural errors.
• Lexer : To verify various tokens, constructs, keywords, indentation and escape sequences.
• Objects are used to reproduce Starlark functionality.

Rust Implementation of Starlark

• Rust Implementation of Starlark was put forth by Google as a Open Source Framework which is also utilized by Facebook for their build system.
• Provides various functionalities like Procedural Macros and also necessary inbuilt constructs to accommodate various data types and functionalities.
• It is very hands on and also has LSP support to provide auto-complete and other functionalities.

Design Choices for Rust Implementation of Starlark

• Garbage Collection mechanism uses Heap.
• Send/Sync Values are frozen and are used to mimic starlark.
• Linter to detect code issues in Starlark format.
• Multiple library components to support macro development and data structure implementations.
• Binary processing of Starlark code using a separate component.

Common Errors when using Starlark

• Exceeding recursion depth.
• Empty Arguments/ Incomplete arguments.
• Data type inconsistencies.
• Attempting to import dependencies.
• Various implementations may not have functionality to directly reproduce starlark components.

​

​

Major Starlark Users

• Google: Bazel build system.
• Meta: Buck build system.
• Pants Build System: Used for monorepos in Python/Go.
• Tilt: Kubernetes automation.
• CI/CD Pipelines: Used in secure build systems.
• Red Hat: Build automation and CI/CD.

Google Bazel Build System

What is Bazel?

• Bazel is a fast and scalable build system.
• Starlark was initially developed to assist developers to remove the bottleneck caused by make files in the Bazel build system.
• Used by Google, Meta, and other large tech companies.
• Supports multiple languages: C++, Java, Python, Go.

Starlark in Bazel

Rules

• Rules in starlark are declarative functions which is used to design how build targets should be processed.

​

• It allows users to reproduce custom build logic in order to ensure correctness while reproduction.

Macros

• Macros are used to dynamically generate the build rules.

​

• It allows developers to reuse build logic and eliminate redundancy in rule declaration.

​

• Macros do not execute during the build phase , but they are invoked as rules during build evaluation.

Advantages of Using Starlark

• For same inputs at any time, same outputs will be generated which is ideal for configurations and builds.
• Can be embedded in code bases which are composed of other languages.
• Allows sandboxed execution in an isolated manner without external resource access.
• Python like syntax.
• Lightweight and Faster when compared to make files.
• Extensive support for custom macros.
• Safe in terms of threading and resource usage.
• Portable across environments depending on interpreter availability.

Limitations of Starlark

• Does not allow complex recursion logics that might need unbounded looping.
• Various implementations of starlark might not be directly mapped across their functional components leading to indirect/complex implementations.
• No support to any existing standard libraries.
• Concurrency is not native, Starlark is single threaded.
• Restricted control flow which does not allow lazy evaluation of iterators.
• Performance Limitations as it is Interpreted and not Complied which makes the process slow.
• Global states once assigned cannot be changed.
• No support for Object Oriented Programming, dictionaries are used to mimic objects.

Limitations of Starlark

• In sandbox environments, Infinite looping cases can not be examined as Starlark doesn’t support infinite loops.
• No access to network, system variables, clock and files no data can be downloaded during execution.
• There is no try/except sort of exception handling support.
• Very limited methods for string and lists to preserve immutability.

Conclusion

• Even though there are multiple limitations of starlark, the functionality it provides in building and testing large monorepos overpowers them. �
• Starlark is very effective in context of system configuration and reproduction in large scale code systems

Q n A

​

​

Thank You