Author: yangguo@chromium.org, @hashseed
Last updated: 2017/11/18
Related Node.js issue: https://github.com/nodejs/node/issues/17058
TL;DR: Let’s make Node.js start up 8x faster.
I disabled commenting because of the notifications I get from people suggesting whitespace changes. Please comment on the Node.js issue.
A big part of Node.js core is implemented in JavaScript. During startup, Node.js creates a v8::Isolate, then a v8::Context, then a node::Environment. It then proceeds to create a process object, among others, and runs bootstrap_node.js to set up the environment. Only after performing all of this, Node.js executes user script.
All of this takes its toll on the startup performance. While a not entirely fair comparison, d8 (V8’s development shell) takes around 50ms to perform d8 -e "" while Node.js takes 200ms to perform node -e ""(on a high-end workstation).
V8’s startup snapshot is a powerful way to speed up creating a new V8 isolate and new V8 contexts. It consists of two parts: the isolate snapshot, and the context snapshot. More information can be found on related blog posts.
In summary, instead of setting up the isolate from scratch, V8 simply deserializes the object graph from a previously serialized isolate. The same applies to the context snapshot. This way, V8 is able to significantly speed up starting up (up to two orders of magnitude).
Node.js benefits from this when creating a new isolate and a new context. However, the subsequent steps during its startup are not part of the snapshot and therefore cause noticeable overhead. If we could create a snapshot from after Node.js has fully started up, but before executing any user script, we could reduce startup time. Basing off the numbers above, up to 4x.
There is precedence of this being implemented in blink, where DOM wrappers are included in the snapshot. Presentation. Tracking issue.
There are a few things that need to be solved to make this work.
V8’s way to create a snapshot is to use the v8::SnapshotCreator API. After creating a SnapshotCreator instance, it provides an isolate that is prepared for serialization. We can then create arbitrary contexts and modify them, and add each of those contexts to the SnapshotCreator instance, before requesting for a serialized blob.
Ideally, we would first perform bootstrapping, and then create the snapshot right before user script would execute. This way, we could bypass bootstrapping when using the resulting snapshot. However, running user script is nested inside of bootstrapping.
This needs to be refactored into a separate, not nested, step. Specifically, parts that depend on runtime input, including command line arguments, need to happen after snapshot.
As previously described, the isolate used to create a snapshot is created differently than one used for normal execution. We would therefore need a code path that performs the same bootstrapping steps as normal execution up to the point where we create the snapshot.
Conversely, when using such a snapshot, we would need a code path that bypasses bootstrapping, under the assumption that creating a new isolate and new context already restores the bootstrapped state from snapshot.
We probably also want to keep the current code path that works without custom startup snapshot.
We could make this distinction by providing the node executable with a new flag to produce snapshots. When consuming the snapshot, we could distinguish based on the number of available context snapshots.
Snapshots that contain two context snapshots have a default vanilla context (there is some more backstory to this, but unimportant for this doc) and the second being the custom bootstrapped context that we use. Snapshots that only contain one context snapshot only contains the default vanilla context that still run the old code path for bootstrapping.
Once this works, we should incorporate creating a bootstrapped snapshot into the build configuration.
Bootstrapping involves loading a number of core modules that have native bindings. V8 cannot serialize these bindings out of the box, but the SnapshotCreator provides an API to register the addresses of the respective C++ function addresses to V8’s serializer. These addresses need to be provided to V8 again, in the same order, for deserialization.
This could be achieved by adding C++ function addresses used in every native module when registering these modules. This list of function addresses is then be passed to V8.
V8’s serializer cannot deal with handles. Even if it knew how to serialize them, there would be no straight-forward way to deserialize them correctly. Therefore it simply asserts that there are no open handles, both scoped ones (v8::Local) and persistent ones (v8::Persistent). However, node::Environment has a set of persistent handles of certain objects, and node::IsolateData has a set of persistent handles of symbols and strings.
For node::Environment, we could move these persistent handles onto the global object before serialization and move them back after deserialization, for example storing them as global properties with symbol keys. These symbol keys would have to be obtained via v8::Symbol::ForApi to make sure we get the same ones before and after serialization/deserialization. For node::IsolateData, we could simply wipe these handles and create these symbols and strings again, after deserialization (again via v8::Symbol::ForApi for symbols).
Currently, node::Environment is created after its corresponding context. This means that at the time the context is deserialized, the environment does not exist yet. But since the context contains a reference to the environment wrapped in a v8::External, this reference cannot be serialized.
We should change the order: first create the environment, then the context, and have an additional init step to set up the cross-references between the two. This way, at the time the context is deserialized, we can already provide the pointer to the environment as native reference.
V8’s snapshot by default does not contain any compiled bytecode to reduce initial memory footprint, and usually makes sense.
To actually run user script after bootstrapping, some more JavaScript code in Node.js core need to be executed, leading up to it. It would make sense to include the compiled bytecode into the snapshot to avoid inevitably having to parse and compile. There is a flag in SnapshotCreator for this, and convenience wrappers in V8’s API that demonstrate how this could be done.
Is the event loop implemented fully in JavaScript? Is it already started during bootstrapping? We may need a way to reinitialize the event loop correctly after deserialization from bootstrapped context. The event loop is not implemented in JavaScript. We’ll just have to make sure it is in a consistent state after deserialization.
Native modules may persist state. Each native module involved in startup therefore needs to be audited to make sure we can restore a consistent state. Some of these native modules interact with V8 for example to install GC callbacks. These callbacks need to be installed after deserialization. We probably should simply eagerly load built-in modules and split initialization into parts that change V8 heap and parts that do not.
Embedder fields may need special attention. If these embedder fields actually reference heap-allocated C++ objects, we may need a way to correctly persist these objects. The V8 API for this exists with embedder field serialization callbacks, but may need to be implemented appropriately.