Using the Loader Hook API for `require()`

Bradley Farias

What is the Loader Hook API

• It intercepts how `import` loads Modules.
• Written to allow loading all kinds of files
• WASM / JSON / CJS / ESM / ...

How could we send `require` requests to it?

• `require` is synchronous, Loader API is asynchronous
• `require` has a different resolution algorithm than `import`
• `require.extensions` and friends will keep being backwards compat constraints

Atomics

Atomics lets other threads *act* synchronous

• Pause main thread using Atomics.wait
• Use SharedArrayBuffer & .postMessage to send shared memory to other threads
• Have threads notify main thread when the shared memory contains the result

Example Synchronous RPC using Atomics

• https://gist.github.com/bmeck/7b43e9dffa114f658563a0e9e0f694ce
• Node can do this under the hood so loader hooks never see it

`import` Workflow

import('fs')

Loaders

Default ESM resolver

.postMessage

Require uses a different resolver than import

• Loaders just call out to parents / the default resolver, they don't know what resolver is their parent
• We can use different "top" Loaders for `import` and `require`
• Loaders in between the main thread and "top" Loader can remain the same

`require` Workflow

require('fs')

Loaders

Default CJS resolver

BUT THATS ON THE MAIN THREAD!?

Require default hooks are on the main thread

• We can call out to the main thread from the resolver thread
• Same way we call out to Loader hooks and act synchronously

`require` Workflow

require('fs')

Loaders

Default CJS resolver

require.resolve

Require has a different cache

• Loader hooks and cache are complex, we should keep working on this for ESM but it might have to be complex due to general mismatch with Web cache workflow.
• I think this is solvable.
• I think it is harder if we expose a way for ESM to match the Web cache workflow rather than enforce it to be how Node has always done things.