1 of 30

🔊 Integrating p5.js with RNBO for Web-Based Audiovisual Experiences

🧑‍💻️ Andrei Antonescu / superblob.studio

Github Repo

2 of 30

Here’s what we’re going to build today - “Mouse Theremin”

3 of 30

Workshop Plan

Overview (10min)
Intro to Max (10 min)
Intro to p5.js and web dev (20 min)
RNBO - converting Max patch and exporting (20 min)
Building Mouse Theremin (30 min)
Using multiple RNBO devices and effects (10 min)
Other RNBO examples (10 min)
Q&A / Open Discussion (10 min)

4 of 30

Overview

Graphical patching environment
DSP / Audio Synthesis/ Video / 3D
Max for Live / MIDI / OSC
“Connect everything”

5 of 30

Overview

Add-on patching environment in Max
Export patches as portable code
Integrate into Web Audio or C++ projects/VST/Raspberry Pi/etc.

Based on the core principles of Processing
Javascript library
Wrapper over Canvas and Web Audio

6 of 30

Intro to Max

Create object (N)

Lock/Unlock (Cmd-E)

Signal vs. Value

Debug / Bangs

Help menu

7 of 30

Simple example to get us started with

Two base frequencies and multiple sets of sine oscillators

Overtones based on frequency divisions

Download (1-Max.maxpat)

Intro to Max

8 of 30

Intro to p5.js

Base website structure (2-starterWeb)

Serving on local server

Setup & Draw

p5.js docs

Connecting to HTML

Mouse Moved

9 of 30

Intro to p5.js - index.html

<!DOCTYPE html>

<head>

</head>

<body>

<main>

<button onClick="doSomething()">Click Me!</button>

</main>

</body>

</html>

Add p5.js library to index.html

Button that calls doSomething() function

Our javascript sketch

10 of 30

Intro to p5.js - sketch.js

function setup() {

createCanvas(windowWidth, windowHeight)

background(200)

fill('green')

ellipse(200, 200, 250)

}

function draw() {

ellipse(random(windowWidth), random(windowHeight), 50)

}

function mouseMoved() {

console.log(mouseX, mouseY)

}

function doSomething() {

console.log("I was clicked")

}

Called once

Called every frame (60fps)

Called when mouse is moved

Our function for interacting with the HTML button

11 of 30

RNBO

Installing RNBO

Creating RNBO patch

Porting from Max

Params and Outputs

Exporting to web

Download (3-RNBO-conv)

12 of 30

RNBO - converting Max patch

13 of 30

RNBO - Params

Params (IN)

Outputs (OUT)

14 of 30

RNBO - Exporting

Web Export

Dependencies needed if using samples

15 of 30

Integrating RNBO and p5.js

Export structure

Adding RNBO to the web

Signal Flow/Devices

Starting Audio on the web

Get/Set Params

Combining with graphics

Download (4-RNBO-Web)

Exported params from patcher

16 of 30

Integrating - Adding RNBO to the web

<!DOCTYPE html>

<head>

</head>

<body>

<main>

<button onClick="doSomething()">Click Me!</button>

</main>

</body>

</html>

Add RNBO library to index.html

17 of 30

Integrating - Devices & Signal Flow

async function setupRNBO() {

const WAContext = window.AudioContext || window.webkitAudioContext

context = new WAContext()

const outputNode = context.createGain()

outputNode.connect(context.destination)

let response = await fetch("export/sketch.export.json")

const myPatcher = await response.json()

const myDevice = await RNBO.createDevice({ context, patcher: myPatcher })

// Connect the devices in series

myDevice.node.connect(outputNode)

Continues…

Get audio context and output

Get patcher and connect to output

18 of 30

Integrating - Devices & Signal Flow

…Continued

// get parameters

freqOne = myDevice.parametersById.get("freqOne")

freqOne.value = 350.0

freqTwo = myDevice.parametersById.get("freqTwo")

freqTwo.value = 500.0

gain = myDevice.parametersById.get("gain")

gain.value = 85.0

context.suspend()

}

Get and set params from patcher

Audio context needs to be resumed after user interaction

19 of 30

Integrating - Starting Audio on the web

function setup() {

createCanvas(innerWidth, innerHeight)

background(200)

setupRNBO()

}

function doSomething() {

context.resume()

console.log("Start Audio")

}

Setup RNBO during setup()

Resume audio context after user interaction

20 of 30

Integrating - Playing with params and graphics

21 of 30

Multiple Devices / Effects

Exporting RNBO effects

Using multiple devices

Download (5-RNBO-Web-Multi)

Download RNBO Guitar Pedals pack

22 of 30

Multiple Devices / Effects

Launch and select an effect, like the ShimmeRev

23 of 30

Select RNBO patch from example and export

24 of 30

Multiple Devices - Creating New & Add to Signal Flow

async function setupRNBO() {

const WAContext = window.AudioContext || window.webkitAudioContext

context = new WAContext()

const outputNode = context.createGain()

outputNode.connect(context.destination)

let response = await fetch("export/Example.export.json")

const myPatcher = await response.json()

const myDevice = await RNBO.createDevice({ context, patcher: myPatcher })

let responseTwo = await fetch("export/Reverb/rnbo.shimmerev.json")

const myPatcherTwo = await responseTwo.json()

const myDeviceTwo = await RNBO.createDevice({ context, patcher: myPatcherTwo })

// Change the signal flow

myDevice.node.connect(myDeviceTwo.node)

myDeviceTwo.node.connect(outputNode)

Continues...

25 of 30

Final touches

Line~

Filter

Download (6-final)

26 of 30

Extra - MIDI/Polyphony/Dependencies

MIDI

Polyphony

Dependencies

Download

27 of 30

Extra - MIDI/Polyphony/Dependencies

28 of 30

Extra - Connecting to p5.Sound / other Web Audio contexts

Instead of creating the audio context, you can grab it from another place/library (Tone JS / p5.Sound / etc.)

For example, p5.js has a function for getting the audio context

context = getAudioContext() // get p5 audio context

See an example here (towards the end).

29 of 30

Resources

30 of 30

Thank you!

🧑‍💻️ Andrei Antonescu / superblob.studio