1 of 22

Sh4der Jockey

a tool for realtime visual performances using Rust and OpenGL

1

Speaker: @sp4ghet

Co-Creator: slerpy

2 of 22

Photosensitivity warning

2

3 of 22

Table of Contents

  1. Background
  2. Demo
  3. Technical Details

3

4 of 22

VJ

4

5 of 22

Livecoding / Algorave

  • IRL events: Algorave Tokyo, Sheffield, etc.
  • Online events: Solstice, Equinox 24 hour streams

Lots of tools e.g. ORC∧, Improviz, Quartz Composer, vvvv, etc.

5

6 of 22

6

7 of 22

Demoscene

Small executable computer graphics and weird/oldschool hardware.

Shader battle tool = Bonzomatic

7

8 of 22

8

9 of 22

Shader Showdown/Royale

9

10 of 22

VRChat Club Scene

10

11 of 22

Similar tools

  • Veda: WebGL, bound to Atom editor.
    • WebGL = No Compute Shader
  • KodeLife: OpenGL, costs money
  • Bonzomatic: OpenGL, competitive live coding environment
    • Single pass fragment shader only
  • Shadertoy: WebGL, youtube for shaders
    • > 4 pass, no vertex shader

​

  • Resolume/VDMX: VJ-ing tools centered around video loops.
    • ISF shaders, basically GLSL + JSON header

​

11

12 of 22

Why a New Tool?

  • Having “my tool” is cool
  • Limitations with WebGL (Mainly compute shaders)
  • Most tools lack vertex shader support
  • Minor details of shader uniforms that become annoying
    • MIDI in veda isn’t ideal
    • NDI video input / Screen Capture
    • EQ and re-bucketing of audio FFT
  • Slow compile time = frame drops

12

13 of 22

Table of Contents

  • Background
  • Demo
  • Technical Details

13

14 of 22

Demo

14

https://github.com/slerpyyy/sh4der-jockey

15 of 22

15

16 of 22

Table of Contents

  • Background
  • Demo
  • Technical Details

16

17 of 22

Rust and OpenGL

  • glium: compile time uniform declaration via macros
    • Usually nice, not for us.
  • Use gl crate with raw unsafe bindings

17

18 of 22

imgui-rs

No support for docking beta branch. (yet https://github.com/imgui-rs/imgui-rs/issues/249 & 462)

Adding support for docking branch seemed relatively trivial, so add support for docking.

Broke builds on linux due to winit version related errors, fix was just to bump the winit version used for various libraries.

18

19 of 22

NDI™ (Network Display Interface)

NewTek NDI™ is a proprietary protocol for sending video/audio over LAN.

The OBS-ndi plugin can be used to send arbitrary video data into sh4der jockey.

No good rust crate for NDI was found. Made a new wrapper crate for the NDI 4 SDK.

​

19

20 of 22

Async Pipeline Build

Building a pipeline can take a long time. (~Minutes if loading big textures in debug mode)

Ameliorate the pipeline build time across multiple frames using Async.

​

Building a pipeline is usually quite short (~10ms) especially in release mode, but can easily go > 16 ms.

20

21 of 22

What’s (possibly) next?

  • Hardware instancing
  • Better timeline + audio/video player
  • UI improvements
    • Color picker, curve editor
  • Cubemaps
  • Loading custom models e.g. fbx, obj, etc

​

Suggestions and feedback welcome, nothing is set in stone yet.

21

22 of 22

That’s all folks

https://discord.gg/Xxdxe2Mj64

https://github.com/slerpyyy/sh4der-jockey

22