WebGL, JavaScript and Gaming

Jason Meisel

Sean Middleditch

Subsonic, LLC.

http://sonargame.com/

About Us

Jason Meisel

�

DigiPen RTIS Junior, experience working on two successful student games.

�

Sean Middleditch

�

DigiPen RTIS Junior, one of the developers on Subsonic, around a decade of professional programming experience in the Web space.

�

�

About Subsonic LLC.

• Formed May 2011
• Created by four DigiPen students
• We made SONAR!

Slides & Contact Information

• Slides will be available online at http://sonargame.com/2011/11/01/new-game-slides/

�

�

�

• Contact us
• Sean Middleditch: sean@sonargame.com
• Jason Meisel: jason@sonargame.com

About             SONAR

The Game We Made

SONAR

• Puzzle-action game based on visual soundwaves
• SONAR is 3D with WebGL
• Complete from scratch (no pre-made engine or toolkit)
• 100% JavaScript codebase
• Papercraft art style
• Silly humor

�

     sonargame.com

About SONAR: The Process

• Made in three months
• Team of 3 programmers, 1 designer/musician, 2 part-time artists
• Chrome Web Store distribution

�

Subsonic: The Student Game

• Subsonic was a DigiPen student game
• Originally 2D with OpenGL and lots of GLSL effects
• Indie Game Challenge 2011 Finalist
• Same action-puzzle gameplay
• Same goofy humor

�

     subsonicinc.com�        digipen.edu

Engine Overview

The Technology Behind SONAR

SONAR Engine: Why a New Engine?

• WebGL has few existing engines
• Data-driven game design
• Soundwave physics mechanic

SONAR Engine: Requirements

• Animated character meshes
• Online account synchronization
• Soundwave "physics"
• Data-driven game object composition
• Achievements / trophies / medals
• Sound & music

SONAR Engine: Overview

• Object-oriented design everywhere (using Joose-like API)
• Reuse libraries where possible (SoundManager 2, glMatrix, requireJS)
• JSON for most data files
• Event-driven game object communication

Object-Oriented JavaScript Wrapper

Component-Based Design

Useful Object-Oriented Techniques for Game Developers

Game Object Compositions

• Game object compositions (the One True Way™)
• a.k.a. Aggregation
• Alternative to traditional inheritance-based architecture
• Simple serialization
• Easy addition of�functionality

Evolve Your Hierarchy:

http://is.gd/H2ezYI - Mick West

�

Components in JavaScript

http://is.gd/Beyekn - Sean Middleditch

Inheritance Tree - Inflexible

Components - Awesome

Spaces

Space is the Place

Spaces Overview

• "Spaces" for game object organization, level management

Subspaces

• Spaces are composed of Subspace objects
• GraphicsSubspace, PhysicsSubspace, etc.
• Subspaces store GameObject components however needed
• Essentially another use of component-based design

• Not all Spaces need all Subspaces
• Introduction animation doesn't need physics

• Subspaces control logic rather than Engine components
• Engine updates once per frame
• Spaces may be updated many times or not at all
• Objects in different Spaces cannot collide

Subspaces

Artificial Intelligence

Action-Oriented Thinking

AI Architecture

• Basic tech: action lists, A*, events

�

• Key behaviors
• Guard location
• Patrol
• Chase player
• Search for player
• Investigate noise
• Stun response
• Camera panning
• Alarm response

Action Lists

• Alternative to Finite State Machines

�

• Queue of Action objects to execute
• Actions can be Blocked or Unblocked
• Actions can be Blocking or Non-blocking
• Actions have Lanes (bitmask of who can be blocked by them)
• Actions can persist for many frames
• Actions can create new Actions and wait for completion

�

http://sonargame.com/2011/06/05/action-lists/

Behavior Composition

• No dependencies between Behaviors
• All interaction defined on blocking status and queue order

• Different enemies have different Behavior actions
• Cameras get PanCamera, do not get ChasePlayer
• Tough Guards do not get the StunResponse Behavior

• Data-driven behavior and Components
• ChasePlayer tells movement component to move
• Move component creates MoveTo actions
• Move component knows movement speed
• Behaviors are independent of movement speed or type

JavaScript Math

The Poetry of Illogical Ideas

Maths

• Used glMatrix

         http://code.google.com/p/glmatrix/

Math is Hard!

• Pass-by-reference vectors are "troublesome"
• Read: burn-your-face-off horrifying
• Bugs appear in random places

�

• Easy to fix, just be careful and meticulous

Pass by Reference Example

Math Optimization

• Optimizing low-level math code is difficult
• Effectively can't see assembler/machine-code output

�

• Use Chrome Profiler
• It's awesome

�

• Test code changes thoroughly
• Some "optimizations" may not work out; don't assume!

Garbage Collection

• vec3.create() creates a new object
• Adds GC pressure
• Avoid creating temporaries

Math is Hard!

• Weak typing gotcha's
• null, false, "", {}, etc. implicitly convert to numbers
• Passing the wrong type (or nothing) to a function won't error, it will just break in horrible ways
• + both means "add" and "concatenate"
• 1 + 1 = 2
• 1 + "1" = "11"

�

• Add manual type-checking around file I/O
• vec3.create([1, 2, "3"])

WebGL

The Pretty Part of Programming Professionally without Proprietary Plug-ins

WebGL: Why Use It?

• No proprietary plugins or APIs
• Cross-browser (not IE)
• Full hardware acceleration

• Subsonic was 2D, we wanted SONAR to be 3D
• Animated characters
• 3D environments
• Less abstract visuals

copyright (C) 2011 Learning WebGL

WebGL: Pros

• Very easy to use (in a low-level way)
• LearningWebGL.com is an amazing tutorial site
• No fixed function -> Proper tutorials

�

• WebGL Inspector by Ben Vanik is great for debugging
• Rivals PIX for DirectX, gDEBugger for OpenGL

          http://benvanik.github.com/WebGL-Inspector/

�

• Based on OpenGL ES 2.0
• Matches DirectX 9 in capabilities
• Lost device is basically handled for you
• No GL extension hell

WebGL: Cons

• No DirectX 10+ or OpenGL 3+ features
• Geometry shaders
• Tesselation
• Instancing
• etc.
• Vague error output (INVALID_OPERATION)
• WebGLDebugUtils helps

          http://www.khronos.org/webgl/wiki/Debugging

• Non-power-of-two textures very limited, hard to debug
• Bad support for compact/efficient vertex buffer updates

JavaScript Game Optimizations

Sore Thumbs on the Web

JavaScript: Web Scripting

• Created by Brendan Eich for Netscape circa 1995
• Dynamic, prototypes, C-like syntax, functional
• Designed for client-side GUI functionality

• Not designed for real-time applications
• V8, SpiderMonkey, Nitro, Chakra -- amazing results despite the language, not because of it

• Only standard runtime/language all browsers support
• Nice high-level features

JavaScript: Game Programming

• We need very fast SIMD math
• Graphics
• Physics
• Game logic (ray casting)

• Real-time application
• Hard ~16ms budget per frame
• Garbage collector is killer 
• Have to avoid all memory allocation during gameplay
• Dynamic VBO updates are inefficient in JavaScript
• Offload as much as possible to the GPU

JavaScript: Garbage Collection Woes

• Avoid garbage collection at all costs
• No memory allocation == no collection

• Avoiding allocations can be tricky
• new is easy to spot and avoid
• Object and Array literals a bit harder: {}, [], ""
• Closures: var x; function() { return x; }
• DOM element modification
• Allocations hidden behind library APIs
• All SIMD/vector types!  vec3.create() // invokes []

�

• Our JavaScript optimization article

http://is.gd/3tS9gn

JavaScript: Garbage Collection Hacks

• Just don't use fancy JavaScript features where possible

• Pre-allocate objects
• Keep a pool of objects for recycling

• Chrome debugger profiling

�

• Allocate "private globals" for temporary scratch space

garbage collection does this

Private Globals

Art Pipeline

Blending Content Creation and Game Engines

Art Pipeline: From Blender to WebGL

• All 3D assets created in Blender
• Textures and image assets created in Photoshop

�

• Pipeline overview
• Model created in Blender
• Exported using custom Python script
• Rolled into TAR package by shell script
• Downloaded with AJAX into FileSystem storage
• Loaded by ChainLoader
• Decoded by ResourceDecoder into Image or VertexBuffer

�

http://sonargame.com/2011/07/23/resource-loading-in-javascript/

Blender

• Pros
• Completely free
• Has some nice animation features
• Extensive Python API
• Extremely leaner in comparison to 3DS/Maya

�

• Cons
• Newer versions lack documentation
• Few professional artists are trained to use Blender
• Training costs can be higher than 3DS Max licenses!

• Blender is a free and Open Source 3D modelling program

     http://blender.org/

copyright (C) 2011 Blender Foundation

3D Model Format

• Custom formats: Binary and JSON
• Binary is hard to work with in Python, JavaScript
• dynamic languages
• Javascript barely has ability to read binary, buggy

�

�

����

• JSON files when compressed are not much larger
• Bottom line: Binary file formats were not worth it

3D Models in JavaScript/WebGL

• Meshes can be loaded into vertex buffer objects (VBOs)

�

• Animated meshes
• Additional information for visualization / debugging
• Animation JSON files describing baked frames of animation
• No CPU side interpolation
• GPU skinning
• JavaScript is way too slow (no SIMD) for CPU skinning
• Removes need for dynamic/streaming VBO updates

Pipeline Integration Woes

• JavaScript is not integration-friendly
• Difficult to work with binary files
• Poor support for loading arbitrary files from disk
• How do artists get models into engine to preview?
• Impossible to monitor files for changes
• Manual steps to re-import files on every change
• Saving files is (almost) impossible
• How do designers save levels into SCM-friendly folder?

�

• Blender exporters vs converters
• We wrote a custom Blender format exporter in Python
• Should have written an FBX converter instead

Autodesk FBX

• "Universal" mesh/animation/material� format
• owned by Autodesk
• Intermediatary format
• Can be exported from 3D Studio Max, Maya, Blender, etc. without custom scripts
• C++ SDK can be used to convert to custom format
• Not suitable for loading in engine
• Very large, most data is extraneous
• No Javascript SDK, hard to extract needed data

• COLLADA is also an option

HTML5

Modern Web for Gaming

HTML5, CSS & Canvas for 2D

• 2D is pervasive even in 3D games
• Menus, HUD, debugging, editor
• HTML5 & CSS makes things super easy
• Don't forget jQuery, Prototype.js, Mootools, etc.
• Canvas is great for 2D rendering
• HUD elements, animated timers, etc.
• Advanced CSS can also do amazing things
• CSS Transformations
• CSS Animations
• CSS can even do 3D!

• Adobe Edge could be used (or any other HTML designer)

HTML5, Canvas, CSS

HTML5, CSS & Canvas for 2D

• Rendering text in (Web)GL is difficult
• HTML5 overlays make it so easy
• Compositing
• Create a WebGL canvas with a low z-index for game
• Create absolutely-positioned <div> element overlays
• Be wary of alpha on your WebGL canvas
•  canvas.getContext('webgl', { alpha : false });

�

• Performance is a problem!
• 2D compositing is not hardware accelerated everywhere
• Even just a few fancy animation overlays can kill FPS
• Keep overlays stationary and simple when possible

Editor

Designing for Success

See Our Pretty Editor

Features

• Tile-based level editor
• Line-based doors and glass drawing
• Trigger zones
• Entity placement and deletion
• Property editing of entities
• Save & load from native file system
• Start and end scripts

�

Implementation

• Canvas for tile rendering
• jQuery for UI controls and support

�

• Was a separate app from the game
• Started on editor same day we started on engine
• Don't do this, make level testing iteration too slow

�

• Scripting
• Horrible hacked-together script engine for triggers
• Using JavaScript was actually not an option
• No sandboxing support
• No coroutines/yield support

Alternative Editor Ideas

• In-game editor
• Quick iteration of level changes
• Same asset pipeline problems

• Existing native tools
• Less development overhead
• Poor support for engine's specialized features

�

• Custom native editor
• Allow pipeline integration (OS feature support)
• Embed Chromium?
• Fast iteration for changes like in-game editor
• Highest development cost
• Probably the ideal solution

Miscellaneous Web APIs

It's the Little Things

Miscellaneous Web APIs for Games

• WebAudio
• Chorus effects, 3D positioning, no Flash
• https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html
• FileSystem
• Store files locally, no cookies, no lost data
• http://www.html5rocks.com/en/tutorials/file/filesystem/
• MouseLock API
• Mandatory for FPS, useful for action games
• http://chromestory.com/2011/10/developer-news-mouse-lock-api/
• Joystick API
• Gamepad support
• http://www.chromium.org/developers/joystick-api

Summary

1. WebGL is easy and awesome
2. JavaScript can be used to make game engines
3. Optimization is tricky but manageable
4. Content pipeline will make or break you
5. Lots of new technologies you should investigate

Questions?

Answers?