1 of 58

Developing games in Go�for Nintendo Switch™

(English ver.)

Go Conference 2021 Autumn (2021-11-13)

Hajime Hoshi

Nintendo Switch is a registered trademark of Nintendo Co., Ltd.

2 of 58

Caution

  • This presentation is based only on open information
  • Hajime Hoshi takes full responsibility for the wording and content of this presentation
  • Please do not ask Nintendo about this presentation
  • (Japanese version)

3 of 58

Agenda

  • Self-introduction
  • About Ebiten
  • What I wanted to do
  • What I did
  • Result
  • Summary

4 of 58

Self-introduction

  • 星一 (Hajime Hoshi, @hajimehoshi)
  • Develops a game engine in Go as a hobby
  • github.com/hajimehoshi
    • Ebiten (A game engine)
    • Oto (An audio playback library)
    • WasmServe (An simple server for Go/Wasm)
    • Asobiba (Go compiler on browsers)
    • go2cpp (A converter from Go to C++)

5 of 58

About Ebiten

6 of 58

Ebiten

7 of 58

Ebiten features

  • Dead simple API
    • Basically rendering an image�onto an image
  • High portability
    • Desktops (Windows, macOS, Linux, FreeBSD)
    • Mobiles (Android, iOS)
    • Browsers (WebAssembly)
    • Consoles (Nintendo Switch)

Go Gopher by Renee French is licensed under the Creative Commons Attribution 3.0 License

8 of 58

Ebiten for Nintendo Switch demo

9 of 58

Bear's Restaurant

  • Odencat Inc.
  • Released in 2018
  • Google Play Indie Festival 2019�TOP 3
  • 1M DLs all over the world
    • The next game "Fishing Paradiso"�has been downloaded 1.3M times
  • Released in various platforms
    • Android, iOS
    • Steam (Win、Mac、Linux)
    • Switch

Copyright 2021 Odencat Inc.

10 of 58

Others (Odencat)

  • Snowman Story
    • Released in 2019
    • Google Indie Game Festival 2020�TOP 10
    • Android, iOS
  • Mousebusters
    • Released in 2020
    • Google Indie Game Festival 2021�TOP 3
    • Android, iOS

Copyright 2021 Odencat Inc.

11 of 58

Ebiten for Nintendo Switch

  • Bear's Restaurant�(Released in June, 2021)
  • Fishing Paradiso�(Will be released in the beginning�of 2022)

  • Can anyone develop games with Ebiten for Nintendo Switch?
    • Yes and no
    • First, you have to get permission to develop Nintendo Switch apps
      • Please send an application at Nintendo Developer Portal

Copyright 2021 Odencat Inc.

12 of 58

What I wanted to do

13 of 58

What I wanted to do

  • In the middle of 2020
    • Daigo "Wouldn't it be nice to make Bear's Restaurant work on�Switch?"
    • Hajime "Well, let's do it."
  • The more platforms Ebiten supports, the happier I am.
  • But how?

We want to make "Bear's Restaurant"�work on Nintendo Switch

14 of 58

Bear's Restaurant

  • Made with Ebiten
  • Pure Go
  • It already has mobile versions
    • It also has desktop and browser versions for�developments

  • Modifying the game for Switch is another topic
    • Supporting landscape mode, gamepads, etc.
  • Supporting Steam is yet another topic

Copyright 2021 Odencat Inc.

15 of 58

Limitations of Go

  • Big runtime
    • Go can compile programs only for specific machines and archs
      • If you want to support new environment, you will have to rewrite the runtime for it
    • The runtime calls system calls directly
      • You cannot create an OS-independent binary which doesn't have any syscalls
      • You can create a C library with c-shared, but it still has syscalls
  • GC (Garbage Collection)

16 of 58

Limitations of Nintendo Switch

  • Everything is non-disclosure by NDA
    • The OS is secret
  • An original environment
  • An original compiler

  • This presentation is based only on open information

Copyright いらすとや

"Secret"

17 of 58

What should I do?

A game

Ebiten

Go runtime

System calls

C functions

Nintendo Switch's original API

Fill this gap

?

18 of 58

Go and system calls

  • What is a system call?
    • A way to invoke an OS kernel feature
    • Differs depending on OSes
      • The runtime chooses syscalls based on OS (GOOS)
  • The Go runtime invokes syscalls via assembly
    • A Go's binary doesn't depend on other libraries (there are some exceptions)
  • JavaScript functions are used in browsers' cases
    • Wasm's "import functions" to be exact

19 of 58

Limitations by the engineer

  • Hajime Hoshi has to develop it alone as a hobby
    • It's not a job!

  • I want to avoid issues
  • Even if issues are found, I want to reduce the cost to fix them
  • I want to reduce maintenance cost for the future
  • I want to make my product OSS (if possible)

Copyright いらすとや

20 of 58

Summary

  • "We want to make Bear's�Restaurant work on switch"
  • Issues
    • The Go runtime is big
    • Switch's environment is non-�disclosure
    • Only one engineer

A game

Ebiten

Go Runtime

Syscalls

C

Nintendo Switch API

Fill this gap

?

21 of 58

What I did

22 of 58

What I did

  • Choosing an approach
  • C++ converter: go2cpp
  • How go2cpp works

23 of 58

Choosing an approach

A Go program

A modified Go compiler

The regular Go compiler

C++

A binary

An original Go compiler

A regular Go binary

Some transforms

1

2

3

4

5

6

24 of 58

1. Go → (An original compiler) → A binary

  • e.g. TinyGo
  • Pros
    • Good performance
  • Cons
    • Quite hard to develop a compiler from�scratch
    • Low portability
  • TinyGo cannot work with Ebiten yet

A Go program

A binary

An original Go compiler

25 of 58

2. Go → (An original compiler) → C++

  • e.g. Gomoku、GopherJS (not C++ but�JavaScript)
  • Pros
    • High portability
  • Cons
    • Quite hard to develop a compiler from�scratch
  • From my experience of maintaining GopherJS:
    • It was hard to follow Go's updates
    • It was hard to fix all the bugs

A Go program

C++

An original Go compiler

26 of 58

3. Go → (A modified Go compiler) → A binary

  • Customizing the Go compiler's backend
  • e.g. TamaGo, Gollvm, Biscuit (OS)
  • Pros
    • The best performance
  • Cons
    • Quite hard to modify the compiler and�the runtime
    • Big maintenance cost
    • Low portability
  • Probably this is the most "straight" way
  • Actually I tried this but gave up. I'd like to try this again if I have time.

A Go program

A binary

A modified Go compiler

27 of 58

4. Go → (A modified Go compiler) → C++

  • Customizing the Go compiler's backend
  • No actual examples (maybe)
  • Pros
    • High portability
  • Cons
    • Quite hard to modify the compiler and�the runtime
    • Big maintenance cost

A Go program

C++

A modified Go compiler

28 of 58

5. Go→(A Go compiler)→A Go binary→(Trans.)→A binary

  • e.g., converting a binary built with�GOOS=linux GOARCH=arm64
  • Pros
    • Good performance
  • Cons
    • This is actually the same as modifying�the runtime (it depends on the immediate�format)
      • Replacing system calls
    • Low portability

A Go program

A regular Go binary

The regular Go compiler

A binary

Some transforms

29 of 58

6. Go→(A Go compiler)→A Go binary→(Trans.)→C++

  • e.g., converting a binary built with�GOOS=js GOARCH=wasm
  • Pros
    • Low maintenance cost�(it depends on the immediate format)
    • High portability
  • Cons
    • Low performance
  • To come to the point, I adopted this
    • Adopted Wasm as the immediate format

A Go program

A regular Go binary

The regular Go compiler

C++

Some transforms

30 of 58

The chosen approach: go2cpp

A Go program

A modified Go compiler

The regular Go compiler

C++

A binary

An original Go compiler

A Wasm binary

go2cpp

1

2

3

4

5

6

31 of 58

go2cpp

  • github.com/hajimehoshi/go2cpp
  • The Go compiler converts a Go program to a Wasm
  • go2cpp converts a Wasm to C++
  • Pros
    • Very high portability
    • Easy to use for users
  • Cons
    • Bad performance (x3-6 worse than native)
    • Single thread
    • A little longer compile time

32 of 58

Wasm (WebAssembly)

  • A binary format that works on browsers
  • The Go compiler officially supports this
    • GOOS=js GOARCH=wasm
  • syscall/js is used as system calls
    • You can call any JavaScript functions
  • Go runtime is included
    • Go's GC is used
    • A binary size is pretty big

The WebAssembly logo is licensed under CC0 1.0 Universal

33 of 58

Why was Wasm adopted as the intermediate format?

  • 1. The specification is compact
    • There are a lot of parsers in Go
    • Porting about 170 opecodes and about 20 import functions
    • In other formats, I'd have to understand the binary format,�the opecodes, and the system calls exactly
  • 2. It is easy to create bindings
    • How to communicate with Nintendo Switch (C++) world
    • You can use syscall/js for Wasm
    • In other formats, binding is not obvious

34 of 58

Comparing GOOS=js GOARCH=wasm and go2cpp

A game�Ebiten�The Go runtime�syscall/js�(Wasm)

wasm_exec.js

import functions (JavaScript)

Web API

C++ for wasm_exec.js

import functions (C++)

Nintendo Switch's API

A game�Ebiten�The Go runtime�syscall/js�(C++ converted from Wasm)

Nintendo Switch driver

Converted by go2cpp

Secret

Generated by go2cpp

35 of 58

go2cpp

  • The Go program works as if it is�on a browser
  • The subsidiary C++ imitates�JavaScript behaviors
    • syscall/js is available
    • e.g., All the numbers are treated�as double

C++ for wasm_exec.js

import functions (C++)

Nintendo Switch's API

A game�Ebiten�The Go runtime�syscall/js�(C++ converted from Wasm)

Nintendo Switch driver

I'm now working on�browsers...

36 of 58

Wasm → C++

  • Maps Wasm's opecodes to C++

local.get 1

global.get 2

i32.wrap_i64

i32.load offset=16

i32.le_u

if ;; label = @55

local.get 1

i32.const 8

i32.sub

local.tee 1

global.set 0

local.get 1

i64.const 357761024

i64.store

i32.const 0

call $runtime.morestack_noctxt

global.get 0

local.set 1

br_if 54 (;@1;)

end

if (static_cast<uint32_t>(local1_) <= static_cast<uint32_t>(mem_->LoadInt32((static_cast<int32_t>(global2_)) + 16))) {

i32_0_ = (static_cast<int32_t>(static_cast<uint32_t>(local1_) - static_cast<uint32_t>(8)));

local1_ = i32_0_;

global0_ = i32_0_;

mem_->StoreInt64((local1_), 357761024LL);

i32_1_ = runtime_2emorestack_5fnoctxt((0));

local1_ = global0_;

if (i32_1_) {

return 1;

}

}

Replace the stack machine expressions with C++'s expressions

37 of 58

Modifying wasm_exec.js

  • wasm_exec.js is a bridge between Wasm and JavaScript
    • See misc/wasm in the Go's repository
  • Porting this to C++
    • Basically this imitates JavaScript behaviors
    • Defining Value class that corresponds with JavaScript values
      • JavaScript's variable doesn't have a type and can take any values
      • Numbers (double), strings, dictionaries, etc.
    • Porting import functions to C++ (getRandomData, valueGet, etc.)
    • Using std::future for setTimeout
      • A task queue is used in the main thread

38 of 58

Imitating a part of Web API

  • Some Web APIs are available as they are
    • navigator.language: Getting the default language
    • localStorage: Saving and loading
    • navigator.getGamepads: Getting the gamepad states
      • Users don't have to care as Ebiten uses this
    • WebGL2RenderingContext
      • The way to get a context is original
      • Only necessary features for Ebiten were implemented
      • Users don't have to care as Ebiten uses this

39 of 58

Binding (communicating between C++ and Go)

class Binding : public

go2cpp_autogen::Game::Binding {

std::vector<uint8_t> Get(const std::string &key)� override {

if (key == "version") {

std::string version_str = "1.0.0";

return std::vector<uint8_t>(� version_str.begin(), version_str.end());

}

return {};

}

};

extern "C" void main() {

go2cpp_autogen::Game game(� std::make_unique<NintendoSwitchDriver>(),

std::make_unique<Binding>());

game.Run({"your_game"});

}

func version() string {

// syscall/js is available

go2cpp := js.Global().Get("go2cpp")

if !go2cpp.Truthy() {

return ""

}

// binding's Get always returns Uint8Array

ver :=� go2cpp.Get("binding").Get("version")

bs := make([]byte, � ver.Get("byteLength").Int())

js.CopyBytesToGo(bs, ver)

// "1.0.0"

return string(bs)

}

syscall/js is available

40 of 58

Why go2cpp?

  • This is not a "straight" way
  • 1. A general judgement
  • 2. Making the product open

Go Gopher by Renee French is licensed under the Creative Commons Attribution 3.0 License

41 of 58

Why go2cpp? 1. A general judgement

  • Compromising for various restrictions
    • I cannot depend on an unreliable way (even though this is my hobby!)
    • I cannot take long time
    • Probably only I have to maintain this
    • I'm not so familiar with low-layers
  • Done is better than perfect
    • Let's rethink after I make things work
    • If I have time, I want to try another way

42 of 58

Various aspects in software development

  • Performance
  • Compile time
  • Uncertainty (Can unexpect issues happen?)
  • Maintenance cost (Is it hard to follow environment changes?)
  • Portability (Does it work on other platforms than Switch?)
  • Ease of use (Can other people use it easily?)

43 of 58

In the case of go2cpp

  • Performance → Not good but it works
  • Compile time → Long but it's tolerable
  • Uncertainty → Low (My plan should work as I planned)
  • Maintenance cost → Low (Wasm's specification is very stable)
  • Portability → Very high (Only the driver part is dependent)
  • Ease of use → Easy (No other tools are required than go2cpp)

44 of 58

Why go2cpp? 2. Making the product open

  • Generated C++ basically depends only on the standard library
  • The native binary's specification is secret
    • Even if I could create a tool generating a binary, it is impossible to make it public
  • In the case of go2cpp
    • C++ generator can be public
    • Only Nintendo Switch driver part cannot be public
    • I made a public GLFW driver for desktops

45 of 58

Summary

  • I developed go2cpp
  • The Go compiler converts a Go�program to a Wasm
  • go2cpp converts a Wasm to C++
  • Go program works as if it is on a�browser
  • C++ imitates JavaScript behaviors
  • I considered various approaches and�adopted go2cpp from the general�judgement

C++ for wasm_exec.js

import functions (C++)

Nintendo Switch's API

A game�Ebiten�The Go runtime�syscall/js�(C++ converted from Wasm)

Nintendo Switch driver

46 of 58

Result

47 of 58

Result

  • Development time
  • Performance
  • GC issue
  • Future works

48 of 58

Development time

  • A half year to one year, in my hobby time
    • Mid. of 2020: Developed Go → C#, but this didn't work on Switch
      • Planned to make games work on Unity as they are
      • This should support various consoles automatically!
    • End of 2020: Developed Go → C++, succeeded to work on Switch
    • Beginning of 2021: Succeeded to make Bear's Restaurant work on Switch
  • I have been working on performance tuning

49 of 58

Performance

  • It is almost the same as Wasm on browsers
  • Benchmarking with fmt package
    • Go 1.17.2 (macOS、GOMAXPROC=1) vs go2cpp@v0.1.3
      • go2cpp is x3-6 slower
    • Go 1.17.2 (Wasm) vs go2cpp@v0.1.3
      • Almost the same
      • Used wasmbrowsertest
      • Chrome vs go2cpp
    • The benchmark data
  • Bear's Restaurant works fine

50 of 58

Benchmark result of fmt package

  • GOOS=darwin vs go2cpp: x3-6 slower

  • GOOS=js (Chrome) vs go2cpp: Almost the same

name old time/op new time/op delta

SprintfEmpty 18.8ns ± 1% 111.3ns ± 3% +490.87% (p=0.008 n=5+5)

SprintfComplex 473ns ± 2% 1247ns ± 1% +163.90% (p=0.008 n=5+5)

name old time/op new time/op delta

SprintfPrefixedInt 464ns ± 1% 597ns ± 2% +28.69% (p=0.008 n=5+5)

ScanInts 1.05ms ± 5% 0.85ms ± 7% -19.09% (p=0.008 n=5+5)

51 of 58

The issue of performance

  • The issues that don't appear in benchmarks
    • GC
    • Single thread
      • If Wasm supports threads… (golang/go#28360)
  • As a final resort, I considered to split Ogg decoder to C++
    • Go's Ogg decoder is still used
    • Now this is still used as it is

52 of 58

GC issues

  • If GC happens, everything stops
    • Single thread
    • Freezes once every two minutes
    • If the heap size is big, the game could freeze�for a few seconds
  • Methods
    • 1. Manual GC
    • 2. Reducing heap allocations
    • 3. Other techniques
  • As a result, Bear's Restaurant works very well

Copyright いらすとや

53 of 58

Methods for GC 1. Manual GC

  • Stop the automatic GC completely�(debug.SetGCPercent(-1))
  • Do GC manually�(runtime.GC())
    • GC when the heap allocation�reaches a threshold (500MiB)
    • GC when the scene fades out
  • The increasing speed of heap�allocations matters

debug.SetGCPercent(-1)

go func() {

const threshold = 500 << 20

var stats runtime.MemStats

for {

runtime.ReadMemStats(&stats)

if stats.HeapAlloc > threshold {

runtime.GC()

}

time.Sleep(time.Second)

}

}()

54 of 58

Methods for GC 2. Reducing heap allocations

  • Checked runtime.mallocgc by Chrome's devtool
    • The profile results in Switch and Chrome are almost same!
  • There are some difficult cases that I�needed escape analysis (-gcflag=-m)
  • As a final resort, I added println in�runtime.mallocgc
  • Heap allocation improvement
    • The first speed was 1.2[MiB/s]
    • After optimization, it became smaller�than 400[KiB/s]
    • Ebiten also become faster

55 of 58

Methods for GC 3. Other techniques

  • Splitting GC process
    • 1. Mark & sweep on Go side
    • 2. finalizing on wasm_exec.js side
      • In the finalize phase, it is not necessary to destroy C++ objects�immediately
      • In finalizeRef, defer destroying Value
  • Overwriting the standard library by -overlay
    • Slices are allocated for each JS function call
    • Improve syscall/js (Adopted a suggestion in golang/go#39740 in advance)

56 of 58

Future works

  • Performance
    • Especially I want to improve GC
    • When I have to care heap allocations, I wonder why I write Go...
  • Compile time
    • It is painful to take 5 minutes to compile
  • If I have time, I want to try a cleverer way
    • What if replacing system calls with C function calls in some ways…?
    • I might want GOOS=libc (?)

57 of 58

Summary

58 of 58

Summary

  • Made Ebiten support Nintendo Switch
  • Bear's Restaurant for Switch was released
  • Developed go2cpp
  • GC issues and solutions (manual GC, reducing�heap allocations)

A Go program

A Wasm binary

The Go compiler

C++

go2cpp

Copyright 2021 Odencat Inc.

Bear's Restaurant for Switch