1 of 34

Solving 3D Problems With F#

​

Olya Samusik | @w0lya | w0lya.s@gmail.com

2 of 34

Agenda

  • 3D graphics programming world overview
    • Domains
    • Challenges
    • Tools, libraries
  • F# & 3D
    • Why F#?
    • Samples / demos
    • Interop with existing tools
    • What has been done by (not only) the F# Community
    • Where to learn? Where to Contribute?

3 of 34

3D basics

Vertex a point in the 3D space (x, y, z)

Edge a line connecting two points

Face a surface defined by at least 3 points

(aka polygon)

Wireframe a shape consisting of vertices

and edges

Mesh a set of polygons forming a shape

​

  • Model (local)
  • World
  • View (eye / camera)
  • Screen

Magic of Matrices

  • Translate
  • Rotate
  • Scale
  • Project

​

​

Representing Objects

Placing Objects: Coordinates

4 of 34

3D basics - continued

Graphics Pipeline

Geometry Pipeline

5 of 34

Rendering Pipeline

6 of 34

(*) - Programmable. Languages: GLSL / HLSL / Cg …

​

Shader Examples

Shader Pipeline

7 of 34

Alternative approach: Raytracing

Very realistic resulting images

8 of 34

GPUs

Forms:

  • Dedicated graphics card
  • Integrated graphics (no dedicated graphics memory, utilizes part of RAM)
  • Hybrid

9 of 34

Challenges besides 3D rendering

  • Real-time rendering with high FPS
  • Real-time rendering across various devices
  • Realism, high level of details vs eating all the computer power
  • Animation
    • Asset-based (static)
      • e.g. based on motion caption + morphing / inbetweening
    • Procedural
      • e.g. based on physics simulation
  • Modelling of (very) complex objects
    • Procedural
  • AI challenges, e.g. object detection
    • AR space
    • 3D object reconstruction from projection
    • ...

​

10 of 34

Domains: Game Development

11 of 34

Domains: Movie Making

12 of 34

Domains: All Kinds of Design

13 of 34

Domains: Architecture, Landscape Design

14 of 34

Domains: Medical, Scientific - All Kinds of Visualizations

15 of 34

Also, pretty much anything can be 3D Printed

16 of 34

VR / AR / MR

17 of 34

3D Graphics (Development) Challenges

Technical

  • Prototyping is not easy -> things get overlooked during Design phase
  • Modelling complex objects, scenes is time-consuming
  • Managing state for animations and simulations is hard
  • You cannot simply debug a scene / simulation
  • Debugging, troubleshooting shaders is hard
  • Maths and related ‘heavy’ computations everywhere
  • Being cross-platform + available on the web, in browser
  • Concurrency

​

​

​

Organizational

  • Communication is hard. Domain and Tech not “outsider-friendly”
  • Difficult scoping
  • Super-unrealistic estimates
  • Managers and clients have overly optimistic expectations

​

18 of 34

Can F# help to address (at least some of) these?

19 of 34

Prototyping

20 of 34

Prototyping - continued

21 of 34

Prototyping - pattern matching

Here and in the previous 2 slides: great examples from Beginning F# book by Robert Pickering

22 of 34

Taking it all to a higher level - a 3D DSL!

A sample made by Tomas Petricek using his OpenGL-based Fun3D library.

See the full demo here.

23 of 34

Shaders in F#

Diffuse shader. Computes the general color of a material when light shines on it. Thanks to SharpShaders.

24 of 34

Interactive and interop example: Quake 3 to F#

​

Will Smith, a.k.a. @TIHan, attempted porting Quake 3 to F#.

  • Running F# scripts in FSI, updating the scene real-time
  • Leveraged F# Compiler Service APIs that allow to implement language bindings

​

  • Interop
    • Mono abstraction
    • F# calls to C - pinvoke
    • C calls to F# - a C macro that calls F# methods, while mapping the unmanaged types into managed (custom)

​

  • Developed ferop library allowing to inline C in F#

​

​

25 of 34

Interactive Scripting - 3D Geometry of Abu-Dhabi

Goswin Rothenthal, Architect, used F# to represent and organize all the cladding elements of the Abu-Dhabi Dome.

Running F# in Tsunami IDE inside Rhino

​

26 of 34

3D Model Manipulation with F# for real-time rendering on mobile

Wael El Oraiby has chosen F# for 3D graphics dev and never regretted it. His Talk at F# Exchange 2018

​

F# look at Shaders

OSS Projects

F# GLFW Bindings

F# 3D Math Lib

F# Rendering Toolkit (Real-Time)

27 of 34

F# and Augmented Reality

The magic of Xamarin, ARKit and F#

​

​

28 of 34

Graphics tools: where is F# scripting possible?

3D Modelling

​

​

​

CAD Software

​

​

​

​

​

​

3Ds Max

MAXScript

MAYA

MEL, Python, C++

Blender

Python

AutoCAD

LISP routines (?!); F# - possible

FreeCAD

Python

Rhino3D

RhinoScript, C++, Python; F# - go here or ask Goswin

29 of 34

Integration of Graphics libraries & tools with F#

General 3D Graphics

​

​

​

​

​

​

​

​

​

OpenGL

OpenTK (C# bindings available), sample in F#

Open Scene Graph

Can either try PInvoke or SWIG

DirectX / Direct3D

Supported

Vulkan

.NET bindings available

Unity

Somewhat supported but no native support officially. However, F# Unity Kit is available. F# + Unity Tutorial. UnityScript deprecated, so no JS option.

Unreal

Officially possible thanks to MonoUE.

Also, web integration (Unreal.js via Fable bindings) is possible.

Urhosharp

RayLib

C# bindings currently available, need to try F# :)

MonoGame

Game Development

30 of 34

Graphics with F# on the Web

How to call JS graphics libraries from F#? Through Fable bindings.

  • Generate Fable bindings based on TypeScript definitions using ts2fable.
  • In your Fable F# project (or SAFE stack project), reference the bindings file or package.
  • Be sure to have the JavaScript library referenced :)
  • Call the library’s functions in F# code and enjoy!

​

31 of 34

F#, 3D Graphics and Fable - example

Generating a Terrain using Three.js, WebGL and a Perlin Noise generator lib.

Source code Live Demo

32 of 34

Summary. Which challenges can F# help to solve?

  • Prototyping
  • Modelling complex scenes
  • Expressing maths

Interactive REPL and F#’s expressiveness: data types, composing, creating DSLs

F# ‘tricks’ you into creating code where you can see the data / transformation flow clearly.

  • State management
  • Concurrency

Functional approach itself helps. Not covered here but Functional Reactive Programming style for graphics or game development is very promising. A FRP game engine has been developed by Bryan Edds in F#! Check it out

  • Debugging, troubleshooting scenes / shaders

Scenes: F# interactive helps

Shaders: write shaders in F#, transpile to shader languages. Try FShade or SharpShaders. Unit testing and friendly IDEs as a bonus.

  • Communication

Being able to easily prototype will help

Nicely written domain logic could help to communicate complex things

Bonus: a much, much better refactoring and maintenance experience.

33 of 34

3D & Gamedev in F#: 3D Graphics Programming and Game Development in F#

​

Apps and Games in F#: Guide - Mobile App and Game Programming with F#

​

Graphics & GameDev in other FP languages: Functional Programming In Games

​

General Computer Graphics Resources: Awesome Graphics

​

General Game Development Resources: Awesome Gamedev

Useful Resources

34 of 34

The end.

​

Questions?