2DGS : 2D Gaussian Splatting for Geometrically Accurate Radiance Fields

Hyunbae Kim

https://www.youtube.com/watch?v=oaHCtB6yiKU

Problems of 3DGS

• Hard to learn Thin Surface
• Do not learn surface normal
• Lack of multi-view consistency

Problems of 3DGS

• Inaccurate affine transformation matrix

Problems of 3DGS

• Bad results of Mesh reconstruction

Contributions of 2DGS

• Present a highly efficient differentiable 2D Gaussian renderer, enabling perspective-correct splatting by leveraging 2D surface modeling, ray-splat intersection, and volumetric integration

​

• Introduce two regularization losses for improved and noise-free surface reconstruction

​

• Achieves state-of-the-art geometry reconstruction and NVS results compared to other explicit representations

3D gaussian Splatting

A : Jacobian of the affine approximation of the projective transformation

W : View transformation

R : Rotation matrix

S : Scale matrix

Modeling of 2DGS

• Central points : pk

​

• Tangential vector : (tu, tv)

​

• Scaling vector : S = (su, sv)

Modeling of 2DGS

• 2D Gaussian value

• 2D Gaussian value

• Rotation / Scale Matrix

Modeling of 2DGS

• Learnable parameters

​

• Central points : pk

​

• Rotation : (tu, tv)

​

• Scale : (su, sv)

​

• Opacity : α

​

• Color : c parameterized with SH

​

​

Splatting

• Screen space points

• ​

• Gaussian density in space points (x, y)

( x : homogeneous ray emitted from the camera and passing through pixel(x,y) )

Splatting

• Gaussian density in space points (x, y)

→ introduce numerical instability especially when the splat degenerates into a line segment

Splatting

Ray-splat Intersection

Given ray x = (x,y)

hx = (-1, 0, 0, x), hy = (0, -1, 0, y)

(1)

(2)

(3)

Ray-splat Intersection

Degenerate Solutions

Degenerate Solutions

• Object space low-pass filter

Rasterization

• Similar with 3DGS

Train

• Pipeline of 3DGS Training

• Pipeline of 2DGS Training

Color Loss

Depth Distortion Loss

Normal Consistency Loss

• i : Indexes over intersected splats along the ray

​

​

​

• ni : Normal of the splat i

​

• N : Normal estimated by the gradient of the depth map

​

Specifically, N is computed with finite differences from nearby depth points

• i : Indexes over intersected splats along the ray

​

​

​

• ni : Normal of the splat i

​

• N : Normal estimated by the gradient of the depth map

​

Specifically, N is computed with finite differences from nearby depth points

Train with Depth distortion / Normal consistency Loss

Ablations - Regularization

Comparison

Comparison

Comparison

Comparison