Colmap Free Gaussian & MonoGS

Boshu Lei

Problem with 3D GS

1. Accurate Camera Pose

2. Good Initialization

Camera Pose Estimation

Pose Estimation

Step 1. Initialize gaussians from monocular depth

Step 2. Optimize camera transform

Lietorch Library for computing SE(3) gradient.

Computation Time 5-10 s

Pose Estimation (MonoGS)

Directly optimize global pose

Backward Gradient Flow Path

Image Gradient

Splatted Gaussian

3D Gaussian

Only means and rotation are relevant to camera pose

Pose Estimation (MonoGS)

Jacobian w.r.t camera pose in SE(3)

Analytical Results

Sliding Window for Tracking (MonoGS)

Optimize Gaussian & Pose

Optimize Gaussian

Mapping

Mapping

Global Pose

Densification

Densification lasts till the last frame arrives.

Densification happens when the new frame is added.

Relative estimation and combination

Mapping (MonoGS)

Mapping (MonoGS)

System Initialization

Randomly initialize depth map and un-project

Insert & Prune Gaussians

Insertion

Insertion based on rendered depth or median depth

Prune

Prune gaussians which have low co-visibility score

Mapping

Mapping

MonoGS

Colmap Free Experiment

Rendering Quality Comparison

Pose Estimation Comparison

Experiment Setup:

1. Initialize the test view pose using the closest training view pose.
2. Fine-tune test view pose.
3. Render image and compute metric

SLAM Results

TUM RGBD

Some Failure Case

T

Comments

Pose Estimation

1. Rely on the assumption that adjacent frame has small displacement.

2. Probably sparse key points matching can give better estimation.

Mapping

1. No loop closure.

2. Sliding window is not appropriate due to discarding past constraints.