Volume

Rendering

2

2

⬄

⬄

Color

Depth

Reconstruct Color

Distill Ranking

Distill Continuity

Consistent neighbors

Rendered RGB image

Rendered Depth map

GT RGB image

Coarse depth map

1-minute quick start of SparseNeRF

Step 1

Conventional NeRF

Volume

Rendering

2

2

⬄

⬄

Color

Depth

Reconstruct Color

Distill Ranking

Distill Continuity

Consistent neighbors

Rendered RGB image

Rendered Depth map

GT RGB image

Coarse depth map

1-minute quick start of SparseNeRF

Step 2

Depth distillation

⬄

⬄

Depth

Distill Ranking

Distill Continuity

Consistent neighbors

Rendered Depth map

Coarse depth map

Step 2

Depth distillation

⬄

⬄

Depth

Distill Ranking

Distill Continuity

Consistent neighbors

Rendered Depth map

Coarse depth map

Step 2.1

Distill Ranking

Rendered depth patch

coarse depth patch

Distill Ranking

Rendered depth patch

coarse depth patch

Algorithm 1:

⬄

⬄

Depth

Distill Ranking

Distill Continuity

Consistent neighbors

Rendered Depth map

Coarse depth map

Step 2.2

K-nearest neighbors

Distill Continuty

Consistent

1. Suppose we sample a bounding box set B={B1, B2,…} in Algorithm 1, then we further sample smaller bounding boxes BB={BB1, BB2, …} from B (see below), where BB1 is the corresponding smaller box sampled from B1.
2. Loss2=0
3. For each smaller bounding box BBi in BB:
4. Compute the k-nearest neighbors of the anchor A on the depth map of DPT
5. Record the corresponding pixel position
6. Encourage these neighbors on the rendered depth of NeRF to be continuous. We randomly select one neighbor A_i, and construct pairs (A,A_i)

we compute the loss2= loss2+ max(|depth’_A-depth’_A_i|-m’, 0)

B1

BB1

BB1

anchor A

k-nearest neighbors

In Algorithm 1, we sample K pairs of points. Now, we use these points as anchors (centers). We further crop smaller boxes BB.

Algorithm 2:

Here, the anchor A is from the sampling points of P in Algorithm 1.

Sampled points in Algorithm 1, anchor, centers of the box