1 of 20

FastPoints: A State-of-the-Art Point Cloud Renderer for Unity

Elias Neuman-Donihue, Yuhao Zhu, Michael Jarvis

2 of 20

The Current State of Point Cloud Rendering

Used in a variety of contexts

Tools required to build point clouds more accessible than ever

Point clouds are becoming more and more popular

3 of 20

The Current State of Point Cloud Rendering

Point cloud rendering has received a lot of attention in the past few years

Large point clouds still pose a number of challenges

Have to be loaded dynamically from disk depending on viewpoint
Generating LOD takes a significant amount of time and/or requires external tools

Many modern renderers suffer from usability issues

It’s no surprise that, as more and more people across academic and industrial disciplines have begun using point clouds in their everyday work, the problem of efficient point cloud rendering has received a lot of attention

A number of breakthroughs have made rendering faster as well as more visually pleasing and useful

That said, large point clouds, by which I mean point clouds that are too large to fit entirely in main or GPU memory on a target platform, still pose a number of problems

The core issue is that a renderer has to be able to display all the points in the cloud - or at least enough points to convey all the visual information a user needs - without moving all those points to GPU memory

The solution to this is to load the cloud dynamically from disk depending on the user’s viewpoint - if you’re dealing with a point cloud of five hundred million points, but the user only has the camera pointed at one corner consisting of a hundred thousand points, the renderer needs to know to only load and render those points that are needed

The most popular way to accomplish this is to generate a level-of-detail structure for the cloud, such as an octree or a kd-tree, that allows a renderer to only load certain nodes or certain levels of subsampling and avoid overtaxing the GPU

But this, in turn, has issues of its own - this generation takes time, and in many cases requires an external application to generate the tree before the renderer can display it

Additionally, many modern renderers, especially state-of-the-art ones integrating newly developed rendering techniques, suffer from usability issues and target programmers rather than non-technical users. The growing popularity of point clouds for visualization outside the tech world has created a need for easily usable, non-programmatic renderers.

4 of 20

A Usability Standard for Point Cloud Renderers

We define four criteria:

Fully non-programmatic interface
Integrable with standard mesh rendering pipelines
Immediately interactive on unordered clouds
Support for popular file formats (PLY, LAS, LAZ)

With improving usability in mind, we set out to define a standard for a new renderer. We use four criteria.

The first is a fully non-programmatic interface. No aspect of our renderer should require a user to know how to code, and all important settings should be interactive through a GUI.

The second is integrability with standard mesh rendering pipelines. This ensures that users who want to involve point clouds in their other 3D digital work are able to do so seamlessly and without requiring different tools and format conversions.

The third is immediate interactivity, even on large point clouds. This means preprocessing for a large cloud doesn’t block the renderer, and the user can interact with other parts of their scene while preprocessing is being performed

The last is support for popular file formats. In particular, we’re thinking of formats such as LAS, LAZ, or PLY that don’t necessarily contain any information about ordering the points they store.

5 of 20

Existing Solutions

We apply our usability standard to three existing solutions for point cloud rendering:

Potree
BA_Pointcloud
CloudCompare

6 of 20

Existing Solutions - Potree

Fully non-programmatic interface
Integrable with standard mesh rendering pipelines
Immediately interactive on unordered clouds
Support for popular file formats

Potree is a state-of-the-art renderer that’s used to show point clouds on the web. It has a bunch of fantastic rendering features, but ultimately suffers a bit from poor UX for non-technical users

Unfortunately, it does not have a fully non-programmatic interface. It supports files that have been converted to the potree format with the external PotreeConverter tool. While the Desktop version does support drag-and-drop of these converted files, in either case a user needs to access the command line to convert the files in the first place.

Potree does not currently support meshes or images. I think I may have read that this will be changing in the future, but for the time being, it only supports point clouds

It’s also not immediately interactive on unordered clouds, as it requires preprocessing via a separate application

And finally, it doesn’t natively support popular file formats - all files must be converted with the PotreeConverter before they can be loaded.

7 of 20

Existing Solutions - BA_PointCloud

Fully non-programmatic interface
Integrable with standard mesh rendering pipelines
Immediately interactive on unordered clouds
Support for popular file formats

8 of 20

Existing Solutions - CloudCompare

Fully non-programmatic interface
Integrable with standard mesh rendering pipelines
Immediately interactive on unordered clouds
Support for popular file formats

9 of 20

FastPoints

We introduce FastPoints, a new renderer, to meet our usability standard

An external plugin for the Unity game engine

Integrates preprocessing through PotreeConverter v2.0

Introduces novel program architecture to allow interactivity during preprocessing

10 of 20

FastPoints

Fully non-programmatic interface
Integrable with standard mesh rendering pipelines
Immediately interactive on unordered clouds
Support for popular file formats

11 of 20

FastPoints - Program Architecture

Asynchronous program architecture

Immediately downsample unordered cloud

Preprocessing performed online while downsampled point cloud is rendered

All preprocessing done on separate thread, allowing user to interact with Unity environment during this time

12 of 20

FastPoints - Program Architecture

13 of 20

FastPoints - Program Architecture

14 of 20

FastPoints - Implementation Details

Implemented with C# plugin and linked native C++ plugin

C++ side handles actual file processing, including decimation and interfacing with LAStools for file reading and conversion and PotreeConverter for octree construction

C# side handles Unity logic and rendering, takes over completely once preprocessing has completed and octree is ready for rendering

15 of 20

FastPoints - Results

Our architecture cuts down on waiting time significantly for even massive point clouds

We found that our decimation strategy allowed much faster visualization than if we waiting for preprocessing to complete before displaying the full cloud.

We tested three clouds of different sizes - first, a 50 million point cloud from NEON; second, a point cloud of the Rush Rhees Library’s Great Hall at the University of Rochester, provided by Professor Michael Jarvis; and third, a point cloud of an Oplontis, a Roman archaelogical site, also provided by Professor Michael Jarvis and clocking in at almost a billion points.

We tested each of these starting in 3 different file formats, converted with LAStools - first, a binary PLY file, then an LAS file, and finally an LAZ file. We found that decimation times remained relatively low on all clouds with all file types, with the largest cloud in LAZ format still finishing decimation in under two minutes

The second table compares the time taken for each step in the pipeline starting from PLY files, which require the additional conversion step. In each case, the decimation time represented only a small fraction of the total preprocessing time.

16 of 20

FastPoints - Decimated Cloud

Exterior of decimated cloud

Interior of decimated cloud

17 of 20

FastPoints - Results

Decimated cloud of 1 million points

Full octree with point budget of 5 million points

18 of 20

FastPoints - Extensibility

Choosing Unity opens up a vast range of possibilities for extending FastPoints

Unity’s compute shader pipeline could be used to implement GPGPU optimizations

The modular nature of the architecture could allow either the octree construction step or rendering algorithm to be swapped out

19 of 20

Questions?

20 of 20

Thank you!