A 6-dimensional hilbert approach to index full waveform lidar data in a distributed computing environment

Laser scanning data are increasingly available across the globe. To maximize the data's usability requires proper storage and indexing. While significant research has been invested in developing storage and indexing solutions for laser scanning point clouds (i.e. using the discrete form of the data), little attention has been paid to developing equivalent solutions for full waveform (FWF) laser scanning data, especially in a distributed computing environment. Given the growing availability of FWF sensors and datasets, FWF data management solutions are increasingly needed. This paper presents an attempt towards establishing a scalable solution for handling large FWF datasets by introducing the distributed computing solution for FWF data. The work involves a FWF database built atop HBase – the distributed database system running on Hadoop commodity clusters. By combining a 6-dimensional (6D) Hilbert spatial code and a temporal index into a compound indexing key, the database system is capable of supporting multiple spatial, temporal, and spatio-temporal queries. Such queries are important for FWF data exploration and dissemination. The proposed spatial decomposition at a fine resolution of 0.05m allows the storage of each LiDAR FWF measurement (i.e. pulse, waves, and points) on a single row of the database, thereby providing the full capabilities to add, modify, and remove each measurement record anatomically. While the feasibility and capabilities of the 6D Hilbert solution are evident, the Hilbert decomposition is not due to the complications from the combination of the data’s high dimensionality, fine resolution, and large spatial extent. These factors lead to a complex set of both attractive attributes and limitation in the proposed solution, which are described in this paper based on experimental tests using a 1.1 billion pulse LiDAR scan of a portion of Dublin, Ireland.