A highly scalable data management system for point cloud and full waveform lidar data

A. V. Vo, D. F. Laefer, M. Trifkovic, C. N.L. Hewage, M. Bertolotto, N. A. Le-Khac, U. Ofterdinger

Research output: Contribution to journalConference articlepeer-review

Abstract

The massive amounts of spatio-temporal information often present in LiDAR data sets make their storage, processing, and visualisation computationally demanding. There is an increasing need for systems and tools that support all the spatial and temporal components and the three-dimensional nature of these datasets for effortless retrieval and visualisation. In response to these needs, this paper presents a scalable, distributed database system that is designed explicitly for retrieving and viewing large LiDAR datasets on the web. The ultimate goal of the system is to provide rapid and convenient access to a large repository of LiDAR data hosted in a distributed computing platform. The system is composed of multiple, share-nothing nodes operating in parallel. Namely, each node is autonomous and has a dedicated set of processors and memory. The nodes communicate with each other via an interconnected network. The data management system presented in this paper is implemented based on Apache HBase, a distributed key-value datastore within the Hadoop eco-system. HBase is extended with new data encoding and indexing mechanisms to accommodate both the point cloud and the full waveform components of LiDAR data. The data can be consumed by any desktop or web application that communicates with the data repository using the HTTP protocol. The communication is enabled by a web servlet. In addition to the command line tool used for administration tasks, two web applications are presented to illustrate the types of user-facing applications that can be coupled with the data system.

Original languageEnglish (US)
Pages (from-to)507-512
Number of pages6
JournalInternational Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives
Volume43
Issue numberB4
DOIs
StatePublished - Aug 6 2020
Event2020 24th ISPRS Congress - Technical Commission IV on Spatial Information Science - Nice, Virtual, France
Duration: Aug 31 2020Sep 2 2020

Keywords

  • Big Data
  • Full Waveform
  • LiDAR
  • Point Cloud
  • Spatial Database
  • Web Service
  • Web-based Visualisation

ASJC Scopus subject areas

  • Information Systems
  • Geography, Planning and Development

Fingerprint

Dive into the research topics of 'A highly scalable data management system for point cloud and full waveform lidar data'. Together they form a unique fingerprint.

Cite this