Lessons learned with laser scanning point cloud management in Hadoop HBase

Anh Vu Vo, Nikita Konda, Neel Chauhan, Harith Aljumaily, Debra F. Laefer

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

While big data technologies are growing rapidly and benefit a wide range of science and engineering domains, many barriers remain for the remote sensing community to fully exploit the benefits provided by these powerful and rapidly developing technologies. To overcome existing barriers, this paper presents the in-depth experience gained when adopting a distributed computing framework – Hadoop HBase – for storage, indexing, and integration of large scale, high resolution laser scanning point cloud data. Four data models were conceptualized, implemented, and rigorously investigated to explore the advantageous features of distributed, key-value database systems. In addition, the comparison of the four models facilitated the reassessment of several well-known point cloud management techniques founded in traditional computing environments in the new context of a distributed, key-value database. The four models were derived from two row-key designs and two columns structures, thereby demonstrating various considerations during the development of a data solution for high-resolution, city-scale aerial laser scan for a portion of Dublin, Ireland. This paper presents lessons learned from the data model design and its implementation for spatial data management in a distributed computing framework. The study is a step towards full exploitation of powerful emerging computing assets for dense spatio-temporal data.

Original languageEnglish (US)
Title of host publicationAdvanced Computing Strategies for Engineering - 25th EG-ICE International Workshop 2018, Proceedings
EditorsBernd Domer, Ian F. Smith
PublisherSpringer Verlag
Pages231-253
Number of pages23
ISBN (Print)9783319916347
DOIs
StatePublished - 2018
Event25th Workshop of the European Group for Intelligent Computing in Engineering, EG-ICE 2018 - Lausanne, Switzerland
Duration: Jun 10 2018Jun 13 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10863 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other25th Workshop of the European Group for Intelligent Computing in Engineering, EG-ICE 2018
CountrySwitzerland
CityLausanne
Period6/10/186/13/18

Keywords

  • Big data
  • Distributed database
  • HBase
  • Hadoop
  • LiDAR
  • Point cloud
  • Spatial data management

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Lessons learned with laser scanning point cloud management in Hadoop HBase'. Together they form a unique fingerprint.

Cite this