TY - JOUR
T1 - ASSESSING LIDAR TRAINING DATA QUANTITIES for CLASSIFICATION MODELS
AU - Majgaonkar, O.
AU - Panchal, K.
AU - Laefer, D.
AU - Stanley, M.
AU - Zaki, Y.
N1 - Funding Information:
Funding for this project was provided by the National Science Foundation awards 1826134 and 1940145. The ModelNet, Vaihingen, and Sunset Park data sets were provided by the Princeton ModelNet Project, German Society for Photogrammetry, Remote Sensing and Geoinformation, and the NYU Center for Urban Science and Progress, respectively.
Publisher Copyright:
© 2021 O. Majgaonkar et al.
PY - 2021/10/7
Y1 - 2021/10/7
N2 - Classifying objects within aerial Light Detection and Ranging (LiDAR) data is an essential task to which machine learning (ML) is applied increasingly. ML has been shown to be more effective on LiDAR than imagery for classification, but most efforts have focused on imagery because of the challenges presented by LiDAR data. LiDAR datasets are of higher dimensionality, discontinuous, heterogenous, spatially incomplete, and often scarce. As such, there has been little examination into the fundamental properties of the training data required for acceptable performance of classification models tailored for LiDAR data. The quantity of training data is one such crucial property, because training on different sizes of data provides insight into a model's performance with differing data sets. This paper assesses the impact of training data size on the accuracy of PointNet, a widely used ML approach for point cloud classification. Subsets of ModelNet ranging from 40 to 9,843 objects were validated on a test set of 400 objects. Accuracy improved logarithmically; decelerating from 45 objects onwards, it slowed significantly at a training size of 2,000 objects, corresponding to 20,000,000 points. This work contributes to the theoretical foundation for development of LiDAR-focused models by establishing a learning curve, suggesting the minimum quantity of manually labelled data necessary for satisfactory classification performance and providing a path for further analysis of the effects of modifying training data characteristics.
AB - Classifying objects within aerial Light Detection and Ranging (LiDAR) data is an essential task to which machine learning (ML) is applied increasingly. ML has been shown to be more effective on LiDAR than imagery for classification, but most efforts have focused on imagery because of the challenges presented by LiDAR data. LiDAR datasets are of higher dimensionality, discontinuous, heterogenous, spatially incomplete, and often scarce. As such, there has been little examination into the fundamental properties of the training data required for acceptable performance of classification models tailored for LiDAR data. The quantity of training data is one such crucial property, because training on different sizes of data provides insight into a model's performance with differing data sets. This paper assesses the impact of training data size on the accuracy of PointNet, a widely used ML approach for point cloud classification. Subsets of ModelNet ranging from 40 to 9,843 objects were validated on a test set of 400 objects. Accuracy improved logarithmically; decelerating from 45 objects onwards, it slowed significantly at a training size of 2,000 objects, corresponding to 20,000,000 points. This work contributes to the theoretical foundation for development of LiDAR-focused models by establishing a learning curve, suggesting the minimum quantity of manually labelled data necessary for satisfactory classification performance and providing a path for further analysis of the effects of modifying training data characteristics.
KW - Learning Curve
KW - LiDAR
KW - Object Classification
KW - Point Cloud
KW - Training
UR - http://www.scopus.com/inward/record.url?scp=85118307725&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85118307725&partnerID=8YFLogxK
U2 - 10.5194/isprs-archives-XLVI-4-W4-2021-101-2021
DO - 10.5194/isprs-archives-XLVI-4-W4-2021-101-2021
M3 - Conference article
AN - SCOPUS:85118307725
SN - 1682-1750
VL - 46
SP - 101
EP - 106
JO - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives
JF - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives
IS - 4/W4-2021
T2 - 16th 3D GeoInfo Conference 2021
Y2 - 11 October 2021 through 14 October 2021
ER -