TY - GEN
T1 - Towards three-dimensional point cloud reconstruction of fish swimming
AU - Karakaya, Mert
AU - Feng, Chen
AU - Porfiri, Maurizio
N1 - Funding Information:
This research was supported by the National Science Foundation under Grant No. CMMI-1901697, by the National Institutes of Health, National Institute on Drug Abuse under grant number 1R21DA042558-01A1, and by the Office of Behavioral and Social Sciences Research that co-founded the National Institute on Drug Abuse grant. The authors are thankful to Vrishin Rajiv Soman for his help in preparing some of the experiments and conducting preliminary analysis.
Publisher Copyright:
© 2020 SPIE
PY - 2020
Y1 - 2020
N2 - Zebrafish is extensively used in behavioral, pharmacological, and neurological studies due to a number of methodological and practical advantages, including genetic and neurobiological homologies with humans and a fully sequenced genome. Critical to a biologically-based understanding of zebrafish behavior is the ability to reconstruct their complex behavioral repertoire in three-dimensions. Toward this aim, several efforts have been made to score their ethogram in three-dimensions, but most of these studies are constrained by a single-view imaging. A promising line of approach to extract refined information about the mechanosensory and perceptual systems of zebrafish is point cloud reconstruction. Here, we provide an initial review of the state of knowledge in zebrafish tracking and we propose a potential methodology that can capture the dynamic three-dimensional geometry of fish swimming. We utilize a stereo vision camera, calibrated with a pinhole camera model with refraction correction to allow for multi-medium imaging. The corrected pinhole camera model accounts for refraction through multiple mediums and allows for more accurate point cloud reconstruction from two cameras. From the point cloud data, we could recreate the three-dimensional geometric model of the fish and analyze its swimming behavior in three dimensions. The extracted dynamic fish geometry should allow for an improved understanding of mechanosensation and perception, which are critical to elucidate how zebrafish process visual cues and perceive flow structures.
AB - Zebrafish is extensively used in behavioral, pharmacological, and neurological studies due to a number of methodological and practical advantages, including genetic and neurobiological homologies with humans and a fully sequenced genome. Critical to a biologically-based understanding of zebrafish behavior is the ability to reconstruct their complex behavioral repertoire in three-dimensions. Toward this aim, several efforts have been made to score their ethogram in three-dimensions, but most of these studies are constrained by a single-view imaging. A promising line of approach to extract refined information about the mechanosensory and perceptual systems of zebrafish is point cloud reconstruction. Here, we provide an initial review of the state of knowledge in zebrafish tracking and we propose a potential methodology that can capture the dynamic three-dimensional geometry of fish swimming. We utilize a stereo vision camera, calibrated with a pinhole camera model with refraction correction to allow for multi-medium imaging. The corrected pinhole camera model accounts for refraction through multiple mediums and allows for more accurate point cloud reconstruction from two cameras. From the point cloud data, we could recreate the three-dimensional geometric model of the fish and analyze its swimming behavior in three dimensions. The extracted dynamic fish geometry should allow for an improved understanding of mechanosensation and perception, which are critical to elucidate how zebrafish process visual cues and perceive flow structures.
KW - Camera calibration
KW - Mechanosensation
KW - Point cloud reconstruction
KW - Stereo-vision
KW - Zebrafish
UR - http://www.scopus.com/inward/record.url?scp=85096355825&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85096355825&partnerID=8YFLogxK
U2 - 10.1117/12.2558095
DO - 10.1117/12.2558095
M3 - Conference contribution
AN - SCOPUS:85096355825
T3 - Proceedings of SPIE - The International Society for Optical Engineering
BT - Bioinspiration, Biomimetics, and Bioreplication X
A2 - Knez, Mato
A2 - Lakhtakia, Akhlesh
A2 - Martin-Palma, Raul J.
PB - SPIE
T2 - Bioinspiration, Biomimetics, and Bioreplication X 2020
Y2 - 27 April 2020 through 8 May 2020
ER -