Towards three-dimensional point cloud reconstruction of fish swimming

Mert Karakaya, Chen Feng, Maurizio Porfiri

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Zebrafish is extensively used in behavioral, pharmacological, and neurological studies due to a number of methodological and practical advantages, including genetic and neurobiological homologies with humans and a fully sequenced genome. Critical to a biologically-based understanding of zebrafish behavior is the ability to reconstruct their complex behavioral repertoire in three-dimensions. Toward this aim, several efforts have been made to score their ethogram in three-dimensions, but most of these studies are constrained by a single-view imaging. A promising line of approach to extract refined information about the mechanosensory and perceptual systems of zebrafish is point cloud reconstruction. Here, we provide an initial review of the state of knowledge in zebrafish tracking and we propose a potential methodology that can capture the dynamic three-dimensional geometry of fish swimming. We utilize a stereo vision camera, calibrated with a pinhole camera model with refraction correction to allow for multi-medium imaging. The corrected pinhole camera model accounts for refraction through multiple mediums and allows for more accurate point cloud reconstruction from two cameras. From the point cloud data, we could recreate the three-dimensional geometric model of the fish and analyze its swimming behavior in three dimensions. The extracted dynamic fish geometry should allow for an improved understanding of mechanosensation and perception, which are critical to elucidate how zebrafish process visual cues and perceive flow structures.

Original languageEnglish (US)
Title of host publicationBioinspiration, Biomimetics, and Bioreplication X
EditorsMato Knez, Akhlesh Lakhtakia, Raul J. Martin-Palma
PublisherSPIE
ISBN (Electronic)9781510635258
DOIs
StatePublished - 2020
EventBioinspiration, Biomimetics, and Bioreplication X 2020 - Virtual, Online, United States
Duration: Apr 27 2020May 8 2020

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Volume11374
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X

Conference

ConferenceBioinspiration, Biomimetics, and Bioreplication X 2020
CountryUnited States
CityVirtual, Online
Period4/27/205/8/20

Keywords

  • Camera calibration
  • Mechanosensation
  • Point cloud reconstruction
  • Stereo-vision
  • Zebrafish

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Towards three-dimensional point cloud reconstruction of fish swimming'. Together they form a unique fingerprint.

Cite this