DiffOcclusion: Differentiable Optimization Based Control Barrier Functions for Occlusion-Free Visual Servoing

Shiqing Wei, Bolun Dai, Rooholla Khorrambakht, Prashanth Krishnamurthy, Farshad Khorrami

Research output: Contribution to journalArticlepeer-review


The visibility (possibly partial) of some image features is crucial to a broad class of visual servoing-based control. In this letter, we consider the setting of image-based visual servoing (IBVS) and address the fundamental problem of keeping a moving object with an unknown motion profile in the field of view while ensuring it remains unobstructed by obstacles. Assuming that the projections of the target and obstacles are both convex polygons, we propose a systematic method for circumscribing these polygons by strictly convex shapes with tunable accuracy. We prove that the minimal scaling factor such that two convex shapes intersect is continuously differentiable with respect to their vertex coordinates. Then, we formulate a control barrier function (CBF) based on this minimal scaling factor and incorporate a motion observer into occlusion-free visual servoing. The effectiveness of our method is validated through both simulation studies and experiments on the Franka Research 3 robotic arm.

Original languageEnglish (US)
Pages (from-to)3235-3242
Number of pages8
JournalIEEE Robotics and Automation Letters
Issue number4
StatePublished - Apr 1 2024


  • robot safety
  • sensor-based control
  • Visual servoing
  • visual tracking

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Biomedical Engineering
  • Human-Computer Interaction
  • Mechanical Engineering
  • Computer Vision and Pattern Recognition
  • Computer Science Applications
  • Control and Optimization
  • Artificial Intelligence


Dive into the research topics of 'DiffOcclusion: Differentiable Optimization Based Control Barrier Functions for Occlusion-Free Visual Servoing'. Together they form a unique fingerprint.

Cite this