Pubs Projects Tools About Home

Interactive Computational Imaging for Deformable Object Analysis (aka Project Squishy)

We describe an interactive approach for visual object analysis which exploits the ability of a robot to manipulate its environment. Knowledge of objects' mechanical properties is important in a host of robotics tasks, but their measurement can be impractical due to perceptual or mechanical limitations. By applying a periodic stimulus and matched video filtering and analysis pipeline, we show that even stiff, fragile, or low-texture objects can be distinguished based on their mechanical behaviours. We construct a novel, linear filter exploiting periodicity of the stimulus to reduce noise, enhance contrast, and amplify motion by a selectable gain - the proposed filter is significantly simpler than previous approaches to motion amplification. We further propose a set of statistics based on dense optical flow derived from the filtered video, and demonstrate visual object analysis based on these statistics for objects offering low contrast and limited deflection. Finally, we analyze 7 object types over 59 trials under varying illumination and pose, demonstrating that objects are linearly distinguishable under this approach, and establish the viability of estimating fluid level in a cup from the same statistics.


•  D. G. Dansereau, S. P. N. Singh, and J. Leitner, “Interactive computational imaging for deformable object analysis,” in Robotics and Automation (ICRA), 2016. Available here.


This work was a collaboration between Donald Dansereau and Juxi Leitner at the Australian Centre for Robotic Vision and Surya Singh from the Robotics Design Laboratory at the University of Queensland.


Interactive poster at


This research was partly supported by the Australian Research Council (ARC) Centre of Excellence for Robotic Vision (CE140100016) and the ARC Biomechanics Meets Robotics Project (DP160100714).