Hyperbolic View Dependency for All-in-Focus Time of Flight Fields

Time of flight (ToF) cameras measure depth and intensity images, enabling improved scene representation and high-level decision making in robotic systems. However, they perform poorly at long ranges and around absorptive or specular objects. ToF fields measured by arrays of cameras have been proposed for mitigating these limitations. In this work we expose a previously undescribed hyperbolic view dependency in ToF fields, and exploit this to construct an all-in-focus filter that improves noise immunity and accuracy of depth estimation.

We discover and describe a hyperbolic view dependency in the distance values of ToF fields,
We propose the first time of flight all-in-focus filter, exploiting this view dependency, offering enhanced noise rejection and depth accuracy compared to previous methods, and
We augment the filter to correctly handle occlusion boundaries and reject saturation from specularities, further improving accuracy and robustness.

These improvements increase the range of conditions in which ToF measurements can be relied upon by robotic systems for higher order algorithms like grasping and human-robot interaction.

Publications

• A. Taras and D. G. Dansereau, “Hyperbolic view dependency for all-in-focus time of flight fields,” Australasian Conference on Robotics and Automation (ACRA), 2022. Available here.

This work was carried out within the Robotic Imaging Group at the Sydney Institute for Robotics and Intelligent Systems, Australian Centre for Field Robotics, University of Sydney.

Acknowledgments

We would like to thank Jia Chen Lu for his suggestions on the implementation of this work and to Chronoptics for their support in using the Kea ToF camera.

Themes

Algorithms & Architectures, Robust Vision

Downloads

The code is on on GitHub here.

Data available for download here. The preview on the left shows the depth (top) and amplitude (bottom) channels of one of 13 scenes.

Our dataset saves both the depth and amplitude images from each view in the camera array. We provide a wide variety of scenes including: planar target, highly reflective objects (plastic boxes, shiny mannequin heads, fake fruit), occluded scenes and even challenging cases with refraction and reflection for future work.

See the dataset readme file here for further details.

Gallery

(click to enlarge)

Single ToF cameras degrade in image quality in challenging conditions, with saturation on specular surfaces (red) and high noise (blue, contrast stretched).

By using multiple view simultaneously, our pipeline produces higher quality depth images.

We capture rich data in a time of flight camera array and expose a hyperbolic view dependency.

This is novel and illustrates why typical light field techniques cannot be directly applied.

Our pipeline sweeps over putative depth, computing the goodness of fit between the theoretical model and the data from all views. The correct depth of the surface minimises the error of the fit. This is repeated for each pixel.

We alo propose a way of removing occluding and unphysical sources of information in the fit. This keeps edges sharp and rejects outliers whilst maintaining computational efficiency.

Overall, our pipeline improves the quality of depth images and removes view dependent saturation. This could allow robots to better represent scenes and succeed at higher order tasks such as human-robot interation or grasping in otherwise prohibative conditions.

Citing

If you find this work useful please cite

@inproceedings{taras2022hyperbolic,
  title = {Hyperbolic View Dependency for All-in-Focus Time of Flight Fields},
  author = {Adam K. Taras and Donald G. Dansereau},
  booktitle = {Australasian Conference on Robotics and Automation (ACRA)},
  year = {2022},
  publisher = {ARAA}
}