Projects: Novel Cameras
- An effective light field segmentation method
- Combines epipolar constraints with the rich semantics learned by the pretrained SAM 2 foundation model for cross-view mask matching
- Produces results of the same quality as the Segment Anything 2 (SAM 2) video tracking baseline, while being 7 times faster
- Can be inferenced in real-time for autonomous driving problems such as object pose tracking
- An end-to-end camera design method that co-designs cameras with perception tasks
- We combine derivative-free and gradient-based optimizers and support continuous, discrete, and categorical parameters
- A camera simulation including virtual environments and a physics-based noise model
- Key step in simplifying the process of designing cameras for robots
- Robotic vision without capturing images or allowing image reconstruction
- We propose guidelines and demonstrate localisation in simulation
- Call to action for advancing inherently private vision systems
- Try the reconstruction challenge!
- Automatically interpreting new cameras by jointly learning novel view sythesis, odometry, and a camera model
- A hypernetwork allows training with a wealth of existing cameras and datasets
- A semi-supervised light field network adapts to newly introduced cameras
- This work is a key step to automated integration of emerging camera technologies
- Unsupervised learning of odometry and depth from sparse 4D light fields
- Encoding sparse LFs for consumption by 2D CNNs for odometry and shape estimation
- Toward unsupervised interpretation of general LF cameras and new imaging devices
- Fast classification of visually similar objects using multiplexed illumination
- Using light stage capture and rendering to drive optimization of multiplexing codes
- Outperforms naive and conventional multiplexing patterns in accuracy and speed
- 138° 72 Mpix LF panoramas through a single, spherical lens
- Spherical parameterization compatible with planar light fields
- World's first single-lens wide-FOV LF camera
Spinning Omnidirectional Stereo Camera
- Live-Streaming 3D Virtual Reality Video
- Project page at Stanford, Example Videos, SIGGRAPH Asia slides (pdf w/embedded video)
- R. Konrad*, D. G. Dansereau*, A. Masood, and G. Wetzstein, “SpinVR: Towards live-streaming 3D virtual reality video,” ACM Transactions on Graphics (TOG), SIGGRAPH ASIA, vol. 36, no. 6, Nov. 2017. Available here.
Mirrored light field video camera adapter
- 3D printed base + laser-cut acrylic mirrors
- Creates a virtual array of cameras that measures a light field
- Optimization scheme finds best mirror array for a specific application
- D. Tsai, D. G. Dansereau, S. Martin, and P. Corke, “Mirrored light field video camera adapter,” Queensland University of Technology, Dec. 2016. Available here.
Exploiting parallax in panoramic capture to construct light fields
- We show that parallax, usually considered a nuisance, can be exploited to build light fields
- We turn an Ocular Robotics camera pointing system into an adaptive light field camera
- We demonstrate light field capture, refocusing and low-light image enhancement
- Best paper award, ACRA 2014
- D. G. Dansereau, D. Wood, S. Montabone, and S. B. Williams, “Exploiting parallax in panoramic capture to construct light fields,” in Australasian Conference on Robotics and Automation (ACRA), 2014. Available here.
