Aerial Single-view Depth Completion: Code + Datasets + Simulator

Youtube

Aerial Single-​View Depth Completion with Image-​Guided Uncertainty Estimation

With mapping for autonomous driving experiencing great boost with the use of deep-​​learning techniques, typical approaches have been using LIDAR measurements as seeds for image-​​based depth completion. The uncertainty of using vision-​based SLAM points as seeds instead, on top of the large viewpoint variations experienced in aerial mapping, however, are still posing major challenges for learning-​​based mapping approaches. Inspired by these shortcomings, we propose a powerful new methodology for single-​​view depth completion providing also confidence values for all scene-​depth estimates produced. This approach is shown to cope well with the challenges of scene perception from aerial platforms, such as large viewpoint changes, large depth variations, and limited computational resources. 

We present evaluations on existing benchmarking datasets as well as new, challenging, photo-​realistic datasets exhibiting a wide range of viewpoints as experienced typically by UAVs with depth and pose ground-​​truth information per image. Our results show that our network trained on our photo-​realistic datasets can be directly deployed on real-​​world outdoor aerial public datasets without fine-​​tuning or style transfer. Finally, we also release our simulator that was used to create such aerial datasets.

We are excited to publicly releast the following results of this work:

• Codebase for aerial, single-​view depth completion and uncertainty estimation: link
• Challenging and photo-​realistic Datasets exhibiting a wide range of viewpoints as experienced by UAV: link
• Visual-​inertial Simulator created for building such aerial datasets, integrating Gazebo, Blender and a UAV Physical library: link

Users of this software/datasets are asked to cite the following letter, where they were introduced:

Lucas Teixeira, Martin R. Oswald, Marc Pollefeys, Margarita Chli, "Aerial Single-​View Depth Completion with Image-​Guided Uncertainty Estimation" in Robotics and Automation Letters (RA-​L), 2020. DOI Research Collection