Recasens David ; Oswald Martin R. ; Pollefeys Marc ; Civera Javier

Abstract

Estimating camera motion in deformable scenes poses a complex and openresearch challenge. Most existing non-rigid structure from motion techniquesassume to observe also static scene parts besides deforming scene parts inorder to establish an anchoring reference. However, this assumption does nothold true in certain relevant application cases such as endoscopies. Deformableodometry and SLAM pipelines, which tackle the most challenging scenario ofexploratory trajectories, suffer from a lack of robustness and properquantitative evaluation methodologies. To tackle this issue with a commonbenchmark, we introduce the Drunkard's Dataset, a challenging collection ofsynthetic data targeting visual navigation and reconstruction in deformableenvironments. This dataset is the first large set of exploratory cameratrajectories with ground truth inside 3D scenes where every surface exhibitsnon-rigid deformations over time. Simulations in realistic 3D buildings lets usobtain a vast amount of data and ground truth labels, including camera poses,RGB images and depth, optical flow and normal maps at high resolution andquality. We further present a novel deformable odometry method, dubbed theDrunkard's Odometry, which decomposes optical flow estimates into rigid-bodycamera motion and non-rigid scene deformations. In order to validate our data,our work contains an evaluation of several baselines as well as a noveltracking error metric which does not require ground truth data. Dataset andcode: https://davidrecasens.github.io/TheDrunkard'sOdometry/