The generalized relative pose and scale problem: View-graph fusion via 2D-2D registration

It is well-known that the relative pose problem can be generalized to non-central cameras. We present a further generalization, denoted the generalized relative pose and scale problem. It has surprising importance for classical problems such as solving similarity transformations for view-graph concatenation in hierarchical structure from motion and loop-closure in visual SLAM, both posed as a 2D-2D registration problem. The relative pose problem and all its generalizations constitute a family of similar symmetric eigenvalue problems, which allow us to compress data and find a geometrically meaningful solution by an efficient search in the space of rotations. While the derivation of a completely general closed-form solver appears intractable, we make use of a simple heuristic global energy minimization scheme based on local minimum suppression, returning outstanding performance in practically relevant scenarios. Efficiency and reliability of our algorithm are demonstrated on both simulated and real data, supporting our claim of superior performance with respect to both generalized 2D-3D and 3D-3D registration approaches. By directly employing image information, we avoid the common noise in point clouds occuring especially along the depth direction.