Critical configurations for projective reconstruction from multiple views

This paper investigates a classical problem in computer vision: Given corresponding points in multiple images, when is there a unique projective reconstruction of the 3D geometry of the scene points and the camera positions? A set of points and cameras is said to be critical when there is more than one way of realizing the resulting image points. For two views, it has been known for almost a century that the critical configurations consist of points and camera lying on a ruled quadric surface. We give a classification of all possible critical configurations for any number of points in three images, and show that in most cases, the ambiguity extends to any number of cameras. The underlying framework for deriving the critical sets is projective geometry. Using a generalization of Pascal's Theorem, we prove that any number of cameras and scene points on an elliptic quartic form a critical set. Another important class of critical configurations consists of cameras and points on rational quartics. The theoretical results are accompanied by many examples and illustrations.