Perspective nonrigid shape and motion recovery

We present a closed form solution to the nonrigid shape and motion (NRSM) problem from point correspondences in multiple perspective uncalibrated views. Under the assumption that the nonrigid object deforms as a linear combination of K rigid shapes, we show that the NRSM problem can be viewed as a reconstruction problem from multiple projections from ℙ^3K to ℙ². Therefore, one can linearly solve for the projection matrices by factorizing a multifocal tensor. However, this projective reconstruction in ℙ^3K does not satisfy the constraints of the NRSM problem, because it is computed only up to a projective transformation in ℙ^3K . Our key contribution is to show that, by exploiting algebraic dependencies among the entries of the projection matrices, one can upgrade the projective reconstruction to determine the affine configuration of the points in ℙ³, and the motion of the camera relative to their centroid. Moreover, if K ≥ 2, then either by using calibrated cameras, or by assuming a camera with fixed internal parameters, it is possible to compute the Euclidean structure by a closed form method.