Project Description

This project studies the interaction of two disparate but inherent aspects of computer vision systems: the temporal evolution of a dynamic scene and the nonlinear representations of features of interest. Specifically, we are interested in dynamical systems where a process of interest evolves over time on a nonlinear manifold and one observes this process only at limited times. The goal is to estimate/predict the remaining process using the observed values under some predetermined criterion. The motivation of such a problem comes from many applications. Consider the evolution of the shape of a human silhouette in a video, in a situation where one has an unobstructed view of the person in only a few frames. Given these observed shapes, along with their observation times, one would like to estimate shapes at some intermediate times and perhaps even predict future shape evolution. Additionally, if the observed shapes were considered noisy (for example, due to the process of extracting shapes from image frames), one would like to smooth the observed shapes using their temporal structure. A similar problem arises in tracking the rigid motion of an object using video data, e.g. in tracking a human speaker using webcam video. Focusing on the rotation component, one observes the object’s orientation at certain times and seeks to estimate and smooth the rotation process over the whole observation interval.

Our Approach

We adopt the framework developed in Samir et al. (2011) that develops a Palais metric-based steepest-decent algorithm applied to the weighted sum of a fitting-related and a regularity-related cost function. Our goal is to find a path that minimizes the energy function


Using the rotation group, the space of positive-definite matrices, and Kendall’s shape space as three representative manifolds, we develop the proposed algorithm for curve fitting. This algorithm requires expressions for exponential maps, inverse exponential maps, parallel transport of tangents, and curvature tensors on the chosen manifolds. These ideas are illustrated using a large number of experimental results on both simulated and real data.


SO(3): Black: true path, yellow: spline on mean tangent space, green: piecewise geodesics, blue: smoothing spline using our method

Related Publications

J. Su, I.L. Dryden, E. Klassen, H. Le and A. Srivastava. Fitting Smoothing Splines to Time-Indexed, Noisy Points on Nonlinear Manifolds. Journal of Image and Vision Computing, accepted, September 2011(doi:10.1016/j.imavis.2011.09.006).