Deformable Contour Tracking

Project Overview:

In this project, we focused on object tracking using contours. Specifically, we attempt to track objects that may deform substantially, where this can be useful is for image segmentation and general object tracking. We formulate the contour as a Hidden Markov Model according to Huang et. al [1]. In each new frame, we allow the states (locations along the boundary) to adapt to new observations of edge detection and normalized correlation coefficient (NCC) score against a patch centered at their location along the previous frame, while constraining smoothness of neighboring transitions as well. To do this, the forward-backward algorithmis is applied to find the global optimum set of contour landmark points in the new frame.

[Project Report]


Sample Results and Videos (Click on the images for video):

   videos require Xvid MPEG-4 codec, can be downloaded from

   A rotating and translating fist that is being tracked.

   Extending thumb and index finger from the fist.

   Applied to 3D tumor segmentation, by tracking 2D tumor contours from consecutive MRI slices.

   (Failed case) Extending all 5 fingers from a fist, apparently the deformation was too large for the algorithm to handle...


   [1] Y. Chen, Y. Rui, and T. S. Huang, "Multicue hmm-ukf for real-time contour tracking." IEEE Transaction on
   Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp. 1525-1529, 2006.