Colour-based rigid body tracking using three-dimensional graphics models

This paper introduces the first stage of a new model-based approach to three-dimensional (3D) human movement tracking. A ‘generate-and-test’ matching procedure was adopted by matching rendered images of a 3D computer graphics model of the human body to target images of rigid body motion. The set of pixels to be compared were just those corresponding to the model of the body in the rendered images. The matching criterion to optimize model position and orientation was based on the minimisation of the RGB (red-green-blue) colour difference between generated model images and associated target images. The method was able to track synthetic image sequences of a half twisting somersault accurately with root-mean-square (rms) errors of less than 5 mm and 0.3∘ for position and orientation estimates respectively. The suitability of the proposed approach for rigid body motion tracking was supported by additional tracking experiments on video image sequences of ‘wooden cross’ trajectories. Comparisons of tracked estimates against manual digitizing estimates returned relatively small rms difference values on both side somersault and twisting somersault movements. The proposed approach has the potential to track video images of a human torso using a rigid body model and hence to track articulated movements by successively adding segments to the model in a hierarchical manner.