Main Content

Tracking and Motion Estimation

Optical flow, activity recognition, motion estimation, and tracking

Motion estimation and tracking are key activities in many computer vision applications, including activity recognition, traffic monitoring, automotive safety, and surveillance.

Computer Vision Toolbox™ provides video tracking algorithms, such as continuously adaptive mean shift (CAMShift) and Kanade-Lucas-Tomasi (KLT). You can use these algorithms for tracking a single object or as building blocks in a more complex tracking system. The toolbox also provides a framework for multiple object tracking that includes Kalman filtering and the Hungarian algorithm for assigning object detections to tracks.

Motion estimation is the process of determining the movement of blocks between adjacent video frames. This toolbox includes motion estimation algorithms, such as optical flow, block matching, and template matching. These algorithms create motion vectors, which can relate to the whole image, blocks, arbitrary patches, or individual pixels. For block and template matching, the evaluation metrics for finding the best match include mean square error (MSE), mean absolute deviation (MAD), maximum absolute difference (MaxAD), sum of absolute difference (SAD), and sum of squared difference (SSD).


expand all

vision.BinaryFileReaderRead video data from binary files
vision.BinaryFileWriterWrite binary video data to files
vision.DeployableVideoPlayerDisplay video
vision.VideoPlayerPlay video or display image
vision.VideoFileReaderRead video frames and audio samples from video file
vision.VideoFileWriterWrite video frames and audio samples to video file
assignDetectionsToTracksAssign detections to tracks for multiobject tracking
bbox2pointsConvert rectangle to corner points list
configureKalmanFilterCreate Kalman filter for object tracking
vision.KalmanFilterCorrection of measurement, state, and state estimation error covariance
vision.HistogramBasedTrackerHistogram-based object tracking
vision.PointTrackerTrack points in video using Kanade-Lucas-Tomasi (KLT) algorithm
vision.BlockMatcherEstimate motion between images or video frames
vision.TemplateMatcherLocate template in image
opticalFlowObject for storing optical flow matrices
opticalFlowFarnebackObject for estimating optical flow using Farneback method
opticalFlowHSObject for estimating optical flow using Horn-Schunck method
opticalFlowLKObject for estimating optical flow using Lucas-Kanade method
opticalFlowLKDoGObject for estimating optical flow using Lucas-Kanade derivative of Gaussian method
vision.BlockMatcherEstimate motion between images or video frames
vision.TemplateMatcherLocate template in image
insertMarkerInsert markers in image or video
insertShapeInsert shapes in image or video
insertObjectAnnotationAnnotate truecolor or grayscale image or video stream
insertTextInsert text in image or video
imshowDisplay image
imshowpairCompare differences between images


Multiple Object Tracking

Tracking is the process of locating a moving object or multiple objects over time in a video stream.

Featured Examples