Tracking of objects in videos consists of giving a label to the same object moving in different frames. This labeling is performed by predicting position of the object given its set of features observed in previous frames.
In this work, we apply the same rationale by considering each connected component in the manuscript as a moving object and to track it so that to minimize the distance and angle of the connected component to its nearest neighbor. The approach was applied to images of ICDAR 2013 handwritten segmentation contest and proved to be robust against text orientation, size and writing script.