Counting the number of flowers in a plant is an example of agricultural quality inspection issues in which a simple 2D image of the product does not suffice. It is essential to see the object under inspection from multiple viewpoints to get a clear estimation of the quality of the product. In order to use multiple viewpoints to obtain a proper quality assessment, a multi-target tracking algorithm that accurately identifies relevant features of the product under inspection is proposed in this paper. The approach is illustrated with an experiment in which the flowers in a number of plants are counted. For the presented method, the plant rotates in front of a camera and a number of consecutive images is taken. The tracking algorithm detects, predicts, and matches the (partially occluded) flowers in the image. The experiments provide a proof of principle of the proposed method. The conclusion of this paper is that the presented multi-target tracking algorithm can be used to solve many similar quality assessment issues for agricultural objects.