2026
Tan, Chenjiao; Li, Changying; Sun, Jin
Dense cotton boll counting with transformer-based video tracking and a customized phenotyping robot for data collection Journal Article
In: Computers and Electronics in Agriculture, vol. 240, pp. 111214, 2026, ISSN: 0168-1699.
Abstract | Links | BibTeX | Tags: detection, Multi-Object Tracking, Optical flow, Point tracking, RT-DETR
@article{TAN2026111214,
title = {Dense cotton boll counting with transformer-based video tracking and a customized phenotyping robot for data collection},
author = {Chenjiao Tan and Changying Li and Jin Sun},
url = {https://www.sciencedirect.com/science/article/pii/S0168169925013201},
doi = {https://doi.org/10.1016/j.compag.2025.111214},
issn = {0168-1699},
year = {2026},
date = {2026-01-01},
journal = {Computers and Electronics in Agriculture},
volume = {240},
pages = {111214},
abstract = {Accurately estimating the number of cotton bolls is vital for plant phenotyping, offering essential insights for both breeders and growers. This trait offers valuable phenotypic information on plant productivity and supports crop management decisions to optimize yield and profitability for growers. Manual counting of bolls in the field, however, is impractical because it is labor-intensive and time-consuming. This study presented a video-based cotton boll counting approach that integrated a transformer-based detector (RT-DETR) with multi-object tracking techniques. To prevent double-counting bolls across frames, two motion estimation methods, FlowFormer and TAPIR were explored to predict the movement of bolls between adjacent frames and a two-stage association process combining Intersection over Union (IoU) and Euclidean distances was developed to track bolls across time. To further enhance counting accuracy, a virtual counting line was introduced to reduce ID switch errors. Experimental results demonstrated the effectiveness of the RT-DETR model, achieving an mAP0.5 exceeding 0.93 for dense boll detection. Furthermore, both FlowFormer and TAPIR can be used for tracking cotton bolls in the videos while the tracking performance of the FlowFormer-based method was slightly higher than that of the TAPIR-based method with an MOTA of 73.36 % and an IDF1 of 79.89 %. The tracking approach integrating RT-DETR and FlowFormer exhibited a relatively strong correlation between the predicted and the ground-truth boll number with an R2 of 0.60 and an MAPE of 14.34 % on multi-plant plots. In single-plant plots, the approach achieved a high correlation with an R2 of 0.97 and a MAPE of 10.33%. These findings indicated the potential of the proposed approach as an effective, automated tool to support breeding programs and yield assessments in cotton production. Both the code and dataset can be accessed at: https://github.com/UGA-BSAIL/Dense_cotton_boll_counting.},
keywords = {detection, Multi-Object Tracking, Optical flow, Point tracking, RT-DETR},
pubstate = {published},
tppubtype = {article}
}
Accurately estimating the number of cotton bolls is vital for plant phenotyping, offering essential insights for both breeders and growers. This trait offers valuable phenotypic information on plant productivity and supports crop management decisions to optimize yield and profitability for growers. Manual counting of bolls in the field, however, is impractical because it is labor-intensive and time-consuming. This study presented a video-based cotton boll counting approach that integrated a transformer-based detector (RT-DETR) with multi-object tracking techniques. To prevent double-counting bolls across frames, two motion estimation methods, FlowFormer and TAPIR were explored to predict the movement of bolls between adjacent frames and a two-stage association process combining Intersection over Union (IoU) and Euclidean distances was developed to track bolls across time. To further enhance counting accuracy, a virtual counting line was introduced to reduce ID switch errors. Experimental results demonstrated the effectiveness of the RT-DETR model, achieving an mAP0.5 exceeding 0.93 for dense boll detection. Furthermore, both FlowFormer and TAPIR can be used for tracking cotton bolls in the videos while the tracking performance of the FlowFormer-based method was slightly higher than that of the TAPIR-based method with an MOTA of 73.36 % and an IDF1 of 79.89 %. The tracking approach integrating RT-DETR and FlowFormer exhibited a relatively strong correlation between the predicted and the ground-truth boll number with an R2 of 0.60 and an MAPE of 14.34 % on multi-plant plots. In single-plant plots, the approach achieved a high correlation with an R2 of 0.97 and a MAPE of 10.33%. These findings indicated the potential of the proposed approach as an effective, automated tool to support breeding programs and yield assessments in cotton production. Both the code and dataset can be accessed at: https://github.com/UGA-BSAIL/Dense_cotton_boll_counting.
2024
Tan, Chenjiao; Sun, Jin; Paterson, Andrew H.; Song, Huaibo; Li, Changying
Three-view cotton flower counting through multi-object tracking and RGB-D imagery Journal Article
In: Biosystems Engineering, vol. 246, pp. 233-247, 2024, ISSN: 1537-5110.
Abstract | Links | BibTeX | Tags: Depth camera, Multiple views, Optical flow, Phenotyping, YOLO
@article{TAN2024233,
title = {Three-view cotton flower counting through multi-object tracking and RGB-D imagery},
author = {Chenjiao Tan and Jin Sun and Andrew H. Paterson and Huaibo Song and Changying Li},
url = {https://www.sciencedirect.com/science/article/pii/S1537511024001880},
doi = {https://doi.org/10.1016/j.biosystemseng.2024.08.010},
issn = {1537-5110},
year = {2024},
date = {2024-01-01},
journal = {Biosystems Engineering},
volume = {246},
pages = {233-247},
abstract = {Monitoring the number of cotton flowers can provide important information for breeders to assess the flowering time and the productivity of genotypes because flowering marks the transition from vegetative growth to reproductive growth and impacts the final yield. Traditional manual counting methods are time-consuming and impractical for large-scale fields. To count cotton flowers efficiently and accurately, a multi-view multi-object tracking approach was proposed by using both RGB and depth images collected by three RGB-D cameras fixed on a ground robotic platform. The tracking-by-detection algorithm was employed to track flowers from three views simultaneously and remove duplicated counting from single views. Specifically, an object detection model (YOLOv8) was trained to detect flowers in RGB images and a deep learning-based optical flow model Recurrent All-pairs Field Transforms (RAFT) was used to estimate motion between two adjacent frames. The intersection over union and distance costs were employed to associate flowers in the tracking algorithm. Additionally, tracked flowers were segmented in RGB images and the depth of each flower was obtained from the corresponding depth image. Those flowers tracked with known depth from two side views were then projected onto the middle image coordinate using camera calibration parameters. Finally, a constrained hierarchy clustering algorithm clustered all flowers in the middle image coordinate to remove duplicated counting from three views. The results showed that the mean average precision of trained YOLOv8x was 96.4%. The counting results of the developed method were highly correlated with those counted manually with a coefficient of determination of 0.92. Besides, the mean absolute percentage error of all 25 testing videos was 6.22%. The predicted cumulative flower number of Pima cotton flowers is higher than that of Acala Maxxa, which is consistent with what breeders have observed. Furthermore, the developed method can also obtain the flower number distributions of different genotypes without laborious manual counting in the field. Overall, the three-view approach provides an efficient and effective approach to count cotton flowers from multiple views. By collecting the video data continuously, this method is beneficial for breeders to dissect genetic mechanisms of flowering time with unprecedented spatial and temporal resolution, also providing a means to discern genetic differences in fecundity, the number of flowers that result in harvestable bolls. The code and datasets used in this paper can be accessed on GitHub: https://github.com/UGA-BSAIL/Multi-view_flower_counting.},
keywords = {Depth camera, Multiple views, Optical flow, Phenotyping, YOLO},
pubstate = {published},
tppubtype = {article}
}
Monitoring the number of cotton flowers can provide important information for breeders to assess the flowering time and the productivity of genotypes because flowering marks the transition from vegetative growth to reproductive growth and impacts the final yield. Traditional manual counting methods are time-consuming and impractical for large-scale fields. To count cotton flowers efficiently and accurately, a multi-view multi-object tracking approach was proposed by using both RGB and depth images collected by three RGB-D cameras fixed on a ground robotic platform. The tracking-by-detection algorithm was employed to track flowers from three views simultaneously and remove duplicated counting from single views. Specifically, an object detection model (YOLOv8) was trained to detect flowers in RGB images and a deep learning-based optical flow model Recurrent All-pairs Field Transforms (RAFT) was used to estimate motion between two adjacent frames. The intersection over union and distance costs were employed to associate flowers in the tracking algorithm. Additionally, tracked flowers were segmented in RGB images and the depth of each flower was obtained from the corresponding depth image. Those flowers tracked with known depth from two side views were then projected onto the middle image coordinate using camera calibration parameters. Finally, a constrained hierarchy clustering algorithm clustered all flowers in the middle image coordinate to remove duplicated counting from three views. The results showed that the mean average precision of trained YOLOv8x was 96.4%. The counting results of the developed method were highly correlated with those counted manually with a coefficient of determination of 0.92. Besides, the mean absolute percentage error of all 25 testing videos was 6.22%. The predicted cumulative flower number of Pima cotton flowers is higher than that of Acala Maxxa, which is consistent with what breeders have observed. Furthermore, the developed method can also obtain the flower number distributions of different genotypes without laborious manual counting in the field. Overall, the three-view approach provides an efficient and effective approach to count cotton flowers from multiple views. By collecting the video data continuously, this method is beneficial for breeders to dissect genetic mechanisms of flowering time with unprecedented spatial and temporal resolution, also providing a means to discern genetic differences in fecundity, the number of flowers that result in harvestable bolls. The code and datasets used in this paper can be accessed on GitHub: https://github.com/UGA-BSAIL/Multi-view_flower_counting.
2022
Tan, Chenjiao; Li, Changying; He, Dongjian; Song, Huaibo
Towards real-time tracking and counting of seedlings with a one-stage detector and optical flow Journal Article
In: Computers and Electronics in Agriculture, vol. 193, pp. 106683, 2022, ISSN: 0168-1699.
Abstract | Links | BibTeX | Tags: Cotton seedling, Counting, Deep convolutional neural network, deep learning, machine learning, object detection, Optical flow
@article{TAN2022106683,
title = {Towards real-time tracking and counting of seedlings with a one-stage detector and optical flow},
author = {Chenjiao Tan and Changying Li and Dongjian He and Huaibo Song},
url = {https://www.sciencedirect.com/science/article/pii/S0168169921007006},
doi = {https://doi.org/10.1016/j.compag.2021.106683},
issn = {0168-1699},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Computers and Electronics in Agriculture},
volume = {193},
pages = {106683},
abstract = {The population of crop seedlings is important for breeders and growers to evaluate the emergence rate of different cultivars and the necessity of replanting, but manual counting of plant seedlings is time-consuming and tedious. Building upon our prior work, we advanced the cotton seedling tracking method by incorporating a one-stage object detection deep neural network and optical flow to improve tracking speed and counting accuracy. Videos of cotton seedlings were captured using consumer-grade video cameras from the top view. You Only Look Once Version 4 (YOLOv4), a one-stage object detection network, was trained to detect cotton seedlings in each frame and to generate bounding boxes. To associate the same seedlings between adjacent frames, an optical flow-based tracking method was adopted to estimate camera motions. By comparing the positions of bounding boxes predicted by optical flow and detected by the YOLOv4 network in the same frame, the number of cotton seedlings was updated. The trained YOLOv4 model achieved high accuracy under conditions of occlusions, blurry images, complex backgrounds, and extreme illuminations. The F1 score of the final detection model was 0.98 and the average precision was 99.12%. Important tracking metrics were compared to evaluate the tracking performance. The Multiple-Object Tracking Accuracy (MOTA) and ID switch of the proposed tracking method were 72.8% and 0.1%, respectively. Counting results showed that the relative error of all testing videos was 3.13%. Compared with the Kalman filter and particle filter-based methods, our optical flow-based method generated fewer errors on testing videos because of higher accuracy of motion estimation. Compared with our previous work, the RMSE of the optical flow-based method decreased by 0.54 and the counting speed increased from 2.5 to 10.8 frames per second. The counting speed can reach 16.6 frames per second if the input resolution was reduced to 1280 × 720 pixels with an only 0.45% reduction in counting accuracy. The proposed method provides an automatic and near real-time tracking approach for counting of multiple cotton seedlings in video frames with improved speed and accuracy, which will benefit plant breeding and precision crop management.},
keywords = {Cotton seedling, Counting, Deep convolutional neural network, deep learning, machine learning, object detection, Optical flow},
pubstate = {published},
tppubtype = {article}
}
The population of crop seedlings is important for breeders and growers to evaluate the emergence rate of different cultivars and the necessity of replanting, but manual counting of plant seedlings is time-consuming and tedious. Building upon our prior work, we advanced the cotton seedling tracking method by incorporating a one-stage object detection deep neural network and optical flow to improve tracking speed and counting accuracy. Videos of cotton seedlings were captured using consumer-grade video cameras from the top view. You Only Look Once Version 4 (YOLOv4), a one-stage object detection network, was trained to detect cotton seedlings in each frame and to generate bounding boxes. To associate the same seedlings between adjacent frames, an optical flow-based tracking method was adopted to estimate camera motions. By comparing the positions of bounding boxes predicted by optical flow and detected by the YOLOv4 network in the same frame, the number of cotton seedlings was updated. The trained YOLOv4 model achieved high accuracy under conditions of occlusions, blurry images, complex backgrounds, and extreme illuminations. The F1 score of the final detection model was 0.98 and the average precision was 99.12%. Important tracking metrics were compared to evaluate the tracking performance. The Multiple-Object Tracking Accuracy (MOTA) and ID switch of the proposed tracking method were 72.8% and 0.1%, respectively. Counting results showed that the relative error of all testing videos was 3.13%. Compared with the Kalman filter and particle filter-based methods, our optical flow-based method generated fewer errors on testing videos because of higher accuracy of motion estimation. Compared with our previous work, the RMSE of the optical flow-based method decreased by 0.54 and the counting speed increased from 2.5 to 10.8 frames per second. The counting speed can reach 16.6 frames per second if the input resolution was reduced to 1280 × 720 pixels with an only 0.45% reduction in counting accuracy. The proposed method provides an automatic and near real-time tracking approach for counting of multiple cotton seedlings in video frames with improved speed and accuracy, which will benefit plant breeding and precision crop management.