An official website of the United States government
Reliable seed yield estimation is an indispensable step in plant breeding programs geared towards cultivar development in major row crops. The objective of this study is to develop a machine learning (ML) approach adept at soybean ( Glycine max L. (Merr.)) pod counting to enable genotype seed yield rank prediction from in-field video data collected by a ground robot. To meet this goal, we developed a multiview image-based yield estimation framework utilizing deep learning architectures. Plant images captured from different angles were fused to estimate the yield and subsequently to rank soybean genotypes for application in breeding decisions. We used data from controlled imaging environment in field, as well as from plant breeding test plots in field to demonstrate the efficacy of our framework via comparing performance with manual pod counting and yield estimation. Our results demonstrate the promise of ML models in making breeding decisions with significant reduction of time and human effort and opening new breeding method avenues to develop cultivars.
more »
« less
This content will become publicly available on March 31, 2026
Robust soybean seed yield estimation using high-throughput ground robot videos
We present a novel method for soybean [Glycine max(L.) Merr.] yield estimation leveraging high-throughput seed counting via computer vision and deep learning techniques. Traditional methods for collecting yield data are labor-intensive, costly, and prone to equipment failures at critical data collection times and require transportation of equipment across field sites. Computer vision, the field of teaching computers to interpret visual data, allows us to extract detailed yield information directly from images. By treating it as a computer vision task, we report a more efficient alternative, employing a ground robot equipped with fisheye cameras to capture comprehensive videos of soybean plots from which images are extracted in a variety of development programs. These images are processed through the P2PNet-Yield model, a deep learning framework, where we combined a feature extraction module (the backbone of the P2PNet-Soy) and a yield regression module to estimate seed yields of soybean plots. Our results are built on 2 years of yield testing plot data—8,500 plots in 2021 and 650 plots in 2023. With these datasets, our approach incorporates several innovations to further improve the accuracy and generalizability of the seed counting and yield estimation architecture, such as the fisheye image correction and data augmentation with random sensor effects. The P2PNet-Yield model achieved a genotype ranking accuracy score of up to 83%. It demonstrates up to a 32% reduction in time to collect yield data as well as costs associated with traditional yield estimation, offering a scalable solution for breeding programs and agricultural productivity enhancement.
more »
« less
- Award ID(s):
- 1954556
- PAR ID:
- 10579965
- Publisher / Repository:
- Frontiers in Plant Science
- Date Published:
- Journal Name:
- Frontiers in Plant Science
- Volume:
- 16
- ISSN:
- 1664-462X
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Plant counting is a critical aspect of crop management, providing farmers with valuable insights into seed germination success and within-field variation in crop population density, both of which are key indicators of crop yield and quality. Recent advancements in Unmanned Aerial System (UAS) technology, coupled with deep learning techniques, have facilitated the development of automated plant counting methods. Various computer vision models based on UAS images are available for detecting and classifying crop plants. However, their accuracy relies largely on the availability of substantial manually labeled training datasets. The objective of this study was to develop a robust corn counting model by developing and integrating an automatic image annotation framework. This study used high-spatial-resolution images collected with a DJI Mavic Pro 2 at the V2–V4 growth stage of corn plants from a field in Wooster, Ohio. The automated image annotation process involved extracting corn rows and applying image enhancement techniques to automatically annotate images as either corn or non-corn, resulting in 80% accuracy in identifying corn plants. The accuracy of corn stand identification was further improved by training four deep learning (DL) models, including InceptionV3, VGG16, VGG19, and Vision Transformer (ViT), with annotated images across various datasets. Notably, VGG16 outperformed the other three models, achieving an F1 score of 0.955. When the corn counts were compared to ground truth data across five test regions, VGG achieved an R2 of 0.94 and an RMSE of 9.95. The integration of an automated image annotation process into the training of the DL models provided notable benefits in terms of model scaling and consistency. The developed framework can efficiently manage large-scale data generation, streamlining the process for the rapid development and deployment of corn counting DL models.more » « less
-
Abstract Because the manual counting of soybean (Glycine max) plants, pods, and seeds/pods is unsuitable for soybean yield predictions, alternative methods are desired. Therefore, the objective was to determine if satellite remote sensing‐based artificial intelligence (AI) models could be used to predict soybean yield. In the study, multiple remote sensing‐based AI models were developed for soybean growth stage ranging from VE/VC (plant emergence) to R6/R7 (full seed to beginning maturity). The ability of the deep neural network (DNN), support vector machine (SVM), random forest (RF), least absolute shrinkage and selection operator (LASSO), and AdaBoost to predict soybean yield, based on blue, green, red, and near‐infrared reflectance data collected by the PlanetScope satellite at six growth stages, was determined. Remote sensing and soybean yield monitor data from three different fields in 2 years (2019 and 2021) were aggregated into 24,282 grid cells that had the dimensions of 10 m by 10 m. A comparison across models showed that the DNN outperformed the other models. Moreover, as crops matured from VE/VC to R4/R5, theR2value of the models increased from 0.26 to over 0.70. These findings indicate that remote sensing data collected at different growth stages can be combined for soybean yield predictions. Moreover, additional work needs to be conducted to assess the model's ability to predict soybean yield with vegetation indices (VIs) data for fields not used to train the model.more » « less
-
Abstract Developments in genomics and phenomics have provided valuable tools for use in cultivar development. Genomic prediction (GP) has been used in commercial soybean [Glycine maxL. (Merr.)] breeding programs to predict grain yield and seed composition traits. Phenomic prediction (PP) is a rapidly developing field that holds the potential to be used for the selection of genotypes early in the growing season. The objectives of this study were to compare the performance of GP and PP for predicting soybean seed yield, protein, and oil. We additionally conducted genome‐wide association studies (GWAS) to identify significant single‐nucleotide polymorphisms (SNPs) associated with the traits of interest. The GWAS panel of 292 diverse accessions was grown in six environments in replicated trials. Spectral data were collected at two time points during the growing season. A genomic best linear unbiased prediction (GBLUP) model was trained on 269 accessions, while three separate machine learning (ML) models were trained on vegetation indices (VIs) and canopy traits. We observed that PP had a higher correlation coefficient than GP for seed yield, while GP had higher correlation coefficients for seed protein and oil contents. VIs with high feature importance were used as covariates in a new GBLUP model, and a new random forest model was trained with the inclusion of selected SNPs. These models did not outperform the original GP and PP models. These results show the capability of using ML for in‐season predictions for specific traits in soybean breeding and provide insights on PP and GP inclusions in breeding programs.more » « less
-
Methods based on upward canopy gap fractions are widely employed to measure in-situ effective LAI (Le) as an alternative to destructive sampling. However, these measurements are limited to point-level and are not practical for scaling up to larger areas. To address the point-to-landscape gap, this study introduces an innovative approach, named NeRF-LAI, for corn and soybean Le estimation that combines gap-fraction theory with the neural radiance field (NeRF) technology, an emerging neural network-based method for implicitly representing 3D scenes using multi-angle 2D images. The trained NeRF-LAI can render downward photorealistic hemispherical depth images from an arbitrary viewpoint in the 3D scene, and then calculate gap fractions to estimate Le. To investigate the intrinsic difference between upward and downward gaps estimations, initial tests on virtual corn fields demonstrated that the downward Le matches well with the upward Le, and the viewpoint height is insensitive to Le estimation for a homogeneous field. Furthermore, we conducted intensive real-world experiments at controlled plots and farmer-managed fields to test the effectiveness and transferability of NeRF-LAI in real-world scenarios, where multi-angle UAV oblique images from different phenological stages were collected for corn and soybeans. Results showed the NeRF-LAI is able to render photorealistic synthetic images with an average peak signal-to-noise ratio (PSNR) of 18.94 for the controlled corn plots and 19.10 for the controlled soybean plots. We further explored three methods to estimate Le from calculated gap fractions: the 57.5° method, the five-ring-based method, and the cell-based method. Among these, the cell-based method achieved the best performance, with the r2 ranging from 0.674 to 0.780 and RRMSE ranging from 1.95 % to 5.58 %. The Le estimates are sensitive to viewpoint height in heterogeneous fields due to the difference in the observable foliage volume, but they exhibit less sensitivity to relatively homogeneous fields. Additionally, the cross-site testing for pixel-level LAI mapping showed the NeRF-LAI significantly outperforms the VI-based models, with a small variation of RMSE (0.71 to 0.95 m2/m2) for spatial resolution from 0.5 m to 2.0 m. This study extends the application of gap fraction-based Le estimation from a discrete point scale to a continuous field scale by leveraging implicit 3D neural representations learned by NeRF. The NeRF-LAI method can map Le from raw multi-angle 2D images without prior information, offering a potential alternative to the traditional in-situ plant canopy analyzer with a more flexible and efficient solution.more » « less
