skip to main content


Title: Deep Multiview Image Fusion for Soybean Yield Estimation in Breeding Applications
Reliable seed yield estimation is an indispensable step in plant breeding programs geared towards cultivar development in major row crops. The objective of this study is to develop a machine learning (ML) approach adept at soybean ( Glycine max L. (Merr.)) pod counting to enable genotype seed yield rank prediction from in-field video data collected by a ground robot. To meet this goal, we developed a multiview image-based yield estimation framework utilizing deep learning architectures. Plant images captured from different angles were fused to estimate the yield and subsequently to rank soybean genotypes for application in breeding decisions. We used data from controlled imaging environment in field, as well as from plant breeding test plots in field to demonstrate the efficacy of our framework via comparing performance with manual pod counting and yield estimation. Our results demonstrate the promise of ML models in making breeding decisions with significant reduction of time and human effort and opening new breeding method avenues to develop cultivars.  more » « less
Award ID(s):
1954556
PAR ID:
10319254
Author(s) / Creator(s):
; ; ; ; ; ; ; ; ; ;
Date Published:
Journal Name:
Plant Phenomics
Volume:
2021
ISSN:
2643-6515
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Because the manual counting of soybean ( Glycine max ) plants, pods, and seeds/pods is unsuitable for soybean yield predictions, alternative methods are desired. Therefore, the objective was to determine if satellite remote sensing − based artificial intelligence (AI) models could be used to predict soybean yield. In the study, multiple remote sensing − based AI models were developed for soybean growth stage ranging from VE/VC (plant emergence) to R6/R7 (full seed to beginning maturity). The ability of the Deep Neural Network (DNN), Support Vector Machine (SVM), Random Forest (RF), Least Absolute Shrinkage and Selection Operator (LASSO), and AdaBoost to predict soybean yield, based on blue, green, red, and near infrared reflectance data collected by the PlanetScope satellite at 6 growth stages, was determined. Remote sensing and soybean yield monitor data from 3 different fields in two years (2019 and 2021) were aggregated into 24,282 grid cells that had the dimensions of 10 by 10m. A comparison across models showed that the DNN outperformed the other models. Moreover, as crops matured from VE/VC to R4/R5, the R 2 value of the models increased from 0.26 to over 0.70. These findings indicate that remote sensing data collected at different growth stages can be combined for soybean yield predictions. Moreover, additional work needs to be conducted to assess the model's ability to predict soybean yield with vegetation indices (VI) data for fields not used to train the model. This article is protected by copyright. All rights reserved 
    more » « less
  2. Abstract

    A combination of drought and heat stress, occurring at the vegetative or reproductive growth phase of many different crops can have a devastating impact on yield. In soybean (Glycine max), a considerable effort has been made to develop genotypes with enhanced yield production under conditions of drought or heat stress. However, how these genotypes perform in terms of growth, physiological responses, and most importantly seed production, under conditions of drought and heat combination is mostly unknown. Here, we studied the impact of water deficit and heat stress combination on the physiology, seed production, and yield per plant of two soybean genotypes, Magellan and Plant Introduction (PI) 548313, that differ in their reproductive responses to heat stress. Our findings reveal that although PI 548313 produced more seeds than Magellan under conditions of heat stress, under conditions of water deficit, and heat stress combination its seed production decreased. Because the number of flowers and pollen germination of PI 548313 remained high under heat or water deficit and heat combination, the reduced seed production exhibited by PI 548313 under the stress combination could be a result of processes that occur at the stigma, ovaries and/or other parts of the flower following pollen germination.

     
    more » « less
  3. Abstract Climate change is causing an increase in the frequency and intensity of droughts, heat waves, and their combinations, diminishing agricultural productivity and destabilizing societies worldwide. We recently reported that during a combination of water deficit (WD) and heat stress (HS), stomata on leaves of soybean (Glycine max) plants are closed, while stomata on flowers are open. This unique stomatal response was accompanied by differential transpiration (higher in flowers, while lower in leaves) that cooled flowers during a combination of WD + HS. Here, we reveal that developing pods of soybean plants subjected to a combination of WD + HS use a similar acclimation strategy of differential transpiration to reduce internal pod temperature by approximately 4 °C. We further show that enhanced expression of transcripts involved in abscisic acid degradation accompanies this response and that preventing pod transpiration by sealing stomata causes a significant increase in internal pod temperature. Using an RNA-Seq analysis of pods developing on plants subjected to WD + HS, we also show that the response of pods to WD, HS, or WD + HS is distinct from that of leaves or flowers. Interestingly, we report that although the number of flowers, pods, and seeds per plant decreases under conditions of WD + HS, the seed mass of plants subjected to WD + HS increases compared to plants subjected to HS, and the number of seeds with suppressed/aborted development is lower in WD + HS compared to HS. Taken together, our findings reveal that differential transpiration occurs in pods of soybean plants subjected to WD + HS and that this process limits heat-induced damage to seed production. 
    more » « less
  4. Abstract Motivation

    Developing new crop varieties with superior performance is highly important to ensure robust and sustainable global food security. The speed of variety development is limited by long field cycles and advanced generation selections in plant breeding programs. While methods to predict yield from genotype or phenotype data have been proposed, improved performance and integrated models are needed.

    Results

    We propose a machine learning model that leverages both genotype and phenotype measurements by fusing genetic variants with multiple data sources collected by unmanned aerial systems. We use a deep multiple instance learning framework with an attention mechanism that sheds light on the importance given to each input during prediction, enhancing interpretability. Our model reaches 0.754 ± 0.024 Pearson correlation coefficient when predicting yield in similar environmental conditions; a 34.8% improvement over the genotype-only linear baseline (0.559 ± 0.050). We further predict yield on new lines in an unseen environment using only genotypes, obtaining a prediction accuracy of 0.386 ± 0.010, a 13.5% improvement over the linear baseline. Our multi-modal deep learning architecture efficiently accounts for plant health and environment, distilling the genetic contribution and providing excellent predictions. Yield prediction algorithms leveraging phenotypic observations during training therefore promise to improve breeding programs, ultimately speeding up delivery of improved varieties.

    Availability and implementation

    Available at https://github.com/BorgwardtLab/PheGeMIL (code) and https://doi.org/doi:10.5061/dryad.kprr4xh5p (data).

     
    more » « less
  5. In this paper, we present a method for creating high-quality 3D models of sorghum panicles for phenotyping in breeding experiments. This is achieved with a novel reconstruc- tion approach that uses seeds as semantic landmarks in both 2D and 3D. To evaluate the performance, we develop a new metric for assessing the quality of reconstructed point clouds without ground-truth. Finally, a counting method is presented where the density of seed centers in the 3D model allows 2D counts from multiple views to be effectively combined into a whole-panicle count. We demonstrate that using this method to estimate seed count and weight for sorghum outperforms count extrapolation from 2D images, an approach used in most state of the art methods for seeds and grains of comparable size. 
    more » « less