Search for: All records

Abstract Aerial photogrammetry is a popular non‐invasive tool to measure the size, body morphometrics and body condition of wild animals. While the method can generate large datasets quickly, the lack of efficient processing tools can create bottlenecks that delay management actions. We developed a machine learning algorithm to automatically measure body morphometrics (body length and widths) of southern right whales (Eubalaena australis, SRWs) from aerial photographs (n = 8,958) collected by unmanned aerial vehicles in Australia. Our approach utilizes two Mask R‐CNN detection models to: (i) generate masks for each whale and (ii) estimate points along the whale's axis. We annotated a dataset of 468 images containing 638 whales to train our models. To evaluate the accuracy of our machine learning approach, we compared the model‐generated body morphometrics to manual measurements. The influence of picture quality (whale posture and water clarity) was also assessed. The model‐generated body length estimates were slightly negatively biased (median error of −1.3%), whereas the body volume estimates had a small (median error of 6.5%) positive bias. After correcting both biases, the resulting model‐generated body length and volume estimates had mean absolute errors of 0.85% (SD = 0.75) and 6.88% (SD = 6.57), respectively. The magnitude of the errors decreased as picture quality increased. When using the model‐generated data to quantify intra‐seasonal changes in body condition of SRW females, we obtained a similar slope parameter (−0.001843, SE = 0.000095) as derived from manual measurements (−0.001565, SE = 0.000079). This indicates that our approach was able to accurately capture temporal trends in body condition at a population level. 

The widespread availability of off-the-shelf machine learning models poses a challenge: which model, of the many available candidates, should be chosen for a given data analysis task? This question of model selection is traditionally answered by collecting and annotating a validation dataset—a costly and time-intensive process. We propose a method for active model selection, using predictions from candidate models to prioritize the labeling of test data points that efficiently differentiate the best candidate. Our method, CODA, performs consensus-driven active model selection by modeling relationships between classifiers, categories, and data points within a probabilistic framework. The framework uses the consensus and disagreement between models in the candidate pool to guide the label acquisition process, and Bayesian inference to update beliefs about which model is best as more information is collected. We validate our approach by curating a collection of 26 benchmark tasks capturing a range of model selection scenarios. CODA outperforms existing methods for active model selection significantly, reducing the annotation effort required to discover the best model by upwards of 70% compared to the previous state-of-the-art. Code and data are available at: https://github.com/justinkay/coda.