Knowing the displacement capacity and mobility patterns of industrially exploited (i.e., fished) marine resources is key to establishing effective conservation management strategies in human-impacted marine ecosystems. Acquiring accurate behavioral information of deep-sea fished ecosystems is necessary to establish the sizes of marine protected areas within the framework of large international societal programs (e.g., European Community H2020, as part of the Blue Growth economic strategy). However, such information is currently scarce, and high-frequency and prolonged data collection is rarely available. Here, we report the implementation of autonomous underwater vehicles and remotely operated vehicles as an aid for acoustic long-baseline localization systems for autonomous tracking of Norway lobster (
In-situ visual observations of marine organisms is crucial to developing behavioural understandings and their relations to their surrounding ecosystem. Typically, these observations are collected via divers, tags, and remotely-operated or human-piloted vehicles. Recently, however, autonomous underwater vehicles equipped with cameras and embedded computers with GPU capabilities are being developed for a variety of applications, and in particular, can be used to supplement these existing data collection mechanisms where human operation or tags are more difficult. Existing approaches have focused on using fully-supervised tracking methods, but labelled data for many underwater species are severely lacking. Semi-supervised trackers may offer alternative tracking solutions because they require less data than fully-supervised counterparts. However, because there are not existing realistic underwater tracking datasets, the performance of semi-supervised tracking algorithms in the marine domain is not well understood. To better evaluate their performance and utility, in this paper we provide (1) a novel dataset specific to marine animals located at
- Award ID(s):
- 2133029
- NSF-PAR ID:
- 10399704
- Publisher / Repository:
- Springer Science + Business Media
- Date Published:
- Journal Name:
- International Journal of Computer Vision
- Volume:
- 131
- Issue:
- 6
- ISSN:
- 0920-5691
- Page Range / eLocation ID:
- p. 1406-1427
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Nephrops norvegicus ), one of the key living resources exploited in European waters. In combination with seafloor moored acoustic receivers, we detected and tracked the movements of 33 tagged lobsters at 400-m depth for more than 3 months. We also identified the best procedures to localize both the acoustic receivers and the tagged lobsters, based on algorithms designed for off-the-shelf acoustic tags identification. Autonomous mobile platforms that deliver data on animal behavior beyond traditional fixed platform capabilities represent an advance for prolonged, in situ monitoring of deep-sea benthic animal behavior at meter spatial scales. -
more » « less
Abstract Imaging underwater environments is of great importance to marine sciences, sustainability, climatology, defense, robotics, geology, space exploration, and food security. Despite advances in underwater imaging, most of the ocean and marine organisms remain unobserved and undiscovered. Existing methods for underwater imaging are unsuitable for scalable, long-term, in situ observations because they require tethering for power and communication. Here we describe underwater backscatter imaging, a method for scalable, real-time wireless imaging of underwater environments using fully-submerged battery-free cameras. The cameras power up from harvested acoustic energy, capture color images using ultra-low-power active illumination and a monochrome image sensor, and communicate wirelessly at net-zero-power via acoustic backscatter. We demonstrate wireless battery-free imaging of animals, plants, pollutants, and localization tags in enclosed and open-water environments. The method’s self-sustaining nature makes it desirable for massive, continuous, and long-term ocean deployments with many applications including marine life discovery, submarine surveillance, and underwater climate change monitoring.
-
more » « less
Abstract Global biotic and abiotic threats, particularly from pervasive human activities, are progressively pushing large, apex carnivorous mammals into the functional role of mesopredator. Hunters are now becoming the hunted. Despite marked impacts on these animals and the ecosystems in which they live, little is known about the physiological repercussions of this role downgrading from ultimate to penultimate predator.
Here we examine how such ecological role reversals alter the physiological processes associated with energy expenditure, and ultimately the cost of survival during peak performance.
Taxonomic group, preferred habitat and domestication affected the capacity of the oxygen pathway to support high levels of aerobic performance by carnivorous mammals. Fear responses associated with anthropogenic threats also impacted aerobic performance.
Allometric trends for three energetic metrics [maximum oxygen consumption, field metabolic rates (FMRs) and the cost per stride or stroke], showed distinct trends in aerobic capacity for different evolutionary lineages of mammalian predators. Cursorial canids that chase down prey demonstrated the highest relative maximum oxygen consumption rates (10–25 times resting levels) and FMRs, while ambush predators (i.e. felids) and diving marine mammals had aerobic capacities that were similar to or lower than sedentary domestic mammals of comparable size.
The maximum energetic cost of performance for apex predators depended on whether the animals were hunters or the hunted. Escape responses were exceptionally costly for marine (narwhal
Monodon monoceros ) and terrestrial (mountain lionPuma concolor ) locomotor specialists, as well as semi‐aquatic (polar bearUrsus maritimus ) species; all showed a nearly two‐fold increase in peak energy expenditure when avoiding threats.As the duration and frequency of threats to wild species continue to grow, cumulative energetic costs are becoming more apparent. In view of this, attention to the energy demands of apex predators will provide vital predictive power to anticipate mismatches between a species' functional design and human‐induced pressures, and allow for the development of conservation strategies based on how species are built to survive.
A free
Plain Language Summary can be found within the Supporting Information of this article. -
Gonzalez, D. (Ed.)more » « less
Today’s research on human-robot teaming requires the ability to test artificial intelligence (AI) algorithms for perception and decision-making in complex real-world environments. Field experiments, also referred to as experiments “in the wild,” do not provide the level of detailed ground truth necessary for thorough performance comparisons and validation. Experiments on pre-recorded real-world data sets are also significantly limited in their usefulness because they do not allow researchers to test the effectiveness of active robot perception and control or decision strategies in the loop. Additionally, research on large human-robot teams requires tests and experiments that are too costly even for the industry and may result in considerable time losses when experiments go awry. The novel Real-Time Human Autonomous Systems Collaborations (RealTHASC) facility at Cornell University interfaces real and virtual robots and humans with photorealistic simulated environments by implementing new concepts for the seamless integration of wearable sensors, motion capture, physics-based simulations, robot hardware and virtual reality (VR). The result is an extended reality (XR) testbed by which real robots and humans in the laboratory are able to experience virtual worlds, inclusive of virtual agents, through real-time visual feedback and interaction. VR body tracking by DeepMotion is employed in conjunction with the OptiTrack motion capture system to transfer every human subject and robot in the real physical laboratory space into a synthetic virtual environment, thereby constructing corresponding human/robot avatars that not only mimic the behaviors of the real agents but also experience the virtual world through virtual sensors and transmit the sensor data back to the real human/robot agent, all in real time. New cross-domain synthetic environments are created in RealTHASC using Unreal Engine™, bridging the simulation-to-reality gap and allowing for the inclusion of underwater/ground/aerial autonomous vehicles, each equipped with a multi-modal sensor suite. The experimental capabilities offered by RealTHASC are demonstrated through three case studies showcasing mixed real/virtual human/robot interactions in diverse domains, leveraging and complementing the benefits of experimentation in simulation and in the real world.
-
null (Ed.)The ocean is a vast three-dimensional space that is poorly explored and understood, and harbors unobserved life and processes that are vital to ecosystem function. To fully interrogate the space, novel algorithms and robotic platforms are required to scale up observations. Locating animals of interest and extended visual observations in the water column are particularly challenging objectives. Towards that end, we present a novel Machine Learning-integrated Tracking (or ML-Tracking) algorithm for underwater vehicle control that builds on the class of algorithms known as tracking-by-detection. By coupling a multi-object detector (trained on in situ underwater image data), a 3D stereo tracker, and a supervisor module to oversee the mission, we show how ML-Tracking can create robust tracks needed for long duration observations, as well as enable fully automated acquisition of objects for targeted sampling. Using a remotely operated vehicle as a proxy for an autonomous underwater vehicle, we demonstrate continuous input from the ML-Tracking algorithm to the vehicle controller during a record, 5+ hr continuous observation of a midwater gelatinous animal known as a siphonophore. These efforts clearly demonstrate the potential that tracking-by-detection algorithms can have on exploration in unexplored environments and discovery of undiscovered life in our ocean.more » « less
