skip to main content


Title: Tools for integrating inertial sensor data with video bio-loggers, including estimation of animal orientation, motion, and position
Abstract Bio-logging devices equipped with inertial measurement units—particularly accelerometers, magnetometers, and pressure sensors—have revolutionized our ability to study animals as necessary electronics have gotten smaller and more affordable over the last two decades. These animal-attached tags allow for fine scale determination of behavior in the absence of direct observation, particularly useful in the marine realm, where direct observation is often impossible, and recent devices can integrate more power hungry and sensitive instruments, such as hydrophones, cameras, and physiological sensors. To convert the raw voltages recorded by bio-logging sensors into biologically meaningful metrics of orientation (e.g., pitch, roll and heading), motion (e.g., speed, specific acceleration) and position (e.g., depth and spatial coordinates), we developed a series of MATLAB tools and online instructional tutorials. Our tools are adaptable for a variety of devices, though we focus specifically on the integration of video, audio, 3-axis accelerometers, 3-axis magnetometers, 3-axis gyroscopes, pressure, temperature, light and GPS data that are the standard outputs from Customized Animal Tracking Solutions (CATS) video tags. Our tools were developed and tested on cetacean data but are designed to be modular and adaptable for a variety of marine and terrestrial species. In this text, we describe how to use these tools, the theories and ideas behind their development, and ideas and additional tools for applying the outputs of the process to biological research. We additionally explore and address common errors that can occur during processing and discuss future applications. All code is provided open source and is designed to be useful to both novice and experienced programmers.  more » « less
Award ID(s):
1643877 2026045
NSF-PAR ID:
10315596
Author(s) / Creator(s):
; ; ; ; ; ; ; ; ; ;
Date Published:
Journal Name:
Animal Biotelemetry
Volume:
9
Issue:
1
ISSN:
2050-3385
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Temporal accuracy is a fundamental characteristic of logging technology and is needed to correlate data streams. Single biologgers sensing animal movement (accelerometers, gyroscope, magnetometers, collectively inertial measurement unit; IMU) have been extensively used to study the ecology of animals. To better capture whole body movement and increase the accuracy of behavior classification, there is a need to deploy multiple loggers on a single individual to capture the movement of multiple body parts. Yet due to temporal drift, accurately aligning multiple IMU datasets can be problematic, especially as deployment duration increases. In this paper we quantify temporal drift and errors in commercially available IMU data loggers using a combination of robotic and animal borne experiments. The variance in drift rate within a tag is over an order of magnitude lower (σ = 0.001 s h−1) than the variance between tags (σ = 0.015 s·h−1), showing that recording frequency is a characteristic of each tag and not a random variable. Furthermore, we observed a large offset (0.54 ± 0.016 s·h−1) between two groups of tags that had differing recording frequencies, and we observed three instances of instantaneous temporal jumps within datasets introducing errors into the data streams. Finally, we show that relative drift rates can be estimated even when deployed on animals displaying various behaviors without the tags needing to be simultaneously moved. For the tags used in this study, drift rates can vary significantly between tags, are repeatable, and can be accurately measured in the field. The temporal alignment of multiple tag datasets allows researchers to deploy multiple tags on an individual animal which will greatly increase our knowledge of movement kinematics and expand the range of movement characteristics that can be used for behavioral classification.

     
    more » « less
  2. null (Ed.)
    Smart bracelets able to interpret the wearer's emotional state and communicate it to a remote decision-support facility will have broad applications in healthcare, elder care, the military, and other fields. While there are existing commercial embedded devices, such as the Apple Watch, that have health-monitoring sensors, such devices cannot sufficiently support a real-time health-monitoring system with battery-efficient remote data delivery. Ongoing R&D is developing solutions capable of monitoring multiple psycho-physiological signals. Possible hardware configurations include wrist-worn devices and sensors across an augmented reality headset (e.g., HoloLens 2). The device should carry an array of sensors of psycho-physiological signals, including a galvanic skin response sensor, motion sensor, skin temperature sensor, and a heart rate sensor. Output from these sensors can be intelligently fused to monitor the affective state and to determine specific trigger events for the wearer. To enable real-time remote monitoring applications, the device needs to be low-power to allow persistent monitoring while prolonging usage before recharging. For many applications, specialized sensor arrays are required, e.g. a galvanic skin response sensor. An application-flexible device would allow adding/removing sensors and would provide a choice of communication modules (e.g., Bluetooth 5.0 low-energy vs ZigBee). Appropriate configurations of the device would support applications in military health monitoring, drug-addiction mitigation, autistic trigger monitoring, and augmented reality exploration. A configuration example is: motion sensors (3-axis accelerometers, gyroscopes, and magnetometers to track steps, falls, and energy usage), a heart-rate sensor (e.g., an optical-based heart rate sensor with a single monitoring zone using the process of photoplethysmography (PPS)), at least a Bluetooth 5.0 (but a different communication device may be needed depending on the use case), and flash memory to temporarily store data when the device is not remotely communicating. The wearables field has greatly advanced in the quality of sensors; the fusion of multi-sensor data is the current frontier. 
    more » « less
  3. Abstract Background Inertial measurement units (IMUs) with high-resolution sensors such as accelerometers are now used extensively to study fine-scale behavior in a wide range of marine and terrestrial animals. Robust and practical methods are required for the computationally-demanding analysis of the resulting large datasets, particularly for automating classification routines that construct behavioral time series and time-activity budgets. Magnetometers are used increasingly to study behavior, but it is not clear how these sensors contribute to the accuracy of behavioral classification methods. Development of effective  classification methodology is key to understanding energetic and life-history implications of foraging and other behaviors. Methods We deployed accelerometers and magnetometers on four species of free-ranging albatrosses and evaluated the ability of unsupervised hidden Markov models (HMMs) to identify three major modalities in their behavior: ‘flapping flight’, ‘soaring flight’, and ‘on-water’. The relative contribution of each sensor to classification accuracy was measured by comparing HMM-inferred states with expert classifications identified from stereotypic patterns observed in sensor data. Results HMMs provided a flexible and easily interpretable means of classifying behavior from sensor data. Model accuracy was high overall (92%), but varied across behavioral states (87.6, 93.1 and 91.7% for ‘flapping flight’, ‘soaring flight’ and ‘on-water’, respectively). Models built on accelerometer data alone were as accurate as those that also included magnetometer data; however, the latter were useful for investigating slow and periodic behaviors such as dynamic soaring at a fine scale. Conclusions The use of IMUs in behavioral studies produces large data sets, necessitating the development of computationally-efficient methods to automate behavioral classification in order to synthesize and interpret underlying patterns. HMMs provide an accessible and robust framework for analyzing complex IMU datasets and comparing behavioral variation among taxa across habitats, time and space. 
    more » « less
  4. Abstract

    Current methods for identifying and predicting infectious disease dynamics in wildlife populations are limited. Pathogen transmission dynamics can be complex, influenced by behavioural interactions between and among hosts, pathogens and their environments. These behaviours may also be influenced directly by observers, with observational research methods being limited to habituated species. Banded mongooseMungos mungoare social, medium size carnivores infected with the novel tuberculosis pathogenMycobacterium mungi. This pathogen is principally transmitted during normal olfactory communication behaviours. Banded mongoose behavioural responses to humans change over the landscape, limiting the use of direct observational approaches in areas where mongoose are threatened and flee.

    The accelerometers in bio‐logging devices have been used previously to identify distinct behaviours in wildlife species, providing a tool to quantifying specific behaviours in ecological studies. We deployed Axy‐5X model accelerometers (TechnoSmArt) on captive mongoose to determine whether accelerometers could be used to identify key mongoose behavioural activities previously associated withM. mungitransmission.

    After two collaring periods, we determined that three distinct behavioural activities could be identified in the accelerometer data: bipedal vertical vigilance, running and scent marking activity; behaviours that have been shown to vary across land type in the banded mongoose.

    Results from this work advance current data analytics and provide modifications to data analysis works flows, updating and expanding upon current methodologies. We also provide preliminary evidence of successful mathematical classification of the target behaviours, supporting the future use of these devices. Methods applied here may allow model estimates ofM. mungitransmission in free‐ranging mongoose to be refined with possible application to other systems where direct observation approaches have limited application.

     
    more » « less
  5. What new questions could ecophysiologists answer if physio-logging research was fully reproducible? We argue that technical debt (computational hurdles resulting from prioritizing short-term goals over long-term sustainability) stemming from insufficient cyberinfrastructure (field-wide tools, standards, and norms for analyzing and sharing data) trapped physio-logging in a scientific silo. This debt stifles comparative biological analyses and impedes interdisciplinary research. Although physio-loggers (e.g., heart rate monitors and accelerometers) opened new avenues of research, the explosion of complex datasets exceeded ecophysiology’s informatics capacity. Like many other scientific fields facing a deluge of complex data, ecophysiologists now struggle to share their data and tools. Adapting to this new era requires a change in mindset, from “data as a noun” (e.g., traits, counts) to “data as a sentence”, where measurements (nouns) are associate with transformations (verbs), parameters (adverbs), and metadata (adjectives). Computational reproducibility provides a framework for capturing the entire sentence. Though usually framed in terms of scientific integrity, reproducibility offers immediate benefits by promoting collaboration between individuals, groups, and entire fields. Rather than a tax on our productivity that benefits some nebulous greater good, reproducibility can accelerate the pace of discovery by removing obstacles and inviting a greater diversity of perspectives to advance science and society. In this article, we 1) describe the computational challenges facing physio-logging scientists and connect them to the concepts of technical debt and cyberinfrastructure , 2) demonstrate how other scientific fields overcame similar challenges by embracing computational reproducibility, and 3) present a framework to promote computational reproducibility in physio-logging, and bio-logging more generally. 
    more » « less