skip to main content

Title: Collections Management and High-Throughput Digitization using Distributed Cyberinfrastructure Resources
Collections digitization relies increasingly upon computational and data management resources that occasionally exceed the capacity of natural history collections and their managers and curators. Digitization of many tens of thousands of micropaleontological specimen slides, as evidenced by the effort presented here by the Indiana University Paleontology Collection, has been a concerted effort in adherence to the recommended practices of multifaceted aspects of collections management for both physical and digital collections resources. This presentation highlights the contributions of distributed cyberinfrastructure from the National Science Foundation-supported Extreme Science and Engineering Discovery Environment (XSEDE) for web-hosting of collections management system resources and distributed processing of millions of digital images and metadata records of specimens from our collections. The Indiana University Center for Biological Research Collections is currently hosting its instance of the Specify collections management system (CMS) on a virtual server hosted on Jetstream, the cloud service for on-demand computational resources as provisioned by XSEDE. This web-service allows the CMS to be flexibly hosted on the cloud with additional services that can be provisioned on an as-needed basis for generating and integrating digitized collections objects in both web-friendly and digital preservation contexts. On-demand computing resources can be used for the manipulation of digital images for automated file I/O, scripted renaming of files for adherence to file naming conventions, derivative generation, and backup to our local tape archive for digital disaster preparedness and long-term storage. Here, we will present our strategies for facilitating reproducible workflows for general collections digitization of the IUPC nomenclatorial types and figured specimens in addition to the gigapixel resolution photographs of our large collection of microfossils using our GIGAmacro system (e.g., this slide of conodonts). We aim to demonstrate the flexibility and nimbleness of cloud computing resources for replicating this, and other, workflows to enhance the findability, accessibility, interoperability, and reproducibility of the data and metadata contained within our collections.  more » « less
Award ID(s):
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
Biodiversity Information Science and Standards
Page Range / eLocation ID:
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Natural history collections are often considered remote and inaccessible without special permission from curators. Digitization of these collections can make them much more accessible to researchers, educators, and general enthusiasts alike, thereby removing the stigma of a lonely specimen on a dusty shelf in the back room of a museum that will never again see the light of day. We are in the process of digitizing the microfossils of the Indiana University Paleontology collection using the GIGAmacro Magnify2 Robotic Imaging System. This suite of software and hardware allows us to automate photography and post-production of high resolution images, thereby severely reducing the amount of time and labor needed to serve the data. Our hardware includes a Canon T6i 24 megapixel DSLR, a Canon MPE 65mm 1X to 5X lens, and a Canon MT26EX Dual Flash, all mounted on a lead system made with high performance precision IGUS Drylin anodized aluminum. The camera and its mount move over the tray of microfossil slides using bearings and rails. The software includes the GIGAmacro Capture Software (photography), GIGAmacro Viewer Software (display and annotation), Zerene Stacker (focus stacking), and Autopano GIGA (stitching). All of the metadata is kept in association with the images, uploaded to Notes from Nature, transcribed by community scientists, then everything is stored in the image archive, Imago. In ~460 hours we have photographed ~10,500 slides and have completed ~65% of our microfossil collection. Using the GIGAmacro system we are able update and store collection information in a more secure and longer lasting digital form. The advantages of this system are numerable and highly recommended for museums who are looking to bring their collections out of the shadows and back into the light. 
    more » « less
  2. Elmer Ottis Wooton (1865–1945) was one of the most important early botanists to work in the Southwestern United States, contributing a great deal of natural history knowledge and botanical research on the flora of New Mexico that shaped many naturalists and scientists for generations. The extensive Wooton legacy includes herbarium collections that he and his famous student Paul Carpenter Standley (1884–1963), prolific botanist and explorer, used for the first Flora of New Mexi co by Wooten and Standley 1915 , along with resources covering botany and range management strategies for the northern Chihuahuan Desert, and an extensive, yet to be digitized, historical archive of correspondence, field notes, vegetation sketches, photographs, and lantern slides, all from his travels and field work in the region. Starting in 1890, the most complete set of Wooton’s herbarium collections were deposited in the NMC herbarium at New Mexico State University (NMSU), and his archives, now stored in a Campus library, have together been underutilized, offline resources. The goals of this ongoing project are to secure, preserve, and promote Wooton’s important historical resources, by fleshing out the botanical history of the region, raising appreciation of herbarium collections within the community, and emphasizing their unique role in facilitating contemporary research aimed at addressing pressing scientific questions such as vegetation responses to global climate change. Students and the general public involved in this project are engaged through hands-on activities including cataloging, databasing and digitization of nearly 10,000 herbarium specimens and Wooton’s archives. These outputs, combined with contemporary data collection and computational biology techniques from an ecological perspective, are being used to document vegetation changes in iconic, climate-sensitive, high-elevation mountainous ecosystems present in southwestern New Mexico. In a later phase of the project, a variety of public audiences will participate through interactive online story maps and citizen science programs such as iNaturalist , Notes from Nature , and BioBlitz . Images of herbarium specimens will be shared via an online database and other relevant biodiversity portals ( Symbiota , iDigBio , JStor ) Community members reached through this project will be better-informed citizens, who may go on to become new stewards of natural history collections, with the potential to influence policies safeguarding the future of our planet’s biodiversity. More locally, the project will support the management of Organ Mountains Desert Peaks National Monument, which was established in 2014 to protect the area's human and environmental resources, and for which knowledge and data are currently limited. 
    more » « less
  3. PmagPy Online: Jupyter Notebooks, the PmagPy Software Package and the Magnetics Information Consortium (MagIC) Database Lisa Tauxe$^1$, Rupert Minnett$^2$, Nick Jarboe$^1$, Catherine Constable$^1$, Anthony Koppers$^2$, Lori Jonestrask$^1$, Nick Swanson-Hysell$^3$ $^1$Scripps Institution of Oceanography, United States of America; $^2$ Oregon State University; $^3$ University of California, Berkely; The Magnetics Information Consortium (MagIC), hosted at is a database that serves as a Findable, Accessible, Interoperable, Reusable (FAIR) archive for paleomagnetic and rock magnetic data. It has a flexible, comprehensive data model that can accomodate most kinds of paleomagnetic data. The PmagPy software package is a cross-platform and open-source set of tools written in Python for the analysis of paleomagnetic data that serves as one interface to MagIC, accommodating various levels of user expertise. It is available through Because PmagPy requires installation of Python, several non-standard Python modules, and the PmagPy software package, there is a speed bump for many practitioners on beginning to use the software. In order to make the software and MagIC more accessible to the broad spectrum of scientists interested in paleo and rock magnetism, we have prepared a set of Jupyter notebooks, hosted on which serve a set of purposes. 1) There is a complete course in Python for Earth Scientists, 2) a set of notebooks that introduce PmagPy (pulling the software package from the github repository) and illustrate how it can be used to create data products and figures for typical papers, and 3) show how to prepare data from the laboratory to upload into the MagIC database. The latter will satisfy expectations from NSF for data archiving and for example the AGU publication data archiving requirements. Getting started To use the PmagPy notebooks online, go to website at Create an Earthref account using your ORCID and log on. [This allows you to keep files in a private work space.] Open the PmagPy Online - Setup notebook and execute the two cells. Then click on File = > Open and click on the PmagPy_Online folder. Open the PmagPy_online notebook and work through the examples. There are other notebooks that are useful for the working paleomagnetist. Alternatively, you can install Python and the PmagPy software package on your computer (see for instructions). Follow the instructions for "Full PmagPy install and update" through section 1.4 (Quickstart with PmagPy notebooks). This notebook is in the collection of PmagPy notebooks. Overview of MagIC The Magnetics Information Consortium (MagIC), hosted at is a database that serves as a Findable, Accessible, Interoperable, Reusable (FAIR) archive for paleomagnetic and rock magnetic data. Its datamodel is fully described here: Each contribution is associated with a publication via the DOI. There are nine data tables: contribution: metadata of the associated publication. locations: metadata for locations, which are groups of sites (e.g., stratigraphic section, region, etc.) sites: metadata and derived data at the site level (units with a common expectation) samples: metadata and derived data at the sample level. specimens: metadata and derived data at the specimen level. criteria: criteria by which data are deemed acceptable ages: ages and metadata for sites/samples/specimens images: associated images and plots. Overview of PmagPy The functionality of PmagPy is demonstrated within notebooks in the PmagPy repository: PmagPy_online.ipynb: serves as an introdution to PmagPy and MagIC (this conference). It highlights the link between PmagPy and the Findable Accessible Interoperable Reusabe (FAIR) database maintained by the Magnetics Information Consortium (MagIC) at Other notebooks of interest are: PmagPy_calculations.ipynb: demonstrates many of the PmagPy calculation functions such as those that rotate directions, return statistical parameters, and simulate data from specified distributions. PmagPy_plots_analysis.ipynb: demonstrates PmagPy functions that can be used to visual data as well as those that conduct statistical tests that have associated visualizations. PmagPy_MagIC.ipynb: demonstrates how PmagPy can be used to read and write data to and from the MagIC database format including conversion from many individual lab measurement file formats. Please see also our YouTube channel with more presentations from the 2020 MagIC workshop here: 
    more » « less
  4. Premise

    The digitization of natural history collections includes transcribing specimen label data into standardized formats. Born‐digital specimen data initially gathered in digital formats do not need to be transcribed, enabling their efficient integration into digitized collections. Modernizing field collection methods for born‐digital workflows requires the development of new tools and processes.

    Methods and Results

    collNotes, a mobile application, was developed for Android andiOSto supplement traditional field journals. Designed for efficiency in the field, collNotes avoids redundant data entries and does not require cellular service. collBook, a companion desktop application, refines field notes into database‐ready formats and produces specimen labels.


    collNotes and collBook can be used in combination as a field‐to‐database solution for gathering born‐digital voucher specimen data for plants and fungi. Both programs are open source and use common file types simplifying either program's integration into existing workflows.

    more » « less
  5. Adam, N. ; Neuhold, E. ; Furuta, R. (Ed.)
    Metadata is a key data source for researchers seeking to apply machine learning (ML) to the vast collections of digitized biological specimens that can be found online. Unfortunately, the associated metadata is often sparse and, at times, erroneous. This paper extends previous research conducted with the Illinois Natural History Survey (INHS) collection (7244 specimen images) that uses computational approaches to analyze image quality, and then automatically generates 22 metadata properties representing the image quality and morphological features of the specimens. In the research reported here, we demonstrate the extension of our initial work to University of the Wisconsin Zoological Museum (UWZM) collection (4155 specimen images). Further, we enhance our computational methods in four ways: (1) augmenting the training set, (2) applying contrast enhancement, (3) upscaling small objects, and (4) refining our processing logic. Together these new methods improved our overall error rates from 4.6 to 1.1%. These enhancements also allowed us to compute an additional set of 17 image-based metadata properties. The new metadata properties provide supplemental features and information that may also be used to analyze and classify the fish specimens. Examples of these new features include convex area, eccentricity, perimeter, skew, etc. The newly refined process further outperforms humans in terms of time and labor cost, as well as accuracy, providing a novel solution for leveraging digitized specimens with ML. This research demonstrates the ability of computational methods to enhance the digital library services associated with the tens of thousands of digitized specimens stored in open-access repositories world-wide by generating accurate and valuable metadata for those repositories. 
    more » « less