Atomic force microscopy (AFM) provides a platform for high-resolution topographical imaging and the mechanical characterization of a wide range of samples, including live cells, proteins, and other biomolecules. AFM is also instrumental for measuring interaction forces and binding kinetics for protein–protein or receptor–ligand interactions on live cells at a single-molecule level. However, performing force measurements and high-resolution imaging with AFM and data analytics are time-consuming and require special skill sets and continuous human supervision. Recently, researchers have explored the applications of artificial intelligence (AI) and deep learning (DL) in the bioimaging field. However, the applications of AI to AFM operations for live-cell characterization are little-known. In this work, we implemented a DL framework to perform automatic sample selection based on the cell shape for AFM probe navigation during AFM biomechanical mapping. We also established a closed-loop scanner trajectory control for measuring multiple cell samples at high speed for automated navigation. With this, we achieved a 60× speed-up in AFM navigation and reduced the time involved in searching for the particular cell shape in a large sample. Our innovation directly applies to many bio-AFM applications with AI-guided intelligent automation through image data analysis together with smart navigation.
more »
« less
Remote Instrumentation Science Environment for Intelligent Image Analytics
Current scientific experiments frequently involve
control of specialized instruments (e.g., scanning electron mi-
croscopes), image data collection from those instruments, and
transfer of the data for processing at simulation centers. This
process requires a “human-in-the-loop” to perform those tasks
manually, which besides requiring a lot of effort and time,
could lead to inconsistencies or errors. Thus, it is essential
to have an automated system capable of performing remote
instrumentation to intelligently control and collect data from
the scientific instruments. In this paper, we propose a Remote
Instrumentation Science Environment (RISE) for intelligent im-
age analytics that provides the infrastructure to securely capture
images, determine process parameters via machine learning, and
provide experimental control actions via automation, under the
premise of “human-on-the-loop”. The machine learning in RISE
aids an iterative discovery process to assist researchers to tune
instrument settings to improve the outcomes of experiments.
Driven by two scientific use cases of image analytics pipelines,
one in material science, and another in biomedical science, we
show how RISE automation leverages a cutting-edge integration
of cloud computing, on-premise HPC cluster, and a Python
programming interface available on a microscope. Using web
services, we implement RISE to perform automated image data
collection/analysis guided by an intelligent agent to provide
real-time feedback control of the microscope using the image
analytics outputs. Our evaluation results show the benefits of
RISE for researchers to obtain higher image analytics accuracy,
save precious time in manually controlling the microscopes, while
reducing errors in operating the instruments.
more »
« less
- Award ID(s):
- 2026847
- PAR ID:
- 10441258
- Date Published:
- Journal Name:
- 2022 IEEE 18th International Conference on e-Science
- Page Range / eLocation ID:
- 84-94
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
Abstract A core goal of the National Ecological Observatory Network (NEON) is to measure changes in biodiversity across the 30‐yr horizon of the network. In contrast to NEON’s extensive use of automated instruments to collect environmental data, NEON’s biodiversity surveys are almost entirely conducted using traditional human‐centric field methods. We believe that the combination of instrumentation for remote data collection and machine learning models to process such data represents an important opportunity for NEON to expand the scope, scale, and usability of its biodiversity data collection while potentially reducing long‐term costs. In this manuscript, we first review the current status of instrument‐based biodiversity surveys within the NEON project and previous research at the intersection of biodiversity, instrumentation, and machine learning at NEON sites. We then survey methods that have been developed at other locations but could potentially be employed at NEON sites in future. Finally, we expand on these ideas in five case studies that we believe suggest particularly fruitful future paths for automated biodiversity measurement at NEON sites: acoustic recorders for sound‐producing taxa, camera traps for medium and large mammals, hydroacoustic and remote imagery for aquatic diversity, expanded remote and ground‐based measurements for plant biodiversity, and laboratory‐based imaging for physical specimens and samples in the NEON biorepository. Through its data science‐literate staff and user community, NEON has a unique role to play in supporting the growth of such automated biodiversity survey methods, as well as demonstrating their ability to help answer key ecological questions that cannot be answered at the more limited spatiotemporal scales of human‐driven surveys.
-
Schmidt, Dirk ; Schreiber, Laura ; Vernet, Elise (Ed.)Uncorrected residual wavefront errors limit the ultimate performance of adaptive optics (AO) systems. We present different contributing factors and techniques to estimate and compensate these wavefront errors in the Keck natural guide star (NGS) AO systems. The error terms include low order static and semi-static aberrations from multiple sources, periodic and random segment piston errors, single-segment low order aberrations, wavefront sensor aliasing, vibrations, calibration drifts, and AO-to-telescope offload related errors. We present the design of a new AO subsystem, a residual wavefront controller (rWFC) to monitor the performance of the AO control loops and the image quality of the AO science instruments and apply the necessary changes to the telescope and AO parameters to minimize the residual wavefront errors. The distributed system consists of components at the telescope, AO bench and the science instruments. A few components of this system are already tested as on-demand standalone tools and will be integrated into a high-level graphical user interface (GUI) to operate the system. The software tool will periodically collect AO telemetry data, perform control loop parameter optimization and update AO parameters such as loop gains, centroid gain, etc. In addition, the system will analyze the science data at the end of each exposure and estimate telescope/AO performance when a bright point source is available in the science field. The benefits of reducing or eliminating the residual wavefront errors have broad implications for optical astronomy. Testing these techniques on a segmented telescope will be extremely useful to the teams developing high contrast AO systems for all extremely large telescopes and future segmented space telescopes.more » « less
-
Scientific instruments have long been a vital part of science, paving pathways to remarkable scientific advancements. Such advancements have changed the world both socially and culturally, especially in the past few decades. Students can be introduced to this idea through the concepts of nature of science (NOS): scientific observations are often filtered through apparatus, inferences can be made through observations, and science is a socially and culturally embedded practice. The curriculum often fails to emphasize the role of instruments in scientific practices, even in teaching laboratories. This study uses semistructured interviews to investigate the cognitive (thoughts) and affective (feelings) domains of first-year university students as they relate to scientific instrumentation, including students’ ideas of instruments. First, the study probed how general chemistry students conceptualize scientific instruments in relation to the three NOS notions. Second, students’ practices related to experimental data evaluation were investigated as data collection is a large part of psychomotor learning in laboratory. Third, students’ affective states toward learning about instruments were queried. The interview results suggested that a majority of participants acknowledge some ideas of NOS, while a few students displayed an advanced understanding when discussing scientific instruments and also tended to have higher interest and motivation toward learning about instruments.more » « less
-
more » « less
Automation transformed various aspects of our human civilization, revolutionizing industries and streamlining processes. In the domain of scientific inquiry, automated approaches emerged as powerful tools, holding promise for accelerating discovery, enhancing reproducibility, and overcoming the traditional impediments to scientific progress. This article evaluates the scope of automation within scientific practice and assesses recent approaches. Furthermore, it discusses different perspectives to the following questions: where do the greatest opportunities lie for automation in scientific practice?; What are the current bottlenecks of automating scientific practice?; and What are significant ethical and practical consequences of automating scientific practice? By discussing the motivations behind automated science, analyzing the hurdles encountered, and examining its implications, this article invites researchers, policymakers, and stakeholders to navigate the rapidly evolving frontier of automated scientific practice.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/eScience55777.2022.00023
