skip to main content


Title: Performance metrics to unleash the power of self-driving labs in chemistry and materials science
Abstract

With the rise of self-driving labs (SDLs) and automated experimentation across chemical and materials sciences, there is a considerable challenge in designing the best autonomous lab for a given problem based on published studies alone. Determining what digital and physical features are germane to a specific study is a critical aspect of SDL design that needs to be approached quantitatively. Even when controlling for features such as dimensionality, every experimental space has unique requirements and challenges that influence the design of the optimal physical platform and algorithm. Metrics such as optimization rate are therefore not necessarily indicative of the capabilities of an SDL across different studies. In this perspective, we highlight some of the critical metrics for quantifying performance in SDLs to better guide researchers in implementing the most suitable strategies. We then provide a brief review of the existing literature under the lens of quantified performance as well as heuristic recommendations for platform and experimental space pairings.

 
more » « less
Award ID(s):
1940959 2208406
NSF-PAR ID:
10490860
Author(s) / Creator(s):
;
Publisher / Repository:
Nature Publishing Group
Date Published:
Journal Name:
Nature Communications
Volume:
15
Issue:
1
ISSN:
2041-1723
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Floorplans often require considering numerous factors, from the layout size to cost, numeric attributes such as room sizes, and other intrinsic properties such as connectivity between visible regions. Representing these complex factors is challenging, but doing so in a representative and efficient way can enable new modes of design exploration. Existing image and graph-based approaches of floorplans’ representation often failed to consider low-level space semantics, structural features, and space utilization with respect to its future inhabitants, which are all the critical elements to analyze design layouts. We present a latent-space representation of floorplans using gated recurrent unit variational autoencoder (GRU-VAE), where floorplans are represented as attributed graphs (encoded with the abovementioned features). Two local search approaches are presented to efficiently explore the latent space for optimizing and generating new floorplans for the given environment. Semantic, structural, and visibility metrics are evaluated individually and as a combined objective for optimizations.

     
    more » « less
  2. null (Ed.)
    Modern cache hierarchies are tangled webs of complexity. Multiple tiers of heterogeneous physical and virtual devices, with many configurable parameters, all contend to optimally serve swarms of requests between local and remote applications. The challenge of effectively designing these systems is exacerbated by continuous advances in hardware, firmware, innovation in cache eviction algorithms, and evolving workloads and access patterns. This rapidly expanding configuration space has made it costly and time-consuming to physically experiment with numerous cache configurations for even a single stable workload. Current cache evaluation techniques (e.g., Miss Ratio Curves) are short-sighted: they analyze only a single tier of cache, focus primarily on performance, and fail to examine the critical relationships between metrics like throughput and monetary cost. Publicly available I/O cache simulators are also lacking: they can only simulate a fixed or limited number of cache tiers, are missing key features, or offer limited analyses. It is our position that best practices in cache analysis should include the evaluation of multi-tier configurations, coupled with more comprehensive metrics that reveal critical design trade-offs, especially monetary costs. We are developing an n-level I/O cache simulator that is general enough to model any cache hierarchy, captures many metrics, provides a robust set of analysis features, and is easily extendable to facilitate experimental research or production level provisioning. To demonstrate the value of our proposed metrics and simulator, we extended an existing cache simulator (PyMimircache). We present several interesting and counter-intuitive results in this paper. 
    more » « less
  3. Modern cache hierarchies are tangled webs of complexity. Multiple tiers of heterogeneous physical and virtual devices, with many configurable parameters, all contend to optimally serve swarms of requests between local and remote applications. The challenge of effectively designing these systems is exacerbated by continuous advances in hardware, firmware, innovation in cache eviction algorithms, and evolving workloads and access patterns. This rapidly expanding configuration space has made it costly and time-consuming to physically experiment with numerous cache configurations for even a single stable workload. Current cache evaluation techniques (e.g., Miss Ratio Curves) are short-sighted: they analyze only a single tier of cache, focus primarily on performance, and fail to examine the critical relationships between metrics like throughput and monetary cost. Publicly available I/O cache simulators are also lacking: they can only simulate a fixed or limited number of cache tiers, are missing key features, or offer limited analyses. It is our position that best practices in cache analysis should include the evaluation of multi-tier configurations, coupled with more comprehensive metrics that reveal critical design trade-offs, especially monetary costs. We are developing an n-level I/O cache simulator that is general enough to model any cache hierarchy, captures many metrics, provides a robust set of analysis features, and is easily extendable to facilitate experimental research or production level provisioning. To demonstrate the value of our proposed metrics and simulator, we extended an existing cache simulator (PyMimircache). We present several interesting and counter-intuitive results in this paper. 
    more » « less
  4. Abstract

    Organic mixed ionic–electronic conductors (OMIECs) have varied performance requirements across a diverse application space. Chemically doping the OMIEC can be a simple, low‐cost approach for adapting performance metrics. However, complex challenges, such as identifying new dopant materials and elucidating design rules, inhibit its realization. Here, these challenges are approached by introducing a new n‐dopant, tetrabutylammonium hydroxide (TBA‐OH), and identifying a new design consideration underpinning its success. TBA‐OH behaves as both a chemical n‐dopant and morphology additive in donor acceptor co‐polymer naphthodithiophene diimide‐based polymer, which serves as an electron transporting material in organic electrochemical transistors (OECTs). The combined effects enhance OECT transconductance, charge carrier mobility, and volumetric capacitance, representative of the key metrics underpinning all OMIEC applications. Additionally, when the TBA+counterion adopts an “edge‐on” location relative to the polymer backbone, Coulombic interaction between the counterion and polaron is reduced, and polaron delocalization increases. This is the first time such mechanisms are identified in doped‐OECTs and doped‐OMIECs. The work herein therefore takes the first steps toward developing the design guidelines needed to realize chemical doping as a generic strategy for tailoring performance metrics in OECTs and OMIECs.

     
    more » « less
  5. Spaceplates are novel flat-optic devices that implement the optical response of a free-space volume over a smaller length, effectively “compressing space” for light propagation. Together with flat lenses such as metalenses or diffractive lenses, spaceplates have the potential to enable the miniaturization of any free-space optical system. While the fundamental and practical bounds on the performance metrics of flat lenses have been well studied in recent years, a similar understanding of the ultimate limits of spaceplates is lacking, especially regarding the issue of bandwidth, which remains as a crucial roadblock for the adoption of this platform. In this work, we derive fundamental bounds on the bandwidth of spaceplates as a function of their numerical aperture and compression ratio (ratio by which the free-space pathway is compressed). The general form of these bounds is universal and can be applied and specialized for different broad classes of space-compression devices, regardless of their particular implementation. Our findings also offer relevant insights into the physical mechanism at the origin of generic space-compression effects and may guide the design of higher performance spaceplates, opening new opportunities for ultra-compact, monolithic, planar optical systems for a variety of applications.

     
    more » « less