Search for: All records

The robustness of neural networks is crucial in safety-critical applications, where identifying a reliable input space is essential for effective model selection, robustness evaluation, and the development of reliable control strategies. Most existing robustness verification methods assess the worst-case output under the assumption that the input space is known. However, precisely identifying a verifiable input space , where no adversarial examples exist, is challenging due to the possible high dimensionality, discontinuity, and non-convex nature of the input space. To address this challenge, we propose a novel framework, LEVIS, comprising LEVIS- and LEVIS-. LEVIS- identifies a single, large verifiable ball that intersects at least two boundaries of a bounded region , while LEVIS- systematically captures the entirety of the verifiable space by integrating multiple verifiable balls. Our contributions are fourfold: we introduce a verification framework, LEVIS, incorporating two optimization techniques for computing nearest and directional adversarial points based on mixed-integer programming (MIP); to enhance scalability, we integrate complementary constrained (CC) optimization with a reduced MIP formulation, achieving up to a 17-fold reduction in runtime by approximating the verifiable region in a principled way; we provide a theoretical analysis characterizing the properties of the verifiable balls obtained through LEVIS-; and we validate our approach across diverse applications, including electrical power flow regression and image classification, demonstrating performance improvements and visualizing the geometric properties of the verifiable region. 

This work proves the feasibility of utilizing steady state and transient in situ/operando spectroscopy to extract mechanistic information that reduces and leads to robust kinetic models. It also opens new avenues to explore kinetics and mechanisms with charge transfer data in heterogeneous catalysis. 

Large language models (LLMs) are notoriously memory-intensive during training, particularly with the popular AdamW optimizer. This memory burden necessitates using more or higher-end GPUs or reducing batch sizes, limiting training scalability and throughput. To address this, various memory-efficient optimizers have been proposed to reduce optimizer memory usage. However, they face critical challenges: (i) reliance on costly SVD operations; (ii) significant performance trade-offs compared to AdamW; and (iii) still substantial optimizer memory overhead to maintain competitive performance. In this work, we identify that AdamW's learning rate adaptation rule can be effectively coarsened as a structured learning rate update. Based on this insight, we propose Approximated Gradient Scaling for Memory-Efficient LLM Optimization (APOLLO), which approximates learning rate scaling using an auxiliary low-rank optimizer state based on pure random projection. This structured learning rate update rule makes APOLLO highly tolerant to further memory reductions while delivering comparable pre-training performance. Even its rank-1 variant, APOLLO-Mini, achieves superior pre-training performance compared to AdamW with SGD-level memory costs. Extensive experiments demonstrate that the APOLLO series performs on-par with or better than AdamW, while achieving greater memory savings by nearly eliminating the optimization states of AdamW. These savings provide significant system-level benefits: (1) Enhanced Throughput: 3x throughput on an 8xA100-80GB setup compared to AdamW by supporting 4x larger batch sizes. (2) Improved Model Scalability: Pre-training LLaMA-13B with naive DDP on A100-80GB GPUs without system-level optimizations. (3) Low-End GPU Friendly Pre-training: Pre-training LLaMA-7B on a single GPU using less than 12 GB of memory with weight quantization. 

Generating molecular structures with desired properties is a critical task with broad applications in drug discovery and materials design. We propose 3M-Diffusion, a novel multi-modal molecular graph generation method, to generate diverse, ideally novel molecular structures with desired properties. 3M-Diffusion encodes molecular graphs into a graph latent space which it then aligns with the text space learned by encoder based LLMs from textual descriptions. It then reconstructs the molecular structure and atomic attributes based on the given text descriptions using the molecule decoder. It then learns a probabilistic mapping from the text space to the latent molecular graph space using a diffusion model. The results of our extensive experiments on several datasets demonstrate that 3M-Diffusion can generate high-quality, novel and diverse molecular graphs that semantically match the textual description provided. The code is available on github. 

The recent emergence of Medical Large Vision Language Models (Med-LVLMs) has enhanced medical diagnosis. However, current Med-LVLMs frequently encounter factual issues, often generating responses that do not align with established medical facts. Retrieval-Augmented Generation (RAG), which utilizes external knowledge, can improve the factual accuracy of these models but introduces two major challenges. First, limited retrieved contexts might not cover all necessary information, while excessive retrieval can introduce irrelevant and inaccurate references, interfering with the model’s generation. Second, in cases where the model originally responds correctly, applying RAG can lead to an over-reliance on retrieved contexts, resulting in incorrect answers. To address these issues, we propose RULE, which consists of two components. First, we introduce a provably effective strategy for controlling factuality risk through the calibrated selection of the number of retrieved contexts. Second, based on samples where over-reliance on retrieved contexts led to errors, we curate a preference dataset to fine-tune the model, balancing its dependence on inherent knowledge and retrieved contexts for generation. We demonstrate the effectiveness of RAFE on three medical VQA datasets, achieving an average improvement of 20.8% in factual accuracy. 

The development of simple in situ spectrokinetic techniques to assess intermediate species nature and adsorption location can benefit catalytic studies by providing insights into the reasons for different catalysts performance and facilitate mechanistic proposals. 

Given the surge in rural logistics services and the disparities between urban and rural delivery services, a compelling necessity emerges to explore innovative drone-based delivery solutions. The challenges inherent in truck-drone delivery due to technological and physical barriers affect service quality for some rural customers, thus magnifying concerns about delivery fairness. To investigated delivery equity, we present a truck-drone cooperative delivery model to analyze rural customers’ accessibility to such innovative delivery technology. This model accommodates rural residents’ delivery preferences while optimizing truck routes. Drones are dispatched from designated trucks to serve customers within their flight distance. Our proposed heuristic algorithm, founded on graph-based truck-drone delivery preferences, solves this intricate problem efficiently. Numerical experiments underscore the efficacy of our approach, highlighting substantial reductions in delivery costs and an impressive 20% increase in drone deliveries on a large-scale network. Through sensitivity analyses exploring drone operational costs and flight distances–affected by government policies and technological advancements–we devise an equity metric that gauges the efficiency and accessibility of rapid rural delivery services under the truck-drone delivery framework. Our research contributes to equity analysis, addressing challenges faced by logistics companies and rural residents. Moreover, it bridges the gap between urban and rural logistics, fostering an inclusive and equitable delivery ecosystem benefiting all customers, regardless of their location. 

Cybersecurity and Artificial Intelligence (AI) are key domains whose intersection gives great promises and poses significant threats. Indeed, the National Academy of Science (NAS), the National Science Foundation (NSF), and othßer respected entities have noted the significant role that AI can play in cybersecurity, and the importance of ensuring the security of AI-enabled algorithms and systems. This minitrack focuses on AI and Cybersecurity that works in broader domains, collaborative inter-organizational realms, shared collaborative domains, or with collaborative technologies. The papers in this minitrack have the potential to offer interesting and impactful solutions to emerging areas, including unmanned aerial vehicles and open source software security.