Search for: All records

Visual Question Answering (VQA) is a fundamental task in computer vision and natural language process fields. Although the “pre-training & finetuning” learning paradigm significantly improves the VQA performance, the adversarial robustness of such a learning paradigm has not been explored. In this paper, we delve into a new problem: using a pre-trained multimodal source model to create adversarial image-text pairs and then transferring them to attack the target VQA models. Correspondingly, we propose a novel VQATTACK model, which can iteratively generate both im- age and text perturbations with the designed modules: the large language model (LLM)-enhanced image attack and the cross-modal joint attack module. At each iteration, the LLM-enhanced image attack module first optimizes the latent representation-based loss to generate feature-level image perturbations. Then it incorporates an LLM to further enhance the image perturbations by optimizing the designed masked answer anti-recovery loss. The cross-modal joint attack module will be triggered at a specific iteration, which updates the image and text perturbations sequentially. Notably, the text perturbation updates are based on both the learned gradients in the word embedding space and word synonym-based substitution. Experimental results on two VQA datasets with five validated models demonstrate the effectiveness of the proposed VQATTACK in the transferable attack setting, compared with state-of-the-art baselines. This work revealsa significant blind spot in the “pre-training & fine-tuning” paradigm on VQA tasks. The source code can be found in the link https://github.com/ericyinyzy/VQAttack.

 

Architected materials design across orders of magnitude length scale intrigues exceptional mechanical responses nonexistent in their natural bulk state. However, the so‐termed mechanical metamaterials, when scaling bottom down to the atomistic or microparticle level, remain largely unexplored and conventionally fall out of their coarse‐resolution, ordered‐pattern design space. Here, combining high‐throughput molecular dynamics (MD) simulations and machine learning (ML) strategies, some intriguing atomistic families of disordered mechanical metamaterials are discovered, as fabricated by melt quenching and exemplified herein by lightweight‐yet‐stiff cellular materials featuring a theoretical limit of linear stiffness–density scaling, whose structural disorder—rather than order—is key to reduce the scaling exponent and is simply controlled by the bonding interactions and their directionality that enable flexible tunability experimentally. Importantly, a systematic navigation in the forcefield landscape reveals that, in‐between directional and non‐directional bonding such as covalent and ionic bonds, modest bond directionality is most likely to promotes disordered packing of polyhedral, stretching‐dominated structures responsible for the formation of metamaterials. This work pioneers a bottom‐down atomistic scheme to design mechanical metamaterials formatted disorderly, unlocking a largely untapped field in leveraging structural disorder in devising metamaterials atomistically and, potentially, generic to conventional upscaled designs.

 

The cell plasma membrane is a two-dimensional, fluid mosaic material composed of lipids and proteins that create a semipermeable barrier defining the cell from its environment. Compared with soluble proteins, the methodologies for the structural and functional characterization of membrane proteins are challenging. An emerging tool for studies of membrane proteins in mammalian systems is a “plasma membrane on a chip,” also known as a supported lipid bilayer. Here, we create the “plant-membrane-on-a-chip,″ a supported bilayer made from the plant plasma membranes of Arabidopsis thaliana, Nicotiana benthamiana, or Zea mays. Membrane vesicles from protoplasts containing transgenic membrane proteins and their native lipids were incorporated into supported membranes in a defined orientation. Membrane vesicles fuse and orient systematically, where the cytoplasmic side of the membrane proteins faces the chip surface and constituents maintain mobility within the membrane plane. We use plant-membrane-on-a-chip to perform fluorescent imaging to examine protein–protein interactions and determine the protein subunit stoichiometry of FLOTILLINs. We report here that like the mammalian FLOTILLINs, FLOTILLINs expressed in Arabidopsis form a tetrameric complex in the plasma membrane. This plant-membrane-on-a-chip approach opens avenues to studies of membrane properties of plants, transport phenomena, biophysical processes, and protein–protein and protein–lipid interactions in a convenient, cell-free platform. 

Numerical simulations have revolutionized material design. However, although simulations excel at mapping an input material to its output property, their direct application to inverse design has traditionally been limited by their high computing cost and lack of differentiability. Here, taking the example of the inverse design of a porous matrix featuring targeted sorption isotherm, we introduce a computational inverse design framework that addresses these challenges, by programming differentiable simulation on TensorFlow platform that leverages automated end-to-end differentiation. Thanks to its differentiability, the simulation is used to directly train a deep generative model, which outputs an optimal porous matrix based on an arbitrary input sorption isotherm curve. Importantly, this inverse design pipeline leverages the power of tensor processing units (TPU)—an emerging family of dedicated chips, which, although they are specialized in deep learning, are flexible enough for intensive scientific simulations. This approach holds promise to accelerate inverse materials design.