Search for: All records

Despite the growing interest in human-AI decision making, experimental studies with domain experts remain rare, largely due to the complexity of working with domain experts and the challenges in setting up realistic experiments. In this work, we conduct an in-depth collaboration with radiologists in prostate cancer diagnosis based on MRI images. Building on existing tools for teaching prostate cancer diagnosis, we develop an interface and conduct two experiments to study how AI assistance and performance feedback shape the decision making of domain experts. In Study 1, clinicians were asked to provide an initial diagnosis (human), then view the AI's prediction, and subsequently finalize their decision (human-AI team). In Study 2 (after a memory wash-out period), the same participants first received aggregated performance statistics from Study 1, specifically their own performance, the AI's performance, and their human-AI team performance, and then directly viewed the AI's prediction before making their diagnosis (i.e., no independent initial diagnosis). These two workflows represent realistic ways that clinical AI tools might be used in practice, where the second study simulates a scenario where doctors can adjust their reliance and trust on AI based on prior performance feedback. Our findings show that, while human-AI teams consistently outperform humans alone, they still underperform the AI due to under-reliance, similar to prior studies with crowdworkers. Providing clinicians with performance feedback did not significantly improve the performance of human-AI teams, although showing AI decisions in advance nudges people to follow AI more. Meanwhile, we observe that the ensemble of human-AI teams can outperform AI alone, suggesting promising directions for human-AI collaboration. 

Abstract Metamaterials have gained important interest in the research community attributable to advances in additive manufacturing enabling their fabrication at reasonable costs. The vast majority of their applications and demonstrations are at micro- and nano-scales, and challenges remained regarding the larger scale applications. In this paper, we are interested by the scalability of metamaterials, targeting structural engineering applications. To do so, we explore mechanisms capable of providing both bending stiffness and high-performance energy dissipation. Our study includes beams constructed with chiral topologies of different structural hierarchy orders, and we also explore three new topologies that we termed chiral friction, chiral-rectangular and chiral-hexagonal design to engineer the beams and the use of friction rods with tunable post-stress that inserted longitudinally through the beams to provide enhanced friction. The mechanical performance of the metamaterial beams is characterized through a series three-point bending tests. Of interest is to evaluate the bending stiffness, shape recoverability, and energy dissipation capabilities. We find that the chiral-hexagonal topology equipped with a non-stressed friction rod exhibit excellent energy dissipation capabilities, showing an improved loss factor by 11.9 times compared to the control beam using 68% of its materials density. Moreover, the use of the post-stress mechanism shows that it is possible to augment both its shape recovery and bending stiffness up to 99.3% and 47.1%, respectively. Overall, our investigation shows that it is possible to engineer scalable metamaterial beams targeting structural engineering applications, and that the use of topology optimization and strategically designed post-tensioning mechanism can allow tuning of mechanical performance. 

A new version of the US National Science Foundation National Center forAtmospheric Research (NSF NCAR) thermosphere-ionosphere-electrodynamicsgeneral circulation model (TIEGCM) has been developed and released. Thispaper describes the changes and improvements of the new version 3.0since its last major release (2.0) in 2016. These include: 1) increasingthe model resolution in both the horizontal and vertical dimensions, aswell as the ionospheric dynamo solver; 2) upward extension of the modelupper boundary to enable more accurate simulations of the topsideionosphere and neutral density in the lower exosphere; 3) improvedparameterization for thermal electron heating rate; 4) resolvingtransport of minor species N(2D); 5) treating helium as a major species;6) parameterization for additional physical processes, such as SAPS andelectrojet turbulent heating; 7) including parallel ion drag in theneutral momentum equation; 8) nudging of prognostic fields near thelower boundary from external data; 9) modification to the NO reactionrate and auroral heating rate; 10) outputs of diagnostic analysis termsof the equations; 11) new functionalities enabling model simulations ofcertain recurrent phenomena, such as solar flares and eclipse. Wepresent examples of the model validation during a moderate storm andcompare simulation results by turning on/off new functionalities todemonstrate the related new model capabilities. Furthermore, the modelis upgraded to comply with the new computer software environment at NSFNCAR for easy installation and run setup and with new visualizationtools. Finally, the model limitations and future development plans arediscussed. 

Explainability is increasingly recognized as an enabling technology for the broader adoption of machine learning (ML), particularly for safety-critical applications. This has given rise to explainable ML, which seeks to enhance the explainability of neural networks through the use of explanators. Yet, the pursuit for better explainability inadvertently leads to increased security and privacy risks. While there has been considerable research into the security risks of explainable ML, its potential privacy risks remain under-explored. To bridge this gap, we present a systematic study of privacy risks in explainable ML through the lens of membership inference. Building on the observation that, besides the accuracy of the model, robustness also exhibits observable differences among member samples and non-member samples, we develop a new membership inference attack. This attack extracts additional membership features from changes in model confidence under different levels of perturbations guided by the importance highlighted by the attribution maps in the explanators. Intuitively, perturbing important features generally results in a bigger loss in confidence for member samples. Using the member-non-member differences in both model performance and robustness, an attack model is trained to distinguish the membership. We evaluated our approach with seven popular explanators across various benchmark models and datasets. Our attack demonstrates there is non-trivial privacy leakage in current explainable ML methods. Furthermore, such leakage issue persists even if the attacker lacks the knowledge of training datasets or target model architectures. Lastly, we also found existing model and output-based defense mechanisms are not effective in mitigating this new attack.