An official website of the United States government
Existing adversarial algorithms for Deep Reinforcement Learning (DRL) have largely focused on identifying an optimal time to attack a DRL agent. However, little work has been explored in injecting efficient adversarial perturbations in DRL environments. We propose a suite of novel DRL adversarial attacks, called ACADIA, representing AttaCks Against Deep reInforcement leArning. ACADIA provides a set of efficient and robust perturbation-based adversarial attacks to disturb the DRL agent's decision-making based on novel combinations of techniques utilizing momentum, ADAM optimizer (i.e., Root Mean Square Propagation, or RMSProp), and initial randomization. These kinds of DRL attacks with novel integration of such techniques have not been studied in the existing Deep Neural Networks (DNNs) and DRL research. We consider two well-known DRL algorithms, Deep-Q Learning Network (DQN) and Proximal Policy Optimization (PPO), under Atari games and MuJoCo where both targeted and non-targeted attacks are considered with or without the state-of-the-art defenses in DRL (i.e., RADIAL and ATLA). Our results demonstrate that the proposed ACADIA outperforms existing gradient-based counterparts under a wide range of experimental settings. ACADIA is nine times faster than the state-of-the-art Carlini & Wagner (CW) method with better performance under defenses of DRL.
more »
« less
This content will become publicly available on March 1, 2026
Deep Learning and Blockchain-Driven Contract Theory: Alleviate Gender Bias in Construction
In the construction industry, the advent of teleoperation and robotic technologies is revolutionizing traditional recruitment prac-tices, introducing new criteria for identifying qualified workers. This evolution presents significant challenges for employers aiming to recruitworkers who can maximize organizational utility. Although contract theory offers a promising solution to these challenges, its inherent self-disclosure property could inadvertently lead to privacy breaches, such as revealing gender-related information. Such disclosure risk mightintensify existing biases, notably gender bias, within the sector. To this end, we proposed deep reinforcement learning (DRL)-based contracttheory. Firstly, the trained DRL model will produce unpredictable contract bundles, restricting employers’ access to workers’ privacy. Sub-sequently, to ensure employers adopt DRL-based contract theory, we utilized blockchain to supervise contract bundle generation. Finally,given that the DRL models are homogenous among employers, we integrated transfer learning to reduce unnecessary overhead. Simulationexperiments conducted using US labor force statistical data demonstrated that our work can effectively mitigate potential gender bias byaugmenting the contract selection rights for female workers from 72.73% and 60% to 96.97% and 95% in comparison with traditionalcontract theory while maximizing employers’ utility. In addition, with the integration of transfer learning, the training overhead ofDRL-based contract theory can decrease by 50%. The meaning and significance of the results lie in the innovative integration of contracttheory, deep reinforcement learning, and transfer learning into the recruitment framework, significantly advancing the body of knowledge inunbiased workforce development. DOI: 10.1061/JCEMD4.COENG-15330. © 2024 American Society of Civil Engineers.
more »
« less
- PAR ID:
- 10615361
- Publisher / Repository:
- Journal of Construction Engineering and Management
- Date Published:
- Journal Name:
- Journal of Construction Engineering and Management
- Volume:
- 151
- Issue:
- 3
- ISSN:
- 0733-9364
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Smart home devices raise privacy concerns among not only primary users but also bystanders like domestic workers. We conducted 25 qualitative interviews with nannies and 16 with parents who employed nannies, in the U.S., to explore and compare their views on and privacy threat models for smart home devices. We found device-specific purposes of use inspired different perspectives among nanny participants. Most were comfortable with employers’ smart speakers and smart TVs, whose purpose had nothing to do with them. However, with indoor smart cameras, nanny participants were often not just bystanders but targets of monitoring; in such situations, they had a wider range of attitudes. In contrast, parent participants tended to have more similar views across devices. We found notable disconnects regarding disclosure, where nanny participants often hesitated to ask about cameras, but parent participants assumed nannies just didn’t care. We recommend prioritizing interventions supporting disclosure, discussion, and sharing control.more » « less
-
Deep reinforcement learning (DRL) has demonstrated impressive success in solving complex control tasks by synthesizing control policies from data. However, the safety and stability of applying DRL to safety-critical systems remain a primary concern and challenging problem. To address the problem, we propose the Phy-DRL: a novel physics-model regulated deep reinforcement learning framework. The Phy-DRL is novel in two architectural designs: a physics-model-regulated reward and residual control, which integrates physics-model-based control and data-driven control. The concurrent designs enable the Phy-DRL to mathematically provable safety and stability guarantees. Finally, the effectiveness of the Phy-DRL is validated by an inverted pendulum system. Additionally, the experimental results demonstrate that the Phy-DRL features remarkably accelerated training and enlarged reward.more » « less
-
Deep Learning (DL) techniques are increasingly being incorporated in critical software systems today. DL software is buggy too. Recent work in SE has characterized these bugs, studied fix patterns, and proposed detection and localization strategies. In this work, we introduce a preventative measure. We propose design by contract for DL libraries, DL Contract for short, to document the properties of DL libraries and provide developers with a mechanism to identify bugs during development. While DL Contract builds on the traditional design by contract techniques, we need to address unique challenges. In particular, we need to document properties of the training process that are not visible at the functional interface of the DL libraries. To solve these problems, we have introduced mechanisms that allow developers to specify properties of the model architecture, data, and training process. We have designed and implemented DL Contract for Python-based DL libraries and used it to document the properties of Keras, a well-known DL library. We evaluate DL Contract in terms of effectiveness, runtime overhead, and usability. To evaluate the utility of DL Contract, we have developed 15 sample contracts specifically for training problems and structural bugs. We have adopted four well-vetted benchmarks from prior works on DL bug detection and repair. For the effectiveness, DL Contract correctly detects 259 bugs in 272 real-world buggy programs, from well-vetted benchmarks provided in prior work on DL bug detection and repair. We found that the DL Contract overhead is fairly minimal for the used benchmarks. Lastly, to evaluate the usability, we conducted a survey of twenty participants who have used DL Contract to find and fix bugs. The results reveal that DL Contract can be very helpful to DL application developers when debugging their code.more » « less
-
Cooperative relays improve reliability and coverage in wireless networks by providing multiple paths for data transmission. Relaying will play an essential role in vehicular networks at higher frequency bands, where mobility and frequent signal blockages cause link outages. To ensure connectivity in a relay-aided vehicular network, the relay selection policy should be designed to efficiently find unblocked relays. Inspired by recent advances in beam management in mobile millimeter wave (mmWave) networks, this paper address the question: how can the best relay be selected with minimal overhead from beam management? In this regard, we formulate a sequential decision problem to jointly optimize relay selection and beam management. We propose a joint relay selection and beam management policy based on deep reinforcement learning (DRL) using the Markov property of beam in- dices and beam measurements. The proposed DRL-based algorithm learns time-varying thresholds that adapt to the dynamic channel conditions and traffic patterns. Numeri- cal experiments demonstrate that the proposed algorithm outperforms baselines without prior channel knowledge. Moreover, the DRL-based algorithm can maintain high spectral efficiency under fast-varying channels.more » « less
