More Like this

1. Rethinking Inverse Reinforcement Learning: from Data Alignment to Task Alignment

   Zhou, Weichao; Li, Wenchao (December 2024, 2024 Conference on Neural Information Processing Systems)

   Many imitation learning (IL) algorithms use inverse reinforcement learning (IRL) to infer a reward function that aligns with the demonstration. However, the inferred reward functions often fail to capture the underlying task objectives. In this paper, we propose a novel framework for IRL-based IL that prioritizes task alignment over conventional data alignment. Our framework is a semi-supervised approach that leverages expert demonstrations as weak supervision to derive a set of candidate reward functions that align with the task rather than only with the data. It then adopts an adversarial mechanism to train a policy with this set of reward functions to gain a collective validation of the policy's ability to accomplish the task. We provide theoretical insights into this framework's ability to mitigate task-reward misalignment and present a practical implementation. Our experimental results show that our framework outperforms conventional IL baselines in complex and transfer learning scenarios. 

   more » « less
2. Attributed Graph Alignment

   https://doi.org/10.1109/ISIT45174.2021.9517967  

   Zhang, Ning; Wang, Weina; Wang, Lele (July 2021, 2021 IEEE International Symposium on Information Theory (ISIT))
3. Multilevel Network Alignment

   https://doi.org/10.1145/3308558.3313484  

   Zhang, Si; Tong, Hanghang; Maciejewski, Ross; Eliassi-Rad, Tina (January 2019, WWW '19 The World Wide Web Conference)

   Network alignment, which aims to find the node correspondence across multiple networks, is a fundamental task in many areas, ranging from social network analysis to adversarial activity detection. The state-of-the-art in the data mining community often view the node correspondence as a probabilistic cross-network node similarity, and thus inevitably introduce an O(n2) lower bound on the computational complexity. Moreover, they might ignore the rich patterns (e.g., clusters) accompanying the real networks. In this paper, we propose a multilevel network alignment algorithm (Moana) which consists of three key steps. It first efficiently coarsens the input networks into their structured representations, and then aligns the coarsest representations of the input networks, followed by the interpolations to obtain the alignment at multiple levels including the node level at the finest granularity. The proposed coarsen-align-interpolate method bears two key advantages. First, it overcomes the O(n2) lower bound, achieving a linear complexity. Second, it helps reveal the alignment between rich patterns of the input networks at multiple levels (e.g., node, clusters, super-clusters, etc.). Extensive experimental evaluations demonstrate the efficacy of the proposed algorithm on both the node-level alignment and the alignment among rich patterns (e.g., clusters) at different granularities. 

   more » « less
4. Diffusion transport alignment

   https://doi.org/10.1007/978-3-031-30047-9_10  

   Duque, A.; Wolf, G.; Moon, K.R. (January 2023, Advances in Intelligent Data Analysis)
5. Value Alignment Verification

   Brown, Daniel; Schneider, Jordan; Dragan, Anca; Niekum, Scott (January 2021, 38th International Conference on Machine Learning)

   As humans interact with autonomous agents to perform increasingly complicated, potentially risky tasks, it is important to be able to efficiently evaluate an agent’s performance and correctness. In this paper we formalize and theoretically analyze the problem of efficient value alignment verification: how to efficiently test whether the behavior of another agent is aligned with a human’s values. The goal is to construct a kind of “driver’s test” that a human can give to any agent which will verify value alignment via a minimal number of queries. We study alignment verification problems with both idealized humans that have an explicit reward function as well as problems where they have implicit values. We analyze verification of exact value alignment for rational agents and propose and analyze heuristic and approximate value alignment verification tests in a wide range of gridworlds and a continuous autonomous driving domain. Finally, we prove that there exist sufficient conditions such that we can verify exact and approximate alignment across an infinite set of test environments via a constant- query-complexity alignment test. 

   more » « less