f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning
- Award ID(s):
- 2006738
- PAR ID:
- 10297561
- Date Published:
- Journal Name:
- Advances in neural information processing systems
- ISSN:
- 1049-5258
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
null (Ed.)Imitation learning (IL) aims to learn a policy from expert demonstrations that minimizes the discrepancy between the learner and expert behaviors. Various imitation learning algorithms have been proposed with different pre-determined divergences to quantify the discrepancy. This naturally gives rise to the following question: Given a set of expert demonstrations, which divergence can recover the expert policy more accurately with higher data efficiency? In this work, we propose f-GAIL – a new generative adversarial imitation learning model – that automatically learns a discrepancy measure from the f-divergence family as well as a policy capable of producing expert-like behaviors. Compared with IL baselines with various predefined divergence measures, f-GAIL learns better policies with higher data efficiency in six physics-based control tasks.more » « less
-
null (Ed.)Abstract We consider the inequality $$f \geqslant f\star f$$ for real functions in $$L^1({\mathbb{R}}^d)$$ where $$f\star f$$ denotes the convolution of $$f$$ with itself. We show that all such functions $$f$$ are nonnegative, which is not the case for the same inequality in $L^p$ for any $$1 < p \leqslant 2$$, for which the convolution is defined. We also show that all solutions in $$L^1({\mathbb{R}}^d)$$ satisfy $$\int _{{\mathbb{R}}^{\textrm{d}}}f(x)\ \textrm{d}x \leqslant \tfrac 12$$. Moreover, if $$\int _{{\mathbb{R}}^{\textrm{d}}}f(x)\ \textrm{d}x = \tfrac 12$$, then $$f$$ must decay fairly slowly: $$\int _{{\mathbb{R}}^{\textrm{d}}}|x| f(x)\ \textrm{d}x = \infty $$, and this is sharp since for all $r< 1$, there are solutions with $$\int _{{\mathbb{R}}^{\textrm{d}}}f(x)\ \textrm{d}x = \tfrac 12$$ and $$\int _{{\mathbb{R}}^{\textrm{d}}}|x|^r f(x)\ \textrm{d}x <\infty $$. However, if $$\int _{{\mathbb{R}}^{\textrm{d}}}f(x)\ \textrm{d}x =: a < \tfrac 12$$, the decay at infinity can be much more rapid: we show that for all $$a<\tfrac 12$$, there are solutions such that for some $$\varepsilon>0$$, $$\int _{{\mathbb{R}}^{\textrm{d}}}e^{\varepsilon |x|}f(x)\ \textrm{d}x < \infty $$.more » « less
An official website of the United States government

