TY - CONF
T1 - Boundary-seeking generative adversarial networks
AU - Hjelm, R. Devon
AU - Jacob, Athul Paul
AU - Che, Tong
AU - Trischler, Adam
AU - Cho, Kyunghyun
AU - Bengio, Yoshua
N1 - Funding Information:
RDH thanks IVADO, MILA, UdeM, NIH grants R01EB006841 and P20GM103472, and NSF grant 1539067 for support. APJ thanks UWaterloo, Waterloo AI lab and MILA for their support and Michael Noukhovitch, Pascal Poupart for constructive discussions. KC thanks AdeptMind, TenCent, eBay, Google (Faculty Awards 2015, 2016), NVIDIA Corporation (NVAIL) and Facebook for their support. YB thanks CIFAR, NSERC, IBM, Google, Facebook and Microsoft for their support. We would like to thank Simon Sebbagh for his input and help with Theorem 2. Finally, we wish to thank the developers of Theano (Al-Rfou et al., 2016), Lasagne http://lasagne.readthedocs. io, and Fuel (Van Merriënboer et al., 2015) for their valuable code-base.
Funding Information:
RDH thanks IVADO, MILA, UdeM, NIH grants R01EB006841 and P20GM103472, and NSF grant 1539067 for support. APJ thanks UWaterloo, Waterloo AI lab and MILA for their support and Michael Noukhovitch, Pascal Poupart for constructive discussions. KC thanks AdeptMind, TenCent, eBay, Google (Faculty Awards 2015, 2016), NVIDIA Corporation (NVAIL) and Facebook for their support. YB thanks CIFAR, NSERC, IBM, Google, Facebook and Microsoft for their support. We would like to thank Simon Sebbagh for his input and help with Theorem 2. Finally, we wish to thank the developers of Theano (Al-Rfou et al., 2016), Lasagne http://lasagne.readthedocs. io, and Fuel (Van Merri?nboer et al., 2015) for their valuable code-base.
Publisher Copyright:
© Learning Representations, ICLR 2018 - Conference Track Proceedings.All right reserved.
PY - 2018/1/1
Y1 - 2018/1/1
N2 - Generative adversarial networks (GANs, Goodfellow et al., 2014) are a learning framework that rely on training a discriminator to estimate a measure of difference between a target and generated distributions. GANs, as normally formulated, rely on the generated samples being completely differentiable w.r.t. the generative parameters, and thus do not work for discrete data. We introduce a method for training GANs with discrete data that uses the estimated difference measure from the discriminator to compute importance weights for generated samples, thus providing a policy gradient for training the generator. The importance weights have a strong connection to the decision boundary of the discriminator, and we call our method boundary-seeking GANs (BGANs). We demonstrate the effectiveness of the proposed algorithm with discrete image and character-based natural language generation. In addition, the boundary-seeking objective extends to continuous data, which can be used to improve stability of training, and we demonstrate this on Celeba, Large-scale Scene Understanding (LSUN) bedrooms, and Imagenet without conditioning.
AB - Generative adversarial networks (GANs, Goodfellow et al., 2014) are a learning framework that rely on training a discriminator to estimate a measure of difference between a target and generated distributions. GANs, as normally formulated, rely on the generated samples being completely differentiable w.r.t. the generative parameters, and thus do not work for discrete data. We introduce a method for training GANs with discrete data that uses the estimated difference measure from the discriminator to compute importance weights for generated samples, thus providing a policy gradient for training the generator. The importance weights have a strong connection to the decision boundary of the discriminator, and we call our method boundary-seeking GANs (BGANs). We demonstrate the effectiveness of the proposed algorithm with discrete image and character-based natural language generation. In addition, the boundary-seeking objective extends to continuous data, which can be used to improve stability of training, and we demonstrate this on Celeba, Large-scale Scene Understanding (LSUN) bedrooms, and Imagenet without conditioning.
UR - http://www.scopus.com/inward/record.url?scp=85057279756&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85057279756&partnerID=8YFLogxK
M3 - Paper
T2 - 6th International Conference on Learning Representations, ICLR 2018
Y2 - 30 April 2018 through 3 May 2018
ER -