TY - JOUR
T1 - Inspirational Adversarial Image Generation
AU - Roziere, Baptiste
AU - Riviere, Morgane
AU - Teytaud, Olivier
AU - Rapin, Jeremy
AU - Lecun, Yann
AU - Couprie, Camille
N1 - Publisher Copyright:
© 1992-2012 IEEE.
PY - 2021
Y1 - 2021
N2 - The task of image generation started receiving some attention from artists and designers, providing inspiration for new creations. However, exploiting the results of deep generative models such as Generative Adversarial Networks can be long and tedious given the lack of existing tools. In this work, we propose a simple strategy to inspire creators with new generations learned from a dataset of their choice, while providing some control over the output. We design a simple optimization method to find the optimal latent parameters corresponding to the closest generation to any input inspirational image. Specifically, we allow the generation given an inspirational image of the user's choosing by performing several optimization steps to recover optimal parameters from the model's latent space. We tested several exploration methods from classical gradient descents to gradient-free optimizers. Many gradient-free optimizers just need comparisons (better/worse than another image), so they can even be used without numerical criterion nor inspirational image, only with human preferences. Thus, by iterating on one's preferences we can make robust facial composite or fashion generation algorithms. Our results on four datasets of faces, fashion images, and textures show that satisfactory images are effectively retrieved in most cases.
AB - The task of image generation started receiving some attention from artists and designers, providing inspiration for new creations. However, exploiting the results of deep generative models such as Generative Adversarial Networks can be long and tedious given the lack of existing tools. In this work, we propose a simple strategy to inspire creators with new generations learned from a dataset of their choice, while providing some control over the output. We design a simple optimization method to find the optimal latent parameters corresponding to the closest generation to any input inspirational image. Specifically, we allow the generation given an inspirational image of the user's choosing by performing several optimization steps to recover optimal parameters from the model's latent space. We tested several exploration methods from classical gradient descents to gradient-free optimizers. Many gradient-free optimizers just need comparisons (better/worse than another image), so they can even be used without numerical criterion nor inspirational image, only with human preferences. Thus, by iterating on one's preferences we can make robust facial composite or fashion generation algorithms. Our results on four datasets of faces, fashion images, and textures show that satisfactory images are effectively retrieved in most cases.
KW - Optimization
KW - generative adversarial networks
KW - similarity search
UR - http://www.scopus.com/inward/record.url?scp=85103241337&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85103241337&partnerID=8YFLogxK
U2 - 10.1109/TIP.2021.3065845
DO - 10.1109/TIP.2021.3065845
M3 - Article
C2 - 33735083
AN - SCOPUS:85103241337
SN - 1057-7149
VL - 30
SP - 4036
EP - 4045
JO - IEEE Transactions on Image Processing
JF - IEEE Transactions on Image Processing
M1 - 9381602
ER -