GeneGAN: Learning Object Transfiguration and Object Subspace from Unpaired Data

Shuchang Zhou, Taihong Xiao, Yi Yang, Dieqiao Feng, Qinyao He and Weiran He

Abstract

Object Transfiguration generates diverse novel images by replacing an object in the given image with particular objects from exemplar images. It offers fine-grained controls of image generation, and can perform tasks like “put exactly those eyeglasses from image A onto the nose of the person in image B”. However, object transfiguration often requires disentanglement of objects from backgrounds in feature space, which is challenging and previously requires learning from paired training data: two images sharing the same background but with different objects. In this work, we propose a deterministic generative model that learns disentangled feature subspaces by adversarial training. The training data are two unpaired sets of images: a positive set containing images that have some kind of object, and a negative set being the opposite. The model encodes an image into two complement features: one for the object, and the other for the background. The object and background features from a “positive” parent and a “negative” parent, can be recombined to produce four children, of which two are exact reproductions, and the other two are crossbreeds. Minimizing the adversarial loss between crossbreeds and parents will ensure the crossbreeds inherit the specific objects of parents. On the other hand, minimizing the reconstruction loss between reproductions and parents can ensure the completeness of the features. Overall, the object and background features are complete and disentangled representations of images. Moreover, the object features are found to constitute a multidimensional attribute subspace.

Session

Posters

Files

Paper (PDF)

DOI

10.5244/C.31.111
https://dx.doi.org/10.5244/C.31.111

Citation

Bibtex

            @inproceedings{BMVC2017_111,
                title={GeneGAN: Learning Object Transfiguration and Object Subspace from Unpaired Data},
                author={Shuchang Zhou, Taihong Xiao, Yi Yang, Dieqiao Feng, Qinyao He and Weiran He},
                year={2017},
                month={September},
                pages={111.1-111.13},
                articleno={111},
                numpages={13},
                booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
                publisher={BMVA Press},
                editor={Tae-Kyun Kim, Stefanos Zafeiriou, Gabriel Brostow and Krystian Mikolajczyk},
                doi={10.5244/C.31.111},
                isbn={1-901725-60-X},
                url={https://dx.doi.org/10.5244/C.31.111}
            }