FACEGAN: Facial attribute controllable rEenactment GAN

Soumya Tripathy, Juho Kannala, Esa Rahtu

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

Abstract

The face reenactment is a popular facial animation method where the person's identity is taken from the source image and the facial motion from the driving image. Recent works have demonstrated high quality results by combining the facial landmark based motion representations with the generative adversarial networks. These models perform best if the source and driving images depict the same person or if the facial structures are otherwise very similar. However, if the identity differs, the driving facial structures leak to the output distorting the reenactment result. We propose a novel Facial Attribute Controllable rEenactment GAN (FACEGAN), which transfers the facial motion from the driving face via the Action Unit (AU) representation. Unlike facial landmarks, the AUs are independent of the facial structure preventing the identity leak. Moreover, AUs provide a human interpretable way to control the reenactment. FACEGAN processes background and face regions separately for optimized output quality. The extensive quantitative and qualitative comparisons show a clear improvement over the state-of-the-art in a single source reenactment task. The results are best illustrated in the reenactment video provided in the supplementary material. The source code will be made available upon publication of the paper.

Original languageEnglish
Title of host publicationProceedings - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021
PublisherIEEE
Pages1328-1337
Number of pages10
ISBN (Electronic)9780738142661
DOIs
Publication statusPublished - Jan 2021
MoE publication typeA4 Article in a conference publication
EventIEEE Winter Conference on Applications of Computer Vision - Virtual, Online, United States
Duration: 5 Jan 20219 Jan 2021

Conference

ConferenceIEEE Winter Conference on Applications of Computer Vision
Abbreviated titleWACV
Country/TerritoryUnited States
CityVirtual, Online
Period05/01/202109/01/2021

Fingerprint

Dive into the research topics of 'FACEGAN: Facial attribute controllable rEenactment GAN'. Together they form a unique fingerprint.

Cite this