Artistic Translations via Generative Adversarial Networks

Abstract

The deep learning models for image-to-image translation established unpaired translation learning. In these models, we focused on developing an N-to-N domains translation model that outputs N-style images from a single content image only by using a single trained model.

When we want to train the model based on existing N-to-N domains translation models to be robust for extreme appearance changing, the model tends to consume a lot of computational costs because these models reuse generators to obtain cycle consistency loss (content loss). Cycle consistency loss is the difference of spatial features between content image and translated one, and this loss is important to established unpaired image-to-image translation learning.

Our study proposed a new N-to-N domains translation model named “Multi-CartoonGAN” which has the potential to learn diverse and large feature mappings within only a small number of training parameters. Multi-CartoonGAN is the developed model of one-to-one domains translation model CartoonGAN. As CartoonGAN succeeded in reducing training parameters by utilizing a pre-trained VGG net to calculate content loss, we developed the model as an N-to-N domains translation model. About the solution of extreme appearance translation, we implemented a new adaptive normalization function: Switch CAdaLIN.

Click to watch the video.

illustrasjon rina

Biography

Dr. Rina Komatsu is a full-time post-doctor for Dr. Tad Gonsalves’s laboratory at Sophia University. Rina has been belonging to Sophia university for 9 years including bachelor’s, master’s, and doctor. Rina received her bachelor’s degree in the department of Information Science and Engineering from the Faculty of Science and Technology. And Rina received a master’s degree in the Department of Science and Technology. Her major study is developing image processing technology using deep learning architecture like GAN (: Generative Adversarial Network).

 

Her Ph.D. is obtained in Engineering from Sophia University in 2022. Her doctor paper proposes the new unsupervised image-to-image translation architecture which deals with N domains translation for difficult translation tasks. Also, her proposed model could save GPU computing costs around 4GB in conditional translation training. Currently, she is participating in USEPE project for developing drone technology using deep learning.

 

02-03-2022