Style transfer has recently received a lot of attention, since it allows to study fundamental challenges in image understanding and synthesis. Recent work has significantly improved the representation of color and texture and computational speed and image resolution. The explicit transformation of image content has, however, been mostly neglected: while artistic style affects formal characteristics of an image, such as color, shape or texture, it also deforms, adds or removes content details. This paper explicitly focuses on a content-and style-aware stylization of a content image. Therefore, we introduce a content transformation module between the encoder and decoder. Moreover, we utilize similar content appearing in photographs and style samples to learn how style alters content details and we generalize this to other class details. Additionally, this work presents a novel normalization layer critical for high resolution image synthesis. The robustness and speed of our model enables a video stylization in real-time and high definition. We perform extensive qualitative and quantitative evaluations to demonstrate the validity of our approach.

Our previous work: A Style-Aware Content Loss for Real-time HD Style Transfer

Main Idea

Content Transformation

Effect of the style specific content transformation block T for a class "person" and style of van Gogh.
Only the cropped out regions containing class "person" are visualized.
The rightmost column contains magnified patches from the real van Gogh paintings.

Image Stylizations

Cezanne Style

Cezanne Style

van Gogh Style

van Gogh Style

van Gogh Style

van Gogh Style

Kirchner Style

Claude Monet Style

Morisot Style

Picasso Style

Video Stylizations


This work has been supported by a hardware donation from NVIDIA Corporation.
This page is based on a design by TEMPLATED.