Style Transfer is a computer vision technique that allows us to recompose the content of an image in the style of another, which is interesting but challenging. Inspired by the power of Convolutional Neural Networks (CNNs), Gatys et al.[1] first studied how to use a CNN to reproduce famous painting styles on natural images, which is the first work of neural style transfer. They proposed to model the content of a photo as the feature responses from a pre-trained CNN, and further model the style of an artwork as the summary feature statistics. The key idea behind their algorithm is to iteratively optimise an image with the objective of matching desired CNN feature distributions, which involves both the photo’s content information and artwork’s style information. We call this type of method as Gatys-style neural style transfer.
The method mentioned above only compare content and stylised images in the CNN feature space to make the stylised image semantically similar to the content image. But since CNN features inevitably lose some lowlevel information contained in the image, there are usually some unappealing distorted structures and irregular artefacts in the stylised results. To preserve the coherence of fine structures during stylisation, Li et al.[2] propose to introduce an additional Laplacian loss, which is defined as the squared euclidean distance between the Laplacian filter responses of a content image and stylised result. We call this type of method as Lapstyle neural style transfer.
In this project, your task can be listed as follows:
Introduction (5 pts): Describe the problem you are working on, why it's important, and an overview of your results.
Related Work (5 pts): Discuss published work that relates to your project. How is your approach similar or different from others?
Data (5 pts): Describe the data you are working with for your project. What type of data is it? Where did it come from? How much data are you working with? Did you have to do any preprocessing, filtering, or other special treatment to use this data in your project?
Methods (30 pts): Describe the approach in paper[1] and targeted improvements in paper[2] according to your understanding. You have learned many classic algorithms in computer vision field. Think actively about whether you can apply these algorithms to neural style transfer. If you can propose some novel ideas, describe them in detail (10 pts). Why is your approach the right thing to do? Did you consider alternative approaches? It may be helpful to include figures, diagrams, or tables to describe your method or compare it with other methods.
Experiments (45 pts):
Besides, if you propose novel ideas, you need to prove your improvements by qualitative or quantitative comparisons with existing methods. (20 pts)
Writing / Formatting (5 pts) Is your paper clearly written and nicely formatted?
If you want read more papers about style transfer to gain inspiration, you can refer to [3] [4]
[5]. Most of them may be GPU-consuming.
Here are some potentially useful links: