MAGNet: Modern Art Generator using Deep Neural Networks



Original Source Here

You can see how the contrast adjustment “punches up” the colors. The source code for adjusting the image contrast in Python is here.

Results

Here are some results of running MAGnet with various text queries. Be sure to check out the appendix to see even more results.

MAGnet rendering of “an abstract painting with circles,” Image by Author
MAGnet rendering of “a painting of a landscape,” Image by Author
MAGnet rendering of “a cubist painting,” Image by Author

Discussion and Future Work

The MAGnet system works fairly well. It seems to do a better job generating abstract paintings than representational paintings. It converges on a solution fairly quickly, within five generations.

Although the results of MAGnet are pretty good, if you play with it, you may see the same visual themes recur occasionally. This may be due to the limited number of source images used during the training (about 4,000 initial images, augmented to over 12,000 via cropping, and filtered to 10,000). Training with more images will probably reduce the frequency of repeated visual themes.

Another area that could be improved is the textures within the images. There seem to be some “computery” artifacts in the flat areas. This could be due to the reimplementation of the CUDA operations.

I noticed that rosinality is currently working on a new open source project on GitHub called alias-free-gan-pytorch. Sounds promising!

Source Code

All of the source code for this project is available on GitHub. Images of the paintings on Kaggle and the source code are released under the CC BY-SA license.

Attribution-ShareAlike

Acknowledgments

I want to thank Jennifer Lim for her help with this article.

References

[1] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al., Learning Transferable Visual Models From Natural Language Supervision (2021)

[2] T. Karras, M. Aittala, J. Hellsten, S. Laine, J. Lehtinen, and T. Aila, “Training Generative Adversarial Networks with Limited Data,” https://arxiv.org/pdf/2006.06676.pdf (2020)

[3] R. Gal, D. Cohen, A. Bermano, and D. Cohen-Or, “SWAGAN: A Style-based WAvelet-driven Generative Model,” https://arxiv.org/pdf/2102.06108.pdf (2021)

[4] F.A. Galatolo, M.G.C.A. Cimino, and G.Vaglini, “Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Search,” https://arxiv.org/pdf/2102.01645.pdf, (2021)

[5] K. Deb, K., A. Pratap, S. Agarwal, and T. Meyarivan, “A fast and elitist multiobjective genetic algorithm: Nsga-ii.”, IEEE transactions on evolutionary computation, (2002)

[6] WikiArt, https://www.wikiart.org, (2008–2021)

[7] O. Patashnik, Z.Wu, E. Shechtman, D.Cohen-Or, and Dani Lischinski “StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery”, https://arxiv.org/pdf/2103.17249.pdf (2021)

[8] M.Mitchell, An Introduction to Genetic Algorithms. Cambridge, MA: MIT Press. (1996)

Appendix — Gallery of MAGnet Images

Here is a sampling of images created by MAGnet. You can click on each image to see a larger version.

AI/ML

Trending AI/ML Article Identified & Digested via Granola by Ramsey Elbasheer; a Machine-Driven RSS Bot

%d bloggers like this: