Finally, AI-Based Painting is Here!

Dear Fellow Scholars, this is Two Minute Papers
with Károly Zsolnai-Fehér. A few years ago, the Generative Adversarial
Network architecture appeared that contains two neural networks that try to outcompete
each other. It has been used extensively for image generation,
and has become a research subfield of its own. For instance, they can generate faces of people
that don’t exist and much, much more. This is great, we should be grateful to live
in a time when breakthroughs like this happen in AI research. However, we should also note that artists
usually have a vision of the work that they would like to create, and instead of just
getting a deluge of new images, most of them would prefer to have some sort of artistic
control over the results. This work offers something that they call
semantic paint brushes. This means that we can paint not in terms
of colors, but in terms of concepts. Now this may sound a little nebulous, so if
you look here, you see that as a result, we can grow trees, change buildings, and do all
kinds of shenanigans without requiring us to be able to draw the results by hand. Look at those marvelous results! It works by compressing down these images
into a latent space. This is a representation that is quite sparse
and captures the essence of these images. One of the key ideas is that this can then
be reconstructed by a generator neural network to get a similar image back, however, the
twist is that while we are in the latent domain, we can apply these intuitive edits to this
image, so when the generator step takes place, it will carry through our changes. If you look at the paper, you will see that
just using one generator network doesn’t yield these great results, therefore this
generator needs to be specific to the image we are currently editing. The included user study shows that the new
method is preferred over the previous techniques. Now, like all of these methods, this is not
without limitations. Here you see that despite trying to remove
the chairs from the scene, amusingly, we get them right back. That’s a bunch of chairs free of charge,
in fact, I am not even sure how many chairs we got here. If you figured that out, make sure to leave
a comment about it, but all in all, that’s not what we asked for, and solving this remains
a challenge for the entire family of these algorithms. And, good news, in fact, when talking about
a paper, probably the best kind of news is that you can try it online through a web demo
right now. Make sure to try it out and post your results
here if you find anything interesting. The authors themselves may also learn something
new from us about interesting new failure cases. It has happened before in this series. This episode has been supported by Weights
& Biases. Weights & Biases provides tools to track your
experiments in your deep learning projects. It is like a shared logbook for your team,
and with this, you can compare your own experiment results, put them next to what your colleagues
did and you can discuss your successes and failures much easier. It takes less than 5 minutes to set up and
is being used by OpenAI, Toyota Research, Stanford and Berkeley. It was also used in this OpenAI project that
you see here, which we covered earlier in the series. They reported that experiment tracking was
crucial in this project and that this tool saved them quite a bit of time and money. If only I had an access to such a tool during
our last research project where I had to compare the performance of neural networks for months
and months. Well, it turns out, I will be able to get
access to these tools, because, get this, it’s free and will always be free for academics
and open source projects. Make sure to visit them through
or just click the link in the video description and sign up for a free demo today. Our thanks to Weights & Biases for helping
us make better videos for you. Thanks for watching and for your generous
support, and I’ll see you next time!

Posts created 5600

66 thoughts on “Finally, AI-Based Painting is Here!

  1. AI this, AI that. It's uniquely fascinating how AI can be applied and generalized into literally everything you throw at it, but there are more scientific breakthroughs around the world in other fields than AI.

  2. Hell yeah! Looking forward to losing all my clients as a freelance artist 🙂 I knew it was gonna happen but its coming much faster than i expected.

  3. Glev Alexandrov told me to warn you about the results at the end of this video:

  4. I feel like you missed a great opportunity here. You're always saying, "two papers down the line". Well this is one paper down the line. I would have appreciate a comparison of what aspect of this research is different from the original GANDissection paper and GAN Paint demo from 5 months ago, along with at least an acknowledgement it existed by a link in the description, I had this weird sense of deja vu, but you never made reference to this initial research. It took a fellow commentor's comment to convince I didn't see this somewhere other than this channel.

    It's clear the research is different, but my limited knowledge of AI doesn't allow me to see the difference for myself. That would have been a valuable service you could have provided. I still don't understand the difference, though I'm interested enough I'll probably try to figure it out on my own.

  5. The results of a house with a lawn/garden that I input had, to put it nicely, "less-than desirable results." It was good with grass and OK with skies, but everything else was wonky. But, to be fair, this is the case with almost all new developing technologies. Can't wait to see how this will end up in just a couple of papers away.

  6. GAN based generators are not the future. We still get good results with VAEs ( We must focus on innovative ideas rather than hype.

  7. I can't wait until this tech works on video. Having alphas generated automatically for item type is going to make Faux Bokeh and other post process effects look spectacular. AI is removing the skill gap for creating good art, and its fucking awesome.

  8. I'm not sure I am happy with the neutral stance of AI researchers. It seems to move from innocent cool science to dark dystopian abuse too fast.

    Perhaps this channel could also treat some papers/books on the ethics and dangers of AI?

  9. "We should be grateful for being alive.."
    And, if we are dead, we should be grateful for being dead as well.
    And if we haven't ever been alive, we should be grateful even more.
    What a time to be or not to be!

  10. I wonder how this AI shenanigans will impact the field of fine arts in the near future. Like, what will happen to old fashioned painting?

  11. Thumbnail lead me to believe it would be the same except for painting faces. That'd be cool if it could be done with the same quality as the face generator.

  12. So how does this solve any real given problem of mankind? Sometimes your euphoria like it is this new holy grail sucks big time. The tech is fun but not more than a gimmick. And AI will never produce real art. Itself is art. But to produce art you need a heart. And no machine will ever do that.

  13. Helló. Egy akadékoskodó subscriber vagyok. Szóval. Tényleg minden videót így fogsz kezdeni… hogy dear fellow scolars…?
    Én bírom ezt a channelt… tényleg jó meg minden…

    De tényleg csak akadémikusok vannak megcélozva… ? én ezt nem értem… miért ez a köszöntés mindig…


  14. I think some people are trying to blow out of proportion AI, similar to the dot com bubble… While it definitely has its uses – no it can't really paint, it can't really create music, it can't even replace a game opponent since people prefer human interaction. I don't even want it in my search engine, since it's a method that allows bias like Google showed…

  15. "Finally, AI-Based Painting is Here"
    Why "finally"? This already existed before and doesnt even look better, at least on the pictures you showed.

  16. This guy sounds like an AI trying to get us to train itself by uploading shared results from around the world. His voice does not sound human

  17. The problem here is that there aren't so many semantic features to paint with. It gets boring quite fast with the examples provided on that website. They need to add more features like bridge, lamp, sign, etc etc.

    What could be exciting is if there were methods to paint code. So that you didn't have to learn programming. Maybe it could be possible using these GANs? Or paint music ? How far can you go with painting with GANs? What do you guys think?

Leave a Reply

Your email address will not be published. Required fields are marked *

Begin typing your search term above and press enter to search. Press ESC to cancel.

Back To Top