Hello. So. Yes. As you've just heard I'm I'm a computer scientist, and not. An artist not a philosopher, so I'll be approaching, this, talk will come from kind of a different. Angle perhaps, from the other ones we've heard today. And. In some sense I'm gonna I'm. Gonna try to look at, almost. Like an opposite perspective so I think what. You'll hear quite a lot about today is how artists. Can use. AI. Can, use neural networks and these technologies, in a creative way and what, I'm gonna talk about is. How. The, internal, processes, that are going on in these, these. Models that generate images and and and. Sounds. And things like that how, those processes in, some way, resemble. Human, creative processes. So. Talks called madness in the machine generative. Models in the context, of human creativity. And. I should stress these these, views. Or these, kind of this, description of. Artificial. Intelligence and, of these. Kinds of algorithms. And networks isn't, necessarily, the one that most people in, the, field for example of deep learning would, would. Reference this is more just it's kind of my way of, thinking. About how these these. Algorithms operate and I'm just kind of hoping that maybe it will it will help to give. You all a peek, under the under the bonnet and what's going on sort of under the hood with these algorithms. Ok. So I think one, really fundamental, thing is. This idea of this kind of interplay, or contrast, between, structure. And noise or you could call it order and chaos madness. And reason things, that are fabricated, versus, things that are found. You. Know this this this feels, like all of creativity somehow, involves this dichotomy, this is why I've called it madness in the machine, and. So. We have these two quotes here from, famous writers the most beautiful things are those that madness prompts, and reason writes. And. Then we have you know this quote from Hemingway the good parts of a book maybe, only something a writer is lucky enough to over here or maybe the wreck of his whole down life, and, I think what, they're both getting at is that there's something kind of arbitrary, about the source of inspiration there's. This idea that something, comes along that really, wasn't under the the, artists, control it wasn't reasoned or planned by them but, is then transformed.
Into Art, by the internal. Processes, of the artist and I think there's definitely a parallel, here with, the way we. Attempt to generate. Things, using algorithms so. Yes like the idea is that the you know the inspiration is random but the outcome is so use this word a lot structured, I don't know if that's kind, of a familiar word here so it means that there's there's a pattern there's a there's a there's, a shape going on here that isn't just something that would occur randomly. Okay. So, it's. All very well to talk, like that but you know what, does that actually mean in practice, so you know, underlying. Most of these, these. AI, techniques, these generative, models what you really have is good. Old-fashioned, statistical. Models so you have machine. Learning algorithms, what do they do they find structure, by more. Or less by fitting probability, distributions, the data, and. So we have here an, example, of a very simple probability, distribution I, probably just took this off Wikipedia it's, it's, a two dimensional Gaussian distribution. And. By the way I should say I'm this talk is going to be very, non-technical there may be one, or two equations. Or graphs like this in it but it's absolutely, not important, to worry. About the details here this is just a sort of illustrative, example of what a simple probability distribution looks, like okay so given. That distribution, which is basically, these kind. Of contour, lines that appear in the graph you can then start generating, things you can generate data by picking, random. Points on that graph that. Doesn't sound very creative, so far right all we've got is a little dot appearing, at some X Y position so how do we get from that to something. That resembles a creative process, well. One. Of the key ideas. To get around is that when, you're creating anything, real anything interesting you're working and what we call. What. We tend to refer to in. So the machine learning is a high dimensional space, a high dimensional space of data what does that mean so, you know when we talk about so it's, it's it's actually in some ways it's kind of misleading it just comes back to our kind. Of human. Tendency, to want to reason about things in, kind of geometric, space to talk about two dimensions, and three dimensions, once, you go beyond three dimensions, calling. It a space is, you. Know kind of, it's. Just a metaphor it's an analogy, so. Anyway, what do I mean by high dimensional space so you know this is famous story the library of Babel, or sure a lot of you have read this, where. Bork, has imagined, this idea, of a library, that held all, possible, books so, he says you know it's shells register, all the possible combinations of the 20-odd. Orthographical. Symbols a number. Which though vast is not infinite, so that's important it's not there's, not an infinite number of possible, books but it's just ridiculously, large and so you know my calculus sort of back. Of envelope calculation, here if we allow up to 10 million. Letters. Essentially. Over 10 million characters, per book and. There's 25 possible, characters then you have this number that is you know impossibly. Larger, than the number of particles in the universe so to actually build this library it's kind of you know it's clearly unthinkable. Another. Another, example here, you know the famous Michelangelo, quote every, block of stone has a statue inside, it and the sculpture, is the, task of the sculpture to discover it so he's taking this big, block of Italian. Marble and, finding. All of these things. Inside. It and so again if you imagine, somehow. Parameterizing. The space of possible statues, that you could get from a block, lots. Of ways you could do that you could think about in terms of you know voxels, or something or you could think about the actual chiseling. Strokes taken by the sculptor obviously. There's an awful lot of there, would be a lot of parameters it would be a very high dimensional space. So. You. Know these are things that unlike. Unlike the example here where we can just plot the whole space on a piece of paper because it's two-dimensional, we can't do that with any kind of realistic data we, can't even enumerate, them we can't count, we can't write down all the different possible options that we have so what. Do we have instead well you know ultimately what, algorithms, have actually, is is the same thing that people have which is a generative, process, so. The simplest example. Of a generative process, in. In. Artificial intelligence I think is there's an auto regressive model and so the kinds of auto regressive models I'm talking about these recent ones are, generally.
They're. Generally neural networks they're parametrized, by neural networks that point in some sense isn't so important, but, the basic concept is that if you've got this very high dimensional data so you've got this you know whole book full of possible. Words what. You can do is you can split that data up into a sequence of very small pieces so you look at each word individually each letter each pixel, each voxel and then, you predict, each piece. Conditioned, on the previous one so very, simple example if you have a language model that is that is modeling, one word at a time, the, probability of the sentence the sky is blue, can be split up into this what we call the prior probability, of the then, the probability, of sky given, the probability of is given the sky and so on and. Then once. You have these probabilities, you can start generating data, by. Guessing what. Will come next, one, step at a time so if you've got this distribution of all the possibilities, you can you can pick a sample from that you can choose something you know maybe. After. The word the. Sky the, word is is, is, quite likely there's, a few other words that could also you. Know you, could. Also think of the, sky over, the sky above something like that all of these will be given a different probability and you're going to pick something according to that probability and then once you've picked it you kind of treat the gas as if it's a real thing you feed it back into the system and then, you guess what comes next and this is so why I think that generating. Auto regressive Li is a little bit like hallucinating. Or dreaming it. Has this flavor of there's. Something that you've just made up and then what having made it up you're going to treat it as if it's real and your brain is going to deal with it as if it's if, it's something that actually happened. And. So what is this this, is a very nice. Illustration. Of an autoregressive model, that I found in the Internet's advice someone. Called dan cat, who. Kindly gave me permission to use this, and. What, it illustrates is, that underneath. These kinds. Of autoregressive models, there's basically a tree of possibilities. So what he's done here is trained, actually, quite a simple language model, so this is not a sort, of state-of-the-art, powerful, neural network language model is a simple, I think, trigram, language, model which means every. Word just depends, on the three words before it. Or. Is it the two words before it's something like that anyway so and it was generated by taking a novel by. Jeff, noon, channel. Skin I think it's called a channel one skin I thought it was anyway. Trained. On that novel he gets a set of sort of probabilities, an. Auto regressive model that kind of has a certain degree, of certain. Belief about which words will follow which, so. Rather like the example we saw before with the generated, poem what, this does is reveal the the kinds of patterns that there are in the text that it's trained on anyway, he then generated, this sample the sentence the whole forest had been in East the dyes and so on I. Think maybe the the first few words were provided. By the author and then it was allowed, to generate and what what this image. Illustrates is, all, of the other branches. All of the other possibilities, that could have been made when. That, sentence. Was generated, so you can see there's these certain points where there's.
Different. Choices the the. Model. Could have made the point when it said anis the dyes that could have said recorded, issued set up you, know lots of other things all, of, which would have been at. Least in in this kind of this. World somewhat, consistent. With the start of the sentence. Her. Temporals could have glowed instead, of her temper temples, wired and so forth and essentially. Any one, of these branches could have been followed to generate some other piece of text and I just find this helpful to keep in mind when thinking about these types of models that this is there's always this tree underneath them and so, for ago, back to this idea of Otto. Suggesting. Text, if you if you're using you know for example your Gmail app and you see that having typed part. Of a sentence it suggests, continuation. For that same well that's basically doing is, taking a tree a much more, you know richer and more. Kind of complex tree than this one and picking. Out branches that are things. Are particularly high probability, things that are really likely for you to actually type of course it could show you farm or it, could give you you, know it could completely overwhelm, you with choices but then you know it would become it, would get in the way basically. So. You. Know that that, basic, model which is very simple as now you know been taken quite far so I don't know how many of you have seen, this unicorn. Story, that's become quite a sort of sort. Of a poster child for, for, generative models this is from earlier, this year from. A group at open AI they, trained a very large very, powerful, neural network on a huge amount of data and then, basically followed the procedure I just said so they gave it in this case they gave it a paragraph. This, the part that says in a shocking finding, down to the. Unicorn spoke perfect, English and then, they allowed it to free generate, conditioned, on that paragraphs is essentially, asking the network to say okay if this is how this, article, starts how should it continue and, it, continues, with this you know. Remarkably. Sort of consistent. Description. Of how the scientists, named this population, their, Ovid's unicorn. He's from the University of La Paz which fits with the fact that it's in the Andes Mountains.
And, They're. Kind of what's, what sort of. It's. A little bit hard to get across the flavor of why this is interesting so it's very easy to look at a generated, image and say okay that looks like a real image that's amazing, generating. Text doesn't seem like such a difficult, thing like a person, can sit down and write a whole bunch of text very quickly, in. A way that we can't generate a photorealistic. Image very quickly but. Actually. Text in some ways is harder for these algorithms because, in order to write. Something, like this it needs to remain. Kind, of contextually. Consistent. It has to keep around the fact that it's talking about unicorns, and the Andes Mountains, that they spoke English, and. Importantly, you know this is something obviously this is something that it had never read before right there were no articles, in the training set about unicorns, and in the Andes Mountains so. It has to given, some new piece of information as to kind of assimilate it and stay. You. Know sort, of relatively, consistent. With it and another thing that I think this interesting, is when. We look at these generative, models we, the, mistakes they make are super interesting, because the mistakes reveal something about what. The network's kind of world model is what it's learned so there's some odd things here one of them is that the, for the unicorns, are for horned, and so you know the one thing the sort of defining property, of unicorns, that has one horn. Even the name you know it's, got uni in it so there's something like that feels, like a mistake that a person. Just wouldn't make you know in the a person, would just sort, of you know have or automatically. Have kind of. Visualize. These things containing, a single horn, okay. So. So. Rewinding. A little bit here to some. Work that I did and. This is by. Deep. Learning standards this is basically a prehistoric papers, from six years ago and. The kind. Of the the point here was just to show that this same method, of autoregressive prediction, can be used with continuous, data so it's not just about predicting, one word after another it, can be used for things that when, I say continuous, you, know things like for. For images and sounds, and things that don't Polly said things don't have consists, of a set of discrete words and. So in this case this was for, online handwriting. And. The, data was recorded as a set of pen coordinates, as a as a person moved there as, a person wrote on. A whiteboard and, what. This image shows is the. Set of predictions, made by the network, so after each blob in this image the. Next blob, is the network's prediction, for where it thinks that will go next the heat map shows you know where the red parts are where it thinks it's most likely that the pen will be next and. And so forth you know further out, and. It. Shows that there's a very just, just by doing just by by following. This this, simple protocol of basically, whatever I've given you try to predict what comes next it shows that network learns a lot of rich structures, it's. Amazing just how much you can learn simply. By predicting, data. And. Yes. Again you can you can think about this as a branching trees of this illustration shows okay given a particular. Series. Of pen strokes which is this kind of nonsensical. Stream. Of letters, you can look at possible, continuations. The branching, points the place where the tree could have could have grown elsewhere, according to this predictive, model.
You. Can also of, course and it's the original sort. Of motivation. For this work was that you can also what, we call condition, or control, the. Predictions, made by the system by feeding it some real text so this is a text this sorry. A text. Synthesis. Handwriting. Synthesis, sorry. Program. Essentially, so the idea is that you know given, given some given some text from his travels it might have been the, network, then actually, produces. This, produces. These images and if there's an online demo I think I haven't, put the, URL. On here it still works after six years it's taken some some some, work to keep it to keep it going on one of the University of Toronto servers, anyway. This. Idea, of conditioning. Is important, because this comes back to the notion of how to artists, use these. Systems as tools so basically you know as long as they receive some input in this case the texts, then, you have some control over what. They do next, you, know in the case of the of the text itself the the input is sort of like a prefix, it's a start to the story and then you let it continue but, of course there's so many other ways we could think about controlling, these systems, and and having more you know, more. Ability to modulate what they do but I think that will be covered a lot more on the other talks today. One. Thing we can also say is that you know, if. You take away that conditioning, signal and just allow it to free generate you get this kind of nonsense but. Within this nonsense, you see little islands, of kind. Of structure, so you see like the word there or the word he. Appearing. In there and what that shows is that even just that this level low level of prediction. The network has learned some of the high level structure, in the system which is something, is. Remarkably. Difficult to do okay. I'm gonna skip through the next few. It's. Never men in their songs. Too. Much fun so a bunch, of you may have heard of, wavenet you know that basically took the same.
Principle. And applied it directly to, you, know raw, audio data. The. First commercial, flight took, place between the United States and Canada in 1919. So, this is now used in production by Google often where if your phone is speaking. To you then it's it's it's running, technology, like this under the hood and, of, course you know that means that. People. Designing, these technologies, can you know they can they can choose what your system says to you to some extent they can modulate how it says them and then, get it you know transferred into speech and so there's lots of creative things you can do there because. It's a raw, waveform, you can also apply this to music. For example so. That's. From a network trained on a whole, bunch of, classical. Piano music and, you, can also run this thing again, take a Behrman, in their songs on a design opens, and uh Martin Anson Brooke Y&M president's 1 1 by row Nomis that's 2 Senate by hearing tilt table so. Once again you hear something that you know it's, gibberish but you can hear little islands, of words, in there and so this is again this is what happens maybe, maybe the key point to take away from this is just how important, that conditioning. Signal is that's the thing that provides, that the, kind of the high-level structure, for. These models and then they sort, of fill in the gaps is basically how they work ok, so one thing you might ask is how would you do this with images right it's all very well with a sequence of words or sequence of audio samples images aren't sequences, well. What you do is you just turn them into sequences, and you predict all the pixels in a particular order one after another so, going. Through this image. Here that's being generated at, each point in time it's predicting, each each one of these pixels, and. Using the kind of all of the previous ones have already been predicted as context, this, bar on the right shows the, the. Basically. The prediction, distribution. Like this shows what. The network thinks will, come next are in terms of this, is for a particular color channel within the image it's just a number between 0 and 255. And, what's sort of interesting about it is you know sometimes it makes these predictions where it's. What we call multimodal, there's, a strong probability that the next thing will be 0 there's some probability that will be 255, so probability of complete. Absence of a particular color or a probability of it very strongly being there and that's. Important. Because it's, that that's, the the branching point these are the decision points so it's back to the tree again the points when it has these two very, distinct decisions are. The points when two. Different images are two different pieces of images kind of branch off from one another and you get it has the possibility. To create different, you know a diversity. Of things. Ok, so again. You know we. Saw before about how quickly gans have advanced, the same thing is true with Auto regressive models, so this is. 2016. This is what they looked like trained, on imagenet. This. Is what you've got an image net in, 2018. And. This. Is what you now get in 2019. So it's basically. Reached the point of photorealism in a few years and, I should credit these are all these are all works by my colleagues, at, deep mines, one, name you might see in a lot of these slides is a Aaron.
Band A Nord we've really been pushing the envelope as far as these autoregressive models go if. You're still not convinced that those faces as well I think. It's safe to say that these are now indistinguishable, from from, real images. So. And. One. Of the key one of the key things that makes this most recent, model so powerful, is that it's it's got a kind of hierarchical, structure so it does this this this next step this tree auto regressive modeling it does that at several different levels in the system and that's important, again because that, way you can kind of create the high level structure first and then fill in the lower level details afterwards, so and these details they. Turn out to be very important, as far as the the actual, sort, of end product, of these models goes so you can think of them I mean maybe I'm stretching things a little bit here but you, could think of this as something like the creative process of the model is the thing that the person when, you're designing these models that's the thing that you're kind of creating right so there's no mathematically. Speaking it's hard. To argue whether it would be better to do it like this. Hierarchically. Or whether it would be better just to predict one pixel at a time this was being done before but intuitively we, had this idea that well, if, I was going to create a complicated, picture like that I'd want a sort of high-level structure. First and then I'd want to go down and put in the details and that's. How this. Is I think that's. A sort of illustrative. Example. Of how it is that in some. Sense human-like, creative, processes, end up in these algorithms because the people designing the algorithms, appeal. To their own kind. Of human intuitions, when they're making these sorts of decisions about how they should work. And. Another interesting point about that particular model is that it's. Relatively. Diverse. In what it generates so this is the ostriches. From from. You're trained on the image that ostrich class real. Images are on the right and you can see they're very diverse some of them are close-ups of ostriches faces, others are you, know ostrich, and ostrich behind a metal fence two ostriches are three together or whatever and. If we look at the generated, ones it's, not quite, as broad. Or as diverse as that but it is still quite there's a lot of variation. There I don't know if you can see there's also one ostrich with two heads so again this is back to this this, there's a it's, is interesting. The kinds of glitches that these systems make so you think, that it has a pretty consistent model, of this bird and then it's generating all these images but then it can make something with two heads and, this is sort of an ongoing area, of research is trying to understand, what, the. Actual, world models, what the actual representations. That these are, under, you know the underlying systems, what have they actually learned about the world they've clearly learned something, or they wouldn't be able to give us back things. That look real but, anyway, the point from the you know this issue of image diversity, is that. They're really trying to kind of, learn. Something, not just about you know a particular image, of an ostrich but really this whole set of images which when you think about it is an extremely difficult thing to do I mean this is you know infinite, number of possible, images. For all of these, you. Know types of animal. Types of building and so forth. And. Other some other models this is this is a Gann model, I'll. Talk about Gans a little bit more later I think but it tends to be a little bit less diverse, and so it, kind of comes back to the idea that.
The. Role the, goal of the Gann is just to create something, that looks very convincing as opposed to attempt actually attempting to model. A complete distribution. And I mean this this can be there are people have done, added specific things to address that but. I find. That there's an interesting parallel there again with you, know human artists in the sense that it's not necessary, so a writer doesn't have to be able to create any sort of character, it's, sort of it's sufficient. For them to be able to create certain types of characters, so you know there's always a sort of a bias. That every. Every. Artist has towards. The the particulars, that they seek. To represent and. I think it's it's. An interesting thought that you, know unlike. The. Sort of traditional statistical. Machine learning, approach. An. Artist, is not attempting, to kind of. Ingest. Everything, and reflect everything, exactly as it is it's fine for them to focus on one fact it's sort of necessary. For, them to focus on one's particular, particular, thing. It. Very, they've. Kind, of high-level terms there so, you. Know what does this mean. You. Know we talk about. Auto. Regressive generation, as being just this this series of things that are somehow dreamed. Or imagined, one after another I feel. Like there's a sort of you know you a direct parallel there, with with you know what you can think of as something like Auto regressive art so I think about Picasso, drawing. You know is. His, horses. Of whatever with a single pencil, line, Jack, Kerouac typing, out on, the road in a few days on a continuous, scroll there's, this quote by Allen Ginsberg first thought best thought this idea of spontaneity, being, important, in the creative process is this idea of surprising, yourself somehow because. And it's again back to this notion of the mixture, of. Of. Structure, and noise and, actually. There's a there's a story that, some of you might have heard about this Picasso or one of these Picasso's, doodles it's, probably apocryphal but I think it's quite interesting. Supposedly. He drew he, was asked to draw a, sketch, on a napkin in, a restaurant and he quickly drew something like that and then handed it to the woman who'd asked them and told, her that will be you know ten, thousand francs or something like that and she said oh but it only took you you know five, seconds, that's crazy he said no it took me my whole life and obviously, what he meant was his.
His Whole life had been spent training himself to be able to do this and this is kind of again that the, process, that we see in these types of models it takes a huge amount of time a huge amount of data to train these models this is where the the, GPUs, and the GPUs, and all the powerful computational hardware, comes and there's a massive amount of number crunching involved, in, learning. All of this structure but then once you have it and you've embedded it in this system at. Least if you're willing to kind of operate. In this very, kind. Of this. Autoregressive. Forward. Way, then. It's it's it's very efficient. You can you can generate things very rapidly, okay. So really. Though you know most novels, aren't written like that they're not written like on the road it's rather this you know slow painful, process of, you. Know planning sketching. Writing, editing, it's, very iterative there's like one little thing changed at a time and then you move on there's. This great great quote from John. Berger that I like he says he's, talking about his own writing process I mean he says I modify the lines change, a word or two and submit them again another confabulation. Begins so there's there's this idea that the words are are talking among one another every. Time you add a new word in that changes, the discussion, and you have to talk for quite a while longer before you can you can sort of you. Can decide on, what. That word should be, and. There's, a there's a this, is perhaps, somewhat tenuous way that you could see this type. Of process reflected. In generative models as well this, happens, when the most models are not Auto regressive it's not just the forward chain of probabilities, but rather you have mutual, interdependence ease, between many variables, so, X depends, on Y and Y depends, on X in, that case, generating. Those things together is not so simple you can't just pick one and then get the other because, they both influence, one another back and forth you end up with something that's more like a sort of a network. Of things that constantly. Interacts. With one another and so the Boltzmann, machine is one of the sort of I guess, one of the canonical examples. Of that type of model. Here's. Another one, this is from 2015, this was a model called draw. That. Again. Was worked. On by my colleagues at deep mind and. It's. A very interesting one in the it's sort of the, way it works internally, it. It, builds, up what we call the canvas. Inside. The network by iteratively, so this the the movement of these pink boxes, shows, the attention, of the system shifts. Around while it's creating, these images. Obviously. Their faces, up here and these are from Street View house numbers, on the right, and. The. Process, of kind, of finishing. These things is is iterative. Here like it maybe it stays in one part of the region and kind of. Crystallizes. That and then moves that moves to another region of the image afterwards, so. There's something it's. If, you think about it as a creative process it's a little bit more holistic, it's a little bit more like take. This this. This, sketch, or this blurry kind of, idea. And gradually. Kind, of make it manifest into something concrete. Another. Model a more recent model, that's kind of works on this principle is, what's called flow based models, and the idea here so apologies, for the the maths here but it's really you know if you just look at this little curve on the left the idea is to take something very simple a simple, you. Know in this case a white noise you. Know standard. Gaussian and, then, iteratively, transform. It until you end up with something complicated, and. So, I think the key thing here is it's. Not Auto, regressive it's not the idea of actually. Output in one pixel at a time or one concrete, thing at a time but, it is still iterative, so there's still this notion that you you can't somehow, you, know someone asks you to create an image it's hard to sort of create a complete, the, complete thing in one go it's rather it's a process it's a series of steps here.
Are Some phases created with a. Flow, model, interesting. Here you can see you, know these phases, are realistic. With their noticeably, more synthetic looking, than some, of the other ones the. Ones I showed before some, other ones we've seen they're, kind of there it's trained on celebrities, so their faces are sort of probably on. You. Know improbably, perfect, to start with but these are just these look too perfect they look airbrush, they look symmetrical. And that's actually like a sort of a common. Failure. Mode in some sense of. AI. Of, generative models is that they have a tendency to take the mean they have a tendency to average things out. And getting. Away from that and creating, things with I think what someone once referred to as digital dirt I think, that's been a really sort of key. Part. Of when artists. Started to get really interested, in generative models when they started to create things that looked kind of messed. Up an interesting as opposed the things that looked sort of too perfect, I'll, go through this you know talk about ganz but it's a we, already mentioned ganz probably. You've heard about it a lot in. Some ways what I think is interesting there ganz is you. Know what we've talked about for so, far them if we think about again the couple think back to you know how this, these things reflect human processes. The models we've seen so locked so far, are. Kind of like the positive mode of creativity they just generate, stuff but we know that when we, are making things ourselves there's, a critic there there's a self-conscious, critical, phase where, we look at what we've done and we, decide if it's good and we modify it accordingly, and so forth and in. Some ways the the gaen bakes. That, notion. Into its structure so it has two networks, a generator, and a discriminator, the. Generator, is trying to make data that, will fool the discriminator, into thinking it's real and the discriminator, is trying to do the opposite trying to distinguish it from realities you can think of it as a sort of battle between a forager, and a detective, or maybe a little more broadly an artist than a critic I mean that's it that's a little bit unfair a critic. Isn't an, artist, isn't attempting, to convince a critic that something is real but, anyway there's. This basic opposition. Between the two things and that that's, it's. Proved incredibly fruitful. So we have so, these were the sort of the, images I think that first caught, everyone's attention in terms of photorealism being, coming, from neural networks and it's funny to think that was only you know two years ago right it seems like a long time to me at least but. This, is you know it's still it just shows how powerful, is this idea of explicitly. Having, a model. That is attempting to. Kind. Of to refine, and to criticize the thing that the system, is doing anyhow. What. I often think of is I have a good time to play this clip. Here so this is from. And. Very. Briefly the cloth is that a bunch of scientists, are orbiting the planet Solaris, and finding that it's creating, things.
It's Sort of like the big generative, moment, and in the very end of the film, the. Hero finds, himself back, in what, appears to be his childhood home with his father he's, looking through the window of it but there's something not quite right with the sea it's. Raining on the inside of the house and the rain appears the pop or something nice feeling, as if it's sort of something. It's. Regenerated. A reality, looks, very convincing, but it's not quite right. It's made by generator, moment. As they, get more and more powerful the, mistakes get more and more high-level. Interesting, and they reveal more. Semantic. Information about, what. It is that they've understood what, it is. So. And. So from there I'm just gonna skip past, so, these you know I had some slides, about possible, sort of now, outmoded, slides, about artistic, tools. Based. On Ganz but I think you're gonna hear so much more interesting stuff about that I think, maybe a one, last important, point I'd like to make is you know I've, talked about these models so far as as if they're as being. Somehow creative. In the human sense but you, know all they're all they are trying to do is to make things that look or sound real right it's just very similitude, it's just take it give, me an image that I can't tell isn't. The real image but, that's not what we you know expect, we expect a lot more than that from human creativity so this is you know some canonical. And I apologize, for my lack, of diversity. And my choice of artists here I'm just I'm reading, lazily. Reaching for canonical, examples, and of course these tend to be dead white European, males. Lots. Of other people you could put on here but the idea was just you know this. Is non-figurative art this, is you know 20th century you. Know obviously. We. Moved away from slavishly. Recreating reality, a long, time ago in terms of our this. Malevich, said this quote trying desperately to free art from the dead weight of the real worlds is that you know an act of like reaction. Against, this idea of just recreating. Things and so, you know the question becomes or one question, obviously, one answer to that is well that's fine the machines are tools that create things and we'll just use those tools but one, question you could ask is could. The machines themselves actually, be creative, and and how would you do that right right you you you you. Can't have the objective functions we've had so far because those are just try. To model reality try, to predict things correctly. We'd. Have to have systems that could actually surprise, us for us to consider them creative so how could we train them to do that well. One kind of theory. Of thought here and this is not specifically, about creativity, it also relates to the idea of exploration. And curiosity. Is, to. Try to make systems that are what we call intrinsically. Motivated. So they're driven to learn, for the sake of learning rather than being extrinsically. Motivated by. Target. Some rewards that we've provided them so as long as as, long as we've specified what the targets are as long as we've said you'll get rewarded for this and not for that it seems pretty hard for the system to really be creative for it really to be open-ended and to really surprise this so it's more like an idea of can. We embody. The notion of art for art's sake okay.
Apologize. For the, equation. Here but it is an extremely fundamental equation. You, know this is from Shannon so paper of 1948, that's kind of like the thing that underpins the whole of the information, age in some sense and, the idea is that. The length of a message is related. To the, log probability, of that message so, they in. Your very simple terms this means that when all of these algorithms have said at the start they're fitting probability, distributions, one. Way you can think about that is that they're they're learning how to compress, data they're building a model that somehow shrinks, the world to something more compact. And, there's, lots of kind of formalizations, of this idea people talk about the Kolmogorov. Complexity, the minimum message message, length minimum description length, that. Has the same flavor that basically, the. The more you can learn about the system the more compact, your description, of it becomes. But. You. Know what happens then if you, want to generate what you want, you want to use this in some sort of creative way so what happens if you're able to create, new data or even just, choose. Where, to look choose where to point your camera. Then. Suddenly this idea of compressing. Doesn't make much sense right if your goal is just them to compress, things and to learn them well then, you should create nothing at all or you should just go and you know sit in a corner and stare at the wall so that you never get any more information but the information you have is very compressible, and so, my PhD, supervisor jurgen, schmidhuber had, this has. This theory about how to kind of. Resolve. This paradox which, is that you're. Not learning, how to just. How to compress everything rather, you're looking for, things that maximize. What he calls compression, progress, so, seek out data that maximizes. The decrease, in bits of everything you. Have ever observed and, in perhaps everything you. And the culture that you're in has ever observed so create the thing that makes the most sense of the world in if, sense is being. Measured as your. Ability to now, further, compress, it right in the sense that you now have a better model and therefore a more compact model and one of his sort. Of bumper sticker, you know slogans, for this as happiness is the first derivative, of life so there's this idea that what makes you happy is this, change is it's, not the it's, not the absolute point of of how well you've modeled things but rather the. Rate at which is changing the way rated which is decreasing, and. I think I mean this is this is as, far as like creativity, goes I think it's fair to say that this is more or less unexplored.
Territory, I mean these these. Jurgen, has these ideas have been around for quite a while but. Obviously it's much harder to to, you know put this sort of thing into practice than it is to explicitly, train to generate, a you know a convincing. Audio signal or something like that and, I guess the other interesting, question we would ask here you, know and this sort, of goes. Back to some of the things Georgia was saying is you know if, we, did allow them to, you, know freely, create stuff. Well for one thing you, know what would we then be doing what would our role in the creative process actually, be right there has to be some way in which we're guiding it otherwise we're just we're, just spectators, to it and the other thing that I think is interesting to think about is would we even recognize, their. Creativity, as our own so even if they were able to do this the machines and able to kind, of create things that help them to make more sense of their world as they've experienced, that would. It be just fundamentally, alien to us or not and I think that's this is a it's a very interesting question as the weather if you left them to, sort of, create. Things in this very open-loop way you would, start to actually you, know there. Would be some common ground, between that and what human artists create and I. Think I'm gonna wrap up there because, I'm, running short of time so, thank, you very much. You.
2020-08-20