Is Generative AI the Future of Game Development? | Artifacts #02

generative AI it's everywhere every new site every social media platform people are talking about it using a suite of powerful new AI tools can create text or Voice or graphics with a simple prompt it has opened up all sorts of exciting applications and the hype is everywhere you can write sheet music even write the lyrics that go along with it write code for vector graphics write code for animations you could even Bing if that's something you want to do or more likely everyone's going to use it to plagiarize that college essay they've been putting off all semester well not you a lot my audience are lovely people who respect academic Integrity don't go proving me wrong now however and amongst all of the big reveals the elaborate press events the tech demos and of course the social media hype it's difficult to truly understand what the state of the art is and whether it is actually going to have an impact on the video games industry as generative AI ushering in a new age of tools that will make game development easier affordable and more accessible will AAA games now only take weeks even days to make instead of years are we going to get the source code and assets for games like Skyrim just by typing a prompt into chat GPT the answer and the reality that goes alongside it is of course much more nuanced than a sound bite that can bounce around on Twitter and at risk of violating the Creed of the internet I'd like to dig a little deeper into what these Technologies are how they work the issues that surround them and their production Readiness in context of how video game development actually works I'm Tommy Thompson and welcome to artifacts here on AI games my new series all about procedural generation and generative AI techniques join me as I tackle the big generative AI question that continues to haunt my news feeds and highlight that the path to generate of AI becoming a useful tool in the games industry will be a slow and gradual process plus stick around for the end as I ask you that's right you for your help and truly understanding the impact this is all having on developers content creators players and everyone else in between [Music] so let's start with the basics when we say generative AI what do we actually mean by this it's a buzzword that has dominated news feeds for much of 2022 and into 2023 but it's on me to explain it for you so here goes generative AI is the more streamlined and marketable term for any AI system typically when built using machine learning that can create an artifact of some sort as its primary function GPT creates text stable diffusion and dialy create images GitHub co-pilot reads code speechify generates voice audio you get the idea in each instance they're provided an input and from that prompt it creates something designed to evoke it whereas we typically assume an AI system creates answers to problems it's seldom creating an output that we as humans can use in Practical and creative ways it's called generative AI because it generates interesting artifacts for the user the term generative AI caught on courtesy of the work by Ian Goodfellow and his collaborators in 2014 on the use of generative adversarial networks or Gans it can be used for image generation Gans sit alongside another form of machine learning known as vaes or variational autoencoders that emerged from research by Dietrich pigma and maxwelling in 2013 and has since proven useful for image generation and natural language processing Gans and vaes have been two driving methodologies for what we now call generative AI while their underlying training approaches differ what they share is a process in which they seek to understand the underlying properties and traits of the data they analyzed and store it in a compressed format a concept known as the latent space if the system can process the latent space sufficiently then it can later seek to decode it in new and interesting ways that still reflect the properties found within the latent space hence they can create for example images that look a lot leak images analyzed when it was being trained now we've already explored applications of Gans for games here on this channel episode 39 of AI and games highlighted use of Gans to generate Mario levels meanwhile in our overview of texture upscaling in episode 61 I looked specifically at how this technology was used as part of the art pipeline for the Mass Effect legendary edition while these breakthroughs are impressive it is important in the interest of historical accuracy that I point out none of what we're seeing in these systems is conceptually new this work is all built upon Decades of existing research in machine learning and computational creativity generative AI is fundamentally a more marketable term for procedural content generation or more specifically procedural content generation using machine learning that doesn't necessarily have to be used in games the idea of systems using Ai and machine learning to generate output for Creative purposes is a well-worn one and it's again a topic we've covered here on this channel at Great length ranging from computationally creative systems like Angelina the AI system that designed its own video games back in episode 20 to Mario level generation like I said already in episode 39 all the way back to particle weapon generation in the shooter Galactic arms race which was way back in episode 5 right at the beginning of my weird and wonderful YouTube experiment while the phrase pcgml has in fact been adopted in some scientific research circles for some years now it clearly doesn't roll off the tongue as effectively as generative AI though it does help squash the argument that I shouldn't be making videos about procedural generation on my channel given it straight up puts AI in the name so thank you to whichever marketing team put that together however the big reason generative AI is making such Headway as the recent gains in deep learning that resulted in significant improvement over previous generations the output of gpt4 is a vast improvement over the same system even a year ago meanwhile the output of the likes of Dali and stable diffusion are very impressive and highlight the potential of this technology to a great degree plus thanks to cloud and Edge computing infrastructures it's now much more accessible for a regular person to utilize with text-based interfaces on web pages and even mobile apps you type questions to check GPT for it to answer you tell mid-journey what type of images you want it to create by creating such a simple interface it helps not only make it more palatable for the average user but also maintains the Mystique of it all that it reads your text and then acts upon it making it seem much more intelligent than it truly is a big part of the Mystique surrounding generative AI is the aforementioned gains and overall output and performance the big application areas of generative AI such as text generation speech to text text-to-speech text to image all of which have been in some form of development for decades now have all seen significant gains courtesy of three main elements as stated already the first one is the improvements in deep learning AI notably in the development and training of large-scale neural network infrastructures secondly there's the access to large-scale data sets which allow for richer understanding of the underlying feature space and the ability to produce a broader range of responses that better align with our expectations which is courtesy once again of that data set lastly there is simply the sheer processing power that cloud and edge-based Computing now affords Training large-scale Systems in the cloud and then deploying them such that they're readily accessible and desktops laptops phones and any other devices that carry an internet connection there is of course the fourth underlying element money in the past 10 years we've seen a huge shift in how AI research development is now funded since the Inception of the field in the 1950s the bulk of r d and AI has actually been conducted in University Research Labs with the majority of funding being provided courtesy of government research grants and perhaps with some larger businesses throwing in money either to support specific projects or research Fields the proportion of AI research coming out of corporate Labs was relatively minimal nowadays as is evident by the owners of man any of these big AI systems it is corporations taking on the bulk of the work and at a scale previously not plausible due to the sheer amount of money being thrown at it this has led to a rat race of sorts as companies ranging from the biggest of Corporations to the smallest of startups pushing hard and fast to make their big announcements investors are pumping billions into the sector with the prospect of massive returns 2021 saw an estimated 70 billion just being invested by Venture capitalists although that did drop turning 46 billion in 2022 however when you consider the other speculative areas such as web 3 nfts and VR are kind of on the way in plus all the recent headlines surrounding generative AI you can bear investment in this area is only going to continue all of this has led to Big pushes for new platforms embarrassing false starts the odd retraction or rework and a mixture of hype enthusiasm but also some understandable apathy and frustration with the state of generative AI given the underlying Financial incentive be it to be the first to market for customers or simply to direct even more investors their way or seeing new AI tools systems features Tech demos and the like being announced on social media or via press release on an almost daily basis capitalism is operating in high gear and only time will tell which of these companies and their products will still be standing when the dust settles so having painted something of a broader picture what value is all of this in the context of games there is a huge amount of potential in moving games forward through use of generative AI techniques using AI to create textures and Sprites generate animations for specific actors writing descriptions for Quest logs or lore Bibles generating storylines for role-playing games creating real-time conversations with non-player characters that are in World relevant and react to the player's input all of these are attainable and in many respects developers of all shapes and sizes can start using tools today that works towards these goals an indie developer could use mid-journey to kick start their ideation process helping establish mood boards and concept art meanwhile a programmer may use generative programming tools such as GitHub co-pilot to assist in writing code for a new feature we're in a new age in which generative AI has the capacity to change how developers make games and as stated it's a notion that many corporations have jumped on with great enthusiasm with the recent announcements of say nvidia's ace platform for NPC creation through to the likes of in-world ai's Origins demo being something you can wish list on Steam it's catching the attention not just of developers but players as well I mean geez if I had a penny for every comment on my Facade video saying Hey imagine this but with chat GPT I well I wouldn't be rich but there's certainly a thread over two in my future but regardless of their desire to Corner this part of the market the path towards success in this space is not a straight line building these tools such that they are safe to use appropriate for developers needs and will not result in the developers suffering public embarrassment or even face litigation is an ongoing one yes as I write this now no doubt many people can point to an example on social media of someone proving it's possible and I'm just being a mood killer and that's Dr mood killer to you thank you very much I mean to make the point for them here's Ben Bonk using chat GPT to generate code for a Unity project or more recently samyam made a game Prototype in an hour using generative tools but as always the path between oh hey we made this thing work for a tick tock and this is a shippable commercial product is vast legally unclear and contains a myriad of pitfalls along the way we'll get into some of these issues that reflect the current state of the art in the moment but it's worth pointing out that there are a myriad of companies out there trying to solve the problems I'm about to address AI don't engine Creator's latitude alongside the likes of hidden door are working on language models for story generation Ubisoft is experimenting in text generation for script writing Activision Blizzard has their own art generation tool chain in World AI convey and character AI are exploring how to make more realistic avatars with workflows that plug into game engines meanwhile Unity is exploring how to interface generative AI into their engine while Roblox already has tools that are available for their creators to use the state of generative AI for games is going to evolve drastically in 2023 and Beyond and you can bet we'll be digging into some of these in future videos here on AI and games but right here right now these are some but not all of the big issues impacting the field okay first and foremost let's just start with the basic reality check well we have seen orders of magnitude Improvement in generative AI systems the technology is often nowhere near as capable or powerful as it is advertised while there are many hype Merchants advocating that generative AI is going to transform the games industry today we're still months if not years away from seeing many of these Technologies be as reliable sustainable practical and seamless as the tech demos suggest it doesn't help of course that this is right off the back of the web 3 and nft hype which also suggests that it was going to transform the industry and yeah I suspect a lot of people went and changed their LinkedIn bios sometime in late 2022 as they opted to change bandwagon but unlike nfts AI does actually have the potential to change game development in a significant way but these Technologies and their accompanying tools and Integrations are all evolving at different speeds all around the world as different companies take on the big challenges of how to make this Tech more palatable with some already having a head start making real gains in the field but this is just the start of a longer Journey as generative AI becomes more palatable for practical and commercial use it's a lot of work to go from Cool Tech demo to establish tool chain ready for developers to use and it's important that when a new demo is promoted that the reality checks kick in sure a tool can generate a cool bit of dialogue and interesting looking texture or even a complete non-player character or game level but does it produce it at high quality 100 of the time every time if there are still issues and risks that require human intervention then that inhibits their efficacy if they're advertised as solving problems without user involvement plus it's worth highlighting that a lot of generative AI tools need to solve problems game developers are actually facing and in a way that game studios operate I'll be coming back to this later in the video but it's important to recognize that games even from the smallest of Indies to the biggest of AAA are complex Productions often with multiple people operating on the same projects in different capacities what generative AI needs to be doing is solving problems that game develop actually face and help bootstrap productivity rather than seek to replace them in my recent AI 101 episode on the use of machine learning in the games industry I highlighted that one of the big issues that prevented it from being used until the last couple of years was that the techniques being created and the problems that researchers were trying to solve or seldom the things that the games industry saw as needing solved it's only now with ML power techniques solving specific production challenges that it has taken on a whole new relevance I'd encourage you watch that video if you haven't already given the arguments made from machine learning for games as a whole are highly relevant to the conversation now moving into the Practical elements perhaps one of the biggest issues that surrounds a lot of generative AI tools and this is not just in the games industry but the field as a whole is the data used to train them text generators need a significant amount of text in order to learn from build upon and subsequently write their own similarly an image generator needs access to a significant Corpus of image data from which it can then generate as well of course the issue then is where does that data come from and critically did the individuals who created that data give their permission the vast majority of generative AI tools are using training data that isn't open source and in some cases has not been outright declared what exactly is in it this leads to two problems for a user of these systems first that if you don't know what's in the training set that leads to unknown outputs and an inherent lack of trust in these systems after all generative AI systems are in essence very complex copycats a text generator mimics the text that reads an image generator mimics the art that is in the training Corpus we want to know it's being trained on relevant data such that we not only get practical outputs but also avoid undesirable ones a non-player character that spews racist sexist homophobic or transphobic rhetoric because it learned to say it thanks to a training set that incorporates the likes of message boards and chat forums isn't going to prove palatable for a big budget Triple E release now compounding the issues of data and access also the legal ramifications of these generative AI tools to say that there are serious legal issues surrounding generative AI is quite frankly a generous understatement the source of the aforementioned data is a big problem image generation tools have been found to contain within their data sets the work of artists who did not provide their consent and researchers found ways to be able to effectively pull complete images out of a generator that are near identical to those in their training data only helping highlight their culpability significant backlash from the online art communities on the likes of art station have Arisen to prevent companies scraping art without consent while also seeking a boycott on air being posted on the platform and outright legal action as artists demand their work be removed and they be summarily compensated and of course that's just in the art world there's the ongoing legal case against Microsoft GitHub and open AI which opens the question of whether their code generation tool learned to program by effectively breaching licenses on copyrighted source code assets which has huge ramifications for the future of data acquisition for generative AI systems this of course brings us to the issues of copyright and fair use if AI generated art is used as part of a game be it concept art character designs textures and more and it is subsequently found to have been derived from an artist who did not give consent this potentially has big ramifications not just for the crater of the generator but any art that could be gleamed to have derived from it no big studios are going to be willing to tackle the legal ramifications of this lest they can guarantee they're building their own internal training data and that is something that many studios are now actively exploring however it doesn't solve the other legal problem claiming ownership of the generated assets currently there is no legal basis for AI generated artifacts to be given the protection of copyright status just because you told chat GPT to write those lyrics doesn't mean you own them similarly asking stable diffusion to create an image that doesn't mean you own the image GitHub co-pilot can write the code but you don't own copyright code in fact nobody does even the generators wouldn't own copyright because well their programs they're not actual individual human beings instead all of these generated assets are free for anyone to use this prevents a lot of these assets from being useful in a production whereby you're building a product given you want to have copyright over all assets and have sought appropriate licenses for use of those that you do not if not then you run risk of other people stealing and using your assets however you see fit this is going to be an interesting development in the coming years as countries around the world seek to tackle the Myriad of legal issues that impact AI systems so with the legal thing out the way there's still what is to me the core problem that the vast majority of media hype conveniently avoids humans you know the people that make the games are the source of creativity in games and generative AI needs to be built to support that much of the discussion around generative AI has focused almost exclusively on using AI to effectively replace human input the aforementioned debacle surrounding generative arts and the theft of assets has helped fuel The Narrative the AI will effectively replace creatives this is of course then fueled by speculative hype suggesting generative AI could replace texture artists concept artists 3D modelers animators sound designers voice actors and even to some extent programmers as well it's a classic case of conservative capitalism advocating for the automation of work that people not only rely on for their livelihood and build entire careers around but also work that they actually enjoy doing and of course many a company has jumped on the bandwagon advocating to investors that they seek to yield increases in profit margins by simply laying off the humans and making the AI do all of the work now in all fairness there's a lot of work out there in every industry that would benefit from being automated and to be realistic about it generative AI is going to change how many jobs and roles are performed not just in games but in virtually every business sector but that's the key part it changes how the job is done and it will do that by supporting a human to do that job now I'm not naive some jobs will be heavily affected and some teams will get downsized but the situations where people advocate for AI to replace developers is downright foolhardy at best and at worst stupid games are a creative medium and we require developers to be at their creative best to get the very best games it's it's that simple sure AI can help support that and facilitate that but critically the AI needs to be built to help creators achieve their creative goals not seek to supplant them this is perhaps indicative of the often Justified backlash against generative AI it is positive as a means to replace someone in a creative field rather than a mechanism to support them to do their job better even for companies that do pursue full automation this ultimately will not work giving you always need humans in the loop as part of that creative process in order for it to work effectively so why not just get the AI to work as a tool that enables them to do that job more effectively returning once again to my AI 101 on machine learning for games I highlighted many examples of how ml is now being deployed in the industry whether it's texture up scaling motion matching cheat detection or any of the other applications I mentioned in that video they all have one thing in common they're helping solve an existing production Challenge in a way that is faster and more effective than throwing more manpower behind it I assume it's also hopefully making some developers lives easier you want a simple example of this and where it should really be going look no further than Speed 3 for those not familiar Speed 3 is a technology used by pretty much every AAA game in existence since Elder Scrolls Oblivion to create trees in their games heck it's even been used in dozens of major Motion Pictures including doing several entries of the Marvel Cinematic Universe since Speed 3 Cinema was released in 2009 you know why because artists don't want to have to sit and manually create tens of thousands of trees and they also don't want to just give a video game level to a generator and ask it to tree it up but if a tool like speed tree can help a level designer get the trees in there faster at higher quality and at scale then they can work to make it an engaging and interesting play Space it's speeding up production by giving an artist a tool to help them do their job faster until generative AI can reach that level of quality and interactivity in a myriad of different forms it's not going to prove all that effective with this video I hope to paint a broader picture of where we are with the generative AI hype given it's going to be an increasing Focus both here on artifacts and across Ai and games in the future as discussed already we're seeing a lot of these Technologies begin to mature and some companies are getting ready to Showcase their products as useful tools for game developers to work with in fact at the time of writing I'm already preparing a handful of video projects in which we're going to see some of these in detail but for now it's worth highlighting what the state of the field is and provide a more grounded take no we're not in some generative AI Nirvana right now nor will we ever be in my opinion but it's clear that there is great potential in a lot of these Technologies enabling game development to be more streamlined higher quality and potentially more accessible but that doesn't shake off many of the Justified criticisms that we've discussed throughout this video and naturally many of you watching will have your own thoughts on this either as a developer or as someone who simply enjoys games I would hope that in a future video we can highlight and discuss some of your perspectives on this topic and to that end I need your help Linked In the video description and in my pinned comment below there's a link to a survey that I've put together about generative AI for games if you can spare 10 minutes please fill in your thoughts on the subject the survey is designed for both game developers and regular players to capture's broader range of opinions as we can I'd love to then come back and discuss more of your thoughts in detail in a future video but for now thanks for watching this episode of artifacts all about generative AI for games given the ever shifting landscape it felt critical to me that we talked about this in some detail and hopefully give a broader account of what all the hype is about and where we currently are in the state of the art as always episodes just like this one wouldn't be possible without my patreon supporters this has been a topic I've been wanting to tackle for some time now and a special thanks to my production and Early Access patrons have provided a lot of useful feedback on this topic as I've been putting it together to support Ai and games please check out the links on screen and in the description and find out how you can sign up on patreon YouTube and even sub stack take it easy folks and I'll be back [Applause] thank you [Music]

2023-06-27 12:38

Other news