KEYNOTE | The Spread and Impact of Game Innovation
Hello again. I was outside and I kept hearing clapping throughout the entire conversation and I kept running to the door going, It’s over? It's over already. I'm glad you enjoyed that.
So, where are we? So we all know now it's common for advanced technology to be utilized in homes and works places. But did you know the games have been a driving force behind this phenomenon? In this presentation, we review the impact and spread leading technologies whose origins are in games. It's my pleasure to bring back to the stage two long time Games for Change community members and inspirations to our to our entire community. Constance Steinkuehler, and Kurt Squire.
Both are professors of informatics at the University of California in Irvine. Please welcome Constance and Kurt. Good morning, New York.
My name is Constance. I'm Kurt and we are Co-directors of the Game Learning and Society Center. We're also married, so we made a basic rule.
As long as we don't get in a fight on stage, we're going to win. Thank you. So to start, I guess I don't have to look up at that other screen. There we go. Oh, no, I'm jumping forward now.
Sorry, I'm already goofing this up. So our project that we're reporting on is all about a last year of work that is an exceedingly ridiculous, ambitious project that was kind of an itch that we wanted to scratch from a long time ago. And it's basically the question of, you know, we talk about games for impact, but how have games actually games and game technologies gained techniques? How have they benefited humanity and other fields writ large, broadly with no boundaries? I mean, what could go wrong with this question? Yeah. So if we've been doing this for a while, you know, we run into a lot of claims. Of course, there's a lot of hope.
A lot of hyperbole, a lot of like all games are revolutionizing education, health care, marketing. And so what we wanted to do was really try to dig in and see, okay, where are games, having an impact as of 2023? And are there some themes and things that we can see that might tell us more about what games are good at, where we can have hope for the future? And then also a little bit in terms of telling even like students like ok. if you, if you're working in games and you want to start applying these skills toward making an impact outside of games, what are some sectors and what are some areas and skills where we think you actually might have some success? And of course that's where my academic empirical researchy hat comes on. And the first question I have is like grabbing it back from you is thinking about, you know, if you're going to ask a question like this, you have to operationalize the research question. So the first question is always, what do we mean by video games? Right. Of course, there's like games themselves that are games, commercial and otherwise we all do games for impact of some sort.
So, of course, we're talking about games, but there's actually more than that. When you look across the literature the way that we did, you find that there's components of games, game engines, physics engines. Is a separate sort of subcomponent of game engines.
There are game design elements. So for example, game design elements being some of the key components of gamification efforts, which are now broad across broad spread across multiple fields. There is input devices, so believe it or not, the kind of input devices that have been mobilized by games created and originated out of games are now used across a broad variety of sectors. And finally, you've got AR, VR, which is a whole. Yeah. And with AR, VR, you know, clearly it's there's part center innovated within the games industry taking Pokemon Go right, which was partly games, partly funded by Google.
So with this search that we start the things that clearly were games and game related, we didn't necessarily just throw it out. If we use the word AR, but we were trying to understand where games really had the most impact. So the next piece of operationalizing, of course, is this notion of contributed.
I mean, what an ambiguous word where the we mean there is there's really a very one. There are five different relationships we're looking at in this study. The first one, of course, is just first order impact of games. So what are the impacts of video games on people, economy, etc.? The next four are really related to the components of games or game industry innovations and technologies and techniques that have helped other fields.
And there you have you know, there are those that we've invented or created. I say we as though I'm the games industry. Here you go. I'm the games industry speaking, you know, and but, you know, technologies and techniques, for example, graphics or rendering techniques that are invented that were invented by games.
But there's also those that have been catalyzed or advanced by the games industry. So, for example, I games did not come up with artificial intelligence, obviously, but we have in fact catalyzed the progress. AI there is applied or transferred games really there we mean more serious games. And then believe it or not, there are studies now of cognition, neuroscience, social interaction, behavior that could now that are now done that would not be possible without games. So games as kind of an in siliicone petri dish for looking at human behavior and cognition.
And so this is back to you. Sorry. And so in the final term, we have to figure out is what do we mean by other fields? Right. And of course, I put in this really tacky graphic because I love it. But, you know, we start off thinking we'll just look at everything everywhere.
It will be great what could go wrong. And yet we're looking out the window. This is so cute and simple. And then you start looking in and you're like, Oh, it's an ever expanding universe. It's complicated. Imagine trying to do a search where there is no boundary to the discipline or domains that you're searching. So that was the enterprise we took over.
It was intended to be about a 14 to 16 week project, and it's been a year. So welcome to our lives. So in order to tackle this, as we said, oh my God, this is a crazy universe. We did what anyone could do is hired a team around a team of undergraduates. So who, who we really and we'll talk about we'll talk more about the methodology in a second, but who helped us identify collect, read articles and bringing all this into place.
And then there's, of course, us too. In the bottom. Method lies a walk through a little bit here. So there were really three phases to the study. The first was just identification of defining what we're going to be our seed terms. We started off with industry documentation. So for example, like Unity or Unreal, actually have some documentation about how it's being used outside of just the games industry.
Then we talked to a bevy, a whole army of amazing research librarians, first in informatics and or computer science related technology fields, but then expanding out. Then we use chat GPT for anyone else, love and hate chat. I have a love hate relationship, so I had many late night conversations with Chat GPT, which sometimes got a little bit exciting and other times exceedingly dull. So we had multiple rounds with Chat. GPT 4 to generate seed terms representing games and game technologies and techniques, but also thinking about trying to generate a list of potential fields, trying to sort of carve the world at its joints in terms of fields of where we should look. We did have our human intern team validate every single one and believe it or not, about 30 to 40% of those offered were not real.
They were all made up answers, which were fascinating. A fun to see what Chat GPT invents about the games industry. Absolutely. You'd be amazed what it will credit to the games industry. Yeah, there's a joke there. But so we validated all of those terms and then went back again to more industry experts, did some interviewing to make sure that we were in the world of just like, Does this look right? Does this look like the things are anything obvious for missing sort of stiff testing? So those are the terms there were a little over 200 terms.
Those were then run through academic search. Complete academic search complete is just an aggregated database that is one of the more comprehensive ones. So it includes not just computer science, but social sciences, arts, you name it.
So it's across all disciplines and academics. We also only use English language as well as trade journals, newspapers. Yes, yes. It includes news and it includes trade journals, dissertations, government documents.
But primarily you can expect this to be a pretty modest estimate of where games have contributed, which I think that's sort of by design. We didn't want to make over claims and then after after we did our search queries to the database, the result was 41,876 potential articles to review. Yeah, that's where the team came in. There was that that's where our team of interns definitely shines. Every single one of those, believe it or not, was at least reviewed in its abstract, and many of them were read in depth.
We read all of the reviews in depth, and we also had pulled some. We put the entire corpus, those that survived our sifting and winnowing. There was 33,142 in the final data corpus. This is the data corpus we're going to talk about a bit today. And again, here are the types of items that are in it, the types of reports that we included and the sidebar gives you a sense of publication. So 1970 was sort of the earliest we wanted to go in terms of digital video games.
But you can see the acceleration of just publication and reports on work going out related to game technologies over time, and that was put in, Zotero says. Zotero is an open source free reference management database for us to manage this many hits. All right.
Yeah, that's you. You? You still. Oh me. So I want to start with first order impacts because it's kind of a separate sort of topic. And it felt as though we would be remiss to talk about sort of the impact of games on humanity without first talking about what do games give us, games themselves, commercial. So looking at first order impacts blowing that out a bit. So here are the main divisions are six main areas where you see a lot of work done in and this represents proportionally what those areas are.
Cognition is someone in cognitive science. I'm well aware of that field, but it may be surprising to see that there is quite a bit of work in social and wellbeing. And again here we're looking exclusively at the positive benefit. So these are the number of articles that are about or projects, reports about games being used for social well-being, etc.
Just a dig in the details a little bit. Of course, many of us know about the work of Green and Babel era and some of their innovative work in neuroscience. Looking at how games have shaped working memory processing speed for visual acuity.
Oh my gosh, self-regulation. In fact, one of their articles in I think it's 2018 was about games and its capacity to actually change how people learn, how to learn the very learning process itself, which is really exciting work. Interestingly, though, that's not all they're being used for in cognition. They're also being used as monitoring devices. I'm just pulling out some of my favorite examples. So thinking about games as a system that could actually monitor the elderly in their homes and look at performance variability to mark, for example, when a physician may need to check in on a patient.
Other examples include the use of games for traumatic brain injury as an ameliorative process and even schizophrenia over in social something nice examples are games for interact interpersonal effectiveness. So here actually doing some work on how games and group play can actually improve interpersonal effectiveness. I'm not sure it improves all of ours or at least mine. Of course improve city communities is another great example.
If you think about making money, if I get the project name right, oh, one of my favorite projects about how you can use games to improve city communities and even this is a bare with me, but even using games to show how this there's a broad theory called bounded generalized reciprocity. And to boil it down and basically bastardize it, it's that it's that when you are in a group, when someone else gifts you or gets basically is kind to you, you are likely to return the kindness that you are less like. You are not likely to return the kindness to someone out of group. Well, it turns out they use games to find out That's not really true. When you're in a group and someone is kind to you, that kindness does transfer out of group. Just if I know there's a lot to feel not so excited about these days.
There's a good point, right? Kindness in group actually bleeds outside of groups. Thank you games! I know. I had to add that I like to find hope where I can find it and I'll let you take it now. Yeah. Okay. So so there we've talked mostly about again, the impact that games have just as people playing them. So this is things like what's the impact of playing Pokemon Go.
This is not someone who's designing a game specifically to have a specific sort of impact. So now we're going to transition over to this next group of categories of things that technologies or impacts that games invented that were sort of new things that they catalyze are advanced or applied and transferred, which are like serious games. And then this games is a petri dish or use games to study what people do in different contexts, right? So this is kind of where we're going now as we went through this. This is kind of the bar or one of the oh, we found again, it was about 3000, 3142 articles.
These are these circles are drawn to scale by Constance here. I'm a German. I like precision. I'm sorry, scale by diameter. If anyone's wearing per pixel, honestly.
Yeah. And and this though overall, some of this probably shouldn't be super surprising to most of us here in the room. Again, we do want to emphasize that this reflects like any sort of data. This reflects our particular methodology and process, particular of this database. You might find more if you or different configurations looking elsewhere. But I think a real key to this one is that there's a lot of hyperbole.
Again, if you've been working in this area at all, you know, a lot of like games are going to revolutionize X and those kinds of articles really did not fall within this dataset. This is really where our games having an impact of some sort like case studies and so on. So the big ones, you might as you might expect to be education and corporate training, they're very, very related. And then cognition, sometimes there's some bleeding overlap. The next biggest one would be health care, medical training, physical health, mental health. I will say we won't.
And we were not going to get into either of these two today because a lot of people here in the room already know these. But I will say you can find games for training, for helping patients with almost any condition and for training people to deal with that condition across about anything you can think of. There's a lot of interesting stuff, but going all the way down into like urban planning and architecture, which we'll talk about today, AI network technology, cyber security, automobiles and transportation kind of got its own category. It's one of the more exciting sort of categories, and it might be people who are working in some of those areas.
And yeah, I'll give you a second just to kind of take it in and get a sense for all the different things that are happening here. Anything that now 24 domains total and and of course, the three biggest ones we're not going to talk about, we're going to pull out some of the other favorite highlights that we've that people might want to see. Okay. Aha.
Thank you. And where are we going to start? Architecture and engineering of the ocean. Okay, so this was one of my favorites, although I had many favorites. And so we found about 170 related to architecture, engineering and construction.
And one of the exciting things about game technologies in this area is that they can you can start with architecture, right? So you can have people starting with the drawing board, so to speak, drawing up designs, handing some of the same files to engineering and construction that go all the way to manufacturing, eventually residing in something like a digital twin. So to really pull together, I think what is exciting about the work in this area is what we call digital twins. And so digital twins are basically the idea that you're going to create a digital representation of a system that you will use as a working model as long as that place is alive. So you might think of one time square, for example, you might think of us having a digital twin of Time Square where we were modeling people moving around. We've got all the various businesses you can look at, say things like, Oh, what's going to be the effect on traffic if you shut down a road and then you can look and let all the simulations run and kind of see what people are going to do, what the economic effects, traffic effects and so on and so on. So some of the work on this, there's work on procedural content generation.
So digital twins where you're using them to create landscapes and architect and environmental systems and then using that as a way of seeing how the landscape is going to evolve. Jet pump experiment. So things getting into literally modeling, what's a particular physical pump going to do within a manufacturing facility, additive manufacturing.
So this is an exciting area where you have an ongoing system and you say, okay, what's going to happen if we swap out a part or change a system like we've got a car and we change out its cooling system, let's say for a whole different type, this allows you to see how that system is going to work without having to break the whole thing. So kind of if you've ever been building a boat as you're sailing, this is allows you to do that more effectively. To me, this idea really came to life when I saw the example of Vancouver's digital twin. So people have seen this example here, but this is the idea they actually built a digital twin of the Vancouver airport to understand how changes in one system are going to affect the others. You can do better sort of policy making and communicating on the fly.
If you do work in this area, you might know that or if you've seen work in this area, a lot of time it's the communication around it. As everyone gets together to build an artifact to say, Oh, you need to remember like the baggage claim system or something. So Vancouver has both internal and external systems too. There's, there's one for Hong Kong as well. Hong Kong has a digital twin for its airport, and it's not hard to believe. This is one of those moments where I thought, okay, now I can see some of the ways in which a metaverse can be helpful.
Not that there aren't others, but this is the kind of thing I think that would not surprise me to see more of. There's a digital twin of Adelaide made by Twin Motion, and so you can see, you can imagine even how having a digital sort of city that would allow you to look at these systems even more broad on broader scales. Okay, so that's digital twins engineering and manufacturing.
One of the another exciting example was looking at how you can actually change manufacturing process gamifying and changing out how the policies work to see how that's going to affect the underlying systems. Yeah. All right quick dive into I topic on a lot of our minds course games are considered sort of the fruit fly of AI research in AI work. Really the primary way that games have contributed have actually been to catalyze or to enable to push the technologies in certain ways.
And so the key areas really here in terms of catalyzing our search algorithms, AI driven game design of course, and being a seedbed for competitions and events. Some exceptions are reports that learning Monte Carlo tree searches and behavior trees, which of course Monte Carlo tree searches underlying what is it, AlphaGo Right. The game that just actually won Go, which is one of the most elegant games I've ever played, ever. And now is a top player and it learns by playing itself.
But that's actually based on Monte Carlo search trees. Another area of impact, as I said, was competition in events where games mean games did not create AI, but games have pushed AI and given it a place for AI to prove itself in its capacity. Of course, we really want to be terrified, which I love, apparently. Of course, last year's news on Meta was that there I can actually now play diplomacy, including negotiate and barter with other players, which is troubling. You're welcome. Possibly. We've got a cultural.
Is this me? Are you? I don't know. Cultural heritage and tourism. I notice here I use cultural heritage, but it actually represents four topics, not one you see work in archeology and tourism museums. This is some really beautiful stuff over in archeology.
An example would be for like looking at using game based technologies to look at. We're going to say this right stratigraphy. Any archeologists in here are offended. If so, I apologize. But the study of strata of rock over and cultural heritage.
Of course, I think many of us know examples like the the Hagia Sophia here's the skinned version or South Tyrol. We're actually modeling some of these precious places and resources before we lose them to climate change. Sorry, guys, but you may not be as familiar with the way in which game technologies are used to. For example, preserve first peoples wildflower names properly pronounced or Pottawatomie games or traditional games by UNESCO broadly, I mean, who doesn't love games? And they're being used in that way.
Also, the obvious one of tourism. We all I think, have gone to bizarre places in order to find that Pokemon, which is real emergency management. Look at the bomb explode. Excellent. Another fun one. All right, so this is another one there. Again, maybe not as many by pure numbers, but I think if you look at this in some of the more interesting, exciting work, I can tell you a lot about where games can have an impact.
So evacuation planning simulation, this is we saw this across actually multiple areas where people were building simulations, using game technologies, oftentimes letting people play them in order to prepare the public or even just engineering and design better methods. So if you design a system, you can actually see how people are going to evacuate where there are bottlenecks in someone, where they're actually people can get out. Debris flow, landslides.
We actually found debris flow, landslides, simulations, literally just the same with bottlenecks. A lot of these are bottlenecks and actually a lot of them are around managing time because once you've got game systems and game technologies, something we see across these areas are really good at simulating systems over time of active shooter response training, something that again, weirdly or sadly a lot of teachers and games for teachers around this and then terrorist response training. So any sort of emergency first responders again which ranges from teachers to to EMT is there are games for their training across all of these areas. And then public preparedness, my favorite one is the fire emergency evacuation, which I believe is the next slide. And then students dorm evacuation. So games to simulate how to get out of a dorm in our own is in the dorm right now.
So I'm particularly tuned in on this about you know what happens again, where are there bottlenecks, how do we get people out? And then using that here's the one from this was done in China to understand how do you get people out of dorms? Apparently, this is this was an issue, at least according to the introduction of the article. So, yes, So that is emergency response, emergency response. And last ones, back to you.
Maybe this is one of my favorite topics, although I have to say there are so many great examples in each of these fields, but I love the ones on science, data visualization, I guess because because working with large data sets is so difficult, especially multivariate ones. But in the world of science, data stimulus simulations, there's really four sort of fundamental kind of areas that people are using. Science is based on gain tech first biological systems.
Obviously Earth, Planet, Globe, Earth Systems is a broader full set. Of course, we're all familiar with Foldit, for example, looking at the structure of proteins as we're in here. Oh, this work is fabulous if you're not familiar with it. I think it's now, gosh, over ten years old. But looking at how game platforms can actually mobilize not just data visualization, but human beings as computational geniuses to figure out new forms of protein folding to lead toward results of pharmaceutical outcomes.
But you also see really cool stuff like electron transfer dynamics, multicellular simulators for synthetic biology. I had to look up what is synthetic biology? Well, vaguely terrifying, but essentially looking at how you can actually purposely alter biology at the at the, you know, at the molecular level. But to lead toward pharmaceutical innovations that can actually save people, you see over an engineering experience, again, you see pharmacy is a big field here. Over in this world, you're actually dealing with a different component of it. But again, you see sort of the imagery of what this looks like. Let's model.
This is using the Unreal engine and back to the full set. I think this might be our last topic, which is phenomenal. We whizzer those we were worried about time, but notice again that the the topics that we actually pulled out were really some of these smaller areas figuring that many of us already know some of the key work here where this is a big conversation. But it is true that game technologies and techniques of design have now been used in all of these other fields that are far beyond our main impact fields of education, training, health and social impact.
So overall, I think that there is no there's not as much hyperbole there as I thought, perhaps in terms of like its contributions in these other ways. What we haven't actually spoken about also is the world of art, some graphics, graphic and rendering techniques, motion capture. Again, you see that games have pushed and innovated. For example, motion captures a really good example for some of that. I think when the final things I definitely notice looking across is that you see at least two main affordances. One is, as you mentioned, any time you want to model systems over time.
So the fact that once you want to look, look at evacuation or you want to look at human behavior, the digital twins are using the real time rendering techniques of of things like unity, unreal and so on to actually do that. And then second, we're beginning once you want to get participatory users doing things. So like, okay, let's see what people do, like have them game where people try to run out of the building and evacuate. Or the same thing with the petri dish stuff that that's studying people's behaviors in systems.
Which games can do? Yes, absolutely. I was going to say the same thing where any also places where you have to shape behavior over time using game type or game mechanics to borrow Sebastian's term to think about how to shape behavior over time. So the runtime dynamics and interactivity of games have actually led to innovations across these field because of those two kind of primary things. One, I think that's what they mean by 4D. Yes, 4D, I'm like, what is the fourth thing? Why aren't I living in it? Apparently I am. It's time. And the second one being especially group dynamics, but behavior over time and those have been sort of the lasting areas.
Oh, we did decide to share the resources. So the left is the citations. There are so many great studies here. We wanted to at least give the citations for the studies that we've cited, but they are barely even just touching the surface of some of the fields here.
I pulled out what I found to be delightful because it's delightful, not representative. And the second one is the full database, which is public. And if you write us, we're happy to add people. I didn't want to make it overly public because we know how that goes, but we can add people and it'd be great to fill in some of the gaps because again, this estimate is by design fairly conservative and it'd be really interesting. There's a couple of fields in particular where we would love to get input into what people know about in those areas.
Oh, and thank you Tencent who enabled us to do the study and patiently put up with a study that went from one quarter to an entire year of 41,000 articles. That's all we got. Thank you.
Come back and rest in the hall. Don't know, do we take questions in like the Yankees doing? I guess they're not yanking us yet. Are there any questions because we could answer them? Yes. Wow. Wow. Oh, wow. So I don't know if anyone heard the question, but how to actually use game technologies techniques to revitalize cultures that may otherwise no longer exists over time.
Correct. We may lose. I think some of what you're seeing in cultural heritage is just exactly that effort. And I do know that there is some there's some expertise and there's expertise inside our State Department that actually is working on similar topics as well. I think some of the effort has been to try to build close partnership with digital twins. I don't know if they're called digital. I don't know if I want to use digital twins for that, but thinking about how to capture them in some digitized way before they disappear from our planet.
Yeah. And and I think, oh, I've worked on one or two of those in my life. And I think it's always a challenge to understand how do you get people designing and speaking for themselves in their own cultures? And those conversations can be really generative and it can. Yeah, I think it's just always a critical thing.
So of course, one key question there is who's doing the authoring? It's much like the mapping. New York geographers dealt with this a lot in terms of mapping. I will point to the work of Elizabeth LaPensée, say who does fabulous mapping and I think now as an industry, but as an example of how to build well in conversation, not further colonization, if I can use that term.
Yes. Yes. I think your instincts might be right.
And I should say that on the marketing topic, we pulled everything that was basically analysis of marketing, of games, looking specifically to how do game tech technologies contribute to marketing. And it's big and it's mostly getting ads in front of eyeballs that are no longer sitting through ads on streamed media. There was yeah, there was also some around essentially the data gathering capacity of games to understand that data modeling. And then I think it may be kind of a no brainer, but just that all the gamification things writ large, I was a little surprised.
I thought, Oh, there's probably an article too, like, Oh, there's a lot there are a lot in marketing around that. And I do suspect you are correct that there's a lot of practice knowledge that doesn't perhaps make its way in as much. It's a it's a really good insight. Yeah, a lot on gamification and how it works and, you know, it's all it has so thoroughly spread throughout everything now I think. Yeah. Yes. Yeah.
I think we're way over. I think we're. But thank you and we'll take questions later. I hope to have the conversation. Thank you so much.
2023-08-16 22:32
