>> It's really a pretty epic time of change in the Game Industry. And you know, a big part of why this has gotten so big is we're now bringing Gaming to Everyone. And when we talk about Gaming for Everyone, sometimes people think of this - this is the Microsoft Adaptive Controller. It's a controller we built the Xbox that allows people with disabilities to play games. So even if you have a physical
issue that makes it hard for you to hold a controller, this can be adapted to make it possible for really anyone to game. And so this is an incredible way of how the game industry is growing to bring in a bigger audience. But this is actually not what I mean when I say Gaming for Everyone, I mean this: I mean casual games, I mean mobile games. I mean the
fact that now the type of entertainment available has changed so much, that we really have attracted this massive audience. If you just look at the revenue from the game industry, you know it's been an amazing climb over the last 6 or 7 years. We're now bigger than virtually every other entertainment medium. You know, we're - the only one bigger than us right now is paid television and the next few years we're projected to pass that too. So gaming is on track to
become the world's biggest source of entertainment. And if you look at sort of what's been driving that, a big part of it really is how games have changed as the internet has really had an impact. When I first got into gaming, it was packaged goods, so you'd make a game and ship it in a store in a box, and that really prevented the game developer from having much interaction with their players. You know we then went to game-as-services, this is sort of where we've been the last maybe 20 years, 10 years. Right now games are digital, you download them, you can now do things like analytics and telemetry to see what your players are doing and it's begun to change the nature of the game experience. And I believe we're now in a new era, a new phase that we're calling Game is Community. Where the games
have truly become communities in their own right - and you know these kinds of games - a lot of things are different - one, the biggest difference of course is now the players are not just you know, sources of data but the players are really at the design table sitting alongside the creators. You know, players have a louder voice, they're often creating their own content. and so the role of the game designer was really changed. When you think of game as packaged goods, the role of the game designers is like a movie director creating a game and giving it to the players and saying, here's my gift to you, players. Enjoy this experience. Now the game designer is more like somebody that runs a nightclub or a cruise ship director, you know it's your job as a game director now to create an environment or create experiences that your players love and can engage in and frankly, stick with. And so it really has been a big shift.
You know it's no longer just about playing the game now it's about competing and chatting and buying, and even watching - you know the entertainment experience now often starts with watching other people play the game before you even play it yourself. Fortnight, obviously. Huge game. In my mind, this is sort of an example of one of these games is communities, you know I think the thing that really put it over the edge was the now famous marshmello concert where you know, the 25 million players, you know put a pause on running around shooting each other for a while to actually enjoy a concert in the game environment. And so I think this is really going to be the moment where we look back and say this is when the game and you really fundamentally change into a new form of entertainment. And I couldn't be more excited about that. If you look at this example, the top grossing games worldwide - 80% of the games in the top 10 were the same games last year.
You know, these are all games making a billion dollars or more, and again the reason why you have such stickiness in these top games, frankly they're so successful, making so much money - is because they're communities, is because they're online, is because they have this sort of continuing notion of live operation to be continually updated. You know, the revenue shows that because look at where the revenue is coming from - 70% is free to play, right? games that are sort of being run as a service and 51% of is mobile. This is really the segment that is defining where that experience is and then it really makes it - and so when we - Microsoft, think about where the game industry is going and sort of what are the key needs a game industry studios have - this is how I break it down. So these are the 5 scenarios we're personally focusing on and focusing delivering awesome solutions so game creators can use these to help them build their games and be more successful, right? So the five things that we see critical to the industry: number one, accelerating game productions. How do you make your games ever more efficient and ever more faster? By moving into the cloud. And it's not just things like you know, using ASA control systems and moving your data to the cloud, it's also things like using machine learning and AI to produce game content which will talk about in a second. Number 2 is creating communities, that notion
again of you know, game developers need help or want help creating multiplayer games, matchmaking, lobbies, all the work that goes into creating those communities as well as keeping them safe and nontoxic and preventing cheating. 3rd one is you know, knowing your players and always know your players better through Analytics and VI. 4th one is growing your player lifetime data. How do you make your players more valuable over time using that data and using live ops techniques so you can boost engagement and boost retention and boost monetization and then finally how do you track new players cost effectively? You know, and how do you do that either through performance marketing or social viral techniques or even just you know publishing your games more effectively into stores having better discovery.
So there's the 5 scenarios that we are really focusing on. I want to talk a little bit about the 4th one the Lite LTV one because live ops to something I personally have a lot of passion for it's why we founded PlayFab and It's a big area that we're investing in. You know live ops is really this process for how you design and build your games, and I think when you talk about gaming of the future, this is really key because I think every game studio needs to learn how to do live ops and how to do it well to run their games. So what is live ops? You know I take the word and break it down into 2 parts here: Part one is building a live game - building a game that is connected to the Internet that gives you the ability to make changes and interact with your players and have the players interact with each other. That is the first part and that is already a shift that requires your engineering team to focus on more than just how to build a fun game that's going build a fun game where players can interact with each other and with you and you can make changes. The second part though, is in some ways the harder part the
ops part is about how do you actaully build your team to do the operations to to continually make those investments and continually make changes and updates to the game and that is really a culture shift - a culture shift and a mindset shift and we've seen - I've seen that game studios are used to building games the old fashioned way, it takes 5 or 6 years to make this transition to being a fully operations-based studio, and to use data effectively. And so when I break this down this is really where I - I personally find this fascinating. This nature of the culture shift in moving to a live ops model. Now when we have a vision for sort of what a platform looks like to provide modern live ops, it looks sort of like this. You know this notion that you've got a back end on the cloud somewhere providing services for the games and these services allow the game experience to be created and then as you're playing the game, you're generating data, the data flows back to the servers, you know you're in the cloud doing the work of analyzing the data to figure out insights - who your players are, what they're doing, and then being able to use those insights to then make changes back into the game experience - and do all that in real time. You know, there was a talk just now prior to mine talking about how you games to 2 in the game difficulty to create a more fun experience for your player, and that's exactly we're saying here. Your goal is
to sort of maximize that player engagement, that player fun. And to do that you don't want your game to be too hard or too easy. And so being able to sork of tweak and tune your game experience on the fly based on what you're learning is critical here in games like Candy Crush tried to do that really effectively but we believe this is the kind of thing all games are learning how to do.
So I want to shift a little bit now continue talk about the future of gaming. That if live ops is something that we're all learning how to do, I think the other kind of big trend that's happening, another big trend is happening, is how the rise of tools have really become democratized you know, so things like Unreal and Unity have made it so much easier now for really anyone to learn how to build a game that it's incredible to me to see just the sheer power that's in the hands of really everyone. But it's not just Unreal and Unity. I also think the roadblocks in Minecraft are examples of of games that themselves have become platforms. And I'm actually really fascinated by this shift, where we're starting to see new almost like gaming devices that are cloud-based and virtual. When I look at Roblox, Roblox
is a platform for game creation. And it's a platform that is really open to everyone. Now the tools themselves are being used in ways outside of gaming. So another thing that is a big shift happening to gaming is the way there our tools that we've created are being used by other industries. This is a screenshot from the
Lion King animated film by Disney that come out a couple years ago. What was really interesting about this movie is they actually produced it in VR and so they were actually using the Unity engine to essentially have a motion capture stage, a VR stage, and they created a virtual environment and then all the cameras were actually being tracked in the stage and so the cinematographer was able to basically put on - you know, move the camera around and record the shots just as if it was a real camera on a real stage. And so they're using all the tools, that they're familiar with, capturing virtual reality and that meant that the actual game producers and the game directors - the movie directors, rather, could actually be in VR and could be seeing the movie being shot in real time. And so I'm
fascinated by this aspect of how game technology is now coming over into movie production. We've also seen those TV productions. So the TV show the Mandalorian, also on Disney., And these guys know that the backdrop there instead of shooting against a green screen which is what they often do for films and then they add special effects later, they were actually using a live LED screen as the backdrop powered by the Unreal engine. So they're actually creating environments in Unreal that are being used as the backdrop for the film production. And again a fascinating way in which movie production is kind of coming over as game technology is coming over and changing how movies are produced. I think about that and I think about another kind of trend this is happening is the sheer investment being made in creating these virtual worlds. We just talked about Mandalorian,
shifting gears now this is Red Dead Redemption right? Read Dead Redemption - huge game obviously from Rock Star, came out a couple years ago. This is fascinating because the sheer amount of investment they made in creating this world for this game was unprecedented and to me, you know given this is basically just one game, it seems like an interesting investment - I'm actually predicting you're going start to see investments in worlds like this one being used for many different games, not just one game. And you are going to start to see how we're going to start to share assets like this in the cloud and open them up to other other game productions. In my mind it's it's almost like the movie - the TV show Westworld, but you know to virtual theme park and then there's many different stories happening in parallel in this virtual theme park but somehow I think about what's going to be happening in gaming is you really are start to see these massive worlds with many different stories and games happening all all in parallel. It goes back to that
comment about live ops so it's the job of the movie director really to create an environment in which players can have these amazing experiences. So I went back to cloud production again, we talk a lot about gaming in the cloud and I think things like our own project xCloud or Stadia are what we talk about, and I mentioned Roblox earlier - in my mind, Roblox and Minecraft are sort of the next generation of cloud platforms, you know this is an example of some of the games inside of Roblox, these are all created by the players themselves. And so you talk about your communities, players creating content, releasing it for other players to consume and and frankly make a living on. Many of these games in Roblox will be top 10 games in their own right, they weren't you know, inside Roblox but were on say iOS or Android where those things just tracked. So I really think this is a huge story, same thing with Minecraft you know, PlayFab is being used now to power the marketplace in Minecraft and frankly we're also being used to power the Analytics platform for Roblox. So we have insight into both of these
platforms, and then the real story here is as we move to the notion of games communities, the sheer amount of creativity and investment coming from the players themselves is really remarkable. And these really are the future of gaming a huge part of it is new platforms, this is sort of like YouTube is to gaming, or YouTube is to say, Hollywood is what's happening now with these sort of player generated games and content is to traditional gaming. And this is a huge, you know shift that's going to be felt for many years. Another way which we see game technology being used outside of gaming is as I mentioned obviously LiveOps we've seen Satya talk about you know LiveOps coming to the enterprise. And we've actually got groups in Microsoft now thinking about how the techniques from gaming can be used in areas like hospitality.
So imagine if you go to your hotel and the hotel is engaging with you in the same way a game would engage with you. We in gaming have so much experience creating these amazing interactive experiences, most businesses these days are transacted through apps, and apps are interactive. And so there's a lot of education and knowledge we have in the gaming industry but how to make those apps more engaging, more fun, more valuable that we can start to apply outside of gaming. So I think you're going start to see more and more of the lessons from gaming make their way into the mainstream which is exciting to see. Another shift that we are looking very closely at here is a machine learning, and how machine learning and AI are being used now through the game life cycle. So it's not just about
say operating, but all the way through build, test, launch, operate. Then the build phase is in some ways one of the most interesting. Using game technology or ML technology to actually help produce the content for the game itself. And so I find that fascinating I find that all the way to June operations, you know, using machine learning for example, to tailor and tune the pricing. Look for fraud, personalizing the game, these are all scenarios we've been experimenting with. And I think you know the modern Game Studios will start to invest in ML, it's an area we're certainly investing in. You know one of the things that's fun about Microsoft is we're big enough we can do things like take machine learning models we built for retail and that are being used bt places like Walmart and other retail chains to sort of make the retail experience more effective. And we're actually experimenting
with taking those models now and putting them in to games. And saying hey, can ML technologies that are used for optimizing traditional retail be used for in-game retail? And those are kind of fun experiments we're doing and hopefully that becomes products we can then release to to everyone to use for their own games. I want to close by talking about you know, the Microsoft Flight Simulator game that came out maybe a month ago now. This is another kind of watershed moment I think in game production because the big news with Microsoft Flight Simulator is how so much of the world for this is actually produced using machine learning from satellite imagery. And so
this is an unprecedented partnership between one hand Bing maps, which has petabytes and petabytes of satellite imagery of the whole planet. Azure which use that imagery to crunch it and run models to create geometry. I think Cognitive services being used to process that and generate the data. And so, If you fly around in that Flight Simulator game, some of the cities like New York City - the geometry of New York City is actually a 3D data set that we've generated you know we have literally crawled all over Manhattan taking photographs of like every square inch and that's what was used to create things like Manhattan so for Manhattan and maybe a few 100 other cities around the world, there are data sets that that that capture the city that we were able to use for Flight Simulator. But only a few, only a few 100
cities have that level of data And so for the rest of the world we had to rely on generating the geometry procedurely because it would have been impossible to generate it by hand. It would have been prohibitively expensive. And so the way we did that - we partnered with a company out of Austria, that basically built machine learming models would take satellite imagery and turn it into real geometry. And so I've got a video here that hopefully will play that will kind of show and illustrate the experience. And so what you're looking as you're looking at literally the transformation process between you know 2D satelite imagery being turned into the 3D geometry. And so what's happening here is and the way
this works is we first you know took a whole bunch of different samples and we build a model that could look for the satellite imagery and detect out of the satellite imagery - detect the features, the buildings. And then we would create a series of models that were used to actually render those. For example in the case of Paris, only 3 buildings in Paris were hand modeled - the Eiffel Tower and the Louvre. The rest are done
through this process. So you can see the feature scanner is running the satellite imagery it's identifying the features its then applying, you know a model to it, and it's not going to be totally realistic, you know, it's it's using ML techniques, so it's going to sort of guess at what the building would look like, not necessarily going to be perfect. It's going to make guesses based on the footprint and the geometry but it's going to be - the important thing is it's going to be realistic. So it may not be perfect but it will be realistic, it will look good to the eye. And it look as if you were flying around a real city even the buildings don't necessarily match. And then the actual buildings it creates, there's different facades available so depending on the region of the world, the geometry is going to be slightly different, so you know buildings in Japan will look different from buildings in Spain or North America and that's OK that's actually the goal because you want it to look realistic and then we took the geometry from the entire planet it takes but 72 hours to run these models across the entire planet and generate the geometry for the entire world. and then that geometry then - and it's only possible again
because the power of the cloud you can spin up thousands of computers, do this pass, you know, in 72 hours. And the result is this. The result is a world you can fly around in that is literally you can fly anywhere in the entire world and experience it. Every one of the shots we see now is generated by that process - it is pretty incredible. And so I'm going to encourage you to play Flight Simulator, but to me that's really the kind of look into the future at the kinds of techniques now that are making their way to game production.
I think it's super, super exciting to see. I'm going to close with a pitch. If you're interested in LiveOps in particular I do have a podcast that my team and I kind of produce, sort of the art of LiveOps, we're in season 2 right now, we've got maybe 35-40 episodes out total. I encourage you to take a listen wherever you go to listen to a podcast. It's a really fun podcast for anyone interested in thinking more about LiveOps and how to apply this to games. And so with that I think I am done - I'm going to switch gears now and take questions. The questions is: Do we
get the geometry for each model separately? So you're talking about the Flight Simulator I assume? So the way it worked is they bsically created models for different features of the world and they created the models by basically training and taking lots of different images, lots of different geometries and they would create and then they were run the model against the geometry to learn and after a few 100 samples it could then sort of go from there on its own. So for example, I remember having conversations with them a few weeks ago and they were working on the model for highways, so highway overpasses you know those very complicated highway exchanges where multiple highways meet in a very complicated structures? They created an ML model just for that, so when it scans the world it could pick out highway interchanges and create geometry for those - that's like one model. They had a model for buildings, they had a model for like trees, they had a model for highway interchanges and the goal is you keep adding more models over time. So as we expand the world they can make the world ever more accurate by creating these new models. And then the geometry that gets created by those models is then fed into the rendering engine. It lives in the cloud, it could stream down to the PC
why you are playing it and then rendered. OK, next person: Any idea how different it is from Mapbox implimentation wise and cost wise? I don't know about Mapbox. I'm sorry I can't tell you. I do know that the company that did the work with us was Blackshark.ai and they are based out of Austria is the vendor we worked with on the ML model. OK: What technical architecture and tools are used for cloud-based gaming, how can we integrate Unity-based games to it? So when to cloud-based gaming I'm not sure - it means a couple different things.
One type of thing is things like xCloud where the game is actually running on servers in the cloud and being streamed down to you know mobile devices or PCs which is great because it means you don't have to have a high end gaming rig, you know on the client device. And we're actually working - we call it pixel streaming. And the generic technology is pixel streaming. Tou know, running something in the cloud and playing it on a device - xCloud is using pixel streaming to create a game service, but pixel streaming is actually available even today in Azure - anyone could fire up a machine with a GPU and there are technologies we have to stream that down to your desktop. So for example, we're talking to clients who are interested in doing things like architectural walkthroughs where you might use a 3D game engine to create a building and if you want your client to walk to the building you may not necessarily know that your client has a high end gaming PC and especially going to things like raytracing or fancy graphics, that has to do it in the cloud and stream the experience down to the device. That's one aspect of cloud gaming. Another aspect of cloud
gaming that - and yes, for that that you can use Unity or Unreal or any any game engine you want, you're just running the game in a server in a data center. If you're meaning things like the Flight Simulator game, where we are using the cloud to produce the content. That's really just using again ML models to spit out the geometry that can then be used in the game engines like Unreal or Unity and that's a case where you know I'm actually going to be looking at whether we can start to publish some white papers getting deeper into exactly how we did Flight Simulator to make it possible to sort of look at doing it yourself. That will be something that my team will be looking at in coming months of producing that kind of content. Flight Simulator amazed me like how...can you mention
an approximate overview of how tough is the production and the manpower used? I don't know the exact number of people. I do know that you know, to be clear - the amount of geometry created could never have been created by people, right? So it really was created by the - it really depended on these machine learning models in order to make that feasible. But is was not a huge team, I don't think Blackshark, I mean I'm speculating here, because I don't know for sure. It was not I don't think,
100's of people and it's sort of like dozens of people doing the machine learning work. And so that really is the kind of point - you can have dozens of people creating machine learning models that can create a game of the scale and size of Flight Simulator and so that's - when I talk about accelerating game production in the cloud, that's really what I'm talking about - using the power of the cloud and machine learning to amplify efforts to create content that could not be possible if you had to do it all by hand. So we get a question: How can you use AI/machine learning from via cloud and gaming? There are a lot of tutorials on machine learning and there are a lot of tutorials on ML for content generation, I would encourage - it is not easy science. But you can follow examples and download scripts and samples and teach yourself - and so I would encourage you to look for some basic machine learning classes and do some googling around machine learning content generation. You know the most common ones are
all machine learning, what has in common is you train a model and then you let the model run and do the work. So training a model means and in the case of Flight Simulator, teaching it these are images, and this is the geometry it turns into. This is the image, here's the geometry. You know you can also do things like I've seen for example, examples where you can take lots of images of you know take take images of thousands of human beings and teach it this is what a human being looks like.
And then you can actually have the model create new images of human beings and create images of humans that don't exist. But that look very realistic because it's very good just sort of pattern matching and generating that kind of content. Current challenge for cloud-based gaming? How's adoption access development teams to work with Stadia? I don't know about Stadia but I know about xCloud. In the case of xCloud, it's just Xbox. You build a game as if it's for Xbox and then you upload it and we can can now stream it from the data center down to Xboxes and there's zero custom work needed. It's like just running on an Xbox that happens to be sitting in the data center. In the case of Stadia,
there is no Xbox, obviously it's just a virtual device that Google has created but it's the same thing - you are just creating it as if it's running on your local PC. It just happens to run in the data center. And of course that's the whole power of this is unlike a mobile game, where you have to build a special version of the game for your mobile device, in some cases you have to build different versions of the game for iOS vs. Android. We have to build different versions of the game for all the different devices we want to target. With cloud gaming, you build it once, runs in the cloud and it's available to literally stream to any device even through a Web browser in some cases. And so that's really the power here is
truly build once, run everywhere through the streaming. Alright, I think this is probably the last question as I think we are almost out of time: Are there solutions that Azure provides to use something like MLP in a game? Can you comment on the feasibility of using this by simple NPC dialog? So MLP center natural language processing absolutely I think we are starting to see people looking at using natural image processing. Couple things that Microsoft has that you can take a look at that are available right now - not quite your your answer, but one of them is we have text to speech generation using virtual speech technology that can create - we can basically sample voices and create what we call sound fonts or voice fonts and you can then basically create the computer speaking in a voice that sounds incredibly realistic, I mean incredibly realistic. And to the point where I wouldn't be surprised if you know many games are able to get away with not hiring actors at all, but having a computer generate dialogue through these sound - these voice fonts. sounds you know, creepily like
a human being speaking. It's not a robotic voice anymore at all. And so if you combine that with ML models, for like there's a lot of text bots out there that can that can type or talk as if it's a human being combine the 2 and yes, I think you could have NPC's in games that sound very realistic and as if the real people but obviously NPCs. And that's where we start to really blur the line. There's a lot of very exciting stuff happening in this space. So that's again going to the Westworld reference. I do
think as we create these ever-bigger worlds, you know multiple stories overlapping, we're going to depending very much on ML and AI to sort of power those worlds and make them feasible.
2021-02-01