[Music] hi everyone thanks for taking the time to join us today for our ask the experts panel on why microsoft's going all in on game development with azure as i'm sure you all know games are overwhelmingly online experiences these days even if the services they use are only for analytics to help tune updates and future titles but uh many of top games are going far deeper with their services and that's what we're here to talk about today so i'm brendan vannis the head of developer success for the gaming vertical and i'll be hosting today's session we've got a fantastic panel of experts from across the microsoft games team so let me start off by introducing them first we have lauren sanchez one of the technical directors at moyang studio makers of minecraft over his 22-year career at microsoft lawrence has worked on both client apps like office and cloud services like microsoft 365 but he finally found his true calling when he joined the minecraft team in 2018 to fulfill his lifelong dream of building video games lawrence's team owns many of the azure based cloud services which power the various minecraft gameplay features and systems including minecraft earth and minecraft realms next up we've got chad lewis one of the producers from turn 10 studios the folks that bring you the forza series on xbox and pc uh he joined the game services team in 2014 to help modernize the services infrastructure focusing on building new capabilities and scale while managing a yearly game launch cadence previously turned 10 he had various program and project manager roles in the microsoft server and tools organization delivering software as a service and infrastructure as a service products to customers finally we have travis bradshaw cto of our gaming cloud travis is the most recent member of the microsoft games team having joined us just this past december with over 15 years of experience in games industry travis previous excuse me travis previously worked for id software on the idtec game engine and the doom quake and wolfenstein franchises specializing in live services in the game production pipelines now this panel is your opportunity to get answers directly from our experts on all aspects of game development and the azure cloud these are the folks working to deliver massively scalable services that both drive online experiences as well as give developers everything they need to monitor analyze operate and evolve their games so if you have any questions at all for a panel feel free to click on the q a tab up top enter them at any point during the session we'll be answering as many as we can live and we can follow up with you post session we'll be able to stick around for a few minutes after to catch up on any that we might have missed or didn't have time for uh and i'll be giving you guys my email address so if you want to connect with me later to ask additional questions i can certainly get you all the information you need okay so uh to kick things off let's go with something for everybody um like with many industries game creators are looking at their games through the lines of cloud native development and things like using microservices continuous delivery containers uh live ops everything in that space so how is this technology the cloud change the way you feel games are being developed so chad do you want to start us off yeah you bet thanks brendan appreciate the chance to talk with everybody today uh yeah so boy there's a lot of things we could talk about in this particular topic but maybe i'll just highlight a couple of things uh that stand out for me um you know for us on the game services team the most important part is to unblock the development team the client development team keep them focused on making a great game finding the fun uh shipping on time and so for us it's really on the services team it's about the flexibility that that cloud native provides us and one of the areas we get uh that kind of flexibility is in-game tweakables um you know with live ops today everybody wants to be able to make changes to the game along the way but also once it goes live uh the fact that we can as a services team provide uh tweakable storage that is very simple to use uh we don't have to worry about the scale of that of that infrastructure and we can really unblock the development team super super early in the process to just you know default to putting the things you think you want to tweak about the game into the cloud and do that by default without worrying too much just gives us a lot more flexibility um uh right off the jump uh i'd say another area that's important to us is the fact that with cloud-based micro services it allows us to kind of automatically accrue benefit to our next game from the game services we're putting together today so for example our our uh for forza our leaderboard service is a shared microservice it's the kind of thing that we're gonna that's gonna show up it's kind of service endpoint that's gonna show up in any of our games we're running that on today's games we're building on top of that for tomorrow's game but we're not starting from scratch and that helps us both from just a timing perspective to get it out there and get it out there early again for the game developer to just start taking the client team to start taking advantage of right away but also we had some other benefits like it's running live today so we're already chewing chewing down scale issues or or performance issues along the way in a currently today running game getting ready for for tomorrow's game fantastic right um lawrence what about you hi good morning everyone uh so i think for me uh so my first kind of gig on the minecraft team uh my team owned the the services that powered uh minecraft earth so for those of you not familiar with with earth it's a it's a new title in uh in the minecraft kind of franchise that uh it was a little different uh first off it was the the only first and only minecraft title that was it was uh online only uh which meant there there's no offline gameplay so to to play minecraft earth you had to to log in using your you know your xpl credentials and from that point forward all gameplay was was online so if you lost your connection uh the game would lock you out and what that did was it enabled us to to sort of build features that would only be possible uh when powered by by the cloud in this case in you know azure services that that we've built um so minecraft earth had two primary game modes uh one and this was really cool and something really excited about uh was ar building so you could using your phone your mobile phone you could build minecraft worlds uh in ar space uh that was really really fun but on the service side i really kind of focused on uh location-based gameplay so item collection and adventures you could play out in parks and the beaches and whatnot similar to pokemon go and that second gameplay mode that kind of geo-location-based gameplay was uh all of that heavy lifting was done on the service on the service side um and uh if you so if you if you log in with your phone it would take your your current geo location and then it would show a minecraft themed map kind of representation of where you were in the real world all of that data the map data and the items we placed in locations and whether or not there was an adventure nearby in a park or on your street all of that was you know obviously very kind of server side intensive um the the mapping for example was powered by bing maps and none of that would have been possible in an offline mode because it just did the sheer amount of data that we'd have to crunch um so azure and cloud services really powered the the the heart and the the soul of of minecraft earth um and then the the key part was because of the the games uh because of the in-game economy in minecraft earth we had to ensure that there was no cheating and there was no kind of funny business happening and that made that the requirement that all transactions in the game uh down to you know item collection had to be server authoritative and that could only be done powered by the cloud um so that was kind of the the key difference in in in a cloud-based game versus you know the traditional kind of offline mode uh and then the other the other thing obviously and and chad touched upon this is the uh the ability to push out changes both in the client and the service was kind of a really big game changer so uh on the service side we were you know we to this day we still deploy once a week at minimum sometimes more frequently and then on the client side we push a new version of the game client every month to the the apple app store and the google play store so that frequency of updates meant we could constantly be evolving the game not just with content but we could add entirely new gameplay features and have those aligned pretty closely between the client and the service makes sense that makes a lot of sense yeah travis how about you yeah thank you brendan um kind of like chad was saying man there's just so many ways that this can change the way that you make games i think that probably in my experience one of those first moments is is what happens when you start to have player telemetry and a bit of a data estate so that you can see what players are doing in your game uh part of the creative process of making a game is you have to choose your bets you have these things you you find the fun you decide where you're going to make your creative stand and you make engagements that uh you hope that players will love and over the course of developing your game you'll make a lot of those bets and it used to be that kind of your sales numbers and your online population numbers might be the only feedback that you get for those decisions that you've made but uh one of the first kind of super powers you can unlock as a game creator through the cloud is you start bringing telemetry in and see how your game players are actually interacting in the game and then once uh you can use that as feedback to see which of your bets were the right bet and where you want to lean in and and really get a better profile of what makes the magic in your own game and uh i think that's really fun uh with the first time game designers really get to see that feedback there's like this eye-opening moment where it's like wow okay and then and then you can kind of go from there so i i think that's probably of the many possible ways that uh cloud development can change the way we make games i think that's the first one that unlocks and and i remember just having a lot of fun with it nice okay well we've got some questions looks like coming in from our attendees so i think this next one's going to be for you again travis um it's a question on are there any tools from microsoft to help indies control their spend when scaling up and out their games specific to the game services that we offer obviously yeah that's a great question so controlling spend is a super important part of operations i think that there are a number of avenues that one can use to do that but in our services specifically well i should back up and say it all starts with your game like what what are the online features that you need for your title and where can we help and then depending on what features you need that's where we can control scale so you mention indie games so a couple of cool services that apply immediately to indie games come from our playfab brand we have the playfab insights feature and we have playfab multiplayer servers for hosting things online multiplayer servers is pretty direct you can choose where you put your servers and you can put limits and scale functions for uh controlling costs there and then in playfab insights there is a azure data explorer cluster and you get nice tunables there for how much spend you would like while you're doing explorations in your data set and it even includes things like at a certain time you can scale up and scale down so if you're doing something intensive and you scale it up to have a really great exploration session you don't accidentally leave the meter running when everyone goes home at night and you can scale that back down so we're constantly looking for ways that we can make uh kind of all the power that's available in the cloud for game creators cost effective so nobody accidentally shoots themselves in the foot right okay well let's go with uh the studios here um so kobe's very much changed the way that applications are being developed and games are which a hugely collaborative process building games usually um they're definitely no exception to this with development teams now widely distributed sometimes all over the world um let's have to have our studios folks address what you guys have found to be the most useful tools and techniques to keep your folks collaborating and delivering uh lawrence do you want to start us off uh sure so uh for my team obviously i'm for every team really uh good community communication is key especially in in complex development when you're coordinating features across client and service uh lack of communication can be a killer and then throw in the fact that we're all kind of working remotely now we've had to really shift gears and and really change the way we work and i think we did so quickly and and successfully uh i think the key is there's no one-size-fits-all tool and uh we we have now kind of evolved this kind of uh hybrid environment where we use a combination of collaboration tools so namely outlook uh for email and and scheduling meetings uh too many meetings in but in my opinion uh teams for is our primary channel for for video conferencing and and that's something we've you know found very successful like scales and all the features you get really really kind of help and then we we also use slack and kind of get a lot of flack for hey why don't you guys only use teams i think the answer is again there's there's no tool that has every feature you need and you kind of pick and choose and let the each uh tools features kind of speak for themselves so now we're in this in this environment where a lot of our ad-hoc communications kind of replacing those hallway conversations that you have um that primarily happens in slack versus scheduled meetings where we have video conferencing and we want to do some kind of face-to-face time that all happens in teams um and then uh uh really when when it comes time to to writing the code and deploying it we rely heavily on on azure devops which was formerly known as visual studio online uh we use a combination of github and ado kit for source control uh we use ado for code reviews and all the kind of process around getting pull requests approved and merged and then we use um ado's deployment and and release pipelines for automated deployments in every environment so that includes you know development pre-production and production and uh really the power of ado is everything's integrated so you have if you have your pull requests being code reviewed uh that happens in ado and then when you kind of approve and you have all your rules around when you when those things merge all that is done online um and i think we've effectively uh removed the need to have you know over-the-shoulder code reviews um you know a lot of kind of face-to-face time that we used to have that we took for granted now we've we've replaced that with with these powerful online tools that in some ways kind of uh kind of surpass the the capabilities you would have in face-to-face code reviews for example right you have a threaded discussion going on in a in a pull request in a code review um that that's pretty hard to replicate in a face-to-face environment unless everyone's kind of writing down and taking notes so i i see some upside here in this new kind of world we're moving into if we have the tools and we have the kind of the forcing function to to to force us to use these tools we can we can achieve these these sorts of new efficiencies in in our productivity and i'm a big fan i'm i love working remotely uh and i think we'll start seeing the evolution of both this kind of new work process powered by the these tools that that we use every day makes sense yeah and that does remind me thanks for talking about um social tools um i should also highlight that uh we do have the microsoft gamestack discord which is our space for community discussions uh we'll be posting a link to that into the the q a panel uh so that everybody has that available to them um for ongoing discussions about you know any technologies we provide to game developers feel free to hit us up there as well even you know long after these sessions are over okay uh chad how about you how is the how's the turn 10 team keeping keeping on the on the path yeah you bet it's been an interesting time hasn't it well lawrence did a great job of highlighting a bunch of areas i plus one almost all all those comments i mean azure devops as a central place for sort of all of our both our process for work items but also our deployments you know kind of like the full end to end deploying and azure for resources i mean i'm not sure we could have stayed productive without that kind of comprehensive ability to do that and to do that online and one of the things that's interesting it's really highlighted some empathy for me is that we do a lot of interaction with folks internationally on on a pretty regular basis we've got some on my specific team some vt members that are in the uk and so one of the things uh leaning in with teams for these discussions is uh our daily discussions is um it's been real democratization of these interactions we used to get into a conference room and then have a few folks you know kind of on the on the big screen at the end of the room through a little camera really hard to interact and so the fact that we're kind of one everybody's one-to-one on a camera and a mic we just really leaned into that making sure that everyone gets heard and it's been fun to to get to know those people in a different way frankly uh on a on a daily basis and then because we've got folks all over the planet to be able to talk through that stuff together and then everyone's got the exact same view panel into adl their access to azure to do the things all looks the same together uh i mean just the thank goodness for that truly allowed us to stay uh productive i'd say just one other thing is it's been interesting as well like you're saying like you don't have those hallway conversations or at the coffee machine or water cooler whatever um so just really also making sure that we've got you know casual and social chat channels and teams that we're encouraging as a lead on the team you know i see as part of my responsibility like keep keep those things going keep keep the fun keep the chat going make sure people feel connected uh reach hanging out to folks as a group and um it's it's been something that is taking some extra investment we got that for free before but but super important yeah too true okay well another one from our audience here um for folks that are new to cloud native tech we were talking earlier about leaderboard microservice um so let's go travis how do that and other microservices connect with each other and my game that's a that's a fun question so there are well first off you know a part of building an online game is is almost putting together a platform specifically for your game uh games are so diverse and they have so many different experiences that depending on the features in your game you'll be composing the thing that's right for your game and that that can go pretty deep and lead to different architectural decisions depending on you know how you want to make that happen um one pattern that i see really commonly is this notion of kind of like a uniform gateway in front of your microservices architecture for instance the playfab services are constructed in this way there's a playfab api you integrate that client or use native web calls to your preference in order to access those services that are all kind of in this uniform api in this front layer that scales to match demand and then those uh kind of proxy you into the microservices which can use a number of different discovery mechanisms one of the reasons that can be really nice and and um chad and lawrence uh i'm sure have similar experiences is that honestly game clients are hard um you're doing c plus code it's super high performance and there are a number of problems that are just easier to solve uh in insert in the services layer and one of those can be like annotating and enriching calls that are coming from the game client so that so that you can provide a nice context for your micro services to operate on without having the client have to to dig that all out of the infrastructure so it's really common to handle like authentication and security and a nice bulkhead pattern that scales with the number of players that leans into your microservices architecture perhaps you'd like to use a service mesh or something uh in the playfab uh architecture we have the play stream which sends a lot of events into kind of a uniform message boss that you can then hook up rules to which is kind of interesting one of the things that's fun and and i use last time i shipped a game on playfab which was do maternal is that you can build microservices in kind of the standard way very very slim and thin and then your playfab playstream can trigger calls into that with with web dispatch or or other or some cloud script or an assure function it's a lot of flexibility here but uh so that can become your backbone for talking to different services and so uh yeah so there's there's choices depending on what's right for your game and what your existing technology is and where you are kind of in your journey to having your games platform but usually there's there's essentially a gateway service that helps your game communicate effectively without a lot of overhead into your online service infrastructure great thanks travis okay we've got a minecraft specific one so lawrence uh the question is what service or actually services i'm going to go with does microsoft use an azure for instrumentation to get player behavioral data that is a great question and the answer is complex because my answer is my go-to answer is always there's there's no one-size-fits-all we we use every trick at our disposal so uh travis mentioned play stream uh minecraft earth relies heavily on play stream both on the client and the service side and uh one of the ways that we use uh play stream events and then the kind of the custom event handling that that playfab provides we use that heavily during prototyping of new features so specifically uh if anyone plays earth there's a there's a pretty complex challenges system um and uh it the the upshot is challenges are defined uh declaratively which gives our con our internal content creators the ability to define all the challenges purely in in json format and we give them tools to do that and uh the way we kind of uh interpret that json and we handle the event completion and in rewards granting for example when you complete a challenge all of that is done on the server in the service layer but we used uh play fabs play stream and the custom event handling to prototype out that feature pretty much in real time so it started out as an idea that came from one of our pms and uh i i kind of got involved which is always a bad idea to have to have you know old-timers like me writing code but i was able to whip up pretty quickly some javascript that would respond to these incoming play stream events from the client and pretty quickly we were able to build up what is effectively an entire feature purely in javascript um in a matter of hours versus you know weeks or months that it takes to define the real feature so i think there are lots of advantages to using playstream not only is it is it easy to to kind of instrument your the the events that you send on the client side but also how you you interpret and handle those events on the server side um we also rely heavily on azure geneva for for a lot of our telemetry and i'm i'm not quite sure if there's i think there's someone from from azure on the call i'm never comfortable with the branding because internally we call it geneva because that's always what we've called it at microsoft i don't know what the external branding the official name for it is but it's basically a massive scale it's basically the all you would need to send in all of your telemetry and metrics and we bet on it across all services in the entire studio it is the big bet so if that's something that that azure exposes uh that's that's something worth investigating but we use it for everything from service telemetry and perf counters to player data basically anything that that can be logged and instrumented we do so through geneva because it's just a super scalable pipeline i think you're muted brandon yeah it had to happen once didn't it um so the uh yes uh microsoft tends to use a lot of internal code names but fortunately that one is public so if you search on geneva analytics that will get you to the right place in the the azure portal okay um so there's an interesting one uh chad let's start with you on this uh the question is what has been the biggest technological advancement regarding game development in recent years has there been something that has moved the industry forward more than anything else yeah i feel like that was teed up let's just go right back to the title of this uh this talk i think the the fact that we've got so much capability online today um as as a group of technologies is is for sure been the most transformative for us um sure i mean graphics uh the power of gpus and cpus have made our games prettier and given us massive capabilities there but when you think about what what really lets us make games better faster and more adaptive to to the market to the players feedback it's it's absolutely the fact that we've got all these technologies online and it's not just that they're you know technologies allow our services team to deliver faster but the fact that they allow teams that would normally not interact with the results of those services uh and allow them to react act and react to them uh easily and seamlessly without a bunch of extra tools development like for a specific example like the pipeline that goes to um the game client wanting to create telemetry for a particular event so they can determine whether or not players are enjoying it or not the fact that they can do that easy and quickly without a lot of interaction and and long api um decisions and strongly type service endpoints and that kind of thing they just instrument their thing it flows through the system and collected through playfab maybe geneva all that infrastructure and then it flows through into power bi for some analysts uh to quickly be able to see the results of that those kinds of that you know one of those kinds of pipelines that's just an example of of how it's just made so many things um the technology kind of get out of the way because it's because it's pretty seamless and we get quickly to okay like is this is this feature working for our players or were we having are we having fun here so i'd say that's that that's an example um yeah let me give some time to some other folks to chime in sure travis why don't you uh why don't you talk more about this yeah man there's so much cool stuff that's happening um i think i have you know kind of a direct answer and a bit of a non-answer i guess um the direct answer yeah like um we've been iterating on game technology and game engines for a long time and and kind of seeing that that ingenuity that's previously been confined to basically a single hardware skew of some sort really get untapped to two different scales i think is amazing and um i guess my kind of non-answer is like i feel like we're just getting started that's why i joined the team and i'm so excited is that some of the coolest things to happening in gaming haven't happened yet when we talk about like the scale that's possible the social interactions that are possible you're starting to see these like transmedia experiences where gaming is less of a cul-de-sac of culture but kind of the main byway and that by the way kind of goes right through the cloud like that's where the capabilities are that are making games uh so fun and uh so fun to work on so i'm really excited when we start seeing distributed compute kind of come into things whether it's distributed simulation or distributed rendering um you know everyone's excited about game streaming i think that's going to be one component of it but uh yeah i think that you know when you have such a passionate and capable group of people that dedicate themselves to game creation kind of all over the world and we've been warming up this toolbox of tools and cloud technology in in in other industries and we're like okay like it's ready now like like come take this and make the new experiences that we're all looking for i think i think that's what gets me most excited about what's coming on fantastic thank you uh lawrence you have anything else you want to add on this yeah i wanted to start just by saying uh writing c plus plus is hard like for for a number of reasons and it's it's becoming a lost art unfortunately well fortunately maybe it's the only language that games are written in and will be written in for a long time so i think with the advent of cloud services i think now we can see this sort of specialization happen where the devs who write c plus game engines you know really kind of get the the heart of the the game play and then the graphics and all the kind of down to the metal technologies i think you can have them focus on what they're best at in the in the tool and c plus writing code in c plus plus and make those clients very specialized and dumb and then you can pow you can couple that with the power of of game services and then you can have add new features ongoing that augment and kind of and kind of work together with these these game clients rather than requiring and taking away from that specialization and kind of muddying the water and having game engine developers working on marketplace or purchasing it just seems like a waste of their talents when that all can be handled all that the logic and all that processing can be handled on the server side so i think that's my clumsy way of saying that in in a world where both technologies exist you have c plus plus you have you know cloud-based you know services i think you can use the technologies as they were intended right fast rendering and z plus plus and all of the kind of high level game logic and maybe even build the menu screens and ui and all the stuff that game developers hate doing you can offload that all onto the service side and then just feed it down to the client i think that's going to be kind of the next big wave you heard it here first makes sense it's uh travis i think you've got one more point on this don't you yeah you know i love that this is a fun panel it's always fun to get together with other with other gaming veterans uh just like lawrence was saying about how we can see more contributions from diverse skill sets in gaming you know we previously game technology has really been dominated by kind of c plus wizards that really know their hardware now we're bringing in uh skills all across web development and cloud development that can also contribute to these experiences and i think that also just represents a trend uh generally towards the democratization of game creation it's it's less specialized and it's less difficult to engage in game creation when there are more tools and more ways to build games and so i think another area where we're going to see some really great experiences coming out of it in a way that the cloud is changing the way games made is i think more people are going to get a chance to make games we're going to see perspectives from different people all over the world and different life experiences and it's it's going to make the landscape of gaming all that much more richer so there's this great continuum of like uh previously you know it was it was more of a niche but like now we're widened the envelope of who can can contribute to games and we're going to keep trying to do that with the with the tools that we're building and and i think the whole industry's on board the more of us to get to mid games the better oh absolutely i think that one of the one of the things that we've all felt the most passionate about is that the fact that we've been able to democratize the the game development process and provide tools that historically have only been in the hands of the largest companies to everyone so that they can all you know build their their most wild crazy dreams into end of the games they want to see and between that and uh what we like to call the the citizen creator capabilities that are coming on these days and games it's just it's broadening the not not just the opportunity for everyone to contribute uh but for the the ability for everyone to to unleash their creativity in new ways well we are already over time i mean this just got away from us fast because it's just it's so much fun to talk to everybody i want to really thank all of our panelists here for taking the time today to talk to everybody and i really want to thank thank all of you our attendees for coming to this session and for providing us with great questions that we could talk to um i'm going to jump over to the q a panel here in just a sec and uh do some follow-up but just a quick reminder again and i mentioned it earlier but uh we'll be posting the link to our gamestack discord if you want to come over there we have ongoing discussions on all the azure technologies that are for game developers uh happy to answer your questions there as well and thank you
2021-05-29