Games of Tomorrow — How VR Developers Are Mixing Their Reality
- Okay, ladies and gentlemen, welcome to Mixing Your Reality, this is the MR/VR Developer Showcase. My name's Kimberly Unger, I am a content sourcing lead and the holder of the mixed reality strategy, for our fresh new Oculus Publishing label. Woo, we're official. (audience cheering) Today I'm here to moderate a showcase of developers, who are mixing up the reality, and they're expanding from straight up VR into MR. Which is, you know, before several months ago, there was not really MR.
You had AR, and you had VR, and MR is kind of where the Venn diagram comes together in the middle, and we're seeing developers from both sides. We're seeing AR developers come add VR stylings, and we're seeing VR developers come and add AR stylings to get this new version of reality that, you know, they're officially calling meta reality, but really is mixed reality. For those of you who are, who are in the game dev biz.
Now these teams are all established VR devs, they all have stuff that's shipped to the store. If you do not know who they are, you should. Please go take a look at their stuff. I'll throw up some QR codes at the very end, so you can take a look. And if you don't know what MR is, or what, you know, our hot off the keyboard APIs look like, stick around for the Presence Platform panel, that's going to be a little bit later this afternoon, that I think is 3:30 in this room. And they're going to lay out all of the new hotness for you guys, and tell you where you can get access to, to the APIs that are coming down the line.
So, super quick, just to frame everything up for you. You know, VR has been around for a bit. It's driving revenue for our developers. The hardware just keeps getting better, and it's about to get a shiny new pair of big kid sneakers.
But in contrast, MR is just getting started. We're going back to ground zero. And, but look, the second kid always learns from the first, and the two of them are going to be asking a lot of new questions together.
And these questions on the differences between VR and MR in some cases are bringing us back to some very fundamental design questions. Which seem to have edited themselves out of my slide. There we go, okay. So things like, how do you take the design learnings from virtual reality, and push them into the real world, right? So that you can integrate your UX or your UI with the things somebody may have in their, in their real life environment, or gravity, or physics. Like how does gravity stop working, when you suddenly have to deal with physics that have to encounter the real world, right? How do you account for a game that maybe fa-noodles the numbers to get a great feel in VR, but then you bring it into the real world and all of a sudden you have to deal with things like real world distances, and real world objects? And then of course, you know, the next one is my favorite current recurring question, as we get from VR, deeply into VR, and we start thinking about things like locomotion and social interactions in the metaverse, what's up with my legs? But stay tuned, that one's coming probably a little bit later in the year. And you know, and there's always answers to all of these things.
Sometimes they're good answers, sometimes they're things like, "Hey, I smash this pot and I get, you know, I get a rupee, I get a jewel." Sometimes they're bad answers, they're things like, "I smash this pot and oh my God, get the super glue, mom is coming home in five minutes." But there's always answers. And as game developers and designers and engineers of all kinds, our job is to not only figure those out, but we're working to build the commonly understood library of game mechanics that everybody can work with going forward.
That foundational library of interactions is going to give rise to the next evolution of game design. And so a clever gimmick right, say a grappling hook in a first person shooter 25 years ago, will become the kind of fundamental art that all games in a category or a class are going to draw on. And those are some of the things that these teams are working on right now. The work they're doing for mixed reality, is going to serve as the foundation, and is going to be art that every design team starts to take a look at from here on out. So without further ado, let's see who we've got here.
We've got Therese from Moon Mode, We've got, is it Hymie or Jamie? - [Jaime] Hymie. - Jamie from Odder's Lab. We've got Thomas from Thomas Van Bouwel, VANBO, and we've got Samantha from Tender Claws, here at the end. And all three of these studios have apps live on platform. Go check them out on your quest, and let's kick this off.
So first off, let's talk to Moon Mode. Therese, would you like me to just hit the button, or would you like to come up here and... - [Therese] You just hit the button. - I'll just hit the button.
All right, perfect. So let's, let's get the mics turned on over here please. All right, so Moon Mode is a small, most of the time three person VR studio.
They've been in the game since 2013, and they are on a mission to prove that VR doesn't have to be niche, complicated or uncomfortable. And I think as we've all seen, these are problems that we've been solving for the past 10 years, and we're finally, finally coming out of the woods. So let me, let me kick this off. Moon Mode. (upbeat funky music) (upbeat funky music) (ambient music) - Holy carrots. - Honestly she's earned that nap.
- Hello, is anyone here? - You caused this whole mess, now you need to help me out. Find those invitations, and get them to the Hatter in the Keep of Hearts. - Follow me. Oh, a narrow passage, I love these. - That is Steve bravest of the Butterfly Lords.
- Okay, I fix, I fix. - You look lost. - All right, there we go.
(audience clapping) - All right these slides are in here, so I don't go fangirling and like forget I'm supposed to be a grown ass adult up here and actually doing the panel. So, so we'll start with Moon Mode. First question, if you could go back in time, what would you tell the bright eyed and bushy-tailed younger you, something you'd wish you'd known at the beginning of working with VR and MR? Like what's the one thing you wish you knew? - So we first started out like eight years ago, so back then it was obviously only VR, not a lot of MR to talk about.
But I think one thing that we've tried to follow for a long time is comfort first. Which is of course there's like as many players as they're, or hopefully eventually developers, and everyone has different preferences for comfort and for, you know, how easily they get sick or not sick. But when we started out, there was not a lot of games that were less extreme, and that's something that we wanted to do, was to focus on that. Because I had, I was trying to show my family like, this is what you know, people in my industry do.
And they were just like, "Oh my god, there's zombies everywhere," and freaking out you know, So I feel like sticking to your guts in a way, that like if you are a tolerant player, or a tolerant player yourself and a developer that you know, can take that then, then stick to that. But if not, then you know, follow your gut and like let's make as many games as their players that you know. Also neck health, like having a huge headset on your head, for a little bit of time, and you're like pop propping it up like this, and you're sitting and you're like doing this you know, it's going to affect your neck. So like do some stretching and be aware of that, yeah. - Perfect, thanks. - Thomas, how about you? - Yeah, I think from early on I wanted to play test my games a lot, to get feedback and make them better.
But one thing I would say to myself back then, is to try to seek out play testing with people outside of the games sphere. Like it's a great place to get started, 'cause everybody's enthusiastic about the same things that you're building. But testing with people who are not gamers or who have never done VR is the most valuable testing you can do, 'cause they're people who don't understand how to like hold a controller or like aren't that familiar with the medium. And if you're trying to make something that's very accessible to people and makes for like a good first time experience, you need that feedback and you will see things that you will not have seen when you play tested with other developers. So, I would've definitely tried to seek out that sort of play testing more, yeah.
- Perfect, Jaime? - So one of the things that we have learned a lot by force is whenever taking a concept that exists and works in real life or other mediums, whenever taking it to VR, making sure that you're not only translating it and to getting to count the limitations of VR, but doing it the other way around, which is what is like, take the size and the capabilities that VR offer, and including that into a working or existing and already working concept, instead of the other way around. - Samantha your turn. Let's look at MR, since I know you guys are doing some cool stuff.
- Yeah, so for MR, I think it's a similar, I think for us at least for VR and MR, I had actually been working in VR since 2002, and some MR in like early research labs. And at that point actually the thing that I would say is it's easy, it was easy to get discouraged over so many years but to kind of, if you find something interesting to kind of stick with it, 'cause there's cycles of things that come around, and like they're developing, technologies developing and getting more and more like intuitive and also more and more accessible. And the thing for MR that I'd say, which is also relevant to VR for practical advice, is to always build in R&D time. Especially if you are a studio working on the edge of emerging media, you kind of want to like have that time to discover and prototype what feels good and what works for you, and then double that time that you think you're going to need. - Sounds perfect. All right, Jaime I'm going to call you up here, to run through your slides next.
This is Odder's Lab, they're the crazy talented team behind OhShape Chess Club and Les Mills Bodycombat. He's come here all the way from Spain. So take it away, just push the button. - Thank you. Thank you guys.
Going to push right here. - [Kimberly] Later, later. - (whispers) There you go. Okay so real quick about Odder's Lab, the name Odder, is we love creating other realities, where possibilities and basically exploiting what VR possibly is bring, that is what made us create the first game in the middle, that is OhShape, then Chess Club here in the right, and Les Mills Bodycombat in the left.
And I want to tell a little bit about the story very quickly how we took the concept of Les Mills Bodycombat, which is a fitness guided class by a huge fitness company, and we took that concept into VR with this video. Which is coming after this. (whispers) There you go. Is it this one? ♪ Woo-Ooh ♪ ♪ I only ever going higher ♪ ♪ Off of a dozen of my desires ♪ ♪ I keep on running I never tire ♪ ♪ Copy, distracted, baby I'm wired ♪ ♪ I keep it 100 to 40 blocks ♪ ♪ Boy you must've interested the crowd ♪ ♪ I would do the work and get the drop ♪ ♪ Trust me, it's different from the top ♪ ♪ Only if you keep it ♪ ♪ Honey, honey, honey, honey ♪ ♪ Honey, honey, honey, honey ♪ ♪ Honey, honey, honey, honey ♪ ♪ Honey, honey, honey, honey ♪ - So this is a quick introduction, that is what Les Mills does in real life. Bodycombat is one of the programs that they had, and from there we took into the VR, and luckily we have the success on the first year, being named the Best App of 2022, on the Quest store. And we faced many challenges, which we will discuss here.
Basically adapting all the moves from the real life Bodycombat into VR. With things like knees, kicks, elbows and things like that, and there were many factors that we had to take into account such as balance, tracking and surroundings. And in terms of MR, this is where we are studying and researching now on how to bring the trainers, into your real space to create a deeper connection with your trainers tried at home. And also align for more moves that it has also the balance, such as kicks that is going to be something super tricky in VR, but MR is going to allow for that, so that would be it, thank you.
(audience clapping) - All right, so sort of drafting what you were showing us there Jaime, so in VR, you have total control, right? In particular in the app you're designing, you control the environment, you control players see, what they hear, not always what they smell, but you know that's an entirely different add-on. So, what design challenges are you starting to notice, in terms of designing for an environment that is different for every single player? - So there's many challenges there. Especially when you have targets coming at you, depth is one of the things that, not everyone at home, it's already hard to have two or three meters wide, so imagine if we're talking that.
The targets in our game come from 10, 15 meters away, so that is something that, if your wall is two meters in front of you, that is super challenging. So we're tweaking a lot with the speed, and maybe instead of coming with depth, they're changing in size while coming at you. So that is one of the biggest challenges that we have, and also particularly when painting those targets and objects that come on to you, with obstacles that you have at home, so that's why for example some technology, such as creating a portal, is helping a lot with adapting to any room in the world.
- Perfect. All right, sorry Samantha I'm going to hit you up next, because I know you guys were working in AR, for quite awhile before you came over into VR in the first place. So when it comes to like moving into MR, what are you guys foreseeing, as some of the challenges with working with those individualized living spaces? - Thanks. Yeah that's actually a really good question, because as you'll see there's going to be a trailer of a project called "Tendar." Which was a AR virtual pet fish, which feasted on your tears.
That turns your emotions into fish food, and it grows as a pet over a few months. And that was using a lot of object detection, and the context of that is that it was meant to be very intimate about your space. I think that is maybe what we're all touching on in MR, is the difference between designing for generic space, and designing for an intimate space, that is your space. That is why we have mixed reality in passing, is to be about your world as well. So there are some challenges in that, in terms of like you know, a aclusion that can be applied across all different room sizes to marking in different like, furniture and objects in the room, and how do you create something that feels like compelling and unique to you but also can be generally applied.
- Perfect. All right, I am going to bump us along a little bit, because we only have 30 minutes in here. So, Thomas you are up next, let's hop over. So Thomas is a one-man studio, the man has never made a API he didn't like, he is one of our top developers on the platform.
He's a developer of "Cubism," which we'll show you in just a second, and come on. (audience clapping) - Appreciate it. Can I play the video? - [Kimberly] One more time and it should start it. - [Thomas] All right, so my name is Thomas. I made a puzzle game called "Cubism," which launched back in 2020.
And this is a very simple game, about putting blocks into shapes. Because it was a very simple game, it was also a really fun playgrounds to experiment with all the cool new API that were coming out of Quest, and so over the years I had updates that added hand tracking support and best support, which you can see here, and there are still updates that are in the works. So in this video for example, you can see scene of the eye support, where you can sort of snap a puzzle onto a table, and then at the end it will persist. Kind of like you would have a Rubik's Cube on your desk, that you can get back to, to do like a quick puzzle, that's the idea there.
So obviously like this was added to "Cubism," but I really wanted to work on something that was native to mix reality, and that really show what you can do with room scale mixed reality. So I'm working on new game called, "Laser Dance," and the idea is you can turn any room of your house, into a laser obstacle course. So yeah, you set up your walls, you set up your furniture, but then you also calibrate these two virtual buttons, which you would see in just a second.
On to the opposite walls of your room, and then the idea is that you basically go back and forth between the buttons. And each time you press them, this parametric pattern of lasers appears, and you have to move your whole body through it. So it's this very physical, you know game, that kind of looks like a dance, when you're just observing it, and that's why it's called "Laser Dance."
So these patterns are sort of adapting to the room, size and layout. If you have furniture, the laser pattern will try to go around it. And depending on the patterns, you'll have to move in very different ways, like your dodging, your dancing, and sometimes you're sort of boot camp crawling on the floor (laughs) which is always the most fun to see. Yeah, so the idea is to try to make that fun and accessible to as many people as possible, to as many spaces as possible. So that's in the works.
(audience clapping) - So, start with a question, I'm going to start with you Thomas. So developing for different room spaces is one thing, what about developing for different people spaces, like people are different sizes, different ages, different levels of mobility, how do you take that into account, when you're building for MR and for VR? - Yeah, I mean it's a challenge for sure. Especially for Laser Dance, which is like inherently very physical game, but when it comes to mix reality, and it comes to room scale mixed reality like this, I'm definitely spending a lot of time building tools to make that I can emulate different situations. For different rooms, I have a system to emulate different rooms, instead of having these side by side as I'm designing the laser patterns, to make sure, okay, what does it look like in a smaller room, in a big room, in a room with a lot of furniture, with no furniture, like narrow hallway and wide space. And so I speak all these parameters, instead of all these bunch of rooms next each other, to like directly compare this. And I also have parameters for body types and mobility limitations.
And so the idea is to try to make all these patterns accessible as possible. So there are things like variables for the players height, or their shoulder widths. But also essentially like how low can they go, not everybody can boot camp crawl on the floor obviously.
So the idea since everything is all ready parametric, like it just ask a few more parameters, it tries to adapt the patterns in that sense, try to make it as successful as possible is the goal. I know it'll be hard for "Laser Dance," because it's by its nature it's a physical game, but it'll be definitely fun to try to get as many people into that as possible. And to get as many people comfortable doing these types of boot camp things, as possible. - Sounds perfect. - Therese, how about, you know, designing for different kinds of players, for things like "Spacefolk City?" - Yeah so, we actually found out quite early on, that we have very wide spectrum of players. We have players who are helping us debug, like coming through like virtual desktop through steam, into you know.
But then we also have players, who never ever played VR before, who never had any kind of game device. And we've been working a lot with our community, and listening to people, and trying to adjust and test a lot. Test with different groups, get your parents, get your family, get your friends, people just like try it out, because it's a very wide group of people, that are getting into VR.
So, I would just say test a lot. - Yeah. - Test a lot. - Perfect. All right, I'm going to jump ahead to our final showcase, Samantha do you want to come up here and talk over your video, or should I just push the button, let me know what you... - Um, yeah. I think I can say a sentence, and you can push a button. - Sounds perfect.
- So yeah we're a studio, we came out of a 20 year art practice, with some of my collaborator, and now were a studio of 15. We work a lot on designing for the edges of emerging technology and the affordances of what they can provide the player, as well as what type of stories they can tell. So we have a lot of narrative experiences, games of the future part of it, is that you'll see some of the AR stuff I mentioned before. And also, we have a game that had live actors in the game as you see it, throughout the different time zones during COVID.
So thanks. - Fantastic, all right. Here we go. (light music) - [Announcer] Let's set the stage.
Put on your headset. Imagine a ship at sea. Enter of a storm at tempest. A production sunk, a theater closed, but I the actor am ready to perform.
My good prosper will bring the play to life, with the help of my magic and you, my spirit. We are ready to perform together. Approach my areal, come. (classical music) (audience clapping) - All right, so I was going to ask the panel a few more questions, but we are down the five minute mark. So, let's see if the audience has any questions, if somebody has a question they want to raise their hands.
Could I get you to walk up here to the mic, so we can record it for posterity? - [Audience Member] Hi, I was wondering if there were any difficulties that you encountered with mix reality, in terms of motion sickness? I know some people experience that with virtual reality, but I was wondering how it differed for mixed reality? - Yeah, there's definitely still things that could cause motion sickness. I think like hitting frame A is still very important and that's something I've seen, that can still affect the comfort level. Even though if you're doing a full mix reality game, but obviously there's also like, it's a bit of a gradient how much your mixing reality, or how much the world you see.
So if you are replacing a bigger part of your room, with a VR scene, there's a bigger chance of risk for motion sickness as well. - Anyone else want to add on? - Yeah, I was going to say that I think it depends also like where your focus lies, where the players focus lie when they play? For example we have "Spacefolk City," which is you build with your hands. And you kind of look at your hands a lot, and there's still like some distortion happening, behind the hand. So some players have said that because they always looking at their hands, and they're building, that can sometimes make some people feel a little bit dizzy, but if you're not like doing this all the time, like it's probably more fine. - Awesome, next question.
- [Audience Member #2] So I feel like everybody said that, MR's important, MR's the future. But what are the best or the biggest advantages do MR games compare to real games? - From playing the scenes so far, it does feel like the barrier to entry for new people, seems a little bit lower. Like when I compare playing "Cubism," to play, "Laser Dance," 'cause there's a lot of people that have not done VR, or don't do that regularly, and for them to get a headset to put on them, but then see the person that is explaining what the experience is, we try to always meet, but then you know when "Laser Dance" releases, it would be the people who are enthusiastic to show their family and friends. That barrier is a bit lower, so I think it just adds to the comforts of the first experience in the headset like that. - [Audience Member #2] Okay, so you saying that the natural interaction is the best advantage. - Excuse me? - [Audience Member #2] You think the natural interaction is the best advantage? - It's one of the advantages for sure.
- [Audience Member #2] Okay, thank you. - All right, next question. - [Audience Member #3] Hey, will developing in MR/VR, do you guys have any tips for like not getting sick yourself with early prototypes? - Don't test for to long.
- I think if you're getting into developing MR/VR right now, I would definitely say like, some people believe in VR lengths, like we generally don't. I believe like you learn your limits, and you learn when to push yourself, and when not to push yourself. I don't want to test early in the morning when I'm hungry, 'cause I'm going to ruin my whole day. But I know that after a meal you know like, if I'm going to test something that has a little frame rate, which happens sometimes during development, you know like learning when you can push, and when you can't push, I think it's really important.
Take breaks a lot. - [Audience Member #3] Thank you. - All right, we have two minutes left. Let's do one more question.
- [Audience Member #4] Have you found running a paucity, between solid objects and transparent like safety concerns, as things get closer to you, have a different way of thinking about that. Like a lot of big things come towards you, they can't really stay that way when they're close, 'cause you can run into things and not see them. How do you guys deal with that? - Well what I'm building in mix reality, at least in "Laser Dance" will also the case in "Cubism," is I try to always draw the binding box around the real things. Very lightly but just so have a frame of preference of what abstract, what does the game know is there, whether you know is there. And at least for my use cases, it doesn't like conflict too much with what's going on with the game, and I do think that helps like safety a little bit. I'm not sure if that directly addresses your question.
Anybody? - Yeah, I think that's actually a very good question. There's something that ties into as I said before, I think that the main event in MR, helps you see the world differently. And like make what is normal to you, kind of like profound and strange in a new way.
And like therefore in the paucity in the way you like want it to be, like clear or near for your, for the point you were saying about safety concerns, is part of what, you know, we're really interested in is how it make feel like windows to new worlds. And therefore it needs to be as like, not transparent as possible. So how do you design for that? And I think part of that is like, you know using the shapes and the constraints of your room, so that people can see much of the room, but as they feel you know, peering into another place.
- [Audience Member #4] Thank you. - All right, last question. - [Audience Member #5] Hi guys, great talk, really interesting. My question is, I work with mix reality as well, or I try to. So my question will be, what is, in your opinion, the main principle, or maybe the general mindset we should have as mix reality designers, as compared to traditional gaming, or VR. What you should change in your, when you're thinking as a designer, how you start a project.
What is the mix reality mindset, if that makes sense? - I think for like any platform, is good to think about what's needed for the platform that you're working on? And what is unique about the parameters that you get with a platform like this, and I think that applies to mix reality as well. Like what does it mean if you can use your entire space? if you can blend virtual and real things, so what are the things that are unique, that you can only do with that? It's very easy to like, use as just an underlay for a game, but there are really things that are unique, when you start to really try to mix reality in using your entire space. - Yeah. - Absolutely, I think it looking at you physical space, and find ways in how to enhance that, instead of coming again from the other angle. - Yeah, we always start by kind of trying to imagine, like what is actual, what is something that feel really amazing to do.
You know if I could just change the world with my mind. You know like I want to have little houses pop out of the wall, and like wouldn't that be amazing? And like being under the ocean, or like all these things, and you know like trying to build on that. On what feels good, what feels exciting. - All right. - Hard second on all that.
- All right, I think our 30 minutes are up. Thank you everybody for coming, this is a fantastic turnout. (audience clapping)
2023-04-23 02:43
