virtual reality and inclusion what does non-visual access to the metaverse mean moderator Bill Curtis Davidson speakers Alexa hoof Brendan Biggs and Aaron Gluck hello everyone I'm Bill Curtis Davidson I'm really excited to be here today and I want to thank the site Tech Global team for allowing us to come talk with you today about the metaverse and accessibility we're really excited to talk about some ways that people with disabilities and as accessibility Advocates are working to make sure these Technologies like augmented virtual and mixed reality accessible to everyone I'm joined by some amazing panelists today who will share their experiences using these Technologies discuss barriers that exist and possibilities that the Technologies create and they include Alexa Huth Aaron and Brandon so I'm going to ask each of them to give a quick introduction right now Alexa would you like to start of course it's wonderful to be here today my name is Alexa hoot and I am a white woman with teal hair and I am Pete's director of communications so that's the partnership on employment and accessible technology and my connection to the metaverse is growing so I started losing my eyesight about 13 years ago due to a condition known as lattice degeneration and I rely on my left eye but I have a lot of visual challenges so I'm excited to talk about them today and how the metaverse might make for a more inclusive future so thank you thank you Alexa Brandon yeah thank you for having me my name is Brandon Biggs and I am a researcher uh that focuses primarily on non-visual maps so Geographic maps and uh virtual reality uh I am blind as well and I'm basically uh totally blind but I do have a little tiny bit of vision I can see shapes um so primarily um uh I I use audio as my primary modality and I'm focused on using audio game conventions and audio games that are games that can be played completely in audio to bring uh accessibility into the VR and spatial information presentation space I'm also interested in making headphones which are the most ubiquitous virtual reality headset a first-class citizen in virtual reality great thank you Brandon and Aaron Glock would you like to introduce yourself yeah hi I'm Aaron Gluck I'm a fourth year PhD candidate at Clemson University um where I actually study accessibility both in autonomous vehicles and in virtual reality um so I started working with VR about seven years ago and only took about two years to start asking the questions of how do we make VR accessible for those that VR is not currently made for foreign I'm really pleased to have each of you here today because you each have such interesting and diverse backgrounds and really to start off this discussion I'd like to ask each of you to comment on some of the reasons why you think these newer immersive Technologies are exciting to you what opportunities do they create and Brandon I'd like to ask you to start us off yeah so uh XR Technologies and virtual reality in particular is a fantastic tool to represent social interactions and information as close to the real world as possible so specifically within social interactions and I find in in the interfaces that I work on that the cognitive load is very very low if you make the interface well and the learnability is really really high uh one one thing that I'm very excited about uh going forward is that blind people will for probably one of the very first times in history be able to socialize with uh everybody around them at the same level as sighted individuals will have access to non-verbal social interactions uh that we don't usually have access to in real life so I think that that's going to expand inclusion uh significantly I'm also extremely excited about the haptic information and technology that is being created as part of this virtual reality expansion to the metaverse so uh I I can't wait till the vibrations are only part of the haptic experience and we've got force feedback and textures as well and I'm very excited that sighted individuals are paying more attention to non-visual experiences as it relates to XR oh thank you so much that that's really great to hear and and um I'd like to ask Alexa to comment next sure um a lot of different reasons that I'm excited about immersive Technologies uh personally I grew up loving PC games um just any sort of experience that took me out of the Daily World and put me in something interesting so when I first started with VR and I tried an art application it gave me that same feeling I had as a kid when I played return to Zork it gave me a sense that I was really entering something new and I was so excited about it and I was able to jump into the world of the art that I was creating and really focus on it with my left eye which is the left eye central vision is primarily what I use and that has so many applications thinking about it from a workplace perspective being able to maybe expand a presentation that I'm seeing so that I can really actually engage with the content instead of what I generally do which is pretend I'm engaging with it um so I think there are a lot of ways to close distance gaps to spark creativity and to bring kind of a closeness among people if the technology is built with inclusion it might of course I I love that you point out the creativity part Alexa as you know I begin my career as a fine artist so I I love the creative aspect of these Technologies uh you make some great points um Aaron what's getting you excited about these Technologies for me it's really that giving us the chance to really do whatever we want to do in this we're no longer uh defied by our movements are no longer defined by physics uh reality doesn't have to be reality anymore uh so it's the question that's always come to me from people that are new into metaverse virtual reality augmented reality is what can you do in those type of spaces and I hate to use it but I always use it the response of there's nothing you can't do so it leads to this open world where anything and everything is possible just because it hasn't been done yet doesn't mean that there's not a way to do it so it really excites me to be able to bring not only new experiences to individuals that maybe wouldn't have them but also to allow people with different disabilities to access these environments that they would potentially have access to as well great thanks to each of you for sharing uh what makes you excited about these Technologies I'd like to talk next about what's attractive to you as a Tech Challenge each of you mentioned so many different aspects of these Technologies from haptics to Audio Only experiences their audio primary experiences and they're each of these has an incredible amount of Technology and Engineering behind it that is moving at lightning speeds so what what's exciting to each of you about what we're working on in terms of making this set of Technologies more inclusive and I guess I'll start um Aaron maybe ask you to comment sure yeah for me it really was a the first moment I put on my virtual real the first virtual reality set um I felt like I was the person I imagined myself to be as a kid I was the hero in the story I was there I was experiencing it the moment I took it off I had was just hit with this sense of I have to get everyone that I know and don't know into VR they need to experience it and so I just spent a lot of time uh in those first couple years just learning and uh growing trying new things in that experience and it wasn't actually until a couple years into it that I was uh talking with my now advisor and accessibility was brought up and I said wait a minute VR is really not an accessible medium it's a especially for uh those with visual impairments since VR is a visual medium so what can we do how can we address that and so I set myself a challenge I said I'm going to figure out something to allow no visuals to be needed to be used in the virtual environment what's nice at least with most modern day headsets or VR systems is you have the ability for head tracking hand tracking uh you've got microphones you've got uh haptic feedback uh you've obvious that 3D spatial audio and so by using those I was able to actually build my first uh non-visual VR experience uh being closing dark prototype and so from there it's just built and grown from that for me you know that was a nice slow exploratory experience I've started exploring what does it look like when you need to without visuals make instantaneous decisions and so what's the best method for that and actually have just started work on a prototype exploring how do you do an equivalent uh first person shooter in a virtual environment so you know the challenge as a programmer and developer for how do we make these things that have for so long been a visual based information processing how do we do that without the visuals stuff that gets me up in the morning and allows me to have uh just a fun time exploring different things well thank you for sharing those stories and examples um Brandon do you have anything you might want to Pepper in here in terms of the work you've been doing and what challenges you've been addressing yeah yeah absolutely so the biggest challenge that I have is that people think uh that that they've been focusing primarily on visuals and so there is the whole field of audio and and haptics that are are very unexplored and significantly underfunded so uh you know the world is twice as big when uh three times as big if you if you add on these these sounds and and haptic experiences and I think they can be just as uh interesting as the visual displays so my challenge has been how do we present information in what's known as a cross-sensory way so that the information is accessible equally visually auditorially and tactilely and so uh most of my my research has been on how do we make these modalities uh independent of one another so that we can use them and then eventually combine them together to create a really incredible experience where you can choose what modality you want to use and so that's the that's the real excitement here is that we've got uh you know the fields of visual data visualization of visual virtual reality they've gone you know for for years and a lot of stuff has been solved there there's so many things that haven't um but you know there's there's been there's a lot of work in that space but there's hardly any work in non-visual virtual reality and that's even bigger than visual virtual reality so that's a that's a really is it super exciting I love what you're pointing out there and I guess I should pause and say to our audience that if you don't know a huge amount about these Technologies one thing that you'll learn more and more is that the the hardware um is becoming more and more capable of sensing the environment right our movement as a person is we're wearing it just like our phone is also capable of that and our phone is also getting more of these capabilities uh as well as even some other types of devices but also sensors of various kinds cameras spatial mapping sensors audio sensors um so as uh we were just hearing some of these Lively examples I think each of these has a Tech Challenge to it right but first we need to ignite our imagination and Brandon I love that you um you say that this is under tapped right people are privileging in a way the visual uh domain but really need to unpack these areas um and I think one way that we can and should do that that is to understand the barriers that people are having right because then we have problems we can solve for and I know each of you are exploring that in your own way um Alexa I would love to ask you to talk about that very topic like what kind of barriers have you felt when you've been using these type of spatial or immersive Technologies sure I I barriers are peripheral vision I find that a lot of these experiences place things outside of my visual reach just by default and I think it would be wonderful if there was a way to customize obviously accessibility options that are never enough but these should be mainstream options as well where do you visually Focus if you do visually Focus would be a really important question for me because it is my left central vision and when I hop into any sort of an immersive experience I'm immediately scrambling uh is there a a menu bar perhaps below my field of vision so I'm looking everywhere it's very overwhelming um in addition to that I am very light sensitive and so at any point this is a very bright room for me that I'm currently in and it's just because we're recording I usually like to be in a very dark environment so when I put on an hmd I am again overwhelmed by how bright it is it's pretty painful if that could be customized down for every immersive experience that I do that would be wonderful so I think really a lot of the barriers are fixable if we are thinking beyond the pretty we're not thinking about the colors and the oh should we put something on the virtual wall that's great but that's something that's going to be a virtual clutter for me and with the outside world the physical world being not built for me already I can only control my apartment and I would love to be able to control my virtual space as well so I think there are a lot of barriers but they are fixable for me personally yeah I think we'll talk a little bit more about design specifically later but I think one of what occurs to me is the first step in inclusive design is recognizing exclusion and then using that as a way to design for problems um and uh what occurs to me is when you talked about dimming the display or focusing uh these are things that have been done in other areas like in gaming for example we have accessibility features in games where you can highlight you know core actors or um to to turn off audio you know uh extra audio you know there's a lot of things that we can learn from other media but it does require us at first to recognize exclusion number one because we don't want exclusion we want inclusion but in order to do that we also need to recognize the value for others Brandon do you have any barriers that you'd like to talk about especially for people who are blind yeah yeah absolutely absolutely none of the mainstream virtual reality platforms are usable for me right now so I'd say that's probably the biggest problem uh and it's not that we don't know how to do this it's that none of the platforms have really um contacted the right people or figured out the they haven't really implemented any of the changes that are needed for this uh and and um you know that part of that problem is we go back to the the idea that virtual reality is visual so I've worked with people uh on the webxr specification and it was so inherently visual initially I had to go through and and um and tell them it's uh you know it's actually not inherently visual we just don't necessarily have the correct displays at the moment uh to to show these um in in different modalities but headphones are there they've been around for a really long time and uh if you talk to blind people about head mounted displays they will say I don't need the camera so the the um the visual aspect of it so why why do we even need this thing it's not useful so the headphones we wear these for 14 hours a day and they work perfectly and a lot of them have head tracking now so they are um full VR headsets uh so um yeah basically uh audio games have suggested many of the conventions that we can use for social interactions and and interacting with the environment and creating an immersive virtual world and so the um the visual virtual reality experiences just need to become as immersive as auditory uh audio games great and and while we're mentioning thank you for all of that Brandon that's really um wonderful points you're making and um while we're mentioning gaming I should say that the adoption of these Technologies of course virtual reality gaming is is there's plenty of that going around but I think the adoption of the hardware is skewing toward Enterprises who are maybe can spend more money on these devices they're still not as commonplace as they will be um and also they're changing rapidly and so um what we're seeing is Enterprises or workplaces beginning to use these in different ways and we'll talk about that more in a minute but as we're talking about barriers maybe to focus a little bit on we talked about the barriers and how uh these exist today and while we're still working on all of it there is an emotional or mental impact when if if you are let's say you do get a new job and and you're offered a training scenario that utilizes virtual reality what happens if you're blind or have low vision or limited vision what if the company rolls that out and hasn't really thought through that I'd like to ask Alexa to comment a little bit on that because we do so much at Pete talking to the employers or a doctor crowd Alexa would you like to expand on that uh to pull that thread that you've started so imagine that I have started a new job and I chose not to disclose my disability which is absolutely fine but then they give me a virtual reality onboarding experience and expect me to complete it within two or three days that's that's not going to happen I can maybe be in an immersive environment for 20 minutes tops so my choices there are either disclose which is a forced disclosure which should not be happening or I would have to fake it until I made it and hope that they're not getting records that perhaps it's taking me a week or longer to complete these modules that they expected to have done these are very exhausting challenges and they're things that people with disabilities are confronted with daily multiple times a day and it shouldn't be happening especially in the workplace because that is just such a barrier and so I'm glad we're talking about the mental and emotional impact of it because there are so many times when I'm confronted with these things that aren't made for me like I said I can only control the apartment environment I can't control the virtual environment or the physical environment and we need to think about them very very carefully before we roll them out to employees or even just people suggesting them and saying oh this will help this will fix it be careful with that language because a lot of times you don't know what set of barriers a person has and so unless the technology is designed flexibly and co-designed with people with disabilities which I know we're going to get into design later but there is no way that you can imagine my experience and so that won't be baked in and I will still be excluded and so I think it's a great thing that people are onboarding with exciting new technology but it does need to be inclusive first great points and I think uh as we're talking about all of these what can you do right um there is fast moving development as and Brandon and really air in it and Lex have mentioned um there while there's no perfect solution there are a lot of things that are happening uh to advance this and I'd like to ask uh each of you to comment on that so I mean these include areas like human interactions right like communicating interacting with other hum real humans through these Technologies or what's happening with hardware and fallback options input options Etc and then also in the content 3D content space um so maybe I can start out with Aaron uh see what comments you have in this area sure so so my specific focus is looking at how do we make commercial off the shelf uh virtual reality systems accessible so you know they they include uh you know an Oculus Quest too if you go to the store and buy one you have a head tracking hand tracking you have controllers that have haptic vibrations uh you have um glass spacers to you know assist with glasses uh you have a microphone that can use a voice user interface um so there's a lot of things that are built in to uh to these uh devices and equipment uh the problem that I specifically see is that we're not using them to help make it accessible they're just there to try and add to the visual experience so it's not like Brandon was saying it's not looking at how do we make these uh uh the main focus rather than and lowering the visual requirement so um you know where the pieces for the most part are there um you can uh it wouldn't it shouldn't be hard to change the the contrast level or the uh the brightness of the display uh that would help uh Alexa out a lot you know those things are there what it takes is the time and the realization that hey we could do this it doesn't take that much uh and I know a lot of researchers are starting to explore these areas um the trick becomes how do we take what's being researched in uh you know academic Labs or in Industry labs how do we get that to the manufacturers and the developers and I know we'll talk about that as we go forward but it's really that disconnect between those of us who are passionate about making this experience as accessible as possible and I know that's going to be a step-by-step you know feature by feature modification by modification uh process it's not sadly going to be a snap your fingers and it's all done overnight but we need to spend that time we need to show that the research is being done we're proving that these things can be done that we can treat uh immersive environments in such a way that they're accessible get that into the hands of those that you know are are producing them that are making the games making the experiences doing the training uh the virtual trainings things like that we need to we need to get them on board and get them connected uh bring everybody together and have those conversations yeah those are great points uh you know having worked in this area in some detail and even my most previous role at Magic leap um I know um a lot of these areas are literally being developed or have recently been developed in the last few years things like voice commands to open up applications instead of using a launcher for example that's Visual and requires a pointing device or having a segmented dimmer which is now available in the release product magically too to focus on content right that is um you know something that you uh you know want to focus on that could be useful but again um these things are being developed literally in real time almost so it's like a moving train um isn't it so Brandon what what thoughts do you have I'm curious about um some literally some of the platforms or developments that are happening that will be useful for accessibility yeah so I mean if you're building one of these tools the thing that's going to get you the fastest uh way to accessibility is to find designers or human computer interaction experts who have the lived experience so very few and far between but they do exist and so if you can find those people they're going to get you there the fastest because they'll they're going to have the the research in the background to really take the platform to the next step the the little bit slower option uh and you know both both um people with lived experience and everybody else should do this is to find other users with the lived experience and test it and test the different interactions but for example with with lived experience dictating you know some of the interactions that really work um the the non-visual experience is something that blind people have solved for themselves they've made virtual reality for the last 20 years and it's really um been extremely immersive and so I'm in the midst of writing up a really comprehensive paper on all all the interactions that take place in these non-virtual non-visual virtual reality experiences and I've asked people how they would you know expand those out to some of the interactions that are in visual virtual reality that aren't necessarily in the auditory virtual reality space so um a lot of it has to do with you know really basic uh um elements and you know contact me if you want more information but it's it's stuff like you know naming every single interactable mesh in the in the area having Collision sounds or having footstep sounds having some kind of way to scan around the area um with you know first using a person's screen reader and having text information uh and having you know menus where you can interact with the different elements in your room uh and so there's there's a bunch of different interactions that you can you can do to make your non-visual uh your your virtual reality experience non-visual and um blind people have been building these for themselves for a really long time and so it's just kind of a matter of looking at what they've already done and almost interactive interaction researchers who are blind themselves will be able to answer this and I'm sure it's the same for other people with lived experiences with different types of disabilities so low vision individuals who are hard of hearing or deaf or you know all those different um different groups of people so um yeah I think that uh that this um this is an area that uh really needs more highlighting uh what what if blind people or people with disabilities done already in this space uh that hasn't hit the mainstream stage yet all right plus one to everything that you just said um to me what's so exciting about this is really that hacking that lived experience is really a lever for Innovation and I love what you said uh if you want to do it the quick way which who doesn't want to do that right uh you need to you need to have people with these diverse lived experiences on your team because they're going to come up with angles that you would never explore on your own because of your lived experience I can speak as someone who does not consider I consider myself an accessibility and inclusion Advocate but I am not a person with a disability so um I um myself count myself in that club right of people who need to interact and feel like that's imperative it's not an option um and I love some of the work we're doing with 3D content descriptions with groups like equal entry in the XR Association where we're literally working with people who are blind or have low vision to figure out how do you actually navigate inside uh containers of space right at the high level going down into an environment going down into a building into rooms and into spaces within rooms and even Furniture um how do you actually experience that number one in the real world the physical world and what would you expect to have in virtual and that is so productive to work with people uh who have the lived experience so I just wanted to kind of emphasize another concrete example of things that are being done right now um with the time we have left I want to quickly ask Alexa to talk a little bit about workplace because uh at Pete we do quite a bit along that line uh working uh in the way we do Alexa what are your comments about the future possibilities and imperatives for workplace I think that there are a lot of interesting applications so as you mentioned uh at p-e-a-t w-o-r-k-s dot o-r-g we have an inclusive XR and hybrid workplace toolkit and that will help you get a sense of exactly what you need to do to be understanding how these Technologies can apply to the workplace in inclusive ways what you should be looking for how you should be using them we also have a joint white paper with the XR Association that brought some really interesting applications for the workplace so the one that I think is exciting is training for especially in the medical industry um so if you're a person like myself and you're often at the eye doctor often they'll have a training doctor come in and look at my retinas and so I usually go through three or four doctors before I'm done with a typical visit if there was skills-based virtual immersive training perhaps they could understand virtually and that would be great I would love training like that not that that replaces lived experience but in some cases because I do have a condition that is often not common I will just say that um and so if people could experience that in an immersive environment and train with it and learn about it uh just even to catch the signs because mine weren't caught early enough because it wasn't something the person was expecting to see so training in the healthcare field I think would be a really great application and it's already happening in in certain certain hospitals and certain training environments so I think the pandemic really sped that up because people weren't able to do the type of Hands-On practice that they were before so that is something that I look forward to not that I would ever begrudge somebody learning from my situation but I think that there are some really great skills-based training and upskilling that can happen in immersive environments [Music] yes thank you so much and and I'm glad you mentioned our P inclusive XR and hybrid work toolkit where we go into the detail for example on meetings which meetings apply to every other use case right which is why we chose that area what do you need to think about before you try to engage with other people what do you think about when you are engaging with them and it's it's broken down into areas like language and communication mode right so if you're communicating for example with sign language how do you actually incorporate that into VR like if you're doing a VR Meetup like we do with equal entry well you actually may have to put a video feed into it and have a sign language interpreter that's always present again how do you do these things is an evolving art um but we need to do it and it's important to look at that for inclusion in the workplace we just have a few minutes left and I'd like to um sort of wrap up by talking about we've talked about all of these challenges all these opportunities and I guess what I'd like to get each of you to comment on is what should designers developers and even implementers do now um if they want to get more involved in this we want a big tent here I think all of us would agree how do you get involved and maybe I can start with you Brandon yeah if you really want to um get more into non-visual experiences of virtual reality I would contact uh the the different research institutions that Focus specifically on that um on that particular uh disability for example the Smith eye Research Institute is all about vision and uh gallaudet University I think has a research institute that's on the individuals who are deaf and hard of hearing and so you know focus on contacting those places um if you want uh you know low vision stuff I'm probably a good person to talk to about that but yeah also you know think about um these different disabilities as uh you know a different challenge to to kind of create this experience for for those individuals um and and what input devices or output devices they'd be using you know I could go back to this uh idea of headphones and you know once we do have these experiences how is that going to increase the accessibility of VR for individuals of all different types of abilities so for example headphones just happen to be in almost every single household so uh you know it it'll decrease the level of adoption required for virtual reality if we can everybody has the tools already to use them yes thank you very much um Erin what are your thoughts yeah I think it's a it's an interesting problem that we're we're at kind of a Tipping Point at this point where uh the information that VR XR AR is just not uh as accessible or Equitable of a technology that more and more people are starting to have these conversations uh I think businesses uh developers are starting to realize that uh the Ada laws are going to uh as we you know had with the web are going to start being enforced in a immersive environment as well and so being able to start looking at these I don't want to call them problems but challenges uh you know that to how do we uh develop systems that are going to be more accessible um how are we going to develop applications that are going to be more accessible you know I think it just leads to kind of a change in the mindset here we're going to you know we're going to take an extra two weeks uh to get this product out and we're going to build in you know x y z to give more uh accessibility to what we're doing and I think as developers and manufacturers start doing these small things they're going to see uh their sales increase I mean uh if you can make something that wasn't accessible to a group of people let's say you know even if it's just a hundred thousand people that have something specific that you can create an accessibility feature for you know you now have a hundred thousand new potential customers and not that it's about that bottom dollar but from a manufacturing development standpoint it often is that you know we have a budget we've got to go with it so uh I think the the manufacturers designers they really like I had mentioned before they need to spend the time reach out to researchers that are looking into that uh reach out to uh the community of people with the disabilities that they're interested in making accessible bring them on you know bring in a consultant do co-designers have someone who can has that lived experience to be able to help get that through I think that's where we're going to really see you know an uptick in those accessibility features as we go forward and there is uh plenty of room for that to be done um so just need to to realize you know taking an extra couple weeks to get a brand new game out uh people can live it's okay you know make it more accessible for everyone thank you so much I totally agree um Alexa please uh give us your thoughts really quick before I do um Brandon had said we could people could reach out to him Brandon could you give an email address or something where they could reach you yeah brandon.biggs at XR navigation dot IO and it should be somewhere in the meeting notes perfect Erin would you like to give contact details to you before I wrap up with my answer sure uh easiest way to get in touch with me is uh am Gluck g-l-u-c-k at clemson.edu great I just didn't want to shout out Pete Works without giving everybody a chance um but I I absolutely think that Brandon and Aaron have have nailed it and I would just encourage people to think about digital curb Cuts so um making sure that you're not thinking about it just is oh I'm helping X people no it's better for the entire design so you need to consider what benefits there will be for everyone closed captioning was a great example so will your product have the next closed captioning if you design with people with disabilities at every stage um so that's just what I'd like to advocate for over and over again great thank you so much um to all three of you for an excellent discussion I feel energized every time I have these discussions and I have a lot of them but this has been really special and I really appreciate each of your unique perspectives um and again please get involved in the community XR access is another community at xraccess.org you can get involved in you can get involved in the XR Association one of the leading trade associations at xra.org
and you can also get involved in groups like equalentry.com who runs in um near monthly accessibility VR Meetup that people like Aaron have spoken at um in the past and they have an incredible library of recorded sessions as well that are accessible and then finally check out Pete that's p-e-a-t-w-o-r-k-s dot org subscribe to our newsletter listen to our podcasts and just stay informed and get involved uh we need everyone to make this as good as possible and it's really important for our future so I'd like to thank the site Tech Global team again for having us today and thank all of you for joining us [Music]
2022-12-12