Cathy Bodine- Cognitive Technologies: Designing for Accessibility with Edge Intelligence
Okay so my, name is Kathy boudin and I'm an associate, professor in bioengineering at. The University, of Colorado and, also. An. Associate, professor in pediatrics, physical medicine and Rehab and orthopedics. So basically. I serve at the nexus, between engineering. And medicine and it's a role that I absolutely, love, and. What I'm here to talk about today is. How. The. Human factors, involved, in designing, technologies. In this, new world of edge computing. You know we're in will explain what edge computing, is for those of you that maybe haven't heard, that yet. We. Only have 30 minutes so I'm gonna move quickly, so. The, first thing I want to make a point of saying is, that, designing. Technologies, for people with cognitive and other impairments is tricky, stuff you. Know it's not for the faint of heart and. It, requires a great deal of care and consideration, in order to do a really truly excellent, job so. We're, gonna start first of all what is that Caminos, what edge computing, is heard. This term yet ooh, got one over there what, is it. Excellent. You go to the head of the class exactly. Edge. Technology. And we just got with my, program, at CU, and Georgia, Tech and University of Delaware, just collaborated, on a National Science Foundation, grant which we just received do, one year planning, on, developing. An engineering research center on edge technology. So I'm pretty excited about it but what is it edge computing. Is basically. This notion if, you think about all the sensor technology, that's out there and all this cool stuff that's happening, the, one thing that all of these technologies. Have in common is, they're, capturing, massive, amounts of data they're manipulating, massive amounts of data and if. You have to send all of that, data in one chunk, to, the cloud what, happens, with your speed. Of processing. You, have lots and lots of data what happens. Slows. Down that's right edge computing. Essentially. Breaks up that data, and it, gets processed locally, in, smaller server, settings, right, and then, it can be uploaded to the cloud that's, where we get our real-time. Instantaneous. Feedback that's, we get everything we need so, if you're in disability, technology like, I am your brain is going nuts because you're thinking all this cool stuff we can now do with. This because it's gonna make this data make sense and I, do want to point out for the first time in the history of cognitive, technologies. In particular we. Now have lots and lots of data, but. We have to know what to do with that data right, in order to create technologies. That are going to be useful and, usable, by, people with, cognitive impairments. And so. It's a really cool new challenge, it enables, us to do things we couldn't do before and, so, for.
The People that are futurist, and folks that are really out there thinking, you, know the wheels are turning right now. What. It means is it's, better and it's faster, and when you think about IOT. Or Internet, of Things all these sensors all the things I'm talking about for example autonomous, vehicles, how many millions of data points are going on every, minute in, an, autonomous, vehicle and, if, we want to have it accessible, for people with cognitive, and, what, does that mean if maybe they don't understand, what they're hearing or maybe they don't know how to say where they want to go what, does that mean for, smart home technologies. Where, we have security, we have healthcare monitoring. We have all kinds of we just heard in our keynote about some really interesting. Home. Service. Provision, and home help that can be occurring, remotely, what about robotics, we're building social assistive robotics in my labs now to help kids birth to three with, severe, neurodevelopmental. Disabilities. How can we use this data faster. Better to, make all these technologies. Work and. More. Importantly, how. Do we have history, repeat itself how. Many of you knew that voice. Output on a computer, was developed not for mainstream. But, for people with disabilities, first did, anybody know that I knew Larry would know the two people yeah okay. It's true back, in the day Addis computers, for people who were blind who needed to hear what was on their computer, I, think. Was jaws originally, wasn't it I can't, remember the original first I think was Jaws jaws, was created, to help people who are blind use a computer, who, uses voice output on their computer, today. Everybody. Okay, same, thing with voice input voice, input was originally, designed for, people with high level spinal cord injury who needed to talk to their computer, who, uses voice input I. Used. It on my way here this morning right okay, that's, what, I mean by saying I want history to repeat itself I want, us to develop the technologies. Of the future that. Everybody's going to want and there's. Good reason for that happening, in terms of the cognitive space because. Of our aging, population right. By, 2035. One, in five Americans, will be over the age of 65, so. That, means we're gonna have those natural things that happen with our cognition, and other body parts as we age well, some of those functional, characteristics. Are going to be reminiscent of some of the functional characteristics, we deal with with other people with cognitive impairments. So it's interesting. But. But, but, there's a big but exclusionary. Design. Matters, and what is exclusionary. Design, exclusionary. Design means we, leave people, out of the pie there's a big chunk missing. That's. Not good. And, so. There's something my friend you - Trevor honest, and if you get a chance to hear her speak she's fantastic. Talks about and that is this notion of, outliers. And so if you're into research you know that when you aggregate all, your data the, idea is everything, kind of regresses, to the mean in other words we have this big old clump of data that kind of shows what's going on with our research but, every, once in a while there's a dot over here on the graph that's. Called an outlier, people. With cognitive disabilities I, believe, are treated as though they're outliers, in many situations. It's, not fair and it's not right, so. What. Do we do about that we want to make sure that, those outliers, are included, and you - talks beautifully. About, how. We can, if we bring those outliers, to the center, and we design, for people at that. Would be previously, known as outliers, as though they are the center then, this design is going to work for everyone does, that make sense. That's. What we want to do so, exclusionary, design, matters we need to call it out when we see it and we need, to know why it's, not working. And. The, thing we also have to bear in mind is that sometimes, it's, not intentional, often. It's not intentional it's not like some engineer sits in their little cubicle, and says by gosh I'm gonna make sure that these people can't use this they, don't think that way but.
Because, Of our own inherent. Biases. Because. Of our own ecosystem. How, we grew up how we're educated, we, don't necessarily think about, all the little bits and pieces that all of us in this room think about on a triple, x daily, basis, because, it's. Just not right. In front of them okay so. I'm, going to talk about three important, human factors, strategies. To avoid exclusionary. Design and talk to you about some of the things that we're doing and why I think it's important. And I have to give a shout-out to my friend Brian up in the upper left corner he has taught my clinic, team and myself more than almost, anyone, we've ever met and. Our years, of doing this so had to give him a shout out all, right so first. Thing that is critical is, user, experience and, usability testing, who knows what that is a good. Handful so, one. Of the things that's now becoming, more. Commonplace. Particularly. In mainstream industry. Is. The notion that we need to include the end user in the design process, we. Know that we've, been all about inclusion our whole lives right, well. One. Of the things I noticed last year at CES in Las Vegas, that massive, tech conference. That's absolutely crazy I. Noticed. Something that really gave me pause and that is, there, are tons and tons of, huge. Corporations. Developing. Technology. For, aging and they're. Doing it without any, knowledge, of, HIPAA, they're. Not doing it with any knowledge of privacy, concerns they. Just don't, know what they don't know and, so I was asked to go and evaluate a number of products while I was there which. I did and it. Was shocking so. On the one hand we, have all this mainstream deluge. Of technology. That's coming at us and. Then we have our groups here, that are trying so hard to. Make it work for other people so. It's. A combination and. But. I will say companies, like Microsoft, IBM, a. Number of these folks are beginning to really include end users in their testing, so what is usability, testing, quite, simply its taking, a product in determining, you know what are the functions of the product what are the tasks you have to do in order to operate the product, and then you create this task list you bring someone in and you have them go through the process of using the technology. The. Way they might use it at home. Usability. Testing, can. Also encompass. Not just the technology itself. But, it can also include the technical, manuals and the technical, support that comes with it so you can actually do a usability, study on the techni stuff, my, favorite was a pill dispenser, and I, swear, the manual, was in three point font. Design. For seniors and those with visual, impairments. Not to mention just normal people so, things like that are really critical, and user. Experience is something we've been doing for about ten years because, to. Re our C's ago one. Of this big grant that we have we realized, that all these products on the market even products. Manufactured, by assistive technology, companies, had, not been, tested. By. And for the very people they were designed, for, not to mention things, like, microwave. Ovens it turns, out that, people with disabilities. Use, the same darn things we do they, use alarm clocks they. Use microwave, ovens they use dishwashers. All of, these things were out there but there really was no data on how. Useful and how use a bowl are they for these folks so, we started, doing mainstream, products, in the first 5 years I think we did like 50 products that we tested, most interesting, had to be smart phones that was quite fun. Now. In this last five years we've been really looking a lot at apps you. Know and particularly. Social media apps Instagram. Facebook all. Of these types, of things Pinterest. I mean the types of things that we all take for granted and, we use is a very important, part of our social fabric right. But. How, useful and how usable are they those. Simple. Apps for, people with cognitive impairments, and we've been learning lots and lots of interesting stuff so, strategy one is to really understand.
Those Human factors, that are involved. With. People with cognitive impairments, in particular for, us what. Evolved. Out of these first few years, of usability. Testing. For. Us, was. A recognition. That, the. Interface, that, people use for these technologies, these apps these whatever just. Are really, difficult for people with cognitive impairments. And so. Strategy. 2 we came up with and. We are implementing, is this. Notion of really do deep, usability. User experience, testing what, does that mean. Well. Let's. See what my next slide is we want to know what specific very. Specific user. Features, impact, technology. Use right, and my. Hypothesis. Is that, when you look at your smartphone. With. All these buttons on it, or. Someone with a cognitive impairment how, easy is that to understand, if you get it right out of the box and then. You start getting apps added to it and then you start looking at these things so we had a lot of concern about how does that work, the other concern and I'll give you an example of one study we did had to do with seniors, voting, so, of course the federal government, decided a few years ago it would be very important, to ensure that everyone in a nursing home or assisted living center, got to vote, independently. Agreed. Right so they put some money out there to get these votes and they, had us do. A study we did this in three locations across the US and we, looked at tablet devices with a voting ballot on it. 85%, of our users made more than 5 errors, with, the user interface. Not. Good and do. You know why for. Seniors as they age their, skin gets drier and the. Capacitance. Decreases so. When they're using a touchscreen it's. Very difficult, for them to access, the touchscreen we, need a new touchscreen interface. Right so, that's a big one secondly. When we work with people with cognitive impairments, we see similar things with some of the different diagnoses, we see people who struggle, with capacitance. In and of itself so we wanted to look at these features and get really into it and so.
What We wanted to do what we did what we have is a simulator, that we have built that allows us to, reconfigure. All, these, features and parts that you see on a typical, touchscreen, and this is our simulation, one is very simple, and, the only thing the end user is told is that they need to press. The one that pressed the button that's different find. The one that's different and so, there's an active button and there are three inactive, buttons you'll notice there's equal spacing, equal-sized, they're, centered on the screen and, they're. Just pretty. Prominent, we wanted to take away everything that might interfere or distract. From what we were trying to look at okay. And. Here's. The kicker these, variables. These values, these things we're looking at are based, on industry recommendations. For how to build the user interface. For. The average Joe, okay. I would. Like to point out that these user interface, manuals, and these design manuals, that everyone uses. We're just put together by teams of engineers and various companies it's, not like they're based on science, but they're based on things, like pretty. Design, good design you know the things that they know and know well but. When it comes to our folks it can be a little bit different so in this level one, simulation. One we're looking at the, button, itself how, big it is how far apart it is what, the target asks that ratio is and that means what's active within the button and around the button so sometimes you touch the edge of the button okay. Just those little simple features and then. We. Are what, we're working our way through is, the, different ways we, can use touch. A button so, we all know how to do direct touch the button but, now they're swiping they're. Squeezing, there's, pinching, they're all these different things and so what we're able to do with our simulator, is we can log time. On tasks we can log error rate we can log pressure, what. Type of gestures, being used, and. What. Happens, when, the person is doing this. So. My. Other hypothesis. We, always talk about an N of one that we need to customize features. For each individual, person, the. Truth of the matter is. We. All kind of have psychological. Or neuropsychological, profiles. So we spent over a year developing. A neuropsychological, test. Battery with our neuro psych department some, incredibly, smart folks that, we are now administering, to every single subject who comes in to do user testing or this. Simulation. User interface, testing, so. My. Theory, is at the end of the day we're going find that, there is a bit of a cognitive profile, type which might encourage, us using, edge technology. To, have, the technology. Self. Populate. Iconic. Features or interface features that work are a good starting, place for that person with a cognitive impairment to. Get started does that make sense. Because. Right now when you just hand somebody out of the box a device or out of the box testing, has just been. Sad. When it comes to. Tablets. And touch and phones and things like that and it's how the whole world interacts, right so, basically what we're looking at is. These. Various variables and we're, measuring as our dependent variable time. On tasks and error, rates now, so, what we can do is basically machine, log all of this data on every, single button press there's, like 250, button presses, per simulation. And. We can then get some idea if there's an error not. What. They were doing and then. We. Can look at our cognitive, test, battery and we can do correlations, between, these. Errors, and their.
Particular, Cognitive profile. Our, goal is to eventually, take all of this massive data and. Bring. It back down. And. See if we can come up with just a few, question. Types if you will are a few activities, that someone might do when they sit down with a touchscreen that. Would then, encourage. This uploading. Of a user interface that is more useful for them so, that's kind of our goal with this particular project and one we're pretty excited. About to be quite honest to see what we're seeing, but. There's, also a third strategy and, this. Is something that I see is totally missing in mainstream large scale companies, something. And I'm working on is that, when in particular when we're working with people with cognitive impairments. It. Does take a village so, yes, they, must be included in, usability. And user experience testing, and in the design process it's. But. Also people. Who are care, providers. And clinicians. Make. A difference, so family care providers, people that get to know these people inside. Out day after day that, can really say no this is going to work no this isn't here, are the things that concern, me and secondarily. And perhaps most importantly, who, sets this stuff up. It. Sure isn't the engineer that designed it, nope, it comes to the, care providers, in some sense right and these, folks are left, struggling. With, what. Is theoretically, intuitive. If, it turns out not so much and then, especially, if we have two set features, and we have to do all these things for folks with cognitive impairments. It makes it really really difficult, the. Other thing that I'm really learning because, I do see companies now that are starting particularly 80 companies strange include care providers is, the. Need to include clinicians. As well, and. So as I said care providers, know the pain points, they know what may or may not go over well they, know what the if it's a community living environment, of some sort they know if there's small kids in the house that are going to disrupt, X they, know these things and they can be very critical to helping designers, think, about things they might not think about if they only talked to the end user but. Here's what I'm really really, starting to understand, as well many, care providers, and certainly. Clinicians. Know. The trajectory of. Cognitive. Life's. Impairment. Lifespans, so for example. There. Are obviously. There are acquired, cognitive. Impairments, there, are. Developmental. Disabilities. That were born with and, there. Are, degenerative. Diseases. And, things that happen to us we, know in Down syndrome, for example that, 20% will, have coexisting. Conditions comorbidities. With autism we. Know on about, that amount will also develop early onset Alzheimer's right. So, one of the things that particularly, families, who are just growing, into, living with someone with a disability they. Can't necessarily see. The trajectory of, what's. Going to happen across the lifespan and this is where I truly believe clinicians. Are critical, along with care providers, to fill. In the. Missing pieces, around. Technology. Design and development, because we know, what. Might happen we, know what happens, if we're older, our. Parents, are becoming older, and the person with the cognitive impairments, still living at home and still needs a lot of care we, know what happens with frail elders, and now at this point in our lives, frail. Elders, are being asked, to set, up and use technology. With. Their. Child, or someone. They care for and that makes it really hard and, there's a lot of us that have done this know how hard this is right, and so, we, have to be thoughtful so, we may have someone, who is setting up the technology or require, are embedded. Within the use of this technology in a very important way who maybe has a hearing loss who, maybe has low vision who maybe has general, natural, cognitive, decline themselves. Or arthritis. Right. And those, are features and functions, that we also need to. Bring. Together and really, think about as we, think. Through how we are going to address all of the human factors not just the individual, with a cognitive, impairment but, there's this whole ecosystem of, human, factors that we really need to be addressing we're, gonna have we, have is. Not gonna we have all this. Data how. Are we using it, how. Are we thinking, about ways. We can modulate our, design, in, a manner that is really going to be useful for lots and lots of people and. So. We. Know that edge computing, has this, immense, promise. But. We have to make it happen and we have to make it happen in a way that is meaningful and, useful and, we also have to take an account and I grew up in the whole 80 industry. You know typically, smaller companies, really. Immensely. Devoted. Developers. People, that owned 80 companies are not in it for the trust me unless you own the wheelchair companies perhaps but nonetheless you're, not in it necessarily, to get rich quick right and they.
Really Care about what they're doing so we have our, group if you will but, over here we have this immense. And I am not kidding trillions, of dollars immense. Industry. And commercial. Technology. Who are all seeing, what, they can do and so we have to get beyond what we know and we have to help get that. Translated. And transferred, into, mainstream. Industry because. If you're going to be independent what are the things you have to do what. Do you guys do every week why you have to do laundry you, have to prepare meals you have to do all these things they're, developing, this new technology, they're collecting, this sensor data the the classic, refrigerator, that, now knows what's in there I for one will never have one because I don't want them to know what's in my refrigerator just, say and they might nudge me a little bit but. The point is and think about all those menus, we've done by hand all these years and care provision, and how we get this whole production about going to the grocery store and all this well, the refrigerator, can now do in order to the store and it can be delivered. Right. Okay. But, how is that gonna work for, someone with a cognitive disability, that's. What we need to translate, and transfer, to these mainstream companies, I saw, one product and I will not name the company that, absolutely. Made me crazy because, it was a tablet device with. Internet, accessible. Video. Streaming, and they, were saying they were selling it to people my age who have elderly parents and the concept, is look you can hang this all over your parents, house and then you can just tap in and watch them all day long it's kind of like when you're pets at home alone well, now you can look at your parents home alone. No. Consideration. Whatsoever for privacy HIPPA, pick. One you. Know you can nag your mom to take her pills whatever. And. It's. Very inexpensive and, I. Can see millions of people buying this but. Here we are sitting over here in our safe world, trying to think about HIPAA and all these other things we, have this mainstream industry, that's just taking off and so. We need to be thoughtful about this, and we need to make it happen in a way that real we need more data we're just one shop, doing, this work we, need lots more people to really be doing this user experience, testing usability testing. And certainly. We need to dig deep, into these specific user interfaces. And the, cognitive, profile. So that we can begin to pull, together all of this data and it can be embedded, within edge. Computing. Design, because. What this can enable us to do then. Is, bring. That outlier, that you to talks about so brilliantly, to, the center designed. For that and then it's going to work so much better for other people now. One of the things I mentioned the ERC planning. Grant we had with Georgia Tech and University of Delaware we're going to be over the next year looking for partners, both industry, community and, research partners, to. Propose, new. Projects. In edge computing, so if anyone has an interest or just, curious, about it I'm happy to talk about it love to talk to you about it so, I was, told I had to keep this to 30 minutes I'm doing my best there's a few minutes left does anybody have any questions or comments because I'd love to hear from you yes sir.
It, Absolutely does and there's two issues there one, blue is the widest color prism so. That's why we chose blue to. Lots. Of people with cognitive impairments. It's really difficult to test their vision, so, in our neuro psych battery. We have a visual perception, test we, have a color perception test and we have some other things so that we can hopefully hopefully, dig, into that a little bit better it's a great question it's one we really struggled with which, is why we had our neuro psych guys help us figure it out. Right. Right. Yeah. Yeah and so what, we want to do is dissect it down to its barest minimum and, then we'll start building up if that makes sense so we expect this to take a while. Yeah. Yeah so it's important but I think it's important to, do this and, which have more people doing it frankly anything. Else comments, questions yes. Yes. One of the things we talk about in in bioengineering and, in our design classes, is inherent, bias, okay. Who typically, does design, work, does any have a guess on that. Engineers. Males. How. Old are they. 30s. 20s and 30s that's kind of mainstream engineering. Design they. Bring to this these kids, today and I say kids I apologize these young adults. Group. This is the ultimate first generation, that does not know what it's like not to have technology it. Is so intuitive to them that, they bring to it their inherent, biases. Is. Fruit number one so, they don't even think about what they don't know what they don't know because they don't know any differently right and they. Don't understand, the experiential, knowledge that. People in particular with cognitive disabilities have, which, is why I think it's so critical to have the care providers, and clinicians. Involved. In the design process. You're, exactly right, and I saw another yes sir. So. One of your main points throughout this this talk is talking about how you get, this data to, all, these, companies that are doing this stuff and I wonder if you can talk a little bit about, how. You want to make that communication because, you're going to be pulling, in lots and lots of data if you have other people building, these things how, do you get Apple or. You. Know Google or any of the smaller companies that are building these apps how. Are you communicating, this, information to them a couple of ways number, one I am blessed, you know when you're around long enough you get to know people I'm. Blessed to know most of the leaders, of the accessibility, initiatives. And the mainstream companies and the big companies. I'm. Also getting, the. Consumer. Technology, Association, put CES on every year and, so, they fly. Me out in this year I'll be addressing the board the. Day before CES, to. Really get an opportunity to talk about disability, and inclusion, so that's on kind of a grand. Scale if you will on the, lower scale what, we are doing, we're actually working on a report format now we're using it's an ISO standardized. Usability, reporting, format that, we're creating reports. On the technology. And so we're debating we're. Working with both marketing. People within a company and we're, trying then also to get them to get us to Arthur lead R&D, designers, for the products we're testing. But. They're how many products, out there right. So, we have to approach a top-down, bottom-up is, kind, of how we're doing it we have to get our industry, the the. 80. Industry, if you will or those of us that care about technology. And disability, also engaged because. It's this mammoth. Rock straight uphill right, but. We're, doing the other thing we're doing and I got, I we. Were blessed there's this guy named Bill. Richardson who. Was he. Was one of the original guys at Sun Microsystems and, he's retired here in Colorado, and bill. Spent, a Thursday, a week for a year with, my team coaching. Us on how. To change, our language and how, to talk. Money, and. Return, on investment, and so. When we approach any of these companies, we don't talk about these people with disabilities. And the need for inclusion we. Talk about return on investment, and we, talk about market share and we, talk about how much more money they can make by. Doing this and how their stakeholders, and shareholders. Are. Going to benefit from this we talk about the cost of retrofitting if your device doesn't work then what, it costs to retrofit, so, you, should hear me it's pretty funny when I get into that mode it's not here, it's, not what how I talk here but but, thank, God for Bill Richardson he, taught us a lot and that has given us some bandwidth, that I don't think we would have had otherwise, so.
Another Hand yes. Sir yes. So. This is a basic question but what you're envisioning in the future is when you buy your iPad or your phone you, pull it out of the box and instead of having to maybe calibrate, the screen you're, going to possibly go through some kind of a, assessment. On there. To determine size, of icons, and all of those things that that's. Our goal okay, yeah. Because it's personalized, you know we did that single sign sign-on profile, thing for a while where you could set your your. Configurations. And then you could theoretically it, would be available to you to download it any whether, you were an ATM or a tablet, or whatever and I'm probably, not saying this all very clearly. And. That's. Difficult. To. Make accessible, for everyone and for everybody to work together on that so yeah that's the concept that you would maybe, have four or five things you do and then your. Device is set with, a baseline, you might still need to tweak for sure thank, you. Thing. Else. You. Guys were awesome thank, you for coming today I'm gonna let you out of here.
2019-01-05 13:22
