Towards Mainstream Brain-Computer Interfaces (BCIs)
You. Okay. Hello everyone thank you for thank, you for coming to this talk. You know series, of BCI related, talks here at MSR it's, my great pleasure to introduce dr. Brandon, Allison, from UC San Diego he's, been working on BCI's for, two, decades and. He's. Gonna talk about mainstream, BCI's, today so without further ado Brandon. Yes. Okay. Well thank you for, the intro and for coming and thanks for inviting me yeah. My name is Brendon Allison, I'm a visiting scholar with, the cognitive science department, at, UC San Diego and also. On a volunteer basis, I'm an officer, with the brain computer, interface Society, which I'll talk, about a bit later however. For. The first part of my disclaimer this talk just, represents, my perspective, I'm not representing, any other person, or entity and. I'm. Not being paid by any one MSR as kindly reimbursing, my travel costs and meals but no. One's paying me in fact. The only entities that have paid me are these two so I'm a visiting scholar with UCSD, I'm loyal. To them but I don't receive any salary from them. So. One. Thing I'd like to do to start and this is a risky, way to start because it involves audience participation, but, we'll see is just. To sort of establish a baseline and especially since this is being recorded, I thought. It would be fun to just run a few questions by the audience, and see. How. You react. And what people think about BC. Is today and, so. It's. Audience participation, time and. The. First one is I just like to ask you to raise your hand if the answer is yes the. First question, is can you name some. BCI's. Alright. So for, those of you at home most people are responding. Yes and then. Do you agree with this statement modern, BC is can be useful, for patients, with ALS or. Other serious, neurological. Conditions please raise your hand so, yeah most people are raising their hand, and. Then. Raise, your hand if you agree with this modern, BCI's can, be useful to healthy people in everyday life. Yeah, so nobody now we have zero and. Then. The last one, can. You raise your hand if you used a BCI in the last month, alright, so we got three. I think. Sorta, three all. Right so this is Speaker. Out on a limb I obviously have not had time to edit this slide I do not have such a fast BCI. So. My, guess is the audience would say yes I can name some BCI's, yes. Modern, BCI's could be useful for some patients but. I was also guessing most. People would say no I don't think modern BCI's, are useful to. Most mainstream people, in everyday life and I. Thought most people would say no I haven't, used a BCI in the last month to. Provide a little additional, detail I've asked the same or similar questions. At recent, brain computer, interface conferences, and at, a talk at Google and, other. Places where you might think these would be early, adopters of BCI's, if there were people, who really knew about computing. Interfaces. Working. With software these. Would be the ones and yet even. Here at Microsoft like. These other entities this. Is a nice baseline right now as of, February, 2020. Even. People who are knowledgeable, who are sort of insiders, in the field or at least would be well connected to them aren't. Using BCI's and and don't see them as being useful except. For patients and. This. Has been the case for quite a while most people have believed this and, I. Generally. Agree with it so, that's. Another thing to help set the tone for this talk my my goal in this talk is not at all to, promote. Mainstream. BCI's as something imminent in fact I'll publicly, say now I don't think they're imminent I don't think these, answers, are likely to change for, most people in the near future, but. When will they change right, so let's. Say I were to give this talk here in 2030. What. Do you think people would respond. 2040. I I hope, I'm still speaking by then and so on when you. Know and I would, guess if I said something way. Out there if I said in the year 3000. You think modern DC. Is as of that time might be useful yeah. I think. Most people here especially, given. Its Microsoft. Recognize. That, keyboards. And mice and other interfaces, are more temporary. And transitory, than they seem right, now we're used to them that's the main way to interact with systems, but, may not be, that way forever, an. Old. Friend of mine worked for Microsoft and. He. Would say he'd be giving a talk and some people in the audience would be critical. And, make. Trouble for him and so he would say look. I understand, that the reason you're doing this is not because of anything related to my talk but just because I'm with Microsoft, and you, want to complain and so that's part of the job that's fine and so. Similarly I'm, not at all the. Avatar. Of BC, is for everyone I'm very interested in it and I always have been but just.
As With my talk before I, very. Much aware there are practical, issues and one of the main ones is of course why, not use something else some. Other way to communicate and that's, on, a later slide. So. This. Raises the question of why why would I use a BCI, unless I basically. Have no other choice and. This. Is the classical, view so for a few slides I'm going to review what. Seems to be the classical, established, view that I get not only from papers but talking with people in conferences, and this. Is the classical, view which. Was consistent. With the little survey I did that is most people here say, yes BCI, is for users with disabilities. Intentionally. These are all older, images this is from 1999, this. Is around, 2000. From Graz this is at, least 10 years old but these are examples, of publications, that are well-established showing. Patients. With disabilities, and for ethical reasons I should say of course these, people have had. Their allowed their pictures to be shown and. Therefore. The classical, view is that widespread. BCI. Adoption, requires. Replacing. Conventional. Interfaces for, conventional, users and conventional, settings that, is a lot of people might look at that slide and say okay there's, a certain setting where BCI's. Are useful it's a patient that. You've got no other choice but, to make. BCI's, useful, for other people, you would need something, where people are doing a common, mainstream, thing, like. For example playing, a computer game and. The. BCI has to replace other, interfaces, and I think, that's a mistake in other words part, of what, I'm getting at in my talk is that although I still don't think mainstream, VC is or imminent I see very consistent. Mistakes being made in, papers. In talks and talking. With conferences, and in. Other people. Business. People or students. Or whoever who. Are trying to think of a mainstream. DCI and they're like well you've, got to come up with this or it's, never going to happen, instead. The, emerging, view is, first. You can replace or supplement a conventional. Interface this is a critical, point in fact I'll go further the, first mainstream BCI, will be something used, in combination. With another interface, or another task so, the idea that you. Would have a BCI, so a key point of this is notice this person has hands. On the table even though there's. Another way to communicate and, and this is not how it's going to happen you're not going to see someone. Saying, wow I need no other way to do anything the BCI is everything. It will be a BCI, combined, with other. Interfaces. Or ways to interact this, is what's called a hybrid BCI. And. Also, for specific, settings and so, here. Again you have an example of a user at home. Free. To use keyboards, no problem, and what if there are certain settings, or certain environments, in which the, cost. Of using a keyboard or another interface is is higher. Than it normally is and this. Is also something that's come up before and was in my 2006, talk, where, people. Have looked at cases such. As, drivers. Where, the. Driver's hands are busy you you, can't. Really use a keyboard, or mouse or texting. On your cell phone or something quite as easily. Mechanics. Or airline mechanics, came up your head is inside, you. Can't speak it's too loud, various. Military, applications, have come up that I haven't worked on soldiers, you want to be silent and you don't want to move how. Do you send information and. Again I'm not saying this is imminent to my knowledge soldiers. Aren't using this either and. Another. Issue with. With. Disability. Is this term that I've learned from the HCI literature, of situational. Disability, that, is in certain situations. A healthy. Person may, be somewhat, like a disabled, user again. If you're driving or for some reason your hands are busy, in. A sense you're like a person, who doesn't have hands available and assistive, technology designed for someone who. Doesn't have the ability to use hands might seem. More practical, for you in that specific setting. And. An. Example. I like to give with, with. This is laziness. And so once. I I, gave a talk, at Philips, headquarters, in Eindhoven. We were working with them on a grant and they you. Know they of course very well known as as.
Electronics. Manufacturers. And so I asked, do. You make any televisions. Anymore without remote, controls, and they said no of, course not you know why would we, the. Remote control is a great example of sort of disability by. Laziness and I'm used, them too but you're, watching TV. You. Could get up you, could get up and change the channel you, could push a button you, could turn that old dial there are lots of ways that, you could do it that you do not have a medical, need for that remote control, yet. I'm, not gonna ask but, if I were to say show of hands and say yeah most people have used them have, any of you actually gone up to your TV and played with it probably not, but. Again I won't I want, to ask that publicly, and if. You think about that that's actually, a tremendous imposition. On your TV viewing experience so, you're, watching TV you wish. To change the channel, you have to, disengage. A little bit from the TV you have to dig around in the couch to find the remote control you, have to look at it you have to orient it the right way you have to push a button. All of this is disrupting, what you're doing otherwise is there, an easier way to do that right now with the BCI no but. This is just an example of an environment, in which people would, not normally think of a TV viewer of someone who, is disabled, or might need an assistive technology but. That remote control example. Is, is. Getting in that direction again. Not quite the same, and. Another. Conventional. View that I want to get, at is that widespread. BCI. Adoption, requires. Dramatically. New capabilities. In other words we need a BCI that. Can do something way beyond what any existing, system can do because, right now they. Can't really do anything useful and this. Is necessary to get around the problem that, wired and ugly systems are unappealing, in other words nobody's. Going to use a, system. With a cap and gel and all that well, of course. That's. All changing, so in particular, practical. Electrodes, by which I mean either dry electrodes, or. Various. Electrodes. That rely, on water instead, of gel or this. Company advanced, brain monitoring they. Have a an, electrode. That releases, gel as needed, over time all. Of these things make BCI is much more practical especially. For people who don't want to put gel in their hair which is basically, everyone except me and, I. Think, there's tremendous fun, so. You have these dry ones and then wireless this is a big deal of course if you've got to get up get up and walk around and their cables that's.
Not Going to work and so. Concordantly. You have a change. In cosmesis, that is cosmetic, appeal and I'm arguing this is of course. A factor for mainstream BCI, adoption, that many people either. Don't think about or like I said they think its, secondary. To capability. In other words first you got to make the ultimate BCI and then, nobody cares about this and that's just not not correct so. These are a few examples of different efforts, to make be CIS that would be more cosmetically. Appealing to, users and in. The next few slides I'm not just getting at. Objective. Cosmesis. Because there's no such thing it's a judgement call if the DCI looks, appealing. To you but, also perceived, cosmesis, what do people think about it so, here one. Way you can do this is you can hire a model, it sounds trivial but if you look a lot of pictures of BCI's, they. Have. Some. Grad student in engineering who's, Shaggy and doesn't, have any. Makeup on. Another. Interesting, case is over here this was a specially, made device. Called the unicorn, and this. Woman, is a fashion. Designer and she. Was trying to get at the problem of persons, with high-functioning autism, so many children with. Autism are, not comfortable with the EEG caps you put on a cap that looks like this and they, don't like the feeling of it pressing, all around their head and so. She designed, this for children with autism it's, called the unicorn and you, can see it most, of the head is exposed, it's, not like you're covered up, right there and it, has this little unicorn thing, on it which is not functional, but, apparently. This was much more popular. With such people if you want to do research with this population you need a hat that they will will wear. Regarding. The, perception, of whether something is cosmetically. Cool these are all examples in the last few. Years or so this is a bit older of. Movies. Or other things in which you have. Electrode. Caps that aren't presented. As sort of, weird. Future, things but sort, of mainstream so, there's, a talk show host Colbert, and there's President, Obama talking to a guy wearing electrode, cap, this. Is from, x-men you. Guys recognize this famous author, science. Fiction hero this. Is Andy weir who wrote the Martian and. When. We, when. He was nominated for an Oscar we, made him these, little 3d printed Oscar statuettes, and put a but he, nicely, got that picture the, point is that this, cap looks the same as other caps this cap here that he was wearing it appears. To be the. Same type of cap that I remember using in the 90s but, the fact that you put it on someone, who's popular and is and, whatnot effects perceived. Cosmesis. Yeah. Right. Yeah. He, he, knows Lord of the Rings better than all nerds. Speaking. Of nerds and I say this in a nice way one of my friends in the field is grooveshark and another. Issue in the common view that i mean to challenge, is that speed, is the, critical factor for broader adoption so you get this a lot and various papers like this one and, the. Idea is well DC eyes are very slow they have to get faster, and, I. Just sort, of adopted, this part here in, other words this this text is mine and not from the image where. I was looking at this in terms of the hype cycle in other words right now, you. Know BC is might be sort, of severely, disabled, and you might see them with people with less severe disabilities, for example there, are cases now where people who are able to use another interface choose. A BCI, for, example, there are cases of patients choosing a BCI over an eye tracker because, the eye tracker is fatiguing, if you. Have. Very limited muscle, control than doing. This looking, around left and right it is exhausting, and so. I was saying then you get eager users and finally. The mainstream healthy users this is just an adaptation of, the classic, hype, cycle, but. In. Response to that 2008, paper I. This. Framework, here, where. I'm trying to get far, beyond, speed, in terms of looking at what matters. In terms of, any user including a mainstream user in terms of BCI adoption, and you see speed, is really. Only a part of it that's only one thing. One. Example, that keeps coming up from, people, who work with patients is that, patients. Will often not choose, to maximize, information, throughput, but instead to maximize, accuracy. Right. This is very well-established so. You. Probably know that you. There's you, you, can alter. Your systems, so that accuracy, goes down and speed goes up or vice-versa and you, can work out the the. Way to maximize bitrate or infamy and transfer, rate and yet, many patients, say even, though it will slow the system down I'm very frustrated by errors and. So. That already starts. To to challenge the, importance, of speed you, know so accuracy, is also there another big one is the size of your alphabet which is a tremendously, ignored issue so if, you can choose one out of ten items versus, one out of hundreds, that's.
A Really big deal and I'm. Happy to talk about this indefinitely. But one. Issue that I mentioned already that I want to get, at I made. Up this term called distraction, quotient, and, what. I define, that as is a. Representation. Of, how much using. The BCI distracts. You from whatever, else you're, doing now. It. Might be that your only goal. Is to use the BCI you you just want to use the BCI because, you're playing a BCI, base game or, something, you have no interest in anything else but generally. Using. The BCI is quite distracting, if you haven't used one a lot. Of them you. Have to count flashes, you have to pay attention to something you have to imagine movement, this makes it much harder for you to type, or speak or do, other things, and. There's. A, type. Of BCI. That's called passive, BCI in which at least theoretically, if, it works the distraction, quotient, is zero meaning, it. Doesn't require anything, for you to get information from the BCI you don't have to count flashes, you can do whatever else you were doing and it. Might give you information such, as alertness. Or, do you think you made an error I'll get into examples, of this and I, think this is also still. Very much ignored people, think about the BCI but not the BCI as part of an integrated. Interactive. Platform, in which, the, goal of the user is to send information, through. Whatever means are most effective, at that point in that situation. Might be brain activity, might be eye activity. A keyboard, or whatever like. The one I mentioned the this, was a paper by Robert Lieb in which he provided. A B C I think it was 2006, but I'm not sure he, provided, a BCI, and an eye tracker to pee people and many. Of the patients said we, we prefer to use the BCI it's, it's just easier and. Another. Interesting, issue is, that the, types of BCI is haven't. Really changed much so. In. 2006. My. Talk was mainly focused on BCI's, themselves, the different types of BCI is rather than mainstream. Applications, and I. Said these were the main types, of BCI's, looking. At it today, hasn't. Really changed so, slow cortical, potential, BCI's are no longer that prevalent but, otherwise. The. These are the same and this is another. Fundamental. Challenge, that has yet to be overcome, is there's something else you can do, mentally, is there's something else you can think something. Else you can pick up with the EEG or other methods that, is useful and. Although. This. Hasn't, developed. A lot I'm not at all saying there's no hope for it in the future as. I was, discussing improved machine learning is. Creating. All kinds of options and. Concordantly. You have an ongoing. Increase. In EEG research, what what, can people do that might influence. The, EEG and how, does it influence the eg you know talk more about this a bit later that's, a link to my talk from before but it's. Mainly for people at home because I think you guys can find it at Microsoft, and, so. I would also say yeah ease is a critical, thing ease and, utility, that is what is the BCI. Doing. For you this, is also a big, thing you okay great you find out that someone just, made a mistake or is looking over there and thinking about something, who. Cares and so these are a few examples, of trying to integrate them with devices, like cell phones or, hats or, whatever which, so far have met with pretty limited, success.
But. I don't. Think that will be forever. So. Inevitably when you're thinking about this question of BC is that, this is one of the first things to ask and this happens all. The time, I'll, talk, to someone and the person will have a great. Idea for, a BCI and I'd say that's great but why. Not use an eye tracker and they say you've destroyed, my idea. So. One, it could, potentially be faster, than, an unavailable, interface so is a BCI, faster, than a keyboard no but if you have no keyboard if. It's not practical. Then. It might be faster now for. A BCI to be more practical you've got to have something on your head there, are various, environments, where people already tend, to have things on their heads we've talked about that, unfortunately. My experience, is that people who do that don't want anything more on their head notably pilots, so my. Grandfather was a pilot he he flew marine one on presidents, Kennedy and Johnson so. He was a senior, officer, and, he basically said that pilots, will not want something, on their head reading their brainwaves no. Matter what and I said don't, you have the authority to order other people lieutenant colonel and he's mmm, I'm, retired anyway, if it's impractical can't. Easily be used or would, take longer to provide the same information so, we chatted, about this over lunch with. Usability. Testing, and another example, as neuromarketing. You, could have someone view, an advertisement. Or use. An interface and then afterward, you can say hey what did you think of that were. There any things in the ad that got your attention when. You are using that interface was there any period, where you were a little confused and disoriented, well, you could get that information or at least some of it but, you. You now have an objective, way you can have someone using something you can say without, interfering. In any way well when, that person used this particular software, or this particular environment, we, saw an increase in these. Sorts. Of problems such as feeling. Disengaged, dozing. Off stuff like that and. You're. Probably aware neuromarketing, has got a lot of attention there's a company. Called neuro focus, that was sold to Nielsen marketing and I don't know what's. Happened with them since then but that, was their focus and I know that two of the people involved, Bob Knight and Jon Polish or, top. Electrophysiologist. Jon Polish was my boss Yunos, p300. Literally, I think better than anyone in the world and. That's. Another example you, want to know in real time what, someone thinks you, can stop them I have been in focus groups and they'll stop you in the middle of an ad and say what do you think of this and this and that and that's disrupting. The whole process it's wasting time you. Have the 600-pound, gorilla problem, there's one guy in the room who says oh that's wonderful and nobody else wants to talk, none. Of these problems exist at, least theoretically if, you're getting information passively. Some. Way through, BCI eye tracking, or other means. Another. Big answer. To why not use something else is that at least in some cases it, might be easier to use than other interfaces, and again I'm not saying this is the case right now but, this is what things, are developing, toward it, might be more practical, in a real world setting TV. Remote control. Another. One is that could potentially be more intuitive, and this is, something. That really is changing in the field and is a dramatic. Development is the. Prospect. Of of auditory. Reconstruction. Which has been getting a lot of attention primarily. With, invasive BCI, but, the. Idea that from electrodes. You can determine what. Someone is thinking about saying what words someone, is thinking or even which sounds. Or music someone. Is thinking about, if. That develops, especially. If it develops non-invasively, then. You have something. BCI's have never been able to do before which is a direct literal, BCI instead. Of spelling, by accounting, this flash or thinking, about this and then thinking about this you, just think of the word this. Keyboard, which everyone, is used to is also a highly, unnatural. Unintuitive, interface, it's just that we're used to it if I, want to convey Apple.
What. Is me pushing these buttons have to do in anyway with Apple Nisour at any anything related, to fruit I it's, it's an arbitrary mapping, that, happens to be the best way to send. Information to a computer that we, have that. Most people think but it's certainly not the best ultimate. Interface and. Privacy. I talked about this a little bit that you. Might be able to. Determine, if, someone. Wants, to pay, attention to a certain region or whatever, that's. Private, in other words most other ways of sending information someone, can watch you they can listen to you talk they can record, it this, way at, least in theory the process, of you sending information nobody, knows. Exactly what you're sending and. One. Example of why this might be useful as some security, settings and then on the other scale. Of sort of fun as is gaming, you're playing a game you don't want the other person to know what. You're doing again, I think this is somewhat further in the future but. It's it's, there and this, one is not in the future so this this has been the answer most. Of the time so. Just. As I've said well you talk about BC is and, you can always get people to say oh well they're useful for patience you, can also always get this out of people you say well what. If people want to use BCI, despite. Everything else yes it's inefficient I don't like it I don't want this stuff on my head I just want to use it for fun you, nobody, has any answer to that except, well there aren't that many people who do so but. That's. Out there I mean there is something cool about putting a thing on your head and communicating, even, though I'm obviously kind. Of biased. So. Now we'll talk about some applications for the public and. Intentionally. A lot of these slides differ, in time. That is some of these are things that have been pursued for decades like alertness monitoring, and then, others are relatively, new. So. This, is one that's been around for a while from Paul Tschida and his colleagues, of image triaging, and. What. They could do is if they presented, images to people very quickly they, could tell from the EEG, which. Images, stood out for example you can show them pictures of trucks and one. Of them is longer, you would have a slightly different. Reaction. To the. Longer truck and so. Although I don't know image triaging, the idea was to reduce the time so instead of someone looking, at images all day and you say oh that one's important, and that. One's important, if you, can present something ten times a second which is of course much faster than you can type that, would be one example not. So much of a mainstream, user but at least of a healthy user where, this is useful as far. As other possible. Applications, for mainstream users, I don't know, yeah. By the way if you're coming, expecting. Me to tell you the killer app for DCIS I mean you should have known better, so, yes I'm not presenting the killer app I'm just talking, about the process, when, someone, does come up with the killer app for DCIS I'm gonna be really really mad that I didn't think of it but, here we are and I.
Thought. Again. I don't, know immediately but studying. So, you have, a list of a hundred words that you want to remember if. You had an EEG system, on that's locked to each of these you could probably determine, from the p300 and other things which, images, you are likely to later remember, this is research from Manny Don Chan dating back a long time you. Need to remember 100 words right. Now you, read the words you test yourself you go back and iteratively now. You get rid of the testing yourself words just keep coming up you, never have to test yourself as. Words. Come up you start noticing boy. Some. Of those initial, words are gone what's. Happening, that you don't know about is it's doing something similar to this that is words, that you are likely to later remember, you. Just. Don't see again again, this assumes the system is accurate in a, few words that you're not likely to remember it just presents them again or or other information, and. Another. Very well much. Better established. Extension. Of, BCI. Is, neurofeedback. Which, has also been around a long time and. Done. Properly, neurofeedback, can, be helpful for some people but. Part, of the problem is that this has become heavily, saturated, so many people and so many companies for. Decades now have been talking about it this is these are just a few examples I haven't, used any of these so, I don't mean to say that they're good or bad these are just examples and. If. You're a an, end user how do you know you're you're. Not an AG expert you see some ads and, there. Are a lot of examples of sort of unethical neurofeedback, I have a little, that later. This. Is one that most people don't know about but, it turns out the p300 and other signals, can. Be used for detecting lies. Or, when I say a lie it's in quotes what. It's really detecting, is whether you perceive, an image, as familiar, or unfamiliar, and so. Larry Farwell, and his team have a company, called brainwave science, where they've tried to use this for forensic, purposes for. Determining. Whether people, actually are, guilty or innocent and a. Lot of people don't realize this, was approved in a court and I of in Iowa over 10 years ago so there. Was a court and they had a trial that addressed, the question of whether brain wave fingerprinting.
Should Be admissible in court the, court ruled yes and so. Unless. Something has changed if, you're get arrested in Iowa you can, use brain wave fingerprinting. As part of your defense I, would. Recommend. Not getting arrested in any state that you know. Neuromarketing. So, I've mentioned this how, to focus groups who react to sounds or images so. First it's not literal it's not like party or brain lights, up and says coca-cola and. This. Is an example that I used. So. This. Is harley davidson cologne and i. Would. Not ask you to raise your hand if you're wearing harley davidson cologne because i'm pretty sure you're not and they don't make it anymore but. I was talking about this with some ad person and this, was apparently, a famous example of a focus group failure, Harley. Davidson, made Cologne and they. Got, focus, groups and they said here. You know try it and it, was a classic 600-pound. Gorilla problem, you've got a few big guys in the room who are bikers and they say yeah Harley, Davidson cologne and it, turned out in fact many people in the focus group were not that positive about it but didn't want to say anything okay, whatever and so. In, theory I don't pretty sure they didn't do neuromarketing, this, might be an example where you would have got a different, result instead, of people. Being afraid to share their opinions, they just, their. Opinions, come out as they see Harley Davidson cologne not, just EEG, alone I mean maybe their mouth turns down or right eye activity. But, a that. Might have prevented, Davidson, from making. Cologne which I assume was bad I don't know. So. Yeah. Another. Confirmation. Technically. Face. Expression. Processing. And, getting. Emotions. Or. Anything, else from the face is way more advanced it and that's. One of the texts which is that niqab use it today instead. Of be shy which is still down. The roles right, yeah, I would say neuromarketing. It I would. Think it would be most effective, using, a, headset. That's sent picking up a lot more than a eg if, I were involved the first thing I would say is hybrid BC III I would. Like to go ahead and put an electrode cap on them if you want but one, of the first things I'm gonna look at is eye activity, which you can get from the EEG facial, muscle contraction. So. Yeah I would be the I think this is a good example of a case where a, EEG, might, be useful as a supplement, to other things but. I'm not saying you just put, a cap on them and get nothing else. But. Yeah you don't have any delays you don't have to wait there's. No distraction, you don't have to stop them you don't have follow-up, question airs you, don't have. Pleasing. The experimenter, so, another example as someone made peach Christmas, lights and, nobody. Wanted to say you're. Such a nice guy but this is terrible you, know. Not. Noticing, forgetting, the, 600-pound, gorilla wow that guy likes Harley perfumes so I'm not gonna argue. Would. You argue with that gorilla, and.
I Talked a little at lunch over error detection, so. There's the signal called the arrow related, negativity, and this. Is an example of a study, from awhile ago 20 years ago and with. Four persons, it was showing that three of the people had, some, overall, improvement, in their information transfer, rate with a BCI, using. This and so, I'd be quite interested in studies, where they're looking at error detection, while someone is not using a BCI but using other things I'll get to this soon with an example from Scott. Mikeg and. So Jung and colleagues and. You. Know the main problem with this is it's. Difficult to get on a single trial basis, and so. I've thought a lot and I've talked with the guy here named Disney tan and the professor, named Anton my holt and others about, if. You're. Not. So concerned with the real-time constraint, then. This, is much more viable and so I mentioned, the example of usability assessment. Not, just for software but other types of settings and then. This is something I actually talked with Eric Orbitz who's still, here on my, way to let my last talk with. This example, of using Microsoft, Word so you're using Microsoft. Word which why. I'm at Microsoft, and you're. Typing, and the. Computer, recognizes. That, you've typed a word that the computer doesn't recognize, what. Happens that, word becomes underlined, in red and now, the user has to stop. Typing and, do. One of two things either, just. Engage from typing and probably use the mouse to right-click on that and correct the error in other, words you're telling the system that you believe an error was made you're telling the system you're right I did make a mistake option. Two is the, opposite, the system is wrong you didn't make a mistake you want it to not underline, that or add, it to dictionary automatically, or in. Some, other way not distract, you from typing I'm in the middle of an important sentence so I got something to say and so. This, is something I was talking about with him is it's not just detection. Or error. But also the absence, of error in other words if you can tell someone thought no error was made that might be useful and I'm also saying this is a largely, underappreciated. Direction. Again. If not not just using, it as an, error correction, system for BCI's, but, for other types of interactions, and. This, is what I was just talking about with Scott McCaig and C Peng Jong and colleagues and. This. Was a study based on only, two EEG, channels, in the late 90s a little, over 20. Years ago and. Red.
Shows, Targets, that were missed by, professional. Sonar operators, so these were people who were naval. Personnel who. Had, a position, of sonar operators, so they they. Worked on submarines, they had experience, and. These. This. Red, thing here is showing. Sort. Of lapses, in in their attention, that the system predicted, and then green is. The. Their. Their hit rate and so you see here with red and blue, what. This is showing is, that blue, is the actual, error rate and red as the predicted, error rate so. There are two, parts of this one of them is was, not so. New which is if people, are in a state where they're more likely to make errors we can pick that up from the EEG with, only two two channels which is pretty good but. The second part which I thought was much more interesting, is they, could predict these alertness, lapses. By. A few seconds, with the EEG that is, they. Could there was a certain time period when there was no other way to know that the person. Was more likely. To make errors except, the EEG or at least no way that we know of if. You could have found something, then that could be real useful you. Call. In a replacement supervisor. Or, you play, the voice of the crew chief yelling, at the guy or, whatever. They do on a sonar, actually ask the sonar people, they say yeah the chief will you let you and. In. The, case of alertness monitoring, here it's even more than a car, I mean a submarine. Hits a mine that's very bad yes. Rosie, one, of them was Ozzie and one of them was fcz, so, they're both midline channels. And. These, two people, so Scott mikeg is still, at UCSD along, with his colleague si ping Jung and. You. Know so this this is going beyond real-time, alertness monitoring, this is almost. Proactive. Alertness monitoring. Usually. You said that I mean it looks to me like the blue is leading the red. So. The blue the actual error rate the red seems to be lagging a little bit so. I mean but I thought what you just said is that no they predicted it ahead of time so I. I'll, check, I might have reversed that but I remember the key point is that they, were able to predict periods. Of increased error likelihood. Ahead. Of time. All. Right so then the question is great. You're. Aware that the user might, be more likely to, make mistakes or, for, some reason you're getting something from the eg well so what what, do you do with it and this. Is an example, from. A group and NASA there's a guy named Alan Pope who retired, a while ago but. He. Had, some. Work on, adaptation. Based on EEG, measures of task engagement, so the idea was that if a. Pilot. Or astronaut, or another person. Was. Overloaded, in other words the EEG indicated. This person, was, already getting too much information, you would reduce the incoming information switch. It to an automated system get. A backup person, whatever and also. Vice versa if the person was bored maybe this is a time to call in increased. Engagement and. Another. Example is, this, Ewing at all paper where, he was, getting, it the same idea for game adaptation. So, if someone is playing a game just. Like doing a neurofeedback task, or a lot of tasks, there's. A risk if the person could be getting really, bored or could be getting overloaded just like the pilot example, and. This. Paper was getting at how you could modify the game difficulty, level, and the incoming information accordingly, and this, is also the sort of thing that I mentioned as neat an here along, with Anthony Holt, and colleagues have been interested, in for, a while that this this adaptive, interface, where. Ideally. From the user's perspective without, any extra work you don't have to say I'm overloaded, or something you just wear this thing on your head from, the users perspective the. Games always really engaging you're, you're, doing really well you're, not perfect you have to work a little bit harder or, else you get bored you're certainly not overloaded. That. Could be very appealing not just for games for fun but training and, other applications. So. He, was presenting, the idea this bio cybernetic. Loop where, the EEG would, effect the difficulty, of a Tetris game but. You get the general idea.
I, Have. Several slides that'll, be quick on gaming and fun so there's not a lot of text these are just examples and. These. Are all systems, that. May. Not necessarily be, BCI, so, they claim that they're working on a eg I have. Some things to say about that but not right now and. This. Is for. Example called, the mind flex so, this was a game and. You're. Supposed to use. The BCI. To manipulate. This ball of the game this, is another one this is called the Star Wars Force, trainer you. Can get it on Amazon for about 50 bucks right, now and the. Star Wars Force, trainer, what. You do is you use the Force to, move this ball up and down to control the height of a ball how. It works as many, people have seen the trick where you if you have a ball and a fan under it the ball appears to be suspended, and so. What you're actually doing is you're using the EEG to control, the strength of that fan so. With the EEG you can make the ball go up and down is that. The same as using the force I don't, know. This. This may be it so it was it was it. Was a long time ago in a galaxy far, far away I mean who knows what they used and. This. Is a company, called it emotive you, may have heard of them where they they have this game and notice, also they're working this headset, doesn't look like a. Classical. Cap they're trying to make it look cool and they got red red, on there, this. Was one. That. I worked on so, a few people have done brain computer, interface, World, of Warcraft's and. This. One was an example that used SS, vep we also had a different one with motor activity, but, basically you can move around the environment with, these four controls, left. Right, forward and select. You. Can do a fair amount largely, because of the good World of Warcraft interface, a key, reason is that this select, icon. Changez two different things depending on where you are we. Didn't do this this was Blizzard, Entertainment, but, it it was a nice example we were thinking well how could you use only four commands, to convey more. Than four potential instructions. In a context, dependent fashion and like well these guys I already did it and the, movement commands I mean you can go. Anywhere in 2d space with just those three and so. We've we. Gave this demonstration, that UC, San Diego and I told the story that I was unable to do it because UCSD. Blocked all the ports for World of Warcraft, so. We had a network guy and. We. Were trying to everything. Was working from a BCI standpoint. And yet we, couldn't actually get online and so the network guy had. About 45, minutes, you know I said could you please talk to a supervisor or something and came, back and said I couldn't like I did talk to his supervisor I did I was saying there's a professor, giving a talk they just couldn't get to that point in other words they're so. Committed. To that they had so many problems with students doing that I also, gave, this talk, at Blizzard. Headquarters, in Irvine California. Unsurprisingly. We have no trouble getting onto there wow sir whoo it was really fast, the, funny story there, is many. Of the people in the room were on the production team so, they're constantly yelling at me you know go laughter shoot, that or like. I'm not demonstrating.
Mastery Of the game I'm just showing this, BCI thing. And. I said I had some slides about fun this, is a great one the necomimi, cat ears I love this so this. Person, is wearing these cat ears and, what. The cat ears are supposed to do you can just see that headband up at the top there is they. Perk up if the user is interested or engaged and they droop if the user is not interested, and so. I thought okay that's a great example of sort of a cute, thing it's not really meant as a practical, one but people might buy it for fun later. On I thought this would be immensely useful for, public speaking so what. If every time you have someone, here everyone. In the audience has to wear one of these right, and so if all the ears start drooping, it's. Time to move on tell. A joke if. All the ears go up you're doing something right I've. Often wanted to do a study, you guys could, do it right here like a you. You kindly bought me a coffee before this and I was wondering you could. Plot that so what is the average time, per talk plotted. Against how much coffee did you get them just before then you. Know and so that would be another one if people, are bored give, them coffee. This. Is sort of the opposite, one where it's a competitive relaxing. Game this is called brain ball and by. The way I'm not with these companies I'm just so. This was a group in Sweden and they, made a competitive, relaxing, game so the ball moves, toward. The more relaxed, person, and so, whoever looks the most relaxed, will win so. This, is kind of a funny image because you can't tell if they're relaxed, or dead but. I'm, just, bringing this up because all these slides are things that are for the most part fairly, well-established people. Have done, them in some. Case has made money from them and yet they're not mainstream. I gave, the example of the force trainer now. If you have an electrode, right there over the forehead is that a pure EEG, system I don't. Need to answer this. Is a new one this is a novel, example, of BCI for fun this, is not, high on my list of likely mainstream, applications, but, I would recommend doing it here in Microsoft, because if you had a remote control cockroach, please do and if, you do put a little camera on them I want to be like I want, to see Bill Gates reaction, when their remote control cockroach goes in this. Is, this. Is true so I edited. A. Book chapter we we have a contest, for the best BCI, research for the annual BCI research, award and this. Was one of the submissions, and I was happy that it was, nominated for a word because I got to edit the book chapter and. In a way they kind of had a BCI, system, for the people and a computer to brain in her face a CBI. I guess for the cockroaches, so. First. They sell, this kit and this is online you can go to backyard brains right now and buy it you, can buy this kit and the. First thing they do which is nice is they make, you anesthetize, the cockroach like okay that's you, put it in ice cuz frankly. After reading this I felt bad for the cockroach and, then. You can. Essentially, put this on which will direct, it to move in a certain direction, the. Part that, they also had, as they then had humans, using a brain-computer interface, to send commands, to to cockroaches, running. A race so. You're. Sitting here and your buddy's right next to you and your job is to use a BCI, to direct, a cockroach, you would probably, prefer to do something else but, direct a cockroach, to move through that and so it's. It's. New I've never heard of that before, and. Slide. It's if anything it's closer to the work from Rao. And, colleagues, here and others with the brain-to-brain, interface but. To be more specific the, human is a brain, to a computer interface and, then the cockroach is a computer, to brain interface, it. Would be intriguing to have the reverse can cockroaches, control people. Probably. So. I thought, this was a good example of fun it's technically, impressive and, in.
The End in the acknowledgments when, I was editing it I thank, their subjects, and I made, him specifically, say we want to thank our subjects , both human and cockroach, because. Honestly, after this there's a lot of poor cockroach, they're. Ugly but yes, just a little sidebar. Dvdt. She's, one of the guys. Who works sort. Of senior, guy. In with a bunch of hardware groups his. Here's. A master's, thing he had a basically. This. Exact, thing with, moss, he. Was controlling moths I. Didn't. Know but, just. Just a little sidebar for you guys missed you. Yeah. Well. In a precursor, to this there was a paper that was published in Nature tower. At all 2002. Which. Was. Interesting. So, what they did is they, had computer. To brain interfaces. I think it was for rats or mice it was it was a rodent they. Implanted, electrodes, in three areas one, was the medial forebrain bundle which. Humans, have also that's a big area that involves, reward and an addiction. And other things and then, the left and right mr. barrel cortex, so rats have an area of the cortex that responds. To touch here or here humans, have this also but in rats it's exaggerated, it's called barrel cortex, because they're really sensitive to these little whisker whisker. Barrels sticking. Out so. They. Implanted. These and they demonstrated, that they could make mice move through a maze or an obstacle course basically, you could if you, you they train the rats so that if they. Stimulate, the left side and then the rat chooses, to move you give it a rewarding stimulus, so pretty soon as, soon as you do that the rat moves to the left if you do them both at the same time the rat moves forward the part. Of it that was particularly, ominous. Toward, the end is they, did the following they, had their rat move up a ramp and moved. To the edge of a precipice, unbeknownst. To the rat there, was a clear plastic panel, under there but the rat didn't know that in other words the rat believed that going. Forward could be fatal, they, gave, it the little zap and the rat didn't move forward they increase the current and the rat jumped off so, the. Rat believed, it was jumping to its death and, I. See no reason this wouldn't work in humans. Not. That I promote that. Another. Issue, that's come up is the idea of some kind of head, mounted display application. And when I say it's come up we. Gave a talk at Google we're, talking about this and a lot of other people had said what. About something involving, an hm D which, makes a lot of sense if you're already wearing glasses on your head or headphones, where it's much easier to put on electrodes, and people. Were working on earbud electrodes, these guys nam were nominated, for an award a few, years ago and so. You. Know with Google glass you, might for example use. It to select among different options so, this is getting back to the question of why would you use this if there's another way to communicate at that, time there. Was this term called glass hole which. You. Know for people who are using Google glass but, we're being visibly, distracting. They would do this or, they would say okay Google all the time and so. This is a case where yes, they could communicate with the interface another way but they might choose to use the BCI, for. Privacy and and courtesy, you, know or maybe you could choose different, parts of an image to my knowledge this wasn't implemented, but, I know some, students with Professor, Cruz e on skis lab we're interested in it and a lot of other people have talked about it and that. Might happen or virtual navigation I guess. I've had to Google slides and they didn't pay me anything. Driver. Alertness I've talked about that a lot where the, the. Question, here is why not use something else can you detect alertness with EEG yes, is that. Better than eye tracking, or other ones. People. Have different answers to that but it's challenging, to give, it a solid yes right now but. To be fair there are people. Who've said no you can get more, depth and detail from, the EEG than, you can from an eye camera, and then. Other. People don't agree so I'm that's getting a bit beyond my hand thing and. I'll. Talk about some emerging, examples, of big.
BCI, Mainstream. BCI, and this. Is also a pretty new thing in the field so prior to 2017. There weren't any big examples, of this of a very, large company or an individual, like Elon Musk who. Not. Only presented. An idea publicly. To. Mainstream, persons, about BC eyes but also was. Talking, about BC eyes for mainstream users you know it's Facebook, was. Talking, about a non-invasive. System that. Could work with. With. Anybody not just with patients and, they. Presented a lot of applications, and our, problem with this and I talked to a lot of BCI, people, is we. Don't really have any information from Facebook and so it's very difficult for, us to comment, they. Gave, a talk at the 2018. BCI. Meeting, and. Similar. To some videos and things they talked about optical, imaging but. We don't really know what they're doing and to, be fair of their, company there are for-profit entity, they're not. Necessarily. Required, to tell us I just, really wish that they would or at least tell me. Yes. Well. This is gnarly. That's here so, this. Is a new effort popularized. By Elon Musk and there you, see talk about neural link and they're a lace and this is a, similar. Case where you have someone who's very. Well established very, well known to, mainstream, people who's. Talking, about this in the, case of neural link there seem, to be focused, on invasive approaches. So specifically, they have this idea that something. Would be implanted. Just behind the ear or somewhere and so. That's kind of a different direction. But. Still these are. Quite. Futuristic ideas. I mean they're not imminent and I'll. Talk about this over the next few slides the, field, does need a breakthrough we could, benefit, from the the scale of research that this, is our people, like me able to come up with the sorts of developments, that. Companies. Could spending. Millions of dollars over the course of several years certainly, not I'm familiar, with a lot of the BCI labs their grants exactly. How much money is involved and of course the, scale matters. Another. Example is, Galvani. Bio electrics which not as many been. As public but they're, a company, that was started by, I believe, laakso SmithKline and verily so these are two big companies I believe. It was a 780 million dollar start, starting, companies so it's quite big and they're. Talking about electronic. Bio medicine which. Seems. To be kind of like the term adaptive, neuro technology, from Wolpaw and colleagues where they're, talking about devices, that both read and write not. Only to the brain but the nervous system for example the, spinal cord to help persons with spinal cord injuries so, although that's sort of broader than what most, people think of as BCI is it's, still related. Sports. Yes. First. My, colleague, Christophe cougar GU, ger, he has some publications involving. Sports and high, athletic. High, altitude, athletics, so for. Example he studied people while they did a simulated, ascent from 3,000. To 5,000, meters. Advanced. Brain monitoring, I think, the author was Chris burka BER, ka they, had some work also involving. Sports. Optimization. And it, was similar to work they did with the US. Marine, Corps they had a study where they were studying. People, as they learned how to fire, rifles. To. Try to help them be more accurate, and, similarly. They were working, on trying to help. Acquisition. Of sports knowledge and, I know there are some other examples I just don't know them right now but. Yeah people have talked about it and this. Is another example of something where, you might have more. Eager early adopters, if, I were to use a BCI, to become, better at tennis. Or something well nobody cares if you're an Olympic athlete even. The tiniest improvement, could, could, matter and could be worth a tremendous amount of money sports.
Another. Another. Sports example, although it's a different application, ge. And Siemens, had a grant, call several years ago for new, concussion, detection, systems so. That's an another, application, with sport if you're wearing a helmet and, you. Can have an EEG system, that can help determine things, like. Concussion. That's, quite helpful of course I would. Say again it should be a hybrid system the first thing I would say is have an accelerometer and the helmet have something that measures neck flexion, but. Given all those things is it possible, that EEG. Would provide additional information not. So much on the injury itself but how it affected the brain well yes that I think that's a reasonable, direction. So that is an application, with sport and if. It makes people safer, oh great. Not. Not boxing, also not, getting punched in the face so. I'm not a medical doctor but, I am a brain scientist, try not to get punched in the face. So. Yeah with these. It's. Hard for me to make a comment. Without knowing more about them. Comment, about exactly, their technology, but, I will, have a few slides where I'm talking about, applications. And so, these are some of the questions that I think need to be asked, critically, not just of them but of other, ones so one. Thing that's not clear is what is the training time great. You have a BCI that is a hundred words per minute that would be an extraordinary breakthrough, I'd. Say ten words per minute I call that pretty good how. Long does it take to train the user how long does it take the train the system, literacy. Does this work on. 90%. Of people who try it is it is it, just one person, who works in Facebook out of all the employees, you screen them all and you found the very best or is it something more Universal. Error. Rates, and error Corrections, so you might show a video where someone spells, perfectly, is that. Representative of, normal, operation, and error. Correction that if an error is made what. Happens that's an issue another. Big one is word, or sentence, completions, so is it, a 100 word per minute BCI in which you're actually thinking each individual, word or is it something where it's. Inferring. Or assuming, some parts of the word based on part of the sentence which, is a common, thing I'm not saying these are bad, I'm just saying these are, examples. Of details that I would need and I think most BCI people, would need to, be able to really, evaluate is. This a breakthrough or what's happening and I mentioned, number of users or takes you know again does it work on everyone I do. Object. To this so none of these entities are really engaging the DCI community, much this is not just my view comes, up all the time and, this. Raises another question of, how does this affect public perception, of BCI's and so I'll get to these slides. But. Here's. An example of where this gets worrisome, and I'm not citing, any of these companies but this is a specific example from, a paper so. A lot, of articles for, me and from others have, raised concerns, about regulation. Or at. Least act, information, about direct-to-consumer. Brain, devices, this. Is an example that was quoted in this article because, the sleep Shepherd lets, you directly control, your brain rate you, can naturally build a better healthier, sleep cycle that allows you to fall asleep faster, stay asleep longer, and have more energy during the day by. The way these comments are not endorsed by the FDA well. A few things first. If. You know what it means to control your brain rate please let me know I'm a professional neuroscientist. I have no idea I've asked, other neuroscientists. They're like we don't know maybe. It means, control. Brain frequencies, like alpha, or beta but does that mean, you, have more activity, at a certain frequency are, you trading for, a different peak frequency, are you training the ratio but. Moreover. If if this, were true wonderful, I mean everybody, would do it it might be true I don't know but, this is an example of something where, you. Could imagine that someone who bought this and then found that their sleep didn't improve would then be a lot more skeptical, not, only about BCI is in general but people like me so.
I've Said this before a. Friend. Of mine from Microsoft, it resonated, with me and I go and people say oh BCI's, are are, stupid. They're not useful for mainstream people, or they're. Unethical. They're, potentially. Dangerous they, don't work and, I. Can. Understand, why many people would think that given what's out there and. So. In. 2018. To talk more about this, mainstream BCI. Or big BCI this. Is a list of the sponsors, of the 2018. BCI, meeting so this was the main conference, for, BC is in 2018. None, of these are big companies with. Respect to them none, of them are like. Elon Musk or Facebook, I know, most of these guys of, course we. Appreciate, the sponsorship, but, this is it's kind of like the, beginning. Of my talk where, I sort of liked having it frozen in time as of. 2020. This, is the involvement, of BCI this is the perception, of BC is as of, 2018. These, are the biggest companies, that are sponsoring. The, biggest conference. Another. Issue from. I call. Science. Fiction about BCI's, it's bifi it's not sci-fi and another. Common, view that we have to address that comes from, this is that basically BCI, is is, evil. Right a BCI, is made by evil people to. Be used for evil purposes a major. Exception, the. First Star Trek pilot this, is a very realistic and altruistic portrayal, of BCI's Captain, Pike is a disabled, person he, was a military hero he saved many cadets, from Delta, radiation, they. Described in detail, how. This works he. Can, do. One thing for yes or two for no he can move around, that's. About right that's about consistent, with the wheelchair level, controller we have for a lot of people but, all these, other ones are examples, where the. BCI is used to to oppress, to. Spy. On people, to cause, harm in some way and the. Worst one is of course more, than 50%, of black mirror episodes, have, BC is I refer. To this as the Bricker BCI, because it's a guy named Charlie Brooker it's. Usually the same it's a tiny thing that you just stick right there on the temple and. There. Appears, to be no limitation, of its capability, it has complete, read/write, capability. Totally, immersive, and. With. Arguably, one exception. They're. Evil I mean they're being used to, in, really, really iffy ways I say. This not theoretically, I've run into people you know and you mentioned what you do and they have, a very bad reaction to it and when you talk to them it's because of.
Things Like this they believe this is what BCI's are the, bigger problem, is that it messes, up public. Discussion, of ethics that, is these. Are. These. Are more pragmatic. And realistic ethical, things we do have ethical, discussion. Boards if anyone's, at the BCI meeting I'm I'm leading, a workshop where, we're trying to work on ethical, guidelines, for. Companies, and mainstream, applications, any of you would be most welcome and. You. Know there are good sides of it big BCI in particular could. Reduce system, cost this is huge it's just like a car. If. You start. Manufacturing EEG. Systems, on the scale of tens of thousands, or millions becomes. Far far cheaper. It. Could foster new applications, including, patient applications, and this came up a lot during lunch that I think even. If a company is purely, focused, on mainstream, users has no interest in patients, whatsoever, indirectly. That will lead, to patient benefit partly, through reduced. System costs, partly, through new application, partly. Also through improved integration so, it might be easier for a disabled person to use Microsoft. Raise. Public awareness that's, good inspire. More funding that's good and, I. Mentioned this before lead to a breakthrough a large-scale. Research has, never really been done before with BCI. But. There are some problems could encourage false claims and hype draw. Attention away from patient, applications, could. Lead to public mistrust or fear, less. Funding and, so. This gets. To the issue of managing, expectations which, is relevant to mainstream, users what if it doesn't work what. If a system causes, harm even, if it's not a real DCI that, the media. Interprets. It as such someone, miss reads the article, you. Know so. There's. Been some, work toward. Real-time. Brain. Stimulation, systems, you, know that and this, was one example that I saw at a conference a company, called focus, where. They were presenting. This as a system for gamers to maintain, vigilance it's a brain stimulation, tool so. It directly influences the brain so. These gamers that stay up already for 72, hours drinking Mountain Dew don't need a system to make them stay up
2020-02-29 01:45
