Welcome back to our main session, and this is the part of our final, leg. Of the, conference, this afternoon and. This. Will be the presentation. By. Sean. Kane this. Is our he is our third technology. Futurist. That we have featured today here at the University, of Colorado. Sean. Kane is an associate, professor in, the department, of computer science. He, directs, what, I think is one of the best names the. Superhuman computing, lab. His. Primary, interests, are accessible. User interfaces. A mobile human-computer, interaction. And explores. Ways to make mobile devices, easier to use especially. For people with disabilities, and people in distracting. Environments. So, with that I am going to hand it over to. Dr. Kane all right thank you everyone. Hear me okay come on like that all right awesome. So, this. Is my first time as a futurist so thank you for indulging, me in that. And. Really. Happy to talk to you about some. Of the work we're doing but also to take. A broader look and talk. About some of the interesting. Things going on in terms of physical. And tangible, computing, and how I think, these are under, explored, areas for, cognitive. Accessibility. In. Just a note so if, you sometimes. I go a little fast, I put, the slides up if you go to Sean cat, slash but you can't see the laser Coleman, 2018, you can see all the slides there I'm, happy to give that to you afterwards, but in. Case you want to follow along or play, play I get along at home so. Just. To give. You a high-level overview of what I wanted to talk about so. This notion of physical or tangible computing, I'm gonna use these terms interchangeably. I think, it's a really exciting place to explore, accessible. Stem new. Types of assistive, tech and to find ways that people can work better together in, learning. About stem and making assistive technology, this. Is something you can start today and I will give you some pointers on how to do that and. I just want to talk about some of the opportunities and some of the challenges. That lie ahead. So so. When I use this term physical computing, this. Is not going to be on the test so, just that I want, to point out a few things that are I think are interesting and worth paying attention to, so, one is you, know finding ways we're removing the notion, of computing, away from, something. That you just you looking at a lit-up rectangle, screen and out, in the physical world a bit more. Part. Of this so I'm a teach, computer science and part of this is thinking about the the process, of doing, computer, science as a combination, of, software challenges and hardware challenges, and. There's a lot of really cool stuff going on in terms of open source hardware and. Affordable. Technology, that you can use to just start making stuff. You. Know today. So. Because I have this cool, lab name and all these great students I just briefly want to plug my. Lab where. At cu-boulder and, the engineering center come. Visit sometime and. A lot of the work we're doing is around looking at multimodal. Interaction. For, education, for expressing, art basically, finding, places where the. Traditional ways of representing, some kind of information, just don't, work well for everybody and trying. To put together, alternative. Ways of exploring and making sense of what's the what's there so, I'll talk about some of the projects in the context of this talk but there's lots more so, you can check out our website or talk, to me or there's, a there's a group of my students kind of clustered over here so you can, you. Can you can talk to them they do the actual work. So. There's. A bunch of interesting things going on in. Terms of you, know access and availability of, this technology the. One is you know we have these toolkits that you can go and you can buy, that. Let you build physical prototypes, again. Easily and quickly. I recommend. Sparkfun because they're local and the. Founder is a CU alum but, you can go to you, know many, places and get kits where you can so, an interactive circuit or snap, together circuits, using magnets, and. So there's a lot of tools now that we have available to play around with and. We're. Also seeing this notion of like sensing, as part of computer science across, the age range so I was at Target a couple of weeks ago and I saw this, image at the bottom here is a toolkit, for, button. Made by a company called canno where, you can build a sensing system, just. Out of the box and it's targeting, younger kids so. Not. To say that you, know kids. Our kids, can do lots of things but the fact that this is now something as an introduction to computing I think is pretty interesting. And. Then, all of this is in the context, of you, know taking. Control of assistive. Technology and partially. Thinking about assistive technology not, just as a thing that some, benefactor. Does to some you. Know inspiring. Person, but, as another, wait for people to engage in solving, their own problems and, solving, problems for their family members and people that they care about and.
So Just a quick, overview here some of the things we're starting to see in terms of. Electronics. Projects that are happening, and being shared online there's. A bunch of 3d, printed Braille and you know you can go and download different. Types of tactile. Learning materials, for instance and 3d print them and some. Of them are made by people who actually, have. A need for these kinds of materials, not. Enough actually, and that's something I'll come back to. So. Here's. Why I'm taking this bundle of. Trends. And tools that we have available I want to highlight some of the the, opportunities. Here that I think are, are. Worth thinking, about and worth you know finding ways perhaps on your own work where you can integrate these, these. Resources that we now have. So. So. One is just that we have this ability to, with. These kinds of technologies, to make, multimodal. Learning something that you can do at home that. You, can we can take technologies. Off the shelf and start. To, bring. This into our home so we're not just watching educational. Videos but, we can actually engage physically, and. So here there's the, top a tool. Called little bits that lets you build circuits by snapping together magnetic, blocks, this. Is a varsha cow shake my students work on building, programming, tools, where you can make little interactive, audio stories by snapping, together different. Stick bricks and, this is you know this cost about $50, worth of parts to have the, entire programming language, so. We can really make multimodal, learning something that you can we, can just all have. So. Briefly. Also just shout out some of the work, that we're doing that I'm really excited about, so. Annika who's sitting. Right there it's been working on projects, were to. Bring multimodal, learning into online, education, and I'm. Gonna show you just. A brief clip from this video where. She, built, some software to take a pen, plotter which is an off-the-shelf, technology. That you might get at a you. Know store that sells fine see kinds of things this. One cost about $300, and built. A tool where we can have online, lessons, that combine, both. Video, but also a, tactile. Sense. But, the point I want to get across is you, know I'm, as. Much, of a skeptic as anybody when so. Self-proclaimed. Futurists. Come in and say here's, the future technology, that's going to solve accessibility. Problems but, this is just software right, so this is stuff that you can buy today and, you. Know taking these these tools that we have in thinking differently about how to use them and so there's a lot we can do potentially in this area of multimodal learning but. I want to draw I want to bring out a few other opportunities. So. Another thing we can do with this notion of physical and tangible, computing, is personalization.
Of Assistive technology and, so here's, another project, from our lab this is from Hallie Profeta, and she. Was building wearable, AAC, devices, so this is a lycra, of leg, strap so it's something that straps around your leg and then she programmed in different buttons that. Can can, speak and an AAC and it was intended for equine, therapy so, riding, around on a horse and and maybe talking to your acquaint Arabist or your family members or what have you and. One of the things I think that's super cool about this is you, know the personalization, is, on many. Different levels in terms of the shape of the technology, where it fits on the body even. The sorts, of things that you can say and, the, visual design, of how it works so this is something that can be from. End to end something that can be optimized for a particular, person and there NEADS. Briefly. I think there's a lot of really fun stuff that, we can do when. We have technology. That can. Actuate, in the physical world this. Is another project that, I had the pleasure. Of working on where we were building drum. Sets that you could play with eye gaze so this was for someone with ALS, but. The idea was you know, let's. Build something where we can have. Fun and express and have something expressive, and. We could do this also with software but there's something about the. Ability to act in the physical world especially. If you're having some you, know if you have ALS for instance and and moving and interacting with the world is a challenge, this, can be a way to maybe get some of that back. Okay. There's. Again. So. I, think there's also the. Opportunity for people, with disabilities, to make assistive. Technology, for themselves and for others that they care about if. You had a chance to see Lorri Dale's talk just before, this one she's, doing a lot of really cool stuff and, what. We're seeing is that people are excited, about building. Assistive, technology, especially people who maybe, have grown up using assistive, technology and have seen personally the benefits in their own lives. And then. Just. One more thing I want to mention here is that you know one. Of the interesting, opportunities, about thinking, about teaching. Computer science as a, task, that has both hardware and software components is, now. We have a task that we can break down and, carve in lots of different ways and so maybe we can find opportunities where, someone. Who might have, difficulty, writing. Out the code can work in the circuits and someone who has difficulty with the circuits can write code and someone, who has difficulty with both of those might do, some user testing or, other kinds of things so so we have these. Computer. Science tasks as something, that use, different types of skills which, mean in some ways I think they naturally, encourage people to work together and to find parts where they can contribute. Okay. And. Then, I I think, there's more opportunities, here I'll, talk more about this later on but that. It's not just technical, contributions, but also things. Like the design and creating. Sounds, and graphics, and other types of things and so with. These kinds of activities often, we, can find many many different ways to participate if. We're if we look. And we know how to take advantage of that, okay. But. There. Are challenges, right, we're not there yet and so I want to highlight a few things that I think are worth. Thinking about and. Places. Where we can possibly innovate, in terms of technology, and. And. In, our teaching methods to actually bring, more of this into the world so people can use it. So. So. A few challenges, I'll highlight so, one is that our curricula, for, teaching, these kinds of materials, can be pretty, primitive, and. You, know basically a lot of these sorts, of projects. Are now documented, online through, some combination of text. And video and maybe some graphics, and. We often don't have good ways of evaluating. The. Difficulty, of a particular, activity or comparing. To you, know if you do, a google search and you say how, do I make a. Door.
Movement Sensor you're gonna find 10 different examples and like how do you know which one to do how do you know which ones you have this skill for so, we don't have great, ways of understanding. And. Leveraging. These kinds of activities yet. From. The perspective of educational, research I think we also don't necessarily have a clear, sense. Of. How. These tasks work so I was saying that you know you can go through these tasks through these, kinds of activities lost through, lots of different ways but, that also means that modeling how someone might learn in, programming. Some. Physical device can, be really complicated all, right so we might have different sorts of branching, paths where, I'm, partially. Working in Hardware and partially writing code and then, when, I go to tests things like, who knows what's gonna happen there's, a lot of different paths that you can take in. Terms of understanding the problem debugging. The problem and fixing things and. I think there's an opportunity here to to, look at this process, of learning how, to do. This sort of computer. Science work but also how to how to teach it and how to learn it. One. Issue that's come up quite often is. Issues. Of when, we're bringing in these physical components, then there's, accessibility, challenges, with that and. A lot of the toolkits, that you can get today have. You, know small finicky, parts right so the top here this is lily pad it's. Also from from, Cu and here. You're threading. Circuits, together with conductive, thread but, again this is a difficult, physical task for many people and. Here we have this. Little bits and we're using magnets but, again the, size of these, different, kind of components. Can be an accessibility, challenge for some people and. So I think finding ways where, this can you, know people can build the projects at the right level of ability. Is. Something that we need to keep looking at. Likewise. I think when, we think about programming, these, types of technologies, we still have this, dichotomy, between wiring. Up circuits and writing, code and I think there's opportunities, to unify. These and. So this is an example of cubelets. To the system, from also. Related to see you where. You can program, by. By. Manipulating, the structure of the, circuit so you're. Not separately, working in code and working, physically, but. Doing something that combines both of these concerns and. You. Know in our group we've been brainstorming different, kinds of ways to. To. Program these sorts of systems and, we've talked about things like assistance. Like if this than that which you might have seen which is thinking about programming, by talking. About certain events that can happen and then how we might react to them and. Also potentially. Modeling, something so I, really want to build a doll smart home doll house that we can start to use as a way of testing. Out things, like smart homes and other kinds of systems because. You, know it can, be often be really abstract and if you're programming some system to do. Something when the you know at. Midnight well. You kind of want a way to simulate that you don't want to have wait till midnight to have to. Have. To see whether your code works. So. Another. Problem is this notion of feed black right. Not feed back but. Feed lack which. Is when the, system fails and we don't have a useful, error message so when you're building circuits the most common thing you, might encounter, this is you build a circuit and you press the start button and nothing happens you don't have an error message it just something doesn't work and these, kinds of problems can be really difficult to debug so, you, know where's the problem is it a coding problem, is it something with the circuit is it something about your design that's problematic like, it's really hard to know and.
This Is a this is a real challenge. That affects not just assistive. Technology but all sorts of problems. And. There's some work going on in terms of building, sort, of augmented, physical, tools and circuits for that. Help you learn about these sorts, of problems and so. Something, like this which is an Augmented, breadboard that is designed, so that it can figure out a circuit that's incorrectly, wired whereas, in the real world an incorrectly, wired circuit often just won't do anything. And. One. Of our challenges here is is with. How. Do we document projects how do we share what's going on and again, you, know an interactive, system is something that's really rich and complicated, and doesn't, necessarily reduce to. Two. Pictures, and text and. So we. Might think about things like making, it easier to show video or. John. Chimel has been doing things where the actual output of these sorts of hacking, workshops their Hardware kits that then someone else can take and build on top of. Okay. So, so. There's a lot of opportunity, and there, are some interesting challenges and I, just wanted to end with some high-level, thoughts, about where I think these. Things are going and things that I want us all to think, about a little bit and. So, you, know why are we teaching you. Know one, of the questions that's come up in some of this work is like how far can we get in terms of teaching of. Computer, science to people, with cognitive disabilities and, I. Think there's a lot of, things. That we you, know a lot of questions, about what are the right sorts of paths you know should we be training people to, have. Careers, as software engineers probably. Some people but not everybody all right where are the places that we can make. This valuable. Not just because, we're computer, science researchers but you know meaningful, to the people who are participating in it and. One thing we've seen consistently. And I'd say that we know now is, building. Assistive technologies, is one place where we're seeing some opportunity, and. Some excitement about this work and. In general we're seeing as is, the case often with computer science is building, things that other people can share and enjoy is something that really, appeals to a lot of people, and. Then, the last no this is something that my. Students and I have been talking a bit about and thinking about is for. Those of you who are computer scientists, as.
We're Talking through this work we've been running into some really interesting questions, about ableism. And so when we talk about how. Do we teach someone, with a cognitive disability computer, science there's all these assumptions, about what that means and for, example we. Know we might talk about well, if you really are successful, at learning computer science you can, understand. And explain a problem, at a high level of abstraction but. In the reality that's not necessarily what you need to be successful, and. Likewise, this notion of a way, to tell whether you're making progress, in learning these tools is whether you can do it without any assistance and I, think there are places here where we we, might want to rethink these assumptions, or start to think about other, ways of measuring progress. Alright, so. I'll. Plug. The, lab and say we're we're. Really working, actively. In the space and we've got a lot of cool stuff so please, come talk to me or my students if, you're interested, in this stuff and we'll find ways to participate. And. I, said, I could help you get started today so if you think, this is cool and you want to do it. Go, buy a micro bit you. Could get it maybe at sparkfun you might have to order it online and. Go try make, code editor but this is sort of the hello world of a. Program. That actually, just, prints out a 11y. So. That's a thing that you can do today so. With, that I, think, that there's a lot of exciting trends. And opportunities going on with. Making computing, more physical, and. There's a lot of interesting problems to solve so. With that I'll be happy to take your questions, Thanks.
2019-01-06