429 - Replacing the white cane with technology?
the latest tech i'm alexa i can answer your questions interviews and we are evolving and we are seeing an amazing opportunity that's happening accessibility accessibility is one of the core values it's even the product mission statement this is double tap tv welcome to this edition of double tap tv thank you guys for being with us each and every single week we're flashing back a bit this week to ces and a company that we featured on our ces show but we wanted to dive in a little bit deeper and give you some more information about why they exist in the first place i am marca flala with stephen scott by my side steven i've got a question for you that's going to really set up this week's show which is how do you get around you know you've got a little bit of vision you obviously do not drive a car so when you need to get from point a to point b how do you do that so i use a white cane uh which you know i don't know what it's like in canada but in the uk you have to be qualified to use yes in the uk we qualify people to use a white stick um and yes i travel around with that and what it does is it gives me feedback via the stick which has a rollerball at the end of it that rollerball moving from left to right in front of me essentially covers the ground where i'm walking and it gives me tactile feedback and also alerts me to any obstacles so tactile feedback first that can be for example any changes in uh the pavement so for example if i was veering off the pavement onto maybe a grassy verge i would instantly get that feedback or perhaps the path changes and it also lets me know for example when i come up against a curb and that drop of a curb i'm able to identify that with my cane but it also allows me to any obstacles so if it's bendy and all the bins are lying out then i know that i can avoid those because i'm going to bump into them first so that's my means of essentially getting around my vision allows me to see a little bit but i have absolutely no depth perception and i have got very little useful vision to tell me what's in front of me or at least to make out something like a pole for example that's one of the biggest challenges is identifying a pole the blur is so much that a pole can disappear in front of me and the cane its purpose in life is to find it now but you're limited in terms of the length of the cane really in terms of how much in advance you're notified to something right when you say tactile feedback on a tech show i'd be like oh that's pretty cool you're going to feel different things but it's literally what's right in front of you like the cane is what maybe five feet maybe four feet so if you feel elevation change or or you you bump into something it's right there like if that cane wasn't there and you didn't react immediately you'd be walking into a post you'd be walking off a sidewalk you'd be who knows what would happen so really technologically speaking has this cane evolved much since you've been using it no the cane hasn't evolved particularly i mean there's been many attempts to evolve into something new something incredible more useful more dynamic more smart all that but truthfully none of these have really caught on because they all override the uh the feedback that you already get from the white cane so as i'm walking down the street and i get that tactile feedback which is essentially the vibration coming from the rollerball up the cane to the handle that's the only feedback i'm getting right but if you add additional elements of tactile feedback to that perhaps through ai or through i don't know radar or whatever it is that they put into these smart canes it adds a huge amount of extra information that i have got to try and disseminate as i'm walking down the street all the while trying to identify what's in front of me and not you know be killed by a bus um and that's where the problem is with a lot of these smart devices they don't really address the real problem because frankly the problem is already solved by the white cane you know i recall a product that you spoke to the company that you spoke to back at techshare pro i think it was in 2019 that attached to the cane right it attached to the cane and added a lidar sensor and through vibrations and haptic feedback it did exactly what you're describing i can imagine sensory wise suddenly you're now trying to figure out you've gotten used to obviously elevation changes and things in front of you so now this new thing is giving you vibrations and things probably a little bit earlier than the other feedback so trying to decipher made it i guess it made it even more confusing it does and you would think it would help right because what it's trying to do and what you're referring to is the we walk smart cane uh which was a really interesting device because what it was trying to do was not just to give you the information through the cane as you would normally get that tactile feedback but you would also get information about what is head height because one of the biggest challenges for a lot of blind people is overhanging branches when you're walking down a street so if you can somehow find out that's coming it means you're not having that hit you in the head which is something the white cane would never be able to identify unless you had one which were holding up in the air at the same time that's not advisable right so you know to avoid that you get this radar which kind of points up into the air and kind of points at your general direction an area of where your head is heading so that the cane is not just handling and giving you information about what's on the ground but also what's ahead of you up above your head um but again i found personally the information that i was giving was just too much and it was all coming through the same handle so that was a challenge i was missing out on the information i needed to get from the ground because it was giving me so much information about what i was having around me and in some cases there was nothing the read i was just maybe pointing a hedge that was walking alongside um what use is that right i mean i'm not planning to walk into the hedge i'm walking beside the hedge so why am i getting information about it so it wasn't a perfect setup and i think we're a long way off from getting the solution in that form what about the lidar sensor that apple introduced on the on the pro models of the iphone which uh made it on the ipad first it seemed promising a couple apps took advantage of that like microsoft's own seeing ai was able to identify things in the room in the proximity you know and that isn't haptic feedback that's you know through an earbud on your on your ear obviously does that prove to be helpful is that something that we're going to see you think evolve down the road i think it will um and i think it already has started i mean like you say it has really been used up until now more for in the home i mean don't get me wrong at the beginning of the pandemic when lidar came along a lot of blind people were using it in order to socially distance from the person in front of them because you can use lidar stay in a store to be able to identify how far away someone is in front of you so you could use that to get your two meter distance so it was really handy for that and it helped a lot of people of course you had to have the iphone 12 pro at the time uh you couldn't just have the regular iphone so that was a bit of a challenge not everyone had that device but now of course we're seeing other apps use it to identify uh products in the home uh or items in the home so for example classic blind move is to put your cup down on the desk and forget where you've put it uh you can use the lidar on the phone not just to identify the cup but also to locate the cup with your hand and use lidar to be able to navigate yourself to the cup so essentially a series of beeps and tones gives you that you know hot and cold approach to finding that mug and making sure you spill the coffee all over the desk yeah it's almost like the metal detectors of the past even though yeah you know what this week we're going back to ces back in january on our hour-long ces special which you could find over on youtube or ami dot ca double tap we spoke to a company called biped ai that is introducing a very cool product that comes in the form of a physical vest goes over your shoulders has a whole bunch of sensors and i guess it pairs with a bluetooth headphone or something to help you navigate the world in front of you but you know what before i make assumptions here let's take a break and let's come back and let's talk to their founder all about this really cool product in great depth it is doubletap tv he is stephen scott i am mark of flallow if you guys want to get involved at home no problem the email address is feedback ami.ca on twitter follow us at doubletap canada use that hashtag which is ask doubletap and when we come back we're gonna reintroduce you to biped ai for more great doubletap tv content visit ami.ca tap this is double tap tv welcome back to double tap tv thank you guys for being with us each and every single week i am mark of lalo with steven scott by my side ces is now just a couple months behind us so to speak uh so a better time to reintroduce you to a company that we had the chance to talk to on that ces special and that company is biped ai that's right mark and mile fabian is the co-founder of biped and he joins us now on doubletap tv now mile great to have you back with us uh you know some people may have missed our episode uh or just caught a little bit of our episode on ces i mean that's obviously a disgrace people must go back and watch that episode in full or else we'll send someone around um but in the meantime mile perhaps you could tell us uh for those who missed that or perhaps you don't know what biped is yep so um but biped is in a sense an a.i co-pilot for blind and visually impaired people um so the roots of biped were lying in the in the innovations that happened currently in the field of autonomous vehicles um and the way those technologies are evolving on the streets just made me think well there has to be a way that can be applied to pedestrians and and more specifically to visually impaired people um because the vehicle is making decisions at a very high frame rate and it can predict what's going to happen in sense in a couple of seconds in advance and that would of course increase safety um for for vision impaired people when they walk in the streets so biped is a small harness that you put on your shoulders it has a set of cameras uh that captured the environment with 170 degrees of field of view around you and after that it's basically replicating what the autonomous vehicle does but at the pedestrian level so it's detecting tracking and predicting trajectories a few seconds in advance and it's providing immersive 3d audio feedback um so you use bone conduction headphones typically and they would uh you would have sounds that are coming from a specific direction with a specific volume that just make you feel okay there's a car here that's coming towards me and there's a pedestrian there that's turning right for example that's the type of information that you're able to get and also those obstacle uh detection for you um so that you can um you're in a sense protected from head to toes well one other really interesting things i find about biped ai um it really is the ai element because this is more than just detecting an obstacle that might be in front of you and giving you a notification it actually determines what is in front of you and how best to avoid it right yeah absolutely what we're aiming for in a sense is environmental awareness rather than just knowing there's something there you want to know what is it and where is it going and does it has a risk to to come in my direction to hit me and so you can anticipate in a sense so it's not only knowing there's a bike but it's knowing there's a bike that's coming in your direction so you hear the sound coming continuously um so we're really trying to merge um what's what can be done for example in object detection and object tracking and artificial intelligence nowadays there's a couple of apps that can do that but merging those two and we're adding a layer of navigation also so that we have a just a companion app where people can just just tell where they want to go and and by having everything um we aim really to have this all in one approach where ranging from 15 centimeters like the closest obstacles to 15 kilometers if you want to like go quite far we can allow you to navigate um quite freely um that's the the the aim of the project in the end now look mile i am a registered blind guy and you know i'm the kind of guy who you know finds it a little bit awkward going out for a walk people love to say that don't they go for a walk it'll clear your mind it'll make you feel better but when you're visually impaired there are a lot of challenges out there and you know i'm very awkward when i walk i'm very conscious of my surroundings i do use the white cane to get around that's my method of getting around the place but um i i don't know i'm always wary of new technology that perhaps gets in the way of what the white cane gives me in terms of feedback so does this device does biped work as a replacement of the white cane or as a complement to the white cane yes it does um there's the tactile information given by the white cane remains i think essential at many levels because it also informs you on the type of ground you're walking on and if you're walking from concrete to grass for example that's something you might want to know in the dance and that's something we can detect but the aim is not to have overwhelming audio feedback so you need to dedicate one channel to what it's good at um and yeah so the the white cane can still be using complement the guide dog can even still be used uh in complement to what biped is doing because the guide dog's never going to tell you what objects in front of you it's not going to give you that environmental awareness it's very good at guiding you but it's not uh it won't be able to tell you what are the things around you where is the um where is that bike located where there where is that car located for example so we essentially try to to bring all of that together on top of what's currently existing and um and having this just this this simple claim that we're um just raising um or improving environmental awareness of blind users um compared with what they currently use as a device so what kind of audio feedback is it really because you know there's such thing as as too much really especially when you're also listening to the world around you right um the main approach that we took is since we embed cameras we're we don't only have ultrasounds uh for example ultrasound sensors that basically tell you there's a bit of like a mass moving but we can characterize what type of object it is if there's for example several people walking towards you any other type of devices as advanced as they might be will tell you about each and every each and every person coming in your direction that might be like if there's ten people walking in a direction that would be um obviously too much um what we do is that we filter based on trajectories and then we group concepts together so if there's a bunch of cars in the parking or there's a bunch of people that together form a crowd we will do not before inform you from the fact that there's a crowd coming in your direction so that's with trajectories and with grouping this information um we are drastically reducing the amount of information you're getting as you're just walking in the streets we are in conversation with mal fabian co-founder and ceo of biped ai steven let's take a quick break and come back with more from male because i want to get into what this device really can precisely detect it is double tap tv we'll be back in just a moment for more great double tap tv content visit ami dot ca double tap this is double tap tv welcome back to double tap tv we're talking all things what are we talking all things i guess obstacle avoidance here this week i am mark of flallow with steven scott and our guest this week is mal fabian he is the co-founder and ceo of biped ai mel you mentioned that uh this device can detect items uh even people um so hang on does that mean for example we could uh get a sense when jane from accounting is coming along absolutely let's um we have a busy road map of things uh no but the point would be to be able to recognize um people um that you have pre-registered for example on the smartphone but also to be able to recognize custom objects uh that you might have stored also it could be car uh could be like uh yeah just like keys could be uh anything that you can think of um that's something we will be able to recognize later um but yeah so the the the road map is a bit like we want to freeze what we have now just to keep improving the current level of those features and then as we have cameras we'll be able to read text to read qr codes um all these are fully open as if you had a smartphone uh but hands-free in a sense you know i've got a lot of voices going on in my head mile at any one time i've got voiceover going on i've got the traffic noise now we're going to add biped to that as well you know it can get a little bit information overload it would be good if there was a way to bring all this information together on my phone for example so that you can almost mentally organize your way through all these different sounds is that something that's possible um essentially it's a matter of how do you stream information from the from the device to the smartphone um there's obviously things that are totally possible um we are thinking of ways to integrate a couple of other technologies or apis that have been developed for the smartphone and be able to stream in a sense the data the the one thing we are paying attention to is the device if it's becoming a personal assistant and this ai co-pilot that we claim will capture data that might be sensitive and that you don't want to be uploaded to servers for example or like leave the device by itself so we took this privacy first approach where every data is just like every image is just captured processed and thrown away by the device and there's no local storage of that information um so so yeah but i mean we we there's definitely ways we could we could embed a couple of other services um and use that as a hands-free uh version of a smartphone you don't have to to hold in your hand anymore can we talk about the hardware here for a second is this a bulky add-on to existing clothing am i am i going to look like a swat police officer yeah um so it it just comes as the final layer um of um of everything you you're you're wearing i have the the device um here um and i'm just wearing it around the around the neck there um so what you would essentially have is you have a small battery module sitting behind the neck and then you have two arms coming on the chest on the on the right arm um of you would have the computation unit of course that's still very very much like a prototype and we're working on improving this and on the other side you have a set of cameras and those cameras uh are these depth cameras um and it's overall just the the whole system is around um 850 grams um so you you barely feel it um to be fair and and um yeah and you can i mean if you were like i mean it's pretty cold these days in in switzerland i guess uh it's also the case in canada and uh and yes i i wear like pretty big winter jackets and i just sit on top and uh and it's it's even with a backpack or or everything you just put that at the final layer are there any controls or buttons that you need to press to make things work or is it all completely hands-free so it's a it's a standalone unit i'd say um there's on off button which is the only thing you have access to on the on the device um otherwise you can even start the device with an app um so the device just connects over bluetooth to your smartphone and then you can start stop you have couple of stats on the the session you just had how many minutes how many kilometers kind of things and then you can also report bugs and feedbacks directly on the app to allow for like a self-improvement of the system um so yeah now you mentioned it's got a camera there in order for it to operate i mean a lot of blind people have issues with night blindness they may be able to see something during the day but not at night does this work at night yes that's one key criteria we wanted to have the the cameras we used they captured notion of distance but i also captured this through the night using infrared sensors um so we do have most of the capabilities of the device still work in in the dark um which is i think something where where that might be quite different with like what the camera of the smartphone for example might be able to capture because once you move to the dark i mean essentially the camera of the smartphone um except for like the the lidar uh ones but i mean basically they won't capture anything um so it it works almost entirely in in the dark um too except that some objects that are a bit further away where you don't have yet a specific shape in in the distance image that they might be detected a bit later but um essentially it works the same way that is mael fabian the co-founder and ceo of biped.ai stephen scott that was a a lot to deal with i'm uh excited for you to get that on your chest so that you can actually feel what it's like to navigate the world with something like that that might actually free you one day from that game yeah well that's the point i mean i i don't know if it'll free me from the cane so much but i like the idea of it being separate from it that's the key um i like the idea of being able to go around get that extra information that i want but without taking away from the white cane we want to be able to add better experiences for blind people moving around more safely more independently in all environments but we don't want to take anything away from already working operations like a white cane or a guide dog so this is this is good i'm excited about this one i'm excited too thank you guys at home for being with us this week and of course each and every single week if you want to get involved or you've got a question or comment about this or any other show feedback at ami.ca is our email address
and if you're not following us already do so on twitter at doubletap canada with the hashtag which is ask doubletap on behalf of our guest ma'al fabian and steven scott i am marco flallo we will speak to you again next week hosted by markaflalo and steven scott editing jordan steves and markaflallo voice over anna vicino integrated described video specialist ron rickford coordinating producer jennifer johnson director production kara nye director programming brian perdue vp content development and programming john melville president and ceo david arrington copyright 2022 accessible media inc
2022-04-03 01:04
