- Well, thank you all for coming. We're really excited here to be presenting at re:MARS and debuting the Retriever and also talking more about what happens under the hood to make basically our ability to move things, both with different types of commands and extend the impact of individuals, but also including being able to move things with your voice, which we do by connecting our device with the Alexa system. So we have a lot to cover, but we're gonna take it at a good pace and if there's Q&A, we definitely can do that as well. So I'm Mike Dooley. I won't hit the mic anymore. I'm CEO and Co-Founder of Labrador Systems.
A little bit later, my Co-Founder Nikolai Romanov, who's my long-term CTO and partner in crime will be up here to cover some of the technology in more depth, but we're gonna start high level about the market need and why we're working on this and go into the individual experience of the product and then roll it into how the technology is enabling this new class of assistive robots to help people live more independently, something that we're very passionate about and think that we really need to be addressing in our society. So to give you some ideas, we'll first cover an intro video. This is taken from individuals who participated in our pilot studies over a year and a half ago now where they used the robot for five to eight weeks in their home so these robots we had at the conference and at other locations, they aren't just demos, they're actually real robots that were in people's homes, using them and interacting with them and getting a lot of data and also really we think making a difference in their lives. Then we're gonna jump into understanding the longevity market, which some people track and others don't. It's something that's, as you get into it and you have folks like my parents, and they start dealing with the issues of aging and folks needing more help, it's a little bit of a shock to the system and what we wanna do is sort of in a nice way, explain what's happening now, and what happens as we get older and why we're working on technologies like this, so that in the future there's more and more options for all of us to live more independently and more happily. Quick, we'll go into our background, then we'll go into the robot, and then we'll go into the demo and the technology stack and everything from there, so it'll be fun.
So first off, we're gonna do our intro video. This is about 3 1/2 minutes and it's four people and I encourage you to listen to each one of 'em. It's about 45 seconds a person. All of 'em have a very strong personal story about how changes in their health or events really started impacting their daily activities because basically in a high level, gravity got heavier for them.
You'll see 'em, you'll hear what they have to say, but when you're looking at 'em, I think what I look at is I'm either seeing, you know, the voice of my mom when someone's talking or I'm seeing potentially what my personal needs might be. And so I asked you to sort of just think about that this is most likely gonna be a situation that in your family you'll be dealing with and why it's so important because this is, it's just something that's gonna affect us all. So we'll kick this off with the intro video. - [Narrator] Four years ago, we began working on a new type of personal robot. We designed it to physically assist people in their daily lives, especially where pain or other health conditions impact their activities. Our mission is to empower individuals to live more independently and to do more of what they want to do.
- I feel very strongly about living independently and living on my own. I have three children who all invite me to be part of their lives, but I don't wanna be dependent on them and so anything that I can find that will help me do that is really, really valuable to my quality of life. It's my aide, it's my assistant that's like a third hand for me. In my mind, I'm still 23, but my body isn't and the robot lightens the load for me and allows me the time and the energy to do the things that are really still very important for me to explore. I still have a lot to do.
- [Woman] Alexa, ask Lab One want to come to the dining room. - [Alexa] Okay, going to the dining room now. - Using the Labrador provides me a lot of independence, whether it's setting the table or getting the laundry into the washing machine.
I'm not lifting anything, I'm not having to bend over. I'm not at risk of falling. It's providing me with the ability to do tasks that normally I would have to ask for help with. It's like having a second person in the house.
It helps that much. - Some of the abilities that I lost following my stroke was the use of my right side. I've never realized how much I couldn't do anymore, but with the Retriever, when it was introduced to me, it was like regaining that whole right side again because it was able to carry and move things for me that I'm unable to do at this at this point. So it's giving me that confidence level to where, with the Retriever, that's my right hand buddy.
You know, it's something that I look forward to and that I could say, "Hey, you know what, let's go on with life, like it doesn't stop here. It continues." - Doing the groceries is just, is what you should be doing anyway. You don't miss bringing in the groceries, you know, things like that until you can't do it anymore. And to not have to ask for help to be able to do it myself, not struggle with trying to lift a bag or put it on my lap and roll into the kitchen one at a time, I could get these things done, the independence and the freedom, and just the self-worth of doing it, not having to feel that you're somebody's burden. The robot gave all that back.
- Great, so the, you know, when you're looking at that, you see individuals, that's Madge, it's Janet, Armando, and Trisha. Those are all individuals that we know their stories deeply. When we were in stealth mode, we were reaching out to different care organizations, doing Facebook posts.
And it started off with just a phone call or a Zoom call about, "Hey, you saw something. What do you think could this be useful for you," and understanding their need. And I think it, you know, what we do when you talk about like Armando saying, "Hey, I got my, you know, this is like getting my right side back and I can deal with this stroke." I don't think we, by no means, do we deal with 100% of the challenges he's facing. I don't think we even deal with about 5%. It's less than that, but that 5% is really important because it's 5% in the right direction and I think whether you're someone that's, hey, you're in the 40s, really viral guy, and all of a sudden you have something happen you didn't expect, to someone who like Madge, who is an in her 70s and is dealing with rheumatoid arthritis as well as those health conditions and things like doing the laundry is competing for our energy for taking time with her grandchildren.
And so these are really sort of universal truths, I think, of the human condition and we just hear the same story over and over in different ways and that's why we're so motivated when we see people use it. First, well, what we talk about and I wanna hit on this is about the longevity market. It's a little bit of a joke, but it has a little bit of bite to it.
When I started pitching this product to Silicon Valley about almost, it's five years ago next month, it was hard to get the empathy for this and I think what I learned after a while is that most VCs don't think they're gonna get old, they're just gonna get rich. And I say that because I think sometimes we think of that too. Well, if I just plan ahead and I have all the resources and work out, which is really important, this doesn't affect me, this is somebody else. Yeah, you know, that's not really true. We all are gonna get old, now some older than others, but we're all gonna be impacted by this and we've met some people that are incredible. I've met someone that was in his 80s that COVID was happening and was really upset because he couldn't be playing on the basketball team in his league, and at the same time, he had a wife at home that he was really worried about of her ability to take care of herself if he wasn't around.
And so it's either gonna be something that's affecting us or affecting people that we're caring for. This is my personal story and what really influenced me quite a bit initially is, this is in 2012, I'm getting married. That's my mom dancing with me and she was really nervous and she wasn't nervous about the wedding. I'm the youngest of six kids, she's been through this. No, that was no sweat.
It's that she was using a cane by that time and starting to use a walker and I knew she was doing that. I could see getting out and we would park closer to the restaurant or do things like that, but what she was nervous about is that I can't take a step back, I can't turn here, that's my weak side, and dancing with her in that wedding, it basically was sort of like really, really hitting me that her constraints, what were those things that she was struggling with? Well, it's five years later, she's full time in a wheelchair and in a hospital bed having caregivers into the home and it wasn't sudden issues. There was a blood clot, she went into the hospital, but not for stroke, they detected it early and great medication, you know, great system. Like there was no, you know, chronic event, but there was atrophy from all the swelling and just being off of her feet and then the pain and everything setting in, went into rehab, again, like world class treatment in terms of like physical rehab. But then when we got home, it was like 1930 because it was just my family, my parents, and us trying to figure out, well, what does mom need now? She can't walk anymore and she's still otherwise healthy, but she needs all this care, moving her into the dining room, bringing in a hospital bed, learning that you could get a hospital bed funded by Medicare, all these basic things.
And I guarantee you, it's a hard thing is that this is a shock to the system, especially happens when you don't think it's gonna be doing it, but the part about it is, is that it's not unique, it's not like that was just an isolated case. 15 million Americans, adults 50 plus, use a mobility aid, such as walker, wheelchair or cane, to move around and that percent increases as we get old. It's a pretty predictable number.
Those charts there show the rates for both, for either individual items like a walker, a cane, or wheelchair, as well as combined. The dots are my mom. So in her 70s, she started using a cane and then into her 80s she started using a walker, and then by mid-80s, she's into a wheelchair. And the news is, is she's still around.
She's now 91 and she's still going strong and this is this idea of getting older, younger is that your health condition and overall, the average that this works out is, is that my mom's experience is so much different than my grandmother. I remember my grandmother passing away in the '80s. She was in a nursing home for a few years, and then she passed away at a relatively younger age compared to my mom, who's now in her 90s. And the period by which my grandmother was in that condition was shorter, it was more compressed. So as we elongate our lifespans, we are in this state where we actually have, we're gonna be using a cane for twice as long as a prior generation, or we're gonna have health issues or other things that we're dealing with that need support and that's complicated because all of a sudden you have a lot of individuals needing help, either on a personal level from their family or from paid caregivers, and as well as on the health system and this is where we've got a big problem and the US has been behind other economies on this.
So Japan's been the most extreme. You have Europe, now we're hitting it, and then China's gonna have a weird inversion pretty soon that they're gonna be facing this as well. So the challenge is, is that the fastest-growing population in the US is 85 plus.
Now you don't seem to think about that when you go to Silicon Valley and see what they're investing in. You don't see that on TV or the hottest trends of products, but it is, that's the baby boom generation moving into that upper curve and the folks that are our ages, for by and large, that are 18 to 64 that are in the working population, and there's people that are working longer than that, but by and large, that mass of the working population, the folks that are available to either care for them or generate income to pay others to care for them is flat. And so you can think, "Well, how do we do that?" We have more people in this upper space, but we're flat in this other part. And the way we do it, it's almost like clutch adjustments. We keep squeezing out productivity in the system. So in the case of my grandmother, she stayed in that nursing home until she passed away.
The system doesn't do that anymore. The system has, you have acute care in hospitals. We turn that around really quickly. If you're not aware of this, my father-in-law had outpatient hip surgery. It takes my car longer to get fixed than my father-in-law took to get a full hip, one side hip replacement and then he does the recovery in the home.
If he's not able to recover in the home, then they'll put you into skilled nursing rehab. And again, Medicare focuses on like, they count the weeks, they count the days you're there and they cap it. And so there's this pressure that we are more efficient now. We have better quality care, and we're also more efficient about it, but there is a cost to that is that when we push people out into the system and we can get them back home, then there's this thing that happened with my mom is how do we deal with this? And so now there's 53 million unpaid caregivers in the US alone and that number is growing at 4%, exactly at the rate of the population above 85 and that's a problem because it's gonna continue to do that.
And if you're tracking, you know, with COVID and everything that was happening, we're losing folks in the care sector. We lost over 200,000 caregivers and skilled nursing alone in the last two years, a huge loss. And when we talk to folks in those industries, they are desperate for help. They are short staffed, they are calling in temps and they don't have enough people to do that. So red hits this influx where we have a huge need. We have just, that's been now, it's a crisis upon a crisis as people have talked about it with what's happening into the industry, and then we have a chance to use automation in two different ways.
One is to help people be more active and more independent so that they can push that sort of heavier need side off even further. Could my mom be more independent? Could she be walking around further and doing more before she got to the level she had? Or could we identify some of the risk factors so she could even earlier treatment? And then as we have folks get to the state my mom is now, could we extend the impact of caregivers to both give her more control over her daily life, but also relieve the burden on either family members or paid caregivers. So our background, really quick and like why, hopefully, you may believe we have a shot at doing this is, this is our attempt to do a threepeat. Myself and my partner, Nikolai, we've both been in the robotics in prior, in his space in aerospace and automation for quite some time. I go back to the 1990s.
I launched Lego Mindstorms, so I was original product manager and marketing manager when that was just a little skunkworks project. Back then we had a big, big vision. We were a small group at Lego that very few people thought would be successful, but we thought that we could change the world by creating a new tool for kids to think with, and it did. The product we launched was the bestselling product in the history of the company and still is, and millions and millions of kids have used it and that was sort of my, it was not just my impression on robotics in a very basic way. I finally learned that driving a motor too hard could purge the firmware out of the memory of a robot and that everything is linked, that sort of system level complexity, that's both very frustrating at times, but incredibly intellectually challenging and interesting as well.
But it also was basically showing the social impact of making a change at scale. So 10 years later, Nikolai and I have now connected at a company called Evolution Robotics and we're launching with a bunch of other of our friends product line called Mint, which is now still on the market and is called Brava. Brava was the first, well, actually one of the first smart robotic floor cleaners that could track where it cleans and actually the first smart mop, and the interesting, the significant part of that wasn't that it could mop your floors, that was really important 'cause that caused people to buy it, but that it was for a very low cost, could localize in a very complex environment. That was a progression. Before that, at that time Roomba was random. And so two years after we launched, Roomba acquired us, so iRobot acquired us, Mint became Brava.
The same product that we shipped in 2011, 10 years later is being shipped today by iRobot. That thing's got a lot of ROI in it. And then on top of it, the algorithms that we have both for Mint and also for the visual SLAM system that Evolution and the engineers there pioneered, now are what make Roomba smart. And iRobot's gone on to do a lot more than that, but that foundation autonomy came from the work out of that.
And so when we're looking at what we were going to do next, we saw that, well, we can progress. I can go from a toy to a floor cleaner to something bigger and what's happening is, is there's the technologies from other categories are enabling us to go into that space and we'll get in more into that as we go through it. So our mission here is really to do a practical robot for the home.
You know, you'll hear this. I mean, the thing that, how to say this, we're not the Hollywood of robotics. When I used to do Lego, we had the base robots that the kids would be able to build and then the videos would be all the stars within the Lego ecosystem that could do like the hero models, a robot that could solve a Rubik's Cube or climb a rock wall, or Lego rock wall at least, or sort M&M's. Now that's not what a 10-year-old could do starting with the product, but they were so motivated by that. And we're all sort of always infatuated by the spectacle about making something that has a face or has something that looks like us or walks interesting or things.
And the problem is, is that we can get so infatuated that we actually don't make robots that do things and so that's something that we have to really sort of be ingrained in is where can we make a difference and what's the mission? So in our mission, it's make a robot that practically does things that make a difference in people's lives, but at the same time has to be ridiculously affordable compared to the other technologies that are out there so they can afford it. And so I'm not trying, I think what we are trying to communicate here is that robotics is moving in a different direction. We are getting serious in terms of this, serious in terms of the skills and the technologies that are being applied and the problems we're trying to solve. And when I go out to the healthcare systems, a lot of people expect that we're gonna roll that out in front of them, that we're gonna have a robot with hands that's gonna come in and help move people, carry them out of the bed, change all sorts, change clothes, all sorts of stuff, and I don't think we're gonna see that in my lifetime. The pace that we robotics work and the technologies, they're very impressive, but it's sort of like the airline industry. When I was 10 years old, I took a flight out to Los Angeles the first time and yes, that plane was a lot different than the planes are today, but it was about the same time and about the same seat and same view outside the window.
Robotics is better than that, but we're not at this level of being transformative of having devices that are going to basically take on human capabilities at a one-to-one scale that can support people. If the people working on those, they're great, but they're 20 to 30 year R&D projects. I'm trying to make a robot that can help my mom before she passes away and help all these other folks that have been testing and are asking for their robot back, by the way. So here's our view of this and this starts from figuring out what you have as building blocks and what your application is and what your current need is.
A lot of folks, when you start talking about care, they go to that sort of left side of the screen where, Hey, I can make a robot that replaces a person. It could cook, it can clean, it could reach up to high shelves, it could carry heavy loads, it could go out and dump the garbage. And the problem is, is we don't have that. We are, you know, we're just at the, we're looking at the other robots like Agility or with Boston Dynamics, they are just starting to pierce into industry where they are talking about some very high-cost solutions, like Boston Dynamics talking about solutions that might be like in an oil rig or at a power plant, or having something at scale, like having Agility that could do package delivery 24/7 out of autonomous car and be used over and over for the same purpose. The problem is, is that that type of economy doesn't happen when you're talking about putting one robot with one person in the home.
And the joke I had when I was talking to someone in senior living is that right now, Amazon uses the robots to bring the shelf to the pick-and-place point where an individual does the loading and you don't have a robot doing that. You don't have that last step. Even that task where you've got a structured environment, a shelf that's been laid out by the company, lighting that's controlled, a whole workstation, and an employee who's trained, they can't basically replace that with automation yet. And basically the joke that the senior living person was saying, "Well, if Amazon ain't getting it, Mom sure isn't," you know, in terms of like having that capability. So we have to start from this idea of grounded in what can we do. And the other part is what should we be doing? And that's what the second question, I find a lot of folks in robotics don't answer that.
It's like, this would be really cool, but is it what we need to be doing? And so this is where we get into this idea of having a robot that's assisting versus replacing. I'm not trying to make a device that's going to do your laundry. I'm trying to make a device, it's basically helping replace the function that's messing up you from not doing your laundry. So that capability to carry a basket that's 20 to 25 pounds is what breaks for a lot of people.
For most everybody in that video, if you watched it, they don't have the ability or it's very straining or risky or everything else to carry the laundry, therefore they don't do it and/or they do it at a cost. It cost manages her time with her kid, her grandkids, because she's tiring out doing it. And so, or it costs Trisha who's in a wheelchair, it costs her a sense of her dignity because she's always having to ask for help. And so what if we could just do that one task? What if you can do like we had in the video with Madge is a robot that can be large enough that you can put an empty laundry basket on it, send it off to the bedroom. You can use Alexa or use your phone or tablet or button, and it goes there and it parks right where your laundry is. We have a video of this with Trisha.
She stripped the bed. It parked right park right next to the bed in her wheelchair. She's taking off all the dirty linens, popping 'em on the robot, the robot's low because she's low to the ground, and then she sends it off to the washer dryer to the laundry room, and then she can basically step by step load in, do the wash, do the dryer, and then take the items out one by one. The same problem we've heard like a 40-year-old guy with a stroke is the exact same issue I heard with an 80-year-old gentleman, very proud guy, former Vietnam vet, former retired police chief with MS.
And he had the same comment when he saw the robot is all he does right now is fold the laundry because that's the only step he can do and his wife has to do everything else. And his ability to do just, hey, if I can replace that function, then we can do, then he's a lot more capable of what he wants to be able to do. So this is an assistive robot.
This is something that's meant to lighten the load of daily activities, bring critical items within reach, water, medication, your cell phone, or other things that you wanna keep close by. Enable hands-free commands using Alexa and the power of that ecosystem to plug into what we're doing. And then longer term, we think we'll basically enhance other applications for home health.
So the way it works, and this is, we've got some patent pending stuff on this, but the genius of this is that it's so simple, is that the way you do it is there's a set of locations. For users, we call those bus stops, and the bus stops are the place you want it to go. Robotics, we call those way points, but bus stops sounds a lot more cool.
And there's a bus stop is basically anywhere you may wanna send the robot. Send it to the kitchen, send it to the laundry room, send it to your side of your bed so it parks at night as your bedside table. That metaphor is super simple, it's universal, everybody has a location where they're sitting and the robot can be close by. And the robot from that point will then self-navigate from one point to the other.
Those blue lines, they're elastic, so it can basically adjust like a pilot adjusting around a storm using the sensors that see obstacles and what you then do is now you have this sort of routine that I can tell when people are using the robot, like for the R&D projects, we have it on Slack so the robot chats with us. And so there's a gentleman up in San Francisco, and I can see he was getting up early in the morning, sent the robot to the kitchen, sent it to his desk. He's making coffee. Someone's sending to the laundry room two times a week, or multiple times, but happening on two times a week they're doing their laundry. It becomes pretty simple when you're sort of figuring that out, but if I would know that that's, if I have an indicator of, hey, my mom is up and they're active, but I'm not spying on her and she's willing to share that information, man, that's a lot more relief than calling and not getting an answer or worried that, you know, something's going wrong.
So what we did with that is we placed the robot, these robots in people's homes for up to eight weeks and we just let 'em use it. It took us about usually about two hours to train it on the environment. The system's not optimized for speed yet, it was really optimized for function, and they would just go at it. And without a schedule, without any requirements, they could have used it zero. And what happened is, is most cases where people engage with it, they're using it 90 times or more per month.
Now, again, my background is product development. It's like, that's amazing. You take a product that didn't exist, there's no precedence for it, you pop it in there, you give them an engineering interface, nothing that's fancy and they engage and they use it persistently to do things like go and get the Instacart order. That wasn't, that idea of bringing in the groceries wasn't on our list, it wasn't an example. They didn't see the video you saw before. They wrote that video, effectively.
There's the scene with the taquitos and tortilla soup. That came because it was late. I think it was a Friday night, my wife's late for home. I'm hungry for dinner, for waiting to make dinner and I get this video of them having this great tortilla soup and taquitos and using it to carry around to sort of set, you know, to have dinner while they're watching TV.
And so other cases like tidying up, things that we think like, Trisha put it really well. You don't think about, you know, you don't miss doing, you know, carrying out the garbage or bringing in the groceries until you can't do it anymore and then it becomes something you miss a lot. And so those are the things that this basic engagement that we show that we just hit this raw nerve of a very sort of strong unmet need. So where does this go? Well, once you have that, once we have this bargain, and this is one of the few technologies we go into like age tech, like this isn't just sitting there with a camera to monitor if you fall or checking like you're, you know, whether you're staying in your bed or other or not and stuff, it's actually helping you, and because of that, it spends a lot of time next to you. It's the whole idea we had for a vision of why we called the company Labrador.
We wanted this to be sort of this trusted, loyal, and actually subordinate being in your home that you could feel comfortable by your side, as if it was your loyal dog that's there. That's why it doesn't look like a robot. That's why it's got wood paneling and all sorts of stuff. And it's why it's it adjusts in heights so that it's not out of place depending on the situation. But if you have that equation and people are naturally using it, then you get a lot more because now you could start saying, "Well, let's add some skills, just like Alexa has skills. Why don't we bring, let's say telehealth connectivity?" Have a whole story about my mom.
She has an Echo Dot in her room and I bought that because everyone was forgetting to turn out the light when they would leave, the caregivers or my dad, and she couldn't get up and turn it out, so really simple. I said, "Hey, mom, here's this device and you could call it," and she was curious about it. She's not a tech adopter. And I said, "Well, what do you wanna call your light?" Well, I call it Mom's lamp. And from that point forward, it's like, "Alexa, turn Mom's lamp off. Alexa, turned Mom's lamp on."
And that was great. It was like, you know, very empowering thing for her. But at the same time, I said, "Well, oh, there's another model has a screen and we could send you pictures from the trip and chat with you." She's like, "Oh no, this is a room where I get bathed and changed and no, I don't want that." And which then I said, what was sort of the inspiration for this is like, "Well, what if we bring the screen to you when you want it and we take it away when you don't?" And that's the same sort of social principle about when we go into a room or not, or if the caregiver's there and we're respecting her privacy. Medication reminders and adherence, physically bringing something to someone.
We have a gentleman who's on our wait list up in the Seattle area, Wounded Warrior. Like this guy is not lazy. He actively competes in competitive archery, even with spinal injury, and didn't make the cut, but he was trying to make it to the Olympics in Sochi on the last round. And he says like, "Even I, I have an app that reminds me take the medication, I click it, I'm fine, and then I get distracted by something shiny and then I realize, 'Oh, it's three hours later, I'm off my cycle on it.'" If you physically present the medication to them, someone gets distracted, it's still right there.
We had the story where Armando was doing laundry and he sent the robot off to the laundry room and forgot about it. And so then he is walking later that afternoon and he sees the robot with the laundry basket sitting right next to the laundry room and he goes, "Ah, I gotta still do the laundry." So this idea of physical reminders, you know, where we have outta sight, outta mind is there, now we have outta reach, outta mind. And so helping people who just really core things like stay hydrated because you're constantly presenting them with water or fluids or a nice cold drink if they wanna get it. And from there, we have a platform that's pretty much the smartest device in the house, so we could go back in time and I could have the robot using its wonderful 3D cameras and stereo vision to look at my Mom walking and it could have probably identified that she was starting to walk differently when that swelling was happening and that would've been great to flag for her, great to flag for the family, and great to flag for the doctor so it didn't even get to a blood clot, so she could have gone to physical therapy earlier.
That's our vision, make life better and let people stay in their own home environment, but increase that quality of care, increase that quality of life and support. Now the data behind this is building as we go, but the consistent pattern we see is this whole idea that people bond and use the product. And so this was an interesting example. We were doing these charts of activity because the whole first question when we started rolling the robot out is would people use it, and then if they would use it, would they be consistent? Would it make it into a routine? And for most people, we saw this sort of thing where they would spike. Some people used it a crazy amount in the first few days, like they were just playing with it, but then it would level into a trend.
And so from there, we would normally see that we averaged out around three or four times a use a day without reinforcement, without scheduling, even without retrieval. This is just a base model which you can send around to things. What was interesting is we saw this one individual that was spiking. They were using a lot on certain days and lower on other days. Now that spikes continued, so she was still really using it the same amount per month, even in the second month of the test, but she kept on doing it differently. And so this is a woman in her 40s with early onset Parkinson's.
And if you're familiar with Parkinson's, there's what they talk about being on and off. Being on means that you're in your zone. Your symptoms are the lowest that you sort of reach.
You have more control over your body. When she's in her zone and she can talk and I can hear about, she can probably get a word out that I can understand in about every three, four seconds. She also has control of her wheelchair. She can be moving around when someone else is not in the house and be more active. When she's off, and it's hard to watch, and we don't take a video of that because it just, it hits us, but we don't wanna exploit it, is that it's 15 seconds for her to get a word out, even sometimes that her caregiver can understand.
At the end, when the caregiver leaves and that's on those days, she's gets put on a mattress in front of the TV with a blanket, with water, with the cell phone and the robot next to her and she's waiting for the next caregiver shift to start or for our sons to come home, tough stuff. And the interesting thing we're talking to her about, and she was super like, she was adamant about getting this robot. She's the third user we had and she was just a wonderful person to work with, is that we asked her, it's like, you know, "Why are you using it more on certain days and not others? Is it that you're going out for therapy? Are you doing certain things?" She says, "No, no, those are my on and off days."
And my on days, again, are ones she has, she's moving around and more active. The off days she's not. And so what we were picking up was her on and off cycles. And I said, "Well, would you be willing? Does your doctor actually ask you to track your on and off cycles?" And she goes, "Yes."
And we've never figured out how to do it, and at that point, her caregiver, who's been on all the calls with us goes, "I'd love to see this because I'm with her Monday to Thursday from like in the afternoon, or morning to afternoon, and then I don't see her the rest of the day and the rest of the weekend and to see this, I could tell that, 'Hey, are we having an issue? Is there something that we tried with new medication or therapy that's something coming on or is she just in a bad spell?'" So the idea there's a give and take. If we make a product that people wanna use, naturally because it benefits them, then the system can benefit from that because it gains us access, it gains us to insight that we can't have before. I talk like this till I'm blue in the face to folks in healthcare about this. When we made Mindstorms to try to help kids sort of expand their thinking and basically get smarter by building and program robots to do what they want, we didn't start by trying to teach them something, we started by trying to get 'em to use the robot and build it and motivate them. Once they did that, there wasn't anything on our side we can educate them about, it's their brain and their ability to sort of construct the world and everything they understand. And so it's a different approach and so what we're documenting is that this approach will work, it does work, and it can have incredible advantages of keeping independent and saving cost in the health system, either by identifying issues early or by reducing the cost of care as people progress.
So that's a lot about us. I really appreciate that 'cause we haven't had this venue that's that long of a time to say that, but this is significant. This will affect your life, either personally or as, the folks around you.
So on that note, I'm gonna bring up Nikolai Romanov, my CTO and Co-Founder can come on up and I'm gonna let him go through some of the technology of this, and I'll chime at times too, but I'm gonna let him drive for a bit and give myself a break there, so come on, Nikolai. - Hi, so my name is Nikolai, Mike's CTO and Co-Founder. So yeah, he talked a lot about the product, but so let's talk about what makes it cheap. So there we started basically, we have a background in a lot of an industry and in consumer products. And so what you see like on this slide, there is a lot of like activity happening on the technology side for like commercial robots, which are deployed in Amazon warehouses and on assembly lines and those robots, we use fairly expensive technology.
They've been around for like 30 or 40 years by now. It's mostly like leader-based navigation systems, which basically scans for environment, does the matching and knows what way where it is. And those robots were great, but we can move fast, but were not are as safe as what needs to be at home.
And when on lower end, there is all this technology which go like in Roomba and vacuum cleaners, are really inexpensive so those robots cost on average, like 10 to $50 a month over a lifetime. So like commercial robots on the other hand, we started like multiple thousand dollars and go to as expensive as like 10 to $100 thousand dollars. And yeah, those organizations can afford it.
And there is this big opening in between, that's where like we fit in. So it's a robot like this size, what you see here on a stage, each now with technologies available today, which are coming from augmented reality and using SLAM design to sort of like position the player in the room, they can be used to position robot in the hall. And at the same time technology developed for where like depth cameras. We allow, again, inexpensive obstacle avoidance so robots can safely navigate and run structured environment of your home and go like around for obstacle, around like pets, you know, toys left on the floor, or like if somebody pulls the chair. And what it allows, we are using like commercial hardware coming from this technology. It's from a cell phones, like cheap processor from augmented reality, it's all this development.
In SLAM, each can run on those processor really efficiently and inexpensively so we can build something like Lab and it can navigate and find its way around your house and it be like in a prices, like in 100 like to a few hundred dollars so it fits right in between. So that's what basically how it looks. So what it does, it use inexpensive cameras and it looks for visual features in the environment and it learns it, and that allows it to navigate through and build pretty detailed maps in which it can instantly recognize where it's located in your house. So it's like what you see, it's basically robot navigating, like early prototype navigating.
I think it's in my house, and on the lower side, it's that so what we call sparse 3D representation of like second floor of my house and it's basically fairly accurate and we can run it all on the edge so we don't need like, we don't use any cloud so it's like inexpensive processor coming from like cell phone can process all this visual information, build a map, and can do it all onboard of robot, so it's even like, it's like important for a home and if like people start to rely on it, so even like if there is no internet connection, the power goes out in a house, it still can run and find its way around and go to bus stops. So here, like we have like a little video which shows, again, it's like earlier in early days, like kind of early prototype, that's when we've been pushing the actually driving robot for one of places where we can test and this in real time when this hardware builds this 3D representation of a house, which looks a little bit like a matrix, so it's like sort of cool, but yeah, each of these dot it's like a unique visual feature, which the software recognizes maps and when can use to navigate and instantly know where, again, it's allocated in the house. So yeah, its, let it play a little bit. And what's important here it's that for our application, yeah, we do a setup once. So we bring robot in somebody's space and we drive it through the environment and it can like build this map and regardless of the changes in the environment, you turn on or off the light, you know, where sun comes out or it's complete darkness at the night, it still can operate.
So we have a National Science Foundation Grant for like advanced increased technology versus what augmented reality does because augmented reality designed just to operate in a fixed environment of a house. You normally have like, you start with game and you play for like 10, 15 minutes, maybe like half an hour, and lightning environment is the same. And even if it's changes and it has a problem, you can always like restart the game. For the robot, yeah, we have to bring it in, set up the map, and it has to be good for operation for like long periods of time for like months or years and regardless of all the light and environment, so we have like some advances where we like, again, because for robot this big, we can like infrared light and we using quite for normalize visual appearance of environment and it allows us to run where augmented reality SLAM and use it for navigation like in any condition. So then again, on a high level, like Mike was saying, we're trying to do it to make it like very simple for when users to operate, so all these complexities are hidden and so here's like a basic where the flow. So it's like you can think about with robot as a simple IoT device or like home automation device.
So all it can do, you can send it like from location to location, you can send it like go to a kitchen or something, or you can ask it to bring something to you. And so on a command flow, it's, again, all these complexities are a hidden and yeah, it's like you issue a command. You can use like voice application. You can use Alexa, you can use like simple web interface or application on your phone, or we give them a simple IoT buttons and you can just push it and the robot will know to come to application. So again, it's get enabled by this SLAM system, which instantly, robot always instantly knows where it is in your house so and once you give it a command, it's has this pre-programed paths, and it will try to follow it as close as possible than using these inexpensive depth cameras, it will avoid where obstacles which can be around in your house. And if it's at all possible, it will get to a destination and when it arrives to a destination, again, it can execute some simple action.
It can give you like a reminder like Mike was talking about, you know, it's time to take your medication or time to take your blood pressure. And you can ask or it can like retrieve things. Here we have like demo where like robot can retrieve drinks from a refrigerator, compatible refrigerator. Like one of my guys can start it. And that's what basically happens. Robot will navigate to the location and then will execute a simple action so it knows it can like pull the tray out and then bring it to you.
So let's see if it works. We've been running it like on the show floor for the last couple days and see if it like. Guess it's decided not to cooperate. Just fine, we'll see if it will work.
Anyway, so, but then we basically can see on this video, it shows what happens inside robot when you give it a command to go somewhere. So on the left, you see like we input from the camera and it's like, again, we are running some simple like object recognition, but what you see on the right, it's like occupancy map and we reach like depth field, which coming from the depth cameras. So that shows basically you see like the house and you see how like the depth cameras show where all the walls are, where all the obstacles is, and what you see, like with green line, it's a flight path. It's bus route Mike was talking about. And that was like pre-programmed, but once again, robot like tries to follow it and we'll avoid all the obstacles. And just all these like inexpensive sensors, cameras, processors, which coming from consumer electronic devices, we enable like this reach perception, which basically enables the robot operate very safely and predictable in the house.
So, yeah, we'll see. We'll leave the camera on and see if it's like works correctly. So here's like, again, it's like probably a little bit too much details, but that shows like a basic software stack which we're using to make it possible. And here, again, like our thing, it's background in the consumer products and when started that we wanted to develop a product. We really didn't want, we were thinking what we can go and find all the technologies which we need out in the wild and when we started to arranging for technologies close, yeah, there is a lot of tools. There is a lot of building blocks develop by unit, but there is like certain missing ingredients and that's sort of our philosophy so we're trying not to reinvent the wheel, so we're using Linux, we're using Cross, we're using a lot of like vision algorithms which are coming out from the community and we're just adding those missing pieces, which are absolutely necessary.
Like we said, like augmented realty SLAM is great, but it doesn't work well in different lighting conditions so that's what part which we have to augment and make it better. So, same thing like on those, there is a lot of modules which already do like path plan and can allow you to send robot around, but a lot of them like really optimized for commercial environment or for larger robots, so again, we do just these like little pieces, missing pieces which we bring in here. And so once again, what makes possible to build the robots which have performance similar to those of, well, like industrial and commercial robots, but at a fraction of a price, like 1/10th of the price or even lower.
Video seems to be cut for whatever reason, but we have like this video, that's basically what robot doing like retrieval from the compatible, what we call pallet. You can put like the pallet anywhere, just a simple piece of plastic, and it can pull the tray which can have like anything on it. You can like pre-stage it, like in this case, like there is some water and the blood monitoring cuff, and once again can act, that's what Mike was saying, as a reminder.
When it's time to take the blood pressure, it can like know it'll go and retrieve the tray, bring it to the person and act as a physical reminder and when we're done it can take it back. At the same time, yeah, you can like, if people have high need, it can be pre-staged with snacks, drinks, you know, like cleaning supplies and you can always ask for robot to bring it around. And all the complexity of the system done for retrieval, it's on robot side. So those pallets and trays are very inexpensive and you can add them throughout the house or facility as needed.
So it's like, yeah, we've done a lot of testing connect, like, yeah, this picture illustrates how we've been testing like at night and making sure like our visual navigation can operate and work in any kind of a condition. Again, it was funded by NSF, so we're very like appreciative of them like recognizing what we're doing and so, that's like in a brief about like the technology and so what our next steps are, right now we have like those robots you see here, those like we build like six of those prototypes so we've been deployed combined for over like 18 months in people homes and people were using them, like individuals up to eight weeks at a time. So we're at now like engage for factory, doing like optimizing design for manufacturing and expecting to have like evaluation units coming from the factory later this year.
Again, hopefully like supply chain will finally start to ease up and we can build more first robots. It's like always frustrating to see what like, yeah, we have like, it's all within reach, but always like situation like delays keeps delaying, then we can have like more hardware. So that's pretty much in a nutshell so I guess like we'll open it up for some Q&A. - Yeah, we're rebooting the demo so we've had demo gods affect us where we had a great run for the last three days down on the floor, but we'll do this to see if we can get one good take of that so you see how it works, but yeah, if there's any questions we can address that.
As Nikolai was saying, some of the other points are, one of the things is that we've kicked off a national tour with Nationwide so they're really one of the pioneering companies in the insurance industry looking at this space and they're doing that because they have long-term care insurance where they fund all the things for care that a lot of things that Medicare and others don't take care of, naturally. Robot's cycling up again right now. We're expanding our pilots in our homes, as well as group care settings.
There's just, every time we go to talk to senior living, it is labor, labor, labor. And so we're looking at ways that we can help folks both in independent living. That shot of the robot you see there in the garage, that's actually an independent living community where folks have cottages and that's a wonderful place.
You have the benefit of a community, but you have your own home, your car, and do your own cooking. A lot of just really, you know, nice amenities for the support you need, but also the independence you want. And so there they literally, first thing they asked us is can the robot go out to the garage and you know, we're in LA and most of the time in LA, the garages are notched down below from the bell to the house 'cause the house is usually raised.
And in Michigan, when they built this community, everything is accessible so it was a level set from one place to the other where the robot could actually navigate all the way out to park by the car so even someone who's still going the grocery store and driving can basically carry items from place to place and you know, and then send it back. So that was actually the robot self-navigating. The bottom picture is a whole different world, same actually organization, but basically has the ability to have the robot operating in skilled nursing, so the other end of the spectrum. See the trick is with the robot and on a demo is not to watch it while it does it.
Just keep on talking and then it's very happy. So just a tip from the future. So anyway, so really happy to be here.
We'll be expanding our pilots and looking at more insights for the data and then pretty much we're gonna open up here to QA. We can either take it here or off, or now that the demo is now decided to be all fun and working, if you wanna come and see the robot, we can show you that too. But any questions before we wrap it up on the video part? Yeah, go ahead, yes. (man speaking faintly in background) Can you say it louder? No, and it's not because like, really, really no. And the challenge is, is that no one has tried to go into this space.
I think the folks that are looking at senior care had a fully different view of what a robot should be. So the people that have put a lot of money on it, they've put it on arms and actuators and trying to have a robot do the dishes and things like that. And you know, which is valuable.
Everybody from Toyota to Google is doing really like PhD and beyond great work there, but those are long term R&D projects and they're well aware away from productization. So, you know, the mission here is, is it's this intersection of, okay, what can you practically do that's gonna make a difference in people's lives and can be made in mass production and be affordable? And that takes everything that Nikolai talked about is first rule is we have to use only components from things that you can buy in Best Buy or Bed Bath & Beyond and then glue them together. And we're solving problems like in AR reality, the patent we just got for that was this persistency of the environment on those features. Well, somebody making a headset playing a video game, they don't really care that we're, you know, when you really re-initialize the headset on your room, did that have to be consistent with what it was like two years ago when it was trained in bright light and now it's dark. Those are problems they weren't trying to solve. So I think we went after an area to say, well, this was this logical gap that this 10, 20, $30 robot per month cost versus that $1,000 robot per month cost and that, you know, senior living doesn't have robots 'cause they can't afford 'em.
They can't afford what's in the hospital. So it's a long answer, but no, no one is competing with us yet and I think in a meaningful way there's some folks that are doing the commercial side for delivery robots like in restaurants and other places, but you know, and what we've put on here is really designed for, you know, it's the world of having a robot left alone, either with a family where they've got a five-year-old that may wanna jump on it and take it for a joy ride, or a cat. For some reason, cats with robots like riding them, or someone like my mom who's on blood thinners and can't have anything sharp that she could get hurt by. So I think we started from the ground up and said, "Hey, where is this issue that we need? What are those key performance requirements we need to solve, and then what are we putting into the robot on the design?" Like something you don't see in industrial robots is every hard surface on this robot on the outside is pressure sensitive. If someone leans on it, we know because we don't want it walking away while they're doing that. So there's just a lot of, it's just this, it's sort of a journey of complexity to make it simple and simple, simpler and simpler for dealing with those environments and I know you're probably, a good chance we announced at CES last year, you'll probably see some folks starting to put wood paneling now on robots.
Maybe that'll be the trend and they'll, they can easily make a boxy robot. No one can stop doing that. To do the stuff we're doing to make it reliable and do things every day, I think that's a lot of work and so that's where it'd be great to get some competition in there to sort of get more attention to it, but it really is an underserved market and that's where we're hoping to raise awareness around it. Cool, all right.
I think we have six seconds, five. That's gonna end it right there. If you have more questions, feel free to come up and we'll show you the robot, show you the sensors, run the demo up here in person and then talk outside too.
Thank you so much for inviting us and being here at re:MARS. (audience applauding) Thank you, thank you.
2022-06-30