Ready. To go okay we're gonna get started now just. Welcome. Everyone. My. Name is Paul, Stubbs I'm a director, of AI, marketing. I own, autonomous, systems sort of strategy, and. One. Of the things that it's really exciting this week to finally, announce and talk about the things that we working on and then, have you guys to help me understand, you know where your errors or interests are and. I'll talk about kind of some of the behind, the scenes efforts, that went on and how are we thinking about this moving forward this is definitely the beginning of, a, long journey that we're just starting this week how, many people have seen the robots over here and over, in the section. Everybody. One. Okay. So today. You, know we're living in this world that is, automated. Right, and sometimes. We, get confused at, what the words are so we're gonna kind of go through what how we're talking about these words so, we live in a world that's automated, in other words if, your, system, your robot, your. Manufacturing. Line can, do something, over and over again in a repetitive fashion and it. Does that very accurately, we consider that as automated. We're. Trying to move to a world that instead, is increasingly. Autonomous. Meaning. That it understands, the world around it's able to use. Sensors to perceive, the world around it it's able to plan. And act upon those sensors, and it's then able to take some, action, into that right so if you think about where like IOT, is IOT sort, of the first part there are sensors, and, maybe, there is some intelligence, there but it's really just about sensing, the world sort. Of autonomous, systems really about sensing. The world planning. Something and then actually manipulating, or doing. Something in the physical world and we're. In this sort, of fourth general. Revolution, they call, it right and so we had sort of you, know, mechanized. Systems. Through production, and manufacturing, then we went through sort of kind of where we are today which is things. That are powered by software, right they're automated by software's controlling, them they may also be manually, operated, right we have a lot of cases that's the case and. Really moving into this world where things are autonomous things, are happening, more. On their own their more independent. But. Autonomy, doesn't mean that. There's. Not a person in the system think. About all the things that we do today that are autonomous but, we don't really think about them that way, let's, think about drones, for example, why. Are drones so, popular, now. When. We've had RC, planes and helicopters, and, such for 30 or 40 years but, why now what's making them popular well I would argue what makes them popular is that I can. Just grab a drone and I can push the stick forward and the drone goes forward I pull it back and the drone goes back and anybody. Can do it well, what's happening behind the scenes is they're autonomous systems, onboard the drone that keep, track of the pitch of the blades and the wind and the engine, speed and all that sort of stuff so those. Things are happening I'm. Still in control as a human a person is still controlling. The drone but. Autonomous systems are in the background manipulating, the drone we, have these systems today for a long time right think about. Your. Braking system in your car right for the anti-lock brakes you. Push the brake down in your car, but. That doesn't really. Make the brake stop the, computer then decides, based. Upon the travel. Conditions whether, it's going to you know pulse the brakes or whether it's actually going to apply the brakes, that's, an autonomous, system on the braking level that enables. Average. Drivers, to drive in, conditions. That they may not be trained on snow, and ice and, rain right so that's an autonomous, system and so we we have lots of examples, where we're thinking about these things in ways that can, make the human more powerful, can take that human operator and do, things that they may not have been able to do on their own over. Here we have the Sarcos Guardian s it's the snake, robot, that's driving around the track and climbing the wall well, if, you manually operate, that that. Robot, it's got this big controller, and you, have some joysticks, but you're trying to articulate. A number of joints, right. And that's easy for a person do it's like driving car right you can drive your car but. If I put a trailer on your car. That.
Becomes A whole different problem right, and so you have lots of joints in the robots that same problem so if I put a trailer in your car that becomes more, difficult to drive probably. Most of us can do it backing. Up is a little bit harder but. What if I put two trailers, on your car now. How back how it's more it's a much more difficult problem for a human to do you, still want to back up the car but you don't know how to articulate two trailers backing up well. The robot is this set this same example except there's five trailers, right, and so trying to manipulate five, trailers, going up over the arch and over stairs becomes, very difficult for, a human operator but, the human operator is still in control because they can just push forward and then when it hits stairs it knows how to go over the stairs and you don't have to worry about all of that detail so, this is kind of the space that we're talking about when we think about, autonomous. Systems. So. We've been thinking, about this in sort of three big buckets right really, the goal here is to scale, human expertise, we, want to be able to leverage the knowledge of, an. Expert, operator, and have, them be able to create. These AI models. Using. Reinforcement learning which I'll show you some of that in a little bit to, then manipulate, and create these things right so we want these people that may be expert. Bulldozer, drivers, to be able to take the knowledge they have about driving the bulldozer and apply. It to, an AI model, they're not data scientists, and data, scientists, don't know how to drive a bulldozer and the bulldozer doesn't know how to drive, AI right and so this, is where we can help them really, trust where the autonomy right this is about being. Understandable. Transparent. Explainable, running. On the secure cloud platform, things like asher and on windows to, be able to have. These good, foundation. For your autonomous, systems and then, real-world scenarios, right a lot of the other. People that are doing things in the RL space or in the autonomous space are, doing it with games and. Video. Games and real board games and things like that which is interesting but. Really we're focusing, on real-world scenarios like real industrial. Automation. And autonomous, scenarios, that can do things like mining. And farming and oil and gas and, HVAC. And all, of those kinds things right those, are the sort of the scenarios were targeting, and so I think this becomes very interesting and, very relevant for a lot of people. So. The five categories. That we're thinking about first and this isn't this. Isn't like the. Entire space but it's a wave to help you think about the problem space a little bit more one. Is around motion, control this is around how do you manipulate the motion of an object in the, case that I'll show you is a horizontal, oil drill right so how do you fly this. Horizontal. Oil drill underground, as if, you would fly a drone in the air except. I'm flying it through for example shale. Rock right. Our. Smart building scenario is very interesting, this is around how do you take a very complex. System. Like. Modern, buildings, that have all, sorts, of sensors, and apply, an AI model, that can train that to be more efficient and I'll show you the demo of that later machine. Calibration, we, have a lot of scenarios where. Imagine. A CNC, machine right so how, many people know what a CNC machine is. So. A 3d. Printer will build something by adding material, a CNC, machine machine, builds things by taking material away right, it's a good way to think about it but over time those, blades, as it cuts get, worn out and then the Machine gets out of calibration because it needs to be in a certain spec and the blades smaller so it gets out of calibration, normally.
That Happens and, people stop and it takes them a few hours to recalibrate the Machine we, can do sort of what we think of as continuous, calibration, where the AI is driving. The calibration, so that it continually stays in calibration process. Control, when you have complex, processes, maybe, you have thinking. About that like managing. The luggage at the airport or, you're. Producing, some product, like maybe shampoo, and that chemical, process, that you have to get all those in right we can make that process a lot more better with autonomy and then, finally this is sort of the canonical example and what we've been showing at the booth which is industrial. Robotics, right and, so the point of all of these things is to make you think differently about when. We're talking about autonomous, systems we're, not talking, about autonomous, robots, or autonomous, trucks or autonomous, planes but, the whole system itself and so, think about that system could be a component, like the brake system you, are a blade. On a bulldozer, it could be the whole thing maybe a floor washing, machine could be autonomous it, could be something, simple like an autonomous. Greenhouse right where the greenhouse has sensors, that, can sense and perceive the world it can open and close the windows and turn on the fan we, think of that as an autonomous system as well so when, we think about these it's more than robots or the robots are pretty cooled all right. Now. Here's sort of the toolchain. Approach to how we would build the the intelligence, for these devices, using. Machine teaching. So. The first piece we have is an, expert, so you have some expert, that can work, with your data scientists, to craft. The. The. Training, it's. Written in the program called inkling so you create this inkling file that, is that, codifies, basically. Your information, about the problem what, the lessons are how you decompose, the problem into smaller steps, because. When we were doing something like the, robot manipulating, a block we, don't program we don't teach the robot to take. The block and build. A house right, there's lots of steps in between and this, is where the expert can come in and decompose, those steps into smaller things right. I think the classic example, we've been talking about is about teaching. Your child how to hit the baseball like. With traditional, machine learning you may send them out in the backyard where the baseball and, the ball and say there's. A baseball and a ball in the backyard go. Figure out what you want to do or, maybe you could sit them down in front of youtube and watch major league pictures, hitting. Baseball, and say hey here's millions, of videos go learn how to hit the baseball right but that's not really how kids, learn right, you learn by breaking, the problem down into smaller things okay first I'm gonna teach you to swing then I'll put the ball on the tee and then I'll teach you to hit it from the tee then I'll throw it under each one of those become a discrete, learning. That. You do, in essence in your, file that you define to set up your model then, once you set the model up you're ready to begin training, now, the way that reinforcement, learning works, is that it's based on a reward system. So. When you do something right you, give the system a reward and then when it when it doesn't right it doesn't get a reward and over, time the system learns. To, maximize. The rewards, right because it wants to end with the highest reward now. The. Way this works is that you would in simulation. Create. An environment, where you can simulate your, your, real-world happenings. And. Then in that case you pass the state information so, here's where my robot is and here's where the stairs are right imagine, you were trying to teach the robot to move to the stairs so. You say here's the robot here's the stairs so the goal is the stairs that's what you would have defined in your your. Teaching. File, so. Then at that moment you pass that information to, the brain to the model, here and say. Hey here's. Where I am and then. The model will return an action, to you say hey move, left, in. Fact it'll really be a vector right so I'm gonna say move left at this speed right but it's say hey move left so then you do, that action in simulation.
And Then, evaluate. What. Happened was I was, i closer or farther way so this is where you generate the reward okay. Plus one I'm getting closer to the stairs and you continue. That process until, the robot. Reaches. The goal which is to arrive, at the stairs. So. Does anybody see a problem, with this process, so far. So. The difficulties, in this system are, really. Evaluating. What. The reward, is because. You. Know kids right if you incentivize. Them the wrong way they're going to try to maximize the reward or, how. Many people here have been. Creative. With their scorecard, for sales in their company right and you have a scorecard, you try to say well maybe, this is included so you're really gaming, the system here so, for example when we were training the the, snake, robot to go to the stairs you, would get the points a maximum, number of points when you reach the stairs so, what the robot did was that came up to the stairs didn't, quite get there got a lot of points and then did a loop around the stairs to get more points and just kept circling the stairs because it, learned hey I can get a lot of points if I just keep going around the stairs but that's not what we were going for so you have to kind of think about what, you're trying to teach it and what the reward structure is so that's an important step, once. That's done, and you do this in simulation, because you can scale and you. Can you, can simulate. Events. That may never occur or a dangerous. Or different. Environments, you can do all that in simulation, once, you're done you can take that model and deploy, it to the edge and, then, run that same process on a real device, so. In that case that's what's running over there on that device we've deployed it to the device and it runs right on the device where instead. Of getting reward now all I'm doing is saying here's where I am and tell me what to do next and it says okay now go left it's okay now here I am now go right and that's, what's happening, and this, process. Here, is like, in gaming it's a game loop you want to do that as fast as you can you want to do this 60, times a second right this is what you want to do when, you are doing this in real life. And. So we have three scenarios that you can do here and you can actually go do these right now at akms, /a s demos and try. These and, I'm gonna show you these what these look like so you know. So, here we have the three demos let me go back and. Show you what these look like. So. These are awesome these are awesome actually demos these really, teach you conceptually, what you're doing here and this is one the smart building example, we, want to be able to maximize the, air the, comfort, and quality for the people while, reducing, energy costs, and reducing, wear and tear on the equipment and in which case we have lots of variables, this is the state information, you. Can imagine in a real scenario you'd have, dozens. If not hundreds. Of variables that you would pass as your, state but, here we simplify, didn't have just a few so we passed these variables to the state and say okay now here's the temperature what I do here's the temperature what I do and it would return some action, you know open. The window close the thing turn the fan on and, turn. The heat up right like that maybe the action that returns then, you say okay now it's still too hot and then so you keep, going through that process right so this is what we do here and what, you can see in the system when you're training the system you'll, have a. Number. Of the, variables, coming in you can see your co2 levels, you can see your energy consumption, and as, you go through each each, run, we call these episodes an episode. Is a series, of iterations. So an iteration may be one loop right, an episode. May be I'm gonna train it to go from here to the corner. And. So, you can see over time it's starting to maximize. The reward. And. Then in the end once you have the Train brain hopefully. It'll, it'll. Operate at, a more consistent, rate. Than maybe an expert operator, might. Another. Really great example here is this one around motion control and here. In this case what we want to do is we want to be able to stay, inside of, the oil pocket, inside of the plan that's been defined and, we, want to go as fast as we can right and the, reason we want to stay inside the plan is that the. Plan is something that's that's defined, by a scientist, he says okay stay here at this depth because it's 3d space right because. You want to avoid other, people's property or maybe you don't have land rights, or maybe there's a natural, boundary. Or, maybe there's other pipes or wires underground, so you want to stay here and so this red line represents that, the true plan that you want to do and the blue line represents our, sort of training path, now.
What We do here is in the beginning we started a random distance and location so then the robot the drill, head has to figure out how, do I get back on path and then, over time as it, goes through these in these episodes you can see that eventually we found, the optimal, kind, of solution, and the, reward, function flattens, out and we know that we're trained at that point and we, can deploy the model and. So. In the end result we end up with a. More. More precise, and a faster, solution to to, drive. That that oil head. Finally. The last piece of this I want to talk about is really. Simulation. And simulation. Is so important, to this process, because, of the, things I talked about earlier you can you, can do things and test scenarios, that may not exist in real life but. Not only that like if you're testing on your real robot or real device those, things have physical, limitations, to the number of cycles right your robot runs for X, number of hours before it breaks so you don't want to break your expensive. Equipment testing, it so you run it in simulation. Also. The other thing about simulation, is in real life in IRL, for you younger kids in IRL. Time. Is one second, equals one second. Right. So if you want to train for a thousand, hours you've. Got a lot of overtime coming. Here. We can train it where we have hundreds. Of thousands, of simulators spun up at the same time and we can train in parallel a much faster rate and, we. All sorts, of simulators right now in the marketplace there are hundreds of simulators, for, every, vertical and, the healthcare industry has simulators, for every part of the body the, oil and gas has similar for everything, Electrical. Engineering has, similar is for everything we. Have simulators, for things like air, drones. And vehicles, and, so all of these are our, simulated, systems, that, can be applied to. Not good. That. We can see here this case this is err sim this is our first. Party simulator for doing, drones. And, vehicles, you, can use either unity, or Unreal. Engine to build these import. Your scenes just like you would any game you can just import standard. Environments. And simulate. All of these environments. Directly, inside of the the. Tool and generate. Training so you can imagine like I said it's like a game right every, frame you're sending the current state to the AI model, and it's telling you to move the car left to right or the drone up and down and you do that in this simulate environment, and so, here you can simulate things like traffic you'll, be able to do lighting conditions, you, can also take photos of, these objects, in here too and use those as training your vision. Systems right so if you want to also capture images, for the vision system you can do that as well we, can do things like snow environmental. Conditions, wind. Leaves. Also, it's a different weather, that, you wouldn't be able to simulate necessarily. Unless you hang around a way from winter again to simulate your car so. We can do all of these things in the simulator, and, it makes it really really, good. We, can also simulate things like sensors, right so if you want to test out sensors in this case lidar if you want to test out a sensor, you could test out sensors, in here you can get different perspectives, maybe you want to see what the driver sees and have, trained, the vision, system or the a model from that we can also look directly down we, can also have multiple actors, so we think of this as fleets or swarms, of robots. Or devices right in a factory you may have multiple robots on an assembly line or you, may have a whole fleet of window-washing. Robots, that you send out to your building we can do this working with a customer to do. Windmill. Inspection. So we can now strain it how to get close to the windmill. And. Finally just, want to leave. You, with some links. You. Can go to aka.ms/offweb. You. The. Demos that I just showed you those three you can go explore those or more details. Learning. We have a full learning course that we just released that teaches you at a 100. Level 200, level, what. Is machine teaching what is reinforcement, learning what, is simulation. How does that work what, is autonomous, system so all the things I kind of talked about today in more, detail and then, finally you could try out some of these yourself with the lab so if you go to the lab you can download some of the source code that I built the demos. With and run, those and. With. That thank you very much and if you have any questions please come up front.
2019-05-10