Erez Dagan: The Key Technologies for Unlocking AVs at Scale
Thank you for the opportunity to speak at the Car of the Future summit by Reuters. In my short talk today I will share with you what we at Mobileye identify as the core technological enablers of autonomous driving at scale. At Mobileye we have been harnessing computer vision and machine learning to promote driver assistance and road safety since 1999. Today our three major business pillars are ADAS, the bread and butter, crowdsourced AV mapping, which targets autonomous driving, driver assistance, and smart cities markets and our self-driving full-stack solution targeting the consumer AV and mobility as a service markets. The combination of all three cortec engines is imperative to unleashing AVs at scale. The first
element is the RSS formal safety model. It's an explicit digitally interpretable and enforceable model for the safety of the decision making of the autonomous vehicle. The second principle is the realization that the AV-ADAS divide is not scope of capabilities, not a different scope of capabilities but rather the mean time between failures in executing those capabilities. We are relying on what we call True Redundancy™ - truly redundant perception subsystems which perceive the environment in parallel to one another and allowing us to both perceive the environment with low failure rate as well as prove that our system has lower failure rate of that of a human driver which allows us in turn to take the driver into a eyes off mind off position. The third element is enabling or unleashing seamless geographical scalability. Autonomous driving has to be deployable
anywhere and in order to make that a reality we've designed a crowdsourced mapping solution which utilizes the plethora of driver assistance cameras that are traveling out there to harvest information and build automatically high refresh rate AV maps to cater for that interest of seamless geographical scalability of the autonomous driving system. What do we mean by True Redundancy? Having two perception subsystems, one comprised of cameras only, and the other one comprised of radar and lidar sensors only, both creating a comprehensive view of the environment covering the four elements that constitute the environment the road users the road boundaries the road geometry and the road semantics. Being able to do that independently in two subsystems gives rise to both the very low failure rate or high mean time between failure that our system yields as well as to the provability of our systems - our ability to prove that our system is exceeding the capabilities of a human driver in terms of a failure rate. Having
two independent subsystems proven on order of magnitude of 10,000 hours each would give rise to a proven safety case of the perception system, of the overall perception systems which is the square around 100 million hours of driving. Coming to design a computer vision subsystem which meantime between failures is at the order of 10,000 hours is not a trivial undertaking. It entails a combination of several standalone mutually informative computer vision cores.
One of these cores is the vidar which produces a high angular resolution and high accuracy depth perception around the vehicle utilizing the multiple cameras placed around the vehicles and their overlap regions. This standalone computer vision engine is of course independent of the more classical pattern recognition methods that we deploy in the surrounding of the vehicle which perceive the vehicles based on the objects of the environment based on their appearance rather than their depth. Another such standalone comprehensive perception engine is designed to directly yield a semantically explicit representation of the host surroundings. As you could see in this slide,
it outputs a top view of the environment road users, road boundaries and even the velocity of vehicles around the host. The resulting computer vision subsystem is actually a product of itself. This is the SuperVision™ product which we designate as a premium driver assistance solution. It is a stand-alone end-to-end stack comprised of computer vision REM™ and mobilized driving policy in RSS layers. The Road Experience Management technology or REM is mobilized crowdsourced AV mapping technology which is designed to seamlessly enable AV everywhere. The technology is comprised of three core stages - the harvesting, the aggregation
and the localization. By harvesting we refer to the process of perceiving the environment through single camera equipped vehicles or driver assistance vehicles, identifying the road geometry and road semantics around us in a way that allows us to transmit with very low bandwidth this information up to the cloud. We interpret the road geometry and road semantics out of the video stream, pack it in small packets of 10 kilobytes per kilometer and shoot them out to the cloud for aggregation of a map. The second stage, the aggregation of the map is taking these snippets of information turning them into a coherent map of the environment - AV map, which contains the road geometry and road semantics with a very high level of accuracy as well as landmarks which will then be used to localize the vehicle within the map, which brings us to the third stage which is localization. An autonomous vehicle or a premium ADAS vehicle such as the SuperVision system
would consume the RoadBook by pulling it from the cloud, identifying landmarks in its vicinity, localizing itself inside of the RoadBook and then gaining that electronic horizon with rich semantics that the AV map has to offer. Having the crowdsourcing technology rely on camera only agents is critical - it's critical for the scalability that we just mentioned but it is done while not compromising on the accuracy. The accuracy of the map is actionable we can drive a vehicle, we can demonstratively drive a vehicle based on the on this map. The road geometry nuances as well as the semantic nuances are well captured to provide a full view of the electronic horizon as it's called. The map is not only accurate - it entails rich semantic layers which leverages both explicit and implicit cues captured by the crowd which allows us to generalize to the driving cultures and traffic rules across the globe. Such semantic information
for example could be the common driving speed, the stopping points within a junction through which human agents are investigating the junction without taking unnecessary risks and could also be the association of different traffic lights to different lanes. Our large harvesting fleet today allows us to harvest 0.7 billion kilometers of road globally, 8 million kilometers of road covered daily. By 2024 based on the set of agreements that we have with our partners we foresee one billion kilometers of road to be covered on a daily basis. Truly redundant
perception system in combination with the AV maps yield a robust model of the environment, covering the road users the road geometry the road boundaries and road semantics. A faithful model of the environment is not sufficient and to safeguard the vehicle decision making from causing an accident we have formulated RSS - an explicit model for a road user's duty of care. Beyond complying with the RSS contract in our own systems we are driving its standardization across the globe. The resulting robustness, geographical scalability, safety and agility of our AV system is not theoretical. It is clearly demonstrated through
our AV deployments in multiple cities across the globe, starting in Israel, Munich, Detroit and shortly upcoming in Tokyo, Shanghai, Paris and New York. Thank you all for your time today. Look for our AVs on the streets and feel free to reach out to continue the conversation. Thanks Erez for that presentation that was brilliant if may I ask - how do you envision the arrival of autonomous vehicles playing out in other words how long until we will be able to actually ride in or purchase a fully autonomous vehicle and what needs to happen before that dream comes true? Great question so there are two streams that are going to bring about autonomy to the end users - first one is robotaxis where a user simply can ride an autonomous vehicle. We are planning to launch such a service already by 2022 next year and we see the industry probably opening up in a larger scale for robotaxis in around 23 24 time frames.
The important next phase of autonomy which already has some sprouts is the introduction of consumer autonomous vehicles. Right now it's consumer autonomy in certain restricted function restrictions or OBD restrictions such as on highways or traffic jams and ultimately this will get us point to point autonomously. Now as for the question of what needs to happen to make that a mass-market reality - of course there are several factors I think primarily I would denote the regulatory discussion that needs to take place, I think I also regarded it in my talk - as for the RSS that contract between the autonomous vehicle and the society that's using the street along with that autonomous vehicle - there has to be a very clear interpretable contract of what is a safe negotiation of road users. Right now this contract is very vague in many ways. It's common practice of humans but expecting computers to take part in that
contract we really need to formalize it and make it something that that the vehicles can rigorously follow to have that market acceptance and all of the safety and efficiency benefits that autonomy can bring in. I think the second challenge that we need to overcome is a geographical scalability - how do we introduce autonomy everywhere not just in the pre-designated geo-fenced areas or roads. Okay, and for instance, while the world waits for AVs to arrive, are there other ways in which the industry and the public could benefit now or in the near future from the technologies being developed for self-driving vehicles? Most certainly - actually Mobileye takes pride in the fact that we trickle down - a lot of the value that we produce while developing the autonomous driving technologies we trickle them down into our driver assistance proposition. For example, our mapping was originally intended to cater only for autonomous vehicles and today is already marketed as part of premium driver assistance systems, and even the RSS model that I talked about which is that contract between the vehicle and the other road users is already being deployed as part of a driver assistance system, so imagine a world where this contract that the autonomous vehicle that was developed originally for autonomous vehicles can now safeguard the conduct of human drivers so a human is fully in control but there is a very clear notion of what is the boundary line between assertive driving and dangerous driving and the vehicle can inhibit certain actions to make sure that the human driver does not overstep and violate that important safety contract, so that's another example.
There are many other examples that concern our perception systems that evolve in giant leaps towards autonomous driving and these values were migrated into the driver assistance arena offering advanced functionalities such as animals detection, free space detection, the REM mapping that I mentioned earlier and then that safety contract of what actions are safe versus actions that are not safe on the most comprehensive sense. Okay, another thing, you mentioned REM, Mobileye seems to be taking a clean sheet approach to mapping for AVs. In what way do REM and the resulting Roadbook differ from other mapping solutions? Excellent, so first and foremost the fact that it is crowdsourced allows us to benefit from a fleet of millions of vehicles equipped with a single camera, no other sensing device, in order to continuously update the map. A map - critical attribute of a map - is that it has to faithfully reflect the reality. If there is a change or a construction in the road that happened five hours ago, the only way to get a map which is faithfully reflecting reality is to have a crowd-sourced approach to it, scanning the roads continuously and updating our understanding of the road structure. So that's the first element - the fact that it's crowdsourced allows that
high refresh rate and seamless geographical scalability anywhere that our driver driving assistance cameras are traveling is being mapped on the way. The second element of the map has to do with the second unique aspect of our mapping approach has to do with the semantics that we are extracting. When you are driving on the road there are explicit cues such as the road lines and the lane marks sorry or the pavement structure or the traffic signs next to you which are very explicit but there is implicit semantics that is very critical to the decisions while driving and I'll give a few examples - the association of which traffic light belongs to which of the lanes seems like a very simple assignment problem that humans solve very easily but when you really try to make a computer resolve that problem in real time you run into difficulties. What a map can do is encode this
information from the crowd that travels the road and understand which lane corresponds with which of the traffic directives - that's a very critical element. Another nice example is when a human is investigating a junction, entering a junction to investigate whether a vehicle is coming, there is a very specific pattern of how do we proceed into the junction while minimizing risks and maximizing our visibility and that pattern of the places in which human drivers are stopping within certain complex junctions is very valuable semantics that we can harvest from the crowd and then have very useful for the autonomous vehicle. So the approach we call them not HD maps but rather AV maps - this category of the crowdsourced rich semantic maps. Okay, but Erez, Mobileye is not the only company pursuing self-driving technologies, right, how does Mobileye's approach stand out from other well-known players in this field? What factors give Mobileye an edge over the competitors? So I won't repeat - I'll just denote the ones that are already mentioned so the REM mapping is one critical factor, the RSS which is the decision-making or driving policy contract of safe negotiation, that's a key factor that enables both the safety as well as the agility of our vehicles, so sitting in the vehicles you will experience a very human-like drive because the boundary line between assertive and dangerous is very clearly formulated and we can maximize the assertiveness and agility. Another factor which is critical and very very fundamental to our approach is the True Redundancy - the True Redundancy between having two perception systems completely independently perceiving the world and giving us both high robustness of our environment model, the environment model is correct to a higher degree and with a lower failure rate and in addition it gives us the ability to validate our perception system with a much more pragmatic approach since because we decouple the system to two independent systems we can validate each of these systems independently to a much lower validation criteria. If one system fails, each system fails every one
thousand hours, both systems will fail every one million hours. And that's a very strong, strong indicator of robustness and validation method. Okay, well thank you very much Erez, I really appreciate your time. Unfortunately, we've now run our time and I wish we had a bit more to carry on and now we're gonna go to a quick five minute break and we'll see to just to our last panel Boosting Safety, Integrating Sensors and Technology. Thank you very much Erez
2021-07-08 12:48
