The promise of autonomous vehicles has been a long time coming. While many are still waiting to see the fruits of all this work, there are some cities like Arizona and San Francisco where autonomous cars are starting to become a reality.

Furthermore, IDTechEx’s new industry report “Autonomous Cars, Robotaxis & Sensors 2024-2044” predicts a coming rapid growth in the number of cities that will offer robotaxi services in the next few years. So, with robotaxis rapidly becoming an everyday reality, the industry and experts must ask, are autonomous robotaxis safe enough?

This summer, the robotaxi industry has seen more commercialisation activity, with both Waymo and Cruise being given the green light by the California Public Utilities Commission (CPUC) to expand their commercial services in San Francisco. But only weeks after that announcement, San Francisco has seen protests around the deployment of autonomous vehicles, and California DMV has halved the number of vehicles that Cruise is permitted to have in testing. Some inhabitants of San Francisco are becoming disenfranchised with the city’s perpetual status as a proving ground for this technology, with a group called Safe Street Rebel leading the protests. Their disruption mechanism is called coning and involves placing a traffic cone on the bonnet of autonomous vehicles, rendering it inoperable until the cone is removed — a somewhat embarrassing situation considering all the vehicles’ technology.

So, are autonomous vehicles really that unsafe and not ready for the news, or is this protest more about the city’s technology testbed status? Waymo claims on its website that it outperforms human drivers when mitigating and avoiding collisions, but what does the data out of California say?

Autonomous vehicle safety is an area that IDTechEx’s autonomous vehicle experts have tracked closely and carefully as autonomous car testing has proliferated. IDTechEx uses data from the California DMV to understand how autonomous vehicles are performing and improving over the years. When assessing the safety of autonomous vehicles, several metrics can be considered: how many testing miles has each company amassed, how often does the safety driver need to intervene with the autonomous system, and how often does the autonomous system cause a crash?

A key metric that IDTechEx uses to monitor autonomous vehicle safety is miles per disengagement. This measures how frequently, or hopefully how infrequently, the autonomous vehicle safety driver needs to intervene with the autonomous system. IDTechEx has measured this since 2015 and has seen exponential growth in the performance of autonomous vehicles. Back in 2015, Waymo recorded 424,000 miles of autonomous testing, during which its safety drivers disengaged the system 341 times, meaning there was an average of approximately 1,200 miles between disengagements. Waymo were the best company by this metric that year. For reference, IDTechEx estimates that human drivers in the US average approximately 200,000 miles between collisions. If it is assumed that each of Waymo’s disengagements would lead to a collision, which is slightly unfair against the autonomous driver, then it would be around 0.5 percent as safe as a human driver.

However, the autonomous vehicle industry has made significant progress since then. In fact, IDTechEx has since the number of miles per disengagement nearly doubled year on year.

In 2022, Cruise were the leader when it came to disengagement performance, with a score of nearly 96,000 miles per disengagement, nearly 50 percent as safe as humans. During its 863,000 miles of testing, safety drivers only needed to intervene with the system nine times. As part of IDTechEx’s research they look closely at the disengagements and collisions in which autonomous vehicles are involved. Doing so uncovers a surprising fact: four out of the nine disengagements were caused by the poor performance of other nearby drivers. If these are removed from the equation, then Cruise’s miles per disengagement score shoots up to over 170,000, 85 percent of the way to the rate at which humans have collisions.
Miles per disengagement is only a proxy for autonomous vehicle safety, though. Since a safety driver has intervened, it is impossible to know whether the car would have collided or not. Instead, perhaps the number of collisions that autonomous vehicles are involved in should be considered.

Between January 2019 and May 2023, the autonomous vehicle companies testing across California submitted more than 450 collision reports. These reports cover a wide range of collision types, from collisions with other vehicles to hitting curbs and even the vehicles being attacked by pedestrians. IDTechEx found that only 3.4 percent of collisions could be attributed to the poor performance of the autonomous system. Another way to look at it is that in 2022, the autonomous driver would cause collisions at a rate of one collision per 1.3 million miles, significantly better than human drivers. But this is with a human behind the wheel monitoring the system. What about when the system has no human safety net? How much do they collide then?

Since 2020, California has allowed driverless autonomous testing on its streets, and two companies have taken advantage of this. Waymo and Cruise. Between 2021 and 2022, Waymo has recorded just under 70,000 miles of driverless activity. On the other hand, Cruise only started recording driverless miles in 2022 but submitted a staggering 590,000 miles. During those miles, the vehicles were involved in 15 collisions, i.e., one collision every ~40,000 miles, or 5 times more often than their human counterparts.

One point of redemption is that these miles were exclusively accumulated in San Francisco, one of the toughest driving environments in the US for autonomous systems. But also tough for humans. With the slower speeds and increased pedestrian presence, IDTechEx estimates that the collision rate amongst human drivers increases from one per ~200,000 miles (the US average across all road types) to one in every 107,000 miles, only half as good, but still better than autonomous drivers.

There is one other statistic that should be considered when talking about the safety performance of autonomous vehicles. Of those 450+ collisions recorded by the companies testing autonomous cars, none involved a major injury or death. In the four years of testing, from 2019 to 2022, that is nearly 14 million miles without a serious injury or fatality. NHTSA say that with human drivers, a fatality happens roughly once per 75 million miles of human driving. So autonomous vehicles still have a way to go to catch up, but it is looking promising.

Whether you look at miles per disengagement, miles per collision, or miles per fatality, humans still have a better track record than autonomous vehicles. However, human safety has been fairly stagnant. The rate at which we crash is not changing that much, and further improvement is mostly coming from crash mitigation technology, such as automatic emergency braking systems and blind spot detection. One thing that can be said for autonomous vehicles is that their safety has been improving at somewhat of an exponential rate. Something that humans are very unlikely to mimic. IDTechEx does not believe that autonomous vehicles are as safe as humans yet, nor are they ready for widespread unsupervised deployment. The rate of improvement that autonomous technologies have shown demonstrates that there is the potential for them to far exceed human levels of safety in the future, leading us toward a world in which we stop questioning whether autonomous cars are ready and start questioning whether human drivers are safe enough.

To find out more about the IDTechEx report “Autonomous Cars, Robotaxis & Sensors 2024-2044”, including downloadable sample pages, please visit www.IDTechEx.com autonomouscars.
 

Author: Dr James Jeffs, Senior Technology Analyst at IDTechEx