Waymo announced they have successfully completed an external audit of their “safety case,” including their remote assistance system. A “safety case” is a detailed document outlining all the procedures and practices to assure safe driving. The audit was done by TÜV SÜD, part of a well known European based safety laboratory. It declares that Waymo has been following good and established practices in keeping their vehicle safe. While not a guarantee of safety or statistical analysis of it, it demonstrates Waymo is doing the expected things seen in the safety field.
This isn’t Waymo’s first external audit. Last year, they allowed Swiss Re, the large Swiss reinsurance company, to analyse all of Waymo’s safety incidents, and classify them as to which would trigger liability (from an auto insurance liability policy standpoint) and how much. They did rather well in that study, with one liability event every 2.3 million miles, which is much better than the typical human. (Not all crashes are liability events. For example, if a Waymo hits a curb and damages itself, that would be covered by collision insurance, not liability insurance. If the vehicle had no fault, it would not have liability. In addition, we could see a robot getting sued for actions which would not trigger that for a human driver.)
It’s very good that Waymo is doing this. Calls to other companies to do the same seem to be received with silence. In particular, Tesla is notoriously close to the vest with data about safety performance of their FSD, Autopilot and Robotaxi systems. (Tesla does release highly misleading data each quarter about airbag deployments with the Autopilot system.) While Tesla’s current robotaxi pilot is supervised, rather than autonomous, many people have asked Tesla to provide data on the safety performance of both the robotaxi and their FSD system in private cars.
No data has been forthcoming, and while all companies are required to report crashes to the federal government, Tesla is the only player who removes as much data as possible from their reports, so only the most basic details can be seen. For a decade, Tesla has avoided the required reporting of “interventions” by safety drivers to the California DMV by saying that their “Full Self Driving” system is not a self-driving system, and thus is not required to report. Some private Tesla drivers try to gather data on their own FSD use, but they are a small group. Most recently, they reported seeing a “critical intervention,” where a driver intervened to prevent a significant safety event or crash, about every 1,400 miles with the newly released FSD version 14. This is arguably 1,500x worse than Waymo’s 2.3 million miles, but liability events and subjectively judged critical interventions are fairly different things. Tesla should work to make it easier to understand.
It is the duty of every robocar team to work towards good safety and minimize unacceptable risk, but it is also their duty to provide data to convince the public they have done this. Waymo is the only company that gets any sort of star here, the others all need to step up their game, and by a lot.
Remote Assist
The audit also examined their remote assistance process. Waymo’s remote assistance is quite different from the supervising safety drivers found in Tesla vehicles and many others. Waymo does not monitor vehicles full time, but connects a remote operations staffer when the vehicle feels it could use human advice and context. The remote assist operator gives strategic advice, such as “turn around” or “take the next left exit” and does not drive the car. Waymo says the recently added the ability to do low speed remote driving which will be used only in freeway operations, when they need to quickly clear a vehicle off the road. Removing full time supervision is the biggest, hardest step in making an actual robocar/robotaxi; it’s not actually autonomous until you do.
Waymo disclosed that remote assist can happen while moving at low speeds, but the vehicle is always able to safetly resolve the situation if the remote assistance doesn’t come, as such they do not require super-low-latency communications (which is good because the cell network can’t always provide them.)
Meanwhile, Waymo growth continues. They recently announced expansion next year to Las Vegas, San Diego and snowy Detroit, and Miami, Washington DC, Nashville, Denver, New York, London and Tokyo are all undergoing testing. In San Francisco, Waymos are now an extremely common sight, you will see several per minute in heavy traffic areas. Estimates predict they are now over 350,000 rides/week and over 100M miles lifetime. (Apollo Go in China just announced 250,000 rides/week there.)
In other news, a San Francisco supervisor has put forward a non-binding resolution to once again ask California to permit cities to regulate robocars within them. There’s a good reason why this is a state matter. Having every city with different robotaxi rules (and bans) would crush the technology which, at least according to Waymo’s audits, is reducing road risk compared to human drivers. The San Francisco action was triggered by a Waymo killing a well loved bodega cat in the SF Mission district. While sad, I can personally report at least 3 of my cats have been hit by human-driven cars, so this is hardly the basis upon which to ban the robots unless there’s some sign they are deliberately or negligently driving with a disregard for pets that is worse than humans.