The National Transportation Safety Board on Thursday provided new details about a March crash in Mountain View, California, that claimed the life of engineer Walter Huang. The Model X had its Autopilot driver assistance system engaged, and, according to the NTSB, the car “began a left steering movement” seven seconds before the crash that put it on a collision course with a concrete lane divider. Then, in the last three seconds before the crash, “the Tesla’s speed increased from 62mph to 70.8mph, with no precrash braking or evasive steering movement detected.”
This isn’t the only recent case where Autopilot steered a Tesla vehicle directly into a stationary object—though thankfully the others didn’t get anyone killed. Back in January, firefighters in Culver City, California, said that a Tesla with Autopilot engaged had plowed into the back of a fire truck at 65mph. In an eerily similar incident last month, a Tesla Model S with Autopilot active crashed into a fire truck at 60mph in the suburbs of Salt Lake City.
A natural reaction to these incidents is to assume that there must be something seriously wrong with Tesla’s Autopilot system. After all, you might expect that avoiding collisions with large, stationary objects like fire engines and concrete lane dividers would be one of the most basic functions of a car’s automatic emergency braking technology.
But while there’s obviously room for improvement, the reality is that the behavior of Tesla’s driver assistance technology here isn’t that different from that of competing systems from other carmakers. As surprising as it might seem, most of the driver-assistance systems on the roads today are simply not designed to prevent a crash in this kind of situation.
Sam Abuelsamid, an industry analyst at Navigant and former automotive engineer, tells Ars that it’s “pretty much universal” that “vehicles are programmed to ignore stationary objects at higher speeds.”
Most adaptive cruise control systems ignore stationary objects
To understand why these systems behave like this, it’s helpful to keep in mind how they evolved. About 20 years ago, carmakers started offering adaptive cruise control systems on their high-end cars. Most of these systems used radar (a few early systems used lasers) to detect the position and speed of the car ahead of it and maintain a safe following distance.
Radar has low angular resolution, so it had only a crude idea of the environment around the vehicle. What radar is quite good at, however, is figuring out how fast objects are moving. And so a key strategy for making the technology work was to ignore anything that wasn’t moving. A car’s radar will detect a lot of stationary objects located somewhere ahead of the car: these might be trees, parked cars, bridges, overhead signs, and so forth.
These systems were designed to work on controlled-access freeways, and, in the vast majority of cases, stationary objects near a freeway would be on the side of the road (or suspended above it) rather than directly in the car’s path. Early adaptive cruise control systems simply didn’t have the capability to distinguish the vast majority of objects that were near the road from the tiny minority that were on the road.
So cars were programmed to focus on maintaining a safe distance from other moving objects—cars—and to ignore stationary objects. Designers assumed it would still be the job of the human driver to pay attention to the road and intervene if there was an obstacle directly in the roadway.
About 15 years ago, companies started offering lane-keeping assistance systems in addition to adaptive cruise control. But while the combination of these two features might seem like a “self-driving” system—after all, the vehicle is controlling both the steering wheel and the gas and brake pedals—it was far from an integrated system.
Abuelsamid points out that a car’s adaptive cruise control system was often a totally separate system—made by a different supplier and using different sensors—from the lane-keeping system. Adaptive cruise control systems are often radar-based, while lane-keeping systems more often use cameras. The two systems don’t necessarily even share data, and on many cars they don’t do any kind of sophisticated path-planning.
And this means that even if a car had a lane-keeping system with some understanding of the stationary objects around the vehicle, that understanding isn’t necessarily shared with the adaptive cruise control system—which is the system that decides whether to hit the brakes. So many cruise control systems on the roads today continue to operate the same way the first ones did 20 years ago: they’ll slow down if a moving car ahead of them does. But they won’t even try to prevent a collision with a parked car, concrete barrier, or other stationary object in the roadway.
Many automatic emergency braking systems won’t stop for stationary objects at freeway speeds
You might have expected this to change in the last decade as cars have gotten increasingly sophisticated automatic emergency braking systems. These systems are designed to do more than adaptive cruise control, detecting impending collisions and slamming on the brakes to prevent or at least mitigate them.
But these systems come with a major caveat: for the most part they’re only designed to work at low speeds. If you try to run a car with AEB technology into a parked car at 20mph, the system is likely to intervene and prevent a collision. But if you try to do the same thing at 70mph, a lot of systems on the market won’t intervene.
The basic reason is that the developers of driver-assistance systems follow a conservative “first do no harm” philosophy. They want to prevent a crash if they can, yes, but more importantly they want to avoid taking an action that will cause a crash that wouldn’t have happened otherwise.
When a car is moving at low speeds, slamming on the brakes isn’t a big risk. A car traveling at 20mph can afford to wait until an object is quite close before slamming on the brakes, making unnecessary stops unlikely. Short stopping distances also mean that a car slamming on the brakes at 20mph is unlikely to get rear-ended.
But the calculation changes for a car traveling at 70mph. In this case, preventing a crash requires slamming on the brakes while the car is still far away from a potential obstacle. That makes it more likely that the car will misunderstand the situation—for example, wrongly interpreting an object that’s merely near the road as being in the road. Sudden braking at high speed can startle the driver, leading to erratic driving behavior. And it also creates a danger that the car behind won’t stop in time, leading to a rear-end collision.
And like adaptive cruise control, automatic emergency braking is often implemented as a separate system from the lane-keeping module. Most AEB systems lack the kind of sophisticated situational awareness a fully self-driving system would have. That means it may not be able to tell if an object 100 meters ahead is in the current travel lane or the next lane over—and whether it’s a temporarily stopped car, a pedestrian, or a bag of garbage.
So a lot of emergency braking systems simply don’t try to brake for obstacles when the vehicle is traveling at high speeds.
“If you’re at lower speeds, at 30mph, and it detects a stationary object, these systems will generally respond and slow the car down and bring it to a stop,” Abuelsamid told us. “When closing speed is above about 50mph, if it sees a stationary car, it’s going to ignore that.”
Carmakers should rethink their driver assistance philosophy
The fundamental issue here is that tendency to treat lane-keeping, adaptive cruise control, and emergency braking as independent systems. As we’ve seen, today’s driver assistance systems have been created in a piecemeal fashion, with each system following a do-no-harm philosophy. They only intervene if they’re confident they can prevent an accident—or at least avoid causing one. If they’re not sure, they do nothing and let the driver make the decision.
The deadly Tesla crash in Mountain View illustrates how dangerous this kind of system can be.
According to the NTSB, Huang’s hands were not detected on the steering wheel for six seconds before the crash, suggesting he wasn’t paying attention to the road in the final seconds.
Perhaps he would not have taken his hands off the steering wheel for so long in a conventional car without Autopilot’s capabilities. But more importantly, even if he had taken his hands off the wheel and his eyes off the road, a conventional car would not have unexpectedly shifted into an adjacent lane. At worst, it would have drifted out of the travel lane in a gradual and predictable fashion, allowing an experienced driver to detect the drift and correct it.
Early driver assistance systems assumed that the driver could monitor the car and intervene if the car made a mistake. But a driver’s ability to monitor a car’s progress depends crucially on reflexes built up over years of driving. Those reflexes depend on cars behaving in consistent and predictable ways: for example, if you take your eyes off the road for a couple of seconds, it will continue traveling in the same direction.
Once a driver-assistance system reaches a certain level of complexity, the assumption that it’s safest for the system to do nothing no longer makes sense. Complex driver assistance systems can behave in ways that surprise and confuse drivers, leading to deadly accidents if the driver’s attention wavers for just a few seconds. At the same time, by handling most situations competently, these systems can lull drivers into a false sense of security and cause them to pay less careful attention to the road.
So the people designing the next generation of autonomous driving systems are going to need a fundamental philosophical shift. Instead of treating cruise control, lane-keeping, and emergency braking as distinct systems, advanced driver assistance systems need to become integrated systems with a sophisticated understanding of the car’s surroundings.
Most driver-assistance systems will activate in a wide variety of circumstances, but they’ll only intervene to prevent certain kinds of crashes. The next generation of driver assistance systems needs to take the opposite approach: they should aim to avoid every obstacle but should only be available for use in areas where the system is confident it can deliver on that goal.
Abuelsamid pointed to Cadillac’s Super Cruise system—which will be rolled out to other GM vehicles over the next few years—as a model for other carmakers to follow. Super Cruise operates as a sophisticated, integrated system, not a collection of separate driver assistance features. In addition to relying on on-board cameras and radar, the system also makes use of high-precision maps to tell exactly where the roadway is supposed to go. So the Super Cruise software can handle a wider range of potential crashes than most other driver assistance technologies on the market.
At the same time, Super Cruise isn’t available on every highway. GM says it has pre-mapped 130,000 miles of highway to an accuracy of 5cm, and Super Cruise will only activate when you’re on one of those highways.
A system like Super Cruise would be unlikely to make the kind of mistake that Walter Huang’s Tesla made. Even if the lanes were poorly marked, the Super Cruise software would have known from its maps that the gore area was not a valid travel lane. And in areas without maps, Super Cruise won’t engage at all, forcing the driver to pay full attention.
We don’t know of any other carmaker that has adopted GM’s approach to driver assistance. But we hope that will change soon.