By Christopher Hart
For decades, the automotive industry has been using technological advances, such as seat belts, air bags, and structural crush resistance, to protect us if we’re in a crash. Those advances have undoubtedly saved thousands of lives a year. Now we have the exciting opportunity to use technological advances to prevent crashes from happening in the first place, which can save tens of thousands of lives a year. But because automation will coexist with human drivers for the foreseeable future, there will be many challenges.
Driverless cars could save many, if not most, of the 33,000 lives that are lost every year on our streets and highways – a very tragic and unacceptable number that has been decreasing for several years but has recently taken a turn in the wrong direction.
Most crashes on our roads are due to driver error. The theory of driverless cars is that if there is no driver, there will be no driver error. Ideally, removing the driver would address at least four issues on the NTSB’s Most Wanted List of Transportation Safety Improvements – fatigue, distractions, impairment, and fitness for duty. The automation in driverless cars would presumably also address a fifth item on our list, namely, improved collision avoidance technologies.
Decades of experience in a variety of contexts has demonstrated that automation can improve safety, reliability, productivity, and efficiency. That experience has also demonstrated that there can be a downside. As noted by Professor James Reason, who is a world-renowned expert in complex human-centric systems:
In their efforts to compensate for the unreliability of human performance, the designers of automated control systems have unwittingly created opportunities for new error types that can be even more serious than those they were seeking to avoid.
Our investigation experience provides three lessons learned that support Prof. Reason’s statement. The first is that the theory of removing human error by removing the human assumes that the automation is working as designed. So, the question becomes: what if the automation fails?
An example of the automation failing without the operator’s knowledge occurred in Washington, DC, with the Metro crash near the Fort Totten Station in 2009 that tragically killed the train operator and 8 passengers. In that accident, a train temporarily became electronically invisible, whereupon the symbol of the train disappeared from the display board in the dispatch center.
Unfortunately, when the train became electronically invisible, there was no alarm in the train behind it regarding the electronic disappearance of the preceding train. By the time the operator saw the stopped train and applied the emergency brake after coming around a curve – which limited her sight distance – it was too late.
Another lesson learned in support of Prof. Reason’s statement is that even if the operator is removed from the loop, humans are still involved in designing, manufacturing, and maintaining the vehicles, as well as the streets and highways they use. Each of these points of human engagement presents opportunities for human error. Moreover, human error in these steps is likely to be more systemic in its effect – possibly involving several vehicles – and more difficult to find and correct. An example of this lesson learned is the collision of an automated – driverless – people mover into a stopped people mover at Miami International Airport in 2008. That collision was caused largely by improper maintenance.
The most fundamental lesson learned from our accident investigation experience in support of Prof. Reason’s statement is that introducing automation into complex human-centric systems can be very challenging. Most of the systems we have investigated are becoming increasingly automated but are not fully automated. As a result, we have seen that the challenges can be even more difficult in a system that is not completely automated but still has substantial human operator involvement.
With that background on how automation can be both the good news and the bad news, how can the NTSB help inform the process of moving toward driverless cars?
First, we offer considerable experience regarding the introduction of automation into complex human-centric systems.
Most of our investigations involve relatively structured systems with professional operators who are trained extensively (including, typically, on the automation) and have various requirements regarding proficiency, fatigue, impairment, distraction, and fitness for duty. Given that human drivers will probably be in the loop for some time to come, I would suggest that as difficult as the transition to more automation has been in the structured and regulated environments we have investigated, it may be even more challenging in a public arena, in which drivers are usually not highly trained and may be fatigued, impaired, distracted, or not medically fit.
The second way that the NTSB can help relates to collaboration. The auto industry has already recognized the importance of collaboration, as most recently shown by their collaborative approach regarding the voluntary installation of autonomous emergency braking by 2022. Our experience with collaboration, especially regarding commercial aviation, may help improve it further.
Although automation has played an important role in the commercial aviation industry’s continuing safety improvement, much of the industry’s exemplary safety record is attributable to collaboration. In the early 1990s, after the industry’s accident rate had been declining rapidly, the accident rate began to flatten on a plateau. Meanwhile, the Federal Aviation Administration was predicting that the volume of flying would double in 15-20 years.
The industry became very concerned that if the volume doubled while the accident rate remained the same, the public would see twice as airplane crashes on the news. That caused the industry to do something that, to my knowledge, has never been done at an industry-wide level in any other industry – they pursued a voluntary collaborative industry-wide approach to improving safety.
The voluntary collaborative process, known as CAST, the Commercial Aviation Safety Team, brings all of the players –airlines, manufacturers, pilots, air traffic controllers, and the regulator – to the table to do four things: Identify the potential safety issues, prioritize those issues – because they would be identifying more issues than they had resources to address, develop interventions for the prioritized issues, and evaluate whether the interventions are working.
This CAST process has been an amazing success. It resulted in a reduction of the aviation fatality rate, from the plateau on which it was stuck, by more than 80% in less than 10 years. As an observer in CAST, the NTSB can help the auto industry determine how much of this aviation industry success story is transferrable to them.
The third way that the NTSB can inform the process of introducing automation relates to on-board event recorders. Our investigations are significantly enhanced when we have event recorders to tell us what happened. Airliners have had “black boxes” – which are actually orange — for decades, to record both the aircraft parameters and the sounds in the cockpit. Other transportation modes are increasingly introducing event recorders as well as audio and video recorders.
There have already been crashes as a new wave of automation has been introduced which handles some or all of the driving task. The more that the industry knows from event recorders about what went right and what went wrong, the more the industry will be able to fashion remedies that effectively address the problems. Accordingly, consistent with another item on our Most Wanted List – Expand the Use of Recorders to Enhance Transportation Safety – we would encourage the use of robust on-board event recorders to help the process.
The NTSB will continue to investigate highway crashes when the investigation can illuminate important safety issues, including issues arising from automation. In addition, we are willing and able to work with the automotive industry before accidents happen.
The NTSB has already engaged with the industry and regulatory agencies to help inform how driverless cars can be safely introduced into America’s transportation system. Our experience in the introduction of automation into human-centric systems, our appreciation of the power of collaboration, and our understanding of the importance of on-board event recorders all position the NTSB to provide valuable assistance to the process.
(Excerpted in large part from a presentation that I gave at the National Press Club on June 30, 2016. Go here for the full speech)
NTSB Safety Compass Blog 