Completed 2019 Research Project

Safety testing for connected and automated vehicles through physical and digital iterative deployment

Principal Investigator
Subhadeep Chakraborty 
University of Tennessee, Knoxville 
View Bio

Co-Principal Investigator
Asad J. Khattak 
University of Tennessee, Knoxville 
View Bio

Co-Principal Investigator
Missy Cummings 
Duke University 
View Bio

Year 1 Final Report: Advancing accelerated testing protocols for safe and reliable deployment of connected and automated vehicles through iterative deployment in physical and digital worlds

Interim Report: Evaluating the Reliability of Tesla Model 3 Driver Assist Functions

Supplemental Report: ADAS Vision Risk Assessment

Project Slide Deck

Research Brief

Summary

Currently, many manufacturers are selling vehicles with SAE Level 2 or 3 automation and testing with more advanced vehicles on public roads (level 4). However, recent crashes have raised critical questions about their safety. Are these vehicles with lower or higher automation safe enough to drive on public roads, and more fundamentally, how do we assess their safety envelope? Currently, there is no consensus about whether testing should exist at the state or federal level, what functions should be tested, and how independent testing should occur, and what constitutes safe thresholds.

In this CSCRS project, we seek to address this gap by developing a comprehensive testing protocol specifically for Level 2 and 3 connected and automated vehicles by using a novel software and physical deployment platform which allows rapid iterative development. With such a testing protocol, automated vehicles can be systematically tested and certified to be generally safe before being tested on public roads. The key objectives of this research are to:

1. Conduct a thorough review of ongoing efforts by standards organizations such as SAE, ISO and others including the UL 4600 standard under development.
2. Develop a testing protocol that will standardize how to systematically and safely test Level 2 and 3 automated vehicles regarding their functions and capabilities with considerations of the driver and environment settings. The protocol will be designed to allow accelerated testing and identification of fringe cases and stress points where automated systems will be prone to failure.
3. Provide safety certification standard recommendations that regulatory agencies at the state and federal levels and the private sector can use, if they so choose.
4. Involve stakeholders, i.e., government agencies and private sector companies, in the CAV testing and enable discussion on safety and certification processes.

Videos

The videos below are from a sequence of tests the Duke research team performed evaluating the behavior of Tesla Model 3s in various partially automated driving situations. Some tests were performed at the NCCAR research test track in Garysburg, NC, while another was performed on a public section of Highway 540 in Cary, NC.

Car2 vid37 highway7 console
Car3 vid11 construction1 console
Car3 vid17 construction3 console
Car2 vid2 construction1 console
Car3 vid19 curve3 console
Car1 vid19 depart3 console
Car1 vid20 depart4 console
Car3 vid32 depart8 console

This is the complete collection of videos.

Publications

• Beck, J., Arvin, R., Lee, S., Khattak, A., & Chakraborty, S. (2023, February).  Automated vehicle data pipeline for accident reconstruction: New insights from LiDAR, camera, and radar data. Accident Analysis and Prevention, 180: 106923.
• Lee, S., Arvin, R., & Khattak, A. J. (2023). Advancing investigation of automated vehicle crashes using text analytics of crash narratives and Bayesian analysis. Accident Analysis and Prevention, 181, 106932. https://doi.org/10.1016/j.aap.2022.106932
• Moradloo, N., Mahdinia, I., Khattak, A., & Bayati, Z. (2023). Safety in higher level automated vehicles: Investigating edge cases in crashes of vehicles equipped with automated driving systems. [Presentation]. Transportation Research Board Annual Meeting.
• SafariTaherkhani M., Patwary, L., & Khattak, A. (2023). Comparison of crash types in automated vehicles with different levels of automation. [Presentation]. Transportation Research Board Annual Meeting. TRBAM-23-05272.
• Moradloo, N., Mahdinia, I., Khattak, A., & SafariTaherkhani, M. (2023). Identifying corner cases in fatal pedestrian-involved crashes: Application of unsupervised machine learning approach. [Presentation]. Transportation Research Board Annual Meeting.
• Huff, S., Chakraborty, S., Beck, J., Nafziger, E., Taylor, C., & Carter, J. (n.d.) Advancing accelerated testing protocols for safe and reliable self-driving operations through iterative deployment in physical and digital worlds. [Presentation]. WCX SAE World Congress Experience – ADAS and Autonomous Vehicle System: Simulation and Testing – Part 1 (AE106)
• Beck J. W., & Chakraborty, S. (n.d.). Perception error modeling towards simulation of automated vehicles. [Presentation]. WCX SAE World Congress Experience – ADAS and Autonomous Vehicle System: Simulation and Testing – Part 1 (AE106)

Project Details