End-to-End Driver Fatigue Detection Model Based on In-Vehicle Vision

Abstract:

Long-term fatigue driving is an important cause of accidents for operational drivers. To ensure driving safety， companies install cameras on operational vehicles to collect drivers’ facial videos， automatically identify the drivers’ fatigue state based on a fatigue detection model， and use voice reminders or even enable remote escort to prevent fatigue. Most of the existing fatigue detection research is based on the extraction of the key points of drivers’ faces， which has high requirements for video quality. However， in the real commercial vehicle environment， the detection of key points easily fails due to the poor light at night， the imperfect position of the camera and the obscured face of the drivers， thus affecting the accuracy of the model. Therefore， this paper proposes an end-to-end fatigue detection model for operational drivers with a high robustness， based on convolutional neural network （CNN） and long short-term memory neural network （LSTM）. The model takes the drivers' facial videos collected by cameras as input， and uses the CNN network to extract the single-frame features of the videos. On this basis， the temporal single-frame features are used as the input of the LSTM network to finally identify the drivers’ fatigue state. The experimental results show that the area under curve （AUC） of the model is 0.9， which is much superior to existing models based on facial key points. In addition， in order to improve the robustness of the model in the actual driving environment， data augmentation is applied to the training data， simulating both light and camera changes. The accuracy and robustness of the model are further improved through model retraining. Before the improvement， the AUC of the model in the actual driving environment for commercial vehicles is reduced by 37.3 %， compared with the laboratory model， However， after the improvement， the AUC is only reduced by 9.7 % which indicates that the robustness of the model is improved and the model can better adapt to the complex naturalistic driving environment for commercial vehicles.