基于车载视觉的端到端驾驶员疲劳检测模型

End-to-End Driver Fatigue Detection Model Based on In-Vehicle Vision

摘要:

营运驾驶员长时间疲劳驾驶是导致事故发生的重要原因，为此，企业在营运车辆上安装相机采集驾驶员面部视频，基于模型和算法自动识别驾驶员的疲劳状态，通过语音提醒甚至启用远程护航进行疲劳干预，以此提高驾驶安全。现有的疲劳检测研究大多数都是基于面部关键点检测的算法，该类算法对面部视频的质量要求严格。在真实的营运行车环境中，夜晚光线过差，相机位置安装不理想，驾驶员面部遮挡等均会造成关键点检测失效，从而影响模型的准确性。基于卷积神经网络（CNN）和长短时记忆神经网络（LSTM）设计了一种端到端营运驾驶员疲劳检测模型，该模型以相机采集的驾驶员面部视频作为输入，使用CNN网络提取视频单帧特征，在此基础上将时序单帧特征作为LSTM网络的输入来最终识别驾驶员的疲劳状态，实验表明，模型的接收者操作特征曲线下面积（AUC）为0.9，远优于现有的面部关键点模型。此外，为了提高该模型在实际行车环境中的鲁棒性，基于光线变化及相机变化的模拟操作在训练数据上进行了数据增强，通过模型重训练进一步提高了模型的精度及鲁棒性。实验结果表明，改进前，营运车辆行车环境下模型的AUC相比实验室模型下降37.3 %，而改进后AUC仅下降9.7 %，模型的鲁棒性得到改善，能够更好地适应复杂的营运车辆自然驾驶环境。

Abstract:

Long-term fatigue driving is an important cause of accidents for operational drivers. To ensure driving safety， companies install cameras on operational vehicles to collect drivers’ facial videos， automatically identify the drivers’ fatigue state based on a fatigue detection model， and use voice reminders or even enable remote escort to prevent fatigue. Most of the existing fatigue detection research is based on the extraction of the key points of drivers’ faces， which has high requirements for video quality. However， in the real commercial vehicle environment， the detection of key points easily fails due to the poor light at night， the imperfect position of the camera and the obscured face of the drivers， thus affecting the accuracy of the model. Therefore， this paper proposes an end-to-end fatigue detection model for operational drivers with a high robustness， based on convolutional neural network （CNN） and long short-term memory neural network （LSTM）. The model takes the drivers' facial videos collected by cameras as input， and uses the CNN network to extract the single-frame features of the videos. On this basis， the temporal single-frame features are used as the input of the LSTM network to finally identify the drivers’ fatigue state. The experimental results show that the area under curve （AUC） of the model is 0.9， which is much superior to existing models based on facial key points. In addition， in order to improve the robustness of the model in the actual driving environment， data augmentation is applied to the training data， simulating both light and camera changes. The accuracy and robustness of the model are further improved through model retraining. Before the improvement， the AUC of the model in the actual driving environment for commercial vehicles is reduced by 37.3 %， compared with the laboratory model， However， after the improvement， the AUC is only reduced by 9.7 % which indicates that the robustness of the model is improved and the model can better adapt to the complex naturalistic driving environment for commercial vehicles.