Extrinsic Calibration of Lidar and Fisheye Camera Based on Deep Learning and Geometric Optimization

Abstract:

Extrinsic calibration plays a crucial role in autonomous driving， intelligent surveillance， and robotic perception. However， existing research primarily focuses on the calibration between LiDAR and pinhole cameras， while studies on LiDAR and fisheye cameras remain relatively limited. To fill this gap， this paper proposes a deep learning-based extrinsic calibration method for a LiDAR and a fisheye camera， incorporating a geometric optimization strategy as post-processing to significantly improve calibration accuracy. The proposed method first utilizes a neural network to predict the extrinsic transformation matrix， followed by geometric optimization to refine the predicted results. Experimental evaluations on 18 335 image–point cloud pairs demonstrate that the deep learning model reduces the point cloud alignment error by 91.6%. With the additional geometric post-processing， the error is further reduced by 52.4%， verifying the high accuracy， reliability， and robustness of the proposed approach in practical applications.