An Image Pre-Processing Method for Visual Simultaneous Localization and Mapping in Dynamic Environments

doi:10.11908/j.issn.0253-374x.23068

Home > Archive>Volume 52, Issue 12, 2024 >1955-1964. DOI:10.11908/j.issn.0253-374x.23068

An Image Pre-Processing Method for Visual Simultaneous Localization and Mapping in Dynamic Environments
DOI:
                        10.11908/j.issn.0253-374x.23068
                    
CSTR:
                        
                    
Author:
                        ZHUO Guirong1,2ZHUO Guirong
School of Automotive Studies， Tongji University， Shanghai 201804， China;Clean Energy Automotive Engineering Center， Tongji University， Shanghai 201804， China
Find this author on All Journals
Find this author on BaiDu
Search for this author on this site
LU Shouyi1,2LU Shouyi
School of Automotive Studies， Tongji University， Shanghai 201804， China;Clean Energy Automotive Engineering Center， Tongji University， Shanghai 201804， China
Find this author on All Journals
Find this author on BaiDu
Search for this author on this site
XIONG Lu1,2XIONG Lu
School of Automotive Studies， Tongji University， Shanghai 201804， China;Clean Energy Automotive Engineering Center， Tongji University， Shanghai 201804， China
Find this author on All Journals
Find this author on BaiDu
Search for this author on this site

                    
Affiliation:1.School of Automotive Studies， Tongji University， Shanghai 201804， China;2.Clean Energy Automotive Engineering Center， Tongji University， Shanghai 201804， China
Clc Number:U469.79
Fund Project:

Article

Figures

Metrics

Reference [30]

Related [11]

Cited by

Materials

Comments

Abstract:

This paper proposes an image preprocessing method for visual simultaneous localization and mapping (SLAM) systems in dynamic environments， which can be easily integrated into existing visual SLAM systems to enable stable， accurate， and continuous operation in highly dynamic environments. First， it proposes a dynamic object recognition algorithm that integrates the use of semantic segmentation networks and optical flow estimation networks to robustly and accurately identify potential dynamic objects in images. Then， to detect shadows associated with dynamic objects， it proposes a shadow recognition algorithm based on region growing. Afterwards， it uses image completion techniques to fill in the gaps left by the removal of dynamic objects from the images. Finally， it combines this image preprocessing method with stereo ORB-SLAM2 and conducts experiments on the KITTI dataset， which demonstrates that the proposed image preprocessing method significantly improves the positioning accuracy of visual SLAM systems. Each module in the image preprocessing method plays an irreplaceable role.

Key words:image pre-processing methods;visual simultaneous localization and mapping;dynamic environment;localization

Reference

[1] KUUTTI S , FALLAH S , KATSAROS K , et al . A survey of the state-of-the-art localization techniques and their potentials for autonomous vehicle applications[J]. IEEE Internet of Things Journal, 2018, 5( 2): 829.

[2] 余卓平, 高乐天, 夏新, 等 . 基于轮胎半径自适应的智能车辆组合定位[J]. 同济大学学报(自然科学版), 2022, 50( 4): 504. YU Zhuoping , GAO Letian , XIA Xin , et al . Intelligent vehicle combined positioning based on adaptive tire radius[J]. Journal of Tongji University (Natural Science), 2022, 50( 4): 504.

[3] FAKHARIAN A , GUSTAFSSON T , MEHRFAM M . Adaptive Kalman filtering based navigation: An IMU/GPS integration approach[C]// 2011 International Conference on Networking, Sensing and Control. [S.l.]: IEEE, 2011: 181- 185.

[4] 周苏, 李伟嘉, 郭军华 . 基于激光雷达的停车场车辆定位算法[J]. 同济大学学报(自然科学版), 2021, 49( 7): 1029. ZHOU Su , LI Weijia , GUO Junhua . Parking lot vehicle positioning algorithm based on LIDAR[J]. Journal of Tongji University (Natural Science), 2021, 49( 7): 1029.

[5] CHEN W , SHANG G , JI A , et al . An overview on visual SLAM: From tradition to ssemantic[J]. Remote Sensing, 2022； 14( 13): 3010.

[6] DAVISON A J , REID I D , MOLTON N D , et al . MonoSLAM: Real-time single camera SLAM[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29( 6): 1052.

[7] KLEIN G , MURRAY D . Parallel tracking and mapping for small AR workspaces[C]// 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality. [S.l.]: IEEE, 2007: 225- 234.

[8] MUR-ARTAL R , TARDOS J D . Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras[J]. IEEE Transactions on Robotics, 2017, 33( 5): 1255.

[9] CAMPOS C , ELVIRA R , RODRIGUEZ J J G , et al . Orb-slam3: An accurate open-source library for visual, visual–inertial, and multimap slam[J]. IEEE Transactions on Robotics, 2021, 37( 6): 1874.

[10] NEWCOMBE R A , LPVEGROVE S J , DAVISON A J . DTAM: Dense tracking and mapping in real-time[C]// 2011 International Conference on Computer Vision. [S.l.]: IEEE, 2011: 2320- 2327.

[11] ENGEL J , SCHOPS T , CREMERS D . LSD-SLAM: Large-scale direct monocular SLAM[C]// European Conference on Computer Vision. [S.l.]: Springer, 2014: 834- 849.

[12] ENGEL J , KOLTUN V , CREMERS D . Direct sparse odometry[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40( 3): 611.

[13] GAO X , WANG R , DEMMEL N , et al . LDSO: Direct sparse odometry with loop closure[C]// 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). [S.l.]: IEEE, 2018: 2198- 2204.

[14] SAPUTRA M R U , MARKHAM A , TRIGONI N . Visual SLAM and structure from motion in dynamic environments: A survey[J]. ACM Computing Surveys (CSUR), 2018, 51( 2): 1.

[15] DERPANIS K G . Overview of the RANSAC Algorithm[J]. Image Rochester NY, 2010, 4( 1): 2.

[16] VYSOTSKA O , STACHNISS C . Exploiting building information from publicly available maps in graph-based SLAM[C]// 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). [S.l.]: IEEE, 2016: 4511- 4516.

[17] RICHARD H , ANDREW Z , Multiple view geometry in computer vision [M]. 2nd ed. Cambridge, U. K.: Cambridge University Press, 2003.

[18] KUNDU A , KRISHNA K M , SIVASWAMY J . Moving object detection by multi-view geometric techniques from a single camera mounted robot[C]// 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems. [S.l.]: IEEE, 2009: 4306- 4312.

[19] 刘钰嵩, 何丽, 袁亮, 等 . 动态场景下基于光流的语义RGBD-SLAM算法[J]. 仪器仪表学报, 2022, 43( 12): 139. LIU Yusong , HE Li , YUAN Liang , et al . Semantic RGBD-SLAM algorithm based on optical flow in dynamic scenes[J]. Journal of Instrumentation, 2022, 43( 12): 139.

[20] 张小勇, 张洪, 高清源, 等 . 室内动态场景下基于稀疏光流与实例分割的视觉SLAM算法[J]. 东华大学学报:自然科学版, 2023, 49( 6): 111. ZHANG Xiaoyong , ZHANG Hong , GAO Qingyuan , et al . Visual SLAM algorithm based on sparse optical flow and instance segmentation in indoor dynamic scenes[J]. Journal of Donghua University (Natural Science), 2023, 49( 6): 111.

[21] YU C , LIU Z , LIU X J , et al . DS-SLAM: A semantic visual SLAM towards dynamic environments[C]// 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). [S.l.]: IEEE, 2018: 1168- 1174.

[22] BESCOS B , FACIL J M , CIVERA J , et al . DynaSLAM: Tracking, mapping, and inpainting in dynamic scenes[J]. IEEE Robotics and Automation Letters, 2018, 3( 4): 4076.

[23] ZHONG F , WANG S , ZHANG Z , et al . Detect-SLAM: Making object detection and SLAM mutually beneficial[C]// 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). [S.l.]: IEEE, 2018: 1001- 1010.

[24] CHENG J , WANG Z , ZHOU H , et al . DM-SLAM: A feature-based SLAM system for rigid dynamic scenes[J]. ISPRS International Journal of Geo-Information, 2020, 9( 4): 202.

[25] FAN M , LAI S , HUANG J , et al . Rethinking BiSeNet for real-time semantic segmentation[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. [S.l.]: IEEE, 2021: 9716- 9725.

[26] CORDTS M , OMRAN M , RAMOS S , et al . The cityscapes dataset for semantic urban scene understanding[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. [S.l.]: IEEE, 2016: 3213- 3223.

[27] GEIGER A , LENZ P , STILLER C , et al . Vision meets robotics: The kitti dataset[J]. The International Journal of Robotics Research, 2013, 32( 11): 1231.

[28] ZHAO S , SHENG Y , DONG Y , et al . Maskflownet: Asymmetric feature matching with learnable occlusion mask[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. [S.l.]: IEEE, 2020: 6278- 6287.

[29] BUTLER D J , WULFF J , STANLEY G B , et al . A naturalistic open source movie for optical flow evaluation[C]// European Conference on Computer Vision. Berlin, Heidelberg: Springer, 2012: 611- 625.

[30] LI Z , LU C Z , QIN J , et al . Towards an end-to-end framework for flow-guided video inpainting[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. [S.l.]: IEEE, 2022: 17562- 17571.

Get Citation

ZHUO Guirong, LU Shouyi, XIONG Lu. An Image Pre-Processing Method for Visual Simultaneous Localization and Mapping in Dynamic Environments[J].同济大学学报(自然科学版),2024,52(12):1955~1964

Copy

Article Metrics

Abstract:37
PDF: 82
HTML: 1042
Cited by: 0

History

Received:March 05,2023
Revised:
Adopted:
Online: January 03,2025
Published:

Get Citation

Share

Article Metrics

History

Article QR Code