Deep Learning Approaches to Bird’s-Eye View Transformation and RGB-Depth Fusion in Autonomous Vehicles

Abstract

Autonomous vehicles depend on accurateand efficient environment representations such assemantically segmented Bird’s Eye View (BEV) for pathplanning and decision-making to achieve safe navigation.Implementing deep learning techniques to generatefront-view to bird’s-eye view transformations with depthinformation and RGB images is often complex due tothe absence of real-world BEV datasets for training.Additionally, model’s performance is often affected bythe semantic class imbalance of the BEV maps at thepixel level. On this study, we propose a sensor fusionblock to integrate RGB and depth features to improveperspective transformation performance. Furthermore,we implement a layer-based data augmentation toaddress the class imbalance challenge. Experimentsto demonstrate that the proposed sensor fusion blockand the layer based data augmentation method improveperspective transformation performance on state of theart deep learning architectures.