Training Socially Aware AVs in Urban Environments

Systems Conversation with Bilal Farooq, Canada Research Chair in Disruptive Transportation Technologies and Services and Associate Professor in Transportation Engineering, Ryerson University

This Systems Conversation was conducted before Dr. Bilal Farooq gave a talk in our systems seminar series. Title of his talk: Training Socially Aware AVs in Urban Environments: Using Immersive Virtual Reality and Interpretable Machine Learning

Currently, the interactions between drivers and pedestrians on urban roads are mainly in the form of a silent agreement. Eye contact between the two agents, or head orientations and body movements of the pedestrians observed by the drivers, are some of the behaviors that establish this silent agreement. Drivers then continuously try to anticipate the next actions of pedestrians based on what they see. At the same time, pedestrians decide on their behavioral choices based on what they perceive from the vehicles and drivers. To replicate these interactions in an automated urban environment, immersive virtual reality experiments are designed to collect detailed behavioral data from pedestrians. Interpretable deep learning models are then developed that can predict pedestrian walking behavior when crossing urban roads. Multi-objective reinforcement learning is used to train the AV that can anticipate the pedestrian’s actions and react while incorporating pedestrian safety as well as passenger comfort. The findings of this study are not only useful for the manufacturers but can also help municipalities in the reassessment of current policies, practices, design, rules and regulations in order to prepare them for the mobility technologies of the future.