An end-to-end approach to autonomous vehicle control using deep learning

Abstract

The objective of this work is to develop an autonomous vehicle controller inside Grand Theft Auto V game, used as a simulation environment. It is used an end-to-end approach, in which the model maps directly the inputs from the image of a car hood camera and a sequence of speed values to three driving commands: steering wheel angle, accelerator pedal pressure and brake pedal pressure. The developedmodel is composed of a convolutional neural network and a recurring neural network. The convolutional network processes the images and the recurrent network processes the speed data. Themodel learns fromdata generated by a human driver´s commands. Two interfaces are developed: one for collecting in-game training data and another to verify the performance of themodel for the autonomous vehicle control. The results show that themodel after training is capable to drive the vehicle as well as a human driver. This proves that a combination of a convolutional network with a recurrent network, using an end-to-end approach, is capable of obtaining a good driving performance even using only images and speed velocity as sensory data.