Camera Based Autonomous Driving Using Machine Learning

  • Uvindu Wijesinghe

Student thesis: Masters ThesisMEng


Autonomous vehicles have been continuously increasing in popularity both among consumers as well as businesses. Many manufacturers are working to build internal capability to deliver increasingly complex automation in their vehicles to keep up with the projected demand. One avenue to attract relevant talent is to sponsor engineering competitions. Formula Student is one such engineering design competition that challenges student teams to build and race a single seater race car. Recently, it has evolved to push competing student teams to build vehicles with autonomous technology, to better equip them with in-demand skills for careers in this sector.

Team Bath Racing Electric (TBRe) is a team that is looking to build an autonomous car for this competition. Due to budgetary constraints, a solution that involves just cameras was preferred over having to use more expensive sensors such as LIDARs. This project investigated the viability of using a convolutional neural network (CNN) to predict steering and throttle inputs based on just images of the track ahead.

A mobile robotics platform was used to build a prototype system as a proof-of-concept. A miniature track was designed and built for the project. An operator used a gamepad to manually drive the robot around the track and collect training images along with steering and throttle inputs. A variety of CNN configurations were tested to find the optimal one for this application.
Once the CNN was trained it was deployed onto the robot. Due to processing power constraints on the robot, the processing was off-loaded to a networked computer to speed up the steering and throttle prediction process. The system was able to perform adequately well and had results comparable with systems described by other teams that had been deployed in the real world in real vehicles.

Further work is required to improve the reliability and resilience of the system. The system can be expanded to incorporate more cameras paired with a more powerful integrated computer. Vehicle integration will also need to be considered when the system is mature enough to deploy on to TBRe’s autonomous car.
Date of Award2019
Original languageEnglish
SupervisorUriel Martinez Hernandez (Supervisor)

Cite this