Bridge CNN defect prediction models using existing image data

Abstract

In South Africa, it is a requirement for all road agencies to conduct principal visual inspections of all bridge structures every five years. Smaller municipalities do not always have the necessary funds available for principal bridge inspections, resulting in either bridge inspections not being executed, or inspections being done by unqualified people. This paper intends to investigate the possibility of using existing bridge inventory and inspection image data to develop Convolutional Neural Network (CNN) models to predict and classify bridge defects autonomously. This research aims to improve the quality of bridge inspections and condition ratings assigned to defects to be more consistent and not reliant on human subjectivity. These models could ultimately be used for quality control in a Bridge Management System (BMS). The CSIR STRUMAN BMS contains inspection and inventory images captured during principal visual bridge inspections. As a proof-of-concept, bridge roadway joints were considered. 600 images of bridge roadway joints captured in the system were classified according to Defect and No Defect datasets. Different CNN classification models were developed to predict whether an image of a bridge roadway joint contained a defect or not. The image datasets were used to train, validate, and test the performance of the CNN models. The performance of the CNN models was evaluated using a Confusion Matrix and Classification report to select the best-performing model. In conclusion, the selected model was evaluated when introduced to new unseen images. The best performing CNN model utilised transfer learning and data augmentation to predict with 95% accuracy from images if a bridge roadway joint had a defect and with 65% accuracy if the bridge roadway joint had no defect.