Repository logo

Correcting Road Image Annotations

Accepted version

Change log


Chen, Weiwei 
De Silva, Lavindra 
Brilakis, Ioannis 


Digitisation provides a promising avenue to meet new socio-economic demands in infrastructure delivery. The current state of practice suggests that human annotators still need to participate substantially in data preparation for digitising infrastructure, such as annotating real-life domain-specific visual datasets for road maintenance. Research in the past focuses on predicting better labels with less human effort, leaving a gap in not maximizing the gains from the subsequent human corrections of pseudo-labels to ground truths. The gap highlights the opportunities for a solution to mimic human corrections by “correcting like instances alike”. We propose an extension to Mask R-CNN to tackle this gap. Our auto-correction method harvests learnings from past corrections and automates corrections in forthcoming images. The method first gauges the corrections made between the pseudo-labels and the final ground truth. The method then compares the feature vectors and attribute data of a new batch of unlabelled images against past corrections. When the deep learning model predicts similarly wrong features, the method will mimic human corrections in the past and prompt additions, deletions or category changes. The method concludes with post-processing to eliminate unwanted predictions. This similarity-based approach improves the precision of the testing batch by 15-70% and reduces the number of mouse clicks by approximately 20%. The solution therefore partially automates the human review after predicting pseudo-labels by similarity-based corrections.



Journal Title

Conference Name

International Conference on Computing in Civil and Building Engineering

Journal ISSN

Volume Title


Publisher DOI

Publisher URL

Engineering and Physical Sciences Research Council (EP/S02302X/1)
EPSRC (EP/V056441/1)
National Highways, Costain and Trimble Solutions through the EPSRC Centre for Doctoral Training in Future Infrastructure and Built Environment: Resilience in a Changing World [grant number EP/S02302X/1] and the Digital Roads [grant number EP/V056441/1].