Correcting Road Image Annotations

Abstract

Digitisation provides a promising avenue to meet new socio-economic demands in infrastructure delivery. The current state of practice suggests that human annotators still need to participate substantially in data preparation for digitising infrastructure, such as annotating real-life domain-specific visual datasets for road maintenance. Research in the past focuses on predicting better labels with less human effort, leaving a gap in not maximizing the gains from the subsequent human corrections of pseudo-labels to ground truths. The gap highlights the opportunities for a solution to mimic human corrections by “correcting like instances alike”. We propose an extension to Mask R-CNN to tackle this gap. Our auto-correction method harvests learnings from past corrections and automates corrections in forthcoming images. The method first gauges the corrections made between the pseudo-labels and the final ground truth. The method then compares the feature vectors and attribute data of a new batch of unlabelled images against past corrections. When the deep learning model predicts similarly wrong features, the method will mimic human corrections in the past and prompt additions, deletions or category changes. The method concludes with post-processing to eliminate unwanted predictions. This similarity-based approach improves the precision of the testing batch by 15-70% and reduces the number of mouse clicks by approximately 20%. The solution therefore partially automates the human review after predicting pseudo-labels by similarity-based corrections.