Repository logo
 

Analysis of artifacts in shell-based image inpainting: why they occur and how to eliminate them

Accepted version
Peer-reviewed

Type

Article

Change log

Authors

Hocking, L Robert 
Holding, Thomas 
Schoenlieb, Carola-Bibiane  ORCID logo  https://orcid.org/0000-0003-0099-6306

Abstract

In this paper we study a class of fast geometric image inpainting methods based on the idea of lling the inpainting domain in successive shells from its boundary inwards. Image pixels are lled by assigning them a color equal to a weighted average of their already lled neighbors. However, there is exibility in terms of the order in which pixels are lled, the weights used for averaging, and the neighborhood that is averaged over. Varying these degrees of freedom leads to di erent algorithms, and indeed the literature contains several methods falling into this general class. All of them are very fast, but at the same time all of them leave undesirable artifacts such as "kinking" (bending) or blurring of extrapolated isophotes. Our objective in this paper is to build a theoretical model in order to understand why these artifacts occur and what, if anything, can be done about them. Our model is based on two distinct limits: a continuum limit in which the pixel width h -> 0 and an asymptotic limit in which h > 0 but h << 1. The former will allow us to explain "kinking" artifacts (and what to do about them) while the latter will allow us to understand blur. Both limits are derived based on a connection between the class of algorithms under consideration and stopped random walks. At the same time, we consider a semi-implicit extension in which pixels in a given shell are solved for simultaneously by solving a linear system. We prove (within the continuum limit) that this extension is able to completely eliminate kinking artifacts, which we also prove must always be present in the direct method. Finally, we show that although our results are derived in the context of inpainting, they are in fact abstract results that apply more generally. As an example, we show how our theory can also be applied to a problem in numerical linear algebra.

Description

Keywords

46 Information and Computing Sciences, 49 Mathematical Sciences, 4603 Computer Vision and Multimedia Computation

Journal Title

Foundations of Computational Mathematics

Conference Name

Journal ISSN

1615-3375
1615-3383

Volume Title

Publisher

Springer Nature
Sponsorship
Engineering and Physical Sciences Research Council (EP/H023348/1)
Engineering and Physical Sciences Research Council (EP/M00483X/1)
Leverhulme Trust (RPG-2015-250)
Engineering and Physical Sciences Research Council (EP/N014588/1)
European Commission Horizon 2020 (H2020) Marie Sk?odowska-Curie actions (691070)
Alan Turing Institute (unknown)
Engineering and Physical Sciences Research Council (EP/J009539/1)
European Commission Horizon 2020 (H2020) Marie Sk?odowska-Curie actions (777826)