Description
Over time, all paintings tend to deteriorate because of aging and storage conditions. Cracks formed on the surface of paintings are the most common types of degradation. The automatic detection and mapping of cracks is a key problem for art restoration, conservation treatments, and analyses such as art authentication. However, this task is challenging, and existing methods often require tedious manual effort in feature engineering and parameter tuning.
Methods based on deep learning (DL) can learn features from data and show promising results. However, DL relies on large sets of previously annotated data, which are not easily available in this application. Furthermore, the high variability of the crack detection task, encompassing different types of paintings and a variety of shapes and patterns of cracks, makes these methods hard to apply to previously unseen paintings.
To address these challenges, we developed a deep active learning (DAL) method for crack detection. DAL methods start with a model trained on little annotated data, perform their tasks, and then suggest to the oracle (here, a human annotator) new samples to annotate before retraining the model. By retraining the model iteratively in an efficient way, our method needs much less data than traditional DL, learns continuously from new annotations by a human-in-the-loop, enables learning from partially annotated data, and performs better on previously unseen paintings. It can also leverage several modalities at the same time, namely photographs in the visible and infrared spectral range and X-ray images.
Our method is equipped with a web interface to be used easily by art restorers and non-experts. It is able to improve itself continuously with the input from all users. We demonstrate the application of the proposed crack detection tool in a concrete use case as a means of supporting the restoration of old master paintings.
