TY - JOUR
T1 - Investigating low-delay deep learning-based cultural image reconstruction
AU - Belhi, Abdelhak
AU - Al-Ali, Abdulaziz Khalid
AU - Bouras, Abdelaziz
AU - Foufou, Sebti
AU - Yu, Xi
AU - Zhang, Haiqing
N1 - Publisher Copyright:
© 2020, Springer-Verlag GmbH Germany, part of Springer Nature.
PY - 2020/12
Y1 - 2020/12
N2 - Numerous cultural assets host a great historical and moral value, but due to their degradation, this value is heavily affected as their attractiveness is lost. One of the solutions that most heritage organizations and museums currently choose is to leverage the knowledge of art and history experts in addition to curators to recover and restore the damaged assets. This process is labor-intensive, expensive and more often results in just an assumption over the damaged or missing region. In this work, we tackle the issue of completing missing regions in artwork through advanced deep learning and image reconstruction (inpainting) techniques. Following our analysis of different image completion and reconstruction approaches, we noticed that these methods suffer from various limitations such as lengthy processing times and hard generalization when trained with multiple visual contexts. Most of the existing learning-based image completion and reconstruction techniques are trained on large datasets with the objective of retrieving the original data distribution of the training samples. However, this distribution becomes more complex when the training data is diverse making the training process difficult and the reconstruction inefficient. Through this paper, we present a clustering-based low-delay image completion and reconstruction approach which combines supervised and unsupervised learning to address the highlighted issues. We compare our technique to the current state of the art using a real-world dataset of artwork collected from various cultural institutions. Our approach is evaluated using statistical methods and a surveyed audience to better interpret our results objectively and subjectively.
AB - Numerous cultural assets host a great historical and moral value, but due to their degradation, this value is heavily affected as their attractiveness is lost. One of the solutions that most heritage organizations and museums currently choose is to leverage the knowledge of art and history experts in addition to curators to recover and restore the damaged assets. This process is labor-intensive, expensive and more often results in just an assumption over the damaged or missing region. In this work, we tackle the issue of completing missing regions in artwork through advanced deep learning and image reconstruction (inpainting) techniques. Following our analysis of different image completion and reconstruction approaches, we noticed that these methods suffer from various limitations such as lengthy processing times and hard generalization when trained with multiple visual contexts. Most of the existing learning-based image completion and reconstruction techniques are trained on large datasets with the objective of retrieving the original data distribution of the training samples. However, this distribution becomes more complex when the training data is diverse making the training process difficult and the reconstruction inefficient. Through this paper, we present a clustering-based low-delay image completion and reconstruction approach which combines supervised and unsupervised learning to address the highlighted issues. We compare our technique to the current state of the art using a real-world dataset of artwork collected from various cultural institutions. Our approach is evaluated using statistical methods and a surveyed audience to better interpret our results objectively and subjectively.
KW - Deep learning
KW - Digital heritage
KW - Image clustering
KW - Image inpainting
KW - Image reconstruction
KW - Low-delay reconstruction
UR - http://www.scopus.com/inward/record.url?scp=85086160973&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85086160973&partnerID=8YFLogxK
U2 - 10.1007/s11554-020-00975-y
DO - 10.1007/s11554-020-00975-y
M3 - Article
AN - SCOPUS:85086160973
SN - 1861-8200
VL - 17
SP - 1911
EP - 1926
JO - Journal of Real-Time Image Processing
JF - Journal of Real-Time Image Processing
IS - 6
ER -