A machine learning framework for enhancing digital experiences in cultural heritage

Abdelhak Belhi, Abdelaziz Bouras, Abdulaziz Khalid Al-Ali, Sebti Foufou

Research output: Contribution to journalArticlepeer-review

Abstract

Purpose: Digital tools have been used to document cultural heritage with high-quality imaging and metadata. However, some of the historical assets are totally or partially unlabeled and some are physically damaged, which decreases their attractiveness and induces loss of value. This paper introduces a new framework that aims at tackling the cultural data enrichment challenge using machine learning. Design/methodology/approach: This framework focuses on the automatic annotation and metadata completion through new deep learning classification and annotation methods. It also addresses issues related to physically damaged heritage objects through a new image reconstruction approach based on supervised and unsupervised learning. Findings: The authors evaluate approaches on a data set of cultural objects collected from various cultural institutions around the world. For annotation and classification part of this study, the authors proposed and implemented a hierarchical multimodal classifier that improves the quality of annotation and increases the accuracy of the model, thanks to the introduction of multitask multimodal learning. Regarding cultural data visual reconstruction, the proposed clustering-based method, which combines supervised and unsupervised learning is found to yield better quality completion than existing inpainting frameworks. Originality/value: This research work is original in sense that it proposes new approaches for the cultural data enrichment, and to the authors’ knowledge, none of the existing enrichment approaches focus on providing an integrated framework based on machine learning to solve current challenges in cultural heritage. These challenges, which are identified by the authors are related to metadata annotation and visual reconstruction.

Original languageEnglish (US)
JournalJournal of Enterprise Information Management
DOIs
StateAccepted/In press - 2020

Keywords

  • Cultural heritage
  • Deep learning
  • Digital heritage
  • Machine learning

ASJC Scopus subject areas

  • Decision Sciences(all)
  • Information Systems
  • Management of Technology and Innovation

Fingerprint Dive into the research topics of 'A machine learning framework for enhancing digital experiences in cultural heritage'. Together they form a unique fingerprint.

Cite this