FaceHack: Attacking Facial Recognition Systems Using Malicious Facial Characteristics

Esha Sarkar, Hadjer Benkraouda, Gopika Krishnan, Homer Gamil, Michail Maniatakos

Research output: Contribution to journalArticlepeer-review


Recent advances in machine learning have opened up new avenues for its extensive use in real-world applications. Facial recognition, specifically, is used from simple friend suggestions in social-media platforms to critical security applications for biometric validation in automated border control at airports. Considering these scenarios, security vulnerabilities of such facial recognition systems pose serious threats with severe outcomes. Recent work demonstrated that Deep Neural Networks (DNNs), typically used in facial recognition systems, are susceptible to backdoor attacks; in other words, the DNNs turn malicious in the presence of a unique trigger. Detection mechanisms have focused on identifying these distinct trigger-based outliers statistically or through reconstructing them. In this work, we propose the use of facial characteristics as triggers to backdoored facial recognition systems. Additionally, we demonstrate that these attacks can be realised on real-time facial recognition systems. Depending on the attack scenario, the changes in the facial attributes may be embedded artificially using social-media filters or introduced naturally through facial muscle movements. We evaluate the success of the attack and validate that it does not interfere with the performance criteria of the model. We also substantiate that our triggers are undetectable by thoroughly testing them on state-of-the-art defense and detection mechanisms.

Original languageEnglish (US)
Pages (from-to)361-372
Number of pages12
JournalIEEE Transactions on Biometrics, Behavior, and Identity Science
Issue number3
StatePublished - Jul 1 2022


  • Machine learning
  • attack
  • backdoor
  • facial recognition
  • privacy
  • security
  • trojan

ASJC Scopus subject areas

  • Instrumentation
  • Computer Vision and Pattern Recognition
  • Computer Science Applications
  • Artificial Intelligence


Dive into the research topics of 'FaceHack: Attacking Facial Recognition Systems Using Malicious Facial Characteristics'. Together they form a unique fingerprint.

Cite this