On the detection of adversarial attacks against deep neural networks

Weiyu Wang; Quanyan Zhu

doi:10.1145/3140368.3140373

On the detection of adversarial attacks against deep neural networks

Weiyu Wang, Quanyan Zhu

Electrical and Computer Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Deep learning model has been widely studied and proven to achieve high accuracy in various pattern recognition tasks, especially in image recognition. However, due to its non-linear architecture and high-dimensional inputs, its ill-posedness [1] towards adversarial perturbations - small deliberately crafted perturbations on the input will lead to completely different outputs, has also attracted researchers' attention. 1 This work takes the traffic sign recognition system on the selfdriving car as an example, and aims at designing an additional mechanism to improve the robustness of the recognition system. It uses a machine learning model which learns the results of the deep learning model's predictions, with human feedback as labels and provides the credibility of current prediction. The mechanism makes use of both the input image and the recognition result as the sample space, querying a human user the True/False of current classification result the least number of times, and completing the task of detecting adversarial attacks.

Original language	English (US)
Title of host publication	SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017
Publisher	Association for Computing Machinery, Inc
Pages	27-30
Number of pages	4
ISBN (Electronic)	9781450352031
DOIs	https://doi.org/10.1145/3140368.3140373
State	Published - Nov 3 2017
Event	10th Workshop on Automated Decision Making for Active Cyber Defense, SafeConfig 2017 - Dallas, United States Duration: Nov 3 2017 → …

Publication series

Name	SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017

Other

Other	10th Workshop on Automated Decision Making for Active Cyber Defense, SafeConfig 2017
Country/Territory	United States
City	Dallas
Period	11/3/17 → …

Keywords

Active Learning
Adversarial Machine Learning
Deep Neural Network
Machine Learning Security
Support Vector Machine

ASJC Scopus subject areas

Artificial Intelligence
Computational Theory and Mathematics
Computer Science Applications

Access to Document

10.1145/3140368.3140373

Cite this

Wang, W., & Zhu, Q. (2017). On the detection of adversarial attacks against deep neural networks. In SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017 (pp. 27-30). (SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017). Association for Computing Machinery, Inc. https://doi.org/10.1145/3140368.3140373

On the detection of adversarial attacks against deep neural networks. / Wang, Weiyu; Zhu, Quanyan.
SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017. Association for Computing Machinery, Inc, 2017. p. 27-30 (SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Wang, W & Zhu, Q 2017, On the detection of adversarial attacks against deep neural networks. in SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017. SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017, Association for Computing Machinery, Inc, pp. 27-30, 10th Workshop on Automated Decision Making for Active Cyber Defense, SafeConfig 2017, Dallas, United States, 11/3/17. https://doi.org/10.1145/3140368.3140373

Wang W, Zhu Q. On the detection of adversarial attacks against deep neural networks. In SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017. Association for Computing Machinery, Inc. 2017. p. 27-30. (SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017). doi: 10.1145/3140368.3140373

Wang, Weiyu ; Zhu, Quanyan. / On the detection of adversarial attacks against deep neural networks. SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017. Association for Computing Machinery, Inc, 2017. pp. 27-30 (SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017).

@inproceedings{7316950f83fa4968b9d7da2e8751c7f7,

title = "On the detection of adversarial attacks against deep neural networks",

abstract = "Deep learning model has been widely studied and proven to achieve high accuracy in various pattern recognition tasks, especially in image recognition. However, due to its non-linear architecture and high-dimensional inputs, its ill-posedness [1] towards adversarial perturbations - small deliberately crafted perturbations on the input will lead to completely different outputs, has also attracted researchers' attention. 1 This work takes the traffic sign recognition system on the selfdriving car as an example, and aims at designing an additional mechanism to improve the robustness of the recognition system. It uses a machine learning model which learns the results of the deep learning model's predictions, with human feedback as labels and provides the credibility of current prediction. The mechanism makes use of both the input image and the recognition result as the sample space, querying a human user the True/False of current classification result the least number of times, and completing the task of detecting adversarial attacks.",

keywords = "Active Learning, Adversarial Machine Learning, Deep Neural Network, Machine Learning Security, Support Vector Machine",

author = "Weiyu Wang and Quanyan Zhu",

note = "Publisher Copyright: {\textcopyright} 2017 Copyright is held by the owner/author(s).; 10th Workshop on Automated Decision Making for Active Cyber Defense, SafeConfig 2017 ; Conference date: 03-11-2017",

year = "2017",

month = nov,

day = "3",

doi = "10.1145/3140368.3140373",

language = "English (US)",

series = "SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017",

publisher = "Association for Computing Machinery, Inc",

pages = "27--30",

booktitle = "SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017",

}

TY - GEN

T1 - On the detection of adversarial attacks against deep neural networks

AU - Wang, Weiyu

AU - Zhu, Quanyan

PY - 2017/11/3

Y1 - 2017/11/3

N2 - Deep learning model has been widely studied and proven to achieve high accuracy in various pattern recognition tasks, especially in image recognition. However, due to its non-linear architecture and high-dimensional inputs, its ill-posedness [1] towards adversarial perturbations - small deliberately crafted perturbations on the input will lead to completely different outputs, has also attracted researchers' attention. 1 This work takes the traffic sign recognition system on the selfdriving car as an example, and aims at designing an additional mechanism to improve the robustness of the recognition system. It uses a machine learning model which learns the results of the deep learning model's predictions, with human feedback as labels and provides the credibility of current prediction. The mechanism makes use of both the input image and the recognition result as the sample space, querying a human user the True/False of current classification result the least number of times, and completing the task of detecting adversarial attacks.

AB - Deep learning model has been widely studied and proven to achieve high accuracy in various pattern recognition tasks, especially in image recognition. However, due to its non-linear architecture and high-dimensional inputs, its ill-posedness [1] towards adversarial perturbations - small deliberately crafted perturbations on the input will lead to completely different outputs, has also attracted researchers' attention. 1 This work takes the traffic sign recognition system on the selfdriving car as an example, and aims at designing an additional mechanism to improve the robustness of the recognition system. It uses a machine learning model which learns the results of the deep learning model's predictions, with human feedback as labels and provides the credibility of current prediction. The mechanism makes use of both the input image and the recognition result as the sample space, querying a human user the True/False of current classification result the least number of times, and completing the task of detecting adversarial attacks.

KW - Active Learning

KW - Adversarial Machine Learning

KW - Deep Neural Network

KW - Machine Learning Security

KW - Support Vector Machine

UR - http://www.scopus.com/inward/record.url?scp=85037084091&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85037084091&partnerID=8YFLogxK

U2 - 10.1145/3140368.3140373

DO - 10.1145/3140368.3140373

M3 - Conference contribution

AN - SCOPUS:85037084091

T3 - SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017

SP - 27

EP - 30

BT - SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017

PB - Association for Computing Machinery, Inc

T2 - 10th Workshop on Automated Decision Making for Active Cyber Defense, SafeConfig 2017

Y2 - 3 November 2017

ER -

On the detection of adversarial attacks against deep neural networks

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this