Adversarial Risk Bounds through Sparsity based Compression

Authors

E. R. Balda, N. Koep, A. Behboodi, R. Mathar,

Abstract

        Neural networks have been shown to be vulnerable against minor adversarial perturbations of their inputs, especially for high dimensional data under $\ell_\infty$ attacks.To combat this problem, techniques like adversarial training have been employed to obtain models that are robust on the training set.However, the robustness of such models against adversarial perturbations may not generalize to unseen data.To study how robustness generalizes, recent works assume that the inputs have bounded $\ell_2$-norm in order to bound the adversarial risk for $\ell_\infty$ attacks with no explicit dimension dependence.In this work, we focus on $\ell_\infty$ attacks with $\ell_\infty$ bounded inputs and prove margin-based bounds.Specifically, we use a compression-based approach that relies on efficiently compressing the set of tunable parameters without distorting the adversarial risk. To achieve this, we apply the concept of effective sparsity and effective joint sparsity on the weight matrices of neural networks.This leads to bounds with no explicit dependence on the input dimension, neither on the number of classes.Our results show that neural networks with approximately sparse weight matrices not only enjoy enhanced robustness but also better generalization. Finally, empirical simulations show that the notion of effective joint sparsity plays a significant role in generalizing robustness to $\ell_\infty$ attacks.

http://proceedings.mlr.press/v108/balda20a.html

BibTEX Reference Entry 

@inproceedings{BaKoBeMa20,
	author = {Emilio Rafael Balda and Niklas Koep and Arash Behboodi and Rudolf Mathar},
	title = "Adversarial Risk Bounds through Sparsity based Compression",
	pages = "3816--3825",
	booktitle = "International Conference on Artificial Intelligence and Statistics (AISTATS)",
	address = {Online},
	month = Aug,
	year = 2020,
	hsb = RWTH-2021-08224 ,
	}

Downloads

 Download paper  Download bibtex-file

This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights there in are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.