Gaussian Class-Conditional Training for Secure and Robust Deep Neural Networks

This chapter presents a Gaussian Class-Conditional (GCC) training strategy for deep neural networks. The approach is based on a novel loss that maps the input data onto Gaussian target distributions in the latent space, where the parameters of the target distributions can be optimized for the specific task. For multiclass classification, the mean values of the learned distributions are centered on the vertices of a simplex such that each class is at the same distance from every other class. For metric learning, the distances between similar and dissimilar instance pairs are mapped on distributions with well-separated means. The proposed strategy has many advantages compared to conventional training. First, the optimal decision surface in the latent space is always a hyperplane, yielding a simple and interpretable decision rule. Second, the regularization of the latent space enforces high inter-class separation and low intra-class separation, minimizing the presence of short paths toward neighboring decision regions. The GCC training strategy is applied to two different multimedia problems. In image classification, GCC training provides both improved accuracy and robustness against adversarial perturbations, outperforming models trained with conventional cross-entropy loss and adversarial training. In biometric verification, GCC training yields lower error rates than other state-of-the-art approaches even on challenging and unconstrained datasets.