Lecture 8 (15 Dec. 2020) - Lecture will take place only online (use link below), no physical presence
Résumé de section
Speaker:
Prof. Peter Kovacs
Department of Numerical Analysis, University of Budapest, Hungary
Title:
Towards model-driven neural networks
Abstract:
Analysis of signals by means of mathematical transformations proved to be an effective method in various aspects, including filtering, system identification, feature extraction, classification, etc. The most widely used method in transform-domain techniques operates with fixed basic functions like trigonometric functions in the Fourier transform, Walsh functions in Walsh–Fourier transform, mother wavelet function for wavelet transforms, etc. These transformations can be used to extract features and to reduce the dimension of the original data. Note that the relevance of the extracted information is in the proper choice of the function system, which also incorporates domain knowledge. However, these handcrafted features are usually suboptimal with respect to the whole learning process. Deep learning (DL) techniques along with representation learning provide good alternatives to extract discriminative information from the raw data. Despite their advantages, DL techniques continue to raise several concerns. Their improved efficiency comes at the cost of losing the explainability. Indeed, due to the large number of nonlinear connections between the model parameters, DL approaches can be considered as black-box methods, where the parameters have no physical meaning and are difficult or impossible to interpret. In this talk, we incorporate the representation abilities of adaptive orthogonal transformations and the prediction abilities of neural networks (NNs) in form of hybrid models. This is a recent trend in signal processing where the mathematical model-based principles and the data-driven machine learning disciplines are combined. In order to demonstrate the potential in these model-driven deep learning techniques, we present two case-studies. First, we consider the problem of thermographic image regression for non-destructive material testing. Then, motivated by the classification of biomedical signals and by the adaptive orthogonal transformations, we introduce VPNet, a novel model-driven NN architecture, which has the advantage of learnable features, interpretable parameters, and compact network structures.
