Mathematical Studies on Deep Neural Networks

Xue-Cheng TAI (Head and Chair Professor, Math)

Current ICTS team members: Xue-Cheng TAI (Leader), Sean HON, Jun FAN

We will research the design of deep neural networks based on partial differential equations (PDEs) for many applications, such as image processing, inverse problems and data analysis.

Deep learning method outperforms classic methods in many applications. However, we lack interpretation and architecture (number of neurons and layers) design guidance for different application problems on hand. In this project, we will use design networks based on PDEs for a lot of applications.

Before the emerging of deep networks, there were artificial models, such as PDEs and variation models for the concerned problems (image processing for example). These models are usually obtained by mathematical analysis, observations and physical laws and principles, etc. These methods can be viewed as an explicit system, i.e. the outputs can be obtained from the input image explicitly by solving the associated variational problems or PDEs. However, these methods are usually too simple and cannot yield satisfactory results because of the complexity of real problems.

The learning method is another methodology to discover the principles implied in the observed data. Usually, this approach views the system that governs the relationship between the input (observation) and output (prediction) as black box, which is determined by the given set of input-output pairs.

Their popularity soared several years ago when deep neural networks (DNNs) outperformed other machine learning methods in many applications, such as natural language processing. However, there is no methodology guiding us to design the networks for different applications. In addition, the existing DNNs lack interpretability.

In this project we research on the design methodology for neural networks architecture designing. Our approach will incorporate an explicit PDEs system and the DNN techniques. Therefore, the proposed method will possess the advantages of PDE system and DNN techniques. Because the general principles of the data from observation or analysis are incorporated in DNNs, which will ease the tasks of network training.