Centrum Wiskunde & Informatica (CWI), Eindhoven University of Technology (TU/e) and Leiden University (UL) have 6 vacancies within the recently approved NWO-ENW (Science domain) Groot project UNRAVEL for talented PhD students, in mathematics, physics, astronomy and computer science on the subject of Unravelling Neural Networks with Structure-Preserving Computing (UNRAVEL).

Our understanding of processes and phenomena in nature and society is being radically transformed by machine learning and the availability of data. This is evident also from the large numbers of researchers embracing deep learning as a tool. At the same time, obstacles and challenges are becoming apparent: most deep-learning approaches require large amounts of data, but in many domains such massive datasets are not available. Furthermore, the emergent behaviour of deep neural networks is usually difficult to interpret. To overcome these drawbacks, the effective use of prior knowledge is key.

The main objective of this project: revealing how neural networks can be made much more effective by incorporating mathematical and physical understanding in their design. The project aims to build a mimetic theory of neural networks that will enable their data-efficient and understandable use for scientific discovery in physics, astronomy and beyond.

To achieve this objective, it is necessary to approach the challenge from different angles. For this reason, our team consists of experts from mathematics, computer science, machine learning, physics and astronomy. The work has been organised in 6 individual projects that will work closely together (more information available from the supervisors):

1. Algorithms for the discovery of interpretable latent variables. This project is about the design and mathematical analysis of algorithms for the discovery of interpretable latent variables (disentangled representations). The PhD project will be on the one hand about the design of such algorithms, taking state-of-the art candidates called Variational Autoencoders as a basis, and on the other hand about mathematically analysing why given algorithms do or do not provide the desired results.
   
   Subproject 1 is executed by Eindhoven University of Technology (TU/e) and is supervised by dr Jim Portegies, Department of Mathematics and Computer Science, Centre for Analysis, Scientific Computing and Applications W&I (CASA). CASA comprises the chairs Scientific Computing (SC) and Applied Analysis (TA). CASA’s major research objective is to develop new and to improve existing mathematical (both analytical and numerical) methods for a wide range of applications in science and engineering.
    
2. Mimetic, hierarchical training algorithms for neural networks. The goal of this subproject is to improve the efficiency and effectiveness of training algorithms in neural networks (algorithms for optimizing neuron weights). Problem-dependent constraints will be incorporated into these training algorithms. The convergence of the mimetic, hierarchical training algorithms will be studied through mathematical analyses. The resulting neural networks will be tested on the basis of discriminating test cases from fluid mechanics (project 5) and astronomy (project 6).
   
   Subproject 2 is executed by Eindhoven University of Technology (TU/e) and is supervised by prof. dr Barry Koren, Department of Mathematics and Computer Science, Centre for Analysis, Scientific Computing and Applications W&I (CASA). CASA comprises the chairs Scientific Computing (SC) and Applied Analysis (TA). CASA’s major research objective is to develop new and to improve existing mathematical (both analytical and numerical) methods for a wide range of applications in science and engineering.
    
3. Dynamic neural networks and their relation to state-space methods. Dynamic neural networks are networks where the action of neurons is described by scalar ordinary differential equations, enabling the simulation of time-dependent phenomena. Previous work on this type of dynamic neural networks revealed an intimate relationship between the structure and parameters of the network with state space models used to describe the underlying system. It implies that the state space system can be used to predict the topology of the neural network. This relation will be exploited to develop an entirely new theory of model reduction techniques directly for neural networks, including structure preservation methods.
   
   Subproject 3 is executed by Eindhoven University of Technology (TU/e) and is supervised by prof. dr Wil Schilders, Department of Mathematics and Computer Science, Centre for Analysis, Scientific Computing and Applications W&I (CASA). CASA comprises the chairs Scientific Computing (SC) and Applied Analysis (TA). CASA’s major research objective is to develop new and to improve existing mathematical (both analytical and numerical) methods for a wide range of applications in science and engineering.
    
4. Machine learning using constrained neural networks and differential equations. In this project mathematical techniques from the fields of ordinary and partial differential equations will be used to better understand and design neural networks. By considering neural networks as the discrete version of a continuous dynamical system, mathematical tools from the field of time integration can be used to analyse the properties of the network, such as its robustness to varying inputs (using nonlinear stability analysis) and the incorporation of constraints in the network (using differential-algebraic equation analysis).
   
   Subproject 4 is executed by Centrum Wiskunde & Informatica (CWI) and supervised by dr Benjamin Sanderse, Scientific Computing group.
    
5. Machine learning for analysis and control of complex fluid flows. In this project we will combine and further develop the scientific and technological know-how of TU/e and SISSA research groups towards the use of Machine Learning techniques. The aim of the project is to develop tools that will allow a deeper understanding, modelling and control capabilities for turbulent flows.
   
   Subproject 5 is executed by University of Technology (TU/e) and is supervised by prof. dr Federico Toschi, Department of Applied Physics, Fluids and Flows (https://www.tue.nl/en/research/research-groups/fluids-and-flows/). It is combined with another PhD position at SISSA in Trieste (Italy). This project will be carried out within the of the and within the PhD programme in Mathematical Analysis, Modelling and Applications at SISSA (Trieste, Italy).
    
6. Neural networks for N-body simulations. In this project, we will solve the gravitational few-body problem using neural networks. This approach is possible because the underlying equations of motion are chaotic. Consequently, solutions obtained using traditional methods on computers only provide a statistical answer, which can as well be obtained by a neural network. We will train deep artificial neural-networks on ensembles of converged solutions to the few-body problem.  The trained network will subsequently be replacing the expensive few-body calculations in large simulations of dense star clusters. This should lead to considerable speed-up compared to more traditional direct integration.
   
   Subproject 6 is executed at Leiden University and supervised by prof. dr Simon Portegies Zwart, Faculty of Science, Computational Astrophysics Leiden, Leiden Observatory.

Each of the topics in itself is of a ground-breaking character. The mathematically inclined projects concentrate on fundamental properties of neural networks that potentially have a big influence on future methodologies for constructing networks and for conducting scientific computational research. The fluid flows and astronomy projects concentrate on specific challenges which serve as test cases for potentially more general strategies.

As a PhD student your tasks are the following:

• perform scientific research in the described domain;
• present results at international conferences;
• publish results in scientific journals;
• participate in activities of the group and the department;
• at the universities: assist staff in teaching undergraduate and graduate courses (at most 20% of the time).

Candidates are required to have a Master’s degree in mathematics, physics, astronomy, computer science or a related discipline, and experience with scientific programming, e.g. Matlab, Python, C and C++. A strong background in applications, knowledge of numerical analysis, ordinary and partial differential equations, discretization techniques, machine learning and neural networks will be beneficial. Preferable qualifications for candidates include proven research talent, an excellent command of English, good academic writing and presentation skills and a creative pro-active team player.

Fixed-term contract: 18 months.

The terms of employment for the PhD student at Centrum Wiskunde & Informatica CWI (subproject 4) are in accordance with the Dutch Collective Labour Agreement for Research Centres ("CAO-onderzoeksinstellingen"). The initial labour agreement will be for a period of 18 months. After a positive evaluation, the agreement will be extended by 30 months. The gross monthly salary, for a PhD student on a full time basis, is €2,407 during the first year and increases to €3,085 over the four year period.

Employees are also entitled to a holiday allowance of 8% of the gross annual salary and a year-end bonus of 8.33%. CWI offers attractive working conditions, including flexible scheduling and help with housing for expat employees.

Please visit our website for more information about our terms of employment: https://www.cwi.nl/jobs/terms-of-employment

The terms of employment for the PhD students at Eindhoven University of Technology (subproject 1, 2, 3 and 5) and Leiden University (subproject 6) are comparable and in accordance with the Collective Labour Agreement of Dutch Universities (“CAO-NU”). For more information about employment conditions:

• For subprojects 1, 2 and 3: Karin Wels-Noordermeer, email: k.h.wels@tue.nl
• For subproject 5: Prof. dr Frederico Tuschi, email: f.toschi@tue.nl
• For subproject 6: Drs. Evelijn Gerstel, email: gerstel@strw.leidenuniv.nl

Centrum Wiskunde & Informatica (CWI)

Centrum Wiskunde & Informatica (CWI) is the Dutch national research institute for mathematics and computer science and is part of the Institutes Organisation of the Dutch Research Council (NWO). The mission of CWI is to conduct pioneering research in mathematics and computer science, generating new knowledge in these fields and conveying it to trade, industry, and society at large.

CWI is an internationally oriented institute, with 160 scientists from approximately 27 countries. The facilities are first-rate and include excellent IT support, career planning, training, and courses.

CWI is located at Science Park Amsterdam, the home of AMS-IX, that is presently developing into a major location of research in the physical sciences in the Netherlands, housing the sciences of the University of Amsterdam as well as several other national research institutes next to CWI.

Scientific Computing

To find more information about the Research group.