The Ph.D. position will focus on the research and development of solutions to test and validate AI-intensive systems. AI-intensive systems involve multiple internal stages, pipelines, models, as well as interaction with other non-AI-based components. Often, they also process multimodal data requiring human interpretation. The question of quality assurance for AI thus is more refined than a single evaluation or testing approach, which often is seen today. In this Ph.D. project, we therefore will focus on bridging evaluation and testing methodologies from the software engineering and applied machine learning domains.

Supervisors: Dr Cynthia C. S. Liem, Dr Annibale Panichella

Related works:

• C. S. Liem and A. Panichella. "Oracle Issues in Machine Learning and Where to Find Them." RAISE 2020.
• Panichella, C. C. S. Liem. "What Are We Really Testing in Mutation Testing for Machine Learning? A Critical Reflection,” ICSE-NIER 2021.
• J. Abdessalem, A. Panichella, S. Nejati, L. C. Briand, and T. Stifter. "Testing autonomous cars for feature interaction failures using many-objective search." ASE 2018.
• C. S. Liem and C. Mostert, “Can't Trust the Feeling? How Open Data Reveals Unexpected Behavior of High-level Music Descriptors,” ISMIR 2020.

This project is a collaboration between the Multimedia Computing (MMC) Group at the Intelligent Systems (INSY) Department, and the Software Engineering Research Group (SERG) at the Software Technology (ST) Department. The INSY and ST departments closely work together in the faculty’s Computer Science research and education.

The Multimedia Computing (MMC) Group conducts research on new methodological and algorithmic concepts for analyzing, interpreting, enriching, modeling, searching and recommending multimedia data available in stand-alone collections or connected in (complex) networked data structures. The work being carried out in the MMC Group encompasses a broad palette of research directions in the domains of multimodal analysis and processing, network data computing and multimedia information systems, with a strong influx of the expertise from the domain of human factors.

The Software Engineering Research Group (SERG) is part of the department of Software Technology, faculty of Electrical Engineering, Mathematics, and Computer Science, and a member of the research school IPA. The main research areas for the group include but are not limited to software analytics, software testing, software quality and maintenance, software evolution and search-based software engineering. The involved researchers have developed numerous techniques to make it easier for software developers to (1) understand, (2) maintain and (3) test existing software systems.

The current PhD position will connect MMC’s research line on validation and validity in multimedia information systems with SERG”s research line on Computational Intelligence for Software Engineering.

An excellent master's degree (or equivalent) in computer science. Required: experience in Software Engineering, Software Testing, and Machine Learning. Preferred: experience in Statistics and Multimedia Processing.

- Strong programming skills, ideally demonstrated through contributions to open-source libraries. Required: Java and Python. Preferred: C++.

- Good communication and presentation skills in English.

- Ability to work in a team.

- Commitment to research and engineering.

Fixed-term contract: 4 jaar.

TU Delft offers PhD-candidates a 4-year contract, with an official go/no go progress assessment after one year. Salary and benefits are in accordance with the Collective Labour Agreement for Dutch Universities, increasing from € 2395 per month in the first year to € 3061 in the fourth year. As a PhD candidate you will be enrolled in the TU Delft Graduate School. The TU Delft Graduate School provides an inspiring research environment with an excellent team of supervisors, academic staff and a mentor. The Doctoral Education Programme is aimed at developing your transferable, discipline-related and research skills.

The TU Delft offers a customisable compensation package, discounts on health insurance and sport memberships, and a monthly work costs contribution. Flexible work schedules can be arranged. For international applicants we offer the Coming to Delft Service and Partner Career Advice to assist you with your relocation.

Delft University of Technology

Delft University of Technology is built on strong foundations. As creators of the world-famous Dutch waterworks and pioneers in biotech, TU Delft is a top international university combining science, engineering and design. It delivers world class results in education, research and innovation to address challenges in the areas of energy, climate, mobility, health and digital society. For generations, our engineers have proven to be entrepreneurial problem-solvers, both in business and in a social context. At TU Delft we embrace diversity and aim to be as inclusive as possible (see our Code of Conduct). Together, we imagine, invent and create solutions using technology to have a positive impact on a global scale.

Challenge. Change. Impact! 

Faculty Electrical Engineering, Mathematics and Computer Science

The Faculty of Electrical Engineering, Mathematics and Computer Science (EEMCS) brings together three disciplines - electrical engineering, mathematics and computer science. Combined, they reinforce each other and are the driving force behind the technology we use in our daily lives. Technology such as the electricity grid, which our faculty is helping to make future-proof. We are also working on a world in which humans and computers reinforce each other. We are mapping out disease processes using single cell data, and using mathematics to simulate gigantic ash plumes after a volcanic eruption. There is plenty of room here for ground-breaking research. We educate innovative engineers and have excellent labs and facilities that underline our strong international position. In total, more than 1,100 employees and 4,000 students work and study in this innovative environment.

Click here to go to the website of the Faculty of Electrical Engineering, Mathematics and Computer Science.