In Natural Language Processing (NLP), human annotators are frequently needed to tell researchers what a given expression “means”, by assigning the expression a label. When human judges disagree about a label (e.g., whether an utterance is offensive or not), it is important that these disagreements be taken into account, as opposed to simply aggregating the values e.g., using reconciliation or majority voting. Such disagreements are now generally recognized to provide information rather than being noise. We also need to recognize that uncertainties may originate from different sources: in particular, they may be due to semantic ambiguity or to subjective bias.

You get the opportunity to partly shape the PhD project based on your own preferences. There are, however, a number of topics we would like to address within the project We aim to investigate:

• whether the differences between various sources of disagreement (e.g., noise, ambiguity, and subjective bias) can be detected using statistical models.
• how to use datasets containing different types of variation to train and evaluate NLP models, starting from datasets such as those created in DALI and LeWiDi. It may also leverage other datasets that have recently appeared, e.g., for natural language inference.
• whether the obvious candidates for soft evaluation metrics (cross-entropy, Kullback-Leibler divergence) apply to all these tasks.
• to what extent variations in one person’s verbal behaviour can be understood mathematically in the same way as variations between different speakers. 

This PhD position is one of six inter-connected PhD positions focussing on uncertainty in NLP, under Utrecht University’s AiNed project “Dealing with Meaning Variation in NLP”, which will be led by Professor Massimo Poesio. We are simultaneously recruiting for another of the six PhD position in this project  We invite you to also check out this interesting vacancy on our website.

You are a motivated researcher with a curious and critical mindset. You hold a Master’s degree in an area relevant to this project; this could be Artificial Intelligence, Computational Cognitive Science, Computing Science, Linguistics, or Statistics. A good mastery of statistics and deep learning is essential; experience with NLP would be beneficial. Excellent English communication skills are necessary.

You take a strong interest in at least two of the three following areas:

1. machine learning;
2. natural language, and;
3. experimental psychology.

A position for 4 years;

• A full-time gross salary that starts at €2,541 and increases to €3,247 per month (scale P of the Collective Labour Agreement Dutch Universities (CAO));
• 8% holiday bonus and 8.3% end-of-year bonus;
• A pension scheme, partially paid parental leave, and flexible employment conditions based on the Collective Labour Agreement Dutch Universities.

In addition to the employment conditions laid down in the CAO for Dutch Universities, Utrecht University has a number of its own arrangements. For example, there are agreements on professional development, leave arrangements and sports. We also give you the opportunity to expand your terms of employment via the Employment Conditions Selection Model. This is how we like to encourage you to continue to grow.

For more information, please visit working at Utrecht University.

Sharing science, shaping tomorrow. A better future for everyone. This ambition motivates our scientists in executing their leading research and inspiring teaching. At Utrecht University, the various disciplines collaborate intensively towards major strategic themes. Our focus is on Dynamics of Youth, Institutions for Open Societies, Life Sciences and Sustainability.

At the Faculty of Science, there are 6 departments to make a fundamental connection with: Biology, Chemistry, Information and Computing Sciences, Mathematics, Pharmaceutical Sciences, and Physics. Each of these is made up of distinct institutes that work together to focus on answering some of humanity's most pressing problems. More fundamental still are the individual research groups – the building blocks of our ambitious scientific projects. Find out more about us.

You will join the Natural Language Processing (NLP) Group, which is part of the AI and Data Science division of the Department of Information and Computing Sciences. In our constantly changing society, the Department of Information and Computing Sciences is constantly looking for new ways to push the boundaries of both science and social application.

The NLP group was founded in 2018, when Professor Kees van Deemter and Dr. Dong Nguyen joined the UU. Over the years, various people joined the NLP group, broadening its range and reinforcing its focus on empirical and theoretical aspects of NLP, covering both symbolic and statistical approaches. An important recent focus is explainability of NLP models.

Our currents research strengths include a theme called NLP and Society, a theme called Natural Language Generation and, connected with the latter, a theme Vision and Language. In all these areas we work closely with the UU’s Language Sciences department. It is foreseen that all PhD projects in the AiNed project “Dealing With Meaning Variation in NLP” will be jointly supervised with Language Sciences. The NLP group contributes to various areas of teaching, for example via the UU’s cross-faculty Bachelor and Master’s degrees in Artificial Intelligence. The group is strongly aligned with the UU’s focus area Human-centred Artificial Intelligence .