Early research on learning from data with disagreement in Natural Language Processing (NLP) was often motivated by findings about “anaphoric” referring expressions such as “he”, “she” or “it”— but it turns out that often people disagree on what these pronouns mean, particularly in conversations. Methods for learning from data with disagreements (`learning from crowds’) have been successfully applied to other types of data containing disagreements, and substantial data sets containing multiple judgments on anaphoric reference now exist. But computational models of referring expression interpretation that can effectively learn from such data sets do not yet exist. Training co-reference models ‘from crowds’ has proven to be challenging to design, and there is no consensus over the question of how to test/evaluate interpretation models that take variation into account. This project will focus on addressing such challenges. It will also develop metrics that do justice to interpretative variation for co-reference, and use these metrics to test models. Ideally, the development of these metrics will be informed by cognitive and behavioural evidence on the processing of reference.

You get the opportunity to partly shape this PhD project based on your own preferences. There are, however, a number of topics we would like to address within the project. These include exploring questions such as:

• Are examples of  anaphoric expressions on which people disagree processed differently from other types of anaphoric expressions?
• How should we evaluate systems when people disagree with each other? Can we develop ‘soft’ evaluation metrics for reference and co-reference? Can we perhaps use cognitive evidence to design such metrics
• Can we develop computational models of co-reference resolution ‘from crowds’?

This position also offers the opportunity to develop teaching skills, next to doing research. Typically, PhD candidates dedicate around 15% of their time to teaching in the department, in the form of tutoring or co-supervision of theses.

You will join the Natural Language Processing (NLP) Group, which is part of the AI & Data Science division of the Department of Information and Computing Sciences. Our currents research strengths include the following themes: NLP and Society, Natural Language Generation and, connected with the latter, Vision and Language. In all these areas we work closely with Utrecht University’’s (UU) Language Sciences department. It is foreseen that all PhD projects in the AiNed project will be jointly supervised with Language Sciences. The NLP group contributes to various areas of teaching, for example via UU’s cross-faculty Bachelor and Master’s degrees in Artificial Intelligence. The group is strongly aligned with UU’s focus area Human-centred Artificial Intelligence .

This PhD position is one of five inter-connected PhD positions focussing on variation in NLP, under Utrecht University’s AiNed project “Dealing with Meaning Variation in NLP”, led by Prof. Massimo Poesio. We are simultaneously recruiting for two other positions in this project. We invite you to also check out these interesting vacancies on our website: PhD position in Natural Language Processing: conflicting interpretations in dialogue and PhD position in Natural Language Processing: subjectivity in the detection of problematic language.

We are looking for a motivated researcher with a curious and critical mindset to join our exciting project. We would also like you to bring:

• a Master’s degree in an area relevant to this project, which lies at the interface between NLP, deep learning, and computational cognitive science. Thus, relevant areas would include  Artificial Intelligence, Deep Learning, Computational Cognitive Science, Computer Science, Linguistics, or Statistics.
• A good mastery of deep learning and of NLP is essential. An understanding of coreference and discourse understanding would be a definite bonus.
• Excellent English communication skills.
• You take a strong interest in at least two of the three following areas: (1) coreference, (2) deep learning, and/or (3) NLP.

• A position for 4 years;
• A full-time gross salary that starts at €2,770 and increases to €3,539 per month (scale P of the Collective Labour Agreement Dutch Universities (CAO));
• 8% holiday bonus and 8.3% end-of-year bonus;
• A pension scheme, partially paid parental leave, and flexible employment conditions based on the Collective Labour Agreement Dutch Universities.

In addition to the employment conditions from the CAO for Dutch Universities, Utrecht University has a number of its own arrangements. These include agreements on professional development, leave arrangements, sports and cultural schemes and you get discounts on software and other IT products. We also give you the opportunity to expand your terms of employment through the Employment Conditions Selection Model. This is how we encourage you to grow.

For more information, please visit working at Utrecht University.

A better future for everyone. This ambition motivates our scientists in executing their leading research and inspiring teaching. At Utrecht University, the various disciplines collaborate intensively towards major strategic themes. Our focus is on Dynamics of Youth, Institutions for Open Societies, Life Sciences and Pathways to Sustainability. Shaping science, sharing tomorrow.

At the Faculty of Science, there are 6 departments to make a fundamental connection with: Biology, Chemistry, Information and Computing Sciences, Mathematics, Pharmaceutical Sciences, and Physics. Each of these is made up of distinct institutes that work together to focus on answering some of humanity’s most pressing problems. More fundamental still are the individual research groups – the building blocks of our ambitious scientific projects. Find out more about us on YouTube.

The Department of Information and Computing Sciences is nationally and internationally known for its research in computer science and information science. The Department provides and contributes to the undergraduate programmes in Computer Science, Information Science, and Artificial Intelligence and a number of research Master's programmes in these fields. It employs over 200 people, working in four divisions: Interaction, Algorithms, Data Science & Artificial Intelligence and Software. The atmosphere is collegial and informal.