PhD Candidate, Formal Methods in Natural Language Processing

Applications are invited for a fully funded PhD candidature on the use of Formal Methods to enhance the efficiency, transparency and the understanding of Transformer-based language models.

While large language models (LLMs) have proven successful in many areas of Natural Language Processing, they suffer from high data and resource usage, and display limited generalization capacity in tasks that humans excel at. In this PhD project you will have the opportunity to investigate how formal methods can help in developing more efficient and more transparent models for Natural Language Understanding. Specifically, you will investigate the use of implicit or explicit structural bias in Transformer-based language models to reduce training data and model parameters; additionally, you will look at novel techniques for evaluating models for their generalization capabilities on Natural Language Understanding tasks such as Natural Language Inference, possibly in a multilingual and multimodal setting.

The specific project content is to be decided between the applicants’ interest and the expertise of the supervisor, dr. Gijs Wijnholds: https://gijswijnholds.github.io
Topics include (but are not limited to):

• Using logical methods to define task-relevant constraints on LLM finetuning;
• Incorporating structured representations in regularized training of smaller language models;
• Assessing the generalization capacity of Transformer-based models in the context of formal language theory, model probing, Natural Language Inference;
• Evaluation of Natural Language Understanding models in the presence of ambiguity and/or annotator disagreement;
• Understanding multilingual Natural Language Inference in Vision-Language Models;

In this position you are expected to develop your own research line; as such we ask for a brief research proposal (max. 1 page) which outlines a project you would like to work on.
You will be embedded in the Text Mining and Retrieval group at Leiden University, supervised by dr. Gijs Wijnholds and dr. Suzan Verberne.

Key responsibilities

• Conduct original research in the field of Natural Language Understanding, focusing on the combination of formal methods and Transformer-based language models;
• Publish and present scientific work at international journals and conferences;
• Engage in collaboration with other researchers within and outside LIACS;
• Assist in relevant teaching activities;

Terms and conditions

We offer a full-time 1-year term position for initially one year. After a positive evaluation of the progress of the thesis, personal capabilities and compatibility the appointment will be extended by a further three years. Salary range from € 2.770, - to € 3.539, - gross per month (pay scale P in accordance with the Collective Labour Agreement for Dutch Universities).

Leiden University offers an attractive benefits package with additional holiday (8%) and end-of-year bonuses (8.3%), training and career development and sabbatical leave. Our individual choices model gives you some freedom to assemble your own set of terms and conditions. Candidates from outside the Netherlands may be eligible for a substantial tax break.

All our PhD students are embedded in the Leiden University Graduate School of Science www.graduateschools.leidenuniv.nl. Our graduate school offers several PhD training courses at three levels: professional courses, skills training and personal effectiveness. In addition, advanced courses to deepen scientific knowledge are offered by the research school.

D&I statement
Diversity and inclusion are core values of Leiden University. Leiden University is committed to becoming an inclusive community which enables all students and staff to feel valued and respected and to develop their full potential. Diversity in experiences and perspectives enriches our teaching and strengthens our research. High quality teaching and research is inclusive.

Research at our faculty

The Faculty of Science is a world-class faculty where staff and students work together in a dynamic international environment. It is a faculty where personal and academic development are top priorities. Our people are committed to expand fundamental knowledge by curiosity and to look beyond the borders of their own discipline; their aim is to benefit science, and to contribute to addressing the major societal challenges of the future.

The research carried out at the Faculty of Science is very diverse, ranging from mathematics, information science, astronomy, physics, chemistry and bio-pharmaceutical sciences to biology and environmental sciences. The research activities are organised in eight institutes. These institutes offer eight bachelor's and twelve master's programmes. The faculty has grown strongly in recent years and now has more than 2.300 staff and almost 5,000 students. We are located at the heart of Leiden's Bio Science Park, one of Europe's biggest science parks, where university and business life come together. For more information, see www.universiteitleiden.nl/en/science and https://workingat.leiden.edu/

The Leiden Institute of Advanced Computer Science (LIACS) is the Artificial Intelligence and Computer Science Institute in the Faculty of Science of Leiden University. We offer courses at the Bachelor and Master of Science level in Artificial Intelligence, Computer Science, ICT in Business, Media Technology, and Bioinformatics. According to an independent research visitation, we are one of the foremost computer science departments of the Netherlands. We strive for excellence in a caring institute, where excellence, fun, and diversity go hand in hand. We offer a clear and inviting career path to young and talented scientists with the ambition to grow. For more information about LIACS, see https://www.cs.leiden.edu