We are looking for an ambitious PhD candidate who is driven by the motivation to understand large language models (e.g., ChatGPT) and is not afraid of a challenge.

The employment is dedicated towards carrying out the research as described below, with the aim to publish several academic articles and/or conference proceedings papers in high-impact academic outlets. The PhD candidate will be based at the Department of Methodology and Statistics and will join an active research group around the topic of behavioural inference from complex data. This project is supervised by Dr Bennett Kleinberg, Dr Inga Schwabe and Dr Joris Mulder. 

The position also involves some contributions to teaching giving the candidate the opportunity to build a teaching portfolio during the PhD project. The position and the academic environment in which the candidate will be embedded provide an optimal starting point for an academic career after having obtained the PhD degree. The candidate will follow a customized track of graduate courses as part of the Interuniversity Graduate School of Psychometrics and Sociometrics (IOPS) program which will further develop the candidate’s research skills, background knowledge, and network. 

You are encouraged to make this research your own, by choosing your own specific focus and project implementations.

Project description

Assessing the artificial mind through the marriage of natural language processing and psychometrics

Background
Recent advances in natural language processing and machine learning have led to landmark achievements in artificial intelligence using large language models (LLMs), most prominently GPT-3 and ChatGPT. LLMs harness deep learning methods and are trained on terabytes of text data to generate output often virtually indistinguishable from human-written content. There is evidence that LLMs hold attitudes and beliefs about the world and are able to argue about them similarly to humans. 

While LLMs are becoming more accessible to the general public, our understanding of how these models represent abstract concepts is still limited. The size of LLMs (e.g., 175 billion parameters for GPT-3) makes it infeasible to study the models analytically. This project marries psychometrics and natural language processing by applying techniques traditionally used to measure attitudes and traits in humans to LMMs to arrive at formal models that map observable model behaviour (e.g., responses of LLMs to psychological assessment tools) to theoretical constructs (e.g., attitudes, cognitive abilities). 

Ultimately, this novel approach to studying the behaviour of complex computational models will help us understand where and how the psychological attributes of AI models differ from those of humans. Furthermore, it will be a step towards a more formalised approach to measuring the behaviour of AI systems. This will pave the way for the study and mitigation of biases in LLMs when these models are used by the general public or in broader societal applications.
 
Project aims
The project has two objectives:

1. Where do the psychometric representations of large language models differ from humans? This question will be addressed by collecting large-scale data on various psychological variables from humans and various LLMs, examining how these constructs are represented (e.g., through latent variable modelling or network approaches), and statistically comparing differences between humans and LLMs.
2. How do the representations of psychological attributes differ between LLMs and humans? To understand differences more granularly, we will investigate how representations can be altered (e.g., through reinforcement learning designs) so that we can i) identify what an optimal change strategy is and ii) infer how psychological constructs must have been manifested for a specific strategy to work.

The ideal candidate would:

• have a (research) MSc in a relevant area such as Cognitive Science, (Applied) Statistics, Data Science, Psychology, Artificial Intelligence, Computer Science, or another related discipline;
• have strong programming skills in R, Python and/or otherwise;
• have a strong interest in at least two of the following: Natural Language Processing, machine learning methods, large language models and psychometrics;
• be enthusiastic about cross-disciplinary research (especially across computational and social/behavioural science);
• possess good communication skills and be an efficient team worker
• Tilburg University is an international university, firmly rooted in the region and therefore bilingual. We expect our employees to meet the language level appropriate for the position. For academics, an excellent command of English is required: CEFR level C2.

It is a big plus if a candidate has experience in developing and/or applying advanced statistical methods, experience in the collection of (online) behavioural data and experience with large language models. Previous research experience is also a plus. However, any candidate that shows a demonstrable interest in the analytical as well as empirical part of this project’s topic is encouraged to apply. Candidates who are currently finishing a (research) master’s degree are strongly encouraged to apply.

Fixed-term contract: 12 and 36 months.

This is a position for 1.0 FTE (40 hours per week) of the full-time working week. 

The salary amounts to a minimum of €2.541 and a maximum of €3.247 gross per month for full time employment contract, based on UFO profile PhD student and salary scale number PhD’s of the Collective Labour Agreement for Dutch Universities. Tilburg University uses a neutral salary system for scaling that is based on relevant education and work experience. 

You are entitled to a vacation allowance of 8% and a year-end bonus of 8.3% of your gross annual income. If you work 40 hours per week, you will receive 41 paid days of leave per year.

Tilburg University offers excellent employment conditions. We think flexibility, professional and personal development, and good employee benefits are important. We make clear agreements about career paths and offer all kinds of facilities and schemes to maintain an optimal balance between work and private life. You can also follow numerous training courses, for example, in the areas of leadership skills, personal effectiveness, and career development.  

Tilburg University values an open and inclusive culture. We embrace diversity and encourage the mutual integration of groups of employees and students. We focus on creating equal opportunities for all our employees and students, so that everyone feels at home in our university community.

Tilburg University has a lively campus in beautiful green surroundings that is easily accessible by public transport. We are committed to a sustainable society and challenge you to make an active contribution.

Please visit working at Tilburg University for more information on our terms of employment. 

Tilburg University

Serving society and making it better for its citizens. That is what drives us. Tilburg University's founder, Martinus Cobbenhagen, believed that anyone who wants to understand society must be actively and consciously involved in it. These ideas still form the basis of our university.

Under the motto Understanding society, our more than 2,800 employees develop and transfer knowledge and bring people from different disciplines and organizations together. In this way, we want to contribute to solving complex social issues. In doing so, we work from the fields of economics, business and entrepreneurship, social and behavioral sciences, law and administration, humanities and digital sciences, and theology

We educate our 20,000 students of 132 nationalities to become Tilburg University Shaped Professionals: responsible and entrepreneurial thinkers, driven by solidarity, a sense of responsibility and empathy, who are able to influence and give direction to a rapidly changing society in an innovative way.

About Tilburg School of Social and Behavioral Sciences
Tilburg School of Social and Behavioral Sciences (TSB) is one of the five faculties of Tilburg University. The teaching and research of the Tilburg School of Social and Behavioral Sciences are organized around the themes of “Adaptive societies, organizations, and workers”, “Healthy life span”, and “Personalized prevention and care”. The School’s inspiring working environment challenges its workers to realize their ambitions, involvement and cooperation are essential to achieve this.   
Tilburg School of Social and Behavioral Sciences | Tilburg University

Department of Methodology and Statistics

The Department of Methodology and Statistics is an internationally renowned group, holding several experts in data science methods, latent variable methods, psychometrics, meta-research, survey methodology, and other applied statistics fields. The department has a strong tradition of working with the other (substantive) research programs in our faculty. The department is responsible for the teaching and research in the area of methodology and statistics for the social and behavioral sciences, the Data Science programs (including the joint BSc in Data Science together with the Technical University Eindhoven), and the Liberal Arts and Science program of Tilburg University. The department is a member of the Interuniversity Graduate School for Psychometrics and Sociometrics (IOPS).