Dr Thomas Robinson

This course is available on the MPhil/PhD in Social Research Methods. This course is available as an outside option to students on other programmes where regulations permit.

Students should have a firm grounding in applied statistics, for example by having completed Applied Regression Analysis (MY452), or similar. It would be helpful, but not essential, to have some prior knowledge of general machine learning concepts (regularisation, cross validation, gradient descent), for example by taking MY474 concurrently.

Recent years have seen huge advancements in the capability and application of machine learning methods, especially within the areas referred to as "deep" and reinforcement learning. Moreover, the rising prominence of generative artificial intellegence models offers not only promising new methods but also new areas of social scientific study.

This course will introduce students to key concepts within deep learning necessary for understanding how these methods are being applied in social scientific research. We will cover the fundamentals of building and training neural networks, including specific model architectures like autoencoder, adversarial, and convolutional neural networks. We will also introduce general features such as embeddings, dropout, and attention. This course will place particular emphasis on how deep and reinforcement learning methods can be used to understand social phenomena, including when our data comes from non-quantitative sources like images and text. We will, for example, examine recent efforts to analyse the content of images, consider how to measure concept similarity in text, and explore how insights from reinforcement learning have been deployed in social scientific and industrial contexts. The course ends with a synthesis of these fundamentals to motivate and explain large-language models and their uses, including for zero-and few-shot classification and synthetic sampling.

Lectures will use applied examples to help students build strong intuitions about how these methods can be used in practice. The course will discuss recent examples from across social science disciplines - both for design and analysis purposes. Students will consider practical issues related to the implementation and computation of large networks, as well as the analytic limitations of prediction models. Students will learn to apply important algorithms to data and to validate and evaluate model results. Students will work directly with applied datasets and analyse these data using Python and industry-standard deep learning APIs.

20 hours of lectures and 10 hours of seminars in the WT.

This course is delivered through a combination of classes and lectures totalling a minimum of 30 hours across Winter Term.

This course has a reading week in Week 6 of WT.

Students will be expected to produce one structured problem set in the WT.

• Argyle, L. P., Busby, E. C., Fulda, N., Gubler, J. R., Rytting, C., & Wingate, D. (2023). Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3), 337-351.
• Burkov, A. (2020). Machine learning engineering (Vol. 1). Montreal, QC, Canada: True Positive Incorporated.
• Egami, N., Hinck, M., Stewart, B. M., & Wei, H. (2023). Using imperfect surrogates for downstream inference: Design-based supervised learning for social science applications of large language models. 37th conference on neural information processing systems (NeurIPS 2023).
• Knox, D., Lucas, C., & Cho, W. K. T. (2022). Testing causal theories with learned proxies. Annual Review of Political Science, 25, 419-441.
• Raff, E. (2022). Inside deep learning: Math, algorithms, models. Simon and Schuster.
• Rodriguez, P. L., & Spirling, A. (2022). Word embeddings: What works, what doesn’t, and how to tell the difference for applied research. The Journal of Politics, 84(1), 101-115.
• Torres, M., & Cantú, F. (2022). Learning to See: Convolutional Neural Networks for the Analysis of Social Science Data. Political Analysis, 30(1), 113–131.

Exam (80%, duration: 2 hours) in the spring exam period.
Take-home assessment (20%) in the WT.