Research Showcase 2025

Aligning large language models (LLMs) with human preferences is critical to recent advances in generative artificial intelligence. Reinforcement learning from human feedback (RLHF) is widely applied to achieve this objective. A key step in RLHF is to learn the reward function from human feedback. However, human feedback is costly and time-consuming, making it essential to collect high-quality conversation data for human teachers to label. Additionally, different human teachers have different levels of expertise. It is thus critical to query the most appropriate teacher for their opinions. In this paper, we use offline reinforcement learning (RL) to formulate the alignment problem. Motivated by the idea of D-optimal design, we first propose a dual active reward learning algorithm for the simultaneous selection of conversations and teachers. Next, we apply pessimistic RL to solve the alignment problem, based on the learned reward estimator. Theoretically, we show that the reward estimator obtained through our proposed adaptive selection strategy achieves minimal generalized variance asymptotically, and prove that the sub-optimality of our pessimistic policy scales as O(1/sqrt{T}) with a given sample budget T. Through simulations and experiments on LLMs, we demonstrate the effectiveness of our algorithm and its superiority over state-of-the-arts.

Training machine learning models requires efficient learning algorithms that economise on the number of labelled data points used for training. In the active learning setting, the objective is to train a model using a small number of labelled data points, thereby minimising the labelling cost. In the data subset selection problem, the goal is to train a model using a limited number of training data points.

We investigate the convergence rates and data sample sizes required to train a machine learning model using the stochastic gradient descent (SGD) algorithm, where data points are sampled based on either their loss value (i.e., loss-based sampling strategies) or their uncertainty value (i.e., uncertainty-based sampling strategies).

For SGD with a constant step size update, we present convergence results for linear classifiers and linearly separable datasets using squared hinge loss and other related training loss functions. Additionally, we extend our analysis to more general classifiers and datasets, considering a broad range of loss-based sampling strategies and smooth convex training loss functions.

We propose a novel algorithm called Adaptive-Weight Sampling (AWS), which utilises SGD with an adaptive step size that achieves stochastic Polyak’s step size in expectation. We establish convergence rate results for AWS for smooth convex training loss functions.

Our numerical experiments demonstrate the efficiency of AWS across various datasets by using either exact or estimated loss values.

We present a unified likelihood ratio-based confidence sequence (CS) for any (self-concordant) generalized linear model (GLM) that is guaranteed to be convex and numerically tight. We show that this is on par or improves upon known CSs for various GLMs, including Gaussian, Bernoulli, and Poisson. In particular, for the first time, our CS for Bernoulli has a poly(S)-free radius where S is the norm of the unknown parameter. Our first technical novelty is its derivation, which utilizes a time-uniform PAC-Bayesian bound with a uniform prior/posterior, despite the latter being a rather unpopular choice for deriving CSs. As a direct application of our new CS, we propose a simple and natural optimistic algorithm called OFUGLB, applicable to any generalized linear bandits (GLB; Filippi et al. (2010)). Our analysis shows that the celebrated optimistic approach simultaneously attains state-of-the-art regrets for various self-concordant (not necessarily bounded) GLBs, and even poly(S)-free for bounded GLBs, including logistic bandits. The regret analysis, our second technical novelty, follows from combining our new CS with a new proof technique that completely avoids the previously widely used self-concordant control lemma (Faury et al., 2020, Lemma 9). Numerically, OFUGLB outperforms or is at par with prior algorithms for logistic bandits.

Recent advancements in artificial intelligence (AI) have significantly transformed weather forecasting, challenging traditional numerical weather prediction (NWP) methods with faster, data-driven alternatives. This talk will explore the latest breakthroughs in AI-driven weather forecasting, highlighting key models such as DeepMind’s GraphCast and GenCast, Huawei’s Pangu-Weather, and NVIDIA’s FourCastNet. These models leverage deep learning architectures—including graph neural networks, generative AI, and transformer-based structures—to improve forecasting accuracy while dramatically reducing computational costs.

We will discuss the fundamental advantages and limitations of AI-based forecasting compared to traditional NWP models, including generalization capabilities, data dependencies, and interpretability concerns. Additionally, we will examine AI’s role in predicting extreme weather events, the integration of AI into hybrid physics-AI models, and its potential applications in real-world decision-making for industries reliant on weather predictions.

By bridging AI and meteorology, this talk aims to provide a structured overview of where the field stands today and where it is headed, offering insights into the future trajectory of AI-driven weather forecasting.

Reinforcement learning (RL)---finding the optimal behaviour (also referred to as policy) maximizing the collected long-term cumulative reward---is among the most influential approaches in machine learning with a large number of successful applications. In several decision problems, however, one faces the possibility of policy switching---changing from the current policy to a new one---which incurs a non-negligible cost, and in the decision one is limited to using historical data without the availability for further online interaction. Despite the inevitable importance of this offline learning scenario, to our best knowledge, very little effort has been made to tackle the key problem of balancing between the gain and the cost of switching in a flexible and principled way. Leveraging ideas from the area of optimal transport, we initialize the systematic study of policy switching in offline RL. We establish fundamental properties and design a Net Actor-Critic algorithm for the proposed novel switching formulation. Numerical experiments demonstrate the efficiency of our approach on multiple robot control benchmarks of the Gymnasium and traffic light control from SUMO-RL.

Rank-based models for equity markets are reduced-form models where stock price dynamics depend on the rank that asset occupies in the investment universe. Such models are able to capture certain stylized macroscopic properties of equity markets, such as stability of capital distribution in the market and rates of stock rank switches. However, when calibrated to real equity data the models possess undesirable features such as an "Atlas stock" effect; namely the smallest security has an unrealistically large average growth rate. Recently, Campbell and Wong (2024) identified that listings and delistings (i.e. entrances and exists) of securities in the market are important drivers for the stability of the capital distribution curve. In this work we develop a framework for ranked-based models with listings and delistings and calibrate them to data. By incorporating listings and delistings the calibration procedure no longer leads to an "Atlas stock" behaviour. Moreover, by studying an appropriate "local model", focusing on a specific target rank, we are able to theoretically connect collisions rates with a notion of particle density, which is more stable and easier to estimate from data than the collision rates. This is joint work with Martin Larsson (CMU) and Licheng Zhang (CMU).

Last passage times have received considerable attention in the recent literature. For instance, in the classic ruin theory (which describes the capital of an insurance company), the moment of ruin is considered as the first time the process is below level zero. However, in more recent literature, the last passage time below zero (g) is treated as the moment of ruin and the Cramér-Lundberg has been generalised to spectrally negative Lévy processes (see e.g. Chiu and Yin (2005)). Moreover, in Paroissin and Rabehasaina (2013), spectrally positive Lévy processes are used for degradation models, and the last passage time above a fixed boundary is considered the failure time.
We use a perturbation method for Lévy processes to derive an Itô formula for the three-dimensional process {(g_t,t,X_t),t≥0} and its infinitesimal generator. We find a solution to a related optimal stopping problem with some applications in corporate bankruptcy. 

Empirical studies show that the marketable orders of HF traders contain significant informational advantage and are correlated with future price fluctuations. There is increasing evidence that some HF traders use their technological advantage to exploit the public and private information before others do - a particular example being the case of a stock traded on different venues that we call market fragmentation. While the traditional market microstructure is mainly concerned with private information with precise signals, the case of superior speed in exploiting information is less studied. We study equilibrium in a financial market in which an HF trader receives a private signal, e.g. news from other market, in addition to publicly available news. Surprisingly, an equilibrium exists even if the HF trader’s signal is initially less precise than the public news. We discuss various liquidity measures as well as deviations from the classical Kyle model. 

Hierarchical factor models, which include the bifactor model as a special case, are useful in social and behavioural sciences for measuring hierarchically structured constructs. Specifying a hierarchical factor model involves imposing hierarchically structured zero constraints on a factor loading matrix, which is a demanding task that can result in misspecification. Therefore, an exploratory analysis is often needed to learn the hierarchical factor structure from data. Unfortunately, we lack an identifiability theory for the learnability of this hierarchical structure and a computationally efficient method with provable performance. The method of Schmid–Leiman transformation, which is often regarded as the default method for exploratory hierarchical factor analysis, is flawed and likely to fail. The contribution of this paper is three-fold. First, an identifiability result is established for general hierarchical factor models, which shows that the hierarchical factor structure is learnable under mild regularity conditions. Second, a computationally efficient divide-and-conquer approach is proposed for learning the hierarchical factor structure. This approach has two building blocks – (1) a constraint-based continuous optimisation algorithm and (2) a search algorithm based on an information criterion – that together explore the structure of factors nested within a given factor. Finally, asymptotic theory is established for the proposed method, showing that it can consistently recover the true hierarchical factor structure as the sample size grows to infinity. The power of the proposed method is shown via simulation studies and a real data example. This is a joint work with Jiawei Qiao (Fudan U) and Zhiliang Ying (Columbia U).

Variable selection plays a key role in modern statistical research and learning. Major classes of variable selection approaches are implemented using Markov chain Monte Carlo methods. These methods may be computationally impractical for large scale problems or complex models and faster approximations are desirable or necessary. We develop an approach to variable selection for heteroscedastic regression models based upon semiparametric mean field variational Bayes. The methodology we propose is suitable for models having linear mean and exponential variance functions with prior specifications that induce sparse solutions on the regression coefficients, namely Bayesian lasso, spike-and-slab and adaptive spike-and-slab lasso. Use of classic mean field variational Bayes leads to the approximating densities having non-standard forms and challenging numerical problems arise in the determination of the optimal approximation. We achieve tractability by imposing a parametric assumption to the approximate marginal posterior densities of variance regression coefficients. Our iterative optimization of the log-likelihood lower bound includes Newton-type steps with analytical derivative expressions for the parametric component of the approximation. This optimization strategy uses new results that solve recurrent issues of constrained optimization involving multivariate skew-normal variational approximations. Illustrations demonstrate our approach is computationally efficient and accurate in comparison to Markov chain Monte Carlo. This is a joint work with Mauro Bernardi and Luca Maestrini. 

Previous cross-sectional research has found correlation in the health outcomes of coresident adults. However, the study of household effects in longitudinal data is challenging due to the complex association structure arising from changes in household membership over time. We propose a ‘grouped’ multilevel model where the groups (called ‘superhouseholds’) are specified to capture changes in household structure. Correlated household random effects are used to capture correlations between households sharing an individual(s), and correlations between household pairs can depend on covariates that describe their connection. We develop a constrained MCMC procedure for model estimation that ensures the group-specific correlation matrices (with dimensions that can vary across groups) are positive definite. The proposed model is evaluated in a simulation study and then applied in analyses of household and area effects on self-rated physical and mental health in the UK using data from a national household panel survey. This is joint work with Siliang Zhang and Paul Clarke.

We build a balance sheet-based model to capture run risk, i.e., a reduced potential to raise capital from liquidity buffers under stress, driven by depositor scrutiny and further fueled by fire sales in response to withdrawals. The setup is inspired by the meltdown of Silicon Valley Bank (SVB) in March 2023. By bringing a time series of SVB's balance sheet data to our model, we are able to demonstrate how changes in the funding and respective asset composition made SVB prone to run risk, as they were increasingly relying on held-to-maturity accounting standards, masking revaluation losses in securities portfolios. Furthermore, we formulate a tractable optimization problem to address the designation of held-to-maturity assets and quantify banks' ability to hold these assets without resorting to remarking. By calibrating this to SVB's balance sheet data, we shed light on the bank's funding risk and implied risk tolerance in the years 2020-22 leading up to its collapse. Finally, we compare this with findings for First Republic Bank, US Bancorp, and PNC Financial Services Group Inc.

Little is known about how long it takes for new medicines to reach countries with different income levels. We analysed data, sourced from IQVIA, on the timing of new drug launches in seventy-five low-, middle-, and high-income markets from 1982 to 2024. The sample captured the majority of essential medicines (as designated by the World Health Organization in the twenty-third Model List of Essential Medicines) that first came into medical use anywhere globally from 1982 onward. We used methods of survival analysis to examine how country income levels and other factors are associated with variability in times to launch across countries and medicines.

At Novo Nordisk, our innovative cardiometabolic and rare disease portfolio requires generation and application of real-world evidence (RWE, derived from large electronic medical and health records) to ultimately bring value to patients. For example, we contexualise clinical trial evidence by bridging the trial cohort to the real-world cohorts. RWE can be used to get a better understanding of the under-characterised populations, their natural histories of disease progression and long-term outcomes. We can also tap into opportunities to develop risk prediction algorithms to better triage patients for personalised treatments and care pathways. We will share a few innovative examples where big medical data informed optimal patient care, and fast-tracked clinical development that cleared regulatory hurdles.

We study the online change detection problem, broadly defined: data are observed in an online fashion where up to some unknown and possibly infinite time the data are drawn from a distribution P, and after this time the data are drawn from a different distribution Q. Both P and Q are unknown, and the goal is to declare a change with the smallest possible delay once it has occurred.

The maximum mean discrepancy (MMD), a semi-metric on the space of probability distributions, provides powerful non-parametric two-sample tests on kernel-enriched domains. However, MMD suffers from quadratic time complexity, making its use impractical for online problems. We introduce a sequential testing procedure in which the MMD is approximated using random Fourier features (RFFs), and which enjoys logarithmic per-iteration time complexity in the worst case. We develop finite sample theory indicating that the RFF based procedure attains the same detection delay as computationally infeasible procedures making use of the MMD, while controlling the multiple testing analogue of size. 

This is joint work with Florian Kalinke (KIT).

Detecting the depedency between two random vectors X and Y is a fundamental and long-standing statistical problem. Building on a geometric insight from Chatterjee's correlation coefficient and leveraging the concept of Monge-Kantorovich ranks, we propose a new coefficient of correlation, which we called coverage correlation coefficient, for measuring the dependency between two random vectors. The proposed coefficient a) has an intuitive geometric interpretation; b) consistently estimate an f-divergence between the joint distribution of X and Y and their product measure, which equals to 0 if and only if X and Y are independent, and equals to 1 if and only if the joint distribution of X and Y is singular to their product measure; c) has a simple asymptotic theory that is entirely distribution-free when X and Y are independent. Unlike many existing measurements, the proposed coefficient is capable of detecting not only functional dependency but also spurious correlation between X and Y though confounding variables. We demonstrate the superior performance of our methods compared to existing methods by extensive simulations and real-data examples.

Recently, Chatterjee introduced a measure of dependence that is as simple as classical coefficients like Pearson's or Spearman's, and consistently estimates a simple and interpretable measure of dependence—being 0 if and only if the variables are independent and 1 if and only if one is a measurable function of the other. While the coefficient possesses many desirable properties, it has been observed that it may lack power against certain local dependency structures. To address this issue, we propose a simple modification that enhances its sensitivity in such cases. We show that the new measure of dependence retains the same set of properties while remaining computationally efficient to estimate. Additionally, we demonstrate its application in variable selection.