Raghav Singhal

For EPFL students: If you are interested in a project, please feel free to reach out via mail. I'm very happy to supervise motivated students!

Email  /   /   /   / 

Research

My research interests currently revolve around pretraining, data, & robust alignment of language models.

Some directions that I am currently interested in are:

• Alignment pretraining: robustly aligning models from the start (token 0) of pretraining, rather than doing so late in mid-training or post-training, as is done usually (see SPP & Model Raising).
• Data-constrained pretraining: how should we approach pretraining when constrained by data, & not by compute?
• On-policy pretraining: pretraining is typically an off-policy imitation of a fixed corpus, & everything on-policy is deferred to post-training. What happens if we move on-policy (RL) signal much earlier, into pretraining itself?
• Post-training & pretraining interaction: how can we design pretraining for better post-training, especially RL, & how do the two stages interact?

Check out my Google Scholar for a complete list of publications. * denotes equal contribution. Selected projects are highlighted.

models & data

Instead of aligning models at the end of training, we install the desired assistant persona from token zero. We annotate pretraining documents with first-person moral reflections derived from a constitution, pretrain on them, then bind the assistant to that persona in post-training.

SPP models are more faithful to the constitution, less misaligned, & more robust to jailbreaks. Compared to models aligned late in pretraining (mid-training), models aligned from token zero better internalize the constitution's underlying principles: they show different value priorities & are better-aligned on OOD moral dilemmas.

We are currently scaling SPP to Apertus 2, a 500B+ parameter MoE.

models

The biggest fully open & compliant training run & LLM at the time.

8B & 70B fully pretrained open-data open-weights models, multilingual in >1000 languages. Performance equivalent or better than corresponding Llama 3 sizes.

RLVR makes models better at reasoning, but this reasoning drifts away from what weaker partners — models, or humans — can follow. In TRL, a trained senior & a frozen weaker junior alternate stochastically to co-generate every rollout, & the combined rollout is rewarded as a whole. TRL matches vanilla GRPO on solo reasoning while giving stronger handoff robustness, less distributional drift, & a chain-of-thought more legible to the junior.

We show that safety alignment in LLMs is not confined to distinct subspaces (but rather, highly entangled with general ability directions), thus fundamentally challenging the foundation of subspace-based defenses.

We achieve exact aggregation in distributed fine-tuning of LLMs, consistently improving over SOTA.

We introduce ABBA, a PEFT method that enhances expressivity by decoupling low-rank updates from pre-trained weights via a Hadamard product, consistently improving over SOTA methods.

We set a new Pareto frontier for distributed fine-tuning of LLMs, achieving SOTA performance, stronger privacy guarantees, & up to 230x lower communication costs.

We provably achieve the best approximation of full fine-tuning in low-rank spaces solely through clever initialization, outperforming LoRA while using up to 90x fewer parameters.

We introduce a multimodal mixup-based contrastive learning framework that effectively captures shared relations across modalities, enabling robust multimodal representation learning.

Source code taken from Jon Barron's website.