Sagnik Mukherjee

PhD Candidate, UIUC

LLMs · Post-training · Optimization · Multi-turn Decision Making

Email CV Scholar Twitter GitHub

I’m a PhD candidate at UIUC, working on Reinforcement Learning for LM post-training with Dr. Dilek Hakkani-Tür and Dr. Hao Peng. I was the first to discover emergent structures in on-policy training, which later spawned a research thread at the intersection of optimization and on-policy training in LLMs. Lately I’ve been thinking about agentic sequential decision making — specifically, the delayed consequences of actions and partial observability.

Feel free to reach out if you’d like to collaborate or chat.

News

Sept 2026New preprint: Understanding LLM Parameter Update Sparsity through the Lens of Fisher — we explain, theoretically, why on-policy data causes sparse updates.
Sept 2026VASR was accepted at NeurIPS 2026.
Sept 2026Gave a talk, “Optimization Geometry in LLM Post-Training”, at the UIUC Machine Learning Seminar.
Jul 2026Presented Do We Need Adam? as an Oral at ICML 2026 in Seoul.
May 2026Started my internship at Microsoft Frontier Tuning. Hosts: Harsh Jhamtani and Jacob Andreas.

Research

Optimization in Post-Training

How does on-policy data shape the optimization landscape and training dynamics of language models? I study the structure that emerges during on-policy training—from sparse parameter updates to preserved spectral structure—and use these insights to develop more efficient methods for LLM post-training.

Reinforcement Learning Finetunes Small Subnetworks in Large Language Models (NeurIPS 2025)
Do We Need Adam? Surprisingly Strong and Sparse Reinforcement Learning with SGD in LLMs (ICML 2026, Oral)
ISO: An RLVR-Native Optimization Stack (Preprint)
Understanding LLM Parameter Update Sparsity through the Lens of Fisher (Preprint)

Generative Modeling for Post-Training

Inference-time alignment and alternate views of post-training.

VASR: Variance-Aware Systematic Resampling for Reward-Guided Diffusion (NeurIPS 2026)
VarPO (soon to appear)

Sequential Decision Making

How does the notion of “delayed consequences of actions” and “partial observability” affect reasoning and decision making in LLMs?

Work in progress.

Publications Google Scholar

Understanding LLM Parameter Update Sparsity through the Lens of Fisher
Yufan Zhang, Sagnik Mukherjee, Hao Peng
Preprint, 2026
Tweet
ISO: An RLVR-Native Optimization Stack
Hanqing Zhu, Wenyan Cong, Zhizhou Sha, Sagnik Mukherjee, Xinyuan Song, David González-Martínez, Xiaoxia Wu, Yuandong Tian, Shiwei Liu, David Z. Pan, Zhangyang Wang
Preprint, 2026
VASR: Variance-Aware Systematic Resampling for Reward-Guided Diffusion
Shivanshu Shekhar, Sagnik Mukherjee, Jia Yi Zhang, Tong Zhang
NeurIPS, 2026
Tweet
Do We Need Adam? Surprisingly Strong and Sparse Reinforcement Learning with SGD in LLMs
Sagnik Mukherjee, Lifan Yuan, Pavan Jayasinha, Dilek Hakkani-Tür, Hao Peng
ICML, 2026 (Oral)
Tweet
Reinforcement Learning Finetunes Small Subnetworks in Large Language Models
Sagnik Mukherjee, Lifan Yuan, Dilek Hakkani-Tür, Hao Peng
NeurIPS, 2025
Tweet
Premise-Augmented Reasoning Chains Improve Error Identification in Math Reasoning with LLMs
Sagnik Mukherjee, Abhinav Chinta, Takyoung Kim, Tarun Anoop Sharma, Dilek Hakkani-Tür
ICML, 2025
Tweet

Mentoring

I have had the opportunity to work with some amazing students.

Pavan Jayasinha — now at OpenAI
Do We Need Adam? Surprisingly Strong and Sparse Reinforcement Learning with SGD in LLMs
Yufan Zhang — visiting student at UIUC
Understanding LLM Parameter Update Sparsity through the Lens of Fisher
Saket Vispute — student at IIT Bombay