Ashwinee Panda

I am a 4th (final) year PhD student at Princeton University working with Prateek Mittal on trustworthy artificial intelligence and privacy preserving machine learning. I am funded by fellowships and grants , most recently the OpenAI Superalignment Fast Grant.

Starting September 2024 I will be a Postdoctoral Associate at UMD working with Tom Goldstein on large language models.

Before Princeton I worked in the UC Berkeley RISE Lab where I was co-advised by Joey Gonzalez and Raluca Ada Popa, researching federated learning.

If you are interested in working with me send me an email at [firstname]_at_[university]_dot_[edu].

CV  /  Google Scholar  /  Twitter  /  Github

profile photo


I am currently working on a number of topics in LLMs, including pretraining, alignment, safety and privacy. The title of my PhD thesis was "Challenges in Augmenting Large Language Models with Private Data".

gpt phish Teach LLMs to Phish: Stealing Private Information from Language Models
Ashwinee Panda, Christopher A. Choquette-Choo, Zhengming Zhang, Yaoqing Yang, Prateek Mittal
At ICLR 2024

We propose a new practical data extraction attack that we call "neural phishing". This attack enables an adversary to target and extract sensitive or personally identifiable information (PII), e.g., credit card numbers, from a model trained on user data.

dp icl Privacy-Preserving In-Context Learning for Large Language Models
Tong Wu*, Ashwinee Panda*, Tianhao Wang*, Prateek Mittal
At ICLR 2024

We propose the first method for performing differentially private in-context learning. Our method generates sentences from in-context learning while keeping the in-context exemplars differentially private, that can be applied to blackbox APIs (ex RAG).

VLM Visual Adversarial Examples Jailbreak Aligned Large Language Models
Xiangyu Qi*, Kaixuan Huang*, Ashwinee Panda, Peter Henderson, Mengdi Wang, Prateek Mittal
At AAAI Conference on Artificial Intelligence, 2024 (Oral)
paper / code

We propose the first method for generating visual adversarial examples that can serve as transferrable universal jailbreaks against aligned large language models.

dp random priors Differentially Private Image Classification by Learning Priors from Random Processes
Xinyu Tang*, Ashwinee Panda*, Vikash Sehwag, Prateek Mittal
At NeurIPS 2023 (Spotlight)
paper / code

We pretrain networks with synthetic images that have strong performance on downstream private computer vision tasks.

dp zo Private Fine-tuning of Large Language Models with Zeroth-order Optimization
Xinyu Tang*, Ashwinee Panda*, Milad Nasr, Saeed Mahloujifar, Prateek Mittal


We propose the first method for performing differentially private fine-tuning of large language models without backpropagation. Our method is the first to provide a nontrivial privacy-utility tradeoff under pure differential privacy.

dp diffusion Differentially Private Generation of High Fidelity Samples From Diffusion Models
Vikash Sehwag*, Ashwinee Panda*, Ashwini Pokle, Xinyu Tang, Saeed Mahloujifar, Mung Chiang, J Zico Kolter, Prateek Mittal
At 40th International Conference on Machine Learning GenAI Workshop
paper / poster

We generate differentially private images from non-privately trained diffusion models by analyzing the inherent privacy of stochastic sampling.

linear scaling A New Linear Scaling Rule for Differentially Private Hyperparameter Optimization
Ashwinee Panda*, Xinyu Tang*, Vikash Sehwag, Saeed Mahloujifar, Prateek Mittal
talk / paper / code

We propose a new hyperparameter optimization method for differentially private machine learning that massively reduces privacy costs and compute costs.

neurotoxin Neurotoxin: Durable Backdoors in Federated Learning
Zhengming Zhang*, Ashwinee Panda*, Linyue Song, Yaoqing Yang, Prateek Mittal, Joseph Gonzalez, Kannan Ramchandran, Michael Mahoney
In Proceedings of the 39th International Conference on Machine Learning
paper / poster / code

Neurotoxin is a novel model poisoning attack for federated learning that stays present in the system for up to 5X longer than the baseline attack.

sparsefed SparseFed: Mitigating Model Poisoning Attacks in Federated Learning via Sparsification
Ashwinee Panda, Saeed Mahloujifar, Arjun Bhagoji, Supriyo Chakraborty, Prateek Mittal
In 25th International Conference on Artificial Intelligence and Statistics
paper / code

SparseFed is a provably robust defense against model poisoning attacks in federated learning that uses server-side sparsification to avoid updating malicious neurons.

fetchsgd FetchSGD: Communication-Efficient Federated Learning with Sketching
Daniel Rothchild*, Ashwinee Panda*, Enayat Ullah, Nikita Ivkin, Ion Stoica, Vladimir Braverman, Joseph Gonzalez, Raman Arora
In Proceedings of the 37th International Conference on Machine Learning
paper / code

FetchSGD is a communication-efficient federated learning algorithm that compresses gradient updates with sketches.

softpbt SoftPBT: Leveraging Experience Replay for Efficient Hyperparameter Schedule Search
Ashwinee Panda, Eric Liang, Richard Liaw, Joey Gonzalez
paper / code

Not Research

WeChat  /  LinkedIn  /  Instagram  /  Yelp  /  Goodreads /  Spotify

I was born and raised in San Jose, California. In high school I taught math, played sax, argued vociferously, sang, danced, and wrote slam poetry. Before studying EECS at Cal I spent the summer in China working at a robotics company. I've been back a couple times.

While at Berkeley I founded DiscreetAI, a venture-backed startup building privacy-preserving machine learning as-a-service. You can check out our ProductHunt launch or our GitHub for more information. Among other things we won the first YCombinator Hackathon and built federated learning solutions for Fortune 500 companies.

  • I gave a lecture on hashing for CS70, UC Berkeley's undergraduate discrete mathematics and probability course. I have served on course staff for Cal's CS70 and CS189, and Princeton's COS432.
  • I worked on R&D at Blockchain at Berkeley. I don't work in crypto anymore, but I'm happy to direct you to any of my amazing friends who have started companies in the space.
  • I read voraciously, about 100 books a year, almost entirely fiction. My favorite genres are xianxia, SFF and horror. My favorite book is The Brothers Karamazov by Fyodor Dostoevsky.
  • I frequently go on food tours and post reviews on Yelp. Feel free to ask me for restaurant recs in NYC, Edison, San Francisco, Los Angeles, and Baltimore.

Website template from Jon Barron.