We analyze safety alignment and show that it is largely shallow. We propose methods for making it deeper.

Refusal tokens enable controlling a single model's refusal rates without further fine-tuning, by selectively intervening during generation.

DynaGuard is a dynamic guardian model that evaluates text based on user-defined policies, surpassing static models in detection accuracy.

We propose the first method for generating visual adversarial examples that can serve as transferrable universal jailbreaks against aligned large language models.

Neurotoxin is a novel model poisoning attack for federated learning that stays present in the system for up to 5X longer than the baseline attack.

SparseFed is a provably robust defense against model poisoning attacks in federated learning that uses server-side sparsification to avoid updating malicious neurons.

We present the first method for doing privacy auditing of LLMs.

We propose a new practical data extraction attack that we call "neural phishing". This attack enables an adversary to target and extract sensitive or personally identifiable information (PII), e.g., credit card numbers, from a model trained on user data.

We propose the first method for performing differentially private fine-tuning of large language models without backpropagation. Our method is the first to provide a nontrivial privacy-utility tradeoff under pure differential privacy.

We propose the first method for performing differentially private in-context learning. Our method generates sentences from in-context learning while keeping the in-context exemplars differentially private, that can be applied to blackbox APIs (ex RAG).

We find that using scaling laws for Differentially Private Hyperparameter Optimization significantly outperforms prior work in privacy and compute cost.

We pretrain networks with synthetic images that have strong performance on downstream private computer vision tasks.

We generate differentially private images from non-privately trained diffusion models by analyzing the inherent privacy of stochastic sampling.

We present a lightweight approximation method that gives the MoE router a dense gradient update while continuing to sparsely activate its parameters.

We release the Gemstone model suite for open-source scaling laws.

LoRI reduces cross-task interference in multi-task low-rank adaptation, enabling better performance when fine-tuning on multiple tasks.

We conduct an in-depth analysis of attention in video diffusion transformers (VDiTs) and report a number of novel findings.

We propose a new definition for attention heads dominated by attention sinks, known as dormant attention heads.

Lottery Ticket Adaptation (LoTA) is a new adaptation method that handles challenging tasks, mitigates catastrophic forgetting, and enables model merging across different tasks.

FetchSGD is a communication-efficient federated learning algorithm that compresses gradient updates with sketches.