Test-Time Safety Alignment

Our approach, Test-Time Safety Alignment (TSA), applies zeroth-order gradient estimation to approximate the gradient of a black-box text-moderation oracle with respect to the prompt embeddings. We perturb input embeddings with i.i.d. Gaussian noise, average the moderation scores, and perform gradient descent to minimize harmfulness. Key components include Gaussian smoothing, gradient normalization to prevent off-manifold drift, and cosine similarity constraints to preserve fidelity to the original prompt direction. The method operates at the sub-lexical level, converging typically within a few gradient steps.

Experiments were conducted on five instruction-tuned models (Gemma 3-1B, Phi-3.5-mini, Llama 3.1-8B, Qwen3-14B, GPT-OSS-20B) using two red-teaming benchmarks: WildJailbreak (adversarial harmful/benign) and HarmBench (direct harmful). TSA nearly eliminates every safety-flagged completion for most models, often within 1-2 gradient steps. On benign prompts, responses remain substantively intact, confirming no blanket refusal bias. Optimized embeddings decode back to original tokens, indicating sub-lexical perturbations are sufficient to steer behavior towards safety. Baselines (SmoothLLM, AdaSteer, RESTA) were less effective, particularly on HarmBench, showing more flagged completions than the undefended model.

The effectiveness of sub-lexical intervention stems from the embedding space's geometry and gradient normalization, allowing precise steering within a token's Voronoi cell. Notably, the optimization cannot be reversed to create jailbreaks, attributed to the refusal basin shaped by safety training. TSA is a drop-in safety layer requiring only input embeddings and a black-box moderation API. Limitations include per-prompt latency (seconds), the need for embedding access (not API-gated services), and effectiveness scaling with the underlying model's existing safety training quality, as observed with Gemma 3-1B.