Prefix Probing: Lightweight Harmful Content Detection for Large Language Models

Prefix Probing demonstrates strong detection performance, achieving F1 scores comparable to or exceeding state-of-the-art external safety models. It consistently surpasses 90% in relative capability scores across various LLMs and benchmarks, indicating effective leverage of the model's intrinsic safety discrimination ability. This high accuracy is achieved without additional models or multi-stage inference.

A key advantage is its minimal computational cost. By utilizing prefix caching, Prefix Probing reduces detection overhead to near first-token latency, making it highly practical for real-time and streaming inference scenarios. It requires no extra model deployment or modifications to the LLM architecture, integrating seamlessly with existing inference systems that support prefix caching.

The search-discovered prefixes significantly enhance detection capability compared to manually designed ones. Prefixes generalize well within the same model family (e.g., smaller to larger models of the same architecture). However, cross-model type generalization is limited, reflecting structural differences in generation processes (e.g., reasoning-oriented vs. conversational models).