AI Steerability 360: A Toolkit for Steering Large Language Models

A core abstraction is the SteeringPipeline class, which serves as a common interface for controls and allows composition of multiple controls. It includes methods like steer() for training and generate() for inference. An example, Contrastive Activation Addition (CAA), is used to demonstrate how steering vectors are computed from paired contrastive examples to modify hidden states, thereby shifting model representations towards or away from targeted behaviors.

The toolkit provides UseCase and Benchmark classes to define tasks and compare steering pipelines. Use cases specify evaluation data and metrics (standard or LLM-as-a-judge). Benchmarks compare pipelines under fixed or variable control parameters, enabling analysis of how different configurations influence model behavior. This facilitates understanding trade-offs, such as between instruction following ability and response quality, as demonstrated with PASTA (Post-hoc Attention Steering).

Additional features include support for composite steering, allowing the study of how multiple steering methods interact. The paper discusses state control abstractions, like ActAdd, ITI, and CAA, which share patterns for constructing activation steering. Limitations include reliance on Hugging Face for inference (slower than vLLM), making large-scale experiments challenging, and the difficulty in defining optimal steering parameters. Ethical considerations revolve around the potential for misuse and the challenge of unforeseen side effects.