GPT-5.1 INSTANT & THINKING

Disallowed Content Performance Benchmarks

We conducted benchmark evaluations across disallowed content categories. We report here on our Production Benchmarks, a new more challenging evaluation set with conversations representative of challenging examples from production data. These evaluations were deliberately created to be difficult and built around cases in which our existing models were not yet giving ideal responses, which is reflected in the scores below.

Overall, both gpt-5.1-thinking and gpt-5.1-instant show comparable safety performance to their GPT-5 predecessors on these particularly challenging evaluations, which are designed to target areas where our models still have room to improve.

The new gpt-5.1-thinking model shows light regressions relative to gpt-5-thinking for content involving harassment and hateful language, as well as disallowed sexual content. We are working on further improvements for these categories.

Early signal on prevalence of undesired responses for sensitive situations: In addition to these offline evaluations, we also share here some very early signal on the prevalence of undesired responses for sensitive situations based on online measurements that we ran during A/B testing. These online measurements have wide error bars, but can help provide early signal on potential improvements or regressions.

Mental health: On offline evaluations, gpt-5.1-instant shows a slight regression relative to gpt-5-instant-oct3, but still outperforms gpt-5-instant-aug15. gpt-5.1-thinking improves relative to gpt-5-thinking. Preliminary online measurements show slight improvements for both, with low statistical confidence.

Emotional reliance: Both gpt-5.1-instant and gpt-5.1-thinking show a slight regression relative to their respective GPT-5 predecessors on offline evaluations, though gpt-5.1-instant improved relative to gpt-5-instant-aug15. Preliminary online measurements show mixed results, with gpt-5.1-thinking showing improvement with high statistical confidence.

Self harm and suicide: Preliminary online measurements were neutral for gpt-5.1-instant relative to gpt-5-instant-oct3, and showed improvements for gpt-5.1-thinking relative to gpt-5-thinking, but these estimates have low statistical confidence.

Jailbreak Attack Robustness

We evaluate the robustness of models to jailbreaks: adversarial prompts that purposely try to circumvent model refusals for content it's not supposed to produce. Below is an adaptation of the academic jailbreak eval, StrongReject [1], which inserts a known jailbreak into an example from disallowed content evals.

We find that gpt-5.1-instant performs better than its predecessor, and that gpt-5.1-thinking is on par with its predecessor in robustness against jailbreak attempts.

Image Input Safety Evaluations

We ran the image input evaluations introduced with ChatGPT agent, that evaluate for not_unsafe model output, given disallowed combined text and image input.

We find that both the instant and thinking variations of GPT-5.1 perform generally on par with their predecessors in vision capabilities. We are observing a regression of gpt-5.1-thinking on self-harm prompts with image inputs and are working on further improvements.

AI Preparedness & Frontier Risk Assessment

GPT-5's frontier capabilities are assessed under the Preparedness Framework as described in the original GPT-5 system card. As we did for GPT-5 at launch, we are continuing to treat GPT-5.1 as High risk in the Biological and Chemical domain, and continuing to apply the corresponding safeguards. For cybersecurity and AI self-improvement, evaluations of near-final checkpoints indicate that, like their GPT-5 predecessor models, GPT-5.1 models do not have a plausible chance of reaching a High threshold.