One Leak Away: How Pretrained Model Exposure Amplifies Jailbreak Risks in Finetuned LLMS
Finetuned LLMs Inherit Critical Vulnerabilities from Pretrained Models, Amplifying Jailbreak Risks
This analysis reveals that finetuned Large Language Models (LLMs) are highly susceptible to jailbreak attacks originating from their publicly available pretrained counterparts. Our novel Probe-Guided Projection (PGP) attack leverages representation-level signals in pretrained models to craft highly transferable adversarial prompts, significantly increasing attack success rates even against safety-tuned systems. This underscores the urgent need for enhanced security measures in the pretrain-finetune paradigm.
Executive Impact
The research highlights a critical security flaw in the prevalent pretrain-finetune paradigm for LLMs. By demonstrating that vulnerabilities present in publicly available pretrained models are directly inherited by their finetuned derivatives, the study exposes a significant attack vector. The proposed PGP attack leverages this inheritance to achieve superior transferability of jailbreak prompts, outperforming existing methods. This necessitates a re-evaluation of current LLM deployment strategies and robust alignment techniques.
0%
PGP Transfer Success Rate
0%
Baseline Transfer Success Rate
0%
Probing Accuracy
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
Jailbreak Attacks
Explores the core mechanisms and transferability of adversarial prompts.
74%
Max Transfer Success (Pretrain to Finetune)
Empirical analysis shows that adversarial prompts optimized on pretrained models transfer most effectively to their finetuned variants, revealing inherited vulnerabilities. This means a direct link exists between the security posture of a base model and its derivatives.
Methodology
Details the Probe-Guided Projection (PGP) attack and its underlying principles.
Enterprise Process Flow
White-box access to Pretrained LLM
→
Generate & Label Jailbreak Prompts (Transferable vs. Untransferable)
→
Train Linear Probes on Hidden States
→
Identify Transferability-Relevant Directions
→
Optimize Prompts using Probe-Guided Projection
→
Transfer Attack Finetuned LLM (Black-box)
The PGP attack systematically guides adversarial prompt generation using representation-level signals from pretrained LLMs.
91.4%
Linear Probe Accuracy
Transferable jailbreak prompts are linearly separable in the pretrained hidden states, indicating that transferability is encoded in the pretrained feature space.
Security Implications
Discusses the broader impact of inherited vulnerabilities and potential defenses.
| Method | Attacker Knowledge | Average TSR (%) |
|---|---|---|
| PGP (ours) | Pretrained Known (Under Threat Model) | 86.8% |
| GCG (adaptation) | Pretrained Known (Under Threat Model) | 12.6% |
| AutoDan (white) | Full White-box (More Info) | 63.4% |
| PAIR | Fully Black-box (No Pretrained Knowledge) | 34.4% |
PGP consistently outperforms existing black-box and even some white-box methods in transferability against finetuned LLMs.
Robustness Against Safety-Tuned Defenses
Even when finetuned with additional safety-aligned examples, PGP's attack success rate remains high (above 0.42). This suggests that data-level alignment alone cannot fully remove inherited vulnerabilities, emphasizing the need for new defense strategies.
Impact: Current safety-tuning practices are insufficient against advanced transfer attacks.
Recommendation: Develop explicit defense mechanisms targeting inherited vulnerabilities from pretrained models.
Advanced ROI Calculator
Estimate the potential efficiency gains and cost savings by strategically implementing AI solutions tailored to your enterprise needs.
Potential Annual Savings
$0
Annual Hours Reclaimed
0
Implementation Roadmap
Our roadmap outlines a strategic approach to addressing the identified vulnerabilities, focusing on proactive defense mechanisms and continuous monitoring.
Vulnerability Assessment & Probing
Conduct deep dives into internal representations of LLMs to identify specific layers and directions correlated with vulnerability inheritance. Develop advanced probing techniques beyond linear models.
Pretrained Model Hardening
Implement security-by-design principles during pretraining. Explore methods to 'scrub' or 'decouple' vulnerability signals from general capabilities before models are publicly released.
Finetuning-time Defenses
Integrate novel defense mechanisms during finetuning that specifically target and mitigate inherited vulnerabilities, rather than relying solely on post-hoc safety alignment.
Runtime Monitoring & Incident Response
Establish continuous monitoring for adversarial prompt patterns and develop rapid response protocols for detected jailbreak attempts, leveraging insights from representation analysis.
Ready to Secure Your LLM Deployments?
Don't let inherited vulnerabilities compromise your enterprise AI. Schedule a consultation with our experts to discuss custom defense strategies and secure your finetuned models.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat