What Matters for Safety Alignment?

Kari Jaaskelainen

08 Jan 2026 — 1 min read

What matters for AI safety alignment?

We ran a large-scale study of 32 popular language and reasoning models to see what truly improves—or harms—safety.

Scope: six intrinsic model traits, three attack styles, five safety datasets, 56 jailbreaks plus four chain‑of‑thought (CoT) attacks; 4.6M API calls.
Safest models: GPT-OSS-20B, Qwen3-Next-80B-A3B-Thinking, and GPT-OSS-120B—suggesting built-in reasoning and self-reflection help guardrails stick.
Training risks: post‑training and knowledge distillation can systematically degrade safety unless safety is treated as a first‑class objective.
Critical vulnerability: a simple CoT “response prefix” can boost attack success 3.34× on average; for Seed-OSS-36B-Instruct it jumps from 0.6% to 96.3%. Text‑completion UIs and features that let users prefill responses are especially risky.
Most effective jailbreaks today: roleplay, prompt injection, and gradient‑based prompt search.

Takeaway: pair stronger model design with deployment safeguards, and optimize explicitly for safety—not just capability.

Paper: https://arxiv.org/abs/2601.03868v1

Paper: https://arxiv.org/abs/2601.03868v1

Register: https://www.AiFeta.com

#AI #AIsafety #SafetyAlignment #LLM #LRM #Jailbreak #ChainOfThought #ResponsibleAI #Security #ML

Meet GRACE: a moral governor for safer, more transparent AI

AI agents are getting powerful—so how do we make sure they do the right thing, not just the effective thing? Meet GRACE, a reason-based moral governor that keeps AI behavior aligned with human norms by separating moral reasoning from goal-driven decision-making. * Moral Module: uses deontic logic and explicit reasons

Why some fine-tuned LLMs miss phishing—and how to fix it

Not all fine-tuned LLMs spot phishing equally. A new study tests Llama 3.1 8B, Gemma 2 9B, and Mistral on high-stakes phishing detection—and uses SHAP and mechanistic interpretability to reveal why models do (or don’t) generalize. * Architecture × data diversity matters: Gemma 2 9B hits state-of-the-art performance (F1

AI that explains itself—by following the science

AI that explains itself—by following the science Black-box AI can be powerful, but it often can’t tell us why it made a decision. Concept Bottleneck Models (CBMs) try to fix that by predicting human-understandable concepts first, then the final answer. The catch: standard CBMs ignore domain-specific cause-and-effect and

Meet GenomAgent: A Team of AI Specialists for Smarter Genomics Q&A

TL;DR: Finding reliable answers in genomics is hard. GenomAgent turns one big AI into a coordinating team of specialists—and beats the current leader by 12% on a key benchmark. Genomic facts live across many databases. Standard chatbots struggle because they can’t flexibly query those sources. GeneGPT added

What matters for AI safety alignment?

Read more

Meet GRACE: a moral governor for safer, more transparent AI

Why some fine-tuned LLMs miss phishing—and how to fix it

AI that explains itself—by following the science

Meet GenomAgent: A Team of AI Specialists for Smarter Genomics Q&A