Why some fine-tuned LLMs miss phishing—and how to fix it

Kari Jaaskelainen

16 Jan 2026 — 1 min read

Not all fine-tuned LLMs spot phishing equally. A new study tests Llama 3.1 8B, Gemma 2 9B, and Mistral on high-stakes phishing detection—and uses SHAP and mechanistic interpretability to reveal why models do (or don’t) generalize.

Architecture × data diversity matters: Gemma 2 9B hits state-of-the-art performance (F1 > 91%) but only when trained on a stylistically diverse, “generalist” dataset.
Generalization is architecture-dependent: Llama 3.1 8B excels in a narrow domain yet stumbles when asked to integrate diverse data, causing a notable drop in performance.
Some models are inherently steadier: Mistral is a consistent, resilient performer across multiple training setups.

Bottom line: Reliable AI isn’t just about fine-tuning—it’s about validating the interplay of model architecture, data diversity, and training strategy, and auditing the flawed heuristics models learn along the way.

Read more: https://arxiv.org/abs/2601.10524v1

Paper: https://arxiv.org/abs/2601.10524v1

Register: https://www.AiFeta.com

AI LLMs Security Phishing Generalization Interpretability NLP MachineLearning

Meet GRACE: a moral governor for safer, more transparent AI

AI agents are getting powerful—so how do we make sure they do the right thing, not just the effective thing? Meet GRACE, a reason-based moral governor that keeps AI behavior aligned with human norms by separating moral reasoning from goal-driven decision-making. * Moral Module: uses deontic logic and explicit reasons

AI that explains itself—by following the science

AI that explains itself—by following the science Black-box AI can be powerful, but it often can’t tell us why it made a decision. Concept Bottleneck Models (CBMs) try to fix that by predicting human-understandable concepts first, then the final answer. The catch: standard CBMs ignore domain-specific cause-and-effect and

Meet GenomAgent: A Team of AI Specialists for Smarter Genomics Q&A

TL;DR: Finding reliable answers in genomics is hard. GenomAgent turns one big AI into a coordinating team of specialists—and beats the current leader by 12% on a key benchmark. Genomic facts live across many databases. Standard chatbots struggle because they can’t flexibly query those sources. GeneGPT added

AI that stays on track for days: ML-Master 2.0

AI is great at quick tasks—but stumbles on week-long projects. This paper tackles that ultra-long-horizon gap. Meet ML-Master 2.0, an autonomous agent for machine learning engineering that stays strategically coherent over days. Its core idea, Hierarchical Cognitive Caching (HCC), treats memory like a multi-level cache and a lab