Beyond URLs: Metadata That Makes LLMs Train Faster

Kari Jaaskelainen

27 Nov 2025 — 1 min read

Smarter LLMs, Faster—thanks to metadata

What if training a large language model didn't just rely on text, but on the context around it? This study shows that adding fine-grained metadata—not just URLs—can meaningfully speed up pretraining and improve quality.

Beyond URLs: detailed quality signals (e.g., source reliability or cleanliness) work best—especially when prepended to the text.
New trick: append metadata and have the model predict it as an auxiliary task; this boosts efficiency too.
No labels? Use learnable meta-tokens trained with masked loss to induce quality-aware structure and recover part of the speedup.

Why it matters: richer, well-placed metadata helps models learn what to trust and what to focus on—cutting compute while maintaining or improving capability.

Takeaway: prefer finer-granularity metadata and thoughtful placement (prepend or append+predict). Probing confirms these signals shape the model’s internal representations.

Paper: https://arxiv.org/abs/2511.21613v1

Paper: https://arxiv.org/abs/2511.21613v1

Register: https://www.AiFeta.com

#AI #LLM #MachineLearning #NLP #DataQuality #Pretraining #Efficiency #DeepLearning

Escaping the Verifier: Learning to Reason via Demonstrations

LLMs can learn to reason—without task verifiers Many real-world problems don’t have automatic checkers to grade answers, even though we have lots of expert solutions. RARO (Relativistic Adversarial Reasoning Optimization) shows how to train reasoning skills from those examples alone. How it works: * A policy (the model) tries

VacuumVLA: A Two-in-One Robot Hand That Grips and Suctions

Robots guided by Vision-Language-Action (VLA) AI are getting better at everyday tasks—but most still use simple two-finger grippers. That limits them on smooth, flat, or handleless objects. VacuumVLA is a low-cost robot hand that merges a standard two-finger gripper with a vacuum suction cup. The robot can switch between

ChatDRex: No‑Code, Conversational AI for Drug Repurposing

Meet ChatDRex Finding new uses for approved drugs can save years and millions. But making those predictions usually takes teams of specialists and a tangle of tools. ChatDRex changes that. It’s a conversation-based, no‑code, multi‑agent system that lets clinicians and researchers ask complex bioinformatics questions in plain

Qwen3‑VL: An AI that understands text, images, and video—at book length

Meet Qwen3-VL, our most capable vision-language model yet. It understands text, images, and video together—and keeps track of up to 256K tokens, so it can follow book-length docs and long videos without losing the thread. What’s new * Stronger brains: Even better at pure text tasks, beating many text-only

Smarter LLMs, Faster—thanks to metadata

Read more

Escaping the Verifier: Learning to Reason via Demonstrations

VacuumVLA: A Two-in-One Robot Hand That Grips and Suctions

ChatDRex: No‑Code, Conversational AI for Drug Repurposing

Qwen3‑VL: An AI that understands text, images, and video—at book length