Teaching AI to Translate Using Pictures: From Words to Sentences

Kari Jaaskelainen

12 Nov 2025 — 1 min read

What if AI could learn to translate without any bilingual textbooks? This research shows how pictures can act as a bridge between languages.

When we look at the same photo, speakers of different languages describe the same objects and actions. The team uses images as “pivots” so a model can connect words across languages—no parallel sentences needed.

The twist: they train progressively. First, the system learns word-level matches grounded in the image (for example, dog ↔ perro). Because words tied to visual content are less varied, this step is cleaner. Then it scales up to full sentences, using the learned word links to filter noise in image–caption pairs and build better translations.

On two benchmark datasets (IAPR-TC12 and Multi30k), this approach beat other leading zero-resource translation methods.

Why it matters: Billions of speakers live in languages with few translated texts. Teaching AI to translate via images could help bring quality translation to low-resource communities.

Paper: http://arxiv.org/abs/1906.00872v1

Paper: http://arxiv.org/abs/1906.00872v1

Register: https://www.AiFeta.com

AI NLP MachineTranslation ComputerVision ZeroResource Multimodal DeepLearning Research

Automating GDPR Compliance: A Roadmap for Companies and Law Firms

GDPR compliance is more than checkboxes. A new roadmap from the Privatech project shows how automation and machine learning can help companies and law firms assess—and even generate—privacy compliance. * Shift the focus to data processors’ real workflows: drafting policies, mapping data uses, documenting decisions. * Break compliance into machine-ready

FPGAs for Faster, Leaner Deep Learning: A Review of CNN Accelerators

Deep learning drives image search, robots, and medical scans. Most systems lean on CPUs and GPUs. This review asks: what if we run convolutional neural networks (CNNs) on FPGAs—reconfigurable chips you can tailor to the model? * Why FPGAs: custom dataflows, low latency, and strong energy efficiency—great for cameras,

Dynamic-K: Recommendations That Know When to Stop

Most apps show a fixed number of “top” items—say 10 movies or 20 products—assuming there are always enough good options. But that’s not always true: sometimes there are few relevant items, or some users are extra picky. The result? Filler recommendations. Dynamic-K flips the script. Instead of

Teaching chatbots to stop contradicting themselves (DECODE)

Teaching chatbots to stop contradicting themselves Ever had a bot say one thing, then the opposite a few turns later? This study introduces DECODE—a new task and dataset for spotting contradictions in everyday conversations, drawn from both human-human and human-bot chats. * New data beats existing natural language inference (NLI)

Read more

Automating GDPR Compliance: A Roadmap for Companies and Law Firms

FPGAs for Faster, Leaner Deep Learning: A Review of CNN Accelerators

Dynamic-K: Recommendations That Know When to Stop

Teaching chatbots to stop contradicting themselves (DECODE)