Klear: Unified audio–video generation, finally in sync
Meet Klear, an AI model that creates video and audio together—so lip movements match words, sounds land on time, and visuals stay coherent. It targets the sync and quality gaps seen in earlier open systems.
What makes it work
- Single‑tower design with unified transformer blocks and an "Omni‑Full Attention" mechanism keeps audio and video tightly aligned and scales smoothly.
- Progressive multi‑task training—random modality masking to joint optimization, plus a staged curriculum—builds robust, balanced A/V skills and avoids unimodal collapse.
- New large‑scale dataset with dense captions; an automated pipeline filters and tightly aligns millions of audio‑video‑caption triplets.
Why it matters
- High‑fidelity, instruction‑following clips in joint or single‑modality modes.
- Strong generalization, including out‑of‑distribution cases.
- Big gains over prior methods; performance comparable to Veo 3 (per authors).
Paper: https://arxiv.org/abs/2601.04151v1
Paper: https://arxiv.org/abs/2601.04151v1
Register: https://www.AiFeta.com
AI GenerativeAI Video Audio Multimodal DeepLearning Research AVSync
