Meet FITRep: Attention-Guided, Transparent Item Deduplication

Meet FITRep: Attention-Guided, Transparent Item Deduplication

Ever scrolled past pages of near-identical listings? Those duplicates hurt user experience and waste ad spend.

FITRep is an attention-guided, white-box way to represent items across image + text, so platforms can tell what’s truly important (primary) versus extra (auxiliary) and cluster near-duplicates with confidence.

  • CHIE: uses Multimodal LLMs to extract hierarchical concepts from each item.
  • SPDR: an adaptive, structure-preserving UMAP-based compression that keeps key relationships.
  • FBC: FAISS-powered clustering that assigns every item a stable cluster ID.
In Meituan’s ad system, FITRep increased CTR by +3.60% and CPM by +4.25% in online A/B tests.

Result: cleaner catalogs, fewer duplicates, more relevant recommendations—and measurable revenue lift.

Paper: https://arxiv.org/abs/2511.21389v1

Authors: Guoxiao Zhang, Ao Li, Tan Qu, Qianlong Xie, Xingxing Wang

Paper: https://arxiv.org/abs/2511.21389v1

Register: https://www.AiFeta.com

MLLM RecommenderSystems InformationRetrieval AdsTech ComputerVision NLP UMAP FAISS Deduplication Ecommerce

Read more