From Pixels to Posts: Retrieval-Augmented Fashion Captioning and Hashtag Generation
Meet a new way to write captions and hashtags for fashion photos—grounded in what is actually in the picture.
- Detects multiple garments in an image with a YOLO-based model.
- Extracts dominant colors and infers fabric and gender by retrieving similar products with CLIP and FAISS.
- Packages these facts as an evidence pack that steers a large language model to stay faithful while sounding stylish.
Why it matters: classic end-to-end captioners often miss attributes or hallucinate. Retrieval-augmented generation keeps the style and improves factual grounding.
Results: the detector reached 0.71 mAP across nine garment types. The RAG-LLM pipeline delivered more attribute-aligned captions and hashtags with higher coverage (including full coverage at the 50% threshold), while a fine-tuned BLIP baseline showed higher word overlap but weaker generalization.
Takeaway: blend vision, retrieval, and LLMs to scale accurate, on-brand fashion copy across products and shoots.
Paper: https://arxiv.org/abs/2511.19149v1
Register: https://www.AiFeta.com
AI FashionTech ComputerVision GenAI RAG LLM YOLO CLIP BLIP ecommerce arXiv
