HyperAIHyperAI

Command Palette

Search for a command to run...

Length-Aware DETR for Robust Moment Retrieval

Seojeong Park1 Jiho Choi1 Kyungjune Baek2 Hyunjung Shim1

Abstract

Video Moment Retrieval (MR) aims to localize moments within a video based ona given natural language query. Given the prevalent use of platforms likeYouTube for information retrieval, the demand for MR techniques issignificantly growing. Recent DETR-based models have made notable advances inperformance but still struggle with accurately localizing short moments.Through data analysis, we identified limited feature diversity in shortmoments, which motivated the development of MomentMix. MomentMix employs twoaugmentation strategies: ForegroundMix and BackgroundMix, each enhancing thefeature representations of the foreground and background, respectively.Additionally, our analysis of prediction bias revealed that short momentsparticularly struggle with accurately predicting their center positions ofmoments. To address this, we propose a Length-Aware Decoder, which conditionslength through a novel bipartite matching process. Our extensive studiesdemonstrate the efficacy of our length-aware approach, especially in localizingshort moments, leading to improved overall performance. Our method surpassesstate-of-the-art DETR-based methods on benchmark datasets, achieving thehighest R1 and mAP on QVHighlights and the highest R1@0.7 on TACoS andCharades-STA (such as a 2.46% gain in R1@0.7 and a 2.57% gain in mAP averagefor QVHighlights). The code is available athttps://github.com/sjpark5800/LA-DETR.


Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing

HyperAI Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp
Length-Aware DETR for Robust Moment Retrieval | Papers | HyperAI