Yin Han ; Xiao Yang ; Bai Jisheng ; Das Rohan Kumar

Abstract

Sound Event Detection (SED) is challenging in noisy environments whereoverlapping sounds obscure target events. Language-queried audio sourceseparation (LASS) aims to isolate the target sound events from a noisy clip.However, this approach can fail when the exact target sound is unknown,particularly in noisy test sets, leading to reduced performance. To addressthis issue, we leverage the capabilities of large language models (LLMs) toanalyze and summarize acoustic data. By using LLMs to identify and selectspecific noise types, we implement a noise augmentation method for noise-robustfine-tuning. The fine-tuned model is applied to predict clip-wise eventpredictions as text queries for the LASS model. Our studies demonstrate thatthe proposed method improves SED performance in noisy environments. This workrepresents an early application of LLMs in noise-robust SED and suggests apromising direction for handling overlapping events in SED. Codes andpretrained models are available athttps://github.com/apple-yinhan/Noise-robust-SED.