Hu Jian ; Lin Jiayi ; Cai Weitong ; Gong Shaogang

Abstract

Camouflaged object detection (COD) approaches heavily rely on pixel-levelannotated datasets. Weakly-supervised COD (WSCOD) approaches use sparseannotations like scribbles or points to reduce annotation effort, but this canlead to decreased accuracy. The Segment Anything Model (SAM) shows remarkablesegmentation ability with sparse prompts like points. However, manual prompt isnot always feasible, as it may not be accessible in real-world application.Additionally, it only provides localization information instead of semanticone, which can intrinsically cause ambiguity in interpreting the targets. Inthis work, we aim to eliminate the need for manual prompt. The key idea is toemploy Cross-modal Chains of Thought Prompting (CCTP) to reason visual promptsusing the semantic information given by a generic text prompt. To that end, weintroduce a test-time adaptation per-instance mechanism called GeneralizableSAM (GenSAM) to automatically enerate and optimize visual prompts the generictask prompt for WSCOD. In particular, CCTP maps a single generic text promptonto image-specific consensus foreground and background heatmaps usingvision-language models, acquiring reliable visual prompts. Moreover, totest-time adapt the visual prompts, we further propose Progressive MaskGeneration (PMG) to iteratively reweight the input image, guiding the model tofocus on the targets in a coarse-to-fine manner. Crucially, all networkparameters are fixed, avoiding the need for additional training. Experimentsdemonstrate the superiority of GenSAM. Experiments on three benchmarksdemonstrate that GenSAM outperforms point supervision approaches and achievescomparable results to scribble supervision ones, solely relying on general taskdescriptions as prompts. our codes is in: https://lwpyh.github.io/GenSAM/.