HyperAIHyperAI

Command Palette

Search for a command to run...

Explicit Visual Prompting for Low-Level Structure Segmentations

Liu Weihuang ; Shen Xi ; Pun Chi-Man ; Cun Xiaodong

Abstract

We consider the generic problem of detecting low-level structures in images,which includes segmenting the manipulated parts, identifying out-of-focuspixels, separating shadow regions, and detecting concealed objects. Whereaseach such topic has been typically addressed with a domain-specific solution,we show that a unified approach performs well across all of them. We takeinspiration from the widely-used pre-training and then prompt tuning protocolsin NLP and propose a new visual prompting model, named Explicit VisualPrompting (EVP). Different from the previous visual prompting which istypically a dataset-level implicit embedding, our key insight is to enforce thetunable parameters focusing on the explicit visual content from each individualimage, i.e., the features from frozen patch embeddings and the input'shigh-frequency components. The proposed EVP significantly outperforms otherparameter-efficient tuning protocols under the same amount of tunableparameters (5.7% extra trainable parameters of each task). EVP also achievesstate-of-the-art performances on diverse low-level structure segmentation taskscompared to task-specific solutions. Our code is available at:https://github.com/NiFangBaAGe/Explicit-Visual-Prompt.


Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing

HyperAI Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp