Exemplar-Free Counting aims to count objects of interest without intensiveannotations of objects or exemplars. To achieve this, we propose a GatedContext-Aware Swin-UNet (GCA-SUNet) to directly map an input image to thedensity map of countable objects. Specifically, a set of Swin transformers forman encoder to derive a robust feature representation, and a Gated Context-AwareModulation block is designed to suppress irrelevant objects or backgroundthrough a gate mechanism and exploit the attentive support of objects ofinterest through a self-similarity matrix. The gate strategy is alsoincorporated into the bottleneck network and the decoder of the Swin-UNet tohighlight the features most relevant to objects of interest. By explicitlyexploiting the attentive support among countable objects and eliminatingirrelevant features through the gate mechanisms, the proposed GCA-SUNet focuseson and counts objects of interest without relying on predefined categories orexemplars. Experimental results on the real-world datasets such as FSC-147 andCARPK demonstrate that GCA-SUNet significantly and consistently outperformsstate-of-the-art methods. The code is available athttps://github.com/Amordia/GCA-SUNet.