Despite the development of various deep learning methods for Wi-Fi sensing,package loss often results in noncontinuous estimation of the Channel StateInformation (CSI), which negatively impacts the performance of the learningmodels. To overcome this challenge, we propose a deep learning model based onBidirectional Encoder Representations from Transformers (BERT) for CSIrecovery, named CSI-BERT. CSI-BERT can be trained in an self-supervised manneron the target dataset without the need for additional data. Furthermore, unliketraditional interpolation methods that focus on one subcarrier at a time,CSI-BERT captures the sequential relationships across different subcarriers.Experimental results demonstrate that CSI-BERT achieves lower error rates andfaster speed compared to traditional interpolation methods, even when facingwith high loss rates. Moreover, by harnessing the recovered CSI obtained fromCSI-BERT, other deep learning models like Residual Network and Recurrent NeuralNetwork can achieve an average increase in accuracy of approximately 15% inWi-Fi sensing tasks. The collected dataset WiGesture and code for our model arepublicly available at https://github.com/RS2002/CSI-BERT.