Noise Contrastive Estimation NCE is a statistical model estimation method proposed by Gutmann and Hyv¨arinen. It is mainly used to solve complex computational problems of neural networks and is currently widely used in the fields of image processing and natural language processing.

The idea of NCE

Compare the real samples with the "noise samples" and find the rules of the real samples from them, that is, "use comparative learning" to transform the probability generation problem into a binary classification problem. In other words, compare the real samples with the wrong samples randomly sampled from a simple distribution, and try to find the difference between the real samples and the wrong samples.

Features of NCE

NCE transforms complex problems into binary classification problems, that is, true samples are judged as 1 and samples sampled from another distribution are judged as 0. In addition, NCE can directly estimate the probability distribution parameters when the normalization factor calculation cannot be completed directly.

Related words: binary classification problem