Wiki

MCMC is an algorithm for sampling from random distributions based on Markov chains. It approximates the posterior distribution of the parameter of interest by randomly sampling in the probability space. The basic theory of MCMC is the Markov process. In related algorithms, in order to sample from a specified distribution, we can simulate the Markov process from any state first.

Evolutionary algorithm is a general problem-solving method that draws on the natural selection and natural genetic mechanisms of the biological world. Basic method: Use simple coding technology to represent various complex structures, use simple genetic operations and natural selection of survival of the fittest to guide learning and determine the search direction; Use population to organize the search, so that […]

Genetic algorithm (GA) is a search algorithm used in computational mathematics to solve optimization problems. It is a type of evolutionary algorithm. Evolutionary algorithms originally borrowed from some phenomena in evolutionary biology, including inheritance, mutation, natural selection, and hybridization. Genetic algorithms are usually implemented in the form of computer simulation. For an optimization problem, if there are […]

Gain ratio usually refers to information gain ratio, which represents the ratio of node information to node split information metric. Gain ratio is usually used as one of the attribute selection methods. The other two common methods are information gain and Gini index. The gain ratio formula is as follows: $latex {GainRatio{ \left( {R} […]

Hilbert space is a complete inner product space, which can be understood as a complete vector space with inner product. Hilbert space is based on finite-dimensional Euclidean space and can be seen as a generalization of the latter. It is not limited to real numbers and finite dimensions, but it is not complete. Like Euclidean space, Hilbert space is also an inner product space, and has distance and angle […]

Hidden Markov Model (HMM) is a probability model about time series, which describes the process of generating an observable random sequence of states from each state. Hidden Markov Model is a statistical model that is used to describe a Markov chain with hidden unknown parameters.

Hidden layers refer to the layers other than the input layer and the output layer in a multi-level feedforward neural network. Hidden layers do not directly receive external signals or send signals to the outside world. They are only needed when data is separated nonlinearly. Neurons on hidden layers can take many forms, such as maximum pooling layers and convolutional layers, which perform different mathematical functions. […]

Hard voting is a voting method that directly outputs class labels, which mainly exists in classification machine learning algorithms. Voting is a combination strategy for classification problems in ensemble learning. Its basic idea is to select the class with the most output in the algorithm. Hard voting is to select the label with the most output by the algorithm. If the number of labels is equal, they are sorted in ascending order. […]

Independent and identically distributed (IID) means that the probability distribution of each variable in a set of random variables is the same, and these random variables are independent of each other. A set of random variables is independent and identically distributed does not mean that the probability of each event in their sample space is the same. For example, the sequence of results obtained by throwing uneven dice is independent and identically distributed, but the probability of each [...]

Incremental learning means that when new data is added, only the new data is updated. Incremental learning can continuously learn new knowledge from new samples while preserving most of the previously learned knowledge. Incremental learning is similar to the human learning model, which is a process of gradual accumulation and updating. The traditional learning method is batch learning, which prepares all the data[…]

A knowledge base is a special database that is used for knowledge management to facilitate the collection, organization and extraction of knowledge in related fields. The knowledge in the database comes from domain experts. It is a collection of knowledge in related fields for solving problems, covering basic facts, rules and other relevant information. A knowledge base is a structured, easy-to-operate, easy-to-use, and comprehensive knowledge base in knowledge engineering.

K-nearest neighbor algorithm KNN is a basic classification and regression algorithm that uses the K points closest to itself to vote to determine the classification of the classification data. KNN characteristics KNN is lazy learning KNN has high computational complexity Different K values will result in different classification results

JS divergence measures the similarity of two probability distributions. It is a variant of KL divergence and solves the asymmetric problem of KL divergence. Generally, JS divergence is symmetric and its value is between 0 and 1. It is defined as follows: There is a problem when measuring KL divergence and JS divergence: If two […]

The least squares regression tree is a commonly used regression tree algorithm. In order to minimize the square error, it is necessary to traverse the values of each feature in turn and calculate the error of each possible split point at the moment. Finally, the point with the smallest split error is selected and the input space is split into two parts. The above steps are repeated recursively until the split is completed. This method splits […]

Latent Dirichlet Allocation (LDA) is a topic model that can express the topic of each document in a document set in the form of probability distribution. It is also an unsupervised learning algorithm that does not require a manually annotated training set for training. It only requires a document set and the number of specified topics K. In addition, for each topic, […]

The minimum description length is also called the minimum description length principle, which means that among multiple data storage hypotheses, the hypothesis that can produce the most data compression effect is the best. The minimum description length principle is a result of the formalization of Occam's razor, introduced by Jorma Rissanen in 1978. Its main point is: in any […]

Meta-learning is a subfield of machine learning, where automatic learning algorithms are applied to the metadata of machine learning experiments. The core of meta-learning is the "learning (training)" process, that is, studying how to make neurons make good use of old knowledge so that they can adjust themselves according to new tasks. The diagram is as follows: The initial parameters of the neural network (blue ■ […]

Max pooling is a commonly used pooling operation that reduces the amount of data by maximizing the value. Its usual operation is to divide the input image into several rectangular regions and output the maximum value for each subdomain. Currently, in addition to max pooling, average pooling is also commonly used. It reduces the complex calculations from the upper hidden layer and can be unaffected by the target […]

The maximum margin refers to the situation in the classification task where the training data interval is the largest. The hyperplane that satisfies the maximum margin can classify the data into positive and negative categories, and the decision confidence is the highest at this time. In the support vector machine, there are hard margin maximization and soft margin maximization, both of which are strategies for finding the maximum margin.

Markov random field is also called Markov network. It is a model described by an undirected graph, which contains a set of nodes. Each node corresponds to a single variable or a set of variables. The links between nodes are undirected. Markov random field describes that among multiple data points, there is no clear relationship between two points in front and back, or in the direction. Although there is a mutual relationship between the two points, […]

Automatic summarization is the process of shortening text documents using software in order to create summaries that contain the main points of the original document. It is currently part of the field of machine learning and data mining, and its purpose is to find subsets of data that contain relevant "information". There are currently two methods for automatic summarization: extraction and abstraction. Extraction is based on words, phrases in the original text […]

Automatic speech recognition technology is a technology that converts human speech into text. Due to the diversity and complexity of speech signals, the current speech recognition system can only achieve satisfactory performance under certain restrictions (it can only be applied to certain specific occasions). Automatic speech recognition definition The goal of automatic speech recognition technology is to enable computers to "[…]

An autoencoder is an artificial neural network used for efficient encoding in unsupervised learning, usually for dimensionality reduction. It is also a data compression algorithm in which the compression and decompression functions of the data are lossy and automatically learned from samples, but in most cases where autoencoders are mentioned, the compression and decompression functions are learned through a neural network […]

The human attention mechanism is based on intuition. It is a means for humans to use limited resources to quickly filter out high-value information from a large amount of information. The attention mechanism in deep learning draws on the human attention thinking mode and is widely used in various scenarios such as natural language processing, image classification, speech recognition, etc., and has achieved remarkable results.