Leave-out methodIt is a model evaluation method that divides the dataset D into two mutually exclusive sets. Assuming that one set is the training set S and the other is the test set T, we have:

D = S ∪ T , S ∩ T = ∅

The division of training/test sets should keep the data distribution as consistent as possible. In order to avoid the additional bias introduced in the data division process affecting the results, stratified sampling is usually used.

Since different division methods will lead to different training/test sets, the corresponding model evaluation results will also be different. Therefore, the estimation results of a single use of the holdout method are mostly unreliable. It usually requires several random divisions, repeated experimental evaluations, and the average value is taken as the evaluation result.