Anomaly detection is the identification of items, events, or observations that do not match the expected pattern or other items in the dataset. Typically, anomaly items turn out to be bank fraud, structural defects, medical problems, text errors, and other types of problems.

Anomaly detection technology application

Anomaly detection technology is used in various fields, such as intrusion detection, fraud detection, fault detection, system health monitoring, sensor network event detection, and ecosystem disturbance detection. It is usually used to remove abnormal data from the dataset in preprocessing, and the dataset without abnormal data will significantly improve the accuracy in supervised learning.

Classification of Anomaly Detection Methods

Unsupervised anomaly detection methods can detect anomalies in unlabeled test data by finding instances that are least matched with other data.

Supervised anomaly detection methods require a dataset that has been labeled as “normal” and “abnormal” and involve training a classifier.

Semi-supervised anomaly detection methods create a model that represents normal behavior based on a given normal training dataset and then detect the likelihood of test instances generated by the learned model.

• Model-based techniques: Many anomaly detection techniques first build a data model. Anomalies are those objects that do not fit the model perfectly.
• Proximity-based techniques: Often a proximity measure can be defined between objects, with outlier objects being those that are far away from most of the other objects.
• Density-based techniques: Density estimates of objects can be computed relatively directly, especially when there is a proximity measure between objects.

Application Scenario

• Fraud Detection: Testing Card Security
• Intrusion detection: Detecting intrusions into computer systems
• Medical field: Testing human health