One-sentence Summary

The authors from Xidian University propose the Reliable Confictive Multi-View Learning (RCML) problem, which requires models to provide decisions with attached reliabilities for conflictive multi-view data, and develop an Evidential Confictive Multi-View Learning (ECML) method that learns view-specific evidence to construct opinions with reliability estimates and introduces a conflictive opinion aggregation strategy with theoretical guarantees for modeling multi-view common and view-specific reliabilities.

Key Contributions

• The paper introduces the Reliable Conflictive Multi-view Learning setting, requiring models to output decisions and associated reliabilities for conflicting multi-view data instead of eliminating or replacing conflicting views.
• The paper presents Evidential Conflictive Multi-view Learning, which learns view-specific evidence, constructs view-specific opinions with decision beliefs and reliability, and aggregates conflicting opinions with a strategy proven to model the relationship between multi-view common and view-specific reliabilities.
• Experiments on six public datasets show that ECML improves accuracy, reliability, and robustness over state-of-the-art multi-view learning baselines.

Introduction

Multi-view AI systems, such as autonomous vehicles and recommender systems, combine complementary information from sensors or generated content, making reliable fusion central to clustering, retrieval, and recommendation. Previous approaches typically assume aligned views or resolve conflicts by detecting outliers, discarding instances, or realigning them, but these strategies are insufficient when applications must still decide on ambiguous, conflicting data. The authors leverage evidential deep learning to address this limitation, defining a Reliable Conflictive Multi-view Learning problem and proposing ECML. Their method learns view-specific evidence, converts it into Dirichlet-based opinions with belief mass and reliability, measures cross-view conflict, and aggregates opinions so that conflicting instances receive lower final reliability estimates.

Dataset

• The authors use six multi-view datasets to evaluate trusted multi-view feature fusion and decision fusion. Each dataset is represented with multiple feature views, some of which are extracted from raw images or text using pretrained or handcrafted feature extractors.

• HandWritten is a handwritten numeral dataset with 2000 instances from classes 0 through 9, with 200 patterns per class. The paper represents this dataset using six feature sets.

• CUB is a fine-grained bird dataset containing 11788 instances associated with text descriptions across 200 bird categories. The authors restrict the experiment to the first 10 categories. They extract image features using GoogleNet and corresponding text features using doc2vec.

• HMDB is a large-scale human action recognition dataset with 6718 instances from 51 action categories. For this dataset, the authors use HOG and MBH features as multiple views.

• Scene15 contains 4485 images from 15 indoor and outdoor scene categories. The authors construct three feature views from GIST, PHOG, and LBP descriptors.

• Caltech101 includes 8677 images from 101 object categories. The authors select the first 10 categories and extract two deep feature views using DECAF and VGG19.

• PIE contains 680 instances belonging to 68 classes. The authors use intensity, LBP, and Gabor features as three views.

• The datasets are used to create multi-view representation tasks where each dataset provides several complementary views for classification. Feature extraction is part of dataset preparation, including GoogleNet and doc2vec for CUB, DECAF and VGG19 for Caltech101, and HOG, MBH, GIST, PHOG, LBP, intensity, and Gabor features for other datasets.

• To build a conflictive test set, the authors add noise to a partial percentage of test instances using Gaussian noise with varying standard deviations. They also create unaligned views by modifying the information of a random view for selected instances, causing that view’s corresponding label to disagree with the true instance label.

• Each method is evaluated over 10 runs, and the authors report mean values and standard deviations to account for variation in these conflictive test settings.

Method

The method addresses the Reliable Conflictive Multi-view Learning problem, where the goal is to predict class labels for test instances and also quantify the reliability of each prediction. The input consists of $V$V views, with normal instances and conflictive instances. For the $n$n-th instance, the feature vector of the $v$v-th view is denoted by $\mathbf{x}_n^v \in \mathbb{R}^{D_v}$xnv​∈RDv​, and its ground-truth label is represented by a one-hot vector $\mathbf{y}_n \in \{0,1\}^K$yn​∈{0,1}K. Normal instances have consistent view information, while conflictive instances may contain noise views or unaligned views that disagree with the main category pattern. The problem setting is shown in the figure below:

The proposed Evidential Conflictive Multi-view Learning framework has two main stages. The first stage learns view-specific evidential representations. The second stage fuses the view-specific opinions through conflictive opinion aggregation to obtain a final multi-view decision and uncertainty estimate. The architecture is shown in the framework diagram below:

In the first stage, each view is processed by a view-specific evidential deep neural network $f^v(\cdot)$fv(⋅). These networks output nonnegative evidence vectors $\mathbf{e}_n^v$env​ that measure the support collected for each category. The evidence is then transformed into Dirichlet distribution parameters:

The Dirichlet strength is defined as

From the Dirichlet parameters, a view-specific multinomial opinion is constructed. The belief mass for class $k$k and the uncertainty mass are computed as

The projected probability of class $k$k is

Thus, each view produces not only a class distribution but also an uncertainty mass. Lower evidence leads to higher uncertainty, while stronger evidence leads to higher confidence.

To encourage consistency among view-specific models during training, the method introduces a conflictive degree between two opinions. For opinions $\mathbf{w}^p$wp and $\mathbf{w}^q$wq, the conflictive degree is

where

measures the projected probability distance between the two views, and

measures the conjunctive certainty. When both views are highly confident but produce conflicting class probabilities, the conflictive degree becomes large. Minimizing this term forces the view-specific networks to learn shared multi-view information and reduces erroneous conflicts caused by unreliable single-view decisions.

In the fusion stage, the method aggregates view-specific opinions using conflictive opinion aggregation. For two opinions $\mathbf{w}^A$wA and $\mathbf{w}^B$wB, the aggregated belief mass is

and the aggregated uncertainty is

The aggregated base rate is

This rule treats conflicting opinions more cautiously than standard fusion methods. If a highly uncertain opinion is combined, the aggregated uncertainty can increase rather than automatically decrease. This behavior is important for conflictive multi-view data, because disagreement among reliable views may indicate an unreliable decision. For multiple views, the final joint opinion is obtained by sequential aggregation:

The fusion stage can be implemented through a simple average pooling layer over view-specific evidence, corresponding to the conflictive opinion aggregation rule.

For training, the softmax layer of conventional deep networks is replaced with a nonnegative activation layer, so the network output can be interpreted as evidence. The classification loss is adapted for Dirichlet-based predictions. For an instance, the evidence-based cross-entropy term is

where $\psi(\cdot)$ψ(⋅) is the digamma function. To penalize misleading evidence that remains on incorrect classes, a KL divergence term is added:

with

The KL weight is annealed during training:

so that the KL regularization gradually increases and prevents premature convergence of misclassified instances to uniform distributions.

The training objective combines three components: a joint evidential loss for the fused Dirichlet prediction, view-specific evidential losses for each single-view prediction, and a consistency loss based on the conflictive degree:

The overall loss is

where

Through this training process, the model learns to collect view-specific evidence, produce calibrated uncertainty estimates, enforce cross-view consistency, and aggregate opinions in a way that reflects both agreement and conflict among views.

Experiment

ECML is evaluated on six real-world multiview datasets against feature-fusion and decision-fusion baselines, with normal and artificially conflictive test sets involving noisy or unaligned views. The experiments validate classification performance, conflictive degree quantification, and uncertainty estimation. The method outperforms baselines on normal data, remains robust when views conflict, and aligns higher uncertainty with greater noise or conflict, supporting its reliability and effectiveness for multi-view learning.

The table summarizes multi-view datasets with varied sizes, class counts, and feature views, including handwritten numerals, birds, actions, scenes, objects, and faces. Some datasets use one or more complementary feature types, such as image and text features for CUB or HOG and MBH features for HMDB. The experimental context also evaluates these datasets under normal, noisy, and misaligned view settings. Dataset feature views differ in modality, with CUB combining image and text features and Scene15 using multiple local feature types. Evaluation includes conflictive views created by adding Gaussian noise and by misaligning information in random views.

Across normal multi-view classification tests, ECML achieves the highest accuracy on every dataset compared with baselines based on decision fusion and multi-view learning. The gains are larger on datasets with more room for improvement, such as HMDB and CUB, while the method still improves accuracy on datasets where baselines already perform well. ECML consistently outperforms all listed baselines on normal test sets. The method shows notable advantages on challenging datasets such as HMDB and CUB, where the accuracy gap over the strongest baselines is comparatively large.

The proposed method achieves the highest accuracy on all evaluated conflictive test sets and maintains an advantage over prior multi-view decision fusion baselines. Relative gains are most pronounced on datasets such as PIE and Scene15, while smaller advantages are seen on HandWritten and Caltech101. The results indicate that modeling view-specific conflicts helps preserve classification performance when view contents are noisy or misaligned. Accuracy generally drops under conflictive evaluation, but the proposed method remains stronger than all compared baselines. The method shows larger relative improvements on conflict-prone or harder datasets, including PIE and Scene15. Conflictive opinion aggregation is associated with awareness of view-specific conflicts and better robustness to noisy or unaligned views.

Experiments evaluate multi-view classification on diverse datasets with varying sizes, classes, feature views, and both normal and conflictive settings involving noise or misalignment. The proposed ECML method consistently outperforms decision fusion and multi-view learning baselines, with especially strong gains on more challenging datasets such as HMDB and CUB. Under conflictive evaluation, it remains the most accurate approach, particularly on view-conflict-sensitive datasets like PIE and Scene15, indicating that modeling view-specific conflicts improves robustness to noisy or unaligned views.