4 个月前

神经层扩散:从拓扑视角看异质性和过平滑在图神经网络中的影响

神经层扩散:从拓扑视角看异质性和过平滑在图神经网络中的影响

摘要

细胞层束(cellular sheaves)通过为图的节点和边分配向量空间和线性映射,赋予图以“几何”结构。图神经网络(GNNs)隐式地假设了一个具有平凡底层层束的图。这一选择反映在图拉普拉斯算子的结构、相关扩散方程的性质以及离散化该方程的卷积模型的特征中。在本文中,我们利用细胞层束理论证明了图的底层几何结构与GNNs在异质亲和设置中的性能及其过度平滑行为之间存在深刻的联系。通过考虑一个逐渐一般化的层束层次结构,我们研究了层束扩散过程在无限时间极限下实现类线性分离的能力如何扩展。同时,我们证明了当层束非平凡时,离散化的参数扩散过程对其渐近行为具有比GNNs更大的控制能力。从实际应用的角度出发,我们探讨了如何从数据中学习层束。所得到的层束扩散模型具备许多理想的特性,解决了经典图扩散方程(及相应的GNN模型)的局限性,并在异质亲和设置中取得了有竞争力的结果。总体而言,我们的工作建立了GNNs与代数拓扑之间的新联系,对这两个领域都具有重要意义。

代码仓库

twitter-research/neural-sheaf-diffusion
官方
pytorch
GitHub 中提及

基准测试

基准方法指标
node-classification-on-actorGen-NSD
Accuracy: 37.80 ± 1.22
node-classification-on-actorDiag-NSD
Accuracy: 37.79 ± 1.01
node-classification-on-actorO(d)-NSD
Accuracy: 37.81 ± 1.15
node-classification-on-chameleonO(d)-NSD
Accuracy: 68.04 ± 1.58
node-classification-on-chameleonGen-NSD
Accuracy: 67.93 ± 1.58
node-classification-on-chameleonDiag-NSD
Accuracy: 68.68 ± 1.73
node-classification-on-citeseer-48-32-20O(d)-NSD
1:1 Accuracy: 76.70 ± 1.57
node-classification-on-citeseer-48-32-20Diag-NSD
1:1 Accuracy: 77.14 ± 1.85
node-classification-on-citeseer-48-32-20Gen-NSD
1:1 Accuracy: 76.32 ± 1.65
node-classification-on-cora-48-32-20-fixedGen-NSD
1:1 Accuracy: 87.30 ± 1.15
node-classification-on-cora-48-32-20-fixedO(d)-NSD
1:1 Accuracy: 86.90 ± 1.13
node-classification-on-cora-48-32-20-fixedDiag-NSD
1:1 Accuracy: 87.14 ± 1.06
node-classification-on-cornellGen-NSD
Accuracy: 85.68 ± 6.51
node-classification-on-cornellO(d)-NSD
Accuracy: 84.86 ± 4.71
node-classification-on-cornellDiag-NSD
Accuracy: 86.49 ± 7.35
node-classification-on-non-homophilic-10Diag-NSD
1:1 Accuracy: 37.79 ± 1.01
node-classification-on-non-homophilic-10Gen-NSD
1:1 Accuracy: 37.80 ± 1.22
node-classification-on-non-homophilic-10O(d)-NSD
1:1 Accuracy: 37.81 ± 1.15
node-classification-on-non-homophilic-11Diag-NSD
1:1 Accuracy: 68.68 ± 1.73
node-classification-on-non-homophilic-11O(d)-NSD
1:1 Accuracy: 68.04 ± 1.58
node-classification-on-non-homophilic-11Gen-NSD
1:1 Accuracy: 67.93 ± 1.58
node-classification-on-non-homophilic-12Gen-NSD
1:1 Accuracy: 53.17 ± 1.31
node-classification-on-non-homophilic-12O(d)-NSD
1:1 Accuracy: 56.34 ± 1.32
node-classification-on-non-homophilic-12Diag-NSD
1:1 Accuracy: 54.78 ± 1.81
node-classification-on-non-homophilic-7Gen-NSD
1:1 Accuracy: 85.68 ± 6.51
node-classification-on-non-homophilic-7O(d) - NSD
1:1 Accuracy: 84.86 ± 4.71
node-classification-on-non-homophilic-7Diag-NSD
1:1 Accuracy: 86.49 ± 7.35
node-classification-on-non-homophilic-8O(d)-NSD
1:1 Accuracy: 89.41 ± 4.74
node-classification-on-non-homophilic-8Diag-NSD
1:1 Accuracy: 88.63 ± 2.75
node-classification-on-non-homophilic-8Gen-NSD
1:1 Accuracy: 89.21 ± 3.84
node-classification-on-non-homophilic-9Gen-NSD
1:1 Accuracy: 82.97 ± 5.13 
node-classification-on-non-homophilic-9O(d)-NSD
1:1 Accuracy: 85.95 ± 5.51
node-classification-on-non-homophilic-9Diag-NSD
1:1 Accuracy: 85.67 ± 6.95
node-classification-on-pubmed-48-32-20-fixedDiag-NSD
1:1 Accuracy: 89.42 ± 0.43
node-classification-on-pubmed-48-32-20-fixedGen-NSD
1:1 Accuracy: 89.33 ± 0.35
node-classification-on-pubmed-48-32-20-fixedO(d)-NSD
1:1 Accuracy: 89.49 ± 0.40
node-classification-on-squirrelDiag-NSD
Accuracy: 54.78 ± 1.81
node-classification-on-squirrelGen-NSD
Accuracy: 53.17 ± 1.31
node-classification-on-squirrelO(d)-NSD
Accuracy: 56.34 ± 1.32
node-classification-on-texasDiag-NSD
Accuracy: 85.67 ± 6.95
node-classification-on-texasGen-NSD
Accuracy: 82.97 ± 5.13
node-classification-on-texasO(d)-NSD
Accuracy: 85.95 ± 5.51
node-classification-on-wisconsinGen-NSD
Accuracy: 89.21 ± 3.84
node-classification-on-wisconsinDiag-NSD
Accuracy: 88.63 ± 2.75
node-classification-on-wisconsinO(d)-NSD
Accuracy: 89.41 ± 4.74

用 AI 构建 AI

从想法到上线——通过免费 AI 协同编程、开箱即用的环境和市场最优价格的 GPU 加速您的 AI 开发

AI 协同编程
即用型 GPU
最优价格
立即开始

Hyper Newsletters

订阅我们的最新资讯
我们会在北京时间 每周一的上午九点 向您的邮箱投递本周内的最新更新
邮件发送服务由 MailChimp 提供
神经层扩散:从拓扑视角看异质性和过平滑在图神经网络中的影响 | 论文 | HyperAI超神经