4 个月前

图注意力网络

图注意力网络

摘要

我们介绍了图注意力网络(Graph Attention Networks, GATs),这是一种新颖的神经网络架构,专门用于处理图结构数据,通过利用掩码自注意力层来克服基于图卷积或其近似方法的先前技术的不足。通过堆叠多层,使得节点能够在这些层中关注其邻域特征,我们能够隐式地为邻域中的不同节点指定不同的权重,而无需进行任何昂贵的矩阵运算(如求逆)或依赖于预先了解图结构。这样,我们同时解决了基于谱的图神经网络的几个关键挑战,并使我们的模型既适用于归纳问题也适用于演绎问题。我们的GAT模型在四个已建立的演绎和归纳图基准测试中取得了或达到了最先进的结果:包括Cora、Citeseer和Pubmed引文网络数据集,以及一个蛋白质-蛋白质相互作用数据集(在训练过程中测试图未被见过)。

代码仓库

taishan1994/pytorch_gat
pytorch
GitHub 中提及
gayanku/FDGATII
pytorch
GitHub 中提及
vailatuts/SSFG-regularization
pytorch
GitHub 中提及
ds4dm/sGat
pytorch
GitHub 中提及
joaopedromattos/gnee
pytorch
GitHub 中提及
tatp22/pytorch-fast-GAT
pytorch
GitHub 中提及
alexOarga/haiku-geometric
jax
GitHub 中提及
fanzhenliu/dagad
pytorch
GitHub 中提及
ColdenChan/GAT_D
tf
GitHub 中提及
marble0117/GNN_models_pytorch
pytorch
GitHub 中提及
zhangbo2008/GAT_network
pytorch
GitHub 中提及
shi27feng/transformers.satisfy
pytorch
GitHub 中提及
gmum/umwpl2021
GitHub 中提及
wan2000/mc-mot
pytorch
GitHub 中提及
WantingZhao/my_GAT
tf
GitHub 中提及
AngusMonroe/GAT-pytorch
pytorch
GitHub 中提及
Diego999/pyGAT
pytorch
GitHub 中提及
fongyk/graph-attention
pytorch
GitHub 中提及
galkampel/HyperNetworks
pytorch
GitHub 中提及
davidpicard/homm
pytorch
GitHub 中提及
BIG-S2/keras-gnm
GitHub 中提及
yururyuru00/GAT_pyG
pytorch
GitHub 中提及
zxhhh97/ABot
pytorch
GitHub 中提及
HazyResearch/hgcn
pytorch
GitHub 中提及
weiyangfb/PyTorchSparseGAT
pytorch
GitHub 中提及
arangoml/fastgraphml
pytorch
GitHub 中提及
404nofound/gat
pytorch
GitHub 中提及
giuseppefutia/link-prediction-code
pytorch
GitHub 中提及
Yindong-Zhang/myGAT
pytorch
GitHub 中提及
marblet/gat-pytorch
pytorch
GitHub 中提及
bowu/GraphSearch
pytorch
GitHub 中提及
4pygmalion/GAT_tensorflow
tf
GitHub 中提及
ChengyuSun/gat
pytorch
GitHub 中提及
GraphSAINT/GraphSAINT
tf
GitHub 中提及
tlmakinen/GAT-noise
pytorch
GitHub 中提及
Kaimaoge/IGNNK
pytorch
GitHub 中提及
Anak2016/GAT
pytorch
GitHub 中提及
qema/orca-py
pytorch
GitHub 中提及
snowkylin/gnn
tf
GitHub 中提及
zhao-tong/GNNs-easy-to-use
pytorch
GitHub 中提及
ydzhang-stormstout/HAT
tf
GitHub 中提及
mlzxzhou/keras-gnm
GitHub 中提及
Anou9531/GAT
tf
GitHub 中提及
snownus/COOP
GitHub 中提及
ds4dm/sparse-gcn
pytorch
GitHub 中提及
meliketoy/graph-cnn.pytorch
pytorch
GitHub 中提及
ChengSashankh/trying-GAT
tf
GitHub 中提及
PetarV-/GAT
官方
tf
GitHub 中提及
anish-lu-yihe/SVRT-by-GAT
pytorch
GitHub 中提及
basiralab/RG-Select
pytorch
GitHub 中提及
danielegrattarola/keras-gat
tf
GitHub 中提及
Negus25/GAT-Mindspore
mindspore
GitHub 中提及
blueberryc/pyGAT
pytorch
GitHub 中提及
isotlaboratory/ml4vrp
pytorch
GitHub 中提及
zhanglab-aim/cancer-net
pytorch
GitHub 中提及
PumpkinYing/GAT
pytorch
GitHub 中提及
dmeoli/neuro-sat
GitHub 中提及
nnaakkaaii/g2-MLP
pytorch
GitHub 中提及
pengsl-lab/mssl2drug
pytorch
GitHub 中提及
dzb1998/pyGAT
pytorch
GitHub 中提及
Messham87/COVID_GAT
pytorch
GitHub 中提及
pengsl-lab/mssl
pytorch
GitHub 中提及
gcucurull/jax-gat
jax
GitHub 中提及
gordicaleksa/pytorch-GAT
pytorch
GitHub 中提及
Kristof-Neys/GAT_Neo4j_DGL
tf
GitHub 中提及
liu6zijian/simplified-gcn-model
pytorch
GitHub 中提及
subercui/pyGConvAT
pytorch
GitHub 中提及
YunseobShin/wiki_GAT
tf
GitHub 中提及

基准测试

基准方法指标
document-classification-on-coraGAT
Accuracy: 83.0%
graph-classification-on-cifar10-100kGAT
Accuracy (%): 65.48
graph-classification-on-ddGAT
Accuracy: 73.109±3.413
graph-classification-on-enzymesGAT
Accuracy: 78.611±1.556
graph-classification-on-imdb-bGAT
Accuracy: 84.250±2.062
graph-classification-on-nci1GAT
Accuracy: 85.109±1.107
graph-classification-on-nci109GAT
Accuracy: 82.560±0.601
graph-classification-on-proteinsGAT
Accuracy: 76.786±1.670
graph-property-prediction-on-ogbg-code2GAT
Ext. data: No
Number of params: 11030210
Test F1 score: 0.1569 ± 0.0010
Validation F1 score: 0.1442 ± 0.0017
graph-regression-on-esr2GAT
R2: 0.666±0.000
RMSE: 0.510±0.666
graph-regression-on-f2GAT
R2: 0.886±0.000
RMSE: 0.343±0.886
graph-regression-on-kitGAT
R2: 0.833±0.000
RMSE: 0.443±0.833
graph-regression-on-lipophilicityGAT
R2: 0.820±0.014
RMSE: 0.536±0.020
graph-regression-on-lipophilicity-1GAT
RMSE: 0.95
graph-regression-on-parp1GAT
R2: 0.921±0.000
RMSE: 0.353±0.921
graph-regression-on-pgrGAT
R2: 0.681±0.000
RMSE: 0.546±0.681
graph-regression-on-zinc-100kGAT
MAE: 0.463
graph-regression-on-zinc-fullGAT
Test MAE: 0.078±0.006
heterogeneous-node-classification-on-acmGAT
Macro-F1: 92.26
Micro-F1: 92.19
heterogeneous-node-classification-on-dblp-2GAT
Macro-F1: 93.83
Micro-F1: 93.39
heterogeneous-node-classification-on-freebaseGAT
Accuracy: 65.26
Macro-F1: 40.74
heterogeneous-node-classification-on-imdbGAT
Macro-F1: 58.94
Micro-F1: 64.86
molecular-property-prediction-on-esolGAT
R2: 0.930±0.007
RMSE: 0.540±0.027
molecular-property-prediction-on-freesolvGAT
R2: 0.959±0.011
RMSE: 0.791±0.101
node-classification-on-brazil-air-trafficGAT (Velickovic et al., 2018)
Accuracy: 0.382
node-classification-on-chameleon-60-20-20GAT
1:1 Accuracy: 63.9 ± 0.46
node-classification-on-citeseerGAT
Accuracy: 72.5 ± 0.7%
Training Split: fixed 20 per node
Validation: YES
node-classification-on-citeseer-05GAT
Accuracy: 38.2%
node-classification-on-citeseer-1GAT
Accuracy: 46.5%
node-classification-on-citeseer-60-20-20GAT
1:1 Accuracy: 67.20 ± 0.46
node-classification-on-citeseer-with-publicGAT
Accuracy: 72.5 ± 0.7%
node-classification-on-coraGAT
Accuracy: 83.0% ± 0.7%
Training Split: fixed 20 per node
Validation: YES
node-classification-on-cora-05GAT
Accuracy: 41.4%
node-classification-on-cora-1GAT
Accuracy: 48.6%
node-classification-on-cora-3GAT
Accuracy: 56.8%
node-classification-on-cora-60-20-20-randomGAT
1:1 Accuracy: 76.70 ± 0.42
node-classification-on-cora-with-public-splitGAT
Accuracy: 83.0 ± 0.7%
node-classification-on-cornell-60-20-20GAT
1:1 Accuracy: 76.00 ± 1.01
node-classification-on-europe-air-trafficGAT (Velickovic et al., 2018)
Accuracy: 42.4
node-classification-on-film-60-20-20-randomGAT
1:1 Accuracy: 35.98 ± 0.23
node-classification-on-flickrGAT (Velickovic et al., 2018)
Accuracy: 0.359
node-classification-on-geniusGAT
Accuracy: 55.80 ± 0.87
node-classification-on-non-homophilicGAT
1:1 Accuracy: 76.00 ± 1.01
node-classification-on-non-homophilic-1GAT
1:1 Accuracy: 71.01 ± 4.66
node-classification-on-non-homophilic-13GAT
1:1 Accuracy: 81.53 ± 0.55
node-classification-on-non-homophilic-16GAT
F1-Score: 59.89 ± 4.12
NMI: 55.80 ± 0.87
node-classification-on-non-homophilic-2GAT
1:1 Accuracy: 78.87 ± 0.86
node-classification-on-non-homophilic-4GAT
1:1 Accuracy: 63.9 ± 0.46
node-classification-on-non-homophilic-6GAT
1:1 Accuracy: 61.09±0.77
node-classification-on-pattern-100kGAT
Accuracy (%): 75.824
node-classification-on-penn94GAT
Accuracy: 81.53 ± 0.55
node-classification-on-ppiGAT
F1: 97.3
node-classification-on-pubmedGAT
Accuracy: 79.0 ± 0.3%
F1-Score: 79.0
Training Split: fixed 20 per node
Validation: YES
node-classification-on-pubmed-003GAT
Accuracy: 50.9%
node-classification-on-pubmed-005GAT
Accuracy: 50.4%
node-classification-on-pubmed-01GAT
Accuracy: 59.6%
node-classification-on-pubmed-60-20-20-randomGAT
1:1 Accuracy: 83.28 ± 0.12
node-classification-on-pubmed-with-publicGAT
Accuracy: 79.0%
node-classification-on-squirrel-60-20-20GAT
1:1 Accuracy: 42.72 ± 0.33
node-classification-on-texas-60-20-20-randomGAT
1:1 Accuracy: 78.87 ± 0.86
node-classification-on-usa-air-trafficGAT (Velickovic et al., 2018)
Accuracy: 58.5
node-classification-on-wiki-voteGAT (Velickovic et al., 2018)
Accuracy: 59.4
node-classification-on-wisconsin-60-20-20GAT
1:1 Accuracy: 71.01 ± 4.66
skeleton-based-action-recognition-on-j-hmbdGAT
10%: 58.1

用 AI 构建 AI

从想法到上线——通过免费 AI 协同编程、开箱即用的环境和市场最优价格的 GPU 加速您的 AI 开发

AI 协同编程
即用型 GPU
最优价格
立即开始

Hyper Newsletters

订阅我们的最新资讯
我们会在北京时间 每周一的上午九点 向您的邮箱投递本周内的最新更新
邮件发送服务由 MailChimp 提供
图注意力网络 | 论文 | HyperAI超神经