3 个月前

基于查询选择器的长期序列预测——稀疏注意力的高效模型

Jacek Klimek Jakub Klimek Witold Kraskiewicz Mateusz Topolewski

摘要

近期，多种基于Transformer的改进模型被用于解决时间序列预测问题。本文提出一种名为Query Selector的高效且确定性的算法，用于构建稀疏注意力矩阵。实验结果表明，该方法在ETT、Helpdesk以及BPI'12数据集上均取得了当前最优的性能表现。

代码仓库

moraieu/query-selector

官方

pytorch

GitHub 中提及

moraieu/query-selector-classification

pytorch

GitHub 中提及

基准测试

基准	方法	指标
multivariate-time-series-forecasting-on	QuerySelector	Accuracy: 0.743
multivariate-time-series-forecasting-on-bpi	QuerySelector	Accuracy: 0.79
time-series-forecasting-on-etth1-168-1	Transformer	MAE: 0.6325 MSE: 0.7146
time-series-forecasting-on-etth1-168-1	QuerySelector	MAE: 0.6088 MSE: 0.6835
time-series-forecasting-on-etth1-168-2	Transformer	MAE: 0.2539 MSE: 0.1049
time-series-forecasting-on-etth1-168-2	QuerySelector	MAE: 0.2371 MSE: 0.0935
time-series-forecasting-on-etth1-24-1	Transformer	MAE: 0.4788 MSE: 0.4496
time-series-forecasting-on-etth1-24-1	QuerySelector	MAE: 0.4627 MSE: 0.4226
time-series-forecasting-on-etth1-24-2	Transformer	MAE: 0.183 MSE: 0.0548
time-series-forecasting-on-etth1-24-2	QuerySelector	MAE: 0.1616 MSE: 0.0436
time-series-forecasting-on-etth1-336-1	Transformer	MAE: 0.7041 MSE: 0.8321
time-series-forecasting-on-etth1-336-1	QuerySelector	MAE: 0.7039 MSE: 0.8503
time-series-forecasting-on-etth1-336-2	Transformer	MAE: 0.3201 MSE: 0.1541
time-series-forecasting-on-etth1-336-2	QuerySelector	MAE: 0.2844 MSE: 0.1267
time-series-forecasting-on-etth1-48-1	Transformer	MAE: 0.4968 MSE: 0.4668
time-series-forecasting-on-etth1-48-1	QuerySelector	MAE: 0.4878 MSE: 0.4581
time-series-forecasting-on-etth1-48-2	Transformer	MAE: 0.2144 MSE: 0.074
time-series-forecasting-on-etth1-48-2	QuerySelector	MAE: 0.2118 MSE: 0.0721
time-series-forecasting-on-etth1-720-1	Transformer	MAE: 0.8399 MSE: 1.108
time-series-forecasting-on-etth1-720-1	QuerySelector	MAE: 0.8428 MSE: 1.115
time-series-forecasting-on-etth1-720-2	Transformer	MAE: 0.4213 MSE: 0.2501
time-series-forecasting-on-etth1-720-2	QuerySelector	MAE: 0.373 MSE: 0.2136
time-series-forecasting-on-etth2-168-1	QuerySelector	MAE: 1.0125 MSE: 1.7385
time-series-forecasting-on-etth2-168-1	Transformer	MAE: 0.9726 MSE: 1.6225
time-series-forecasting-on-etth2-168-2	QuerySelector	MAE: 0.3322 MSE: 0.1753
time-series-forecasting-on-etth2-168-2	Transformer	MAE: 0.3547 MSE: 0.1974
time-series-forecasting-on-etth2-24-1	QuerySelector	MAE: 0.4864 MSE: 0.4124
time-series-forecasting-on-etth2-24-1	Transformer	MAE: 0.5013 MSE: 0.4237
time-series-forecasting-on-etth2-24-2	QuerySelector	MAE: 0.2239 MSE: 0.0843
time-series-forecasting-on-etth2-24-2	Transformer	MAE: 0.2479 MSE: 0.0999
time-series-forecasting-on-etth2-336-1	Transformer	MAE: 1.2189 MSE: 2.6617
time-series-forecasting-on-etth2-336-1	QuerySelector	MAE: 1.1859 MSE: 2.3168
time-series-forecasting-on-etth2-336-2	Transformer	MAE: 0.3805 MSE: 0.2191
time-series-forecasting-on-etth2-336-2	QuerySelector	MAE: 0.371 MSE: 0.2088
time-series-forecasting-on-etth2-48-1	Transformer	MAE: 0.9488 MSE: 1.522
time-series-forecasting-on-etth2-48-1	QuerySelector	MAE: 0.9317 MSE: 1.4074
time-series-forecasting-on-etth2-48-2	QuerySelector	MAE: 0.2622 MSE: 0.1117
time-series-forecasting-on-etth2-48-2	Transformer	MAE: 0.2763 MSE: 0.1218
time-series-forecasting-on-etth2-720-1	QuerySelector	MAE: 1.3084 MSE: 3.0664
time-series-forecasting-on-etth2-720-1	Transformer	MAE: 1.3668 MSE: 3.1805
time-series-forecasting-on-etth2-720-2	Transformer	MAE: 0.434 MSE: 0.2853
time-series-forecasting-on-etth2-720-2	QuerySelector	MAE: 0.413 MSE: 0.2585

用 AI 构建 AI

从想法到上线——通过免费 AI 协同编程、开箱即用的环境和市场最优价格的 GPU 加速您的 AI 开发

AI 协同编程

即用型 GPU

最优价格

立即开始

Hyper Newsletters

订阅我们的最新资讯

我们会在北京时间 每周一的上午九点 向您的邮箱投递本周内的最新更新

邮件发送服务由 MailChimp 提供