3 个月前

基于线性专家混合的长期时间序列预测

基于线性专家混合的长期时间序列预测

摘要

长期时间序列预测(Long-term Time Series Forecasting, LTSF)旨在根据历史观测值预测未来的时间序列值。目前,该任务的最先进(State-of-the-Art, SOTA)方法在某些情况下由以线性映射为核心的模型实现,这类模型通常包含一个线性映射层。然而,由于其固有的结构简单性,这些模型难以适应时间序列模式中的周期性变化,无法动态调整其预测规则。为应对这一挑战,我们提出了一种面向线性中心模型的“专家混合”(Mixture-of-Experts)式增强方法,并进一步构建了线性专家混合模型(Mixture-of-Linear-Experts, MoLE)。与训练单一模型不同,MoLE同时训练多个线性中心模型(即“专家”)以及一个路由模型(router),后者负责对各专家的输出进行加权并融合。尽管整个框架采用端到端的方式进行训练,但每个专家能够专注于学习特定的时间模式,而路由模型则具备自适应地组合专家输出的能力。实验结果表明,在我们评估的超过78%的数据集和设置中,MoLE显著降低了现有线性中心模型(包括DLinear、RLinear和RMLP)的预测误差。在与PatchTST进行对比的68%的实验场景中,通过引入MoLE,原有线性中心模型能够达到SOTA的长期时间序列预测性能;相比之下,传统的单头线性中心模型仅在25%的情况下达到SOTA水平。

代码仓库

rogerni/mole
官方
pytorch
GitHub 中提及

基准测试

基准方法指标
time-series-forecasting-on-electricity-192MoLE-DLinear
MSE: 0.147
time-series-forecasting-on-electricity-336MoLE-DLinear
MSE: 0.162
time-series-forecasting-on-electricity-720MoLE-RMLP
MSE: 0.178
time-series-forecasting-on-electricity-720MoLE-DLinear
MSE: 0.18
time-series-forecasting-on-electricity-96MoLE-DLinear
MSE: 0.131
time-series-forecasting-on-electricity-96MoLE-RMLP
MSE: 0.129
time-series-forecasting-on-etth1-192-1MoLE-DLinear
MSE: 0.453
time-series-forecasting-on-etth1-192-1MoLE-RLinear
MSE: 0.403
time-series-forecasting-on-etth1-336-1MoLE-DLinear
MSE: 0.469
time-series-forecasting-on-etth1-336-1MoLE-RLinear
MSE: 0.43
time-series-forecasting-on-etth1-720-1MoLE-DLinear
MSE: 0.505
time-series-forecasting-on-etth1-720-1MoLE-RLinear
MSE: 0.449
time-series-forecasting-on-etth1-96-1MoLE-DLinear
MSE: 0.377
time-series-forecasting-on-etth1-96-1MoLE-RLinear
MSE: 0.375
time-series-forecasting-on-etth2-192-1MoLE-RLinear
MSE: 0.336
time-series-forecasting-on-etth2-192-1MoLE-DLinear
MSE: 0.362
time-series-forecasting-on-etth2-336-1MoLE-RLinear
MSE: 0.371
time-series-forecasting-on-etth2-336-1MoLE-DLinear
MSE: 0.419
time-series-forecasting-on-etth2-720-1MoLE-DLinear
MSE: 0.605
time-series-forecasting-on-etth2-720-1MoLE-RLinear
MSE: 0.409
time-series-forecasting-on-etth2-96-1MoLE-RLinear
MSE: 0.273
time-series-forecasting-on-etth2-96-1MoLE-DLinear
MSE: 0.287
time-series-forecasting-on-ettm1-192-1MoLE-DLinear
MSE: 0.328
time-series-forecasting-on-ettm1-336-1MoLE-DLinear
MSE: 0.38
time-series-forecasting-on-ettm1-720-1MoLE-DLinear
MSE: 0.447
time-series-forecasting-on-ettm1-96-1MoLE-DLinear
MSE: 0.286
time-series-forecasting-on-ettm2-192-1MoLE-DLinear
MSE: 0.233
time-series-forecasting-on-ettm2-336-1MoLE-DLinear
MSE: 0.289
time-series-forecasting-on-ettm2-720-1MoLE-DLinear
MSE: 0.399
time-series-forecasting-on-ettm2-96-1MoLE-DLinear
MSE: 0.168
time-series-forecasting-on-weather-192MoLE-DLinear
MSE: 0.203
time-series-forecasting-on-weather-192MoLE-RMLP
MSE: 0.19
time-series-forecasting-on-weather-336MoLE-DLinear
MSE: 0.238
time-series-forecasting-on-weather-720MoLE-DLinear
MSE: 0.314
time-series-forecasting-on-weather-96MoLE-DLinear
MSE: 0.147
time-series-forecasting-on-weather2k114-192MoLE-DLinear
MSE: 0.405
time-series-forecasting-on-weather2k114-336MoLE-DLinear
MSE: 0.415
time-series-forecasting-on-weather2k114-720MoLE-DLinear
MSE: 0.425
time-series-forecasting-on-weather2k114-96MoLE-DLinear
MSE: 0.391
time-series-forecasting-on-weather2k1786-192MoLE-RLinear
MSE: 0.581
time-series-forecasting-on-weather2k1786-192MoLE-DLinear
MSE: 0.601
time-series-forecasting-on-weather2k1786-336MoLE-DLinear
MSE: 0.603
time-series-forecasting-on-weather2k1786-720MoLE-RLinear
MSE: 0.628
time-series-forecasting-on-weather2k1786-720MoLE-DLinear
MSE: 0.66
time-series-forecasting-on-weather2k1786-96MoLE-RLinear
MSE: 0.535
time-series-forecasting-on-weather2k1786-96MoLE-DLinear
MSE: 0.535
time-series-forecasting-on-weather2k79-192MoLE-DLinear
MSE: 0.566
time-series-forecasting-on-weather2k79-336MoLE-DLinear
MSE: 0.546
time-series-forecasting-on-weather2k79-720MoLE-DLinear
MSE: 0.535
time-series-forecasting-on-weather2k79-96MoLE-DLinear
MSE: 0.555
time-series-forecasting-on-weather2k850-192MoLE-DLinear
MSE: 0.484
time-series-forecasting-on-weather2k850-336MoLE-DLinear
MSE: 0.474
time-series-forecasting-on-weather2k850-720MoLE-DLinear
MSE: 0.461
time-series-forecasting-on-weather2k850-96MoLE-RLinear
MSE: 0.471
time-series-forecasting-on-weather2k850-96MoLE-DLinear
MSE: 0.474

用 AI 构建 AI

从想法到上线——通过免费 AI 协同编程、开箱即用的环境和市场最优价格的 GPU 加速您的 AI 开发

AI 协同编程
即用型 GPU
最优价格
立即开始

Hyper Newsletters

订阅我们的最新资讯
我们会在北京时间 每周一的上午九点 向您的邮箱投递本周内的最新更新
邮件发送服务由 MailChimp 提供
基于线性专家混合的长期时间序列预测 | 论文 | HyperAI超神经