HyperAIHyperAI

Command Palette

Search for a command to run...

4 months ago

Prioritized Experience Replay

Tom Schaul; John Quan; Ioannis Antonoglou; David Silver

Prioritized Experience Replay

Abstract

Experience replay lets online reinforcement learning agents remember and reuse experiences from the past. In prior work, experience transitions were uniformly sampled from a replay memory. However, this approach simply replays transitions at the same frequency that they were originally experienced, regardless of their significance. In this paper we develop a framework for prioritizing experience, so as to replay important transitions more frequently, and therefore learn more efficiently. We use prioritized experience replay in Deep Q-Networks (DQN), a reinforcement learning algorithm that achieved human-level performance across many Atari games. DQN with prioritized experience replay achieves a new state-of-the-art, outperforming DQN with uniform replay on 41 out of 49 games.

Code Repositories

snhwang/p3_collab-compet
pytorch
Mentioned in GitHub
MathPhysSim/PER-NAF
tf
Mentioned in GitHub
nbopardi/smb
tf
Mentioned in GitHub
CharlotteMorrison/Baxter-Research
pytorch
Mentioned in GitHub
VictorZuanazzi/Project_RL
pytorch
Mentioned in GitHub
snhwang/p1_navigation_SNH
pytorch
Mentioned in GitHub
CSCI4850/S20-team3-project
Mentioned in GitHub
VasaKiDD/TD3-deep-rl-research
pytorch
Mentioned in GitHub
Howuhh/prioritized_experience_replay
pytorch
Mentioned in GitHub
JuliaPOMDP/DeepQLearning.jl
Mentioned in GitHub
utarumo/RL_implementation
tf
Mentioned in GitHub
snhwang/p2-continuous-control-SNH
pytorch
Mentioned in GitHub
kayuksel/pytorch-ars
pytorch
Mentioned in GitHub
iDataist/Navigation-with-Deep-Q-Network
pytorch
Mentioned in GitHub
guillaumeboniface/bananaland
pytorch
Mentioned in GitHub
Clement-Hui/Q-Learning
pytorch
Mentioned in GitHub
chainer/chainerrl
pytorch
Mentioned in GitHub
justinmaojones/starr
Mentioned in GitHub
dtak/hip-mdp-public
tf
Mentioned in GitHub
xinjinghao/sparrow-v1
pytorch
Mentioned in GitHub
olonok69/Udacity_Banana_Unity
pytorch
Mentioned in GitHub
1jsingh/rl_navigation
pytorch
Mentioned in GitHub
ACampero/dopamine
tf
Mentioned in GitHub
tensorlayer/RLzoo
tf
Mentioned in GitHub
ku2482/soft-actor-critic.pytorch
pytorch
Mentioned in GitHub
eddynelson/dqn
tf
Mentioned in GitHub
rybread1/deep-rl-trex
tf
Mentioned in GitHub
CharlotteMorrison/Baxter-VREP
pytorch
Mentioned in GitHub
seacevedo/ReinforcementLearningProjects
pytorch
Mentioned in GitHub
tphanson/xupr-drl
tf
Mentioned in GitHub
Adrelf/DRL-navigation
pytorch
Mentioned in GitHub
MEOWMEOW114/nd893-p1-navigation-banana
pytorch
Mentioned in GitHub
yusme/DDPG
tf
Mentioned in GitHub
KatyNTsachi/Hierarchical-RL
tf
Mentioned in GitHub
mindspore-courses/Rainbow-MindSpore
mindspore
Mentioned in GitHub
kmdanielduan/DQN_Family_PyTorch
pytorch
Mentioned in GitHub
xusophia/DataSciFinalProj
pytorch
Mentioned in GitHub
rybread1/DeepRlTrex
tf
Mentioned in GitHub
sunfex/weighted-sac
pytorch
Mentioned in GitHub
Arrabonae/openai_DDDQN
pytorch
Mentioned in GitHub
instadeepai/flashbax
jax
Mentioned in GitHub
CharlotteMorrison/Baxter-VREP-Version-2
pytorch
Mentioned in GitHub
anhtu293/NeurIPS-2019-Challenge
tf
Mentioned in GitHub
yzheng51/rl-dino-run
pytorch
Mentioned in GitHub
SimonRamstedt/ddpg
tf
Mentioned in GitHub
HussonnoisMaxence/RL_Algorithms
pytorch
Mentioned in GitHub
NervanaSystems/coach
tf
Mentioned in GitHub
Suryavf/SelfDrivingCar
pytorch
Mentioned in GitHub
toshikwa/soft-actor-critic.pytorch
pytorch
Mentioned in GitHub
shashwatsaxena571/DRL-navigation
pytorch
Mentioned in GitHub
Guillaume-Cr/lunar_lander_per
pytorch
Mentioned in GitHub
austinsilveria/Banana-Collection-DQN
pytorch
Mentioned in GitHub
ku2482/sac-discrete.pytorch
pytorch
Mentioned in GitHub
Damcy/prioritized-experience-replay
pytorch
Mentioned in GitHub
Brandon-Rozek/DeepRL
Mentioned in GitHub
V0LsTeR/DQN_heap
tf
Mentioned in GitHub
SayhoKim/tetrisRL
tf
Mentioned in GitHub

Benchmarks

BenchmarkMethodologyMetrics
atari-games-on-atari-2600-alienPrior noop
Score: 4203.8
atari-games-on-atari-2600-alienPrior hs
Score: 1334.7
atari-games-on-atari-2600-amidarPrior hs
Score: 129.1
atari-games-on-atari-2600-amidarPrior noop
Score: 1838.9
atari-games-on-atari-2600-assaultPrior noop
Score: 7672.1
atari-games-on-atari-2600-assaultPrior hs
Score: 6548.9
atari-games-on-atari-2600-asterixPrior hs
Score: 22484.5
atari-games-on-atari-2600-asterixPrior noop
Score: 31527
atari-games-on-atari-2600-asteroidsPrior noop
Score: 2654.3
atari-games-on-atari-2600-asteroidsPrior hs
Score: 1745.1
atari-games-on-atari-2600-atlantisPrior noop
Score: 357324.0
atari-games-on-atari-2600-atlantisPrior hs
Score: 330647.0
atari-games-on-atari-2600-bank-heistPrior hs
Score: 876.6
atari-games-on-atari-2600-bank-heistPrior noop
Score: 1054.6
atari-games-on-atari-2600-battle-zonePrior hs
Score: 25520.0
atari-games-on-atari-2600-battle-zonePrior noop
Score: 31530.0
atari-games-on-atari-2600-beam-riderPrior noop
Score: 23384.2
atari-games-on-atari-2600-beam-riderPrior hs
Score: 31181.3
atari-games-on-atari-2600-berzerkPrior noop
Score: 1305.6
atari-games-on-atari-2600-berzerkPrior hs
Score: 865.9
atari-games-on-atari-2600-bowlingPrior hs
Score: 52
atari-games-on-atari-2600-bowlingPrior noop
Score: 47.9
atari-games-on-atari-2600-boxingPrior noop
Score: 95.6
atari-games-on-atari-2600-boxingPrior hs
Score: 72.3
atari-games-on-atari-2600-breakoutPrior noop
Score: 373.9
atari-games-on-atari-2600-breakoutPrior hs
Score: 343.0
atari-games-on-atari-2600-centipedePrior noop
Score: 4463.2
atari-games-on-atari-2600-centipedePrior hs
Score: 3489.1
atari-games-on-atari-2600-chopper-commandPrior hs
Score: 4635.0
atari-games-on-atari-2600-chopper-commandPrior noop
Score: 8600.0
atari-games-on-atari-2600-crazy-climberPrior noop
Score: 141161.0
atari-games-on-atari-2600-crazy-climberPrior hs
Score: 127512.0
atari-games-on-atari-2600-demon-attackPrior noop
Score: 71846.4
atari-games-on-atari-2600-demon-attackPrior hs
Score: 61277.5
atari-games-on-atari-2600-double-dunkPrior noop
Score: 18.5
atari-games-on-atari-2600-double-dunkPrior hs
Score: 16.0
atari-games-on-atari-2600-enduroPrior noop
Score: 2093.0
atari-games-on-atari-2600-enduroPrior hs
Score: 1831.0
atari-games-on-atari-2600-fishing-derbyPrior hs
Score: 9.8
atari-games-on-atari-2600-fishing-derbyPrior noop
Score: 39.5
atari-games-on-atari-2600-freewayPrior hs
Score: 28.9
atari-games-on-atari-2600-freewayPrior noop
Score: 33.7
atari-games-on-atari-2600-frostbitePrior hs
Score: 3510.0
atari-games-on-atari-2600-frostbitePrior noop
Score: 4380.1
atari-games-on-atari-2600-gopherPrior hs
Score: 34858.8
atari-games-on-atari-2600-gopherPrior noop
Score: 32487.2
atari-games-on-atari-2600-gravitarPrior noop
Score: 548.5
atari-games-on-atari-2600-gravitarPrior hs
Score: 269.5
atari-games-on-atari-2600-heroPrior hs
Score: 20889.9
atari-games-on-atari-2600-heroPrior noop
Score: 23037.7
atari-games-on-atari-2600-ice-hockeyPrior hs
Score: -0.2
atari-games-on-atari-2600-ice-hockeyPrior noop
Score: 1.3
atari-games-on-atari-2600-james-bondPrior hs
Score: 3961.0
atari-games-on-atari-2600-james-bondPrior noop
Score: 5148.0
atari-games-on-atari-2600-kangarooPrior noop
Score: 16200.0
atari-games-on-atari-2600-kangarooPrior hs
Score: 12185.0
atari-games-on-atari-2600-krullPrior hs
Score: 6872.8
atari-games-on-atari-2600-krullPrior noop
Score: 9728.0
atari-games-on-atari-2600-kung-fu-masterPrior noop
Score: 39581.0
atari-games-on-atari-2600-kung-fu-masterPrior hs
Score: 31676.0
atari-games-on-atari-2600-montezumas-revengePrior hs
Score: 51
atari-games-on-atari-2600-ms-pacmanPrior noop
Score: 6518.7
atari-games-on-atari-2600-ms-pacmanPrior hs
Score: 1865.9
atari-games-on-atari-2600-name-this-gamePrior hs
Score: 10497.6
atari-games-on-atari-2600-name-this-gamePrior noop
Score: 12270.5
atari-games-on-atari-2600-pongPrior hs
Score: 18.9
atari-games-on-atari-2600-pongPrior noop
Score: 20.6
atari-games-on-atari-2600-private-eyePrior hs
Score: 670.7
atari-games-on-atari-2600-private-eyePrior noop
Score: 200.0
atari-games-on-atari-2600-qbertPrior noop
Score: 16256.5
atari-games-on-atari-2600-qbertPrior hs
Score: 9944
atari-games-on-atari-2600-river-raidPrior hs
Score: 11807.2
atari-games-on-atari-2600-river-raidPrior noop
Score: 14522.3
atari-games-on-atari-2600-road-runnerPrior hs
Score: 52264.0
atari-games-on-atari-2600-road-runnerPrior noop
Score: 57608.0
atari-games-on-atari-2600-robotankPrior noop
Score: 62.6
atari-games-on-atari-2600-robotankPrior hs
Score: 56.2
atari-games-on-atari-2600-seaquestPrior noop
Score: 26357.8
atari-games-on-atari-2600-seaquestPrior hs
Score: 25463.7
atari-games-on-atari-2600-space-invadersPrior noop
Score: 2865.8
atari-games-on-atari-2600-space-invadersPrior hs
Score: 3912.1
atari-games-on-atari-2600-star-gunnerPrior noop
Score: 63302.0
atari-games-on-atari-2600-star-gunnerPrior hs
Score: 61582.0
atari-games-on-atari-2600-tennisPrior hs
Score: -5.3
atari-games-on-atari-2600-tennisPrior noop
Score: 0.0
atari-games-on-atari-2600-time-pilotPrior hs
Score: 5963.0
atari-games-on-atari-2600-time-pilotPrior noop
Score: 9197.0
atari-games-on-atari-2600-tutankhamPrior noop
Score: 204.6
atari-games-on-atari-2600-tutankhamPrior hs
Score: 56.9
atari-games-on-atari-2600-up-and-downPrior noop
Score: 16154.1
atari-games-on-atari-2600-up-and-downPrior hs
Score: 12157.4
atari-games-on-atari-2600-venturePrior hs
Score: 94.0
atari-games-on-atari-2600-venturePrior noop
Score: 54.0
atari-games-on-atari-2600-video-pinballPrior hs
Score: 295972.8
atari-games-on-atari-2600-video-pinballPrior noop
Score: 282007.3
atari-games-on-atari-2600-wizard-of-worPrior noop
Score: 4802.0
atari-games-on-atari-2600-wizard-of-worPrior hs
Score: 5727.0
atari-games-on-atari-2600-zaxxonPrior hs
Score: 9474.0
atari-games-on-atari-2600-zaxxonPrior noop
Score: 10469.0

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp
Prioritized Experience Replay | Papers | HyperAI