Download PDFOpen PDF in browserCurrent version

A Review for Deep Reinforcement Learning in Atari: Benchmarks, Challenges and Solutions

EasyChair Preprint 6985, version 1

Versions: 12history
25 pagesDate: November 3, 2021

Abstract

The Arcade Learning Environment (ALE) is proposed as an evaluation platform for empirically assessing the generality of agents across dozens of Atari 2600 games. ALE offers various challenging problems and has drawn significant attention from the deep reinforcement learning (RL) community. From Deep Q-Networks (DQN) to Agent57, RL agents seem to achieve superhuman performance in ALE. However, is this the case? In this paper, to explore this problem, we first review the current evaluation metrics in the Atari benchmarks and then reveal that the current evaluation criteria of achieving superhuman performance are inappropriate, which underestimated the human performance relative to what is possible. To handle those problems and promote the development of RL research, we propose a novel Atari benchmark based on human world records (HWR), which puts forward higher requirements for RL agents on both final performance and learning efficiency.
Furthermore, we summarize the state-of-the-art (SOTA) methods in Atari benchmarks and provide benchmark results over new evaluation metrics based on human world records. We concluded that at least four open challenges hinder RL agents from achieving superhuman performance from those new benchmark results. Finally, we also discuss some promising ways to handle those problems.

Keyphrases: Human World Records Benchmark, Reinforcement Learning, Superhuman Agents, The Arcade Learning Environment

BibTeX entry
BibTeX does not have the right entry for preprints. This is a hack for producing the correct reference:
@booklet{EasyChair:6985,
  author    = {Jiajun Fan},
  title     = {A Review for Deep Reinforcement Learning in Atari: Benchmarks, Challenges and Solutions},
  howpublished = {EasyChair Preprint 6985},
  year      = {EasyChair, 2021}}
Download PDFOpen PDF in browserCurrent version