| A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | |||||||||||||||||||||||||
2 | |||||||||||||||||||||||||
3 | |||||||||||||||||||||||||
4 | Atari Learning Environment (via OpenAI gym) | ||||||||||||||||||||||||
5 | Used environments are the "NoFrameskip-v4" variants. | ||||||||||||||||||||||||
6 | |||||||||||||||||||||||||
7 | |||||||||||||||||||||||||
8 | Uniform Random Policy | Human | Ours @ 10M | google/ dopamine @ 10M | DQN (Mnih et al. 2015) @ 200M | google/ dopamine @ 200M | Rainbow Paper @ 200M | PFRL Rainbow @ 200M | better than google/dopamine @10M? | better than google/dopamine @200M? | better than rainbow paper @200M? | better than human? | Human normalized score | evaluation score 95% CI (for our results) | Notes | ||||||||||
9 | Alien | 227.8 | 6,875.4 | 15,053.8 | 1,250.0 | 3,069.3 | 3,500.0 | 9,491.7 | 10,255.4 | 1 | 1 | 1 | 1 | 223.0 | 634 | ||||||||||
10 | Amidar | 5.8 | 1,675.8 | 2,083.4 | 350.0 | 739.5 | 2,400.0 | 5,131.2 | 4,284.6 | 1 | 0 | 0 | 1 | 124.4 | |||||||||||
11 | Assault | 222.4 | 1,496.4 | 10,898.4 | 1,500.0 | 3,358.6 | 3,250.0 | 14,198.5 | 15,331.9 | 1 | 1 | 0 | 1 | 838.0 | 770.83 | ||||||||||
12 | Asterix | 210.0 | 8,503.3 | 164,942.0 | 3,000.0 | 6,011.6 | 18,000.0 | 428,200.3 | 550,307.6 | 1 | 1 | 0 | 1 | 1,986.3 | 28210.57 | ||||||||||
13 | Asteroids | 719.1 | 13,156.7 | 8,398.4 | 800.0 | 1,629.3 | 1,500.0 | 2,712.8 | 3,399.6 | 1 | 1 | 1 | 0 | 61.7 | 365.2121287 | ||||||||||
14 | Atlantis | 12,850.0 | 29,028.1 | 825,445.8 | 170,000.0 | 85,950.0 | 830,000.0 | 826,659.5 | 883,073.0 | 1 | 0 | 0 | 1 | 5,022.8 | 11049.46 | ||||||||||
15 | BankHeist | 14.2 | 734.4 | 1,124.9 | 900.0 | 429.7 | 1,150.0 | 1,358.0 | 1,272.7 | 1 | 0 | 0 | 1 | 154.2 | 4.33 | ||||||||||
16 | BattleZone | 2,360.0 | 37,800.0 | 67,806.5 | 22,500.0 | 26,300.0 | 40,000.0 | 62,010.0 | 202,382.5 | 1 | 1 | 1 | 1 | 184.7 | 2145 | ||||||||||
17 | BeamRider | 363.9 | 5,774.7 | 21,404.1 | 5,700.0 | 6,845.9 | 6,200.0 | 16,850.2 | 21,661.5 | 1 | 1 | 1 | 1 | 388.9 | 1618.18 | ||||||||||
18 | Berzerk | 123.7 | 2,630.4 | 1,801.3 | 480.0 | 830.0 | 2,545.6 | 6,018.1 | 1 | 1 | 0 | 0 | 66.9 | 46.26 | |||||||||||
19 | Bowling | 23.1 | 154.8 | 28.9 | 40.0 | 42.4 | 40.0 | 30.0 | 62.3 | 0 | 0 | 0 | 0 | 4.4 | 0.52 | ||||||||||
20 | Boxing | 0.1 | 4.3 | 99.5 | 77.0 | 71.8 | 98.0 | 99.6 | 99.9 | 1 | 1 | 0 | 1 | 2,366.0 | 0.04 | ||||||||||
21 | Breakout | 1.7 | 31.8 | 681.7 | 55.0 | 401.2 | 110.0 | 417.5 | 317.8 | 1 | 1 | 1 | 1 | 2,259.1 | 36.62030224 | ||||||||||
22 | Centipede | 2,090.9 | 11,963.2 | 8,847.6 | 5,000.0 | 8,309.4 | 6,500.0 | 8,167.3 | 7,546.8 | 1 | 1 | 1 | 0 | 68.4 | 185.19 | ||||||||||
23 | ChopperCommand | 811.0 | 9,881.8 | 2,922.5 | 2,500.0 | 6,686.7 | 11,500.0 | 16,654.0 | 21,014.5 | 1 | 0 | 0 | 0 | 23.3 | 371.1 | ||||||||||
24 | CrazyClimber | 10,780.5 | 35,410.5 | 132,129.2 | 110,000.0 | 114,103.3 | 145,000.0 | 168,788.5 | 174,025.0 | 1 | 0 | 0 | 1 | 492.7 | 2413.43 | ||||||||||
25 | DemonAttack | 152.1 | 3,401.3 | 110,345.6 | 3,000.0 | 9,711.2 | 16,000.0 | 111,185.2 | 100,980.6 | 1 | 1 | 0 | 1 | 3,391.4 | 11472.31 | ||||||||||
26 | DoubleDunk | -18.6 | -15.5 | -1.0 | -18.0 | -18.1 | 22.0 | -0.3 | -0.1 | 1 | 0 | 0 | 1 | 567.7 | 0.36 | ||||||||||
27 | Enduro | 0.0 | 309.6 | 2,216.6 | 1,700.0 | 301.8 | 2,200.0 | 2,125.9 | 2,281.9 | 1 | 1 | 1 | 1 | 715.9 | 63.16 | ||||||||||
28 | FishingDerby | -91.7 | 5.5 | 33.6 | 20.0 | -0.8 | 42.0 | 31.3 | 39.1 | 1 | 0 | 1 | 1 | 128.9 | 0.55 | ||||||||||
29 | Freeway | 0.0 | 29.6 | 33.0 | 32.0 | 30.3 | 33.0 | 34.0 | 33.6 | 1 | 0 | 0 | 1 | 111.5 | 0 | ||||||||||
30 | Frostbite | 65.2 | 4,334.7 | 4,983.4 | 2,700.0 | 328.3 | 8,400.0 | 9,590.5 | 11,046.3 | 1 | 0 | 0 | 1 | 115.2 | 76.94 | ||||||||||
31 | Gravitar | 173.0 | 2,672.0 | 2,847.3 | 210.0 | 306.7 | 1,250.0 | 1,419.3 | 1,387.2 | 1 | 1 | 1 | 1 | 107.0 | 23.88 | ||||||||||
32 | Hero | 1,027.0 | 25,762.5 | 19,903.0 | 14,000.0 | 19,950.3 | 47,000.0 | 55,887.4 | 34,234.5 | 1 | 0 | 0 | 0 | 76.3 | 40.65 | ||||||||||
33 | IceHockey | -11.2 | 0.9 | 5.5 | -5.0 | -1.6 | 2.0 | 1.1 | 6.6 | 1 | 1 | 1 | 1 | 137.9 | |||||||||||
34 | Kangaroo | 52.0 | 3,035.0 | 14,280.0 | 6,000.0 | 6,740.0 | 13,000.0 | 14,637.5 | 13,726.5 | 1 | 1 | 0 | 1 | 477.0 | 192.676021 | ||||||||||
35 | Krull | 1,598.0 | 2,394.6 | 9,143.3 | 6,100.0 | 3,804.7 | 6,100.0 | 8,741.5 | 7,844.4 | 1 | 1 | 1 | 1 | 947.2 | 106.35 | ||||||||||
36 | KungFuMaster | 258.5 | 22,736.2 | 22,552.4 | 19,000.0 | 23,270.0 | 27,000.0 | 52,181.0 | 54,835.5 | 1 | 0 | 0 | 0 | 99.2 | 658.26 | ||||||||||
37 | MsPacman | 307.3 | 15,693.4 | 5,322.8 | 2,400.0 | 2,311.0 | 4,000.0 | 5,380.4 | 5,277.3 | 1 | 1 | 0 | 0 | 32.6 | 117.34 | ||||||||||
38 | NameThisGame | 2,292.3 | 4,076.2 | 19,562.5 | 9,000.0 | 7,256.7 | 9,000.0 | 13,136.0 | 14,679.0 | 1 | 1 | 1 | 1 | 968.1 | 536.33 | ||||||||||
39 | Phoenix | 761.4 | 7,242.6 | 24,866.1 | 5,000.0 | 8,200.0 | 108,528.6 | 147,467.9 | 1 | 1 | 0 | 1 | 371.9 | 1555.24 | |||||||||||
40 | Pitfall | -229.4 | 6,463.7 | -1.6 | -3.0 | -3.0 | 0.0 | -3.4 | 1 | 1 | 0 | 0 | 3.4 | 0.19 | |||||||||||
41 | Pong | -20.7 | 9.3 | 21.0 | 16.0 | 18.9 | 20.5 | 20.9 | 21.0 | 1 | 1 | 1 | 1 | 139.0 | 0.01 | ||||||||||
42 | PrivateEye | 24.9 | 69,571.3 | 360.6 | 0.0 | 1,787.6 | 22,000.0 | 4,234.0 | 101.7 | 1 | 0 | 0 | 0 | 0.5 | 175.79 | ||||||||||
43 | Qbert | 163.9 | 13,455.0 | 15,529.9 | 7,000.0 | 10,595.8 | 17,500.0 | 33,817.5 | 42,518.7 | 1 | 0 | 0 | 1 | 115.6 | 87.47 | *incomplete run, only 3.6M frames | |||||||||
44 | RoadRunner | 11.5 | 7,845.0 | 57,881.1 | 35,000.0 | 18,256.7 | 55,000.0 | 62,041.0 | 67,638.5 | 1 | 1 | 0 | 1 | 738.7 | *incomplete run, only 9M frames | ||||||||||
45 | Robotank | 2.2 | 11.9 | 36.7 | 22.5 | 51.6 | 65.0 | 61.4 | 74.0 | 1 | 0 | 0 | 1 | 355.6 | 1.07 | *incomplete run, only 3.6M frames | |||||||||
46 | Seaquest | 68.4 | 20,181.8 | 53,149.5 | 2,000.0 | 5,286.0 | 10,000.0 | 15,898.9 | 5,277.4 | 1 | 1 | 1 | 1 | 263.9 | 4576.54 | ||||||||||
47 | Skiing | -17,098.1 | -4,336.9 | -16,516.1 | -22,000.0 | -22,000.0 | -12,957.8 | -29,974.7 | 1 | 1 | 0 | 0 | 4.6 | 113.64 | |||||||||||
48 | Solaris | 1,236.3 | 12,326.7 | 2,510.2 | 1,400.0 | 2,200.0 | 3,560.3 | 6,730.1 | 1 | 1 | 0 | 0 | 11.5 | 170.72 | |||||||||||
49 | SpaceInvaders | 148.0 | 1,652.3 | 17,708.9 | 800.0 | 1,975.5 | 4,000.0 | 18,789.0 | 2,823.1 | 1 | 1 | 0 | 1 | 1,167.4 | 2007.81 | ||||||||||
50 | StarGunner | 664.0 | 10,250.0 | 327,543.0 | 2,000.0 | 57,996.7 | 58,000.0 | 127,029.0 | 155,248.8 | 1 | 1 | 1 | 1 | 3,410.0 | 13594.58 | ||||||||||
51 | Tennis | -23.8 | -8.9 | 23.8 | -3.0 | -2.5 | 0.0 | 0.0 | -0.1 | 1 | 1 | 1 | 1 | 319.5 | 0.03 | ||||||||||
52 | TimePilot | 3,568.0 | 5,925.0 | 31,407.9 | 3,000.0 | 5,946.7 | 12,000.0 | 12,926.0 | 24,038.2 | 1 | 1 | 1 | 1 | 1,181.2 | 1106.38 | ||||||||||
53 | Tutankham | 11.4 | 167.6 | 103.8 | 200.0 | 186.7 | 240.0 | 241.0 | 258.0 | 0 | 0 | 0 | 0 | 59.2 | 0.24 | ||||||||||
54 | Venture | 0.0 | 1,187.5 | 52.9 | 0.0 | 380.0 | 1,550.0 | 5.5 | 2.5 | 1 | 0 | 1 | 0 | 4.5 | 14.32 | *incomplete run, only 7.2M frames | |||||||||
55 | VideoPinball | 16,256.9 | 17,297.6 | 202,204.1 | 40,000.0 | 42,684.1 | 450,000.0 | 533,936.5 | 292,835.7 | 1 | 0 | 0 | 1 | 17,867.5 | 27680.36611 | ||||||||||
56 | WizardOfWor | 563.5 | 4,756.5 | 18,406.7 | 2,800.0 | 3,393.3 | 7,500.0 | 17,862.5 | 21,341.2 | 1 | 1 | 1 | 1 | 425.5 | 838.82 | ||||||||||
57 | YarsRevenge | 3,092.9 | 54,576.9 | 102,667.8 | 10,000.0 | 45,000.0 | 102,557.0 | 93,877.2 | 1 | 1 | 1 | 1 | 193.4 | 1364.55 | |||||||||||
58 | Zaxxon | 32.5 | 9,173.3 | 17,187.2 | 4,800.0 | 4,976.7 | 14,500.0 | 22,209.5 | 25,084.0 | 1 | 1 | 0 | 1 | 187.7 | 251.21 | ||||||||||
59 | |||||||||||||||||||||||||
60 | random and human scores from Mnih et al. (2015) where available, otherwise from Badia, et al. (2020) | ||||||||||||||||||||||||
61 | 96% | 64% | 40% | 72% | mean HNS = | 979.2 | |||||||||||||||||||
62 | median HNS = | 190.5 | |||||||||||||||||||||||
63 | |||||||||||||||||||||||||
64 | |||||||||||||||||||||||||
65 | |||||||||||||||||||||||||
66 | |||||||||||||||||||||||||
67 | procgen (distribution_mode=hard), incomplete results due to time constraints | ||||||||||||||||||||||||
68 | |||||||||||||||||||||||||
69 | |||||||||||||||||||||||||
70 | Uniform Random Policy | Ours @ 10M | CleanRL PPO @ 100M | Procgen Paper (approx) @ 200M | |||||||||||||||||||||
71 | bigfish | 0.19 | 0.40 | 21.00 | |||||||||||||||||||||
72 | bossfight | 0.03 | 0.44 | 7.78 | 10.50 | ||||||||||||||||||||
73 | caveflyer | 1.57 | 2.80 | 4.54 | 10.00 | ||||||||||||||||||||
74 | chaser | 0.64 | 4.58 | 2.53 | 8.00 | ||||||||||||||||||||
75 | climber | 1.28 | 3.82 | 10.00 | |||||||||||||||||||||
76 | coinrun | 2.55 | 6.00 | 7.78 | 8.25 | ||||||||||||||||||||
77 | dodgeball | 0.63 | 1.71 | 10.00 | |||||||||||||||||||||
78 | fruitbot | 0.70 | 0.41 | 24.00 | |||||||||||||||||||||
79 | heist | 1.90 | 2.30 | 9.00 | |||||||||||||||||||||
80 | jumper | 0.95 | 1.94 | 5.08 | 7.00 | ||||||||||||||||||||
81 | leaper | 0.88 | 2.02 | 9.00 | |||||||||||||||||||||
82 | maze | 2.72 | 3.37 | 10.00 | |||||||||||||||||||||
83 | miner | 1.13 | 1.23 | 1.96 | 16.00 | ||||||||||||||||||||
84 | ninja | 1.15 | 3.97 | 9.50 | |||||||||||||||||||||
85 | plunder | 2.95 | 5.73 | 15.00 | |||||||||||||||||||||
86 | starpilot | 0.87 | 9.55 | 23.70 | 20.00 | ||||||||||||||||||||
87 | |||||||||||||||||||||||||
88 | |||||||||||||||||||||||||
89 | |||||||||||||||||||||||||
90 | |||||||||||||||||||||||||
91 | |||||||||||||||||||||||||
92 | |||||||||||||||||||||||||
93 | |||||||||||||||||||||||||
94 | |||||||||||||||||||||||||
95 | |||||||||||||||||||||||||
96 | |||||||||||||||||||||||||
97 | |||||||||||||||||||||||||
98 | |||||||||||||||||||||||||
99 | |||||||||||||||||||||||||
100 |