If artificial intelligence is going to excel at driving cars or performing other complex tasks that we humans take for granted, then it needs to learn how to respond to unknown circumstances. That is the task of machine learning, which needs real-world examples to study.
So far, however, most data used to train machine-learning systems comes from virtual environments. A group of researchers, including a Microsoft Research scientist from the U.K., have set out to change that by using game replay data that can show an AI how humans tackle complex problems.
The researchers used Atari 2600 game replays to provide real-world data to a deep learning system that uses trial and error, or reinforcement learning (RL), to tackle new tasks in a previously unknown environment. The data used in the study represents what the researchers called the “largest and most diverse such data set” that has ever been publicly released.
The data was gathered by making a web-based Atari 2600 emulator, called the Atari Grand Challenge, available using the Javatari tool written in Javascript. The researchers used a form of gamified crowdsourcing, which leveraged people’s desire to play games in order to be helpful along with a reward mechanism that ranked each player’s performance.
Around 9.7 million frames or about 45 hours of gameplay time were collected and analyzed. Five games were used in creating the data based on their varying levels of difficulty and complexity: Video Pinball, Qbert, Space Invaders, Ms. Pac-Man, and Montezuma’s Revenge.
The results have been promising so far. By feeding information into the system like player actions taken during the games, in-game rewards, and current scores, the researchers were able to demonstrate the value of using this kind of data to train machine learning systems. Going forward, the researchers hope to use professional players to improve the data’s ability to train AI that is even better at responding to unknown situations.
