Index of /besson/publis/gym-nes-mario-bros.git

[ICO]NameLast modifiedSizeDescription
[PARENTDIR]Parent Directory  - Root of the website (in English or in French)
[DIR]src/2018-07-06 18:04 -  
[DIR]images/2018-03-22 15:44 -  
[   ]_config.yml2018-03-22 17:23 25  
[TXT]README.md2018-03-22 15:47 5.1KSome details about this page
[   ]Makefile2018-03-05 15:25 208  
[   ]LICENSE2018-02-24 14:58 1.1K 
# OpenAI Gym for NES games + DQN with Keras to learn Mario Bros. from raw pixels

- An **EXPERIMENTAL** [openai-gym]( wrapper for NES games.
- With a Double Deep Q Network to learn how to play **Mario Bros.** game from 1983.

# Installation
> You can use a [`virtualenv`]( or a [`pipenv`]( if you want to install the dependencies in an isolated environment.

0. Use Python 3 only.
1. Install [openai-gym]( and [keras]( with [tensorflow]( backend (with [`pip`](, and `cv2` (OpenCV module, on Debian/Ubuntu, `sudo pip install opencv-python`, see [this SO question](
2. Install the [`fceux` NES emulator]( and make sure `fceux` is in your `$PATH`. In Debian/Ubuntu, you can simple use `sudo apt install fceux`. Version 2 at least is needed.
3. Find a `.nes` ROM for [Mario]( [Bros.]( game (any dump for the Nintendo NES will do). Save it to [`src/roms/mario_bros.nes`](src/roms/).
4. Copy state files from `roms/fcs/*` to your `~/.fceux/fcs/` (faster loading for the beginning of the game).

# Example usage
For instance to load the Mario Bros. environment:

# import nesgym to register environments to gym
import nesgym
env = gym.make('nesgym/MarioBros-v0')
obs = env.reset()

for step in range(10000):
    action = env.action_space.sample()
    obs, reward, done, info = env.step(action)
    ... # your awesome reinforcement learning algorithm is here

# Examples for training dqn
An implementation of dqn is in [`src/dqn`](src/dqn), using [keras](

You can train dqn model for Atari with [``](src/ and for NES with [``](src/ or [``](src/

# Integrating new NES games?
You need to write two files:

1. a [lua interface]( file,
2. and an [openai gym environment]( class (python) file.

The lua file needs to get the reward from emulator (typically extracting from a memory location), and the python file defines the game specific environment.

For an example of lua file, see [`src/lua/soccer.lua`](src/lua/soccer.lua); for an example of gym env file, see [`src/nesgym/`](src/nesgym/

> [This website]( gives RAM mapping for the most well-known NES games, this is very useful to extract easily the score or lives directly from the NES RAM memory, to use it as a reward for the reinforcement learning loop. See for instance [for Mario Bros.](

# Gallery
## Training Atari games

## Training NES games
### Mario Bros. game
> *That's new!*

Architecture of the DQN playning mario:


Overview of the experimentation with 3 emulators:


### Soccer game


### :scroll: License ? [![GitHub license](](
This (small) repository is published under the terms of the [MIT license]( (file [LICENSE](LICENSE)).
© [Lilian Besson](, 2018.

[![Ask Me Anything !](](


[![ForTheBadge built-with-swag](](
[![ForTheBadge uses-badges](](
[![ForTheBadge uses-git](](