Welcome to NGS, an augmented version of the repository originally developed by tkn-tub.
In this augmented version we have managed to additionally integrate the SUMO traffic simulator. Hence all SUMO traffic scenarios are also supported.
Additionally, we performed all of our reinforcement learning using the agents present in the stable-baselines repository. Which is a repository with high-quality implementations of well known Deep RL algorithms.
Continuing to scroll down, you will find the three different subsections that will explains the required dependencies for:
- SUMO
- Stable Baselines
- ns3-gym
Note: You can go through the steps below for a more hands on installation process or you can run the installation helper script by entering the following command in the terminal:
./install_depThe above command must be run the working directory of ns3-gym.
# The following two commands are needed only
# if this is the first time running
# ------------------------------------------
./waf configure --enable-examples
./waf build
# ------------------------------------------
# 1 = circle traffic scenario
# 2 = two lane highway merge scenario
# 3 = square traffic scenario
./launch_sumo scenario=[ 1 | 2 | 3 ]You must have two terminal windows open to perform the following:
Terminal 1 (ns3 side):
# This should be run the main working directory
./launch_sumo scenario=[ 1 | 2 | 3 ]Terminal 2 (gym side):
# This should be run under rl_fyp/gym_rsu
python3 script.py train online [ alg_name ] scenario=[name] [ policy_kwargs ]Below you can find a UML diagram giving a high level explanation of how NGS works:
"Simulation of Urban MObility" (SUMO) is an open source, highly portable, microscopic traffic simulation package designed to handle large road networks and different modes of transport.
It is mainly developed by employees of the Institute of Transportation Systems at the German Aerospace Center.
You can download SUMO via our downloads site.
As the program is still under development and is extended continuously, we advice you to use the latest sources from our GitHub repository. Using a command line client the following command should work:
git clone --recursive https://github.com/eclipse/sumo
Stable Baselines is a set of improved implementations of reinforcement learning algorithms based on OpenAI Baselines.
Documentation is available online: https://stable-baselines.readthedocs.io/
RL Baselines Zoo. is a collection of pre-trained Reinforcement Learning agents using Stable-Baselines.
Note: Stabe-Baselines supports Tensorflow versions from 1.8.0 to 1.14.0. Support for Tensorflow 2 API is planned.
Baselines requires python3 (>=3.5) with the development headers. You'll also need system packages CMake, OpenMPI and zlib. Those can be installed as follows
sudo apt-get update && sudo apt-get install cmake libopenmpi-dev python3-dev zlib1g-devInstallation of system packages on Mac requires Homebrew. With Homebrew installed, run the following:
brew install cmake openmpiTo install stable-baselines on Windows, please look at the documentation.
Install the Stable Baselines package:
pip install stable-baselines[mpi]
This includes an optional dependency on MPI, enabling algorithms DDPG, GAIL, PPO1 and TRPO. If you do not need these algorithms, you can install without MPI:
pip install stable-baselines
Please read the documentation for more details and alternatives (from source, using docker).
OpenAI Gym is a toolkit for reinforcement learning (RL) widely used in research. The network simulator ns–3 is the de-facto standard for academic and industry studies in the areas of networking protocols and communication technologies. ns3-gym is a framework that integrates both OpenAI Gym and ns-3 in order to encourage usage of RL in networking research.
- Install all required dependencies required by ns-3.
# minimal requirements for C++:
apt-get install gcc g++ python
see https://www.nsnam.org/wiki/Installation
- Install ZMQ and Protocol Buffers libs:
# to install protobuf-3.6 on ubuntu 16.04:
sudo add-apt-repository ppa:maarten-fonville/protobuf
sudo apt-get update
apt-get install libzmq5 libzmq5-dev
apt-get install libprotobuf-dev
apt-get install protobuf-compiler
- Configure and build ns-3 project (if you are going to use Python virtual environment, please execute these commands inside it):
# Opengym Protocol Buffer messages (C++ and Python) are build during configure
./waf configure
./waf build
- Install ns3gym located in src/opengym/model/ns3gym (Python3 required)
pip3 install ./src/opengym/model/ns3gym
-
(Optional) Install all libraries required by your agent (like tensorflow, keras, etc.).
-
Run example:
cd ./scratch/opengym
./simple_test.py
- (Optional) Start ns-3 simulation script and Gym agent separately in two terminals (useful for debugging):
# Terminal 1
./waf --run "opengym"
# Terminal 2
cd ./scratch/opengym
./test.py --start=0
- American University of Beirut:
- Rayyan Nasr: [email protected]
- Jihad Eddine Al Khurfan: [email protected]
Please use the following bibtex :
@inproceedings{ns3gym,
Title = {{ns-3 meets OpenAI Gym: The Playground for Machine Learning in Networking Research}},
Author = {Gaw{\l}owicz, Piotr and Zubow, Anatolij},
Booktitle = {{ACM International Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems (MSWiM)}},
Year = {2019},
Location = {Miami Beach, USA},
Month = {November},
Url = {http://www.tkn.tu-berlin.de/fileadmin/fg112/Papers/2019/gawlowicz19_mswim.pdf}
}