LPSim is a discrete time-driven simulation platform that enables microsimulation analysis for network traffic assignment for both cars and aircrafts. Its architecture incorporates a highly parallelized GPU implementation that provides efficient execution of large-scale simulations on demand and networks with hundreds of thousands of nodes and edges, as well as millions of trips. The computational performance of LPSim is assessed by testing the platform to simulate the entire Bay Area metropolitan region during morning hours, utilizing half-second time steps. The runtime for the nine-county Bay Area simulation, excluding routing and initialization, is just over within minutes depending on how many GPUs are available to be used.
The concept of implementing a multi-GPU simulation can be elucidated as follows: initially, the network will undergo partitioning into distinct GPU units, following which, the simulation of individuals will be executed independently within separate GPUs for multiple time-steps, prior to any communication between these subunits.
sudo apt update
sudo apt install qtchooser
sudo apt-get install qt5-default
sudo apt-get install libglew-dev
sudo apt-get install build-essential
sudo apt-get install libfontconfig1
sudo apt-get install mesa-common-dev
sudo apt-get install wget
sudo apt-get install pciutils
sudo apt install git- Boost 1.59
wget http://sourceforge.net/projects/boost/files/boost/1.59.0/boost_1_59_0.tar.gz
sudo tar xf boost_1_59_0.tar.gz -C /usr/local- CUDA (used versions: 9.0 in Ubuntu) (If you are using Google Cloud Platform, please follow these instructions)
- g++ (used versions: 6.4.0 in Ubuntu)
- Qt5 (used versions: 5.9.5 in Ubuntu)
- qmake (used versions: 3.1 in Ubuntu)
- Python (used versions: 3.6.5 in Ubuntu)
- pytest (used versions: 6.1.1 in Ubuntu)
- pytest-cov (used versions: 2.10.1 in Ubuntu)
- pytest-remotedata (used versions: 0.3.2 in Ubuntu)
- psutil (used versions: 5.7.2 in Ubuntu)
- xlwt (used versions: 1.3.0 in Ubuntu)
Once the necessary dependencies are installed, you can use the following lines to make sure the correct versions of each one are used:
export PATH=/usr/local/cuda-9.0/bin:$PATH
export LIBRARY_PATH=/usr/local/cuda-9.0/lib64:$LIBRARY_PATH
export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/{YOUR_USERNAME}/pandana/srcYou can also add the export lines at the end of your user's ~/.bashrc to
avoid re-entering them in each session.
Clone the repo in your home directory with:
git clone [email protected]:Xuan-1998/LPSim.git ~/LPSim && cd ~/LPSimClone the Pandana repository to your home directory stay on the main branch, since LPSim now uses a fast contraction hierarchies framework for shortest path routing. Previously implemented shortest path frameworks include Johnson's all pairs shortest path and a parallelized Dijkstra's priority queue.
Create Makefile and compile with:
sudo qmake LivingCity/LivingCity.proImportantly, because LPSim uses a shared library from Pandana, a Pandana makefile must be created (to create a shared object file) and the LPSim makefile must be modified.
Pandana Makefile:
- Create Makefile in
pandana/src/containing the following:
CC = gcc # C compiler
CXX = g++
CPPFLAGS = -DLINUX -DMAC -std=c++0x -c -fPIC -g -O3 -Wall -pedantic -fopenmp # C flags
LDFLAGS = -shared # linking flags
RM = rm -f # rm command
TARGET_LIB = libchrouting.so # target lib
SRCS = accessibility.cpp graphalg.cpp contraction_hierarchies/src/libch.cpp
OBJS = $(SRCS:.cpp=.o)
.PHONY: all
all: ${TARGET_LIB}
$(TARGET_LIB): $(OBJS)
$(CXX) ${LDFLAGS} -o $@ $^
.PHONY: clean
clean:
-${RM} ${TARGET_LIB} ${OBJS}
- Run
make.
LPSim Makefile:
- Add
-I/home/{YOUR_USERNAME}/pandana/srctoINCPATH. - Add
-L/home/{YOUR_USERNAME}/pandana/src -lchroutingtoLIBS. - Run
sudo make -j.
- Make sure that you have Docker, its NVidia container toolkit and the necessary permissions:
sudo apt install docker.io
sudo groupadd docker
sudo usermod -aG docker {YOUR_USERNAME}
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
sudo systemctl restart docker
sudo apt-get install -y nvidia-container-toolkit- You also need to check the status of NVIDIA GPU on your instance or local machine
nvidia-smiIf error occurs e.g. on GCloud, you can check and install the correct NVIDIA driver version for your GPU on https://www.nvidia.cn/Download/index.aspx.
- You can build it yourself and run it
docker run -it --rm --gpus all -v "$PWD":/lpsim -w /lpsim xuanjiang1998/lpsim:v1 bash- Once inside the container, compile and run
qmake LivingCity/LivingCity.pro
make
cd LivingCity
./LivingCityBefore running everything, you need the appropriate data:
- Network
- Demand
The networks currently reside in LPSim/LivingCity/berkeley_2018, and the default directory is the full SF Bay Area network in new_full_network/. This contains the nodes.csv and edges.csv files to create the network.
The demand is not in new_full_network/, but needs to reside there in order to run it. Please contact Pavan Yedavalli to procure real or sample demands.
If you wish to edit the microsimulation configuration, modify LPSim/LivingCity/command_line_options.ini, which contains the following:
GUI=false
USE_CPU=false
NETWORK_PATH=berkeley_2018/new_full_network/
USE_JOHNSON_ROUTING=false
USE_SP_ROUTING=true
USE_PREV_PATHS=true
LIMIT_NUM_PEOPLE=256000
ADD_RANDOM_PEOPLE=false
NUM_PASSES=1
TIME_STEP=0.5
START_HR=5
END_HR=12
Here, you can modify the:
GUI- deprecated. Do not touch.USE_CPU- deprecated. Do not touch.NETWORK_PATH- specific path to the network files. Default isberkeley_2018/new_full_network/.USE_JOHNSON_ROUTING- uses Johnson's all pairs shortest path routing. This should always be set tofalse.USE_SP_ROUTING- uses new SP routing framework. This should always be set totrue.USE_PREV_PATHS- uses paths already produced and saved to file. Set tofalseif running for the first time. Set totrueif the simulation was already run and it was saved to file.LIMIT_NUM_PEOPLE- deprecated. Do not touch.ADD_RANDOM_PEOPLE- deprecated. Do not touch.NUM_PASSES- the number of times the simulation is run. Set to 1.TIME_STEP- timestep. Default is .5 seconds.START_HR- start hour of the simulation. Default is 5am.END_HR- end hour of the simulation. Default is 12pm.
Run with:
cd LivingCity
./LivingCityShould you wish to make any changes, please create a new branch. In addition, once the original Makefile is created, you can simply run sudo make -j from the LPSim/ directory to compile any new changes.
If necessary, you can checkout a different existing branch from main (edge_speeds_over_time, for instance):
git checkout edge_speeds_over_timeFor debugging we recommend cuda-memcheck ./LivingCity for out-of-bounds memory bugs in the CUDA section and cuda-gdb for more advanced features such as breakpoints.
In order to use cuda-gdb, LPSim/Makefile must be modified by adding the flag -G to enable debugging and changing -O3 to -O to avoid optimizations that restrict the use of the debugger.
For example, to enable debugging at LivingCity/traffic/b18CUDA_trafficSimulator.cu, its compilation at the line LPSim/Makefile:1756:
/usr/local/cuda-9.0/bin/nvcc -m64 -O3 -arch=sm_50 -c --compiler-options -f
no-strict-aliasing -use_fast_math --ptxas-options=-v -Xcompiler -fopenmp -I/u
sr/include/opencv2/ -I/opt/local/include/ -I/usr/local/boost_1_59_0/ -I/home/
{YOUR_USERNAME}/LPSim/LivingCity/glew/include/ -I/usr/local/cuda-9.0/include -L/opt/l
ocal/lib -lopencv_imgcodecs -lopencv_core -lopencv_imgproc -lcudart -lcuda -g -lgomp
LivingCity/traffic/b18CUDA_trafficSimulator.cu -o
${OBJECTS_DIR}b18CUDA_trafficSimulator_cuda.o
must be modified to:
/usr/local/cuda-9.0/bin/nvcc -m64 -O -arch=sm_50 -c --compiler-options -f
no-strict-aliasing -use_fast_math --ptxas-options=-v -Xcompiler -fopenmp -I/u
sr/include/opencv2/ -I/opt/local/include/ -I/usr/local/boost_1_59_0/ -I/home/
{YOUR_USERNAME}/LPSim/LivingCity/glew/include/ -I/usr/local/cuda-9.0/include -L/opt/l
ocal/lib -lopencv_imgcodecs -lopencv_core -lopencv_imgproc -lcudart -lcuda -g -G
-lgomp LivingCity/traffic/b18CUDA_trafficSimulator.cu -o
${OBJECTS_DIR}b18CUDA_trafficSimulator_cuda.o
After this modification, sudo make clean and sudo make -j must be run.
Please keep in mind that this alteration slows the program down. For more information about cuda-gdb, please refer to the official Website and Documentation.
In order to run all tests you should first move to LPSim/LivingCity
cd LivingCityand then run
sh runAllTests.shIn order to obtain a profiling of each component of the simulation, you should run
python3 LivingCity/benchmarking/runBenchmarks.pyIf you wish to specify the name of the benchmark outputs and/or the number of iterations, just run:
python3 LivingCity/benchmarking/runBenchmarks.py --name={name_of_benchmark} --runs={number_of_iterations_to_run}The script will run LivingCity the specified number of times while polling the system resources. For each component, its resource and time consumption will be saved into a csv file, a plot and a xls file in LPSim/LivingCity/benchmarking/. The profiling of each version is encouraged to be stored in here.
nvidia-smi -l 1 >> logfile.txt
docker pull your-dockerhub-username/lpsim:latest
docker run -it your-dockerhub-username/lpsim:latest /bin/bash
/usr/local/cuda-12.3/bin/nvcc -m64 -O2 -arch=sm_80 -c --compiler-options -fno-strict-aliasing -use_fast_math --ptxas-options=-v -Xcompiler -fopenmp --expt-relaxed-constexpr -I/usr/include/opencv4/ -I/opt/local/include/ -I/usr/local/boost_1_59_0/ -I/lpsim/LivingCity/glew/include/ -I/usr/include/pandana/src -I/usr/local/cuda-12.3/include -L/opt/local/lib -lopencv_imgcodecs -lopencv_core -lopencv_imgproc -lm -ldl -L/usr/include/pandana/src -lchrouting -lcudart -lcuda -lgomp LivingCity/traffic/b18CUDA_trafficSimulator.cu -o ${OBJECTS_DIR}b18CUDA_trafficSimulator_cuda.o
curl https://raw.githubusercontent.com/GoogleCloudPlatform/compute-gpu-installation/main/linux/install_gpu_driver.py --output install_gpu_driver.py
sudo python3 install_gpu_driver.py
nvidia-smi --loop-ms=1000 --filename=output_gpuusage.txt
while true; do nvidia-smi >> output_gpu.txt; sleep 10; done
run docker with:
docker run -it --rm --privileged --gpus all -v "$PWD":/LPSim -w /LPSim gcr.io/blissful-jet-303616/LPSim:latest bash
then:
nvprof --print-summary ./LivingCity >> profile_g3.txt 2>&1 (summary)
nvprof --print-gpu-trace ./LivingCity >> output_g3.txt 2>&1 (breakdown)
nvprof --metrics flop_count_sp,flop_count_dp ./LivingCity >> output_g3.txt 2>&1 (flop ratio)
This repository and code have been developed and maintained by Xuan Jiang, Jiaying Li, Yibo Zhao, Chonghe Jiang, Xin Peng, Johan Agerup, Yuhan Tang, and Raja Sengupta. This work is based on Pavan Yedavalli's Microsimulation analysis for network traffic assignment project.
If this code is used in any shape or form for your project, please cite this paper accordingly:
Jiang, X., Sengupta, R., Demmel, J., & Williams, S. (2024). Large scale multi-GPU based parallel traffic simulation for accelerated traffic assignment and propagation. Transportation Research Part C: Emerging Technologies, 169, 104873. Available: https://www.sciencedirect.com/science/article/pii/S0968090X24003942
Jiang, X., Tang, Y., Cao, J., Bulusu, V., Yang, H., Peng, X., Zheng, Y., Zhao, J., Sengupta, R. (2023). Simulating Integration of Urban Air Mobility into Existing Transportation Systems: Survey. Journal of Air Transportation, 1-11. Available: https://arc.aiaa.org/doi/10.2514/1.D0431
Jiang, X., Cao, S., Mo, B., Cao, J., Yang, H., Tang, Y., Hansen, M., Zhao, J. and Sengupta, R., 2024. Simulation-Based Optimization for Vertiport Location Selection: A Surrogate Model With Machine Learning Method. Transportation Research Record, p.03611981241277755. Available: https://journals.sagepub.com/doi/full/10.1177/03611981241277755
Thank you!
This project is licensed under the MIT License.
MIT License
Copyright (c) 2024 Xuan Jiang, Jiaying Li, Chonghe Jiang, Dingyi Zhuang, Jinhua Zhao, Raja Sengupta
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
![@dependabot[bot]](https://avatars.githubusercontent.com/in/29110?s=64&v=4){kind=small}
