Accelerating Object Classification Using FPGAs

The goal of this project is to run the LeNet5 model trained for classifying traffic signs on a FPGA. Here is a high-level description of the design tree. For more details please refer to the final report in the docs/ directory.

A brief video demo of the design can be found here.

Repository Structure

G3_FPGA_Object_Classification_Accelerator
├── README.md
├── docs                                        # Documentation
│   ├── final_report.pdf                        # Final group report
│   └── final_presentation.pptx                 # Final presentation
├── ml_training
│   └── pytorch_edlenet.ipynb                   # PyTorch training code for Lenet-5 model
├── server                                      # Python server and GUI that interacts with both FPGAs
│   ├── __init__.py
│   ├── fpga.py                                 # Main server file
│   ├── fpga_gui.ui                             # Python GUI
│   ├── sign_label_images.py
│   └── signnames.csv
└── src                                         # Hardware and Software components of the design
    ├── lenet5                                  # Video board component containing the Neural Core
    │   ├── lenet
    │   │   ├── lenet.srcs
    │   │   │   ├── lenet.bd                    # Top-level Vivado block diagram
    │   │   │   └── lenet_wrapper.v             # Top-level Verilog wrapper generated from the block diagram
    │   │   └── sw                              # SDK software component
    │   │       ├── lscript.ld                  # Linker script
    │   │       └── main.c                      # Main code containing LWIP client, DMA controller, interrupt handler
    │   ├── neural_core                         # Custom Neural Core IP
    │   │   ├── component.xml                   # Custom IP-XACT file generated by Vivado IP integrator
    │   │   └── neural_core.srcs                # Most important Verilog source files (not everything)
    │   │       ├── cnn_core.v                  # Top-level module for Convolutional Layer
    │   │       ├── fully_connected_network.sv  # Top-level module for Fully Connected Layer
    │   │       ├── neural_core.v               # Top-level module of the Neural Core subsystem
    │   │       └── weight_bias                 # Memory files containing weights and biases
    │   └── scripts                             # Useful automation scripts
    │       ├── install_opencv.bat              # Install OpenCV, pyqt5, PyTorch, Pandas
    │       ├── sdk.bat                         # Open vivado in batch mode and execute sdk.tcl
    │       └── sdk.tcl                         # Launch SDK with correct hardware definition file
    └── vga                                     # DDR board component containing VGA controller code
        ├── sw                                  # SDK software component
        │   ├── classes.h                       # Contains arrays of classification labels, used to draw on VGA
        │   ├── lscript.ld                      # Linker script
        │   └── main.c                          # Main code containing LWIP client, VGA controller, image drawing logic
        └── tft_vga_ctrl.srcs
            ├── constrs_1
            │   └── constrs.xdc                 # constraint file for RGB pins of TFT controller and reset signals
            └── sources_1
                ├── tft_vga_ctrl.bd             # Top-level Vivado block diagram
                └── tft_vga_ctrl_wrapper.v      # Top-level Verilog wrapper generated from the block diagram  

Authors

Hammad Mohiudin

Mehdi Mousavi

Ahanaf Rakin

Farid Chalabi