EdgeCam: A Distributed Camera System for Inference Scheduling and Continuous Learning in Video Analytics
EdgeCam, an open-source distributed camera system that incorporates inference scheduling and continuous learning for video analytics. The system comprises multiple edge nodes and the cloud, enabling collaborative video analytics. The edge node also gathers drift data to support continuous learning and maintain recognition accuracy.
Firstly, for the video collected from on-edge camera, our system support a filter engine that can determine whether a video frame needs to be filtered or not, so as to save the downstream resource costs. Then, our system have a decision engine to intelligently select offloading strategies for video frames, while adjusting their resolutions and encoding qualities adaptively. Moreover, EdgeCam implements continuous learning to enhance the accuracy of lightweight models on edge nodes, enabling them to handle data drift.
The main implementation details of EdgeCam are shown below.
| modules | Description |
|---|---|
| Video Reader | Provides user-friendly interfaces for reading video streams using OpenCV. Supports offline video files and network cameras using the Real-Time Streaming Protocol (RTSP). |
| Filter | Aims to filter out the redundant video frames. Implements four methods to compute differences. |
| Decision Engine | Encapsulates different offloading and video processing policies, each governing the workflow of video frames. |
| Standalone Worker | A thread that continuously extracts tasks from the local queue and then uses the locally deployed DNN model to perform inference. |
| Offloading Worker | Each offloading task will be assigned an offloading thread from the thread pool. The offloading thread dispatches the task to the corresponding edge server or offloads it to the cloud server. |
| Drift Data Collector | Periodically selects a subset of video frames with lower average confidence in the predicted results from locally inferred frames. |
| Model Retraining Worker | A thread that sends the selected frames for retraining to the cloud to get ground truth accuracy. Utilizes these frames and ground truth to retrain the current model. |
| Request Handler | A thread that listens to the requests from the edge nodes. |
On edge
Please install the following libraries on each edge node.
- Install the deep learning framework pytorch and opencv-python on the Jetson.
- Install dependent libraries.
pip3 install munch
pip3 install grpcio
pip3 install grpcio-tools
pip3 install loguru
pip3 install mysql-connector-python
pip3 install mapcalc
pip3 install APScheduler
pip3 install imutils
pip3 install PyYAMLOn cloud
Similar to the installation on the edge node, install the corresponding version of Pytorch and required libraries.
Database
- Installthe MySQL database.
sudo apt install mysql-server- The MySQL database is configured to allow remote connections.
Video Source
If the video source is a video file, please configure the path of the video file.
video_path: your video path
If the video source is a network camera, please configure the account, password, and IP address.
rtsp:
label: True
account: your account
password: your password
ip_address: your camera ip
channel: 1
Feature Type
One can choose different features to calculate video frame difference, including pixel, edge, area, and corner features.
feature: edge
IP configuration
Please configure the IP address of the cloud server.
server_ip: 'server ip:50051'
Please configure the number and IP addresses of edge nodes.
edge_id: the edge node ID
edge_num: the number of edge nodes
destinations: {'id': [1, 2, ...], 'ip':[ip1, ip2, ...]}
Deployed Model
The models deployed on the edge node and the cloud can be configured by specifying model names. The pre-trained model directory is model_management/models.
lightweight: fasterrcnn_mobilenet_v3_large_fpn
golden: fasterrcnn_resnet50_fpn
Retraining configuration
The users can configure whether to collect frames for model retraining, the window size of retraining, the number of collected frames, the number of training epochs, etc.
Database
To be able to connect to the database, please configure user name, password, and ip address of the database.
connection: {'user': 'your name', 'password': 'your password', 'host': 'database ip', 'raise_on_warnings': True}
Offloading policy
Please configure offloading policy.
policy: Edge-Cloud-Assited
For example:
Edge-Local: Video frames received by an edge node are processed exclusively by that node.
Edge-Shortest: Video frames are dispatched from an edge node to the edge node with the shortest inference queue. The selected edge node performs the inference.
Shortest-Cloud-Threshold: If the local inference queue lengths of all edge nodes surpass a specified threshold, the edge node directly offloads the video frame to the cloud for inference. Otherwise, it is dispatched to the edge node with the shortest queue for inference.
Edge-Cloud-Assisted: Inference initially takes place on the edge node using a lightweight DNN. Regions of the video frame with low recognition confidence below a threshold are offloaded to the cloud for inference.
cd ~/EdgeCam
python3 cloud_server.pyPlease use the following command for each edge node.
cd ~/EdgeCam
python3 edge_client.pyIf you have any questions, please feel free to contact us. Email: [email protected]; [email protected]; [email protected]
MIT License
Copyright (c) 2023 Multimedia Systems and Networking Group
A copy of the license is available at the following link: