Concept

Goals and tasks

The goal of the project is to develop a software for measuring the performance of a wide range of deep learning models inferring on various popular frameworks and various hardware, as well as regularly publishing the obtained data.

Tasks

1. To develop the software architecture that allows you to extend a set of supported frameworks and hardware platforms.
2. To implement the software. Inference implementation is based on the OpenVINO toolkit or another inference frameworks.
3. To develop an infrastructure to automatically deploy a benchmarking system and collect inference performance metrics.
4. To integrate other frameworks to infer deep models (Caffe, TensorFlow, MXNet, etc.).
5. To gather performance metrics for deep models regularly.
6. To extend the set of models for which inference performance is measured regularly.