Home

Deep Learning Inference Benchmark Wiki

General resources

• Getting started
• Official site (RU, ENG)

How to build

• Linux Systems

How to deploy and run

• DL‐Benchmark system setup
• Model preparing
• Quick deployment guide
• Quick startup guide
• Step-by-step manual (from deployment to inference performance and deep model accuracy)

How to infer deep models

• Examples of command lines

Developer documentation

• Concept
• Architecture
• Contribution guide
• Coding style guide
• Tests
• Papers
• New feature ideas

Benchmarking results

Datasets:

• Image classification: ImageNet
• Object detection: MS COCO, PASCAL VOC
• Semantic segmentation: MS COCO, PASCAL VOC
• Instance segmentation: MS COCO
• Semantic segmentation of on-road scenes: Cityscapes
• Face detection: LFW, VGGFace2, Wider Face
• Other tasks: public images from the Internet

Checked models for supported frameworks:

• Checked models inferring by Intel® Distribution of OpenVINO™ toolkit (public models, Intel models)
• Checked models inferring by Caffe
• Checked models inferring by TensorFlow
• Checked models inferring by TensorFlow Lite
• Checked models inferring by ONNX Runtime
• Checked models inferring by MXNet (Gluon API)
• Checked models inferring by OpenCV DNN
• Checked models inferring by PyTorch
• Checked models inferring by Apache TVM
• Checked models inferring by ncnn
• Checked models inferring by PaddlePaddle

Parameters:

• List of parameters

Metrics:

• Performance metrics
• Accuracy metrics

Results:

• Performance
• Accuracy