Skip to content

Commit

Permalink
Grammar check; removed obsolexte parts not relevant to ovc; better wo…
Browse files Browse the repository at this point in the history
…rding
  • Loading branch information
slyalin committed Sep 1, 2023
1 parent e452984 commit a4f1cdb
Show file tree
Hide file tree
Showing 9 changed files with 53 additions and 132 deletions.
20 changes: 7 additions & 13 deletions docs/Documentation/model_introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
.. meta::
:description: Preparing models for OpenVINO Runtime. Learn how to convert and compile models from different frameworks or read them directly.


.. toctree::
:maxdepth: 1
:hidden:
Expand All @@ -15,7 +14,6 @@
openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition
openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide


Every deep learning workflow begins with obtaining a model. You can choose to prepare a custom one, use a ready-made solution and adjust it to your needs, or even download and run a pre-trained network from an online database, such as `TensorFlow Hub <https://tfhub.dev/>`__, `Hugging Face <https://huggingface.co/>`__, `Torchvision models <https://pytorch.org/hub/>`__.

OpenVINO™ :doc:`supports several model representations <Supported_Model_Formats>` and allows converting them to it's own representation, `openvino.Model <api/ie_python_api/_autosummary/openvino.Model.html>`__ (`ov.Model <api/ie_python_api/_autosummary/openvino.runtime.Model.html>`__ ), providing a conversion API to this task. Converted model can be used for inference using one or multiple OpenVINO Hardware plugins. This chapter describes two variants of using conversion API: using a Python program or calling `ovc` command line tool.
Expand All @@ -27,7 +25,6 @@ OpenVINO™ :doc:`supports several model representations <Supported_Model_Format
and `mo` correspondingly, which are considered to be legacy now. All new users are recommended to use these new methods instead of the old methods. Please note that the new API and old API do not
provide the same level of features, that means the new tools are not always backward compatible with the old ones. Please consult with `Model Conversion API Transition Guide <openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition>`.


Convert a model in Python: `convert_model`
##########################################

Expand Down Expand Up @@ -180,36 +177,34 @@ Model conversion API is exposed in Python by means of ``openvino.convert_model``
# run the inference
result = compiled_model(input_data)


In Option 1, where `openvino.save_model` function is used, an OpenVINO model will be serialized in the file system as a pair of files with extensions `.xml` and `.bin`. This pair of files is called OpenVINO Intermediate Representation format (OpenVINO IR, or just IR) and useful for efficient model deployment. OpenVINO IR is intended to be loaded in another application with the `openvino.Core.read_model` for inference, please see details in :doc:`OpenVINO™ Runtime documentation <openvino_docs_OV_UG_OV_Runtime_User_Guide>`.

Option 2, where ``openvino.compile_model`` is used, provides a convenient way to quickly switch from framework-based code to OpenVINO-based code in your already existing Python inference application. In this case, the converted model isn't saved to the IR, instead the model is compiled and used for inference in the same application.
Option 2, where ``openvino.compile_model`` is used, provides a convenient way to quickly switch from framework-based code to OpenVINO-based code in your already existing Python inference application. In this case, the converted model isn't saved to the IR, instead, the model is compiled and used for inference in the same application.

Following the option 1 means separating the model conversion and the model inference into two different applications. This approach addresses deployment scenarious where it is required to minimize extra dependencies and speedup model loading in the end inference application. For example, to convert a PyTorch model to OpenVINO, `torch` Python module is required as a dependency and the conversion must be performed in Python. It takes extra time and memory that wouldn't be required for inference of the converted model. But when the converted model is saved into the IR files with `openvino.save_model`, it can be loaded in a separate application without `torch` dependency and without need to spend time for model conversion. Also the inference appication can be written in other programming languages supported by OpenVINO, for example, in C++, and Python is not required to be installed for the inference application in this case.
Following option 1 means separating the model conversion and the model inference into two different applications. This approach addresses deployment scenarios where it is required to minimize extra dependencies and speed up model loading in the end inference application. For example, to convert a PyTorch model to OpenVINO, `torch` Python module is required as a dependency and the conversion must be performed in Python. It takes extra time and memory that wouldn't be required for inference of the converted model. But when the converted model is saved into the IR files with `openvino.save_model`, it can be loaded in a separate application without `torch` dependency and without the need to spend time for model conversion. Also, the inference application can be written in other programming languages supported by OpenVINO, for example, in C++, and Python is not required to be installed for the inference application in this case.

.. image:: _static/images/model_conversion_diagram.svg
:alt: model conversion diagram

Convert a model in CLI: ``ovc``
###############################

Another option to convert a model is to use ``ovc`` command-line tool. `ovc` stands for OpenVINO Model Converter and combines both `openvino.convert_model` and `openvino.save_model` together. It is convenient to use if the original model is ready for inference and represeted in a file of one of the supported formats: ONNX, TensorFlow, TensorFlow Lite, or PaddlePaddle. As the result of the conversion, `ovc` produces OpenVINO IR as a pair of `.xml` and `.bin` files, which needs to be read with the ``ov.read_model()`` method. Then, you can compile and infer the ``ov.Model`` later with :doc:`OpenVINO™ Runtime <openvino_docs_OV_UG_OV_Runtime_User_Guide>`
Another option to convert a model is to use ``ovc`` command-line tool. `ovc` stands for OpenVINO Model Converter and combines both `openvino.convert_model` and `openvino.save_model` together. It is convenient to use if the original model is ready for inference and represented in a file of one of the supported formats: ONNX, TensorFlow, TensorFlow Lite, or PaddlePaddle. As the result of the conversion, `ovc` produces OpenVINO IR as a pair of `.xml` and `.bin` files, which needs to be read with the ``ov.read_model()`` method. Then, you can compile and infer the ``ov.Model`` later with :doc:`OpenVINO™ Runtime <openvino_docs_OV_UG_OV_Runtime_User_Guide>`

.. note::
PyTorch models cannot be converted with `ovc`, use `openvino.convert_model` instead.

The figure below illustrates the typical workflow for deploying a trained deep learning model:
The figure below illustrates the typical workflow for deploying a trained deep-learning model:

**TODO: Update BASIC_FLOW_MO_simplified.svg and replace 'mo' with 'ovc'**
.. image:: _static/images/BASIC_FLOW_MO_simplified.svg

The results of both ``ovc`` and ``openvino.convert_model``/``openvino.save_model`` conversion methods described above are the same. You can choose one of them, depending on what is most convenient for you. Keep in mind that there should not be any differences in the results of model conversion if the same set of parameters is used and model is saved into OpenVINO IR.

The results of both ``ovc`` and ``openvino.convert_model``/``openvino.save_model`` conversion methods described above are the same. You can choose one of them, depending on what is most convenient for you. Keep in mind that there should not be any differences in the results of model conversion if the same set of parameters is used and the model is saved into OpenVINO IR.

Cases when Model Preparation is not Required
Cases When Model Preparation is Not Required
############################################

If model is represented as a single file from ONNX, PaddlePaddle, TensorFlow and TensorFlow Lite (check :doc:`TensorFlow Frontend Capabilities and Limitations <openvino_docs_MO_DG_TensorFlow_Frontend>`), then it doesn't not require a separate step for model conversion and saving to IR, that is ``openvino.convert_model`` and ``openvino.save_model``, or `ovc`. OpenVINO provides C++ and Python APIs for reading such models by just calling the ``openvino.Core.read_model`` or ``openvino.Core.compile_model`` methods. These methods perform conversion of the model from the original representation. Besides this conversion takes some extra time in comparison to reading the same model from prepared OpenVINO IR, it may be convenient in cases when it is required to read a model in the original format in C++ as `openvino.convert_model` is available in Python only. Preparing OpenVINO IR as a dedicated step and then using this IR in an application dedicated for inference is still recommend way for the efficient model deployment for OpenVINO runtime.
If a model is represented as a single file from ONNX, PaddlePaddle, TensorFlow and TensorFlow Lite (check :doc:`TensorFlow Frontend Capabilities and Limitations <openvino_docs_MO_DG_TensorFlow_Frontend>`), then it doesn't require a separate step for model conversion and saving to IR, that is ``openvino.convert_model`` and ``openvino.save_model``, or `ovc`. OpenVINO provides C++ and Python APIs for reading such models by just calling the ``openvino.Core.read_model`` or ``openvino.Core.compile_model`` methods. These methods perform conversion of the model from the original representation. Besides this conversion takes some extra time in comparison to reading the same model from prepared OpenVINO IR, it may be convenient in cases when it is required to read a model in the original format in C++ as `openvino.convert_model` is available in Python only. Preparing OpenVINO IR as a dedicated step and then using this IR in an application dedicated to inference is still the recommended way for the efficient model deployment for OpenVINO runtime.

This section describes how to obtain and prepare your model for work with OpenVINO to get the best inference results:

Expand All @@ -218,4 +213,3 @@ This section describes how to obtain and prepare your model for work with OpenVI
* :doc:`Transition guide from mo / ov.tools.mo.convert_model to OVC / openvino.convert_model <openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition>`

@endsphinxdirective

Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
:description: Model Conversion API provides a way to import pre-trained models
from various deep learning frameworks to OpenVINO runtime for accelerated inference.


To convert a model to OpenVINO model, you can use the following commands as described in `Model preparation <openvino_docs_model_processing_introduction>` chapter:

.. tab-set::
Expand Down Expand Up @@ -43,15 +42,15 @@ To convert a model to OpenVINO model, you can use the following commands as desc

ovc path_to_your_model

Providing just a path to the model or model object as `openvino.convert_model` argument is frequently enough to make a succesfull conversion. However, depending on the model topology and original deep learning framework, additional parameters may be required:
Providing just a path to the model or model object as `openvino.convert_model` argument is frequently enough to make a successful conversion. However, depending on the model topology and original deep learning framework, additional parameters may be required:

- `example_input` parameter available in Python `openvino.convert_model` only is intended to trace the model to obtain its graph representation. This parameter is crucial for succesful conversion of model from PyTorch and sometimes requried for TensorFlow. Refer to the :doc:`PyTorch Model Conversion <openvino_docs_OV_Converter_UG_prepare_model_convert_model_Convert_Model_From_PyTorch>` or `TensorFlow Model Conversion <openvino_docs_OV_Converter_UG_prepare_model_convert_model_Convert_Model_From_TensoFlow>`.
- `example_input` parameter available in Python `openvino.convert_model` only is intended to trace the model to obtain its graph representation. This parameter is crucial for the successful conversion of a model from PyTorch and is sometimes required for TensorFlow. Refer to the :doc:`PyTorch Model Conversion <openvino_docs_OV_Converter_UG_prepare_model_convert_model_Convert_Model_From_PyTorch>` or `TensorFlow Model Conversion <openvino_docs_OV_Converter_UG_prepare_model_convert_model_Convert_Model_From_TensoFlow>`.

- `input` parameter to set or override shapes for model inputs. It configures dynamic and static dimensions in model inputs depending on your needs for the most efficient inference. For more information about using this parameter, refer to the :doc:`Setting Input Shapes <openvino_docs_OV_Converter_UG_prepare_model_convert_model_Converting_Model>` guide.

- `output` parameter to select one or multiple outputs from the original model. Useful if the model has outputs that are not required for inference in a deployment scenario. Specifying only necessary outputs may lead to more compact model that infer faster.
- `output` parameter to select one or multiple outputs from the original model. Useful if the model has outputs that are not required for inference in a deployment scenario. Specifying only necessary outputs may lead to a more compact model that infers faster.

- `compress_to_fp16` parameter that are provided by `ovc` CLI tool and `openvino.save_model` Python function, gives controls over compression of model weights to FP16 format when saving OpenVINO model to IR. This option is enabled by default that means all produced IRs are saved using FP16 data type for weights which saves up to 2x storage space for the model file and in most cases doesn't sacrifice model accuracy. In case if it does affect accuracy, the compression can be disabled by using this flag. Refer to the :doc:`Compression of a Model to FP16 <openvino_docs_OV_Converter_UG_FP16_Compression>` guide.
- `compress_to_fp16` parameter that is provided by `ovc` CLI tool and `openvino.save_model` Python function, gives controls over the compression of model weights to FP16 format when saving OpenVINO model to IR. This option is enabled by default which means all produced IRs are saved using FP16 data type for weights which saves up to 2x storage space for the model file and in most cases doesn't sacrifice model accuracy. In case it does affect accuracy, the compression can be disabled by setting this flag to `False`. Refer to the :doc:`Compression of a Model to FP16 <openvino_docs_OV_Converter_UG_FP16_Compression>` guide.

- `extension` parameter which makes possible conversion of the models consisting of operations that are not supported by OpenVINO out-of-the-box. It requires implementing of an OpenVINO extension first, please refer to :doc:`Setting Input Shapes <openvino_docs_Extensibility_UG_Frontend_Extensions>` guide.

Expand Down
4 changes: 2 additions & 2 deletions docs/OV_Converter_UG/prepare_model/FP16_Compression.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ To disable compression, use the ``compress_to_fp16=False`` option:

ovc path_to_your_model --compress_to_fp16=False


For details on how plugins handle compressed ``FP16`` models, see
:doc:`Working with devices <openvino_docs_OV_UG_Working_with_devices>`.

Expand All @@ -40,5 +39,6 @@ For details on how plugins handle compressed ``FP16`` models, see
Refer to the :doc:`Post-training optimization <**TODO: LINK TO NNCF>` guide for more
information about that.


@endsphinxdirective


Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,10 @@
:description: Learn how to convert a model from the
ONNX format to the OpenVINO Model.


Introduction to ONNX
####################

`ONNX <https://github.com/onnx/onnx>`__ is a representation format for deep learning models that allows AI developers to easily transfer models between different frameworks. It is hugely popular among deep learning tools, like PyTorch, Caffe2, Apache MXNet, Microsoft Cognitive Toolkit, and many others.
`ONNX <https://github.com/onnx/onnx>`__ is a representation format for deep learning models that allows AI developers to easily transfer models between different frameworks.

.. note:: ONNX model file can be loaded by `openvino.Core.read_model` or `openvino.Core.compile_model` methods by OpenVINO runtime API without preparing OpenVINO IR first. Refer to the :doc:`inference example <openvino_docs_OV_UG_Integrate_OV_with_your_application>` for more details. Using ``openvino.convert_model`` is still recommended if model load latency matters for the inference application.

Expand All @@ -36,13 +35,12 @@ To convert an ONNX model, run model conversion with the path to the input model

ovc your_model_file.onnx

Multi-File Models Support
External Data Files
#########################

ONNX models that consist of multiple parts are supported. It is relevant to big models that exeed 2GB protobuf limitation and cannot be represented as a single protobuf file in standard ONNX. Instead of a single file, such a model is represented as one file with `.onnx` extension and multiple sattelite files that hold model weights that are located in the same directory or in a subdirectory of a directory where main `.onnx` file is located.

OpenVINO model conversion API supports such multi-file ONNX representation. In this case only a single file with `.onnx` extension should be passed as `ovc` or `openvino.convert_model` parameter while other files will be found and loaded automatically during the mode conversion. The resulting OpenVINO model represented as IR no the filesystem will have usual structure with a single `.xml` file and a single `.bin` file where all the original model weights are copied to and packed together.
ONNX models may consist of multiple files when the total size of the model exceeds 2GB allowed by Protobuf. According to `ONNX< https://github.com/onnx/onnx/blob/main/docs/ExternalData.md>`, instead of a single file, such a model is represented as one file with `.onnx` extension and multiple separate files with external data which are located in the same directory where the main `.onnx` file is located or in another directory.

OpenVINO model conversion API supports ONNX models with external data representation. In this case, only the main file with `.onnx` extension should be passed as `ovc` or `openvino.convert_model` parameter while other files will be found and loaded automatically during the mode conversion. The resulting OpenVINO model represented as IR in the filesystem will have the usual structure with a single `.xml` file and a single `.bin` file where all the original model weights are copied and packed together.

Supported ONNX Layers
#####################
Expand All @@ -54,6 +52,4 @@ Additional Resources

**TODO: LINK TO NOTEBOOKS** page for a set of tutorials providing step-by-step instructions for converting specific ONNX models.


@endsphinxdirective

Loading

0 comments on commit a4f1cdb

Please sign in to comment.