diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 06990f58..89573c94 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -1,7 +1,7 @@ --- -name: TensorRT OSS Bug Report -about: Report any bugs to help us improve TensorRT. -title: '' +name: Report a TensorRT issue +about: The more information you share, the more feedback we can provide. +title: 'XXX failure of TensorRT X.Y when running XXX on GPU XXX' labels: '' assignees: '' @@ -9,36 +9,59 @@ assignees: '' ## Description - + ## Environment -**TensorRT Version**: -**NVIDIA GPU**: -**NVIDIA Driver Version**: -**CUDA Version**: -**CUDNN Version**: -**Operating System**: -**Python Version (if applicable)**: -**Tensorflow Version (if applicable)**: -**PyTorch Version (if applicable)**: -**Baremetal or Container (if so, version)**: + + +**TensorRT Version**: + +**NVIDIA GPU**: + +**NVIDIA Driver Version**: + +**CUDA Version**: + +**CUDNN Version**: + + +Operating System: + +Python Version (if applicable): + +Tensorflow Version (if applicable): + +PyTorch Version (if applicable): + +Baremetal or Container (if so, version): ## Relevant Files +**Model link**: + ## Steps To Reproduce - +**Commands or scripts**: + +**Have you tried [the latest release](https://developer.nvidia.com/tensorrt)?**: + +**Can this model run on other frameworks?** For example run ONNX model with ONNXRuntime (`polygraphy run --onnxrt`): diff --git a/CHANGELOG.md b/CHANGELOG.md index 1e8abf23..1c92f9f9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,10 +1,21 @@ # TensorRT OSS Release Changelog -## [8.6.0 EA](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html#tensorrt-8) - 2023-03-14 +## [8.6.1 GA](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-6-1) - 2023-05-02 + +TensorRT OSS release corresponding to TensorRT 8.6.1.6 GA release. +- Updates since [TensorRT 8.6.0 EA release](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-6-0-EA). +- Please refer to the [TensorRT 8.6.1.6 GA release notes](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-6-1) for more information. + +Key Features and Updates: + +- Added a new flag `--use-cuda-graph` to demoDiffusion to improve performance. +- Optimized GPT2 and T5 HuggingFace demos to use fp16 I/O tensors for fp16 networks. + +## [8.6.0 EA](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-6-0-EA) - 2023-03-10 TensorRT OSS release corresponding to TensorRT 8.6.0.12 EA release. -- Updates since [TensorRT 8.5.3 GA release](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html#rel-8-5-3). -- Please refer to the [TensorRT 8.6.0.12 EA release notes](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html#tensorrt-8) for more information. +- Updates since [TensorRT 8.5.3 GA release](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-5-3). +- Please refer to the [TensorRT 8.6.0.12 EA release notes](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-6-0-EA) for more information. Key Features and Updates: @@ -12,11 +23,11 @@ Key Features and Updates: - The following plugins have been removed accordingly: GroupNorm, LayerNorm, MultiHeadCrossAttention, MultiHeadFlashAttention, SeqLen2Spatial, and SplitGeLU. - Added a new sample called onnx_custom_plugin. -## [8.5.3 GA](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html#rel-8-5-3) - 2023-01-30 +## [8.5.3 GA](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-5-3) - 2023-01-30 TensorRT OSS release corresponding to TensorRT 8.5.3.1 GA release. -- Updates since [TensorRT 8.5.2 GA release](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html#rel-8-5-2). -- Please refer to the [TensorRT 8.5.3 GA release notes](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html#rel-8-5-3) for more information. +- Updates since [TensorRT 8.5.2 GA release](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-5-2). +- Please refer to the [TensorRT 8.5.3 GA release notes](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-5-3) for more information. Key Features and Updates: @@ -24,11 +35,11 @@ Key Features and Updates: - Added nvinfer1::plugin namespace - Optimized KV Cache performance for T5 -## [8.5.2 GA](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html#rel-8-5-2) - 2022-12-12 +## [8.5.2 GA](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-5-2) - 2022-12-12 TensorRT OSS release corresponding to TensorRT 8.5.2.2 GA release. -- Updates since [TensorRT 8.5.1 GA release](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html#rel-8-5-1). -- Please refer to the [TensorRT 8.5.2 GA release notes](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html#rel-8-5-2) for more information. +- Updates since [TensorRT 8.5.1 GA release](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-5-1). +- Please refer to the [TensorRT 8.5.2 GA release notes](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-5-2) for more information. Key Features and Updates: @@ -51,11 +62,11 @@ Key Features and Updates: ### Removed - None -## [8.5.1 GA](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html#rel-8-5-1) - 2022-11-01 +## [8.5.1 GA](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-5-1) - 2022-11-01 TensorRT OSS release corresponding to TensorRT 8.5.1.7 GA release. - Updates since [TensorRT 8.4.1 GA release](https://github.com/NVIDIA/TensorRT/releases/tag/8.4.1). -- Please refer to the [TensorRT 8.5.1 GA release notes](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html#rel-8-5-1) for more information. +- Please refer to the [TensorRT 8.5.1 GA release notes](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-5-1) for more information. Key Features and Updates: @@ -84,7 +95,7 @@ Key Features and Updates: ## [22.08](https://github.com/NVIDIA/TensorRT/releases/tag/22.08) - 2022-08-16 -Updated TensorRT version to 8.4.2 - see the [TensorRT 8.4.2 release notes](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html#rel-8-4-2) for more information +Updated TensorRT version to 8.4.2 - see the [TensorRT 8.4.2 release notes](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-4-2) for more information ### Changed - Updated default protobuf version to 3.20.x @@ -114,11 +125,11 @@ Updated TensorRT version to 8.4.2 - see the [TensorRT 8.4.2 release notes](https ### Removed - None -## [8.4.1 GA](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html#rel-8-4-1) - 2022-06-14 +## [8.4.1 GA](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-4-1) - 2022-06-14 TensorRT OSS release corresponding to TensorRT 8.4.1.5 GA release. - Updates since [TensorRT 8.2.1 GA release](https://github.com/NVIDIA/TensorRT/releases/tag/8.2.1). -- Please refer to the [TensorRT 8.4.1 GA release notes](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html#rel-8-4-1) for more information. +- Please refer to the [TensorRT 8.4.1 GA release notes](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-4-1) for more information. Key Features and Updates: @@ -258,11 +269,11 @@ Key Features and Updates: ### Removed - Unused source file(s) in demo/BERT -## [8.2.1 GA](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html#rel-8-2-1) - 2021-11-24 +## [8.2.1 GA](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-2-1) - 2021-11-24 TensorRT OSS release corresponding to TensorRT 8.2.1.8 GA release. - Updates since [TensorRT 8.2.0 EA release](https://github.com/NVIDIA/TensorRT/releases/tag/8.2.0-EA). -- Please refer to the [TensorRT 8.2.1 GA release notes](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html#rel-8-2-1) for more information. +- Please refer to the [TensorRT 8.2.1 GA release notes](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-2-1) for more information. - ONNX parser [v8.2.1](https://github.com/onnx/onnx-tensorrt/releases/tag/release%2F8.2-GA) - Removed duplicate constant layer checks that caused some performance regressions @@ -316,7 +327,7 @@ TensorRT OSS release corresponding to TensorRT 8.2.1.8 GA release. - Updated Python documentation for `add_reduce`, `add_top_k`, and `ISoftMaxLayer` - Renamed default GitHub branch to `main` and updated hyperlinks -## [8.2.0 EA](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html#rel-8-2-0-EA) - 2021-10-05 +## [8.2.0 EA](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#rel-8-2-0-EA) - 2021-10-05 ### Added - [Demo applications](demo/HuggingFace) showcasing TensorRT inference of [HuggingFace Transformers](https://huggingface.co/transformers). - Support is currently extended to GPT-2 and T5 models. @@ -426,7 +437,7 @@ TensorRT OSS release corresponding to TensorRT 8.2.1.8 GA release. ## [21.07](https://github.com/NVIDIA/TensorRT/releases/tag/21.07) - 2021-07-21 Identical to the TensorRT-OSS [8.0.1](https://github.com/NVIDIA/TensorRT/releases/tag/8.0.1) Release. -## [8.0.1](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html#tensorrt-8) - 2021-07-02 +## [8.0.1](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/#tensorrt-8) - 2021-07-02 ### Added - Added support for the following ONNX operators: `Celu`, `CumSum`, `EyeLike`, `GatherElements`, `GlobalLpPool`, `GreaterOrEqual`, `LessOrEqual`, `LpNormalization`, `LpPool`, `ReverseSequence`, and `SoftmaxCrossEntropyLoss` [details](). - Rehauled `Resize` ONNX operator, now fully supporting the following modes: diff --git a/CMakeLists.txt b/CMakeLists.txt index 4847eaf5..66f4201b 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -48,7 +48,7 @@ set(CMAKE_SKIP_BUILD_RPATH True) project(TensorRT LANGUAGES CXX CUDA VERSION ${TRT_VERSION} - DESCRIPTION "TensorRT is a C++ library that facilitates high performance inference on NVIDIA GPUs and deep learning accelerators." + DESCRIPTION "TensorRT is a C++ library that facilitates high-performance inference on NVIDIA GPUs and deep learning accelerators." HOMEPAGE_URL "https://github.com/NVIDIA/TensorRT") if(CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT) @@ -88,8 +88,8 @@ endif() ############################################################################################ # Dependencies -set(DEFAULT_CUDA_VERSION 11.3.1) -set(DEFAULT_CUDNN_VERSION 8.2) +set(DEFAULT_CUDA_VERSION 12.0.1) +set(DEFAULT_CUDNN_VERSION 8.8) set(DEFAULT_PROTOBUF_VERSION 3.20.1) # Dependency Version Resolution diff --git a/README.md b/README.md index e3585685..d31f2c4c 100644 --- a/README.md +++ b/README.md @@ -25,8 +25,8 @@ You can skip the **Build** section to enjoy TensorRT with Python. ## Prerequisites To build the TensorRT-OSS components, you will first need the following software packages. -**TensorRT EA build** -* [TensorRT](https://developer.nvidia.com/nvidia-tensorrt-download) v8.6.0.12 +**TensorRT GA build** +* [TensorRT](https://developer.nvidia.com/nvidia-tensorrt-download) v8.6.1.6 **System Packages** * [CUDA](https://developer.nvidia.com/cuda-toolkit) @@ -48,8 +48,8 @@ To build the TensorRT-OSS components, you will first need the following software * (Cross compilation for Jetson platform) [NVIDIA JetPack](https://developer.nvidia.com/embedded/jetpack) >= 5.0 (current support only for TensorRT 8.4.0 and TensorRT 8.5.2) * (Cross compilation for QNX platform) [QNX Toolchain](https://blackberry.qnx.com/en) * PyPI packages (for demo applications/tests) - * [onnx](https://pypi.org/project/onnx/) 1.9.0 - * [onnxruntime](https://pypi.org/project/onnxruntime/) 1.8.0 + * [onnx](https://pypi.org/project/onnx/) + * [onnxruntime](https://pypi.org/project/onnxruntime/) * [tensorflow-gpu](https://pypi.org/project/tensorflow/) >= 2.5.1 * [Pillow](https://pypi.org/project/Pillow/) >= 9.0.1 * [pycuda](https://pypi.org/project/pycuda/) < 2021.1 @@ -70,18 +70,18 @@ To build the TensorRT-OSS components, you will first need the following software git submodule update --init --recursive ``` -2. #### (Optional - if not using TensorRT container) Specify the TensorRT EA release build path +2. #### (Optional - if not using TensorRT container) Specify the TensorRT GA release build path If using the TensorRT OSS build container, TensorRT libraries are preinstalled under `/usr/lib/x86_64-linux-gnu` and you may skip this step. - Else download and extract the TensorRT EA build from [NVIDIA Developer Zone](https://developer.nvidia.com/nvidia-tensorrt-download). + Else download and extract the TensorRT GA build from [NVIDIA Developer Zone](https://developer.nvidia.com/nvidia-tensorrt-download). **Example: Ubuntu 20.04 on x86-64 with cuda-12.0** ```bash cd ~/Downloads - tar -xvzf TensorRT-8.6.0.12.Linux.x86_64-gnu.cuda-12.0.tar.gz - export TRT_LIBPATH=`pwd`/TensorRT-8.6.0.12 + tar -xvzf TensorRT-8.6.1.6.Linux.x86_64-gnu.cuda-12.0.tar.gz + export TRT_LIBPATH=`pwd`/TensorRT-8.6.1.6 ``` @@ -111,9 +111,9 @@ For Linux platforms, we recommend that you generate a docker container for build ```bash ./docker/build.sh --file docker/ubuntu-cross-aarch64.Dockerfile --tag tensorrt-jetpack-cuda11.4 ``` - **Example: Ubuntu 20.04 on aarch64 with cuda-11.4.2** + **Example: Ubuntu 20.04 on aarch64 with cuda-11.8** ```bash - ./docker/build.sh --file docker/ubuntu-20.04-aarch64.Dockerfile --tag tensorrt-aarch64-ubuntu20.04-cuda11.4 + ./docker/build.sh --file docker/ubuntu-20.04-aarch64.Dockerfile --tag tensorrt-aarch64-ubuntu20.04-cuda11.8 --cuda 11.8.0 ``` 2. #### Launch the TensorRT-OSS build container. @@ -143,7 +143,7 @@ For Linux platforms, we recommend that you generate a docker container for build yum -y install centos-release-scl yum-config-manager --enable rhel-server-rhscl-7-rpms yum -y install devtoolset-8 - export PATH="/opt/rh/devtoolset-8/root/bin:${PATH} + export PATH="/opt/rh/devtoolset-8/root/bin:${PATH}" ``` **Example: Linux (aarch64) build with default cuda-12.0** @@ -174,14 +174,14 @@ For Linux platforms, we recommend that you generate a docker container for build > NOTE: The latest JetPack SDK v5.1 only supports TensorRT 8.5.2. > NOTE: -
1. The default CUDA version used by CMake is 11.8.0. To override this, for example to 10.2, append `-DCUDA_VERSION=10.2` to the cmake command. +
1. The default CUDA version used by CMake is 12.0.1. To override this, for example to 11.8, append `-DCUDA_VERSION=11.8` to the cmake command.
2. If samples fail to link on CentOS7, create this symbolic link: `ln -s $TRT_OUT_DIR/libnvinfer_plugin.so $TRT_OUT_DIR/libnvinfer_plugin.so.8` * Required CMake build arguments are: - `TRT_LIB_DIR`: Path to the TensorRT installation directory containing libraries. - `TRT_OUT_DIR`: Output directory where generated build artifacts will be copied. * Optional CMake build arguments: - `CMAKE_BUILD_TYPE`: Specify if binaries generated are for release or debug (contain debug symbols). Values consists of [`Release`] | `Debug` - - `CUDA_VERISON`: The version of CUDA to target, for example [`11.7.1`]. + - `CUDA_VERSION`: The version of CUDA to target, for example [`11.7.1`]. - `CUDNN_VERSION`: The version of cuDNN to target, for example [`8.6`]. - `PROTOBUF_VERSION`: The version of Protobuf to use, for example [`3.0.0`]. Note: Changing this will not configure CMake to use a system version of Protobuf, it will configure CMake to download and try building that version. - `CMAKE_TOOLCHAIN_FILE`: The path to a toolchain file for cross compilation. diff --git a/VERSION b/VERSION index 2f889e47..811e1c1d 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -8.6.0.12 +8.6.1.6 diff --git a/cmake/toolchains/cmake_aarch64-native.toolchain b/cmake/toolchains/cmake_aarch64-native.toolchain index b15f024c..fd4e30cc 100644 --- a/cmake/toolchains/cmake_aarch64-native.toolchain +++ b/cmake/toolchains/cmake_aarch64-native.toolchain @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/BERT/CMakeLists.txt b/demo/BERT/CMakeLists.txt index ef5cffbb..cc2c8fc9 100644 --- a/demo/BERT/CMakeLists.txt +++ b/demo/BERT/CMakeLists.txt @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/BERT/README.md b/demo/BERT/README.md old mode 100644 new mode 100755 index 6f2f6aa5..ff55e1b0 --- a/demo/BERT/README.md +++ b/demo/BERT/README.md @@ -64,7 +64,7 @@ Since the tokenizer and projection of the final predictions are not nearly as co The tokenizer splits the input text into tokens that can be consumed by the model. For details on this process, see [this tutorial](https://mccormickml.com/2019/05/14/BERT-word-embeddings-tutorial/). -To run the BERT model in TensorRT, we construct the model using TensorRT APIs and import the weights from a pre-trained TensorFlow checkpoint from [NGC](https://ngc.nvidia.com/models/nvidian:bert_tf_v2_large_fp16_128). Finally, a TensorRT engine is generated and serialized to the disk. The various inference scripts then load this engine for inference. +To run the BERT model in TensorRT, we construct the model using TensorRT APIs and import the weights from a pre-trained TensorFlow checkpoint from [NGC](https://catalog.ngc.nvidia.com/orgs/nvidia/models/bert_tf_ckpt_large_qa_squad2_amp_128). Finally, a TensorRT engine is generated and serialized to the disk. The various inference scripts then load this engine for inference. Lastly, the tokens predicted by the model are projected back to the original text to get a final result. @@ -586,3 +586,4 @@ Results were obtained by running `scripts/inference_benchmark.sh --gpu Ampere` o | 384 | 32 | 40.79 | 40.97 | 40.46 | | 384 | 64 | 78.04 | 78.41 | 77.51 | | 384 | 128 | 151.33 | 151.62 | 150.76 | + diff --git a/demo/BERT/builder.py b/demo/BERT/builder.py index f9b49901..c6d15d00 100755 --- a/demo/BERT/builder.py +++ b/demo/BERT/builder.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/BERT/builder_utils.py b/demo/BERT/builder_utils.py index accef397..248bee80 100644 --- a/demo/BERT/builder_utils.py +++ b/demo/BERT/builder_utils.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -15,7 +15,7 @@ # limitations under the License. # -import re +import re import pickle import numpy as np @@ -161,7 +161,7 @@ def onnx_to_trt_name(onnx_name): if toks[0] == 'bert': #embeddings or encoder if toks[1] == 'encoder': #transformer # Token conversions for sparse checkpoints - if toks[-2] == 'dense_act': + if toks[-2] == 'dense_act': toks[-2] = 'dense' elif toks[-3] == 'dense_act': if toks[-2] == 'input_quantizer': @@ -187,7 +187,7 @@ def onnx_to_trt_name(onnx_name): toks[-2] = 'kernel' elif toks[-2] == 'input_quantizer': toks[-2] = 'input' - + if 'final_input_quantizer' not in toks[2]: ind = toks.index('layers')+1 if 'layers' in toks else 3 toks = toks[ind:] @@ -229,14 +229,14 @@ def get_onnx_weight_dict(tensor_dict, config): Bqkv[0,:] = tensor Bqkv[1,:] = tensor_dict[prefix + BK] Bqkv[2,:] = tensor_dict[prefix + BV] - + if config.use_int8 and getattr(config, 'interleaved', False): Wqkv = np.ascontiguousarray(Wqkv.reshape((3, N, H, N, H))) Bqkv = np.ascontiguousarray(Bqkv.reshape((3, N, H))) else: Wqkv = np.ascontiguousarray(Wqkv.reshape((3, N, H, N, H)).transpose((1,0,2,3,4))) Bqkv = np.ascontiguousarray(Bqkv.reshape((3, N, H)).transpose((1,0,2))) - + weights_dict[prefix + WQKV] = trt.Weights(Wqkv) weights_dict[prefix + BQKV] = trt.Weights(Bqkv) weights_dict[prefix + WQKV + "_notrans"] = trt.Weights(np.ascontiguousarray(Wqkv.T)) @@ -261,7 +261,7 @@ def load_onnx_weights_and_quant(path, config): model = onnx.load(path) weights = model.graph.initializer tensor_dict = dict((onnx_to_trt_name(w.name), np.frombuffer(w.raw_data, np.int8).reshape(w.dims)) - if w.name.split('_')[-1] == 'mask' else + if w.name.split('_')[-1] == 'mask' else (onnx_to_trt_name(w.name), np.frombuffer(w.raw_data, np.float32).reshape(w.dims)) for w in weights) return get_onnx_weight_dict(tensor_dict, config) diff --git a/demo/BERT/builder_varseqlen.py b/demo/BERT/builder_varseqlen.py index b9afa5e8..0c1aeaac 100755 --- a/demo/BERT/builder_varseqlen.py +++ b/demo/BERT/builder_varseqlen.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -385,11 +385,11 @@ def emb_layernorm(builder, network, config, weights_dict, builder_config, max_se if config.use_int8 and config.use_qat: dr_input = weights_dict['l0_attention_self_query_input_amax'] set_output_range(emb_layer, dr_input, out_idx=0) - + if config.use_megatron: dr_skln1_res_in = weights_dict['l0_attention_output_add_residual_input_quantizer_amax'] set_output_range(emb_layer, dr_skln1_res_in, out_idx=1) - + set_output_name(emb_layer, "embeddings_", "output") return emb_layer, cu_seqlens, max_seqlen @@ -409,7 +409,7 @@ def build_engine(batch_sizes, workspace_size, sequence_length, config, weights_d if verbose: builder_config.profiling_verbosity = trt.ProfilingVerbosity.DETAILED - # speed up the engine build for trt major version >= 8 + # speed up the engine build for trt major version >= 8 # 1. disable cudnn tactic # 2. load global timing cache if trt_version[0] >= 8: @@ -438,7 +438,7 @@ def build_engine(batch_sizes, workspace_size, sequence_length, config, weights_d mask_idx = None else: mask_idx = emb_layer.get_output(1) - + if config.use_megatron: # megatron currently only supports int8 and interleaved shuffler = network.add_shuffle(emb_layer.get_output(1)) shuffler.second_transpose = (2, 1, 0, 3) @@ -503,8 +503,8 @@ def main(): cc = pycuda.autoinit.device.compute_capability() if cc[0] * 10 + cc[1] < 72: raise RuntimeError("This variable-length BERT demo only support Xavier+ GPU.") - - if args.megatron: + + if args.megatron: if not (args.interleaved and args.int8): raise RuntimeError("Megatron BERT currently only supports int8 and interleaved.") if not args.pickle: diff --git a/demo/BERT/helpers/calibrator.py b/demo/BERT/helpers/calibrator.py index 1ebf0793..beacc625 100644 --- a/demo/BERT/helpers/calibrator.py +++ b/demo/BERT/helpers/calibrator.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/BERT/helpers/data_processing.py b/demo/BERT/helpers/data_processing.py index dd7d1733..88459ebf 100644 --- a/demo/BERT/helpers/data_processing.py +++ b/demo/BERT/helpers/data_processing.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/BERT/helpers/tokenization.py b/demo/BERT/helpers/tokenization.py index 41186ea7..434f411d 100644 --- a/demo/BERT/helpers/tokenization.py +++ b/demo/BERT/helpers/tokenization.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/BERT/infer_c/bert_infer.h b/demo/BERT/infer_c/bert_infer.h index db1e1352..827f9ba9 100644 --- a/demo/BERT/infer_c/bert_infer.h +++ b/demo/BERT/infer_c/bert_infer.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/BERT/infer_c/common.h b/demo/BERT/infer_c/common.h index b20d993a..b5280e2a 100644 --- a/demo/BERT/infer_c/common.h +++ b/demo/BERT/infer_c/common.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/BERT/infer_c/infer_c.cpp b/demo/BERT/infer_c/infer_c.cpp index 7ccc1c8f..b868a661 100644 --- a/demo/BERT/infer_c/infer_c.cpp +++ b/demo/BERT/infer_c/infer_c.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/BERT/infer_c/logging.cpp b/demo/BERT/infer_c/logging.cpp index 748a8b3b..b6b14298 100644 --- a/demo/BERT/infer_c/logging.cpp +++ b/demo/BERT/infer_c/logging.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/BERT/infer_c/logging.h b/demo/BERT/infer_c/logging.h index 0fad4642..2c36d039 100644 --- a/demo/BERT/infer_c/logging.h +++ b/demo/BERT/infer_c/logging.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/BERT/infer_c/perf.cpp b/demo/BERT/infer_c/perf.cpp index 365006f7..bbc6de76 100644 --- a/demo/BERT/infer_c/perf.cpp +++ b/demo/BERT/infer_c/perf.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/BERT/inference.py b/demo/BERT/inference.py index e4742d0b..2116de8f 100644 --- a/demo/BERT/inference.py +++ b/demo/BERT/inference.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/BERT/inference_c.py b/demo/BERT/inference_c.py index 3022b3dc..e2bda9af 100644 --- a/demo/BERT/inference_c.py +++ b/demo/BERT/inference_c.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/BERT/inference_varseqlen.py b/demo/BERT/inference_varseqlen.py index e5205f38..9cd08519 100644 --- a/demo/BERT/inference_varseqlen.py +++ b/demo/BERT/inference_varseqlen.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/BERT/perf.py b/demo/BERT/perf.py index 541337e4..5943b41b 100644 --- a/demo/BERT/perf.py +++ b/demo/BERT/perf.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/BERT/perf_varseqlen.py b/demo/BERT/perf_varseqlen.py index 48af2ce1..a1680797 100644 --- a/demo/BERT/perf_varseqlen.py +++ b/demo/BERT/perf_varseqlen.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/BERT/squad/evaluate-v1.1.py b/demo/BERT/squad/evaluate-v1.1.py index d2c5ecc9..c73db423 100644 --- a/demo/BERT/squad/evaluate-v1.1.py +++ b/demo/BERT/squad/evaluate-v1.1.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/BERT/squad/evaluate-v2.0.py b/demo/BERT/squad/evaluate-v2.0.py index 4ec1b887..e36d3e9f 100644 --- a/demo/BERT/squad/evaluate-v2.0.py +++ b/demo/BERT/squad/evaluate-v2.0.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -183,7 +183,7 @@ def make_precision_recall_eval(scores, na_probs, num_true_pos, qid_to_has_ans, plot_pr_curve(precisions, recalls, out_image, title) return {'ap': 100.0 * avg_prec} -def run_precision_recall_analysis(main_eval, exact_raw, f1_raw, na_probs, +def run_precision_recall_analysis(main_eval, exact_raw, f1_raw, na_probs, qid_to_has_ans, out_image_dir): if out_image_dir and not os.path.exists(out_image_dir): os.makedirs(out_image_dir) @@ -277,7 +277,7 @@ def main(): if OPTS.na_prob_file: find_all_best_thresh(out_eval, preds, exact_raw, f1_raw, na_probs, qid_to_has_ans) if OPTS.na_prob_file and OPTS.out_image_dir: - run_precision_recall_analysis(out_eval, exact_raw, f1_raw, na_probs, + run_precision_recall_analysis(out_eval, exact_raw, f1_raw, na_probs, qid_to_has_ans, OPTS.out_image_dir) histogram_na_prob(na_probs, has_ans_qids, OPTS.out_image_dir, 'hasAns') histogram_na_prob(na_probs, no_ans_qids, OPTS.out_image_dir, 'noAns') @@ -292,5 +292,5 @@ def main(): if OPTS.out_image_dir: import matplotlib matplotlib.use('Agg') - import matplotlib.pyplot as plt + import matplotlib.pyplot as plt main() diff --git a/demo/DeBERTa/deberta_onnx_modify.py b/demo/DeBERTa/deberta_onnx_modify.py index 4c61dd6e..234c4659 100644 --- a/demo/DeBERTa/deberta_onnx_modify.py +++ b/demo/DeBERTa/deberta_onnx_modify.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -52,7 +52,7 @@ if args.output is None: model_output = os.path.splitext(model_input)[0] + ("_plugin" if use_plugin else "_original") + os.path.splitext(model_input)[-1] else: - model_output = args.output + model_output = args.output def remove_uint8_cast(graph): ''' @@ -72,7 +72,7 @@ def remove_uint8_cast(graph): # node.attrs["to"] = TensorProto.INT64 return graph - + @gs.Graph.register() def insert_disentangled_attention(self, inputs, outputs, factor, span): ''' @@ -80,7 +80,7 @@ def insert_disentangled_attention(self, inputs, outputs, factor, span): inputs: list of plugin inputs outputs: list of plugin outputs - factor: scaling factor of disentangled attention, sqrt(3d), converted from a division factor to a multiplying factor + factor: scaling factor of disentangled attention, sqrt(3d), converted from a division factor to a multiplying factor span: relative distance span, k ''' # disconnect previous output from flow (the previous subgraph still exists but is effectively dead since it has no link to an output tensor, and thus will be cleaned up) @@ -102,21 +102,21 @@ def insert_disentangled_attention_all(graph): layers = [(nodes[2*i+0], nodes[2*i+1]) for i in range(len(nodes)//2)] # 2 gatherelements in 1 layer for l, (left,right) in enumerate(layers): print(f"Fusing layer {l}") - + # CAVEAT! MUST cast to list() when setting the inputs & outputs. graphsurgeon's default for X.inputs and X.outputs is `onnx_graphsurgeon.util.misc.SynchronizedList`, i.e. 2-way node-tensor updating mechanism. If not cast, when we remove the input nodes of a tensor, the tensor itself will be removed as well... - + # inputs: (data0, data1, data2), input tensors for c2c add and 2 gathers inputs = list(left.o().o().o().o().i().inputs)[0:1] + list(left.inputs)[0:1] + list(right.inputs)[0:1] - + # outputs: (result), output tensors after adding 3 gather results outputs = list(left.o().o().o().o().outputs) - + # constants: scaling factor, relative distance span factor = left.o().inputs[1].inputs[0].attrs["value"].values.item() span = right.i(1,0).i().i().i().inputs[1].inputs[0].attrs["value"].values.item() - # insert plugin layer - graph.insert_disentangled_attention(inputs, outputs, factor, span) + # insert plugin layer + graph.insert_disentangled_attention(inputs, outputs, factor, span) return graph @@ -142,7 +142,7 @@ def correctness_check_models(graph): end_node.outputs[0].dtype = graph_raw.outputs[0].dtype # need to explicitly specify dtype and shape of graph output tensor end_node.outputs[0].shape = ['batch_size*num_heads', seq_len, seq_len] original_output_all.append(end_node.outputs[0]) - + graph_raw.outputs = graph_raw.outputs + original_output_all # add plugin outputs to graph output ## for modified graph with plugin @@ -165,13 +165,13 @@ def correctness_check_models(graph): factor = left.o().inputs[1].inputs[0].attrs["value"].values.item() span = right.i(1,0).i().i().i().inputs[1].inputs[0].attrs["value"].values.item() - # insert plugin layer - graph.insert_disentangled_attention(inputs, outputs, factor, span) + # insert plugin layer + graph.insert_disentangled_attention(inputs, outputs, factor, span) graph.outputs = graph.outputs + plugin_output_all # add plugin outputs to graph output return graph_raw, graph - + def check_model(model_name): # Load the ONNX model model = onnx.load(model_name) @@ -200,7 +200,7 @@ def check_model(model_name): # don't check model because 'DisentangledAttention_TRT' is not a registered op -elif correctness_check: +elif correctness_check: # correctness check, save two models (original and w/ plugin) with intermediate output nodes inserted graph_raw, graph = correctness_check_models(graph) @@ -213,7 +213,7 @@ def check_model(model_name): model_output2 = os.path.splitext(model_input)[0] + "_correctness_check_plugin" + os.path.splitext(model_input)[-1] onnx.save_model(gs.export_onnx(graph_raw), model_output1) onnx.save_model(gs.export_onnx(graph), model_output2) - + print(f"Saving models for correctness check to {model_output1} (original) and {model_output2} (with plugin)") check_model(model_output1) diff --git a/demo/DeBERTa/deberta_ort_inference.py b/demo/DeBERTa/deberta_ort_inference.py index 24bf966c..17378989 100644 --- a/demo/DeBERTa/deberta_ort_inference.py +++ b/demo/DeBERTa/deberta_ort_inference.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -18,21 +18,21 @@ """ Test ORT-TRT engine of DeBERTa model. Different precisions are supported. -Usage: +Usage: Test model inference time: - python deberta_ort_inference.py --onnx=./test/deberta.onnx --test fp16 Correctness check by comparing original model and model with plugin: - python deberta_ort_inference.py --onnx=./test/deberta --correctness-check fp16 -Notes: +Notes: - supported precisions are fp32/fp16. For test, you can specify more than one precisions, and TensorRT engine of each precision will be built sequentially. - - engine files are saved at `./engine_cache/[Model name]_[GPU name]_[Precision]/`. Note that TensorRT engine is specific to both GPU architecture and TensorRT version. + - engine files are saved at `./engine_cache/[Model name]_[GPU name]_[Precision]/`. Note that TensorRT engine is specific to both GPU architecture and TensorRT version. - if in --correctness-check mode, the argument for --onnx is the stem name for the model without .onnx extension. """ -import os, argparse -import onnxruntime as ort +import os, argparse +import onnxruntime as ort import numpy as np import torch from time import time @@ -44,10 +44,10 @@ def GPU_ABBREV(name): ''' Map GPU device query name to abbreviation. - + ::param str name Device name from torch.cuda.get_device_name(). ::return str GPU abbreviation. - ''' + ''' GPU_LIST = [ 'V100', @@ -56,13 +56,13 @@ def GPU_ABBREV(name): 'A100', 'A10G', 'A10' - ] + ] # Partial list, can be extended. The order of A100, A10G, A10 matters. They're put in a way to not detect substring A10 as A100 - + for i in GPU_LIST: if i in name: - return i - + return i + return 'GPU' # for names not in the partial list, use 'GPU' as default gpu_name = GPU_ABBREV(torch.cuda.get_device_name()) @@ -79,7 +79,7 @@ def GPU_ABBREV(name): args = parser.parse_args() -ONNX_MODEL = args.onnx +ONNX_MODEL = args.onnx MODEL_STEM = os.path.splitext(args.onnx)[0].split('/')[-1] TEST = args.test CORRECTNESS = args.correctness_check @@ -96,8 +96,8 @@ def GPU_ABBREV(name): def test_engine(): for precision in TEST: - - engine_cachepath = '/'.join([ENGINE_PATH, '_'.join([MODEL_STEM, gpu_name, precision, 'ort'])]) + + engine_cachepath = '/'.join([ENGINE_PATH, '_'.join([MODEL_STEM, gpu_name, precision, 'ort'])]) providers = [ ('TensorrtExecutionProvider', { @@ -110,7 +110,7 @@ def test_engine(): so = ort.SessionOptions() - sess = ort.InferenceSession(ONNX_MODEL, sess_options=so, providers=providers) + sess = ort.InferenceSession(ONNX_MODEL, sess_options=so, providers=providers) print(f'Running inference on engine {engine_cachepath}') @@ -121,7 +121,7 @@ def test_engine(): input_ids = torch.randint(0, vocab, (batch_size, seq_len), dtype=torch.long) attention_mask = torch.randint(0, 2, (batch_size, seq_len), dtype=torch.long) inputs = { - 'input_ids': input_ids.numpy(), + 'input_ids': input_ids.numpy(), 'attention_mask': attention_mask.numpy() } @@ -137,7 +137,7 @@ def test_engine(): print(f'Average Inference time (ms) of {nreps} runs: {duration/nreps*1000:.3f}. For more accurate test, please use the onnxruntime_perf_test commands.') def correctness_check_engines(): - + for precision in CORRECTNESS: engine_cachepath1 = '/'.join([ENGINE_PATH, '_'.join([MODEL_STEM, 'original', gpu_name, precision, 'ort'])]) @@ -145,7 +145,7 @@ def correctness_check_engines(): if not os.path.exists(engine_cachepath1) or not os.path.exists(engine_cachepath2): print('At least one of the original and/or plugin engines do not exist. Please build them first by --test') - return + return print(f'Running inference on original engine {engine_cachepath1} and plugin engine {engine_cachepath2}') @@ -158,7 +158,7 @@ def correctness_check_engines(): 'trt_engine_cache_enable': True, 'trt_engine_cache_path': engine_cachepath1 }), - 'CUDAExecutionProvider'] + 'CUDAExecutionProvider'] providers2 = [ ('TensorrtExecutionProvider', { @@ -167,7 +167,7 @@ def correctness_check_engines(): 'trt_engine_cache_enable': True, 'trt_engine_cache_path': engine_cachepath2 }), - 'CUDAExecutionProvider'] + 'CUDAExecutionProvider'] sess1 = ort.InferenceSession(ONNX_MODEL+'_original.onnx', sess_options=so, providers=providers1) sess2 = ort.InferenceSession(ONNX_MODEL+'_plugin.onnx', sess_options=so, providers=providers2) @@ -179,7 +179,7 @@ def correctness_check_engines(): input_ids = torch.randint(0, vocab, (batch_size, seq_len), dtype=torch.long) attention_mask = torch.randint(0, 2, (batch_size, seq_len), dtype=torch.long) inputs = { - 'input_ids': input_ids.numpy(), + 'input_ids': input_ids.numpy(), 'attention_mask': attention_mask.numpy() } diff --git a/demo/DeBERTa/deberta_pytorch2onnx.py b/demo/DeBERTa/deberta_pytorch2onnx.py index 74c3ccbb..51546b29 100644 --- a/demo/DeBERTa/deberta_pytorch2onnx.py +++ b/demo/DeBERTa/deberta_pytorch2onnx.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -22,8 +22,8 @@ python deberta_pytorch2onnx.py [--filename xx.onnx] [--variant microsoft/deberta-xx] [--seq-len xx] ''' -import os, time, argparse -from transformers import DebertaV2Tokenizer, DebertaV2Config, DebertaV2ForSequenceClassification +import os, time, argparse +from transformers import DebertaV2Tokenizer, DebertaV2Config, DebertaV2ForSequenceClassification # DEBERTA V2 implementation, https://github.com/huggingface/transformers/blob/master/src/transformers/models/deberta_v2/modeling_deberta_v2.py import torch, onnxruntime as ort, numpy as np @@ -34,14 +34,14 @@ args = parser.parse_args() onnx_filename = args.filename -model_variant = args.variant +model_variant = args.variant sequence_length = args.seq_len assert not args.variant or (args.variant and not args.seq_len), "--variant and --seq-len cannot be used together!" assert torch.cuda.is_available(), "CUDA not available!" def randomize_model(model): - for module_ in model.named_modules(): + for module_ in model.named_modules(): if isinstance(module_[1],(torch.nn.Linear, torch.nn.Embedding)): module_[1].weight.data.normal_(mean=0.0, std=model.config.initializer_range) elif isinstance(module_[1], torch.nn.LayerNorm): @@ -55,7 +55,7 @@ def export(): parent_dir = os.path.dirname(onnx_filename) if not os.path.exists(parent_dir): os.makedirs(parent_dir) - + if model_variant is None: # default model hyper-params batch_size = 1 @@ -71,7 +71,7 @@ def export(): relative_attention=True max_relative_positions = 256 # k pos_att_type = ["p2c", "c2p"] - + deberta_config = DebertaV2Config(vocab_size=vocab_size, hidden_size=hidden_size, num_hidden_layers=layers, num_attention_heads=heads, intermediate_size=intermediate_size, type_vocab_size=type_vocab_size, max_position_embeddings=max_position_embeddings, relative_attention=relative_attention, max_relative_positions=max_relative_positions, pos_att_type=pos_att_type) deberta_model = DebertaV2ForSequenceClassification(deberta_config) deberta_model = randomize_model(deberta_model) @@ -82,21 +82,21 @@ def export(): batch_size = 1 seq_len = deberta_config.max_position_embeddings vocab_size = deberta_config.vocab_size - + deberta_model.cuda().eval() # input/output gpu = torch.device('cuda') input_ids = torch.randint(0, vocab_size, (batch_size, seq_len), dtype=torch.long, device=gpu) attention_mask = torch.randint(0, 2, (batch_size, seq_len), dtype=torch.long, device=gpu) - input_names = ['input_ids', 'attention_mask'] + input_names = ['input_ids', 'attention_mask'] output_names = ['output'] - dynamic_axes={'input_ids' : {0 : 'batch_size'}, - 'attention_mask' : {0 : 'batch_size'}, + dynamic_axes={'input_ids' : {0 : 'batch_size'}, + 'attention_mask' : {0 : 'batch_size'}, 'output' : {0 : 'batch_size'}} - + # ONNX export - torch.onnx.export(deberta_model, # model + torch.onnx.export(deberta_model, # model (input_ids, attention_mask), # model inputs onnx_filename, export_params=True, @@ -105,7 +105,7 @@ def export(): input_names = input_names, output_names = output_names, dynamic_axes = dynamic_axes) - + # full precision inference num_trials = 10 @@ -115,7 +115,7 @@ def export(): end = time.time() print("Average PyTorch FP32(TF32) time: {:.2f} ms".format((end - start)/num_trials*1000)) - + # half precision inference (do this after onnx export, otherwise the export ONNX model is with FP16 weights...) deberta_model_fp16 = deberta_model.half() start = time.time() diff --git a/demo/DeBERTa/deberta_tensorrt_inference.py b/demo/DeBERTa/deberta_tensorrt_inference.py index b17806a8..6a579a1c 100644 --- a/demo/DeBERTa/deberta_tensorrt_inference.py +++ b/demo/DeBERTa/deberta_tensorrt_inference.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -18,7 +18,7 @@ """ Build and test TensorRT engines generated from the DeBERTa model. Different precisions are supported. -Usage: +Usage: Build and test a model: - build: python deberta_tensorrt_inference.py --onnx=xx.onnx --build fp16 # build TRT engines - test: python deberta_tensorrt_inference.py --onnx=xx.onnx --test fp16 # test will measure the inference time @@ -30,15 +30,15 @@ - [3] build plugin model: python deberta_tensorrt_inference.py --onnx=xx_correctness_check_plugin.onnx --build fp16 - [4] correctness check: python deberta_tensorrt_inference.py --onnx=deberta --correctness_check fp16 -Notes: +Notes: - supported precisions are fp32/tf32/fp16. For both --build and --test, you can specify more than one precisions, and TensorRT engines of each precision will be built sequentially. - - engine files are saved as `**/[Model name]_[GPU name]_[Precision].engine`. Note that TensorRT engines are specific to both GPU architecture and TensorRT version, and therefore are not compatible cross-version nor cross-device. + - engine files are saved as `**/[Model name]_[GPU name]_[Precision].engine`. Note that TensorRT engines are specific to both GPU architecture and TensorRT version, and therefore are not compatible cross-version nor cross-device. - in --correctness-check mode, the argument for --onnx is the `root` name for the models [root]_correctness_check_original/plugin.onnx """ import torch import tensorrt as trt -import os, sys, argparse +import os, sys, argparse import numpy as np import pycuda.driver as cuda import pycuda.autoinit # without this, "LogicError: explicit_context_dependent failed: invalid device context - no currently active context?" @@ -49,10 +49,10 @@ def GPU_ABBREV(name): ''' Map GPU device query name to abbreviation. - + ::param str name Device name from torch.cuda.get_device_name(). ::return str GPU abbreviation. - ''' + ''' GPU_LIST = [ 'V100', @@ -61,13 +61,13 @@ def GPU_ABBREV(name): 'A100', 'A10G', 'A10' - ] + ] # Partial list, can be extended. The order of A100, A10G, A10 matters. They're put in a way to not detect substring A10 as A100 - + for i in GPU_LIST: if i in name: - return i - + return i + return 'GPU' # for names not in the partial list, use 'GPU' as default gpu_name = GPU_ABBREV(torch.cuda.get_device_name()) @@ -86,7 +86,7 @@ def GPU_ABBREV(name): args = parser.parse_args() -ONNX_MODEL = args.onnx +ONNX_MODEL = args.onnx MODEL_NAME = os.path.splitext(args.onnx)[0] BUILD = args.build TEST = args.test @@ -127,7 +127,7 @@ def __repr__(self): return self.__str__() def __init__(self, engine_path): - self.engine_path = engine_path + self.engine_path = engine_path self.logger = trt.Logger(trt.Logger.WARNING) self.runtime = trt.Runtime(self.logger) @@ -159,7 +159,7 @@ def load_engine(self): with open(self.engine_path, 'rb') as f: engine = self.runtime.deserialize_cuda_engine(f.read()) return engine - + def allocate_buffers(self, engine): ''' Allocates all buffers required for an engine, i.e. host/device inputs/outputs. @@ -172,7 +172,7 @@ def allocate_buffers(self, engine): for binding in engine: # binding is the name of input/output size = trt.volume(engine.get_binding_shape(binding)) * engine.max_batch_size dtype = trt.nptype(engine.get_binding_dtype(binding)) - + # Allocate host and device buffers host_mem = cuda.pagelocked_empty(size, dtype) # page-locked memory buffer (won't swapped to disk) device_mem = cuda.mem_alloc(host_mem.nbytes) @@ -202,7 +202,7 @@ def __call__(self, model_inputs: list, timing=False): TORCH = True else: assert False, 'Unsupported input data format!' - + # batch size consistency check if NUMPY: batch_size = np.unique(np.array([i.shape[0] for i in model_inputs])) @@ -226,18 +226,18 @@ def __call__(self, model_inputs: list, timing=False): np.copyto(self.inputs[i].host, model_input.ravel()) elif TORCH: if timing: - cuda.memcpy_dtod(self.inputs[i].device, model_input.data_ptr(), model_input.element_size() * model_input.nelement()) + cuda.memcpy_dtod(self.inputs[i].device, model_input.data_ptr(), model_input.element_size() * model_input.nelement()) else: # for Torch GPU tensor it's easier, can just do Device to Device copy cuda.memcpy_dtod_async(self.inputs[i].device, model_input.data_ptr(), model_input.element_size() * model_input.nelement(), self.stream) # dtod need size in bytes - if NUMPY: + if NUMPY: if timing: [cuda.memcpy_htod(inp.device, inp.host) for inp in self.inputs] else: # input, Host to Device [cuda.memcpy_htod_async(inp.device, inp.host, self.stream) for inp in self.inputs] - + duration = 0 if timing: start_time = time() @@ -258,7 +258,7 @@ def __call__(self, model_inputs: list, timing=False): # synchronize to ensure completion of async calls self.stream.synchronize() - if NUMPY: + if NUMPY: return [out.host.reshape(batch_size,-1) for out in self.outputs], duration elif TORCH: return [torch.from_numpy(out.host.reshape(batch_size,-1)) for out in self.outputs], duration @@ -284,19 +284,19 @@ def build_engine(): print(onnx_parser.get_error(idx)) if not parse_success: sys.exit('ONNX model parsing failed') - + ## build TRT engine (configuration options at: https://docs.nvidia.com/deeplearning/tensorrt/api/python_api/infer/Core/BuilderConfig.html#ibuilderconfig) config = TRT_BUILDER.create_builder_config() - + seq_len = network.get_input(0).shape[1] - + # handle dynamic shape (min/opt/max): https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#work_dynamic_shapes # by default batch dim set as 1 for all min/opt/max. If there are batch need, change the value for opt and max accordingly - profile = TRT_BUILDER.create_optimization_profile() - profile.set_shape("input_ids", (1,seq_len), (1,seq_len), (1,seq_len)) - profile.set_shape("attention_mask", (1,seq_len), (1,seq_len), (1,seq_len)) + profile = TRT_BUILDER.create_optimization_profile() + profile.set_shape("input_ids", (1,seq_len), (1,seq_len), (1,seq_len)) + profile.set_shape("attention_mask", (1,seq_len), (1,seq_len), (1,seq_len)) config.add_optimization_profile(profile) - + if TRT_VERSION >= 84: config.set_memory_pool_limit(trt.MemoryPoolType.WORKSPACE, 4096 * (1 << 20)) # 4096 MiB, syntax after TRT 8.4 else: @@ -312,7 +312,7 @@ def build_engine(): # build serialized_engine = TRT_BUILDER.build_serialized_network(network, config) - + ## save TRT engine with open(engine_filename, 'wb') as f: f.write(serialized_engine) @@ -326,7 +326,7 @@ def test_engine(): print(f'Running inference on engine {engine_filename}') model = TRTModel(engine_filename) - + ## psuedo-random input test batch_size = 1 seq_len = model.engine.get_binding_shape(0)[1] @@ -343,7 +343,7 @@ def test_engine(): for _ in range(nreps): outputs, duration = model(inputs, timing=True) duration_total += duration - + print(f'Average Inference time (ms) of {nreps} runs: {duration_total/nreps*1000:.3f}') def correctness_check_engines(): @@ -351,7 +351,7 @@ def correctness_check_engines(): ## load and deserialize TRT engine engine_filename1 = '_'.join([ONNX_MODEL, 'correctness_check_original', gpu_name, precision]) + '.engine' engine_filename2 = '_'.join([ONNX_MODEL, 'correctness_check_plugin', gpu_name, precision]) + '.engine' - + assert os.path.exists(engine_filename1), f'Engine file {engine_filename1} does not exist. Please build the engine first by --build' assert os.path.exists(engine_filename2), f'Engine file {engine_filename2} does not exist. Please build the engine first by --build' @@ -359,7 +359,7 @@ def correctness_check_engines(): model1 = TRTModel(engine_filename1) model2 = TRTModel(engine_filename2) - + ## psuedo-random input test batch_size = 1 seq_len = model1.engine.get_binding_shape(0)[1] @@ -369,7 +369,7 @@ def correctness_check_engines(): input_ids = torch.randint(0, vocab, (batch_size, seq_len), dtype=torch.long, device=gpu) attention_mask = torch.randint(0, 2, (batch_size, seq_len), dtype=torch.long, device=gpu) inputs = [input_ids, attention_mask] - + outputs1, _ = model1(inputs) outputs2, _ = model2(inputs) diff --git a/demo/DeBERTa/requirements.txt b/demo/DeBERTa/requirements.txt index 5046d50c..59b63433 100644 --- a/demo/DeBERTa/requirements.txt +++ b/demo/DeBERTa/requirements.txt @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Diffusion/README.md b/demo/Diffusion/README.md index ad9aa0d2..daaeeb09 100644 --- a/demo/Diffusion/README.md +++ b/demo/Diffusion/README.md @@ -93,7 +93,10 @@ Use `--input-image=` to specify your image. Otherwise the example ### Generate an inpainted image guided by an image, mask and single text prompt ```bash -python3 demo_inpaint.py "a mecha robot sitting on a bench" --hf-token=$HF_TOKEN -v +# Create separate onnx/engine directories when switching versions +mkdir -p onnx-1.5 engine-1.5 + +python3 demo_inpaint.py "a mecha robot sitting on a bench" --hf-token=$HF_TOKEN --version=1.5 --onnx-dir=onnx-1.5 --engine-dir=engine-1.5 -v ``` Use `--input-image=` and `--mask-image=` to specify your inputs. They must have the same dimensions. Otherwise the example image and mask will be downloaded from the Internet. @@ -102,4 +105,4 @@ Use `--input-image=` and `--mask-image=` to specify - One can set schdeuler using `--scheduler=EulerA`. Note that some schedulers are not available for some pipelines or version. - To accelerate engine building time one can use `--timing-cache=`. This cache file will be created if does not exist. Note, that it may influence the performance if the cache file created on the other hardware is used. It is suggested to use this flag only during development. To achieve the best perfromance during deployment, please, build engines without timing cache. - To switch between versions or pipelines one needs either to clear onnx and engine dirs, or to specify `--force-onnx-export --force-onnx-optimize --force-engine-build` or to create new dirs and to specify `--onnx-dir= --engine-dir=`. - +- Inference performance can be improved by enabling [CUDA graphs](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#cuda-graphs) using `--use-cuda-graph`. Enabling CUDA graphs requires fixed input shapes, so this flag must be combined with `--build-static-batch` and cannot be combined with `--build-dynamic-shape`. diff --git a/demo/Diffusion/demo_img2img.py b/demo/Diffusion/demo_img2img.py index c15c099b..963babee 100755 --- a/demo/Diffusion/demo_img2img.py +++ b/demo/Diffusion/demo_img2img.py @@ -74,6 +74,9 @@ def parseArgs(): if batch_size > max_batch_size: raise ValueError(f"Batch size {len(prompt)} is larger than allowed {max_batch_size}. If dynamic shape is used, then maximum batch size is 4") + if args.use_cuda_graph and (not args.build_static_batch or args.build_dynamic_shape): + raise ValueError(f"Using CUDA graph requires static dimensions. Enable `--build-static-batch` and do not specify `--build-dynamic-shape`") + # Initialize demo demo = Img2ImgPipeline( scheduler=args.scheduler, @@ -95,6 +98,10 @@ def parseArgs(): timing_cache=args.timing_cache, onnx_refit_dir=args.onnx_refit_dir) demo.loadResources(image_height, image_width, batch_size, args.seed) + if args.use_cuda_graph: + # inference once to get cuda graph + images = demo.infer(prompt, negative_prompt, input_image, image_height, image_width, strength=0.75, warmup=True) + print("[I] Warming up ..") for _ in range(args.num_warmup_runs): images = demo.infer(prompt, negative_prompt, input_image, image_height, image_width, strength=0.75, warmup=True) diff --git a/demo/Diffusion/demo_inpaint.py b/demo/Diffusion/demo_inpaint.py index 27eedf2b..1fa8219a 100755 --- a/demo/Diffusion/demo_inpaint.py +++ b/demo/Diffusion/demo_inpaint.py @@ -34,7 +34,7 @@ def parseArgs(): print("[I] Initializing StableDiffusion inpainting demo using TensorRT") args = parseArgs() - # Inpainting is currently only supported for v1.5 and v2.1 + # Inpainting is currently only supported for v1.5 and v2.0 if args.version not in ("1.5", "2.0"): raise ValueError(f"Inpainting not supported in version {args.version}. Use v2.0, or v1.5") @@ -84,6 +84,9 @@ def parseArgs(): if batch_size > max_batch_size: raise ValueError(f"Batch size {len(prompt)} is larger than allowed {max_batch_size}. If dynamic shape is used, then maximum batch size is 4") + if args.use_cuda_graph and (not args.build_static_batch or args.build_dynamic_shape): + raise ValueError(f"Using CUDA graph requires static dimensions. Enable `--build-static-batch` and do not specify `--build-dynamic-shape`") + # Initialize demo demo = InpaintPipeline( scheduler=args.scheduler, @@ -105,6 +108,11 @@ def parseArgs(): timing_cache=args.timing_cache) demo.loadResources(image_height, image_width, batch_size, args.seed) + + if args.use_cuda_graph: + # inference once to get cuda graph + images = demo.infer(prompt, negative_prompt, input_image, mask_image, image_height, image_width, strength=0.75, warmup=True) + print("[I] Warming up ..") for _ in range(args.num_warmup_runs): images = demo.infer(prompt, negative_prompt, input_image, mask_image, image_height, image_width, strength=0.75, warmup=True) diff --git a/demo/Diffusion/demo_txt2img.py b/demo/Diffusion/demo_txt2img.py index 219e4792..4491c45e 100644 --- a/demo/Diffusion/demo_txt2img.py +++ b/demo/Diffusion/demo_txt2img.py @@ -61,6 +61,9 @@ def parseArgs(): if batch_size > max_batch_size: raise ValueError(f"Batch size {len(prompt)} is larger than allowed {max_batch_size}. If dynamic shape is used, then maximum batch size is 4") + if args.use_cuda_graph and (not args.build_static_batch or args.build_dynamic_shape): + raise ValueError(f"Using CUDA graph requires static dimensions. Enable `--build-static-batch` and do not specify `--build-dynamic-shape`") + # Initialize demo demo = Txt2ImgPipeline( scheduler=args.scheduler, @@ -70,7 +73,8 @@ def parseArgs(): hf_token=args.hf_token, verbose=args.verbose, nvtx_profile=args.nvtx_profile, - max_batch_size=max_batch_size) + max_batch_size=max_batch_size, + use_cuda_graph=args.use_cuda_graph) # Load TensorRT engines and pytorch modules demo.loadEngines(args.engine_dir, args.onnx_dir, args.onnx_opset, @@ -82,6 +86,10 @@ def parseArgs(): timing_cache=args.timing_cache, onnx_refit_dir=args.onnx_refit_dir) demo.loadResources(image_height, image_width, batch_size, args.seed) + if args.use_cuda_graph: + # inference once to get cuda graph + images = demo.infer(prompt, negative_prompt, image_height, image_width, warmup=True, verbose=False) + print("[I] Warming up ..") for _ in range(args.num_warmup_runs): images = demo.infer(prompt, negative_prompt, image_height, image_width, warmup=True, verbose=False) diff --git a/demo/Diffusion/inpaint_pipeline.py b/demo/Diffusion/inpaint_pipeline.py index 3931609f..3a1ade5a 100755 --- a/demo/Diffusion/inpaint_pipeline.py +++ b/demo/Diffusion/inpaint_pipeline.py @@ -42,7 +42,7 @@ def __init__( if scheduler != "PNDM": raise ValueError(f"Inpainting only supports PNDM scheduler") - + super(InpaintPipeline, self).__init__(*args, **kwargs, \ inpaint=True, scheduler=scheduler, stages=[ 'vae_encoder', 'clip', 'unet', 'vae']) @@ -132,3 +132,4 @@ def infer( if not warmup: self.print_summary(self.denoising_steps, e2e_tic, e2e_toc, vae_enc=True) self.save_image(images, 'inpaint', prompt) + diff --git a/demo/Diffusion/models.py b/demo/Diffusion/models.py index 3a5c4947..bcf69b32 100644 --- a/demo/Diffusion/models.py +++ b/demo/Diffusion/models.py @@ -383,7 +383,7 @@ def __init__(self, token, device, path): super().__init__() self.path = path self.vae_encoder = AutoencoderKL.from_pretrained(self.path, subfolder="vae", use_auth_token=token).to(device) - + def forward(self, x): return self.vae_encoder.encode(x).latent_dist.sample() diff --git a/demo/Diffusion/stable_diffusion_pipeline.py b/demo/Diffusion/stable_diffusion_pipeline.py index 9fd77fa2..7632995a 100755 --- a/demo/Diffusion/stable_diffusion_pipeline.py +++ b/demo/Diffusion/stable_diffusion_pipeline.py @@ -24,7 +24,7 @@ import onnx from polygraphy import cuda import torch -from utilities import Engine, device_view, save_image +from utilities import Engine, save_image from utilities import DPMScheduler, DDIMScheduler, EulerAncestralDiscreteScheduler, LMSDiscreteScheduler, PNDMScheduler class StableDiffusionPipeline: @@ -45,6 +45,7 @@ def __init__( hf_token=None, verbose=False, nvtx_profile=False, + use_cuda_graph=False, ): """ Initializes the Diffusion pipeline. @@ -76,6 +77,8 @@ def __init__( Enable verbose logging. nvtx_profile (bool): Insert NVTX profiling markers. + use_cuda_graph (bool): + Use CUDA graph to capture engine execution and then launch inference """ self.denoising_steps = denoising_steps @@ -125,11 +128,15 @@ def __init__( self.stages = stages self.inpaint = inpaint + self.use_cuda_graph = use_cuda_graph - self.stream = None # loaded in loadResources() - self.tokenizer = None # loaded in loadResources() - self.models = {} # loaded in loadEngines() - self.engine = {} # loaded in loadEngines() + # initialized in loadResources() + self.stream = None + self.tokenizer = None + # initialized in loadEngines() + self.models = {} + self.engine = {} + self.shared_device_memory = None def loadResources(self, image_height, image_width, batch_size, seed): # Initialize noise generator @@ -157,6 +164,9 @@ def teardown(self): for engine in self.engine.values(): del engine + if self.shared_device_memory: + self.shared_device_memory.free() + self.stream.free() del self.stream @@ -301,18 +311,23 @@ def loadEngines( self.engine[model_name] = engine # Load and activate TensorRT engines + max_device_memory = 0 for model_name, obj in self.models.items(): engine = self.engine[model_name] engine.load() + max_device_memory = max(max_device_memory, engine.engine.device_memory_size) if onnx_refit_dir: onnx_refit_path = self.getOnnxPath(model_name, onnx_refit_dir) if os.path.exists(onnx_refit_path): engine.refit(onnx_opt_path, onnx_refit_path) - engine.activate() + + self.shared_device_memory = cuda.DeviceArray.raw((max_device_memory,)) + for engine in self.engine.values(): + engine.activate(reuse_device_memory=self.shared_device_memory.ptr) def runEngine(self, model_name, feed_dict): engine = self.engine[model_name] - return engine.infer(feed_dict, self.stream) + return engine.infer(feed_dict, self.stream, use_cuda_graph=self.use_cuda_graph) def initialize_latents(self, batch_size, unet_channels, latent_height, latent_width): latents_dtype = torch.float32 # text_embeddings.dtype @@ -357,7 +372,7 @@ def encode_prompt(self, prompt, negative_prompt): return_tensors="pt", ).input_ids.type(torch.int32).to(self.device) - text_input_ids_inp = device_view(text_input_ids) + text_input_ids_inp = text_input_ids # NOTE: output tensor for CLIP must be cloned because it will be overwritten when called again for negative prompt text_embeddings = self.runEngine('clip', {"input_ids": text_input_ids_inp})['text_embeddings'].clone() @@ -369,7 +384,7 @@ def encode_prompt(self, prompt, negative_prompt): truncation=True, return_tensors="pt", ).input_ids.type(torch.int32).to(self.device) - uncond_input_ids_inp = device_view(uncond_input_ids) + uncond_input_ids_inp = uncond_input_ids uncond_embeddings = self.runEngine('clip', {"input_ids": uncond_input_ids_inp})['text_embeddings'] # Concatenate the unconditional and text embeddings into a single batch to avoid doing two forward passes for classifier free guidance @@ -404,9 +419,9 @@ def denoise_latent(self, latents, text_embeddings, timesteps=None, step_offset=0 embeddings_dtype = np.float16 timestep_float = timestep.float() if timestep.dtype != torch.float32 else timestep - sample_inp = device_view(latent_model_input) - timestep_inp = device_view(timestep_float) - embeddings_inp = device_view(text_embeddings) + sample_inp = latent_model_input + timestep_inp = timestep_float + embeddings_inp = text_embeddings noise_pred = self.runEngine('unet', {"sample": sample_inp, "timestep": timestep_inp, "encoder_hidden_states": embeddings_inp})['latent'] if self.nvtx_profile: nvtx.end_range(nvtx_unet) @@ -431,7 +446,7 @@ def encode_image(self, init_image): if self.nvtx_profile: nvtx_vae = nvtx.start_range(message='vae_encoder', color='red') cudart.cudaEventRecord(self.events['vae_encoder-start'], 0) - init_latents = self.runEngine('vae_encoder', {"images": device_view(init_image)})['latent'] + init_latents = self.runEngine('vae_encoder', {"images": init_image})['latent'] cudart.cudaEventRecord(self.events['vae_encoder-stop'], 0) if self.nvtx_profile: nvtx.end_range(nvtx_vae) @@ -443,7 +458,7 @@ def decode_latent(self, latents): if self.nvtx_profile: nvtx_vae = nvtx.start_range(message='vae', color='red') cudart.cudaEventRecord(self.events['vae-start'], 0) - images = self.runEngine('vae', {"latent": device_view(latents)})['images'] + images = self.runEngine('vae', {"latent": latents})['images'] cudart.cudaEventRecord(self.events['vae-stop'], 0) if self.nvtx_profile: nvtx.end_range(nvtx_vae) diff --git a/demo/Diffusion/utilities.py b/demo/Diffusion/utilities.py index cdccbf7a..59cea047 100644 --- a/demo/Diffusion/utilities.py +++ b/demo/Diffusion/utilities.py @@ -35,6 +35,7 @@ import torch import requests from io import BytesIO +from cuda import cudart TRT_LOGGER = trt.Logger(trt.Logger.ERROR) @@ -59,9 +60,13 @@ # Map of torch dtype -> numpy dtype torch_to_numpy_dtype_dict = {value : key for (key, value) in numpy_to_torch_dtype_dict.items()} -def device_view(t): - return cuda.DeviceView(ptr=t.data_ptr(), shape=t.shape, dtype=torch_to_numpy_dtype_dict[t.dtype]) - +def CUASSERT(cuda_ret): + err = cuda_ret[0] + if err != cudart.cudaError_t.cudaSuccess: + raise RuntimeError(f"CUDA ERROR: {err}, error code reference: https://nvidia.github.io/cuda-python/module/cudart.html#cuda.cudart.cudaError_t") + if len(cuda_ret) > 1: + return cuda_ret[1] + return None class Engine(): def __init__( @@ -73,6 +78,7 @@ def __init__( self.context = None self.buffers = OrderedDict() self.tensors = OrderedDict() + self.cuda_graph_instance = None # cuda graph def __del__(self): [buf.free() for buf in self.buffers.values() if isinstance(buf, cuda.DeviceArray) ] @@ -219,8 +225,12 @@ def load(self): print(f"Loading TensorRT engine: {self.engine_path}") self.engine = engine_from_bytes(bytes_from_path(self.engine_path)) - def activate(self): - self.context = self.engine.create_execution_context() + def activate(self, reuse_device_memory=None): + if reuse_device_memory: + self.context = self.engine.create_execution_context_without_device_memory() + self.context.device_memory = reuse_device_memory + else: + self.context = self.engine.create_execution_context() def allocate_buffers(self, shape_dict=None, device='cuda'): for idx in range(trt_util.get_bindings_per_profile(self.engine)): @@ -234,19 +244,32 @@ def allocate_buffers(self, shape_dict=None, device='cuda'): self.context.set_binding_shape(idx, shape) tensor = torch.empty(tuple(shape), dtype=numpy_to_torch_dtype_dict[dtype]).to(device=device) self.tensors[binding] = tensor - self.buffers[binding] = cuda.DeviceView(ptr=tensor.data_ptr(), shape=shape, dtype=dtype) - def infer(self, feed_dict, stream): - start_binding, end_binding = trt_util.get_active_profile_bindings(self.context) - # shallow copy of ordered dict - device_buffers = copy(self.buffers) + def infer(self, feed_dict, stream, use_cuda_graph=False): for name, buf in feed_dict.items(): - assert isinstance(buf, cuda.DeviceView) - device_buffers[name] = buf - bindings = [0] * start_binding + [buf.ptr for buf in device_buffers.values()] - noerror = self.context.execute_async_v2(bindings=bindings, stream_handle=stream.ptr) - if not noerror: - raise ValueError(f"ERROR: inference failed.") + self.tensors[name].copy_(buf) + + for name, tensor in self.tensors.items(): + self.context.set_tensor_address(name, tensor.data_ptr()) + + if use_cuda_graph: + if self.cuda_graph_instance is not None: + CUASSERT(cudart.cudaGraphLaunch(self.cuda_graph_instance, stream.ptr)) + CUASSERT(cudart.cudaStreamSynchronize(stream.ptr)) + else: + # do inference before CUDA graph capture + noerror = self.context.execute_async_v3(stream.ptr) + if not noerror: + raise ValueError(f"ERROR: inference failed.") + # capture cuda graph + CUASSERT(cudart.cudaStreamBeginCapture(stream.ptr, cudart.cudaStreamCaptureMode.cudaStreamCaptureModeGlobal)) + self.context.execute_async_v3(stream.ptr) + self.graph = CUASSERT(cudart.cudaStreamEndCapture(stream.ptr)) + self.cuda_graph_instance = CUASSERT(cudart.cudaGraphInstantiate(self.graph, 0)) + else: + noerror = self.context.execute_async_v3(stream.ptr) + if not noerror: + raise ValueError(f"ERROR: inference failed.") return self.tensors @@ -1186,8 +1209,11 @@ def add_arguments(parser): parser.add_argument('--num-warmup-runs', type=int, default=5, help="Number of warmup runs before benchmarking performance") parser.add_argument('--nvtx-profile', action='store_true', help="Enable NVTX markers for performance profiling") parser.add_argument('--seed', type=int, default=None, help="Seed for random generator to get consistent results") + parser.add_argument('--use-cuda-graph', action='store_true', help="Enable cuda graph") parser.add_argument('--output-dir', default='output', help="Output directory for logs and image artifacts") parser.add_argument('--hf-token', type=str, help="HuggingFace API access token for downloading model checkpoints") parser.add_argument('-v', '--verbose', action='store_true', help="Show verbose output") return parser + + diff --git a/demo/EfficientDet/notebooks/EfficientDet-TensorRT8.ipynb b/demo/EfficientDet/notebooks/EfficientDet-TensorRT8.ipynb index 5eba10cb..a38ef2c4 100644 --- a/demo/EfficientDet/notebooks/EfficientDet-TensorRT8.ipynb +++ b/demo/EfficientDet/notebooks/EfficientDet-TensorRT8.ipynb @@ -347,28 +347,29 @@ "\n", "To generate an ONNX model file, first find the input shape that corresponds to the model you're converting:\n", "\n", - "| **Model** | **Input Shape** |\n", - "| -----------------|-----------------|\n", - "| EfficientDet D0 | N,512,512,3 |\n", - "| EfficientDet D1 | N,640,640,3 |\n", - "| EfficientDet D2 | N,768,768,3 |\n", - "| EfficientDet D3 | N,896,896,3 |\n", - "| EfficientDet D4 | N,1024,1024,3 |\n", - "| EfficientDet D5 | N,1280,1280,3 |\n", - "| EfficientDet D6 | N,1280,1280,3 |\n", - "| EfficientDet D7 | N,1536,1536,3 |\n", - "| EfficientDet D7x | N,1536,1536,3 |\n", - "\n", - "\n", - "Where **N** is the batch size you would like to run inference at, such as `8,512,512,3` for a batch size of 8.\n", - "\n", - "The conversion process supports both NHWC and NCHW input formats, so if your input source is an `NCHW` data format, you can use the corresponding input shape, i.e. `1,512,512,3` -> `1,3,512,512`.\n", + "| **Model** | **Input Size** |\n", + "| --------------------|----------------|\n", + "| efficientdet-d0 | 512,512 |\n", + "| efficientdet-d1 | 640,640 |\n", + "| efficientdet-d2 | 768,768 |\n", + "| efficientdet-d3 | 896,896 |\n", + "| efficientdet-d4 | 1024,1024 |\n", + "| efficientdet-d5 | 1280,1280 |\n", + "| efficientdet-d6 | 1280,1280 |\n", + "| efficientdet-d7 | 1536,1536 |\n", + "| efficientdet-d7x | 1536,1536 |\n", + "| efficientdet-lite0 | 320,320 |\n", + "| efficientdet-lite1 | 384,384 |\n", + "| efficientdet-lite2 | 448,448 |\n", + "| efficientdet-lite3 | 512,512 |\n", + "| efficientdet-lite3x | 640,640 |\n", + "| efficientdet-lite4 | 640,640 |\n", "\n", "To create the ONNX graph, execute efficientdet/create_onnx.py script which takes the following arguments:\n", "```\n", "* --saved_model /path/to/tf_model \n", "* --onnx /path/to/onnx.model\n", - "* --input_shape One of the input shapes corresponding to the model mentioned previously\n", + "* --input_size One of the input shapes corresponding to the model mentioned previously\n", "```" ] }, @@ -388,7 +389,7 @@ "!python3 $TRT_OSSPATH/samples/python/efficientdet/create_onnx.py \\\n", " --saved_model ./tf_model/ \\\n", " --onnx ./onnx_model/model.onnx \\\n", - " --input_shape '1,512,512,3'" + " --input_size '512,512'" ] }, { @@ -403,7 +404,7 @@ "The script has a few additional arguments:\n", "\n", "* `--nms_threshold` allows overriding the NMS score threshold value. The runtime latency of the EfficientNMS plugin is sensitive to the score threshold used, so it's a good practice to set this value as high as possible, while still fulfilling your application requirements, to reduce latency as much as possible.\n", - "* `--legacy_plugins` allows falling back to older plugins on systems where a version lower than TensorRT 8.0.1 is installed. This will result in substantially slower inference times however.\n" + "* `--preprocessor [imagenet,scale_range]` allows switching between two possible image preprocessing methods. Most EfficientDet models use the `imagenet` method, which this argument defaults to, and corresponds to standard ImageNet mean subtraction and standard deviation normalization. The `scale_range` method instead normalizes the image to a range of [-1,+1]. Please use this method only when converting the **AdvProp** pre-trained checkpoints, as they were created with this preprocessor operation.\n" ] }, { diff --git a/demo/HuggingFace/BART/BARTModelConfig.py b/demo/HuggingFace/BART/BARTModelConfig.py index 8994e7e5..f8ea3bd7 100755 --- a/demo/HuggingFace/BART/BARTModelConfig.py +++ b/demo/HuggingFace/BART/BARTModelConfig.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -98,8 +98,8 @@ class BARTModelTRTConfig(NNConfig): TARGET_MODELS = ["facebook/bart-base", "facebook/bart-large", "facebook/bart-large-cnn", "facebook/mbart-large-50"] MAX_DECODER_WORKSPACE_MB = { - TARGET_MODELS[0]: 3072, - TARGET_MODELS[1]: 3072, + TARGET_MODELS[0]: 3072, + TARGET_MODELS[1]: 3072, TARGET_MODELS[2]: 3072, TARGET_MODELS[3]: 3072, } @@ -108,23 +108,23 @@ class BARTModelTRTConfig(NNConfig): # bart-large: 24-layer, 1024-hidden, 406M parameters # in all bart variants, # of encoder layers and # of decoder layers are the same NUMBER_OF_LAYERS = { - TARGET_MODELS[0]: 12, - TARGET_MODELS[1]: 24, + TARGET_MODELS[0]: 12, + TARGET_MODELS[1]: 24, TARGET_MODELS[2]: 24, TARGET_MODELS[3]: 24, - } - + } + NUMBER_OF_DECODER_LAYERS = { - TARGET_MODELS[0]: 6, - TARGET_MODELS[1]: 12, + TARGET_MODELS[0]: 6, + TARGET_MODELS[1]: 12, TARGET_MODELS[2]: 12, TARGET_MODELS[3]: 12, - } - + } + # in all bart variants, # of heads in encoder and decoder are the same NUMBER_OF_HEADS = { - TARGET_MODELS[0]: 12, - TARGET_MODELS[1]: 16, + TARGET_MODELS[0]: 12, + TARGET_MODELS[1]: 16, TARGET_MODELS[2]: 16, TARGET_MODELS[3]: 16, } @@ -142,7 +142,7 @@ class BARTModelTRTConfig(NNConfig): TARGET_MODELS[1]: 1024, TARGET_MODELS[2]: 1024, TARGET_MODELS[3]: 1024, - } + } # To achieve identical results with original HuggingFace implementation, the min_length in model config should be consistent with each model variant # see task-specific params in config.json of each variant model @@ -151,7 +151,7 @@ class BARTModelTRTConfig(NNConfig): TARGET_MODELS[1]: 0, TARGET_MODELS[2]: 56, TARGET_MODELS[3]: 0, - } + } #TODO: this might better be an inference time input like the `max_length` arg in generate() and greedy_search(). The change needed is in NNDF/interface.py:__call__ so it's a fundamental change affecting GPT2 and T5 code. Here I just put this option in BART model config for now. But it's also reasonable to treat this as a model config, because the TRT engine building may need this to have fixed dimension (e.g., to enable KV-cache) # see task-specific params in config.json of each variant model @@ -160,9 +160,9 @@ class BARTModelTRTConfig(NNConfig): TARGET_MODELS[1]: 1024, TARGET_MODELS[2]: 142, TARGET_MODELS[3]: 200, - } + } - # BART specific configs: https://huggingface.co/facebook/bart-base/blob/main/config.json + # BART specific configs: https://huggingface.co/facebook/bart-base/blob/main/config.json NO_REPEAT_NGRAM_SIZE = 3 BOS_TOKEN_ID = 0 EOS_TOKEN_ID = 2 @@ -178,7 +178,7 @@ class BARTModelTRTConfig(NNConfig): NETWORK_DECODER_SEGMENT_NAME = "decoder" NETWORK_ENCODER_SEGMENT_NAME = "encoder" NETWORK_SEGMENTS = [NETWORK_DECODER_SEGMENT_NAME, NETWORK_ENCODER_SEGMENT_NAME] - + def __init__(self): precision_fp16 = [False, True] kv_caches = [False, True] @@ -214,7 +214,7 @@ def get_metadata_string(self, metadata: NetworkMetadata) -> str: # Remove redundant bart name prefix if "mbart" in metadata.variant: metadata = metadata._replace(variant=metadata.variant.replace("facebook/mbart-","mbart-")) - else: + else: metadata = metadata._replace(variant=metadata.variant.replace("facebook/bart-","")) return super().get_metadata_string(metadata) @@ -240,13 +240,13 @@ def get_input_dims(metadata) -> Dict: if metadata.other.kv_cache: # for KV cache version, we need add per-layer KV cache inputs. `past_key_values` at each layer is (self-attention K, self-attention V, cross-attention K, cross-attention V) for i in range(BARTModelTRTConfig.NUMBER_OF_DECODER_LAYERS[metadata.variant]): - # decoder self-attention KV cache (dim[0] & dim[2] are dynamic, and dim[2] varies at each decoding timestep) + # decoder self-attention KV cache (dim[0] & dim[2] are dynamic, and dim[2] varies at each decoding timestep) self_attention_past_kv_dims = (Dims.BATCH, "num_heads", Dims.create_new_sequence_dim("past_decoder_length"), "embedding_size_per_head") decoder_inputs_dict[f"past_key_values.{i}.decoder.key"] = self_attention_past_kv_dims decoder_inputs_dict[f"past_key_values.{i}.decoder.value"] = self_attention_past_kv_dims - + # encoder-decoder cross-attention KV cache (dim[0] & dim[2] are dynamic, but dim[2] is constant at each decoding timestep) - cross_attention_past_kv_dims = (Dims.BATCH, "num_heads", Dims.create_new_sequence_dim("encoder_length"), "embedding_size_per_head") + cross_attention_past_kv_dims = (Dims.BATCH, "num_heads", Dims.create_new_sequence_dim("encoder_length"), "embedding_size_per_head") decoder_inputs_dict[f"past_key_values.{i}.encoder.key"] = cross_attention_past_kv_dims decoder_inputs_dict[f"past_key_values.{i}.encoder.value"] = cross_attention_past_kv_dims @@ -273,16 +273,16 @@ def get_output_dims(metadata) -> Dict: if metadata.other.kv_cache: # for KV cache version, we need add per-layer KV cache inputs. `past_key_values` at each layer is (self-attention K, self-attention V, cross-attention K, cross-attention V) - + # for all BART variants, # encoder layers = # decoder layers, so just divide total # layers by 2 for i in range(BARTModelTRTConfig.NUMBER_OF_DECODER_LAYERS[metadata.variant]): - # decoder self-attention KV cache (dim[0] & dim[2] are dynamic, and dim[2] varies at each decoding timestep) + # decoder self-attention KV cache (dim[0] & dim[2] are dynamic, and dim[2] varies at each decoding timestep) self_attention_present_kv_dims = (Dims.BATCH, "num_heads", Dims.create_new_sequence_dim("decoder_length"), "embedding_size_per_head") decoder_outputs_dict[f"present_key_values.{i}.decoder.key"] = self_attention_present_kv_dims decoder_outputs_dict[f"present_key_values.{i}.decoder.value"] = self_attention_present_kv_dims - + # encoder-decoder cross-attention KV cache (dim[0] & dim[2] are dynamic, but dim[2] is constant at each decoding timestep) - cross_attention_present_kv_dims = (Dims.BATCH, "num_heads", Dims.create_new_sequence_dim("encoder_length"), "embedding_size_per_head") + cross_attention_present_kv_dims = (Dims.BATCH, "num_heads", Dims.create_new_sequence_dim("encoder_length"), "embedding_size_per_head") decoder_outputs_dict[f"present_key_values.{i}.encoder.key"] = cross_attention_present_kv_dims decoder_outputs_dict[f"present_key_values.{i}.encoder.value"] = cross_attention_present_kv_dims diff --git a/demo/HuggingFace/BART/checkpoint.toml b/demo/HuggingFace/BART/checkpoint.toml index d7ddcafd..52add215 100755 --- a/demo/HuggingFace/BART/checkpoint.toml +++ b/demo/HuggingFace/BART/checkpoint.toml @@ -9,9 +9,6 @@ label = "NVIDIA TensorRT-based applications perform up to 36X faster than CPU-on [BART.all."facebook/bart-large".all.summarization] -[BART.all."facebook/mbart-large-50".all.summarization] - -label = "NVIDIA TensorRT-based applications perform up to 36X faster than CPU-only platforms during inference, enabling developers to optimize neural network models trained on all major frameworks, calibrate for lower precision with high accuracy, and deploy to hyperscale data centers, embedded platforms, or automotive product platforms. TensorTM, built on the NVIDIA CUDA parallel programming model, enables developers of applications to optimise inference by leveraging libraries, development tools, and technologies in CUDA-X for AI, autonomous machines, high performance computing, and graphics. With new NVIDIA Ampere Architecture GPUs, Tensor RT also uses sparse tensor cores for an additional performance boost." label = "NVIDIA TensorRT-based applications perform up to 36X faster than CPU-only platforms during inference, enabling developers to optimize neural network models trained on all major frameworks, calibrate for lower precision with high accuracy, and deploy to hyperscale data centers, embedded platforms, or automotive product platforms. Tensor RT is the first GPU-based inference platform to use NVIDIA's CUDA-X architecture. TenseRT, built on the NVIDIA CUDA parallel programming model, enables developers to analyze neural network data and perform inference by leveraging libraries, development tools, and technologies in CUDA, including CUDA for AI, autonomous machines, high performance computing, and graphics. With new NVIDIA Ampere Architecture GPUs, TensorRex also uses sparse tensor cores for an additional performance boost." [BART.all."facebook/bart-large-cnn".all.summarization] diff --git a/demo/HuggingFace/BART/export.py b/demo/HuggingFace/BART/export.py index faa8b945..f3730178 100755 --- a/demo/HuggingFace/BART/export.py +++ b/demo/HuggingFace/BART/export.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -41,7 +41,7 @@ # TRT-HuggingFace from BART.BARTModelConfig import BARTModelTRTConfig -from NNDF.tensorrt_utils import clamp_weights_onnx_to_fp16_bounds, move_t5_cast_op +from NNDF.tensorrt_utils import OnnxProcessOperation, process_onnx from NNDF.networks import NetworkMetadata, Precision, Dims from NNDF.logger import G_LOGGER from NNDF.models import ( @@ -354,13 +354,14 @@ def _export_forward(input_ids, encoder_hidden_states, use_cache): if network_metadata.precision.fp16: G_LOGGER.debug("Clamping FP16 weights for BART") - # move_t5_cast_op(output_fpath, output_fpath) # BART doesn't have T5's Add-Cast-Pow ordering issue + # BART doesn't have T5's Add-Cast-Pow ordering issue if network_metadata.other.kv_cache: # both onnx files need clamp - clamp_weights_onnx_to_fp16_bounds(non_kv_fpath, non_kv_fpath) - clamp_weights_onnx_to_fp16_bounds(kv_fpath, kv_fpath) + process_onnx([OnnxProcessOperation.CLAMP_WEIGHTS], kv_fpath, kv_fpath) + process_onnx([OnnxProcessOperation.CLAMP_WEIGHTS], non_kv_fpath, non_kv_fpath) + else: - clamp_weights_onnx_to_fp16_bounds(output_fpath, output_fpath) + process_onnx([OnnxProcessOperation.CLAMP_WEIGHTS], output_fpath, output_fpath) return BARTDecoderONNXFile(output_fpath, network_metadata) @@ -412,7 +413,7 @@ def torch_to_onnx( if network_metadata.precision.fp16: G_LOGGER.debug("Clamping FP16 weights for BART") - # move_t5_cast_op(output_fpath, output_fpath) # BART doesn't have T5's Add-Cast-Pow ordering issue - clamp_weights_onnx_to_fp16_bounds(output_fpath, output_fpath) + # BART doesn't have T5's Add-Cast-Pow ordering issue + process_onnx([OnnxProcessOperation.CLAMP_WEIGHTS], output_fpath, output_fpath) return BARTEncoderONNXFile(output_fpath, network_metadata) diff --git a/demo/HuggingFace/BART/frameworks.py b/demo/HuggingFace/BART/frameworks.py index 69fecb72..3df3e908 100644 --- a/demo/HuggingFace/BART/frameworks.py +++ b/demo/HuggingFace/BART/frameworks.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -68,16 +68,16 @@ def __init__(self): def generate_and_download_framework( self, metadata: NetworkMetadata, workspace: NNFolderWorkspace ) -> NetworkModels: - + cache_variant = False if metadata.other.kv_cache: cache_variant = True - + trt_BART_config = self.config metadata_serialized = trt_BART_config.get_metadata_string(metadata) workspace_dir, encoder_onnx_root, decoder_onnx_root = workspace.set_model_path(metadata_serialized, is_encoder_decoder = True) pytorch_model_dir = os.path.join(workspace_dir, "pytorch_model") - + # We keep track of the generated torch location for cleanup later self.torch_BART_dir = pytorch_model_dir @@ -111,8 +111,8 @@ def generate_and_download_framework( model = MBartForConditionalGeneration.from_pretrained(pytorch_model_dir) model.config.use_cache = cache_variant # somehow the use_cache config automatically set to True even though specified in tfm_config before. Force change - - # These ONNX models can be converted using special encoder and decoder classes. + + # These ONNX models can be converted using special encoder and decoder classes. encoder_onnx_model_fpath = os.path.join(encoder_onnx_root, metadata_serialized + "-encoder.onnx") decoder_onnx_model_fpath = os.path.join(decoder_onnx_root, metadata_serialized + "-decoder-with-lm-head.onnx") @@ -262,7 +262,7 @@ def execute_inference( batch_size=batch_size, use_cache=metadata.other.kv_cache, ) - + # Prepare runtime results. runtime=[ NetworkRuntime( @@ -370,4 +370,4 @@ def run_framework( if __name__ == "__main__": result = RUN_CMD() - print("Results: {}".format(result)) \ No newline at end of file + print("Results: {}".format(result)) diff --git a/demo/HuggingFace/BART/hf.py b/demo/HuggingFace/BART/hf.py index 0e8dc48b..ae79b64c 100755 --- a/demo/HuggingFace/BART/hf.py +++ b/demo/HuggingFace/BART/hf.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -21,7 +21,7 @@ Usage: python3 hf.py --variant facebook/bart-base [--enable-kv-cache] [--fp16] """ -import time +import time from transformers import BartTokenizer, BartForConditionalGeneration import argparse @@ -63,6 +63,6 @@ output = tokenizer.decode(summary_ids[-1,:], skip_special_tokens=True) -print('BART output: ', output) +print('BART output: ', output) print(f"Input sequence length: {input_ids.size(1)}, Output sequence length: {summary_ids[-1,:].size(0)}") print("Average run time: {:.2f} ms".format((end - start)/trials*1000)) diff --git a/demo/HuggingFace/BART/measurements.py b/demo/HuggingFace/BART/measurements.py index 06dd7c1b..54f809b0 100644 --- a/demo/HuggingFace/BART/measurements.py +++ b/demo/HuggingFace/BART/measurements.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -96,7 +96,7 @@ def full_inference_greedy( stopping_criteria = StoppingCriteriaList([MaxLengthCriteria(max_length)]) no_repeat_ngram_size = BARTModelTRTConfig.NO_REPEAT_NGRAM_SIZE logits_processor = LogitsProcessorList([ - NoRepeatNGramLogitsProcessor(no_repeat_ngram_size), + NoRepeatNGramLogitsProcessor(no_repeat_ngram_size), MinLengthLogitsProcessor(min_length, tokenizer.convert_tokens_to_ids(tokenizer.eos_token)), ForcedBOSTokenLogitsProcessor(tokenizer.convert_tokens_to_ids(tokenizer.bos_token)), ForcedEOSTokenLogitsProcessor(max_length, tokenizer.convert_tokens_to_ids(tokenizer.eos_token)) @@ -167,7 +167,7 @@ def full_inference_beam( stopping_criteria = StoppingCriteriaList([MaxLengthCriteria(max_length)]) no_repeat_ngram_size = BARTModelTRTConfig.NO_REPEAT_NGRAM_SIZE logits_processor = LogitsProcessorList([ - NoRepeatNGramLogitsProcessor(no_repeat_ngram_size), + NoRepeatNGramLogitsProcessor(no_repeat_ngram_size), MinLengthLogitsProcessor(min_length, tokenizer.convert_tokens_to_ids(tokenizer.eos_token)), ForcedBOSTokenLogitsProcessor(tokenizer.convert_tokens_to_ids(tokenizer.bos_token)), ForcedEOSTokenLogitsProcessor(max_length, tokenizer.convert_tokens_to_ids(tokenizer.eos_token)) @@ -194,7 +194,7 @@ def _e2e(): ) encoder_last_hidden_state = BART_encoder(input_ids=input_ids) - + encoder_last_hidden_state = expand_inputs_for_beam_search(encoder_last_hidden_state, expand_size=num_beams) decoder_output_beam = BART_decoder.beam_search( @@ -219,9 +219,9 @@ def _e2e_trt(): ) encoder_last_hidden_state = BART_encoder(input_ids=input_ids) - + encoder_last_hidden_state = expand_inputs_for_beam_search(encoder_last_hidden_state, expand_size=num_beams) - + BART_decoder.set_encoder_hidden_states_for_inference_cycle(encoder_last_hidden_state) decoder_output_beam = BART_decoder.beam_search( input_ids=decoder_input_ids, diff --git a/demo/HuggingFace/BART/onnxrt.py b/demo/HuggingFace/BART/onnxrt.py index 0d8cbed2..b7523e0d 100644 --- a/demo/HuggingFace/BART/onnxrt.py +++ b/demo/HuggingFace/BART/onnxrt.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -125,7 +125,7 @@ def execute_inference( benchmarking_mode: bool = False, benchmarking_args: BARTBenchmarkingArgs = None, ) -> NetworkResult: - + if "mbart" not in metadata.variant: tokenizer = BartTokenizer.from_pretrained(metadata.variant) else: @@ -207,7 +207,7 @@ def execute_inference( onnx=list(onnx_fpaths.values()), trt=None ) - + # Skip result checking in benchmarking mode since the input data is random. if benchmarking_mode: return BenchmarkingResult(median_runtime=runtime, models=models) diff --git a/demo/HuggingFace/BART/trt.py b/demo/HuggingFace/BART/trt.py index b8be8d81..85bb2790 100644 --- a/demo/HuggingFace/BART/trt.py +++ b/demo/HuggingFace/BART/trt.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -278,7 +278,7 @@ def __init__( # non kv-cache mode: False. Then in forward(), trt_context and bindings are set to the default ones # kv-cache mode: True. By default 1st decoding step starts with non-kv engine's context and binding; then flag gets updated in prepare_inputs_for_generation() - self.return_device = "cuda" + self.return_device = torch.device('cuda') self.variant = network_metadata.variant # record variant name to later index the vocab_size in forward() @@ -806,14 +806,15 @@ def execute_calculate_perplexity( metadata: NetworkMetadata, encoder_input: str, decoder_input: str, + batch_size: int, ): if "mbart" not in metadata.variant: tokenizer = BartTokenizer.from_pretrained(metadata.variant) else: tokenizer = MBart50Tokenizer.from_pretrained(metadata.variant, src_lang="en_XX") - encoder_input_ids = tokenizer([encoder_input], padding=True, return_tensors="pt").input_ids - decoder_input_ids = tokenizer([decoder_input], padding=True, return_tensors="pt").input_ids + encoder_input_ids = tokenizer([encoder_input] * batch_size, padding=True, return_tensors="pt").input_ids + decoder_input_ids = tokenizer([decoder_input] * batch_size, padding=True, return_tensors="pt").input_ids perplexity = calculate_perplexity( self.BART_trt_encoder, self.BART_trt_decoder, tokenizer, encoder_input_ids, decoder_input_ids, @@ -821,21 +822,6 @@ def execute_calculate_perplexity( ) return perplexity - def _setup_workspace(self, metadata: NetworkMetadata, working_directory: str) -> NNFolderWorkspace: - return NNFolderWorkspace( - self.frameworks_cmd.config.network_name, metadata, working_directory - ) - - def _download_models( - self, - workspace: NNFolderWorkspace, - metadata: NetworkMetadata, - ) -> Tuple[NetworkModel]: - # No fpath provided for onnx files, download them from HuggingFace repo. - return self.frameworks_cmd.generate_and_download_framework( - metadata, workspace - ).onnx - def _setup_engines( self, metadata: NetworkMetadata, @@ -967,7 +953,7 @@ def _setup_engines( if num_beams > 1: engine_tag += "-beam{}".format(num_beams) - preview_features = [] + preview_features = [PreviewFeature.DISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805] if disable_preview_dynamic_shapes: engine_tag += "-noPreviewFasterDynamicShapes" else: @@ -1080,7 +1066,7 @@ def run_trt( else: for ei, di in zip(network_input, perplexity_reference): ppl_results.append( - self.execute_calculate_perplexity(metadata, ei, di) + self.execute_calculate_perplexity(metadata, ei, di, batch_size) ) else: @@ -1106,7 +1092,7 @@ def run_trt( assert benchmarking_args.output_seq_len <= benchmarking_args.output_profile_max_len, "output_seq_len should <= output_profile_max_len = {} for benchmarking mode".format(benchmarking_args.output_profile_max_len) assert benchmarking_args.input_profile_max_len <= max_input_seq_len, "Model config restrict input_profile_max_len <= {} for benchmark mode".format(max_input_seq_len) assert benchmarking_args.output_profile_max_len <= max_output_seq_len, "Model config restrict output_profile_max_len <= {} for benchmark mode".format(max_output_seq_len) - + self._setup_engines(metadata, hash_onnx_fpath, batch_size, args.num_beams, disable_preview_dynamic_shapes, benchmarking_args, seq_tag) inference_results = self.execute_inference( metadata, hash_onnx_fpath, None, timing_profile, batch_size, args.num_beams, True, benchmarking_args diff --git a/demo/HuggingFace/GPT2/GPT2ModelConfig.py b/demo/HuggingFace/GPT2/GPT2ModelConfig.py index 3d83024b..a0edca9f 100644 --- a/demo/HuggingFace/GPT2/GPT2ModelConfig.py +++ b/demo/HuggingFace/GPT2/GPT2ModelConfig.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -51,7 +51,7 @@ def add_args(parser: argparse.ArgumentParser) -> None: network_group.add_argument( "--num-beams", type=int, default=1, help="Enables beam search during decoding." ) - + network_group.add_argument( "--fp16", action="store_true", help="Enables fp16 TensorRT tactics." ) diff --git a/demo/HuggingFace/GPT2/export.py b/demo/HuggingFace/GPT2/export.py index 513de75b..cbd06964 100644 --- a/demo/HuggingFace/GPT2/export.py +++ b/demo/HuggingFace/GPT2/export.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -60,7 +60,7 @@ def __init__(self, transformer, lm_head, config): self.transformer = transformer self.lm_head = lm_head self.config = config - self.device = "cuda" # WAR to avoid beam search in framework + self.device = torch.device('cuda') # WAR to avoid beam search in framework self.main_input_name = "input_ids" # For better HuggingFace version compatibility def prepare_inputs_for_generation(self, input_ids, past = None, use_cache=None, **kwargs): @@ -143,6 +143,17 @@ def pairwise(iterable): l_next.precision = trt.float32 l_next.set_output_type(0, trt.float32) + if self.network_metadata.precision.fp16: + for i in range(network_definition[1].num_inputs): + t = network_definition[1].get_input(i) + if t.dtype == trt.float32: + t.dtype = trt.float16 + + for i in range(network_definition[1].num_outputs): + t = network_definition[1].get_output(i) + if t.dtype == trt.float32: + t.dtype = trt.float16 + return network_definition # Converters diff --git a/demo/HuggingFace/GPT2/frameworks.py b/demo/HuggingFace/GPT2/frameworks.py index f5d0abd9..d430a056 100644 --- a/demo/HuggingFace/GPT2/frameworks.py +++ b/demo/HuggingFace/GPT2/frameworks.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -148,10 +148,10 @@ def setup_tokenizer_and_model( # By default, HuggingFace model structure is one giant file. gpt2_torch_fpath = network_fpaths.torch[0].fpath gpt2_model = AutoModelForCausalLM.from_pretrained(gpt2_torch_fpath) + # Framework fp16 does not support cpu mode for GPT2 - # TODO: Enable true fp16. Using cuda 11.4 with PyTorch 1.13 will cause issue for this function. - # if metadata.precision.fp16: - # gpt2_model = gpt2_model.cuda().half() + if metadata.precision.fp16: + gpt2_model = gpt2_model.cuda().half() gpt2_torch = GPT2TorchFile.TorchModule( gpt2_model.transformer, gpt2_model.lm_head, gpt2_model.config @@ -185,9 +185,9 @@ def execute_inference( # get single decoder iteration inference timing profile _, decoder_e2e_time = gpt2_inference( - gpt2_torch, - input_ids, - timing_profile, + gpt2_torch, + input_ids, + timing_profile, use_cuda=(not use_cpu), use_cache = metadata.other.kv_cache, ) diff --git a/demo/HuggingFace/GPT2/measurements.py b/demo/HuggingFace/GPT2/measurements.py index 740b5e69..f783f872 100644 --- a/demo/HuggingFace/GPT2/measurements.py +++ b/demo/HuggingFace/GPT2/measurements.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -64,14 +64,14 @@ def full_inference( if isinstance(gpt2, TRTNativeRunner): gpt2.set_return_device("cuda" if use_cuda else "cpu") - + def _e2e(): with torch.no_grad(): output = gpt2.generate( - input_ids, - max_length=max_length, - min_length=min_length, - batch_size=batch_size, + input_ids, + max_length=max_length, + min_length=min_length, + batch_size=batch_size, num_beams=num_beams, use_cache=use_cache, early_stopping=early_stopping diff --git a/demo/HuggingFace/GPT2/trt.py b/demo/HuggingFace/GPT2/trt.py index f215fe2a..411eb72c 100644 --- a/demo/HuggingFace/GPT2/trt.py +++ b/demo/HuggingFace/GPT2/trt.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -122,7 +122,7 @@ def __init__( ): super().__init__(trt_engine_file, network_metadata, hf_config, batch_size = batch_size) self.network_metadata = network_metadata - self.data_type = torch.float32 + self.data_type = torch.float32 if not network_metadata.precision.fp16 else torch.float16 # In benchmarking mode, if input_profile_max is provided, should use that as max_sequence_length if benchmarking_args is not None: if benchmarking_args.input_profile_max_len is not None: @@ -153,7 +153,7 @@ def __init__( if self.config.use_cache: self.bindings[self.trt_engine.get_binding_index("logits") + self.num_bindings] = self.logits.data_ptr() - + # Setting input and output the same does not work for GPT2. Needs separate cache and copy the memory address after each iteration self.self_attention_cache_1 = {} self.self_attention_cache_2 = {} @@ -172,9 +172,9 @@ def __init__( input_idx = self.trt_engine.get_binding_index("past_" + self_attention_name) output_idx = self.trt_engine.get_binding_index("present_" + self_attention_name) - + self.bindings[input_idx] = kv_buffer_1.data_ptr() # Generation phase - self.bindings[output_idx] = kv_buffer_2.data_ptr() + self.bindings[output_idx] = kv_buffer_2.data_ptr() # Context mode will always use buffer 1 as output self.bindings[input_idx + self.num_bindings] = 0 # Context phase, should be 0 @@ -184,17 +184,17 @@ def __init__( self.past_decoder_length = 0 self.use_cache_1_as_input = True self._set_context_mode_trt_context() - + self.context_mode = self.config.use_cache - self.return_device = "cuda" - self.device = "cuda" + self.return_device = torch.device('cuda') + self.device = torch.device('cuda') def reset(self): ''' Resets the input specific fields after finishing a task. ''' self.context_mode = self.config.use_cache - + def _switch_input_output_binding(self): ''' For kv cache mode, switch input and output pointers to avoid data concurrency issue and D2D copy @@ -212,7 +212,7 @@ def _switch_input_output_binding(self): self.bindings[output_idx] = self.bindings[input_idx] self.bindings[input_idx] = temp self.use_cache_1_as_input = not self.use_cache_1_as_input - + def prepare_inputs_for_generation(self, input_ids, past = None, use_cache = None, **kwargs): # TODO: add position_ids, token_type_ids support if past is not None: @@ -220,7 +220,7 @@ def prepare_inputs_for_generation(self, input_ids, past = None, use_cache = None self.context_mode = False else: self.context_mode = self.config.use_cache - + return { "input_ids": input_ids, "past_key_values": past, @@ -244,7 +244,7 @@ def _reorder_cache(self, past: Tuple[Tuple[torch.Tensor]], beam_idx: torch.Tenso tuple(past_state.index_select(0, beam_idx.to(past_state.device)) for past_state in layer_past) for layer_past in past ) - + def _set_context_mode_trt_context(self): # Create TRT context for context mode (1st decoder run) with optimization profile = 1 self.context_trt_context = self.trt_engine.create_execution_context() @@ -260,7 +260,7 @@ def forward(self, input_ids, *args, **kwargs): if is_cpu_mode: input_ids = input_ids.int().cuda() - + # Set the binding shape of input_ids, which should be (bs, input_length). if not self.context_mode: self.bindings[0] = input_ids.int().data_ptr() @@ -269,7 +269,7 @@ def forward(self, input_ids, *args, **kwargs): self.bindings[self.num_bindings] = input_ids.int().data_ptr() self.context_trt_context.set_binding_shape(self.num_bindings, input_ids.shape) - if self.config.use_cache: + if self.config.use_cache: if self.context_mode: self.past_decoder_length = 0 @@ -284,7 +284,7 @@ def forward(self, input_ids, *args, **kwargs): # Optimization Profile 0 is context phase with kv inputs self.context_trt_context.set_binding_shape(self.kv_cache_binding_offset+2*i + self.num_bindings, self_attention_kv_shape) self.context_trt_context.set_binding_shape(self.kv_cache_binding_offset+2*i + 1 + self.num_bindings, self_attention_kv_shape) - + # Launch TRT inference. if not self.context_mode: assert self.trt_context.all_binding_shapes_specified @@ -292,7 +292,7 @@ def forward(self, input_ids, *args, **kwargs): else: assert self.context_trt_context.all_binding_shapes_specified self.context_trt_context.execute_v2(bindings=self.bindings) - + # For bs > 1, this is required, so cannnot avoid this D2D copy logits_length = bs * input_length * self.config.vocab_size logits = self.logits.flatten()[:logits_length].view(bs, input_length, self.config.vocab_size) @@ -306,7 +306,7 @@ def forward(self, input_ids, *args, **kwargs): present_key_values = () self_attention_cache = self.self_attention_cache_1 if self.use_cache_1_as_input or (self.profile_idx == 0) else self.self_attention_cache_2 - + for i in range(self.num_decoder_layers): self_attention_k_output = self_attention_cache[f"key_values.{i}.decoder.key"] @@ -316,7 +316,7 @@ def forward(self, input_ids, *args, **kwargs): self_attention_k_output = self_attention_k_output.cpu() self_attention_v_output = self_attention_v_output.cpu() - present_key_values += ((self_attention_k_output, self_attention_v_output),) + present_key_values += ((self_attention_k_output, self_attention_v_output),) self._switch_input_output_binding() return CausalLMOutputWithPast(logits=logits.to(self.return_device), past_key_values = present_key_values) @@ -344,6 +344,33 @@ def cleanup( self.frameworks_cmd.cleanup(workspace, keep_onnx_model, keep_torch_model) + def generate( + self, + input_ids, + min_length: int = None, + max_length: int = None, + num_beams: int = 1, + use_cache: bool = False, + early_stopping: bool = True, + ): + if max_length is None: + max_length = GPT2ModelTRTConfig.MAX_OUTPUT_LENGTH[self.metadata.variant] + + if min_length is None: + min_length = GPT2ModelTRTConfig.MIN_OUTPUT_LENGTH[self.metadata.variant] + + output = self.gpt2_trt.generate( + input_ids, + max_length=max_length, + min_length=min_length, + num_beams=num_beams, + use_cache=use_cache, + early_stopping=early_stopping + ) + + self.gpt2_trt.reset() + return output + def execute_inference( self, metadata: NetworkMetadata, @@ -379,7 +406,7 @@ def execute_inference( timing_profile, use_cache = metadata.other.kv_cache, ) - + # get complete decoder inference result and its timing profile sample_output, full_e2e_runtime = full_inference( self.gpt2_trt, @@ -439,6 +466,7 @@ def execute_calculate_perplexity( self, metadata: NetworkMetadata, reference: str, + batch_size: int, ): tokenizer = GPT2Tokenizer.from_pretrained(metadata.variant) @@ -446,7 +474,7 @@ def execute_calculate_perplexity( # replace with EOS token when using generating mode tokenizer.add_special_tokens({"pad_token": "[PAD]"}) reference = reference.replace("\\n", "\n") - ppl_input_ids = tokenizer([reference], padding=False, return_tensors="pt").input_ids + ppl_input_ids = tokenizer([reference] * batch_size, padding=False, return_tensors="pt").input_ids perplexity = calculate_perplexity( self.gpt2_trt, ppl_input_ids, GPT2ModelTRTConfig.MAX_LENGTH[metadata.variant] @@ -495,7 +523,7 @@ def _setup_engines( max_output_length = benchmarking_args.output_profile_max_len opt_input_seq_len = benchmarking_args.input_seq_len opt_output_seq_len = benchmarking_args.output_seq_len - + if not hf_config.use_cache: # If not using kv cache, only input_ids is passed decoder_profiles = [Profile().add( @@ -531,7 +559,7 @@ def _setup_engines( opt=(batch_size * num_beams, 1), max=(batch_size * num_beams, 1), ) - + self_attention_profile_generation = { "min": (batch_size * num_beams, num_heads, 1, embedding_size_per_head), "opt": (batch_size * num_beams, num_heads, opt_output_seq_len - 1, embedding_size_per_head), @@ -554,11 +582,11 @@ def _setup_engines( f"past_key_values.{i}.decoder.value", **self_attention_profile_generation ) - + # TensorRT accepts multiple optimization engines for the same model. # Profile 1 is only used in the first decoder iterations. decoder_profiles = [dec_profiles_generation, dec_profiles_context] - + # Convert ONNX models to TRT engines. if benchmarking_args is None: engine_tag = "bs{}".format(batch_size) @@ -572,12 +600,12 @@ def _setup_engines( if num_beams > 1: engine_tag += "-beam{}".format(num_beams) - preview_features = [] + preview_features = [PreviewFeature.DISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805] if disable_preview_dynamic_shapes: engine_tag += "-noPreviewFasterDynamicShapes" else: preview_features.append(PreviewFeature.FASTER_DYNAMIC_SHAPES_0805) - + self.gpt2_trt_engine = GPT2ONNXFile( decoder_onnx_fpath, metadata ).as_trt_engine( @@ -606,15 +634,11 @@ def run_trt( perplexity_reference: List[str] = None, ) -> Union[List[NetworkResult], BenchmarkingResult]: - workspace = NNFolderWorkspace( - self.frameworks_cmd.config.network_name, metadata, working_directory - ) + workspace = self._setup_workspace(metadata, working_directory) # no fpath provided for onnx files, download them if len(onnx_fpaths) == 0: - onnx_fpaths = self.frameworks_cmd.generate_and_download_framework( - metadata, workspace - ).onnx + onnx_fpaths = self._download_models(workspace, metadata) else: keep_onnx_model = True keep_torch_model = True @@ -642,7 +666,7 @@ def run_trt( else: for r in perplexity_reference: ppl_results.append( - self.execute_calculate_perplexity(metadata, r) + self.execute_calculate_perplexity(metadata, r, batch_size) ) else: hf_config = AutoConfig.from_pretrained(metadata.variant, use_cache = metadata.other.kv_cache) diff --git a/demo/HuggingFace/NNDF/checkpoints.py b/demo/HuggingFace/NNDF/checkpoints.py index c690a314..3c94ea32 100644 --- a/demo/HuggingFace/NNDF/checkpoints.py +++ b/demo/HuggingFace/NNDF/checkpoints.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/HuggingFace/NNDF/cuda_bootstrapper.py b/demo/HuggingFace/NNDF/cuda_bootstrapper.py new file mode 100644 index 00000000..e9fdb26d --- /dev/null +++ b/demo/HuggingFace/NNDF/cuda_bootstrapper.py @@ -0,0 +1,101 @@ +# +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +""" +Holds logic for modifying and removing invalid CUDA libraries in LD_LIBRARY_PATH. + +Users may have CUDA libraries in LD_LIBRARY_PATH which causes issues with Torch cublas. +This problem only occurs on Linux. +See: + https://github.com/pytorch/pytorch/issues/94294 + https://github.com/pytorch/pytorch/issues/64097 +""" + +import os +import sys +import glob +import shutil + +import subprocess as sp +from NNDF.logger import G_LOGGER + +def bootstrap_ld_library_path() -> bool: + """ + Modifies the LD_LIBRARY_PATH if applicable and then spawns a child process + using first "poetry" and then "python3"/"python" if "poetry" fails. + """ + if os.environ.get("TRT_OSS_DISABLE_BOOTSTRAP") or "linux" not in sys.platform: + return False + + # Walk through each path in environment to see if there are cublas libraries being loaded. + paths = os.environ.get("LD_LIBRARY_PATH", "").split(os.pathsep) + new_paths = [] + modified_path = False + for path in paths: + for lib in ("cublas", "cudart", "cublasLt"): + g = glob.glob(os.path.join(path, f"lib{lib}.so.*")) + if g: + modified_path = True + G_LOGGER.warning(f"Discarding `{path}` from LD_LIBRARY_PATH since it contains CUDA libraries.") + break + else: + new_paths.append(path) + + + if not modified_path: + return False + else: + warning_msg = ("Attempting to bootstrap altered LD_LIBRARY_PATH. " + "\nYou can disable this with TRT_OSS_DISABLE_BOOTSTRAP=1 however frameworks performance may be impacted. " + "\nThere are known issues with cuBLAS loading and PyTorch compatability " + "that is still being resolved for most CUDA <= 12.1 and Torch setups. See: " + "\n - https://github.com/pytorch/pytorch/issues/94294" + "\n - https://github.com/pytorch/pytorch/issues/64097\n") + G_LOGGER.warning(warning_msg) + + G_LOGGER.info(f"CUDA detected in path. Restarting scripts with modified LD_LIBRARY_PATH: {new_paths}") + os.environ["LD_LIBRARY_PATH"] = os.pathsep.join(new_paths) + # To prevent potential recursion, we add one more modification just in case. + os.environ["TRT_OSS_DISABLE_BOOTSTRAP"] = "1" + + # Spawn a new child process instead. + try: + # Use the same python exe that invoked this script + default_python = sys.executable + + # Demo supports both poetry and python3 invocation. + # Check if poetry works first. + cmd = [default_python] + list(sys.argv) + if shutil.which("poetry") is not None: + poetry_cmd = ["poetry", "run"] + cmd + + # Poetry command will be tried. If it fails, we ignore the error and fallback to default python. + try: + # Instantiate a secondary child process. + sp.check_call(" ".join(poetry_cmd), env=dict(os.environ), cwd=os.getcwd(), shell=True) + return True + except: + pass + + # Default python fallback. + sp.check_call(" ".join(cmd), env=dict(os.environ), cwd=os.getcwd(), shell=True) + except Exception as e: + G_LOGGER.error("Unable to start a new process with modified LD_LIBRARY_PATH. Consider removing CUDA lib in LD_LIBRARY_PATH manually.") + G_LOGGER.error(str(e)) + G_LOGGER.warning("Attempting to continue with demo.") + + return True diff --git a/demo/HuggingFace/NNDF/general_utils.py b/demo/HuggingFace/NNDF/general_utils.py index e606c143..f8fb9897 100644 --- a/demo/HuggingFace/NNDF/general_utils.py +++ b/demo/HuggingFace/NNDF/general_utils.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -16,8 +16,8 @@ # """Common utils used by demo folder. -Note: -- For now, users/developers that are contributing to TensorRT OSS should NOT import non-default Python packages in this file, because the test pipeline's boot-up process cannot load extra dependencies. In the near future, alternative solutions such as creating a separate boot-up util list can be possible. +Note: +- For now, users/developers that are contributing to TensorRT OSS should NOT import non-default Python packages in this file, because the test pipeline's boot-up process cannot load extra dependencies. In the near future, alternative solutions such as creating a separate boot-up util list can be possible. - Users/developers that are just using the TensorRT OSS without contributing are still free to modify this file and customize for deployment. """ @@ -170,7 +170,7 @@ def simple_percentile(data, p): Temporary replacement for numpy.percentile() because TRT CI/CD pipeline requires additional packages to be added at boot up in this general_utils.py file. """ assert p >= 0 and p <= 100, "Percentile must be between 1 and 99" - + rank = len(data) * p / 100 if rank.is_integer(): return sorted(data)[int(rank)] @@ -229,7 +229,7 @@ def set_model_path(self, metadata_serialized, is_encoder_decoder: bool) -> str: os.makedirs(self.decoder_path, exist_ok=True) if is_encoder_decoder: self.encoder_path = os.path.join(self.model_path, "encoder") - os.makedirs(self.encoder_path, exist_ok=True) + os.makedirs(self.encoder_path, exist_ok=True) # For decoder only models, there is no encoder else: self.encoder_path = None @@ -242,19 +242,19 @@ def set_model_path(self, metadata_serialized, is_encoder_decoder: bool) -> str: os.makedirs(self.decoder_kv_path, exist_ok=True) return self.model_path, self.encoder_path, self.decoder_path - + def get_path(self) -> str: return self.dpath - + def get_model_path(self) -> str: return self.model_path - + def get_encoder_path(self) -> str: return self.encoder_path - + def get_decoder_path(self) -> str: return self.decoder_path - + def get_decoder_path_kv(self) -> (str, str): if not self.metadata.other.kv_cache: raise RuntimeError("Trying to access kv specific folder in non kv mode") @@ -267,7 +267,7 @@ def cleanup(self, force_remove: bool = False) -> None: ''' if force_remove: return shutil.rmtree(self.dpath) - + if self.is_encoder_decoder_path_set: if self.encoder_path is not None: remove_if_empty(self.encoder_path) @@ -281,6 +281,6 @@ def cleanup(self, force_remove: bool = False) -> None: remove_if_empty( self.decoder_path ) - + remove_if_empty(self.model_path) remove_if_empty(self.dpath) diff --git a/demo/HuggingFace/NNDF/interface.py b/demo/HuggingFace/NNDF/interface.py index 9635fcbd..8d1a739c 100644 --- a/demo/HuggingFace/NNDF/interface.py +++ b/demo/HuggingFace/NNDF/interface.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -35,6 +35,7 @@ TimingProfile, ) from NNDF.logger import G_LOGGER +from NNDF.general_utils import NNFolderWorkspace # externals # None, there should be no external dependencies for testing purposes. @@ -322,6 +323,21 @@ def __init__( # Should be set by self.frameworks_cmd = frameworks_cmd() + def _setup_workspace(self, metadata: NetworkMetadata, working_directory: str) -> NNFolderWorkspace: + return NNFolderWorkspace( + self.frameworks_cmd.config.network_name, metadata, working_directory + ) + + def _download_models( + self, + workspace: NNFolderWorkspace, + metadata: NetworkMetadata, + ) -> Tuple[NetworkModel]: + # No fpath provided for onnx files, download them from HuggingFace repo. + return self.frameworks_cmd.generate_and_download_framework( + metadata, workspace + ).onnx + @abstractmethod def run_trt( self, diff --git a/demo/HuggingFace/NNDF/logger.py b/demo/HuggingFace/NNDF/logger.py index 396280c9..220394b7 100644 --- a/demo/HuggingFace/NNDF/logger.py +++ b/demo/HuggingFace/NNDF/logger.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/HuggingFace/NNDF/models.py b/demo/HuggingFace/NNDF/models.py index dc2dd851..8a51392b 100644 --- a/demo/HuggingFace/NNDF/models.py +++ b/demo/HuggingFace/NNDF/models.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -422,7 +422,7 @@ def as_torch_model( return converter.torch_class(output_fpath, self.network_metadata) return converter.onnx_to_torch(output_fpath, self.fpath, self.network_metadata) - + def _cleanup_onnx_folder(self, folder_dir): for d in os.listdir(folder_dir): fpath = os.path.join(folder_dir, d) diff --git a/demo/HuggingFace/NNDF/networks.py b/demo/HuggingFace/NNDF/networks.py index 9f21fac1..ff8700fc 100644 --- a/demo/HuggingFace/NNDF/networks.py +++ b/demo/HuggingFace/NNDF/networks.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/HuggingFace/NNDF/tensorrt_utils.py b/demo/HuggingFace/NNDF/tensorrt_utils.py index 702d213d..74226ae8 100644 --- a/demo/HuggingFace/NNDF/tensorrt_utils.py +++ b/demo/HuggingFace/NNDF/tensorrt_utils.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -19,6 +19,7 @@ from typing import Dict, List from functools import reduce +from enum import Enum # polygraphy from polygraphy.backend.trt import engine_from_bytes, TrtRunner @@ -28,6 +29,7 @@ # tensorrt import tensorrt as trt +import os # ONNX import onnx @@ -70,21 +72,19 @@ def set_kv_data(kv_dict, past_or_present, layer_id, segment_value_dict): kv_dict: Dict[str, tuple/torch.dtype], the dict to modify within the function past_or_present: str, either "past" or "present" layer_id: int, need kv cache for each decoder layer - segment_value_dict: Dict[str, tuple/torch.dtype], example: + segment_value_dict: Dict[str, tuple/torch.dtype], example: kvcache type: {"encoder": torch.float32, "decoder": torch.float32} kvcache shape: {"encoder": cross_attention_kv_shape, "decoder": self_attention_kv_shape} ''' for segment, value in segment_value_dict.items(): for code in ['key', 'value']: kv_dict[f"{past_or_present}_key_values.{layer_id}.{segment}.{code}"] = value - -def clamp_weights_onnx(onnx_input_fpath: str, onnx_output_fpath: str, min: float, max: float, ignore_nodes: List = None): +def clamp_weights_onnx(graph, min: float, max: float, ignore_nodes: List = None): """ Clamps given onnx model to targeted upper and lower bounds. """ - graph = gs.import_onnx(onnx.load(onnx_input_fpath)) if ignore_nodes is None: ignore_nodes = {} else: @@ -103,24 +103,22 @@ def clamp_weights_onnx(onnx_input_fpath: str, onnx_output_fpath: str, min: float if node_attr is not None: np.clip(node_attr.values, min, max, out=node_attr.values) - - model = gs.export_onnx(graph) - onnx.save(model, onnx_output_fpath, save_as_external_data=False) + + return graph -def clamp_weights_onnx_to_fp16_bounds(onnx_input_fpath: str, onnx_output_fpath: str, ignore_nodes: List = None): +def clamp_weights_onnx_to_fp16_bounds(graph, ignore_nodes: List = None): upper_bound = 65504 - return clamp_weights_onnx(onnx_input_fpath, onnx_output_fpath, -upper_bound, upper_bound, ignore_nodes) + return clamp_weights_onnx(graph, -upper_bound, upper_bound, ignore_nodes) -def move_t5_cast_op(onnx_input_fpath: str, onnx_output_fpath: str): +def move_t5_cast_op(graph): """ T5 encoder and decoder have cast ops after residual add operation. Moving the cast operation before add helps with FP16 accuracy as addition operation can cause overflow in FP16. """ - graph = gs.import_onnx(onnx.load(onnx_input_fpath)) cast_nodes = [node for node in graph.nodes if node.op == "Cast"] # Version check for backward compatibility torch_version_major = int(torch.__version__.split('.')[0]) @@ -180,13 +178,47 @@ def move_t5_cast_op(onnx_input_fpath: str, onnx_output_fpath: str): n.inputs = outs graph.cleanup().toposort() + return graph + +# The current operations would require loading/unloading onnx files twice, +class OnnxProcessOperation(Enum): + CLAMP_WEIGHTS = 1 + MOVE_CAST_OP = 2 + +def process_onnx(config: List[OnnxProcessOperation], onnx_input_fpath, onnx_output_fpath, keep_input = False, **kwargs): + graph = gs.import_onnx(onnx.load(onnx_input_fpath)) + folder = os.path.split(onnx_input_fpath)[0] + for op in config: + if op == OnnxProcessOperation.CLAMP_WEIGHTS: + graph = clamp_weights_onnx_to_fp16_bounds(graph, **kwargs) + elif op == OnnxProcessOperation.MOVE_CAST_OP: + graph = move_t5_cast_op(graph) + model = gs.export_onnx(graph) - onnx.save(model, onnx_output_fpath, save_as_external_data=False) + folder = os.path.split(onnx_input_fpath)[0] + model_size = 0 + for filename in os.listdir(folder): + file_path = os.path.join(folder, filename) + try: + if os.path.isfile(file_path) or os.path.islink(file_path): + model_size += os.stat(file_path).st_size + if not keep_input: + os.unlink(file_path) + + except Exception as e: + print('Failed to delete %s. Reason: %s' % (file_path, e)) + + # Save the weights as external data only when model > 2GB + if model_size >= 1.8 * 1024 * 1024 * 1024: + onnx.save_model(model, onnx_output_fpath, save_as_external_data=True, all_tensors_to_one_file = False, convert_attribute=False) + else: + onnx.save_model(model, onnx_output_fpath, save_as_external_data=False) # Helper Classes class TRTNativeRunner: """TRTNativeRunner avoids the high overheads with Polygraphy runner providing performance comparable to C++ implementation.""" def __init__(self, trt_engine_file: TRTEngineFile, network_metadata: NetworkMetadata): + self.network_metadata = network_metadata self.trt_engine_file = trt_engine_file self.trt_logger = trt.Logger() diff --git a/demo/HuggingFace/NNDF/torch_utils.py b/demo/HuggingFace/NNDF/torch_utils.py index e8569e3f..f3b2fadc 100644 --- a/demo/HuggingFace/NNDF/torch_utils.py +++ b/demo/HuggingFace/NNDF/torch_utils.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/HuggingFace/README.md b/demo/HuggingFace/README.md index a025ee1a..cbc8e9c2 100644 --- a/demo/HuggingFace/README.md +++ b/demo/HuggingFace/README.md @@ -18,11 +18,13 @@ Currently, this repository supports the following models: ## Setup + Follow the setup steps in the TensorRT OSS repository. It is recommended to experiment inside Docker container. For a smoother setup experience, it is recommended to use [Poetry](https://python-poetry.org/) to install requirements and execute: ```bash poetry install # one-time setup +poetry add # see top level repo README.md on how to get TensorRT wheels. poetry run python run.py # execute program ``` @@ -120,7 +122,7 @@ Notes: ## How to run with K-V cache -For all the models (GPT2/BART/T5), use `--enable-kv-cache` option to get the same effect of HuggingFace's `use_cache` option. For encoder-decoder models, this option will use key & value cache in decoder for uni-directional self-attention and encoder-decoder cross-attention. KV cache could reduce the size of `input_ids` and improve runtime performance when `input_ids` is long. Current benchmarking result shows that at `input_seq_len = 1024` and `output_seq_len = 1024`, t5-large model with kv cache could achieve 3x faster than without kv cache in single NVIDIA A100 GPU. +For all the models (GPT2/BART/T5), use `--enable-kv-cache` option to get the same effect of HuggingFace's `use_cache` option. For encoder-decoder models, this option will use key & value cache in decoder for uni-directional self-attention and encoder-decoder cross-attention. KV cache could reduce the size of `input_ids` and improve runtime performance when `input_ids` is long. Current benchmarking result shows that at `input_seq_len = 1024` and `output_seq_len = 1024`, t5-large model with kv cache could achieve 3x faster than without kv cache in single NVIDIA A100 GPU. ```python python3 run.py run BART [frameworks | trt] --variant facebook/bart-base --working-dir temp --enable-kv-cache @@ -131,7 +133,7 @@ Notes: * For BART, we will be porting similar optimization from T5, but currently, K-V cache decoder with TensorRT requires exporting 2 onnx files and building separate engines respectively, called "non-kv" and "kv". For the first decoder run, KV Cache needs to be generated with only `input_ids` and `encoder_hidden_states`(if encoder_decoder), which is named "non-kv". For the other decoder iterations, previous KV Cache and other inputs are passed into the model to generate the updated KV Cache and decoder_hidden_states, which is named "kv". Because current onnx export cannot handle dynamic number of inputs, 2 onnx files with slightly different configurations are used together. -* For GPT2, since it is decoder only, only self attention kv is needed, and it has 2 mode, corresonding to 2 optimization profiles for a single TensorRT engine: context mode which takes in `input_ids` with various length only and outputs `hidden_states` and self attention cache; generation mode, which takes in `input_ids` with seq_len = 1 and entire self attention kv cache, and outputs `hidden_states` with seq_len = 1 and kv cache with cum_seq_len (`past_decoder_length`) + 1. It has some memory concurrency issue that cannot let self attention input and output point to the same memory location, so it requires dual cache. +* For GPT2, since it is decoder only, only self attention kv is needed, and it has 2 mode, corresonding to 2 optimization profiles for a single TensorRT engine: context mode which takes in `input_ids` with various length only and outputs `hidden_states` and self attention cache; generation mode, which takes in `input_ids` with seq_len = 1 and entire self attention kv cache, and outputs `hidden_states` with seq_len = 1 and kv cache with cum_seq_len (`past_decoder_length`) + 1. It has some memory concurrency issue that cannot let self attention input and output point to the same memory location, so it requires dual cache. ## How to run with beam search @@ -179,3 +181,15 @@ pytest ``` It is recommended to use Pytest `4.6.x`. Your Python environment must have already had the setup completed. + + +## Troubleshooting + +### cuBLAS Errors + +``` +CUDA error: CUBLAS_STATUS_INVALID_VALUE when calling `cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc)` +``` + +It is possible that your LD_LIBRARY_PATH has a competing CUDA version stored inside, causing PyTorch to read the incorrect library. +Consider modifying LD_LIBRARY_PATH and removing your CUDA path. diff --git a/demo/HuggingFace/T5/T5ModelConfig.py b/demo/HuggingFace/T5/T5ModelConfig.py index 2db90363..5490fb4b 100644 --- a/demo/HuggingFace/T5/T5ModelConfig.py +++ b/demo/HuggingFace/T5/T5ModelConfig.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -103,7 +103,7 @@ class T5ModelTRTConfig(NNConfig): TARGET_MODELS[3]: 4096, TARGET_MODELS[4]: 5120, } - + MAX_SEQUENCE_LENGTH = { TARGET_MODELS[0]: 512, TARGET_MODELS[1]: 768, @@ -139,7 +139,7 @@ class T5ModelTRTConfig(NNConfig): TARGET_MODELS[2]: 24, TARGET_MODELS[3]: 24, TARGET_MODELS[4]: 24, - } + } NETWORK_FULL_NAME = "full" NETWORK_DECODER_SEGMENT_NAME = "decoder" NETWORK_ENCODER_SEGMENT_NAME = "encoder" @@ -215,12 +215,12 @@ def get_input_dims(metadata) -> Dict: self_attention_past_kv_dims = (Dims.BATCH, "num_heads", Dims.create_new_sequence_dim("past_decoder_length"), "embedding_size_per_head") decoder_inputs_dict[f"past_key_values.{i}.decoder.key"] = self_attention_past_kv_dims decoder_inputs_dict[f"past_key_values.{i}.decoder.value"] = self_attention_past_kv_dims - + # encoder-decoder cross-attention KV cache (dim[0] & dim[2] are dynamic, but dim[2] is constant at each decoding timestep) - cross_attention_past_kv_dims = (Dims.BATCH, "num_heads", Dims.create_new_sequence_dim("encoder_length"), "embedding_size_per_head") + cross_attention_past_kv_dims = (Dims.BATCH, "num_heads", Dims.create_new_sequence_dim("encoder_length"), "embedding_size_per_head") decoder_inputs_dict[f"past_key_values.{i}.encoder.key"] = cross_attention_past_kv_dims decoder_inputs_dict[f"past_key_values.{i}.encoder.value"] = cross_attention_past_kv_dims - + decoder_inputs = [Dims(context_inputs_dict), Dims(decoder_inputs_dict)] else: decoder_inputs_dict = OrderedDict( @@ -262,12 +262,12 @@ def get_output_dims(metadata) -> Dict: self_attention_present_kv_dims = (Dims.BATCH, "num_heads", Dims.create_new_sequence_dim("past_decoder_length"), "embedding_size_per_head") decoder_outputs_dict[f"present_key_values.{i}.decoder.key"] = self_attention_present_kv_dims decoder_outputs_dict[f"present_key_values.{i}.decoder.value"] = self_attention_present_kv_dims - + # encoder-decoder cross-attention KV cache (dim[0] & dim[2] are dynamic, but dim[2] is constant at each decoding timestep) cross_attention_present_kv_dims = (Dims.BATCH, "num_heads", Dims.create_new_sequence_dim("encoder_length"), "embedding_size_per_head") context_outputs_dict[f"present_key_values.{i}.encoder.key"] = cross_attention_present_kv_dims context_outputs_dict[f"present_key_values.{i}.encoder.value"] = cross_attention_present_kv_dims - + decoder_outputs = [Dims(context_outputs_dict), Dims(decoder_outputs_dict)] else: decoder_outputs_dict = OrderedDict( diff --git a/demo/HuggingFace/T5/export.py b/demo/HuggingFace/T5/export.py index 5a8c8bc8..63b7a73e 100644 --- a/demo/HuggingFace/T5/export.py +++ b/demo/HuggingFace/T5/export.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -44,7 +44,7 @@ # TRT-HuggingFace from T5.T5ModelConfig import T5ModelTRTConfig -from NNDF.tensorrt_utils import clamp_weights_onnx_to_fp16_bounds, move_t5_cast_op +from NNDF.tensorrt_utils import OnnxProcessOperation, process_onnx from NNDF.networks import NetworkMetadata, Precision, Dims from NNDF.logger import G_LOGGER from NNDF.models import ( @@ -123,17 +123,18 @@ def __init__(self, decoder, lm_head, config, is_trt = False): self.decoder = decoder self.lm_head = lm_head self.config = config - self.device = "cuda" # HuggingFace's beam search requires to set self.device. Set it to avoid application crash + # HuggingFace's beam search requires to set self.device. Set it to avoid application crash + self.device = torch.device('cuda') # Use hardcoded value to extend compatibility with older HF versions. self.main_input_name = "input_ids" # trt uses cached and precomputed cross attention vs. framework uses the entire kv cache as output. Need to treat them differently. self.is_trt = is_trt def prepare_inputs_for_generation( - self, - input_ids, - past=None, - use_cache=None, + self, + input_ids, + past=None, + use_cache=None, **kwargs ): # cut decoder_input_ids if past is used @@ -148,10 +149,10 @@ def prepare_inputs_for_generation( } def forward( - self, - input_ids, - encoder_hidden_states, - use_cache = None, + self, + input_ids, + encoder_hidden_states, + use_cache = None, past_key_values = None, return_dict = None, **kwargs, @@ -181,9 +182,9 @@ def forward( if not return_dict: return (logits, past_key_values) - + return Seq2SeqLMOutput( - logits=logits, + logits=logits, past_key_values=past_key_values ) @@ -206,16 +207,16 @@ def forward(self, encoder_hidden_states): dummy_hidden_states = torch.zeros(1,1).to(self.device) dummy_position_bias = torch.zeros(1, layer_module.layer[1].EncDecAttention.n_heads, 1, encoder_hidden_states.shape[1]).to(self.device) cross_attention_outputs = layer_module.layer[1]( - hidden_states=dummy_hidden_states, - key_value_states=encoder_hidden_states, - use_cache=True, + hidden_states=dummy_hidden_states, + key_value_states=encoder_hidden_states, + use_cache=True, past_key_value=None, position_bias=dummy_position_bias ) present_key_values = present_key_values + cross_attention_outputs[1] - + return present_key_values - + def __call__(self, *args, **kwargs): return self.forward(*args, **kwargs) @@ -256,9 +257,20 @@ class T5DecoderTRTEngine(TRTEngineFile): def __init__(self, model, network_metadata): super().__init__(model, T5DecoderConverter, network_metadata) self.max_trt_workspace = T5ModelTRTConfig.MAX_DECODER_WORKSPACE_MB[network_metadata.variant] - + def get_network_definition(self, network_definition): + if self.network_metadata.precision.fp16: + for i in range(network_definition[1].num_inputs): + t = network_definition[1].get_input(i) + if t.dtype == trt.float32: + t.dtype = trt.float16 + + for i in range(network_definition[1].num_outputs): + t = network_definition[1].get_output(i) + if t.dtype == trt.float32: + t.dtype = trt.float16 + return add_extra_fp32(network_definition) def use_obey_precision_constraints(self): @@ -393,11 +405,10 @@ def _export_forward(input_ids, encoder_hidden_states, past_key_values): ) if network_metadata.precision.fp16: - clamp_weights_onnx_to_fp16_bounds(output_fpath_kv_generator, output_fpath_kv_generator) + process_onnx([OnnxProcessOperation.CLAMP_WEIGHTS], output_fpath_kv_generator, output_fpath_kv_generator) if network_metadata.precision.fp16: - move_t5_cast_op(output_fpath, output_fpath) - clamp_weights_onnx_to_fp16_bounds(output_fpath, output_fpath) + process_onnx([OnnxProcessOperation.MOVE_CAST_OP, OnnxProcessOperation.CLAMP_WEIGHTS], output_fpath, output_fpath) return T5DecoderONNXFile(output_fpath, network_metadata) @@ -467,7 +478,6 @@ def torch_to_onnx( ) if network_metadata.precision.fp16: - move_t5_cast_op(output_fpath, output_fpath) - clamp_weights_onnx_to_fp16_bounds(output_fpath, output_fpath) + process_onnx([OnnxProcessOperation.MOVE_CAST_OP, OnnxProcessOperation.CLAMP_WEIGHTS], output_fpath, output_fpath) return T5EncoderONNXFile(output_fpath, network_metadata) diff --git a/demo/HuggingFace/T5/frameworks.py b/demo/HuggingFace/T5/frameworks.py index c98e4f9b..2f06128d 100644 --- a/demo/HuggingFace/T5/frameworks.py +++ b/demo/HuggingFace/T5/frameworks.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -91,8 +91,8 @@ def generate_and_download_framework( pytorch_model_dir, use_cache = metadata.other.kv_cache ) - - # These ONNX models can be converted using special encoder and decoder classes. + + # These ONNX models can be converted using special encoder and decoder classes. encoder_onnx_model_fpath = os.path.join(encoder_onnx_root, metadata_serialized + "-encoder.onnx") decoder_onnx_model_fpath = os.path.join(decoder_onnx_root, metadata_serialized + "-decoder-with-lm-head.onnx") @@ -162,10 +162,8 @@ def setup_tokenizer_and_model( # By default, huggingface model structure is one giant file. t5_torch_fpath = network_fpaths.torch[0].fpath t5_model = T5ForConditionalGeneration.from_pretrained(t5_torch_fpath, use_cache=metadata.other.kv_cache) - # Framework fp16 does not support cpu mode for T5 - # TODO: Enable true frameworks fp16. CUDA 11.4 so far does not support model.half() for PyTorch 1.13. - # if metadata.precision.fp16: - # t5_model = t5_model.cuda().half() + if metadata.precision.fp16: + t5_model = t5_model.cuda().half() t5_torch_encoder = T5EncoderTorchFile.TorchModule(t5_model.encoder) t5_torch_decoder = T5DecoderTorchFile.TorchModule( @@ -203,7 +201,7 @@ def execute_inference( t5_torch_encoder, input_ids, timing_profile, use_cuda=(not use_cpu) ) - # Need to feed the decoder a new empty input_ids for text generation. + # Need to feed the decoder a new empty input_ids for text generation. decoder_output_len = output_seq_len // 2 if (not metadata.other.kv_cache) else 1 decoder_input_ids = torch.full( diff --git a/demo/HuggingFace/T5/measurements.py b/demo/HuggingFace/T5/measurements.py index 3f4a8043..3b30e8c1 100644 --- a/demo/HuggingFace/T5/measurements.py +++ b/demo/HuggingFace/T5/measurements.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -41,7 +41,7 @@ def decoder_inference( def decoder_stmt(): t5_decoder( - input_ids=input_ids, encoder_hidden_states=encoder_last_hidden_state, use_cache=use_cache, + input_ids=input_ids, encoder_hidden_states=encoder_last_hidden_state, use_cache=use_cache, past_key_values=past_key_values ) @@ -90,7 +90,7 @@ def _e2e(): encoder_outputs = BaseModelOutput(last_hidden_state = encoder_last_hidden_state), ) return decoder_output - + if isinstance(t5_decoder, TRTNativeRunner): t5_decoder.set_return_device("cuda" if use_cuda else "cpu") diff --git a/demo/HuggingFace/T5/onnxrt.py b/demo/HuggingFace/T5/onnxrt.py index f33b251d..499ba2a5 100644 --- a/demo/HuggingFace/T5/onnxrt.py +++ b/demo/HuggingFace/T5/onnxrt.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -146,7 +146,7 @@ def execute_inference( self.t5_ort_encoder, input_ids, timing_profile ) - # Need to feed the decoder a new empty input_ids for text generation. + # Need to feed the decoder a new empty input_ids for text generation. decoder_output_len = output_seq_len // 2 decoder_input_ids = torch.full( diff --git a/demo/HuggingFace/T5/trt.py b/demo/HuggingFace/T5/trt.py index 3f61d41b..3a2decc2 100644 --- a/demo/HuggingFace/T5/trt.py +++ b/demo/HuggingFace/T5/trt.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -129,7 +129,7 @@ def __init__( # In benchmarking mode, the max_sequence_length should be the designated input_profile_max_len if benchmarking_args is not None and benchmarking_args.input_profile_max_len is not None: self.max_sequence_length = benchmarking_args.input_profile_max_len - else: + else: self.max_sequence_length = hf_config.d_model self.encoder_hidden_size = hf_config.d_model self.main_input_name = "input_ids" @@ -201,20 +201,21 @@ def __init__( benchmarking_args: T5TRTBenchmarkingArgs = None, ): super().__init__(trt_engine_file, network_metadata, hf_config, batch_size = batch_size) - self.data_type = torch.float32 + self.data_type = torch.float32 if not network_metadata.precision.fp16 else torch.float16 # In benchmarking mode, the max_sequence_length should be the user-provided input_profile_max_len if benchmarking_args is not None and benchmarking_args.input_profile_max_len is not None: self.max_input_length = benchmarking_args.input_profile_max_len - else: + else: self.max_input_length = hf_config.d_model - + # Similarly, the max_output_length should be the user-provided output_profile_max_len if benchmarking_args is not None and benchmarking_args.output_profile_max_len is not None: self.max_output_length = benchmarking_args.output_profile_max_len - else: + else: self.max_output_length = hf_config.d_model - + + self.device = torch.device('cuda') self.main_input_name = "input_ids" self.encoder_hidden_size = hf_config.d_model self.num_heads = hf_config.num_heads @@ -246,7 +247,7 @@ def __init__( input_idx = self.trt_engine.get_binding_index("past_" + self_attention_name) self.self_attention_cache[self_attention_name] = input_buffer self.bindings[input_idx] = input_buffer.data_ptr() - + output_idx = self.trt_engine.get_binding_index("present_" + self_attention_name) self.bindings[output_idx] = input_buffer.data_ptr() @@ -262,22 +263,21 @@ def __init__( # Optimization bit self.persist_encoder_hidden_states = False - self.encoder_hidden_states = None + self.encoder_hidden_states = torch.zeros((self.batch_size * num_beams * self.max_input_length * self.encoder_hidden_size), dtype=self.data_type).cuda() + self.bindings[1] = self.encoder_hidden_states.data_ptr() self.persist_cross_attention_kv_cache = False - self.return_device = "cuda" + self.return_device = torch.device('cuda') self.variant = network_metadata.variant # record variant name to later index the vocab_size in forward() - + def set_encoder_hidden_states_for_inference_cycle(self, encoder_hidden_states): """Used to cache encoder hidden state runs across same encoder sessions""" - if encoder_hidden_states.device == torch.device("cpu"): - self.encoder_hidden_states = encoder_hidden_states.cuda() - else: - self.encoder_hidden_states = encoder_hidden_states - - self.bindings[1] = self.encoder_hidden_states.data_ptr() + # Use in-place assignment so that the memory location of self.encoder_hidden_states will never change. + # PyTorch will handle the FP32->FP16 conversion automatically if that is needed. + self.encoder_hidden_states[:encoder_hidden_states.numel()] = encoder_hidden_states.flatten() self.persist_encoder_hidden_states = True + self.trt_context.set_binding_shape(1, encoder_hidden_states.shape) def set_cross_attention_kv_cache_engine(self, cross_attention_kv_generator): self.cross_attention_kv_generator = cross_attention_kv_generator @@ -285,6 +285,12 @@ def set_cross_attention_kv_cache_engine(self, cross_attention_kv_generator): trt_runtime = trt.Runtime(self.trt_logger) self.cross_attention_kv_generator_trt_engine = trt_runtime.deserialize_cuda_engine(f.read()) self.cross_attention_kv_generator_trt_context = self.cross_attention_kv_generator_trt_engine.create_execution_context() + self.cross_attention_bindings = [None] * self.cross_attention_kv_generator_trt_engine.num_bindings + self.cross_attention_bindings[0] = self.encoder_hidden_states.data_ptr() + # Cross attention cache as outputs + for i in range(self.num_decoder_layers): + self.cross_attention_bindings[2*i+1] = self.cross_attention_cache[f"past_key_values.{i}.encoder.key"].data_ptr() + self.cross_attention_bindings[2*i+2] = self.cross_attention_cache[f"past_key_values.{i}.encoder.value"].data_ptr() def set_cross_attention_kv_cache_for_inference_cycle(self, encoder_hidden_states): """ @@ -293,17 +299,8 @@ def set_cross_attention_kv_cache_for_inference_cycle(self, encoder_hidden_states Unlike self-attention cache, cross attention is constant during the decoding process, so we only need to set its bindings once at the first decoding step, and skip in all later steps (by self.persist_cross_attention_kv_cache flag) """ self.cross_attention_kv_generator_trt_context.set_binding_shape(0, encoder_hidden_states.shape) - bindings = [None] * self.cross_attention_kv_generator_trt_engine.num_bindings - bindings[0] = encoder_hidden_states.data_ptr() assert self.cross_attention_kv_generator_trt_context.all_binding_shapes_specified - - cross_attention_kv_shape_output = (encoder_hidden_states.shape[0], self.num_heads, self.max_input_length, self.embedding_size_per_head) - # Cross attention cache as outputs - for i in range(self.num_decoder_layers): - bindings[2*i+1] = self.cross_attention_cache[f"past_key_values.{i}.encoder.key"].data_ptr() - bindings[2*i+2] = self.cross_attention_cache[f"past_key_values.{i}.encoder.value"].data_ptr() - - self.cross_attention_kv_generator_trt_context.execute_v2(bindings=bindings) + self.cross_attention_kv_generator_trt_context.execute_v2(bindings=self.cross_attention_bindings) self.persist_cross_attention_kv_cache = True def set_return_device(self, return_device): @@ -343,7 +340,7 @@ def _reorder_cache(self, past, beam_idx): reordered_decoder_past = reordered_decoder_past + (reordered_layer_past_states,) return reordered_decoder_past - def forward(self, input_ids, encoder_hidden_states, *args, **kwargs): + def forward(self, input_ids, encoder_hidden_states, encoder_outputs=None, *args, **kwargs): # Get the batch size. bs = input_ids.shape[0] # in beam search mode, bs is batch_size * num_beams @@ -366,8 +363,6 @@ def forward(self, input_ids, encoder_hidden_states, *args, **kwargs): if not self.persist_encoder_hidden_states: self.set_encoder_hidden_states_for_inference_cycle(encoder_hidden_states) - self.trt_context.set_binding_shape(1, self.encoder_hidden_states.shape) - if self.config.use_cache: if (kwargs.get("past_key_values") is None): self.past_decoder_length = 0 @@ -388,7 +383,9 @@ def forward(self, input_ids, encoder_hidden_states, *args, **kwargs): assert self.trt_context.all_binding_shapes_specified self.trt_context.execute_v2(bindings=self.bindings) - logits = self.hidden_states[:,:input_length,:] + # For bs > 1, this is required, so cannot avoid this D2D copy + logits_length = bs * input_length * self.config.vocab_size + logits = self.hidden_states.flatten()[:logits_length].view(bs, input_length, self.config.vocab_size) if is_cpu_mode: logits = logits.cpu() @@ -414,7 +411,7 @@ def forward(self, input_ids, encoder_hidden_states, *args, **kwargs): def prepare_inputs_for_generation(self, input_ids, past=None, use_cache=None, **kwargs): # In HuggingFace generation_utils.py, this function will be called at each decoding step, before running the decoder's forward(). - + if past is not None: input_ids = input_ids[:, -1:] @@ -428,13 +425,13 @@ def prepare_inputs_for_generation(self, input_ids, past=None, use_cache=None, ** ret["past_key_values"] = past return ret - + def reset(self): ''' You should always call this function after a use case because T5TRTDecoder does not clear the cached encoder_hidden_states or cross_attention itself. ''' self.persist_encoder_hidden_states = False - self.encoder_hidden_states = None + self.encoder_hidden_states.zero_() if self.config.use_cache: self.persist_cross_attention_kv_cache = False @@ -487,9 +484,9 @@ def generate( if min_length is None: min_length = T5ModelTRTConfig.MIN_OUTPUT_LENGTH[self.metadata.variant] - + encoder_last_hidden_state = self.t5_trt_encoder(input_ids=input_ids).to("cuda") - + decoder_output = self.t5_trt_decoder.generate( input_ids, max_length = max_length, @@ -504,7 +501,7 @@ def generate( self.t5_trt_decoder.reset() return decoder_output - + def execute_inference( self, metadata: NetworkMetadata, @@ -526,7 +523,7 @@ def execute_inference( else: input_seq_len = benchmarking_args.input_seq_len output_seq_len = benchmarking_args.output_seq_len - + input_ids = torch.randint(0, hf_config.vocab_size, (batch_size, input_seq_len)) encoder_last_hidden_state, encoder_e2e_time = encoder_inference( @@ -599,7 +596,7 @@ def execute_inference( # Remove the padding and end tokens. semantic_outputs = tokenizer.decode( - decoder_output[-1, :], skip_special_tokens=True + decoder_output[0, :], skip_special_tokens=True ) if isinstance(semantic_outputs, list): @@ -618,10 +615,11 @@ def execute_calculate_perplexity( metadata: NetworkMetadata, encoder_input: str, decoder_input: str, + batch_size: int, ): tokenizer = T5Tokenizer.from_pretrained(metadata.variant) - encoder_input_ids = tokenizer([encoder_input], padding=True, return_tensors="pt").input_ids - decoder_input_ids = tokenizer([decoder_input], padding=True, return_tensors="pt").input_ids + encoder_input_ids = tokenizer([encoder_input] * batch_size, padding=True, return_tensors="pt").input_ids + decoder_input_ids = tokenizer([decoder_input] * batch_size, padding=True, return_tensors="pt").input_ids perplexity = calculate_perplexity( self.t5_trt_encoder, self.t5_trt_decoder, tokenizer, encoder_input_ids, decoder_input_ids, @@ -629,21 +627,6 @@ def execute_calculate_perplexity( ) return perplexity - def _setup_workspace(self, metadata: NetworkMetadata, working_directory: str) -> NNFolderWorkspace: - return NNFolderWorkspace( - self.frameworks_cmd.config.network_name, metadata, working_directory - ) - - def _download_models( - self, - workspace: NNFolderWorkspace, - metadata: NetworkMetadata, - ) -> Tuple[NetworkModel]: - # No fpath provided for onnx files, download them from HuggingFace repo. - return self.frameworks_cmd.generate_and_download_framework( - metadata, workspace - ).onnx - def _setup_engines( self, metadata: NetworkMetadata, @@ -703,7 +686,7 @@ def _setup_engines( # Set up the non kv engine, used for non-kv mode and kv mode generation phase (1st decoder run uses the non-kv profile to generate kv cache) dec_profiles = Profile() - + # for beam search, decoder engine's inputs are expanded `num_beams` times # optimization profiles should be changed accordingly, but onnx models can be shared across greedy/beam because the first dim (batch size) is already a dynamic value, so no change needed in export.py if not hf_config.use_cache: @@ -727,7 +710,7 @@ def _setup_engines( opt=(batch_size * num_beams, opt_input_seq_len, encoder_hidden_size), max=(batch_size * num_beams, max_input_length, encoder_hidden_size), ) - + if hf_config.use_cache: num_heads = hf_config.num_heads @@ -761,7 +744,7 @@ def _setup_engines( f"past_key_values.{i}.encoder.value", **cross_attention_profile ) - + decoder_profiles = [dec_profiles] # Convert ONNX models to TRT engines. @@ -777,7 +760,7 @@ def _setup_engines( if num_beams > 1: engine_tag += "-beam{}".format(num_beams) - preview_features = [] + preview_features = [PreviewFeature.DISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805] if disable_preview_dynamic_shapes: engine_tag += "-noPreviewFasterDynamicShapes" else: @@ -825,7 +808,7 @@ def _setup_engines( profiles=cross_attention_kv_generation_profiles, preview_features=preview_features ) - + self.t5_trt_decoder.set_cross_attention_kv_cache_engine(self.t5_trt_cross_attention_kv_generator) def run_trt( @@ -876,7 +859,7 @@ def run_trt( else: for ei, di in zip(network_input, perplexity_reference): ppl_results.append( - self.execute_calculate_perplexity(metadata, ei, di) + self.execute_calculate_perplexity(metadata, ei, di, batch_size) ) self.t5_trt_decoder.reset() diff --git a/demo/HuggingFace/notebooks/bart-playground.ipynb b/demo/HuggingFace/notebooks/bart-playground.ipynb index c59c5264..59e0e20f 100644 --- a/demo/HuggingFace/notebooks/bart-playground.ipynb +++ b/demo/HuggingFace/notebooks/bart-playground.ipynb @@ -7,7 +7,7 @@ "metadata": {}, "outputs": [], "source": [ - "# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.\n", + "# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.\n", "# SPDX-License-Identifier: Apache-2.0\n", "#\n", "# Licensed under the Apache License, Version 2.0 (the \"License\");\n", diff --git a/demo/HuggingFace/notebooks/bart.ipynb b/demo/HuggingFace/notebooks/bart.ipynb index dbd574af..5a9dd70b 100644 --- a/demo/HuggingFace/notebooks/bart.ipynb +++ b/demo/HuggingFace/notebooks/bart.ipynb @@ -7,7 +7,7 @@ "metadata": {}, "outputs": [], "source": [ - "# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.\n", + "# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.\n", "# SPDX-License-Identifier: Apache-2.0\n", "#\n", "# Licensed under the Apache License, Version 2.0 (the \"License\");\n", @@ -673,7 +673,7 @@ "if num_beams > 1:\n", " engine_tag += \"-beam{}\".format(num_beams)\n", "\n", - "preview_features = []\n", + "preview_features = [PreviewFeature.DISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805]\n", "if disable_preview_dynamic_shapes:\n", " engine_tag += \"-noPreviewFasterDynamicShapes\"\n", "else:\n", diff --git a/demo/HuggingFace/notebooks/gpt2.ipynb b/demo/HuggingFace/notebooks/gpt2.ipynb index cf96475c..745b996b 100644 --- a/demo/HuggingFace/notebooks/gpt2.ipynb +++ b/demo/HuggingFace/notebooks/gpt2.ipynb @@ -509,10 +509,11 @@ "opt_length = input_id.shape[1] if use_input_length else max_length // 2 \n", "# Create different engine tags for different configurations\n", "engine_tag = f\"bs{batch_size}\"\n", - "preview_features = [PreviewFeature.FASTER_DYNAMIC_SHAPES_0805]\n", + "preview_features = [PreviewFeature.DISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805]\n", "if disable_preview_dynamic_shapes:\n", - " engine_tag += \"-disableFasterDynamicShapes\"\n", - " preview_features = []\n", + " engine_tag += \"-noPreviewFasterDynamicShapes\"\n", + "else:\n", + " preview_features += [PreviewFeature.FASTER_DYNAMIC_SHAPES_0805]\n", "\n", "profiles = [Profile().add(\n", " \"input_ids\",\n", diff --git a/demo/HuggingFace/notebooks/t5.ipynb b/demo/HuggingFace/notebooks/t5.ipynb index b752108e..c708e04e 100644 --- a/demo/HuggingFace/notebooks/t5.ipynb +++ b/demo/HuggingFace/notebooks/t5.ipynb @@ -351,10 +351,14 @@ "outputs": [], "source": [ "onnx_model_path = './models/{}/ONNX'.format(T5_VARIANT)\n", - "!mkdir -p $onnx_model_path\n", "\n", "metadata=NetworkMetadata(variant=T5_VARIANT, precision=Precision(fp16=True), other=T5Metadata(kv_cache=False))\n", "\n", + "encoder_onnx_model_path = os.path.join(onnx_model_path, \"encoder\")\n", + "decoder_onnx_model_path = os.path.join(onnx_model_path, \"decoder\")\n", + "!mkdir -p $encoder_onnx_model_path\n", + "!mkdir -p $decoder_onnx_model_path\n", + "\n", "encoder_onnx_model_fpath = T5_VARIANT + \"-encoder.onnx\"\n", "decoder_onnx_model_fpath = T5_VARIANT + \"-decoder-with-lm-head.onnx\"\n", "\n", @@ -362,10 +366,10 @@ "t5_decoder = T5DecoderTorchFile(t5_model.to('cpu'), metadata)\n", "\n", "onnx_t5_encoder = t5_encoder.as_onnx_model(\n", - " os.path.join(onnx_model_path, encoder_onnx_model_fpath), force_overwrite=False\n", + " os.path.join(encoder_onnx_model_path, encoder_onnx_model_fpath), force_overwrite=False\n", ")\n", "onnx_t5_decoder = t5_decoder.as_onnx_model(\n", - " os.path.join(onnx_model_path, decoder_onnx_model_fpath), force_overwrite=False\n", + " os.path.join(decoder_onnx_model_path, decoder_onnx_model_fpath), force_overwrite=False\n", ")" ] }, @@ -444,7 +448,7 @@ "if num_beams > 1:\n", " engine_tag += \"-beam{}\".format(num_beams)\n", "\n", - "preview_features = []\n", + "preview_features = [PreviewFeature.DISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805]\n", "if disable_preview_dynamic_shapes:\n", " engine_tag += \"-noFasterDynamicShapes\"\n", "else:\n", @@ -454,7 +458,7 @@ "decoder_engine_name = os.path.join(tensorrt_model_path, decoder_onnx_model_fpath) + f\"-{engine_tag}.engine\"\n", "\n", "if not os.path.exists(encoder_engine_name):\n", - " t5_trt_encoder_engine = T5EncoderONNXFile(os.path.join(onnx_model_path, encoder_onnx_model_fpath), metadata).as_trt_engine(\n", + " t5_trt_encoder_engine = T5EncoderONNXFile(os.path.join(encoder_onnx_model_path, encoder_onnx_model_fpath), metadata).as_trt_engine(\n", " encoder_engine_name,\n", " profiles=[encoder_profile],\n", " preview_features=preview_features)\n", @@ -462,7 +466,7 @@ " t5_trt_encoder_engine = T5EncoderTRTEngine(encoder_engine_name, metadata)\n", "\n", "if not os.path.exists(decoder_engine_name):\n", - " t5_trt_decoder_engine = T5DecoderONNXFile(os.path.join(onnx_model_path, decoder_onnx_model_fpath), metadata).as_trt_engine(\n", + " t5_trt_decoder_engine = T5DecoderONNXFile(os.path.join(decoder_onnx_model_path, decoder_onnx_model_fpath), metadata).as_trt_engine(\n", " decoder_engine_name,\n", " profiles=[decoder_profile],\n", " preview_features=preview_features)\n", @@ -621,6 +625,14 @@ "\n", "If you are interested in further details of the conversion process, check out [T5/trt.py](../T5/trt.py)" ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b6a8b7c8", + "metadata": {}, + "outputs": [], + "source": [] } ], "metadata": { @@ -639,7 +651,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.10" + "version": "3.10.6" }, "vscode": { "interpreter": { diff --git a/demo/HuggingFace/requirements.txt b/demo/HuggingFace/requirements.txt index db19f166..30d9cdb1 100644 --- a/demo/HuggingFace/requirements.txt +++ b/demo/HuggingFace/requirements.txt @@ -18,12 +18,13 @@ huggingface-hub==0.11.0; python_version>="3.7" huggingface-hub==0.4.0; python_version<"3.7" transformers==4.20.0; python_version>="3.7" transformers==4.18.0; python_version<"3.7" -torch <= 1.11 +torch==1.13.1; python_version>="3.7" +torch==1.10; python_version<"3.7" sentencepiece==0.1.95; python_version<"3.10" sentencepiece==0.1.97; python_version>="3.10" --extra-index-url https://pypi.ngc.nvidia.com onnx==1.9.0; python_version<"3.8" -onnx==1.12.0; python_version>="3.8" +onnx==1.13.1; python_version>="3.8" polygraphy>=0.42.2 tabulate toml diff --git a/demo/HuggingFace/run.py b/demo/HuggingFace/run.py index d9752128..3521b57f 100644 --- a/demo/HuggingFace/run.py +++ b/demo/HuggingFace/run.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -45,6 +45,7 @@ # NNDF from NNDF.general_utils import process_per_result_entries, process_results, register_network_folders, RANDOM_SEED from NNDF.logger import G_LOGGER +from NNDF.cuda_bootstrapper import bootstrap_ld_library_path # huggingface from transformers import set_seed @@ -298,6 +299,12 @@ def main() -> None: # Delegate parser to action specifics action = get_action(known_args.action, networks, parser) known_args, _ = parser.parse_known_args() + + # If bootstrap occurs, then the spawned process completes the rest of demo. + # We can exit safely. We spawn after parsing basic args to reduce loading churn on rudimentary help commands. + if bootstrap_ld_library_path(): + sys.exit(0) + return action.execute(known_args) diff --git a/demo/HuggingFace/tests/test_interface.py b/demo/HuggingFace/tests/test_interface.py index ebf1cca7..9dda902f 100644 --- a/demo/HuggingFace/tests/test_interface.py +++ b/demo/HuggingFace/tests/test_interface.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/common/audio_processing.py b/demo/Tacotron2/common/audio_processing.py index ac7f4b12..090581d5 100644 --- a/demo/Tacotron2/common/audio_processing.py +++ b/demo/Tacotron2/common/audio_processing.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/common/layers.py b/demo/Tacotron2/common/layers.py index 2996e849..cbeb4910 100644 --- a/demo/Tacotron2/common/layers.py +++ b/demo/Tacotron2/common/layers.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/common/stft.py b/demo/Tacotron2/common/stft.py index 77d33ade..59700e99 100644 --- a/demo/Tacotron2/common/stft.py +++ b/demo/Tacotron2/common/stft.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/common/utils.py b/demo/Tacotron2/common/utils.py index a874259f..6cccbf22 100644 --- a/demo/Tacotron2/common/utils.py +++ b/demo/Tacotron2/common/utils.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/data_functions.py b/demo/Tacotron2/data_functions.py index 8e2c0011..623e5af6 100644 --- a/demo/Tacotron2/data_functions.py +++ b/demo/Tacotron2/data_functions.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/inference.py b/demo/Tacotron2/inference.py index 8aa7d0ec..77bbccc1 100644 --- a/demo/Tacotron2/inference.py +++ b/demo/Tacotron2/inference.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/inference_perf.py b/demo/Tacotron2/inference_perf.py index 3d07a0bf..cb13463e 100644 --- a/demo/Tacotron2/inference_perf.py +++ b/demo/Tacotron2/inference_perf.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/loss_functions.py b/demo/Tacotron2/loss_functions.py index 1cae61fc..7ee1a5b2 100644 --- a/demo/Tacotron2/loss_functions.py +++ b/demo/Tacotron2/loss_functions.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/main.py b/demo/Tacotron2/main.py index ac309808..2fee8563 100644 --- a/demo/Tacotron2/main.py +++ b/demo/Tacotron2/main.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/models.py b/demo/Tacotron2/models.py index c3d91ab5..fad8af46 100644 --- a/demo/Tacotron2/models.py +++ b/demo/Tacotron2/models.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/multiproc.py b/demo/Tacotron2/multiproc.py index 4bfd3bc6..d3eb63ad 100644 --- a/demo/Tacotron2/multiproc.py +++ b/demo/Tacotron2/multiproc.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/preprocess_audio2mel.py b/demo/Tacotron2/preprocess_audio2mel.py index e57709ec..32026325 100644 --- a/demo/Tacotron2/preprocess_audio2mel.py +++ b/demo/Tacotron2/preprocess_audio2mel.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/tacotron2/arg_parser.py b/demo/Tacotron2/tacotron2/arg_parser.py index e0044088..2a450ef6 100644 --- a/demo/Tacotron2/tacotron2/arg_parser.py +++ b/demo/Tacotron2/tacotron2/arg_parser.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/tacotron2/data_function.py b/demo/Tacotron2/tacotron2/data_function.py index f89e8793..5d2c0064 100644 --- a/demo/Tacotron2/tacotron2/data_function.py +++ b/demo/Tacotron2/tacotron2/data_function.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/tacotron2/loss_function.py b/demo/Tacotron2/tacotron2/loss_function.py index 2ef0f15d..07b3610e 100644 --- a/demo/Tacotron2/tacotron2/loss_function.py +++ b/demo/Tacotron2/tacotron2/loss_function.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/tacotron2/model.py b/demo/Tacotron2/tacotron2/model.py index e90bbd4e..c8ba9f96 100644 --- a/demo/Tacotron2/tacotron2/model.py +++ b/demo/Tacotron2/tacotron2/model.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/tacotron2/text/cleaners.py b/demo/Tacotron2/tacotron2/text/cleaners.py index f34a56fc..4cbcb015 100644 --- a/demo/Tacotron2/tacotron2/text/cleaners.py +++ b/demo/Tacotron2/tacotron2/text/cleaners.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/tacotron2/text/cmudict.py b/demo/Tacotron2/tacotron2/text/cmudict.py index a18ccf6a..b359b235 100644 --- a/demo/Tacotron2/tacotron2/text/cmudict.py +++ b/demo/Tacotron2/tacotron2/text/cmudict.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/tacotron2/text/numbers.py b/demo/Tacotron2/tacotron2/text/numbers.py index 365b9e64..43df588d 100644 --- a/demo/Tacotron2/tacotron2/text/numbers.py +++ b/demo/Tacotron2/tacotron2/text/numbers.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/tacotron2/text/symbols.py b/demo/Tacotron2/tacotron2/text/symbols.py index 090ef204..604626ec 100644 --- a/demo/Tacotron2/tacotron2/text/symbols.py +++ b/demo/Tacotron2/tacotron2/text/symbols.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/tensorrt/convert_onnx2trt.py b/demo/Tacotron2/tensorrt/convert_onnx2trt.py index 64f83a68..ec43cb05 100644 --- a/demo/Tacotron2/tensorrt/convert_onnx2trt.py +++ b/demo/Tacotron2/tensorrt/convert_onnx2trt.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -64,7 +64,6 @@ def parse_args(parser): required=False) parser.add_argument("--disable-preview-dynamic-shapes", action="store_true", help="Disable dynamic shape preview feature.") parser.set_defaults(loop=int(trt.__version__[0]) >= 8) - return parser diff --git a/demo/Tacotron2/tensorrt/convert_tacotron22onnx.py b/demo/Tacotron2/tensorrt/convert_tacotron22onnx.py index 09420e1c..361a2221 100644 --- a/demo/Tacotron2/tensorrt/convert_tacotron22onnx.py +++ b/demo/Tacotron2/tensorrt/convert_tacotron22onnx.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/tensorrt/convert_waveglow2onnx.py b/demo/Tacotron2/tensorrt/convert_waveglow2onnx.py index 8894c2f0..4b9aecbc 100644 --- a/demo/Tacotron2/tensorrt/convert_waveglow2onnx.py +++ b/demo/Tacotron2/tensorrt/convert_waveglow2onnx.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/tensorrt/generate_decoder.py b/demo/Tacotron2/tensorrt/generate_decoder.py index ec7c2c34..62f8b04e 100644 --- a/demo/Tacotron2/tensorrt/generate_decoder.py +++ b/demo/Tacotron2/tensorrt/generate_decoder.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -209,4 +209,4 @@ def insert_decoder_loop(decoder_iter_onnx_path, output_dir, decoder_out_name, fp if args.decoder_out == None: args.decoder_out = "decoder_with_outer_loop_{}.onnx".format("fp16" if args.fp16 else "fp32") - insert_decoder_loop(args.model_path, args.output_dir, args.decoder_out, args.fp16) \ No newline at end of file + insert_decoder_loop(args.model_path, args.output_dir, args.decoder_out, args.fp16) diff --git a/demo/Tacotron2/tensorrt/inference_trt.py b/demo/Tacotron2/tensorrt/inference_trt.py index 87fd7324..4f5f76d3 100644 --- a/demo/Tacotron2/tensorrt/inference_trt.py +++ b/demo/Tacotron2/tensorrt/inference_trt.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -244,7 +244,7 @@ def infer_tacotron2_trt(encoder, decoder_iter, postnet, mel_outputs = torch.tensor(result[0], device=device) mel_lengths = torch.tensor(result[1], device=device) - else: + else: print("Running Tacotron2 Decoder with loop") decoder_tensors = { "inputs" : @@ -449,10 +449,10 @@ def main(): with encoder_context, postnet_context: pass - + if decoder_context is not None: with decoder_context: pass - + if waveglow_context is not None: with waveglow_context: pass diff --git a/demo/Tacotron2/tensorrt/test_infer_trt.py b/demo/Tacotron2/tensorrt/test_infer_trt.py index 36dbc6f9..7023f02f 100644 --- a/demo/Tacotron2/tensorrt/test_infer_trt.py +++ b/demo/Tacotron2/tensorrt/test_infer_trt.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -194,7 +194,7 @@ def main(): with MeasureTime(measurements, "waveglow_latency"): audios = infer_waveglow_trt(waveglow, waveglow_context, mel, measurements, args.fp16) - + num_mels = mel.size(0)*mel.size(2) num_samples = audios.size(0)*audios.size(1) diff --git a/demo/Tacotron2/tensorrt/trt_utils.py b/demo/Tacotron2/tensorrt/trt_utils.py index 06b072fe..3e1d534a 100644 --- a/demo/Tacotron2/tensorrt/trt_utils.py +++ b/demo/Tacotron2/tensorrt/trt_utils.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -27,7 +27,7 @@ def parse_dynamic_size(dim): split = str(dim).split(',') assert len(split) in (1,3) , "Dynamic size input must be either 1 or 3 comma-separated integers" ints = [int(i) for i in split] - + if len(ints) == 1: ints *= 3 diff --git a/demo/Tacotron2/test_infer.py b/demo/Tacotron2/test_infer.py index 5c5217db..81254d37 100644 --- a/demo/Tacotron2/test_infer.py +++ b/demo/Tacotron2/test_infer.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/train.py b/demo/Tacotron2/train.py index c3cee862..55a9e56f 100644 --- a/demo/Tacotron2/train.py +++ b/demo/Tacotron2/train.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/waveglow/arg_parser.py b/demo/Tacotron2/waveglow/arg_parser.py index ef18272b..7002bf6d 100644 --- a/demo/Tacotron2/waveglow/arg_parser.py +++ b/demo/Tacotron2/waveglow/arg_parser.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/waveglow/data_function.py b/demo/Tacotron2/waveglow/data_function.py index f41a31c0..62076eba 100644 --- a/demo/Tacotron2/waveglow/data_function.py +++ b/demo/Tacotron2/waveglow/data_function.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/waveglow/denoiser.py b/demo/Tacotron2/waveglow/denoiser.py index dc798ee1..5dc2d789 100644 --- a/demo/Tacotron2/waveglow/denoiser.py +++ b/demo/Tacotron2/waveglow/denoiser.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/waveglow/loss_function.py b/demo/Tacotron2/waveglow/loss_function.py index f3f5ce1c..75620df9 100644 --- a/demo/Tacotron2/waveglow/loss_function.py +++ b/demo/Tacotron2/waveglow/loss_function.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/demo/Tacotron2/waveglow/model.py b/demo/Tacotron2/waveglow/model.py index eee0145c..00a26421 100644 --- a/demo/Tacotron2/waveglow/model.py +++ b/demo/Tacotron2/waveglow/model.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -18,6 +18,7 @@ import torch from torch.autograd import Variable import torch.nn.functional as F +import numpy as np @torch.jit.script @@ -293,7 +294,13 @@ def infer_onnx(self, spect, z, sigma=0.9): audio = z[:, :self.n_remaining_channels, :, :] z = z[:, self.n_remaining_channels:self.n_group, :, :] - audio = sigma*audio + + # Convert sigma to a torch tensor to ensure constant is exported properly + if audio.type() == 'torch.cuda.HalfTensor' or audio.type() == 'torch.HalfTensor': + sigma = torch.tensor(np.float16(sigma)) + else: + sigma = torch.tensor(np.float32(sigma)) + audio = sigma * audio for k in reversed(range(self.n_flows)): n_half = int(audio.size(1) // 2) diff --git a/docker/build.sh b/docker/build.sh index 39d37a32..6b28fd09 100755 --- a/docker/build.sh +++ b/docker/build.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -18,7 +18,6 @@ arg_dockerfile=docker/ubuntu-20.04.Dockerfile arg_imagename=tensorrt-ubuntu -arg_cudaversion=12.0.1 arg_help=0 while [[ "$#" -gt 0 ]]; do case $1 in @@ -38,7 +37,13 @@ if [ "$arg_help" -eq "1" ]; then exit; fi -docker_args="-f $arg_dockerfile --build-arg CUDA_VERSION=$arg_cudaversion --build-arg uid=$(id -u) --build-arg gid=$(id -g) --tag=$arg_imagename ." +if [ -z "$arg_cudaversion" ] +then + echo "--cuda not specified, so not passing in --build-arg CUDA_VERSION to Dockerfile" + docker_args="-f $arg_dockerfile --build-arg uid=$(id -u) --build-arg gid=$(id -g) --tag=$arg_imagename ." +else + docker_args="-f $arg_dockerfile --build-arg CUDA_VERSION=$arg_cudaversion --build-arg uid=$(id -u) --build-arg gid=$(id -g) --tag=$arg_imagename ." +fi echo "Building container:" echo "> docker build $docker_args" diff --git a/docker/centos-7.Dockerfile b/docker/centos-7.Dockerfile index 164b1533..ff27d6d2 100644 --- a/docker/centos-7.Dockerfile +++ b/docker/centos-7.Dockerfile @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -16,12 +16,11 @@ # ARG CUDA_VERSION=12.0.1 -ARG OS_VERSION=7 -FROM nvidia/cuda:${CUDA_VERSION}-cudnn8-devel-centos${OS_VERSION} +FROM nvidia/cuda:${CUDA_VERSION}-cudnn8-devel-centos7 LABEL maintainer="NVIDIA CORPORATION" -ENV TRT_VERSION 8.6.0.12 +ENV TRT_VERSION 8.6.1.6 SHELL ["/bin/bash", "-c"] # Setup user account @@ -48,19 +47,26 @@ RUN yum -y install \ # Install python3 RUN yum install -y python36 python3-devel +# yum needs to use python2 +RUN sed -i "1s/python/python2/" /usr/bin/yum + # Install TensorRT RUN if [ "${CUDA_VERSION}" = "10.2" ] ; then \ v="${TRT_VERSION%.*}-1.cuda${CUDA_VERSION}" &&\ yum-config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/cuda-rhel7.repo &&\ yum -y install libnvinfer8-${v} libnvparsers8-${v} libnvonnxparsers8-${v} libnvinfer-plugin8-${v} \ libnvinfer-devel-${v} libnvparsers-devel-${v} libnvonnxparsers-devel-${v} libnvinfer-plugin-devel-${v} \ - python3-libnvinfer-${v}; \ + python3-libnvinfer-=${v} libnvinfer-dispatch8-=${v} libnvinfer-dispatch-devel-=${v} libnvinfer-lean8-=${v} \ + libnvinfer-lean-devel-=${v} libnvinfer-vc-plugin8-=${v} libnvinfer-vc-plugin-devel-=${v} \ + libnvinfer-headers-devel-=${v} libnvinfer-headers-plugin-devel-=${v}; \ else \ v="${TRT_VERSION}-1.cuda${CUDA_VERSION%.*}" &&\ yum-config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/cuda-rhel7.repo &&\ yum -y install libnvinfer8-${v} libnvparsers8-${v} libnvonnxparsers8-${v} libnvinfer-plugin8-${v} \ libnvinfer-devel-${v} libnvparsers-devel-${v} libnvonnxparsers-devel-${v} libnvinfer-plugin-devel-${v} \ - python3-libnvinfer-${v}; \ + python3-libnvinfer-=${v} libnvinfer-dispatch8-=${v} libnvinfer-dispatch-devel-=${v} libnvinfer-lean8-=${v} \ + libnvinfer-lean-devel-=${v} libnvinfer-vc-plugin8-=${v} libnvinfer-vc-plugin-devel-=${v} \ + libnvinfer-headers-devel-=${v} libnvinfer-headers-plugin-devel-=${v}; \ fi # Install dev-toolset-8 for g++ version that supports c++14 diff --git a/docker/launch.sh b/docker/launch.sh index 13763d24..2fe9d299 100755 --- a/docker/launch.sh +++ b/docker/launch.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/docker/ubuntu-18.04.Dockerfile b/docker/ubuntu-18.04.Dockerfile index cf42b792..8c246126 100644 --- a/docker/ubuntu-18.04.Dockerfile +++ b/docker/ubuntu-18.04.Dockerfile @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -16,12 +16,11 @@ # ARG CUDA_VERSION=12.0.1 -ARG OS_VERSION=18.04 -FROM nvidia/cuda:${CUDA_VERSION}-cudnn8-devel-ubuntu${OS_VERSION} +FROM nvidia/cuda:${CUDA_VERSION}-cudnn8-devel-ubuntu18.04 LABEL maintainer="NVIDIA CORPORATION" -ENV TRT_VERSION 8.6.0.12 +ENV TRT_VERSION 8.6.1.6 SHELL ["/bin/bash", "-c"] # Setup user account @@ -70,14 +69,18 @@ RUN if [ "${CUDA_VERSION}" = "10.2" ] ; then \ apt-get update &&\ sudo apt-get install libnvinfer8=${v} libnvonnxparsers8=${v} libnvparsers8=${v} libnvinfer-plugin8=${v} \ libnvinfer-dev=${v} libnvonnxparsers-dev=${v} libnvparsers-dev=${v} libnvinfer-plugin-dev=${v} \ - python3-libnvinfer=${v}; \ + python3-libnvinfer=${v} libnvinfer-dispatch8=${v} libnvinfer-dispatch-dev=${v} libnvinfer-lean8=${v} \ + libnvinfer-lean-dev=${v} libnvinfer-vc-plugin8=${v} libnvinfer-vc-plugin-dev=${v} \ + libnvinfer-headers-dev=${v} libnvinfer-headers-plugin-dev=${v}; \ else \ v="${TRT_VERSION}-1+cuda${CUDA_VERSION%.*}" &&\ apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub &&\ apt-get update &&\ sudo apt-get -y install libnvinfer8=${v} libnvonnxparsers8=${v} libnvparsers8=${v} libnvinfer-plugin8=${v} \ libnvinfer-dev=${v} libnvonnxparsers-dev=${v} libnvparsers-dev=${v} libnvinfer-plugin-dev=${v} \ - python3-libnvinfer=${v}; \ + python3-libnvinfer=${v} libnvinfer-dispatch8=${v} libnvinfer-dispatch-dev=${v} libnvinfer-lean8=${v} \ + libnvinfer-lean-dev=${v} libnvinfer-vc-plugin8=${v} libnvinfer-vc-plugin-dev=${v} \ + libnvinfer-headers-dev=${v} libnvinfer-headers-plugin-dev=${v}; \ fi # Install PyPI packages diff --git a/docker/ubuntu-20.04-aarch64.Dockerfile b/docker/ubuntu-20.04-aarch64.Dockerfile index c0cda07e..540943cd 100644 --- a/docker/ubuntu-20.04-aarch64.Dockerfile +++ b/docker/ubuntu-20.04-aarch64.Dockerfile @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -15,10 +15,12 @@ # limitations under the License. # +ARG CUDA_VERSION=12.0.1 + # Multi-arch container support available in non-cudnn containers. -FROM nvidia/cuda:12.0.1-devel-ubuntu20.04 +FROM nvidia/cuda:${CUDA_VERSION}-devel-ubuntu20.04 -ENV TRT_VERSION 8.6.0.12 +ENV TRT_VERSION 8.6.1.6 SHELL ["/bin/bash", "-c"] # Setup user account @@ -72,7 +74,9 @@ RUN v="${TRT_VERSION}-1+cuda${CUDA_VERSION%.*}" &&\ apt-get update &&\ sudo apt-get -y install libnvinfer8=${v} libnvonnxparsers8=${v} libnvparsers8=${v} libnvinfer-plugin8=${v} \ libnvinfer-dev=${v} libnvonnxparsers-dev=${v} libnvparsers-dev=${v} libnvinfer-plugin-dev=${v} \ - python3-libnvinfer=${v}; + python3-libnvinfer=${v} libnvinfer-dispatch8=${v} libnvinfer-dispatch-dev=${v} libnvinfer-lean8=${v} \ + libnvinfer-lean-dev=${v} libnvinfer-vc-plugin8=${v} libnvinfer-vc-plugin-dev=${v} \ + libnvinfer-headers-dev=${v} libnvinfer-headers-plugin-dev=${v}; # Install Cmake RUN cd /tmp && \ diff --git a/docker/ubuntu-20.04.Dockerfile b/docker/ubuntu-20.04.Dockerfile index b4143713..65605b47 100644 --- a/docker/ubuntu-20.04.Dockerfile +++ b/docker/ubuntu-20.04.Dockerfile @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -16,12 +16,11 @@ # ARG CUDA_VERSION=12.0.1 -ARG OS_VERSION=20.04 -FROM nvidia/cuda:${CUDA_VERSION}-cudnn8-devel-ubuntu${OS_VERSION} +FROM nvidia/cuda:${CUDA_VERSION}-cudnn8-devel-ubuntu20.04 LABEL maintainer="NVIDIA CORPORATION" -ENV TRT_VERSION 8.6.0.12 +ENV TRT_VERSION 8.6.1.6 SHELL ["/bin/bash", "-c"] # Setup user account @@ -76,14 +75,18 @@ RUN if [ "${CUDA_VERSION}" = "10.2" ] ; then \ apt-get update &&\ sudo apt-get install libnvinfer8=${v} libnvonnxparsers8=${v} libnvparsers8=${v} libnvinfer-plugin8=${v} \ libnvinfer-dev=${v} libnvonnxparsers-dev=${v} libnvparsers-dev=${v} libnvinfer-plugin-dev=${v} \ - python3-libnvinfer=${v}; \ + python3-libnvinfer=${v} libnvinfer-dispatch8=${v} libnvinfer-dispatch-dev=${v} libnvinfer-lean8=${v} \ + libnvinfer-lean-dev=${v} libnvinfer-vc-plugin8=${v} libnvinfer-vc-plugin-dev=${v} \ + libnvinfer-headers-dev=${v} libnvinfer-headers-plugin-dev=${v}; \ else \ v="${TRT_VERSION}-1+cuda${CUDA_VERSION%.*}" &&\ apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub &&\ apt-get update &&\ sudo apt-get -y install libnvinfer8=${v} libnvonnxparsers8=${v} libnvparsers8=${v} libnvinfer-plugin8=${v} \ libnvinfer-dev=${v} libnvonnxparsers-dev=${v} libnvparsers-dev=${v} libnvinfer-plugin-dev=${v} \ - python3-libnvinfer=${v}; \ + python3-libnvinfer=${v} libnvinfer-dispatch8=${v} libnvinfer-dispatch-dev=${v} libnvinfer-lean8=${v} \ + libnvinfer-lean-dev=${v} libnvinfer-vc-plugin8=${v} libnvinfer-vc-plugin-dev=${v} \ + libnvinfer-headers-dev=${v} libnvinfer-headers-plugin-dev=${v}; \ fi # Install PyPI packages diff --git a/docker/ubuntu-cross-aarch64.Dockerfile b/docker/ubuntu-cross-aarch64.Dockerfile index 0ff1fbc7..cf5f31d9 100644 --- a/docker/ubuntu-cross-aarch64.Dockerfile +++ b/docker/ubuntu-cross-aarch64.Dockerfile @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -16,12 +16,12 @@ # ARG CUDA_VERSION=11.4.1 -ARG OS_VERSION=20.04 -FROM nvidia/cuda:${CUDA_VERSION}-devel-ubuntu${OS_VERSION} +# Multi-arch container support available in non-cudnn containers. +FROM nvidia/cuda:${CUDA_VERSION}-devel-ubuntu20.04 LABEL maintainer="NVIDIA CORPORATION" -ENV TRT_VERSION 8.6.0.12 +ENV TRT_VERSION 8.5.2 ENV DEBIAN_FRONTEND=noninteractive ARG uid=1000 @@ -75,14 +75,16 @@ RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/ # Install CUDA cross compile toolchain RUN dpkg -i /pdk_files/cuda-repo-cross-aarch64*.deb /pdk_files/cuda-repo-ubuntu*_amd64.deb \ + && cp /var/cuda-repo-cross*/cuda-*-keyring.gpg /usr/share/keyrings/ \ + && cp /var/cuda-repo-ubuntu*/cuda-*-keyring.gpg /usr/share/keyrings/ \ && apt-get update \ && apt-get install -y cuda-cross-aarch64 \ && rm -rf /var/lib/apt/lists/* # Unpack cudnn -RUN dpkg -x /pdk_files/cudnn-local-repo*.deb /pdk_files/cudnn_extract \ - && dpkg -x /pdk_files/cudnn_extract/var/cudnn-local-repo*/libcudnn[7-8]_*-1+cuda11.[0-9]_arm64.deb /pdk_files/cudnn \ - && dpkg -x /pdk_files/cudnn_extract/var/cudnn-local-repo*/libcudnn[7-8]-dev_*-1+cuda11.[0-9]_arm64.deb /pdk_files/cudnn \ +RUN dpkg -x /pdk_files/cudnn-local-tegra-repo*.deb /pdk_files/cudnn_extract \ + && dpkg -x /pdk_files/cudnn_extract/var/cudnn-local-tegra-repo*/libcudnn[7-8]_*-1+cuda11.[0-9]_arm64.deb /pdk_files/cudnn \ + && dpkg -x /pdk_files/cudnn_extract/var/cudnn-local-tegra-repo*/libcudnn[7-8]-dev_*-1+cuda11.[0-9]_arm64.deb /pdk_files/cudnn \ && cd /pdk_files/cudnn/usr/lib/aarch64-linux-gnu \ && cd /pdk_files/cudnn \ && ln -s usr/include/aarch64-linux-gnu include \ @@ -98,6 +100,8 @@ RUN dpkg -x /pdk_files/cudnn-local-repo*.deb /pdk_files/cudnn_extract \ && ln -s /pdk_files/cudnn/usr/include/aarch64-linux-gnu/cudnn_version_v[7-9].h /usr/include/cudnn_version.h # Unpack libnvinfer +RUN dpkg -x /pdk_files/nv-tensorrt-local-repo-l4t-[0-8].[0-9].[0-9]-cuda-11.[0-9]_*_arm64.deb /pdk_files/tensorrt +RUN mv /pdk_files/tensorrt/var/nv-tensorrt-local-repo-l4t-[0-8].[0-9].[0-9]-cuda-11.[0-9]/*.deb /pdk_files RUN dpkg -x /pdk_files/libnvinfer[0-8]_*-1+cuda11.[0-9]_arm64.deb /pdk_files/tensorrt \ && dpkg -x /pdk_files/libnvinfer-dev_*-1+cuda11.[0-9]_arm64.deb /pdk_files/tensorrt \ && dpkg -x /pdk_files/libnvparsers[6-8]_*-1+cuda11.[0-9]_arm64.deb /pdk_files/tensorrt \ diff --git a/include/NvInfer.h b/include/NvInfer.h index 4a1ee65d..064bc93b 100644 --- a/include/NvInfer.h +++ b/include/NvInfer.h @@ -8971,8 +8971,8 @@ enum class MemoryPoolType : int32_t kDLA_GLOBAL_DRAM = 3, //! - //! kTACTIC_DRAM is the host DRAM used by the optimizer to - //! run tactics. On embedded devices, where host and device memory are unified, this includes all device + //! kTACTIC_DRAM is the device DRAM used by the optimizer to + //! run tactics. On embedded devices, where host and device memory are unified, this includes all host //! memory required by TensorRT to build the network up to the point of each memory allocation. //! This defaults to 75% of totalGlobalMem as reported by cudaGetDeviceProperties when //! cudaGetDeviceProperties.embedded is true, and 100% otherwise. diff --git a/include/NvInferImpl.h b/include/NvInferImpl.h index 522163cb..ddc6a89a 100644 --- a/include/NvInferImpl.h +++ b/include/NvInferImpl.h @@ -306,6 +306,12 @@ class VCudaEngine : public VRoot virtual HardwareCompatibilityLevel getHardwareCompatibilityLevel() const noexcept = 0; virtual ICudaEngine* getPImpl() noexcept = 0; virtual int32_t getNbAuxStreams() const noexcept = 0; + + virtual int32_t getTensorBytesPerComponentV2(char const* tensorName, int32_t profileIndex) const noexcept = 0; + virtual int32_t getTensorComponentsPerElementV2(char const* tensorName, int32_t profileIndex) const noexcept = 0; + virtual TensorFormat getTensorFormatV2(char const* tensorName, int32_t profileIndex) const noexcept = 0; + virtual char const* getTensorFormatDescV2(char const* tensorName, int32_t profileIndex) const noexcept = 0; + virtual int32_t getTensorVectorizedDimV2(char const* tensorName, int32_t profileIndex) const noexcept = 0; }; class VExecutionContext : public VRoot diff --git a/include/NvInferRuntime.h b/include/NvInferRuntime.h index fffbdeef..3850ab93 100644 --- a/include/NvInferRuntime.h +++ b/include/NvInferRuntime.h @@ -388,7 +388,7 @@ class IPluginV2DynamicExt : public nvinfer1::IPluginV2Ext //! \brief Return true if plugin supports the format and datatype for the input/output indexed by pos. //! //! For this method inputs are numbered 0..(nbInputs-1) and outputs are numbered nbInputs..(nbInputs+nbOutputs-1). - //! Using this numbering, pos is an index into InOut, where 0 <= pos < nbInputs+nbOutputs-1. + //! Using this numbering, pos is an index into InOut, where 0 <= pos < nbInputs+nbOutputs. //! //! TensorRT invokes this method to ask if the input/output indexed by pos supports the format/datatype specified //! by inOut[pos].format and inOut[pos].type. The override should return true if that format/datatype at inOut[pos] @@ -406,8 +406,8 @@ class IPluginV2DynamicExt : public nvinfer1::IPluginV2Ext //! * A definition for a plugin that supports only FP16 NCHW for its two inputs, //! and FP32 NCHW for its single output: //! - //! return inOut.format[pos] == TensorFormat::kLINEAR && (inOut.type[pos] == pos < 2 ? DataType::kHALF : - //! DataType::kFLOAT); + //! return inOut.format[pos] == TensorFormat::kLINEAR && (inOut.type[pos] == (pos < 2 ? DataType::kHALF : + //! DataType::kFLOAT)); //! //! * A definition for a "polymorphic" plugin with two inputs and one output that supports //! any format or type, but the inputs and output must have the same format and type: @@ -1896,14 +1896,36 @@ class ICudaEngine : public INoCopy //! \param tensorName The name of an input or output tensor. //! //! \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator. + //! \warning The function can only return the result of profile 0, and issues a warning message when there are + //! multiple profiles in the engine, use getTensorBytesPerComponent with profileIndex when there are multiple + //! profiles. //! //! \see getTensorVectorizedDim() + //! \see getTensorBytesPerComponent(tensorName, profileIndex) //! int32_t getTensorBytesPerComponent(char const* tensorName) const noexcept { return mImpl->getTensorBytesPerComponent(tensorName); } + //! + //! \brief Return the number of bytes per component of an element of given profile, or -1 if the provided name does + //! not map to an input or output tensor. + //! + //! The vector component size is returned if getTensorVectorizedDim(tensorName, profileIndex) != -1. + //! + //! \param tensorName The name of an input or output tensor. + //! \param profileIndex The profile index to query + //! + //! \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator. + //! + //! \see getTensorVectorizedDim(tensorName, profileIndex) + //! + int32_t getTensorBytesPerComponent(char const* tensorName, int32_t profileIndex) const noexcept + { + return mImpl->getTensorBytesPerComponentV2(tensorName, profileIndex); + } + //! //! \brief Return the number of components included in one element. //! @@ -1929,14 +1951,36 @@ class ICudaEngine : public INoCopy //! \param tensorName The name of an input or output tensor. //! //! \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator. + //! \warning The function can only return the result of profile 0, and issues a warning message when there + //! are multiple profiles in the engine, use getTensorComponentsPerElement with profileIndex when there are + //! multiple profiles. //! //! \see getTensorVectorizedDim() + //! \see getTensorComponentsPerElement(tensorName, profileIndex) //! int32_t getTensorComponentsPerElement(char const* tensorName) const noexcept { return mImpl->getTensorComponentsPerElement(tensorName); } + //! + //! \brief Return the number of components included in one element of given profile, or -1 if the provided name does + //! not map to an input or output tensor. + //! + //! The number of elements in the vectors is returned if getTensorVectorizedDim(tensorName, profileIndex) != -1. + //! + //! \param tensorName The name of an input or output tensor. + //! \param profileIndex The profile index to query + //! + //! \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator. + //! + //! \see getTensorVectorizedDim(tensorName, profileIndex) + //! + int32_t getTensorComponentsPerElement(char const* tensorName, int32_t profileIndex) const noexcept + { + return mImpl->getTensorComponentsPerElementV2(tensorName, profileIndex); + } + //! //! \brief Return the binding format. //! @@ -1952,18 +1996,34 @@ class ICudaEngine : public INoCopy } //! - //! \brief Return the binding format, or TensorFormat::kLINEAR if the provided name does not map to an input or + //! \brief Return the tensor format, or TensorFormat::kLINEAR if the provided name does not map to an input or //! output tensor. //! - //! \param tensorName The name of an input or output tensor. - //! //! \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator. + //! \warning This API can only return the tensor format of profile 0, and issues a warning message when there are + //! multiple profiles in the engine, use getTensorFormat with profileIndex when there are multiple profiles. + //! + //! \see getTensorFormat(tensorName, profileIndex) //! TensorFormat getTensorFormat(char const* tensorName) const noexcept { return mImpl->getTensorFormat(tensorName); } + //! + //! \brief Return the tensor format of given profile, or TensorFormat::kLINEAR if the provided name does not map to + //! an input or output tensor. + //! + //! \param tensorName The name of an input or output tensor. + //! \param profileIndex The profile index to query the format for. + //! + //! \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator. + //! + TensorFormat getTensorFormat(char const* tensorName, int32_t profileIndex) const noexcept + { + return mImpl->getTensorFormatV2(tensorName, profileIndex); + } + //! //! \brief Return the human readable description of the tensor format, or nullptr if the provided name does not //! map to an input or output tensor. @@ -2004,12 +2064,37 @@ class ICudaEngine : public INoCopy //! \param tensorName The name of an input or output tensor. //! //! \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator. + //! \warning The function can only return the result of profile 0, and issues a warning message when there are + //! multiple profiles in the engine, use getTensorFormatDesc with profileIndex when there are multiple profiles. //! char const* getTensorFormatDesc(char const* tensorName) const noexcept { return mImpl->getTensorFormatDesc(tensorName); } + //! + //! \brief Return the human readable description of the tensor format of given profile, or empty string if the + //! provided name does not map to an input or output tensor. + //! + //! The description includes the order, vectorization, data type, and strides. + //! Examples are shown as follows: + //! Example 1: kCHW + FP32 + //! "Row major linear FP32 format" + //! Example 2: kCHW2 + FP16 + //! "Two wide channel vectorized row major FP16 format" + //! Example 3: kHWC8 + FP16 + Line Stride = 32 + //! "Channel major FP16 format where C % 8 == 0 and H Stride % 32 == 0" + //! + //! \param tensorName The name of an input or output tensor. + //! \param profileIndex The profile index to query the format for. + //! + //! \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator. + //! + char const* getTensorFormatDesc(char const* tensorName, int32_t profileIndex) const noexcept + { + return mImpl->getTensorFormatDescV2(tensorName, profileIndex); + } + //! //! \brief Return the dimension index that the buffer is vectorized, or -1 is the name is not found. //! @@ -2035,12 +2120,30 @@ class ICudaEngine : public INoCopy //! \param tensorName The name of an input or output tensor. //! //! \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator. + //! \warning The function can only return the result of profile 0, and issues a warning message when there are + //! multiple profiles in the engine, use getTensorVectorizedDim with profileIndex when there are multiple profiles. //! int32_t getTensorVectorizedDim(char const* tensorName) const noexcept { return mImpl->getTensorVectorizedDim(tensorName); } + //! + //! \brief Return the dimension index that the buffer is vectorized of given profile, or -1 if the provided name + //! does not map to an input or output tensor. + //! + //! Specifically -1 is returned if scalars per vector is 1. + //! + //! \param tensorName The name of an input. + //! \param profileIndex The profile index to query the format for. + //! + //! \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator. + //! + int32_t getTensorVectorizedDim(char const* tensorName, int32_t profileIndex) const noexcept + { + return mImpl->getTensorVectorizedDimV2(tensorName, profileIndex); + } + //! //! \brief Returns the name of the network associated with the engine. //! diff --git a/include/NvInferRuntimeCommon.h b/include/NvInferRuntimeCommon.h index 261cecce..10dc5640 100644 --- a/include/NvInferRuntimeCommon.h +++ b/include/NvInferRuntimeCommon.h @@ -89,43 +89,6 @@ class IPluginRegistry AsciiChar const* const pluginNamespace = "") noexcept = 0; - //! - //! \brief Return whether the parent registry will be searched if a plugin is not found in this registry - //! default: true - //! - //! \return bool variable indicating whether parent search is enabled. - //! - //! \see setParentSearchEnabled - //! - virtual bool isParentSearchEnabled() const = 0; - - //! - //! \brief Set whether the parent registry will be searched if a plugin is not found in this registry. - //! - //! \param enabled The bool variable indicating whether parent search is enabled. - //! - //! \see isParentSearchEnabled - //! - virtual void setParentSearchEnabled(bool const enabled) = 0; - - //! - //! \brief Load and register a shared library of plugins. - //! - //! \param pluginPath the plugin library path. - //! - //! \return The loaded plugin library handle. The call will fail and return - //! nullptr if any of the plugins are already registered. - //! - virtual PluginLibraryHandle loadLibrary(AsciiChar const* pluginPath) noexcept = 0; - - //! - //! \brief Deregister plugins associated with a library. Any resources acquired when the library - //! was loaded will be released. - //! - //! \param handle the plugin library handle to deregister. - //! - virtual void deregisterLibrary(PluginLibraryHandle handle) noexcept = 0; - // @cond SuppressDoxyWarnings IPluginRegistry() = default; IPluginRegistry(IPluginRegistry const&) = delete; @@ -189,6 +152,43 @@ class IPluginRegistry //! - Thread-safe: Yes //! virtual bool deregisterCreator(IPluginCreator const& creator) noexcept = 0; + + //! + //! \brief Return whether the parent registry will be searched if a plugin is not found in this registry + //! default: true + //! + //! \return bool variable indicating whether parent search is enabled. + //! + //! \see setParentSearchEnabled + //! + virtual bool isParentSearchEnabled() const = 0; + + //! + //! \brief Set whether the parent registry will be searched if a plugin is not found in this registry. + //! + //! \param enabled The bool variable indicating whether parent search is enabled. + //! + //! \see isParentSearchEnabled + //! + virtual void setParentSearchEnabled(bool const enabled) = 0; + + //! + //! \brief Load and register a shared library of plugins. + //! + //! \param pluginPath the plugin library path. + //! + //! \return The loaded plugin library handle. The call will fail and return + //! nullptr if any of the plugins are already registered. + //! + virtual PluginLibraryHandle loadLibrary(AsciiChar const* pluginPath) noexcept = 0; + + //! + //! \brief Deregister plugins associated with a library. Any resources acquired when the library + //! was loaded will be released. + //! + //! \param handle the plugin library handle to deregister. + //! + virtual void deregisterLibrary(PluginLibraryHandle handle) noexcept = 0; }; } // namespace nvinfer1 diff --git a/include/NvInferRuntimePlugin.h b/include/NvInferRuntimePlugin.h index f722d921..fbe578ff 100644 --- a/include/NvInferRuntimePlugin.h +++ b/include/NvInferRuntimePlugin.h @@ -636,7 +636,7 @@ class IPluginV2IOExt : public IPluginV2Ext //! \brief Return true if plugin supports the format and datatype for the input/output indexed by pos. //! //! For this method inputs are numbered 0..(nbInputs-1) and outputs are numbered nbInputs..(nbInputs+nbOutputs-1). - //! Using this numbering, pos is an index into InOut, where 0 <= pos < nbInputs+nbOutputs-1. + //! Using this numbering, pos is an index into InOut, where 0 <= pos < nbInputs+nbOutputs. //! //! TensorRT invokes this method to ask if the input/output indexed by pos supports the format/datatype specified //! by inOut[pos].format and inOut[pos].type. The override should return true if that format/datatype at inOut[pos] diff --git a/include/NvInferVersion.h b/include/NvInferVersion.h index cb3d790f..670b5e1b 100644 --- a/include/NvInferVersion.h +++ b/include/NvInferVersion.h @@ -20,21 +20,24 @@ #define NV_TENSORRT_MAJOR 8 //!< TensorRT major version. #define NV_TENSORRT_MINOR 6 //!< TensorRT minor version. -#define NV_TENSORRT_PATCH 0 //!< TensorRT patch version. -#define NV_TENSORRT_BUILD 12 //!< TensorRT build number. +#define NV_TENSORRT_PATCH 1 //!< TensorRT patch version. +#define NV_TENSORRT_BUILD 5 //!< TensorRT build number. #define NV_TENSORRT_LWS_MAJOR 0 //!< TensorRT LWS major version. #define NV_TENSORRT_LWS_MINOR 0 //!< TensorRT LWS minor version. #define NV_TENSORRT_LWS_PATCH 0 //!< TensorRT LWS patch version. +// This #define is deprecated in TensorRT 8.6 and will be removed in 10.0. Use NV_TENSORRT_MAJOR. #define NV_TENSORRT_SONAME_MAJOR 8 //!< Shared object library major version number. +// This #define is deprecated in TensorRT 8.6 and will be removed in 10.0. Use NV_TENSORRT_MINOR. #define NV_TENSORRT_SONAME_MINOR 6 //!< Shared object library minor version number. -#define NV_TENSORRT_SONAME_PATCH 0 //!< Shared object library patch version number. +// This #define is deprecated in TensorRT 8.6 and will be removed in 10.0. Use NV_TENSORRT_PATCH. +#define NV_TENSORRT_SONAME_PATCH 1 //!< Shared object library patch version number. #define NV_TENSORRT_RELEASE_TYPE_EARLY_ACCESS 0 //!< An early access release #define NV_TENSORRT_RELEASE_TYPE_RELEASE_CANDIDATE 1 //!< A release candidate #define NV_TENSORRT_RELEASE_TYPE_GENERAL_AVAILABILITY 2 //!< A final release -#define NV_TENSORRT_RELEASE_TYPE NV_TENSORRT_RELEASE_TYPE_EARLY_ACCESS //!< TensorRT release type +#define NV_TENSORRT_RELEASE_TYPE NV_TENSORRT_RELEASE_TYPE_GENERAL_AVAILABILITY //!< TensorRT release type #endif // NV_INFER_VERSION_H diff --git a/parsers/CMakeLists.txt b/parsers/CMakeLists.txt index 955e109d..5dab1c9f 100644 --- a/parsers/CMakeLists.txt +++ b/parsers/CMakeLists.txt @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/CMakeLists.txt b/parsers/caffe/CMakeLists.txt index 74fd358f..f6abda79 100644 --- a/parsers/caffe/CMakeLists.txt +++ b/parsers/caffe/CMakeLists.txt @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/NvCaffeParser.cpp b/parsers/caffe/NvCaffeParser.cpp index 7e1906d3..2a9737e6 100644 --- a/parsers/caffe/NvCaffeParser.cpp +++ b/parsers/caffe/NvCaffeParser.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/binaryProtoBlob.h b/parsers/caffe/binaryProtoBlob.h index d0f52d20..79ec2976 100644 --- a/parsers/caffe/binaryProtoBlob.h +++ b/parsers/caffe/binaryProtoBlob.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/blobNameToTensor.h b/parsers/caffe/blobNameToTensor.h index a4929e56..d685cced 100644 --- a/parsers/caffe/blobNameToTensor.h +++ b/parsers/caffe/blobNameToTensor.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeMacros.h b/parsers/caffe/caffeMacros.h index f403de3d..d9cca466 100644 --- a/parsers/caffe/caffeMacros.h +++ b/parsers/caffe/caffeMacros.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/caffeParser.cpp b/parsers/caffe/caffeParser/caffeParser.cpp index d447c5ca..9e8722b2 100644 --- a/parsers/caffe/caffeParser/caffeParser.cpp +++ b/parsers/caffe/caffeParser/caffeParser.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/caffeParser.h b/parsers/caffe/caffeParser/caffeParser.h index 5a24f636..bd79967b 100644 --- a/parsers/caffe/caffeParser/caffeParser.h +++ b/parsers/caffe/caffeParser/caffeParser.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/opParsers.h b/parsers/caffe/caffeParser/opParsers/opParsers.h index ee6a38c0..b4641bcd 100644 --- a/parsers/caffe/caffeParser/opParsers/opParsers.h +++ b/parsers/caffe/caffeParser/opParsers/opParsers.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseAbsVal.cpp b/parsers/caffe/caffeParser/opParsers/parseAbsVal.cpp index 5814e92f..6b9415c6 100644 --- a/parsers/caffe/caffeParser/opParsers/parseAbsVal.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseAbsVal.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseBNLL.cpp b/parsers/caffe/caffeParser/opParsers/parseBNLL.cpp index 751fa9f0..6ff13917 100644 --- a/parsers/caffe/caffeParser/opParsers/parseBNLL.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseBNLL.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseBatchNorm.cpp b/parsers/caffe/caffeParser/opParsers/parseBatchNorm.cpp index 4c4aa14c..b16552d7 100644 --- a/parsers/caffe/caffeParser/opParsers/parseBatchNorm.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseBatchNorm.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseClip.cpp b/parsers/caffe/caffeParser/opParsers/parseClip.cpp index c94f351e..108acf4f 100644 --- a/parsers/caffe/caffeParser/opParsers/parseClip.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseClip.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseConcat.cpp b/parsers/caffe/caffeParser/opParsers/parseConcat.cpp index 44291c6e..a682b798 100644 --- a/parsers/caffe/caffeParser/opParsers/parseConcat.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseConcat.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseConv.cpp b/parsers/caffe/caffeParser/opParsers/parseConv.cpp index 87421476..ae7372ad 100644 --- a/parsers/caffe/caffeParser/opParsers/parseConv.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseConv.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseCrop.cpp b/parsers/caffe/caffeParser/opParsers/parseCrop.cpp index 46531c30..9f907c38 100644 --- a/parsers/caffe/caffeParser/opParsers/parseCrop.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseCrop.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseDeconv.cpp b/parsers/caffe/caffeParser/opParsers/parseDeconv.cpp index 366425f2..92ef6816 100644 --- a/parsers/caffe/caffeParser/opParsers/parseDeconv.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseDeconv.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseELU.cpp b/parsers/caffe/caffeParser/opParsers/parseELU.cpp index 90afc517..ad67af98 100644 --- a/parsers/caffe/caffeParser/opParsers/parseELU.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseELU.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseEltwise.cpp b/parsers/caffe/caffeParser/opParsers/parseEltwise.cpp index 82601ffc..5f572075 100644 --- a/parsers/caffe/caffeParser/opParsers/parseEltwise.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseEltwise.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseInnerProduct.cpp b/parsers/caffe/caffeParser/opParsers/parseInnerProduct.cpp index 4819c8a2..a13c0d6a 100644 --- a/parsers/caffe/caffeParser/opParsers/parseInnerProduct.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseInnerProduct.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseLRN.cpp b/parsers/caffe/caffeParser/opParsers/parseLRN.cpp index 2b7ff20b..b9587afe 100644 --- a/parsers/caffe/caffeParser/opParsers/parseLRN.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseLRN.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parsePReLU.cpp b/parsers/caffe/caffeParser/opParsers/parsePReLU.cpp index d2805988..f31e204a 100644 --- a/parsers/caffe/caffeParser/opParsers/parsePReLU.cpp +++ b/parsers/caffe/caffeParser/opParsers/parsePReLU.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parsePermute.cpp b/parsers/caffe/caffeParser/opParsers/parsePermute.cpp index 33163011..8803b4b0 100644 --- a/parsers/caffe/caffeParser/opParsers/parsePermute.cpp +++ b/parsers/caffe/caffeParser/opParsers/parsePermute.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parsePooling.cpp b/parsers/caffe/caffeParser/opParsers/parsePooling.cpp index fb0cac87..5e69e419 100644 --- a/parsers/caffe/caffeParser/opParsers/parsePooling.cpp +++ b/parsers/caffe/caffeParser/opParsers/parsePooling.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parsePower.cpp b/parsers/caffe/caffeParser/opParsers/parsePower.cpp index 0e77baeb..492c36d8 100644 --- a/parsers/caffe/caffeParser/opParsers/parsePower.cpp +++ b/parsers/caffe/caffeParser/opParsers/parsePower.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseReLU.cpp b/parsers/caffe/caffeParser/opParsers/parseReLU.cpp index 756e9b4c..d37e5fbe 100644 --- a/parsers/caffe/caffeParser/opParsers/parseReLU.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseReLU.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseReduction.cpp b/parsers/caffe/caffeParser/opParsers/parseReduction.cpp index ce6b8c0c..c3bf0742 100644 --- a/parsers/caffe/caffeParser/opParsers/parseReduction.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseReduction.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseReshape.cpp b/parsers/caffe/caffeParser/opParsers/parseReshape.cpp index cfe233d3..a31698a2 100644 --- a/parsers/caffe/caffeParser/opParsers/parseReshape.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseReshape.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseScale.cpp b/parsers/caffe/caffeParser/opParsers/parseScale.cpp index 1cdb8e0c..bd1efa94 100644 --- a/parsers/caffe/caffeParser/opParsers/parseScale.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseScale.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseSigmoid.cpp b/parsers/caffe/caffeParser/opParsers/parseSigmoid.cpp index a94a4181..46d4b9d2 100644 --- a/parsers/caffe/caffeParser/opParsers/parseSigmoid.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseSigmoid.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseSoftMax.cpp b/parsers/caffe/caffeParser/opParsers/parseSoftMax.cpp index ac1407e0..0d88de7f 100644 --- a/parsers/caffe/caffeParser/opParsers/parseSoftMax.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseSoftMax.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/opParsers/parseTanH.cpp b/parsers/caffe/caffeParser/opParsers/parseTanH.cpp index f7bb501e..e3c6a3dd 100644 --- a/parsers/caffe/caffeParser/opParsers/parseTanH.cpp +++ b/parsers/caffe/caffeParser/opParsers/parseTanH.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeParser/readProto.h b/parsers/caffe/caffeParser/readProto.h index 09bae595..b9276819 100644 --- a/parsers/caffe/caffeParser/readProto.h +++ b/parsers/caffe/caffeParser/readProto.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeWeightFactory/caffeWeightFactory.cpp b/parsers/caffe/caffeWeightFactory/caffeWeightFactory.cpp index bff42a35..5d4fe134 100644 --- a/parsers/caffe/caffeWeightFactory/caffeWeightFactory.cpp +++ b/parsers/caffe/caffeWeightFactory/caffeWeightFactory.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeWeightFactory/caffeWeightFactory.h b/parsers/caffe/caffeWeightFactory/caffeWeightFactory.h index 4644f80d..e32e979d 100644 --- a/parsers/caffe/caffeWeightFactory/caffeWeightFactory.h +++ b/parsers/caffe/caffeWeightFactory/caffeWeightFactory.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/caffe/caffeWeightFactory/weightType.h b/parsers/caffe/caffeWeightFactory/weightType.h index 71f300cc..6377d592 100644 --- a/parsers/caffe/caffeWeightFactory/weightType.h +++ b/parsers/caffe/caffeWeightFactory/weightType.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/common/half.h b/parsers/common/half.h index 8ca4891d..7497459a 100644 --- a/parsers/common/half.h +++ b/parsers/common/half.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/common/ieee_half.h b/parsers/common/ieee_half.h index dd4963c1..071aee09 100644 --- a/parsers/common/ieee_half.h +++ b/parsers/common/ieee_half.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/parsers/common/parserUtils.h b/parsers/common/parserUtils.h index 25b54da4..115a2efa 100644 --- a/parsers/common/parserUtils.h +++ b/parsers/common/parserUtils.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/CMakeLists.txt b/plugin/CMakeLists.txt index 02e055d5..393d4891 100644 --- a/plugin/CMakeLists.txt +++ b/plugin/CMakeLists.txt @@ -135,7 +135,7 @@ set_target_properties(${SHARED_TARGET} PROPERTIES if (MSVC) set_target_properties(${SHARED_TARGET} PROPERTIES LINK_FLAGS "/DEF:${PLUGIN_EXPORT_DEF}") else() - set_target_properties(${SHARED_TARGET} PROPERTIES LINK_FLAGS "-Wl,--exclude-libs,ALL -Wl,--version-script=${PLUGIN_EXPORT_MAP} -Wl,--no-undefined") + set_target_properties(${SHARED_TARGET} PROPERTIES LINK_FLAGS "-Wl,--exclude-libs,ALL -Wl,-Bsymbolic -Wl,--version-script=${PLUGIN_EXPORT_MAP} -Wl,--no-undefined") endif() set_target_properties(${SHARED_TARGET} PROPERTIES DEBUG_POSTFIX ${TRT_DEBUG_POSTFIX}) diff --git a/plugin/api/inferPlugin.cpp b/plugin/api/inferPlugin.cpp index 8e16dfe3..b55f9388 100644 --- a/plugin/api/inferPlugin.cpp +++ b/plugin/api/inferPlugin.cpp @@ -176,43 +176,44 @@ extern "C" { bool initLibNvInferPlugins(void* logger, const char* libNamespace) { - initializePlugin(logger, libNamespace); - initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); + initializePlugin(logger, libNamespace); + initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); - initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); + initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); - initializePlugin(logger, libNamespace); - initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); + initializePlugin(logger, libNamespace); + initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); + initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); - initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); + initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); + initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); - initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); - initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); + initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); initializePlugin(logger, libNamespace); diff --git a/plugin/batchTilePlugin/batchTilePlugin.h b/plugin/batchTilePlugin/batchTilePlugin.h index 1f0bc199..4f2bf37a 100644 --- a/plugin/batchTilePlugin/batchTilePlugin.h +++ b/plugin/batchTilePlugin/batchTilePlugin.h @@ -42,7 +42,7 @@ class BatchTilePlugin : public IPluginV2Ext int32_t initialize() noexcept override; void terminate() noexcept override; - size_t getWorkspaceSize(int) const noexcept override; + size_t getWorkspaceSize(int32_t) const noexcept override; int32_t enqueue(int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; diff --git a/plugin/batchedNMSPlugin/gatherNMSOutputs.h b/plugin/batchedNMSPlugin/gatherNMSOutputs.h index e3393a08..f245eb93 100644 --- a/plugin/batchedNMSPlugin/gatherNMSOutputs.h +++ b/plugin/batchedNMSPlugin/gatherNMSOutputs.h @@ -18,8 +18,8 @@ #define TRT_BATCHED_NMS_HELPER_H #include "common/plugin.h" -pluginStatus_t gatherNMSOutputs(cudaStream_t stream, bool shareLocation, int numImages, int numPredsPerClass, - int numClasses, int topK, int keepTopK, nvinfer1::DataType DT_BBOX, nvinfer1::DataType DT_SCORE, +pluginStatus_t gatherNMSOutputs(cudaStream_t stream, bool shareLocation, int32_t numImages, int32_t numPredsPerClass, + int32_t numClasses, int32_t topK, int32_t keepTopK, nvinfer1::DataType DT_BBOX, nvinfer1::DataType DT_SCORE, void const* indices, void const* scores, void const* bboxData, void* keepCount, void* nmsedBoxes, void* nmsedScores, void* nmsedClasses, bool clipBoxes, float const scoreShift); diff --git a/plugin/bertQKVToContextPlugin/CustomQKVToContextPluginDynamic_PluginConfig.yaml b/plugin/bertQKVToContextPlugin/CustomQKVToContextPluginDynamic_PluginConfig.yaml index 0931c67b..6b56ebea 100644 --- a/plugin/bertQKVToContextPlugin/CustomQKVToContextPluginDynamic_PluginConfig.yaml +++ b/plugin/bertQKVToContextPlugin/CustomQKVToContextPluginDynamic_PluginConfig.yaml @@ -5,14 +5,12 @@ versions: "1": inputs: - input - - input_mask outputs: - output input_dims: input: 5 - input_mask: 1 output_dims: - output: "input_0, input_mask_0, hidden_size_0, 1, 1" + output: "input_0, input_1, hidden_size_0, 1, 1" attributes: - type_id - hidden_size @@ -55,6 +53,85 @@ versions: - hidden_size - num_heads - has_mask + golden_io_path: "plugin/bertQKVToContextPlugin/CustomQKVToContextPluginDynamic_PluginGoldenIO.json" + abs_tol: 1e-5 + rel_tol: 1e-5 + fp16_atol: 1e-2 + fp16_rtol: 1e-2 + configs: + config1: + input_types: + input: float16 + attribute_options: + type_id: + value: 1 + shape: "1" + hidden_size: + value: 768 + shape: "1" + num_heads: + value: 12 + shape: "1" + has_mask: + value: 0 + shape: "1" + output_types: + output: float16 + config2: + input_types: + input: float16 + attribute_options: + type_id: + value: 1 + shape: "1" + hidden_size: + value: 1024 + shape: "1" + num_heads: + value: 16 + shape: "1" + has_mask: + value: 0 + shape: "1" + output_types: + output: float16 + config5: + input_types: + input: float16 + attribute_options: + type_id: + value: 1 + shape: "1" + hidden_size: + value: 384 + shape: "1" + num_heads: + value: 12 + shape: "1" + has_mask: + value: 0 + shape: "1" + output_types: + output: float16 + has_mask: + input_types: + input: float32 + input_mask: int32 + attribute_options: + type_id: + value: 0 + shape: "1" + hidden_size: + value: 3 + shape: "1" + num_heads: + value: 3 + shape: "1" + has_mask: + value: 1 + shape: "1" + output_types: + output: float32 "2": inputs: - input diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/include/fused_multihead_attention.h b/plugin/bertQKVToContextPlugin/fused_multihead_attention/include/fused_multihead_attention.h index 0eede2aa..9f541aaf 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/include/fused_multihead_attention.h +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/include/fused_multihead_attention.h @@ -108,7 +108,7 @@ struct Fused_multihead_attention_params float* max_scratch_ptr{}; float* sum_scratch_ptr{}; // Scratch buffer to finalize the output (not needed for FP16). - int* o_scratch_ptr{}; + int32_t* o_scratch_ptr{}; }; //////////////////////////////////////////////////////////////////////////////////////////////////// @@ -398,7 +398,7 @@ class TFusedMultiHeadAttentionXMMAKernel } } mFunctions.insert({kernelKey, funcInfo}); - const int s = static_cast(kernelMeta.mS); + const int32_t s = static_cast(kernelMeta.mS); if (mValidSequences.find(s) == mValidSequences.end()) { mValidSequences.insert(s); @@ -425,7 +425,7 @@ class TFusedMultiHeadAttentionXMMAKernel } } - bool isValid(int s) const + bool isValid(int32_t s) const { return (mValidSequences.find(s) != mValidSequences.end()); } @@ -487,7 +487,7 @@ class TFusedMultiHeadAttentionXMMAKernel CUfunction mDeviceFunction; }; std::unordered_map mFunctions; - std::set mValidSequences; + std::set mValidSequences; }; template diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_128_64_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_128_64_kernel.sm75.cpp index e3e3e986..af45426d 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_128_64_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_128_64_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_128_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_128_64_kernel.sm80.cpp index 111eed80..3e5031b1 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_128_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_128_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_128_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_128_64_kernel.sm87.cpp index f76a4b0c..0d0a6ed7 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_128_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_128_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_128_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_128_64_kernel.sm90.cpp index 4cd98ea5..a5134aaf 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_128_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_128_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm75.cpp index ea2872a4..e2604633 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm80.cpp index 353df03d..035270eb 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm86.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm86.cpp index ac48fd26..81f7a887 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm86.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm86.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm87.cpp index e9b441fd..929c0a4b 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm90.cpp index 4bbf6da7..a9592f3f 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_384_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_512_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_512_64_kernel.sm90.cpp index 61b71d35..a5a19772 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_512_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_512_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_64_64_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_64_64_kernel.sm75.cpp index a59a3033..9dc6ffa6 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_64_64_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_64_64_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_64_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_64_64_kernel.sm80.cpp index 3dcb178e..588d5dc8 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_64_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_64_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_64_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_64_64_kernel.sm87.cpp index dc477b4c..4d6308d3 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_64_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_64_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_64_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_64_64_kernel.sm90.cpp index 0c40a36a..fd292683 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_64_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_64_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_96_64_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_96_64_kernel.sm75.cpp index 7a097f31..238e9fbd 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_96_64_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_96_64_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_96_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_96_64_kernel.sm80.cpp index afd69591..a2eb24f7 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_96_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_96_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_96_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_96_64_kernel.sm87.cpp index 0908bb48..5b39da95 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_96_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_96_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_96_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_96_64_kernel.sm90.cpp index 82cb6ce4..1af3e96a 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_96_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_96_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_128_64_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_128_64_kernel.sm75.cpp index 79c0c662..a18e4874 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_128_64_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_128_64_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_128_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_128_64_kernel.sm80.cpp index 23045fa7..0c079b17 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_128_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_128_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_128_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_128_64_kernel.sm87.cpp index 819677d4..b88a696d 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_128_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_128_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_128_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_128_64_kernel.sm90.cpp index 92767aa1..457af2b6 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_128_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_128_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_384_64_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_384_64_kernel.sm75.cpp index 7e077a94..22611907 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_384_64_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_384_64_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_384_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_384_64_kernel.sm80.cpp index f8422eb6..bf716793 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_384_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_384_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_384_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_384_64_kernel.sm87.cpp index d2634d20..c4376f86 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_384_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_384_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_384_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_384_64_kernel.sm90.cpp index aa56892d..44f159a7 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_384_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_384_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_512_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_512_64_kernel.sm90.cpp index 49f55601..fd51119e 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_512_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_512_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_64_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_64_64_kernel.sm80.cpp index 935b3ba2..062ce999 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_64_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_64_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_96_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_96_64_kernel.sm80.cpp index f81cd7fa..017f6862 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_96_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_int8_96_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_32_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_32_kernel.sm75.cpp index aaf7f56f..373f496a 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_32_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_32_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_32_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_32_kernel.sm80.cpp index 7e5cb15a..1e3ff7c6 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_32_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_32_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm75.cpp index 8946f2c0..ece2d0eb 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm80.cpp index be4daab0..dbc34090 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm86.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm86.cpp index 810d1624..ff794f09 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm86.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm86.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm87.cpp index 0cf510c9..d957a175 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm90.cpp index 23d90611..910c2772 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_128_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_32_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_32_kernel.sm75.cpp index 3c13456b..f466437c 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_32_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_32_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_32_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_32_kernel.sm80.cpp index 18ac26ac..643f3abe 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_32_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_32_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm75.cpp index cbaadce9..b193aac5 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm80.cpp index a83fad85..eedf762f 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm86.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm86.cpp index 4ed00888..17cdf962 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm86.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm86.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm87.cpp index 1133758c..3943f07e 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm90.cpp index 18d4dbd6..8aebf6e4 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_256_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm75.cpp index cc7fd264..47d6f8b4 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm80.cpp index 46f4e7fa..2c0141c6 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm86.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm86.cpp index 3580a4cd..007b0ca5 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm86.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm86.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm87.cpp index 5d5bd4b5..e47a0eb5 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm90.cpp index 56e80a0f..71047e0d 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_384_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_32_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_32_kernel.sm75.cpp index 74157890..e424fd93 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_32_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_32_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_32_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_32_kernel.sm80.cpp index e51279fc..f3b2aec9 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_32_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_32_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_64_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_64_kernel.sm75.cpp index 23c1d966..6706f1e1 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_64_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_64_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_64_kernel.sm80.cpp index c4471a00..57d31338 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_64_kernel.sm90.cpp index 510227d9..d9bbd955 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_512_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm75.cpp index 6eae08a1..a93f1f80 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm80.cpp index 41eb73f3..fc6e825e 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm86.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm86.cpp index 2b194a06..dc64aaf1 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm86.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm86.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm87.cpp index ee6fb986..17394f7b 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm90.cpp index 0786bb1a..30a6a139 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_64_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm75.cpp index f74ef32c..75826861 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm80.cpp index 4998d6a8..a5a9db91 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm86.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm86.cpp index 95ef8d73..5c0e4792 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm86.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm86.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm87.cpp index 46d56b52..75cca5b0 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm90.cpp index 437a2849..05ed3a7d 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_fp16_96_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_128_32_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_128_32_kernel.sm80.cpp index 5d2091fc..7377bb87 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_128_32_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_128_32_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_128_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_128_64_kernel.sm87.cpp index aa3364cd..c486ba74 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_128_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_128_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_128_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_128_64_kernel.sm90.cpp index 93211a19..ff8b71b7 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_128_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_128_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_192_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_192_64_kernel.sm87.cpp index 1857e517..b55a9b29 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_192_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_192_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_192_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_192_64_kernel.sm90.cpp index b9ab1cbd..a486db0f 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_192_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_192_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_256_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_256_64_kernel.sm87.cpp index 6a8cffaa..dcac39f3 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_256_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_256_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_256_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_256_64_kernel.sm90.cpp index d1477730..9826a2c2 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_256_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_256_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_384_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_384_64_kernel.sm87.cpp index cc930e3c..b6659f16 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_384_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_384_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_384_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_384_64_kernel.sm90.cpp index 8228fbe0..bbb5eeeb 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_384_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_384_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_64_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_64_64_kernel.sm80.cpp index 5fc60b51..f9fd6183 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_64_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_64_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_64_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_64_64_kernel.sm87.cpp index 09603ff8..6441c74a 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_64_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_64_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_64_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_64_64_kernel.sm90.cpp index 3668a80d..df8cda25 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_64_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_64_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_96_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_96_64_kernel.sm80.cpp index 66963771..e62d93aa 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_96_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_96_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_96_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_96_64_kernel.sm87.cpp index 91ad85db..590c0df4 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_96_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_96_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_96_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_96_64_kernel.sm90.cpp index 05a7d0fe..be698b64 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_96_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_il_int8_96_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_32_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_32_kernel.sm75.cpp index 95f20196..ce3baa27 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_32_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_32_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_32_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_32_kernel.sm80.cpp index b545c838..0abcf4e3 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_32_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_32_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm72.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm72.cpp index 9852a3e0..fbf16481 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm72.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm72.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm75.cpp index b26d41ba..56cb1930 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm80.cpp index b1612ed2..f7b86091 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm86.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm86.cpp index 6a863153..fe49aaa3 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm86.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm86.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm87.cpp index 88108faa..b84b0dc8 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm90.cpp index b4535fd8..6f889451 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_128_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm72.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm72.cpp index 8d1795b6..3c3735d1 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm72.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm72.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm75.cpp index 52997298..dfe6d8ce 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm80.cpp index 0818097f..8a1d2d2c 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm86.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm86.cpp index d86ceeb4..31dd3150 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm86.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm86.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm87.cpp index 3571ce08..aa2a81c9 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm90.cpp index 54caea8a..a5e4c65e 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_192_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_32_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_32_kernel.sm75.cpp index c74af8c6..2a729502 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_32_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_32_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_32_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_32_kernel.sm80.cpp index 922856c5..aeac0ebd 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_32_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_32_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm72.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm72.cpp index 6aed0d37..a62c2cf9 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm72.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm72.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm75.cpp index 647339c7..3fa33ae5 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm80.cpp index f99cbf52..f597a37e 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm86.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm86.cpp index 59950b12..24d31716 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm86.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm86.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm87.cpp index d17b2eaf..b70f696d 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm90.cpp index 7efa519a..07f7b870 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_256_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm72.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm72.cpp index d33402a0..2d62254b 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm72.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm72.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm75.cpp index 3f29bde8..b373a064 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm80.cpp index 6e6e2404..86517581 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm86.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm86.cpp index a5b12391..c9196880 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm86.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm86.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm87.cpp index e1102282..70e699f8 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm90.cpp index b8d41328..848c68be 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_384_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_32_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_32_kernel.sm75.cpp index d6d21949..baaf7441 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_32_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_32_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_32_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_32_kernel.sm80.cpp index 94fc8adf..68204bf6 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_32_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_32_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_64_kernel.sm75.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_64_kernel.sm75.cpp index 7fa46bb1..8ee4ced0 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_64_kernel.sm75.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_64_kernel.sm75.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_64_kernel.sm80.cpp index 838e3bf9..e9bd8613 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_64_kernel.sm90.cpp index 2b08ac33..48644b36 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_512_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_64_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_64_64_kernel.sm80.cpp index 3c34dc60..77ccb240 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_64_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_64_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_64_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_64_64_kernel.sm87.cpp index 55da2568..2eb5c132 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_64_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_64_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_64_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_64_64_kernel.sm90.cpp index 38442aab..2280de3b 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_64_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_64_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_96_64_kernel.sm80.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_96_64_kernel.sm80.cpp index 5fac09b8..b7a7f1db 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_96_64_kernel.sm80.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_96_64_kernel.sm80.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_96_64_kernel.sm87.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_96_64_kernel.sm87.cpp index 5c7f08c8..c2e6aca4 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_96_64_kernel.sm87.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_96_64_kernel.sm87.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_96_64_kernel.sm90.cpp b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_96_64_kernel.sm90.cpp index 98eecaae..a4516a2d 100644 --- a/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_96_64_kernel.sm90.cpp +++ b/plugin/bertQKVToContextPlugin/fused_multihead_attention_v2/src/fused_multihead_attention_v2_int8_96_64_kernel.sm90.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/bertQKVToContextPlugin/qkvToContext.cu b/plugin/bertQKVToContextPlugin/qkvToContext.cu index 6281fb36..6ebb98ea 100644 --- a/plugin/bertQKVToContextPlugin/qkvToContext.cu +++ b/plugin/bertQKVToContextPlugin/qkvToContext.cu @@ -109,7 +109,7 @@ __global__ void maskedSoftmax(const float rsqrtHeadSize, const T* input, T* outp rZ = (1.f) / Z; } __syncthreads(); - local[it] *= rZ; + local[it] = (threadIdx.x < lastValid) ? local[it] * rZ : 0.F; } #pragma unroll @@ -1264,3 +1264,4 @@ bool FusedMHARunnerInt8v2::isValid(int s) const } // namespace bert } // namespace plugin } // namespace nvinfer1 + diff --git a/plugin/bertQKVToContextPlugin/qkvToContextInt8InterleavedPlugin.cpp b/plugin/bertQKVToContextPlugin/qkvToContextInt8InterleavedPlugin.cpp index d23a1261..61ef88f7 100644 --- a/plugin/bertQKVToContextPlugin/qkvToContextInt8InterleavedPlugin.cpp +++ b/plugin/bertQKVToContextPlugin/qkvToContextInt8InterleavedPlugin.cpp @@ -80,9 +80,9 @@ QKVToContextInterleavedPlugin::QKVToContextInterleavedPlugin(std::string const& deserialize_value(&data, &length, &mUseInt8ScaleMax); } -int QKVToContextInterleavedPlugin::getSMVersion() const noexcept +int32_t QKVToContextInterleavedPlugin::getSMVersion() const noexcept { - int device{-1}; + int32_t device{-1}; PLUGIN_CHECK(cudaGetDevice(&device)); cudaDeviceProp props; PLUGIN_CHECK(cudaGetDeviceProperties(&props, device)); @@ -108,7 +108,7 @@ nvinfer1::IPluginV2DynamicExt* QKVToContextInterleavedPlugin::clone() const noex } DimsExprs QKVToContextInterleavedPlugin::getOutputDimensions( - int outputIndex, DimsExprs const* inputs, int nbInputs, IExprBuilder& exprBuilder) noexcept + int32_t outputIndex, DimsExprs const* inputs, int32_t nbInputs, IExprBuilder& exprBuilder) noexcept { // Input SHAPE is 1x(3*N*H)xTotalx1 (NCHW) // Output SHAPE is 1x(N*H)xTotalx1 @@ -124,7 +124,7 @@ DimsExprs QKVToContextInterleavedPlugin::getOutputDimensions( return output; } bool QKVToContextInterleavedPlugin::supportsFormatCombination( - int pos, PluginTensorDesc const* inOut, int nbInputs, int nbOutputs) noexcept + int32_t pos, PluginTensorDesc const* inOut, int32_t nbInputs, int32_t nbOutputs) noexcept { PLUGIN_ASSERT(nbInputs == 3); PLUGIN_ASSERT(nbOutputs == 1); @@ -140,7 +140,7 @@ bool QKVToContextInterleavedPlugin::supportsFormatCombination( if (pos == 1) { - // cuSeqlens is a int array of size B+1 + // cuSeqlens is a int32_t array of size B+1 auto const* seqlens = &inOut[pos]; return (seqlens->type == DataType::kINT32) && (seqlens->format == TensorFormat::kLINEAR); } @@ -153,19 +153,19 @@ bool QKVToContextInterleavedPlugin::supportsFormatCombination( } void QKVToContextInterleavedPlugin::configurePlugin( - DynamicPluginTensorDesc const* in, int nbInputs, DynamicPluginTensorDesc const* out, int nbOutputs) noexcept + DynamicPluginTensorDesc const* in, int32_t nbInputs, DynamicPluginTensorDesc const* out, int32_t nbOutputs) noexcept { } size_t QKVToContextInterleavedPlugin::getWorkspaceSize( - PluginTensorDesc const* inputs, int nbInputs, PluginTensorDesc const* outputs, int nbOutputs) const noexcept + PluginTensorDesc const* inputs, int32_t nbInputs, PluginTensorDesc const* outputs, int32_t nbOutputs) const noexcept { return 0; } // IPluginV2Ext Methods DataType QKVToContextInterleavedPlugin::getOutputDataType( - int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { PLUGIN_ASSERT(index == 0); return DataType::kINT8; @@ -182,12 +182,12 @@ char const* QKVToContextInterleavedPlugin::getPluginVersion() const noexcept return kQKV_TO_CONTEXT_INTERLEAVED_PLUGIN_VERSION; } -int QKVToContextInterleavedPlugin::getNbOutputs() const noexcept +int32_t QKVToContextInterleavedPlugin::getNbOutputs() const noexcept { return 1; } -int QKVToContextInterleavedPlugin::initialize() noexcept +int32_t QKVToContextInterleavedPlugin::initialize() noexcept { return 0; } @@ -227,14 +227,14 @@ char const* QKVToContextInterleavedPlugin::getPluginNamespace() const noexcept return mNamespace.c_str(); } -int QKVToContextInterleavedPlugin::enqueue(PluginTensorDesc const* inputDesc, PluginTensorDesc const* outputDesc, +int32_t QKVToContextInterleavedPlugin::enqueue(PluginTensorDesc const* inputDesc, PluginTensorDesc const* outputDesc, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept { - int const total = inputDesc[0].dims.d[2]; - int const B = inputDesc[1].dims.d[0] - 1; - int const maxS = inputDesc[2].dims.d[0]; - int S = 384; + int32_t const total = inputDesc[0].dims.d[2]; + int32_t const B = inputDesc[1].dims.d[0] - 1; + int32_t const maxS = inputDesc[2].dims.d[0]; + int32_t S = 384; if (maxS <= 128) { S = 128; @@ -257,7 +257,7 @@ int QKVToContextInterleavedPlugin::enqueue(PluginTensorDesc const* inputDesc, Pl params.o_ptr = outputs[0]; params.qkv_ptr = const_cast(inputs[0]); - params.cu_seqlens = static_cast(const_cast(inputs[1])); + params.cu_seqlens = static_cast(const_cast(inputs[1])); float scaleQkv = inputDesc[0].scale; float scaleCtx = outputDesc[0].scale; diff --git a/plugin/bertQKVToContextPlugin/qkvToContextInt8InterleavedPlugin.h b/plugin/bertQKVToContextPlugin/qkvToContextInt8InterleavedPlugin.h index 53800df6..98985646 100644 --- a/plugin/bertQKVToContextPlugin/qkvToContextInt8InterleavedPlugin.h +++ b/plugin/bertQKVToContextPlugin/qkvToContextInt8InterleavedPlugin.h @@ -53,26 +53,26 @@ class QKVToContextInterleavedPlugin : public nvinfer1::IPluginV2DynamicExt // IPluginV2DynamicExt Methods nvinfer1::IPluginV2DynamicExt* clone() const noexcept override; - nvinfer1::DimsExprs getOutputDimensions(int outputIndex, nvinfer1::DimsExprs const* inputs, int nbInputs, + nvinfer1::DimsExprs getOutputDimensions(int32_t outputIndex, nvinfer1::DimsExprs const* inputs, int32_t nbInputs, nvinfer1::IExprBuilder& exprBuilder) noexcept override; bool supportsFormatCombination( - int pos, nvinfer1::PluginTensorDesc const* inOut, int nbInputs, int nbOutputs) noexcept override; - void configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int nbInputs, - nvinfer1::DynamicPluginTensorDesc const* out, int nbOutputs) noexcept override; - size_t getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int nbInputs, - nvinfer1::PluginTensorDesc const* outputs, int nbOutputs) const noexcept override; - int enqueue(nvinfer1::PluginTensorDesc const* inputDesc, nvinfer1::PluginTensorDesc const* outputDesc, + int32_t pos, nvinfer1::PluginTensorDesc const* inOut, int32_t nbInputs, int32_t nbOutputs) noexcept override; + void configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int32_t nbInputs, + nvinfer1::DynamicPluginTensorDesc const* out, int32_t nbOutputs) noexcept override; + size_t getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int32_t nbInputs, + nvinfer1::PluginTensorDesc const* outputs, int32_t nbOutputs) const noexcept override; + int32_t enqueue(nvinfer1::PluginTensorDesc const* inputDesc, nvinfer1::PluginTensorDesc const* outputDesc, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; // IPluginV2Ext Methods nvinfer1::DataType getOutputDataType( - int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; // IPluginV2 Methods char const* getPluginType() const noexcept override; char const* getPluginVersion() const noexcept override; - int getNbOutputs() const noexcept override; - int initialize() noexcept override; + int32_t getNbOutputs() const noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; size_t getSerializationSize() const noexcept override; void serialize(void* buffer) const noexcept override; @@ -82,18 +82,18 @@ class QKVToContextInterleavedPlugin : public nvinfer1::IPluginV2DynamicExt protected: void createMHARunner() noexcept; - int getSMVersion() const noexcept; + int32_t getSMVersion() const noexcept; private: std::string const& mLayerName; std::string mNamespace; - int mS; - int mB; - int mSM; - int mHeadSize; - int mHiddenSize; - int mNumHeads; + int32_t mS; + int32_t mB; + int32_t mSM; + int32_t mHeadSize; + int32_t mHiddenSize; + int32_t mNumHeads; FusedMultiHeadAttentionXMMAKernelV2 const* mXmmaKernel; diff --git a/plugin/bertQKVToContextPlugin/qkvToContextPlugin.cpp b/plugin/bertQKVToContextPlugin/qkvToContextPlugin.cpp index 2e9861a8..ed67ef64 100644 --- a/plugin/bertQKVToContextPlugin/qkvToContextPlugin.cpp +++ b/plugin/bertQKVToContextPlugin/qkvToContextPlugin.cpp @@ -388,7 +388,7 @@ size_t QKVToContextPluginDynamic::getSerializationSize() const noexcept { PLUGIN_ASSERT(unfusedDispatcher.get()); return sizeof(mNumHeads) + sizeof(mHeadSize) + sizeof(DataType) + sizeof(mHasImask) + sizeof(mHiddenSize) - + sizeof(mSM) + sizeof(mS) + sizeof(mB) + sizeof(mDqProbs) + sizeof(int) + + sizeof(mSM) + sizeof(mS) + sizeof(mB) + sizeof(mDqProbs) + sizeof(int32_t) + unfusedDispatcher->getSerializationSize(); } @@ -785,7 +785,7 @@ bool QKVToContextVarSeqlenPlugin::supportsFormatCombination( PLUGIN_ASSERT(mUseVarSeqlen); if (pos == 2) { // must be cuSeqlens - // cuSeqlens is a int array of size B+1 + // cuSeqlens is a int32_t array of size B+1 auto const* seqlens = &inOut[pos]; return (seqlens->type == DataType::kINT32) && (seqlens->format == TensorFormat::kLINEAR); } diff --git a/plugin/bertQKVToContextPlugin/qkvToContextPlugin.h b/plugin/bertQKVToContextPlugin/qkvToContextPlugin.h index 74b8d02c..7af05d87 100644 --- a/plugin/bertQKVToContextPlugin/qkvToContextPlugin.h +++ b/plugin/bertQKVToContextPlugin/qkvToContextPlugin.h @@ -108,7 +108,8 @@ class MHARunner float mRsqrtHeadSize; }; -std::pair tuneBatchedGemm(const int32_t B, const int32_t S, const int32_t numHeads, const int32_t headSize); +std::pair tuneBatchedGemm( + const int32_t B, const int32_t S, const int32_t numHeads, const int32_t headSize); template int32_t computeScaledSoftmax(cudaStream_t stream, const int32_t ld, const int32_t B, const int32_t N, @@ -116,7 +117,7 @@ int32_t computeScaledSoftmax(cudaStream_t stream, const int32_t ld, const int32_ template int32_t computeMaskedScaledSoftmax(cudaStream_t stream, const int32_t ld, const int32_t B, const int32_t N, - float const rsqrtHeadSize, int const* maskIdx, T const* input, T* output); + float const rsqrtHeadSize, int32_t const* maskIdx, T const* input, T* output); // One of the preferred ways of making TensorRT to be able to see // our custom layer requires extending IPluginV2 and IPluginCreator classes. diff --git a/plugin/clipPlugin/clip.h b/plugin/clipPlugin/clip.h index b69b3983..70a53143 100644 --- a/plugin/clipPlugin/clip.h +++ b/plugin/clipPlugin/clip.h @@ -18,7 +18,7 @@ #define TRT_CLIP_H #include "NvInfer.h" -int clipInference( - cudaStream_t stream, int n, float clipMin, float clipMax, void const* input, void* output, nvinfer1::DataType type); +int32_t clipInference(cudaStream_t stream, int32_t n, float clipMin, float clipMax, void const* input, void* output, + nvinfer1::DataType type); #endif // TRT_CLIP_H diff --git a/plugin/clipPlugin/clipPlugin.cpp b/plugin/clipPlugin/clipPlugin.cpp index 3c1f28d1..8b7d22b5 100644 --- a/plugin/clipPlugin/clipPlugin.cpp +++ b/plugin/clipPlugin/clipPlugin.cpp @@ -67,30 +67,30 @@ char const* ClipPlugin::getPluginVersion() const noexcept return kCLIP_PLUGIN_VERSION; } -int ClipPlugin::getNbOutputs() const noexcept +int32_t ClipPlugin::getNbOutputs() const noexcept { return 1; } -Dims ClipPlugin::getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept +Dims ClipPlugin::getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept { PLUGIN_ASSERT(nbInputDims == 1); PLUGIN_ASSERT(index == 0); return *inputs; } -int ClipPlugin::initialize() noexcept +int32_t ClipPlugin::initialize() noexcept { return 0; } -int ClipPlugin::enqueue( - int batchSize, void const* const* inputs, void* const* outputs, void*, cudaStream_t stream) noexcept +int32_t ClipPlugin::enqueue( + int32_t batchSize, void const* const* inputs, void* const* outputs, void*, cudaStream_t stream) noexcept { try { void* output = outputs[0]; - int status = pluginStatus_t::STATUS_FAILURE; + int32_t status = pluginStatus_t::STATUS_FAILURE; status = clipInference(stream, mInputVolume * batchSize, mClipMin, mClipMax, inputs[0], output, mDataType); if (status != pluginStatus_t::STATUS_SUCCESS) @@ -130,8 +130,8 @@ void ClipPlugin::serialize(void* buffer) const noexcept PLUGIN_ASSERT(d == a + getSerializationSize()); } -void ClipPlugin::configureWithFormat(Dims const* inputs, int nbInputs, Dims const* outputs, int nbOutputs, - DataType type, PluginFormat format, int) noexcept +void ClipPlugin::configureWithFormat(Dims const* inputs, int32_t nbInputs, Dims const* outputs, int32_t nbOutputs, + DataType type, PluginFormat format, int32_t) noexcept { PLUGIN_ASSERT(nbOutputs == 1); PLUGIN_API_CHECK_ENUM_RANGE(DataType, type); @@ -139,7 +139,7 @@ void ClipPlugin::configureWithFormat(Dims const* inputs, int nbInputs, Dims cons mDataType = type; size_t volume = 1; - for (int i = 0; i < inputs->nbDims; i++) + for (int32_t i = 0; i < inputs->nbDims; i++) { volume *= inputs->d[i]; } diff --git a/plugin/clipPlugin/clipPlugin.h b/plugin/clipPlugin/clipPlugin.h index 8ffbfad2..f7e218ce 100644 --- a/plugin/clipPlugin/clipPlugin.h +++ b/plugin/clipPlugin/clipPlugin.h @@ -39,28 +39,28 @@ class ClipPlugin : public nvinfer1::pluginInternal::BasePlugin ClipPlugin() = delete; - int getNbOutputs() const noexcept override; + int32_t getNbOutputs() const noexcept override; - Dims getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept override; + Dims getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept override; - int initialize() noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; - size_t getWorkspaceSize(int) const noexcept override + size_t getWorkspaceSize(int32_t) const noexcept override { return 0; }; - int enqueue(int batchSize, void const* const* inputs, void* const* outputs, void* workspace, + int32_t enqueue(int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; size_t getSerializationSize() const noexcept override; void serialize(void* buffer) const noexcept override; - void configureWithFormat(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, DataType type, - PluginFormat format, int maxBatchSize) noexcept override; + void configureWithFormat(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, + DataType type, PluginFormat format, int32_t maxBatchSize) noexcept override; bool supportsFormat(DataType type, PluginFormat format) const noexcept override; diff --git a/plugin/common/bboxUtils.h b/plugin/common/bboxUtils.h index a49602b4..028eeb81 100644 --- a/plugin/common/bboxUtils.h +++ b/plugin/common/bboxUtils.h @@ -40,10 +40,10 @@ template struct BboxInfo { T conf_score; - int label; - int bbox_idx; + int32_t label; + int32_t bbox_idx; bool kept; - BboxInfo(T conf_score, int label, int bbox_idx, bool kept) + BboxInfo(T conf_score, int32_t label, int32_t bbox_idx, bool kept) : conf_score(conf_score) , label(label) , bbox_idx(bbox_idx) @@ -72,7 +72,7 @@ int8_t* nextWorkspacePtr(int8_t* ptr, uintptr_t previousWorkspaceSize); size_t dataTypeSize(nvinfer1::DataType dtype); -void setUniformOffsets(cudaStream_t stream, int num_segments, int offset, int* d_offsets); +void setUniformOffsets(cudaStream_t stream, int32_t num_segments, int32_t offset, int32_t* d_offsets); } // namespace plugin } // namespace nvinfer1 #endif diff --git a/plugin/common/bertCommon.h b/plugin/common/bertCommon.h index a72fb65d..68de8c07 100644 --- a/plugin/common/bertCommon.h +++ b/plugin/common/bertCommon.h @@ -91,19 +91,19 @@ namespace plugin namespace bert { -inline int getSMVersion() +inline int32_t getSMVersion() { - int device{-1}; + int32_t device{-1}; PLUGIN_CHECK(cudaGetDevice(&device)); cudaDeviceProp props; PLUGIN_CHECK(cudaGetDeviceProperties(&props, device)); return nvinfer1::plugin::getTrtSMVersionDec(props.major, props.minor); } -inline int getMHAMaskPackedSize(int smVersion, nvinfer1::DataType dataType, int sequenceLength) +inline int32_t getMHAMaskPackedSize(int32_t smVersion, nvinfer1::DataType dataType, int32_t sequenceLength) { // this code must match EmbLayerNormPluginDynamic::getOutputDimensions in embLayerNormPlugin.cpp - int packedSize = unfusedMaskSize; + int32_t packedSize = unfusedMaskSize; bool isSmOK = (smVersion == kSM_75 || smVersion == kSM_80 || smVersion == kSM_86 || smVersion == kSM_87 || smVersion == kSM_90); bool isPrecisionOK = (dataType == nvinfer1::DataType::kINT8 || dataType == nvinfer1::DataType::kHALF); @@ -196,36 +196,38 @@ inline T* devToDev(T const* data, size_t nbElem) } template -cublasStatus_t inline cublasGemm(cublasHandle_t handle, cublasOperation_t transa, cublasOperation_t transb, int m, - int n, int k, const T alpha, T const* A, int lda, T const* B, int ldb, const T beta, T* C, int ldc); +cublasStatus_t inline cublasGemm(cublasHandle_t handle, cublasOperation_t transa, cublasOperation_t transb, int32_t m, + int32_t n, int32_t k, const T alpha, T const* A, int32_t lda, T const* B, int32_t ldb, const T beta, T* C, + int32_t ldc); template <> -cublasStatus_t inline cublasGemm(cublasHandle_t handle, cublasOperation_t transa, cublasOperation_t transb, int m, - int n, int k, float const alpha, float const* A, int lda, float const* B, int ldb, float const beta, float* C, - int ldc) +cublasStatus_t inline cublasGemm(cublasHandle_t handle, cublasOperation_t transa, cublasOperation_t transb, int32_t m, + int32_t n, int32_t k, float const alpha, float const* A, int32_t lda, float const* B, int32_t ldb, float const beta, + float* C, int32_t ldc) { return cublasSgemm(handle, transa, transb, m, n, k, &alpha, A, lda, B, ldb, &beta, C, ldc); } template <> -cublasStatus_t inline cublasGemm(cublasHandle_t handle, cublasOperation_t transa, cublasOperation_t transb, int m, - int n, int k, const half alpha, half const* A, int lda, half const* B, int ldb, const half beta, half* C, int ldc) +cublasStatus_t inline cublasGemm(cublasHandle_t handle, cublasOperation_t transa, cublasOperation_t transb, int32_t m, + int32_t n, int32_t k, const half alpha, half const* A, int32_t lda, half const* B, int32_t ldb, const half beta, + half* C, int32_t ldc) { return cublasHgemm(handle, transa, transb, m, n, k, &alpha, A, lda, B, ldb, &beta, C, ldc); } template cublasStatus_t inline cublasGemmStridedBatchedEx(cublasHandle_t handle, cublasOperation_t transa, - cublasOperation_t transb, int m, int n, int k, const T alpha, T const* A, int lda, long long int strideA, - T const* B, int ldb, long long int strideB, const T beta, T* C, int ldc, long long int strideC, int batchCount, + cublasOperation_t transb, int32_t m, int32_t n, int32_t k, const T alpha, T const* A, int32_t lda, int64_t strideA, + T const* B, int32_t ldb, int64_t strideB, const T beta, T* C, int32_t ldc, int64_t strideC, int32_t batchCount, cublasGemmAlgo_t algo); template <> cublasStatus_t inline cublasGemmStridedBatchedEx(cublasHandle_t handle, cublasOperation_t transa, - cublasOperation_t transb, int m, int n, int k, float const alpha, float const* A, int lda, long long int strideA, - float const* B, int ldb, long long int strideB, float const beta, float* C, int ldc, long long int strideC, - int batchCount, cublasGemmAlgo_t algo) + cublasOperation_t transb, int32_t m, int32_t n, int32_t k, float const alpha, float const* A, int32_t lda, + int64_t strideA, float const* B, int32_t ldb, int64_t strideB, float const beta, float* C, int32_t ldc, + int64_t strideC, int32_t batchCount, cublasGemmAlgo_t algo) { return ::cublasGemmStridedBatchedEx(handle, transa, transb, m, n, k, &alpha, A, CUDA_R_32F, lda, strideA, B, @@ -234,9 +236,9 @@ cublasStatus_t inline cublasGemmStridedBatchedEx(cublasHandle_t handle, cublasOp template <> cublasStatus_t inline cublasGemmStridedBatchedEx(cublasHandle_t handle, cublasOperation_t transa, - cublasOperation_t transb, int m, int n, int k, const half alpha, half const* A, int lda, long long int strideA, - half const* B, int ldb, long long int strideB, const half beta, half* C, int ldc, long long int strideC, - int batchCount, cublasGemmAlgo_t algo) + cublasOperation_t transb, int32_t m, int32_t n, int32_t k, const half alpha, half const* A, int32_t lda, + int64_t strideA, half const* B, int32_t ldb, int64_t strideB, const half beta, half* C, int32_t ldc, + int64_t strideC, int32_t batchCount, cublasGemmAlgo_t algo) { return ::cublasGemmStridedBatchedEx(handle, transa, transb, m, n, k, &alpha, A, CUDA_R_16F, lda, strideA, B, CUDA_R_16F, ldb, strideB, &beta, C, CUDA_R_16F, ldc, strideC, batchCount, CUDA_R_16F, algo); @@ -244,14 +246,14 @@ cublasStatus_t inline cublasGemmStridedBatchedEx(cublasHandle_t handle, cublasOp template cublasStatus_t inline cublasGemmStridedBatched(cublasHandle_t handle, cublasOperation_t transa, - cublasOperation_t transb, int m, int n, int k, const T alpha, T const* A, int lda, long long int strideA, - T const* B, int ldb, long long int strideB, const T beta, T* C, int ldc, long long int strideC, int batchCount); + cublasOperation_t transb, int32_t m, int32_t n, int32_t k, const T alpha, T const* A, int32_t lda, int64_t strideA, + T const* B, int32_t ldb, int64_t strideB, const T beta, T* C, int32_t ldc, int64_t strideC, int32_t batchCount); template <> cublasStatus_t inline cublasGemmStridedBatched(cublasHandle_t handle, cublasOperation_t transa, - cublasOperation_t transb, int m, int n, int k, float const alpha, float const* A, int lda, long long int strideA, - float const* B, int ldb, long long int strideB, float const beta, float* C, int ldc, long long int strideC, - int batchCount) + cublasOperation_t transb, int32_t m, int32_t n, int32_t k, float const alpha, float const* A, int32_t lda, + int64_t strideA, float const* B, int32_t ldb, int64_t strideB, float const beta, float* C, int32_t ldc, + int64_t strideC, int32_t batchCount) { return cublasSgemmStridedBatched( @@ -260,9 +262,9 @@ cublasStatus_t inline cublasGemmStridedBatched(cublasHandle_t handle, cublasOper template <> cublasStatus_t inline cublasGemmStridedBatched(cublasHandle_t handle, cublasOperation_t transa, - cublasOperation_t transb, int m, int n, int k, const half alpha, half const* A, int lda, long long int strideA, - half const* B, int ldb, long long int strideB, const half beta, half* C, int ldc, long long int strideC, - int batchCount) + cublasOperation_t transb, int32_t m, int32_t n, int32_t k, const half alpha, half const* A, int32_t lda, + int64_t strideA, half const* B, int32_t ldb, int64_t strideB, const half beta, half* C, int32_t ldc, + int64_t strideC, int32_t batchCount) { return cublasHgemmStridedBatched( handle, transa, transb, m, n, k, &alpha, A, lda, strideA, B, ldb, strideB, &beta, C, ldc, strideC, batchCount); diff --git a/plugin/common/checkMacrosPlugin.cpp b/plugin/common/checkMacrosPlugin.cpp index a6364396..f94d90d0 100644 --- a/plugin/common/checkMacrosPlugin.cpp +++ b/plugin/common/checkMacrosPlugin.cpp @@ -30,7 +30,7 @@ namespace plugin ILogger* gLogger{}; template -int LogStream::Buf::sync() +int32_t LogStream::Buf::sync() { std::string s = str(); while (!s.empty() && s.back() == '\n') @@ -53,7 +53,7 @@ LogStream gLogInfo; LogStream gLogVerbose; // break-pointable -void throwCudaError(char const* file, char const* function, int line, int status, char const* msg) +void throwCudaError(char const* file, char const* function, int32_t line, int32_t status, char const* msg) { CudaError error(file, function, line, status, msg); error.log(gLogError); @@ -62,7 +62,7 @@ void throwCudaError(char const* file, char const* function, int line, int status } // break-pointable -void throwCublasError(char const* file, char const* function, int line, int status, char const* msg) +void throwCublasError(char const* file, char const* function, int32_t line, int32_t status, char const* msg) { if (msg == nullptr) { @@ -88,7 +88,7 @@ void throwCublasError(char const* file, char const* function, int line, int stat } // break-pointable -void throwCudnnError(char const* file, char const* function, int line, int status, char const* msg) +void throwCudnnError(char const* file, char const* function, int32_t line, int32_t status, char const* msg) { CudnnError error(file, function, line, status, msg); error.log(gLogError); @@ -97,7 +97,7 @@ void throwCudnnError(char const* file, char const* function, int line, int statu } // break-pointable -void throwPluginError(char const* file, char const* function, int line, int status, char const* msg) +void throwPluginError(char const* file, char const* function, int32_t line, int32_t status, char const* msg) { PluginError error(file, function, line, status, msg); reportValidationFailure(msg, file, line); @@ -105,13 +105,13 @@ void throwPluginError(char const* file, char const* function, int line, int stat throw error; } -void logError(char const* msg, char const* file, char const* fn, int line) +void logError(char const* msg, char const* file, char const* fn, int32_t line) { gLogError << "Parameter check failed at: " << file << "::" << fn << "::" << line; gLogError << ", condition: " << msg << std::endl; } -void reportValidationFailure(char const* msg, char const* file, int line) +void reportValidationFailure(char const* msg, char const* file, int32_t line) { std::ostringstream stream; stream << "Validation failed: " << msg << "\n" << file << ':' << line << "\n"; @@ -127,7 +127,7 @@ void reportValidationFailure(char const* msg, char const* file, int line) } // break-pointable -void reportAssertion(char const* msg, char const* file, int line) +void reportAssertion(char const* msg, char const* file, int32_t line) { std::ostringstream stream; stream << "Assertion failed: " << msg << "\n" diff --git a/plugin/common/checkMacrosPlugin.h b/plugin/common/checkMacrosPlugin.h index 4629d1e1..3b28ec08 100644 --- a/plugin/common/checkMacrosPlugin.h +++ b/plugin/common/checkMacrosPlugin.h @@ -37,7 +37,7 @@ class LogStream : public std::ostream class Buf : public std::stringbuf { public: - int sync() override; + int32_t sync() override; }; Buf buffer; @@ -87,23 +87,23 @@ extern LogStream gLogWarning; extern LogStream gLogInfo; extern LogStream gLogVerbose; -void reportValidationFailure(char const* msg, char const* file, int line); -void reportAssertion(char const* msg, char const* file, int line); -void logError(char const* msg, char const* file, char const* fn, int line); +void reportValidationFailure(char const* msg, char const* file, int32_t line); +void reportAssertion(char const* msg, char const* file, int32_t line); +void logError(char const* msg, char const* file, char const* fn, int32_t line); [[noreturn]] void throwCudaError( - char const* file, char const* function, int line, int status, char const* msg = nullptr); + char const* file, char const* function, int32_t line, int32_t status, char const* msg = nullptr); [[noreturn]] void throwCudnnError( - char const* file, char const* function, int line, int status, char const* msg = nullptr); + char const* file, char const* function, int32_t line, int32_t status, char const* msg = nullptr); [[noreturn]] void throwCublasError( - char const* file, char const* function, int line, int status, char const* msg = nullptr); + char const* file, char const* function, int32_t line, int32_t status, char const* msg = nullptr); [[noreturn]] void throwPluginError( - char const* file, char const* function, int line, int status, char const* msg = nullptr); + char const* file, char const* function, int32_t line, int32_t status, char const* msg = nullptr); class TRTException : public std::exception { public: - TRTException(char const* fl, char const* fn, int ln, int st, char const* msg, char const* nm) + TRTException(char const* fl, char const* fn, int32_t ln, int32_t st, char const* msg, char const* nm) : file(fl) , function(fn) , line(ln) @@ -121,8 +121,8 @@ class TRTException : public std::exception protected: char const* file{nullptr}; char const* function{nullptr}; - int line{0}; - int status{0}; + int32_t line{0}; + int32_t status{0}; char const* message{nullptr}; char const* name{nullptr}; }; @@ -130,7 +130,7 @@ class TRTException : public std::exception class CudaError : public TRTException { public: - CudaError(char const* fl, char const* fn, int ln, int stat, char const* msg = nullptr) + CudaError(char const* fl, char const* fn, int32_t ln, int32_t stat, char const* msg = nullptr) : TRTException(fl, fn, ln, stat, msg, "Cuda") { } @@ -139,7 +139,7 @@ class CudaError : public TRTException class CudnnError : public TRTException { public: - CudnnError(char const* fl, char const* fn, int ln, int stat, char const* msg = nullptr) + CudnnError(char const* fl, char const* fn, int32_t ln, int32_t stat, char const* msg = nullptr) : TRTException(fl, fn, ln, stat, msg, "Cudnn") { } @@ -148,7 +148,7 @@ class CudnnError : public TRTException class CublasError : public TRTException { public: - CublasError(char const* fl, char const* fn, int ln, int stat, char const* msg = nullptr) + CublasError(char const* fl, char const* fn, int32_t ln, int32_t stat, char const* msg = nullptr) : TRTException(fl, fn, ln, stat, msg, "cuBLAS") { } @@ -157,7 +157,7 @@ class CublasError : public TRTException class PluginError : public TRTException { public: - PluginError(char const* fl, char const* fn, int ln, int stat, char const* msg = nullptr) + PluginError(char const* fl, char const* fn, int32_t ln, int32_t stat, char const* msg = nullptr) : TRTException(fl, fn, ln, stat, msg, "Plugin") { } @@ -189,9 +189,9 @@ inline void caughtError(std::exception const& e) } \ } -#define PLUGIN_API_CHECK_ENUM_RANGE(Type, val) PLUGIN_API_CHECK(int(val) >= 0 && int(val) < EnumMax()) +#define PLUGIN_API_CHECK_ENUM_RANGE(Type, val) PLUGIN_API_CHECK(int32_t(val) >= 0 && int32_t(val) < EnumMax()) #define PLUGIN_API_CHECK_ENUM_RANGE_RETVAL(Type, val, retval) \ - PLUGIN_API_CHECK_RETVAL(int(val) >= 0 && int(val) < EnumMax(), retval) + PLUGIN_API_CHECK_RETVAL(int32_t(val) >= 0 && int32_t(val) < EnumMax(), retval) #define PLUGIN_CHECK_CUDA(call) \ do \ diff --git a/plugin/common/common.cuh b/plugin/common/common.cuh index 1f20e590..5541bb8c 100644 --- a/plugin/common/common.cuh +++ b/plugin/common/common.cuh @@ -243,9 +243,9 @@ __device__ inline kv_half2 operator+(const kv_half2& a, const kv_half2& b) template using kvp = cub::KeyValuePair; -template +template __device__ inline void layerNorm( - const kvp& threadData, const int ld, const int offset, const P* beta, const P* gamma, T* output) + const kvp& threadData, const int32_t ld, const int32_t offset, const P* beta, const P* gamma, T* output) { // Assuming threadData is already divided by ld @@ -263,9 +263,9 @@ __device__ inline void layerNorm( } __syncthreads(); - for (int i = threadIdx.x; i < ld; i += TPB) + for (int32_t i = threadIdx.x; i < ld; i += TPB) { - const int idx = offset + i; + const int32_t idx = offset + i; const R val = output[idx]; const R g(gamma[i]); const R b(beta[i]); @@ -273,9 +273,9 @@ __device__ inline void layerNorm( } } -template +template __device__ inline void layerNormSmall( - const T val, const kvp& threadData, const int ld, const int idx, const P* beta, const P* gamma, T* output) + const T val, const kvp& threadData, const int32_t ld, const int32_t idx, const P* beta, const P* gamma, T* output) { // Assuming threadData is already divided by ld // Small settings: the block covers the leading dimension TPB >= ld. The input @@ -305,7 +305,7 @@ __device__ inline void layerNormSmall( template __device__ inline void scaledSoftmaxSmall( - const int ld, const int lastValid, const float rsqrtHeadSize, const T* input, T* output) + const int32_t ld, const int32_t lastValid, const float rsqrtHeadSize, const T* input, T* output) { using BlockReduce = cub::BlockReduce; @@ -315,13 +315,13 @@ __device__ inline void scaledSoftmaxSmall( __shared__ float rZ; __shared__ float fMax; - const int offset = (blockIdx.y * gridDim.x + blockIdx.x) * ld; + const int32_t offset = (blockIdx.y * gridDim.x + blockIdx.x) * ld; const float w(rsqrtHeadSize); cub::Sum sum; float threadData(-FLT_MAX); - const int idx = offset + threadIdx.x; + const int32_t idx = offset + threadIdx.x; if (threadIdx.x < lastValid) { threadData = input[idx]; @@ -353,14 +353,14 @@ __device__ inline void scaledSoftmaxSmall( if (threadIdx.x < ld) { - // this will be 0 for threadIdx.x >= lastValid - output[idx] = T(threadData * rZ); + float const val = (threadIdx.x < lastValid) ? threadData * rZ : 0.F; + output[idx] = static_cast(val); } } template __device__ inline void scaledSoftmax( - const int ld, const int lastValid, const float rsqrtHeadSize, const T* input, T* output) + const int32_t ld, const int32_t lastValid, const float rsqrtHeadSize, const T* input, T* output) { using BlockReduce = cub::BlockReduce; __shared__ typename BlockReduce::TempStorage tmpStorage; @@ -368,7 +368,7 @@ __device__ inline void scaledSoftmax( __shared__ float rZ; __shared__ float fMax; - const int offset = (blockIdx.y * gridDim.x + blockIdx.x) * ld; + const int32_t offset = (blockIdx.y * gridDim.x + blockIdx.x) * ld; const float w(rsqrtHeadSize); cub::Sum sum; @@ -378,9 +378,9 @@ __device__ inline void scaledSoftmax( { threadData = 0; } - for (int i = threadIdx.x; i < lastValid; i += TPB) + for (int32_t i = threadIdx.x; i < lastValid; i += TPB) { - const int idx = offset + i; + const int32_t idx = offset + i; threadData = max(static_cast(input[idx]), threadData); } @@ -393,9 +393,9 @@ __device__ inline void scaledSoftmax( threadData = 0; - for (int i = threadIdx.x; i < lastValid; i += TPB) + for (int32_t i = threadIdx.x; i < lastValid; i += TPB) { - const int idx = offset + i; + const int32_t idx = offset + i; threadData += exp((static_cast(input[idx]) - fMax) * w); } @@ -407,9 +407,9 @@ __device__ inline void scaledSoftmax( } __syncthreads(); - for (int i = threadIdx.x; i < ld; i += TPB) + for (int32_t i = threadIdx.x; i < ld; i += TPB) { - const int idx = offset + i; + const int32_t idx = offset + i; const float val = (i < lastValid) ? exp((static_cast(input[idx]) - fMax) * w) * rZ : 0.f; output[idx] = T(val); } @@ -426,7 +426,7 @@ constexpr HDI IntType alignTo(IntType a, IntType b) return ceildiv(a, b) * b; } -template +template struct BytesToType; template <> @@ -450,7 +450,7 @@ struct BytesToType<16> using type = float4; }; -template +template __device__ inline void copy(const void* local, void* data) { using T = typename BytesToType::type; @@ -500,7 +500,7 @@ static inline __device__ uint32_t float4_to_char4(float x, inline __device__ char quantize(const float x, const float qScale) { - int tmpq = __float2int_rn(qScale * x); // scale and round + int32_t tmpq = __float2int_rn(qScale * x); // scale and round char tmpq8 = min(127, max(-127, tmpq)); // clip and cast return tmpq8; } diff --git a/plugin/common/cub_helper.h b/plugin/common/cub_helper.h index 3ac17480..ee8402c4 100644 --- a/plugin/common/cub_helper.h +++ b/plugin/common/cub_helper.h @@ -17,13 +17,13 @@ #include "common/kernels/kernel.h" #include template -size_t cubSortPairsWorkspaceSize(int num_items, int num_segments) +size_t cubSortPairsWorkspaceSize(int32_t num_items, int32_t num_segments) { size_t temp_storage_bytes = 0; cub::DeviceSegmentedRadixSort::SortPairsDescending((void*) NULL, temp_storage_bytes, (KeyT const*) NULL, (KeyT*) NULL, (ValueT const*) NULL, (ValueT*) NULL, num_items, // # items num_segments, // # segments - (int const*) NULL, (int const*) NULL); + (int32_t const*) NULL, (int32_t const*) NULL); return temp_storage_bytes; } diff --git a/plugin/common/cudaDriverWrapper.cpp b/plugin/common/cudaDriverWrapper.cpp index dc143567..a4ce78dd 100644 --- a/plugin/common/cudaDriverWrapper.cpp +++ b/plugin/common/cudaDriverWrapper.cpp @@ -74,7 +74,7 @@ CUresult CUDADriverWrapper::cuGetErrorName(CUresult error, char const** pStr) co return (*_cuGetErrorName)(error, pStr); } -CUresult CUDADriverWrapper::cuFuncSetAttribute(CUfunction hfunc, CUfunction_attribute attrib, int value) const +CUresult CUDADriverWrapper::cuFuncSetAttribute(CUfunction hfunc, CUfunction_attribute attrib, int32_t value) const { return (*_cuFuncSetAttribute)(hfunc, attrib, value); } diff --git a/plugin/common/cudaDriverWrapper.h b/plugin/common/cudaDriverWrapper.h index cdde0531..f92fd99e 100644 --- a/plugin/common/cudaDriverWrapper.h +++ b/plugin/common/cudaDriverWrapper.h @@ -42,7 +42,7 @@ class CUDADriverWrapper CUresult cuGetErrorName(CUresult error, char const** pStr) const; - CUresult cuFuncSetAttribute(CUfunction hfunc, CUfunction_attribute attrib, int value) const; + CUresult cuFuncSetAttribute(CUfunction hfunc, CUfunction_attribute attrib, int32_t value) const; CUresult cuLinkComplete(CUlinkState state, void** cubinOut, size_t* sizeOut) const; @@ -73,24 +73,24 @@ class CUDADriverWrapper private: void* handle; CUresult (*_cuGetErrorName)(CUresult, char const**); - CUresult (*_cuFuncSetAttribute)(CUfunction, CUfunction_attribute, int); + CUresult (*_cuFuncSetAttribute)(CUfunction, CUfunction_attribute, int32_t); CUresult (*_cuLinkComplete)(CUlinkState, void**, size_t*); CUresult (*_cuModuleUnload)(CUmodule); CUresult (*_cuLinkDestroy)(CUlinkState); - CUresult (*_cuLinkCreate)(unsigned int, CUjit_option*, void**, CUlinkState*); + CUresult (*_cuLinkCreate)(uint32_t, CUjit_option*, void**, CUlinkState*); CUresult (*_cuModuleLoadData)(CUmodule*, void const*); CUresult (*_cuModuleGetFunction)(CUfunction*, CUmodule, char const*); - CUresult (*_cuLinkAddFile)(CUlinkState, CUjitInputType, char const*, unsigned int, CUjit_option*, void**); + CUresult (*_cuLinkAddFile)(CUlinkState, CUjitInputType, char const*, uint32_t, CUjit_option*, void**); CUresult (*_cuLinkAddData)( - CUlinkState, CUjitInputType, void*, size_t, char const*, unsigned int, CUjit_option*, void**); - CUresult (*_cuLaunchCooperativeKernel)(CUfunction, unsigned int, unsigned int, unsigned int, unsigned int, - unsigned int, unsigned int, unsigned int, CUstream, void**); + CUlinkState, CUjitInputType, void*, size_t, char const*, uint32_t, CUjit_option*, void**); + CUresult (*_cuLaunchCooperativeKernel)( + CUfunction, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, CUstream, void**); CUresult (*_cuLaunchKernel)(CUfunction f, uint32_t gridDimX, uint32_t gridDimY, uint32_t gridDimZ, uint32_t blockDimX, uint32_t blockDimY, uint32_t blockDimZ, uint32_t sharedMemBytes, CUstream hStream, void** kernelParams, void** extra); }; -inline void cuErrCheck_(CUresult stat, CUDADriverWrapper const& wrap, char const* file, int line) +inline void cuErrCheck_(CUresult stat, CUDADriverWrapper const& wrap, char const* file, int32_t line) { if (stat != CUDA_SUCCESS) { diff --git a/plugin/common/kernels/decodeBbox3DKernels.cu b/plugin/common/kernels/decodeBbox3DKernels.cu index 25d934e7..30ead81c 100644 --- a/plugin/common/kernels/decodeBbox3DKernels.cu +++ b/plugin/common/kernels/decodeBbox3DKernels.cu @@ -22,8 +22,9 @@ namespace nvinfer1 { namespace plugin { -#define checkCudaErrors(status) \ +#define checkCudaErrors(status_) \ { \ + auto const status = status_; \ if (status != 0) \ { \ std::cout << "Cuda failure: " << cudaGetErrorString(status) \ @@ -42,7 +43,7 @@ namespace plugin __device__ float sigmoid(const float x) { return 1.0f / (1.0f + expf(-x)); } __global__ void postprocess_kernal(const float *cls_input, - float *box_input, + float const* box_input, const float *dir_cls_input, float *anchors, float *anchors_bottom_height, @@ -96,7 +97,7 @@ __global__ void postprocess_kernal(const float *cls_input, float *anchor_ptr = anchors + ith_anchor * 4; float z_offset = anchor_ptr[2] / 2 + anchors_bottom_height[ith_anchor / 2]; float anchor[7] = {x_offset, y_offset, z_offset, anchor_ptr[0], anchor_ptr[1], anchor_ptr[2], anchor_ptr[3]}; - float *box_encodings = box_input + box_offset; + float const* box_encodings = box_input + box_offset; float xa = anchor[0]; float ya = anchor[1]; float za = anchor[2]; @@ -136,7 +137,7 @@ __global__ void postprocess_kernal(const float *cls_input, void decodeBbox3DLaunch( const int batch_size, const float *cls_input, - float *box_input, + const float *box_input, const float *dir_cls_input, float *anchors, float *anchors_bottom_height, diff --git a/plugin/common/kernels/kernel.h b/plugin/common/kernels/kernel.h index 94bbab5b..7777402c 100644 --- a/plugin/common/kernels/kernel.h +++ b/plugin/common/kernels/kernel.h @@ -241,12 +241,12 @@ int32_t proposalInference_gpu(cudaStream_t stream, void const* rpn_prob, void co size_t _get_workspace_size( int32_t N, int32_t anc_size_num, int32_t anc_ratio_num, int32_t H, int32_t W, int32_t nmsMaxOut); -void decodeBbox3DLaunch(int32_t const batch_size, float const* cls_input, float* box_input, float const* dir_cls_input, - float* anchors, float* anchors_bottom_height, float* bndbox_output, int32_t* object_counter, - float const min_x_range, float const max_x_range, float const min_y_range, float const max_y_range, - int32_t const feature_x_size, int32_t const feature_y_size, int32_t const num_anchors, int32_t const num_classes, - int32_t const num_box_values, float const score_thresh, float const dir_offset, float const dir_limit_offset, - int32_t const num_dir_bins, cudaStream_t stream = 0); +void decodeBbox3DLaunch(int32_t const batch_size, float const* cls_input, float const* box_input, + float const* dir_cls_input, float* anchors, float* anchors_bottom_height, float* bndbox_output, + int32_t* object_counter, float const min_x_range, float const max_x_range, float const min_y_range, + float const max_y_range, int32_t const feature_x_size, int32_t const feature_y_size, int32_t const num_anchors, + int32_t const num_classes, int32_t const num_box_values, float const score_thresh, float const dir_offset, + float const dir_limit_offset, int32_t const num_dir_bins, cudaStream_t stream = 0); template int32_t pillarScatterKernelLaunch(int32_t batch_size, int32_t max_pillar_num, int32_t num_features, diff --git a/plugin/common/kernels/maskRCNNKernels.h b/plugin/common/kernels/maskRCNNKernels.h index 5b706150..9763816f 100644 --- a/plugin/common/kernels/maskRCNNKernels.h +++ b/plugin/common/kernels/maskRCNNKernels.h @@ -38,7 +38,7 @@ inline size_t nAlignDown(size_t x, size_t align) inline size_t dimVolume(const nvinfer1::Dims& dims) { size_t volume = 1; - for (int i = 0; i < dims.nbDims; ++i) + for (int32_t i = 0; i < dims.nbDims; ++i) volume *= dims.d[i]; return volume; @@ -63,14 +63,14 @@ inline size_t typeSize(const nvinfer1::DataType type) struct RefineNMSParameters { - int backgroundLabelId, numClasses, keepTopK; + int32_t backgroundLabelId, numClasses, keepTopK; float scoreThreshold, iouThreshold; }; struct RefineDetectionWorkSpace { - RefineDetectionWorkSpace( - const int batchSize, const int sampleCount, const RefineNMSParameters& param, const nvinfer1::DataType type); + RefineDetectionWorkSpace(const int32_t batchSize, const int32_t sampleCount, const RefineNMSParameters& param, + const nvinfer1::DataType type); RefineDetectionWorkSpace() = default; @@ -98,8 +98,8 @@ struct RefineDetectionWorkSpace struct ProposalWorkSpace { - ProposalWorkSpace(const int batchSize, const int inputCnt, const int sampleCount, const RefineNMSParameters& param, - const nvinfer1::DataType type); + ProposalWorkSpace(const int32_t batchSize, const int32_t inputCnt, const int32_t sampleCount, + const RefineNMSParameters& param, const nvinfer1::DataType type); ProposalWorkSpace() = default; @@ -134,7 +134,7 @@ struct ProposalWorkSpace struct MultilevelProposeROIWorkSpace { - MultilevelProposeROIWorkSpace(const int batchSize, const int inputCnt, const int sampleCount, + MultilevelProposeROIWorkSpace(const int32_t batchSize, const int32_t inputCnt, const int32_t sampleCount, const RefineNMSParameters& param, const nvinfer1::DataType type); MultilevelProposeROIWorkSpace() = default; @@ -168,7 +168,8 @@ struct MultilevelProposeROIWorkSpace struct ConcatTopKWorkSpace { - ConcatTopKWorkSpace(const int batchSize, const int concatCnt, const int topK, const nvinfer1::DataType inType); + ConcatTopKWorkSpace( + const int32_t batchSize, const int32_t concatCnt, const int32_t topK, const nvinfer1::DataType inType); ConcatTopKWorkSpace() = default; @@ -185,18 +186,18 @@ struct ConcatTopKWorkSpace size_t totalSize = 0; }; -cudaError_t RefineBatchClassNMS(cudaStream_t stream, int N, int samples, nvinfer1::DataType dtype, +cudaError_t RefineBatchClassNMS(cudaStream_t stream, int32_t N, int32_t samples, nvinfer1::DataType dtype, const RefineNMSParameters& param, const RefineDetectionWorkSpace& refineOffset, void* workspace, const void* inScores, const void* inDelta, const void* inCountValid, const void* inROI, void* outDetections); -cudaError_t DetectionPostProcess(cudaStream_t stream, int N, int samples, const float* regWeight, +cudaError_t DetectionPostProcess(cudaStream_t stream, int32_t N, int32_t samples, const float* regWeight, const float inputHeight, const float inputWidth, nvinfer1::DataType dtype, const RefineNMSParameters& param, const RefineDetectionWorkSpace& refineOffset, void* workspace, const void* inScores, const void* inDelta, const void* inCountValid, const void* inROI, void* outDetections); -cudaError_t proposalRefineBatchClassNMS(cudaStream_t stream, int N, - int inputCnt, // candidate anchors - int samples, // preNMS_topK +cudaError_t proposalRefineBatchClassNMS(cudaStream_t stream, int32_t N, + int32_t inputCnt, // candidate anchors + int32_t samples, // preNMS_topK nvinfer1::DataType dtype, const RefineNMSParameters& param, const ProposalWorkSpace& proposalOffset, void* workspace, const void* inScores, const void* inDelta, const void* inCountValid, const void* inAnchors, void* outProposals); @@ -205,9 +206,9 @@ cudaError_t proposalRefineBatchClassNMS(cudaStream_t stream, int N, // inDelta: [N, anchorsCnt, 4] // outScores: [N, topK, 1] // outBbox: [N, topK, 4] -cudaError_t MultilevelPropose(cudaStream_t stream, int N, - int inputCnt, // candidate anchors number among feature map - int samples, // pre nms cnt +cudaError_t MultilevelPropose(cudaStream_t stream, int32_t N, + int32_t inputCnt, // candidate anchors number among feature map + int32_t samples, // pre nms cnt const float* regWeight, const float inputHeight, const float inputWidth, nvinfer1::DataType dtype, const RefineNMSParameters& param, const MultilevelProposeROIWorkSpace& proposalOffset, void* workspace, const void* inScore, const void* inDelta, void* inCountValid, const void* inAnchors, void* outScores, @@ -216,35 +217,33 @@ cudaError_t MultilevelPropose(cudaStream_t stream, int N, // inScores: [N, topK, 1] * featureCnt // inBboxes: [N, topK, 4] * featureCnt // outProposals: [N, topK, 4] -cudaError_t ConcatTopK(cudaStream_t stream, int N, int featureCnt, int topK, nvinfer1::DataType dtype, void* workspace, - const ConcatTopKWorkSpace& spaceOffset, void** inScores, void** inBBox, void* outProposals); +cudaError_t ConcatTopK(cudaStream_t stream, int32_t N, int32_t featureCnt, int32_t topK, nvinfer1::DataType dtype, + void* workspace, const ConcatTopKWorkSpace& spaceOffset, void** inScores, void** inBBox, void* outProposals); -cudaError_t DecodeBBoxes(cudaStream_t stream, int N, - int samples, // number of anchors per image +cudaError_t DecodeBBoxes(cudaStream_t stream, int32_t N, + int32_t samples, // number of anchors per image const float* regWeight, const float inputHeight, const float inputWidth, const void* anchors, // [N, anchors, (y1, x1, y2, x2)] const void* delta, //[N, anchors, (dy, dx, log(dh), log(dw)] - void* outputBbox, - nvinfer1::DataType dtype - ); + void* outputBbox, nvinfer1::DataType dtype); -cudaError_t ApplyDelta2Bboxes(cudaStream_t stream, int N, - int samples, // number of anchors per image +cudaError_t ApplyDelta2Bboxes(cudaStream_t stream, int32_t N, + int32_t samples, // number of anchors per image const void* anchors, // [N, anchors, (y1, x1, y2, x2)] const void* delta, //[N, anchors, (dy, dx, log(dh), log(dw)] void* outputBbox); struct xy_t { - int y; - int x; + int32_t y; + int32_t x; xy_t() : y(0) , x(0) { } - xy_t(int y_, int x_) + xy_t(int32_t y_, int32_t x_) : y(y_) , x(x_) { @@ -256,16 +255,17 @@ cudaError_t roiAlign(cudaStream_t const stream, int32_t const batchSize, xy_t co bool const absCoords, bool const swapCoords, bool const plusOneCoords, int32_t const samplingRatio, void const* rois, void const* const layers[], xy_t const* layerDims, void* pooled, xy_t const poolDims); -cudaError_t roiAlignHalfCenter(cudaStream_t stream, int batchSize, int featureCount, int roiCount, float firstThreshold, +cudaError_t roiAlignHalfCenter(cudaStream_t stream, int32_t batchSize, int32_t featureCount, int32_t roiCount, + float firstThreshold, - int inputHeight, int inputWidth, const void* rois, const void* const layers[], const xy_t* layerDims, + int32_t inputHeight, int32_t inputWidth, const void* rois, const void* const layers[], const xy_t* layerDims, void* pooled, const xy_t poolDims, const nvinfer1::DataType dtype); // RESIZE NEAREST -void resizeNearest(dim3 grid, dim3 block, cudaStream_t stream, int nbatch, float scale, int2 osize, float const* idata, - int istride, int ibatchstride, float* odata, int ostride, int obatchstride); +void resizeNearest(dim3 grid, dim3 block, cudaStream_t stream, int32_t nbatch, float scale, int2 osize, + float const* idata, int32_t istride, int32_t ibatchstride, float* odata, int32_t ostride, int32_t obatchstride); // SPECIAL SLICE -void specialSlice(cudaStream_t stream, int batch_size, int boxes_cnt, const void* idata, void* odata); +void specialSlice(cudaStream_t stream, int32_t batch_size, int32_t boxes_cnt, const void* idata, void* odata); #endif // TRT_MASKRCNN_UTILS_H diff --git a/plugin/common/kernels/reducedMathPlugin.h b/plugin/common/kernels/reducedMathPlugin.h index f52ea72a..777a5e51 100644 --- a/plugin/common/kernels/reducedMathPlugin.h +++ b/plugin/common/kernels/reducedMathPlugin.h @@ -32,7 +32,7 @@ namespace plugin namespace detail { -void findDivisor(int denom, unsigned int& mul_coeff, unsigned int& shift_coeff); +void findDivisor(int32_t denom, uint32_t& mul_coeff, uint32_t& shift_coeff); __host__ __device__ __forceinline__ uint32_t umulhi(uint32_t x, uint32_t y) { @@ -46,51 +46,49 @@ __host__ __device__ __forceinline__ uint32_t umulhi(uint32_t x, uint32_t y) // This is a weird implementation that returns div_up(0,1)=0 but // div_up(0,2)=1 (wrong) -- just do not use it with a=0. -__host__ __device__ inline int div_up(int a, int b) +__host__ __device__ inline int32_t div_up(int32_t a, int32_t b) { return (a - 1) / b + 1; } -} //end namespace detail +} // end namespace detail class ReducedDivisor { public: ReducedDivisor() {} - __host__ __forceinline__ - ReducedDivisor(int _y) + __host__ __forceinline__ ReducedDivisor(int32_t _y) : y(_y) { detail::findDivisor(y, mul_coeff, shift_coeff); } - __host__ __device__ __forceinline__ - ReducedDivisor(unsigned _mul_coeff, unsigned _shift_coeff, int _y) + __host__ __device__ __forceinline__ ReducedDivisor(uint32_t _mul_coeff, uint32_t _shift_coeff, int32_t _y) : mul_coeff(_mul_coeff) , shift_coeff(_shift_coeff) , y(_y) { } - __host__ __device__ __forceinline__ int div(int x) const + __host__ __device__ __forceinline__ int32_t div(int32_t x) const { // if dividing by 1, then findDivisor wouldn't have worked because // mul_coeff would have had to be 2^32, which can't be represented, // so we have to special case that one. return (y != 1) ? detail::umulhi((uint32_t) x, mul_coeff) >> shift_coeff : x; } - __host__ __device__ __forceinline__ int mod(int x) const + __host__ __device__ __forceinline__ int32_t mod(int32_t x) const { return x - (div(x) * y); } - __host__ __device__ __forceinline__ void divmod(int x, int& q, int& mod) const + __host__ __device__ __forceinline__ void divmod(int32_t x, int32_t& q, int32_t& mod) const { q = div(x); mod = x - (q * y); } - __host__ __device__ __forceinline__ int get() const + __host__ __device__ __forceinline__ int32_t get() const { return y; } - inline __host__ void get_mul_shift(unsigned& mul, unsigned& shift) + inline __host__ void get_mul_shift(uint32_t& mul, uint32_t& shift) { mul = mul_coeff; shift = shift_coeff; @@ -99,7 +97,7 @@ class ReducedDivisor protected: uint32_t mul_coeff{}; uint32_t shift_coeff{}; - int y{}; + int32_t y{}; }; } // namespace plugin diff --git a/plugin/common/mrcnn_config.h b/plugin/common/mrcnn_config.h index 93c3cccd..5b3673ca 100644 --- a/plugin/common/mrcnn_config.h +++ b/plugin/common/mrcnn_config.h @@ -26,8 +26,8 @@ namespace MaskRCNNConfig static const nvinfer1::Dims3 IMAGE_SHAPE{3, 1024, 1024}; // Pooled ROIs -static int const POOL_SIZE = 7; -static int const MASK_POOL_SIZE = 14; +static int32_t const POOL_SIZE = 7; +static int32_t const MASK_POOL_SIZE = 14; // Threshold to determine the mask area out of final convolution output static float const MASK_THRESHOLD = 0.5F; @@ -37,7 +37,7 @@ static float const RPN_BBOX_STD_DEV[] = {0.1F, 0.1F, 0.2F, 0.2F}; static float const BBOX_STD_DEV[] = {0.1F, 0.1F, 0.2F, 0.2F}; // Max number of final detections -static int const DETECTION_MAX_INSTANCES = 100; +static int32_t const DETECTION_MAX_INSTANCES = 100; // Minimum probability value to accept a detected instance // ROIs below this threshold are skipped @@ -51,13 +51,13 @@ static float const DETECTION_NMS_THRESHOLD = 0.3F; static const std::vector BACKBONE_STRIDES = {4.F, 8.F, 16.F, 32.F, 64.F}; // Size of the fully-connected layers in the classification graph -static int const FPN_CLASSIF_FC_LAYERS_SIZE = 1024; +static int32_t const FPN_CLASSIF_FC_LAYERS_SIZE = 1024; // Size of the top-down layers used to build the feature pyramid -static int const TOP_DOWN_PYRAMID_SIZE = 256; +static int32_t const TOP_DOWN_PYRAMID_SIZE = 256; // Number of classification classes (including background) -static int const NUM_CLASSES = 1 + 80; // COCO has 80 classes +static int32_t const NUM_CLASSES = 1 + 80; // COCO has 80 classes // Length of square anchor side in pixels static const std::vector RPN_ANCHOR_SCALES = {32.F, 64.F, 128.F, 256.F, 512.F}; @@ -69,18 +69,18 @@ static float const RPN_ANCHOR_RATIOS[] = {0.5F, 1.F, 2.F}; // Anchor stride // If 1 then anchors are created for each cell in the backbone feature map. // If 2, then anchors are created for every other cell, and so on. -static int const RPN_ANCHOR_STRIDE = 1; +static int32_t const RPN_ANCHOR_STRIDE = 1; // Although Python impementation uses 6000, // TRT fails if this number larger than kMAX_TOPK_K defined in engine/checkMacros.h -static int const MAX_PRE_NMS_RESULTS = 1024; // 3840; +static int32_t const MAX_PRE_NMS_RESULTS = 1024; // 3840; // Non-max suppression threshold to filter RPN proposals. // You can increase this during training to generate more propsals. static float const RPN_NMS_THRESHOLD = 0.7F; // ROIs kept after non-maximum suppression (training and inference) -static int const POST_NMS_ROIS_INFERENCE = 1000; +static int32_t const POST_NMS_ROIS_INFERENCE = 1000; // COCO Class names static const std::vector CLASS_NAMES = { diff --git a/plugin/common/nmsHelper.cpp b/plugin/common/nmsHelper.cpp index 2104b098..8b0cc9ca 100644 --- a/plugin/common/nmsHelper.cpp +++ b/plugin/common/nmsHelper.cpp @@ -24,7 +24,7 @@ namespace nvinfer1 namespace plugin { -size_t detectionForwardBBoxDataSize(int N, int C1, DataType DT_BBOX) +size_t detectionForwardBBoxDataSize(int32_t N, int32_t C1, DataType DT_BBOX) { if (DT_BBOX == DataType::kFLOAT) { @@ -39,7 +39,7 @@ size_t detectionForwardBBoxDataSize(int N, int C1, DataType DT_BBOX) return (size_t) -1; } -size_t detectionForwardBBoxPermuteSize(bool shareLocation, int N, int C1, DataType DT_BBOX) +size_t detectionForwardBBoxPermuteSize(bool shareLocation, int32_t N, int32_t C1, DataType DT_BBOX) { if (DT_BBOX == DataType::kFLOAT) { @@ -54,15 +54,15 @@ size_t detectionForwardBBoxPermuteSize(bool shareLocation, int N, int C1, DataTy return (size_t) -1; } -size_t detectionForwardPreNMSSize(int N, int C2) +size_t detectionForwardPreNMSSize(int32_t N, int32_t C2) { - PLUGIN_ASSERT(sizeof(float) == sizeof(int)); + PLUGIN_ASSERT(sizeof(float) == sizeof(int32_t)); return N * C2 * sizeof(float); } -size_t detectionForwardPostNMSSize(int N, int numClasses, int topK) +size_t detectionForwardPostNMSSize(int32_t N, int32_t numClasses, int32_t topK) { - PLUGIN_ASSERT(sizeof(float) == sizeof(int)); + PLUGIN_ASSERT(sizeof(float) == sizeof(int32_t)); return N * numClasses * topK * sizeof(float); } } // namespace plugin diff --git a/plugin/common/nmsUtils.h b/plugin/common/nmsUtils.h index 6270f351..28a4aa7e 100644 --- a/plugin/common/nmsUtils.h +++ b/plugin/common/nmsUtils.h @@ -23,8 +23,8 @@ namespace nvinfer1 { namespace plugin { -size_t detectionInferenceWorkspaceSize(bool shareLocation, int N, int C1, int C2, int numClasses, int numPredsPerClass, - int topK, nvinfer1::DataType DT_BBOX, nvinfer1::DataType DT_SCORE); +size_t detectionInferenceWorkspaceSize(bool shareLocation, int32_t N, int32_t C1, int32_t C2, int32_t numClasses, + int32_t numPredsPerClass, int32_t topK, nvinfer1::DataType DT_BBOX, nvinfer1::DataType DT_SCORE); } // namespace plugin } // namespace nvinfer1 #endif diff --git a/plugin/common/plugin.h b/plugin/common/plugin.h index 3d9d7078..a043339d 100644 --- a/plugin/common/plugin.h +++ b/plugin/common/plugin.h @@ -26,6 +26,7 @@ #include #include +// Enumerator for status typedef enum { STATUS_SUCCESS = 0, diff --git a/plugin/common/reducedMathPlugin.cpp b/plugin/common/reducedMathPlugin.cpp index 36d24458..4e33680a 100644 --- a/plugin/common/reducedMathPlugin.cpp +++ b/plugin/common/reducedMathPlugin.cpp @@ -23,9 +23,9 @@ namespace detail { // Count leading zeros - start from most significant bit. -int clz(int x) +int32_t clz(int32_t x) { - for (int i = 31; i >= 0; --i) + for (int32_t i = 31; i >= 0; --i) { if ((1U << i) & x) { @@ -37,9 +37,9 @@ int clz(int x) #define CUDNN_IS_POW_2(x) (0 == ((x) & ((x) -1))) -int find_log_2(int x, bool round_up = false) +int32_t find_log_2(int32_t x, bool round_up = false) { - int a = 31 - clz(x); + int32_t a = 31 - clz(x); if (round_up) { a += !CUDNN_IS_POW_2(x); @@ -47,7 +47,7 @@ int find_log_2(int x, bool round_up = false) return a; } -void findDivisor(int denom, unsigned int& mul_coeff, unsigned int& shift_coeff) +void findDivisor(int32_t denom, uint32_t& mul_coeff, uint32_t& shift_coeff) { if (denom == 0) { @@ -56,7 +56,7 @@ void findDivisor(int denom, unsigned int& mul_coeff, unsigned int& shift_coeff) if (denom == 1) { // if dividing by 1, reduced math doesn't work because mul_coeff would - // need to be 2^32, which doesn't fit into unsigned int. the div() + // need to be 2^32, which doesn't fit into uint32_t. the div() // routine handles this special case separately. mul_coeff = 0; shift_coeff = 0; diff --git a/plugin/common/templates.h b/plugin/common/templates.h index 3629b5f0..298bb8c2 100644 --- a/plugin/common/templates.h +++ b/plugin/common/templates.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/common/vfcCommon.cpp b/plugin/common/vfcCommon.cpp index 17c1283b..11375350 100644 --- a/plugin/common/vfcCommon.cpp +++ b/plugin/common/vfcCommon.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/common/vfcCommon.h b/plugin/common/vfcCommon.h index 2791017d..a2015177 100644 --- a/plugin/common/vfcCommon.h +++ b/plugin/common/vfcCommon.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/coordConvACPlugin/coordConvACPlugin.cpp b/plugin/coordConvACPlugin/coordConvACPlugin.cpp index d958a265..443ba6d5 100644 --- a/plugin/coordConvACPlugin/coordConvACPlugin.cpp +++ b/plugin/coordConvACPlugin/coordConvACPlugin.cpp @@ -35,7 +35,8 @@ std::vector CoordConvACPluginCreator::mPluginAttributes; CoordConvACPlugin::CoordConvACPlugin() {} -CoordConvACPlugin::CoordConvACPlugin(nvinfer1::DataType iType, int iC, int iH, int iW, int oC, int oH, int oW) +CoordConvACPlugin::CoordConvACPlugin( + nvinfer1::DataType iType, int32_t iC, int32_t iH, int32_t iW, int32_t oC, int32_t oH, int32_t oW) : iType(iType) , iC(iC) , iH(iH) @@ -63,19 +64,19 @@ void CoordConvACPlugin::deserialize(uint8_t const* data, size_t length) PLUGIN_VALIDATE(d == data + length); } -int CoordConvACPlugin::getNbOutputs() const noexcept +int32_t CoordConvACPlugin::getNbOutputs() const noexcept { return 1; } -int CoordConvACPlugin::initialize() noexcept +int32_t CoordConvACPlugin::initialize() noexcept { return STATUS_SUCCESS; } void CoordConvACPlugin::terminate() noexcept {} -Dims CoordConvACPlugin::getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept +Dims CoordConvACPlugin::getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept { PLUGIN_ASSERT(index == 0); PLUGIN_ASSERT(nbInputDims == 1); @@ -92,7 +93,7 @@ Dims CoordConvACPlugin::getOutputDimensions(int index, Dims const* inputs, int n return dimsOutput; } -size_t CoordConvACPlugin::getWorkspaceSize(int maxBatchSize) const noexcept +size_t CoordConvACPlugin::getWorkspaceSize(int32_t maxBatchSize) const noexcept { return 0; } @@ -100,7 +101,7 @@ size_t CoordConvACPlugin::getWorkspaceSize(int maxBatchSize) const noexcept size_t CoordConvACPlugin::getSerializationSize() const noexcept { // iType, iC, iH, iW, oC, oH, oW - return sizeof(nvinfer1::DataType) + sizeof(int) * 6; + return sizeof(nvinfer1::DataType) + sizeof(int32_t) * 6; } void CoordConvACPlugin::serialize(void* buffer) const noexcept @@ -116,9 +117,9 @@ void CoordConvACPlugin::serialize(void* buffer) const noexcept PLUGIN_ASSERT(d == a + getSerializationSize()); } -void CoordConvACPlugin::configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, - DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, nvinfer1::PluginFormat format, int maxBatchSize) noexcept +void CoordConvACPlugin::configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, + int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, + bool const* outputIsBroadcast, nvinfer1::PluginFormat format, int32_t maxBatchSize) noexcept { PLUGIN_ASSERT(nbInputs == 1); PLUGIN_ASSERT(nbOutputs == 1); @@ -180,18 +181,18 @@ char const* CoordConvACPlugin::getPluginNamespace() const noexcept } nvinfer1::DataType CoordConvACPlugin::getOutputDataType( - int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { return inputTypes[0]; } bool CoordConvACPlugin::isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept { return false; } -bool CoordConvACPlugin::canBroadcastInputAcrossBatch(int inputIndex) const noexcept +bool CoordConvACPlugin::canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept { return false; } diff --git a/plugin/coordConvACPlugin/coordConvACPlugin.h b/plugin/coordConvACPlugin/coordConvACPlugin.h index 3b9905b8..a710ba44 100644 --- a/plugin/coordConvACPlugin/coordConvACPlugin.h +++ b/plugin/coordConvACPlugin/coordConvACPlugin.h @@ -35,32 +35,32 @@ class CoordConvACPlugin : public IPluginV2Ext public: CoordConvACPlugin(); - CoordConvACPlugin(DataType iType, int iC, int iH, int iW, int oC, int oH, int oW); + CoordConvACPlugin(DataType iType, int32_t iC, int32_t iH, int32_t iW, int32_t oC, int32_t oH, int32_t oW); CoordConvACPlugin(void const* data, size_t length); ~CoordConvACPlugin() override = default; - int getNbOutputs() const noexcept override; + int32_t getNbOutputs() const noexcept override; - Dims getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept override; + Dims getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept override; - int initialize() noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; - size_t getWorkspaceSize(int maxBatchSize) const noexcept override; + size_t getWorkspaceSize(int32_t maxBatchSize) const noexcept override; - int enqueue(int batchSize, void const* const* inputs, void* const* outputs, void* workspace, + int32_t enqueue(int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; size_t getSerializationSize() const noexcept override; void serialize(void* buffer) const noexcept override; - void configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, + void configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept override; + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept override; bool supportsFormat(DataType type, PluginFormat format) const noexcept override; @@ -73,16 +73,16 @@ class CoordConvACPlugin : public IPluginV2Ext IPluginV2Ext* clone() const noexcept override; nvinfer1::DataType getOutputDataType( - int index, nvinfer1::DataType const* inputType, int nbInputs) const noexcept override; + int32_t index, nvinfer1::DataType const* inputType, int32_t nbInputs) const noexcept override; void setPluginNamespace(char const* pluginNamespace) noexcept override; char const* getPluginNamespace() const noexcept override; bool isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept override; + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept override; - bool canBroadcastInputAcrossBatch(int inputIndex) const noexcept override; + bool canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept override; private: void deserialize(uint8_t const* data, size_t length); diff --git a/plugin/decodeBbox3DPlugin/decodeBbox3D.cpp b/plugin/decodeBbox3DPlugin/decodeBbox3D.cpp index 01cfb22c..c24a4d20 100644 --- a/plugin/decodeBbox3DPlugin/decodeBbox3D.cpp +++ b/plugin/decodeBbox3DPlugin/decodeBbox3D.cpp @@ -16,117 +16,111 @@ */ #include "decodeBbox3D.h" +#include "common/bboxUtils.h" +#include "common/checkMacrosPlugin.h" +#include "common/kernels/kernel.h" #include "common/templates.h" -#include -#include namespace nvinfer1 { namespace plugin { -#define checkCudaErrors(status) \ - { \ - if ((status) != 0) \ - { \ - std::cout << "Cuda failure: " << cudaGetErrorString(status) << " at line " << __LINE__ << " in file " \ - << __FILE__ << " error status: " << (status) << std::endl; \ - abort(); \ - } \ - } using nvinfer1::plugin::DecodeBbox3DPlugin; using nvinfer1::plugin::DecodeBbox3DPluginCreator; +namespace +{ static char const* const kPLUGIN_VERSION{"1"}; static char const* const kPLUGIN_NAME{"DecodeBbox3DPlugin"}; +} // namespace // Static class fields initialization PluginFieldCollection DecodeBbox3DPluginCreator::mFC{}; std::vector DecodeBbox3DPluginCreator::mPluginAttributes; -DecodeBbox3DPlugin::DecodeBbox3DPlugin(float x_min, float x_max, float y_min, float y_max, float z_min, float z_max, - int32_t num_dir_bins, float dir_offset, float dir_limit_offset, std::vector const& anchor_bottom_height, - std::vector const& anchors, float score_thresh) - : min_x_range_(x_min) - , max_x_range_(x_max) - , min_y_range_(y_min) - , max_y_range_(y_max) - , min_z_range_(z_min) - , max_z_range_(z_max) - , num_dir_bins_(num_dir_bins) - , dir_offset_(dir_offset) - , dir_limit_offset_(dir_limit_offset) - , score_thresh_(score_thresh) +DecodeBbox3DPlugin::DecodeBbox3DPlugin(float xMin, float xMax, float yMin, float yMax, float zMin, float zMax, + int32_t numDirBins, float dirOffset, float dirLimitOffset, std::vector const& anchorBottomHeight, + std::vector const& anchors, float scoreThreshold) + : mMinXRange(xMin) + , mMaxXRange(xMax) + , mMinYRange(yMin) + , mMaxYRange(yMax) + , mMinZRange(zMin) + , mMaxZRange(zMax) + , mNumDirBins(numDirBins) + , mDirOffset(dirOffset) + , mDirLimitOffset(dirLimitOffset) + , mScoreThreashold(scoreThreshold) { - anchor_bottom_height_.clear(); - for (size_t i = 0; i < anchor_bottom_height.size(); i++) - anchor_bottom_height_.push_back(anchor_bottom_height[i]); - anchors_.clear(); - for (size_t i = 0; i < anchors.size(); i++) - anchors_.push_back(anchors[i]); - num_classes_ = int(anchor_bottom_height_.size()); - PLUGIN_VALIDATE(num_classes_ > 0); - PLUGIN_VALIDATE(static_cast(num_classes_) * 2 * 4 == anchors_.size()); + mAnchorBottomHeight = anchorBottomHeight; + mAnchors = anchors; + mNumClasses = static_cast(mAnchorBottomHeight.size()); + PLUGIN_VALIDATE(static_cast(mNumClasses) * 2 * 4 == mAnchors.size()); } -DecodeBbox3DPlugin::DecodeBbox3DPlugin(float x_min, float x_max, float y_min, float y_max, float z_min, float z_max, - int num_dir_bins, float dir_offset, float dir_limit_offset, std::vector const& anchor_bottom_height, - std::vector const& anchors, float score_thresh, int feature_h, int feature_w) - : min_x_range_(x_min) - , max_x_range_(x_max) - , min_y_range_(y_min) - , max_y_range_(y_max) - , min_z_range_(z_min) - , max_z_range_(z_max) - , num_dir_bins_(num_dir_bins) - , dir_offset_(dir_offset) - , dir_limit_offset_(dir_limit_offset) - , score_thresh_(score_thresh) - , feature_h_(feature_h) - , feature_w_(feature_w) +DecodeBbox3DPlugin::DecodeBbox3DPlugin(float xMin, float xMax, float yMin, float yMax, float zMin, float zMax, + int32_t numDirBins, float dirOffset, float dirLimitOffset, std::vector const& anchorBottomHeight, + std::vector const& anchors, float scoreThreshold, int32_t feature_h, int32_t feature_w) + : mMinXRange(xMin) + , mMaxXRange(xMax) + , mMinYRange(yMin) + , mMaxYRange(yMax) + , mMinZRange(zMin) + , mMaxZRange(zMax) + , mNumDirBins(numDirBins) + , mDirOffset(dirOffset) + , mDirLimitOffset(dirLimitOffset) + , mScoreThreashold(scoreThreshold) + , mFeatureH(feature_h) + , mFeatureW(feature_w) { - anchor_bottom_height_.clear(); - for (size_t i = 0; i < anchor_bottom_height.size(); i++) - anchor_bottom_height_.push_back(anchor_bottom_height[i]); - anchors_.clear(); - for (size_t i = 0; i < anchors.size(); i++) - anchors_.push_back(anchors[i]); - num_classes_ = int(anchor_bottom_height_.size()); - PLUGIN_VALIDATE(num_classes_ > 0); - PLUGIN_VALIDATE(static_cast(num_classes_) * 2 * 4 == anchors_.size()); + mAnchorBottomHeight = anchorBottomHeight; + mAnchors = anchors; + mNumClasses = static_cast(mAnchorBottomHeight.size()); + PLUGIN_VALIDATE(static_cast(mNumClasses) * 2 * 4 == mAnchors.size()); } DecodeBbox3DPlugin::DecodeBbox3DPlugin(void const* data, size_t length) { - char const* d = reinterpret_cast(data); - min_x_range_ = readFromBuffer(d); - max_x_range_ = readFromBuffer(d); - min_y_range_ = readFromBuffer(d); - max_y_range_ = readFromBuffer(d); - min_z_range_ = readFromBuffer(d); - max_z_range_ = readFromBuffer(d); - num_dir_bins_ = readFromBuffer(d); - dir_offset_ = readFromBuffer(d); - dir_limit_offset_ = readFromBuffer(d); - score_thresh_ = readFromBuffer(d); - num_classes_ = readFromBuffer(d); - feature_h_ = readFromBuffer(d); - feature_w_ = readFromBuffer(d); - anchor_bottom_height_.clear(); - anchors_.clear(); - for (int i = 0; i < num_classes_; i++) - anchor_bottom_height_.push_back(readFromBuffer(d)); - for (int i = 0; i < num_classes_ * 2 * 4; i++) - anchors_.push_back(readFromBuffer(d)); + PLUGIN_VALIDATE(data != nullptr); + auto const* d = reinterpret_cast(data); + mMinXRange = readFromBuffer(d); + mMaxXRange = readFromBuffer(d); + mMinYRange = readFromBuffer(d); + mMaxYRange = readFromBuffer(d); + mMinZRange = readFromBuffer(d); + mMaxZRange = readFromBuffer(d); + mNumDirBins = readFromBuffer(d); + mDirOffset = readFromBuffer(d); + mDirLimitOffset = readFromBuffer(d); + mScoreThreashold = readFromBuffer(d); + mNumClasses = readFromBuffer(d); + mFeatureH = readFromBuffer(d); + mFeatureW = readFromBuffer(d); + + mAnchorBottomHeight.resize(mNumClasses); + for (int32_t i = 0; i < mNumClasses; i++) + { + mAnchorBottomHeight[i] = readFromBuffer(d); + } + + mAnchors.resize(mNumClasses * 2 * 4); + for (int32_t i = 0; i < mNumClasses * 2 * 4; i++) + { + mAnchors[i] = readFromBuffer(d); + } + + PLUGIN_VALIDATE(d == reinterpret_cast(data) + length); } nvinfer1::IPluginV2DynamicExt* DecodeBbox3DPlugin::clone() const noexcept { try { - auto* plugin = new DecodeBbox3DPlugin(min_x_range_, max_x_range_, min_y_range_, max_y_range_, min_z_range_, - max_z_range_, num_dir_bins_, dir_offset_, dir_limit_offset_, anchor_bottom_height_, anchors_, score_thresh_, - feature_h_, feature_w_); + auto* plugin = new DecodeBbox3DPlugin(mMinXRange, mMaxXRange, mMinYRange, mMaxYRange, mMinZRange, mMaxZRange, + mNumDirBins, mDirOffset, mDirLimitOffset, mAnchorBottomHeight, mAnchors, mScoreThreashold, mFeatureH, + mFeatureW); plugin->setPluginNamespace(mNamespace.c_str()); return plugin; } @@ -137,36 +131,56 @@ nvinfer1::IPluginV2DynamicExt* DecodeBbox3DPlugin::clone() const noexcept return nullptr; } -nvinfer1::DimsExprs DecodeBbox3DPlugin::getOutputDimensions( - int outputIndex, nvinfer1::DimsExprs const* inputs, int nbInputs, nvinfer1::IExprBuilder& exprBuilder) noexcept +nvinfer1::DimsExprs DecodeBbox3DPlugin::getOutputDimensions(int32_t outputIndex, nvinfer1::DimsExprs const* inputs, + int32_t nbInputs, nvinfer1::IExprBuilder& exprBuilder) noexcept { - PLUGIN_ASSERT(this->getNbOutputs() == 2); - PLUGIN_ASSERT(outputIndex >= 0 && outputIndex < this->getNbOutputs()); - auto feature_h = inputs[0].d[1]; - auto feature_w = inputs[0].d[2]; - auto batch_size = inputs[0].d[0]; - if (outputIndex == 0) + try { - nvinfer1::DimsExprs dim0{}; - dim0.nbDims = 3; - dim0.d[0] = batch_size; - dim0.d[1] = exprBuilder.operation(nvinfer1::DimensionOperation::kPROD, feature_h[0], - exprBuilder.operation( - nvinfer1::DimensionOperation::kPROD, feature_w[0], exprBuilder.constant(num_classes_ * 2)[0])[0]); - dim0.d[2] = exprBuilder.constant(9); - return dim0; + PLUGIN_VALIDATE(getNbOutputs() == 2); + PLUGIN_VALIDATE(outputIndex >= 0 && outputIndex < getNbOutputs()); + PLUGIN_VALIDATE(inputs != nullptr); + auto const& featureH = inputs[0].d[1]; + auto const& featureW = inputs[0].d[2]; + auto const& batchSize = inputs[0].d[0]; + if (outputIndex == 0) + { + nvinfer1::DimsExprs dim0{}; + dim0.nbDims = 3; + dim0.d[0] = batchSize; + dim0.d[1] = exprBuilder.operation(nvinfer1::DimensionOperation::kPROD, featureH[0], + exprBuilder.operation( + nvinfer1::DimensionOperation::kPROD, featureW[0], exprBuilder.constant(mNumClasses * 2)[0])[0]); + dim0.d[2] = exprBuilder.constant(9); + return dim0; + } + nvinfer1::DimsExprs dim1{}; + dim1.nbDims = 1; + dim1.d[0] = batchSize; + return dim1; } - nvinfer1::DimsExprs dim1{}; - dim1.nbDims = 1; - dim1.d[0] = batch_size; - return dim1; + catch (std::exception const& e) + { + caughtError(e); + } + return nvinfer1::DimsExprs{}; } bool DecodeBbox3DPlugin::supportsFormatCombination( - int pos, nvinfer1::PluginTensorDesc const* inOut, int nbInputs, int nbOutputs) noexcept + int32_t pos, nvinfer1::PluginTensorDesc const* inOut, int32_t nbInputs, int32_t nbOutputs) noexcept { - PLUGIN_ASSERT(nbInputs == 3); - PLUGIN_ASSERT(nbOutputs == 2); + try + { + PLUGIN_VALIDATE(nbInputs == 3); + PLUGIN_VALIDATE(nbOutputs == 2); + PLUGIN_VALIDATE(inOut != nullptr); + PLUGIN_VALIDATE((pos >= 0) && (pos < nbInputs + nbOutputs)); + } + catch (std::exception const& e) + { + caughtError(e); + return false; + } + PluginTensorDesc const& in = inOut[pos]; if (pos == 0) // cls_preds { @@ -191,58 +205,90 @@ bool DecodeBbox3DPlugin::supportsFormatCombination( return false; } -void DecodeBbox3DPlugin::configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int nbInputs, - nvinfer1::DynamicPluginTensorDesc const* out, int nbOutputs) noexcept +void DecodeBbox3DPlugin::configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int32_t nbInputs, + nvinfer1::DynamicPluginTensorDesc const* out, int32_t nbOutputs) noexcept { - feature_h_ = in[0].desc.dims.d[1]; - feature_w_ = in[0].desc.dims.d[2]; + try + { + PLUGIN_VALIDATE(in != nullptr); + mFeatureH = in[0].desc.dims.d[1]; + mFeatureW = in[0].desc.dims.d[2]; + } + catch (std::exception const& e) + { + caughtError(e); + } } -size_t DecodeBbox3DPlugin::getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int nbInputs, - nvinfer1::PluginTensorDesc const* outputs, int nbOutputs) const noexcept +size_t DecodeBbox3DPlugin::getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int32_t nbInputs, + nvinfer1::PluginTensorDesc const* outputs, int32_t nbOutputs) const noexcept { - size_t anchors_size = num_classes_ * 2 * 4 * sizeof(float); - size_t anchor_bottom_height_size = num_classes_ * sizeof(float); + size_t mAnchorsSize = mNumClasses * 2 * 4 * sizeof(float); + size_t mAnchorBottomHeightSize = mNumClasses * sizeof(float); size_t workspaces[2]; - workspaces[0] = anchors_size; - workspaces[1] = anchor_bottom_height_size; + workspaces[0] = mAnchorsSize; + workspaces[1] = mAnchorBottomHeightSize; return calculateTotalWorkspaceSize(workspaces, 2); } -int DecodeBbox3DPlugin::enqueue(nvinfer1::PluginTensorDesc const* inputDesc, +int32_t DecodeBbox3DPlugin::enqueue(nvinfer1::PluginTensorDesc const* inputDesc, nvinfer1::PluginTensorDesc const* outputDesc, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept { - int batchSize = inputDesc[0].dims.d[0]; - // Inputs - float* cls_input = const_cast((float const*) (inputs[0])); - float* box_input = const_cast((float const*) inputs[1]); - float* dir_cls_input = const_cast((float const*) (inputs[2])); - // Outputs - float* bndbox_output = (float*) (outputs[0]); - int* box_num = (int*) (outputs[1]); - // Initialize workspaces - float* anchors = (float*) workspace; - size_t anchors_size = num_classes_ * 2 * 4 * sizeof(float); - float* anchor_bottom_height = (float*) nextWorkspacePtr((int8_t*) anchors, anchors_size); - size_t anchor_bottom_height_size = num_classes_ * sizeof(float); - checkCudaErrors(cudaMemcpyAsync(anchors, &anchors_[0], anchors_size, cudaMemcpyHostToDevice, stream)); - checkCudaErrors(cudaMemcpyAsync( - anchor_bottom_height, &anchor_bottom_height_[0], anchor_bottom_height_size, cudaMemcpyHostToDevice, stream)); - // Initialize box_num to 0 - checkCudaErrors(cudaMemsetAsync(box_num, 0, batchSize * sizeof(int), stream)); - decodeBbox3DLaunch(batchSize, cls_input, box_input, dir_cls_input, anchors, anchor_bottom_height, bndbox_output, - box_num, min_x_range_, max_x_range_, min_y_range_, max_y_range_, feature_w_, feature_h_, num_classes_ * 2, - num_classes_, 7, score_thresh_, dir_offset_, dir_limit_offset_, num_dir_bins_, stream); - return 0; + try + { + int32_t batchSize = inputDesc[0].dims.d[0]; + + // Inputs + auto const* clsInput = static_cast(inputs[0]); + auto const* boxInput = static_cast(inputs[1]); + auto const* dirClsInput = static_cast(inputs[2]); + + // Outputs + auto* bndboxOutput = static_cast(outputs[0]); + auto* boxNum = static_cast(outputs[1]); + + // Initialize workspaces + auto* anchors = static_cast(workspace); + size_t anchorsSize = mNumClasses * 2 * 4 * sizeof(float); + auto* anchorBottomHeight + = reinterpret_cast(nextWorkspacePtr(reinterpret_cast(anchors), anchorsSize)); + size_t anchorBottomHeightSize = mNumClasses * sizeof(float); + PLUGIN_CUASSERT(cudaMemcpyAsync(anchors, &mAnchors[0], anchorsSize, cudaMemcpyHostToDevice, stream)); + PLUGIN_CUASSERT(cudaMemcpyAsync( + anchorBottomHeight, &mAnchorBottomHeight[0], anchorBottomHeightSize, cudaMemcpyHostToDevice, stream)); + // Initialize boxNum to 0 + PLUGIN_CUASSERT(cudaMemsetAsync(boxNum, 0, batchSize * sizeof(int32_t), stream)); + + decodeBbox3DLaunch(batchSize, clsInput, boxInput, dirClsInput, anchors, anchorBottomHeight, bndboxOutput, + boxNum, mMinXRange, mMaxXRange, mMinYRange, mMaxYRange, mFeatureW, mFeatureH, mNumClasses * 2, mNumClasses, + 7, mScoreThreashold, mDirOffset, mDirLimitOffset, mNumDirBins, stream); + return cudaPeekAtLastError(); + } + catch (std::exception const& e) + { + caughtError(e); + } + return STATUS_FAILURE; } nvinfer1::DataType DecodeBbox3DPlugin::getOutputDataType( - int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { - if (index == 0) - return inputTypes[0]; - return nvinfer1::DataType::kINT32; + try + { + PLUGIN_VALIDATE(inputTypes != nullptr); + if (index == 0) + { + return inputTypes[0]; + } + return nvinfer1::DataType::kINT32; + } + catch (std::exception const& e) + { + caughtError(e); + } + return nvinfer1::DataType{}; } char const* DecodeBbox3DPlugin::getPluginType() const noexcept @@ -255,12 +301,12 @@ char const* DecodeBbox3DPlugin::getPluginVersion() const noexcept return kPLUGIN_VERSION; } -int DecodeBbox3DPlugin::getNbOutputs() const noexcept +int32_t DecodeBbox3DPlugin::getNbOutputs() const noexcept { return 2; } -int DecodeBbox3DPlugin::initialize() noexcept +int32_t DecodeBbox3DPlugin::initialize() noexcept { return 0; } @@ -269,31 +315,38 @@ void DecodeBbox3DPlugin::terminate() noexcept {} size_t DecodeBbox3DPlugin::getSerializationSize() const noexcept { - size_t scalar_size = 9 * sizeof(float) + 4 * sizeof(int); - size_t vector_size = num_classes_ * 9 * sizeof(float); - return scalar_size + vector_size; + size_t scalarSize = 9 * sizeof(float) + 4 * sizeof(int32_t); + size_t vectorSize = mNumClasses * 9 * sizeof(float); + return scalarSize + vectorSize; } void DecodeBbox3DPlugin::serialize(void* buffer) const noexcept { - char* d = reinterpret_cast(buffer); - writeToBuffer(d, min_x_range_); - writeToBuffer(d, max_x_range_); - writeToBuffer(d, min_y_range_); - writeToBuffer(d, max_y_range_); - writeToBuffer(d, min_z_range_); - writeToBuffer(d, max_z_range_); - writeToBuffer(d, num_dir_bins_); - writeToBuffer(d, dir_offset_); - writeToBuffer(d, dir_limit_offset_); - writeToBuffer(d, score_thresh_); - writeToBuffer(d, num_classes_); - writeToBuffer(d, feature_h_); - writeToBuffer(d, feature_w_); - for (int i = 0; i < num_classes_; i++) - writeToBuffer(d, anchor_bottom_height_[i]); - for (int i = 0; i < num_classes_ * 2 * 4; i++) - writeToBuffer(d, anchors_[i]); + PLUGIN_ASSERT(buffer != nullptr); + auto* d = reinterpret_cast(buffer); + auto* const start = d; + writeToBuffer(d, mMinXRange); + writeToBuffer(d, mMaxXRange); + writeToBuffer(d, mMinYRange); + writeToBuffer(d, mMaxYRange); + writeToBuffer(d, mMinZRange); + writeToBuffer(d, mMaxZRange); + writeToBuffer(d, mNumDirBins); + writeToBuffer(d, mDirOffset); + writeToBuffer(d, mDirLimitOffset); + writeToBuffer(d, mScoreThreashold); + writeToBuffer(d, mNumClasses); + writeToBuffer(d, mFeatureH); + writeToBuffer(d, mFeatureW); + for (int32_t i = 0; i < mNumClasses; i++) + { + writeToBuffer(d, mAnchorBottomHeight[i]); + } + for (int32_t i = 0; i < mNumClasses * 2 * 4; i++) + { + writeToBuffer(d, mAnchors[i]); + } + PLUGIN_ASSERT(d == start + getSerializationSize()); } void DecodeBbox3DPlugin::destroy() noexcept @@ -303,7 +356,15 @@ void DecodeBbox3DPlugin::destroy() noexcept void DecodeBbox3DPlugin::setPluginNamespace(char const* libNamespace) noexcept { - mNamespace = libNamespace; + try + { + PLUGIN_VALIDATE(libNamespace != nullptr); + mNamespace = libNamespace; + } + catch (std::exception const& e) + { + caughtError(e); + } } char const* DecodeBbox3DPlugin::getPluginNamespace() const noexcept @@ -341,74 +402,73 @@ PluginFieldCollection const* DecodeBbox3DPluginCreator::getFieldNames() noexcept return &mFC; } -IPluginV2* DecodeBbox3DPluginCreator::createPlugin(char const* name, PluginFieldCollection const* fc) noexcept +IPluginV2* DecodeBbox3DPluginCreator::createPlugin(char const* /*name*/, PluginFieldCollection const* fc) noexcept { try { + PLUGIN_VALIDATE(fc != nullptr); PluginField const* fields = fc->fields; - int nbFields = fc->nbFields; - float point_cloud_range[6] = {0.0F}; + + // Initialize default values for attributes. + float pointCloudRange[6] = {0.F}; std::vector anchors{}; - std::vector anchor_bottom_height{}; - float dir_offset = 0.78539F; - float dir_limit_offset = 0.0F; - int num_dir_bins = 2; - float score_thresh = 0.1F; - for (int i = 0; i < nbFields; ++i) + std::vector anchorBottomHeight{}; + float dirOffset = 0.78539F; + float dirLimitOffset = 0.F; + int32_t numDirBins = 2; + float scoreThreshold = 0.F; + + for (int32_t i = 0; i < fc->nbFields; ++i) { char const* attr_name = fields[i].name; if (!strcmp(attr_name, "point_cloud_range")) { - float const* d = static_cast(fields[i].data); - point_cloud_range[0] = d[0]; - point_cloud_range[1] = d[1]; - point_cloud_range[2] = d[2]; - point_cloud_range[3] = d[3]; - point_cloud_range[4] = d[4]; - point_cloud_range[5] = d[5]; + auto const* d = static_cast(fields[i].data); + for (int32_t pointCloudIdx = 0; pointCloudIdx < 6; pointCloudIdx++) + { + pointCloudRange[pointCloudIdx] = d[pointCloudIdx]; + } } else if (!strcmp(attr_name, "anchors")) { - float const* as = static_cast(fields[i].data); - for (int j = 0; j < fields[i].length; ++j) + auto const* d = static_cast(fields[i].data); + for (int32_t j = 0; j < fields[i].length; ++j) { - anchors.push_back(*as); - ++as; + anchors.push_back(d[j]); } } else if (!strcmp(attr_name, "anchor_bottom_height")) { - float const* ah = static_cast(fields[i].data); - for (int j = 0; j < fields[i].length; ++j) + auto const* d = static_cast(fields[i].data); + for (int32_t j = 0; j < fields[i].length; ++j) { - anchor_bottom_height.push_back(*ah); - ++ah; + anchorBottomHeight.push_back(d[j]); } } else if (!strcmp(attr_name, "dir_offset")) { - float const* d = static_cast(fields[i].data); - dir_offset = d[0]; + auto const* d = static_cast(fields[i].data); + dirOffset = d[0]; } else if (!strcmp(attr_name, "dir_limit_offset")) { - float const* d = static_cast(fields[i].data); - dir_limit_offset = d[0]; + auto const* d = static_cast(fields[i].data); + dirLimitOffset = d[0]; } else if (!strcmp(attr_name, "num_dir_bins")) { - int const* d = static_cast(fields[i].data); - num_dir_bins = d[0]; + auto const* d = static_cast(fields[i].data); + numDirBins = d[0]; } else if (!strcmp(attr_name, "score_thresh")) { - float const* d = static_cast(fields[i].data); - score_thresh = d[0]; + auto const* d = static_cast(fields[i].data); + scoreThreshold = d[0]; } } - IPluginV2* plugin = new DecodeBbox3DPlugin(point_cloud_range[0], point_cloud_range[3], point_cloud_range[1], - point_cloud_range[4], point_cloud_range[2], point_cloud_range[5], num_dir_bins, dir_offset, - dir_limit_offset, anchor_bottom_height, anchors, score_thresh); + IPluginV2* plugin = new DecodeBbox3DPlugin(pointCloudRange[0], pointCloudRange[3], pointCloudRange[1], + pointCloudRange[4], pointCloudRange[2], pointCloudRange[5], numDirBins, dirOffset, dirLimitOffset, + anchorBottomHeight, anchors, scoreThreshold); return plugin; } catch (std::exception const& e) @@ -419,7 +479,7 @@ IPluginV2* DecodeBbox3DPluginCreator::createPlugin(char const* name, PluginField } IPluginV2* DecodeBbox3DPluginCreator::deserializePlugin( - char const* name, void const* serialData, size_t serialLength) noexcept + char const* /*name*/, void const* serialData, size_t serialLength) noexcept { try { @@ -434,7 +494,15 @@ IPluginV2* DecodeBbox3DPluginCreator::deserializePlugin( void DecodeBbox3DPluginCreator::setPluginNamespace(char const* libNamespace) noexcept { - mNamespace = libNamespace; + try + { + PLUGIN_VALIDATE(libNamespace != nullptr); + mNamespace = libNamespace; + } + catch (std::exception const& e) + { + caughtError(e); + } } char const* DecodeBbox3DPluginCreator::getPluginNamespace() const noexcept diff --git a/plugin/decodeBbox3DPlugin/decodeBbox3D.h b/plugin/decodeBbox3DPlugin/decodeBbox3D.h index 6b2bebee..ea85785a 100644 --- a/plugin/decodeBbox3DPlugin/decodeBbox3D.h +++ b/plugin/decodeBbox3DPlugin/decodeBbox3D.h @@ -19,8 +19,6 @@ #define _DECODE_BBOX_3D_H_ #include "NvInferPlugin.h" -#include "common/bboxUtils.h" -#include "common/kernels/kernel.h" #include #include #include @@ -34,33 +32,33 @@ class DecodeBbox3DPlugin : public nvinfer1::IPluginV2DynamicExt { public: DecodeBbox3DPlugin() = delete; - DecodeBbox3DPlugin(float x_min, float x_max, float y_min, float y_max, float z_min, float z_max, int num_dir_bins, - float dir_offset, float dir_limit_offset, std::vector const& anchor_bottom_height, - std::vector const& anchors, float score_thresh); - DecodeBbox3DPlugin(float x_min, float x_max, float y_min, float y_max, float z_min, float z_max, int num_dir_bins, - float dir_offset, float dir_limit_offset, std::vector const& anchor_bottom_height, - std::vector const& anchors, float score_thresh, int feature_h, int feature_w); + DecodeBbox3DPlugin(float xMin, float xMax, float yMin, float yMax, float zMin, float zMax, int32_t numDirBins, + float dirOffset, float dirLimitOffset, std::vector const& anchorBottomHeight, + std::vector const& anchors, float scoreThresh); + DecodeBbox3DPlugin(float xMin, float xMax, float yMin, float yMax, float zMin, float zMax, int32_t numDirBins, + float dirOffset, float dirLimitOffset, std::vector const& anchorBottomHeight, + std::vector const& anchors, float scoreThresh, int32_t featureH, int32_t featureW); DecodeBbox3DPlugin(void const* data, size_t length); // IPluginV2DynamicExt Methods nvinfer1::IPluginV2DynamicExt* clone() const noexcept override; - nvinfer1::DimsExprs getOutputDimensions(int outputIndex, nvinfer1::DimsExprs const* inputs, int nbInputs, + nvinfer1::DimsExprs getOutputDimensions(int32_t outputIndex, nvinfer1::DimsExprs const* inputs, int32_t nbInputs, nvinfer1::IExprBuilder& exprBuilder) noexcept override; bool supportsFormatCombination( - int pos, nvinfer1::PluginTensorDesc const* inOut, int nbInputs, int nbOutputs) noexcept override; - void configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int nbInputs, - nvinfer1::DynamicPluginTensorDesc const* out, int nbOutputs) noexcept override; - size_t getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int nbInputs, - nvinfer1::PluginTensorDesc const* outputs, int nbOutputs) const noexcept override; - int enqueue(nvinfer1::PluginTensorDesc const* inputDesc, nvinfer1::PluginTensorDesc const* outputDesc, + int32_t pos, nvinfer1::PluginTensorDesc const* inOut, int32_t nbInputs, int32_t nbOutputs) noexcept override; + void configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int32_t nbInputs, + nvinfer1::DynamicPluginTensorDesc const* out, int32_t nbOutputs) noexcept override; + size_t getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int32_t nbInputs, + nvinfer1::PluginTensorDesc const* outputs, int32_t nbOutputs) const noexcept override; + int32_t enqueue(nvinfer1::PluginTensorDesc const* inputDesc, nvinfer1::PluginTensorDesc const* outputDesc, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; // IPluginV2Ext Methods nvinfer1::DataType getOutputDataType( - int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; // IPluginV2 Methods char const* getPluginType() const noexcept override; char const* getPluginVersion() const noexcept override; - int getNbOutputs() const noexcept override; - int initialize() noexcept override; + int32_t getNbOutputs() const noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; size_t getSerializationSize() const noexcept override; void serialize(void* buffer) const noexcept override; @@ -70,21 +68,21 @@ class DecodeBbox3DPlugin : public nvinfer1::IPluginV2DynamicExt private: std::string mNamespace; - float min_x_range_; - float max_x_range_; - float min_y_range_; - float max_y_range_; - float min_z_range_; - float max_z_range_; - int num_dir_bins_; - float dir_offset_; - float dir_limit_offset_; - int num_classes_; - std::vector anchor_bottom_height_; - std::vector anchors_; - float score_thresh_; - int feature_h_; - int feature_w_; + float mMinXRange; + float mMaxXRange; + float mMinYRange; + float mMaxYRange; + float mMinZRange; + float mMaxZRange; + int32_t mNumDirBins; + float mDirOffset; + float mDirLimitOffset; + int32_t mNumClasses; + std::vector mAnchorBottomHeight; + std::vector mAnchors; + float mScoreThreashold; + int32_t mFeatureH; + int32_t mFeatureW; }; class DecodeBbox3DPluginCreator : public nvinfer1::IPluginCreator diff --git a/plugin/detectionLayerPlugin/detectionLayerPlugin.cpp b/plugin/detectionLayerPlugin/detectionLayerPlugin.cpp index 0a993b7b..840156cd 100644 --- a/plugin/detectionLayerPlugin/detectionLayerPlugin.cpp +++ b/plugin/detectionLayerPlugin/detectionLayerPlugin.cpp @@ -253,14 +253,18 @@ DetectionLayer::DetectionLayer(void const* data, size_t length) mType = DataType::kFLOAT; } -bool DetectionLayer::checkValidInputs(nvinfer1::Dims const* inputs, int32_t nbInputDims) +void DetectionLayer::checkValidInputs(nvinfer1::Dims const* inputs, int32_t nbInputDims) { // classifier_delta_bbox[N, anchors, num_classes*4, 1, 1] // classifier_class[N, anchors, num_classes, 1, 1] // rpn_rois[N, anchors, 4] - return (nbInputDims != 3) && (inputs[0].nbDims == 4 && inputs[0].d[1] == mNbClasses * 4) && // delta_bbox - (inputs[1].nbDims == 4 && inputs[1].d[1] == mNbClasses) && // score - (inputs[2].nbDims == 2 && inputs[2].d[1] == 4); // roi + PLUGIN_VALIDATE(nbInputDims == 3); + // delta_bbox + PLUGIN_VALIDATE(inputs[0].nbDims == 4 && inputs[0].d[1] == mNbClasses * 4); + // score + PLUGIN_VALIDATE(inputs[1].nbDims == 4 && inputs[1].d[1] == mNbClasses); + // roi + PLUGIN_VALIDATE(inputs[2].nbDims == 2 && inputs[2].d[1] == 4); } size_t DetectionLayer::getWorkspaceSize(int32_t batchSize) const noexcept @@ -271,13 +275,18 @@ size_t DetectionLayer::getWorkspaceSize(int32_t batchSize) const noexcept Dims DetectionLayer::getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept { - if (!(checkValidInputs(inputs, nbInputDims) && (index == 0))) + try { - return Dims{}; + checkValidInputs(inputs, nbInputDims); + PLUGIN_VALIDATE(index == 0); + // [N, anchors, (y1, x1, y2, x2, class_id, score)] + return {2, {mKeepTopK, 6}}; } - - // [N, anchors, (y1, x1, y2, x2, class_id, score)] - return {2, {mKeepTopK, 6}}; + catch (std::exception const& e) + { + caughtError(e); + } + return Dims{}; } int32_t DetectionLayer::enqueue( @@ -334,12 +343,19 @@ void DetectionLayer::configurePlugin(Dims const* inputDims, int32_t nbInputs, Di DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept { - PLUGIN_ASSERT(checkValidInputs(inputDims, nbInputs)); - PLUGIN_ASSERT(inputDims[0].d[0] == inputDims[1].d[0] && inputDims[1].d[0] == inputDims[2].d[0]); + try + { + checkValidInputs(inputDims, nbInputs); + PLUGIN_VALIDATE(inputDims[0].d[0] == inputDims[1].d[0] && inputDims[1].d[0] == inputDims[2].d[0]); - mAnchorsCnt = inputDims[2].d[0]; - mType = inputTypes[0]; - mMaxBatchSize = maxBatchSize; + mAnchorsCnt = inputDims[2].d[0]; + mType = inputTypes[0]; + mMaxBatchSize = maxBatchSize; + } + catch (std::exception const& e) + { + caughtError(e); + } } // Attach the plugin object to an execution context and grant the plugin the access to some context resource. diff --git a/plugin/detectionLayerPlugin/detectionLayerPlugin.h b/plugin/detectionLayerPlugin/detectionLayerPlugin.h index 82cd77b4..adbf535d 100644 --- a/plugin/detectionLayerPlugin/detectionLayerPlugin.h +++ b/plugin/detectionLayerPlugin/detectionLayerPlugin.h @@ -88,7 +88,7 @@ class DetectionLayer : public IPluginV2Ext void detachFromContext() noexcept override; private: - bool checkValidInputs(nvinfer1::Dims const* inputs, int32_t nbInputDims); + void checkValidInputs(nvinfer1::Dims const* inputs, int32_t nbInputDims); int32_t mBackgroundLabel; int32_t mNbClasses; diff --git a/plugin/disentangledAttentionPlugin/disentangledAttentionPlugin.cpp b/plugin/disentangledAttentionPlugin/disentangledAttentionPlugin.cpp index bbb7db79..4a1c1fc8 100644 --- a/plugin/disentangledAttentionPlugin/disentangledAttentionPlugin.cpp +++ b/plugin/disentangledAttentionPlugin/disentangledAttentionPlugin.cpp @@ -31,20 +31,10 @@ std::vector DisentangledAttentionPluginCreator::mPluginAttributes; REGISTER_TENSORRT_PLUGIN(DisentangledAttentionPluginCreator); -#define CHECK_CUDNN(call) \ - do \ - { \ - cudnnStatus_t status = call; \ - if (status != CUDNN_STATUS_SUCCESS) \ - { \ - return status; \ - } \ - } while (0) - namespace { -constexpr char const* DEBERTA_NAME{"DisentangledAttention_TRT"}; -constexpr char const* DEBERTA_VERSION{"1"}; +constexpr char const* kDEBERTA_PLUGIN_NAME{"DisentangledAttention_TRT"}; +constexpr char const* kDEBERTA_PLUGIN_VERSION{"1"}; } // namespace DisentangledAttentionPlugin::DisentangledAttentionPlugin() {} @@ -62,11 +52,6 @@ DisentangledAttentionPlugin::DisentangledAttentionPlugin(void const* serialData, deserialize_value(&serialData, &serialLength, &mFactor); } -DisentangledAttentionPlugin::~DisentangledAttentionPlugin() -{ - terminate(); -} - int32_t DisentangledAttentionPlugin::getNbOutputs() const noexcept { return 1; @@ -74,62 +59,41 @@ int32_t DisentangledAttentionPlugin::getNbOutputs() const noexcept int32_t DisentangledAttentionPlugin::initialize() noexcept { - // if need large amount of GPU memory, recommend to specify in getWorkspaceSize so TRT allocates it. If not, when a - // plugin is called many times, the memory manually allocated by this initialize() is repeated many times -- may - // overflow return 0; } char const* DisentangledAttentionPlugin::getPluginType() const noexcept { - return DEBERTA_NAME; + return kDEBERTA_PLUGIN_NAME; } char const* DisentangledAttentionPlugin::getPluginVersion() const noexcept { - return DEBERTA_VERSION; + return kDEBERTA_PLUGIN_VERSION; } // IPluginV2DynamicExt Methods nvinfer1::DimsExprs DisentangledAttentionPlugin::getOutputDimensions( int32_t index, nvinfer1::DimsExprs const* inputs, int32_t nbInputs, nvinfer1::IExprBuilder& exprBuilder) noexcept { - nvinfer1::DimsExprs output; - - PLUGIN_ASSERT(nbInputs == 3); // 3 inputs - output = inputs[0]; // same as input[0], i.e. data0 - - PLUGIN_ASSERT(index < 1); // only one output - - return output; -} - -void DisentangledAttentionPlugin::attachToContext( - cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept -{ -} - -// Detach the plugin object from its execution context. -void DisentangledAttentionPlugin::detachFromContext() noexcept {} - -template -TDataType const* DisentangledAttentionPlugin::pointer_const_cast(void const* const p) -{ - return static_cast(p); + try + { + PLUGIN_VALIDATE(inputs != nullptr); + PLUGIN_VALIDATE(index == 0); // Only one output + return inputs[0]; + } + catch (std::exception const& e) + { + caughtError(e); + } + return nvinfer1::DimsExprs{}; } template -TDataType* DisentangledAttentionPlugin::pointer_cast(void* const p) -{ - return static_cast(p); -} - -int32_t DisentangledAttentionPlugin::enqueue(nvinfer1::PluginTensorDesc const* inputDesc, - nvinfer1::PluginTensorDesc const* outputDesc, void const* const* inputs, void* const* outputs, void* workspace, - cudaStream_t stream) noexcept +void DisentangledAttentionPlugin::enqueueType(nvinfer1::PluginTensorDesc const* inputDesc, + nvinfer1::PluginTensorDesc const* outputDesc, void const* const* inputs, void* const* outputs, cudaStream_t stream, + TDataType factor) { - -#if kDISENTANGLED_VERSION == 1 nvinfer1::Dims dims0 = inputDesc[0].dims; nvinfer1::Dims dims1 = inputDesc[1].dims; nvinfer1::Dims dims2 = inputDesc[2].dims; @@ -138,84 +102,45 @@ int32_t DisentangledAttentionPlugin::enqueue(nvinfer1::PluginTensorDesc const* i dim3 dimData2(dims2.d[0], dims2.d[1], dims2.d[2]); dim3 dimResult(dimData0); - dim3 block_optimized(kDISENTANGLED_TILESIZE_V1, kDISENTANGLED_BLOCKDIMY_V1); - dim3 grid_optimized((dimResult.z - 1) / kDISENTANGLED_TILESIZE_V1 + 1, - (dimResult.y - 1) / kDISENTANGLED_TILESIZE_V1 + 1, dimResult.x); + dim3 blockOptimized(kDISENTANGLED_TILESIZE, kDISENTANGLED_BLOCKDIMY); + dim3 gridOptimized( + (dimResult.z - 1) / kDISENTANGLED_TILESIZE + 1, (dimResult.y - 1) / kDISENTANGLED_TILESIZE + 1, dimResult.x); - if (inputDesc[0].type == nvinfer1::DataType::kFLOAT) - { - auto const* data0 = pointer_const_cast(inputs[0]); - auto const* data1 = pointer_const_cast(inputs[1]); - auto const* data2 = pointer_const_cast(inputs[2]); - auto* result = pointer_cast(outputs[0]); - disentangled_kernel_wrapper(data0, data1, data2, - result, dimData0, dimData1, dimData2, dimResult, mFactor, mSpan, block_optimized, grid_optimized, stream); - } - else if (inputDesc[0].type == nvinfer1::DataType::kHALF) - { - auto const* data0 = pointer_const_cast<__half>(inputs[0]); - auto const* data1 = pointer_const_cast<__half>(inputs[1]); - auto const* data2 = pointer_const_cast<__half>(inputs[2]); - auto* result = pointer_cast<__half>(outputs[0]); - __half factor = __float2half(mFactor); - disentangled_kernel_wrapper<__half, kDISENTANGLED_TILESIZE_V1, kDISENTANGLED_BLOCKDIMY_V1>(data0, data1, data2, - result, dimData0, dimData1, dimData2, dimResult, factor, mSpan, block_optimized, grid_optimized, stream); - } - else if (inputDesc[0].type == nvinfer1::DataType::kINT8) - { - auto const* data0 = pointer_const_cast(inputs[0]); - auto const* data1 = pointer_const_cast(inputs[1]); - auto const* data2 = pointer_const_cast(inputs[2]); - auto* result = pointer_cast(outputs[0]); - int8_t factor = int8_t(mFactor); - disentangled_kernel_wrapper(data0, data1, data2, - result, dimData0, dimData1, dimData2, dimResult, factor, mSpan, block_optimized, grid_optimized, stream); - } -#elif kDISENTANGLED_VERSION == 2 - nvinfer1::Dims dims0 = inputDesc[0].dims; - nvinfer1::Dims dims1 = inputDesc[1].dims; - nvinfer1::Dims dims2 = inputDesc[2].dims; - dim3 dimData0(dims0.d[0], dims0.d[1], dims0.d[2]); - dim3 dimData1(dims1.d[0], dims1.d[1], dims1.d[2]); - dim3 dimData2(dims2.d[0], dims2.d[1], dims2.d[2]); - dim3 dimResult(dimData0); - - dim3 block_optimized(kDISENTANGLED_TILESIZE_V2, kDISENTANGLED_BLOCKDIMY_V2); - dim3 grid_optimized((dimResult.z - 1) / kDISENTANGLED_TILESIZE_V2 + 1, - (dimResult.y - 1) / kDISENTANGLED_TILESIZE_V2 + 1, dimResult.x); + auto const* data0 = static_cast(inputs[0]); + auto const* data1 = static_cast(inputs[1]); + auto const* data2 = static_cast(inputs[2]); + auto* result = static_cast(outputs[0]); + disentangled_kernel_wrapper(data0, data1, data2, result, + dimData0, dimData1, dimData2, dimResult, factor, mSpan, blockOptimized, gridOptimized, stream); +} - if (inputDesc[0].type == nvinfer1::DataType::kFLOAT) - { - auto const* data0 = pointer_const_cast(inputs[0]); - auto const* data1 = pointer_const_cast(inputs[1]); - auto const* data2 = pointer_const_cast(inputs[2]); - auto* result = pointer_cast(outputs[0]); - disentangled_kernel_wrapper(data0, data1, data2, - result, dimData0, dimData1, dimData2, dimResult, mFactor, mSpan, block_optimized, grid_optimized, stream); - } - else if (inputDesc[0].type == nvinfer1::DataType::kHALF) +int32_t DisentangledAttentionPlugin::enqueue(nvinfer1::PluginTensorDesc const* inputDesc, + nvinfer1::PluginTensorDesc const* outputDesc, void const* const* inputs, void* const* outputs, + void* /* workspace */, cudaStream_t stream) noexcept +{ + try { - auto const* data0 = pointer_const_cast<__half>(inputs[0]); - auto const* data1 = pointer_const_cast<__half>(inputs[1]); - auto const* data2 = pointer_const_cast<__half>(inputs[2]); - auto* result = pointer_cast<__half>(outputs[0]); - __half factor = __float2half(mFactor); - disentangled_kernel_wrapper<__half, kDISENTANGLED_TILESIZE_V2, kDISENTANGLED_BLOCKDIMY_V2>(data0, data1, data2, - result, dimData0, dimData1, dimData2, dimResult, factor, mSpan, block_optimized, grid_optimized, stream); + PLUGIN_VALIDATE(inputDesc && outputDesc && inputs && outputs); + switch (inputDesc[0].type) + { + case nvinfer1::DataType::kFLOAT: + enqueueType(inputDesc, outputDesc, inputs, outputs, stream, mFactor); + break; + case nvinfer1::DataType::kHALF: + enqueueType<__half>(inputDesc, outputDesc, inputs, outputs, stream, __float2half(mFactor)); + break; + case nvinfer1::DataType::kINT8: + enqueueType(inputDesc, outputDesc, inputs, outputs, stream, static_cast(mFactor)); + break; + default: PLUGIN_VALIDATE(false, "Unsupported Datatype"); break; + } + return cudaPeekAtLastError(); } - else if (inputDesc[0].type == nvinfer1::DataType::kINT8) + catch (std::exception const& e) { - auto const* data0 = pointer_const_cast(inputs[0]); - auto const* data1 = pointer_const_cast(inputs[1]); - auto const* data2 = pointer_const_cast(inputs[2]); - auto* result = pointer_cast(outputs[0]); - int8_t factor = int8_t(mFactor); - disentangled_kernel_wrapper(data0, data1, data2, - result, dimData0, dimData1, dimData2, dimResult, factor, mSpan, block_optimized, grid_optimized, stream); + caughtError(e); + return STATUS_FAILURE; } -#endif - - return cudaPeekAtLastError(); } size_t DisentangledAttentionPlugin::getSerializationSize() const noexcept @@ -256,7 +181,7 @@ IPluginV2DynamicExt* DisentangledAttentionPlugin::clone() const noexcept try { auto* plugin = new DisentangledAttentionPlugin(mSpan, mFactor); - plugin->setPluginNamespace(mPluginNamespace); + plugin->setPluginNamespace(mNamespace.c_str()); return plugin; } catch (std::exception const& e) @@ -269,41 +194,57 @@ IPluginV2DynamicExt* DisentangledAttentionPlugin::clone() const noexcept void DisentangledAttentionPlugin::configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int32_t nbInputs, nvinfer1::DynamicPluginTensorDesc const* out, int32_t nbOutputs) noexcept { - - // inputs - PLUGIN_ASSERT(nbInputs == 3); // 3 inputs - - // check for valid input dimensions - PLUGIN_ASSERT(in[0].desc.dims.nbDims == 3); - PLUGIN_ASSERT(in[1].desc.dims.nbDims == 3); - PLUGIN_ASSERT(in[2].desc.dims.nbDims == 3); - - // check BN (batch_size * num_heads) dimension consistency - PLUGIN_ASSERT(in[0].desc.dims.d[0] == in[1].desc.dims.d[0]); - PLUGIN_ASSERT(in[0].desc.dims.d[0] == in[2].desc.dims.d[0]); - - // check S (sequence_length) dimension consistency - PLUGIN_ASSERT(in[0].desc.dims.d[1] == in[1].desc.dims.d[1]); - PLUGIN_ASSERT(in[0].desc.dims.d[1] == in[2].desc.dims.d[1]); - PLUGIN_ASSERT(in[0].desc.dims.d[1] == in[0].desc.dims.d[2]); - - // check K (2 * span) dimension consistency for in[1] and in[2] - PLUGIN_ASSERT(in[1].desc.dims.d[2] == 2 * mSpan); - PLUGIN_ASSERT(in[2].desc.dims.d[2] == 2 * mSpan); - - // Outputs (same dimension as in[0]) - PLUGIN_ASSERT(nbOutputs == 1); - PLUGIN_ASSERT(out[0].desc.dims.nbDims == 3); - PLUGIN_ASSERT(in[0].desc.dims.d[0] == out[0].desc.dims.d[0]); - PLUGIN_ASSERT(in[0].desc.dims.d[1] == out[0].desc.dims.d[1]); - PLUGIN_ASSERT(in[0].desc.dims.d[2] == out[0].desc.dims.d[2]); + try + { + // inputs + PLUGIN_VALIDATE(nbInputs == 3); // 3 inputs + + // check for valid input dimensions + PLUGIN_VALIDATE(in[0].desc.dims.nbDims == 3); + PLUGIN_VALIDATE(in[1].desc.dims.nbDims == 3); + PLUGIN_VALIDATE(in[2].desc.dims.nbDims == 3); + + // check BN (batch_size * num_heads) dimension consistency + PLUGIN_VALIDATE(in[0].desc.dims.d[0] == in[1].desc.dims.d[0]); + PLUGIN_VALIDATE(in[0].desc.dims.d[0] == in[2].desc.dims.d[0]); + + // check S (sequence_length) dimension consistency + PLUGIN_VALIDATE(in[0].desc.dims.d[1] == in[1].desc.dims.d[1]); + PLUGIN_VALIDATE(in[0].desc.dims.d[1] == in[2].desc.dims.d[1]); + PLUGIN_VALIDATE(in[0].desc.dims.d[1] == in[0].desc.dims.d[2]); + + // check K (2 * span) dimension consistency for in[1] and in[2] + PLUGIN_VALIDATE(in[1].desc.dims.d[2] == 2 * mSpan); + PLUGIN_VALIDATE(in[2].desc.dims.d[2] == 2 * mSpan); + + // Outputs (same dimension as in[0]) + PLUGIN_VALIDATE(nbOutputs == 1); + PLUGIN_VALIDATE(out[0].desc.dims.nbDims == 3); + PLUGIN_VALIDATE(in[0].desc.dims.d[0] == out[0].desc.dims.d[0]); + PLUGIN_VALIDATE(in[0].desc.dims.d[1] == out[0].desc.dims.d[1]); + PLUGIN_VALIDATE(in[0].desc.dims.d[2] == out[0].desc.dims.d[2]); + } + catch (std::exception const& e) + { + caughtError(e); + } } nvinfer1::DataType DisentangledAttentionPlugin::getOutputDataType( int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { - PLUGIN_ASSERT(inputTypes && nbInputs > 0 && index < 1); - return inputTypes[0]; // version 1, same as data1; version 2, same as data0 + try + { + PLUGIN_VALIDATE(inputTypes != nullptr); + PLUGIN_VALIDATE(nbInputs > 0); + PLUGIN_VALIDATE(index == 0); + return inputTypes[0]; // version 1, same as data1; version 2, same as data0 + } + catch (std::exception const& e) + { + caughtError(e); + } + return nvinfer1::DataType{}; } size_t DisentangledAttentionPlugin::getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int32_t nbInputs, @@ -314,12 +255,20 @@ size_t DisentangledAttentionPlugin::getWorkspaceSize(nvinfer1::PluginTensorDesc void DisentangledAttentionPlugin::setPluginNamespace(char const* libNamespace) noexcept { - mPluginNamespace = libNamespace; + try + { + PLUGIN_VALIDATE(libNamespace != nullptr); + mNamespace = libNamespace; + } + catch (std::exception const& e) + { + caughtError(e); + } } char const* DisentangledAttentionPlugin::getPluginNamespace() const noexcept { - return mPluginNamespace; + return mNamespace.c_str(); } DisentangledAttentionPluginCreator::DisentangledAttentionPluginCreator() @@ -336,12 +285,12 @@ DisentangledAttentionPluginCreator::DisentangledAttentionPluginCreator() char const* DisentangledAttentionPluginCreator::getPluginName() const noexcept { - return DEBERTA_NAME; + return kDEBERTA_PLUGIN_NAME; } char const* DisentangledAttentionPluginCreator::getPluginVersion() const noexcept { - return DEBERTA_VERSION; + return kDEBERTA_PLUGIN_VERSION; } PluginFieldCollection const* DisentangledAttentionPluginCreator::getFieldNames() noexcept @@ -356,32 +305,42 @@ char const* DisentangledAttentionPluginCreator::getPluginNamespace() const noexc void DisentangledAttentionPluginCreator::setPluginNamespace(char const* libNamespace) noexcept { - mNamespace = libNamespace; + try + { + PLUGIN_VALIDATE(libNamespace != nullptr); + mNamespace = libNamespace; + } + catch (std::exception const& e) + { + caughtError(e); + } } IPluginV2DynamicExt* DisentangledAttentionPluginCreator::createPlugin( - char const* name, PluginFieldCollection const* fc) noexcept + char const* /*name*/, PluginFieldCollection const* fc) noexcept { try { + PLUGIN_VALIDATE(fc != nullptr); + // Set default invalid values (for assert in case when attributes are missing) int32_t span = 0; - float factor = 0.0F; + float factor = 0.F; for (int32_t i = 0; i < fc->nbFields; i++) { - std::string field_name(fc->fields[i].name); - if (field_name.compare("span") == 0) + std::string fieldName = fc->fields[i].name; + if (fieldName.compare("span") == 0) { span = *static_cast(fc->fields[i].data); } - if (field_name.compare("factor") == 0) + if (fieldName.compare("factor") == 0) { factor = *static_cast(fc->fields[i].data); } } - PLUGIN_ASSERT(span >= 0); - PLUGIN_ASSERT(factor > 0.0F && factor < 1.0F); // factor is 1/sqrt(3d), therefore must less than 1 + PLUGIN_VALIDATE(span >= 0); + PLUGIN_VALIDATE(factor > 0.F && factor < 1.F); // factor is 1/sqrt(3d), therefore must less than 1 DisentangledAttentionPlugin* plugin = new DisentangledAttentionPlugin(span, factor); plugin->setPluginNamespace(mNamespace.c_str()); @@ -396,7 +355,7 @@ IPluginV2DynamicExt* DisentangledAttentionPluginCreator::createPlugin( } IPluginV2DynamicExt* DisentangledAttentionPluginCreator::deserializePlugin( - char const* name, void const* serialData, size_t serialLength) noexcept + char const* /*name*/, void const* serialData, size_t serialLength) noexcept { try { diff --git a/plugin/disentangledAttentionPlugin/disentangledAttentionPlugin.h b/plugin/disentangledAttentionPlugin/disentangledAttentionPlugin.h index 789943b2..72223755 100644 --- a/plugin/disentangledAttentionPlugin/disentangledAttentionPlugin.h +++ b/plugin/disentangledAttentionPlugin/disentangledAttentionPlugin.h @@ -36,13 +36,16 @@ namespace plugin // using namespace nvinfer1; -#define kDISENTANGLED_VERSION 2 // Version 1: regular relative position index // Version 2: log bucket relative position index -constexpr int32_t kDISENTANGLED_TILESIZE_V1 = 32; -constexpr int32_t kDISENTANGLED_BLOCKDIMY_V1 = 8; -constexpr int32_t kDISENTANGLED_TILESIZE_V2 = 64; -constexpr int32_t kDISENTANGLED_BLOCKDIMY_V2 = 4; +#define kDISENTANGLED_VERSION 2 +#if kDISENTANGLED_VERSION == 1 +constexpr int32_t kDISENTANGLED_TILESIZE = 32; +constexpr int32_t kDISENTANGLED_BLOCKDIMY = 8; +#elif kDISENTANGLED_VERSION == 2 +constexpr int32_t kDISENTANGLED_TILESIZE = 64; +constexpr int32_t kDISENTANGLED_BLOCKDIMY = 4; +#endif template void disentangled_kernel_wrapper(TDataType const* data0, TDataType const* data1, TDataType const* data2, @@ -58,14 +61,6 @@ class DisentangledAttentionPlugin final : public nvinfer1::IPluginV2DynamicExt DisentangledAttentionPlugin(void const* serialData, size_t serialLength); - ~DisentangledAttentionPlugin() override; - - template - TDataType const* pointer_const_cast(void const* const p); - - template - TDataType* pointer_cast(void* p); - int32_t getNbOutputs() const noexcept override; // DynamicExt plugins returns DimsExprs class instead of Dims @@ -79,9 +74,9 @@ class DisentangledAttentionPlugin final : public nvinfer1::IPluginV2DynamicExt size_t getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int32_t nbInputs, nvinfer1::PluginTensorDesc const* outputs, int32_t nbOutputs) const noexcept override; + // This is where the plugin work is done. int32_t enqueue(nvinfer1::PluginTensorDesc const* inputDesc, nvinfer1::PluginTensorDesc const* outputDesc, - void const* const* inputs, void* const* outputs, void* workspace, - cudaStream_t stream) noexcept override; // this is where the plugin work is done + void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; size_t getSerializationSize() const noexcept override; @@ -101,11 +96,6 @@ class DisentangledAttentionPlugin final : public nvinfer1::IPluginV2DynamicExt nvinfer1::DataType getOutputDataType( int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; - void attachToContext( - cudnnContext* cudnn, cublasContext* cublas, nvinfer1::IGpuAllocator* allocator) noexcept override; - - void detachFromContext() noexcept override; - void setPluginNamespace(char const* pluginNamespace) noexcept override; char const* getPluginNamespace() const noexcept override; @@ -114,7 +104,11 @@ class DisentangledAttentionPlugin final : public nvinfer1::IPluginV2DynamicExt nvinfer1::DynamicPluginTensorDesc const* out, int32_t nbOutputs) noexcept override; private: - char const* mPluginNamespace; + // Helper method for enqueue() + template + void enqueueType(nvinfer1::PluginTensorDesc const* inputDesc, nvinfer1::PluginTensorDesc const* outputDesc, + void const* const* inputs, void* const* outputs, cudaStream_t stream, TDataType factor); + std::string mNamespace; // attributes diff --git a/plugin/disentangledAttentionPlugin/disentangledKernel.cu b/plugin/disentangledAttentionPlugin/disentangledKernel.cu index c7024ad6..fb26db33 100644 --- a/plugin/disentangledAttentionPlugin/disentangledKernel.cu +++ b/plugin/disentangledAttentionPlugin/disentangledKernel.cu @@ -263,25 +263,16 @@ void disentangled_kernel_wrapper(TDataType const* data0, TDataType const* data1, data0, data1, data2, result, dimData0, dimData1, dimData2, dimResult, factor, span); } -template void disentangled_kernel_wrapper( +template void disentangled_kernel_wrapper( float const*, float const*, float const*, float*, dim3, dim3, dim3, dim3, float, int32_t, dim3, dim3, cudaStream_t); -template void disentangled_kernel_wrapper<__half, kDISENTANGLED_TILESIZE_V1, kDISENTANGLED_BLOCKDIMY_V1>(__half const*, +template void disentangled_kernel_wrapper<__half, kDISENTANGLED_TILESIZE, kDISENTANGLED_BLOCKDIMY>(__half const*, __half const*, __half const*, __half*, dim3, dim3, dim3, dim3, __half, int32_t, dim3, dim3, cudaStream_t); -template void disentangled_kernel_wrapper(int8_t const*, - int8_t const*, int8_t const*, int8_t*, dim3, dim3, dim3, dim3, int8_t, int32_t, dim3, dim3, cudaStream_t); - -template void disentangled_kernel_wrapper( - float const*, float const*, float const*, float*, dim3, dim3, dim3, dim3, float, int32_t, dim3, dim3, cudaStream_t); - -template void disentangled_kernel_wrapper<__half, kDISENTANGLED_TILESIZE_V2, kDISENTANGLED_BLOCKDIMY_V2>(__half const*, - __half const*, __half const*, __half*, dim3, dim3, dim3, dim3, __half, int32_t, dim3, dim3, cudaStream_t); - -template void disentangled_kernel_wrapper(int8_t const*, +template void disentangled_kernel_wrapper(int8_t const*, int8_t const*, int8_t const*, int8_t*, dim3, dim3, dim3, dim3, int8_t, int32_t, dim3, dim3, cudaStream_t); #undef IND -} /* plugin */ +} // namespace plugin } // namespace nvinfer1 diff --git a/plugin/efficientNMSPlugin/efficientNMSInference.h b/plugin/efficientNMSPlugin/efficientNMSInference.h index 3ba002ef..d9ec3192 100644 --- a/plugin/efficientNMSPlugin/efficientNMSInference.h +++ b/plugin/efficientNMSPlugin/efficientNMSInference.h @@ -22,7 +22,8 @@ #include "efficientNMSParameters.h" -size_t EfficientNMSWorkspaceSize(int batchSize, int numScoreElements, int numClasses, nvinfer1::DataType datatype); +size_t EfficientNMSWorkspaceSize( + int32_t batchSize, int32_t numScoreElements, int32_t numClasses, nvinfer1::DataType datatype); pluginStatus_t EfficientNMSInference(nvinfer1::plugin::EfficientNMSParameters param, void const* boxesInput, void const* scoresInput, void const* anchorsInput, void* numDetectionsOutput, void* nmsBoxesOutput, diff --git a/plugin/efficientNMSPlugin/efficientNMSParameters.h b/plugin/efficientNMSPlugin/efficientNMSParameters.h index 30f2528a..89829089 100644 --- a/plugin/efficientNMSPlugin/efficientNMSParameters.h +++ b/plugin/efficientNMSPlugin/efficientNMSParameters.h @@ -30,27 +30,27 @@ struct EfficientNMSParameters // Related to NMS Options float iouThreshold = 0.5F; float scoreThreshold = 0.5F; - int numOutputBoxes = 100; - int numOutputBoxesPerClass = -1; + int32_t numOutputBoxes = 100; + int32_t numOutputBoxesPerClass = -1; bool padOutputBoxesPerClass = false; - int backgroundClass = -1; + int32_t backgroundClass = -1; bool scoreSigmoid = false; bool clipBoxes = false; - int boxCoding = 0; + int32_t boxCoding = 0; bool classAgnostic = false; // Related to NMS Internals - int numSelectedBoxes = 4096; - int scoreBits = -1; + int32_t numSelectedBoxes = 4096; + int32_t scoreBits = -1; bool outputONNXIndices = false; // Related to Tensor Configuration // (These are set by the various plugin configuration methods, no need to define them during plugin creation.) - int batchSize = -1; - int numClasses = 1; - int numBoxElements = -1; - int numScoreElements = -1; - int numAnchors = -1; + int32_t batchSize = -1; + int32_t numClasses = 1; + int32_t numBoxElements = -1; + int32_t numScoreElements = -1; + int32_t numAnchors = -1; bool shareLocation = true; bool shareAnchors = true; bool boxDecoder = false; diff --git a/plugin/efficientNMSPlugin/efficientNMSPlugin.cpp b/plugin/efficientNMSPlugin/efficientNMSPlugin.cpp index b6e9f65b..2f5d428b 100644 --- a/plugin/efficientNMSPlugin/efficientNMSPlugin.cpp +++ b/plugin/efficientNMSPlugin/efficientNMSPlugin.cpp @@ -59,7 +59,7 @@ char const* EfficientNMSPlugin::getPluginVersion() const noexcept return kEFFICIENT_NMS_PLUGIN_VERSION; } -int EfficientNMSPlugin::getNbOutputs() const noexcept +int32_t EfficientNMSPlugin::getNbOutputs() const noexcept { if (mParam.outputONNXIndices) { @@ -71,7 +71,7 @@ int EfficientNMSPlugin::getNbOutputs() const noexcept return 4; } -int EfficientNMSPlugin::initialize() noexcept +int32_t EfficientNMSPlugin::initialize() noexcept { if (!initialized) { @@ -131,7 +131,7 @@ char const* EfficientNMSPlugin::getPluginNamespace() const noexcept } nvinfer1::DataType EfficientNMSPlugin::getOutputDataType( - int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { if (mParam.outputONNXIndices) { @@ -164,7 +164,7 @@ IPluginV2DynamicExt* EfficientNMSPlugin::clone() const noexcept } DimsExprs EfficientNMSPlugin::getOutputDimensions( - int outputIndex, DimsExprs const* inputs, int nbInputs, IExprBuilder& exprBuilder) noexcept + int32_t outputIndex, DimsExprs const* inputs, int32_t nbInputs, IExprBuilder& exprBuilder) noexcept { try { @@ -234,7 +234,7 @@ DimsExprs EfficientNMSPlugin::getOutputDimensions( } bool EfficientNMSPlugin::supportsFormatCombination( - int pos, PluginTensorDesc const* inOut, int nbInputs, int nbOutputs) noexcept + int32_t pos, PluginTensorDesc const* inOut, int32_t nbInputs, int32_t nbOutputs) noexcept { if (inOut[pos].format != PluginFormat::kLINEAR) { @@ -246,7 +246,7 @@ bool EfficientNMSPlugin::supportsFormatCombination( PLUGIN_ASSERT(nbInputs == 2); PLUGIN_ASSERT(nbOutputs == 1); - // detection_indices output: int + // detection_indices output: int32_t if (pos == 2) { return inOut[pos].type == DataType::kINT32; @@ -268,8 +268,8 @@ bool EfficientNMSPlugin::supportsFormatCombination( PLUGIN_ASSERT(0 <= pos && pos <= 6); } - // num_detections and detection_classes output: int - int const posOut = pos - nbInputs; + // num_detections and detection_classes output: int32_t + int32_t const posOut = pos - nbInputs; if (posOut == 0 || posOut == 3) { return inOut[pos].type == DataType::kINT32 && inOut[pos].format == PluginFormat::kLINEAR; @@ -281,7 +281,7 @@ bool EfficientNMSPlugin::supportsFormatCombination( } void EfficientNMSPlugin::configurePlugin( - DynamicPluginTensorDesc const* in, int nbInputs, DynamicPluginTensorDesc const* out, int nbOutputs) noexcept + DynamicPluginTensorDesc const* in, int32_t nbInputs, DynamicPluginTensorDesc const* out, int32_t nbOutputs) noexcept { try { @@ -308,7 +308,7 @@ void EfficientNMSPlugin::configurePlugin( mParam.numScoreElements = in[1].desc.dims.d[1] * in[1].desc.dims.d[2]; mParam.numClasses = in[1].desc.dims.d[2]; - // When pad per class is set, the total ouput boxes size may need to be reduced. + // When pad per class is set, the total output boxes size may need to be reduced. // This operation is also done in getOutputDimension(), but for dynamic shapes, the // numOutputBoxes param can't be set until the number of classes is fully known here. if (mParam.padOutputBoxesPerClass && mParam.numOutputBoxesPerClass > 0) @@ -359,15 +359,15 @@ void EfficientNMSPlugin::configurePlugin( } size_t EfficientNMSPlugin::getWorkspaceSize( - PluginTensorDesc const* inputs, int nbInputs, PluginTensorDesc const* outputs, int nbOutputs) const noexcept + PluginTensorDesc const* inputs, int32_t nbInputs, PluginTensorDesc const* outputs, int32_t nbOutputs) const noexcept { - int batchSize = inputs[1].dims.d[0]; - int numScoreElements = inputs[1].dims.d[1] * inputs[1].dims.d[2]; - int numClasses = inputs[1].dims.d[2]; + int32_t batchSize = inputs[1].dims.d[0]; + int32_t numScoreElements = inputs[1].dims.d[1] * inputs[1].dims.d[2]; + int32_t numClasses = inputs[1].dims.d[2]; return EfficientNMSWorkspaceSize(batchSize, numScoreElements, numClasses, mParam.datatype); } -int EfficientNMSPlugin::enqueue(PluginTensorDesc const* inputDesc, PluginTensorDesc const* outputDesc, +int32_t EfficientNMSPlugin::enqueue(PluginTensorDesc const* inputDesc, PluginTensorDesc const* outputDesc, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept { try @@ -562,7 +562,7 @@ IPluginV2DynamicExt* EfficientNMSONNXPluginCreator::createPlugin( try { PluginField const* fields = fc->fields; - for (int i = 0; i < fc->nbFields; ++i) + for (int32_t i = 0; i < fc->nbFields; ++i) { char const* attrName = fields[i].name; if (!strcmp(attrName, "score_threshold")) @@ -578,12 +578,12 @@ IPluginV2DynamicExt* EfficientNMSONNXPluginCreator::createPlugin( if (!strcmp(attrName, "max_output_boxes_per_class")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - mParam.numOutputBoxesPerClass = *(static_cast(fields[i].data)); + mParam.numOutputBoxesPerClass = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "center_point_box")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - mParam.boxCoding = *(static_cast(fields[i].data)); + mParam.boxCoding = *(static_cast(fields[i].data)); } } diff --git a/plugin/efficientNMSPlugin/efficientNMSPlugin.h b/plugin/efficientNMSPlugin/efficientNMSPlugin.h index c40c9635..afceec01 100644 --- a/plugin/efficientNMSPlugin/efficientNMSPlugin.h +++ b/plugin/efficientNMSPlugin/efficientNMSPlugin.h @@ -20,7 +20,7 @@ #include #include "common/plugin.h" -#include "efficientNMSParameters.h" +#include "efficientNMSPlugin/efficientNMSParameters.h" namespace nvinfer1 { @@ -37,8 +37,8 @@ class EfficientNMSPlugin : public IPluginV2DynamicExt // IPluginV2 methods char const* getPluginType() const noexcept override; char const* getPluginVersion() const noexcept override; - int getNbOutputs() const noexcept override; - int initialize() noexcept override; + int32_t getNbOutputs() const noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; size_t getSerializationSize() const noexcept override; void serialize(void* buffer) const noexcept override; @@ -48,19 +48,19 @@ class EfficientNMSPlugin : public IPluginV2DynamicExt // IPluginV2Ext methods nvinfer1::DataType getOutputDataType( - int index, nvinfer1::DataType const* inputType, int nbInputs) const noexcept override; + int32_t index, nvinfer1::DataType const* inputType, int32_t nbInputs) const noexcept override; // IPluginV2DynamicExt methods IPluginV2DynamicExt* clone() const noexcept override; DimsExprs getOutputDimensions( - int outputIndex, DimsExprs const* inputs, int nbInputs, IExprBuilder& exprBuilder) noexcept override; + int32_t outputIndex, DimsExprs const* inputs, int32_t nbInputs, IExprBuilder& exprBuilder) noexcept override; bool supportsFormatCombination( - int pos, PluginTensorDesc const* inOut, int nbInputs, int nbOutputs) noexcept override; - void configurePlugin(DynamicPluginTensorDesc const* in, int nbInputs, DynamicPluginTensorDesc const* out, - int nbOutputs) noexcept override; - size_t getWorkspaceSize(PluginTensorDesc const* inputs, int nbInputs, PluginTensorDesc const* outputs, - int nbOutputs) const noexcept override; - int enqueue(PluginTensorDesc const* inputDesc, PluginTensorDesc const* outputDesc, void const* const* inputs, + int32_t pos, PluginTensorDesc const* inOut, int32_t nbInputs, int32_t nbOutputs) noexcept override; + void configurePlugin(DynamicPluginTensorDesc const* in, int32_t nbInputs, DynamicPluginTensorDesc const* out, + int32_t nbOutputs) noexcept override; + size_t getWorkspaceSize(PluginTensorDesc const* inputs, int32_t nbInputs, PluginTensorDesc const* outputs, + int32_t nbOutputs) const noexcept override; + int32_t enqueue(PluginTensorDesc const* inputDesc, PluginTensorDesc const* outputDesc, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; protected: diff --git a/plugin/efficientNMSPlugin/tftrt/efficientNMSExplicitTFTRTPlugin.cpp b/plugin/efficientNMSPlugin/tftrt/efficientNMSExplicitTFTRTPlugin.cpp index 9183e5a2..f5c86365 100644 --- a/plugin/efficientNMSPlugin/tftrt/efficientNMSExplicitTFTRTPlugin.cpp +++ b/plugin/efficientNMSPlugin/tftrt/efficientNMSExplicitTFTRTPlugin.cpp @@ -102,18 +102,18 @@ IPluginV2DynamicExt* EfficientNMSExplicitTFTRTPluginCreator::createPlugin( try { const PluginField* fields = fc->fields; - for (int i = 0; i < fc->nbFields; ++i) + for (int32_t i = 0; i < fc->nbFields; ++i) { const char* attrName = fields[i].name; if (!strcmp(attrName, "max_output_size_per_class")) { PLUGIN_ASSERT(fields[i].type == PluginFieldType::kINT32); - mParam.numOutputBoxesPerClass = *(static_cast(fields[i].data)); + mParam.numOutputBoxesPerClass = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "max_total_size")) { PLUGIN_ASSERT(fields[i].type == PluginFieldType::kINT32); - mParam.numOutputBoxes = *(static_cast(fields[i].data)); + mParam.numOutputBoxes = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "iou_threshold")) { @@ -128,12 +128,12 @@ IPluginV2DynamicExt* EfficientNMSExplicitTFTRTPluginCreator::createPlugin( if (!strcmp(attrName, "pad_per_class")) { PLUGIN_ASSERT(fields[i].type == PluginFieldType::kINT32); - mParam.padOutputBoxesPerClass = *(static_cast(fields[i].data)); + mParam.padOutputBoxesPerClass = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "clip_boxes")) { PLUGIN_ASSERT(fields[i].type == PluginFieldType::kINT32); - mParam.clipBoxes = *(static_cast(fields[i].data)); + mParam.clipBoxes = *(static_cast(fields[i].data)); } } diff --git a/plugin/efficientNMSPlugin/tftrt/efficientNMSImplicitTFTRTPlugin.cpp b/plugin/efficientNMSPlugin/tftrt/efficientNMSImplicitTFTRTPlugin.cpp index 3c89b044..25c8e0ef 100644 --- a/plugin/efficientNMSPlugin/tftrt/efficientNMSImplicitTFTRTPlugin.cpp +++ b/plugin/efficientNMSPlugin/tftrt/efficientNMSImplicitTFTRTPlugin.cpp @@ -60,12 +60,12 @@ const char* EfficientNMSImplicitTFTRTPlugin::getPluginVersion() const noexcept return EFFICIENT_NMS_IMPLICIT_TFTRT_PLUGIN_VERSION; } -int EfficientNMSImplicitTFTRTPlugin::getNbOutputs() const noexcept +int32_t EfficientNMSImplicitTFTRTPlugin::getNbOutputs() const noexcept { return 4; } -int EfficientNMSImplicitTFTRTPlugin::initialize() noexcept +int32_t EfficientNMSImplicitTFTRTPlugin::initialize() noexcept { return STATUS_SUCCESS; } @@ -106,7 +106,8 @@ const char* EfficientNMSImplicitTFTRTPlugin::getPluginNamespace() const noexcept return mNamespace.c_str(); } -Dims EfficientNMSImplicitTFTRTPlugin::getOutputDimensions(int outputIndex, const Dims* inputs, int nbInputs) noexcept +Dims EfficientNMSImplicitTFTRTPlugin::getOutputDimensions( + int32_t outputIndex, const Dims* inputs, int32_t nbInputs) noexcept { try { @@ -117,7 +118,7 @@ Dims EfficientNMSImplicitTFTRTPlugin::getOutputDimensions(int outputIndex, const PLUGIN_ASSERT(inputs[1].nbDims == 2); if (mParam.padOutputBoxesPerClass && mParam.numOutputBoxesPerClass > 0) { - const int numClasses = inputs[1].d[1]; + const int32_t numClasses = inputs[1].d[1]; if (mParam.numOutputBoxesPerClass * numClasses < mParam.numOutputBoxes) { mParam.numOutputBoxes = mParam.numOutputBoxesPerClass * numClasses; @@ -157,12 +158,12 @@ Dims EfficientNMSImplicitTFTRTPlugin::getOutputDimensions(int outputIndex, const return Dims{}; } -size_t EfficientNMSImplicitTFTRTPlugin::getWorkspaceSize(int maxBatchSize) const noexcept +size_t EfficientNMSImplicitTFTRTPlugin::getWorkspaceSize(int32_t maxBatchSize) const noexcept { return EfficientNMSWorkspaceSize(maxBatchSize, mParam.numScoreElements, mParam.numClasses, mParam.datatype); } -int EfficientNMSImplicitTFTRTPlugin::enqueue(int batchSize, void const* const* inputs, +int32_t EfficientNMSImplicitTFTRTPlugin::enqueue(int32_t batchSize, void const* const* inputs, EfficientNMSImplicitTFTRTOutputsDataType outputs, void* workspace, cudaStream_t stream) noexcept { try @@ -188,13 +189,13 @@ int EfficientNMSImplicitTFTRTPlugin::enqueue(int batchSize, void const* const* i return -1; } -bool EfficientNMSImplicitTFTRTPlugin::canBroadcastInputAcrossBatch(int inputIndex) const noexcept +bool EfficientNMSImplicitTFTRTPlugin::canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept { return false; } DataType EfficientNMSImplicitTFTRTPlugin::getOutputDataType( - int index, const DataType* inputTypes, int nbInputs) const noexcept + int32_t index, const DataType* inputTypes, int32_t nbInputs) const noexcept { // num_detections and detection_classes use integer outputs if (index == 0 || index == 3) @@ -221,13 +222,13 @@ IPluginV2IOExt* EfficientNMSImplicitTFTRTPlugin::clone() const noexcept } bool EfficientNMSImplicitTFTRTPlugin::isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept { return false; } bool EfficientNMSImplicitTFTRTPlugin::supportsFormatCombination( - int pos, const PluginTensorDesc* inOut, int nbInputs, int nbOutputs) const noexcept + int32_t pos, const PluginTensorDesc* inOut, int32_t nbInputs, int32_t nbOutputs) const noexcept { if (inOut[pos].format != PluginFormat::kLINEAR) { @@ -241,8 +242,8 @@ bool EfficientNMSImplicitTFTRTPlugin::supportsFormatCombination( PLUGIN_ASSERT(0 <= pos && pos <= 5); } - // num_detections and detection_classes output: int - const int posOut = pos - nbInputs; + // num_detections and detection_classes output: int32_t + const int32_t posOut = pos - nbInputs; if (posOut == 0 || posOut == 3) { return inOut[pos].type == DataType::kINT32 && inOut[pos].format == PluginFormat::kLINEAR; @@ -254,7 +255,7 @@ bool EfficientNMSImplicitTFTRTPlugin::supportsFormatCombination( } void EfficientNMSImplicitTFTRTPlugin::configurePlugin( - const PluginTensorDesc* in, int nbInputs, const PluginTensorDesc* out, int nbOutputs) noexcept + const PluginTensorDesc* in, int32_t nbInputs, const PluginTensorDesc* out, int32_t nbOutputs) noexcept { try { @@ -335,18 +336,18 @@ IPluginV2IOExt* EfficientNMSImplicitTFTRTPluginCreator::createPlugin( try { const PluginField* fields = fc->fields; - for (int i = 0; i < fc->nbFields; ++i) + for (int32_t i = 0; i < fc->nbFields; ++i) { const char* attrName = fields[i].name; if (!strcmp(attrName, "max_output_size_per_class")) { PLUGIN_ASSERT(fields[i].type == PluginFieldType::kINT32); - mParam.numOutputBoxesPerClass = *(static_cast(fields[i].data)); + mParam.numOutputBoxesPerClass = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "max_total_size")) { PLUGIN_ASSERT(fields[i].type == PluginFieldType::kINT32); - mParam.numOutputBoxes = *(static_cast(fields[i].data)); + mParam.numOutputBoxes = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "iou_threshold")) { @@ -361,12 +362,12 @@ IPluginV2IOExt* EfficientNMSImplicitTFTRTPluginCreator::createPlugin( if (!strcmp(attrName, "pad_per_class")) { PLUGIN_ASSERT(fields[i].type == PluginFieldType::kINT32); - mParam.padOutputBoxesPerClass = *(static_cast(fields[i].data)); + mParam.padOutputBoxesPerClass = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "clip_boxes")) { PLUGIN_ASSERT(fields[i].type == PluginFieldType::kINT32); - mParam.clipBoxes = *(static_cast(fields[i].data)); + mParam.clipBoxes = *(static_cast(fields[i].data)); } } diff --git a/plugin/efficientNMSPlugin/tftrt/efficientNMSImplicitTFTRTPlugin.h b/plugin/efficientNMSPlugin/tftrt/efficientNMSImplicitTFTRTPlugin.h index ca797c26..51b09148 100644 --- a/plugin/efficientNMSPlugin/tftrt/efficientNMSImplicitTFTRTPlugin.h +++ b/plugin/efficientNMSPlugin/tftrt/efficientNMSImplicitTFTRTPlugin.h @@ -47,8 +47,8 @@ class EfficientNMSImplicitTFTRTPlugin : public nvinfer1::IPluginV2IOExt // IPluginV2 methods const char* getPluginType() const noexcept override; const char* getPluginVersion() const noexcept override; - int getNbOutputs() const noexcept override; - int initialize() noexcept override; + int32_t getNbOutputs() const noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; size_t getSerializationSize() const noexcept override; void serialize(void* buffer) const noexcept override; @@ -56,24 +56,25 @@ class EfficientNMSImplicitTFTRTPlugin : public nvinfer1::IPluginV2IOExt void setPluginNamespace(const char* libNamespace) noexcept override; const char* getPluginNamespace() const noexcept override; - nvinfer1::Dims getOutputDimensions(int outputIndex, const nvinfer1::Dims* inputs, int nbInputs) noexcept override; - size_t getWorkspaceSize(int maxBatchSize) const noexcept override; - int enqueue(int batchSize, void const* const* inputs, EfficientNMSImplicitTFTRTOutputsDataType outputs, + nvinfer1::Dims getOutputDimensions( + int32_t outputIndex, const nvinfer1::Dims* inputs, int32_t nbInputs) noexcept override; + size_t getWorkspaceSize(int32_t maxBatchSize) const noexcept override; + int32_t enqueue(int32_t batchSize, void const* const* inputs, EfficientNMSImplicitTFTRTOutputsDataType outputs, void* workspace, cudaStream_t stream) noexcept override; // IPluginV2Ext methods - bool canBroadcastInputAcrossBatch(int inputIndex) const noexcept override; + bool canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept override; nvinfer1::DataType getOutputDataType( - int index, const nvinfer1::DataType* inputType, int nbInputs) const noexcept override; + int32_t index, const nvinfer1::DataType* inputType, int32_t nbInputs) const noexcept override; nvinfer1::IPluginV2IOExt* clone() const noexcept override; bool isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept override; + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept override; // IPluginV2IOExt methods - bool supportsFormatCombination( - int pos, const nvinfer1::PluginTensorDesc* inOut, int nbInputs, int nbOutputs) const noexcept override; - void configurePlugin(const nvinfer1::PluginTensorDesc* in, int nbInputs, const nvinfer1::PluginTensorDesc* out, - int nbOutputs) noexcept override; + bool supportsFormatCombination(int32_t pos, const nvinfer1::PluginTensorDesc* inOut, int32_t nbInputs, + int32_t nbOutputs) const noexcept override; + void configurePlugin(const nvinfer1::PluginTensorDesc* in, int32_t nbInputs, const nvinfer1::PluginTensorDesc* out, + int32_t nbOutputs) noexcept override; protected: void deserialize(int8_t const* data, size_t length); diff --git a/plugin/embLayerNormPlugin/CustomEmbLayerNormPluginDynamic_PluginConfig.yaml b/plugin/embLayerNormPlugin/CustomEmbLayerNormPluginDynamic_PluginConfig.yaml index 0a3d0b7d..d5f1594a 100644 --- a/plugin/embLayerNormPlugin/CustomEmbLayerNormPluginDynamic_PluginConfig.yaml +++ b/plugin/embLayerNormPlugin/CustomEmbLayerNormPluginDynamic_PluginConfig.yaml @@ -66,13 +66,13 @@ versions: bert_embeddings_token_type_embeddings: 2 bert_embeddings_position_embeddings: 2 attribute_dim_range: - output_fp16: + output_fp16: - min: "=1" - max: "=1" - full_mask: + full_mask: - min: "=1" - max: "=1" - mha_type_id: + mha_type_id: - min: "=1" - max: "=1" bert_embeddings_layernorm_beta: @@ -91,29 +91,29 @@ versions: - min: "=1, =1" - max: "=pinf, =pinf" attribute_options: - output_fp16: + output_fp16: - 0 - 1 - full_mask: + full_mask: - 0 - 1 - mha_type_id: + mha_type_id: - 0 - 1 - 2 - bert_embeddings_layernorm_beta: + bert_embeddings_layernorm_beta: min: "=ninf" max: "=pinf" - bert_embeddings_layernorm_gamma: + bert_embeddings_layernorm_gamma: min: "=ninf" max: "=pinf" - bert_embeddings_word_embeddings: + bert_embeddings_word_embeddings: min: "=ninf" max: "=pinf" - bert_embeddings_token_type_embeddings: + bert_embeddings_token_type_embeddings: min: "=ninf" max: "=pinf" - bert_embeddings_position_embeddings: + bert_embeddings_position_embeddings: min: "=ninf" max: "=pinf" attributes_required: @@ -122,4 +122,33 @@ versions: - bert_embeddings_word_embeddings - bert_embeddings_token_type_embeddings - bert_embeddings_position_embeddings + golden_reference_script: "plugin/embLayerNormPlugin/CustomEmbLayerNormPluginDynamic_PluginReference.py" + abs_tol: 1e-5 + rel_tol: 1e-5 + configs: + config1: + input_types: + token_id: int32 + segment_id: int32 + input_mask: int32 + attribute_options: + output_fp16: + value: 0 + shape: "1" + full_mask: + value: 0 + shape: "1" + mha_type_id: + value: 0 + shape: "1" + bert_embeddings_layernorm_beta: + shape: "128" + bert_embeddings_layernorm_gamma: + shape: "128" + bert_embeddings_word_embeddings: + shape: "100, 128" + bert_embeddings_token_type_embeddings: + shape: "2, 128" + bert_embeddings_position_embeddings: + shape: "20, 128" ... diff --git a/plugin/embLayerNormPlugin/embLayerNormPlugin.cpp b/plugin/embLayerNormPlugin/embLayerNormPlugin.cpp index 94693861..ef9738d0 100644 --- a/plugin/embLayerNormPlugin/embLayerNormPlugin.cpp +++ b/plugin/embLayerNormPlugin/embLayerNormPlugin.cpp @@ -329,8 +329,8 @@ int32_t EmbLayerNormPluginDynamic::enqueue(PluginTensorDesc const* inputDesc, Pl auto const wordEmb = static_cast(mWordEmbDev.get()); auto const tokEmb = static_cast(mTokEmbDev.get()); auto const posEmb = static_cast(mPosEmbDev.get()); - status = embSkipLayerNorm(stream, static_cast(mLd), batchSize, S, inputIds, segmentIds, beta, - gamma, wordEmb, posEmb, tokEmb, mWordVocabSize, mTokVocabSize, output); + status = embSkipLayerNorm(stream, static_cast(mLd), batchSize, S, inputIds, segmentIds, + beta, gamma, wordEmb, posEmb, tokEmb, mWordVocabSize, mTokVocabSize, output); if (status != cudaSuccess) { @@ -343,7 +343,7 @@ int32_t EmbLayerNormPluginDynamic::enqueue(PluginTensorDesc const* inputDesc, Pl auto const wordEmb = static_cast(mWordEmbDev.get()); auto const tokEmb = static_cast(mTokEmbDev.get()); auto const posEmb = static_cast(mPosEmbDev.get()); - status = embSkipLayerNorm(stream, static_cast(mLd), batchSize, S, inputIds, segmentIds, beta, + status = embSkipLayerNorm(stream, static_cast(mLd), batchSize, S, inputIds, segmentIds, beta, gamma, wordEmb, posEmb, tokEmb, mWordVocabSize, mTokVocabSize, output); if (status != cudaSuccess) @@ -353,7 +353,7 @@ int32_t EmbLayerNormPluginDynamic::enqueue(PluginTensorDesc const* inputDesc, Pl } else { - gLogError << "Unsupported type error, expected [kHALF,kFLOAT], but received " << static_cast(mType) + gLogError << "Unsupported type error, expected [kHALF,kFLOAT], but received " << static_cast(mType) << std::endl; return STATUS_NOT_SUPPORTED; diff --git a/plugin/embLayerNormPlugin/embLayerNormPlugin.h b/plugin/embLayerNormPlugin/embLayerNormPlugin.h index ba3cc18f..eb21d268 100644 --- a/plugin/embLayerNormPlugin/embLayerNormPlugin.h +++ b/plugin/embLayerNormPlugin/embLayerNormPlugin.h @@ -35,7 +35,7 @@ namespace plugin namespace bert { -int32_t computeMaskIdx(cudaStream_t stream, int32_t const S, int32_t const B, int32_t const* mask, int* maskIdx); +int32_t computeMaskIdx(cudaStream_t stream, int32_t const S, int32_t const B, int32_t const* mask, int32_t* maskIdx); template int32_t embSkipLayerNorm(cudaStream_t stream, int32_t ld, int32_t B, int32_t S, int32_t const* inputIds, diff --git a/plugin/exports-vfc_plugin.map b/plugin/exports-vfc_plugin.map index c81bfe23..70ee8938 100644 --- a/plugin/exports-vfc_plugin.map +++ b/plugin/exports-vfc_plugin.map @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/exports.map b/plugin/exports.map index b0b1d3c5..64de08ba 100644 --- a/plugin/exports.map +++ b/plugin/exports.map @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/plugin/fcPlugin/CustomFCPluginDynamic_PluginConfig.yaml b/plugin/fcPlugin/CustomFCPluginDynamic_PluginConfig.yaml index 44a26e8c..0939c434 100644 --- a/plugin/fcPlugin/CustomFCPluginDynamic_PluginConfig.yaml +++ b/plugin/fcPlugin/CustomFCPluginDynamic_PluginConfig.yaml @@ -57,5 +57,20 @@ versions: - out_dims - type_id - W -... - + golden_io_path: "plugin/fcPlugin/CustomFCPluginDynamic_PluginGoldenIO.json" + golden_reference_script: "plugin/fcPlugin/CustomFCPluginDynamic_PluginReference.py" + abs_tol: 1e-5 + rel_tol: 1e-5 + fp16_atol: 1e-3 + fp16_rtol: 1e-3 + configs: + config1: + input_types: + input: float16 + attribute_options: + "type_id": + value: 1 + shape: "1" + output_types: + output: float16 +... \ No newline at end of file diff --git a/plugin/flattenConcat/flattenConcat.cpp b/plugin/flattenConcat/flattenConcat.cpp index 11e451af..6ecc9f61 100644 --- a/plugin/flattenConcat/flattenConcat.cpp +++ b/plugin/flattenConcat/flattenConcat.cpp @@ -36,15 +36,15 @@ static char const* const kFLATTENCONCAT_PLUGIN_NAME{"FlattenConcat_TRT"}; PluginFieldCollection FlattenConcatPluginCreator::mFC{}; std::vector FlattenConcatPluginCreator::mPluginAttributes; -FlattenConcat::FlattenConcat(int concatAxis, bool ignoreBatch) +FlattenConcat::FlattenConcat(int32_t concatAxis, bool ignoreBatch) : mIgnoreBatch(ignoreBatch) , mConcatAxisID(concatAxis) { PLUGIN_VALIDATE(mConcatAxisID == 1 || mConcatAxisID == 2 || mConcatAxisID == 3); } -FlattenConcat::FlattenConcat(int concatAxis, bool ignoreBatch, int numInputs, int outputConcatAxis, - int const* inputConcatAxis, size_t const* copySize, nvinfer1::Dims const& chwDims) +FlattenConcat::FlattenConcat(int32_t concatAxis, bool ignoreBatch, int32_t numInputs, int32_t outputConcatAxis, + int32_t const* inputConcatAxis, size_t const* copySize, nvinfer1::Dims const& chwDims) : mCopySize(numInputs) , mInputConcatAxis(numInputs) , mIgnoreBatch(ignoreBatch) @@ -64,13 +64,13 @@ FlattenConcat::FlattenConcat(void const* data, size_t length) char const* d = static_cast(data); char const* const a = d; mIgnoreBatch = read(d); - mConcatAxisID = read(d); + mConcatAxisID = read(d); PLUGIN_VALIDATE(mConcatAxisID >= 1 && mConcatAxisID <= 3); - mOutputConcatAxis = read(d); - mNumInputs = read(d); + mOutputConcatAxis = read(d); + mNumInputs = read(d); mInputConcatAxis.resize(mNumInputs); - std::for_each(mInputConcatAxis.begin(), mInputConcatAxis.end(), [&](int& inp) { inp = read(d); }); + std::for_each(mInputConcatAxis.begin(), mInputConcatAxis.end(), [&](int32_t& inp) { inp = read(d); }); mCHW = read(d); @@ -82,12 +82,12 @@ FlattenConcat::FlattenConcat(void const* data, size_t length) FlattenConcat::~FlattenConcat() {} -int FlattenConcat::getNbOutputs() const noexcept +int32_t FlattenConcat::getNbOutputs() const noexcept { return 1; } -Dims FlattenConcat::getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept +Dims FlattenConcat::getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept { try { @@ -97,11 +97,11 @@ Dims FlattenConcat::getOutputDimensions(int index, Dims const* inputs, int nbInp mNumInputs = nbInputDims; mCopySize.resize(mNumInputs); mInputConcatAxis.resize(mNumInputs); - int outputConcatAxis = 0; + int32_t outputConcatAxis = 0; - for (int i = 0; i < nbInputDims; ++i) + for (int32_t i = 0; i < nbInputDims; ++i) { - int flattenInput = 0; + int32_t flattenInput = 0; PLUGIN_ASSERT(inputs[i].nbDims == 3); if (mConcatAxisID != 1) { @@ -129,20 +129,20 @@ Dims FlattenConcat::getOutputDimensions(int index, Dims const* inputs, int nbInp return Dims{}; } -int FlattenConcat::initialize() noexcept +int32_t FlattenConcat::initialize() noexcept { return STATUS_SUCCESS; } void FlattenConcat::terminate() noexcept {} -size_t FlattenConcat::getWorkspaceSize(int) const noexcept +size_t FlattenConcat::getWorkspaceSize(int32_t) const noexcept { return 0; } -int FlattenConcat::enqueue( - int batchSize, void const* const* inputs, void* const* outputs, void*, cudaStream_t stream) noexcept +int32_t FlattenConcat::enqueue( + int32_t batchSize, void const* const* inputs, void* const* outputs, void*, cudaStream_t stream) noexcept { try { @@ -157,11 +157,11 @@ int FlattenConcat::enqueue( } auto* output = static_cast(outputs[0]); - int offset = 0; - for (int i = 0; i < mNumInputs; ++i) + int32_t offset = 0; + for (int32_t i = 0; i < mNumInputs; ++i) { auto const* input = static_cast(inputs[i]); - for (int n = 0; n < numConcats; ++n) + for (int32_t n = 0; n < numConcats; ++n) { auto status = cublasScopy(mCublas, mInputConcatAxis[i], input + n * mInputConcatAxis[i], 1, output + (n * mOutputConcatAxis + offset), 1); @@ -185,7 +185,7 @@ int FlattenConcat::enqueue( size_t FlattenConcat::getSerializationSize() const noexcept { - return sizeof(bool) + sizeof(int) * (3 + mNumInputs) + sizeof(nvinfer1::Dims) + return sizeof(bool) + sizeof(int32_t) * (3 + mNumInputs) + sizeof(nvinfer1::Dims) + (sizeof(decltype(mCopySize)::value_type) * mNumInputs); } @@ -197,12 +197,12 @@ void FlattenConcat::serialize(void* buffer) const noexcept write(d, mConcatAxisID); write(d, mOutputConcatAxis); write(d, mNumInputs); - for (int i = 0; i < mNumInputs; ++i) + for (int32_t i = 0; i < mNumInputs; ++i) { write(d, mInputConcatAxis[i]); } write(d, mCHW); - for (int i = 0; i < mNumInputs; ++i) + for (int32_t i = 0; i < mNumInputs; ++i) { write(d, mCopySize[i]); } @@ -221,13 +221,13 @@ void FlattenConcat::detachFromContext() noexcept {} // Return true if output tensor is broadcast across a batch. bool FlattenConcat::isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept { return false; } // Return true if plugin can use input that is broadcast across batch without replication. -bool FlattenConcat::canBroadcastInputAcrossBatch(int inputIndex) const noexcept +bool FlattenConcat::canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept { return false; } @@ -251,15 +251,16 @@ char const* FlattenConcat::getPluginNamespace() const noexcept } // Return the DataType of the plugin output at the requested index -DataType FlattenConcat::getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept +DataType FlattenConcat::getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { PLUGIN_ASSERT(index < 3); return DataType::kFLOAT; } -void FlattenConcat::configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, +void FlattenConcat::configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept { try { @@ -269,9 +270,9 @@ void FlattenConcat::configurePlugin(Dims const* inputDims, int nbInputs, Dims co PLUGIN_ASSERT(inputDims[0].nbDims == 3); mInputConcatAxis.resize(mNumInputs); - for (int i = 0; i < nbInputs; ++i) + for (int32_t i = 0; i < nbInputs; ++i) { - int flattenInput = 0; + int32_t flattenInput = 0; PLUGIN_ASSERT(inputDims[i].nbDims == 3); if (mConcatAxisID != 1) { @@ -291,7 +292,7 @@ void FlattenConcat::configurePlugin(Dims const* inputDims, int nbInputs, Dims co } mCopySize.resize(mNumInputs); - for (int i = 0; i < nbInputs; ++i) + for (int32_t i = 0; i < nbInputs; ++i) { mCopySize[i] = inputDims[i].d[0] * inputDims[i].d[1] * inputDims[i].d[2] * sizeof(float); } @@ -374,7 +375,7 @@ IPluginV2Ext* FlattenConcatPluginCreator::createPlugin(char const* name, PluginF if (!strcmp(attrName, "axis")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - mConcatAxisID = *(static_cast(fields[i].data)); + mConcatAxisID = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "ignoreBatch")) { diff --git a/plugin/flattenConcat/flattenConcat.h b/plugin/flattenConcat/flattenConcat.h index eb4dc133..e0bcc008 100644 --- a/plugin/flattenConcat/flattenConcat.h +++ b/plugin/flattenConcat/flattenConcat.h @@ -33,10 +33,10 @@ namespace plugin class FlattenConcat : public IPluginV2Ext { public: - FlattenConcat(int concatAxis, bool ignoreBatch); + FlattenConcat(int32_t concatAxis, bool ignoreBatch); - FlattenConcat(int concatAxis, bool ignoreBatch, int numInputs, int outputConcatAxis, int const* inputConcatAxis, - size_t const* copySize, nvinfer1::Dims const& chwDims); + FlattenConcat(int32_t concatAxis, bool ignoreBatch, int32_t numInputs, int32_t outputConcatAxis, + int32_t const* inputConcatAxis, size_t const* copySize, nvinfer1::Dims const& chwDims); FlattenConcat(void const* data, size_t length); @@ -44,33 +44,34 @@ class FlattenConcat : public IPluginV2Ext FlattenConcat() = delete; - int getNbOutputs() const noexcept override; + int32_t getNbOutputs() const noexcept override; - Dims getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept override; + Dims getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept override; - int initialize() noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; - size_t getWorkspaceSize(int) const noexcept override; + size_t getWorkspaceSize(int32_t) const noexcept override; - int enqueue(int batchSize, void const* const* inputs, void* const* outputs, void* workspace, + int32_t enqueue(int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; - DataType getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + DataType getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; size_t getSerializationSize() const noexcept override; void serialize(void* buffer) const noexcept override; bool isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept override; + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept override; - bool canBroadcastInputAcrossBatch(int inputIndex) const noexcept override; + bool canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept override; - void configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, + void configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept override; + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept override; bool supportsFormat(DataType type, PluginFormat format) const noexcept override; @@ -98,9 +99,9 @@ class FlattenConcat : public IPluginV2Ext Weights deserializeToDevice(char const*& hostBuffer, size_t count) noexcept; std::vector mCopySize; - std::vector mInputConcatAxis; + std::vector mInputConcatAxis; bool mIgnoreBatch{false}; - int mConcatAxisID{0}, mOutputConcatAxis{0}, mNumInputs{0}; + int32_t mConcatAxisID{0}, mOutputConcatAxis{0}, mNumInputs{0}; nvinfer1::Dims mCHW; std::string mPluginNamespace; cublasHandle_t mCublas{nullptr}; @@ -126,7 +127,7 @@ class FlattenConcatPluginCreator : public nvinfer1::pluginInternal::BaseCreator private: static PluginFieldCollection mFC; bool mIgnoreBatch{false}; - int mConcatAxisID; + int32_t mConcatAxisID; static std::vector mPluginAttributes; }; diff --git a/plugin/geluPlugin/CustomGeluPluginDynamic_PluginConfig.yaml b/plugin/geluPlugin/CustomGeluPluginDynamic_PluginConfig.yaml index 90f32375..3a41b9ed 100644 --- a/plugin/geluPlugin/CustomGeluPluginDynamic_PluginConfig.yaml +++ b/plugin/geluPlugin/CustomGeluPluginDynamic_PluginConfig.yaml @@ -48,4 +48,24 @@ versions: max: "=pinf" attributes_required: - type_id + abs_tol: 1e-2 + rel_tol: 1e-2 + golden_reference_script: "plugin/geluPlugin/CustomGeluPluginDynamic_PluginReference.py" + configs: + config1: + input_types: + input: float32 + attribute_options: + type_id: + value: 0 + bias: + shape: "1, 1, 128, 1, 1" + config2: + input_types: + input: float16 + attribute_options: + type_id: + value: 1 + bias: + shape: "1, 1, 784, 1, 1" ... diff --git a/plugin/geluPlugin/geluPlugin.cpp b/plugin/geluPlugin/geluPlugin.cpp index 13e53f79..9d1d0cf1 100644 --- a/plugin/geluPlugin/geluPlugin.cpp +++ b/plugin/geluPlugin/geluPlugin.cpp @@ -51,8 +51,8 @@ GeluPluginDynamic::GeluPluginDynamic(const std::string name, const DataType type if (mHasBias) { void* cudaMem{nullptr}; - PLUGIN_CHECK(cudaMalloc(&cudaMem, getWeightsSize(bias, mType))); - PLUGIN_CHECK(cudaMemcpy(cudaMem, bias.values, getWeightsSize(bias, mType), cudaMemcpyHostToDevice)); + PLUGIN_CUASSERT(cudaMalloc(&cudaMem, getWeightsSize(bias, mType))); + PLUGIN_CUASSERT(cudaMemcpy(cudaMem, bias.values, getWeightsSize(bias, mType), cudaMemcpyHostToDevice)); make_cuda_shared(mBiasDev, cudaMem); } } @@ -89,15 +89,39 @@ nvinfer1::IPluginV2DynamicExt* GeluPluginDynamic::clone() const noexcept return nullptr; } -nvinfer1::DimsExprs GeluPluginDynamic::getOutputDimensions( - int outputIndex, nvinfer1::DimsExprs const* inputs, int nbInputs, nvinfer1::IExprBuilder& exprBuilder) noexcept +nvinfer1::DimsExprs GeluPluginDynamic::getOutputDimensions(int32_t outputIndex, nvinfer1::DimsExprs const* inputs, + int32_t nbInputs, nvinfer1::IExprBuilder& exprBuilder) noexcept { - return inputs[0]; + try + { + PLUGIN_VALIDATE(inputs != nullptr); + PLUGIN_VALIDATE(nbInputs == 1); + PLUGIN_VALIDATE(outputIndex == 0); + return inputs[0]; + } + catch (std::exception const& e) + { + caughtError(e); + } + return DimsExprs{}; } bool GeluPluginDynamic::supportsFormatCombination( - int pos, nvinfer1::PluginTensorDesc const* inOut, int nbInputs, int nbOutputs) noexcept + int32_t pos, nvinfer1::PluginTensorDesc const* inOut, int32_t nbInputs, int32_t nbOutputs) noexcept { + try + { + PLUGIN_VALIDATE(inOut != nullptr); + PLUGIN_VALIDATE(nbInputs == 1); + PLUGIN_VALIDATE(nbOutputs == 1); + PLUGIN_VALIDATE(pos >= 0); + PLUGIN_VALIDATE(pos < nbInputs + nbOutputs); + } + catch (std::exception const& e) + { + caughtError(e); + return false; + } PluginTensorDesc const& input = inOut[0]; if (pos == 0) @@ -112,77 +136,93 @@ bool GeluPluginDynamic::supportsFormatCombination( return false; } -void GeluPluginDynamic::configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int nbInputs, - nvinfer1::DynamicPluginTensorDesc const* out, int nbOutputs) noexcept +void GeluPluginDynamic::configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int32_t nbInputs, + nvinfer1::DynamicPluginTensorDesc const* out, int32_t nbOutputs) noexcept { gLogVerbose << "GeluPluginDynamic configurePlugin\n"; - PLUGIN_ASSERT(mType == in[0].desc.type); + + try + { + PLUGIN_VALIDATE(in != nullptr); + PLUGIN_VALIDATE(nbInputs == 1); + PLUGIN_VALIDATE(mType == in[0].desc.type); + } + catch (std::exception const& e) + { + caughtError(e); + } } -size_t GeluPluginDynamic::getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int nbInputs, - nvinfer1::PluginTensorDesc const* outputs, int nbOutputs) const noexcept +size_t GeluPluginDynamic::getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int32_t nbInputs, + nvinfer1::PluginTensorDesc const* outputs, int32_t nbOutputs) const noexcept { return 0; } -int GeluPluginDynamic::enqueue(nvinfer1::PluginTensorDesc const* inputDesc, - nvinfer1::PluginTensorDesc const* outputDesc, void const* const* inputs, void* const* outputs, void* workspace, - cudaStream_t stream) noexcept -{ - int const inputVolume = volume(inputDesc[0].dims); - int status = -1; +template +int32_t GeluPluginDynamic::enqueueTyped( + void const* input_, void* output_, int32_t const inputVolume, cudaStream_t stream) noexcept +{ + TDataType const* input = static_cast(input_); + TDataType* output = static_cast(output_); + int32_t const cols = inputVolume / mLd; + int32_t const rows = mLd; - // Our plugin outputs only one tensor - // Launch CUDA kernel wrapper and save its return value - if (mType == DataType::kFLOAT) + if (mHasBias) { - float const* input = static_cast(inputs[0]); - float* output = static_cast(outputs[0]); - if (mHasBias) - { - float const* bias = static_cast(mBiasDev.get()); - int const cols = inputVolume / mLd; - int const rows = mLd; - status = computeGeluBias(output, input, bias, rows, cols, stream); - } - else - { - status = computeGelu(stream, inputVolume, input, output); - } + TDataType const* bias = static_cast(mBiasDev.get()); + return computeGeluBias(output, input, bias, rows, cols, stream); } - else if (mType == DataType::kHALF) + else { - half const* input = static_cast(inputs[0]); - - half* output = static_cast(outputs[0]); + return computeGelu(stream, inputVolume, input, output); + } +} - if (mHasBias) - { - half const* bias = static_cast(mBiasDev.get()); - int const cols = inputVolume / mLd; - int const rows = mLd; - status = computeGeluBias(output, input, bias, rows, cols, stream); - } - else - { - status = computeGelu(stream, inputVolume, input, output); - } +int32_t GeluPluginDynamic::enqueue(nvinfer1::PluginTensorDesc const* inputDesc, + nvinfer1::PluginTensorDesc const* outputDesc, void const* const* inputs, void* const* outputs, void* workspace, + cudaStream_t stream) noexcept +{ + try + { + PLUGIN_VALIDATE(inputDesc != nullptr); + PLUGIN_VALIDATE(inputs != nullptr); + PLUGIN_VALIDATE(outputs != nullptr); } - else + catch (std::exception const& e) { + caughtError(e); return STATUS_FAILURE; } - return status; + int32_t const inputVolume = volume(inputDesc[0].dims); + + // Our plugin outputs only one tensor. + // Launch CUDA kernel wrapper and save its return value. + switch (mType) + { + case DataType::kFLOAT: return enqueueTyped(inputs[0], outputs[0], inputVolume, stream); + case DataType::kHALF: return enqueueTyped(inputs[0], outputs[0], inputVolume, stream); + default: return STATUS_FAILURE; + } } // IPluginV2Ext Methods nvinfer1::DataType GeluPluginDynamic::getOutputDataType( - int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { - PLUGIN_ASSERT(index == 0); - PLUGIN_ASSERT(inputTypes[0] == DataType::kFLOAT || inputTypes[0] == DataType::kHALF); - return inputTypes[0]; + try + { + PLUGIN_VALIDATE(index == 0); + PLUGIN_VALIDATE(inputTypes != nullptr); + PLUGIN_VALIDATE(inputTypes[0] == DataType::kFLOAT || inputTypes[0] == DataType::kHALF); + return inputTypes[0]; + } + catch (std::exception const& e) + { + caughtError(e); + } + return DataType{}; } // IPluginV2 Methods @@ -197,12 +237,12 @@ char const* GeluPluginDynamic::getPluginVersion() const noexcept return kGELU_PLUGIN_VERSION; } -int GeluPluginDynamic::getNbOutputs() const noexcept +int32_t GeluPluginDynamic::getNbOutputs() const noexcept { return 1; } -int GeluPluginDynamic::initialize() noexcept +int32_t GeluPluginDynamic::initialize() noexcept { gLogVerbose << "GeluPluginDynamic initalize\n"; return 0; @@ -243,7 +283,15 @@ void GeluPluginDynamic::destroy() noexcept void GeluPluginDynamic::setPluginNamespace(char const* libNamespace) noexcept { - mNamespace = libNamespace; + try + { + PLUGIN_VALIDATE(libNamespace != nullptr); + mNamespace = libNamespace; + } + catch (std::exception const& e) + { + caughtError(e); + } } char const* GeluPluginDynamic::getPluginNamespace() const noexcept @@ -283,6 +331,7 @@ IPluginV2* GeluPluginDynamicCreator::createPlugin(char const* name, PluginFieldC try { gLogVerbose << "GeluPluginDynamicCreator createPlugin\n"; + PLUGIN_VALIDATE(fc != nullptr); Weights bias{DataType::kFLOAT, nullptr, 0}; int32_t typeId = -1; @@ -290,13 +339,14 @@ IPluginV2* GeluPluginDynamicCreator::createPlugin(char const* name, PluginFieldC for (int32_t i = 0; i < fc->nbFields; i++) { - std::string field_name(fc->fields[i].name); + PLUGIN_VALIDATE(fc->fields[i].name != nullptr); + std::string fieldName(fc->fields[i].name); - if (field_name.compare("type_id") == 0) + if (fieldName.compare("type_id") == 0) { typeId = *static_cast(fc->fields[i].data); } - if (field_name.compare("bias") == 0) + if (fieldName.compare("bias") == 0) { bias.values = fc->fields[i].data; bias.count = fc->fields[i].length; @@ -337,7 +387,15 @@ IPluginV2* GeluPluginDynamicCreator::deserializePlugin( void GeluPluginDynamicCreator::setPluginNamespace(char const* libNamespace) noexcept { - mNamespace = libNamespace; + try + { + PLUGIN_VALIDATE(libNamespace != nullptr); + mNamespace = libNamespace; + } + catch (std::exception const& e) + { + caughtError(e); + } } char const* GeluPluginDynamicCreator::getPluginNamespace() const noexcept diff --git a/plugin/geluPlugin/geluPlugin.h b/plugin/geluPlugin/geluPlugin.h index 20e27f28..009797c3 100644 --- a/plugin/geluPlugin/geluPlugin.h +++ b/plugin/geluPlugin/geluPlugin.h @@ -33,15 +33,15 @@ namespace plugin namespace bert { -int computeGelu(cudaStream_t stream, int n, float const* input, float* output); +int32_t computeGelu(cudaStream_t stream, int32_t n, float const* input, float* output); -int computeGelu(cudaStream_t stream, int n, half const* input, half* output); +int32_t computeGelu(cudaStream_t stream, int32_t n, half const* input, half* output); -int computeGeluBias( - float* output, float const* input, float const* bias, int const ld, int const cols, cudaStream_t stream); +int32_t computeGeluBias( + float* output, float const* input, float const* bias, int32_t const ld, int32_t const cols, cudaStream_t stream); -int computeGeluBias( - half* output, half const* input, half const* bias, int const ld, int const cols, cudaStream_t stream); +int32_t computeGeluBias( + half* output, half const* input, half const* bias, int32_t const ld, int32_t const cols, cudaStream_t stream); class GeluPluginDynamic : public nvinfer1::IPluginV2DynamicExt { @@ -56,26 +56,26 @@ class GeluPluginDynamic : public nvinfer1::IPluginV2DynamicExt // IPluginV2DynamicExt Methods nvinfer1::IPluginV2DynamicExt* clone() const noexcept override; - nvinfer1::DimsExprs getOutputDimensions(int outputIndex, nvinfer1::DimsExprs const* inputs, int nbInputs, + nvinfer1::DimsExprs getOutputDimensions(int32_t outputIndex, nvinfer1::DimsExprs const* inputs, int32_t nbInputs, nvinfer1::IExprBuilder& exprBuilder) noexcept override; bool supportsFormatCombination( - int pos, nvinfer1::PluginTensorDesc const* inOut, int nbInputs, int nbOutputs) noexcept override; - void configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int nbInputs, - nvinfer1::DynamicPluginTensorDesc const* out, int nbOutputs) noexcept override; - size_t getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int nbInputs, - nvinfer1::PluginTensorDesc const* outputs, int nbOutputs) const noexcept override; - int enqueue(nvinfer1::PluginTensorDesc const* inputDesc, nvinfer1::PluginTensorDesc const* outputDesc, + int32_t pos, nvinfer1::PluginTensorDesc const* inOut, int32_t nbInputs, int32_t nbOutputs) noexcept override; + void configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int32_t nbInputs, + nvinfer1::DynamicPluginTensorDesc const* out, int32_t nbOutputs) noexcept override; + size_t getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int32_t nbInputs, + nvinfer1::PluginTensorDesc const* outputs, int32_t nbOutputs) const noexcept override; + int32_t enqueue(nvinfer1::PluginTensorDesc const* inputDesc, nvinfer1::PluginTensorDesc const* outputDesc, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; // IPluginV2Ext Methods nvinfer1::DataType getOutputDataType( - int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; // IPluginV2 Methods char const* getPluginType() const noexcept override; char const* getPluginVersion() const noexcept override; - int getNbOutputs() const noexcept override; - int initialize() noexcept override; + int32_t getNbOutputs() const noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; size_t getSerializationSize() const noexcept override; void serialize(void* buffer) const noexcept override; @@ -84,6 +84,10 @@ class GeluPluginDynamic : public nvinfer1::IPluginV2DynamicExt char const* getPluginNamespace() const noexcept override; private: + // Helper method for enqueue() + template + int32_t enqueueTyped(void const* input, void* output, int32_t const inputVolume, cudaStream_t stream) noexcept; + const std::string mLayerName; std::string mNamespace; diff --git a/plugin/generateDetectionPlugin/generateDetectionPlugin.cpp b/plugin/generateDetectionPlugin/generateDetectionPlugin.cpp index 4a9b0984..574f2ba2 100644 --- a/plugin/generateDetectionPlugin/generateDetectionPlugin.cpp +++ b/plugin/generateDetectionPlugin/generateDetectionPlugin.cpp @@ -125,8 +125,8 @@ IPluginV2Ext* GenerateDetectionPluginCreator::deserializePlugin( return nullptr; } -GenerateDetection::GenerateDetection( - int num_classes, int keep_topk, float score_threshold, float iou_threshold, nvinfer1::Dims const& image_size) +GenerateDetection::GenerateDetection(int32_t num_classes, int32_t keep_topk, float score_threshold, float iou_threshold, + nvinfer1::Dims const& image_size) : mNbClasses(num_classes) , mKeepTopK(keep_topk) , mScoreThreshold(score_threshold) @@ -150,12 +150,12 @@ GenerateDetection::GenerateDetection( mType = DataType::kFLOAT; } -int GenerateDetection::getNbOutputs() const noexcept +int32_t GenerateDetection::getNbOutputs() const noexcept { return 1; } -int GenerateDetection::initialize() noexcept +int32_t GenerateDetection::initialize() noexcept { // Init the regWeight [10, 10, 5, 5] mRegWeightDevice = std::make_shared>(4); @@ -163,12 +163,12 @@ int GenerateDetection::initialize() noexcept static_cast(TLTMaskRCNNConfig::DETECTION_REG_WEIGHTS), sizeof(float) * 4, cudaMemcpyHostToDevice)); //@Init the mValidCnt and mDecodedBboxes for max batch size - std::vector tempValidCnt(mMaxBatchSize, mAnchorsCnt); + std::vector tempValidCnt(mMaxBatchSize, mAnchorsCnt); - mValidCnt = std::make_shared>(mMaxBatchSize); + mValidCnt = std::make_shared>(mMaxBatchSize); - PLUGIN_CUASSERT(cudaMemcpy( - mValidCnt->mPtr, static_cast(tempValidCnt.data()), sizeof(int) * mMaxBatchSize, cudaMemcpyHostToDevice)); + PLUGIN_CUASSERT(cudaMemcpy(mValidCnt->mPtr, static_cast(tempValidCnt.data()), + sizeof(int32_t) * mMaxBatchSize, cudaMemcpyHostToDevice)); return 0; } @@ -220,7 +220,7 @@ char const* GenerateDetection::getPluginNamespace() const noexcept size_t GenerateDetection::getSerializationSize() const noexcept { - return sizeof(int) * 2 + sizeof(float) * 2 + sizeof(int) * 2 + sizeof(nvinfer1::Dims); + return sizeof(int32_t) * 2 + sizeof(float) * 2 + sizeof(int32_t) * 2 + sizeof(nvinfer1::Dims); } void GenerateDetection::serialize(void* buffer) const noexcept @@ -267,7 +267,7 @@ void GenerateDetection::deserialize(int8_t const* data, size_t length) mType = DataType::kFLOAT; } -void GenerateDetection::check_valid_inputs(nvinfer1::Dims const* inputs, int nbInputDims) noexcept +void GenerateDetection::check_valid_inputs(nvinfer1::Dims const* inputs, int32_t nbInputDims) noexcept { // classifier_delta_bbox[N, anchors, num_classes*4, 1, 1] // classifier_class[N, anchors, num_classes, 1, 1] @@ -282,13 +282,13 @@ void GenerateDetection::check_valid_inputs(nvinfer1::Dims const* inputs, int nbI PLUGIN_ASSERT(inputs[2].nbDims == 2 && inputs[2].d[1] == 4); } -size_t GenerateDetection::getWorkspaceSize(int batch_size) const noexcept +size_t GenerateDetection::getWorkspaceSize(int32_t batch_size) const noexcept { RefineDetectionWorkSpace refine(batch_size, mAnchorsCnt, mParam, mType); return refine.totalSize; } -Dims GenerateDetection::getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept +Dims GenerateDetection::getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept { check_valid_inputs(inputs, nbInputDims); @@ -322,7 +322,7 @@ int32_t GenerateDetection::enqueue( } DataType GenerateDetection::getOutputDataType( - int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { // Only DataType::kFLOAT is acceptable by the plugin layer return DataType::kFLOAT; @@ -330,21 +330,21 @@ DataType GenerateDetection::getOutputDataType( // Return true if output tensor is broadcast across a batch. bool GenerateDetection::isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept { return false; } // Return true if plugin can use input that is broadcast across batch without replication. -bool GenerateDetection::canBroadcastInputAcrossBatch(int inputIndex) const noexcept +bool GenerateDetection::canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept { return false; } // Configure the layer with input and output data types. -void GenerateDetection::configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, - DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept +void GenerateDetection::configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, + int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept { check_valid_inputs(inputDims, nbInputs); PLUGIN_ASSERT(inputDims[0].d[0] == inputDims[1].d[0] && inputDims[1].d[0] == inputDims[2].d[0]); diff --git a/plugin/generateDetectionPlugin/generateDetectionPlugin.h b/plugin/generateDetectionPlugin/generateDetectionPlugin.h index 30825328..75dd50f3 100644 --- a/plugin/generateDetectionPlugin/generateDetectionPlugin.h +++ b/plugin/generateDetectionPlugin/generateDetectionPlugin.h @@ -36,24 +36,24 @@ namespace plugin class GenerateDetection : public IPluginV2Ext { public: - GenerateDetection( - int num_classes, int keep_topk, float score_threshold, float iou_threshold, nvinfer1::Dims const& image_size); + GenerateDetection(int32_t num_classes, int32_t keep_topk, float score_threshold, float iou_threshold, + nvinfer1::Dims const& image_size); GenerateDetection(void const* data, size_t length); ~GenerateDetection() noexcept override = default; - int getNbOutputs() const noexcept override; + int32_t getNbOutputs() const noexcept override; - Dims getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept override; + Dims getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept override; - int initialize() noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; void destroy() noexcept override; - size_t getWorkspaceSize(int maxBatchSize) const noexcept override; + size_t getWorkspaceSize(int32_t maxBatchSize) const noexcept override; int32_t enqueue(int32_t batch_size, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; @@ -74,25 +74,26 @@ class GenerateDetection : public IPluginV2Ext char const* getPluginNamespace() const noexcept override; - DataType getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + DataType getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; bool isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept override; + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept override; - bool canBroadcastInputAcrossBatch(int inputIndex) const noexcept override; + bool canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept override; void attachToContext( cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept override; - void configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, + void configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept override; + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept override; void detachFromContext() noexcept override; private: void deserialize(int8_t const* data, size_t length); - void check_valid_inputs(nvinfer1::Dims const* inputs, int nbInputDims) noexcept; + void check_valid_inputs(nvinfer1::Dims const* inputs, int32_t nbInputDims) noexcept; int32_t mBackgroundLabel{}; int32_t mNbClasses{}; diff --git a/plugin/gridAnchorPlugin/gridAnchorPlugin.cpp b/plugin/gridAnchorPlugin/gridAnchorPlugin.cpp index 14cbf260..33afca2d 100644 --- a/plugin/gridAnchorPlugin/gridAnchorPlugin.cpp +++ b/plugin/gridAnchorPlugin/gridAnchorPlugin.cpp @@ -38,28 +38,28 @@ char const* const kGRID_ANCHOR_PLUGIN_VERSION = "1"; PluginFieldCollection GridAnchorBasePluginCreator::mFC{}; std::vector GridAnchorBasePluginCreator::mPluginAttributes; -GridAnchorGenerator::GridAnchorGenerator(GridAnchorParameters const* paramIn, int numLayers, char const* name) +GridAnchorGenerator::GridAnchorGenerator(GridAnchorParameters const* paramIn, int32_t numLayers, char const* name) : mPluginName(name) , mNumLayers(numLayers) { - PLUGIN_CUASSERT(cudaMallocHost((void**) &mNumPriors, mNumLayers * sizeof(int))); + PLUGIN_CUASSERT(cudaMallocHost((void**) &mNumPriors, mNumLayers * sizeof(int32_t))); PLUGIN_CUASSERT(cudaMallocHost((void**) &mDeviceWidths, mNumLayers * sizeof(Weights))); PLUGIN_CUASSERT(cudaMallocHost((void**) &mDeviceHeights, mNumLayers * sizeof(Weights))); mParam.resize(mNumLayers); - for (int id = 0; id < mNumLayers; id++) + for (int32_t id = 0; id < mNumLayers; id++) { mParam[id] = paramIn[id]; PLUGIN_VALIDATE(mParam[id].numAspectRatios >= 0 && mParam[id].aspectRatios != nullptr); mParam[id].aspectRatios = (float*) malloc(sizeof(float) * mParam[id].numAspectRatios); - for (int i = 0; i < paramIn[id].numAspectRatios; ++i) + for (int32_t i = 0; i < paramIn[id].numAspectRatios; ++i) { mParam[id].aspectRatios[i] = paramIn[id].aspectRatios[i]; } - for (int i = 0; i < 4; ++i) + for (int32_t i = 0; i < 4; ++i) { mParam[id].variance[i] = paramIn[id].variance[i]; } @@ -67,7 +67,7 @@ GridAnchorGenerator::GridAnchorGenerator(GridAnchorParameters const* paramIn, in std::vector tmpScales(mNumLayers + 1); // Calculate the scales of SSD model for each layer - for (int i = 0; i < mNumLayers; i++) + for (int32_t i = 0; i < mNumLayers; i++) { tmpScales[i] = (mParam[id].minSize + (mParam[id].maxSize - mParam[id].minSize) * id / (mNumLayers - 1)); } @@ -82,7 +82,7 @@ GridAnchorGenerator::GridAnchorGenerator(GridAnchorParameters const* paramIn, in // The first layer is different if (id == 0) { - for (int i = 0; i < mParam[id].numAspectRatios; i++) + for (int32_t i = 0; i < mParam[id].numAspectRatios; i++) { aspect_ratios.push_back(mParam[id].aspectRatios[i]); scales.push_back(scale0[i]); @@ -92,7 +92,7 @@ GridAnchorGenerator::GridAnchorGenerator(GridAnchorParameters const* paramIn, in else { - for (int i = 0; i < mParam[id].numAspectRatios; i++) + for (int32_t i = 0; i < mParam[id].numAspectRatios; i++) { aspect_ratios.push_back(mParam[id].aspectRatios[i]); } @@ -100,7 +100,7 @@ GridAnchorGenerator::GridAnchorGenerator(GridAnchorParameters const* paramIn, in aspect_ratios.push_back(1.0); // scales - for (int i = 0; i < mParam[id].numAspectRatios; i++) + for (int32_t i = 0; i < mParam[id].numAspectRatios; i++) { scales.push_back(tmpScales[id]); } @@ -115,7 +115,7 @@ GridAnchorGenerator::GridAnchorGenerator(GridAnchorParameters const* paramIn, in std::vector tmpWidths; std::vector tmpHeights; // Calculate the width and height of the prior boxes - for (int i = 0; i < mNumPriors[id]; i++) + for (int32_t i = 0; i < mNumPriors[id]; i++) { float sqrt_AR = sqrt(aspect_ratios[i]); tmpWidths.push_back(scales[i] * sqrt_AR); @@ -131,30 +131,30 @@ GridAnchorGenerator::GridAnchorGenerator(void const* data, size_t length, char c : mPluginName(name) { char const *d = reinterpret_cast(data), *a = d; - mNumLayers = read(d); - PLUGIN_CUASSERT(cudaMallocHost((void**) &mNumPriors, mNumLayers * sizeof(int))); + mNumLayers = read(d); + PLUGIN_CUASSERT(cudaMallocHost((void**) &mNumPriors, mNumLayers * sizeof(int32_t))); PLUGIN_CUASSERT(cudaMallocHost((void**) &mDeviceWidths, mNumLayers * sizeof(Weights))); PLUGIN_CUASSERT(cudaMallocHost((void**) &mDeviceHeights, mNumLayers * sizeof(Weights))); mParam.resize(mNumLayers); - for (int id = 0; id < mNumLayers; id++) + for (int32_t id = 0; id < mNumLayers; id++) { // we have to deserialize GridAnchorParameters by hand mParam[id].minSize = read(d); mParam[id].maxSize = read(d); - mParam[id].numAspectRatios = read(d); + mParam[id].numAspectRatios = read(d); mParam[id].aspectRatios = (float*) malloc(sizeof(float) * mParam[id].numAspectRatios); - for (int i = 0; i < mParam[id].numAspectRatios; ++i) + for (int32_t i = 0; i < mParam[id].numAspectRatios; ++i) { mParam[id].aspectRatios[i] = read(d); } - mParam[id].H = read(d); - mParam[id].W = read(d); - for (int i = 0; i < 4; ++i) + mParam[id].H = read(d); + mParam[id].W = read(d); + for (int32_t i = 0; i < 4; ++i) { mParam[id].variance[i] = read(d); } - mNumPriors[id] = read(d); + mNumPriors[id] = read(d); mDeviceWidths[id] = deserializeToDevice(d, mNumPriors[id]); mDeviceHeights[id] = deserializeToDevice(d, mNumPriors[id]); } @@ -164,7 +164,7 @@ GridAnchorGenerator::GridAnchorGenerator(void const* data, size_t length, char c GridAnchorGenerator::~GridAnchorGenerator() { - for (int id = 0; id < mNumLayers; id++) + for (int32_t id = 0; id < mNumLayers; id++) { PLUGIN_CUERROR(cudaFree(const_cast(mDeviceWidths[id].values))); PLUGIN_CUERROR(cudaFree(const_cast(mDeviceHeights[id].values))); @@ -175,12 +175,12 @@ GridAnchorGenerator::~GridAnchorGenerator() PLUGIN_CUERROR(cudaFreeHost(mDeviceHeights)); } -int GridAnchorGenerator::getNbOutputs() const noexcept +int32_t GridAnchorGenerator::getNbOutputs() const noexcept { return mNumLayers; } -Dims GridAnchorGenerator::getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept +Dims GridAnchorGenerator::getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept { // Particularity of the PriorBox layer: no batchSize dimension needed // 2 channels. First channel stores the mean of each prior coordinate. @@ -188,23 +188,23 @@ Dims GridAnchorGenerator::getOutputDimensions(int index, Dims const* inputs, int return Dims3(2, mParam[index].H * mParam[index].W * mNumPriors[index] * 4, 1); } -int GridAnchorGenerator::initialize() noexcept +int32_t GridAnchorGenerator::initialize() noexcept { return STATUS_SUCCESS; } void GridAnchorGenerator::terminate() noexcept {} -size_t GridAnchorGenerator::getWorkspaceSize(int maxBatchSize) const noexcept +size_t GridAnchorGenerator::getWorkspaceSize(int32_t maxBatchSize) const noexcept { return 0; } -int GridAnchorGenerator::enqueue( - int batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept +int32_t GridAnchorGenerator::enqueue( + int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept { // Generate prior boxes for each layer - for (int id = 0; id < mNumLayers; id++) + for (int32_t id = 0; id < mNumLayers; id++) { void* outputData = outputs[id]; pluginStatus_t status = anchorGridInference( @@ -219,10 +219,10 @@ int GridAnchorGenerator::enqueue( size_t GridAnchorGenerator::getSerializationSize() const noexcept { - size_t sum = sizeof(int); // mNumLayers - for (int i = 0; i < mNumLayers; i++) + size_t sum = sizeof(int32_t); // mNumLayers + for (int32_t i = 0; i < mNumLayers; i++) { - sum += 4 * sizeof(int); // mNumPriors, mParam[i].{numAspectRatios, H, W} + sum += 4 * sizeof(int32_t); // mNumPriors, mParam[i].{numAspectRatios, H, W} sum += (6 + mParam[i].numAspectRatios) * sizeof(float); // mParam[i].{minSize, maxSize, aspectRatios, variance[4]} sum += mDeviceWidths[i].count * sizeof(float); @@ -235,19 +235,19 @@ void GridAnchorGenerator::serialize(void* buffer) const noexcept { char *d = reinterpret_cast(buffer), *a = d; write(d, mNumLayers); - for (int id = 0; id < mNumLayers; id++) + for (int32_t id = 0; id < mNumLayers; id++) { // we have to serialize GridAnchorParameters by hand write(d, mParam[id].minSize); write(d, mParam[id].maxSize); write(d, mParam[id].numAspectRatios); - for (int i = 0; i < mParam[id].numAspectRatios; ++i) + for (int32_t i = 0; i < mParam[id].numAspectRatios; ++i) { write(d, mParam[id].aspectRatios[i]); } write(d, mParam[id].H); write(d, mParam[id].W); - for (int i = 0; i < 4; ++i) + for (int32_t i = 0; i < 4; ++i) { write(d, mParam[id].variance[i]); } @@ -309,7 +309,7 @@ char const* GridAnchorGenerator::getPluginNamespace() const noexcept #include // Return the DataType of the plugin output at the requested index DataType GridAnchorGenerator::getOutputDataType( - int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { PLUGIN_ASSERT(index < mNumLayers); return DataType::kFLOAT; @@ -317,21 +317,21 @@ DataType GridAnchorGenerator::getOutputDataType( // Return true if output tensor is broadcast across a batch. bool GridAnchorGenerator::isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept { return false; } // Return true if plugin can use input that is broadcast across batch without replication. -bool GridAnchorGenerator::canBroadcastInputAcrossBatch(int inputIndex) const noexcept +bool GridAnchorGenerator::canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept { return false; } // Configure the layer with input and output data types. -void GridAnchorGenerator::configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, - DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept +void GridAnchorGenerator::configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, + int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept { PLUGIN_ASSERT(nbOutputs == mNumLayers); PLUGIN_ASSERT(outputDims[0].nbDims == 3); @@ -400,20 +400,20 @@ IPluginV2Ext* GridAnchorBasePluginCreator::createPlugin(char const* name, Plugin try { float minScale = 0.2F, maxScale = 0.95F; - int numLayers = 6; + int32_t numLayers = 6; std::vector aspectRatios; - std::vector fMapShapes; + std::vector fMapShapes; std::vector layerVariances; PluginField const* fields = fc->fields; bool const isFMapRect = (kGRID_ANCHOR_PLUGIN_NAMES[1] == mPluginName); - for (int i = 0; i < fc->nbFields; ++i) + for (int32_t i = 0; i < fc->nbFields; ++i) { char const* attrName = fields[i].name; if (!strcmp(attrName, "numLayers")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - numLayers = static_cast(*(static_cast(fields[i].data))); + numLayers = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "minSize")) { @@ -428,10 +428,10 @@ IPluginV2Ext* GridAnchorBasePluginCreator::createPlugin(char const* name, Plugin else if (!strcmp(attrName, "variance")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kFLOAT32); - int size = fields[i].length; + int32_t size = fields[i].length; layerVariances.reserve(size); auto const* lVar = static_cast(fields[i].data); - for (int j = 0; j < size; j++) + for (int32_t j = 0; j < size; j++) { layerVariances.push_back(*lVar); lVar++; @@ -440,10 +440,10 @@ IPluginV2Ext* GridAnchorBasePluginCreator::createPlugin(char const* name, Plugin else if (!strcmp(attrName, "aspectRatios")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kFLOAT32); - int size = fields[i].length; + int32_t size = fields[i].length; aspectRatios.reserve(size); auto const* aR = static_cast(fields[i].data); - for (int j = 0; j < size; j++) + for (int32_t j = 0; j < size; j++) { aspectRatios.push_back(*aR); aR++; @@ -452,11 +452,11 @@ IPluginV2Ext* GridAnchorBasePluginCreator::createPlugin(char const* name, Plugin else if (!strcmp(attrName, "featureMapShapes")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - int size = fields[i].length; + int32_t size = fields[i].length; PLUGIN_VALIDATE(!isFMapRect || (size % 2 == 0)); fMapShapes.reserve(size); - int const* fMap = static_cast(fields[i].data); - for (int j = 0; j < size; j++) + int32_t const* fMap = static_cast(fields[i].data); + for (int32_t j = 0; j < size; j++) { fMapShapes.push_back(*fMap); fMap++; @@ -468,13 +468,13 @@ IPluginV2Ext* GridAnchorBasePluginCreator::createPlugin(char const* name, Plugin std::vector firstLayerAspectRatios; PLUGIN_VALIDATE(numLayers > 0); - int const numExpectedLayers = static_cast(fMapShapes.size()) >> (isFMapRect ? 1 : 0); + int32_t const numExpectedLayers = static_cast(fMapShapes.size()) >> (isFMapRect ? 1 : 0); PLUGIN_VALIDATE(numExpectedLayers == numLayers); - int numFirstLayerARs = 3; + int32_t numFirstLayerARs = 3; // First layer only has the first 3 aspect ratios from aspectRatios firstLayerAspectRatios.reserve(numFirstLayerARs); - for (int i = 0; i < numFirstLayerARs; ++i) + for (int32_t i = 0; i < numFirstLayerARs; ++i) { firstLayerAspectRatios.push_back(aspectRatios[i]); } @@ -482,21 +482,22 @@ IPluginV2Ext* GridAnchorBasePluginCreator::createPlugin(char const* name, Plugin std::vector boxParams(numLayers); // One set of box parameters for one layer - for (int i = 0; i < numLayers; i++) + for (int32_t i = 0; i < numLayers; i++) { - int hOffset = (isFMapRect ? i * 2 : i); - int wOffset = (isFMapRect ? i * 2 + 1 : i); + int32_t hOffset = (isFMapRect ? i * 2 : i); + int32_t wOffset = (isFMapRect ? i * 2 + 1 : i); // Only the first layer is different if (i == 0) { - boxParams[i] = {minScale, maxScale, firstLayerAspectRatios.data(), (int) firstLayerAspectRatios.size(), - fMapShapes[hOffset], fMapShapes[wOffset], + boxParams[i] = {minScale, maxScale, firstLayerAspectRatios.data(), + (int32_t) firstLayerAspectRatios.size(), fMapShapes[hOffset], fMapShapes[wOffset], {layerVariances[0], layerVariances[1], layerVariances[2], layerVariances[3]}}; } else { - boxParams[i] = {minScale, maxScale, aspectRatios.data(), (int) aspectRatios.size(), fMapShapes[hOffset], - fMapShapes[wOffset], {layerVariances[0], layerVariances[1], layerVariances[2], layerVariances[3]}}; + boxParams[i] = {minScale, maxScale, aspectRatios.data(), (int32_t) aspectRatios.size(), + fMapShapes[hOffset], fMapShapes[wOffset], + {layerVariances[0], layerVariances[1], layerVariances[2], layerVariances[3]}}; } } diff --git a/plugin/gridAnchorPlugin/gridAnchorPlugin.h b/plugin/gridAnchorPlugin/gridAnchorPlugin.h index 496de7c7..b0e1fc16 100644 --- a/plugin/gridAnchorPlugin/gridAnchorPlugin.h +++ b/plugin/gridAnchorPlugin/gridAnchorPlugin.h @@ -31,23 +31,23 @@ namespace plugin class GridAnchorGenerator : public IPluginV2Ext { public: - GridAnchorGenerator(GridAnchorParameters const* param, int numLayers, char const* version); + GridAnchorGenerator(GridAnchorParameters const* param, int32_t numLayers, char const* version); GridAnchorGenerator(void const* data, size_t length, char const* version); ~GridAnchorGenerator() override; - int getNbOutputs() const noexcept override; + int32_t getNbOutputs() const noexcept override; - Dims getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept override; + Dims getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept override; - int initialize() noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; - size_t getWorkspaceSize(int maxBatchSize) const noexcept override; + size_t getWorkspaceSize(int32_t maxBatchSize) const noexcept override; - int enqueue(int batchSize, void const* const* inputs, void* const* outputs, void* workspace, + int32_t enqueue(int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; size_t getSerializationSize() const noexcept override; @@ -68,19 +68,20 @@ class GridAnchorGenerator : public IPluginV2Ext char const* getPluginNamespace() const noexcept override; - DataType getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + DataType getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; bool isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept override; + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept override; - bool canBroadcastInputAcrossBatch(int inputIndex) const noexcept override; + bool canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept override; void attachToContext( cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept override; - void configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, + void configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept override; + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept override; void detachFromContext() noexcept override; @@ -94,9 +95,9 @@ class GridAnchorGenerator : public IPluginV2Ext Weights deserializeToDevice(char const*& hostBuffer, size_t count) noexcept; - int mNumLayers; + int32_t mNumLayers; std::vector mParam; - int* mNumPriors; + int32_t* mNumPriors; Weights *mDeviceWidths, *mDeviceHeights; std::string mPluginNamespace; }; diff --git a/plugin/groupNormalizationPlugin/GroupNormalizationPlugin_PluginConfig.yaml b/plugin/groupNormalizationPlugin/GroupNormalizationPlugin_PluginConfig.yaml index 76574792..bec760a5 100644 --- a/plugin/groupNormalizationPlugin/GroupNormalizationPlugin_PluginConfig.yaml +++ b/plugin/groupNormalizationPlugin/GroupNormalizationPlugin_PluginConfig.yaml @@ -3,6 +3,37 @@ name: GroupNormalizationPlugin interface: "IPluginV2DynamicExt" versions: "1": + inputs: + - input + - scale + - bias + outputs: + - output + input_dims: + input: 4 + scale: 1 + bias: 1 + input_dim_constraints: + - "input_1 MULTIPLE_OF num_groups_0" + - "scale_0 == input_1" + - "bias_0 == scale_0" + input_dim_range: + input: + min: "=1, =1, =1, =1" + max: "=pinf, =pinf, =pinf, =pinf" + scale: + min: "=1" + max: "=pinf" + bias: + min: "=1" + max: "=pinf" + supported_input_types: + - combination1: + input: float32 + scale: float32 + bias: float32 + output_dims: + output: "input_0, input_1, input_2, input_3" attributes: - eps - num_groups @@ -12,12 +43,53 @@ versions: attribute_length: eps: 1 num_groups: 1 + attribute_dim_range: + eps: + min: "=1" + max: "=1" + num_groups: + min: "=1" + max: "=1" attribute_options: eps: min: "0" max: "=pinf" num_groups: - min: "0" + min: "=1" max: "=pinf" attributes_required: [] + abs_tol: 1e-2 + rel_tol: 1e-2 + golden_reference_script: "plugin/groupNormalizationPlugin/GroupNormalizationPlugin_PluginReference.py" + configs: + config1: + input_types: + input: float32 + scale: float32 + bias: float32 + attribute_options: + eps: + value: 0.0001 + num_groups: + value: 1 + config2: + input_types: + input: float32 + scale: float32 + bias: float32 + attribute_options: + eps: + value: 0.001 + num_groups: + value: 2 + config3: + input_types: + input: float32 + scale: float32 + bias: float32 + attribute_options: + eps: + value: 0.01 + num_groups: + value: 3 ... diff --git a/plugin/groupNormalizationPlugin/GroupNormalizationPlugin_PluginReference.py b/plugin/groupNormalizationPlugin/GroupNormalizationPlugin_PluginReference.py new file mode 100644 index 00000000..0e7fb510 --- /dev/null +++ b/plugin/groupNormalizationPlugin/GroupNormalizationPlugin_PluginReference.py @@ -0,0 +1,27 @@ +import numpy as np + +def ref(inputs, attributes, version = "1"): + assert version == "1" + num_groups = attributes["num_groups"][0] + epsilon = attributes["eps"][0] + input = inputs["input"] + bias = inputs["bias"] + scale = inputs["scale"] + output = input.copy() + + assert len(input.shape) == 4 + B, C, H, W = input.shape + + # Groups are a subdivision of the channel dimension. + assert C % num_groups == 0 + + # Normalize every group. + output = output.reshape((B * num_groups, -1)) + output -= np.mean(output, axis=-1, keepdims=True) + output /= np.sqrt(epsilon + np.var(output, axis=-1, keepdims=True)) + + # Apply per-channel scale and bias. + output = output.reshape(input.shape) + output = bias.reshape(1, C, 1, 1) + scale.reshape(1, C, 1, 1) * output + + return {"output": output} diff --git a/plugin/groupNormalizationPlugin/groupNormalizationPlugin.cpp b/plugin/groupNormalizationPlugin/groupNormalizationPlugin.cpp index 44e0b881..d68f4018 100644 --- a/plugin/groupNormalizationPlugin/groupNormalizationPlugin.cpp +++ b/plugin/groupNormalizationPlugin/groupNormalizationPlugin.cpp @@ -17,6 +17,7 @@ #include "groupNormalizationPlugin.h" #include "common/dimsHelpers.h" +#include "common/serialize.hpp" #include #include @@ -25,16 +26,6 @@ using namespace nvinfer1; using nvinfer1::plugin::GroupNormalizationPlugin; using nvinfer1::plugin::GroupNormalizationPluginCreator; -#define PLUGIN_CHECK_CUDNN(call) \ - do \ - { \ - cudnnStatus_t status = call; \ - if (status != CUDNN_STATUS_SUCCESS) \ - { \ - return status; \ - } \ - } while (0) - namespace { constexpr char const* kGROUP_NORM_VERSION{"1"}; @@ -47,33 +38,17 @@ std::vector GroupNormalizationPluginCreator::mPluginAttri REGISTER_TENSORRT_PLUGIN(GroupNormalizationPluginCreator); -GroupNormalizationPlugin::GroupNormalizationPlugin(float epsilon, int nbGroups) +GroupNormalizationPlugin::GroupNormalizationPlugin(float epsilon, int32_t nbGroups) : mEpsilon(epsilon) , mNbGroups(nbGroups) { PLUGIN_VALIDATE(mEpsilon > 0.0F); - // Number of groups should be positive PLUGIN_VALIDATE(mNbGroups > 0); } -int GroupNormalizationPlugin::initialize() noexcept +int32_t GroupNormalizationPlugin::initialize() noexcept { - auto allocScaleBias = [this](std::shared_ptr>& buf, float value) { - PLUGIN_VALIDATE(mNbScaleBias > 0); - if (!buf || !buf->mPtr || buf->mSize != mNbScaleBias) - { - // Allocate device memory. - buf = std::make_shared>(mNbScaleBias); - - // Initialize values. - std::vector const values(mNbScaleBias, value); - PLUGIN_CUASSERT(cudaMemcpy(buf->mPtr, values.data(), sizeof(float) * mNbScaleBias, cudaMemcpyHostToDevice)); - } - }; - - allocScaleBias(mBnScales, 1.F); - allocScaleBias(mBnBias, 0.F); - return 0; + return STATUS_SUCCESS; } GroupNormalizationPlugin::GroupNormalizationPlugin(void const* data, size_t length) @@ -81,7 +56,6 @@ GroupNormalizationPlugin::GroupNormalizationPlugin(void const* data, size_t leng // Deserialize in the same order as serialization deserialize_value(&data, &length, &mEpsilon); deserialize_value(&data, &length, &mNbGroups); - deserialize_value(&data, &length, &mNbScaleBias); } char const* GroupNormalizationPlugin::getPluginType() const noexcept @@ -94,113 +68,153 @@ char const* GroupNormalizationPlugin::getPluginVersion() const noexcept return kGROUP_NORM_VERSION; } -int GroupNormalizationPlugin::getNbOutputs() const noexcept +int32_t GroupNormalizationPlugin::getNbOutputs() const noexcept { return 1; } nvinfer1::DimsExprs GroupNormalizationPlugin::getOutputDimensions( - int index, nvinfer1::DimsExprs const* inputs, int nbInputs, nvinfer1::IExprBuilder& exprBuilder) noexcept + int32_t index, nvinfer1::DimsExprs const* inputs, int32_t nbInputs, nvinfer1::IExprBuilder& exprBuilder) noexcept { - // Input (from previous layer), scale and bias are the three inputs to the plugin. - PLUGIN_ASSERT(nbInputs == 3); - PLUGIN_ASSERT(index == 0); - nvinfer1::DimsExprs output(inputs[0]); - return output; + try + { + // Input (from previous layer), scale and bias are the three inputs to the plugin. + PLUGIN_VALIDATE(nbInputs == 3); + PLUGIN_VALIDATE(index == 0); + return inputs[0]; + } + catch (std::exception const& e) + { + caughtError(e); + return DimsExprs{}; + } } void GroupNormalizationPlugin::attachToContext( cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept { - PLUGIN_ASSERT(cudnnContext); - _cudnn_handle = cudnnContext; - PLUGIN_CUDNNASSERT(cudnnCreateTensorDescriptor(&desc)); - PLUGIN_CUDNNASSERT(cudnnCreateTensorDescriptor(&bnDesc)); + try + { + PLUGIN_VALIDATE(cudnnContext); + mCudnnHandle = cudnnContext; + PLUGIN_CUDNNASSERT(cudnnCreateTensorDescriptor(&mTensorDesc)); + PLUGIN_CUDNNASSERT(cudnnCreateTensorDescriptor(&mBNTensorDesc)); + } + catch (std::exception const& e) + { + caughtError(e); + } } // Detach the plugin object from its execution context. void GroupNormalizationPlugin::detachFromContext() noexcept { - PLUGIN_CUDNNASSERT(cudnnDestroyTensorDescriptor(desc)); - PLUGIN_CUDNNASSERT(cudnnDestroyTensorDescriptor(bnDesc)); + try + { + PLUGIN_CUDNNASSERT(cudnnDestroyTensorDescriptor(mTensorDesc)); + PLUGIN_CUDNNASSERT(cudnnDestroyTensorDescriptor(mBNTensorDesc)); + mCudnnHandle = nullptr; + } + catch (std::exception const& e) + { + caughtError(e); + } } -int GroupNormalizationPlugin::enqueue(nvinfer1::PluginTensorDesc const* inputDesc, +int32_t GroupNormalizationPlugin::enqueue(nvinfer1::PluginTensorDesc const* inputDesc, nvinfer1::PluginTensorDesc const* outputDesc, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept { - // Get the input dimensions - nvinfer1::Dims input_dims = inputDesc[0].dims; - int batchSize = input_dims.d[0]; - int nbChannels = input_dims.d[1]; - - // Calculate size of each group - int groupSize = nbChannels / mNbGroups; - - mChannelVolume = pluginInternal::volume(input_dims, /*start*/ 2, /*stop*/ inputDesc[0].dims.nbDims); - - PLUGIN_CHECK_CUDNN(cudnnSetTensor4dDescriptor(desc, // descriptor - CUDNN_TENSOR_NCHW, // tensor format - CUDNN_DATA_FLOAT, // type - 1, // Batchsize - batchSize * mNbGroups, // Channels - groupSize, // Height - mChannelVolume // Width - )); + try + { + PLUGIN_VALIDATE(inputDesc != nullptr); + PLUGIN_VALIDATE(inputs != nullptr); + PLUGIN_VALIDATE(outputs != nullptr); + PLUGIN_VALIDATE(mBnScales != nullptr && mBnScales->mPtr != nullptr); + PLUGIN_VALIDATE(mBnBias != nullptr && mBnBias->mPtr != nullptr); + PLUGIN_VALIDATE(mCudnnHandle != nullptr); + PLUGIN_VALIDATE(mTensorDesc != nullptr); + PLUGIN_VALIDATE(mBNTensorDesc != nullptr); + } + catch (std::exception const& e) + { + caughtError(e); + return STATUS_FAILURE; + } - PLUGIN_CHECK_CUDNN(cudnnDeriveBNTensorDescriptor(bnDesc, desc, CUDNN_BATCHNORM_SPATIAL)); - PLUGIN_CHECK_CUDNN(cudnnSetStream(_cudnn_handle, stream)); - - // Reshape the data according in the cudnnSetTensor4dDescriptor. - PLUGIN_ASSERT(mBnScales && mBnScales->mPtr); - PLUGIN_ASSERT(mBnBias && mBnBias->mPtr); - float a = 1.F; - float b = 0.F; - PLUGIN_CHECK_CUDNN(cudnnBatchNormalizationForwardTraining(_cudnn_handle, // handle - CUDNN_BATCHNORM_SPATIAL, // BatchNormMode_t, try also non persistent - &a, // - &b, // - desc, // in/out descriptor - inputs[0], // input - desc, // in/out descriptor - outputs[0], // output - bnDesc, // - mBnScales->mPtr, // 1 - mBnBias->mPtr, // 0 - 0.0, // exponential average factor - nullptr, // resultRunningMean - nullptr, // resultRunningVar - mEpsilon, // eps - nullptr, // resultSaveMean - nullptr // resultSaveInvVar + PLUGIN_CHECK_CUDNN(cudnnSetStream(mCudnnHandle, stream)); + + // The tensor descriptors were set up in configurePlugin() to make Batch Normalization actually + // perform Group Normalization. This was done by setting the tensor descriptor shape to + // (1, batch*num_groups, channels_per_group, volume_of_spatial_dims). + // cudnnBatchNorm will normalize over the last two dimensions. + float const one = 1.F; + float const zero = 0.F; + PLUGIN_CHECK_CUDNN(cudnnBatchNormalizationForwardTraining(mCudnnHandle, // handle + CUDNN_BATCHNORM_SPATIAL, // BatchNormMode_t, try also non persistent + &one, // + &zero, // + mTensorDesc, // in/out descriptor + inputs[0], // input + mTensorDesc, // in/out descriptor + outputs[0], // output + mBNTensorDesc, // + mBnScales->mPtr, // 1 + mBnBias->mPtr, // 0 + 0.0, // exponential average factor + nullptr, // resultRunningMean + nullptr, // resultRunningVar + mEpsilon, // eps + nullptr, // resultSaveMean + nullptr // resultSaveInvVar )); - float* output = static_cast(outputs[0]); + // Apply an additional scale and bias on each channel. + nvinfer1::Dims inputDims = inputDesc[0].dims; + int32_t batchSize = inputDims.d[0]; + int32_t nbChannels = inputDims.d[1]; + auto* output = static_cast(outputs[0]); return scaleShiftChannelsInplace(output, batchSize, nbChannels, mChannelVolume, static_cast(inputs[2]), static_cast(inputs[1]), stream); // mBetaDev, mGammaDev, } size_t GroupNormalizationPlugin::getSerializationSize() const noexcept { - return sizeof(mNbGroups) + sizeof(mEpsilon) + sizeof(mNbScaleBias); + return sizeof(mNbGroups) + sizeof(mEpsilon); } void GroupNormalizationPlugin::serialize(void* buffer) const noexcept { + PLUGIN_ASSERT(buffer != nullptr); + auto* const start = reinterpret_cast(buffer); serialize_value(&buffer, mEpsilon); serialize_value(&buffer, mNbGroups); - serialize_value(&buffer, mNbScaleBias); + PLUGIN_ASSERT(start + getSerializationSize() == reinterpret_cast(buffer)); } bool GroupNormalizationPlugin::supportsFormatCombination( - int pos, nvinfer1::PluginTensorDesc const* inOut, int nbInputs, int nbOutputs) noexcept + int32_t pos, nvinfer1::PluginTensorDesc const* inOut, int32_t nbInputs, int32_t nbOutputs) noexcept { - PLUGIN_ASSERT(inOut && pos < (nbInputs + nbOutputs)); - return ((inOut[pos].type == nvinfer1::DataType::kFLOAT) && inOut[pos].format == nvinfer1::PluginFormat::kLINEAR - && inOut[pos].type == inOut[0].type); + try + { + PLUGIN_VALIDATE(inOut != nullptr); + PLUGIN_VALIDATE(pos < nbInputs + nbOutputs); + PLUGIN_VALIDATE(pos >= 0); + return ((inOut[pos].type == nvinfer1::DataType::kFLOAT) && inOut[pos].format == nvinfer1::PluginFormat::kLINEAR + && inOut[pos].type == inOut[0].type); + } + catch (std::exception const& e) + { + caughtError(e); + return false; + } } -void GroupNormalizationPlugin::terminate() noexcept {} +void GroupNormalizationPlugin::terminate() noexcept +{ + mBnScales.reset(); + mBnBias.reset(); +} void GroupNormalizationPlugin::destroy() noexcept { @@ -213,10 +227,11 @@ IPluginV2DynamicExt* GroupNormalizationPlugin::clone() const noexcept try { auto* plugin = new GroupNormalizationPlugin(mEpsilon, mNbGroups); - plugin->setPluginNamespace(mPluginNamespace); + plugin->setPluginNamespace(mNamespace.c_str()); plugin->mNbScaleBias = mNbScaleBias; plugin->mBnScales = mBnScales; plugin->mBnBias = mBnBias; + plugin->mChannelVolume = mChannelVolume; return plugin; } catch (std::exception const& e) @@ -226,34 +241,109 @@ IPluginV2DynamicExt* GroupNormalizationPlugin::clone() const noexcept return nullptr; } -void GroupNormalizationPlugin::configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int nbInputs, - nvinfer1::DynamicPluginTensorDesc const* out, int nbOutputs) noexcept +void GroupNormalizationPlugin::configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int32_t nbInputs, + nvinfer1::DynamicPluginTensorDesc const* out, int32_t nbOutputs) noexcept { - int32_t const batchSize = in[0].desc.dims.d[0] <= 0 ? in[0].max.d[0] : in[0].desc.dims.d[0]; - mNbScaleBias = batchSize * mNbGroups; + try + { + PLUGIN_VALIDATE(in != nullptr); + PLUGIN_VALIDATE(out != nullptr); + PLUGIN_VALIDATE(nbInputs == 3); + PLUGIN_VALIDATE(nbOutputs == getNbOutputs()); + + nvinfer1::Dims inputDims = in[0].desc.dims; + int32_t const batchSize = inputDims.d[0]; + int32_t const nbChannels = inputDims.d[1]; + + if (batchSize <= 0 || nbChannels <= 0) + { + // Input size not yet known, nothing to configure. + return; + } + + if (mTensorDesc == nullptr) + { + // Not yet attached to context. + return; + } + + // Allocate scale/bias tensors needed for cudnnBatchNorm. + mNbScaleBias = batchSize * mNbGroups; + auto allocScaleBias = [this](std::shared_ptr>& buf, float value) { + PLUGIN_VALIDATE(mNbScaleBias > 0); + if (!buf || !buf->mPtr || buf->mSize != mNbScaleBias) + { + // Allocate device memory. + buf = std::make_shared>(mNbScaleBias); + + // Initialize values. + std::vector const values(mNbScaleBias, value); + PLUGIN_CUASSERT( + cudaMemcpy(buf->mPtr, values.data(), sizeof(float) * mNbScaleBias, cudaMemcpyHostToDevice)); + } + }; + allocScaleBias(mBnScales, 1.F); + allocScaleBias(mBnBias, 0.F); + + // Calculate size of each group + int32_t groupSize = nbChannels / mNbGroups; + mChannelVolume = pluginInternal::volume(inputDims, /*start*/ 2, /*stop*/ inputDims.nbDims); + + // Set tensor descriptor in a way that cudnnBatchNorm will perform Group Normalization. + PLUGIN_CUDNNASSERT(cudnnSetTensor4dDescriptor(mTensorDesc, // descriptor + CUDNN_TENSOR_NCHW, // tensor format + CUDNN_DATA_FLOAT, // type + 1, // Batchsize + batchSize * mNbGroups, // Channels + groupSize, // Height + mChannelVolume // Width + )); + PLUGIN_CUDNNASSERT(cudnnDeriveBNTensorDescriptor(mBNTensorDesc, mTensorDesc, CUDNN_BATCHNORM_SPATIAL)); + } + catch (std::exception const& e) + { + caughtError(e); + } } nvinfer1::DataType GroupNormalizationPlugin::getOutputDataType( - int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { - PLUGIN_ASSERT(inputTypes && nbInputs > 0 && index == 0); - return inputTypes[0]; + try + { + PLUGIN_VALIDATE(inputTypes != nullptr); + PLUGIN_VALIDATE(index == 0); + return inputTypes[0]; + } + catch (std::exception const& e) + { + caughtError(e); + return DataType{}; + } } -size_t GroupNormalizationPlugin::getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int nbInputs, - nvinfer1::PluginTensorDesc const* outputs, int nbOutputs) const noexcept +size_t GroupNormalizationPlugin::getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int32_t nbInputs, + nvinfer1::PluginTensorDesc const* outputs, int32_t nbOutputs) const noexcept { return 0; } void GroupNormalizationPlugin::setPluginNamespace(char const* libNamespace) noexcept { - mPluginNamespace = libNamespace; + try + { + PLUGIN_VALIDATE(libNamespace != nullptr); + mNamespace = libNamespace; + } + catch (std::exception const& e) + { + caughtError(e); + } } char const* GroupNormalizationPlugin::getPluginNamespace() const noexcept { - return mPluginNamespace; + return mNamespace.c_str(); } GroupNormalizationPluginCreator::GroupNormalizationPluginCreator() @@ -288,7 +378,15 @@ char const* GroupNormalizationPluginCreator::getPluginNamespace() const noexcept void GroupNormalizationPluginCreator::setPluginNamespace(char const* libNamespace) noexcept { - mNamespace = libNamespace; + try + { + PLUGIN_VALIDATE(libNamespace != nullptr); + mNamespace = libNamespace; + } + catch (std::exception const& e) + { + caughtError(e); + } } IPluginV2DynamicExt* GroupNormalizationPluginCreator::createPlugin( @@ -296,19 +394,22 @@ IPluginV2DynamicExt* GroupNormalizationPluginCreator::createPlugin( { try { + PLUGIN_VALIDATE(fc != nullptr); + // Set default values - int nbGroups{1}; + int32_t nbGroups{1}; float epsilon{0.00001F}; - for (int i = 0; i < fc->nbFields; i++) + for (int32_t i = 0; i < fc->nbFields; i++) { - std::string field_name(fc->fields[i].name); - if (field_name.compare("eps") == 0) + PLUGIN_VALIDATE(fc->fields[i].name != nullptr); + std::string fieldName(fc->fields[i].name); + if (fieldName.compare("eps") == 0) { epsilon = *static_cast(fc->fields[i].data); } - if (field_name.compare("num_groups") == 0) + if (fieldName.compare("num_groups") == 0) { - nbGroups = *static_cast(fc->fields[i].data); + nbGroups = *static_cast(fc->fields[i].data); } } diff --git a/plugin/groupNormalizationPlugin/groupNormalizationPlugin.h b/plugin/groupNormalizationPlugin/groupNormalizationPlugin.h index 23d3e40d..001720aa 100644 --- a/plugin/groupNormalizationPlugin/groupNormalizationPlugin.h +++ b/plugin/groupNormalizationPlugin/groupNormalizationPlugin.h @@ -19,9 +19,8 @@ #define TRT_GROUP_NORM_PLUGIN_H #include "common/plugin.h" -#include "common/serialize.hpp" + #include -#include #include #include @@ -34,13 +33,13 @@ namespace plugin { template -cudaError_t scaleShiftChannelsInplace(T* inOut, int const B, int const C, int const channelVolume, float const* beta, - float const* gamma, cudaStream_t stream); +cudaError_t scaleShiftChannelsInplace(T* inOut, int32_t const B, int32_t const C, int32_t const channelVolume, + float const* beta, float const* gamma, cudaStream_t stream); class GroupNormalizationPlugin final : public nvinfer1::IPluginV2DynamicExt { public: - GroupNormalizationPlugin(float epsilon, int const nbGroups); + GroupNormalizationPlugin(float epsilon, int32_t const nbGroups); GroupNormalizationPlugin(void const* data, size_t length); @@ -48,20 +47,20 @@ class GroupNormalizationPlugin final : public nvinfer1::IPluginV2DynamicExt // delete default constructor. GroupNormalizationPlugin() = delete; - int getNbOutputs() const noexcept override; + int32_t getNbOutputs() const noexcept override; // DynamicExt plugins returns DimsExprs class instead of Dims - DimsExprs getOutputDimensions(int index, nvinfer1::DimsExprs const* inputs, int nbInputDims, + DimsExprs getOutputDimensions(int32_t index, nvinfer1::DimsExprs const* inputs, int32_t nbInputDims, nvinfer1::IExprBuilder& exprBuilder) noexcept override; - int initialize() noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; - size_t getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int nbInputs, - nvinfer1::PluginTensorDesc const* outputs, int nbOutputs) const noexcept override; + size_t getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int32_t nbInputs, + nvinfer1::PluginTensorDesc const* outputs, int32_t nbOutputs) const noexcept override; - int enqueue(nvinfer1::PluginTensorDesc const* inputDesc, nvinfer1::PluginTensorDesc const* outputDesc, + int32_t enqueue(nvinfer1::PluginTensorDesc const* inputDesc, nvinfer1::PluginTensorDesc const* outputDesc, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; size_t getSerializationSize() const noexcept override; @@ -69,7 +68,7 @@ class GroupNormalizationPlugin final : public nvinfer1::IPluginV2DynamicExt void serialize(void* buffer) const noexcept override; bool supportsFormatCombination( - int pos, nvinfer1::PluginTensorDesc const* inOut, int nbInputs, int nbOutputs) noexcept override; + int32_t pos, nvinfer1::PluginTensorDesc const* inOut, int32_t nbInputs, int32_t nbOutputs) noexcept override; char const* getPluginType() const noexcept override; @@ -79,7 +78,8 @@ class GroupNormalizationPlugin final : public nvinfer1::IPluginV2DynamicExt void destroy() noexcept override; - DataType getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + DataType getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; void attachToContext( cudnnContext* cudnn, cublasContext* cublas, nvinfer1::IGpuAllocator* allocator) noexcept override; @@ -90,21 +90,22 @@ class GroupNormalizationPlugin final : public nvinfer1::IPluginV2DynamicExt char const* getPluginNamespace() const noexcept override; - void configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int nbInputs, - nvinfer1::DynamicPluginTensorDesc const* out, int nbOutputs) noexcept override; + void configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int32_t nbInputs, + nvinfer1::DynamicPluginTensorDesc const* out, int32_t nbOutputs) noexcept override; private: - char const* mPluginNamespace; std::string mNamespace; float mEpsilon; - int mNbGroups; - int mChannelVolume; + int32_t mNbGroups; + int32_t mChannelVolume; + + cudnnHandle_t mCudnnHandle{}; - cudnnHandle_t _cudnn_handle; // Describes input and output. - cudnnTensorDescriptor_t desc; - cudnnTensorDescriptor_t bnDesc; + cudnnTensorDescriptor_t mTensorDesc{}; + cudnnTensorDescriptor_t mBNTensorDesc{}; + // These are buffers initialized to 1 and 0 respectively std::shared_ptr> mBnScales{}; std::shared_ptr> mBnBias{}; diff --git a/plugin/instanceNormalizationPlugin/instanceNormalizationPlugin.h b/plugin/instanceNormalizationPlugin/instanceNormalizationPlugin.h index d736f64c..f00a083c 100644 --- a/plugin/instanceNormalizationPlugin/instanceNormalizationPlugin.h +++ b/plugin/instanceNormalizationPlugin/instanceNormalizationPlugin.h @@ -25,7 +25,7 @@ #include #include -typedef unsigned short half_type; +typedef uint16_t half_type; namespace nvinfer1 { diff --git a/plugin/leakyReluPlugin/lReluPlugin.cpp b/plugin/leakyReluPlugin/lReluPlugin.cpp index 84346bc0..0fe1af5b 100644 --- a/plugin/leakyReluPlugin/lReluPlugin.cpp +++ b/plugin/leakyReluPlugin/lReluPlugin.cpp @@ -39,24 +39,24 @@ LReLU::LReLU(void const* buffer, size_t length) { char const *d = reinterpret_cast(buffer), *a = d; mNegSlope = read(d); - mBatchDim = read(d); + mBatchDim = read(d); PLUGIN_VALIDATE(d == a + length); } -int LReLU::getNbOutputs() const noexcept +int32_t LReLU::getNbOutputs() const noexcept { return 1; } -Dims LReLU::getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept +Dims LReLU::getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept { PLUGIN_ASSERT(nbInputDims == 1); PLUGIN_ASSERT(index == 0); return inputs[0]; } -int LReLU::enqueue( - int batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept +int32_t LReLU::enqueue( + int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept { void const* inputData = inputs[0]; void* outputData = outputs[0]; @@ -67,7 +67,7 @@ int LReLU::enqueue( size_t LReLU::getSerializationSize() const noexcept { // mNegSlope, mBatchDim - return sizeof(float) + sizeof(int); + return sizeof(float) + sizeof(int32_t); } void LReLU::serialize(void* buffer) const noexcept @@ -78,13 +78,13 @@ void LReLU::serialize(void* buffer) const noexcept PLUGIN_ASSERT(d == a + getSerializationSize()); } -void LReLU::configureWithFormat(Dims const* inputDims, int /* nbInputs */, Dims const* /* outputDims */, int nbOutputs, - DataType type, PluginFormat format, int) noexcept +void LReLU::configureWithFormat(Dims const* inputDims, int32_t /* nbInputs */, Dims const* /* outputDims */, + int32_t nbOutputs, DataType type, PluginFormat format, int32_t) noexcept { PLUGIN_ASSERT(type == DataType::kFLOAT && format == PluginFormat::kLINEAR); PLUGIN_ASSERT(mBatchDim == 1); PLUGIN_ASSERT(nbOutputs == 1); - for (int i = 0; i < inputDims[0].nbDims; ++i) + for (int32_t i = 0; i < inputDims[0].nbDims; ++i) { mBatchDim *= inputDims[0].d[i]; } @@ -95,14 +95,14 @@ bool LReLU::supportsFormat(DataType type, PluginFormat format) const noexcept return (type == DataType::kFLOAT && format == PluginFormat::kLINEAR); } -int LReLU::initialize() noexcept +int32_t LReLU::initialize() noexcept { return 0; } void LReLU::terminate() noexcept {} -size_t LReLU::getWorkspaceSize(int /* maxBatchSize */) const noexcept +size_t LReLU::getWorkspaceSize(int32_t /* maxBatchSize */) const noexcept { return 0; } diff --git a/plugin/leakyReluPlugin/lReluPlugin.h b/plugin/leakyReluPlugin/lReluPlugin.h index c4b68b1d..39c973b5 100644 --- a/plugin/leakyReluPlugin/lReluPlugin.h +++ b/plugin/leakyReluPlugin/lReluPlugin.h @@ -37,25 +37,25 @@ class LReLU : public nvinfer1::pluginInternal::BasePlugin ~LReLU() override = default; - int getNbOutputs() const noexcept override; + int32_t getNbOutputs() const noexcept override; - Dims getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept override; + Dims getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept override; - int initialize() noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; - size_t getWorkspaceSize(int maxBatchSize) const noexcept override; + size_t getWorkspaceSize(int32_t maxBatchSize) const noexcept override; - int enqueue(int batchSize, void const* const* inputs, void* const* outputs, void* workspace, + int32_t enqueue(int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; size_t getSerializationSize() const noexcept override; void serialize(void* buffer) const noexcept override; - void configureWithFormat(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, DataType type, - PluginFormat format, int maxBatchSize) noexcept override; + void configureWithFormat(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, + DataType type, PluginFormat format, int32_t maxBatchSize) noexcept override; bool supportsFormat(DataType type, PluginFormat format) const noexcept override; @@ -69,7 +69,7 @@ class LReLU : public nvinfer1::pluginInternal::BasePlugin private: float mNegSlope; - int mBatchDim; + int32_t mBatchDim; }; class LReluPluginCreator : public nvinfer1::pluginInternal::BaseCreator diff --git a/plugin/modulatedDeformConvPlugin/modulatedDeformConvPlugin.h b/plugin/modulatedDeformConvPlugin/modulatedDeformConvPlugin.h index 3e2d5687..53b653fa 100644 --- a/plugin/modulatedDeformConvPlugin/modulatedDeformConvPlugin.h +++ b/plugin/modulatedDeformConvPlugin/modulatedDeformConvPlugin.h @@ -55,7 +55,7 @@ class ModulatedDeformableConvPluginDynamic : public nvinfer1::IPluginV2DynamicEx ModulatedDeformableConvPluginDynamic() = delete; - ~ModulatedDeformableConvPluginDynamic(); + ~ModulatedDeformableConvPluginDynamic() override; nvinfer1::IPluginV2DynamicExt* clone() const noexcept override; nvinfer1::DimsExprs getOutputDimensions(int32_t outputIndex, nvinfer1::DimsExprs const* inputs, int32_t nbInputs, diff --git a/plugin/multilevelCropAndResizePlugin/multilevelCropAndResizePlugin.cpp b/plugin/multilevelCropAndResizePlugin/multilevelCropAndResizePlugin.cpp index 711d2fd8..3dc980b8 100644 --- a/plugin/multilevelCropAndResizePlugin/multilevelCropAndResizePlugin.cpp +++ b/plugin/multilevelCropAndResizePlugin/multilevelCropAndResizePlugin.cpp @@ -108,7 +108,7 @@ IPluginV2Ext* MultilevelCropAndResizePluginCreator::deserializePlugin( return nullptr; } -MultilevelCropAndResize::MultilevelCropAndResize(int pooled_size, nvinfer1::Dims const& imageSize) +MultilevelCropAndResize::MultilevelCropAndResize(int32_t pooled_size, nvinfer1::Dims const& imageSize) : mPooledSize({pooled_size, pooled_size}) { @@ -122,12 +122,12 @@ MultilevelCropAndResize::MultilevelCropAndResize(int pooled_size, nvinfer1::Dims mThresh = (224 * 224) / (4.0F); } -int MultilevelCropAndResize::getNbOutputs() const noexcept +int32_t MultilevelCropAndResize::getNbOutputs() const noexcept { return 1; } -int MultilevelCropAndResize::initialize() noexcept +int32_t MultilevelCropAndResize::initialize() noexcept { return 0; } @@ -139,7 +139,7 @@ void MultilevelCropAndResize::destroy() noexcept delete this; } -size_t MultilevelCropAndResize::getWorkspaceSize(int) const noexcept +size_t MultilevelCropAndResize::getWorkspaceSize(int32_t) const noexcept { return 0; } @@ -182,7 +182,7 @@ char const* MultilevelCropAndResize::getPluginNamespace() const noexcept return mNameSpace.c_str(); } -void MultilevelCropAndResize::check_valid_inputs(nvinfer1::Dims const* inputs, int nbInputDims) noexcept +void MultilevelCropAndResize::check_valid_inputs(nvinfer1::Dims const* inputs, int32_t nbInputDims) noexcept { // to be compatible with tensorflow node's input: // roi: [N, anchors, 4], @@ -193,7 +193,7 @@ void MultilevelCropAndResize::check_valid_inputs(nvinfer1::Dims const* inputs, i PLUGIN_ASSERT(rois.nbDims == 2); PLUGIN_ASSERT(rois.d[1] == 4); - for (int i = 1; i < nbInputDims; ++i) + for (int32_t i = 1; i < nbInputDims; ++i) { nvinfer1::Dims dims = inputs[i]; @@ -202,7 +202,7 @@ void MultilevelCropAndResize::check_valid_inputs(nvinfer1::Dims const* inputs, i } } -Dims MultilevelCropAndResize::getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept +Dims MultilevelCropAndResize::getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept { check_valid_inputs(inputs, nbInputDims); @@ -241,7 +241,8 @@ int32_t MultilevelCropAndResize::enqueue( size_t MultilevelCropAndResize::getSerializationSize() const noexcept { - return sizeof(int) * 2 + sizeof(int) * 4 + sizeof(float) + sizeof(int) * 2 * mFeatureMapCount + sizeof(DataType); + return sizeof(int32_t) * 2 + sizeof(int32_t) * 4 + sizeof(float) + sizeof(int32_t) * 2 * mFeatureMapCount + + sizeof(DataType); } void MultilevelCropAndResize::serialize(void* buffer) const noexcept @@ -254,7 +255,7 @@ void MultilevelCropAndResize::serialize(void* buffer) const noexcept write(d, mInputHeight); write(d, mInputWidth); write(d, mThresh); - for (int i = 0; i < mFeatureMapCount; i++) + for (int32_t i = 0; i < mFeatureMapCount; i++) { write(d, mFeatureSpatialSize[i].y); write(d, mFeatureSpatialSize[i].x); @@ -276,7 +277,7 @@ void MultilevelCropAndResize::deserialize(int8_t const* data, size_t length) mInputHeight = read(d); mInputWidth = read(d); mThresh = read(d); - for (int i = 0; i < mFeatureMapCount; i++) + for (int32_t i = 0; i < mFeatureMapCount; i++) { mFeatureSpatialSize[i].y = read(d); mFeatureSpatialSize[i].x = read(d); @@ -287,7 +288,7 @@ void MultilevelCropAndResize::deserialize(int8_t const* data, size_t length) // Return the DataType of the plugin output at the requested index DataType MultilevelCropAndResize::getOutputDataType( - int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { // Only DataType::kFLOAT is acceptable by the plugin layer // return DataType::kFLOAT; @@ -300,21 +301,21 @@ DataType MultilevelCropAndResize::getOutputDataType( // Return true if output tensor is broadcast across a batch. bool MultilevelCropAndResize::isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept { return false; } // Return true if plugin can use input that is broadcast across batch without replication. -bool MultilevelCropAndResize::canBroadcastInputAcrossBatch(int inputIndex) const noexcept +bool MultilevelCropAndResize::canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept { return false; } // Configure the layer with input and output data types. -void MultilevelCropAndResize::configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, - int nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept +void MultilevelCropAndResize::configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, + int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept { PLUGIN_ASSERT(supportsFormat(inputTypes[0], floatFormat)); check_valid_inputs(inputDims, nbInputs); diff --git a/plugin/multilevelCropAndResizePlugin/multilevelCropAndResizePlugin.h b/plugin/multilevelCropAndResizePlugin/multilevelCropAndResizePlugin.h index d67802f3..c2f615b4 100644 --- a/plugin/multilevelCropAndResizePlugin/multilevelCropAndResizePlugin.h +++ b/plugin/multilevelCropAndResizePlugin/multilevelCropAndResizePlugin.h @@ -36,23 +36,23 @@ namespace plugin class MultilevelCropAndResize : public IPluginV2Ext { public: - MultilevelCropAndResize(int pooled_size, nvinfer1::Dims const& image_size); + MultilevelCropAndResize(int32_t pooled_size, nvinfer1::Dims const& image_size); MultilevelCropAndResize(void const* data, size_t length); ~MultilevelCropAndResize() noexcept override = default; - int getNbOutputs() const noexcept override; + int32_t getNbOutputs() const noexcept override; - Dims getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept override; + Dims getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept override; - int initialize() noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; void destroy() noexcept override; - size_t getWorkspaceSize(int) const noexcept override; + size_t getWorkspaceSize(int32_t) const noexcept override; int32_t enqueue(int32_t batch_size, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; @@ -73,25 +73,26 @@ class MultilevelCropAndResize : public IPluginV2Ext char const* getPluginNamespace() const noexcept override; - DataType getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + DataType getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; bool isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept override; + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept override; - bool canBroadcastInputAcrossBatch(int inputIndex) const noexcept override; + bool canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept override; void attachToContext( cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept override; - void configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, + void configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept override; + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept override; void detachFromContext() noexcept override; private: void deserialize(int8_t const* data, size_t length); - void check_valid_inputs(nvinfer1::Dims const* inputs, int nbInputDims) noexcept; + void check_valid_inputs(nvinfer1::Dims const* inputs, int32_t nbInputDims) noexcept; xy_t mPooledSize{}; static const int32_t mFeatureMapCount = 5; // p2, p3, p4, p5, p6(Maxpooling) @@ -124,7 +125,7 @@ class MultilevelCropAndResizePluginCreator : public nvinfer1::pluginInternal::Ba private: static PluginFieldCollection mFC; - int mPooledSize; + int32_t mPooledSize; static std::vector mPluginAttributes; }; } // namespace plugin diff --git a/plugin/multilevelProposeROI/multilevelProposeROIPlugin.cpp b/plugin/multilevelProposeROI/multilevelProposeROIPlugin.cpp index 8fb0e426..d9ad8add 100644 --- a/plugin/multilevelProposeROI/multilevelProposeROIPlugin.cpp +++ b/plugin/multilevelProposeROI/multilevelProposeROIPlugin.cpp @@ -129,7 +129,7 @@ IPluginV2Ext* MultilevelProposeROIPluginCreator::deserializePlugin( } MultilevelProposeROI::MultilevelProposeROI( - int prenms_topk, int keep_topk, float fg_threshold, float iou_threshold, const nvinfer1::Dims imageSize) + int32_t prenms_topk, int32_t keep_topk, float fg_threshold, float iou_threshold, const nvinfer1::Dims imageSize) : mPreNMSTopK(prenms_topk) , mKeepTopK(keep_topk) , mFGThreshold(fg_threshold) @@ -137,7 +137,8 @@ MultilevelProposeROI::MultilevelProposeROI( , mImageSize(imageSize) { mBackgroundLabel = -1; - PLUGIN_VALIDATE(mPreNMSTopK > 0 && mPreNMSTopK <= 4096); + PLUGIN_VALIDATE(mPreNMSTopK > 0); + PLUGIN_VALIDATE(mPreNMSTopK <= 4096); PLUGIN_VALIDATE(mKeepTopK > 0); PLUGIN_VALIDATE(mIOUThreshold >= 0.0F); PLUGIN_VALIDATE(mFGThreshold >= 0.0F); @@ -157,12 +158,12 @@ MultilevelProposeROI::MultilevelProposeROI( generate_pyramid_anchors(mImageSize); } -int MultilevelProposeROI::getNbOutputs() const noexcept +int32_t MultilevelProposeROI::getNbOutputs() const noexcept { return 1; } -int MultilevelProposeROI::initialize() noexcept +int32_t MultilevelProposeROI::initialize() noexcept { // Init the regWeight [1, 1, 1, 1] mRegWeightDevice = std::make_shared>(4); @@ -171,22 +172,22 @@ int MultilevelProposeROI::initialize() noexcept sizeof(float) * 4, cudaMemcpyHostToDevice)); // Init the mValidCnt of max batch size - std::vector tempValidCnt(mMaxBatchSize, mPreNMSTopK); + std::vector tempValidCnt(mMaxBatchSize, mPreNMSTopK); - mValidCnt = std::make_shared>(mMaxBatchSize); + mValidCnt = std::make_shared>(mMaxBatchSize); - PLUGIN_CUASSERT(cudaMemcpy( - mValidCnt->mPtr, static_cast(tempValidCnt.data()), sizeof(int) * mMaxBatchSize, cudaMemcpyHostToDevice)); + PLUGIN_CUASSERT(cudaMemcpy(mValidCnt->mPtr, static_cast(tempValidCnt.data()), + sizeof(int32_t) * mMaxBatchSize, cudaMemcpyHostToDevice)); // Init the anchors for batch size: - for (int i = 0; i < mFeatureCnt; i++) + for (int32_t i = 0; i < mFeatureCnt; i++) { - int i_anchors_cnt = mAnchorsCnt[i]; + int32_t i_anchors_cnt = mAnchorsCnt[i]; auto i_anchors_host = mAnchorBoxesHost[i].data(); auto i_anchors_device = std::make_shared>(i_anchors_cnt * 4 * mMaxBatchSize); - int batch_offset = sizeof(float) * i_anchors_cnt * 4; + int32_t batch_offset = sizeof(float) * i_anchors_cnt * 4; uint8_t* device_ptr = static_cast(i_anchors_device->mPtr); - for (int i = 0; i < mMaxBatchSize; i++) + for (int32_t i = 0; i < mMaxBatchSize; i++) { PLUGIN_CUASSERT(cudaMemcpy(static_cast(device_ptr + i * batch_offset), static_cast(i_anchors_host), batch_offset, cudaMemcpyHostToDevice)); @@ -197,7 +198,7 @@ int MultilevelProposeROI::initialize() noexcept // Init the temp storage for proposals from feature maps before concat std::vector score_tp; std::vector box_tp; - for (int i = 0; i < mFeatureCnt; i++) + for (int32_t i = 0; i < mFeatureCnt; i++) { if (mType == DataType::kFLOAT) { @@ -276,7 +277,7 @@ char const* MultilevelProposeROI::getPluginNamespace() const noexcept size_t MultilevelProposeROI::getSerializationSize() const noexcept { - return sizeof(int) * 2 + sizeof(float) * 2 + sizeof(int) * (mFeatureCnt + 1) + sizeof(nvinfer1::Dims) + return sizeof(int32_t) * 2 + sizeof(float) * 2 + sizeof(int32_t) * (mFeatureCnt + 1) + sizeof(nvinfer1::Dims) + sizeof(DataType); } @@ -288,7 +289,7 @@ void MultilevelProposeROI::serialize(void* buffer) const noexcept write(d, mFGThreshold); write(d, mIOUThreshold); write(d, mMaxBatchSize); - for (int i = 0; i < mFeatureCnt; i++) + for (int32_t i = 0; i < mFeatureCnt; i++) { write(d, mAnchorsCnt[i]); } @@ -302,15 +303,15 @@ MultilevelProposeROI::MultilevelProposeROI(void const* data, size_t length) mFeatureCnt = TLTMaskRCNNConfig::MAX_LEVEL - TLTMaskRCNNConfig::MIN_LEVEL + 1; char const *d = reinterpret_cast(data), *a = d; - int prenms_topk = read(d); - int keep_topk = read(d); + int32_t prenms_topk = read(d); + int32_t keep_topk = read(d); float fg_threshold = read(d); float iou_threshold = read(d); - mMaxBatchSize = read(d); + mMaxBatchSize = read(d); PLUGIN_VALIDATE(mAnchorsCnt.size() == 0); - for (int i = 0; i < mFeatureCnt; i++) + for (int32_t i = 0; i < mFeatureCnt; i++) { - mAnchorsCnt.push_back(read(d)); + mAnchorsCnt.push_back(read(d)); } mImageSize = read(d); mType = read(d); @@ -331,14 +332,14 @@ MultilevelProposeROI::MultilevelProposeROI(void const* data, size_t length) generate_pyramid_anchors(mImageSize); } -void MultilevelProposeROI::check_valid_inputs(nvinfer1::Dims const* inputs, int nbInputDims) noexcept +void MultilevelProposeROI::check_valid_inputs(nvinfer1::Dims const* inputs, int32_t nbInputDims) noexcept { // x=2,3,4,5,6 // foreground_delta_px [N, h_x * w_x * anchors_per_location, 4, 1], // foreground_score_px [N, h_x * w_x * anchors_per_location, 1, 1], // anchors should be generated inside PLUGIN_ASSERT(nbInputDims == 2 * mFeatureCnt); - for (int i = 0; i < 2 * mFeatureCnt; i += 2) + for (int32_t i = 0; i < 2 * mFeatureCnt; i += 2) { // foreground_delta PLUGIN_ASSERT(inputs[i].nbDims == 3 && inputs[i].d[1] == 4); @@ -347,13 +348,13 @@ void MultilevelProposeROI::check_valid_inputs(nvinfer1::Dims const* inputs, int } } -size_t MultilevelProposeROI::getWorkspaceSize(int batch_size) const noexcept +size_t MultilevelProposeROI::getWorkspaceSize(int32_t batch_size) const noexcept { size_t total_size = 0; PLUGIN_ASSERT(mAnchorsCnt.size() == static_cast(mFeatureCnt)); // workspace for propose on each feature map - for (int i = 0; i < mFeatureCnt; i++) + for (int32_t i = 0; i < mFeatureCnt; i++) { MultilevelProposeROIWorkSpace proposal(batch_size, mAnchorsCnt[i], mPreNMSTopK, mParam, mType); @@ -367,7 +368,7 @@ size_t MultilevelProposeROI::getWorkspaceSize(int batch_size) const noexcept return total_size; } -Dims MultilevelProposeROI::getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept +Dims MultilevelProposeROI::getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept { check_valid_inputs(inputs, nbInputDims); @@ -387,10 +388,10 @@ void MultilevelProposeROI::generate_pyramid_anchors(nvinfer1::Dims const& imageS // Generate anchors strides and scales std::vector anchor_scales; - std::vector anchor_strides; - for (int i = min_level; i < max_level + 1; i++) + std::vector anchor_strides; + for (int32_t i = min_level; i < max_level + 1; i++) { - int stride = static_cast(pow(2.0, i)); + int32_t stride = static_cast(pow(2.0, i)); anchor_strides.push_back(stride); anchor_scales.push_back(stride * anchor_scale); } @@ -402,11 +403,11 @@ void MultilevelProposeROI::generate_pyramid_anchors(nvinfer1::Dims const& imageS for (size_t s = 0; s < anchor_scales.size(); ++s) { float scale = anchor_scales[s]; - int stride = anchor_strides[s]; + int32_t stride = anchor_strides[s]; std::vector s_anchors; - for (int y = stride / 2; y < image_dims.d[1]; y += stride) - for (int x = stride / 2; x < image_dims.d[2]; x += stride) + for (int32_t y = stride / 2; y < image_dims.d[1]; y += stride) + for (int32_t x = stride / 2; x < image_dims.d[2]; x += stride) for (auto r : aspect_ratios) { float h = scale * r.second; @@ -433,7 +434,7 @@ int32_t MultilevelProposeROI::enqueue( std::vector mTempScores; std::vector mTempBboxes; - for (int i = 0; i < mFeatureCnt; i++) + for (int32_t i = 0; i < mFeatureCnt; i++) { if (mType == DataType::kFLOAT) { @@ -447,7 +448,7 @@ int32_t MultilevelProposeROI::enqueue( } } - for (int i = 0; i < mFeatureCnt; i++) + for (int32_t i = 0; i < mFeatureCnt; i++) { MultilevelProposeROIWorkSpace proposal_ws(batch_size, mAnchorsCnt[i], mPreNMSTopK, mParam, mType); status = MultilevelPropose(stream, batch_size, mAnchorsCnt[i], mPreNMSTopK, @@ -477,7 +478,7 @@ int32_t MultilevelProposeROI::enqueue( // Return the DataType of the plugin output at the requested index DataType MultilevelProposeROI::getOutputDataType( - int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { // Only DataType::kFLOAT is acceptable by the plugin layer if ((inputTypes[0] == DataType::kFLOAT) || (inputTypes[0] == DataType::kHALF)) @@ -487,29 +488,29 @@ DataType MultilevelProposeROI::getOutputDataType( // Return true if output tensor is broadcast across a batch. bool MultilevelProposeROI::isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept { return false; } // Return true if plugin can use input that is broadcast across batch without replication. -bool MultilevelProposeROI::canBroadcastInputAcrossBatch(int inputIndex) const noexcept +bool MultilevelProposeROI::canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept { return false; } // Configure the layer with input and output data types. -void MultilevelProposeROI::configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, - DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept +void MultilevelProposeROI::configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, + int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept { check_valid_inputs(inputDims, nbInputs); mAnchorsCnt.clear(); - for (int i = 0; i < mFeatureCnt; i++) + for (int32_t i = 0; i < mFeatureCnt; i++) { mAnchorsCnt.push_back(inputDims[2 * i].d[0]); - PLUGIN_ASSERT(mAnchorsCnt[i] == (int) (mAnchorBoxesHost[i].size() / 4)); + PLUGIN_ASSERT(mAnchorsCnt[i] == (int32_t) (mAnchorBoxesHost[i].size() / 4)); } mMaxBatchSize = maxBatchSize; diff --git a/plugin/multilevelProposeROI/multilevelProposeROIPlugin.h b/plugin/multilevelProposeROI/multilevelProposeROIPlugin.h index 31277758..653958e6 100644 --- a/plugin/multilevelProposeROI/multilevelProposeROIPlugin.h +++ b/plugin/multilevelProposeROI/multilevelProposeROIPlugin.h @@ -35,24 +35,24 @@ namespace plugin class MultilevelProposeROI : public IPluginV2Ext { public: - MultilevelProposeROI( - int prenms_topk, int keep_topk, float fg_threshold, float iou_threshold, const nvinfer1::Dims image_size); + MultilevelProposeROI(int32_t prenms_topk, int32_t keep_topk, float fg_threshold, float iou_threshold, + const nvinfer1::Dims image_size); MultilevelProposeROI(void const* data, size_t length); ~MultilevelProposeROI() noexcept override = default; - int getNbOutputs() const noexcept override; + int32_t getNbOutputs() const noexcept override; - Dims getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept override; + Dims getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept override; - int initialize() noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; void destroy() noexcept override; - size_t getWorkspaceSize(int maxBatchSize) const noexcept override; + size_t getWorkspaceSize(int32_t maxBatchSize) const noexcept override; int32_t enqueue(int32_t batch_size, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; @@ -73,36 +73,37 @@ class MultilevelProposeROI : public IPluginV2Ext char const* getPluginNamespace() const noexcept override; - DataType getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + DataType getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; bool isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept override; + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept override; - bool canBroadcastInputAcrossBatch(int inputIndex) const noexcept override; + bool canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept override; void attachToContext( cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept override; - void configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, + void configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept override; + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept override; void detachFromContext() noexcept override; private: - void check_valid_inputs(nvinfer1::Dims const* inputs, int nbInputDims) noexcept; + void check_valid_inputs(nvinfer1::Dims const* inputs, int32_t nbInputDims) noexcept; void generate_pyramid_anchors(nvinfer1::Dims const& imageSize); - int mBackgroundLabel; - int mPreNMSTopK; - int mKeepTopK; - int mFeatureCnt; + int32_t mBackgroundLabel; + int32_t mPreNMSTopK; + int32_t mKeepTopK; + int32_t mFeatureCnt; float mFGThreshold; float mIOUThreshold; - int mMaxBatchSize; - std::vector mAnchorsCnt; - std::shared_ptr> mValidCnt; // valid cnt = number of input roi for every image. + int32_t mMaxBatchSize; + std::vector mAnchorsCnt; + std::shared_ptr> mValidCnt; // valid cnt = number of input roi for every image. std::vector>> mAnchorBoxesDevice; // [N, anchors(261888 for resnet101 + 1024*1024), (y1, x1, y2, x2)] std::vector> mAnchorBoxesHost; @@ -140,8 +141,8 @@ class MultilevelProposeROIPluginCreator : public nvinfer1::pluginInternal::BaseC private: static PluginFieldCollection mFC; - int mPreNMSTopK; - int mKeepTopK; + int32_t mPreNMSTopK; + int32_t mKeepTopK; float mFGThreshold; float mIOUThreshold; static std::vector mPluginAttributes; diff --git a/plugin/multilevelProposeROI/tlt_mrcnn_config.h b/plugin/multilevelProposeROI/tlt_mrcnn_config.h index c3f6b345..13c8abfe 100644 --- a/plugin/multilevelProposeROI/tlt_mrcnn_config.h +++ b/plugin/multilevelProposeROI/tlt_mrcnn_config.h @@ -27,8 +27,8 @@ namespace TLTMaskRCNNConfig static const nvinfer1::Dims3 IMAGE_SHAPE{3, 832, 1344}; // Pooled ROIs -static int const POOL_SIZE = 7; -static int const MASK_POOL_SIZE = 14; +static int32_t const POOL_SIZE = 7; +static int32_t const MASK_POOL_SIZE = 14; // Threshold to determine the mask area out of final convolution output static float const MASK_THRESHOLD = 0.5; @@ -37,7 +37,7 @@ static float const MASK_THRESHOLD = 0.5; static float const DETECTION_REG_WEIGHTS[] = {10, 10, 5, 5}; // Max number of final detections -static int const DETECTION_MAX_INSTANCES = 100; +static int32_t const DETECTION_MAX_INSTANCES = 100; // Minimum probability value to accept a detected instance // ROIs below this threshold are skipped @@ -47,18 +47,18 @@ static float const DETECTION_MIN_CONFIDENCE = 0; static float const DETECTION_NMS_THRESHOLD = 0.5; // Size of the fully-connected layers in the classification graph -static int const FPN_CLASSIF_FC_LAYERS_SIZE = 1024; +static int32_t const FPN_CLASSIF_FC_LAYERS_SIZE = 1024; // Size of the top-down layers used to build the feature pyramid -static int const TOP_DOWN_PYRAMID_SIZE = 256; +static int32_t const TOP_DOWN_PYRAMID_SIZE = 256; // Number of classification classes (including background) -static int const NUM_CLASSES = 1 + 90; +static int32_t const NUM_CLASSES = 1 + 90; // Min and max level of fpn feature pyramids: // p2, p3, p4, p5, p6. -static int const MIN_LEVEL = 2; -static int const MAX_LEVEL = 6; +static int32_t const MIN_LEVEL = 2; +static int32_t const MAX_LEVEL = 6; // Length of minimum square anchor side in pixels static float const RPN_ANCHOR_SCALE = 8; @@ -70,17 +70,17 @@ static const std::vector> ANCHOR_RATIOS // Anchor stride // If 1 then anchors are created for each cell in the backbone feature map. // If 2, then anchors are created for every other cell, and so on. -static int const RPN_ANCHOR_STRIDE = 1; +static int32_t const RPN_ANCHOR_STRIDE = 1; // TRT fails if this number larger than kMAX_TOPK_K defined in engine/checkMacros.h -static int const MAX_PRE_NMS_RESULTS = 1000; // 3840; +static int32_t const MAX_PRE_NMS_RESULTS = 1000; // 3840; // Non-max suppression threshold to filter RPN proposals. // You can increase this during training to generate more propsals. static float const RPN_NMS_THRESHOLD = 0.7F; // ROIs kept after non-maximum suppression (training and inference) -static int const POST_NMS_ROIS_INFERENCE = 1000; +static int32_t const POST_NMS_ROIS_INFERENCE = 1000; // COCO Class names static const std::vector CLASS_NAMES = { diff --git a/plugin/multiscaleDeformableAttnPlugin/multiscaleDeformableAttn.cu b/plugin/multiscaleDeformableAttnPlugin/multiscaleDeformableAttn.cu index 8b923d2f..d6843c64 100644 --- a/plugin/multiscaleDeformableAttnPlugin/multiscaleDeformableAttn.cu +++ b/plugin/multiscaleDeformableAttnPlugin/multiscaleDeformableAttn.cu @@ -36,8 +36,8 @@ #include "multiscaleDeformableIm2ColCuda.cuh" -int32_t ms_deform_attn_cuda_forward(cudaStream_t stream, const float* value, const int32_t* spatialShapes, - const int32_t* levelStartIndex, const float* samplingLoc, const float* attnWeight, float* output, int32_t batch, +int32_t ms_deform_attn_cuda_forward(cudaStream_t stream, float const* value, int32_t const* spatialShapes, + int32_t const* levelStartIndex, float const* samplingLoc, float const* attnWeight, float* output, int32_t batch, int32_t mSpatialSize, int32_t mNumHeads, int32_t mChannels, int32_t mNumLevels, int32_t mNumQuery, int32_t mNumPoint) { auto perValueSize = mSpatialSize * mNumHeads * mChannels; @@ -57,8 +57,8 @@ int32_t ms_deform_attn_cuda_forward(cudaStream_t stream, const float* value, con return 0; } -int32_t ms_deform_attn_cuda_forward(cudaStream_t stream, const __half* value, const int32_t* spatialShapes, - const int32_t* levelStartIndex, const __half* samplingLoc, const __half* attnWeight, __half* output, int32_t batch, +int32_t ms_deform_attn_cuda_forward(cudaStream_t stream, __half const* value, int32_t const* spatialShapes, + int32_t const* levelStartIndex, __half const* samplingLoc, __half const* attnWeight, __half* output, int32_t batch, int32_t mSpatialSize, int32_t mNumHeads, int32_t mChannels, int32_t mNumLevels, int32_t mNumQuery, int32_t mNumPoint) { auto perValueSize = mSpatialSize * mNumHeads * mChannels; diff --git a/plugin/multiscaleDeformableAttnPlugin/multiscaleDeformableAttnPlugin.cpp b/plugin/multiscaleDeformableAttnPlugin/multiscaleDeformableAttnPlugin.cpp index 81f7afd3..939590ae 100644 --- a/plugin/multiscaleDeformableAttnPlugin/multiscaleDeformableAttnPlugin.cpp +++ b/plugin/multiscaleDeformableAttnPlugin/multiscaleDeformableAttnPlugin.cpp @@ -44,7 +44,7 @@ nvinfer1::IPluginV2DynamicExt* MultiscaleDeformableAttnPlugin::clone() const PLU plugin->setPluginNamespace(getPluginNamespace()); return plugin; } - catch (const std::exception& e) + catch (std::exception const& e) { caughtError(e); } @@ -140,11 +140,11 @@ int32_t MultiscaleDeformableAttnPlugin::enqueue(nvinfer1::PluginTensorDesc const } else if (inputDesc[0].type == nvinfer1::DataType::kHALF) { - const __half* value = static_cast(inputs[0]); + __half const* value = static_cast<__half const*>(inputs[0]); int32_t const* spatialShapes = static_cast(inputs[1]); int32_t const* levelStartIndex = static_cast(inputs[2]); - const __half* samplingLoc = static_cast(inputs[3]); - const __half* attnWeight = static_cast(inputs[4]); + __half const* samplingLoc = static_cast<__half const*>(inputs[3]); + __half const* attnWeight = static_cast<__half const*>(inputs[4]); __half* output = static_cast<__half*>(outputs[0]); rc = ms_deform_attn_cuda_forward(stream, value, spatialShapes, levelStartIndex, samplingLoc, attnWeight, output, @@ -244,7 +244,7 @@ IPluginV2* MultiscaleDeformableAttnPluginCreator::createPlugin( MultiscaleDeformableAttnPlugin* plugin = new MultiscaleDeformableAttnPlugin(); return plugin; } - catch (const std::exception& e) + catch (std::exception const& e) { caughtError(e); } @@ -260,7 +260,7 @@ IPluginV2* MultiscaleDeformableAttnPluginCreator::deserializePlugin( plugin->setPluginNamespace(getPluginNamespace()); return plugin; } - catch (const std::exception& e) + catch (std::exception const& e) { caughtError(e); } diff --git a/plugin/multiscaleDeformableAttnPlugin/multiscaleDeformableIm2ColCuda.cuh b/plugin/multiscaleDeformableAttnPlugin/multiscaleDeformableIm2ColCuda.cuh index eaa670fd..370c4cd1 100644 --- a/plugin/multiscaleDeformableAttnPlugin/multiscaleDeformableIm2ColCuda.cuh +++ b/plugin/multiscaleDeformableAttnPlugin/multiscaleDeformableIm2ColCuda.cuh @@ -39,7 +39,7 @@ #define CUDA_KERNEL_LOOP(i, n) for (int32_t i = blockIdx.x * blockDim.x + threadIdx.x; i < (n); i += blockDim.x * gridDim.x) -constexpr int32_t CUDA_NUM_THREADS = 768; +constexpr int32_t kCUDA_NUM_THREADS{768}; inline int32_t GET_BLOCKS(int32_t const N, int32_t const numThreads) { return (N + numThreads - 1) / numThreads; @@ -98,26 +98,26 @@ __device__ scalar_t ms_deform_attn_im2col_bilinear(scalar_t const*& bottomData, } template <> -__device__ __half ms_deform_attn_im2col_bilinear<__half>(const __half*& bottomData, int32_t const& height, int32_t const& width, - int32_t const& nHeads, int32_t const& channels, const __half& h, const __half& w, int32_t const& m, int32_t const& c) +__device__ __half ms_deform_attn_im2col_bilinear<__half>(__half const*& bottomData, int32_t const& height, int32_t const& width, + int32_t const& nHeads, int32_t const& channels, __half const& h, __half const& w, int32_t const& m, int32_t const& c) { int32_t const hLow = __half2int_rd(h); int32_t const wLow = __half2int_rd(w); int32_t const hHigh = hLow + 1; int32_t const wHigh = wLow + 1; - const __half kZERO = __int2half_rz(0); - const __half one = __int2half_rz(1); + __half const kZERO = __int2half_rz(0); + __half const one = __int2half_rz(1); #if __CUDA_ARCH__>=530 - const __half lh = __hsub(h, __int2half_rd(hLow)); - const __half lw = __hsub(w, __int2half_rd(wLow)); - const __half hh = __hsub(one, lh), hw = __hsub(one, lw); + __half const lh = __hsub(h, __int2half_rd(hLow)); + __half const lw = __hsub(w, __int2half_rd(wLow)); + __half const hh = __hsub(one, lh), hw = __hsub(one, lw); #else - const __half lh = __float2half(__half2float(h) - hLow); - const __half lw = __float2half(__half2float(w) - wLow); - const __half hh = __float2half(__half2float(one) - __half2float(lh)); - const __half hw = __float2half(__half2float(one) - __half2float(lw)); + __half const lh = __float2half(__half2float(h) - hLow); + __half const lw = __float2half(__half2float(w) - wLow); + __half const hh = __float2half(__half2float(one) - __half2float(lh)); + __half const hw = __float2half(__half2float(one) - __half2float(lw)); #endif int32_t const wStride = nHeads * channels; int32_t const hStride = width * wStride; @@ -161,7 +161,7 @@ __device__ __half ms_deform_attn_im2col_bilinear<__half>(const __half*& bottomDa w1 = __hadd(w1, w2); w3 = __hadd(w3, w4); - const __half val = __hadd(w1, w3); + __half const val = __hadd(w1, w3); #else __half w1 = __float2half((__half2float(hh) * __half2float(hw)) * __half2float(v1)); __half w2 = __float2half((__half2float(hh) * __half2float(lw)) * __half2float(v2)); @@ -171,7 +171,7 @@ __device__ __half ms_deform_attn_im2col_bilinear<__half>(const __half*& bottomDa w1 = __float2half(__half2float(w1) + __half2float(w2)); w3 = __float2half(__half2float(w3) + __half2float(w4)); - const __half val = __float2half(__half2float(w1) + __half2float(w3)); + __half const val = __float2half(__half2float(w1) + __half2float(w3)); #endif return val; } @@ -373,9 +373,9 @@ __global__ void ms_deformable_im2col_gpu_kernel(int32_t const n, scalar_t const* } template <> -__global__ void ms_deformable_im2col_gpu_kernel<__half>(int32_t const n, const __half* dataValue, - int32_t const* dataSpatialShapes, int32_t const* dataLevelStartIndex, const __half* dataSamplingLoc, - const __half* dataAttnWeight, int32_t const batchSize, int32_t const spatialSize, int32_t const numHeads, int32_t const channels, +__global__ void ms_deformable_im2col_gpu_kernel<__half>(int32_t const n, __half const* dataValue, + int32_t const* dataSpatialShapes, int32_t const* dataLevelStartIndex, __half const* dataSamplingLoc, + __half const* dataAttnWeight, int32_t const batchSize, int32_t const spatialSize, int32_t const numHeads, int32_t const channels, int32_t const numLevels, int32_t const numQuery, int32_t const numPoint, __half* dataCol) { CUDA_KERNEL_LOOP(index, n) @@ -394,9 +394,9 @@ __global__ void ms_deformable_im2col_gpu_kernel<__half>(int32_t const n, const _ int32_t dataLocWPtr = dataWeightPtr << 1; int32_t const qidStride = numHeads * channels; int32_t const dataValuePtrInitOffset = bCol * spatialSize * qidStride; - const __half kZERO_POINT_FIVE = __float2half(0.5f); - const __half kMINUS_ONE = __float2half(-1.0f); - const __half kZERO = __int2half_rz(0); + __half const kZERO_POINT_FIVE = __float2half(0.5f); + __half const kMINUS_ONE = __float2half(-1.0f); + __half const kZERO = __int2half_rz(0); __half tpVal = kZERO; __half col = kZERO; @@ -406,17 +406,17 @@ __global__ void ms_deformable_im2col_gpu_kernel<__half>(int32_t const n, const _ int32_t const spatialHPtr = lCol << 1; int32_t const spatialH = dataSpatialShapes[spatialHPtr]; int32_t const spatialW = dataSpatialShapes[spatialHPtr + 1]; - const __half spatialHHalf = __int2half_rd(spatialH); - const __half spatialWHalf = __int2half_rd(spatialW); - const __half* dataValuePtr = dataValue + (dataValuePtrInitOffset + levelStartId * qidStride); + __half const spatialHHalf = __int2half_rd(spatialH); + __half const spatialWHalf = __int2half_rd(spatialW); + __half const* dataValuePtr = dataValue + (dataValuePtrInitOffset + levelStartId * qidStride); for (int32_t pCol = 0; pCol < numPoint; ++pCol) { - const __half locW = dataSamplingLoc[dataLocWPtr]; - const __half locH = dataSamplingLoc[dataLocWPtr + 1]; - const __half weight = dataAttnWeight[dataWeightPtr]; + __half const locW = dataSamplingLoc[dataLocWPtr]; + __half const locH = dataSamplingLoc[dataLocWPtr + 1]; + __half const weight = dataAttnWeight[dataWeightPtr]; #if __CUDA_ARCH__ >= 530 - const __half hIm = __hsub(__hmul(locH, spatialHHalf), kZERO_POINT_FIVE); - const __half wIm = __hsub(__hmul(locW, spatialWHalf), kZERO_POINT_FIVE); + __half const hIm = __hsub(__hmul(locH, spatialHHalf), kZERO_POINT_FIVE); + __half const wIm = __hsub(__hmul(locW, spatialWHalf), kZERO_POINT_FIVE); if (__hgt(hIm, kMINUS_ONE) && __hgt(wIm, kMINUS_ONE) && __hlt(hIm, spatialHHalf) && __hlt(wIm, spatialWHalf)) @@ -426,8 +426,8 @@ __global__ void ms_deformable_im2col_gpu_kernel<__half>(int32_t const n, const _ col = __hadd(col, __hmul(tpVal, weight)); } #else - const __half hIm = __float2half(__half2float(locH) * __half2float(spatialHHalf) - __half2float(kZERO_POINT_FIVE)); - const __half wIm = __float2half(__half2float(locW) * __half2float(spatialWHalf) - __half2float(kZERO_POINT_FIVE)); + __half const hIm = __float2half(__half2float(locH) * __half2float(spatialHHalf) - __half2float(kZERO_POINT_FIVE)); + __half const wIm = __float2half(__half2float(locW) * __half2float(spatialWHalf) - __half2float(kZERO_POINT_FIVE)); if((__half2float(hIm)>__half2float(kMINUS_ONE)) && (__half2float(wIm)>__half2float(kMINUS_ONE)) && (__half2float(hIm)<__half2float(spatialHHalf)) && (__half2float(wIm)<__half2float(spatialWHalf))) @@ -996,7 +996,7 @@ void ms_deformable_im2col_cuda(cudaStream_t stream, scalar_t const* dataValue, i { int32_t const numKernels = batchSize * numQuery * numHeads * channels; int32_t const numActualKernels = batchSize * numQuery * numHeads * channels; - int32_t const numThreads = CUDA_NUM_THREADS; + int32_t const numThreads = kCUDA_NUM_THREADS; cudaError_t err = cudaSuccess; ms_deformable_im2col_gpu_kernel<<>>( @@ -1016,7 +1016,7 @@ void ms_deformable_col2im_cuda(cudaStream_t stream, scalar_t const* grad_col, sc int32_t const numLevels, int32_t const numQuery, int32_t const numPoint, scalar_t* gradValue, scalar_t* gradSamplingLoc, scalar_t* gradAttnWeight) { - int32_t const numThreads = (channels > CUDA_NUM_THREADS) ? CUDA_NUM_THREADS : channels; + int32_t const numThreads = (channels > kCUDA_NUM_THREADS) ? kCUDA_NUM_THREADS : channels; int32_t const numKernels = batchSize * numQuery * numHeads * channels; int32_t const numActualKernels = batchSize * numQuery * numHeads * channels; if (channels > 1024) @@ -1157,7 +1157,7 @@ __global__ void float2half_input(int32_t const nData1, int32_t const nData2, int } } -__global__ void half2float_output(int32_t const n_data, const __half* data_half, float* data_float) +__global__ void half2float_output(int32_t const n_data, __half const* data_half, float* data_float) { CUDA_KERNEL_LOOP(index, n_data) { diff --git a/plugin/nmsPlugin/nmsPlugin.cpp b/plugin/nmsPlugin/nmsPlugin.cpp index 40f00ccc..14a5e2eb 100644 --- a/plugin/nmsPlugin/nmsPlugin.cpp +++ b/plugin/nmsPlugin/nmsPlugin.cpp @@ -54,7 +54,7 @@ DetectionOutputDynamic::DetectionOutputDynamic(DetectionOutputParameters params) { } -DetectionOutput::DetectionOutput(DetectionOutputParameters params, int C1, int C2, int numPriors) +DetectionOutput::DetectionOutput(DetectionOutputParameters params, int32_t C1, int32_t C2, int32_t numPriors) : param(params) , C1(C1) , C2(C2) @@ -64,7 +64,8 @@ DetectionOutput::DetectionOutput(DetectionOutputParameters params, int C1, int C { } -DetectionOutputDynamic::DetectionOutputDynamic(DetectionOutputParameters params, int C1, int C2, int numPriors) +DetectionOutputDynamic::DetectionOutputDynamic( + DetectionOutputParameters params, int32_t C1, int32_t C2, int32_t numPriors) : param(params) , C1(C1) , C2(C2) @@ -81,12 +82,12 @@ DetectionOutput::DetectionOutput(void const* data, size_t length) param = read(d); // Channel size of the locData tensor // numPriors * numLocClasses * 4 - C1 = read(d); + C1 = read(d); // Channel size of the confData tensor // numPriors * param.numClasses - C2 = read(d); + C2 = read(d); // Number of bounding boxes per sample - numPriors = read(d); + numPriors = read(d); // data type of this plugin mType = read(d); // mScoreBits @@ -100,12 +101,12 @@ DetectionOutputDynamic::DetectionOutputDynamic(void const* data, size_t length) param = read(d); // Channel size of the locData tensor // numPriors * numLocClasses * 4 - C1 = read(d); + C1 = read(d); // Channel size of the confData tensor // numPriors * param.numClasses - C2 = read(d); + C2 = read(d); // Number of bounding boxes per sample - numPriors = read(d); + numPriors = read(d); // data type of this plugin mType = read(d); // mScoreBits @@ -113,24 +114,24 @@ DetectionOutputDynamic::DetectionOutputDynamic(void const* data, size_t length) PLUGIN_VALIDATE(d == a + length); } -int DetectionOutput::getNbOutputs() const noexcept +int32_t DetectionOutput::getNbOutputs() const noexcept { // Plugin layer has 2 outputs return 2; } -int DetectionOutputDynamic::getNbOutputs() const noexcept +int32_t DetectionOutputDynamic::getNbOutputs() const noexcept { // Plugin layer has 2 outputs return 2; } -int DetectionOutput::initialize() noexcept +int32_t DetectionOutput::initialize() noexcept { return STATUS_SUCCESS; } -int DetectionOutputDynamic::initialize() noexcept +int32_t DetectionOutputDynamic::initialize() noexcept { return STATUS_SUCCESS; } @@ -140,7 +141,7 @@ void DetectionOutput::terminate() noexcept {} void DetectionOutputDynamic::terminate() noexcept {} // Returns output dimensions at given index -Dims DetectionOutput::getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept +Dims DetectionOutput::getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept { PLUGIN_ASSERT(nbInputDims == 3); PLUGIN_ASSERT(index == 0 || index == 1); @@ -155,7 +156,7 @@ Dims DetectionOutput::getOutputDimensions(int index, Dims const* inputs, int nbI } DimsExprs DetectionOutputDynamic::getOutputDimensions( - int outputIndex, DimsExprs const* inputs, int nbInputs, IExprBuilder& exprBuilder) noexcept + int32_t outputIndex, DimsExprs const* inputs, int32_t nbInputs, IExprBuilder& exprBuilder) noexcept { PLUGIN_ASSERT(nbInputs == 3); PLUGIN_ASSERT(outputIndex >= 0 && outputIndex < this->getNbOutputs()); @@ -165,8 +166,8 @@ DimsExprs DetectionOutputDynamic::getOutputDimensions( PLUGIN_ASSERT(inputs[1].nbDims == 4); // prior data PLUGIN_ASSERT(inputs[2].nbDims == 4); - int const C1_idx = param.inputOrder[0]; - int const C2_idx = param.inputOrder[1]; + int32_t const C1_idx = param.inputOrder[0]; + int32_t const C2_idx = param.inputOrder[1]; if (inputs[C1_idx].d[0]->isConstant() && inputs[C1_idx].d[1]->isConstant() && inputs[C1_idx].d[2]->isConstant() && inputs[C1_idx].d[3]->isConstant()) { @@ -207,22 +208,22 @@ DimsExprs DetectionOutputDynamic::getOutputDimensions( } // Returns the workspace size -size_t DetectionOutput::getWorkspaceSize(int maxBatchSize) const noexcept +size_t DetectionOutput::getWorkspaceSize(int32_t maxBatchSize) const noexcept { return detectionInferenceWorkspaceSize( param.shareLocation, maxBatchSize, C1, C2, param.numClasses, numPriors, param.topK, mType, mType); } size_t DetectionOutputDynamic::getWorkspaceSize( - PluginTensorDesc const* inputs, int nbInputs, PluginTensorDesc const* outputs, int nbOutputs) const noexcept + PluginTensorDesc const* inputs, int32_t nbInputs, PluginTensorDesc const* outputs, int32_t nbOutputs) const noexcept { return detectionInferenceWorkspaceSize( param.shareLocation, inputs[0].dims.d[0], C1, C2, param.numClasses, numPriors, param.topK, mType, mType); } // Plugin layer implementation -int DetectionOutput::enqueue( - int batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept +int32_t DetectionOutput::enqueue( + int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept { // Input order {loc, conf, prior} void const* const locData = inputs[param.inputOrder[0]]; @@ -263,13 +264,13 @@ int32_t DetectionOutputDynamic::enqueue(PluginTensorDesc const* inputDesc, Plugi size_t DetectionOutput::getSerializationSize() const noexcept { // DetectionOutputParameters, C1, C2, numPriors, mType, mScoreBits - return sizeof(DetectionOutputParameters) + sizeof(int) * 3 + sizeof(DataType) + sizeof(int32_t); + return sizeof(DetectionOutputParameters) + sizeof(int32_t) * 3 + sizeof(DataType) + sizeof(int32_t); } size_t DetectionOutputDynamic::getSerializationSize() const noexcept { // DetectionOutputParameters, C1, C2, numPriors, mType, mScoreBits - return sizeof(DetectionOutputParameters) + sizeof(int) * 3 + sizeof(DataType) + sizeof(int32_t); + return sizeof(DetectionOutputParameters) + sizeof(int32_t) * 3 + sizeof(DataType) + sizeof(int32_t); } // Serialization of plugin parameters @@ -304,7 +305,7 @@ bool DetectionOutput::supportsFormat(DataType type, PluginFormat format) const n } bool DetectionOutputDynamic::supportsFormatCombination( - int pos, PluginTensorDesc const* inOut, int nbInputs, int nbOutputs) noexcept + int32_t pos, PluginTensorDesc const* inOut, int32_t nbInputs, int32_t nbOutputs) noexcept { // 3 inputs, 2 outputs, so 5 input/output in total PLUGIN_ASSERT(0 <= pos && pos < 5); @@ -437,7 +438,7 @@ char const* DetectionOutputDynamic::getPluginNamespace() const noexcept // Return the DataType of the plugin output at the requested index. DataType DetectionOutput::getOutputDataType( - int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { // Two outputs PLUGIN_ASSERT(index == 0 || index == 1); @@ -448,12 +449,12 @@ DataType DetectionOutput::getOutputDataType( return inputTypes[0]; } // keepCount: use kFLOAT instead as they have same sizeof(type) - PLUGIN_ASSERT(sizeof(int) == sizeof(float)); + PLUGIN_ASSERT(sizeof(int32_t) == sizeof(float)); return DataType::kFLOAT; } DataType DetectionOutputDynamic::getOutputDataType( - int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { // Two outputs PLUGIN_ASSERT(index == 0 || index == 1); @@ -464,19 +465,19 @@ DataType DetectionOutputDynamic::getOutputDataType( return inputTypes[0]; } // keepCount: use kFLOAT instead as they have same sizeof(type) - PLUGIN_ASSERT(sizeof(int) == sizeof(float)); + PLUGIN_ASSERT(sizeof(int32_t) == sizeof(float)); return DataType::kFLOAT; } // Return true if output tensor is broadcast across a batch. bool DetectionOutput::isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept { return false; } // Return true if plugin can use input that is broadcast across batch without replication. -bool DetectionOutput::canBroadcastInputAcrossBatch(int inputIndex) const noexcept +bool DetectionOutput::canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept { return false; } @@ -489,21 +490,21 @@ bool DetectionOutput::canBroadcastInputAcrossBatch(int inputIndex) const noexcep // type: DataType configuration for the plugin layer // format: format NCHW, NHWC etc // maxbatchSize: maximum batch size for the plugin layer -void DetectionOutput::configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, - DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept +void DetectionOutput::configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, + int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept { PLUGIN_ASSERT(nbInputs == 3); PLUGIN_ASSERT(nbOutputs == 2); // Verify all the input dimensions - for (int i = 0; i < nbInputs; i++) + for (int32_t i = 0; i < nbInputs; i++) { PLUGIN_ASSERT(inputDims[i].nbDims == 3); } // Verify all the output dimensions - for (int i = 0; i < nbOutputs; i++) + for (int32_t i = 0; i < nbOutputs; i++) { PLUGIN_ASSERT(outputDims[i].nbDims == 3); } @@ -513,9 +514,9 @@ void DetectionOutput::configurePlugin(Dims const* inputDims, int nbInputs, Dims C1 = inputDims[param.inputOrder[0]].d[0]; C2 = inputDims[param.inputOrder[1]].d[0]; - int const nbBoxCoordinates = 4; + int32_t const nbBoxCoordinates = 4; numPriors = inputDims[param.inputOrder[2]].d[1] / nbBoxCoordinates; - int const numLocClasses = param.shareLocation ? 1 : param.numClasses; + int32_t const numLocClasses = param.shareLocation ? 1 : param.numClasses; // Verify C1 PLUGIN_ASSERT(numPriors * numLocClasses * nbBoxCoordinates == inputDims[param.inputOrder[0]].d[0]); @@ -528,19 +529,19 @@ void DetectionOutput::configurePlugin(Dims const* inputDims, int nbInputs, Dims } void DetectionOutputDynamic::configurePlugin( - DynamicPluginTensorDesc const* in, int nbInputs, DynamicPluginTensorDesc const* out, int nbOutputs) noexcept + DynamicPluginTensorDesc const* in, int32_t nbInputs, DynamicPluginTensorDesc const* out, int32_t nbOutputs) noexcept { PLUGIN_ASSERT(nbInputs == 3); PLUGIN_ASSERT(nbOutputs == 2); // Verify all the input dimensions - for (int i = 0; i < nbInputs; i++) + for (int32_t i = 0; i < nbInputs; i++) { PLUGIN_ASSERT(in[i].desc.dims.nbDims == 4); } // Verify all the output dimensions - for (int i = 0; i < nbOutputs; i++) + for (int32_t i = 0; i < nbOutputs; i++) { PLUGIN_ASSERT(out[i].desc.dims.nbDims == 4); } @@ -550,9 +551,9 @@ void DetectionOutputDynamic::configurePlugin( C1 = in[param.inputOrder[0]].desc.dims.d[1]; C2 = in[param.inputOrder[1]].desc.dims.d[1]; - int const nbBoxCoordinates = 4; + int32_t const nbBoxCoordinates = 4; numPriors = in[param.inputOrder[2]].desc.dims.d[2] / nbBoxCoordinates; - int const numLocClasses = param.shareLocation ? 1 : param.numClasses; + int32_t const numLocClasses = param.shareLocation ? 1 : param.numClasses; // Verify C1 PLUGIN_ASSERT(numPriors * numLocClasses * nbBoxCoordinates == in[param.inputOrder[0]].desc.dims.d[1]); @@ -639,38 +640,38 @@ IPluginV2Ext* NMSPluginCreator::createPlugin(char const* name, PluginFieldCollec mScoreBits = 16; // Read configurations from each fields - for (int i = 0; i < fc->nbFields; ++i) + for (int32_t i = 0; i < fc->nbFields; ++i) { char const* attrName = fields[i].name; if (!strcmp(attrName, "shareLocation")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.shareLocation = static_cast(*(static_cast(fields[i].data))); + params.shareLocation = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "varianceEncodedInTarget")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.varianceEncodedInTarget = static_cast(*(static_cast(fields[i].data))); + params.varianceEncodedInTarget = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "backgroundLabelId")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.backgroundLabelId = static_cast(*(static_cast(fields[i].data))); + params.backgroundLabelId = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "numClasses")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.numClasses = static_cast(*(static_cast(fields[i].data))); + params.numClasses = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "topK")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.topK = static_cast(*(static_cast(fields[i].data))); + params.topK = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "keepTopK")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.keepTopK = static_cast(*(static_cast(fields[i].data))); + params.keepTopK = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "confidenceThreshold")) { @@ -684,18 +685,18 @@ IPluginV2Ext* NMSPluginCreator::createPlugin(char const* name, PluginFieldCollec } else if (!strcmp(attrName, "confSigmoid")) { - params.confSigmoid = static_cast(*(static_cast(fields[i].data))); + params.confSigmoid = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "isNormalized")) { - params.isNormalized = static_cast(*(static_cast(fields[i].data))); + params.isNormalized = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "inputOrder")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - int const size = fields[i].length; - int const* o = static_cast(fields[i].data); - for (int j = 0; j < size; j++) + int32_t const size = fields[i].length; + int32_t const* o = static_cast(fields[i].data); + for (int32_t j = 0; j < size; j++) { params.inputOrder[j] = *o; o++; @@ -704,7 +705,7 @@ IPluginV2Ext* NMSPluginCreator::createPlugin(char const* name, PluginFieldCollec else if (!strcmp(attrName, "codeType")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.codeType = static_cast(*(static_cast(fields[i].data))); + params.codeType = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "scoreBits")) { @@ -714,7 +715,7 @@ IPluginV2Ext* NMSPluginCreator::createPlugin(char const* name, PluginFieldCollec else if (!strcmp(attrName, "isBatchAgnostic")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.isBatchAgnostic = static_cast(*(static_cast(fields[i].data))); + params.isBatchAgnostic = static_cast(*(static_cast(fields[i].data))); } } @@ -744,38 +745,38 @@ IPluginV2DynamicExt* NMSDynamicPluginCreator::createPlugin(char const* name, Plu mScoreBits = 16; // Read configurations from each fields - for (int i = 0; i < fc->nbFields; ++i) + for (int32_t i = 0; i < fc->nbFields; ++i) { char const* attrName = fields[i].name; if (!strcmp(attrName, "shareLocation")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.shareLocation = static_cast(*(static_cast(fields[i].data))); + params.shareLocation = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "varianceEncodedInTarget")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.varianceEncodedInTarget = static_cast(*(static_cast(fields[i].data))); + params.varianceEncodedInTarget = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "backgroundLabelId")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.backgroundLabelId = static_cast(*(static_cast(fields[i].data))); + params.backgroundLabelId = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "numClasses")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.numClasses = static_cast(*(static_cast(fields[i].data))); + params.numClasses = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "topK")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.topK = static_cast(*(static_cast(fields[i].data))); + params.topK = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "keepTopK")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.keepTopK = static_cast(*(static_cast(fields[i].data))); + params.keepTopK = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "confidenceThreshold")) { @@ -789,18 +790,18 @@ IPluginV2DynamicExt* NMSDynamicPluginCreator::createPlugin(char const* name, Plu } else if (!strcmp(attrName, "confSigmoid")) { - params.confSigmoid = static_cast(*(static_cast(fields[i].data))); + params.confSigmoid = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "isNormalized")) { - params.isNormalized = static_cast(*(static_cast(fields[i].data))); + params.isNormalized = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "inputOrder")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - int const size = fields[i].length; - int const* o = static_cast(fields[i].data); - for (int j = 0; j < size; j++) + int32_t const size = fields[i].length; + int32_t const* o = static_cast(fields[i].data); + for (int32_t j = 0; j < size; j++) { params.inputOrder[j] = *o; o++; @@ -809,7 +810,7 @@ IPluginV2DynamicExt* NMSDynamicPluginCreator::createPlugin(char const* name, Plu else if (!strcmp(attrName, "codeType")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.codeType = static_cast(*(static_cast(fields[i].data))); + params.codeType = static_cast(*(static_cast(fields[i].data))); } else if (!strcmp(attrName, "scoreBits")) { diff --git a/plugin/nmsPlugin/nmsPlugin.h b/plugin/nmsPlugin/nmsPlugin.h index 2075aeb5..70d3d5ba 100644 --- a/plugin/nmsPlugin/nmsPlugin.h +++ b/plugin/nmsPlugin/nmsPlugin.h @@ -32,23 +32,23 @@ class DetectionOutput : public IPluginV2Ext public: DetectionOutput(DetectionOutputParameters param); - DetectionOutput(DetectionOutputParameters param, int C1, int C2, int numPriors); + DetectionOutput(DetectionOutputParameters param, int32_t C1, int32_t C2, int32_t numPriors); DetectionOutput(void const* data, size_t length); ~DetectionOutput() override = default; - int getNbOutputs() const noexcept override; + int32_t getNbOutputs() const noexcept override; - Dims getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept override; + Dims getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept override; - int initialize() noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; - size_t getWorkspaceSize(int maxBatchSize) const noexcept override; + size_t getWorkspaceSize(int32_t maxBatchSize) const noexcept override; - int enqueue(int batchSize, void const* const* inputs, void* const* outputs, void* workspace, + int32_t enqueue(int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; size_t getSerializationSize() const noexcept override; @@ -69,19 +69,20 @@ class DetectionOutput : public IPluginV2Ext char const* getPluginNamespace() const noexcept override; - DataType getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + DataType getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; bool isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept override; + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept override; - bool canBroadcastInputAcrossBatch(int inputIndex) const noexcept override; + bool canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept override; void attachToContext( cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept override; - void configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, + void configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept override; + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept override; void detachFromContext() noexcept override; @@ -89,7 +90,7 @@ class DetectionOutput : public IPluginV2Ext private: DetectionOutputParameters param; - int C1, C2, numPriors; + int32_t C1, C2, numPriors; DataType mType; int32_t mScoreBits; std::string mPluginNamespace; @@ -99,15 +100,15 @@ class DetectionOutputDynamic : public IPluginV2DynamicExt { public: DetectionOutputDynamic(DetectionOutputParameters param); - DetectionOutputDynamic(DetectionOutputParameters param, int C1, int C2, int numPriors); + DetectionOutputDynamic(DetectionOutputParameters param, int32_t C1, int32_t C2, int32_t numPriors); DetectionOutputDynamic(void const* data, size_t length); ~DetectionOutputDynamic() override = default; // IPluginV2 methods char const* getPluginType() const noexcept override; char const* getPluginVersion() const noexcept override; - int getNbOutputs() const noexcept override; - int initialize() noexcept override; + int32_t getNbOutputs() const noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; size_t getSerializationSize() const noexcept override; void serialize(void* buffer) const noexcept override; @@ -117,24 +118,25 @@ class DetectionOutputDynamic : public IPluginV2DynamicExt void setScoreBits(int32_t scoreBits) noexcept; // IPluginV2Ext methods - DataType getOutputDataType(int index, nvinfer1::DataType const* inputType, int nbInputs) const noexcept override; + DataType getOutputDataType( + int32_t index, nvinfer1::DataType const* inputType, int32_t nbInputs) const noexcept override; // IPluginV2DynamicExt methods IPluginV2DynamicExt* clone() const noexcept override; DimsExprs getOutputDimensions( - int outputIndex, DimsExprs const* inputs, int nbInputs, IExprBuilder& exprBuilder) noexcept override; + int32_t outputIndex, DimsExprs const* inputs, int32_t nbInputs, IExprBuilder& exprBuilder) noexcept override; bool supportsFormatCombination( - int pos, PluginTensorDesc const* inOut, int nbInputs, int nbOutputs) noexcept override; - void configurePlugin(DynamicPluginTensorDesc const* in, int nbInputs, DynamicPluginTensorDesc const* out, - int nbOutputs) noexcept override; - size_t getWorkspaceSize(PluginTensorDesc const* inputs, int nbInputs, PluginTensorDesc const* outputs, - int nbOutputs) const noexcept override; + int32_t pos, PluginTensorDesc const* inOut, int32_t nbInputs, int32_t nbOutputs) noexcept override; + void configurePlugin(DynamicPluginTensorDesc const* in, int32_t nbInputs, DynamicPluginTensorDesc const* out, + int32_t nbOutputs) noexcept override; + size_t getWorkspaceSize(PluginTensorDesc const* inputs, int32_t nbInputs, PluginTensorDesc const* outputs, + int32_t nbOutputs) const noexcept override; int32_t enqueue(PluginTensorDesc const* inputDesc, PluginTensorDesc const* outputDesc, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; private: DetectionOutputParameters param; - int C1, C2, numPriors; + int32_t C1, C2, numPriors; DataType mType; int32_t mScoreBits; std::string mPluginNamespace; diff --git a/plugin/normalizePlugin/normalizePlugin.cpp b/plugin/normalizePlugin/normalizePlugin.cpp index 86640406..b400f35a 100644 --- a/plugin/normalizePlugin/normalizePlugin.cpp +++ b/plugin/normalizePlugin/normalizePlugin.cpp @@ -35,7 +35,7 @@ char const* const kNORMALIZE_PLUGIN_NAME{"Normalize_TRT"}; PluginFieldCollection NormalizePluginCreator::mFC{}; std::vector NormalizePluginCreator::mPluginAttributes; -Normalize::Normalize(Weights const* weights, int nbWeights, bool acrossSpatial, bool channelShared, float eps) +Normalize::Normalize(Weights const* weights, int32_t nbWeights, bool acrossSpatial, bool channelShared, float eps) : acrossSpatial(acrossSpatial) , channelShared(channelShared) , eps(eps) @@ -47,8 +47,8 @@ Normalize::Normalize(Weights const* weights, int nbWeights, bool acrossSpatial, mScalarScale = static_cast(weights[0].values)[0]; } -Normalize::Normalize(Weights const* weights, int nbWeights, float scalarScale, bool acrossSpatial, bool channelShared, - float eps, int C, int H, int W) +Normalize::Normalize(Weights const* weights, int32_t nbWeights, float scalarScale, bool acrossSpatial, + bool channelShared, float eps, int32_t C, int32_t H, int32_t W) : mScalarScale(scalarScale) , acrossSpatial(acrossSpatial) , channelShared(channelShared) @@ -67,27 +67,27 @@ Normalize::Normalize(void const* buffer, size_t length) { char const* d = static_cast(buffer); char const* a = d; - C = read(d); - H = read(d); - W = read(d); + C = read(d); + H = read(d); + W = read(d); acrossSpatial = read(d); channelShared = read(d); eps = read(d); - mNbWeights = read(d); - int count = read(d); + mNbWeights = read(d); + int32_t count = read(d); std::memcpy(&mScalarScale, d, sizeof(float)); mWeights = deserializeToDevice(d, count); PLUGIN_VALIDATE(d == a + length); } -int Normalize::getNbOutputs() const noexcept +int32_t Normalize::getNbOutputs() const noexcept { // Plugin layer has 1 output return 1; } -Dims Normalize::getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept +Dims Normalize::getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept { PLUGIN_ASSERT(nbInputDims == 1); PLUGIN_ASSERT(index == 0); @@ -95,20 +95,20 @@ Dims Normalize::getOutputDimensions(int index, Dims const* inputs, int nbInputDi return Dims3(inputs[0].d[0], inputs[0].d[1], inputs[0].d[2]); } -int Normalize::initialize() noexcept +int32_t Normalize::initialize() noexcept { return STATUS_SUCCESS; } void Normalize::terminate() noexcept {} -size_t Normalize::getWorkspaceSize(int maxBatchSize) const noexcept +size_t Normalize::getWorkspaceSize(int32_t maxBatchSize) const noexcept { return normalizePluginWorkspaceSize(acrossSpatial, C, H, W); } -int Normalize::enqueue( - int batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept +int32_t Normalize::enqueue( + int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept { void const* inputData = inputs[0]; void* outputData = outputs[0]; @@ -133,7 +133,8 @@ int Normalize::enqueue( size_t Normalize::getSerializationSize() const noexcept { // C,H,W, acrossSpatial,channelShared, eps, mWeights.count,mWeights.values - return sizeof(int) * 3 + sizeof(bool) * 2 + sizeof(float) + sizeof(int) * 2 + mWeights.count * sizeof(float); + return sizeof(int32_t) * 3 + sizeof(bool) * 2 + sizeof(float) + sizeof(int32_t) * 2 + + mWeights.count * sizeof(float); } void Normalize::serialize(void* buffer) const noexcept @@ -145,8 +146,8 @@ void Normalize::serialize(void* buffer) const noexcept write(d, acrossSpatial); write(d, channelShared); write(d, eps); - write(d, (int) mNbWeights); - write(d, (int) mWeights.count); + write(d, (int32_t) mNbWeights); + write(d, (int32_t) mWeights.count); serializeFromDevice(d, mWeights); PLUGIN_ASSERT(d == a + getSerializationSize()); @@ -191,7 +192,8 @@ char const* Normalize::getPluginNamespace() const noexcept } // Return the DataType of the plugin output at the requested index -DataType Normalize::getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept +DataType Normalize::getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { PLUGIN_ASSERT(index == 0); return DataType::kFLOAT; @@ -199,21 +201,21 @@ DataType Normalize::getOutputDataType(int index, nvinfer1::DataType const* input // Return true if output tensor is broadcast across a batch. bool Normalize::isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept { return false; } // Return true if plugin can use input that is broadcast across batch without replication. -bool Normalize::canBroadcastInputAcrossBatch(int inputIndex) const noexcept +bool Normalize::canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept { return false; } // Configure the layer with input and output data types. -void Normalize::configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, +void Normalize::configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept { PLUGIN_ASSERT(*inputTypes == DataType::kFLOAT && floatFormat == PluginFormat::kLINEAR); C = inputDims[0].d[0]; @@ -314,13 +316,13 @@ IPluginV2Ext* NormalizePluginCreator::createPlugin(char const* name, PluginField { std::vector weightValues; PluginField const* fields = fc->fields; - for (int i = 0; i < fc->nbFields; ++i) + for (int32_t i = 0; i < fc->nbFields; ++i) { char const* attrName = fields[i].name; if (!strcmp(attrName, "nbWeights")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - mNbWeights = *(static_cast(fields[i].data)); + mNbWeights = *(static_cast(fields[i].data)); } else if (!strcmp(attrName, "acrossSpatial")) { @@ -340,10 +342,10 @@ IPluginV2Ext* NormalizePluginCreator::createPlugin(char const* name, PluginField else if (!strcmp(attrName, "weights")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kFLOAT32); - int size = fields[i].length; + int32_t size = fields[i].length; weightValues.reserve(size); auto const* w = static_cast(fields[i].data); - for (int j = 0; j < size; j++) + for (int32_t j = 0; j < size; j++) { weightValues.push_back(*w); w++; diff --git a/plugin/normalizePlugin/normalizePlugin.h b/plugin/normalizePlugin/normalizePlugin.h index f41ff478..622b44d6 100644 --- a/plugin/normalizePlugin/normalizePlugin.h +++ b/plugin/normalizePlugin/normalizePlugin.h @@ -31,26 +31,26 @@ namespace plugin class Normalize : public IPluginV2Ext { public: - Normalize(Weights const* weights, int nbWeights, bool acrossSpatial, bool channelShared, float eps); + Normalize(Weights const* weights, int32_t nbWeights, bool acrossSpatial, bool channelShared, float eps); - Normalize(Weights const* weights, int nbWeights, float scalarScale, bool acrossSpatial, bool channelShared, - float eps, int C, int H, int W); + Normalize(Weights const* weights, int32_t nbWeights, float scalarScale, bool acrossSpatial, bool channelShared, + float eps, int32_t C, int32_t H, int32_t W); Normalize(void const* buffer, size_t length); ~Normalize() override = default; - int getNbOutputs() const noexcept override; + int32_t getNbOutputs() const noexcept override; - Dims getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept override; + Dims getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept override; - int initialize() noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; - size_t getWorkspaceSize(int maxBatchSize) const noexcept override; + size_t getWorkspaceSize(int32_t maxBatchSize) const noexcept override; - int enqueue(int batchSize, void const* const* inputs, void* const* outputs, void* workspace, + int32_t enqueue(int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; size_t getSerializationSize() const noexcept override; @@ -71,19 +71,20 @@ class Normalize : public IPluginV2Ext char const* getPluginNamespace() const noexcept override; - DataType getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + DataType getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; bool isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept override; + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept override; - bool canBroadcastInputAcrossBatch(int inputIndex) const noexcept override; + bool canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept override; void attachToContext( cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept override; - void configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, + void configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept override; + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept override; void detachFromContext() noexcept override; @@ -95,14 +96,14 @@ class Normalize : public IPluginV2Ext cublasHandle_t mCublas; Weights mWeights{}; // mWeights.values is on the device - int mNbWeights{}; + int32_t mNbWeights{}; float mScalarScale{}; // keep track of scale on the host (for when channelShared is true) bool acrossSpatial{}; bool channelShared{}; float eps{}; - int C{}; - int H{}; - int W{}; + int32_t C{}; + int32_t H{}; + int32_t W{}; std::string mPluginNamespace; }; @@ -128,7 +129,7 @@ class NormalizePluginCreator : public nvinfer1::pluginInternal::BaseCreator bool mAcrossSpatial{}; bool mChannelShared{}; float mEps{}; - int mNbWeights{}; + int32_t mNbWeights{}; static std::vector mPluginAttributes; }; } // namespace plugin diff --git a/plugin/nvFasterRCNN/nvFasterRCNNPlugin.cpp b/plugin/nvFasterRCNN/nvFasterRCNNPlugin.cpp index 89d976ee..85b2dee4 100644 --- a/plugin/nvFasterRCNN/nvFasterRCNNPlugin.cpp +++ b/plugin/nvFasterRCNN/nvFasterRCNNPlugin.cpp @@ -132,7 +132,7 @@ RPROIPlugin::~RPROIPlugin() } } -int RPROIPlugin::initialize() noexcept +int32_t RPROIPlugin::initialize() noexcept { return STATUS_SUCCESS; } @@ -146,12 +146,12 @@ size_t RPROIPlugin::getSmemSize() const noexcept return prop.sharedMemPerBlockOptin; } -int RPROIPlugin::getNbOutputs() const noexcept +int32_t RPROIPlugin::getNbOutputs() const noexcept { return 2; } -Dims RPROIPlugin::getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept +Dims RPROIPlugin::getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept { PLUGIN_ASSERT(index >= 0 && index < 2); PLUGIN_ASSERT(nbInputDims == 4); @@ -165,13 +165,13 @@ Dims RPROIPlugin::getOutputDimensions(int index, Dims const* inputs, int nbInput return Dims4(params.nmsMaxOut, inputs[2].d[0], params.poolingH, params.poolingW); } -size_t RPROIPlugin::getWorkspaceSize(int maxBatchSize) const noexcept +size_t RPROIPlugin::getWorkspaceSize(int32_t maxBatchSize) const noexcept { return RPROIInferenceFusedWorkspaceSize(maxBatchSize, A, H, W, params.nmsMaxOut); } -int RPROIPlugin::enqueue( - int batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept +int32_t RPROIPlugin::enqueue( + int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept { // Bounding box (region proposal) objectness scores. void const* const scores = inputs[0]; @@ -230,7 +230,7 @@ void RPROIPlugin::serialize(void* buffer) const noexcept PLUGIN_ASSERT(d == a + getSerializationSize()); } -float* RPROIPlugin::copyToHost(void const* srcHostData, int count) noexcept +float* RPROIPlugin::copyToHost(void const* srcHostData, int32_t count) noexcept { float* dstHostPtr = nullptr; PLUGIN_CHECK(cudaMallocHost(&dstHostPtr, count * sizeof(float))); @@ -238,7 +238,7 @@ float* RPROIPlugin::copyToHost(void const* srcHostData, int count) noexcept return dstHostPtr; } -int RPROIPlugin::copyFromHost(char* dstHostBuffer, void const* source, int count) const noexcept +int32_t RPROIPlugin::copyFromHost(char* dstHostBuffer, void const* source, int32_t count) const noexcept { PLUGIN_CHECK(cudaMemcpy(dstHostBuffer, source, count * sizeof(float), cudaMemcpyHostToHost)); return count * sizeof(float); @@ -318,7 +318,8 @@ char const* RPROIPlugin::getPluginNamespace() const noexcept } // Return the DataType of the plugin output at the requested index. -DataType RPROIPlugin::getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept +DataType RPROIPlugin::getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { // Two outputs PLUGIN_ASSERT(index == 0 || index == 1); @@ -327,13 +328,13 @@ DataType RPROIPlugin::getOutputDataType(int index, nvinfer1::DataType const* inp // Return true if output tensor is broadcast across a batch. bool RPROIPlugin::isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept { return false; } // Return true if plugin can use input that is broadcast across batch without replication. -bool RPROIPlugin::canBroadcastInputAcrossBatch(int inputIndex) const noexcept +bool RPROIPlugin::canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept { return false; } @@ -432,45 +433,45 @@ IPluginV2Ext* RPROIPluginCreator::createPlugin(char const* name, PluginFieldColl try { PluginField const* fields = fc->fields; - int nbFields = fc->nbFields; + int32_t nbFields = fc->nbFields; - for (int i = 0; i < nbFields; ++i) + for (int32_t i = 0; i < nbFields; ++i) { char const* attrName = fields[i].name; if (!strcmp(attrName, "poolingH")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.poolingH = *(static_cast(fields[i].data)); + params.poolingH = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "poolingW")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.poolingW = *(static_cast(fields[i].data)); + params.poolingW = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "featureStride")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.featureStride = *(static_cast(fields[i].data)); + params.featureStride = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "preNmsTop")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.preNmsTop = *(static_cast(fields[i].data)); + params.preNmsTop = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "nmsMaxOut")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.nmsMaxOut = *(static_cast(fields[i].data)); + params.nmsMaxOut = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "anchorsRatioCount")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.anchorsRatioCount = *(static_cast(fields[i].data)); + params.anchorsRatioCount = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "anchorsScaleCount")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.anchorsScaleCount = *(static_cast(fields[i].data)); + params.anchorsScaleCount = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "iouThreshold")) { @@ -492,7 +493,7 @@ IPluginV2Ext* RPROIPluginCreator::createPlugin(char const* name, PluginFieldColl PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kFLOAT32); anchorsRatios.reserve(params.anchorsRatioCount); float const* ratios = static_cast(fields[i].data); - for (int j = 0; j < params.anchorsRatioCount; ++j) + for (int32_t j = 0; j < params.anchorsRatioCount; ++j) { anchorsRatios.push_back(*ratios); ratios++; @@ -503,7 +504,7 @@ IPluginV2Ext* RPROIPluginCreator::createPlugin(char const* name, PluginFieldColl PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kFLOAT32); anchorsScales.reserve(params.anchorsScaleCount); float const* scales = static_cast(fields[i].data); - for (int j = 0; j < params.anchorsScaleCount; ++j) + for (int32_t j = 0; j < params.anchorsScaleCount; ++j) { anchorsScales.push_back(*scales); scales++; diff --git a/plugin/nvFasterRCNN/nvFasterRCNNPlugin.h b/plugin/nvFasterRCNN/nvFasterRCNNPlugin.h index 4e05582c..30f3c099 100644 --- a/plugin/nvFasterRCNN/nvFasterRCNNPlugin.h +++ b/plugin/nvFasterRCNN/nvFasterRCNNPlugin.h @@ -40,17 +40,17 @@ class RPROIPlugin : public IPluginV2IOExt ~RPROIPlugin() override; - int getNbOutputs() const noexcept override; + int32_t getNbOutputs() const noexcept override; - Dims getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept override; + Dims getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept override; - int initialize() noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; - size_t getWorkspaceSize(int maxBatchSize) const noexcept override; + size_t getWorkspaceSize(int32_t maxBatchSize) const noexcept override; - int enqueue(int batchSize, void const* const* inputs, void* const* outputs, void* workspace, + int32_t enqueue(int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; size_t getSerializationSize() const noexcept override; @@ -72,12 +72,13 @@ class RPROIPlugin : public IPluginV2IOExt char const* getPluginNamespace() const noexcept override; - DataType getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + DataType getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; bool isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept override; + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept override; - bool canBroadcastInputAcrossBatch(int inputIndex) const noexcept override; + bool canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept override; void attachToContext( cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept override; @@ -89,9 +90,9 @@ class RPROIPlugin : public IPluginV2IOExt private: void deserialize(int8_t const* data, size_t length); - float* copyToHost(void const* srcHostData, int count) noexcept; + float* copyToHost(void const* srcHostData, int32_t count) noexcept; - int copyFromHost(char* dstHostBuffer, void const* source, int count) const noexcept; + int32_t copyFromHost(char* dstHostBuffer, void const* source, int32_t count) const noexcept; size_t getSmemSize() const noexcept; diff --git a/plugin/pillarScatterPlugin/pillarScatter.cpp b/plugin/pillarScatterPlugin/pillarScatter.cpp index 93f162df..f1a0aa16 100644 --- a/plugin/pillarScatterPlugin/pillarScatter.cpp +++ b/plugin/pillarScatterPlugin/pillarScatter.cpp @@ -59,8 +59,8 @@ nvinfer1::IPluginV2DynamicExt* PillarScatterPlugin::clone() const noexcept return nullptr; } -nvinfer1::DimsExprs PillarScatterPlugin::getOutputDimensions( - int outputIndex, nvinfer1::DimsExprs const* inputs, int nbInputs, nvinfer1::IExprBuilder& exprBuilder) noexcept +nvinfer1::DimsExprs PillarScatterPlugin::getOutputDimensions(int32_t outputIndex, nvinfer1::DimsExprs const* inputs, + int32_t nbInputs, nvinfer1::IExprBuilder& exprBuilder) noexcept { PLUGIN_ASSERT(outputIndex == 0); nvinfer1::DimsExprs output; @@ -74,7 +74,7 @@ nvinfer1::DimsExprs PillarScatterPlugin::getOutputDimensions( } bool PillarScatterPlugin::supportsFormatCombination( - int pos, nvinfer1::PluginTensorDesc const* inOut, int nbInputs, int nbOutputs) noexcept + int32_t pos, nvinfer1::PluginTensorDesc const* inOut, int32_t nbInputs, int32_t nbOutputs) noexcept { PLUGIN_ASSERT(nbInputs == 3); PLUGIN_ASSERT(nbOutputs == 1); @@ -99,36 +99,36 @@ bool PillarScatterPlugin::supportsFormatCombination( return false; } -void PillarScatterPlugin::configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int nbInputs, - nvinfer1::DynamicPluginTensorDesc const* out, int nbOutputs) noexcept +void PillarScatterPlugin::configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int32_t nbInputs, + nvinfer1::DynamicPluginTensorDesc const* out, int32_t nbOutputs) noexcept { return; } -size_t PillarScatterPlugin::getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int nbInputs, - nvinfer1::PluginTensorDesc const* outputs, int nbOutputs) const noexcept +size_t PillarScatterPlugin::getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int32_t nbInputs, + nvinfer1::PluginTensorDesc const* outputs, int32_t nbOutputs) const noexcept { return 0; } -int PillarScatterPlugin::enqueue(nvinfer1::PluginTensorDesc const* inputDesc, +int32_t PillarScatterPlugin::enqueue(nvinfer1::PluginTensorDesc const* inputDesc, nvinfer1::PluginTensorDesc const* outputDesc, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept { try { - int batchSize = inputDesc[0].dims.d[0]; - int maxPillarNum = inputDesc[0].dims.d[1]; - int numFeatures = inputDesc[0].dims.d[2]; + int32_t batchSize = inputDesc[0].dims.d[0]; + int32_t maxPillarNum = inputDesc[0].dims.d[1]; + int32_t numFeatures = inputDesc[0].dims.d[2]; nvinfer1::DataType inputType = inputDesc[0].type; - auto coords_data = static_cast(inputs[1]); - auto params_data = static_cast(inputs[2]); + auto coords_data = static_cast(inputs[1]); + auto params_data = static_cast(inputs[2]); - unsigned int featureY = feature_y_size_; - unsigned int featureX = feature_x_size_; + uint32_t featureY = feature_y_size_; + uint32_t featureX = feature_x_size_; - int status = -1; + int32_t status = -1; if (inputType == nvinfer1::DataType::kHALF) { @@ -159,7 +159,7 @@ int PillarScatterPlugin::enqueue(nvinfer1::PluginTensorDesc const* inputDesc, } nvinfer1::DataType PillarScatterPlugin::getOutputDataType( - int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { return inputTypes[0]; } @@ -174,12 +174,12 @@ char const* PillarScatterPlugin::getPluginVersion() const noexcept return kPLUGIN_VERSION; } -int PillarScatterPlugin::getNbOutputs() const noexcept +int32_t PillarScatterPlugin::getNbOutputs() const noexcept { return 1; } -int PillarScatterPlugin::initialize() noexcept +int32_t PillarScatterPlugin::initialize() noexcept { return 0; } diff --git a/plugin/pillarScatterPlugin/pillarScatter.h b/plugin/pillarScatterPlugin/pillarScatter.h index 690f6974..cdaf0454 100644 --- a/plugin/pillarScatterPlugin/pillarScatter.h +++ b/plugin/pillarScatterPlugin/pillarScatter.h @@ -37,24 +37,24 @@ class PillarScatterPlugin : public nvinfer1::IPluginV2DynamicExt PillarScatterPlugin(size_t h, size_t w); // IPluginV2DynamicExt Methods nvinfer1::IPluginV2DynamicExt* clone() const noexcept override; - nvinfer1::DimsExprs getOutputDimensions(int outputIndex, nvinfer1::DimsExprs const* inputs, int nbInputs, + nvinfer1::DimsExprs getOutputDimensions(int32_t outputIndex, nvinfer1::DimsExprs const* inputs, int32_t nbInputs, nvinfer1::IExprBuilder& exprBuilder) noexcept override; bool supportsFormatCombination( - int pos, nvinfer1::PluginTensorDesc const* inOut, int nbInputs, int nbOutputs) noexcept override; - void configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int nbInputs, - nvinfer1::DynamicPluginTensorDesc const* out, int nbOutputs) noexcept override; - size_t getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int nbInputs, - nvinfer1::PluginTensorDesc const* outputs, int nbOutputs) const noexcept override; - int enqueue(nvinfer1::PluginTensorDesc const* inputDesc, nvinfer1::PluginTensorDesc const* outputDesc, + int32_t pos, nvinfer1::PluginTensorDesc const* inOut, int32_t nbInputs, int32_t nbOutputs) noexcept override; + void configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int32_t nbInputs, + nvinfer1::DynamicPluginTensorDesc const* out, int32_t nbOutputs) noexcept override; + size_t getWorkspaceSize(nvinfer1::PluginTensorDesc const* inputs, int32_t nbInputs, + nvinfer1::PluginTensorDesc const* outputs, int32_t nbOutputs) const noexcept override; + int32_t enqueue(nvinfer1::PluginTensorDesc const* inputDesc, nvinfer1::PluginTensorDesc const* outputDesc, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; // IPluginV2Ext Methods nvinfer1::DataType getOutputDataType( - int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; // IPluginV2 Methods char const* getPluginType() const noexcept override; char const* getPluginVersion() const noexcept override; - int getNbOutputs() const noexcept override; - int initialize() noexcept override; + int32_t getNbOutputs() const noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; size_t getSerializationSize() const noexcept override; void serialize(void* buffer) const noexcept override; diff --git a/plugin/proposalLayerPlugin/proposalLayerPlugin.cpp b/plugin/proposalLayerPlugin/proposalLayerPlugin.cpp index f384a8f3..1335ea66 100644 --- a/plugin/proposalLayerPlugin/proposalLayerPlugin.cpp +++ b/plugin/proposalLayerPlugin/proposalLayerPlugin.cpp @@ -117,14 +117,16 @@ IPluginV2Ext* ProposalLayerPluginCreator::deserializePlugin(char const* name, vo return nullptr; } -ProposalLayer::ProposalLayer(int prenms_topk, int keep_topk, float iou_threshold, nvinfer1::Dims const& imageSize) +ProposalLayer::ProposalLayer( + int32_t prenms_topk, int32_t keep_topk, float iou_threshold, nvinfer1::Dims const& imageSize) : mPreNMSTopK(prenms_topk) , mKeepTopK(keep_topk) , mIOUThreshold(iou_threshold) , mImageSize(imageSize) { mBackgroundLabel = -1; - PLUGIN_VALIDATE(mPreNMSTopK > 0 && mPreNMSTopK <= 1024); + PLUGIN_VALIDATE(mPreNMSTopK > 0); + PLUGIN_VALIDATE(mPreNMSTopK <= 1024); PLUGIN_VALIDATE(mKeepTopK > 0); PLUGIN_VALIDATE(iou_threshold > 0.0F); PLUGIN_VALIDATE(mImageSize.nbDims == 3); @@ -141,26 +143,26 @@ ProposalLayer::ProposalLayer(int prenms_topk, int keep_topk, float iou_threshold generate_pyramid_anchors(imageSize); } -int ProposalLayer::getNbOutputs() const noexcept +int32_t ProposalLayer::getNbOutputs() const noexcept { return 1; } -int ProposalLayer::initialize() noexcept +int32_t ProposalLayer::initialize() noexcept { // Init the mValidCnt of max batch size - std::vector tempValidCnt(mMaxBatchSize, mPreNMSTopK); + std::vector tempValidCnt(mMaxBatchSize, mPreNMSTopK); - mValidCnt = std::make_shared>(mMaxBatchSize); + mValidCnt = std::make_shared>(mMaxBatchSize); - PLUGIN_CUASSERT(cudaMemcpy( - mValidCnt->mPtr, static_cast(tempValidCnt.data()), sizeof(int) * mMaxBatchSize, cudaMemcpyHostToDevice)); + PLUGIN_CUASSERT(cudaMemcpy(mValidCnt->mPtr, static_cast(tempValidCnt.data()), + sizeof(int32_t) * mMaxBatchSize, cudaMemcpyHostToDevice)); // Init the anchors for batch size: mAnchorBoxesDevice = std::make_shared>(mAnchorsCnt * 4 * mMaxBatchSize); - int batch_offset = sizeof(float) * mAnchorsCnt * 4; + int32_t batch_offset = sizeof(float) * mAnchorsCnt * 4; uint8_t* device_ptr = static_cast(mAnchorBoxesDevice->mPtr); - for (int i = 0; i < mMaxBatchSize; i++) + for (int32_t i = 0; i < mMaxBatchSize; i++) { PLUGIN_CUASSERT(cudaMemcpy(static_cast(device_ptr + i * batch_offset), static_cast(mAnchorBoxesHost.data()), batch_offset, cudaMemcpyHostToDevice)); @@ -218,7 +220,7 @@ char const* ProposalLayer::getPluginNamespace() const noexcept size_t ProposalLayer::getSerializationSize() const noexcept { - return sizeof(int) * 2 + sizeof(float) + sizeof(int) * 2 + sizeof(nvinfer1::Dims); + return sizeof(int32_t) * 2 + sizeof(float) + sizeof(int32_t) * 2 + sizeof(nvinfer1::Dims); } void ProposalLayer::serialize(void* buffer) const noexcept @@ -265,7 +267,7 @@ void ProposalLayer::deserialize(int8_t const* data, size_t length) generate_pyramid_anchors(mImageSize); } -void ProposalLayer::check_valid_inputs(nvinfer1::Dims const* inputs, int nbInputDims) +void ProposalLayer::check_valid_inputs(nvinfer1::Dims const* inputs, int32_t nbInputDims) { // object_score[N, anchors, 2, 1], // foreground_delta[N, anchors, 4, 1], @@ -277,14 +279,14 @@ void ProposalLayer::check_valid_inputs(nvinfer1::Dims const* inputs, int nbInput PLUGIN_ASSERT(inputs[1].nbDims == 3 && inputs[1].d[1] == 4); } -size_t ProposalLayer::getWorkspaceSize(int batch_size) const noexcept +size_t ProposalLayer::getWorkspaceSize(int32_t batch_size) const noexcept { ProposalWorkSpace proposal(batch_size, mAnchorsCnt, mPreNMSTopK, mParam, mType); return proposal.totalSize; } -Dims ProposalLayer::getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept +Dims ProposalLayer::getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept { check_valid_inputs(inputs, nbInputDims); @@ -312,10 +314,10 @@ void ProposalLayer::generate_pyramid_anchors(nvinfer1::Dims const& imageDims) for (size_t s = 0; s < scales.size(); ++s) { float scale = scales[s]; - int stride = strides[s]; + int32_t stride = strides[s]; - for (int y = 0; y < imageDims.d[1]; y += anchor_stride * stride) - for (int x = 0; x < imageDims.d[2]; x += anchor_stride * stride) + for (int32_t y = 0; y < imageDims.d[1]; y += anchor_stride * stride) + for (int32_t x = 0; x < imageDims.d[2]; x += anchor_stride * stride) for (float r : ratios) { float sqrt_r = sqrt(r); @@ -330,8 +332,8 @@ void ProposalLayer::generate_pyramid_anchors(nvinfer1::Dims const& imageDims) PLUGIN_VALIDATE(anchors.size() % 4 == 0); } -int ProposalLayer::enqueue( - int batch_size, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept +int32_t ProposalLayer::enqueue( + int32_t batch_size, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept { void* proposals = outputs[0]; @@ -352,7 +354,8 @@ int ProposalLayer::enqueue( } // Return the DataType of the plugin output at the requested index -DataType ProposalLayer::getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept +DataType ProposalLayer::getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { // Only DataType::kFLOAT is acceptable by the plugin layer return DataType::kFLOAT; @@ -360,27 +363,27 @@ DataType ProposalLayer::getOutputDataType(int index, nvinfer1::DataType const* i // Return true if output tensor is broadcast across a batch. bool ProposalLayer::isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept { return false; } // Return true if plugin can use input that is broadcast across batch without replication. -bool ProposalLayer::canBroadcastInputAcrossBatch(int inputIndex) const noexcept +bool ProposalLayer::canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept { return false; } // Configure the layer with input and output data types. -void ProposalLayer::configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, +void ProposalLayer::configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept { check_valid_inputs(inputDims, nbInputs); PLUGIN_ASSERT(inputDims[0].d[0] == inputDims[1].d[0]); mAnchorsCnt = inputDims[0].d[0]; - PLUGIN_ASSERT(mAnchorsCnt == (int) (mAnchorBoxesHost.size() / 4)); + PLUGIN_ASSERT(mAnchorsCnt == (int32_t) (mAnchorBoxesHost.size() / 4)); mMaxBatchSize = maxBatchSize; } diff --git a/plugin/proposalLayerPlugin/proposalLayerPlugin.h b/plugin/proposalLayerPlugin/proposalLayerPlugin.h index d6880d00..68a0d136 100644 --- a/plugin/proposalLayerPlugin/proposalLayerPlugin.h +++ b/plugin/proposalLayerPlugin/proposalLayerPlugin.h @@ -34,33 +34,33 @@ namespace plugin class ProposalLayer : public IPluginV2Ext { public: - ProposalLayer(int prenms_topk, int keep_topk, float iou_threshold, nvinfer1::Dims const& image_size); + ProposalLayer(int32_t prenms_topk, int32_t keep_topk, float iou_threshold, nvinfer1::Dims const& image_size); ProposalLayer(void const* data, size_t length); ~ProposalLayer() override = default; - int getNbOutputs() const noexcept override; + int32_t getNbOutputs() const noexcept override; - Dims getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept override; + Dims getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept override; - int initialize() noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; void destroy() noexcept override; - size_t getWorkspaceSize(int maxBatchSize) const noexcept override; + size_t getWorkspaceSize(int32_t maxBatchSize) const noexcept override; - int enqueue(int batch_size, void const* const* inputs, void* const* outputs, void* workspace, + int32_t enqueue(int32_t batch_size, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; size_t getSerializationSize() const noexcept override; void serialize(void* buffer) const noexcept override; - // void configureWithFormat(const Dims* inputs, int nbInputs, const Dims* outputDims, int nbOutputs, - // nvinfer1::DataType type, nvinfer1::PluginFormat format, int maxBatchSize) override; + // void configureWithFormat(const Dims* inputs, int32_t nbInputs, const Dims* outputDims, int32_t nbOutputs, + // nvinfer1::DataType type, nvinfer1::PluginFormat format, int32_t maxBatchSize) override; bool supportsFormat(DataType type, PluginFormat format) const noexcept override; @@ -74,25 +74,26 @@ class ProposalLayer : public IPluginV2Ext char const* getPluginNamespace() const noexcept override; - DataType getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + DataType getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; bool isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept override; + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept override; - bool canBroadcastInputAcrossBatch(int inputIndex) const noexcept override; + bool canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept override; void attachToContext( cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept override; - void configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, + void configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept override; + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept override; void detachFromContext() noexcept override; private: void deserialize(int8_t const* data, size_t length); - void check_valid_inputs(nvinfer1::Dims const* inputs, int nbInputDims); + void check_valid_inputs(nvinfer1::Dims const* inputs, int32_t nbInputDims); void generate_pyramid_anchors(nvinfer1::Dims const& imageDims); int32_t mBackgroundLabel{}; @@ -133,8 +134,8 @@ class ProposalLayerPluginCreator : public nvinfer1::pluginInternal::BaseCreator private: static PluginFieldCollection mFC; - int mPreNMSTopK; - int mKeepTopK; + int32_t mPreNMSTopK; + int32_t mKeepTopK; float mIOUThreshold; static std::vector mPluginAttributes; }; diff --git a/plugin/pyramidROIAlignPlugin/pyramidROIAlignPlugin.cpp b/plugin/pyramidROIAlignPlugin/pyramidROIAlignPlugin.cpp index 86ca4482..486377c0 100644 --- a/plugin/pyramidROIAlignPlugin/pyramidROIAlignPlugin.cpp +++ b/plugin/pyramidROIAlignPlugin/pyramidROIAlignPlugin.cpp @@ -132,11 +132,6 @@ IPluginV2Ext* PyramidROIAlignPluginCreator::createPlugin(char const* name, Plugi PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); samplingRatio = *(static_cast(fields[i].data)); PLUGIN_VALIDATE(samplingRatio >= 0); - } - if (!strcmp(attrName, "legacy")) - { - PLUGIN_ASSERT(fields[i].type == PluginFieldType::kINT32); - legacy = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "legacy")) { @@ -202,7 +197,7 @@ void PyramidROIAlign::destroy() noexcept delete this; } -size_t PyramidROIAlign::getWorkspaceSize(int) const noexcept +size_t PyramidROIAlign::getWorkspaceSize(int32_t) const noexcept { return 0; } @@ -290,7 +285,6 @@ Dims PyramidROIAlign::getOutputDimensions(int32_t index, Dims const* inputs, int int32_t PyramidROIAlign::enqueue( int32_t batch_size, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept - { void* const pooled = outputs[0]; cudaError_t status; @@ -331,18 +325,18 @@ int32_t PyramidROIAlign::enqueue( size_t PyramidROIAlign::getSerializationSize() const noexcept { - return sizeof(int) * 2 // mPooledSize - + sizeof(int) * 2 // mImageSize - + sizeof(int) // mFeatureLength - + sizeof(int) // mROICount - + sizeof(int) // mFPNScale - + sizeof(int) // mTransformCoords - + sizeof(bool) // mAbsCoords - + sizeof(bool) // mSwapCoords - + sizeof(bool) // mPlusOneCoords - + sizeof(int) // mSamplingRatio - + sizeof(bool) // mIsLegacy - + sizeof(int) * 8; // mFeatureSpatialSize + return sizeof(int32_t) * 2 // mPooledSize + + sizeof(int32_t) * 2 // mImageSize + + sizeof(int32_t) // mFeatureLength + + sizeof(int32_t) // mROICount + + sizeof(int32_t) // mFPNScale + + sizeof(int32_t) // mTransformCoords + + sizeof(bool) // mAbsCoords + + sizeof(bool) // mSwapCoords + + sizeof(bool) // mPlusOneCoords + + sizeof(int32_t) // mSamplingRatio + + sizeof(bool) // mIsLegacy + + sizeof(int32_t) * 8; // mFeatureSpatialSize } void PyramidROIAlign::serialize(void* buffer) const noexcept diff --git a/plugin/pyramidROIAlignPlugin/pyramidROIAlignPlugin.h b/plugin/pyramidROIAlignPlugin/pyramidROIAlignPlugin.h index feca0b7b..dde6a309 100644 --- a/plugin/pyramidROIAlignPlugin/pyramidROIAlignPlugin.h +++ b/plugin/pyramidROIAlignPlugin/pyramidROIAlignPlugin.h @@ -52,7 +52,7 @@ class PyramidROIAlign : public IPluginV2Ext void destroy() noexcept override; - size_t getWorkspaceSize(int) const noexcept override; + size_t getWorkspaceSize(int32_t) const noexcept override; int32_t enqueue(int32_t batch_size, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; diff --git a/plugin/regionPlugin/regionPlugin.cpp b/plugin/regionPlugin/regionPlugin.cpp index 61e2c889..c6f709eb 100644 --- a/plugin/regionPlugin/regionPlugin.cpp +++ b/plugin/regionPlugin/regionPlugin.cpp @@ -37,7 +37,7 @@ void safeFree(T* ptr) } template -void allocateChunk(T*& ptr, int count) +void allocateChunk(T*& ptr, int32_t count) { ptr = static_cast(malloc(count * sizeof(T))); } @@ -55,7 +55,7 @@ struct SoftmaxTreeDeleter safeFree(smTree->group); if (smTree->name) { - for (int i = 0; i < smTree->n; i++) + for (int32_t i = 0; i < smTree->n; i++) { safeFree(smTree->name[i]); } @@ -83,7 +83,7 @@ Region::Region(RegionParameters params) { } -Region::Region(RegionParameters params, int C, int H, int W) +Region::Region(RegionParameters params, int32_t C, int32_t H, int32_t W) : num(params.num) , coords(params.coords) , classes(params.classes) @@ -97,12 +97,12 @@ Region::Region(RegionParameters params, int C, int H, int W) Region::Region(void const* buffer, size_t length) { char const *d = reinterpret_cast(buffer), *a = d; - C = read(d); - H = read(d); - W = read(d); - num = read(d); - classes = read(d); - coords = read(d); + C = read(d); + H = read(d); + W = read(d); + num = read(d); + classes = read(d); + coords = read(d); bool softmaxTreePresent = read(d); bool leafPresent = read(d); bool parentPresent = read(d); @@ -117,7 +117,7 @@ Region::Region(void const* buffer, size_t length) // need to read each element individually allocateChunk(smTreeTemp, 1); - smTreeTemp->n = read(d); + smTreeTemp->n = read(d); if (leafPresent) { @@ -152,23 +152,23 @@ Region::Region(void const* buffer, size_t length) smTreeTemp->group = nullptr; } - for (int i = 0; i < smTreeTemp->n; i++) + for (int32_t i = 0; i < smTreeTemp->n; i++) { if (leafPresent) { - smTreeTemp->leaf[i] = read(d); + smTreeTemp->leaf[i] = read(d); } if (parentPresent) { - smTreeTemp->parent[i] = read(d); + smTreeTemp->parent[i] = read(d); } if (childPresent) { - smTreeTemp->child[i] = read(d); + smTreeTemp->child[i] = read(d); } if (groupPresent) { - smTreeTemp->group[i] = read(d); + smTreeTemp->group[i] = read(d); } } @@ -183,17 +183,17 @@ Region::Region(void const* buffer, size_t length) if (namePresent) { - for (int i = 0; i < smTreeTemp->n; i++) + for (int32_t i = 0; i < smTreeTemp->n; i++) { allocateChunk(smTreeTemp->name[i], 256); - for (int j = 0; j < 256; j++) + for (int32_t j = 0; j < 256; j++) { smTreeTemp->name[i][j] = read(d); } } } - smTreeTemp->groups = read(d); + smTreeTemp->groups = read(d); if (groupSizePresent) { allocateChunk(smTreeTemp->groupSize, smTreeTemp->groups); @@ -210,15 +210,15 @@ Region::Region(void const* buffer, size_t length) { smTreeTemp->groupOffset = nullptr; } - for (int i = 0; i < smTreeTemp->groups; i++) + for (int32_t i = 0; i < smTreeTemp->groups; i++) { if (groupSizePresent) { - smTreeTemp->groupSize[i] = read(d); + smTreeTemp->groupSize[i] = read(d); } if (groupOffsetPresent) { - smTreeTemp->groupOffset[i] = read(d); + smTreeTemp->groupOffset[i] = read(d); } } smTree = std::shared_ptr(smTreeTemp, SoftmaxTreeDeleter()); @@ -230,20 +230,20 @@ Region::Region(void const* buffer, size_t length) PLUGIN_VALIDATE(d == a + length); } -int Region::getNbOutputs() const noexcept +int32_t Region::getNbOutputs() const noexcept { return 1; } -Dims Region::getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept +Dims Region::getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept { PLUGIN_ASSERT(nbInputDims == 1); PLUGIN_ASSERT(index == 0); return inputs[0]; } -int Region::enqueue( - int batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept +int32_t Region::enqueue( + int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept { void const* inputData = inputs[0]; void* outputData = outputs[0]; @@ -263,26 +263,26 @@ int Region::enqueue( size_t Region::getSerializationSize() const noexcept { // C, H, W, num, classes, coords, smTree !nullptr and other array members !nullptr, softmaxTree members - size_t count = 6 * sizeof(int) + 8 * sizeof(bool); + size_t count = 6 * sizeof(int32_t) + 8 * sizeof(bool); if (smTree.get()) { - count += 2 * sizeof(int); + count += 2 * sizeof(int32_t); if (smTree->leaf) { - count += smTree->n * sizeof(int); + count += smTree->n * sizeof(int32_t); } if (smTree->parent) { - count += smTree->n * sizeof(int); + count += smTree->n * sizeof(int32_t); } if (smTree->child) { - count += smTree->n * sizeof(int); + count += smTree->n * sizeof(int32_t); } if (smTree->group) { - count += smTree->n * sizeof(int); + count += smTree->n * sizeof(int32_t); } if (smTree->name) { @@ -290,11 +290,11 @@ size_t Region::getSerializationSize() const noexcept } if (smTree->groupSize) { - count += smTree->groups * sizeof(int); + count += smTree->groups * sizeof(int32_t); } if (smTree->groupOffset) { - count += smTree->groups * sizeof(int); + count += smTree->groups * sizeof(int32_t); } } return count; @@ -321,7 +321,7 @@ void Region::serialize(void* buffer) const noexcept if (smTree) { write(d, smTree->n); - for (int i = 0; i < smTree->n; i++) + for (int32_t i = 0; i < smTree->n; i++) { if (smTree->leaf) { @@ -342,17 +342,17 @@ void Region::serialize(void* buffer) const noexcept } if (smTree->name) { - for (int i = 0; i < smTree->n; i++) + for (int32_t i = 0; i < smTree->n; i++) { char const* str = smTree->name[i]; - for (int j = 0; j < 256; j++) + for (int32_t j = 0; j < 256; j++) { write(d, str[j]); } } } write(d, smTree->groups); - for (int i = 0; i < smTree->groups; i++) + for (int32_t i = 0; i < smTree->groups; i++) { if (smTree->groupSize) { @@ -372,7 +372,7 @@ bool Region::supportsFormat(DataType type, PluginFormat format) const noexcept return (type == DataType::kFLOAT && format == PluginFormat::kLINEAR); } -int Region::initialize() noexcept +int32_t Region::initialize() noexcept { return STATUS_SUCCESS; } @@ -389,7 +389,7 @@ char const* Region::getPluginVersion() const noexcept return kREGION_PLUGIN_VERSION; } -size_t Region::getWorkspaceSize(int maxBatchSize) const noexcept +size_t Region::getWorkspaceSize(int32_t maxBatchSize) const noexcept { return 0; } @@ -429,28 +429,29 @@ char const* Region::getPluginNamespace() const noexcept } // Return the DataType of the plugin output at the requested index -DataType Region::getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept +DataType Region::getOutputDataType(int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { PLUGIN_ASSERT(index == 0); return DataType::kFLOAT; } // Return true if output tensor is broadcast across a batch. -bool Region::isOutputBroadcastAcrossBatch(int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept +bool Region::isOutputBroadcastAcrossBatch( + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept { return false; } // Return true if plugin can use input that is broadcast across batch without replication. -bool Region::canBroadcastInputAcrossBatch(int inputIndex) const noexcept +bool Region::canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept { return false; } // Configure the layer with input and output data types. -void Region::configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, +void Region::configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept { PLUGIN_ASSERT(*inputTypes == DataType::kFLOAT && floatFormat == PluginFormat::kLINEAR); PLUGIN_ASSERT(nbInputs == 1); @@ -507,23 +508,23 @@ IPluginV2Ext* RegionPluginCreator::createPlugin(char const* name, PluginFieldCol try { PluginField const* fields = fc->fields; - for (int i = 0; i < fc->nbFields; ++i) + for (int32_t i = 0; i < fc->nbFields; ++i) { char const* attrName = fields[i].name; if (!strcmp(attrName, "num")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.num = *(static_cast(fields[i].data)); + params.num = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "coords")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.coords = *(static_cast(fields[i].data)); + params.coords = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "classes")) { PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32); - params.classes = *(static_cast(fields[i].data)); + params.classes = *(static_cast(fields[i].data)); } if (!strcmp(attrName, "smTree")) { diff --git a/plugin/regionPlugin/regionPlugin.h b/plugin/regionPlugin/regionPlugin.h index 9ec4cc33..7af234f1 100644 --- a/plugin/regionPlugin/regionPlugin.h +++ b/plugin/regionPlugin/regionPlugin.h @@ -32,23 +32,23 @@ class Region : public IPluginV2Ext public: Region(RegionParameters params); - Region(RegionParameters params, int C, int H, int W); + Region(RegionParameters params, int32_t C, int32_t H, int32_t W); Region(void const* buffer, size_t length); ~Region() override = default; - int getNbOutputs() const noexcept override; + int32_t getNbOutputs() const noexcept override; - Dims getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept override; + Dims getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept override; - int initialize() noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; - size_t getWorkspaceSize(int maxBatchSize) const noexcept override; + size_t getWorkspaceSize(int32_t maxBatchSize) const noexcept override; - int enqueue(int batchSize, void const* const* inputs, void* const* outputs, void* workspace, + int32_t enqueue(int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; size_t getSerializationSize() const noexcept override; @@ -69,19 +69,20 @@ class Region : public IPluginV2Ext char const* getPluginNamespace() const noexcept override; - DataType getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + DataType getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; bool isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept override; + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept override; - bool canBroadcastInputAcrossBatch(int inputIndex) const noexcept override; + bool canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept override; void attachToContext( cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept override; - void configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, + void configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept override; + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept override; void detachFromContext() noexcept override; @@ -91,11 +92,11 @@ class Region : public IPluginV2Ext } private: - int num; - int coords; - int classes; + int32_t num; + int32_t coords; + int32_t classes; std::shared_ptr smTree; - int C, H, W; + int32_t C, H, W; bool hasSoftmaxTree; std::string mPluginNamespace; }; diff --git a/plugin/reorgPlugin/reorgPlugin.cpp b/plugin/reorgPlugin/reorgPlugin.cpp index 15bd98f0..b1d1ff21 100644 --- a/plugin/reorgPlugin/reorgPlugin.cpp +++ b/plugin/reorgPlugin/reorgPlugin.cpp @@ -25,7 +25,7 @@ static char const* const kREORG_PLUGIN_NAME{"Reorg_TRT"}; PluginFieldCollection ReorgPluginCreator::mFC{}; std::vector ReorgPluginCreator::mPluginAttributes; -Reorg::Reorg(int C, int H, int W, int stride) +Reorg::Reorg(int32_t C, int32_t H, int32_t W, int32_t stride) : C(C) , H(H) , W(W) @@ -33,7 +33,7 @@ Reorg::Reorg(int C, int H, int W, int stride) { } -Reorg::Reorg(int stride) +Reorg::Reorg(int32_t stride) : stride(stride) { } @@ -41,27 +41,27 @@ Reorg::Reorg(int stride) Reorg::Reorg(void const* buffer, size_t length) { char const *d = reinterpret_cast(buffer), *a = d; - C = read(d); - H = read(d); - W = read(d); - stride = read(d); + C = read(d); + H = read(d); + W = read(d); + stride = read(d); PLUGIN_VALIDATE(d == a + length); } -int Reorg::getNbOutputs() const noexcept +int32_t Reorg::getNbOutputs() const noexcept { return 1; } -Dims Reorg::getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept +Dims Reorg::getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept { PLUGIN_ASSERT(nbInputDims == 1); PLUGIN_ASSERT(index == 0); return Dims3(inputs[0].d[0] * stride * stride, inputs[0].d[1] / stride, inputs[0].d[2] / stride); } -int Reorg::enqueue( - int batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept +int32_t Reorg::enqueue( + int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept { void const* inputData = inputs[0]; void* outputData = outputs[0]; @@ -72,7 +72,7 @@ int Reorg::enqueue( size_t Reorg::getSerializationSize() const noexcept { // C, H, W, stride - return sizeof(int) * 4; + return sizeof(int32_t) * 4; } void Reorg::serialize(void* buffer) const noexcept @@ -90,14 +90,14 @@ bool Reorg::supportsFormat(DataType type, PluginFormat format) const noexcept return (type == DataType::kFLOAT && format == PluginFormat::kLINEAR); } -int Reorg::initialize() noexcept +int32_t Reorg::initialize() noexcept { return STATUS_SUCCESS; } void Reorg::terminate() noexcept {} -size_t Reorg::getWorkspaceSize(int maxBatchSize) const noexcept +size_t Reorg::getWorkspaceSize(int32_t maxBatchSize) const noexcept { return 0; } @@ -129,7 +129,7 @@ char const* Reorg::getPluginNamespace() const noexcept } // Return the DataType of the plugin output at the requested index -DataType Reorg::getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept +DataType Reorg::getOutputDataType(int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { // Only 1 input and 1 output from the plugin layer PLUGIN_ASSERT(index == 0); @@ -139,21 +139,22 @@ DataType Reorg::getOutputDataType(int index, nvinfer1::DataType const* inputType } // Return true if output tensor is broadcast across a batch. -bool Reorg::isOutputBroadcastAcrossBatch(int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept +bool Reorg::isOutputBroadcastAcrossBatch( + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept { return false; } // Return true if plugin can use input that is broadcast across batch without replication. -bool Reorg::canBroadcastInputAcrossBatch(int inputIndex) const noexcept +bool Reorg::canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept { return false; } // Configure the layer with input and output data types. -void Reorg::configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, +void Reorg::configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept { PLUGIN_ASSERT(*inputTypes == DataType::kFLOAT && floatFormat == PluginFormat::kLINEAR); PLUGIN_ASSERT(nbInputs == 1); diff --git a/plugin/reorgPlugin/reorgPlugin.h b/plugin/reorgPlugin/reorgPlugin.h index 405c2808..f0f999b5 100644 --- a/plugin/reorgPlugin/reorgPlugin.h +++ b/plugin/reorgPlugin/reorgPlugin.h @@ -29,25 +29,25 @@ namespace plugin class Reorg : public IPluginV2Ext { public: - Reorg(int stride); + Reorg(int32_t stride); - Reorg(int C, int H, int W, int stride); + Reorg(int32_t C, int32_t H, int32_t W, int32_t stride); Reorg(void const* buffer, size_t length); ~Reorg() override = default; - int getNbOutputs() const noexcept override; + int32_t getNbOutputs() const noexcept override; - Dims getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept override; + Dims getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept override; - int initialize() noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; - size_t getWorkspaceSize(int maxBatchSize) const noexcept override; + size_t getWorkspaceSize(int32_t maxBatchSize) const noexcept override; - int enqueue(int batchSize, void const* const* inputs, void* const* outputs, void* workspace, + int32_t enqueue(int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; size_t getSerializationSize() const noexcept override; @@ -68,25 +68,26 @@ class Reorg : public IPluginV2Ext char const* getPluginNamespace() const noexcept override; - DataType getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + DataType getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; bool isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept override; + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept override; - bool canBroadcastInputAcrossBatch(int inputIndex) const noexcept override; + bool canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept override; void attachToContext( cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept override; - void configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, + void configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept override; + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept override; void detachFromContext() noexcept override; private: - int C{}, H{}, W{}; - int stride{}; + int32_t C{}, H{}, W{}; + int32_t stride{}; std::string mPluginNamespace; }; @@ -109,7 +110,7 @@ class ReorgPluginCreator : public nvinfer1::pluginInternal::BaseCreator private: static PluginFieldCollection mFC; - int stride{}; + int32_t stride{}; static std::vector mPluginAttributes; }; } // namespace plugin diff --git a/plugin/resizeNearestPlugin/resizeNearestPlugin.cpp b/plugin/resizeNearestPlugin/resizeNearestPlugin.cpp index 22f56ffb..3f88a17b 100644 --- a/plugin/resizeNearestPlugin/resizeNearestPlugin.cpp +++ b/plugin/resizeNearestPlugin/resizeNearestPlugin.cpp @@ -103,23 +103,23 @@ ResizeNearest::ResizeNearest(float scale) PLUGIN_VALIDATE(mScale > 0); } -int ResizeNearest::getNbOutputs() const noexcept +int32_t ResizeNearest::getNbOutputs() const noexcept { return 1; } -Dims ResizeNearest::getOutputDimensions(int index, Dims const* inputDims, int nbInputs) noexcept +Dims ResizeNearest::getOutputDimensions(int32_t index, Dims const* inputDims, int32_t nbInputs) noexcept { PLUGIN_ASSERT(nbInputs == 1); nvinfer1::Dims const& input = inputDims[0]; PLUGIN_ASSERT(index == 0); nvinfer1::Dims output{}; output.nbDims = input.nbDims; - for (int d = 0; d < input.nbDims; ++d) + for (int32_t d = 0; d < input.nbDims; ++d) { if (d == input.nbDims - 2 || d == input.nbDims - 1) { - output.d[d] = int(input.d[d] * mScale); + output.d[d] = int32_t(input.d[d] * mScale); } else { @@ -129,7 +129,7 @@ Dims ResizeNearest::getOutputDimensions(int index, Dims const* inputDims, int nb return output; } -int ResizeNearest::initialize() noexcept +int32_t ResizeNearest::initialize() noexcept { return 0; } @@ -141,7 +141,7 @@ void ResizeNearest::destroy() noexcept delete this; } -size_t ResizeNearest::getWorkspaceSize(int) const noexcept +size_t ResizeNearest::getWorkspaceSize(int32_t) const noexcept { return 0; } @@ -149,7 +149,7 @@ size_t ResizeNearest::getWorkspaceSize(int) const noexcept size_t ResizeNearest::getSerializationSize() const noexcept { // scale, dimensions: 3 * 2 - return sizeof(float) + sizeof(int) * 3 * 2; + return sizeof(float) + sizeof(int32_t) * 3 * 2; } void ResizeNearest::serialize(void* buffer) const noexcept @@ -225,17 +225,17 @@ bool ResizeNearest::supportsFormat(DataType type, PluginFormat format) const noe return (type == DataType::kFLOAT && format == PluginFormat::kLINEAR); } -int ResizeNearest::enqueue( - int batch_size, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept +int32_t ResizeNearest::enqueue( + int32_t batch_size, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept { - int nchan = mOutputDims.d[0]; + int32_t nchan = mOutputDims.d[0]; float scale = mScale; int2 osize = {mOutputDims.d[2], mOutputDims.d[1]}; - int istride = mInputDims.d[2]; - int ostride = mOutputDims.d[2]; - int ibatchstride = mInputDims.d[1] * istride; - int obatchstride = mOutputDims.d[1] * ostride; + int32_t istride = mInputDims.d[2]; + int32_t ostride = mOutputDims.d[2]; + int32_t ibatchstride = mInputDims.d[1] * istride; + int32_t obatchstride = mOutputDims.d[1] * ostride; dim3 block(32, 16); dim3 grid((osize.x - 1) / block.x + 1, (osize.y - 1) / block.y + 1, std::min(batch_size * nchan, 65535)); @@ -246,7 +246,8 @@ int ResizeNearest::enqueue( } // Return the DataType of the plugin output at the requested index -DataType ResizeNearest::getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept +DataType ResizeNearest::getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { // Only 1 input and 1 output from the plugin layer PLUGIN_ASSERT(index == 0); @@ -257,21 +258,21 @@ DataType ResizeNearest::getOutputDataType(int index, nvinfer1::DataType const* i // Return true if output tensor is broadcast across a batch. bool ResizeNearest::isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept { return false; } // Return true if plugin can use input that is broadcast across batch without replication. -bool ResizeNearest::canBroadcastInputAcrossBatch(int inputIndex) const noexcept +bool ResizeNearest::canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept { return false; } // Configure the layer with input and output data types. -void ResizeNearest::configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, +void ResizeNearest::configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept { PLUGIN_ASSERT(nbInputs == 1); mInputDims = inputDims[0]; diff --git a/plugin/resizeNearestPlugin/resizeNearestPlugin.h b/plugin/resizeNearestPlugin/resizeNearestPlugin.h index ed8ff38a..5db5fc49 100644 --- a/plugin/resizeNearestPlugin/resizeNearestPlugin.h +++ b/plugin/resizeNearestPlugin/resizeNearestPlugin.h @@ -39,19 +39,19 @@ class ResizeNearest : public IPluginV2Ext ~ResizeNearest() override = default; - int getNbOutputs() const noexcept override; + int32_t getNbOutputs() const noexcept override; - Dims getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept override; + Dims getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept override; - int initialize() noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; void destroy() noexcept override; - size_t getWorkspaceSize(int) const noexcept override; + size_t getWorkspaceSize(int32_t) const noexcept override; - int enqueue(int batch_size, void const* const* inputs, void* const* outputs, void* workspace, + int32_t enqueue(int32_t batch_size, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; size_t getSerializationSize() const noexcept override; @@ -70,19 +70,20 @@ class ResizeNearest : public IPluginV2Ext char const* getPluginNamespace() const noexcept override; - DataType getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + DataType getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; bool isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept override; + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept override; - bool canBroadcastInputAcrossBatch(int inputIndex) const noexcept override; + bool canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept override; void attachToContext( cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept override; - void configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, + void configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept override; + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept override; void detachFromContext() noexcept override; diff --git a/plugin/scatterPlugin/scatterPlugin.cpp b/plugin/scatterPlugin/scatterPlugin.cpp index 3dbe44ae..694d4cfe 100644 --- a/plugin/scatterPlugin/scatterPlugin.cpp +++ b/plugin/scatterPlugin/scatterPlugin.cpp @@ -37,7 +37,7 @@ PluginFieldCollection ScatterNDPluginCreator::mFC{}; ScatterND::ScatterND() {} -int ScatterND::getNbOutputs() const noexcept +int32_t ScatterND::getNbOutputs() const noexcept { // Plugin layer has 1 output return 1; @@ -51,7 +51,7 @@ DimsExprs ScatterND::getOutputDimensions( return ret; } -int ScatterND::initialize() noexcept +int32_t ScatterND::initialize() noexcept { return 0; } @@ -90,7 +90,7 @@ void ScatterND::configurePlugin( int32_t ScatterND::calculateNumSlices(Dims indexTensorDims) const noexcept { int32_t nSlices = 1; - for (int i = 0; i < indexTensorDims.nbDims - 1; i++) + for (int32_t i = 0; i < indexTensorDims.nbDims - 1; i++) { nSlices *= indexTensorDims.d[i]; } @@ -106,7 +106,7 @@ size_t ScatterND::getWorkspaceSize( } void ScatterND::calculateTransformCoeff( - Dims const& dataTensorDims, int indexRank, int32_t* transformCoeff) const noexcept + Dims const& dataTensorDims, int32_t indexRank, int32_t* transformCoeff) const noexcept { std::vector pitches; for (int32_t i = indexRank - 1, nIndx = 1; i >= 0; i--) @@ -123,7 +123,7 @@ void ScatterND::calculateTransformCoeff( int32_t ScatterND::calculateCopySize(Dims const& dataDims) const noexcept { int32_t copySize = 1; - for (int i = 0; i < dataDims.nbDims; i++) + for (int32_t i = 0; i < dataDims.nbDims; i++) { copySize *= dataDims.d[i]; } @@ -158,7 +158,7 @@ int32_t ScatterND::enqueue(PluginTensorDesc const* inputDesc, PluginTensorDesc c case DataType::kFP8: PLUGIN_FAIL("FP8 not supported"); break; } - for (int i = indexRank; i < dataDims.nbDims; i++) + for (int32_t i = indexRank; i < dataDims.nbDims; i++) { rowSize *= dataDims.d[i]; } @@ -194,7 +194,8 @@ char const* ScatterND::getPluginNamespace() const noexcept } // Return the DataType of the plugin output at the requested index -DataType ScatterND::getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept +DataType ScatterND::getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { PLUGIN_ASSERT(index == 0); return inputTypes[dataTensorIdx]; diff --git a/plugin/scatterPlugin/scatterPlugin.h b/plugin/scatterPlugin/scatterPlugin.h index d5b23c0e..3545aa57 100644 --- a/plugin/scatterPlugin/scatterPlugin.h +++ b/plugin/scatterPlugin/scatterPlugin.h @@ -35,7 +35,7 @@ class ScatterND : public IPluginV2DynamicExt ~ScatterND() override = default; - int getNbOutputs() const noexcept override; + int32_t getNbOutputs() const noexcept override; DimsExprs getOutputDimensions( int32_t outputIndex, DimsExprs const* inputs, int32_t nbInputs, IExprBuilder& exprBuilder) noexcept override; @@ -49,7 +49,7 @@ class ScatterND : public IPluginV2DynamicExt size_t getWorkspaceSize(PluginTensorDesc const* inputs, int32_t nbInputs, PluginTensorDesc const* outputs, int32_t nbOutputs) const noexcept override; - int initialize() noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; @@ -72,7 +72,8 @@ class ScatterND : public IPluginV2DynamicExt char const* getPluginNamespace() const noexcept override; - DataType getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + DataType getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; void attachToContext( cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept override; @@ -83,12 +84,12 @@ class ScatterND : public IPluginV2DynamicExt // calculate how many slices we need to scatter = reduce_mul(indexTensor.shape[:-1]) int32_t calculateNumSlices(Dims indexTensorDims) const noexcept; int32_t calculateCopySize(Dims const& dataDims) const noexcept; - void calculateTransformCoeff(Dims const& dataTensorDims, int indexRank, int32_t* transformCoeff) const noexcept; + void calculateTransformCoeff(Dims const& dataTensorDims, int32_t indexRank, int32_t* transformCoeff) const noexcept; std::string mPluginNamespace; - static constexpr int indexTensorIdx = 1; - static constexpr int updateTensorIdx = 2; - static constexpr int dataTensorIdx = 0; + static constexpr int32_t indexTensorIdx = 1; + static constexpr int32_t updateTensorIdx = 2; + static constexpr int32_t dataTensorIdx = 0; }; class ScatterNDPluginCreator : public nvinfer1::pluginInternal::BaseCreator diff --git a/plugin/skipLayerNormPlugin/CustomSkipLayerNormPluginDynamic_PluginConfig.yaml b/plugin/skipLayerNormPlugin/CustomSkipLayerNormPluginDynamic_PluginConfig.yaml index 3ae0de8f..a2f309c3 100644 --- a/plugin/skipLayerNormPlugin/CustomSkipLayerNormPluginDynamic_PluginConfig.yaml +++ b/plugin/skipLayerNormPlugin/CustomSkipLayerNormPluginDynamic_PluginConfig.yaml @@ -88,4 +88,38 @@ versions: - ld - beta - gamma + golden_reference_script: "plugin/skipLayerNormPlugin/CustomSkipLayerNormPluginDynamic_PluginReference.py" + abs_tol: 1e-2 + rel_tol: 1e-2 + configs: + config1: + input_types: + input: float32 + skip: float32 + attribute_options: + type_id: + value: 0 + ld: + value: 128 + beta: + shape: "1, 1, 128" + gamma: + shape: "1, 1, 128" + bias: + shape: "1, 1, 128" + config2: + input_types: + input: float16 + skip: float16 + attribute_options: + type_id: + value: 1 + ld: + value: 768 + beta: + shape: "1, 1, 768" + gamma: + shape: "1, 1, 768" + bias: + shape: "1, 1, 768" ... diff --git a/plugin/skipLayerNormPlugin/skipLayerNormPlugin.cpp b/plugin/skipLayerNormPlugin/skipLayerNormPlugin.cpp index 37ab791a..d95f0df6 100644 --- a/plugin/skipLayerNormPlugin/skipLayerNormPlugin.cpp +++ b/plugin/skipLayerNormPlugin/skipLayerNormPlugin.cpp @@ -331,7 +331,7 @@ int32_t SkipLayerNormPluginDynamic::enqueue(PluginTensorDesc const* inputDesc, P else { PLUGIN_ERROR(("Unsupported type error, expected [kINT8,kHALF,kFLOAT], but received " - + std::to_string(static_cast(iType))) + + std::to_string(static_cast(iType))) .c_str()); } } @@ -857,7 +857,7 @@ int32_t SkipLayerNormVarSeqlenPlugin::enqueue(PluginTensorDesc const* inputDesc, else { PLUGIN_VALIDATE(("Unsupported type error, expected [kINT8,kHALF,kFLOAT], but received " - + std::to_string(static_cast(iType))) + + std::to_string(static_cast(iType))) .c_str()); } } diff --git a/plugin/specialSlicePlugin/specialSlicePlugin.cpp b/plugin/specialSlicePlugin/specialSlicePlugin.cpp index a8760027..c8721596 100644 --- a/plugin/specialSlicePlugin/specialSlicePlugin.cpp +++ b/plugin/specialSlicePlugin/specialSlicePlugin.cpp @@ -79,7 +79,7 @@ IPluginV2Ext* SpecialSlicePluginCreator::deserializePlugin(char const* name, voi return nullptr; } -size_t SpecialSlice::getWorkspaceSize(int) const noexcept +size_t SpecialSlice::getWorkspaceSize(int32_t) const noexcept { return 0; } @@ -126,7 +126,7 @@ char const* SpecialSlice::getPluginNamespace() const noexcept size_t SpecialSlice::getSerializationSize() const noexcept { - return sizeof(int); + return sizeof(int32_t); } void SpecialSlice::serialize(void* buffer) const noexcept @@ -139,23 +139,23 @@ void SpecialSlice::serialize(void* buffer) const noexcept SpecialSlice::SpecialSlice(void const* data, size_t length) { char const *d = reinterpret_cast(data), *a = d; - mBboxesCnt = read(d); + mBboxesCnt = read(d); PLUGIN_VALIDATE(d == a + length); } SpecialSlice::SpecialSlice() {} -int SpecialSlice::initialize() noexcept +int32_t SpecialSlice::initialize() noexcept { return 0; } -int SpecialSlice::getNbOutputs() const noexcept +int32_t SpecialSlice::getNbOutputs() const noexcept { return 1; } -void SpecialSlice::check_valid_inputs(nvinfer1::Dims const* inputs, int nbInputDims) +void SpecialSlice::check_valid_inputs(nvinfer1::Dims const* inputs, int32_t nbInputDims) { PLUGIN_ASSERT(nbInputDims == 1); @@ -163,7 +163,7 @@ void SpecialSlice::check_valid_inputs(nvinfer1::Dims const* inputs, int nbInputD PLUGIN_ASSERT(inputs[0].nbDims == 2 && inputs[0].d[1] == 6); } -Dims SpecialSlice::getOutputDimensions(int index, Dims const* inputDims, int nbInputs) noexcept +Dims SpecialSlice::getOutputDimensions(int32_t index, Dims const* inputDims, int32_t nbInputs) noexcept { PLUGIN_ASSERT(index == 0); @@ -180,8 +180,8 @@ Dims SpecialSlice::getOutputDimensions(int index, Dims const* inputDims, int nbI return output; } -int SpecialSlice::enqueue( - int batch_size, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept +int32_t SpecialSlice::enqueue( + int32_t batch_size, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept { specialSlice(stream, batch_size, mBboxesCnt, inputs[0], outputs[0]); @@ -190,7 +190,8 @@ int SpecialSlice::enqueue( } // Return the DataType of the plugin output at the requested index -DataType SpecialSlice::getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept +DataType SpecialSlice::getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept { // Only 1 input and 1 output from the plugin layer PLUGIN_ASSERT(index == 0); @@ -201,21 +202,21 @@ DataType SpecialSlice::getOutputDataType(int index, nvinfer1::DataType const* in // Return true if output tensor is broadcast across a batch. bool SpecialSlice::isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept { return false; } // Return true if plugin can use input that is broadcast across batch without replication. -bool SpecialSlice::canBroadcastInputAcrossBatch(int inputIndex) const noexcept +bool SpecialSlice::canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept { return false; } // Configure the layer with input and output data types. -void SpecialSlice::configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, +void SpecialSlice::configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept { PLUGIN_ASSERT(nbInputs == 1); diff --git a/plugin/specialSlicePlugin/specialSlicePlugin.h b/plugin/specialSlicePlugin/specialSlicePlugin.h index abd43107..5fbe2835 100644 --- a/plugin/specialSlicePlugin/specialSlicePlugin.h +++ b/plugin/specialSlicePlugin/specialSlicePlugin.h @@ -39,13 +39,13 @@ class SpecialSlice : public IPluginV2Ext ~SpecialSlice() override = default; - int getNbOutputs() const noexcept override; + int32_t getNbOutputs() const noexcept override; - void check_valid_inputs(nvinfer1::Dims const* inputs, int nbInputDims); + void check_valid_inputs(nvinfer1::Dims const* inputs, int32_t nbInputDims); - Dims getOutputDimensions(int index, Dims const* inputs, int nbInputDims) noexcept override; + Dims getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept override; - int initialize() noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override{}; @@ -54,9 +54,9 @@ class SpecialSlice : public IPluginV2Ext delete this; } - size_t getWorkspaceSize(int) const noexcept override; + size_t getWorkspaceSize(int32_t) const noexcept override; - int enqueue(int batch_size, void const* const* inputs, void* const* outputs, void* workspace, + int32_t enqueue(int32_t batch_size, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; size_t getSerializationSize() const noexcept override; @@ -75,24 +75,25 @@ class SpecialSlice : public IPluginV2Ext char const* getPluginNamespace() const noexcept override; - DataType getOutputDataType(int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; + DataType getOutputDataType( + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; bool isOutputBroadcastAcrossBatch( - int outputIndex, bool const* inputIsBroadcasted, int nbInputs) const noexcept override; + int32_t outputIndex, bool const* inputIsBroadcasted, int32_t nbInputs) const noexcept override; - bool canBroadcastInputAcrossBatch(int inputIndex) const noexcept override; + bool canBroadcastInputAcrossBatch(int32_t inputIndex) const noexcept override; void attachToContext( cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept override; - void configurePlugin(Dims const* inputDims, int nbInputs, Dims const* outputDims, int nbOutputs, + void configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast, - bool const* outputIsBroadcast, PluginFormat floatFormat, int maxBatchSize) noexcept override; + bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept override; void detachFromContext() noexcept override; private: - int mBboxesCnt; + int32_t mBboxesCnt; std::string mNameSpace; }; diff --git a/plugin/splitPlugin/split.cu b/plugin/splitPlugin/split.cu index a3389e5b..9cdf2af8 100644 --- a/plugin/splitPlugin/split.cu +++ b/plugin/splitPlugin/split.cu @@ -17,7 +17,8 @@ #include #include -#include "splitPlugin.h" + +#include "split.h" using namespace nvinfer1; using nvinfer1::plugin::SplitPlugin; @@ -73,6 +74,64 @@ void split_kernel(int nsegment, } } +bool SplitPlugin::supportsFormatCombination( + int pos, const nvinfer1::PluginTensorDesc* inOut, int nbInputs, int nbOutputs) noexcept +{ + PLUGIN_ASSERT(inOut && pos < (nbInputs + nbOutputs)); + return (inOut[pos].format == nvinfer1::PluginFormat::kLINEAR); +} + +nvinfer1::DataType SplitPlugin::getOutputDataType(int index, const nvinfer1::DataType* inputTypes, int nbInputs) const + noexcept +{ + PLUGIN_ASSERT(inputTypes && nbInputs > 0); + return inputTypes[0]; +} + +int SplitPlugin::initialize() noexcept +{ + return 0; +} + +void SplitPlugin::terminate() noexcept +{ + +} + +void SplitPlugin::configurePlugin(const nvinfer1::DynamicPluginTensorDesc* in, int nbInputs, + const nvinfer1::DynamicPluginTensorDesc* out, int nbOutputs) noexcept +{ + std::vector segment_offsets(1, 0); + for( int i = 0; i < nbOutputs; ++i ) + { + segment_offsets.push_back(segment_offsets.back() + _output_lengths[i]); + } + _d_segment_offsets = segment_offsets; + + for (int i = 0; i < nbInputs; i++) + { + for (int j = 0; j < in[0].desc.dims.nbDims; j++) + { + // Do not support dynamic dimensions + PLUGIN_ASSERT(in[0].desc.dims.d[j] != -1); + } + } + + nvinfer1::Dims dims = in[0].desc.dims; + _nx = 1; + for( int i = dims.nbDims-1; i > _axis; --i ) + { + _nx *= dims.d[i]; + } + _ny = dims.d[_axis]; + _nz = 1; + for( int i = _axis-1; i >= 0; --i ) + { + _nz *= dims.d[i]; + } + _d_output_ptrs.resize(nbOutputs, nullptr); +} + int SplitPlugin::enqueue(const nvinfer1::PluginTensorDesc* inputDesc, const nvinfer1::PluginTensorDesc* outputDesc, const void* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept { @@ -109,3 +168,9 @@ int SplitPlugin::enqueue(const nvinfer1::PluginTensorDesc* inputDesc, const nvin return cudaGetLastError() != cudaSuccess; } +nvinfer1::DimsExprs SplitPlugin::getOutputDimensions(int outputIndex, const nvinfer1::DimsExprs* inputs, int nbInputs, nvinfer1::IExprBuilder& exprBuilder) noexcept +{ + nvinfer1::DimsExprs output(inputs[0]); + output.d[_axis] = exprBuilder.constant(_output_lengths[outputIndex]); + return output; +} diff --git a/plugin/splitPlugin/split.h b/plugin/splitPlugin/split.h index 53e1cef9..76ff63b4 100644 --- a/plugin/splitPlugin/split.h +++ b/plugin/splitPlugin/split.h @@ -38,11 +38,11 @@ namespace plugin { class SplitPlugin final : public nvinfer1::IPluginV2DynamicExt { - int _axis; - std::vector _output_lengths; - int _nx, _ny, _nz; - int _x_stride, _y_stride, _z_stride; - thrust::device_vector _d_segment_offsets; + int32_t _axis; + std::vector _output_lengths; + int32_t _nx, _ny, _nz; + int32_t _x_stride, _y_stride, _z_stride; + thrust::device_vector _d_segment_offsets; thrust::device_vector _d_output_ptrs; using IPluginV2::getOutputDimensions; @@ -67,13 +67,13 @@ class SplitPlugin final : public nvinfer1::IPluginV2DynamicExt } public: - SplitPlugin(int axis, int* const& output_lengths, int noutput) + SplitPlugin(int32_t axis, int32_t* const& output_lengths, int32_t noutput) : _axis(axis) - , _output_lengths(std::vector(output_lengths, output_lengths + noutput)) + , _output_lengths(std::vector(output_lengths, output_lengths + noutput)) { PLUGIN_ASSERT(axis <= nvinfer1::Dims::MAX_DIMS); } - SplitPlugin(int axis, std::vector output_lengths) + SplitPlugin(int32_t axis, std::vector output_lengths) : _axis(axis) , _output_lengths(output_lengths) { @@ -85,16 +85,16 @@ class SplitPlugin final : public nvinfer1::IPluginV2DynamicExt } bool supportsFormatCombination( - int pos, nvinfer1::PluginTensorDesc const* inOut, int nbInputs, int nbOutputs) noexcept override; + int32_t pos, nvinfer1::PluginTensorDesc const* inOut, int32_t nbInputs, int32_t nbOutputs) noexcept override; nvinfer1::DataType getOutputDataType( - int index, nvinfer1::DataType const* inputTypes, int nbInputs) const noexcept override; - int initialize() noexcept override; + int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override; + int32_t initialize() noexcept override; void terminate() noexcept override; - void configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int nbInputs, - nvinfer1::DynamicPluginTensorDesc const* out, int nbOutputs) noexcept override; - int enqueue(PluginTensorDesc const* inputDesc, PluginTensorDesc const* outputDesc, void const* const* inputs, + void configurePlugin(nvinfer1::DynamicPluginTensorDesc const* in, int32_t nbInputs, + nvinfer1::DynamicPluginTensorDesc const* out, int32_t nbOutputs) noexcept override; + int32_t enqueue(PluginTensorDesc const* inputDesc, PluginTensorDesc const* outputDesc, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; - nvinfer1::DimsExprs getOutputDimensions(int outputIndex, nvinfer1::DimsExprs const* inputs, int nbInputs, + nvinfer1::DimsExprs getOutputDimensions(int32_t outputIndex, nvinfer1::DimsExprs const* inputs, int32_t nbInputs, nvinfer1::IExprBuilder& exprBuilder) noexcept override; nvinfer1::IPluginV2DynamicExt* clone() const noexcept override @@ -113,8 +113,8 @@ class SplitPlugin final : public nvinfer1::IPluginV2DynamicExt { return SPLIT_PLUGIN_NAME; } - size_t getWorkspaceSize(nvinfer1::PluginTensorDesc const* /*inputs*/, int /*nbInputs*/, - nvinfer1::PluginTensorDesc const* /*outputs*/, int /*nbOutputs*/) const noexcept override + size_t getWorkspaceSize(nvinfer1::PluginTensorDesc const* /*inputs*/, int32_t /*nbInputs*/, + nvinfer1::PluginTensorDesc const* /*outputs*/, int32_t /*nbOutputs*/) const noexcept override { return 0; } @@ -123,7 +123,7 @@ class SplitPlugin final : public nvinfer1::IPluginV2DynamicExt { return ""; } - int getNbOutputs() const noexcept override + int32_t getNbOutputs() const noexcept override { return _output_lengths.size(); } diff --git a/plugin/splitPlugin/splitPlugin.cpp b/plugin/splitPlugin/splitPlugin.cpp deleted file mode 100644 index 82a768af..00000000 --- a/plugin/splitPlugin/splitPlugin.cpp +++ /dev/null @@ -1,89 +0,0 @@ -/* - * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. - * SPDX-License-Identifier: Apache-2.0 - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#include -#include - -#include "splitPlugin.h" - -using namespace nvinfer1; -using nvinfer1::plugin::SplitPlugin; - -bool SplitPlugin::supportsFormatCombination( - int pos, const nvinfer1::PluginTensorDesc* inOut, int nbInputs, int nbOutputs) noexcept -{ - PLUGIN_ASSERT(inOut && pos < (nbInputs + nbOutputs)); - return (inOut[pos].format == nvinfer1::PluginFormat::kLINEAR); -} - -nvinfer1::DataType SplitPlugin::getOutputDataType(int index, const nvinfer1::DataType* inputTypes, int nbInputs) const - noexcept -{ - PLUGIN_ASSERT(inputTypes && nbInputs > 0); - return inputTypes[0]; -} - -int SplitPlugin::initialize() noexcept -{ - return 0; -} - -void SplitPlugin::terminate() noexcept -{ - -} - -void SplitPlugin::configurePlugin(const nvinfer1::DynamicPluginTensorDesc* in, int nbInputs, - const nvinfer1::DynamicPluginTensorDesc* out, int nbOutputs) noexcept -{ - std::vector segment_offsets(1, 0); - for( int i = 0; i < nbOutputs; ++i ) - { - segment_offsets.push_back(segment_offsets.back() + _output_lengths[i]); - } - _d_segment_offsets = segment_offsets; - - for (int i = 0; i < nbInputs; i++) - { - for (int j = 0; j < in[0].desc.dims.nbDims; j++) - { - // Do not support dynamic dimensions - PLUGIN_ASSERT(in[0].desc.dims.d[j] != -1); - } - } - - nvinfer1::Dims dims = in[0].desc.dims; - _nx = 1; - for( int i = dims.nbDims-1; i > _axis; --i ) - { - _nx *= dims.d[i]; - } - _ny = dims.d[_axis]; - _nz = 1; - for( int i = _axis-1; i >= 0; --i ) - { - _nz *= dims.d[i]; - } - //_d_output_ptrs.resize(nbOutputs, nullptr); -} - -nvinfer1::DimsExprs SplitPlugin::getOutputDimensions(int outputIndex, const nvinfer1::DimsExprs* inputs, int nbInputs, nvinfer1::IExprBuilder& exprBuilder) noexcept -{ - nvinfer1::DimsExprs output(inputs[0]); - output.d[_axis] = exprBuilder.constant(_output_lengths[outputIndex]); - return output; -} diff --git a/plugin/splitPlugin/splitPlugin.h b/plugin/splitPlugin/splitPlugin.h deleted file mode 100644 index cc7af917..00000000 --- a/plugin/splitPlugin/splitPlugin.h +++ /dev/null @@ -1,187 +0,0 @@ -/* - * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. - * SPDX-License-Identifier: Apache-2.0 - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#ifndef TRT_SPLIT_PLUGIN_H -#define TRT_SPLIT_PLUGIN_H -#include - -#include "common/checkMacrosPlugin.h" -#include "common/serialize.hpp" - -#include -#include -#include - -namespace -{ -constexpr const char* SPLIT_PLUGIN_VERSION{"1"}; -constexpr const char* SPLIT_PLUGIN_NAME{"Split"}; -} // namespace - -template -__global__ void split_kernel(int nsegment, - int const* __restrict__ segment_offsets, - T const* __restrict__ idata, - T* const* odatas, - int nx, - int src_ny, - int nz); - -namespace nvinfer1 -{ -namespace plugin -{ -class SplitPlugin final : public nvinfer1::IPluginV2DynamicExt -{ - int _axis; - std::vector _output_lengths; - int _nx, _ny, _nz; - int _x_stride, _y_stride, _z_stride; - thrust::device_vector _d_segment_offsets; - thrust::device_vector _d_output_ptrs; - -protected: - void deserialize(void const* serialData, size_t serialLength) noexcept - { - deserialize_value(&serialData, &serialLength, &_axis); - deserialize_value(&serialData, &serialLength, &_output_lengths); - } - size_t getSerializationSize() const noexcept override - { - return serialized_size(_axis) + serialized_size(_output_lengths); - } - void serialize(void* buffer) const noexcept override - { - serialize_value(&buffer, _axis); - serialize_value(&buffer, _output_lengths); - } - -public: - SplitPlugin(int axis, int* const& output_lengths, int noutput) - : _axis(axis) - , _output_lengths(std::vector(output_lengths, output_lengths + noutput)) - { - PLUGIN_ASSERT(axis <= nvinfer1::Dims::MAX_DIMS); - } - SplitPlugin(int axis, std::vector output_lengths) - : _axis(axis) - , _output_lengths(output_lengths) - { - PLUGIN_ASSERT(axis <= nvinfer1::Dims::MAX_DIMS); - } - SplitPlugin(void const* serialData, size_t serialLength) - { - this->deserialize(serialData, serialLength); - } - - bool supportsFormatCombination(int pos, const nvinfer1::PluginTensorDesc* inOut, int nbInputs, int nbOutputs) noexcept override; - nvinfer1::DataType getOutputDataType(int index, const nvinfer1::DataType* inputTypes, int nbInputs) const noexcept override; - int initialize() noexcept override; - void terminate() noexcept override; - void configurePlugin(const nvinfer1::DynamicPluginTensorDesc* in, int nbInputs, - const nvinfer1::DynamicPluginTensorDesc* out, int nbOutputs) noexcept override; - int enqueue(const PluginTensorDesc* inputDesc, const PluginTensorDesc* outputDesc, const void* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept override; - nvinfer1::DimsExprs getOutputDimensions(int outputIndex, const nvinfer1::DimsExprs* inputs, int nbInputs, nvinfer1::IExprBuilder& exprBuilder) noexcept override; - - nvinfer1::IPluginV2DynamicExt* clone() const noexcept override - { - return new SplitPlugin{_axis, _output_lengths}; - } - void destroy() noexcept override - { - delete this; - } - const char* getPluginVersion() const noexcept override - { - return SPLIT_PLUGIN_VERSION; - } - const char* getPluginType() const noexcept override - { - return SPLIT_PLUGIN_NAME; - } - size_t getWorkspaceSize(const nvinfer1::PluginTensorDesc* /*inputs*/, int /*nbInputs*/, - const nvinfer1::PluginTensorDesc* /*outputs*/, int /*nbOutputs*/) const noexcept override - { - return 0; - } - void setPluginNamespace(const char* /*pluginNamespace*/) noexcept override {} - const char* getPluginNamespace() const noexcept override - { - return ""; - } - int getNbOutputs() const noexcept override - { - return _output_lengths.size(); - } - void attachToContext( - cudnnContext* /*cudnn*/, cublasContext* /*cublas*/, nvinfer1::IGpuAllocator* /*allocator*/) noexcept override - { - } - void detachFromContext() noexcept override {} -}; - -class SplitPluginCreator : public nvinfer1::IPluginCreator -{ -public: - SplitPluginCreator() {} - - ~SplitPluginCreator() {} - - const char* getPluginName() const noexcept - { - return SPLIT_PLUGIN_NAME; - } - - const char* getPluginVersion() const noexcept - { - return SPLIT_PLUGIN_VERSION; - } - - const nvinfer1::PluginFieldCollection* getFieldNames() noexcept - { - std::cerr << "Function not implemented" << std::endl; - return nullptr; - } - - nvinfer1::IPluginV2DynamicExt* createPlugin(const char* /*name*/, const nvinfer1::PluginFieldCollection* /*fc*/) noexcept - { - std::cerr << "Function not implemented" << std::endl; - return nullptr; - } - - nvinfer1::IPluginV2DynamicExt* deserializePlugin(const char* /*name*/, const void* serialData, size_t serialLength) noexcept - { - return new SplitPlugin{serialData, serialLength}; - } - - void setPluginNamespace(const char* libNamespace) noexcept - { - mNamespace = libNamespace; - } - - const char* getPluginNamespace() const noexcept - { - return mNamespace.c_str(); - } - -private: - std::string mNamespace; -}; - -} // namespace plugin -} // namespace nvinfer1 -#endif // TRT_SPLIT_PLUGIN_H diff --git a/python/CMakeLists.txt b/python/CMakeLists.txt index 580191f9..723c9318 100644 --- a/python/CMakeLists.txt +++ b/python/CMakeLists.txt @@ -40,7 +40,7 @@ set(CMAKE_CXX_STANDARD ${CPP_STANDARD}) if (NOT MSVC) # This allows us to use TRT libs shipped with standalone wheels. - set(CMAKE_SHARED_LINKER_FLAGS -Wl,-rpath=$ORIGIN) + set(CMAKE_SHARED_LINKER_FLAGS -Wl,-rpath=$ORIGIN:$ORIGIN/../${TENSORRT_MODULE}_libs) endif() # -------- PATHS -------- diff --git a/python/docstrings/infer/pyCoreDoc.h b/python/docstrings/infer/pyCoreDoc.h index 49580cf8..82878ac2 100644 --- a/python/docstrings/infer/pyCoreDoc.h +++ b/python/docstrings/infer/pyCoreDoc.h @@ -1141,9 +1141,10 @@ constexpr char const* descr = R"trtdoc( For example, to enable faster dynamic shapes, call :func:`set_preview_feature` with ``PreviewFeature.FASTER_DYNAMIC_SHAPES_0805`` )trtdoc"; constexpr char const* FASTER_DYNAMIC_SHAPES_0805 = R"trtdoc( - Optimize runtime dimensions with TensorRT's DL Compiler. + [DEPRECATED - will be removed in TensorRT 9.0] Optimize runtime dimensions with TensorRT's DL Compiler. Potentially reduces run time and decreases device memory usage and engine size. Models most likely to benefit from enabling ``FASTER_DYNAMIC_SHAPES_0805`` are transformer-based models, and models containing dynamic control flows. + The default value for this flag is on. Turning it off is deprecated. )trtdoc"; constexpr char const* DISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805 = R"trtdoc( Disable usage of cuDNN/cuBLAS/cuBLASLt tactics in the TensorRT core library. diff --git a/python/docstrings/infer/pyGraphDoc.h b/python/docstrings/infer/pyGraphDoc.h index ff09d9aa..bc69b28e 100644 --- a/python/docstrings/infer/pyGraphDoc.h +++ b/python/docstrings/infer/pyGraphDoc.h @@ -98,7 +98,7 @@ constexpr const char* descr = R"trtdoc( constexpr const char* LINEAR = R"trtdoc( Row major linear format. - For a tensor with dimensions {N, C, H, W}, the W axis always has unit stride, and the stride of every other axis is at least the the product of of the next dimension times the next stride. the strides are the same as for a C array with dimensions [N][C][H][W]. + For a tensor with dimensions {N, C, H, W}, the W axis always has unit stride, and the stride of every other axis is at least the product of the next dimension times the next stride. the strides are the same as for a C array with dimensions [N][C][H][W]. )trtdoc"; constexpr const char* CHW2 = R"trtdoc( @@ -232,14 +232,14 @@ constexpr const char* reset_dynamic_range = R"trtdoc( constexpr const char* set_dimension_name = R"trtdoc( Name a dimension of an input tensor. - + Associate a runtime dimension of an input tensor with a symbolic name. - Dimensions with the same non-empty name must be equal at runtime. + Dimensions with the same non-empty name must be equal at runtime. Knowing this equality for runtime dimensions may help the TensorRT optimizer. Both runtime and build-time dimensions can be named. If the function is called again, with the same index, it will overwrite the previous name. If None is passed as name, it will clear the name of the dimension. - + For example, setDimensionName(0, "n") associates the symbolic name "n" with the leading dimension. :arg index: index of the dimension. diff --git a/python/docstrings/parsers/pyCaffeDoc.h b/python/docstrings/parsers/pyCaffeDoc.h index 1f282941..9b2e1ebd 100644 --- a/python/docstrings/parsers/pyCaffeDoc.h +++ b/python/docstrings/parsers/pyCaffeDoc.h @@ -89,7 +89,7 @@ constexpr const char* is_plugin_v2 = R"trtdoc( :arg layer_name: Name of the layer which the user wishes to validate. - :returns: True if the the layer configuration is provided by an :class:`IPluginV2` . + :returns: True if the layer configuration is provided by an :class:`IPluginV2` . )trtdoc"; constexpr const char* create_plugin = R"trtdoc( diff --git a/python/docstrings/parsers/pyOnnxDoc.h b/python/docstrings/parsers/pyOnnxDoc.h index f0d6f0ea..65694747 100644 --- a/python/docstrings/parsers/pyOnnxDoc.h +++ b/python/docstrings/parsers/pyOnnxDoc.h @@ -134,10 +134,10 @@ namespace OnnxParserFlagDoc constexpr const char* descr = R"trtdoc( Flags that control how an ONNX model gets parsed. )trtdoc"; -constexpr const char* VERSION_COMPATIBLE = R"trtdoc( - Parse the ONNX model into the INetworkDefinition with the intention of building a version-compatible engine in TensorRT 8.6. - This flag is planned to be deprecated in TensorRT 8.7, and removed in TensorRT 9.0. - This will choose TensorRT's native InstanceNormalization implementation over the plugin implementation. +constexpr const char* NATIVE_INSTANCENORM = R"trtdoc( + Parse the ONNX model into the INetworkDefinition with the intention of using TensorRT's native layer implementation over the plugin implementation for InstanceNormalization nodes. + This flag is planned to be deprecated in TensorRT 8.7 and removed in TensorRT 9.0. + This flag is required when building version-compatible or hardware-compatible engines. There may be performance degradations when this flag is enabled. )trtdoc"; } // namespace OnnxParserFlagDoc diff --git a/python/packaging/LICENSE.txt b/python/packaging/bindings_wheel/LICENSE.txt similarity index 100% rename from python/packaging/LICENSE.txt rename to python/packaging/bindings_wheel/LICENSE.txt diff --git a/python/packaging/setup.cfg b/python/packaging/bindings_wheel/setup.cfg similarity index 100% rename from python/packaging/setup.cfg rename to python/packaging/bindings_wheel/setup.cfg diff --git a/python/packaging/bindings_wheel/setup.py b/python/packaging/bindings_wheel/setup.py new file mode 100644 index 00000000..b5b1b121 --- /dev/null +++ b/python/packaging/bindings_wheel/setup.py @@ -0,0 +1,50 @@ +# +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import os + +from setuptools import setup + +tensorrt_module = "##TENSORRT_MODULE##" + +# This file expects the following to be passed from the environment when using standalone wheels: +# - STANDALONE: Whether we are building a standalone wheel +IS_STANDALONE = os.environ.get("STANDALONE") == "1" +if IS_STANDALONE: + tensorrt_module += "_bindings" + +setup( + name=tensorrt_module, + version="##TENSORRT_PYTHON_VERSION##", + description="A high performance deep learning inference library", + long_description="A high performance deep learning inference library", + author="NVIDIA Corporation", + license="Proprietary", + classifiers=[ + "License :: Other/Proprietary License", + "Intended Audience :: Developers", + "Programming Language :: Python :: 3", + ], + packages=[tensorrt_module], + extras_require={"numpy": "numpy"}, + package_data={tensorrt_module: ["*.so*", "*.pyd", "*.pdb"]}, + include_package_data=True, + zip_safe=True, + keywords="nvidia tensorrt deeplearning inference", + url="https://developer.nvidia.com/tensorrt", + download_url="https://github.com/nvidia/tensorrt/tags", +) diff --git a/python/packaging/tensorrt/__init__.py b/python/packaging/bindings_wheel/tensorrt/__init__.py similarity index 86% rename from python/packaging/tensorrt/__init__.py rename to python/packaging/bindings_wheel/tensorrt/__init__.py index 0482c061..0192bc55 100644 --- a/python/packaging/tensorrt/__init__.py +++ b/python/packaging/bindings_wheel/tensorrt/__init__.py @@ -22,34 +22,28 @@ import warnings -def try_load(library): - try: - ctypes.CDLL(library) - except OSError: - pass - - -# Try loading all packaged libraries. This is a nop if there are no libraries packaged. -CURDIR = os.path.realpath(os.path.dirname(__file__)) -for lib in glob.iglob(os.path.join(CURDIR, "*.so*")): - try_load(lib) - - -# On Windows, we need to manually open the TensorRT libraries - otherwise we are unable to -# load the bindings. -def find_lib(name): - paths = os.environ["PATH"].split(os.path.pathsep) - for path in paths: - libpath = os.path.join(path, name) - if os.path.isfile(libpath): - return libpath - - raise FileNotFoundError( - "Could not find: {:}. Is it on your PATH?\nNote: Paths searched were:\n{:}".format(name, paths) - ) +# For standalone wheels, attempt to import the wheel containing the libraries. +try: + import ##TENSORRT_MODULE##_libs +except (ImportError, ModuleNotFoundError): + pass if sys.platform.startswith("win"): + # On Windows, we need to manually open the TensorRT libraries - otherwise we are unable to + # load the bindings. + def find_lib(name): + paths = os.environ["PATH"].split(os.path.pathsep) + for path in paths: + libpath = os.path.join(path, name) + if os.path.isfile(libpath): + return libpath + + raise FileNotFoundError( + "Could not find: {:}. Is it on your PATH?\nNote: Paths searched were:\n{:}".format(name, paths) + ) + + # Order matters here because of dependencies LIBRARIES = {"tensorrt": [ "nvinfer.dll", diff --git a/python/packaging/frontend_sdist/LICENSE.txt b/python/packaging/frontend_sdist/LICENSE.txt new file mode 100644 index 00000000..08f07f9f --- /dev/null +++ b/python/packaging/frontend_sdist/LICENSE.txt @@ -0,0 +1,180 @@ +Abstract +This document is the Software License Agreement (SLA) for NVIDIA TensorRT. This document contains specific license terms and conditions for NVIDIA TensorRT. By accepting this agreement, you agree to comply with all the terms and conditions applicable to the specific product(s) included herein. + +If you are receiving TensorRT under the NVIDIA Prerelease License Agreement (also known as NPLA) or under the NVIDIA Software License Agreement (previously known as the NVIDIA Tegra Software License Agreement), your use of TensorRT is governed by such applicable terms and conditions. All other uses of TensorRT are governed by the terms and conditions of the below license agreement. + +NVIDIA SOFTWARE LICENSE AGREEMENT +Important: READ BEFORE DOWNLOADING, INSTALLING, COPYING OR USING THE LICENSED SOFTWARE +This Software License Agreement ("SLA”), made and entered into as of the time and date of click through action (“Effective Date”),is a legal agreement between you and NVIDIA Corporation ("NVIDIA") and governs the use of the NVIDIA computer software and the documentation made available for use with such NVIDIA software. By downloading, installing, copying, or otherwise using the NVIDIA software and/or documentation, you agree to be bound by the terms of this SLA. If you do not agree to the terms of this SLA, do not download, install, copy or use the NVIDIA software or documentation. IF YOU ARE ENTERING INTO THIS SLAON BEHALF OF A COMPANY OR OTHER LEGAL ENTITY, YOU REPRESENT THAT YOU HAVE THE LEGAL AUTHORITY TO BIND THE ENTITY TO THIS SLA, IN WHICH CASE “YOU” WILL MEAN THE ENTITY YOU REPRESENT. IF YOU DON’T HAVE SUCH AUTHORITY, OR IF YOU DON’T ACCEPT ALL THE TERMS AND CONDITIONS OF THIS SLA, THEN NVIDIA DOES NOT AGREETO LICENSE THE LICENSED SOFTWARETO YOU, AND YOU MAY NOT DOWNLOAD, INSTALL, COPY OR USE IT. + +Preface +This document is the Software License Agreement (SLA) for NVIDIA TensorRT. This document contains specific license terms and conditions for NVIDIA TensorRT. By accepting this agreement, you agree to comply with all the terms and conditions applicable to the specific product(s) included herein. + +If you are receiving TensorRT under the NVIDIA Prerelease License Agreement (also known as NPLA) or under the NVIDIA Software License Agreement (previously known as the NVIDIA Tegra Software License Agreement), your use of TensorRT is governed by such applicable terms and conditions. All other uses of TensorRT are governed by the terms and conditions of the below license agreement. + +NVIDIA SOFTWARE LICENSE AGREEMENT +Important: READ BEFORE DOWNLOADING, INSTALLING, COPYING OR USING THE LICENSED SOFTWARE +This Software License Agreement ("SLA”), made and entered into as of the time and date of click through action (“Effective Date”),is a legal agreement between you and NVIDIA Corporation ("NVIDIA") and governs the use of the NVIDIA computer software and the documentation made available for use with such NVIDIA software. By downloading, installing, copying, or otherwise using the NVIDIA software and/or documentation, you agree to be bound by the terms of this SLA. If you do not agree to the terms of this SLA, do not download, install, copy or use the NVIDIA software or documentation. IF YOU ARE ENTERING INTO THIS SLAON BEHALF OF A COMPANY OR OTHER LEGAL ENTITY, YOU REPRESENT THAT YOU HAVE THE LEGAL AUTHORITY TO BIND THE ENTITY TO THIS SLA, IN WHICH CASE “YOU” WILL MEAN THE ENTITY YOU REPRESENT. IF YOU DON’T HAVE SUCH AUTHORITY, OR IF YOU DON’T ACCEPT ALL THE TERMS AND CONDITIONS OF THIS SLA, THEN NVIDIA DOES NOT AGREETO LICENSE THE LICENSED SOFTWARETO YOU, AND YOU MAY NOT DOWNLOAD, INSTALL, COPY OR USE IT. + +1. LICENSE. +1.1. License Grant +Subject to the terms of the AGREEMENT, NVIDIA hereby grants you a non-exclusive, non-transferable license, without the right to sublicense (except as expressly set forth in a Supplement), during the applicable license term unless earlier terminated as provided below, to have Authorized Users install and use the Software, including modifications (if expressly permitted in a Supplement), in accordance with the Documentation. You are only licensed to activate and use Licensed Software for which you a have a valid license, even if during the download or installation you are presented with other product options. No Orders are binding on NVIDIA until accepted by NVIDIA. Your Orders are subject to the AGREEMENT. + +SLA Supplements: Certain Licensed Software licensed under this SLA may be subject to additional terms and conditions that will be presented to you in a Supplement for acceptance prior to the delivery of such Licensed Software under this SLA and the applicable Supplement. Licensed Software will only be delivered to you upon your acceptance of all applicable terms. + +1.2. Limited Purpose Licenses +If your license is provided for one of the purposes indicated below, then notwithstanding contrary terms in License Grant or in a Supplement, such licenses are for internal use and do not include any right or license to sub-license and distribute the Licensed Software or its output in any way in any public release, however limited, and/or in any manner that provides third parties with use of or access to the Licensed Software or its functionality or output, including (but not limited to) external alpha or beta testing or development phases. Further: +Evaluation License. You may use evaluation licenses solely for your internal evaluation of the Licensed Software for broader adoption within your Enterprise or in connection with a NVIDIA product purchase decision, and such licenses have an expiration date as indicated by NVIDIA in its sole discretion (or ninety days from the date of download if no other duration is indicated). +Educational/Academic License. You may use educational/academic licenses solely for educational purposes and all users must be enrolled or employed by an academic institution. If you do not meet NVIDIA’s academic program requirements for educational institutions, you have no rights under this license. +Test/Development License. You may use test/development licenses solely for your internal development, testing and/or debugging of your software applications or for interoperability testing with the Licensed Software, and such licenses have an expiration date as indicated by NVIDIA in its sole discretion (or one year from the date of download if no other duration is indicated). NVIDIA Confidential Information under the AGREEMENT includes output from Licensed Software developer tools identified as “Pro” versions, where the output reveals functionality or performance data pertinent to NVIDIA hardware or software products. +1.3. Pre-Release Licenses +With respect to alpha, beta, preview, and other pre-release Software and Documentation (“Pre-Release Licensed Software”) delivered to you under the AGREEMENT you acknowledge and agree that such Pre-Release Licensed Software (i) may not be fully functional, may contain errors or design flaws, and may have reduced or different security, privacy, accessibility, availability, and reliability standards relative to commercially provided NVIDIA software and documentation, and (ii) use of such Pre-Release Licensed Software may result in unexpected results, loss of data, project delays or other unpredictable damage or loss. THEREFORE, PRE-RELEASE LICENSED SOFTWARE IS NOT INTENDED FOR USE, AND SHOULD NOT BE USED, IN PRODUCTION OR BUSINESS-CRITICAL SYSTEMS. NVIDIA has no obligation to make available a commercial version of any Pre-Release Licensed Software and NVIDIA has the right to abandon development of Pre-Release Licensed Software at any time without liability. + +1.4. Enterprise and Contractor Usage +You may allow your Enterprise employees and Contractors to access and use the Licensed Software pursuant to the terms of the AGREEMENT solely to perform work on your behalf, provided further that with respect to Contractors: (i) you obtain a written agreement from each Contractor which contains terms and obligations with respect to access to and use of Licensed Software no less protective of NVIDIA than those set forth in the AGREEMENT, and (ii) such Contractor’s access and use expressly excludes any sublicensing or distribution rights for the Licensed Software. You are responsible for the compliance with the terms and conditions of the AGREEMENT by your Enterprise and Contractors. Any act or omission that, if committed by you, would constitute a breach of the AGREEMENT shall be deemed to constitute a breach of the AGREEMENT if committed by your Enterprise or Contractors. + +1.5. Services +Except as expressly indicated in an Order, NVIDIA is under no obligation to provide support for the Licensed Software or to provide any patches, maintenance, updates or upgrades under the AGREEMENT. Unless patches, maintenance, updates or upgrades are provided with their separate governing terms and conditions, they constitute Licensed Software licensed to you under the AGREEMENT. + +2. LIMITATIONS. +2.1. License Restrictions +Except as expressly authorized in the AGREEMENT, you agree that you will not (nor authorize third parties to): (i) copy and use Software that was licensed to you for use in one or more NVIDIA hardware products in other unlicensed products (provided that copies solely for backup purposes are allowed); (ii) reverse engineer, decompile, disassemble (except to the extent applicable laws specifically require that such activities be permitted) or attempt to derive the source code, underlying ideas, algorithm or structure of Software provided to you in object code form; (iii) sell, transfer, assign, distribute, rent, loan, lease, sublicense or otherwise make available the Licensed Software or its functionality to third parties (a) as an application services provider or service bureau, (b) by operating hosted/virtual system environments, (c) by hosting, time sharing or providing any other type of services, or (d) otherwise by means of the internet; (iv) modify, translate or otherwise create any derivative works of any Licensed Software; (v) remove, alter, cover or obscure any proprietary notice that appears on or with the Licensed Software or any copies thereof; (vi) use the Licensed Software, or allow its use, transfer, transmission or export in violation of any applicable export control laws, rules or regulations; (vii) distribute, permit access to, or sublicense the Licensed Software as a stand-alone product; (viii) bypass, disable, circumvent or remove any form of copy protection, encryption, security or digital rights management or authentication mechanism used by NVIDIA in connection with the Licensed Software, or use the Licensed Software together with any authorization code, serial number, or other copy protection device not supplied by NVIDIA directly or through an authorized reseller; (ix) use the Licensed Software for the purpose of developing competing products or technologies or assisting a third party in such activities; (x) use the Licensed Software with any system or application where the use or failure of such system or application can reasonably be expected to threaten or result in personal injury, death, or catastrophic loss including, without limitation, use in connection with any nuclear, avionics, navigation, military, medical, life support or other life critical application (“Critical Applications”), unless the parties have entered into a Critical Applications agreement; (xi) distribute any modification or derivative work you make to the Licensed Software under or by reference to the same name as used by NVIDIA; or (xii) use the Licensed Software in any manner that would cause the Licensed Software to become subject to an Open Source License. Nothing in the AGREEMENT shall be construed to give you a right to use, or otherwise obtain access to, any source code from which the Software or any portion thereof is compiled or interpreted. You acknowledge that NVIDIA does not design, test, manufacture or certify the Licensed Software for use in the context of a Critical Application and NVIDIA shall not be liable to you or any third party, in whole or in part, for any claims or damages arising from such use. You agree to defend, indemnify and hold harmless NVIDIA and its Affiliates, and their respective employees, contractors, agents, officers and directors, from and against any and all claims, damages, obligations, losses, liabilities, costs or debt, fines, restitutions and expenses (including but not limited to attorney’s fees and costs incident to establishing the right of indemnification) arising out of or related to you and your Enterprise, and their respective employees, contractors, agents, distributors, resellers, end users, officers and directors use of Licensed Software outside of the scope of the AGREEMENT or any other breach of the terms of the AGREEMENT. + +2.2. Third Party License Obligations +You acknowledge and agree that the Licensed Software may include or incorporate third party technology (collectively “Third Party Components”), which is provided for use in or with the Software and not otherwise used separately. If the Licensed Software includes or incorporates Third Party Components, then the third-party pass-through terms and conditions (“Third Party Terms”) for the particular Third Party Component will be bundled with the Software or otherwise made available online as indicated by NVIDIA and will be incorporated by reference into the AGREEMENT. In the event of any conflict between the terms in the AGREEMENT and the Third Party Terms, the Third Party Terms shall govern. Copyright to Third Party Components are held by the copyright holders indicated in the copyright notices indicated in the Third Party Terms. + +Audio/Video Encoders and Decoders: You acknowledge and agree that it is your sole responsibility to obtain any additional third party licenses required to make, have made, use, have used, sell, import, and offer for sale your products or services that include or incorporate any Third Party Components and content relating to audio and/or video encoders and decoders from, including but not limited to, Microsoft, Thomson, Fraunhofer IIS, Sisvel S.p.A., MPEG-LA, and Coding Technologies as NVIDIA does not grant to you under the AGREEMENT any necessary patent or other rights with respect to audio and/or video encoders and decoders. + +2.3. Limited Rights +Your rights in the Licensed Software are limited to those expressly granted under the AGREEMENT and no other licenses are granted whether by implication, estoppel or otherwise. NVIDIA reserves all rights, title and interest in and to the Licensed Software not expressly granted under the AGREEMENT. + +3. CONFIDENTIALITY +Neither party will use the other party’s Confidential Information, except as necessary for the performance of the AGREEMENT, nor will either party disclose such Confidential Information to any third party, except to personnel of NVIDIA and its Affiliates, you, your Enterprise, your Enterprise Contractors, and each party’s legal and financial advisors that have a need to know such Confidential Information for the performance of the AGREEMENT, provided that each such personnel, employee and Contractor is subject to a written agreement that includes confidentiality obligations consistent with those set forth herein. Each party will use all reasonable efforts to maintain the confidentiality of all of the other party’s Confidential Information in its possession or control, but in no event less than the efforts that it ordinarily uses with respect to its own Confidential Information of similar nature and importance. The foregoing obligations will not restrict either party from disclosing the other party’s Confidential Information or the terms and conditions of the AGREEMENT as required under applicable securities regulations or pursuant to the order or requirement of a court, administrative agency, or other governmental body, provided that the party required to make such disclosure (i) gives reasonable notice to the other party to enable it to contest such order or requirement prior to its disclosure (whether through protective orders or otherwise), (ii) uses reasonable effort to obtain confidential treatment or similar protection to the fullest extent possible to avoid such public disclosure, and (iii) discloses only the minimum amount of information necessary to comply with such requirements. + +4. OWNERSHIP +You are not obligated to disclose to NVIDIA any modifications that you, your Enterprise or your Contractors make to the Licensed Software as permitted under the AGREEMENT. As between the parties, all modifications are owned by NVIDIA and licensed to you under the AGREEMENT unless otherwise expressly provided in a Supplement. The Licensed Software and all modifications owned by NVIDIA, and the respective Intellectual Property Rights therein, are and will remain the sole and exclusive property of NVIDIA or its licensors, whether the Licensed Software is separate from or combined with any other products or materials. You shall not engage in any act or omission that would impair NVIDIA’s and/or its licensors’ Intellectual Property Rights in the Licensed Software or any other materials, information, processes or subject matter proprietary to NVIDIA. NVIDIA’s licensors are intended third party beneficiaries with the right to enforce provisions of the AGREEMENT with respect to their Confidential Information and/or Intellectual Property Rights. + +5. FEEDBACK +You have no obligation to provide Feedback to NVIDIA. However, NVIDIA and/or its Affiliates may use and include any Feedback that you provide to improve the Licensed Software or other NVIDIA products, technologies or materials. Accordingly, if you provide Feedback, you agree that NVIDIA and/or its Affiliates, at their option, may, and may permit their licensees, to make, have made, use, have used, reproduce, license, distribute and otherwise commercialize the Feedback in the Licensed Software or in other NVIDIA products, technologies or materials without the payment of any royalties or fees to you. All Feedback becomes the sole property of NVIDIA and may be used in any manner NVIDIA sees fit, and you hereby assign to NVIDIA all of your right, title and interest in and to any Feedback. NVIDIA has no obligation to respond to Feedback or to incorporate Feedback into the Licensed Software. + +6. NO WARRANTIES +THE LICENSED SOFTWARE AND ANY OTHER CONFIDENTIAL INFORMATION AND/OR SERVICES ARE PROVIDED BY NVIDIA “AS IS” AND “WITH ALL FAULTS,” AND NVIDIA EXPRESSLY DISCLAIMS ALL OTHER WARRANTIES OF ANY KIND OR NATURE, WHETHER EXPRESS, IMPLIED OR STATUTORY, INCLUDING, BUT NOT LIMITED TO, ANY WARRANTIES OF OPERABILITY, CONDITION, VALUE, ACCURACY OF DATA, OR QUALITY, AS WELL AS ANY WARRANTIES OF MERCHANTABILITY, SYSTEM INTEGRATION, WORKMANSHIP, SUITABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, OR THE ABSENCE OF ANY DEFECTS THEREIN, WHETHER LATENT OR PATENT. NO WARRANTY IS MADE BY NVIDIA ON THE BASIS OF TRADE USAGE, COURSE OF DEALING OR COURSE OF TRADE. NVIDIA DOES NOT WARRANT THAT THE LICENSED SOFTWARE OR ANY OTHER CONFIDENTIAL INFORMATION AND/OR SERVICES PROVIDED BY NVIDIA UNDER THE AGREEMENT WILL MEET YOUR REQUIREMENTS OR THAT THE OPERATION THEREOF WILL BE UNINTERRUPTED OR ERROR-FREE, OR THAT ALL ERRORS WILL BE CORRECTED. YOU ACKNOWLEDGE THAT NVIDIA’S OBLIGATIONS UNDER THE AGREEMENT ARE FOR THE BENEFIT OF YOU ONLY. Nothing in this warranty section affects any statutory rights of consumers or other recipients to the extent that they cannot be waived or limited by contract under applicable law. + +7. LIMITATION OF LIABILITY +TO THE MAXIMUM EXTENT PERMITTED BY LAW, NVIDIA OR ITS LICENSORS SHALL NOT BE LIABLE FOR ANY SPECIAL, INCIDENTAL, PUNITIVE OR CONSEQUENTIAL DAMAGES, OR ANY LOST PROFITS, LOSS OF USE, LOSS OF DATA OR LOSS OF GOODWILL, OR THE COSTS OF PROCURING SUBSTITUTE PRODUCTS, ARISING OUT OF OR IN CONNECTION WITH THE AGREEMENT OR THE USE OR PERFORMANCE OF THE LICENSED SOFTWARE AND ANY OTHER CONFIDENTIAL INFORMATION AND/OR SERVICES PROVIDED BY NVIDIA UNDER THE AGREEMENT, WHETHER SUCH LIABILITY ARISES FROM ANY CLAIM BASED UPON BREACH OF CONTRACT, BREACH OF WARRANTY, TORT (INCLUDING NEGLIGENCE), PRODUCT LIABILITY OR ANY OTHER CAUSE OF ACTION OR THEORY OF LIABILITY. IN NO EVENT WILL NVIDIA’S TOTAL CUMULATIVE LIABILITY UNDER OR ARISING OUT OF THE AGREEMENT EXCEED THE NET AMOUNTS RECEIVED BY NVIDIA FOR YOUR USE OF THE PARTICULAR LICENSED SOFTWARE DURING THE TWELVE (12) MONTHS BEFORE THE LIABILITY AROSE (or up to US$10.00 if you acquired the Licensed Software for no charge). THE NATURE OF THE LIABILITY, THE NUMBER OF CLAIMS OR SUITS OR THE NUMBER OF PARTIES WITHIN YOUR ENTERPRISE THAT ACCEPTED THE TERMS OF THE AGREEMENT SHALL NOT ENLARGE OR EXTEND THIS LIMIT. THE FOREGOING LIMITATIONS SHALL APPLY REGARDLESS OF WHETHER NVIDIA OR ITS LICENSORS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES AND REGARDLESS OF WHETHER ANY REMEDY FAILS ITS ESSENTIAL PURPOSE. The disclaimers, exclusions and limitations of liability set forth in the AGREEMENT form an essential basis of the bargain between the parties, and, absent any such disclaimers, exclusions or limitations of liability, the provisions of the AGREEMENT, including, without limitation, the economic terms, would be substantially different. + +8. TERM AND TERMINATION. +8.1. AGREEMENT, Licenses and Services +This SLA shall become effective upon the Effective Date, each Supplement upon their acceptance, and both this SLA and Supplements shall continue in effect until your last access or use of the Licensed Software and/or services hereunder, unless earlier terminated as provided in this “Term and Termination” section. Each Licensed Software license ends at the earlier of (a) the expiration of the applicable license term, or (b) termination of such license or the AGREEMENT. Each service ends at the earlier of (x) the expiration of the applicable service term, (y) termination of such service or the AGREEMENT, or (z) expiration or termination of the associated license and no credit or refund will be provided upon the expiration or termination of the associated license for any service fees paid. + +8.2. Termination and Effect of Expiration or Termination +NVIDIA may terminate the AGREEMENT in whole or in part: (i) if you breach any term of the AGREEMENT and fail to cure such breach within thirty (30) days following notice thereof from NVIDIA (or immediately if you violate NVIDIA’s Intellectual Property Rights); (ii) if you become the subject of a voluntary or involuntary petition in bankruptcy or any proceeding relating to insolvency, receivership, liquidation or composition for the benefit of creditors, if that petition or proceeding is not dismissed with prejudice within sixty (60) days after filing, or if you cease to do business; or (iii) if you commence or participate in any legal proceeding against NVIDIA, with respect to the Licensed Software that is the subject of the proceeding during the pendency of such legal proceeding. If you or your authorized NVIDIA reseller fail to pay license fees or service fees when due then NVIDIA may, in its sole discretion, suspend or terminate your license grants, services and any other rights provided under the AGREEMENT for the affected Licensed Software, in addition to any other remedies NVIDIA may have at law or equity. Upon any expiration or termination of the AGREEMENT, a license or a service provided hereunder, (a) any amounts owed to NVIDIA become immediately due and payable, (b) you must promptly discontinue use of the affected Licensed Software and/or service, and (c) you must promptly destroy or return to NVIDIA all copies of the affected Licensed Software and all portions thereof in your possession or control, and each party will promptly destroy or return to the other all of the other party’s Confidential Information within its possession or control. Upon written request, you will certify in writing that you have complied with your obligations under this section. Upon expiration or termination of the AGREEMENT all provisions survive except for the license grant provisions. + +9. CONSENT TO COLLECTION AND USE OF INFORMATION. +You hereby agree and acknowledge that the Software may access, collect non-personally identifiable information about your Enterprise computer systems in order to properly optimize such systems for use with the Software. To the extent that you use the Software, you hereby consent to all of the foregoing, and represent and warrant that you have the right to grant such consent. In addition, you agree that you are solely responsible for maintaining appropriate data backups and system restore points for your Enterprise systems, and that NVIDIA will have no responsibility for any damage or loss to such systems (including loss of data or access) arising from or relating to (a) any changes to the configuration, application settings, environment variables, registry, drivers, BIOS, or other attributes of the systems (or any part of such systems) initiated through the Software; or (b) installation of any Software or third party software patches initiated through the Software. In certain systems you may change your system update preferences by unchecking "Automatically check for updates" in the "Preferences" tab of the control panel for the Software. + +In connection with the receipt of the Licensed Software or services you may receive access to links to third party websites and services and the availability of those links does not imply any endorsement by NVIDIA. NVIDIA encourages you to review the privacy statements on those sites and services that you choose to visit so that you can understand how they may collect, use and share personal information of individuals. NVIDIA is not responsible or liable for: (i) the availability or accuracy of such links; or (ii) the products, services or information available on or through such links; or (iii) the privacy statements or practices of sites and services controlled by other companies or organizations. + +To the extent that you or members of your Enterprise provide to NVIDIA during registration or otherwise personal information, you acknowledge that such information will be collected, used and disclosed by NVIDIA in accordance with NVIDIA's privacy policy, available at URL http://www.nvidia.com/object/privacy_policy.html. + +10. GENERAL. +This SLA, any Supplements incorporated hereto, and Orders constitute the entire agreement of the parties with respect to the subject matter hereto and supersede all prior negotiations, conversations, or discussions between the parties relating to the subject matter hereto, oral or written, and all past dealings or industry custom. Any additional and/or conflicting terms and conditions on purchase order(s) or any other documents issued by you are null, void, and invalid. Any amendment or waiver under the AGREEMENT must be in writing and signed by representatives of both parties. + +The AGREEMENT and the rights and obligations thereunder may not be assigned by you, in whole or in part, including by merger, consolidation, dissolution, operation of law, or any other manner, without written consent of NVIDIA, and any purported assignment in violation of this provision shall be void and of no effect. NVIDIA may assign, delegate or transfer the AGREEMENT and its rights and obligations hereunder, and if to a non-Affiliate you will be notified. + +Each party acknowledges and agrees that the other is an independent contractor in the performance of the AGREEMENT, and each party is solely responsible for all of its employees, agents, contractors, and labor costs and expenses arising in connection therewith. The parties are not partners, joint ventures or otherwise affiliated, and neither has any authority to make any statements, representations or commitments of any kind to bind the other party without prior written consent. + +Neither party will be responsible for any failure or delay in its performance under the AGREEMENT (except for any payment obligations) to the extent due to causes beyond its reasonable control for so long as such force majeure event continues in effect. + +The AGREEMENT will be governed by and construed under the laws of the State of Delaware and the United States without regard to the conflicts of law provisions thereof and without regard to the United Nations Convention on Contracts for the International Sale of Goods. The parties consent to the personal jurisdiction of the federal and state courts located in Santa Clara County, California. You acknowledge and agree that a breach of any of your promises or agreements contained in the AGREEMENT may result in irreparable and continuing injury to NVIDIA for which monetary damages may not be an adequate remedy and therefore NVIDIA is entitled to seek injunctive relief as well as such other and further relief as may be appropriate. If any court of competent jurisdiction determines that any provision of the AGREEMENT is illegal, invalid or unenforceable, the remaining provisions will remain in full force and effect. Unless otherwise specified, remedies are cumulative. + +The Licensed Software has been developed entirely at private expense and is “commercial items” consisting of “commercial computer software” and “commercial computer software documentation” provided with RESTRICTED RIGHTS. Use, duplication or disclosure by the U.S. Government or a U.S. Government subcontractor is subject to the restrictions set forth in the AGREEMENT pursuant to DFARS 227.7202-3(a) or as set forth in subparagraphs (c)(1) and (2) of the Commercial Computer Software - Restricted Rights clause at FAR 52.227-19, as applicable. Contractor/manufacturer is NVIDIA, 2701 San Tomas Expressway, Santa Clara, CA 95050. + +You acknowledge that the Licensed Software described under the AGREEMENT is subject to export control under the U.S. Export Administration Regulations (EAR) and economic sanctions regulations administered by the U.S. Department of Treasury’s Office of Foreign Assets Control (OFAC). Therefore, you may not export, reexport or transfer in-country the Licensed Software without first obtaining any license or other approval that may be required by BIS and/or OFAC. You are responsible for any violation of the U.S. or other applicable export control or economic sanctions laws, regulations and requirements related to the Licensed Software. By accepting this SLA, you confirm that you are not a resident or citizen of any country currently embargoed by the U.S. and that you are not otherwise prohibited from receiving the Licensed Software. + +Any notice delivered by NVIDIA to you under the AGREEMENT will be delivered via mail, email or fax. Please direct your legal notices or other correspondence to NVIDIA Corporation, 2701 San Tomas Expressway, Santa Clara, California 95050, United States of America, Attention: Legal Department. + +11. GLOSSARY OF TERMS +Certain capitalized terms, if not otherwise defined elsewhere in this SLA, shall have the meanings set forth below: +“Affiliate” +“Affiliate” means any legal entity that Owns, is Owned by, or is commonly Owned with a party. “Own” means having more than 50% ownership or the right to direct the management of the entity. +“AGREEMENT” +“AGREEMENT” means this SLA and all associated Supplements entered by the parties referencing this SLA. +“Authorized Users” +“Authorized Users” means your Enterprise individual employees and any of your Enterprise’s Contractors, subject to the terms of the “Enterprise and Contractors Usage” section. +“Confidential Information” +“Confidential Information” means the Licensed Software (unless made publicly available by NVIDIA without confidentiality obligations), and any NVIDIA business, marketing, pricing, research and development, know-how, technical, scientific, financial status, proposed new products or other information disclosed by NVIDIA to you which, at the time of disclosure, is designated in writing as confidential or proprietary (or like written designation), or orally identified as confidential or proprietary or is otherwise reasonably identifiable by parties exercising reasonable business judgment, as confidential. Confidential Information does not and will not include information that: (i) is or becomes generally known to the public through no fault of or breach of the AGREEMENT by the receiving party; (ii) is rightfully known by the receiving party at the time of disclosure without an obligation of confidentiality; (iii) is independently developed by the receiving party without use of the disclosing party’s Confidential Information; or (iv) is rightfully obtained by the receiving party from a third party without restriction on use or disclosure. +“Contractor” +“Contractor” means an individual who works primarily for your Enterprise on a contractor basis from your secure network. means an individual who works primarily for your Enterprise on a contractor basis from your secure network. +“Documentation” +“Documentation” means the NVIDIA documentation made available for use with the Software, including (without limitation) user manuals, datasheets, operations instructions, installation guides, release notes and other materials provided to you under the AGREEMENT. +“Enterprise” +“Enterprise” means you or any company or legal entity for which you accepted the terms of this SLA, and their subsidiaries of which your company or legal entity owns more than fifty percent (50%) of the issued and outstanding equity. +“Feedback” +“Feedback” means any and all suggestions, feature requests, comments or other feedback regarding the Licensed Software, including possible enhancements or modifications thereto. +“Intellectual Property Rights” +“Intellectual Property Rights” means all patent, copyright, trademark, trade secret, trade dress, trade names, utility models, mask work, moral rights, rights of attribution or integrity service marks, master recording and music publishing rights, performance rights, author’s rights, database rights, registered design rights and any applications for the protection or registration of these rights, or other intellectual or industrial property rights or proprietary rights, howsoever arising and in whatever media, whether now known or hereafter devised, whether or not registered, (including all claims and causes of action for infringement, misappropriation or violation and all rights in any registrations and renewals), worldwide and whether existing now or in the future. +“Licensed Software” +“Licensed Software” means Software, Documentation and all modifications owned by NVIDIA. +“Open Source License” +“Open Source License” includes, without limitation, a software license that requires as a condition of use, modification, and/or distribution of such software that the Software be (i) disclosed or distributed in source code form; (ii) be licensed for the purpose of making derivative works; or (iii) be redistributable at no charge. +“Order” +“Order” means a purchase order issued by you, a signed purchase agreement with you, or other ordering document issued by you to NVIDIA or a NVIDIA authorized reseller (including any on-line acceptance process) that references and incorporates the AGREEMENT and is accepted by NVIDIA. +“Software” +“Software” means the NVIDIA software programs licensed to you under the AGREEMENT including, without limitation, libraries, sample code, utility programs and programming code. +“Supplement” +“Supplement” means the additional terms and conditions beyond those stated in this SLA that apply to certain Licensed Software licensed hereunder. +12. TensorRT SUPPLEMENT TO SOFTWARE LICENSE AGREEMENT +TensorRT SUPPLEMENT TO SOFTWARE LICENSE AGREEMENT +The terms set forth in this TensorRT Supplement (“Supplement”) govern your use of the NVIDIA GPU inference engine (the “TensorRT Licensed Software”) under the terms of your software license agreement (“SLA”) as modified by this Supplement. This Supplement is an exhibit to the SLA and is hereby incorporated as an integral part thereto. Capitalized terms used but not defined herein shall have the meaning assigned to them in the SLA. In the event of conflict between the terms in this Supplement and the terms in the SLA, this Supplement shall control. + +12.1. TensorRT DISTRIBUTION +Subject to the terms of the SLA and this Supplement, NVIDIA hereby grants you a non-exclusive, nontransferable license during the applicable license term unless earlier terminated pursuant to the SLA, to distribute the libnvinfer, libnvinfer_plugin, and libnvparsers libraries when delivered to you as part of the TensorRT Licensed Software in source code form or binary form (but not when provided to you as part of a hardware product), subject to the following: such distribution is solely in binary form to your licensees (“Customers”) only as a component of your own software products having additional material functionality beyond the TensorRT Licensed Software (each, a “Licensee Application"). Subject to the terms and conditions of the SLA and this Supplement, you may further authorize Customers to redistribute the libnvinfer, libnvinfer_plugin, and libnvparsers libraries as incorporated into a Licensee Application, solely in binary form, provided, however, that you shall require in your agreements with your Customers that their distributions be on terms at least as restrictive as those applicable for your use of such TensorRT Licensed Software within a Licensee Application. The expiration or termination of your licenses to the above described TensorRT Licensed Software under the SLA and this Supplement will not affect rights previously granted by you to recipients that were in compliance with the SLA and this Supplement. + +In addition to the rights above, for parties that are developing software intended solely for use on Jetson development kits or Jetson modules and running Linux for Tegra software the following shall apply: TensorRT Licensed Software licensed hereunder may be distributed in its entirety, as provided by NVIDIA and without separation of its components, for you and/or your licensees to create software development kits for use only on the Jetson platform and running Linux for Tegra software. You shall require in your agreements with your licensees that their distributions be on terms at least as restrictive as those applicable for your distribution of TensorRT Licensed Software as described in this Section 1. + +In addition to the rights above, for parties that are developing software intended solely for use on Jetson development kits or Jetson modules and running Linux for Tegra software the following shall apply: TensorRT Licensed Software licensed hereunder may be distributed in its entirety, as provided by NVIDIA and without separation of its components, for you and/or your licensees to create software development kits for use only on the Jetson platform and running Linux for Tegra software. You shall require in your agreements with your licensees that their distributions be on terms at least as restrictive as those applicable for your distribution of TensorRT Licensed Software as described in this Section 1. + +12.2. LICENSE DURATION +Each TensorRT Licensed Software is licensed to you for an initial duration of one year starting from the date of delivery or download. The licenses granted will automatically renew for successive one year periods, provided that NVIDIA reserves the right to terminate licenses upon ninety days (90) days written notice to you prior to the commencement of a renewal year in addition to the termination rights set forth in the SLA. + +12.3. EXPIRATION OF TERMINATION OF THIS SUPPLEMENT +Your failure to comply with the terms of this Supplement is ground for termination for breach by NVIDIA under the SLA. This Supplement will automatically expire or terminate upon the expiration or termination of your rights to TensorRT Licensed Software under the SLA or this Supplement. + +Notices +Notice +This document is provided for information purposes only and shall not be regarded as a warranty of a certain functionality, condition, or quality of a product. NVIDIA Corporation (“NVIDIA”) makes no representations or warranties, expressed or implied, as to the accuracy or completeness of the information contained in this document and assumes no responsibility for any errors contained herein. NVIDIA shall have no liability for the consequences or use of such information or for any infringement of patents or other rights of third parties that may result from its use. This document is not a commitment to develop, release, or deliver any Material (defined below), code, or functionality. + +NVIDIA reserves the right to make corrections, modifications, enhancements, improvements, and any other changes to this document, at any time without notice. + +Customer should obtain the latest relevant information before placing orders and should verify that such information is current and complete. + +NVIDIA products are sold subject to the NVIDIA standard terms and conditions of sale supplied at the time of order acknowledgement, unless otherwise agreed in an individual sales agreement signed by authorized representatives of NVIDIA and customer (“Terms of Sale”). NVIDIA hereby expressly objects to applying any customer general terms and conditions with regards to the purchase of the NVIDIA product referenced in this document. No contractual obligations are formed either directly or indirectly by this document. + +NVIDIA products are not designed, authorized, or warranted to be suitable for use in medical, military, aircraft, space, or life support equipment, nor in applications where failure or malfunction of the NVIDIA product can reasonably be expected to result in personal injury, death, or property or environmental damage. NVIDIA accepts no liability for inclusion and/or use of NVIDIA products in such equipment or applications and therefore such inclusion and/or use is at customer’s own risk. + +NVIDIA makes no representation or warranty that products based on this document will be suitable for any specified use. Testing of all parameters of each product is not necessarily performed by NVIDIA. It is customer’s sole responsibility to evaluate and determine the applicability of any information contained in this document, ensure the product is suitable and fit for the application planned by customer, and perform the necessary testing for the application in order to avoid a default of the application or the product. Weaknesses in customer’s product designs may affect the quality and reliability of the NVIDIA product and may result in additional or different conditions and/or requirements beyond those contained in this document. NVIDIA accepts no liability related to any default, damage, costs, or problem which may be based on or attributable to: (i) the use of the NVIDIA product in any manner that is contrary to this document or (ii) customer product designs. + +No license, either expressed or implied, is granted under any NVIDIA patent right, copyright, or other NVIDIA intellectual property right under this document. Information published by NVIDIA regarding third-party products or services does not constitute a license from NVIDIA to use such products or services or a warranty or endorsement thereof. Use of such information may require a license from a third party under the patents or other intellectual property rights of the third party, or a license from NVIDIA under the patents or other intellectual property rights of NVIDIA. + +Reproduction of information in this document is permissible only if approved in advance by NVIDIA in writing, reproduced without alteration and in full compliance with all applicable export laws and regulations, and accompanied by all associated conditions, limitations, and notices. + +THIS DOCUMENT AND ALL NVIDIA DESIGN SPECIFICATIONS, REFERENCE BOARDS, FILES, DRAWINGS, DIAGNOSTICS, LISTS, AND OTHER DOCUMENTS (TOGETHER AND SEPARATELY, “MATERIALS”) ARE BEING PROVIDED “AS IS.” NVIDIA MAKES NO WARRANTIES, EXPRESSED, IMPLIED, STATUTORY, OR OTHERWISE WITH RESPECT TO THE MATERIALS, AND EXPRESSLY DISCLAIMS ALL IMPLIED WARRANTIES OF NONINFRINGEMENT, MERCHANTABILITY, AND FITNESS FOR A PARTICULAR PURPOSE. TO THE EXTENT NOT PROHIBITED BY LAW, IN NO EVENT WILL NVIDIA BE LIABLE FOR ANY DAMAGES, INCLUDING WITHOUT LIMITATION ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, PUNITIVE, OR CONSEQUENTIAL DAMAGES, HOWEVER CAUSED AND REGARDLESS OF THE THEORY OF LIABILITY, ARISING OUT OF ANY USE OF THIS DOCUMENT, EVEN IF NVIDIA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Notwithstanding any damages that customer might incur for any reason whatsoever, NVIDIA’s aggregate and cumulative liability towards customer for the products described herein shall be limited in accordance with the Terms of Sale for the product. + +VESA DisplayPort +DisplayPort and DisplayPort Compliance Logo, DisplayPort Compliance Logo for Dual-mode Sources, and DisplayPort Compliance Logo for Active Cables are trademarks owned by the Video Electronics Standards Association in the United States and other countries. + +HDMI +HDMI, the HDMI logo, and High-Definition Multimedia Interface are trademarks or registered trademarks of HDMI Licensing LLC. + +ARM +ARM, AMBA and ARM Powered are registered trademarks of ARM Limited. Cortex, MPCore and Mali are trademarks of ARM Limited. All other brands or product names are the property of their respective holders. "ARM" is used to represent ARM Holdings plc; its operating company ARM Limited; and the regional subsidiaries ARM Inc.; ARM KK; ARM Korea Limited.; ARM Taiwan Limited; ARM France SAS; ARM Consulting (Shanghai) Co. Ltd.; ARM Germany GmbH; ARM Embedded Technologies Pvt. Ltd.; ARM Norway, AS and ARM Sweden AB. + +OpenCL +OpenCL is a trademark of Apple Inc. used under license to the Khronos Group Inc. + +Trademarks +NVIDIA, the NVIDIA logo, and cuBLAS, CUDA, CUDA Toolkit, cuDNN, DALI, DIGITS, DGX, DGX-1, DGX-2, DGX Station, DLProf, GPU, JetPack, Jetson, Kepler, Maxwell, NCCL, Nsight Compute, Nsight Systems, NVCaffe, NVIDIA Ampere GPU architecture, NVIDIA Deep Learning SDK, NVIDIA Developer Program, NVIDIA GPU Cloud, NVLink, NVSHMEM, PerfWorks, Pascal, SDK Manager, T4, Tegra, TensorRT, TensorRT Inference Server, Tesla, TF-TRT, Triton Inference Server, Turing, and Volta are trademarks and/or registered trademarks of NVIDIA Corporation in the United States and other countries. Other company and product names may be trademarks of the respective companies with which they are associated. + +Copyright +© 2021 NVIDIA Corporation. All rights reserved. diff --git a/python/packaging/frontend_sdist/setup.cfg b/python/packaging/frontend_sdist/setup.cfg new file mode 100644 index 00000000..32a8c1c0 --- /dev/null +++ b/python/packaging/frontend_sdist/setup.cfg @@ -0,0 +1,5 @@ +[metadata] +license_files = LICENSE.txt + +[bdist_wheel] +universal = 1 diff --git a/python/packaging/frontend_sdist/setup.py b/python/packaging/frontend_sdist/setup.py new file mode 100644 index 00000000..7d647ffb --- /dev/null +++ b/python/packaging/frontend_sdist/setup.py @@ -0,0 +1,70 @@ +# +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import sys + +from setuptools import setup +from setuptools.command.install import install +import subprocess as sp + +tensorrt_module = "##TENSORRT_MODULE##" + + +class InstallCommand(install): + def run(self): + def install_dep(package_name): + status = sp.run( + [ + sys.executable, + "-m", + "pip", + "install", + "{:}==##TENSORRT_PYTHON_VERSION##".format(package_name), + "--index-url", + "https://pypi.nvidia.com", + ] + ) + status.check_returncode() + + install_dep("{:}_libs".format(tensorrt_module)) + install_dep("{:}_bindings".format(tensorrt_module)) + + install.run(self) + + +setup( + name=tensorrt_module, + version="##TENSORRT_PYTHON_VERSION##", + description="A high performance deep learning inference library", + long_description="A high performance deep learning inference library", + author="NVIDIA Corporation", + license="Proprietary", + classifiers=[ + "License :: Other/Proprietary License", + "Intended Audience :: Developers", + "Programming Language :: Python :: 3", + ], + packages=[tensorrt_module], + extras_require={"numpy": "numpy"}, + package_data={tensorrt_module: ["*.so*", "*.pyd", "*.pdb"]}, + include_package_data=True, + zip_safe=True, + keywords="nvidia tensorrt deeplearning inference", + url="https://developer.nvidia.com/tensorrt", + download_url="https://github.com/nvidia/tensorrt/tags", + cmdclass={"install": InstallCommand}, +) diff --git a/python/packaging/frontend_sdist/tensorrt/__init__.py b/python/packaging/frontend_sdist/tensorrt/__init__.py new file mode 100644 index 00000000..ee379126 --- /dev/null +++ b/python/packaging/frontend_sdist/tensorrt/__init__.py @@ -0,0 +1,18 @@ +# +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +from tensorrt_bindings import * diff --git a/python/packaging/libs_wheel/LICENSE.txt b/python/packaging/libs_wheel/LICENSE.txt new file mode 100644 index 00000000..08f07f9f --- /dev/null +++ b/python/packaging/libs_wheel/LICENSE.txt @@ -0,0 +1,180 @@ +Abstract +This document is the Software License Agreement (SLA) for NVIDIA TensorRT. This document contains specific license terms and conditions for NVIDIA TensorRT. By accepting this agreement, you agree to comply with all the terms and conditions applicable to the specific product(s) included herein. + +If you are receiving TensorRT under the NVIDIA Prerelease License Agreement (also known as NPLA) or under the NVIDIA Software License Agreement (previously known as the NVIDIA Tegra Software License Agreement), your use of TensorRT is governed by such applicable terms and conditions. All other uses of TensorRT are governed by the terms and conditions of the below license agreement. + +NVIDIA SOFTWARE LICENSE AGREEMENT +Important: READ BEFORE DOWNLOADING, INSTALLING, COPYING OR USING THE LICENSED SOFTWARE +This Software License Agreement ("SLA”), made and entered into as of the time and date of click through action (“Effective Date”),is a legal agreement between you and NVIDIA Corporation ("NVIDIA") and governs the use of the NVIDIA computer software and the documentation made available for use with such NVIDIA software. By downloading, installing, copying, or otherwise using the NVIDIA software and/or documentation, you agree to be bound by the terms of this SLA. If you do not agree to the terms of this SLA, do not download, install, copy or use the NVIDIA software or documentation. IF YOU ARE ENTERING INTO THIS SLAON BEHALF OF A COMPANY OR OTHER LEGAL ENTITY, YOU REPRESENT THAT YOU HAVE THE LEGAL AUTHORITY TO BIND THE ENTITY TO THIS SLA, IN WHICH CASE “YOU” WILL MEAN THE ENTITY YOU REPRESENT. IF YOU DON’T HAVE SUCH AUTHORITY, OR IF YOU DON’T ACCEPT ALL THE TERMS AND CONDITIONS OF THIS SLA, THEN NVIDIA DOES NOT AGREETO LICENSE THE LICENSED SOFTWARETO YOU, AND YOU MAY NOT DOWNLOAD, INSTALL, COPY OR USE IT. + +Preface +This document is the Software License Agreement (SLA) for NVIDIA TensorRT. This document contains specific license terms and conditions for NVIDIA TensorRT. By accepting this agreement, you agree to comply with all the terms and conditions applicable to the specific product(s) included herein. + +If you are receiving TensorRT under the NVIDIA Prerelease License Agreement (also known as NPLA) or under the NVIDIA Software License Agreement (previously known as the NVIDIA Tegra Software License Agreement), your use of TensorRT is governed by such applicable terms and conditions. All other uses of TensorRT are governed by the terms and conditions of the below license agreement. + +NVIDIA SOFTWARE LICENSE AGREEMENT +Important: READ BEFORE DOWNLOADING, INSTALLING, COPYING OR USING THE LICENSED SOFTWARE +This Software License Agreement ("SLA”), made and entered into as of the time and date of click through action (“Effective Date”),is a legal agreement between you and NVIDIA Corporation ("NVIDIA") and governs the use of the NVIDIA computer software and the documentation made available for use with such NVIDIA software. By downloading, installing, copying, or otherwise using the NVIDIA software and/or documentation, you agree to be bound by the terms of this SLA. If you do not agree to the terms of this SLA, do not download, install, copy or use the NVIDIA software or documentation. IF YOU ARE ENTERING INTO THIS SLAON BEHALF OF A COMPANY OR OTHER LEGAL ENTITY, YOU REPRESENT THAT YOU HAVE THE LEGAL AUTHORITY TO BIND THE ENTITY TO THIS SLA, IN WHICH CASE “YOU” WILL MEAN THE ENTITY YOU REPRESENT. IF YOU DON’T HAVE SUCH AUTHORITY, OR IF YOU DON’T ACCEPT ALL THE TERMS AND CONDITIONS OF THIS SLA, THEN NVIDIA DOES NOT AGREETO LICENSE THE LICENSED SOFTWARETO YOU, AND YOU MAY NOT DOWNLOAD, INSTALL, COPY OR USE IT. + +1. LICENSE. +1.1. License Grant +Subject to the terms of the AGREEMENT, NVIDIA hereby grants you a non-exclusive, non-transferable license, without the right to sublicense (except as expressly set forth in a Supplement), during the applicable license term unless earlier terminated as provided below, to have Authorized Users install and use the Software, including modifications (if expressly permitted in a Supplement), in accordance with the Documentation. You are only licensed to activate and use Licensed Software for which you a have a valid license, even if during the download or installation you are presented with other product options. No Orders are binding on NVIDIA until accepted by NVIDIA. Your Orders are subject to the AGREEMENT. + +SLA Supplements: Certain Licensed Software licensed under this SLA may be subject to additional terms and conditions that will be presented to you in a Supplement for acceptance prior to the delivery of such Licensed Software under this SLA and the applicable Supplement. Licensed Software will only be delivered to you upon your acceptance of all applicable terms. + +1.2. Limited Purpose Licenses +If your license is provided for one of the purposes indicated below, then notwithstanding contrary terms in License Grant or in a Supplement, such licenses are for internal use and do not include any right or license to sub-license and distribute the Licensed Software or its output in any way in any public release, however limited, and/or in any manner that provides third parties with use of or access to the Licensed Software or its functionality or output, including (but not limited to) external alpha or beta testing or development phases. Further: +Evaluation License. You may use evaluation licenses solely for your internal evaluation of the Licensed Software for broader adoption within your Enterprise or in connection with a NVIDIA product purchase decision, and such licenses have an expiration date as indicated by NVIDIA in its sole discretion (or ninety days from the date of download if no other duration is indicated). +Educational/Academic License. You may use educational/academic licenses solely for educational purposes and all users must be enrolled or employed by an academic institution. If you do not meet NVIDIA’s academic program requirements for educational institutions, you have no rights under this license. +Test/Development License. You may use test/development licenses solely for your internal development, testing and/or debugging of your software applications or for interoperability testing with the Licensed Software, and such licenses have an expiration date as indicated by NVIDIA in its sole discretion (or one year from the date of download if no other duration is indicated). NVIDIA Confidential Information under the AGREEMENT includes output from Licensed Software developer tools identified as “Pro” versions, where the output reveals functionality or performance data pertinent to NVIDIA hardware or software products. +1.3. Pre-Release Licenses +With respect to alpha, beta, preview, and other pre-release Software and Documentation (“Pre-Release Licensed Software”) delivered to you under the AGREEMENT you acknowledge and agree that such Pre-Release Licensed Software (i) may not be fully functional, may contain errors or design flaws, and may have reduced or different security, privacy, accessibility, availability, and reliability standards relative to commercially provided NVIDIA software and documentation, and (ii) use of such Pre-Release Licensed Software may result in unexpected results, loss of data, project delays or other unpredictable damage or loss. THEREFORE, PRE-RELEASE LICENSED SOFTWARE IS NOT INTENDED FOR USE, AND SHOULD NOT BE USED, IN PRODUCTION OR BUSINESS-CRITICAL SYSTEMS. NVIDIA has no obligation to make available a commercial version of any Pre-Release Licensed Software and NVIDIA has the right to abandon development of Pre-Release Licensed Software at any time without liability. + +1.4. Enterprise and Contractor Usage +You may allow your Enterprise employees and Contractors to access and use the Licensed Software pursuant to the terms of the AGREEMENT solely to perform work on your behalf, provided further that with respect to Contractors: (i) you obtain a written agreement from each Contractor which contains terms and obligations with respect to access to and use of Licensed Software no less protective of NVIDIA than those set forth in the AGREEMENT, and (ii) such Contractor’s access and use expressly excludes any sublicensing or distribution rights for the Licensed Software. You are responsible for the compliance with the terms and conditions of the AGREEMENT by your Enterprise and Contractors. Any act or omission that, if committed by you, would constitute a breach of the AGREEMENT shall be deemed to constitute a breach of the AGREEMENT if committed by your Enterprise or Contractors. + +1.5. Services +Except as expressly indicated in an Order, NVIDIA is under no obligation to provide support for the Licensed Software or to provide any patches, maintenance, updates or upgrades under the AGREEMENT. Unless patches, maintenance, updates or upgrades are provided with their separate governing terms and conditions, they constitute Licensed Software licensed to you under the AGREEMENT. + +2. LIMITATIONS. +2.1. License Restrictions +Except as expressly authorized in the AGREEMENT, you agree that you will not (nor authorize third parties to): (i) copy and use Software that was licensed to you for use in one or more NVIDIA hardware products in other unlicensed products (provided that copies solely for backup purposes are allowed); (ii) reverse engineer, decompile, disassemble (except to the extent applicable laws specifically require that such activities be permitted) or attempt to derive the source code, underlying ideas, algorithm or structure of Software provided to you in object code form; (iii) sell, transfer, assign, distribute, rent, loan, lease, sublicense or otherwise make available the Licensed Software or its functionality to third parties (a) as an application services provider or service bureau, (b) by operating hosted/virtual system environments, (c) by hosting, time sharing or providing any other type of services, or (d) otherwise by means of the internet; (iv) modify, translate or otherwise create any derivative works of any Licensed Software; (v) remove, alter, cover or obscure any proprietary notice that appears on or with the Licensed Software or any copies thereof; (vi) use the Licensed Software, or allow its use, transfer, transmission or export in violation of any applicable export control laws, rules or regulations; (vii) distribute, permit access to, or sublicense the Licensed Software as a stand-alone product; (viii) bypass, disable, circumvent or remove any form of copy protection, encryption, security or digital rights management or authentication mechanism used by NVIDIA in connection with the Licensed Software, or use the Licensed Software together with any authorization code, serial number, or other copy protection device not supplied by NVIDIA directly or through an authorized reseller; (ix) use the Licensed Software for the purpose of developing competing products or technologies or assisting a third party in such activities; (x) use the Licensed Software with any system or application where the use or failure of such system or application can reasonably be expected to threaten or result in personal injury, death, or catastrophic loss including, without limitation, use in connection with any nuclear, avionics, navigation, military, medical, life support or other life critical application (“Critical Applications”), unless the parties have entered into a Critical Applications agreement; (xi) distribute any modification or derivative work you make to the Licensed Software under or by reference to the same name as used by NVIDIA; or (xii) use the Licensed Software in any manner that would cause the Licensed Software to become subject to an Open Source License. Nothing in the AGREEMENT shall be construed to give you a right to use, or otherwise obtain access to, any source code from which the Software or any portion thereof is compiled or interpreted. You acknowledge that NVIDIA does not design, test, manufacture or certify the Licensed Software for use in the context of a Critical Application and NVIDIA shall not be liable to you or any third party, in whole or in part, for any claims or damages arising from such use. You agree to defend, indemnify and hold harmless NVIDIA and its Affiliates, and their respective employees, contractors, agents, officers and directors, from and against any and all claims, damages, obligations, losses, liabilities, costs or debt, fines, restitutions and expenses (including but not limited to attorney’s fees and costs incident to establishing the right of indemnification) arising out of or related to you and your Enterprise, and their respective employees, contractors, agents, distributors, resellers, end users, officers and directors use of Licensed Software outside of the scope of the AGREEMENT or any other breach of the terms of the AGREEMENT. + +2.2. Third Party License Obligations +You acknowledge and agree that the Licensed Software may include or incorporate third party technology (collectively “Third Party Components”), which is provided for use in or with the Software and not otherwise used separately. If the Licensed Software includes or incorporates Third Party Components, then the third-party pass-through terms and conditions (“Third Party Terms”) for the particular Third Party Component will be bundled with the Software or otherwise made available online as indicated by NVIDIA and will be incorporated by reference into the AGREEMENT. In the event of any conflict between the terms in the AGREEMENT and the Third Party Terms, the Third Party Terms shall govern. Copyright to Third Party Components are held by the copyright holders indicated in the copyright notices indicated in the Third Party Terms. + +Audio/Video Encoders and Decoders: You acknowledge and agree that it is your sole responsibility to obtain any additional third party licenses required to make, have made, use, have used, sell, import, and offer for sale your products or services that include or incorporate any Third Party Components and content relating to audio and/or video encoders and decoders from, including but not limited to, Microsoft, Thomson, Fraunhofer IIS, Sisvel S.p.A., MPEG-LA, and Coding Technologies as NVIDIA does not grant to you under the AGREEMENT any necessary patent or other rights with respect to audio and/or video encoders and decoders. + +2.3. Limited Rights +Your rights in the Licensed Software are limited to those expressly granted under the AGREEMENT and no other licenses are granted whether by implication, estoppel or otherwise. NVIDIA reserves all rights, title and interest in and to the Licensed Software not expressly granted under the AGREEMENT. + +3. CONFIDENTIALITY +Neither party will use the other party’s Confidential Information, except as necessary for the performance of the AGREEMENT, nor will either party disclose such Confidential Information to any third party, except to personnel of NVIDIA and its Affiliates, you, your Enterprise, your Enterprise Contractors, and each party’s legal and financial advisors that have a need to know such Confidential Information for the performance of the AGREEMENT, provided that each such personnel, employee and Contractor is subject to a written agreement that includes confidentiality obligations consistent with those set forth herein. Each party will use all reasonable efforts to maintain the confidentiality of all of the other party’s Confidential Information in its possession or control, but in no event less than the efforts that it ordinarily uses with respect to its own Confidential Information of similar nature and importance. The foregoing obligations will not restrict either party from disclosing the other party’s Confidential Information or the terms and conditions of the AGREEMENT as required under applicable securities regulations or pursuant to the order or requirement of a court, administrative agency, or other governmental body, provided that the party required to make such disclosure (i) gives reasonable notice to the other party to enable it to contest such order or requirement prior to its disclosure (whether through protective orders or otherwise), (ii) uses reasonable effort to obtain confidential treatment or similar protection to the fullest extent possible to avoid such public disclosure, and (iii) discloses only the minimum amount of information necessary to comply with such requirements. + +4. OWNERSHIP +You are not obligated to disclose to NVIDIA any modifications that you, your Enterprise or your Contractors make to the Licensed Software as permitted under the AGREEMENT. As between the parties, all modifications are owned by NVIDIA and licensed to you under the AGREEMENT unless otherwise expressly provided in a Supplement. The Licensed Software and all modifications owned by NVIDIA, and the respective Intellectual Property Rights therein, are and will remain the sole and exclusive property of NVIDIA or its licensors, whether the Licensed Software is separate from or combined with any other products or materials. You shall not engage in any act or omission that would impair NVIDIA’s and/or its licensors’ Intellectual Property Rights in the Licensed Software or any other materials, information, processes or subject matter proprietary to NVIDIA. NVIDIA’s licensors are intended third party beneficiaries with the right to enforce provisions of the AGREEMENT with respect to their Confidential Information and/or Intellectual Property Rights. + +5. FEEDBACK +You have no obligation to provide Feedback to NVIDIA. However, NVIDIA and/or its Affiliates may use and include any Feedback that you provide to improve the Licensed Software or other NVIDIA products, technologies or materials. Accordingly, if you provide Feedback, you agree that NVIDIA and/or its Affiliates, at their option, may, and may permit their licensees, to make, have made, use, have used, reproduce, license, distribute and otherwise commercialize the Feedback in the Licensed Software or in other NVIDIA products, technologies or materials without the payment of any royalties or fees to you. All Feedback becomes the sole property of NVIDIA and may be used in any manner NVIDIA sees fit, and you hereby assign to NVIDIA all of your right, title and interest in and to any Feedback. NVIDIA has no obligation to respond to Feedback or to incorporate Feedback into the Licensed Software. + +6. NO WARRANTIES +THE LICENSED SOFTWARE AND ANY OTHER CONFIDENTIAL INFORMATION AND/OR SERVICES ARE PROVIDED BY NVIDIA “AS IS” AND “WITH ALL FAULTS,” AND NVIDIA EXPRESSLY DISCLAIMS ALL OTHER WARRANTIES OF ANY KIND OR NATURE, WHETHER EXPRESS, IMPLIED OR STATUTORY, INCLUDING, BUT NOT LIMITED TO, ANY WARRANTIES OF OPERABILITY, CONDITION, VALUE, ACCURACY OF DATA, OR QUALITY, AS WELL AS ANY WARRANTIES OF MERCHANTABILITY, SYSTEM INTEGRATION, WORKMANSHIP, SUITABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, OR THE ABSENCE OF ANY DEFECTS THEREIN, WHETHER LATENT OR PATENT. NO WARRANTY IS MADE BY NVIDIA ON THE BASIS OF TRADE USAGE, COURSE OF DEALING OR COURSE OF TRADE. NVIDIA DOES NOT WARRANT THAT THE LICENSED SOFTWARE OR ANY OTHER CONFIDENTIAL INFORMATION AND/OR SERVICES PROVIDED BY NVIDIA UNDER THE AGREEMENT WILL MEET YOUR REQUIREMENTS OR THAT THE OPERATION THEREOF WILL BE UNINTERRUPTED OR ERROR-FREE, OR THAT ALL ERRORS WILL BE CORRECTED. YOU ACKNOWLEDGE THAT NVIDIA’S OBLIGATIONS UNDER THE AGREEMENT ARE FOR THE BENEFIT OF YOU ONLY. Nothing in this warranty section affects any statutory rights of consumers or other recipients to the extent that they cannot be waived or limited by contract under applicable law. + +7. LIMITATION OF LIABILITY +TO THE MAXIMUM EXTENT PERMITTED BY LAW, NVIDIA OR ITS LICENSORS SHALL NOT BE LIABLE FOR ANY SPECIAL, INCIDENTAL, PUNITIVE OR CONSEQUENTIAL DAMAGES, OR ANY LOST PROFITS, LOSS OF USE, LOSS OF DATA OR LOSS OF GOODWILL, OR THE COSTS OF PROCURING SUBSTITUTE PRODUCTS, ARISING OUT OF OR IN CONNECTION WITH THE AGREEMENT OR THE USE OR PERFORMANCE OF THE LICENSED SOFTWARE AND ANY OTHER CONFIDENTIAL INFORMATION AND/OR SERVICES PROVIDED BY NVIDIA UNDER THE AGREEMENT, WHETHER SUCH LIABILITY ARISES FROM ANY CLAIM BASED UPON BREACH OF CONTRACT, BREACH OF WARRANTY, TORT (INCLUDING NEGLIGENCE), PRODUCT LIABILITY OR ANY OTHER CAUSE OF ACTION OR THEORY OF LIABILITY. IN NO EVENT WILL NVIDIA’S TOTAL CUMULATIVE LIABILITY UNDER OR ARISING OUT OF THE AGREEMENT EXCEED THE NET AMOUNTS RECEIVED BY NVIDIA FOR YOUR USE OF THE PARTICULAR LICENSED SOFTWARE DURING THE TWELVE (12) MONTHS BEFORE THE LIABILITY AROSE (or up to US$10.00 if you acquired the Licensed Software for no charge). THE NATURE OF THE LIABILITY, THE NUMBER OF CLAIMS OR SUITS OR THE NUMBER OF PARTIES WITHIN YOUR ENTERPRISE THAT ACCEPTED THE TERMS OF THE AGREEMENT SHALL NOT ENLARGE OR EXTEND THIS LIMIT. THE FOREGOING LIMITATIONS SHALL APPLY REGARDLESS OF WHETHER NVIDIA OR ITS LICENSORS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES AND REGARDLESS OF WHETHER ANY REMEDY FAILS ITS ESSENTIAL PURPOSE. The disclaimers, exclusions and limitations of liability set forth in the AGREEMENT form an essential basis of the bargain between the parties, and, absent any such disclaimers, exclusions or limitations of liability, the provisions of the AGREEMENT, including, without limitation, the economic terms, would be substantially different. + +8. TERM AND TERMINATION. +8.1. AGREEMENT, Licenses and Services +This SLA shall become effective upon the Effective Date, each Supplement upon their acceptance, and both this SLA and Supplements shall continue in effect until your last access or use of the Licensed Software and/or services hereunder, unless earlier terminated as provided in this “Term and Termination” section. Each Licensed Software license ends at the earlier of (a) the expiration of the applicable license term, or (b) termination of such license or the AGREEMENT. Each service ends at the earlier of (x) the expiration of the applicable service term, (y) termination of such service or the AGREEMENT, or (z) expiration or termination of the associated license and no credit or refund will be provided upon the expiration or termination of the associated license for any service fees paid. + +8.2. Termination and Effect of Expiration or Termination +NVIDIA may terminate the AGREEMENT in whole or in part: (i) if you breach any term of the AGREEMENT and fail to cure such breach within thirty (30) days following notice thereof from NVIDIA (or immediately if you violate NVIDIA’s Intellectual Property Rights); (ii) if you become the subject of a voluntary or involuntary petition in bankruptcy or any proceeding relating to insolvency, receivership, liquidation or composition for the benefit of creditors, if that petition or proceeding is not dismissed with prejudice within sixty (60) days after filing, or if you cease to do business; or (iii) if you commence or participate in any legal proceeding against NVIDIA, with respect to the Licensed Software that is the subject of the proceeding during the pendency of such legal proceeding. If you or your authorized NVIDIA reseller fail to pay license fees or service fees when due then NVIDIA may, in its sole discretion, suspend or terminate your license grants, services and any other rights provided under the AGREEMENT for the affected Licensed Software, in addition to any other remedies NVIDIA may have at law or equity. Upon any expiration or termination of the AGREEMENT, a license or a service provided hereunder, (a) any amounts owed to NVIDIA become immediately due and payable, (b) you must promptly discontinue use of the affected Licensed Software and/or service, and (c) you must promptly destroy or return to NVIDIA all copies of the affected Licensed Software and all portions thereof in your possession or control, and each party will promptly destroy or return to the other all of the other party’s Confidential Information within its possession or control. Upon written request, you will certify in writing that you have complied with your obligations under this section. Upon expiration or termination of the AGREEMENT all provisions survive except for the license grant provisions. + +9. CONSENT TO COLLECTION AND USE OF INFORMATION. +You hereby agree and acknowledge that the Software may access, collect non-personally identifiable information about your Enterprise computer systems in order to properly optimize such systems for use with the Software. To the extent that you use the Software, you hereby consent to all of the foregoing, and represent and warrant that you have the right to grant such consent. In addition, you agree that you are solely responsible for maintaining appropriate data backups and system restore points for your Enterprise systems, and that NVIDIA will have no responsibility for any damage or loss to such systems (including loss of data or access) arising from or relating to (a) any changes to the configuration, application settings, environment variables, registry, drivers, BIOS, or other attributes of the systems (or any part of such systems) initiated through the Software; or (b) installation of any Software or third party software patches initiated through the Software. In certain systems you may change your system update preferences by unchecking "Automatically check for updates" in the "Preferences" tab of the control panel for the Software. + +In connection with the receipt of the Licensed Software or services you may receive access to links to third party websites and services and the availability of those links does not imply any endorsement by NVIDIA. NVIDIA encourages you to review the privacy statements on those sites and services that you choose to visit so that you can understand how they may collect, use and share personal information of individuals. NVIDIA is not responsible or liable for: (i) the availability or accuracy of such links; or (ii) the products, services or information available on or through such links; or (iii) the privacy statements or practices of sites and services controlled by other companies or organizations. + +To the extent that you or members of your Enterprise provide to NVIDIA during registration or otherwise personal information, you acknowledge that such information will be collected, used and disclosed by NVIDIA in accordance with NVIDIA's privacy policy, available at URL http://www.nvidia.com/object/privacy_policy.html. + +10. GENERAL. +This SLA, any Supplements incorporated hereto, and Orders constitute the entire agreement of the parties with respect to the subject matter hereto and supersede all prior negotiations, conversations, or discussions between the parties relating to the subject matter hereto, oral or written, and all past dealings or industry custom. Any additional and/or conflicting terms and conditions on purchase order(s) or any other documents issued by you are null, void, and invalid. Any amendment or waiver under the AGREEMENT must be in writing and signed by representatives of both parties. + +The AGREEMENT and the rights and obligations thereunder may not be assigned by you, in whole or in part, including by merger, consolidation, dissolution, operation of law, or any other manner, without written consent of NVIDIA, and any purported assignment in violation of this provision shall be void and of no effect. NVIDIA may assign, delegate or transfer the AGREEMENT and its rights and obligations hereunder, and if to a non-Affiliate you will be notified. + +Each party acknowledges and agrees that the other is an independent contractor in the performance of the AGREEMENT, and each party is solely responsible for all of its employees, agents, contractors, and labor costs and expenses arising in connection therewith. The parties are not partners, joint ventures or otherwise affiliated, and neither has any authority to make any statements, representations or commitments of any kind to bind the other party without prior written consent. + +Neither party will be responsible for any failure or delay in its performance under the AGREEMENT (except for any payment obligations) to the extent due to causes beyond its reasonable control for so long as such force majeure event continues in effect. + +The AGREEMENT will be governed by and construed under the laws of the State of Delaware and the United States without regard to the conflicts of law provisions thereof and without regard to the United Nations Convention on Contracts for the International Sale of Goods. The parties consent to the personal jurisdiction of the federal and state courts located in Santa Clara County, California. You acknowledge and agree that a breach of any of your promises or agreements contained in the AGREEMENT may result in irreparable and continuing injury to NVIDIA for which monetary damages may not be an adequate remedy and therefore NVIDIA is entitled to seek injunctive relief as well as such other and further relief as may be appropriate. If any court of competent jurisdiction determines that any provision of the AGREEMENT is illegal, invalid or unenforceable, the remaining provisions will remain in full force and effect. Unless otherwise specified, remedies are cumulative. + +The Licensed Software has been developed entirely at private expense and is “commercial items” consisting of “commercial computer software” and “commercial computer software documentation” provided with RESTRICTED RIGHTS. Use, duplication or disclosure by the U.S. Government or a U.S. Government subcontractor is subject to the restrictions set forth in the AGREEMENT pursuant to DFARS 227.7202-3(a) or as set forth in subparagraphs (c)(1) and (2) of the Commercial Computer Software - Restricted Rights clause at FAR 52.227-19, as applicable. Contractor/manufacturer is NVIDIA, 2701 San Tomas Expressway, Santa Clara, CA 95050. + +You acknowledge that the Licensed Software described under the AGREEMENT is subject to export control under the U.S. Export Administration Regulations (EAR) and economic sanctions regulations administered by the U.S. Department of Treasury’s Office of Foreign Assets Control (OFAC). Therefore, you may not export, reexport or transfer in-country the Licensed Software without first obtaining any license or other approval that may be required by BIS and/or OFAC. You are responsible for any violation of the U.S. or other applicable export control or economic sanctions laws, regulations and requirements related to the Licensed Software. By accepting this SLA, you confirm that you are not a resident or citizen of any country currently embargoed by the U.S. and that you are not otherwise prohibited from receiving the Licensed Software. + +Any notice delivered by NVIDIA to you under the AGREEMENT will be delivered via mail, email or fax. Please direct your legal notices or other correspondence to NVIDIA Corporation, 2701 San Tomas Expressway, Santa Clara, California 95050, United States of America, Attention: Legal Department. + +11. GLOSSARY OF TERMS +Certain capitalized terms, if not otherwise defined elsewhere in this SLA, shall have the meanings set forth below: +“Affiliate” +“Affiliate” means any legal entity that Owns, is Owned by, or is commonly Owned with a party. “Own” means having more than 50% ownership or the right to direct the management of the entity. +“AGREEMENT” +“AGREEMENT” means this SLA and all associated Supplements entered by the parties referencing this SLA. +“Authorized Users” +“Authorized Users” means your Enterprise individual employees and any of your Enterprise’s Contractors, subject to the terms of the “Enterprise and Contractors Usage” section. +“Confidential Information” +“Confidential Information” means the Licensed Software (unless made publicly available by NVIDIA without confidentiality obligations), and any NVIDIA business, marketing, pricing, research and development, know-how, technical, scientific, financial status, proposed new products or other information disclosed by NVIDIA to you which, at the time of disclosure, is designated in writing as confidential or proprietary (or like written designation), or orally identified as confidential or proprietary or is otherwise reasonably identifiable by parties exercising reasonable business judgment, as confidential. Confidential Information does not and will not include information that: (i) is or becomes generally known to the public through no fault of or breach of the AGREEMENT by the receiving party; (ii) is rightfully known by the receiving party at the time of disclosure without an obligation of confidentiality; (iii) is independently developed by the receiving party without use of the disclosing party’s Confidential Information; or (iv) is rightfully obtained by the receiving party from a third party without restriction on use or disclosure. +“Contractor” +“Contractor” means an individual who works primarily for your Enterprise on a contractor basis from your secure network. means an individual who works primarily for your Enterprise on a contractor basis from your secure network. +“Documentation” +“Documentation” means the NVIDIA documentation made available for use with the Software, including (without limitation) user manuals, datasheets, operations instructions, installation guides, release notes and other materials provided to you under the AGREEMENT. +“Enterprise” +“Enterprise” means you or any company or legal entity for which you accepted the terms of this SLA, and their subsidiaries of which your company or legal entity owns more than fifty percent (50%) of the issued and outstanding equity. +“Feedback” +“Feedback” means any and all suggestions, feature requests, comments or other feedback regarding the Licensed Software, including possible enhancements or modifications thereto. +“Intellectual Property Rights” +“Intellectual Property Rights” means all patent, copyright, trademark, trade secret, trade dress, trade names, utility models, mask work, moral rights, rights of attribution or integrity service marks, master recording and music publishing rights, performance rights, author’s rights, database rights, registered design rights and any applications for the protection or registration of these rights, or other intellectual or industrial property rights or proprietary rights, howsoever arising and in whatever media, whether now known or hereafter devised, whether or not registered, (including all claims and causes of action for infringement, misappropriation or violation and all rights in any registrations and renewals), worldwide and whether existing now or in the future. +“Licensed Software” +“Licensed Software” means Software, Documentation and all modifications owned by NVIDIA. +“Open Source License” +“Open Source License” includes, without limitation, a software license that requires as a condition of use, modification, and/or distribution of such software that the Software be (i) disclosed or distributed in source code form; (ii) be licensed for the purpose of making derivative works; or (iii) be redistributable at no charge. +“Order” +“Order” means a purchase order issued by you, a signed purchase agreement with you, or other ordering document issued by you to NVIDIA or a NVIDIA authorized reseller (including any on-line acceptance process) that references and incorporates the AGREEMENT and is accepted by NVIDIA. +“Software” +“Software” means the NVIDIA software programs licensed to you under the AGREEMENT including, without limitation, libraries, sample code, utility programs and programming code. +“Supplement” +“Supplement” means the additional terms and conditions beyond those stated in this SLA that apply to certain Licensed Software licensed hereunder. +12. TensorRT SUPPLEMENT TO SOFTWARE LICENSE AGREEMENT +TensorRT SUPPLEMENT TO SOFTWARE LICENSE AGREEMENT +The terms set forth in this TensorRT Supplement (“Supplement”) govern your use of the NVIDIA GPU inference engine (the “TensorRT Licensed Software”) under the terms of your software license agreement (“SLA”) as modified by this Supplement. This Supplement is an exhibit to the SLA and is hereby incorporated as an integral part thereto. Capitalized terms used but not defined herein shall have the meaning assigned to them in the SLA. In the event of conflict between the terms in this Supplement and the terms in the SLA, this Supplement shall control. + +12.1. TensorRT DISTRIBUTION +Subject to the terms of the SLA and this Supplement, NVIDIA hereby grants you a non-exclusive, nontransferable license during the applicable license term unless earlier terminated pursuant to the SLA, to distribute the libnvinfer, libnvinfer_plugin, and libnvparsers libraries when delivered to you as part of the TensorRT Licensed Software in source code form or binary form (but not when provided to you as part of a hardware product), subject to the following: such distribution is solely in binary form to your licensees (“Customers”) only as a component of your own software products having additional material functionality beyond the TensorRT Licensed Software (each, a “Licensee Application"). Subject to the terms and conditions of the SLA and this Supplement, you may further authorize Customers to redistribute the libnvinfer, libnvinfer_plugin, and libnvparsers libraries as incorporated into a Licensee Application, solely in binary form, provided, however, that you shall require in your agreements with your Customers that their distributions be on terms at least as restrictive as those applicable for your use of such TensorRT Licensed Software within a Licensee Application. The expiration or termination of your licenses to the above described TensorRT Licensed Software under the SLA and this Supplement will not affect rights previously granted by you to recipients that were in compliance with the SLA and this Supplement. + +In addition to the rights above, for parties that are developing software intended solely for use on Jetson development kits or Jetson modules and running Linux for Tegra software the following shall apply: TensorRT Licensed Software licensed hereunder may be distributed in its entirety, as provided by NVIDIA and without separation of its components, for you and/or your licensees to create software development kits for use only on the Jetson platform and running Linux for Tegra software. You shall require in your agreements with your licensees that their distributions be on terms at least as restrictive as those applicable for your distribution of TensorRT Licensed Software as described in this Section 1. + +In addition to the rights above, for parties that are developing software intended solely for use on Jetson development kits or Jetson modules and running Linux for Tegra software the following shall apply: TensorRT Licensed Software licensed hereunder may be distributed in its entirety, as provided by NVIDIA and without separation of its components, for you and/or your licensees to create software development kits for use only on the Jetson platform and running Linux for Tegra software. You shall require in your agreements with your licensees that their distributions be on terms at least as restrictive as those applicable for your distribution of TensorRT Licensed Software as described in this Section 1. + +12.2. LICENSE DURATION +Each TensorRT Licensed Software is licensed to you for an initial duration of one year starting from the date of delivery or download. The licenses granted will automatically renew for successive one year periods, provided that NVIDIA reserves the right to terminate licenses upon ninety days (90) days written notice to you prior to the commencement of a renewal year in addition to the termination rights set forth in the SLA. + +12.3. EXPIRATION OF TERMINATION OF THIS SUPPLEMENT +Your failure to comply with the terms of this Supplement is ground for termination for breach by NVIDIA under the SLA. This Supplement will automatically expire or terminate upon the expiration or termination of your rights to TensorRT Licensed Software under the SLA or this Supplement. + +Notices +Notice +This document is provided for information purposes only and shall not be regarded as a warranty of a certain functionality, condition, or quality of a product. NVIDIA Corporation (“NVIDIA”) makes no representations or warranties, expressed or implied, as to the accuracy or completeness of the information contained in this document and assumes no responsibility for any errors contained herein. NVIDIA shall have no liability for the consequences or use of such information or for any infringement of patents or other rights of third parties that may result from its use. This document is not a commitment to develop, release, or deliver any Material (defined below), code, or functionality. + +NVIDIA reserves the right to make corrections, modifications, enhancements, improvements, and any other changes to this document, at any time without notice. + +Customer should obtain the latest relevant information before placing orders and should verify that such information is current and complete. + +NVIDIA products are sold subject to the NVIDIA standard terms and conditions of sale supplied at the time of order acknowledgement, unless otherwise agreed in an individual sales agreement signed by authorized representatives of NVIDIA and customer (“Terms of Sale”). NVIDIA hereby expressly objects to applying any customer general terms and conditions with regards to the purchase of the NVIDIA product referenced in this document. No contractual obligations are formed either directly or indirectly by this document. + +NVIDIA products are not designed, authorized, or warranted to be suitable for use in medical, military, aircraft, space, or life support equipment, nor in applications where failure or malfunction of the NVIDIA product can reasonably be expected to result in personal injury, death, or property or environmental damage. NVIDIA accepts no liability for inclusion and/or use of NVIDIA products in such equipment or applications and therefore such inclusion and/or use is at customer’s own risk. + +NVIDIA makes no representation or warranty that products based on this document will be suitable for any specified use. Testing of all parameters of each product is not necessarily performed by NVIDIA. It is customer’s sole responsibility to evaluate and determine the applicability of any information contained in this document, ensure the product is suitable and fit for the application planned by customer, and perform the necessary testing for the application in order to avoid a default of the application or the product. Weaknesses in customer’s product designs may affect the quality and reliability of the NVIDIA product and may result in additional or different conditions and/or requirements beyond those contained in this document. NVIDIA accepts no liability related to any default, damage, costs, or problem which may be based on or attributable to: (i) the use of the NVIDIA product in any manner that is contrary to this document or (ii) customer product designs. + +No license, either expressed or implied, is granted under any NVIDIA patent right, copyright, or other NVIDIA intellectual property right under this document. Information published by NVIDIA regarding third-party products or services does not constitute a license from NVIDIA to use such products or services or a warranty or endorsement thereof. Use of such information may require a license from a third party under the patents or other intellectual property rights of the third party, or a license from NVIDIA under the patents or other intellectual property rights of NVIDIA. + +Reproduction of information in this document is permissible only if approved in advance by NVIDIA in writing, reproduced without alteration and in full compliance with all applicable export laws and regulations, and accompanied by all associated conditions, limitations, and notices. + +THIS DOCUMENT AND ALL NVIDIA DESIGN SPECIFICATIONS, REFERENCE BOARDS, FILES, DRAWINGS, DIAGNOSTICS, LISTS, AND OTHER DOCUMENTS (TOGETHER AND SEPARATELY, “MATERIALS”) ARE BEING PROVIDED “AS IS.” NVIDIA MAKES NO WARRANTIES, EXPRESSED, IMPLIED, STATUTORY, OR OTHERWISE WITH RESPECT TO THE MATERIALS, AND EXPRESSLY DISCLAIMS ALL IMPLIED WARRANTIES OF NONINFRINGEMENT, MERCHANTABILITY, AND FITNESS FOR A PARTICULAR PURPOSE. TO THE EXTENT NOT PROHIBITED BY LAW, IN NO EVENT WILL NVIDIA BE LIABLE FOR ANY DAMAGES, INCLUDING WITHOUT LIMITATION ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, PUNITIVE, OR CONSEQUENTIAL DAMAGES, HOWEVER CAUSED AND REGARDLESS OF THE THEORY OF LIABILITY, ARISING OUT OF ANY USE OF THIS DOCUMENT, EVEN IF NVIDIA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Notwithstanding any damages that customer might incur for any reason whatsoever, NVIDIA’s aggregate and cumulative liability towards customer for the products described herein shall be limited in accordance with the Terms of Sale for the product. + +VESA DisplayPort +DisplayPort and DisplayPort Compliance Logo, DisplayPort Compliance Logo for Dual-mode Sources, and DisplayPort Compliance Logo for Active Cables are trademarks owned by the Video Electronics Standards Association in the United States and other countries. + +HDMI +HDMI, the HDMI logo, and High-Definition Multimedia Interface are trademarks or registered trademarks of HDMI Licensing LLC. + +ARM +ARM, AMBA and ARM Powered are registered trademarks of ARM Limited. Cortex, MPCore and Mali are trademarks of ARM Limited. All other brands or product names are the property of their respective holders. "ARM" is used to represent ARM Holdings plc; its operating company ARM Limited; and the regional subsidiaries ARM Inc.; ARM KK; ARM Korea Limited.; ARM Taiwan Limited; ARM France SAS; ARM Consulting (Shanghai) Co. Ltd.; ARM Germany GmbH; ARM Embedded Technologies Pvt. Ltd.; ARM Norway, AS and ARM Sweden AB. + +OpenCL +OpenCL is a trademark of Apple Inc. used under license to the Khronos Group Inc. + +Trademarks +NVIDIA, the NVIDIA logo, and cuBLAS, CUDA, CUDA Toolkit, cuDNN, DALI, DIGITS, DGX, DGX-1, DGX-2, DGX Station, DLProf, GPU, JetPack, Jetson, Kepler, Maxwell, NCCL, Nsight Compute, Nsight Systems, NVCaffe, NVIDIA Ampere GPU architecture, NVIDIA Deep Learning SDK, NVIDIA Developer Program, NVIDIA GPU Cloud, NVLink, NVSHMEM, PerfWorks, Pascal, SDK Manager, T4, Tegra, TensorRT, TensorRT Inference Server, Tesla, TF-TRT, Triton Inference Server, Turing, and Volta are trademarks and/or registered trademarks of NVIDIA Corporation in the United States and other countries. Other company and product names may be trademarks of the respective companies with which they are associated. + +Copyright +© 2021 NVIDIA Corporation. All rights reserved. diff --git a/python/packaging/libs_wheel/setup.cfg b/python/packaging/libs_wheel/setup.cfg new file mode 100644 index 00000000..32a8c1c0 --- /dev/null +++ b/python/packaging/libs_wheel/setup.cfg @@ -0,0 +1,5 @@ +[metadata] +license_files = LICENSE.txt + +[bdist_wheel] +universal = 1 diff --git a/python/packaging/setup.py b/python/packaging/libs_wheel/setup.py similarity index 59% rename from python/packaging/setup.py rename to python/packaging/libs_wheel/setup.py index d73d9fd7..eb45a258 100644 --- a/python/packaging/setup.py +++ b/python/packaging/libs_wheel/setup.py @@ -15,20 +15,11 @@ # limitations under the License. # -try: - from setuptools import setup -except ImportError: - from distutils.core import setup import os -tensorrt_module = "##TENSORRT_MODULE##" +from setuptools import setup -def is_standalone(): - return os.environ.get("STANDALONE") == "1" - - -def is_dla(): - return os.environ.get("ENABLE_DLA") == "1" +module_name = "##TENSORRT_MODULE##_libs" def get_requirements(): @@ -41,27 +32,20 @@ def get_vers(var): cuda_major, _ = get_vers("CUDA") return "-cu{cuda_major}".format(cuda_major=cuda_major) - if is_standalone(): - reqs = [ "nvidia-cuda-runtime" + get_version_range() ] - if tensorrt_module == "tensorrt": - reqs += [ - "nvidia-cudnn" + get_version_range(), - "nvidia-cublas" + get_version_range(), - ] - return reqs - return [] - + reqs = ["nvidia-cuda-runtime" + get_version_range()] + if "##TENSORRT_MODULE##" == "tensorrt": + reqs += [ + "nvidia-cudnn" + get_version_range(), + "nvidia-cublas" + get_version_range(), + ] + return reqs -name = tensorrt_module -# Only standalone wheels need to be disambiguated. Otherwise, the entire tar/deb/rpm is DLA/non-DLA. -if is_standalone() and is_dla(): - name += "-dla" setup( - name=name, + name=module_name, version="##TENSORRT_PYTHON_VERSION##", - description="A high performance deep learning inference library", - long_description="A high performance deep learning inference library", + description="TensorRT Libraries", + long_description="TensorRT Libraries", author="NVIDIA Corporation", license="Proprietary", classifiers=[ @@ -69,10 +53,9 @@ def get_vers(var): "Intended Audience :: Developers", "Programming Language :: Python :: 3", ], - packages=[tensorrt_module], + packages=[module_name], install_requires=get_requirements(), - extras_require={"numpy": "numpy"}, - package_data={tensorrt_module: ["*.so*", "*.pyd", "*.pdb"]}, + package_data={module_name: ["*.so*", "*.pyd", "*.pdb"]}, include_package_data=True, zip_safe=True, keywords="nvidia tensorrt deeplearning inference", diff --git a/python/packaging/libs_wheel/tensorrt_libs/__init__.py b/python/packaging/libs_wheel/tensorrt_libs/__init__.py new file mode 100644 index 00000000..83df0448 --- /dev/null +++ b/python/packaging/libs_wheel/tensorrt_libs/__init__.py @@ -0,0 +1,33 @@ +# +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import ctypes +import glob +import os + + +def try_load(library): + try: + ctypes.CDLL(library) + except OSError: + pass + + +# Try loading all packaged libraries. This is a nop if there are no libraries packaged. +CURDIR = os.path.realpath(os.path.dirname(__file__)) +for lib in glob.iglob(os.path.join(CURDIR, "*.so*")): + try_load(lib) diff --git a/python/requirements.txt b/python/requirements.txt index 3c9b272d..43663c1a 100644 --- a/python/requirements.txt +++ b/python/requirements.txt @@ -5,6 +5,6 @@ numpy==1.19.4; python_version < "3.8" and platform_system != "Windows" numpy==1.23.0; python_version >= "3.8" and python_version < "3.10" numpy==1.23.1; python_version >= "3.10" Pillow; python_version<"3.6" -##PYTHON_BUILDDIR##/tensorrt-py3.##PYTHON3_MINOR##/dist/tensorrt-##TENSORRT_PYTHON_VERSION##-cp3##PYTHON3_MINOR##-none-linux_##TARGET##.whl ; python_version=="3.##PYTHON3_MINOR##" +##PYTHON_BUILDDIR##/tensorrt_bindings-py3.##PYTHON3_MINOR##/dist/tensorrt-##TENSORRT_PYTHON_VERSION##-cp3##PYTHON3_MINOR##-none-linux_##TARGET##.whl ; python_version=="3.##PYTHON3_MINOR##" ##TENSORRT_ROOT##/python_builds/uff/uff-##UFF_VERSION##-py2.py3-none-any.whl ##TENSORRT_ROOT##/python_builds/graphsurgeon/graphsurgeon-##GRAPHSURGEON_VERSION##-py2.py3-none-any.whl diff --git a/python/src/infer/pyCore.cpp b/python/src/infer/pyCore.cpp index cca5504c..555668e3 100644 --- a/python/src/infer/pyCore.cpp +++ b/python/src/infer/pyCore.cpp @@ -966,15 +966,73 @@ void bindCore(py::module& m) .def("get_tensor_shape", &ICudaEngine::getTensorShape, "name"_a, ICudaEngineDoc::get_tensor_shape) .def("get_tensor_dtype", &ICudaEngine::getTensorDataType, "name"_a, ICudaEngineDoc::get_tensor_dtype) .def("get_tensor_location", &ICudaEngine::getTensorLocation, "name"_a, ICudaEngineDoc::get_tensor_location) - .def("get_tensor_bytes_per_component", &ICudaEngine::getTensorBytesPerComponent, "name"_a, - ICudaEngineDoc::get_tensor_bytes_per_component) - .def("get_tensor_components_per_element", &ICudaEngine::getTensorComponentsPerElement, "name"_a, - ICudaEngineDoc::get_tensor_components_per_element) - .def("get_tensor_format", &ICudaEngine::getTensorFormat, "name"_a, ICudaEngineDoc::get_tensor_format) - .def("get_tensor_format_desc", &ICudaEngine::getTensorFormatDesc, "name"_a, - ICudaEngineDoc::get_tensor_format_desc) - .def("get_tensor_vectorized_dim", &ICudaEngine::getTensorVectorizedDim, "name"_a, - ICudaEngineDoc::get_tensor_vectorized_dim) + + .def( + "get_tensor_bytes_per_component", + [](ICudaEngine& self, std::string const& name) -> int32_t { + return self.getTensorBytesPerComponent(name.c_str()); + }, + "name"_a, ICudaEngineDoc::get_tensor_bytes_per_component) + .def( + "get_tensor_bytes_per_component", + [](ICudaEngine& self, std::string const& name, int32_t profileIndex) -> int32_t { + return self.getTensorBytesPerComponent(name.c_str(), profileIndex); + }, + "name"_a, "profile_index"_a, ICudaEngineDoc::get_tensor_bytes_per_component) + + .def( + "get_tensor_components_per_element", + [](ICudaEngine& self, std::string const& name) -> int32_t { + return self.getTensorComponentsPerElement(name.c_str()); + }, + "name"_a, ICudaEngineDoc::get_tensor_components_per_element) + .def( + "get_tensor_components_per_element", + [](ICudaEngine& self, std::string const& name, int32_t profileIndex) -> int32_t { + return self.getTensorComponentsPerElement(name.c_str(), profileIndex); + }, + "name"_a, "profile_index"_a, ICudaEngineDoc::get_tensor_components_per_element) + + .def( + "get_tensor_format", + [](ICudaEngine& self, std::string const& name) -> TensorFormat { + return self.getTensorFormat(name.c_str()); + }, + "name"_a, ICudaEngineDoc::get_tensor_format) + + .def( + "get_tensor_format", + [](ICudaEngine& self, std::string const& name, int32_t profileIndex) -> TensorFormat { + return self.getTensorFormat(name.c_str(), profileIndex); + }, + "name"_a, "profile_index"_a, ICudaEngineDoc::get_tensor_format) + + .def( + "get_tensor_format_desc", + [](ICudaEngine& self, std::string const& name) -> const char* { + return self.getTensorFormatDesc(name.c_str()); + }, + "name"_a, ICudaEngineDoc::get_tensor_format_desc) + .def( + "get_tensor_format_desc", + [](ICudaEngine& self, std::string const& name, int32_t profileIndex) -> const char* { + return self.getTensorFormatDesc(name.c_str(), profileIndex); + }, + "name"_a, "profile_index"_a, ICudaEngineDoc::get_tensor_format_desc) + + .def( + "get_tensor_vectorized_dim", + [](ICudaEngine& self, std::string const& name) -> int32_t { + return self.getTensorVectorizedDim(name.c_str()); + }, + "name"_a, ICudaEngineDoc::get_tensor_vectorized_dim) + .def( + "get_tensor_vectorized_dim", + [](ICudaEngine& self, std::string const& name, int32_t profileIndex) -> int32_t { + return self.getTensorVectorizedDim(name.c_str(), profileIndex); + }, + "name"_a, "profile_index"_a, ICudaEngineDoc::get_tensor_vectorized_dim) + .def("get_tensor_profile_shape", lambdas::get_tensor_profile_shape, "name"_a, "profile_index"_a, ICudaEngineDoc::get_tensor_profile_shape) // End of enqueueV3 related APIs. @@ -982,8 +1040,8 @@ void bindCore(py::module& m) py::cpp_function(&ICudaEngine::setErrorRecorder, py::keep_alive<1, 2>{})) .def_property_readonly("tactic_sources", &ICudaEngine::getTacticSources) .def_property_readonly("profiling_verbosity", &ICudaEngine::getProfilingVerbosity) - .def("create_engine_inspector", - &ICudaEngine::createEngineInspector, ICudaEngineDoc::create_engine_inspector, py::keep_alive<0, 1>{}) + .def("create_engine_inspector", &ICudaEngine::createEngineInspector, ICudaEngineDoc::create_engine_inspector, + py::keep_alive<0, 1>{}) .def_property_readonly("hardware_compatibility_level", &ICudaEngine::getHardwareCompatibilityLevel) .def_property_readonly("num_aux_streams", &ICudaEngine::getNbAuxStreams) .def("__del__", &utils::doNothingDel); diff --git a/python/src/infer/pyGraph.cpp b/python/src/infer/pyGraph.cpp index ed355fc5..ae66ed12 100644 --- a/python/src/infer/pyGraph.cpp +++ b/python/src/infer/pyGraph.cpp @@ -867,141 +867,113 @@ namespace tensorrt .def("mark_output", &INetworkDefinition::markOutput, "tensor"_a, INetworkDefinitionDoc::mark_output) // Layers .def("add_input", &INetworkDefinition::addInput, "name"_a, "dtype"_a, "shape"_a, - INetworkDefinitionDoc::add_input, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_input, py::return_value_policy::reference_internal) .def("add_convolution", utils::deprecate(lambdas::add_convolution, "add_convolution_nd"), "input"_a, "num_output_maps"_a, "kernel_shape"_a, "kernel"_a, "bias"_a=nullptr, py::keep_alive<1, 5>{}, py::keep_alive<1, 6>{}, INetworkDefinitionDoc::add_convolution, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("add_convolution_nd", lambdas::add_convolution_nd, "input"_a, "num_output_maps"_a, "kernel_shape"_a, "kernel"_a, "bias"_a=nullptr, py::keep_alive<1, 5>{}, py::keep_alive<1, 6>{}, - INetworkDefinitionDoc::add_convolution_nd, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_convolution_nd, py::return_value_policy::reference_internal) .def("add_fully_connected", utils::deprecate(lambdas::add_fully_connected, "add_matrix_multiply"), "input"_a, "num_outputs"_a, "kernel"_a, "bias"_a=nullptr, py::keep_alive<1, 4>{}, py::keep_alive<1, 5>{}, INetworkDefinitionDoc::add_fully_connected, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("add_activation", &INetworkDefinition::addActivation, "input"_a, "type"_a, - INetworkDefinitionDoc::add_activation, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_activation, py::return_value_policy::reference_internal) .def("add_pooling", utils::deprecateMember(&INetworkDefinition::addPooling, "add_pooling_nd"), "input"_a, "type"_a, "window_size"_a, - INetworkDefinitionDoc::add_pooling, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_pooling, py::return_value_policy::reference_internal) .def("add_pooling_nd", &INetworkDefinition::addPoolingNd, "input"_a, "type"_a, "window_size"_a, - INetworkDefinitionDoc::add_pooling_nd, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_pooling_nd, py::return_value_policy::reference_internal) .def("add_lrn", &INetworkDefinition::addLRN, "input"_a, "window"_a, "alpha"_a, "beta"_a, "k"_a, - INetworkDefinitionDoc::add_lrn, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_lrn, py::return_value_policy::reference_internal) .def("add_scale", lambdas::add_scale, "input"_a, "mode"_a, "shift"_a=nullptr, "scale"_a=nullptr, "power"_a=nullptr, py::keep_alive<1, 4>{}, py::keep_alive<1, 5>{}, py::keep_alive<1, 6>{}, INetworkDefinitionDoc::add_scale, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("add_scale_nd", lambdas::add_scale_nd, "input"_a, "mode"_a, "shift"_a=nullptr, "scale"_a=nullptr, "power"_a=nullptr, "channel_axis"_a, py::keep_alive<1, 4>{}, py::keep_alive<1, 5>{}, py::keep_alive<1, 6>{}, INetworkDefinitionDoc::add_scale_nd, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("add_softmax", &INetworkDefinition::addSoftMax, "input"_a, INetworkDefinitionDoc::add_softmax, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("add_concatenation", lambdas::add_concatenation, "inputs"_a, INetworkDefinitionDoc::add_concatenation, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("add_deconvolution", utils::deprecate(lambdas::add_deconvolution, "add_deconvolution_nd"), "input"_a, "num_output_maps"_a, "kernel_shape"_a, "kernel"_a, "bias"_a=nullptr, py::keep_alive<1, 5>{}, py::keep_alive<1, 6>{}, - INetworkDefinitionDoc::add_deconvolution, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_deconvolution, py::return_value_policy::reference_internal) .def("add_deconvolution_nd", lambdas::add_deconvolution_nd, "input"_a, "num_output_maps"_a, "kernel_shape"_a, "kernel"_a, "bias"_a=nullptr, py::keep_alive<1, 5>{}, py::keep_alive<1, 6>{}, - INetworkDefinitionDoc::add_deconvolution_nd, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_deconvolution_nd, py::return_value_policy::reference_internal) .def("add_elementwise", &INetworkDefinition::addElementWise, "input1"_a, "input2"_a, "op"_a, - INetworkDefinitionDoc::add_elementwise, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_elementwise, py::return_value_policy::reference_internal) .def("add_unary", &INetworkDefinition::addUnary, "input"_a, "op"_a, INetworkDefinitionDoc::add_unary, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("add_padding", utils::deprecateMember(&INetworkDefinition::addPadding, "add_padding_nd"), "input"_a, "pre_padding"_a, "post_padding"_a, - INetworkDefinitionDoc::add_padding, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_padding, py::return_value_policy::reference_internal) .def("add_padding_nd", &INetworkDefinition::addPaddingNd, "input"_a, "pre_padding"_a, "post_padding"_a, - INetworkDefinitionDoc::add_padding_nd, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_padding_nd, py::return_value_policy::reference_internal) .def("add_shuffle", &INetworkDefinition::addShuffle, "input"_a, INetworkDefinitionDoc::add_shuffle, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("add_slice", &INetworkDefinition::addSlice, "input"_a, "start"_a, "shape"_a, "stride"_a, - INetworkDefinitionDoc::add_slice, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_slice, py::return_value_policy::reference_internal) .def("add_reduce", &INetworkDefinition::addReduce, "input"_a, "op"_a, "axes"_a, "keep_dims"_a, - INetworkDefinitionDoc::add_reduce, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_reduce, py::return_value_policy::reference_internal) .def("add_topk", &INetworkDefinition::addTopK, "input"_a, "op"_a, "k"_a, "axes"_a, - INetworkDefinitionDoc::add_topk, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_topk, py::return_value_policy::reference_internal) .def("add_gather", &INetworkDefinition::addGather, "input"_a, "indices"_a, "axis"_a, - INetworkDefinitionDoc::add_gather, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_gather, py::return_value_policy::reference_internal) .def("add_scatter", &INetworkDefinition::addScatter, "data"_a, "indices"_a, "updates"_a, "mode"_a, - INetworkDefinitionDoc::add_scatter, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_scatter, py::return_value_policy::reference_internal) .def("add_gather_v2", &INetworkDefinition::addGatherV2, "input"_a, "indices"_a, "mode"_a, - INetworkDefinitionDoc::add_gather_v2, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_gather_v2, py::return_value_policy::reference_internal) .def("add_ragged_softmax", &INetworkDefinition::addRaggedSoftMax, "input"_a, "bounds"_a, - INetworkDefinitionDoc::add_ragged_softmax, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_ragged_softmax, py::return_value_policy::reference_internal) .def("add_matrix_multiply", static_cast(&INetworkDefinition::addMatrixMultiply), "input0"_a, "op0"_a, "input1"_a, "op1"_a, INetworkDefinitionDoc::add_matrix_multiply, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("add_constant", &INetworkDefinition::addConstant, "shape"_a, "weights"_a, py::keep_alive<1, 3>{}, INetworkDefinitionDoc::add_constant, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("add_rnn_v2", utils::deprecateMember(&INetworkDefinition::addRNNv2, "addLoop"), "input"_a, "layer_count"_a, - "hidden_size"_a, "max_seq_length"_a, "op"_a, - py::keep_alive<1, 0>{}, INetworkDefinitionDoc::add_rnn_v2) + "hidden_size"_a, "max_seq_length"_a, "op"_a, INetworkDefinitionDoc::add_rnn_v2) .def("add_identity", &INetworkDefinition::addIdentity, "input"_a, - INetworkDefinitionDoc::add_identity, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_identity, py::return_value_policy::reference_internal) .def("add_cast", &INetworkDefinition::addCast, "input"_a, "to_type"_a, - INetworkDefinitionDoc::add_cast, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_cast, py::return_value_policy::reference_internal) .def("add_plugin_v2", lambdas::add_plugin_v2, "inputs"_a, "plugin"_a, - INetworkDefinitionDoc::add_plugin_v2, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_plugin_v2, py::return_value_policy::reference_internal) .def("add_parametric_relu", &INetworkDefinition::addParametricReLU, "input"_a, - "slopes"_a, INetworkDefinitionDoc::add_parametric_relu, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + "slopes"_a, INetworkDefinitionDoc::add_parametric_relu, py::return_value_policy::reference_internal) .def("add_resize", &INetworkDefinition::addResize, "input"_a, INetworkDefinitionDoc::add_resize, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("add_loop", &INetworkDefinition::addLoop, INetworkDefinitionDoc::add_loop, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("add_shape", &INetworkDefinition::addShape, "input"_a, INetworkDefinitionDoc::add_shape, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("add_select", &INetworkDefinition::addSelect, "condition"_a, "then_input"_a, - "else_input"_a, INetworkDefinitionDoc::add_select, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + "else_input"_a, INetworkDefinitionDoc::add_select, py::return_value_policy::reference_internal) .def("add_assertion", &INetworkDefinition::addAssertion, "condition"_a, "message"_a, INetworkDefinitionDoc::add_assertion, INetworkDefinitionDoc::add_assertion, py::return_value_policy::reference_internal) .def("add_grid_sample", &INetworkDefinition::addGridSample, "input"_a, "grid"_a, INetworkDefinitionDoc::add_grid_sample, py::return_value_policy::reference_internal) .def("add_nms", &INetworkDefinition::addNMS, "boxes"_a, - "scores"_a, "max_output_boxes_per_class"_a, INetworkDefinitionDoc::add_nms, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + "scores"_a, "max_output_boxes_per_class"_a, INetworkDefinitionDoc::add_nms, py::return_value_policy::reference_internal) .def("add_fill", &INetworkDefinition::addFill, "shape"_a, "op"_a, INetworkDefinitionDoc::add_fill) .def("add_quantize", &INetworkDefinition::addQuantize, "input"_a, "scale"_a, - INetworkDefinitionDoc::add_quantize, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_quantize, py::return_value_policy::reference_internal) .def("add_dequantize", &INetworkDefinition::addDequantize, "input"_a, "scale"_a, - INetworkDefinitionDoc::add_dequantize, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_dequantize, py::return_value_policy::reference_internal) .def("add_if_conditional", &INetworkDefinition::addIfConditional, INetworkDefinitionDoc::add_if_conditional, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("add_einsum", lambdas::add_einsum, "inputs"_a, "equation"_a, INetworkDefinitionDoc::add_einsum, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("add_one_hot", &INetworkDefinition::addOneHot, "indices"_a, "values"_a, "depth"_a, "axis"_a, - INetworkDefinitionDoc::add_one_hot, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + INetworkDefinitionDoc::add_one_hot, py::return_value_policy::reference_internal) .def("add_non_zero", &INetworkDefinition::addNonZero, "input"_a, INetworkDefinitionDoc::add_non_zero, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("add_reverse_sequence", &INetworkDefinition::addReverseSequence, "input"_a, "sequence_lens"_a, INetworkDefinitionDoc::add_reverse_sequence, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("add_normalization", &INetworkDefinition::addNormalization, "input"_a, "scale"_a, "bias"_a, "axesMask"_a, INetworkDefinitionDoc::add_normalization, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("remove_tensor", &INetworkDefinition::removeTensor, "tensor"_a, INetworkDefinitionDoc::remove_tensor) .def("unmark_output", &INetworkDefinition::unmarkOutput, "tensor"_a, INetworkDefinitionDoc::unmark_output) .def("mark_output_for_shapes", &INetworkDefinition::markOutputForShapes, "tensor"_a, INetworkDefinitionDoc::mark_output_for_shapes) @@ -1009,15 +981,15 @@ namespace tensorrt .def("set_weights_name", &INetworkDefinition::setWeightsName, "weights"_a, "name"_a, INetworkDefinitionDoc::set_weights_name) // Getters .def("get_layer", &INetworkDefinition::getLayer, "index"_a, INetworkDefinitionDoc::get_layer, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("get_input", &INetworkDefinition::getInput, "index"_a, INetworkDefinitionDoc::get_input, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("get_output", &INetworkDefinition::getOutput, "index"_a, INetworkDefinitionDoc::get_output, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) // Note: the builder is the _parent_ of the INetworkDefinition, so a reference_internal policy (which would // keep the INetworkDefinition alive while the builder is referenced) is unnecessary here. .def_property_readonly("builder", &INetworkDefinition::getBuilder, INetworkDefinitionDoc::builder, - py::keep_alive<1, 0>{}, py::return_value_policy::reference) + py::return_value_policy::reference) #if ENABLE_INETWORK_SERIALIZE // Serialization @@ -1026,7 +998,7 @@ namespace tensorrt // Allow iteration over the layers of a network .def("__len__", &INetworkDefinition::getNbLayers) .def("__getitem__", lambdas::network_getitem, py::return_value_policy::reference_internal, - py::keep_alive<1, 0>{}, py::return_value_policy::reference_internal) + py::return_value_policy::reference_internal) .def("__del__", &utils::doNothingDel) ; diff --git a/python/src/parsers/pyOnnx.cpp b/python/src/parsers/pyOnnx.cpp index 57887e87..ec9bc0c8 100644 --- a/python/src/parsers/pyOnnx.cpp +++ b/python/src/parsers/pyOnnx.cpp @@ -17,7 +17,7 @@ // Implementation of PyBind11 Binding Code for OnnxParser #include "ForwardDeclarations.h" -#include "onnxOpenSource/NvOnnxParser.h" +#include "onnx/NvOnnxParser.h" #include "parsers/pyOnnxDoc.h" #include "utils.h" #include @@ -98,7 +98,7 @@ void bindOnnx(py::module& m) py::class_(m, "OnnxParser", OnnxParserDoc::descr, py::module_local()) .def(py::init(&nvonnxparser::createParser), "network"_a, "logger"_a, OnnxParserDoc::init, - py::keep_alive<1, 2>{}, py::keep_alive<1, 3>{}, py::keep_alive<2, 1>{}) + py::keep_alive<1, 3>{}, py::keep_alive<2, 1>{}) .def("parse", lambdas::parse, "model"_a, "path"_a = nullptr, OnnxParserDoc::parse, py::call_guard{}) .def("parse_with_weight_descriptors", lambdas::parse_with_weight_descriptors, "model"_a, @@ -119,7 +119,7 @@ void bindOnnx(py::module& m) .def("__del__", &utils::doNothingDel); py::enum_(m, "OnnxParserFlag", OnnxParserFlagDoc::descr, py::module_local()) - .value("VERSION_COMPATIBLE", OnnxParserFlag::kVERSION_COMPATIBLE, OnnxParserFlagDoc::VERSION_COMPATIBLE); + .value("NATIVE_INSTANCENORM", OnnxParserFlag::kNATIVE_INSTANCENORM, OnnxParserFlagDoc::NATIVE_INSTANCENORM); py::enum_(m, "ErrorCode", ErrorCodeDoc::descr, py::module_local()) .value("SUCCESS", ErrorCode::kSUCCESS) diff --git a/quickstart/IntroNotebooks/3. Using Tensorflow 2 through ONNX.ipynb b/quickstart/IntroNotebooks/3. Using Tensorflow 2 through ONNX.ipynb index 4bc794e6..aa8f6328 100644 --- a/quickstart/IntroNotebooks/3. Using Tensorflow 2 through ONNX.ipynb +++ b/quickstart/IntroNotebooks/3. Using Tensorflow 2 through ONNX.ipynb @@ -8,7 +8,7 @@ "\n", "The ONNX path to getting a TensorRT engine is a high-performance approach to TensorRT conversion that works with a variety of frameworks - including Tensorflow and Tensorflow 2.\n", "\n", - "TensorRT's ONNX parser is an all-or-nothing parser for ONNX models that ensures an optimal, single TensorRT engine and is great for exporting to the TensorRT API runtimes. ONNX models can be easily generated from Tensorflow models using the ONNX project's keras2onnx and tf2onnx tools.\n", + "TensorRT's ONNX parser is an all-or-nothing parser for ONNX models that ensures an optimal, single TensorRT engine and is great for exporting to the TensorRT API runtimes. ONNX models can be easily generated from Tensorflow models using the ONNX project's tf2onnx tool.\n", "\n", "In this notebook we will take a look at how ONNX models can be generated from a Keras/TF2 ResNet50 model, how we can convert those ONNX models to TensorRT engines using trtexec, and finally how we can use the native Python TensorRT runtime to feed a batch of data into the TRT engine at inference time.\n", "\n", @@ -196,7 +196,7 @@ }, { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD8CAYAAAB3lxGOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAEAAElEQVR4nOz9WYxtWZrfh/3WsMczxhx3zLmGrDG7qrrJbplkAyRNGjYEw4ApGbAeDFh+0YMBP5jQkwG9+MED/GSbhgDDpiHLgA3TosmmRLYotbrZVd1dVV2VWZmV053vjenEGfe4Jj+sfaOachVFil1gAp0LCOSNyIgz7LPXt77hP4gQAp+vz9fn68/ukv+qX8Dn6/P1+fpXuz4PAp+vz9ef8fV5EPh8fb7+jK/Pg8Dn6/P1Z3x9HgQ+X5+vP+Pr8yDw+fp8/Rlfv7QgIIT4a0KInwohPhZC/M1f1vN8vj5fn69/uSV+GTgBIYQCPgT+CvAU+APg3wwh/ORP/ck+X5+vz9e/1PplZQK/CnwcQvg0hNAD/3fgX/8lPdfn6/P1+fqXWPqX9Lh3gCd/4vunwK/9whehZdAKRmWJQGCsxXmPdQ6pFEIIgg+IAAiBDx7nLFIKhAQlJd57QACCEEAKifcBrRTee0IIeO9IkoQ0TbHW0vcdUim8d2RphkQSBHSmx4f4s+A94uZxA23XgxBkWYIUAu88goB3HqUUWmt6a3DOgRAIpSCAcx6tFEJACJ4sTTHGYKxFaY1Sir7vQQSyLKVrW7RW6ESSppKubzk8PEDLlIuLa0wHzkGaAMqQFwLnJW0NSTLC+x7T10gJwidIkRNEwPke7+N1lUqQpglCCJqmxXlPlmWMRmPq3Y6u7ZBS4kMABEKI4RMTKCWRQmKtRWuFsQYpBEWRIxAIqTC9wRgDAtJUgQg4Z/E+oJSOn5EL8b13PW74fJx38fMUkKYpQgicdTjrIATKskBKycHBQXxu03F1dYX3EucC1nmss4QQyPOc/YN9lstr2rZhOh2T5xnT6ZjO1iyuV9R1T1nkCCHpe0vwYE18vuADPgSElGRZRp6nOG+x1mKtwYeA9wEhBePxlKZpQARCCAQf4jVyDq00UsQ7SSrJ4eEBm/WaqqoBCYKfXV8hEAK8j1m6khIhBNYN1wWQUkCAsizoe0PXdvF3dUKRF3RtS6olSivatmd/f48nz86uQghH/3/777/2Nv+XXEKIfxv4twGSRPL2lw956+5rSCRt33N2vWCx3SLzjFxn7E9muLbH+kBrGozt2FVrxpOE/f05zjnW6x2CBCkypEjoq44izciyjLquqOsdt26f8s473+C9n7xL27ZcXl6ytzfj3u27CCPopWexvaYyNV98403q9Q7XGvrOYoLk6fklJIq7p/uYpqGrGgqdkkrFq6+8wmgy5ve++z0u1tdMDvdIixxjFM22ZX9vSpYqunrH7dunXFyec7VYsHd4iFSa88tLmn7Dt7/zOrazNE3D3XuHCLljPJX8jb/x3+PO6Rv81n/0Pf72//m36OuEu7cyiv1r5rccXZ+jxBd58NGGansG/gw6Q2JuM8lfxRc9Vb/gen1FMcoYTQoOjw+w3nF+eUmaZnzhS1/i3t37/Od//z/h4uKSICRKpSR5wWa75Vfe+TYEuHjxIm5K50lzjRMd1nbcv3uHu6d3cUby4JOn7KqK+f6E6X7C+eIBL86eE1zCbLoHTjIppuRJydnZOVVdcXh8iA2GXbPD+Rggx+WE2XhKrlNs0/Hnv/Md7pwec+v2EUfHB/znv/vbnF+84N33zrlcVBjnWW13bOuKk1un3L5zTFXP+erXvsB/86/+JfbmI14sHrD1T7m82mD6hO3G8v57D7m+rLg4W7M8X9JsHUVaoPOSq/WWv/7X/zL37p/w6OEHPH76gOvVgqazzA/22DYdt+/e5/LqmnysSXSCaQzbTU2iUn7zL/wmP/7hD1lfX/HX/tpfIUkS/v2/9bd55dUJzms8HkfAhYAQgiRJ0EqRKI0Wkq5paawlLUqc62mbHWVZ8tf+6l/h+3/0Qz756FMSXXByfIff+I2/wMcfvIdZn/Ptb3+Lf/Ld32cym/Hk2dmjn7cXf1lB4Blw7098f3f42c0KIfwt4G8B7B9Mw2/8+r+GtoHHD5/QNA3b7Tae1FKgEMzmE84ePccLwd7ePuUo5ZNPP8B7S9d17O3tEYJkvaoQMtycKt578jzHmJ48z9Fa8+DBQ6RQOOfI85x79+4RbMD0PctmS+d6qrphs92SCInSGlu1eBSj0QiZJYggcMaT6ZRUJ3hjaaqGqm7IsowvfenL1LajMQYhwDpDCAEBWBs3uDUOrRK0VGyrit4YkkQQfEOWFbx4vo0X7+4xtgt8+JMz/sP/63/M9//oQ3aVQjMiyyd0bUpf5ayrlm98/RbOaN770VNm49usqzVajtGpYtc3VHVF27ToRNA2grMXZ7zy2qscH8HZ+SWLqwVlMSZJUpx3SCVwePp6S5InHJ0ccnVxhfEOEQI+eDbVhje+fJe+azi/fEaWJOzNjijKFKUlOlFsthuauiXRKfloGk/b3hAKqOs6ZlJS0fcGh6XvepIsIUkTjDU0TcNkf8TJwQldb5ju7fOTDz6k+1HF4dE+u13FrvqEtq0RWqOSQF5q0hxsaPlv/Xf+Kl/8wn32D0co6chzwdnVFu8dk8keDz55wHvv/pTNyrO6ahHeM8pTbt++jUoKrjctT58+4/mzB7Tdhu22ouss870p9++9wicPH7HbVkzGU2yoCM6hlKJtW2ZHc9566y1+/MOf8M1f+TZf/fo3+Fv/x/8DxVjSGwcIrHdY73DBEwDvPbKIGY9znrpp6J0nzXP8kLHdvn0LKRXG9CitkFJgTE+aJYzGYx4/+4jnl+e0pufq0cNfuFl/WUHgD4C3hBCvETf/vwH8D37RL3vn+fCnHzHLR4QQUyhnLZPpBCsCiZYURUbAkqYFeZEjREBJjXGGtu0pipI8H7NePYgpqlIkiSZYj7V2SHtjWg5gjMUYy+HhIY8ePeHV+68wGiVcbK5BBpq65npxzaQYUagUayyti+mfEiBcQAVQQuCtw/Q95+fn9NYw29/j6PSUj588BALeOvquwzuHdzGFbtsW7xxJmpAkKdZuCM7jFewfltQ7gfPw/MmaPNnj9Tdu84//0Q8pi5JJOaerKhIFSa7Q8phf/dZf5Gpzwfd//IcsrhakhUTrGUUxQvkMmViCjyVQkiTMZjN8iAF0tV5xcXlJmuY0TcuTJ0/pnYmllwgURUZelqRpxvf+8LuMRiNu3TlFenj08CHT+YiDkxkXFzVXq0vSVJPnOS706ESz227Z1JfIJDCdzpAU1LuGPC9xxmM6R1EUjCcThJa0piVNc7I8ZVNvSVRCcAHbG+pNhe06kkTzwx/8Id/5zq/w1a99k+9+73s0piFIi9AK3xnKUcLewZjpLMeFik11yf6RpCg0R9mUnzyq+OTjp9S7B+y2kpPju7S7CwQtUii0SvDBgbd8652vUxQZjx9/RFEqRmVGbyreeuMtsmKEM57a1oymmjxPqesG72P637Ytn3zyCavNgmL0Fr//vT/g2fMz0rygbw1KxtNfaYUUGiEEUkoIEEIsaYsixzYdddMgROC1V+/z9a9/nfPzC4yxZFkGXmFsR13t6ExL5w2Pnz7CuJ5yXPzCzfpLaQyGECzw7wD/AHgf+H+EEN77Z/2NsZ4sK9hstjjnef31N9BK432s/5u2BgKj8QhrLI8ePaFpO6RQKKWw1iGlQusEKeUQQV2spaxhNBpRliVd13FwcMhsNqOu6hhIjGE8HlOOSqSUZGlGmmX4AF3bEUKs1bquo+9b+m4oyK2jq1v6piXVKVmaMh5PcN7z6cMHrDcbnHPUTY3zDiFjXyHWuBYfAlmaoqSEIHDOMx4Hvv2dbyJEYLep6VpBvRUk4ZCL5y3vfO03+Df+xr/Fwf4B682CIFt2254//IMPefz4jIurR8hsjc4bNtsKqUbUXUVtFhSFpiwKDg72uX3rNuVohJCSq4tLkiRjvrfParnmwYMHnJ2fo7QiEDg+PeJLX/4SAU9vO6azKdYbrlcL6rbi9v0TrKhp+g0yDbRmR9WsafuKXbWl72PAI4AUiqbpyLKcLMtp25aqqui6HmMsTdNirUPrDO9jf8g4g1ASISUeaHvDw4ePsNbjUXzve9/ngw8/pWladKoYjTOk8hjXYlzL1775RS6vn/GDH/0Tvv/Hv8fj5x/y/OwBVbXl8OAAYxyPHj3n0cMz1uuWJMnxHoRUSClp6pr7d+/yza9/nXe+8Q7Hh8e43nL75Dbf/Oo32J8fcDA/wPaWro59FCkkIsSeRlVV/MH3/wjnPT9+7z3+4W//Z+TjCZ31+BB7RDEIaNI0JU3iPWydxdgYjNM0BTx93+Fc3PTz+Zzz87Ob3kdRFLH0bXZst2tkIlGZJi0zVPaLz/tfWk8ghPD3gL/3z/XLQrC/d8jdO/dYXi3ZP9zj9qv3+fD/+3fIJiMmZYm1PVKCVoreBapdRV7kJKkgeFhcXSOEvmniGWMRNjAqCrI8Yzyd0DQNvTV0xnDvlVe4vLpitVlzfHLKaDTG1fGEScqcY3nMdDymWseUXGmNAALgrMV1BokgVZrgPQzNszzPebG4pLId6aggCInpOoosoyxynOlItCKEgJKSIi+GJpPFGcf+fESik9gos4GutohQYPsU02f8p7/9+3znO+8QvKLrLElhmKg5L86uyKZb3v76MSe3cn7ywwXvP14wTlKc3zBOEpJsRqI1idYoJXHWUe0qDo8O+cY77/Dpw0fsdhVN3VIkCqSImcd0wnq74sXFc77yla+yP9/no59+yNXFFc5YVtsl67Dl6NYebbdld71F54L9wzlnT68QAsq8RKeO+XwPO0nY3ztgebViu6xo25Y8y0nSlDTN0GmC0IK2a5FSUdUNZTainIyYjqe0bYOzCePpnB//+D2urhd4L+n6nnIyZjKdstyu2awXPH3+iB++WzCdKerqEuuvcWHNwcGMpq55/OiaD396xYtnPfVOYo3GG4+1kGY5ewd7TC3YvkdLwav37pMlgsvzFxzsHaJQJDLhYO+As7MFwXm2mw1SKISQSCEw3nJ5eUmRj7hcXGOMI5MpMsmQ0qF8QGgFOjYAAyCGBqBAEIKPt9fQkA0hcH5+zuPHT+i6Dq01AolEoZOEq6tLttUWlac4EfBSoLX6hdvvX1lj8E8ugcD0lqpqAMFicc3TyzO0Tjg+OmJU5OgQKEc5velo+ljnz2ZjpILNZsVqtUMgSZIcQsA5RyJjRzrNMtq2xRhDmqa0TYtzjldefZWHDx+SpjlKaVTmKcoClaWkZUGRZexWWwgxPdOJJhcC6z3eOnBQ5CWm62nbls5aUmupmxaVpxRZwa5p8N4zn48ZlSNWyxqIaV6WZUwmU65XK0xv8D5wfV3xD/7+f4YzKQf7B5y3FbY35FmJ6QKPHpxxfHjG5fmGk8PbJBnstlu2u5bZacrBUcJsP3BwkqLLhiIPHOwfM59JLs9a+q6n7RuW1zmm7ynynL29faxxtHXLZDzFO8F0UlDXNWVZstqsePz0KcW4YFutmUzG7B/sUVcVfWd49OQxxVHP/funnNzeZ1LkTGYjMjHh+nJFU7VkecJkOubOnXsIWUIQrK63CCEZjUYUeUFelOwfHtI7Q9VUZDns7e9R72rSNMEYw/VqSTCOUTmi7zqsNfS9YzrZY121eC/oh1LP+zjNaduGNM85u7jE+Q3zecqrr93hYP+AP/zep6yWHbYXVDtHqjOSJCdLM4QQ7KodRTbi8aOHrJcLxuMC61q0TKh3Nd//ox+CUiyvrmOgSFJ0KhAagvNYY8AHrLWxtECg0oS67XDOMSsLpI0lDEoMkxhQSYJSCjHcyyHEe94GR9+1LJdLPvroo2HSohAIpNAoKVgsLrHOkBaaTV1hraVMy1+4/z4TQaBtO+qmpeu62ARpK86uLxgfzlFa0tQVWIuQkt1ux67uSdM4yhqNC9q2xbkGrdKhGehQQZIohTEG5yxXV0u898xmM9brDSEErHW88spr7O3t0fcWaS3G9HTeIVON6XrqqmFeTAghkKYJeZ6xrWqEF3R9T5okOO9x3oOF0HWAYLfdEaSgGXoBo7Ikz9MYPAg3QWBUllxeLbDGQhCcPTMsLp/y9pfv0HU9+3sT0kzQ9juyNOPenXu0dWCz6ijzCU+fXpHqfdI8cPvOASE8YblcgQgcHKbcPTrki1+4R3ANFy9+grWWznRsNjvSLOHNN96kHE/46KOPqXaxsUkA6zxpnmGt4+zsBbt6w1/4zf8GH330ETpV7B/s8eDTBzGjCJ6ub3j67DG3T0752he/yuLFjnbbkGhN7QNSKPKsJFEpnQm8eP6czWZDkiSc3rnHvbt3OTg84u4rr/Di4pwXZy8oxyWzozGXlxdcPD9js1ihhWK73CBEHFnudjXzeZwOJWlJ21jqbsHyeotMdUyPq5bXXr/FF976i1ycfwpI1ust1oA1kCY5e3sFfddhWolSCXvzOc5seXF2xv78gFE+x/SW1fWSNFUcHxxTtRVPHj0iLUvOzi9IEkWWKO7eOY2Nyu0O0xuCDySpGk50H0vWQiGAvqlJRTzHCeC8v2lmJ1pj+h5rDFoqyjKnahvCkHUulyvKLI+jxRAQEooyZ71ek+UJInFYPCDZNtUv3H+fiSAgBJjesFwuKYqc1WZJksT59XazwVtDJhUJKs7bdcKbb77BV7/2Nlme8A9+67domx4pFUqlpKmndR3OO6wxrDdrnPf0pqceLmKaZWR5hpCK58/PmE3GTFLBbrejmE7p+562bgkh0PcW7wKj8ZiTu3d49/2f0DYdcpjrChnrOaQkzTKO51N2bUPV1jjn0DrejCHEOXlRlAQXm4XVrsL0Buc8wQe6VpIoyeq6JUsTyjxDyIa+3zCfj8mznE8+fkxwKX0nCS4nLQre+dZXmR0FHr14wv7REUo0INbMD0aMJhPOnm0JQcZmKfG58yLl8OCI69WKtu7QOsG7aqjdNUIJmram7Vq0Uvylv/QXWCyu+M53vsXf+4/+Pkorml1NOhaUZUnb9jjrkUJS7XasFy1pmjEejxHSs1lvabtHeFKMscznc+588Ta/8s13+PKXv8xs/4BiNKbuWjabNTrRlPOMtq05e/qMj977KRfPz1hdLxEePvjJ+0wnE4IHJRQEBQgWl1e0jSVBIULCelWxWu6QwnF5sUFLiTUPObu4Zn++By7Q1Dmr6wVt3ZEXOa+8+grnZ49YXm+xxnDr1VvcOj6mbSvG44zRuGBbb3ny7AkfP3iAEB4hBbduHfOFN97i2fPnmM4QnENISZpolIDGxGYpztM0LZkSGGvIEh3vD2tpux49lG3OudgkROC9J3mZIQxjRB9CzBYGHMU3vvE1fvjDH+JxEUugBKCGcvXnr89EEAAI+FjfSHkD5KHVeNOiAF0UJDLWWHmeIaXC+0C1a+haAyjEUIcpGdN3Z3qcczRtS1mWFGWJUgpjLJPpDKUUT548Zb1a8613vo51BmsNIXiUTuLr8rG5Y41FCslsNiVNE+ymRQuJCwEPBCnRaUJWFMwP90mqHd1FjwgtWkqSRNHUFcYaZnqM8R5jYi+AALZ3JEmKFJpUKe7eeo3t9hyE4eCwYDpL6NsM23e8eHEGQWJa2J++ScBy9/Y9anPJxVPPfHRCSst2fcn19QoZCp4/WeCcpMhzptMxNkQAy3q95vnz57RtS5aVEGJDq8zH7OotXdujdYJONKvlitu3TpmMxzR1TfAeZz1pkjMpx4yKnN2mjmPeuqfMx8i0QMuMpq3YVTvazoIqODo85Ctvv82vfOObvPWFL3Kwt49OUlCKPaU5dbcI3mNEg/dTDmd73Dk6ZbNY0lQVTx89YbNa8eUvv81qteXDjz5kMppifeDSXyPQCK9QImN9veWD9x5gjEXKFuED5y8WeBSpHrNZX7JeVfR9h04kUga22xVKCXSi6fqOPMvYPzigqVKKMmU8KSjGBUmmObt8wWoXMN5wfLJHolKuL5dsVhskguA9fdcgBaQKvO0JHrI0YZSmbNfLCDwS4EMgSeK2tC6WD0miEQGcADGk/+ElAE1rrOljc5nAdrthPClZrK6RWuNDIACoXzwD+EwEAaUUSgmM7cnLEZPxiMX2GmEMxngSKaEo4SWa0ASePn3GdrvFOcN6s8M7j8EhMISh4/qyEx8nDA6lNCA4OjpCIOi7nq7rqZuG7XaHLmPDrq4qyumUNM3oiOOquqppmprl8hoICKXwHmwI2BBwDOhEYLer6Lue4ALBBhKtGZUF1XaDVoo0SWibBq3iZCIEKPIcGyBTY6bTKXdv3+fd959RVSsOD6ccH8/pasvZ8zUhGCaTAiWhryaotOGPf/A+UlsunwlCu6PpGuqt48XzBaYes1kFbAQ7kqYp0hm89zx/9pzLiyvSrEBrj5TxNK3rlr6z5FlBkim8sPze7/4u89mMn7z7HtVuh7MRi2H7QLv1KGPYmYom6ZjkexwcHGI7RZ44lJQY05KXJftHt3jl/n2+8+1v8+W3v8y4LJEhELwltl4FUkiCEgQDpreoINibzRglmr6dcry3R5nn3L1zjxcvzsmyhMfPL3l+domWGiUUIigUCfvzIxLpCaIiTXLqrcWYjiQrMK5hcbmmaTRpmjOdTkiThMX1FUdHUwJT6k1sUAqpSJIUEPHAyhRKS3ywjGcFvTME0bPd7Hj27AW73WY4xQPe9nStJ0tzskRxsH/EbDbn+uqK3WY14CMCeZEzGo/p+z5OoULAMaT7iUKIMIwQI4q2qRvqpqYscgiOn/70fb705S9StTtWTR0zVCD4z3gmkCQaKUESSFPNZDphtBnjtaI33ZDJSLwXsYE1nPCr1ZKXBChjPM6Zm3Ge9x7vbIT2hthpTdOU3a5ib38/jqLcANZRKdvtjoPJHlmWsqp26KJglI9pk/ZmbltXOx49ehSxDInG9obe+wj0cA7lPb5tuVou40gnCEQQpEnKeDyib2uKIgKWnLEUkwIzlBqT8ZTNbkeiMk6Ob9N1ESDTNC3n52dU1Q7rLBcXL0iTwGgEXddwdV4z24c/+O4POD05oV3nfHq5pek2CJGDTxmPjuh2KevtmuBD7D8I6LuezW4bN/M4GcoACUh26wqpBXmW4qwhK1M+/fgTXrl/nw/f/5C6qsnSgiRL2DYb6pWllz04QyNa1H5JeWdMYx2dqxmPpty+c8rR6TH3XnuDWycnvPraq0wmY7zp8SFmrt4FjDEonSKkRCIRHpyx2C7OwOvtjtlozBffeiOOdvOMo8N9Hj2/4o9+8CPquuLy+prOGqRQHO2fMJ1lmH5D12/Q0iF1xnpds91VpElKno9RuiBJS6RU7LaGvYM55Uhz3l+h0wwQyEQjZKCqKzI0dbNDqMCrr92lanc03YZmI2mqBmcsKkvQiYLgybKEr7z9JaSIB8jR0SGL8/MbwI/DI5ViOp2yvL6mNSZmWxhwAV2mqFQjBCil0ToGK2t7Ei0Bx3bXs7c3Z7wYcbHZkKu4xZ39jAeBWFMLyrSg61uapqEoCjrhkc4AcQzoh3mqQGCtjThz54dN6m9O/ZeBobeWNFEopTk4OuTo8Jh33/sJZ2fnjEYjlEpiHZ9n1E2DUgfMplOWux3eB8pyRFe2ZGnGeDzhYrmgXi6ZH+5juljfCxmwHiye1vTooOlNj5QSJQSZ1qRaoVUc0UghMdYCERvedz193+M8JDrhYG+PNJVcXp4jhSDLMjarhmdPLhgV+3RtRV4EvFiDMti+wVuNDJLgJF0VaBtH7yAtU06Ob/HWW2/huo/YLGIdWRQ5PniaTYM1ljRNSVRCayzWxFqyby3lKEehadqaLFOs1itOj08iGEmnFGnOKJ9gjMU1lsm0QMie7WqFHwtsF/AuNt7u3b/N21//ErfunDI/OGQ0HkUOQCzowXuC8LiXtXIBaZqhpCJRCabv8cZie4PpWpZd7NdoCQTH8eE+hye3yfOUk9NDLq6uefDkMZeLK4ILJDKjswItMsZF5G0sFi3eSg4PD8mKKZ1xbDY10iuSFKSCsizJsjzW4VKCl7hgsd5RLdd4Ybn/6h2mB2POrzydaXj+rL7B+EMg1QpnPHvTGf/6f/uvc352zm/9/f8YYQ3L60XkwIh4j3dtN/At4mEWfKA3BgL4HjI94CWG7FYIgVKx5pdKkucJi+tL6qbCBxAiZrc+2F+4/z4TQUArTZomKC3ZrDYsrhd4LbEi4D3xNPfgvCBLc7yw7HY7xuNJHAM1zZCmxc3vfXzDSisQgvFkwv7BAWKYsV5cXjI3hul0jk5S0jSnqmqM6SlHJYlOCCFG2ywrKMoRSaJwD128YUWsz9Khx+DaFoKPUFovyYsSCfiXxBEp6bo21n3e4V1EyKVJQtN0KBk/qPnBHvfunnJ+8RwhW4q8RAjYrhuW1x13v3LEeDxitdnS9hvSDLSumEzmGJdjuwbTVQQvSDQkSjAeZUxmGpX0COFIk5TZdErV1Fhrh9FnSgixOWudRwWPRCOCAi/QUmN7S111BOfRUpHIFNtbRCaZFHM22y0ne/eQ9NitZW96RNdYymLG7Tt3+MrXvsT91+4wmpaoLEUpicATvEXgbpq4bWfYVi06zZCqRCBBOWwIeGcRwSPxERXX9lTbDVIm5HlBUk6YTQt+9TvfpLeBjz79lPd/+lOulwskglE+oiimSOHY2ooi36MoBSrLyIqSbV3z9NkTrPWURcF6fU2R5zCQdwIR0CVlIMszlptzklJw+94pdbeh6bfs6prFlSMEgdYJiZKkicJ4w+nxPvfv3qKrdnHacb2krRt0lscpC4F2GP8x3M1DRT+Q1SIy9SXRqaHl5OSEzWZF3zeoINBJzieffMRqt8OT4l0cR3v3Gc8ElJakaUyRvXf0fU8ICkMgEBBSxYYVEYeeCoHWMR0iCPrODAysgA+OEGIgzbOMEDxJkrLb7Viv1rRtOzQeY4YhEIiB6dV1XXwcEfH9xhikkHRth1Lx8ZMkpelarIfJfI4A6q7FhhA7wUimo4KDvT36tuPs7AwI7HZbnHOkaUpZ5KjRiDTN6I0jCM1oPGPv4AAhHLvdJVmmODreQ20TpJ9S7zymi4Gt6bbM9hWHxylm2zKeBtbbjs16RVFqxuM5nakRiWdXLXj67H1Wmyf40KPUCOc8bdsSfEQsIiLTUrysxUMgTVKc9dRVQ5IrvLNMxmPapkEEkEHQdD2NrCnyKVYLsJosTyjzKfPpAePRjDde/wJvvPkmr7x+j2KcojOJ0BKCJ3hPcBbbd1gT8RXrbU3TWSbzfYQUCC9QQqJkzKa0FDhrqHdbqqrCGMt0usdms6Lzz+iMZe/gmCQree3V+8zmU54+e0rfN4xHKVI6Ls+fkyYjpJ7T9i3bZocQgTRTBOEwtsU6Sd3scM7hQ6Dre7oBlzAaZ6RZEkFU1RX5VLGtVtTtlrPzDcaW5FlOliucbSB4ppMRb7x2j7NnTzh//hRnehrbc7h/SNU7jHekKh5a1tqbrDbuD4VEYIPDGIPWKQLQWnP79h36vqOpdygV+QPb3QZrPagc50LskXnxC/ffZyIIxM47JGkCQiCVRChNZw2JToabQOKtx/aRaDGbx+6+dyGezD6OUoQQSCVumidSSNq24cWLlrpu6PouEoKcQ4iA1prgXcQZytgZDz7Qty2m73Hecn5xzmhUkCQRh9D2FtCRhuwcfW+o6wYRAqrUSK354pe+TFPXvDg/xzvYrnakqWY8mTAeFWid0HUdPnisNRweHZGkGR998hOC6hE6YzqboVTByeGM1bJmu9silOPo1oSvfP2E01slH7/rcFRM9wsaU3HrcJ833/oCH378Iav1il215MWZYFcvEMojtWS727DbbkmSBKk03VB7ZmkCWHpjiKxdB85RjEY4AQcHM/q+IxDQSWyKGW9IvCXPC87OLzjYG1OWEybjCa/cu8vbX/4Ch8cn5EWOShOklghhYxBwsZwxTY1znmqzYXG1xAYdiTQhILwYGnIZNknj9XKG3rR0XUO9q6h3O4SQOK2xztP1PaCZ7R8wH4+YfvELOG/IMkVvWqaTCW1v+OTRCx4+fkBVV/QYgggURYJSkGiFlGD6DiE1LjjqrkYIDzKj7Rtme1POl49RlRsg4AVts0JqwWQ6phwlrBYd3luydESepTx7+pS6rrh9eoQUgte+8CX+4I9/Qtd7RqOS8XTKdlfFA4l4b2sVpwPxwAAVuBm9ZnmK8wYfYnaptUBrgfUChEIGInJRJb9w/30mgkDfdWRSM5nMeP7iHIsgSxK0jyevMxYlAnXf0FlHkAqVpFgX66Ku7xFCoEWs46QQIALexL4BzsUbnbgJnAsY29N1LbdPTrFtw7rdEVRKniq00mx3Fd4YwLNrtvS2i131oCiUZnwwo6mrCBn2Fte3MXUdyoKQKrQs6QSUIaGvHPuTfebTMQiHx3F5fcnZ5TnWgQuWy7NrNu2SkzszJIradtRdQzmaoLIGr9dkU8e3vvEr7B1p2q4lKXq21YZ8XrCfTLn/xjH7pwX9w5plvaGYTNBygpY7OrnBqzhPdsFBiNcpWIeQnmJU4ENL19d4GUFXzhlQGVJ4yjKlqnfko4Ss0NT9jqTI6OyGyXifbmvY9TW3Tw65ff+Et966zdFBRlF4lAaVlQgZCH0Tv0xN6HaEtsIbS7dasjq/ROVzgg84OwQBoZBK4wVUfUvjOpwMuGAwpqVabijyAjmeEaTG1g3eBTZdR16OmO7NGI1HJEUOespoNme5XvLg2VPqdkvb1+zMLiJB8UglCd5h+o5E6pgFKoMJDc5aqsslu+qKO/cOqPotdtsynk4Z1yDDApn2HJyMEMIj5RRrOpbbHT/+yQd842tf4/4rr7G/f0Jwlr29Oe/++Ad0sudgOmfvaMZuu8K7HjE0tKUI8YCzmhAUIcSDRsjAcnNF0+1AWdICVOJQ2qGtR3uB7OMYUYh/BdyBf5HlrUX4OPronSPIl8IgCukDRVEQgsOYlt56ZJJTVTuyrAAEzvsb8YsowCDxzsRmCQJEFG0IAtI8ZbvZkSYZm82aLNUcHx6wXl5Rdz0nJ6cUWcFyscRZE5uPIQYRIRR9azk8OuLeq6/w3nvvonVMleUguiGkoO07fvrxR7gAje0pVU6RlJyenGJcg/OGbbUaPsCKvYMjetdycXnG/umc8d6U3WrHarPFGc9yfYVzLcvNC5I8cOvebZ6+eMj3f/Axpd9HJoA1JKOUdJzw+PwRV+slq23HndMMZxOcTfCAlx6VKpTWVNsqCnsET5ol9H1N3+8wtiEvZhjjMI1BJQzXMACe+XxGQID2ONmTFjn5RJGUc7TSnN69xf1X7nDr9gFJYpDSoFOF1AneG3zfY+odod8R2i1dvaFtWqrlmmp1TTrRBGMjq9IGpAqRUQeY4GmNwXhHb3ps3yHwCO9wvcELh7AOLSX9rsU2Fc12xXh/j8nhIeV8jkoKgtxSjlKyQuNWsRQxzmOtv/ks+64lyUsQlrpbo7ZQ1TvqZoNQPXde22f/aI73AdOD6QTBSvJCcHxrj+ViwSuv3eX6asGzp5d8/OkT3nrjTWbzff613/iLCODdH/8hiewoEk+RCVItsKan71tARhCUAKUFwugBEBUFc+qmout3uNCT5Yqs0OhUojqBUgJvLM4HZPADAennr89EEAhEhl7YbiMSSic4N7DtnGc2m9I19dBAc1hrb9h/LxVvQoj9g1j/RFWiTL5kngWCj8o/SZJSqZoQArvdjqqq2d/bw4fAbrvj/v0oQpIkMV2P8GJLOiC6IpgoQp2LoiBLU87cixi0ZKxdvXM8fPiQ3jqMNfjEsn8w4+TkkE8efoCnp+0qrO04OJpydLQHwnJ0PCMpJcurDZvljpPDU27dPmG3XnN1cYl1Lad3Zvzk3Z/y6OkjHj8859Ys4fBoD9tbdpsdz59ccrW4ZrfuyJICETQX55c0dY1QAuctUqUEYhmSJmmcXAi4ulpF+m+qBqUkzcHhmOPjE1bra5yNaM08K1httiQ6ZberuXU8R0tFlqS89uqr/Mo3vsbd27co8jyOfrVCJpqAx9keZzr6vsO3LbauqTZbttsd6/WOtmkJaYc1HQSPRyAGZZ8kScmSFBkEwQa89TgXSLIcpKbv49jXKU2RpngXpw2WQNO3dM4yDwGV56gguH16l9V6y3K9paq7m3sJIZEy9qh6F0vVF+fP2FYbvHe03Y5bdw44PT3lzp1jttuK3/md32dxuUP4yCfRQrHd7DjeP6JrDVpognN88skDZuM9vvn1dyjygrPzc3SiyMjojcU6x2Q2oen72BRXkRQkpSCO/FVko4o4NbBNG/UF8gStYkPb+Qj79sJgQ+QdvJxO/bz1mQgCANZZ+u02jveEwBpLANIkIQwXI4SYFtkQhsgmb2b43vuhq29puxbwFEkewRLBx/EOgBBMJlPqXRVhybsth/sHKK1ZXC/Y7WKDpSzLKPwxdNCjPFkUc1gsFjxfXHB0vI8xhizLaOs29iKIGIWm6bA+QmilAp0EdtWKpt3R9Ft6U4NwzOY5eSlR2nH77gEfPXjE4ydnSARH87uM8imXzxfUu55EC5TM+OiDT3DCkmrw1lEWYzb1huvLHc4qjPF4I9mbzkl1wWZ3jfcepaKAR6rtjTxbnmcopai6mt4E0jxhPp+wWNSMJ1OOjw6ZzfYiUce0GONp6pZqWwEC18c+TTHNOTk55ptf/Srf/OpXmJUZioiVV2mKGEA1zvYRz+Etpu+oq4pqV7Hd7thudzRtC3l3Qw6SUg2NMU2aJEyKEdusoBcbEhm1GLxzGGvp+i4iMLVGhxxc5HQYP5BwROwvFNMZSMG0nPLqvde5vt6w3bWsdpvYTcfh8Xjn6IxFSYXdrNizc2bTCVJ7skyzWi0ZjXJEUFydX7NetZRpgRSSq4srNqsNz56+YLPacHpyRL3b8PjRE8pszG//o3/MyfERQkiKyQRXt0QtONjb26PpDF1n4vRGJRFIpUAy9LqUIASB8wHrAyFI2t7hvaVqDKZ3SO1wysVGt/yMjwiliOOU3RDV3KClFoina9O2KBHYVQ06zwewhY/Y6EH7ryxLiiJjtV7inUcnEjPwsaWKvHAXAt450jShQUQu/dWCO7fuMJlMuLh6ymazQSlFnueRMzCQOYKPwSZJEparDdfbJeUoIwyPF0VLYqAxXY/pDUiBVprxqADhePb8MbtqTWt2JJlivj9Gac9kVmCtZ7m+4PzsktV1Q5lnrK93vJAXPH96FsEeXoGT5OmI2/eO2K4/4PDggMlowmq1QbqU3bUhy0tSMWacTRgXY9pRz3q9QGmJ89D13YDNUHR9x2QyplQ5b8znuNATpCPdetI0IyDoe8tkMmNx3bPdVEwne4zLKYvFNVlaIILi1ukpX/vK27z56n3GRYZWsckqk5SkLBBKRg0G3+P7Fryl7zp2u11s2HY9XRsxE4k1mL6jNy06TUhVDs4ivKFQmlle0kpNrxN8lrFarWnaFmsbQnB4Uiw2Vi8IgonlnPGegCAgyMsRtveM8wn3777K9WrNrm6wzseyz3RD03eA7qaB1myRjWM2LREyovOscYzyGV1twAayMuNwb45WioO9QxaXC5SQnJwec2YdXdPx6MFjzp5d8Y2vv82v/vlvko9GNNZTTKZ01lK3LUmaEm7kPgTGxoAWBAjUULIMmW6QOC+o66gf6azABYFMQOpY7vWh/4X77792EBBC3AP+L8AJMaP/WyGE/50Q4n8B/I+By+FX/91BW+AXBwGpyNKMaiABEeII0A+peOQUCKwNCOfQWUZvDFJExpW1loODA6QSdF2LUjFa2t5itSMZgkAIPxOAHE/GtFXDYnlN28fU/uVjFUXBZrOhLMuB9OMYj6ZkWcZ2uyXLUvbSecTbD4Kh3vt44xNLhe2uIgDHpyccHh1SjnOWq0t2uy06g/nelHJcstqsCcGyXK94/vyCtunJ0xyNpq16rlmihCYM9WGiMqbjObdP79B92fHKrS/GjnpvyVTJcrVD+pJxNmM+OWBvtkdbNVxetaSjhLIosZ1lXW+iSk2A0TjyKu69cpfl5pqLq0vK0RhrPUVRonVK3zU448ELDvePOD4+5bf/0X9KUYwZF2MO9vY52t9nOipJhcD2cfJR5gVCJxBcbHT5juB7ZPB4GwUyrXH4QSDUuXgCe2sHGHGcEti6ol4tMNsN2hgS70kIpEJgTBu7+BhkcGA9IUROhhAK23Z0DnzbE4KIUGiZ0LUdvTUcTA94/f4bbHcNV6trHH6Q+RLoTJKmisk0xTnDcn1NklqatmC32VFtO3K9RXhFniTcu3WXg1tTTG/QIuHqxRVHt46Zjmf4w8D14prVck3fLRiPco7vHrLc7PBCkqQZl9drrhZrhNSx70LAB27kx4QSKCEIQg6wGImQCu8FbRuBXlIWSN0jtUSqOOJu+vZPPwgAFvifhRC+L4SYAH8khPhPhv/3vw0h/K/+eR8opvRRIWg0GtHbQF6WjMdTVstr8ixhNplw5/Zd3v/4I3SacHp6irOBqop86TzPI4c6zfC+H2rXOK9FCPrKRKEO5ymKgsl4QqKS+KFs1qy3GyBKQY1GI4QQ7O3tsV6vWa/X7O8dcnh4yGq1QipFrjIAsjSlpR5EQhRKKgSRDWasZzaZcnR0RLtbUVUN1gVUUEBs8BjjODs/48HDa9q2pe8TEqUInoHtZphOxhBSppOcRGkcCYlO+OrbX2Bz1XN29hyBQAmN8BqcJMtzJuWEUZFDsIzKFJ3IG/ZZ11usdRR5hrGWfrfharFgW28x1mBMGNSHc6xxrFcbvBMUxRhnA1/6wtt8/3s/REpNluQUaca0LMi1RAaDdYbOWPJI5ABn4mnuehIJQYIUL9PbCKP1zkdk3MCfT5REix5b1azOX7B48ox6saDb7qi3W/quoesaZLBkaRTVcF0PzgJRycgHAdZgW4cRFlBsshIpU7xMMJ1Bpwl3Tu/Qdj3hk4+4WF6hpEYXySBIkzEaaZ4/e44LPdbWVPWGIi85nN2iqwPtNpLZDvcOuHv3lMePnrDdbOk7S7BwfnYFPlAWY0zv8G7L9fWSv/db/4CL9RWTyRj0jq636CSlabs4FZGxIRuEiNMtEVA6jsxfKlprpUGoQflZo7WGgX2YqITa1fTdLyETCCG8AF4M/94KId4nSo3/i69BXjmEwGQ8wQVJlpccHB5SVxVSCg4PD3nzzTf55PEjVJLyjW98g+AF6/Wa7XZL13V0fcvR0RGLxQWTyQScH5BeITKyBFEqWyryvODVV1+jbztOTm+RZZrZLGc2m5HnOWmaxmxkyA6MMTfBSitobMd4PKKuGmazKYmMqkZpmjKfzsiynLrtONjbY1SO2a1XeC9IkhznLM4KLi+XXC527B/OAEXbJjgrQThSpZnPpwTrKIuMzbrh5PgI8NR1xfJ6ifeGj95/hDWBcrRHte0Y5SXeOLx2bDYrDg7GKGV54417eA3L1Yq+Mdy6dYI3gc1qTW8Mxaig73u22y2rzYpqlzCfzfnkk0/x3jKdjMmyhODgg5/8lL/wG7/J3vyA7bZiVI7Z399nbz5lXOYoLLaLk5zI5QjDl0P6HiUCFo8kTnNeimk65+MIJ4TIuEsU2JpqueDy8Sc8++hTrp4+p68qlBRRaswZlJYkWYpE0ZiANwYhAR+bhzKAMx3GWbxXLILCOpge3EJ5iWl6iiLnlTv3ubi64unzZ/jgGY0mTGZTshzSLOA8ZJlm/2CKMx1eFty9ex/pco7md3j+9BnjcoRCsFou2ay3lOWIvrdsVkvGoxHee8ajCbPpjJNbR3z3h3+ISyyr7Y7OCWbzA4rRjGfPXgzgH0nAR8Ja8IhgUSqP0vXWR/n7vIiEOBuABILCW4mwKjY4u4Btf8mIQSHEq8A7wHeB3wD+HSHEvwX8ITFbWP5X/D15XnD14BGvv3XITCZ89NEnLK6vo1Zbqnn48BFPHj/BWkfdbfjxj3/MyfFp3Gx1zWazoW2jFLkQgjRJWG6WdG1HlmdRwz5JaNt+6CXA6ekpm82GW7dvcXl5xlu332J//4CnT58yHo8HAk/kMUgp2WwiK2w6GhF6SVVV4Byj0QFf/fJXWVwucNZxeus2B4cHXC+XXF1f8+Lsgq4xaJ0TpGA0KSjLESfze5ycdoynY+7eeYOmc1RbQ7ACbzy3T0/56IMPOTza40tfuk+gJ8gOv7I8GDjsQViyPCdNNLPJmNsne5y9uKSqdixCx8F+ThCG6d4B49mcNM05f3HOdDylb2G7a1Ey8iScczRNR9tavFPsdhXGdrz5xuu0TRXRnG0MFL/9D3+b1fWGruu4e/c+t2/fIc8yMi3p6w68i+o+DKexN9iuxjRbUt9FbIUxeOsQXsRywMQu9qgsGY0KlBZsLl5w/uQhV88es3j+mKcPHtDtqoi8nExI8gzvJELF6ZCSEp0kaCni67U9tvfIEMAF6t2OujV0vQORM5qM6dsGYQyjUc6rd+6zWF5zvV0DkrqqcS4Gstu35sz3cqrtNT5EnUjTOUZpyv7+AalOWa+XfPzdD9hstkzGE1575YTZZMZ7774fhW6NAyl451u/wmodRW0D4AO0reE0H+EGfExRRnHQpm3i5ElFRqLWir6LE5bZdE6e5SyuFvSdochHMQMygXpZ43OH2VmmowlL1j93//1LBwEhxBj4fwL/0xDCRgjxvwf+PWLo//eA/zXwP/o5f3fjOzAqM6qqou97tNKUZTrU5VsO9vYIPsTavW2QSYYeBBmbpmW5XPHixQu22w3WWkKwpJliV1V0pqfuWrIiJ00SrHGUZQEBri4v+aM/+iO22x3nl1dcnD0nU3cxxrJcLjHGUNdRkkzKSDLZbDbMZlN0mrG4XA0GEJLF4opMJ3gTx2oX5+fUdUXTdixXK1QAawzGGuq2Ic0L2sYzneeURc6onDCdJRRlQaILZuM5TdXw/PFTpPKc3jrizp1jLi6ekmQ5T88FvTF861vv0FQNP/rjnxD8liSZcOv0mLZuGJUps70ovyZVGOS9Jmyrhq43NFXPbLrP8fExu92WuulIMsne/j7TvTnBT/DWMptNePvtL/P+++/Rtw3lfkFwgo8+/ISDvQPu3LrLdDJDacV8PkNoi2l21LuK1kmEUtGAxTpEVyNdS9/V7DY7NqsldV0TfCxl8rxAIdnf36fIM1xbcfX0AdfPHrM8f8H6+oK22tLWDU1doxJNWmQIPLbrEWrAl0gxGNMolJSEYOOG7cE4j+sDni06u4oZhbN0TRSyvX/7Lr21fP/dP+ZytSBIgSRhcnzIycmE09MJf/C936cYFVTLjuX1Blem7M8PuHt3ynZ7zdOnjwHJdLJHlme88dYXODg85ez5Cz766Kec3DohyRJ+/N67BAmeaMKikwhk8y4wm8zwwWOcGUpdi/NRqr6pdwihojbEZEyWZKwWS6qqRVKzfzDHdAZhDKIPyB6C8v/lLfinEwSEEMkQAP5vIYT/F0AI4fxP/P//E/B3f97f/knfgaP9aWi7FqUUL168QKiUg8NDur6PmIA0jfLPrsYLG8UXfYin3SLW0lonvOSip1lCXVckacpE6+joIiUHB3OKouDZ0+fUTcP60SOElAPlFK6urri6WgwKxTY6AsHNaDBJkhu58O12x3hUEoRnt1rz6e5TZpMZWiVUVRWdk2SczRZZRt92pHkGKJrasFxteHFxRW96dJbQ94ZyVKJ1JBxpBDjQieQnH7zL1eKAXbXk9TdfoSwnPHm24OGjp/RtF3X49wtM63j08FOurhbcunXKrdunLJZnrDdrnj17znrXYQzk+YjWd0wnUySCi4sL6rZm73CPvf0DRpMRJ6dvcLC/z6gsaNuax49K9qZTfu3Xfo3f/73v8vTRc956463IpWgaJvMZ+cE+obrEmpbddgPlnCwr8NbS77bY9QLR71heX3H+4oLF5TVta1Eqw4uEJM1QUnN8fEyRZVTrNVdPnrBdXNJsN7RtDUqQjUqsC/Q+0LQ9Mk3QUiKFIiiJEgGlQKjoYtQZj3MdfefoncdL8KJmu1pQ5AlpnmPqhq0zTOUh909vs9vtaD9o2TUNRwenvPnq6zi/5umTp1ycVRzuK5zVnJ9f0o4EMiTsqsDV9Tku2Mh1kfDeT37CJx8/YjKeMpvP6axhNJkgE83T52ekM40NgUQllOWIpq6j9FuaDbB0DzL2SxKt6PBY0zMqx0gh2a7XbALUdYVpPPlcYZqWIsnoG4NUgkJlhM796QcBERk7/z7wfgjhf/Mnfn5r6BcA/HeBd/85HuxG9ODp06c4JLdv30VIiTGWZNgYLy3G2rbhyZMnKJUQQkQU7nY7mqZlb3/KrVvHeG9v8AQvnj7H9j1f/uKXIcDHH3481J+BoizRSUKW6mGDRyMT59wNVTNJEq6vr9nb2+PDDz+k7fuofDQZIbyIzbM2/qy1Dc56kjQ2bJI0QYhAEBHznmWRJ7/Z7ggC2q5DKEVV12RFRpIIhAhkSYbvHbdPTnj48EnEt/cVXgTyIkHKgh/98Yf0nWM2G7Fe7whOsrhYErxgca04OJ6zWK64vFry4jIq6cxmB+RpwWqxZHG5ZFSMyPKCJEtxLvD87BxxKfjhH38csx4pGI9K2rbBG8dv/8Pf5upyQVcbrq6u2Jvuc3pyi3uvvBLBSE1FtV2yWS/Zmx4ikwTTNLTbLduzF9hmxdnFC85eXLBdN4SgyYspQuc4FCqXTOdzlFZsNxvWV9d0u5rgAsgEpzOSskCjcCLQC00iErTKYtZBQAoXxfSDQ2nIModODaFuaPuOICGRiqbeUm1zErVHsH1UNxaKwzu3eOerX8c6x7vvf8DixZI/bt/j1p0RVbNiPk/4whfe5Ozxjlwf4nrJf/F7v0ugpigj5yDPC775zjf59NMnfPjBJ/S9Zble0/Udl4srdKawwaJDrNuTJCPVOV3T0ewaQgZJluC8offR7yL4ftCfiHDm4H0kwgWB8IH79484Pjzk/PwCrSQyLxiV5T91oP2pBgFi7f8/BH4shPjh8LN/F/g3hRDfJB7LD4H/yX/VAwkEdd1grYtEkc5weXUZacDT2Y1jjxBR7ikoGWfdGPK8YDabAVFkwznPer3GB4cLgdNbt6I01nVMPU3bR3ZhH0U7qroiL0t88LTdzxqAL3UJXuq5XV9fc+dO7Hv2fcQeJElK3e4AbrwPTNvfoAYJgSzL8b5nPI3qvSFInIvINE8kd2itKcoiRv5gMX1PWY6joUXbc3rrDsb0JOmI9brFBUVe7LGrHSk93mlWq4rjg2Om0whQUlpzfnHFbtfiQhy36iRDq5QsLSgKw3q14brdoLWiGBVxqmECwYUBRtyxbWpMH/UOrTVcnJ8jhWZ/b4/rqwX78wO+9PbbjPf28NUlm6tzHnz6CdebhsNXvkTwRFUnAl21ZXXxjMXlOdv1hqa2SJGjVYF3AhskeZKgBbRNxeLqktXlCms6jBcYkbKxFc50SJ1Fwkw2QWlN5yVJECgxKPF4i3Ce4AI6zShKj9oZur7BBk8hFVJ52irDljm+b+naqAOxvS7YPznhW199hywpePTsOVIFvvTWV6iafT765Mfcu32XWSnZrQTPHl+xq7dkhafuG4pxgXOB1WaFCw5HoOk7jLX4YDlfXLBpVoxmBSpPkcQsxhqLs57D/UMm4wlVU7Gro7S9ShR90xJcoKk8ImzJEk2ZF9SVQUvNK/fuIKXgm9/4Kp9+/AlGtRRFxnbTkSS/BBZhCOG/IDqA/pfXP5/XwD/9WAQf0DpBBEkS4kabTKYkSULwnqIcoZSktTbqCCp1Ax2+vLykbduYUQRPVdXM5mPOFwvSVc5kOmW9WvPhhx+iiEIdQmk609M08XSwfYfMNWpI4ZMkYTIZ0zQtu12Fkgm73Y7bt+/gnz+n2kbDUEKg73tSqQfN+HAjBum9j8YkBRyd7JEkJyyuVzx+8jTOcIUkEOmjSsdMJM1S2q5jb37Iwav7fPTBRzgTS6AsTxmLFERO2/aYXuOcJ00KZvMxQSQkaU6WpNx75T4PHj+maSyQoRNFkib0xqFFj04yppN5pGLb4YSRHmMdnoDWUaTy4PAA7wxNXTObTjHK0mxr9Eih05TvfOtbvP3lLxPw+OC4vrrg4aefIPMJk+kUZy279Zaw2yGtIRWBIksZFQWmq24MROO1i5Dduq7xl4GL8zO2mxbnDEalVE6y7ByrqqZ3O0ZFyeE8MB9PmYxG4AJBeqQaBDlEFCpBQJJm6CTFWEfTW1AKpRN86MGbGKTqit5YgogaC8d37vLOV7/JfHZIVqbMJwVp0nDr+JQ8y2k1PPj0Y8bjA9759q/w7MWHLJZPybNJ7Cv88PuU5Zz5/pym7qiaiulsTNO1GN8wnU9Y1zVCJWidxoaedfG5pjNGoxK9UixWUQ9hPBpFHcddFanFxhK8oywSBPD86WNGZcndW7eYT8fYLKGuKpp+R57nv3D/fSYQgyH4YXyn6Dsz+NJJynJ0oxO4v7/PannN5voaLyPgQ0pB25obffyyLPDe0PU1VSUxxrBarTiY76O15mqxokwSkrQbWIiOEALb3Q5vevbyGUIIjDHDRo61+stpwnK55PXXX+fy6gpCoBscil++xkhciDzvZJAib5qa8XzKvXunjCZTqh9t2VVbVJJErPgwuVEv1ZWrijwfMZ7Mme8dUjfv4Y1jOplSVR3G1sz2TxCiRCeGqlqiRUBOc9arNZLYVMqyAtM72tbR24DOJVk+SLIHwW5bYXtLopPBlKUnKzOyrERnUeQlkrQSRuWMF8+eslwuGZdj9vf3cH3PpJzw7W/9CulkjPMtrms5Oz+jqja8evdVpgeHeBvYbrc0Z+f4zZpMK27dOiHPdyi1QZBTjA8QqqBzgWwyiepLpsdZh1I5fe+pjGfZWBatY9laqqaHVcPZouJoPuf+rTvMC0meAZlGaRmFZ2XULhQykqaEkhjX0XUtee7w1uCdpchTUq1o2pZgLKbu2C43eCE4mB5QTsfUmzNEojg5ucWzp895/HBDVfd87Wvf4dnzBzx/fsb+YUkxGSNVx9MnF/igECJjvdugE002KjC2IfgI8c2LgjQdgwfpFcEbdtsdWmjG0zHRMbpBWcV0UjAaFZRZhtYJy+sl2+2ON994E7znhz94wHze8O6P/pg0Tbhz75R3330XE2BSfNa5AyJKfRsXT1GHoDc/k05q22jg8VI4RKsI4qir5mfgl8EiDBHhxF3XkqYpXddzeXlJ13UkOpYT1jqkjlp2SZpCiAKPDAjFzWYzAIbUjVeBNR5jDNPpbFCETWiaGuEjn8EYQxiQb3VdR9qxjDZeo1HGrTsnnF9c8fjJI6TyWBdTz0xr6qYl2Ej53Nvb5/j4lKdPn3F9tcL0DhEk69VuUEvOUConSwVa9aSpwVhHQFEUI9q6AiF49PAxVdVincB6STBExZxcE1zcmMHB3jynKEdol+JF1EYwzkbDjsEMNE5eDBBo25ZURFm3r3zlbV77yttgo+LSZrXixYvnKKU4vXuXtMjpW4eWimq3Y3d+xt5Ycev111EqI0lGzOanHB7dozWw3Fak4zF7+3t44ZnP52yKCW3vqOua5bZm1Rh2JlA78L2laSra2mKN4M5+wXycwSSHXJKIwTYeEXn4Wg8KVD1d39O2lrre0TQVB/vloCZ0xep6RTGa0TcdIdEUecl6taVzO2b7gjIf0eWBPOv5wluvobXi4wef4IXFeMhDVABSKlrdW9OTpPHQWSyuGI1zEJHN+Y13vsXquub5k2eYvicZtAMWiwXrzYqqrWnrlmKcs9tuaat41hweHjCZjFFSMRmVBB/Y20vIMx1l0aeH3L53mx/86IfoPKGclMDq526/z0QQ8MEjM0Vft0iR4LXGO9jUUf1mbzpF64y2XaBVTjmeRF01X5OnCSFEpZxExeah845gIE0iiaVvenASnaZIlbHZrggi5WB+RN00uM6TpRnrXYWUkr0sxweBkhG+XNc1INisVxAcEgemJysyrpZXJEqT5RkdhoCl6lsCGZNyTJHnvHrrNXyT8OG7D7m+qlBpQds0+GDRXpCrJGLBDRRoquWaerOl3mxJ0kiRLouCuq4IxtJ1G9IsQ0mL3V1SFDmhr2jqhlFZst3u2Gwrmqoh1Sm4mDY65fFJLCGUCkgt6Wx0Q9aJZjqdsq12sf8iNSIEGu8Y5Tm+TxAIdFFyvap45bXX+PW//JfJJmO6viO4msXV+2jVkxS3ufPGnyfoBO8vePbpe/zOP/4BT54/5zvvvMn+LdB6xP03XmF++xV0PoFdRV6mjEZjxrMpzvSUWcre8R6N7bDLLX1t0C7BtY7gU7I8Q0pJ1fd8enVNUhvk3phCnVJmY3rbIVxDrhVp8OTSkScJShU4AU56eg+dMTRNjZaePBX0/ZbV6gJVJpTzPbbVNaPJjLCboVzC8sUlm3XPq7e+xNViSb29xlQ1o3SfjJyjyTHUV7z69huY1vDpJw85PTxi7+iQH/zoR3Q7QzEpMUKxt3fCevMpvW+QuaRuarI0Z9Nsca0g0SVaz+lqCGXCarvBmBaZ9RRlQi88P330kIP9KUf39uMhtt5Qm44f/fGPCM6Dd6yXPx8jAJ+VIOAcQQ7sweAATZJluBDou47FteOia/HOkBVF5LkPsFetFFJBlqWkiR6syqPNGESZ8USnECLt2LrYX8iyaDratX20vq4bQugG/EEzjGT04GDkfoZnD548z6JLrDF89StfwRjDRx99FKcJIZCnOmoJWoPpwHSGxeWKF88vYopubKy7kySCXrLoqdDWDV3TDKIY7obr4H2cLCSpAjxdHxmLWgnyXNH3Lc470jzFhUBV1bRth3WOJI2SbEJAoqLrTfAOOcgseB/nzzJEvzqtFUpJsmIUNe6FxhqPd1EIlSApyjFf+erXeO3NNyI7UwlEcOQpzCcl09Et9g9vEYTnyeMP+Ye/9Xf5g9/9KS4b88rrd9jVUVq+nM7JZ3Oc1NTLJXXfsX90QJIq2mpQdgo29ixsBz6gpQQX8MaBzgevP0dvOjZdxVR72mrMLgkoDDq0CBsNOBIVgzqDX0RAEAb9wCjvTdSSdAHron1Z5saURU5wljTJ2G07ri53pGlOUYw5PcnY7rYorWlXPdZ4XrufIVHsT/d4fP2YUZ7zpS9+kawo+Z3f+QEHt0akacF0f59nT5+z3axARpei5bqmqmsEkiCjelWSJpjOMypnGOdouy7iPUyDs9E/c7VdURYF1lmquuXoSHG9XOIhCuzwGdcYdIOWmtYa46O8mFQpUijSkaata0zfRa31ENN4b22kkBqD9EO33Q+a9VLdsBG1Tm7osiGE2GDLykFWLNxAM6N9eXQK6k3sS4QsGzQLYgrWdR2r1QqtNNPJjOVqydHRcTTUzJ9ExR5jSHRKIiIPvGlazs4vCBeXbDYbhOCmzNGDyWmSaIo0w3Q9Td/H7vawXlKlm6ahLMvBpyAMWAZHmuUgXpY13DQXq7qJMm0SMp0hlUAl8oYQhfgTOgwhwqu7vv/ZWFRHGzYhJV0bZdfTLME5x+3bt/nWt77F0fEReIscdPWno5xJkTGZniKl4PryjE8+/ICHn37CZrVgelygpKBzgTzJUHkBw1Rlu9vSNDUyS2NmV+/o6orddhvl1vsecCgRYg8kOFxXgZFI15NrKHS06hIyxJNdGFTo6L1F6wxrMwge5yP12Pvo4ee9x1iDGhyuFX7o+UQH6nEx4nKxpKprLi7O2GxXfPvb34plX56xd3BAWY7w4ZLpbMazF2dsVisOv3PEhx99zMHREX/uz/05LhYLJrOM0XjEK6++SjYa8eDhRyAM49GIvfkek9GE87MLql0bIdcyoFNFCLGwURKyPAHvIUhunZ4iRODFi+cE6xmVI7TUZElGIyt0oqP03j9DT+CXYk3+L7pCCBBExJEP97+SUa+/riu6ponKwfGXSeRgwDi4EFkbxUmbpo2NRQ8gcYPb70sfwWQQZ3BD8Oj7boAs5zene1EUP9MnlJIw9CLKsiCEwIMHD6ib6L4TQuDRo0csFgtmszmTyYQ8LwiDJXWSphyfnBCAx0+e3EwB5vM5ycBLaLtusOO2MNiZ/UnGoxhEVSGO2rquYzQaMR6PabsW5wJZmqNkMgRByWw2I0sTsjxFSNBakqbJzSQjGrHEG97fMCs9XdfeBJm+72/YkdY58jy/IbPcuXuH1994gyzPsNZExfCuoUwViQzkWmCqDcvzc2zf8Pr9O5wczHF9EwPebJ/R3gHJaIoXkr7rcNaSaIHUEmvbCDFud7Fsco5EKzKtKBLJrEwYZxLlWoSpKVTgeFZyenLA0fEBo7JAK0GSDH5/pqNuKpq2xgdPYAiEARgEZ17iQl7K1Xsf6/qujSrR5aigrne8ePGC58+eU9cNSqes1mvG0wnT2YxXX3+dvYMDQDKazhhPpvgAk+ksZrbek5c5fQ+379xlu92yWW/om462brHGcuv0FkVRYKwhCI8PliAsQnuqKsqOTcqCNNEUWcbpySl5muNtYL2qKbIxs8mMpm4jFX+wdP9nBYHPRCYAMRu4IZwNXgOjckyiFflkRFNF8cWsKOKbGkRFIcQGXIi9gKhIFB/TDzezGPzdXzbw+sHyOppaRsUarQVSxpvee0+SDTc9kGUZ49GYtmk5O4t+8D1RbOTF8xc3tamAKIiqouqLRPD1r32d3lje//CnpFmKCbFE8cOJ7JzDWoOxFqUUqZK47mcb9WVAeOnN+PL1FEWBMRaNilzzEEjSSHXOfVSneYluRHo88XnsELxevjc/CK5IFRWXX4qoGGMQRHy8dw6VRmfn/f193nrrTQ4PDyAEvHeRQu0dSZqQKkHf1AjTMRnlvPXm6xyqknFywPm65otf+iJH915lNJuSTKYEIXHek6SaMk9JZMD3Da6vaXYbujb6II5HBZPeI5MoDpsoqGqLEjAe5Rzuz9jPPUWeEJSgSFNGZYLrJd3Ws6ta2q4BUpJUIZ0cSqOXaXJ8Ly+DAj4SnJzp6Zqag+MTdrst19d7GNuz3mzonWWxvEamKR8/eBA1LYCu71Eq4eNPH1C1LYv1mr/9H/wHXC02dMYwLqPFmBSS+WSKty27aoc6khwfHfLJp5/igyPVMROwvidISd81JCryYvzAvbi6uGS1XOGNp60sXWNwJnB5sUBIPxwm4ga9+vPWZyQIxAJVyPiCvA84Y8izlK999StcLxY8ffwYa4d01XMjxyxEVBQWSmJtrJ+FjDgCqeQN6EdKOQQLjxt8CZIkjamg6ZEyJQR3szFfbkAfAqPRKPYc0jSir4zBS8/tu3c5Pz+jaaLJqR4soossw3Y9venprWGz3UZ14xBl0pumJvj4Wl+y6MLgGSeG4KaUoigKlFLUdX1TFmit2Ww2g8NRIEmKqN4j1TANWbCrKiaTcbRL7zukEkPt7yJSEgbSir8xQpFB3gTAACjh0fpn71fKDtP3vPHGG3zlK19hOpsiRMTpe++QgzWb0joyP5VmNJmgOGRkNcKM6XXBm1/9ItPDA3RRIJMUY3oCgiLPYxAIlrZr6Oot9W4dJ0MukGpNmSWoBLJUk6pAX8bsYFRkcXwme6SIwhtZljMa5YQUEhxtF0VJjSOWRnqQEhsIji+zoZdZICFE3r611Lsto/GYw/053f17TKYTirKkalquVxvWux1VXdP2Pb/5zjvsNte89+57fPjxJ6R5gfGOTz56iEehMk1V76h2Fb/6q7/Kd3/vd9iuFgjg1ulpJHLZyIxMswQhJW1rhh6GiaAqE3sXwTkefvKMpmnJs4Q0EVxeXFHvOqTyyJQIroOhR/bz12ckCAyyyINKsJYh+q5L2JtPefr4EUIKppPJwP5i+MC4kRaTf0JqTA26bCHIKAIqJVqrQbNAo1S0JyvLkr7v2e0qwJOm3NTEYvhdgiDRKd6DkpogQclAUaZMJlMW19c0bYdUCuuiSCXZAESxjp9++BG7uo4SUL1BJRo7oAqNsQQRGWTGe5x1N3LpaZqS5/k/RWd+GQQuLi4oyzJiGaSmKHMQis5YkBJjzQ3hJM0TxqMRVVUTeksIg0VbCPHLR4ML4QJprtEpsc8yHJA+RD+CEAK3bt3inXfe4c033yDJM2DoTRgblZaHr13To/KSotQouUN3kuKLJySzA9K9MSLPEUmCD+B8QMhYbo0yjTAdptnR7Nao4AaacZT9zhOJRpKnUCQCyMlTRaYliZaUMo36e0qgk2GcnKgYwLOEQE3bNRirAR8vPGKQMY8iJFJqNArrY38H2ZFIwXazZDqbUxY5xk3YVTWWwJOnz6nbhslsjpSSN996i76t+d3f/UPOz5fs7U1xPjCdzVhuNhhjODo95Ytf/CLvvPMODz76gKP5lMPjQ+7eu8PzsxeD/0V05tKpQtgenEMIh+0NWnpGxZTgJcZEGzlV5KRZwmZd4YynnOQEbCyZ8bS/DD2BP8310v77JTeAEDVVqs2aJ48f0zTRw88Zi1axq25tH8dYQtD1/WDlJHHe4q3FhRDHeQMI6WWNLaW4+X40ihtpu90Opg76RjD0pQd8BA/ZwaMvNuNiOl6yXFzTd2YQpnzZjPQ0bXS+lR4uF9f01qK0xgeig4/S2OH0fmmEIlXEgnvcDSbiZQYzGo1u+gFa6wgeGXQQu6pFiMiahMiVcC5mH85ZJpMxs/kM6xxNGyXU3RAEGPoNznuEkJSjMSEENptNNKsIgSzNKJKMsij41rd+ha997SvsHx5AcENGFcuwRKUEkeCDpnMC0oIgIeiUtJyQ6oJs74AwykAp0FE0FhkDdqoKUunxdUU/fE3GI6azKcascAjKPMXFj4WQScCSaYmWIPEkitgFVwpJiBMVHd9jWZbotMa4GuuBIPDOD05RHiMsSkqUEiig6w1926C8BxFod5I8y5BAU9d89OAh49k8Ctb0hrppuXfvNl3X84ff+yM22xqlBb31mF10e/IB0ixjVI5YrVZ89NMPOTo4xHYNf/7X/jwez4/ffTf2Y6zFOkuqMnQqCdaBhuCiS7EclLnLIkNLOZDqQAQRLfakpHV2uIflTYP7563PRBCIMmKDeUiArmkoihHVbhNlv3VMM5d1PXxQUBZ5VBEOAecdSiusszeDEOcdYVCKfXmSvkQgvmwU+uAwJo4FhYivIU0iECkMIidK6cgKHJRcjDFDk0yw3e5izTw0z14+r28d3kZhECUjLFgoibOOYGx01hleZ/7S5spahJLIof5/iVp8qbc4Ho+BOEF52bTLsox6W9P2kVueFxlZkdN1AWOi23KaJTRdExtiIUTVIOsQMmYbIQSwljTPuX///o1Ii9IJQkBeFBRpyltvvMlbb73JyelJ5PDfNK4CUmms0eikpDaQT/fx6YjedwRdkI5Sgi4QaY5Ic7wIIAUixACN1mAMru9pthuWiwVd05Bqxagcs00b2r4nHWzl5EAOCs6TyJgxKkEkMCEi/fflNR5KqyzLyLIcKRucsxHJN7BRgx/ETIgZJoASkX/gjaCrHS5Y8jwjS3KctVwvrllvK05OTylGJe+//wF13fDhTz/ivfc+IM+z2MPKy8jwcy7KfyvFfD7n4uycjz/8iOkoZ28+GUpVOD+/oKoago9TMGN6pBLIIHAikCSKRKvBcDdqMIQQ9RikkBRF9E40pkMrjXdD2ak00P3c/feZCAIvazIlJM5Z8B4liIafeBAq1urWDLTiyNzr+p7eGJSKzDsRdUfjGjwAvI/69d1AVY6fcRRt6PueZkDGxR6jG6ycBT4wdNujZqAcmI5pmkVfAx+13EO1w/Q9etB1F8Pc2RoTyxuiLwLO32z2YEPMDFwgLTLUULcJJenbniyLH4vWmvV6jRCC8XgcNwwxqL3UQEQJbPBR1DRJUIkmIbrXVk1F03WsNhuCh4DADrZUSioSnUavOyFRSmOMpWk7vA8RpGQd3nn29/Y4OjpkNpsxnowH8cpormpNj1AZNigIml3rmB+f4FSOJSMtJmjhETqHvCBohRjkw0SIiD5JlHI3VU1b10MTuKet6sEPwEbEooBkMHiVSIKUCG9RBJSIsuRSSNygxhszPn8zOtZao5MEpEGEaG92ozsZXh4W0V9CD0EkOIexPdb1dGXJaJaSKs1sMuNytYpmrC5mYG3T8fHHn0SPS51ytbjm+PiEclQyn8/4J9/9LtZafuPXfwNnLX/n7/y/edjs+MbX3ua9995j72Cf6+sVIQiUjpD11DryURYBWcEzGk8o8pKmamnaOFl66T1Z5hlqGKNbH23j0ySN7Mh/xiDwsxEEBn+BJNe0TU2eZ3gXfQBj97yPIpTeQ3CkaWz6tW0UEdFJpBwzBIHoqyeR0RAo4gsINE0TnW6Vupkg3Ci2hICSacRq1y3JWNP3hq7rI0hGSoQIfyI1NyDCcBN41BDI0jSl7yzWB1zM/4mvJo4ijYkjS+883vlYXpQlfqivdeJvRoLxJDCkacp2u70ZYb1UPNrb2xsynzCMguINgIgeizpJEEPvQcqoq3BzkZDDKR6FP4y1vP/+B7FMAKyNAKldVUUSVwjcuXOH0WQc5b9UZEG+3EBCpphNQ9M7bo3neKFxMsEJjRUuGowmGWgQIRCsIQSL8B6cw3Yd2/WaZr3BWU/fGa6ulqzXLW3bDZ37aLKqhYpiIB4CPvoTuFjSBEXsDxFt4QkC0zusCzf8Aa3B+yFTGF5/vM/itVcqjuFFiA1r27f4LtAWJUpnKCE4OjxkXTdcXFxRVVXsF3nPcnGNDIqmaXnjjTf49V//dfb29xBS8vjpU5qm4a3X3+SnH7xPtd1R1xuePnuKUJLsaRTSuXV6m9ZEyrEg0oSDiwY7RVGQ5wXL6w3rVR2DuI/SbS4EgnfD/RV9MYs8J/ifNYB/3vpMBIFoChqtnwVi+IBj469r42nQGz9QbWM5IBUoFcikRiUJq+12aOy8NI8Q4GPqKGTEIMSJQrhh+PnBueglaShISVmWtG1706V/OV3QSg3KNXG0V7VVbMAZO4wb4yZ07mcnjxuAOEIqsiKnLErapom1v/dxO4rYXW+NIQCz8ZhExbr/5TLG0LZt5BpkUVnJDUKSLlhciPgK46KFaz+wI411ERVpLCK4iLS0wxiUKMcltbppEtZ1TZ7nAy3a3jgrd33P8fExd+/dIUk0wZlB/DIgxNBbUAlN1WCMYzKNmolBKBxE1JpSBKUg2BsTUi3jGC4Q6Jqas+cvuD4/o68qtuuKzaaharqh8Rtl+aUAQYi7lJcjMB/LvsHIU6jIqgsv90cQN5nQS5QgA0jIOx9LAhmGeyKOCZ0DJaNasTc91hna3RaVFAOrMwabumlxNrpPR/Kbpd60pFnK/XuvkCY5T588Y7leUWQ5qUr40Y9+xHd/7/fYXK+4de+Y49MTemv58N2f4HxgNtsjNz27qiYZmLVlltP2nqppaDtL3Xa4oXcWBsORIERUSfYWqSRaaUxv8NajxWd8RBiIqaWznixLsX2EQjoXbZS8j9bVOk1IEoXWgjTVCDmmNxbjHFIGhJAgI2mDALbrb+bhSRodWqQcPAjccMMMGYG1UYSkLEc3GoX7+/vREce/ZAnyM1ciY6ibGqSMGASt483cdjcMtjgWhDRPyPIcPwigeBsJMT9TK3JRG94YnCvRMtxMPeII09yAhtI0vRkd9n2PUAEhozJwLG3iWG+z3SGlYKIia9AYE9+jjXBr9bI5aAfkIFAURXw9LmrtCwGj0Yj5bM7R0RHO2RsnYZTABYtUEX4slYybMEA5GkfXICLoxgcXhT+Fx5qWdrPC9g1FkaIzjZIyOjg/f8GjTz/FNh3CQ9u5qOXnAsh4MvvBSOSl4vYNGFZEGTERXuJExIA7EbHpGlwElg2CplJER+vYzH0ZyLkJDN4FhBaARziHdBbfRyVj3xu2yw1920fthSSCpkzXxGDcW+bzParNjv/P3/m7XC8XHB4fMBmPuLy45B/8/d/i/NkLTOt4/bVXefvrX+F3f/e7XFxdURQll5cLfAiRVq8ybt86oap2nHXP2e4a+m6DFJrxZITpLaY3sc80lEFCx2uKg6qpML2JBLlfsD4bQSBEHnuqHXmWcb3dxJmwUHTWRWUhnTIaj5lN8ug3EAyjMkf1Pe16E/HxUmEGSKyxFkRE00WmYUqaZpg+puMvG5FxEhDLAobpQZT4Ukwm0xicgqXvLdZEKXOpYg2dpiku+JtpgrHxJM6yDGssTVdFI5NBMKTZVYyKMgaQuiZLs+H9EzUPNhvqpkaJEZPJBK012+2W6+vrIU2NUOPpdErTNOx2W7JMx1m9eQmMGsatUqCUHv4d1Y2EjH0JHwJayGGs6RCCgWeQ3vRSlNAoAVmes7e/x9XVJYvFgpPTk8HazeGcRaeRaCSDR4tAphVJniGcRUgbxT3wIBwEi2m2bK8vCc6S6TnkGqEVbV2zuLzi/OwS2xmKrIhBW0mMdcPJHzAmxGlAEj836+LpPQwcBtyFwgvFy0AGEufA2RADCvHaJEIgkFjropy3jH8fm4UBj4yqPc6igVRKMqURoaetG/q2xzhPkqWkOiHYOH0p8oK+Mzz45AGPXzxnPBkxHk0wfc9uu+X64gLhQcuU6WzOp48e8umjaFvnBzNV5wPOGWbTebRLW16zXW/Z7Xpa45iMSvK0oG16un4DBHobvTjyLIkBs7Xxv85j+1/idEAI8RDYAg6wIYRvCyH2gf8QeJWoLvTf/2cpDscKlnjji6gYmyYp5XjEcr0GYZFpzmQ6YTxKaesV3sSJgE4UOpEUKo/yUn0fTwAJ0meMRiVSSvK8QAhJr0ycAqTxRuj7Pjb5spREJWidUBQl3nmuF9fx1JAqAm8GTwLpY/BIk5Sm76ODbvA3/QOpNMrH5t+oHHF0dBwputbdlDxSRI+C3jpEEkU2d3WDaVt2ux3T6ZSiKG5MUV5CiLuuuznRje2RqRz0GGJ67kKk7uZFSaITjHH4EANDkqYkSRoDoFJonbyUQCD2HAVFWcb6uI0a9l3bcn5+zna94s/9+q8BIebJzg5gJ4UzLvoJiECZaWSqwHSIZGja4cB2OGdod2u6aoeSgLcwTHGqXcVus8V0BmfB6Ti9QIL2HryJEwGGaZLzUV7Lxoax1oFEiajtKFS0mI8pJj685JVIXprTKqlIlCRRCufiOE7KyJkQAB6CcHgb8NaQpYoiTZCAMxFnoqWmNS3WGIoiJ89y+rYmS1KcdXRdz/7ePLpWLZfszWfMpzPOdxX7szlKZUgpePD4IUIGhITeGJLBBVolBYf7RwQb6JoOJfWA7oz/DSL2fWIjO8QDRAmyXMcSCUmapZFJ+Iv5Q39qmcBvhhCu/sT3fxP4RyGE/6UQ4m8O3//Pf9EfJ0nC4eE+r7/+KrvVNa6vGY0K9g/20UnMBlA61j1DI3A2ncVZqo9KNV4IhNKxLveeVCT0IRqEJmmKUpq2bYcUMcRZ6zCnf+ljOJ/NB1BOPCEvLi4Yj8dYY6iqKjrB1k1k2iUaY1z0IUhSgnMooaPteR9xBaPRmKPDI/YPDjg7O0Moya6qsF0fBS5edi5FnD075wgI6jrOlXe7CqXkDa4BoGlqkiR+bFEpqBvSYzVAiyMMN80iLXez3t7oNMZsKKFt26HUCAOrTsSGU1lw995dltdLLl5cxN7H/4+5P4nRbd3T/KDf2632ayNi96e9bd7KzKrCxkYIBMYWCBDCYmLBgF7AhBkDmgEDPEEIhBggBEgIMwDJIFnYCBDYwkKis8slV1GuvHkz895zT7ebaL9mtW/H4L8i7q10nnSZrJTOkrb23nH2jrPji2+96988z++ZJ/7oF7/gB599Slk4cpJSPAb/VHWk5CHMKBJ1VaCtIoYJ7cBphUqRNHUMc6A7ydpXaU2YZiYyvjsz9iMZjbGPJh+F9wlXGgmUCYoYRtEIsJjOlsNNaYs2Vg7CzDJzQQayefkLi5ZD9B6/pTbVojT1fl7OBwmR0Uv+gcR6zeiqxhrNOPQcj2LLLauCcVm5+vkxX9EzDRO2cPg5sNquGOaR92/f8erFc64uL7l++57f+93flbZOG66urri/OzCOj+EhDq0069WK/X7P+/fXsjoceuZZKFMoQ0YtlabkEIxDTwppaSUWNsdCbRJc+Z99/WW1A/8k8I8tv/5ngH+ZP+cQKArDjz9/zqcf7XmrzmxXr0BF6trhbI2rSmKGu/sHUAG7Knj9as84jDw8HCmNI2b5pjulGH2SmzSt8aPIWYd5ZJj7RSWXiJTkpJlm0Vrv92vW6x13t7dM04w1Gh8jry4vuP5wQ0ZTVDVdN2CTptCZjJaSXmmUtehKJL7TMNM2DevVCmsc53NHPwyEFBhnzzxNVGUpPalOGKU4d4JMr1xNDqCU5XA8UxaO1Vo0AjEFQvTMQXp/lCLMSoamWrFer5Ypd6AsC4xWaLVMjmOisBprNWVZMM+BGCJVWZGAsqhpqoamqDgqRVKZ0Uus1/F0T9v8mO5wR/Iv0FkxDT2uqRB5TiCpQMaw2mxRaiIpJ5WZrglpAh/ox0Q/W0K0dFNg+nAmeM/D3R3fvuvoY4MqtigEd5aTxxhwVmThOWpQiYQMja2xWPsYVGtARXzS6JB4pFWhjBwOLJLynNExYuwCI13aFFk1ZHKIKGckAFdBIuEJJBvwauT21HF3uiNQgIWkMj5F8jQRw0wKgbKWVKeH48Ann79h8jV/+IcfeLi/59nlBcYathd7fuenP+YXX/0xaU7kAFVREWKmaetlNgW39/ccTg/000g/zEyDX1o88XugoF21oh0IXlbTy2Ys5UjX9+I+tH+5M4EM/F+UUhn4ny4o8Re/RRx+h+QV/j3Xb+cObNYVLy8VpTnw4pmladbM80RMHh/OrJpE3dQ8v1jT+RmfEus1XO5WXKyNRC11A8dzRzd4eq9xpWXWKx7CiFKJKXZUDXTTWZ4ABRhVsFptOB0GmnqJsk5JfOxGKgZlRHxiyoKmXtGPEeNKAYK4EmsL2SYohdWGQgvlaImM5ObmlvHdgE+BlCNaKXSp8DoQkpSgWllQkcJBWzfEKbPebCirinkWqtJuv+Hh4Ratk2jIyXR9h1EXGDR+nCl2FaPvCfNM6QzTeAZmCid9c84ZV2Q2puF06lm1K6qy4XQ6MZ570jzzR6cj3k8knclG0nyMAZUG+sMNOfQS/3Y8sCufQU4olUkWsq1ptjugJxUFQa0pzI6gR7LO5KJGhxXD6Wtu7u85nh64vrnj3dsPHO/vmc8zZa6odAF6IKYZnRcxstbowgIzMUdIkiblnEMvk++E2GZJAUMQHYISt2DMkdnPEAI2egoKtNWgPIqIVRmVFSomtJV2IWsICrwJ+MLT64EH3zPknpEErsBUDj8OWFdCUhjn8HnG1Y5tXfPJJ284Hg/8yR8pvvjVLyF5sk78nV/8XfYv9lx/uEFFzT/0e3+dfhwIKXFzd4eyhruHew7DkYT8O2JGqMsmL9J2udGNU7K9ygqdNCZajFUM+Ux0IjYz7V9uJfDvzTl/o5R6DvxflVI//+3/mHPOywHBn/r4U+7AJ28ucl1b9hcbPl69FEiDXjPNI8fTHcbCalWx3m64vn9A24L9ZkuhHfUbiYLu+5Gb21uOXU820i+9/+Y9thgoGstH1ZpkIsc+YJ1jt7tkVW/YtBd88+Vb7q47umPH0N9jrAxlZOAUeP36ghAGnM0Ym7i8XDGMR2KcF6GSyDyrssAYT1kBzHTnmQxYK6q0oigY+p7NZg1IsszFxU62IFHaom1T0p/uKFxgt11xf++ZppGhO9HWtXju0cQ5EUNC2562KZgMhHBGMWPN4+rSYbSnLN1TiZkjMlwzFc+u9pRlDXmQHtJIXLgxitoVBB8Zp5mqbRmnAVcY5nEgo2X19MhvULKitFbAlvI1O+acMdpSFhUxRZpyxRR7TqcT79695du3H/jm7Xs+vL9h6jpUmGksXK0bGqdF3r9ImmWVq8l5sY8vmwpJma5wrpB5jIcUw7L686SsUdpJib846XKSzYWIr9RvTGgLJzIs7YqyFlJEK4Mra2xRUtXQrhRzn1ivNlxctfz8538oA0lkSDr5yGa7fUq4evf+PZvtinEcePvuHdY5rq9v+Bf/xX8RYy2f/eBzfvrT36GqKzb7Hf/s//Z/x5/8+lcydF7eyxlwVtq5sMxirHX4JCvS8/nEOI04bVj6JLz3NE3N3d2Ji4tL/tLwYjnnb5afPyil/jngHwXeP+YPKKVeAR/+vM/hCstf/4f+CrvdjtlPbPuGzXbNhw/vCLFntW65uLgQSKbKuLJFA5fbPRrN6XDm9ZsXXF5dMHqPz4lz17HbRFJu2OxXrPYth/4Bbd/QrlYURU2YMnW55uULxfkA776yzPM9SsH9vXACnz9bs9nsmMYj3Xnk+bMVb95c8O7dif3FczabLe/ff2AcJ9br9ilq2y/46KauKNuakD1t23A6abyfWK1b6qri00+ecXi4Z5zGBXYZKMsJ72+Z55a2UazaFd4HVquWnCJGL16BnCHd42xDXWn8fId1hot1xel0QKmZ16+uhCyTIlVRkpJi6CesUcCZnDxF4cXK21Tk7BjHgckY+i6jG8dm2+DDyO3dDQ8PL9hsd5RlJU8/ZcnZExfZ8/l4JE0zdlVQmpKQFit30xC94/bmlp///Od8+dXXXN/cc/dwpO9HwjShoif5RF1oCluhEOUbLLZvvUzrs6g3fQqLK3Lp9+2iW8hS+aQ4k5Jk+aUgWgOjJMzT2MVluoiFUOpRYsSSnYZKCZUyRVXi6pbRZ7phQhlH3ViyUuz3Oy72e06HB9F2pIitSlwhLcGXf/RHEo7jNMY5DqcTP/nJj6mqii+//DWffPIJN9c3/OLnf8hnP/icoqw4PBw5n860qxaNJkRpbawWSlb20hIYY0k2Ef28zLvk6/fRk9BMY6CpJeW7qdvvvP/+oglELaCXQNIW+A8B/x3gnwf+s8B/d/n5f//nfZ6ydFw921I3FUOfePnyAqUVqJmqFmJsUze0qxXNekXXzzzcPbBaN0LMLTX7/UaMRTlz7nuGcUXxeUvdgLaAy4x+z2q7IgN9N/H+7S3n4y2F1fzuX/khn77eUNeZh4d7/ubfvOWHP/yIH//4Dc6VrFa/zzh4yrLm5uaGfjD8I//Iz3j+/AVffPEFXTdQliXv373nw4cb+n5gGEbqGnb7mn5MkAc2a8vtzYHL/RWr1Z7nzzcY1ZGSIaWAn05stpkQBvr+RFk0XFw8p20uyNlIT6sMQ9dTNzXby8Q4jFxdXYCSHr1tay4vLW3b8OzZFYfDAx8+vCdnjzEWrUeUGhnHAaUaynKmqmtWK7ERhzBQb9ZY7TEriyYSwsiXv/6Cj968Ybe/pK4LtBJoaZJIYXCWeZ6YTh2r54KEn6eOAtFX/PEvv+H/86/8Df7W3/pb3N7e00+e2UdQRnwT2uLDyLE/Y02iMhCTXqTNcmM+bkkee3Z5gj+WunITaKGwEb0cBpOfmL0mxYhG4bTBGiObhpiWfz+gf5OObZSwFEgJV1Rk5bg7nPnm/TXZtNh6w/X1e4Z+RpGZp0EoSwq0Nsyz5/r6hg8fPlBVFcMoZp7Vas1ut8MYw263Zxon3r//wN3dPXf3D8ze8+HdO5y2iykLITdFAZ/IZumR8yDZjWTRqBgrVcE0jxSuICfRrVSlozt333n//UUrgRfAP7fALizwv845/5+VUv8q8M8qpf6LwK+Bf+rP+ySimpvZbq/ouwMQxCNdWdarF4yTyGS9H9luN5RV5M3LFzjrONw/SHqtkkjnwjhinFitd1TFDmdhDhPn8cyq2tCYhuvbW/pO09rn2CJyfz7he2gq+N2ffc71dcv7t3/C7//uD/l3/SO/z+3tPavG4mzJD3/4E/7Vf/VfYdVOfP7ZC3JKXF3UvHi+JYbI2D9AWlNVz7m9u8Nax8efvuHm7pqvv/6azXbFdv2ai4uGnCOn4wfWrebFy5e8ffstQ+9pmx373QXznHi4P/Hq5Qs260vu78/s1iuaesXFdsPFxZ7NVU8MkbatF16Apygsz55fLUj2gd3W4uxE3w2igUBRWMdud8GbNx89zTTmeeL29obLy5LLj17SnwemrufDt++o6h0hzuQs0tqQxFugVJJJukLasBi5/fCB5tNAtCKYHrqeP/jX/w3+b/+Pv80f/OEvefjwFf0wETKSpKuMDCyNIqfIoR+wJuM2rRwAyxVzXNgJmsIWKIXcLLYUE5hRZJ1QtiCoQJhFRelDwnslVu3lJuJRKuy9zAEeLdVakqOUEiehQkxko890Y1iSfyMqJmII/OqXf0LbtIR5ZNXUWKuZs+F0OmKt4Opc4ciz6En2+x3eS/BNWRbc397hvWcaJ5xz3NzeMnQ9RV0t2ggwiM0+BLE7K8QV6PxMURYLjSsTougmjJkpS0e1xMoXzvH+3XcX43+hQyDn/Evgr/0ZH78F/om/38+TUuLu9pa2rQl+BkSmO0/jgpYqmNTE2A+EACjLar/jw/v3HI8n6rrm5uY97WpFVTdAorCObfuSr778lofDifNw5vmLKxqz492vvqWqVkKknUZsKDBpy4cPX8r6T3l+9IOPefFsy6qxvP3myNdf/gnPn72kLhTPLtZcXPyAwgYe7g989PqK9VripC/3LW9ePuP16zd8eH9NjIEpTNxcz9SF5uPXz/j44zfE6OmHM9988yWbiy2ff/IKw7z0uCU/+tGPaeo1X/76G55dvcLamvdvbwFLWTacPn7Jfr9Dlx8Wk0zCOYP3M11/5vNPX2KM5nB4IJNpa808eZq65f7+gRgSH330MT/4wQ95eDgQgudweOCLL8TZePXJ5wQfefv1t1RW8YOPP+f5i2eSlKQNMSRsVmhtyWkWWKmXf//XX3/F678WUFZhtOVXX37NP/8v/Av8v/7GHzJ7hWMkLGKcfvb4mCgLR6oK7CJCmlNEFxZjxaeQVV70EglrzVM0nbMOZwussURELut0gVGBsRMTmoiFhLisFIsZLOJjFrVgVou6UWYFeiFWkcWEo7Th4Txw7AbQFqUNKUWePbtiHEeMilSFpSyEVoRxHI49SmXRnziJmReremQcBzabDePYS/pUVXN/f8/x/oBVmtIVGKUF9PqIBlOK2QMuYczC3ljk0nMMiBxeYYyiKA1l5XC5kNVllorgu67vhWIQRMH3x3/0S8qqpKxqpmni/fv3aKV58+YNztWUpabvB06nB+5vrvn2m28pipJnz55xOBw5HB8Wdv8eZzUpGG4+dDw8dPRDT+0iD9df8s2v7tjuYN1YVDakyXG6H/HTQJgVNx/esdu0GJ3pu9NiWskUVvH226+Yx4711vHt22+4u5O1j8qBoTvx0etXfPzRx7x6+ZrT5x3jOPKHf/SHjMMzPvv4Na9evcCHifvzgdcvnlGYzKtXL/j49Uu64z2b9ZaiKFhVNXVV8uqFyE3rusXqhLMVxjjUmwvatuHt9S0hyF68qQuKbcvtbaA/H7m82LNZ1XjvWTc19cUlz66ecf/wAFmx3WxZ1QVxdjjX8vL5BZd72cxUF3uMdmQ/8uPPPuP5xQs0BRmY/EyIinoJXU0pkYJHzyMKuLuRSPmsND4kvn37Lb/4xS+4vr4BU7JywhEIaMZpZJg8KddYA9qJdyTkSFJIUpOW3X7+rbogLy5PHwI5q8Vg5tEWbOXEOqu0QEtgQW9njFZPKVPxSUIsO/dHKIpCiUjJGOqqwK1XHIaBYVpcq4svojCGZ1c7VIyEyjKNA5DJ2bFqazJZZiUkVq2E5Vxff6As7KJuVDR1CVloSTHM1E1NjB5lNW6xRKOWf13pKKzFR1HDOmexVok+oy7YrkqqoqCwFqMtw+CpVyWHw5HtesNbbv70jQd8Tw4BpTSvXn3ML/7wF8xTQuUHDscDMUb2+z3BQ86aoqqIc8fY9YyTsOeqqpShVyVWXj97op/pTgfm/kvmuSPGke2mwerE6XCkNECYqAp4/uw5P//5HzF0B/b7PdM08OL5c2Y/sWolJbZtaj755GP2u0u++NUvUUrRj3c83D+gUHTnjsPDgc16xZvXH+NcIUEl6xXrVUtT/1V+76/8lK47M449b999i1Wa7CN1UXKx3XF/c4dVBqcdV/vnpJR4uLvj3bt36NcilFFE6qpAqXn5f9wJY2+z4Xg8Mg0DVisu9xfiPYiJoRuFmjvMNEVDoR3PL64AuZHOh4MAK3SkbRvajz7icDgyWcVms8b9+IfsNxeo6MjRSOk6zWgrYS0ZeRrFJI5AoxRdd8YUJWEZ4A39SNf1i+GJJzFLiEHgHd5TRMGlx5TJMRJiXGTYohrUWmOCJgSezFMaMYkFIskHJj+JczQGSqMWJoIEfhqbQXnJuNAyZExZlIIiepKVICiMNcL4qwqqpsTWJa01tKsVPp+xzsoO/nRPUzfsLnbEMHN3myisYcrVk++jH3qK0kLyIkVvaxSBh/sbUvT4eaIsStqmIuWEMdDWpfAWtByIy9wSV4i2ZPKTMBlUpi0LCrPm5fM9bVVTlwVj35GjojvP4q5NAr7le30IoFnVWy52z4gxUtoGpycqZ6nciugVzpWo7ChdxcvnLxa3mF6SXcJCw4lQllgjarSg7ylrzxQ8q1VJzmeuLkvWqwusLbi8bLi8KPjVF6NYR2koi4q2abi9vcYow831Dbvtju1mi3MFCs1uu+Orb255dvWctllRFiXr1Ya2XeN94O7unpvrW9brDc+fP2O/3TGOPeO5w6B5/fwlzy8vOZ8OlBtHd+z4xS9+zk9/+ju09QarS2xh6U4DZEXhjCTROoWxnvP5CAOcTifqesN+e4FCy0Q8JaxxS4ISODPhmgKrSnabHU29JqVE153wIcjwC8l3KFzBdrslNpmqcOx3G3btFrKhP3uaeoWztYA+y2KBe6SFaCwu0Kosif7IPHn63FMXjk8++ZQf/ejHfPXub8mO38lBEFNc6L/gg2eaNIREoRM5l8QQRd9ARhmZ6MekZGPgWfIoxYEaY6YfPSl55smwW1cYV6KNA/TimwgLg9IIdTpl8U6gBD7ixImXtYhtmqambiuGHNhsVrx+85L87VsympTgYrfheDjgbGa9aiW4ZBwJWrB1Xd+zah273Y5pGtnvNux3rRwg3ZnCWurCsFnvnshR/dhjaiFHp/yY0ShzM6dljWxtYg4zsw9oAtt1w2azWijcmamXKsHtdozDwGrVEr7vZCFjDHXZ8vmnP1jw2xGyTJeneSKHSL2rqIqaXAWMTiijGOdp4QbYBdwJVhuC9zjrKCrNFDJttrStJcVIXRbUlXAK5ume+4eZ9UZyDkIwrDdbpnFgt9uLf7uqePXyFcdjzzjOvH71mv1+T9NYVqsWpTR3d3c0dSuruwWTHWOi73ti8KzahmHopMwuS5wTrNlDWZKzUIA/+ehzfucnP6Nd7TmeOuq6ZBwm9GuRDTvnqCpHWVkOR5mVKOXRGLSyrFvJUTwcD/g5UuxqMcOsNXVdYbWlsEJFHsceY0pyUoQs7MKiKJZQTI01BUVVC+Rl8epbW1DVDU29oSwLbOnQzghmzMvKMRvZsz/c3XH3/gNq95zSWF6//oh//B//x/nimyNffP2WxCQ9epaSOyaR+k7jiLJQN8WThj/lKE9rLVsCcfrJAfLoEclZE33k3M/MfqLQGWcN66YEJbkGIYoTNcOiFBTjWs7i+4BEMJmQIeRMVRas9htsoen9zGpVUdQlEJknsX2/fvOaX/7JL0lxRKFpm4IYxB05WVg1jrqp2O/3dN2Z58+vnuYab9+9JacsTENjQBna1mFPkio9+1mIQY++Ea1QyVMuDsG6KgnJEXMgTD1jB0MURWT0M9ZpCespJBw3Fu4777/vxSEActJtNjs2GzgeTzhXyYsx+2XoVWF0QVU0KOWFkxcmjBMQ6Gq1oXAOP45MQ09bN9i6ZfQJawq2m43Yeq3jeHjAFY4QA9fX79FasVo1oFp2uw3ffvsNV1eXwngrCoy1zH4CpXj2/BlN01DWhq6TtNf1ZsN2t1uchDP3D3coDU1boEwiRFEtrtdCLX7MOihLaRu0Vnzy8WdsNhtWqx3jOFMUJVVVs16vljI2UDcFVeXYrDdoI/kJlj1KWcpSUGbjKIaoVbNZtgXgbEXbtJImFAJVpXGu5Hw+4oPg0uq6wdmCYZgX2XFm6Caiz1hT07YrdrsLUlRLqYqEfcZAjhNET/IT8zRyd3vH3e09b55/AkqGcc451ts19v01KYwCXNFSySkjbAVLwmkoXUFVFLgnkpKg5FMWkvBjZoJYgRVhmhmHiXMv5e9AYNWKLkNZR/CROUbm4AkpLOIbublY2hNyxofIFDxFLmgrUQMqB01Z0a4qtvs9r19e8uH6mq7rWbcFP/jsNe/fv2e9rlEgN7JbCVD0+TNpsdoV0zQwzxMpywBXI98nlKHvRzGy6YyzGef0U/6FD/5pjakTtLUBbYlIfNqpm9Epk+YRP3usNtRFiXWWMabFMfvdQBH4nhwCKSXGcQDUkzmmaWoKV8ju2HtilKGOqxoUkZhlf1s1FVlBW7c4azmfjlhlWLdrgqm52G+wWqMzGKByjuwNTdsQUiDEd2irqcqGsl5JuovOrLcruv4MOXN3f83D4Y623bDdrTDG0axKSUrSmhADTdswjj27iy3T3DPPE7v9mrZtyDFCWujFyNPPGBhGT7tqhCxcVmRgGEXr3XUnpmlgtdqzv3jG/f0tWoFzNfvdM8pKwlaTLwHFer1ejCICXy2KSuTL2hFDZp4iduEKFq5g9gPOOXa7/TK1zguJWWNtQeEaQhTsttEFTbWiaVq6blxyH4AkZCdlMswJPw5LpWO5vb3lI6WlLC1rzl3P7e0d4zRRKDFKPd6ERhu0gkIbSqtwyuDQOG1wVqGM+i2LtEGrR7efJsyZ4GeGcWSYIjEpdIZ+Svio0LYkIZbfKXimKJgyltwBrRUpqgXIkZlCoEyJoq1YXWxxleaylrSk7a7l/sHjlOdq36JMZv/xC9rG8vLlq9/EmSVH3YgD1BrD/mJP15349u23aGVxhaNyz0k5kZKiH+fFoJbYbYXp8Ehz8gtPcpom1lXD88tnGOf45u1bvn7/jlVTPCHxCAGjFaWT90BZGoIPC/Tlew4VAXGlpTjzcD4yTvNT8Iiw86W819pisEQvfW/ZNBR1IYQfG8Xvr52U5sqSTMWm3VG6gqnrMVkAEbvtc4xThOh58eI1rrDMwVNUFejMdtuy3a5ABfp+QClYrSuUShSleYovu7i4QAClE85plC6wzrBafcrxdAASMY4YBXVbQVYyXbYG6xT7/UZciNZirOF0PKKsYrMVOGVRavEbaE1VNbLnzuXCmC+YhoOkLTsBjUiUWoHQZQNpYSLKJdRkpbRMlBUUhcM6caDNc1haApEYo2uGYUZbhzYV1paAoahqXFmgrFmUghM5hWVdpbBFSXfu+IN/4w/40T/072bV7mnWaz7/8Y959vwZX7+7lnI+Z0JI+JhRylBai7NWeARaY5WhtAXGRHHz5fwkGspqcQAqg9YCC/Fz4Nx7fMwUOtANnq6fl0m/IabEHIMMG7VCRYNhcSGiFuKaSHR9DFSrlo9/8CnttsY4/eS4nPojn37yiouLC7765luquuT1q59yefWclDNd3+Oyo2kb5nmi73uuLtfstxXWRBTSyoCEw9TtmqpZcXd/z/l0ktlWzqza1SJmipCEFbFvV1xdXmKLkqJQhCTqRVdUgKI79/jJY41lGKbFYRJpqoaiLL7z3vteHAI5J8apAyQauywKnFWczgfGYURrS+EkeLNQBpJCGYPVjhRgHKV/Cj5hlP1NVltRkBe5aV3XqBg53N3QtjXC5YPNtmX2I2Pf0W5bfPTsLtYYl6lqyzCKdfby2Z6hnyQNJsHYJ1zhWLUNxknmfVFaIRwZKMPSeKKwSlOXMqCqojzBxnFgu90zjWLJjWFe3G4eYy1161B6i9FuSQOyGOPQuiHPnhQcKZR4f8IY6fPFIpyX/y9IZSXf/MLJZDnMM3OKWCeZAuM4LfZVS0yB8TzStiu0Lok+UbYVxpRYU5Bixi6WXUF7RWKYCPOITgFjHefuzJe//ppx/QvOp45XLy4prOPjjz7iZz/7GX/y62/ojgPJCxPvaTfvHIV1lM5SOkPlHJV1QFwGiAgYBXhMjso5MQd5SvbDwOkc6MaZttRsN5F+nCksoLV4HVMiLf2/tAPyGskrJm1O1gqfI7YquHr5gssXW7rzgRQ8dVVxsfsRZVUxjAP399e4wvD5Z59QNQ1FVYm1egySbKQS9w8P7DbN4uqMWGvwwS9o+ETVrinqhv3FhtPxyP39PTkJy1K0C4nSSZ7Crm6oyxJXVlxcbvnBjz7n1I84W1JVLV03cHt9yzzN3N09cJpmjqcTq1X7BM39s67vxyGQEt6PdN2J9WpNXUsakPcjMYXFC51Ji4hkHmaMM1QpgdOczx1lWaC1EQOMdZDkTRPStAAmYZ4HhvGEcxHrNHMcqV2FLRKrTUm7qun6SF2v8GGmbkoSa8Zh4vJKbthpmogx0537hcU3kYnEDOeupywtKXtS9mgNVVVilWLoz+SM5AYaxzQPnM+GnBR1XQmbb73i1N9wOt8Ro6CxyrbCe8/5PFFVLQon5iBlKdya7U60+X3fEVNivV4TwhJkssBR59mL4cQsdOGc0MZSOLuQmBIpibV4nDxlWUEKhDnhNgXaFgt7YUlmnjNlVWAMArZMksYcg2c4nZinaQGYCKrMak1T1/zkJz/h9d/+u/yqv8GHhF1YCSCiH2tEzuu0EfLRIuxRC2bcaP00Xyit0H3HceR4PHI8HJkmx7kbsGqpXLQcyrBAUVm4iFpYg4/OSh4Vj8gPiSVQEjRTOKq6pC3XsHAoYkpMU+Z3fvIjjCu4vLrAFA5txT5t6rxUF4mqclRVzTyPXF7tqatSchqcbACKumVOCa0STV1Q11LVrZqWGMICxxVykc0ZnTJFJWa6V69f8827D4yT59Wrjyhtxf3dA2TFw/HIwzDy4cMHdjvhZPyP/2f/pz/z/vteHAKgaasthpEU4HToZCc7RaYxoJJDkzC6WIATiXGYOQ4dgczkZy4u9lRFyTiNdKcelTIualxTMZEZup7T4UDwE+mc5XCxin4OPH/+DB8mwqjoD4HVuiLOklWwqq9wecAP8mY+9mfp2UikOTCdE6vNCqWh8zNkxdh5+n6gdA4THdnAPMg03KkldSdphvPAarVFZS2WXWspTcu5OzFPsleuty0P5yOEJBWAn7m7fsc4TlxcXnD1YsPDw2Gh/FgJpzCOwpbM08y09JuhimzWm6VG0FhdkgtQs6XrOxSLWw9RA6ZykK2BjpTWYRMo5ZnShEfoN2bWuDGgh4nQn5iGCNWKj370Qz755CP26zUqO2Kqybbl2asrPv70Ge/frYjxjJ/TExDEFJbkFMmBqQ1FpXEm45ys8gAKZzEadJaEpMJZpmz54nbktnfMMaGcplqVFKVBG6n2HmPFSApNgaIkZzkAU1ILX9AsfoNAkcSjkZfKp90/oz+el3Wb8CPKsmW3vwSEd1CWVlqBxeVXuIJx9FT1irpqmSfFfnv5BKoty5JpGpY0qkipFVVbU1mB4EiAaIEr7BMOffYDRiHQ0Jjpu4nCaD7c31Pakh/+4IewW9PUNRcXK5KG7rMXkkwcvucxZMZYVs0OZyZSDkJ5yZqmWpHCgNEF62ZHVdV03QEfJhSG09CT4MnNVpYtcxEYZgkLSSERp5mzD5zPZ8ZxlDJMGc5DT1GUFFnR95njcUIFjx+gizPT7IUr0GiMtZzO58VKO1I5S+McU5rJPlBoA0bjbMU8ek6HgRgyq6JBpwLvJ1R2EikWJVsgJyFlWlPQdwOHw4kYM9YUhFFjVEVpC3JQEMAgII95Gjkfb/niiy/Y7ffM4VNOpzN11UiO/Ryf4KohpCdSsZ/9gtxaJuFzYPaTJBGHQFNXgABOFZpZDxSqQCdPlUFNnoBHmRltEwZFnhJ0kfBwJM8d4xw4+Uw3e1ZtTZomjC4JWaNcg6ss+4sVu/2eOEf6MEhJngXOEVUiGoVyYJ2SSiNL2yFZjUtqdfBM88TgHcfeczPAh9FQpJFNZWjaEmXESpwTkKN47VmwY1GTlBKbsdLCI0iKLHxVUogEHzBKo7PG6IJhiASvMVYSrNP5geAzRWGYxgFFhODJBG6P91hTcD6P1OWWVRsZB/E99L0YzZQyjENmGI5sVgYNhFmMWGGembzAQbURgdboZ1T2WBImGYyWIbLKUjmNXUd3PjH0HeNwIOfMetOw3wgnIzn1Xbff9+MQkIw9xzzPOOMWJHimqirZ/ZuC/cVeAkems6CltEw8bVlQtw3GWMqqILY1Tou2eoozYRFh5JyXF19RljIss9bSti3DMDL0A2phB87zjAK22y193yNWVi0EWoRXl2N+cq/NPhDnzDCMyw0nkVaCupKhGwoKUyyRVcMy6FwUePPMuTuTEYfgw8NBnGbW8fAgKCvh6kXGruPi8pKb21vmeearr74SpNgTM18m/fM8U5Y1j9mMIURurm8Qjbl5CtrwQZKM7AIxLQonGw8fUEkxMlOlEYJixuPLiFm5JezVM02DCF905Hw+8/7de96+e4sm8as//mMunr+RgaGz0hZUFbv1muFwJoyBFAVOSZIknUeM2uPXLGGxCEUppsUvrxknz/u7I9e3ExG1KA0DlSsorUblCEkxT+Hp+y9A2CWCXClIApZNSdgMRssNFVNkHAa895zOJ2Yv1OqiLBbGhH7CqmldMJ5PxCiJUX0/MI4dxsjhez4N3N3eE6OhLAUndnFxwbmblpxMwb5HL9yIEBNosQ+XVSXUYCXI/DAnAhHbFrSrLeOQOBw6Xrx4ASiOhwdSCsu6fbWg2mUIbJ6clv/m63txCEi8l0yuh6EXhLU2BL+UMCrT9yfO3Ynj8cD9/R3nocdVJa6qWG3WWCuDw0ykqCxTL9FbjyBGayXa+3Q6MU0zq9Watm1F2tkLvWcYzwvWSabxu90eAGMUGUeMnqpq0cqQVMA5B0ZIw+fzmX4YMFbShHOM9F2PtzMhCjsuuEwIonCxthARR0oLRdjQ9x0piXouLVPk0/lEVcka1BYOmyKb3ZZPP/sUHwL3h1tRMi7sdOfs8gaVQ1Qt/MAQAvd3t0sOgqw1UYqqqhfpdca6AusKfIgkLWTePoxYfcLgUKUmBI9TTnBfZHyUr88PHafTmQ8f3vP65Quc0fztf/1v8nt/7R/G1SVWK+qyYNXU1KVo3GdryUYUeh6ZK5DzUy6DJER7yko0ECprUlLCH8wSfHrzcMDHkqquaXzPqnZUVmPIkCJ939F1HfNCmX5kTOYs8mSd1SLKkaBbk4XXn5bvqT1oikLaK2sMx9NhSTUWDqEPMrS0tljox5mydBJyYlb0nWeapEKYJs/V1TNWm4Lz+YSxUunM88w8CyOgrksSoLxHISj8aZqWzAyFMY4YMw/3B66vr+m7iRcvXmGtoOvbVYtzlu12jfeyXnyUMH/X9b04BFJKdP1I01TEDH6U3AGFoixruv7MueuYJ8/xeODm5gMPxwOXz68whWO9aSnLUoAbKEprGacOV7c07Upy/haoZIyJYRjY7RxN03J9fb3AQh3OZNl7awtkzufT8nTV5PyI8RYzS1FJAEjICW2F8lJVUJaF2D1zJsx+SY6RN0UMCq0dq1WNtUuU+DwJ5tzJdL6qKzbbjQzVUkAZxRwmvJcBqbGK0/lIu26EIFwKHcfPctA4555SlqqqXoRWjrIsF1mxHKyPT8a6rinLUsAX1hJ8wDoL2aAwhJAY0kRdapqyZk6T5PRJCohwHzQ83AtU4+72ht//vb/K1J/56uuvebi95sXHb3BGs9ts2K5WVNZQW4d38oa2VcWUZqLyQvONsiWaVMYqGdxpZchZMc2RaQ4MHs5z4u440KVEs11xUTXsmoLaIX8vRvrzifO5x0dFTurpAFg2goBUd9IGyXuxbmo2283Te3OaJoGRGEVMgWEU2KxSinHqCTFSVkJpLjKE2ImDsCkXj0eW19JHqkZTlpmUDdM08nBzR+kcSzADrRXyNIuOQqqUgA8Bq2S1entzy93dkfu7E6vVhhiDaA/qAmNkiDqOw1N2xeNB8l3X9+IQyBmGcRYbsFKM0wQKqrrClY53H95DFqHIOI3EmGjbVgQ2KvNwfwcqUxUFTVWRoyErwWqnlJmmmZwzTdOw3e6YZ884TkzTzDCM7Pf7RQc/yRrMarSBrjuz2WwlnUiaSym9Y6JcVczBQ4qUdY0ylnk5dVWGyhVo1JLVV5CVllRhYygr2X4AfPj662W1qGjahs1my3othqB5nlmtW4ZhYO4n4hilRRk6Li4uSCmx2WzkKT+KZkA4DLI3ZolrFwVgKSGsSXIdH1uHsiwXBZ6VfbsPtKs1g09YXeBMhQkFxsq03wZJ/JUVq8YVcsD148jt7Q3T0PPq2RV//OuvCaPm7sN7Xrx5idWKzXrFbr2hcSVNURKKkq4bMMj3LmtDacU+G2LEe6hb4Tg+pkFNc6CfI/eniQ/3HafRM8VMUc/sryq2taUy4JBQ2HkUJHjKhpyWbMIcAYuzZskhWHInZKNLu1rx7NlztrsdVV2Qk4Bbc86LsCpxPg/LATEK1suVyyHq6UOPNQ5VCck5xhkwrDYr+v6BfDdijGYYBw7He+qyecoenOaZ/cUlSmuctVhXoJW4LU+He/rgxU9SFDx7/oyL/RX7iwumcTGKzSPr9ZqUxDiVF+rwb7dZf/r6//sQUEr9FMkWeLx+APy3gR3wXwKul4//t3LO/8c/73PJ03lC6wPD2DEMI26h1CgU0yQZgtYVNG1NVVWSiacSs5/op56+O7PZrDAqMY0J5yzTPHLuJ7quI8bIZrNhvV4TY+L29o559jKHcAVlWZGip2gqUlIL6ltuIjlFlej8YxCKTJSboKwrnCvxMZFnQYoVziHJN4ayqAlJE5JapsiSoqRU5tyd+PLLr9hfbLm6uqBpaqyzhBgkdsxIdl5ZVSitOZ1OWOfQ829mHSkK9jzGyGOc2uMcQL42t5TVQSLElpJYUnAFoe4XabaUsBatLaWpMMlS2JqqrCms6AMkEFigGlprCd6oSlJOfPGrX9HUJTnM3H14S7l6zv31O8LwY2wj5harRTdRaE3jCrpwYup63KrCOUPhNM4ur73WOFtitSVGz+Q9k0+MXvHu7sg3NydClkzCGDzrqqItFM6AToF5GiEJfz8uqcOCCpBAMqMlmEM8CUrWgW3FZrtlvdksceYa5yrWbcP9/T0pZ5ySmLhxHAHoup638f2y7swc52uuLq9o2y05S3s3Tz1tW9H3Z46ne8k4cA5XWMZ5wi3EZDFThUX9KdyEtITdhqrFz/MiO6+pyobVak1V1xyRNW/GojVYZzifOnKW2Vbb/iXgxXLOfwj8dXlhlQG+Af454D8P/A9zzv/9v+/PlRKHw4H7+xs2m5UIKhY4hrXy5Bz6gYeHA029Yre/pGmkTTDJU5el4J6jTLz9PElf5iwp2aeEoK7rnnLaH/v4x5SfnD3T1LHZbLi7O5LiiNaO0+nE6XRehpRx+eYPkDJVXaGsg3Gi64al7RCYZ4gRi9ycX379lq6fefb8is2mZZ6gXYsbr121NE1D3/dst2tijDwcbjmdTmy3W46nE2UpPe/sZag4e8/9w4MQZkG2HEVBURQ0TV7EPoK4eowfDyFQN9UyzBSIxjhO9ENPTpmyrJfDN9EPk6xoxyD9bila9IyXft17iBJvrq0FYxinmQ/X1+y2G26u32GJ+O7E4eY9w/lAYxq6h3vOh+OS3iNtRQ6RcejxaaZIJU6VYMuFIKyXFXFiGDr6eSTbgs4rvrk+cneS9inNM4WGykJhElYbYvDM47QcarIeZLmx0AarNUay0ckYtHU0qxWXL57x8tVLXFGIurMsKcqCqqlwvcPP84L6Fi+DtY7dfg8o7u/vWa1W+DmLwMqWKBTbjWKeg1Rc1gglOEWqqqKpVzzcnyjLGmvdEqNeLAPGUazDs5Tyu90OloNbsGoIkdsLSn69aYGWm5trrNuJVfx8pmkaSbD+jusfVDvwTwB/knP+9eOA6t/WpWCeZ6Z54MXLZ4Dl3J/ISdJ/jXEcjodlgAfrzY6+75+eeptmTeEM/XAm5bjw1mc5LetKBms5c39/z9u3b3n+/DmbzebJoTVNE9N0FrKQsUxj4HweqaqGaZQ/U1U14zhJZFWGjGacAome3Et46GNWYPaRGCKByNgPfPP1W27vDozjxG63pqwsL80zqsrx8UcfU5aWb99+xTzPGFssIApELNL3aKXk9Rllnff4Cu+2W+qqwi46+sf+f7PZUpYV4ziJFXU5BJpV+xQ64n0gjRMh5mVIumKab5jmwDjNNK3ocxNgnMMUC8duVszDiHU9RVHjY6AbR87njrZp2K7X3F+/py5q7g8nHt6/5eHtt1jznNPDA+fjkaHr8eNMaQt0zkTvGQ4jejKEsUKvWtxKzD/zecTqxPFwB9bw/KNP+dCduDtNjEFRaotWM+vKUtmM01loO34WSpV0cU8+BbcoTa0xWJVwxqCNo2hadpeXvProDS9evpR0aR+ockEInkRiuxeEXIwJ4xxlLRSm/f5C2itn2W63bOc17aolJyPDRl1wcbEFMoXrRJ7t5ClvjKMoJP9yGmf6vqdpWoqiWKApHj/LBkch1WVVVfR9L1sLIzZuY/XTDKHve9brDdvt9t9yKAj/4A6B/yTwv/mt3/9XlVL/GeBvAP+1Py+CDES/fXF1SUoeVzqmcZbeaL8l58w3X39DyokXL1+Ss/RHYoRxOCf9+2OGXAxRtPNh4nx7w3qdxU2mFIfDA7/+9a8ZR4mNds4xDD11XXM+n0kpcTwdqMoVt3e35HRPTpo3rz/CGklzEblnJGclh8LiSV9v1otsOaGyou8HjtOBeZxp65awUXSnM+NwZrNpaeuStKoY+x6jajarNRqYhpG2blAo+k7yCoMPUtKHRbizpBe55cmRYnp6Y9R18zQHkCj037QHwQdO57MMsIpqgWo+DhItFxeXklbkHP2poywaqqqmaCq0Etuu6uD+5pbudOLF608ZJ88wzaAUn376CZC5v70nxYxDc/3Nr/njv/v/par+KmpRFhojKjqbNRrxCUzBM3YzfhwIXU/qN8x1hSWQ/cDtzTUvPnrNq9cf8yfXPyeoAu0yKcO6Kdi1BZXNGISRIBj3hPdJBoLLfWCMBqMwKmOVBKmWTc16t+Pi6ordfsdqvaJcDtecM+M4Mw4jTdtQV83Ccaip64aqqnDO0TQNH3/8Md57Lq+e03VncQxaR9/d0zSOTMTaEgEoG4bBA56iLMUBuKRNpwVP771HK+n/y6J4Souepon7+ztSeqQLGeq6xTlH13Ws15vFN1LIOvJ85ng8fuf99w8ii7AA/uPAf3P50P8E+KcRBeY/DfwPgP/Cn/H3nsJHXr244NmzC1IK9P0JpRXzshZp2xWXl1eitnMFVb3m9uZ+cbs5zuczisw4DXLTVAXaKMqq4f3791hbst9vFpdW4Ec/+oH465c0HmO0rKFKx37/EfvdJb/85a+5u70lZ0NOmpcvX/Hhw3s22y0g09Z5cTY6pSRSTGnp/Y4nrJG4K2ccQXkudjtevfqIeR45HO5wRou01sI8TqQw0zQlVhvGOUCWfL/gI2S/mIQcbbPCGktZlVLeKfBe5hWP6cXTNC4JxHHR12f2+z0oRdfJuixnIelI7qJImZu6JbjfJCBX1lA4iSbTpSPHQBpHzscjN2/f46qS3cVLymrFerPj+csXUo11R4ieuRso7Zbj3Q1//Ad/l9cfv6S9aNmsNzx7/pJ1vebm23dSvVhLpUtJXiITfeZ46NA+s2lLVNJSOlc1/TBzf+iJ2aILTQyeolZsa4vKA/M0EiOQhW+YiU9gMgWy0WAJKc+LM9VairKkrEps4ZbNh1oO04gFzqee0+lMWVaEEJ4Gq0ophqF/irM7n88UxZp5yrJlwdDUa6xxeA9Gl/h55nQaF2oyxDRgF31M07R475ecCJlZsDzhjZHk5Jzj8pSPzH7k4UEyIJqmIadMVdY4W3A6nR7vtb+cweBvXf8R4G/mnN8DPP68/M//58D/4c/6S78dPvL7P/ssu0KGaYeTxzlNzJHD8chms2Wz2TKME0VRst5s6PuRru9JUWLMY5gxVlO4CoVis9mSNVT1gapawB3R0zQVRVFweXm5bAtq2lbimeZZk1Kg63q8j6xWa/puompWFEXB7e1bYpJvfl3XHI5HrBOia1nXT1P6eRgxKNkO5MzUj5y7CVdWhNJhdEbrLPJPYyisY54HdFORFhRU3wmAcr/dihqwLGmWwU7hCtYrSSwepuHppk2JJ6VgSonT6fj0sRDC00xBqhiZWeiFY6+0JP9Ok/T8Wht2V3ucK9FFs2ghZMAVY5S5QMikpGjWWwiZ8PoNhgRxwqqMH3ooCsYxcf3Wcn/zgd3VD9lsNnz2wx/hsuJfOf0/pboqS4IH5gnjLFZLe6OVw+gS7TKXF89wpuDbb9/z/uaOkMUdGKaetixZF5kUPdF7YlRkDCihBoOkJGkeb+5HrsgiV1Ri9skLskugaRL3js/CIkySYGS05FFkpI2s61rWrUpR183T5gCk+ri/f6Bta5wrOJ87pvG3NQENPsqW4eH8QFGUuCUzU8RpAnSZpongPfuLPatVKw9AZ6RNSXEJqPXk/Dgfaokxcj7fPN6HT7OwP+v6B3EI/Kf4rVbgMXRk+e1/Avg7/1afIMRAN5xkzddUGANXz55J+ZMyWSnmEJnmnlM3UlcNIXjapmHWmru7jlW5oi1KjLOs1nvmMHNxcYE2mZvba6qq4uWrFwzDwDB2hBCpm4p21S7kHsXNzTccHt7jXMGnn37KNEbW6y0XF5fEGDkchMq7WsvTb7Vac3F5wWotPSAZkg/kEHHaoFMmx8S5H5jnGT/PrFcV49gx+5GmrNiu1jwcZoauZ+h7EhbnKmwhNKCmaiic49mz59zd3VHYgsIUjMNAiqJySznjCvsE23ic/ltr0Fpzc3PDZruhqGqUMYspRzMvIRaME8PomafAaiE2l5sNVhlylrAUs9wwVVHw4vIZSVnpD1xFWa+om1aGZdHjNKg4E4aOpBLd8cCHt9/y8ecfsd1sUZ9f8mx3wbdffsPP/+7PBaSZBQ5TFiWlsbSmoG3WVE1J61bstxW3h3t+/eVXdP2EMi3JzzgFF21FxUwKXvp/BSFCzIq8RJRrwQgv+YgKjbQrWuvF/BSXmypLyIli2ZgIrLQwIhUPwVNUJQ8PD1IROMMw9MzzzI9+9CP2ly/pz4YYPTmLWElpOWSmaaRpGub5SPCJ4BL9OLK7aMldzzRNzMxM48Q4ThRLQvaj5mQaB1arGqUfK8DEfr9nHGdBqAdZ/WptGBZQyaNY6C9NJ7AEjvwHgf/Kb334v6eU+utIO/DFn/pvf+blg+fcSca6IJgCdf2CeZ45Hs+kCKt2Lb3ZJKrAspC1Xr30zForXFFQ1hUhZIwueP78Bcfjgb7vl4ogUVXF04T/9vaGcZQ+2lrNyxev6fuB+7ujPA13G5wraJqazz//jNvbW6ZpYrNds794SdOuqKoK4xYFXcqoAlRM5BBJw4gyju2+IvkJFrFO350Y5x7nDHG9IoeJ4+nIPI/EXLB/80JQZccz2+2W4D2ng8Ajh26iOw3kJFZmVWeGYZA9f0x0XYfWIkRarzdPA1HvvVhqU6IsqyefQYwJcqCqGlarimfPX7BqV7IKTBEfI4fzmVVdCx23qqmuntENgWkMrLBkJevIYYlvd9ZQFZZ5CvgwMZ0O3H34QPSe3W5Hudvy+vkrfvq7v88vfv4LvvjVr4SW3KwkPTkrSlfhTElVrtiuHVeXLe8+vOfu7kG4gcoS40RdFlysa2zsSYTFQaqIPhNiJmNBJZSWp7U2QvDVgFMatfw+I0/7RJbDYimhjdEYJYyGtl2hlMiuh354wr5Zu2L2E9oobm4/sFt/xDTPgGK1qpn9wFdfvUcpxYsXryiKAteUlFXF4XzL7D0XV5d0p06AsePE/d09bdMuSHWHXVucM4zTQNedGIYO6ywXlxcUhaPvZEPlXKZwivNZDibRjkhux3ddf9HcgQ64/FMf+0//2/08KcoKKATPZtMQlp1214nqSQZeK1arNVfPrjgeD+L80gqrLHUlYMfClnifiD5SNfWTBbVZ0nOPxwdevHhB2zZ89dVXfPjwjqurK+Z54nQ6sl2vePXqI07Hgb4fKFxNjPFpIGOtJkb5nBeXlzgJHRRnmA8kL5N7kxVhnBjPHSpnysKSU0BbScitqoKqtqTgUZsNhdUio51HdLFmu90JZDILM9GWhrvbO3a7HefhzOl8Yrfd4ayFJXUmhsg4joJp14Znz56xXst0uGkajBUOo7WWpmmpq4ZUSRthjGW7kfJ/tdmSY2K6eY+tK6It6PqOunQin21asAXTfGYOCYzEwocUcc7KbGW7hjlxe90zjDNKWbEvA+vNhlXzivVqy09/52d8+df/Hdzc3jLdy0GktcElReFqjClR2jKHzP39gdvbB6bRo23FOM14H6lbcXTmuUOVDokfUiQCMSliVqRFBfQba7J0APkxiXipcnLOpMVbEEIkk3BOUVSFiJqMZbVakXOS3j2MKC1eFDtrDod7/s7f+dv89MeJy6tLnNP4oBjnGW0Sxliur99RVStevXpF4SreXX/N7e0tn3/+AzJnCldQlTUpCqTliZfpRF/S92ceDvdSYSwq2O1mD2hhKE6eVbunKKQ1LopCFI/fe7KQgnkemeeJX3/5xYJHFinsRx99yjgEunOP1ha0TL5zSDRlJdbhriPGJIgoWKLIT9zfvUWpgHMF1i3ru8XJpbXsX7331HXN3e0df/Nv/Gv8/u//VVat9Pfn85nVasMvf/lLxnHgcHhgf7Fld3mx4LGU+OuVCJ5SyqiUJSdv8qQFzxW9Z/aDePajlJ2Pqx0UVGVN+Vz0+017wTSLbr4tWzncCmkBrLXkAE3ZsN1u0UYxF/LEcbZgniWTLsa0xI0JUHW324ksetEbVHWNURZtHNYUoAx1uxH0eM483N9z+uZrLl+9pL68kihwazCuYBko4KyDogJt8VNAG8PF5SVq7mEcMFHTHwMPxxMpeMZ+YOh71i8Lis0GPwdW6zWffPIZz5+/BK1FcwAUGCrlcK7EmILDwy1f3X3Fh+tbtC0Q/cCItVq4/XGmsIJHC1nckwm9fF8yKUqPb5bkIZUl2stoLa2BfsSXSf8sv+CJglxQotBPlVNZOpoGMiVdf6Lrztzf37LZbNjuNhxPD7x4eUkmkLIHEpeXe6qq4ttv35Nz4HB4ICYRnOmomecJHyN1U9PULfvtXh4sUVKVh2EgYRkneR9pLcEqXXdms97SNC3TmHi4P9I2e4qiZLvb0jQNDw8P33/ZsEJWaj7MnD6cFtljou/GBQ/unnTsd7f3zOPIN199zasXL9isBKIxzRPH7kyzXhFz5nA60Q93rFYV3gu4UynFr774AoUAQ/th4P5wz+QneVJG+NUff8HnP/gB4zDztntLVTb80R//CZeXV7Rty09+5/f4/Ce/T4yOsOCo9KPizGpUSsRZDEACv7B4PzJNnhi93Pgohm4gpbiUp5qmqRd8tqMo9CIUyaxXG3JOrJ+vefv2LUop1putbFBCYEweYwrqZo0rAttdj1aatm1pV+sn5Fhd16w2G1xVkROE2cshUNZI0kZBigNhmujPJx4OBzbPnrG2lqoopMy2lpwHwas5hykLmHqGTtqn1XrLfHpgaFp8M1NVGo2nO018/dWXvH9/w4sf/R5aW07nI+fjSQ6pzZbVekU/jnTnjgKN9ZlSGWpXcNcP/OrbazqfKNctYcpoldjWFReritpNtEVJtBbv80LREdtxTkm2A4+moSz6B200hStR1qGtXYAmkv5TFqU4TXUg5YCxhk21lpWsNrTtihgDh+M9Shnads3xdKRqaj797HP8BInENA6YRT8Qoud4krWtK0o+3HwghMRm19K0Ig0uC0keCt6z2+6Yp5nDwwNhDjw8PGB7gykUTbumXa3JOdG2DUVV0qxa+uHA3cMd2paSh7C/XB4ApRC8v+P6XhwCKWV+8YdfoJT0suNwpiprjGm4u+0EuOjFG75pdnz1xTVffPEtt+/PXD27YLdfs9o0qDny4fYdp/MBbTTrpiVOibZd01Zrgs9UruGbb7+lKDyb7UtQiZuba5p6yz/8e/8efBgoUFSrinVrGYaZH3z2Kbv9R/yVv/KP8vEnPyP5NedChj+F8rg0oZnIeiaoCdOUFEahBo+KGU9FsHuUFVS6mie0KUhpJkVxxb19P6G14eWbShRqZYVKkRxntII0zTRFgbWKOc4iE9aO2W+XoV/DeDyQVU/IiaK5wpYVZSMmobauRVMwebKXAdocZlm9lgV5GiDMqDDi6NBNQ7lak+bApqxwCXKEMSqyK9A24ownvv8FZu7Rdcs0ZFK9odpfMHQPlPbEtumIWXP99lv+4A++5Wf/zoZ2isSHEzdffs2vfv4LSm34D/z7/v38+stf86//a/8aRcys65JGG4opcThMfJlKhssdZtXS9W/ZlyM/cp4fWsWbukBFD9FQ5Cz5E0h2ITngtJbUnqxR2WIw6OyYR4MtM9rN6Dig44ROidKUlEXDEE6cxiN2grWuaIsd4ziQ7hTtaocyW5xq2OxbxqwIeRRMXtR88c1XyxO6oV6VnM8dp1OHUoaVK9hcrCUiLUwMpyM6qSUPMYHVEutmFVlncHDx4pLruzON3dFuVxSFxjpFyp4pB2oTmFTPmx8+l8/rItmseXvT052lGvyu63txCPjg8V5wYk3TsNvun6bcbdtye3vLMEzUdcPsPavNip/8zk85HO/o+o7L53vR1EcrsVLa4pzsfp21NE2DNoZp6MnZP4EaVpuWcRxkjx4z1jgaW1JVDnQk5Uy7Sny2vuSjj3/CxeUn2EJ08qWGFAMmByxRDEjGoRYKjLIGXZXoAHUJbiV3kU6RHD0pNIRpQjQsinqaMLagWTlimMnZUxSyIsspMo6yrqyMISCMf1vWTMGJI9EV7HZGKh4Qj0Up/804KwfKOBGmER8jpigoqkoqoBCIswwujdGs24bRZ1xVLwRiJyXygrZKM6R5lMzAWSo4a0pwlqosCYW0LlVdC+i0jyhtGBdf/+nhnrfffMNXX37FzYdryrrk6uoKW1hu3r/j9OEWE+V18VEgM8F70mxg1JTGsLY1m1XFbrvFGJk7xKQWLYAi+kAI8urGRRkKsIAFF04hT6YhAZdqqcaWP6a1oSxKckpc7PcoHPcPdyJAQtOsVriipd3UnLt7zsMk6cE+4ecZ1lvqqsaHCPTsdjvadi2T/JQpC4mWGzpHCOHJ9feo+fBelIJlWdK2LZtdJCkDRIpSUxSaaY50nef29oZV2wKKrhslcCSOXF+/pa4bSWD6jut7cQgoFC9evMB7z/NnL1ivxR55Wqal1ha8fLknxUyOcHV1Kbz8psCVVuAbhyPj1JOV5tnzF4vLsMD7eaHRziQydV1xcfUZ0yRBIGSRkpyPZ1b1mhg9D8cTwzhSVhVNs+bi6iVXL9+gdEFaJKRlVuQ8ouIEMZBVAqx4CbQCY1DOQMgYPEZHiDPZS2CKwkEolt5TU80e4xzWZaZTT0ozxq4wpiAF8atX1QpTt0Q02VhMVRNiWijLGm02bLdXQELl8PQmzynhhxEVZjlcF559HDJFU0ns17IG1DlTWMNud0FRN9I/WyECZzLaWlQWqzEhPTELjULoSkZjjRB8H8VKIQSCjpz7gRgz54c7bq5vuLtdPBL7Lc16jSkcz58/J5576CdImaRBOUVTOawrUNqAMdRasW4F4BnDuNibgaiBJWwk/ptBJfwpK7HAS3+jF3hEeeWUxRzV7FB5hJSJybNerZjnzN3drZi3rObU3XN9fU1RKS73l5weOi62O3a7C+Zp4nzqSN6z3q6I3uNcwWq9FkNXnCjslnGcmGcxuymlF0q0DB1lPX3EFg2rpmT2IzHM9N4zzQMf3r8np8Tz5y/YbHbUVUHfPbBaWZ4/3/Dhw3vevT995/33vTgEANpm9SR5HYZxsbnWYgAxBmsLtNPCzJ9mRj+w2W0oqxJ0ptUtw1Az+5mqrshK4cqGjDz1i0pTlOIlKIuC29trisKw2W6YphFjDLowzKPn7nDi7u6B3e6S3eXH7C5eoVxNmgXQaZ0GeogdeZ6IkxB8c1WhVxsoapQuycmSjIJ8QtETolBunU6CHDJCkdEaSm0kGCF2ZH8gzBOTCjhXk3NBxqFci6p28tRVFqzFFQGjJxlwGYFbEj1pzqTgySmQYhDjSwqYBCElIpmyrpYM+xEVPc5o+Tt+pm62KFOIC89Eye6LUZxqj9FgGlmxGdDLiu0xITmmSEwZyQFVkkGIJqEYhwFyXlSVhu1mQ2UNp8OAQhR8QUk6dTZgC8uqKQmVIy6z/E1ZsmlqUhRqsXYanYS9J7Fdwn1ISeLRtVaoxaOfUxTkmFoqByQVKMWI97JW04MgvwoDOSYeHh5EPKVLuvOZaQrcxYgrLdpm7u/u8EEUqxebC+Zx4rCsZq0tuNjv2ay3nM+dzCxiIubAOA54L8gxrS2gF+9AKSKhxavQNA1oRdcfGIYObcD7kdu7az5cv18qwQ23t2KTL4oKY2Q2ZGzgw4d333nvfS8OAdm3Fmw2G25v7+j7ge12J44sH9FKM43SB7dt4tyfmeYJly3deGa92XBxuadqGoZxYBhGYgjMPjIvhp+qXPTvOdIPHeM0cHsjq7XbuxusseSklkmyoWr3XL34hI8+/gmb3UtyNktgpAIVIJxhOpOHkTDOjFG2AkW1wlYl2dbEaNEoAah6US2qLKGYaRqZxx5FWgxASpJw5num8z3zFAjTiCs2uGJLwGCiwlCAXQPCStAWTLFMtKMnzSJDTiFAkh9KSfJSmGaOhxNzijTrlWgRzmemvqMpjUzeUySECa0FkU5eMGpaQ4pLbx1lGKrzUxKRIeFTeMogSFkAGGiDLRQqW1brtQhxcqaw4uev65LVgtd+uLvjdDxIYk7OKKMIJpJVojQak0UeXil4vlmzaWvm6YHKKTAWhQFEOWeMgEZT8Mv0Xy2R42lpCRbtgFYolXmMg5uniXme0ZNFqwjWolLgeDxQlTV9d0uIcHn5gpgS2ii8H8kxEmdPtoa6qrh/eFgOjoLd9mIxhEWqsuLu/oHz6cx6vQZk6Lff7ynLkt1u9+QDkZbFME0Ciy2bkq4/8/Bwx/5ii3WKeR7IOQixSmU+XL/l/ftr3rx5w+R7mnPNxdWemPvvvP++F4eAWCgt3gdCiEzTzOl0JifxC2QNIQjDbhg7EiKT9FEwWnPwT+EhAp7wWOcYp8DxeGKaRtqmpyzsgtaWWKZ+6DgcDlxf37C/2HP3cEThqJsdH3/2CT/9ye/x4s1naFeRU0QXCiwkP8D5gXQ64rsJ77PYX5cSWCkt+v+shVuT8pJAO6DCTMYzdUe644EwDTK/KArR0aczce7wPoFKxODQdktEkY0DU4KtQBlyCJBHuUFzIvgJPw6QAoYoz12tgMw0Dtx+eM/xdGS121E2LRkxNeU4Y6pWyuIcIcyitM8KlOTk5UWma62VTAed0DlJ9SJSSVQKaOTGYuH+pZzR1mER1aNBeATR+6WNMaiUOB+P3N/dcjoeCfOMTQntLLhENmLM0lEOxv2q4bJt0DnSDz3WWFQCkiItWQJyOqkFBiMdmoBj5LzUSkCmjwAZRSJGT4hyaBSFZC3WZUEIgeH8IGlAc+b5i9esN62YrQrL7C05XzHN9ZJpkVivWiFPT5NkU4bI2I9YW+CnmbIsWTUt2mS67kgIYQmRlQNrnucnMVKMkdPpxIYWYzJlZTBazEPPn19xPgtQ5nC4o64LmqbkdLrneJS1pbWJ9ab67vvvL/He/vu+HnkCh8OBqlrCOu/v6fuBVbsiJVitVmitxC6MX2S/4uI6njuGacQWBcZIgk7McD53PDzcMw5nyspRlZYUA+PQ09Qtzhbc3x8ga4LPHA93tKsLPvvhR/z4p3+VF68+w5br5T0V0coTfI8fzqSHe4aHB/zg0aak2taUdY12xVISi+c+R8hxIs8zcRpJU09Ms3yOsaM/HrBGodbCtSd2+GlgnBJJZUxVgB1x7QpTlEugSiIBUSV09iJyCTPz2JHChDNqcTMmSJ6hO3P94QO3N9cUZcFqs6FoWnwQzmBd1xRliQoT0U/4scOmR9ugzCzk3lLSk6Nh6bUTiRQ8wScZlCp5CgtRJ+GDtFCla3j15iPIMPU93enEPIyonDmfjvz6iy+4ublmmkaIAbfcwNkqkkqYLEYeoxXP1y21UQydQEAjggvPXvgHaE1eDD0sPb5Wi1wamVlYa55+CB/gN+KhwlrKoiAbcWA6a5iAaZ6o6zV1XdH3J1KCslxRFgWb9YqHh5l5munOHau2Zb3ePFWxc5rRaM6nDmsc26tnOGvohhOFKxb/gWaOM8ZYSZhajGOFK6ARkVpVW6pyu8w5Ivvthh//8IfCt9QaV5TcG4WfJ9abNVYrrj+85/Li8jvvv+/FITB7z+FwoG0FsJFS4ng4EkNcGIBCBbbWMgWJVipKS9vWlFVNP0+C7D6dqOuW/f6CcZy4vbllnkZ8GEnZM46R7nwixcjUzqxXO2KEomiZfaJdrfnBD3/E7/zu7/Pi9Q8wuiUltZzO8iYf+xP96UDses6njjgG2tbR2oKirFBWE3NYpMMiYldpQIWA9jNxGohhRPuZSoNyBmMUFUni1CZPfxoZ54QuHc5KUGZTlujCgo7E2BOyIpOxWRJ+wzQwzR1V5bAqQQqkONEfDxwf7pjGnu12w/bykvV2h0bjJ6Ho1k2LLSzZD4RxYOpO6MWDnpbg0LywwZNMzmSgmtJi2fXkyT8O3nnMDEQptDWY5Njutrx+/Vqeag8HTscj8zSRU+J4ODD/8k+4Py56fBDtfs74HJn9RAoThS5pS8vlqiGHmaE/oxVECRUiR0mmVoInlnmAWiCiRgjUeqFUay0EX2cN2hnSQgVKMYgCMwaGqWOMicLJatdqQ1FaTt2BaQlpub0bxTg2S0UXo7wuNzc3T0YhgegaonWCn0dxPp94eHjgeHqgqssnwrZbUGplWTEMoyQgZ0Q6bONT69J15ycY62azFbtxWTIMA2VZ8PrVK/b7SyFZn8+i2vyO63txCIDYXV++fIFzYs5Yb9YUhcRzF4WAHZyzPLu6JOaBzXZHVTfSOpwtSmXpL9cNOUfO3Yn7+3uauiLGSN+PGJs4HO5Yr9ZM88R8e49WJev1mrIq+ewnn/BXfvbXefbiE4wtSUmD0uIuQyi4OcnPXjlyUeOspl7vKdq1yIKjR+cZYoaQwEc0soO3KUKMEIJ46xXUTQ050Z8lh87PEqrpyhVlvaPZXtJcXNHuNminSHnAx0gEjDWS8BsnpvFMzl78BGEmpJF57DgcruUA2GzYXLzAVS1aO1KSexm0WGKXO3gaOsbziXLJwIsxoo1eqhuBcubHyT+Pk3e1RKU/4k4UxjmKqqZdReJU8OL1a168eM7hoef+9pbTwwPzMBC953w6cR47puCfnsYg4qvR94xTj4qBQpVsqorSCBtinnrq2jLNgQKDUTK3QQl8A6UlOTmEpzWgNUrWgDk9IbyMNWRjlrZzZp5H7GTpxiMmJnwhevyyKNFW0fUnQhCrcN8LmWkcez76+CPOpxNKweF4IMYo7EHjaBphOxhjKArH+XxiGCemaUBpSVWSSrgS5P3ChBiG4YkYrRScz0dubm44Hk9sNutlAPoIlClklVyVNM2KslhR1wZnBXX+Xdf34hBwzvHTn/54QXhlbm5K9vtXOFcwDGKr7LuedtWwv2zph3vatlogDIGmKXn18jntak1VN7Ju8RPrtmW7XfNwCPTDyKptmaeGsiqpqxW3NyfKouKTT3/Eqzev+fgHr7m8fIVxpayWQHLhlVoINSI1Lcsau7+iWe0otaMqakwh68McJnKOpBhQPmBSQpHQcaZYPO0iEpEeOvqZaZEkC0NwxcXVK5rNBapsqS+fU2y2mFUDOhFDT4iepDJEhUkjfu45HW/Zrlty6EgxEOaevjsQ44wrNPWqpmxbtKmIIOtWZTC2lBI/eVSMDOczQ39mk5J8/eS/h8zzmAOoWG4ypD3IC4Q1L3ZdYwvadk2Mltgp3rx5TVWVvD2/53w6cj4emaeR4GdiCqQgsV+i7F0CG5RhmEdiipTO0haWVVXi55HT+UjKHlNoTEg4Y0E/IrfECeicw8f0tAVQKj/pA0QZvHwcyavMS9UwTwN2tiiVqCuHtlF8/zni48zkR7yPdNcdpSuo64a1W2O0aFuGcWC1Wi0Pm3m5yS1Ky5AvLMCTqiopSkeMAWsdp9OZw0EAsykJD0Ki7OUmH8eBOUj0mkJjbcnF/oIQBHt3eDjhQ6QqG3JW9J1H68TQR+r6ex5IChCTnNZaKy4udjRNLfSZBe1sjPyAKHDGsYc8UjcNlxc7ERvFhCJyebmjaUpeXD4Tt52NzL7g+ctLmtYx+8R28wxj1+z3b/jZX/uHefH8BfXaYrQEdyqlFwr0kl2XRVfijME2Ldk2KJ8wEXRM5ODJKZOI5BzIYQQ/w1JOq5whBvzY4/sOnTykgJ9HUgxUZUVRltTtJUW1xZQ15XZPsdlBUYDVpDgT44wSpYA8yuPMNJ6YxhNu3xLmAZUi09gzjsNySArPMISEs3LzJhTGlZSPT00JP8T7WUr9ha68zBVlr27MsnXIi7DG4kyBV4Y5pqdVoLEFZVXTrtcoXXKee6xSTN2JoTsz9QNj35NClNARlYkBsBJH/lhPpJSYxgkFVEVB5Rx14fBhJCRPUomQhGsQo5LoMKUWiIiAPNUkAza9BNLID9BK5hopgk6PQaeQohdnaappVxWbTUU2AWM1wzASciQpWe/1i39hGAfW6zV939N1PVOYefnyJdvdjrffviOmhLKasihxRYH3EXxYtgNwe3tL08h6vO8HrHVM0yMWTAbN0+Tx6UxVay52F0yTZEUGn3l29ZL7BzEVla5aKp4CZ2sODwcOhyNx89333vfkEMg8PNwtp55it9twPsuTX+i8BdvtejH8nFGLOCVEDzlSVwUp9czzRDYyiGsqx7Ze0587wn7P7B3rtiZxgdIFq/aSF69WvHn9E958+mMKazFqImfR9mPyUpqmRUAi+3GNRhlLtlIeEwJ5uYFQkPMMOUCcUHGWJ2myYot+uKU/3GPw1IW89NoayroSpZ8xzNHSDSOFdTSV3PzKGaIficETc0BbeULnFEhhYu7O1JXDGUXwmRgC4zhJqo4pKMqGmDW+GzDFmmwtGYV2VsQ/OsgAcxmiFc6hVZYUn0VwpBMovSC6I6BBZYvTUu4OKRP9TIgZW5TUTUtdt2gKnBm5v/nA8e6GOIndeB7HpwNFYB7LU3l5arqkCDEwjCOFc6yKinp5cgY/Y+uSOXoSihQVac54LYKrx6+DxWUnDxGN0fppFqCVxK3JWaeegCMyD5DqabttKWtFNwt0VhdCi4o50Q09zUpi2vtuoM3iWZFPl+nHgXKq6MdevBrRkya42F+QEhwejozzRF3V4vI09okA/agSHMfxSTLvvWfyA9OcKIuKohDL/DxFxtFjdEFZZEKYmeaJqtRs1jtUNvT9xPn8vV8RWpqmRCmh2mhrBfjR1MJ7X/hwq1XDPEW6bpAUnRQZh46mXaFIlIXEkjsLMc7MkxwWF7sNd4eRmDx1XVFUK5pmy7Nnn/Py5Y8o6xVpjgiBRsl+Oway+k0gRU5edtdZoSJkG6Qfn2fUHNFKXIUxzygVUETUIgryMfJwuOfh4RZLpG4Fr52CxlbFAkJV9ONIPwqDwFaaECfy3GMKRxikBdBGvPBqKc+nacKPI1cvXqCVwRqY+4mhn9HKSclZrZlCIsyBdiHrqvxYFksmYopZHJFJuHhW3DdSMCu1RG2DRvPUtmcheBhlxfI7jwQfsY8JPNZAoSiLgtPhgeF0IKfAPI74aYIFhS4jgGX1mIWdqKOEyHbnjqqo2Kw3tIXB2IzGULQVadbi5IxLUaQCTmnZnKS0FGFCWzKLU/A3lYDs4PWSRWiNUJMfSx9rwFqF9wPn7oiNIrG2sv2lnwYu6hpbOtwyc0g5YwuHnxRfff2VQGG16D9iSlgn/zZjDcYZzueOwhU0TfN0w4N8T/XSzlhr8f43aPxzd8/Qe1btht32Eq0ttzcH6romBXkIKBxVWaPQrNdb1qcz7z98+O777+/nJlVK/S+A/xjwIef8e8vHLpDcgc8QeMg/lXO+VzId+h8B/1GgB/5zOee/+ed+fq1o2lqm3EHQzJeXe+CRntrRnTv2F3vWq4YUe4ZhIEYvWXVKJuUSihNkp82SL4elqSqGqZCQSmTg4sqSqxcvqVcbcjZPJXGKkZAC2IxywnQjpeWA0hA1cZ4YpzvG7gyjx2XZ8yssIc1kHbFL/59jpOsHzsMZ7TSbzZq6sMR5JC+GKayl63qO44BzhnZToRxMoafWK3IS6o+rhDbEMtRCKaZuJPpE3axhlqATHzJj77m4uMRojXEtOUyiTixKtCvRGVJOchPI8Yf3sjIsikKEVdEvwzUnVRGip09ZCdA1RaKXNGNblKg54OOM9xLPFWJEG0vTVJw8zNNAjhJhnnMii5yQlDNZS0S6UpCN9Mnd+UwIkf16Tds0WBUJcSSqjCodlbOEfsZPkdo4ks7MISxrTNlsaK2f/u0ZYS/kLAeDXjImH8NX5BCQV2OeR7oukc0ISta1KXrxP6REIjGHmSJ6XFkwec/Qn6RNcoJ+M85ycXVF3/UYZ6nbhmEaKVyBcZY5ePqhh5Tp+55Xr15hjOH+/h4hXFfs9ntSStzf3ZGyJuWZeZI0ZWtLSlfjp0xZ1JAFcSbMikYkyMhQsa6+Wyfw3SmFf+/1vwT+w3/qY/8N4F/KOf8Y+JeW34MwB3+8/PgvI+DRP/d6jGs2VnM8PTD7CWuFoTaOPX0vCr/D4YHDwwOHh3uhrcZIDJ7z6QAp4JwBEsfDA935SGEN0c8oEptVuwBAJenn4vKS9XpHygofMygHWZKMU4g8puHG5AlxJqWwlIswdgO377/hdP+B5EesRXb9v8WmCznjU6KfZsZ5oqgL9s8uaNYrsoGowFYlpq4YY+Q4DCRjqNY12EjIE6YAZRUhSN6AQCcVcY7kAOiC8dTLUz0vqbrZEJMmRmjqNcpUKF2icJiiEuiGMkhPsfzaWKSzCRhtKMpSpMbjSJjF9IOS6kMBOWWCj0QfZcCYJevRlSUpwzALLdqHQFUWtG1DUxVMfUeOUm1opZYQ0ix6/hRFSZniU3TWvFhqnz27WijPgcnPhBzBGJrNGm0sfvIQxQAUfBAl32OiEPKETik/MftSijK/WeaPPAaToJ5gnuPYczwdSFlmKplEPw48HB9AK66eXy0y3n5pPSQ0pGqkRbi4umR/ccFqtaJdryWlSmuGceTcd/gQyCRCDByPR87nMyCGudVqhbVWYKZWUqK22y3OFaxXGy72l1zsLtEYxnGmbdZYU2K1o3AlKYjHxs8Td3e3wh9Mf0Ercc75/66U+uxPffifBP6x5df/DPAvA//15eP/qyy1zf9bKbX7U9zBf/PnT3A6j7iioB9Ff308H7HGieJqnCmqin7oGIbI/d0dIQb2F1tyjmSVKXKkMCJQmYOXia9LJD8RvcKWFcl7snK8efMjXr78BGdKxs6TzQyFgtCTZ9HhG6yUyBlyEK03CtI8MY89eE1drES/4ERuqwgUOsiQMwb8PBHijK0cla2etPnjHCiKCls4MgrvE8En2tUKra2sQ2tLUVqm/sQce8KcyLEmeOkRy6pCJRi7Ufh+HonZ0gblGkzZoosGbSK4kqJYQBsxkuPyNMyi/X8U1MzzhNFgNUzHW4oUcfUKlZslwAVA5MA+BAqTyHjU8gR3Swsw9xOzj4SsicriyooiZbrjNTrV2HGkzgqvBJeelEIbi1ZGBpcGkk2s9jW7q4LCavxRAkoyCZLCKEPlamY30uWOoBPayoFgY3wKXSHJ10vOpKwJGXQ2RLVIonOWuULWaJTceFpeE43BqAqna9FHGQ0W1pUQgZPy4ltpRKSmcs16XaNNzbrZ0lQN4zCQY2T0ZxSIzDcpSThqV9SuJCyBJjFFSl2w3e3ouw5tNEPfLcnYEnbarnbkqDGmxE9yYysNp+M9IcycTqK6dcYx+Zn3H77BWQkz+QsdAt9xvfitG/sd8GL59Rvgq9/6c18vH/vuQyArplHx4fqOTEJbRcieh7szd7cPeB/Yuy0+BZyrwK0xLjBG0UtXpWOMM/3DJJPXLDHQ89BhciTMAR8NPtYUqy37y89pmueooClixDCippHYvSXMCV2tMFWNzsXyBll2/zmQ/IBVicvtJxSVRdtISB2JGZs9Jgrw0vuJ5APaWoq6gZgJPpBjImXJuU8hEbzHDwGXLZUqSD6hy5qyaAjTTNcdSbqgadcUNuJDIMRApUuin0hzor1YkaMmRA3aoqsVtt2RygoVA8rZBcYRfoPU8kGefCh0VqiYCNOE0ZkcR/oPX+JyFAR3mMm6FKuuUSRj8CrgTCD6M9aI2i5HT2ENU0zMAeZseRg9HjAuM5zfoXpN1XfklJi1Iy1yfkWB0Zay1JgioWrLs6sdznQc768Jflhcio65T5iocEGgI31lmMol/m02JDJaI0yHlBefvlQzSSmCEucpRqONIWqLUWZRREoWQV23bDZbGS4Gx/P9jt6cmcuBdVvx7duviFPPxcUFl/uWECJGWYyaaYorvPaooAlTAJ15++3XrDZbnC3wPrFRO5wtiClSVI7G1BirSQgCrlyi3WIMdMNZDtnSsd5e4WdFCgo/95JnkCf68R5nNOMwkqPh+sMHsvEMw4FRO5z9S5YN55yzUuq7Dct/xvXbuQNXF1t8iNwfbzFWnvAxRm5ubnh4OLHZbBnHgRCFs37/cBINdTCs1pJk3J+HBbLp2G4vUFh8CJgM3ge63lNsdvzgBz9jvb7EmhIVM9omCBO+65jP94RsKF0p7jmr0BiSySgiaepJKVA2Na7ZonIg5Q6LvIHy5AnDmbh49pW1GO2Is1QmOSbGYaIuSyEFIwi14/FEYZZE4CUGO8dEPwyMw0S1LlitW4zKzCkQgwcyyc80TbMkMidmHzBG4sWLukZbJ+tJLZPxx/xEEfdIbqI1BWZRAYq+XoJb+vOBarWlSYHsPVEpPEaAHNZgjMSbhxBkqEaWrIjF9TZ7T0hZXv8lAFVmGx5nE6MKQKKqHcTMTFx680xVGarWQR7x5wmVA+SAUm4Bqg5kpTBKCzquLFCFBLJY5zBG/As5LwpBZ2TmuKxGRfoldCHr5O+ljHg/tKZpJOOvbdbYqsBYy7rZkIO8RnVVs11vIScudnuUUlRlSXfuKeqGw0OHc5YUJ1KEqigJPnM+dbx8uaXvT5yOnbgSyVRlhVkYF94HycXM+UkvUBYFZVVSVBVKOVIMWFvSthZjEnrhHBqlqCrF0AXmyaMrz+XVBSlpEb59x/UXOQTeP5b5SqlXwOP48Rvg49/6cx8tH/t7rt/OHfjs4xf59uaWsgHvPXd3dzw8dE8QCmMMIUq+/M31LR8+3FHXjq1qaNuaafAcjmcUmtV2Q1Wu8HPk1A2EaWZKimqz56PnH/Hy5cdyKoa0aPtnwtDTH+7x0xllG0yYcWFGuWKRvkoIxDiNspJsanRhiZMk3ioCOczEYWDuexkYKkW1XqNSYhw7WenMnnkc2bYrwjwTk8SfT9NE0VpSihROCLMxZoL3GKVo6xpbVMRxJnu5KUmR5D2rVYu1hmU5JY7BolhItfKkYVH2ycz20TefpTdexDkqS6aBigUxytfl/cRSLMtMREu7JXODijAMIqpKEh1utGUez3Tn7in5SDwEkteQyITsaS9qZhUhKuqyRo+ePI8Y4zFq2ecnRYgDfh4XY5Z60iuEGBZpssKVBVXd4JzBoHAuYjRoY0nL12yMhKnmJW2AZZCMyigjDsmsWPrvmqZd0TQrmXPYErRkCUoRpRb69Za+77FW0oOqtUjMy7Ihlo66qZimAbJHq4Km3lBWFUZXaDWhlRNLeJrZbNZorRl6SRY2Wi/gm5kUI0VRUlc17WqND3Duhv8fc38Wa2ma7vlBv3f6pjXuMSIyIqfKqjqzj3vAlhhsgy9AlgAZIWRuAMMFluw7JKQWSCAsSyCmGyQuEFwgAZYlS2AhLjDiwjR22+52d5+xhqzMrMzImPa0pm98Jy6eb0eV8ck+h3P6oFpSKiN2rFg79t7rHZ7n+f9/fwyR9WrFclEyjnsWzYqcIqVbUthI09TcHl9SVTWLZsPu4S8ngejfAP7rwP94/v//5Zc+/i8ppf5V4B8H9v+gfgBIPTqMPdppFkvJZn88YapKE4LHOnkDn07tnMba0NSVjAy7I8djy2q1wehCPnbqCYcjh4cduIp/5JPf5ONPf42yWEBSci0OgTh19O0DXXuPzgGtvAh/op9VayJM92FknAYKlVFWiSkoTuTkSWki9D2+axn7lvZ4nBNtasYpEBNkYxm7VuSqZJTKdH3L6XiQN7DVFGVB4ax0n2PCaSMOQ+dIg8STh5iJkyf0PWGaZNSYAgY3C6vAKE1TNxLpnaIs/CybaI4B7SR+HcQzL5E9YvxJSs+GIWEipuhFL5BlcYYYUAYKV5J9QTSCfDdaU5cV76aJ/X6PUhpXlkxTIOSECgFbZDCwvloQbKJKGkxFOmVSrwipJ/hIe5hojxPWZPI0Se2emOXLsnkZKwvXOkvd1KQcMVmRrBUdiTZom/GTRI9lRCGplJbsAYXkVkotMluPRdWntaWpF2hrOeyPjN5T1ZIfqJVlmiJF0eBsRd97QvDUdUYrsTNvNmcsmppu6NjvQSnL5cUTmsWSlOH8rGCzPsPYgr59kLJMiQXZzCNNHgnWKVGVmnEYKcsFSjuaasE0RPpuJEWPNomqXsg0xUu/5+Lymtbf40NktVqx352+c/39WUeE/yekCXiplHoJ/A/mxf+vKaX+W8DPgf/K/PT/GzIe/BwZEf7zf9rra6N59uwpP//mJ6w3TynLknGMxPniNk0jSlse5/hPrp+y2iyoKs0wtvjgWSwgRei6gbLQTFOk60dOw8QHVy/45LPfYL15AtmiAW0SmR4/tYzDjmk8oLPCoCnmiYIyzC45CUuN2aMLQ0qeOA0olQRDPmaZeYdAPweNFK4gTp5x8hTVAnxk6gdWq6XcQEKadeFa9P5aUy8X5AzTOEkasDYi0BlGwugJs0Emh4hvW9HrWyumFOcwtsTHSB4nqXlRaG2lSa4UkVEWEBljpNQxOb33MeSsmHwg+IDSmRgnxu6E0xW5WJLnrn2KeSY6VWg9yOefJwXBB7nSzuiy4dSSFfhZ6pwdKJcoVprSlEwBbEjoEEhDT3c6EvwJayKF01glw1bBpYuGQ2lZ2CnLFMCW4rh7RHCnmKQEMm5ON1bvFzqzKMdYIw7sLO8yjCFrTVaayQeOp5bFakkMYggypqAqS1L2tKcDerVgtTpnHCQl+j7t8VOi7ycuNwWFK7HOSZDI2FMWNVXRgDKsVyVFIcAcpxPTKFkDdqYtT9M0U6nlBlU4xziNDEOPNklGjECKkWmcGKcDxiQOhyNhyoQJDseWru+YpsA4TlT1XzCVOOf8X/2OP/qn/4TnZuBf/LO87i/9JYzVuEKy3e4fHni4P3B58RTvw1zTRZSBq+srtptLYhoJMUpNXImgYvewn8dVcqVNGM4un/Ibv/1XePL8E4gGXTboGICJnERjH6YTYWqJE7iYqRZr8dUnT8qQkpfmoFOYwjCFkcl76sKglTTEHmfeYhopqKsKlTImZgqlGbqe5AOFLYghvnfQLZbSadYzCzCME3Ge1xuliT4yhBPGFihjST5Qu5I8SbS3cyV9P83kYnHB+flzmpgAhS4KFAZt+lkBKSXCLzP29ExwSoj3qagKlIb2dKRxC8pyKeNPHwhEmqpBKUvG4iMoH+YutqKqatQsTxYzz9yHUCK0aceWaBOurMhDQNlAyiMxjZyOD+TYsVnVpMnj1S8mGczTCWct2om6zoco9bwRO7HWWizQIIusKAnBz5J0wZprI6M3bbWExjxqhpVBGccUMvcPe4qqZr1aM3gxsfkgITfTGEgLJS7UAG07MQ6S8TgOgbfDa8GPb7Zst1umSaYrbS+EaVcIEakdRowS9F3X9cIw0IpxmohBbldVKbmTVVWTlaLtelIYKYslZelIKdOPwnZwzlEWBeOQ8GFisVhS18xS5F9xF2HKibc3b7l+co0rHW9u3kr9pg3KZkJKqJSoSsfHn3xIjgW3d0f82KN0KbBLP+EKS1UWPOKiVpstH334KZ/94DeoqiV+UpikyTFD9DNc02NMwNqEjhZjNM4ZtE7kaZCZs5WPZe2whaPzHcpmlE7EYWRsO0I/yNw8w6JpKIqK/tSikiL5wGG3lzFbjIT5RBV+XMIj2QfTOOIHYethrCy4mMjzZmCdoz91XFxfEcIkSTwaQhjJvcYkSNmSgGQiSUsdq4zF2Fkbn9NM/3lcWHoWQ2Wsc1hXklxBYwuU1XTdEdtsWRhDePyBZY3WVkRZs9bAaEfKCmMdRV2LXl5pVps14wzICElGapEAzoIFU1psZdBWNtvoJwqjKa0TQlKa1z4SN46WE/wxK2AaJ6Eru0LgKMbNTUExEVV1RdsldEyoOaZN4IIaZTXGivEILbkKMUOYYbWPO2TXduyPB7qu5dmzpxRFTVXWaCXSXD8JlnwaPdvNGc6UAJzavcTcWcU4ikfkcDoRY2KxWos2IAriPZPmgFyHs2buBTixQsfMYrHEp4mub1Eqo7XYibv+SFkp6kVJJjKNgTzfno0tUEnWT12571x/vxKbgHDxKxbLZg7pSHz6yfdoW09RFHTdCZMUYDAGju0JbTLLqsI6xf5w4ng8cn52SVUXxACXF2esl+d87we/zubsEq0MZdVAhBTSe82/MVAWFtUUqKLBlA3NqkFZSwgDMWWcqygLQ0pO3HsKqqbAxMgwDUxdR/bS+Q4xYYqSjKI7dVS2oEsdh8OR8/Nz2SiQhlVZVOz2O4wxVHXN4XggTxGjDDpm8RKME0obfOhQWnM8tpxvN6K7N4E4WoauJakJVyVM0YAVyW+OiWn0cjVGUpDSewSY0IJyDKRxkDBUpTG2QJsCU8qinvwos23ZWrDGzNdnN7cMFdo4XKUovceV3Wy+kpl71dRwOmGc5Xg8onIiGyPaCi0jvyYuOJ1aDnuFNY6q0OSkMLp4L+QhIUk+RSENvbmhGWMgpCQTEuRGoLIiEUX9WJb0w0B6NIUpM1ugE2BQxoiAyjjKeoFxJZMXutXD/Y7FshYNxThyOByEiuw0fS/Aj9VKEHibzYrT6cTl5SXEwO6w5/7+Hm2uKcqKGCe0kY2ubfcslzVlpfGDlDBaG4mEQ1yVj47HaeYBWFsQUoufTkwTgs+bPNZlzs7XLFcVOwt95zk7X7JYLjl1DxwOBxSZYfhLiiH7h/UIITCMPYdDJCQJEu2HgdOp5+zsgovLS1LqaRaON29foShARawr0CYjm3mm7Y5SY9ua7faCFx9+zNWTZ7iqmUnFI0Y7yDJH9jEIxkkpClfQ9QFbKUzhZmhFhy5KlK2lY58Uh9t3OKcEwtFPcnqPE4uy4HQ6ECO4qpHGlLa4ouJuf8CHyHp7RlEU7A8CUIlZXGIoTdv2TGPEKjULdzzX19e0xzeM00SzWHCYmXW+a9HWMPYtWSty8EzRkzHUtiR7T7YBn6A9HvFhwl2eEWIgxQmXpfmao2A7U5TEWuuclAVzEGaIsNE1wXvCOFCsV0xegKnEyDgFhkk882IE0tRNg9ISGV4VgsYqZ6aDtZbd/R1+mlhvHCmNrBY1H7y4ZhwCu7sddbPAad5nLeZH8IGFPHnqqiLMzc6ubUFpCuvou0EckVpIx4+hqzFFrHWgEtoarJPNLaQonf65b+LKCltWGOtIGfw8DbJOc3a+wfUWYzXWWlbLJa4QX8b9/T0hTNzdycL92c9+xrKRxKD9w55hPPHk+plsVEVNjhMeD0pMZNlrXnz4EXd3N3TdkfXmnGkaJENw6ORmUBTsdnuU7ciMHI5HjJLcyM1mibGBV6+/4vXrt5xtr9Da8vXXN/gw0iyWBJ9YLpffuf5+JTaBnLNAPnYtzknwY1GWbLYVMWXqwnJqJ/ox0vUD45BxhSYmifUexp6qLpkGz83NDU294unTD9ienVHWi9nnnslIt1WXhniamKaRRwXcNAVs0VAvGrL3jNNARs0ClAmMot3tGLqeYrsgK6m76rJkeXHF4UGyEZrljO7qR8pmiUlKTo2ixBjLODcL6wb6bqDrB8qyfE+giTlj5pHY6XSi76Weq8uSh+BnutHpfcTYEIOAUaulZAY2K5arFTFH4dq1R/rTgeXCCSs0CZlGZSQLcQZrKBwpSE1cVDUhJJS2OGtJSc36WqErWSNJyD4ErCuorZLmoDEUdUOzWpPblpRlHGeNnQk/iW6m5RjTY21FcVGz2mxZrtdCONKaPI1oImjYbhsWywUvX32LK53cvOZGsUWBVnN894AuC6qimOX/UnLlnKVJqsRQlHOmKCsKowWMqswv/tMWZYtZKh6xrsZqgXluzpZcPTnndOroh46r6494eHjgeDwSwoS1Al7ZH/a8DnfUdcPu4cCZuuSblyNlWbPZnjH5ga7r+dnU8fyD53ifGMaCxXJJs6wIcWQKE7vDjrbtubp8wursjNOh5f7+DcYG/Nhx7I+sNw3ea7746me8ffuK7dkZ/dDyzcs31LVEmG+2Z0xTz49//JPvXH+/IptAoqpLxsnLrWAYqcpI06z50Y9+zEcff8BqtZpxyx3b9TXH0463b+/ROnFqTxRFyaJZyzWrXKCVoSqbOYxD3Hxa6Vl4okhR5rH1Yk30mhwVhV1RrFaEydMPI8VihXGWGDwqKXb3D+QcMHpNCB6jxXjSd3tev3zFkD3nz5+BdQyhw5U1w/7EGCLnV5cUTcN4PJKVRlvHMHmUMiLwsZK/mKKEnpRFwel0wvuJZdOI9LRtCclTWf2e4zcoAZp+sNpye/uA0pazi0umUYCqYZLI82k4o6jNTEcSWpBSWUaWWrwHYT7NjasFgaakWaiS6PxJ4vqzzhFTwMdAXRVUVcmwfyBrTdk0VAsJick541yBVoopT0zDSN/26KwZuoGiNKisKV3FZnPGxeU1Oif293cYldE5sT5b8vGnH3HzcE8CukHKRenlWKxxZDRxDHgg1RWFcXMfIaJmf4LSCuEiznTrqkKnhLJy81GmQBmHdg4TEtaKhFkbRVUXtN2Bu3vRraQI3//B98kklsuG5fISV1hOpyOT7/n25S3aXHBz+5rDYU/MRoJuTyfKsubsbIMPnpQCwzjRtgPWKUKcmKZOyoacKMqCfhzZPTy8pwk3TcnzFx/QdZ7D4cDv/8Hfw4eOuilYb9Z0rWe323F+cSWbXVXTj56H/e4719+vxCaglOLDD19wPD1wc/OW/X5P8HB+bqjrmsNhT1YFxmaKoqQsa97dvOPdzR1lKWq1srDU1ZLt9or16pynTz+kaZaQIXgJ7ZQTL0sDSSdUXWASxFhg3ZJ6swVXEIdWBDHOoXOW9KJxJHovxg5liQjK248Tu5sHdvcHzp5fU6+2dMETjaUuNIfxAVOULDZbsrEkZdCuJCQYfaRqljjrxA4bRbATgwelJE4c6ZkcjjvGoaPvO2onzMUYE9FU7A5Hzs4uyN7z5uU3PHvyAVkb/DAK09APJN+TioIQPTYFrHGza1ba7jGmOcseyOBMScyzUSjLKU6MGCW4bu+lAWWLAlM4fAasQ1uxKmM0BlHXKdLMEOjJMeCKBcx5B217oh966sWCyyfXWKXo+o7SGqqqIOSBs6sr1tsNU/CUucT7wNDLpmJNSVk4dJ676jGhq7nzjzSUT6eTNBRn5FgIER0TdbMgawtFIRuBNhgreoDlqsIVFmsVTW1597Dj1atXrFZbrHF88/WX3N/fi9uvdrS7A5MfMFbz4oNnnJ9fEXzi7ZtbnCtE2ahlZGmMo6pqpnHi9u6O42ng2bMrQhy5f7ijbaX3cHX1lHGM7I4HUXYaQ3toMbahcCXtqccYx3pzhQ8D7ann449/AGqBD4nLqzXWWs7PL/id3/lHvnP9/UpsAimn951QGTNpiqJgGkcWiwXDeOR4HChKhVIN45BYLc4AxTgKQHOxWHJx/oQXzz/l/PwJZ9tLXFGTYyQFYbZhLBBRKqKdIkyJ9tQRRk9R1qiyIvW9aPPrRq6944RaLPD7A6UtUFoJoaeyTIcToRtwxnJ5dsnFtaDJumPPYrnG+oxyJauqxJQV3eQJgCsrhtETQmK5XOPHidOpYxgmlpdnhEnj/cQwTXjvGaaeruswRon0dhpompJh6NCFYex63nzzLeuzC3b7A3fvXrNebxm6jmk4olIk9CdcufwFLIRIjJ5HQFgKMudHKXyUn0fX9XR9QBcL0danPMuFBZElEWcanyJYg9FSk4unW6Pn7jwpMqeQUBiLm1WEU5g47B84tQeaZsn2Yiv8xixJvOfrFW/ePOBDYHO25ebuhrIq8MHT9wN931G4mkW5onIlp7GVkir9IlNAz+pGHx7dhcIa8D7hlAXtpBFqSrQR9WFVaaoClguHM4lTe6RpHE+fXZGzNCy/eflz6dhPI1134utvvmK9XrBer1jVH1JXC9arc75cfktRNMRkePLkKS9fvuKrr37MkyfXuEISlVxREMlszrZk7Rmmln7sGaeR1focZ2vGwXPz+p6bN98Qg6aoVhRFzaJes2hKqvpKSMuq4Mn1Cx52O4q5BA0+Udf1d66/X4lNYBxHfvz5TzieHtjvH/jww0+5fvaU3UNLiJ5Ipu9ODPcntosPKI1hvbpksVjRdgfquqYsKpp6y3Zzxfn2CUWxQCG3hJwjKQdU0OQcsAq5egXP6BOlW9AsRcp7f3ePKUrKWmrulDM2BGHFF6XYnic5SZTSs602sF5taRYrHrqePkSWVcUURoFtrlfYqhHxR9lQFyWnwxFb1KxXZxzTgXEYyEkSlhSZ9nRiGAb6thVFoRZ9+lSWkCN+GvHTiMNTWcvbV684O7/gfLNhaE+slgvC1JH8gDGK6AfIDdba2WMfmMYRZzTOSBxaIqO04NyCT0yDJ/qMq6x00Yki2/UenyK2sCgr8lZTVmQvzH4zC2UKY+bRnZd/v3PUZUkcPTGIZXjoTwz9ie3ZBlctmYYR5RT9NGLqkikn3j3cYQo3ZyhAWRY0dU0KHTpDYSyrZkEfR+lV+ImsZwCSgqauOfU9YRKOgZr9FNMQ0KXBFWK8ykrjioK6LkihpSgdOo/s9vcE7VksK+pqiVKGvpvYbLfsd3sWywpjwBUG6wQR9urbGzbrC5ypWC3O+Oqrb/CTYNCmITH0gWlKbC4uqZaGrjtwdrlhc3YGGk7tid3xQFaOGFv8lNC6pCyWvL69Zb1ecH15xRQG/OT57LNPMNZxe3ekKtc8uXqBjwfqeoGfeo7Hv6Bi8C/7kXPm/v6eaepm0UbmdDyRM1xdXfHzr3ccDgfGsWVVwuk4cti3aJMx1hCDxqM4Hkbub1sWdRQDjdbkGfGVcyTGCdLEkCJTP5ETVPWCwtZobdnv7tgfT6zWhil4TAatLd39AylEqqIUKkxpmeKJMA60pxPtseP88oqYNad+oFgI3dVPXsw8TYWpKpSXN3+9WRNiZOk9pqmpvKcsCvqum70KRjrgSuFjJMVEXVUMbZT8upxoT6f3yOymqjgcO25v3nF+/XRm9nmCH4hhFBNU9ECaT0dhAsToyQEKJzPxPPP5jS3ww4jSmrKyOOek0TaKki1GKQWMlSbcOAewhnHAuZKiLLCFoy4KCVWZMs7I6xTWMk3zNCNlgh9ouwM+XFA1DTFqqkVFP/UoK9TiV2/fYJWirGvag8SgV2VFmuR2EbynsAVlUcBsGMs6Sz6hkhBVa630PObZv1KPpiENyshEwAf6fqQsNSpH+r4lh56YPFMaMMngCsPQe4xVtO2Rl9/+nMPxjpvbt1gH49gxnloe7vY8NB1alXRt5t3bB/LbPZ98+ilPnz6Xen8QJqGrK7wPvHv3lpQmhrGfYaKecUwMXUBrx4urNc8/+JjSnbFcnvPRx59ineHd3WuOx4Ht+QLnFuRkubvf0w3vaI4tm82W9XbznevvV2ITKMuSy6tLprHj1B7Z7fbc3R74/me/IaqwyXN9dc39ww1KCUjhdDqyWtWCGE+a29sdRjdU1ZKqXKICJAJyHjDPXZN09aeBYRgoTYNzBXEKHLp2zpgbWaY0U3ZEl7B7eKCuKlxR4OoaKoufWm5vbjm9u6XRlsV2yzijslfbM5ntx0RVlKLFV8ImJEiJYFxJUTWA5CwWpcWae2JIaGeElluIH6CqxGU2DCMgHf6h7zg/27JoGnLbs16tePXyW9bbc8pmwdC19G0rNJ/sSNELaQeBZ8hD2AQplDIqzOIaEGFPwlmhEGENYRrxBOKcnMss2NGPISFkJu8pywLrLMZYqqoWpV0MwtBDYbVGFZZhGlFkQvR0/YlTeyBrAXPYwhJyIOSAKxwPDzsWTUVd1+x3O5wWA5BSmmEcMIcDdbWUZinMxig9p6hlhmEQEVOhmcIjwFNCXo2RXkBOMI3SbCuLzHKhORyOpNBiysjZ+ZbNdsvp1HNzc8fp2PHhRx+z2z1w8+4tw9Tx8Ucfslw33L56x3I5G9k83NzckSIsVitiBIX4D5QSwdU4jdze3/LN733JxeWasnR0fc9qsWa1ctSLhnEIfP7TL3jx5Am/8zu/S98Ffv7VS2ypcYVi8h7jOk7HAUXkYX/P6B/4/Kc/45NPv8dy+d2k0V+RTaBh7BqOx4mpLzAKqqqgfdhzf/NOhB25pq7O0A6KasTWiZwPTGmibx3Wrrm++h6XFx9R1VsxjZggoZjZo7PB5ozykLqB2B/JeU/UWkQ1w8ju5p4QAsVZQxU1qWs5TAGdMq5YYuoGVTYyWjtODIcTeVlh1xd0y5pu9NT1BUu1YeoGbHGGNxHrLBZLQmFLh46QsmZ1fo1PimBKumkiVgt6NNtmCXZPXWvubu/xMbOwFeMoYRN13XAcOrZmCVrT9h1Pnj7l62++5e3L1/zgh79Of+yJnZRAKWeMqlGpQbNAZS3GqOAJ/YG0sHJjKEQ8E1KiVY6irLHW4WMkDEeKsibHzNT24k5bL4hdj98fcFqzKgsRHoWAMwaMxlYVh/bEpBW5rpnajnEMqKqhnK/fQzdyOpwIU8I6ByFTu4bu2KOUYZoCTnv8EClsTX/qyFiUU8Q4cVQZrxLNwqGSJ8WJFBI+SmJS1ogxx9Uko/HKEExB4cQPYGwBWaOSBi/X9bSoiLZGlwUpHWnvDugxsT+0pH4iD4F+13O1/oCz80tCAqdqxqNC6ydkDK5cU9SAdpxfKU5tyxdf/QGr5ZqyrKnqkqfX55hCkcYlsX9Ke+w5vms5255RpjXrYgtkjqd7bu5vqBrLZvct5MQwvWE4ilR7c36GcQts43Glo7254dzV/N7nX5DajhcvXnzn+vuV2ATE799yf/dAYTWrRc16ueJ0OpBU4nDYceqOEiyxtiQ76+RzQKVAVS3Zrq9ZLtcCnIwRU1rRrs+wSZMNmjg76eL7TrdOif1hx/39Dh+1ZLdpiH5knAKHU0+1XFNVBQpRaqUoCUN1WeMKhVs0mLKkcZIRqJISA6I2knFfFTirGFMSr/0wMo0ToLHGMM7a8bJpMDpxOJ7mHsXw3tI6jmI9ljAUjfeJmBTaWuH7KcN2Tl5q25aUpRYeJ/EodG1HsQlgZENU2ZP8RPATOXq5JCslgM4kslZBjonL7lFl18+5emTpvOcwcwJTQCGk4+gldBWYGftxTpa2uKIQrsEcExZjJA8jx/0BP3mWKzmxjJYMAO+9jHZnK3FZlExmoh9HYtYY55hiImRJQtY5kdXjOFA4CTlEYp6k+WdL0HbOGWBmKIp11xqJH5Owm4BPnmVdoXxJGHoe7o6Mk+fq+gnnZ+AjbLYbqrrCuRo7lyOH/ZIQJrZnZ9zf33J1fcGpPXF3f8uPfvTH/PCHP2TRRG5u33HqHlhtGwpX84PPPsP7THvsubi4pFk07Pc7co5cX12yXFSUpeXdzTtIgYe7W96+fc1yucTVJe/evSVbx3g8sN8/UFaJ8+0579684x9E+/iV2ASUhmfPr1B4lk2F0Ug0V1HiSsux2zFMgzRzfKDvRrbbJcY6QogoY1hv1yw3S2xpSSS8n1A6zjprMFlBzHNu3gwnTZFu7Dkej0zTxPb8CU3TzKOyeQyWInUpja4YPWTZZMa2oyxLKu2wxmK0ZMpnFNFHjFZMc5dbZvoiXokxCmAkJ0Fnm4pxHOj7lvPzS4zOvH1zy9XFObu+p2oaEnBqW0IUeawxTkoVW0jqstKc+o7t2Rk3dw8cjkfKqmK9XrM/7emHgb4/sSWKeShDThK3lWIk+YhxIiMOs624aRaMw0AkYYwlA13XMYyjXKGVEh3CDBERWTGC8UoRRRYN/EwwftT7O2vQhUWRmfycK6A1XXsEMsvlksIZCifj3JREQ++cY6KX+b2z7I9HUlRoJ98/HxKhzBRWUVixDOcsakM9h40666XEEQ0T+jEsPQk0lRSJYcJPBm012iqs0WgqsAsgs97UnG3PsUXJqeuEBBwDWY1oK4zD58+fc3PzjpQSL1++5LPvfywKwv0OYzSH/R5nS66vrrClWOnrusJow3q14sMXH1KWIldu2wMoWK42lGWBc46hb3m4u2G3OzKNkfUHZ9T1gpfffIspa24fduxv95Tnms++/wO+/fYlV9dX37n+fiU2AWs110/OUExUznE6Hri7u+HFiw9QRrFYLsidMPpXqzW7+4MYV+A9kx2VcYWlbkrK2QGXckDn9J4YnPzE1Hf4aaIwkkq82+2IIbDdbFg0jZh2pgmtDT5GnC0oC0sYBgIaUyimsac7nXCmoKgalHVSZ+c8NyEDWsPkByEYhwIfJoa2xViLstIgs3aO/5pPqxwDrqwQTYMlSrAewzDfGlyJdRXKylhLGUs3egn+8F7CUJH6MCtwgyEGT9eeaLqWnCNaC8sAJZTZPJ/GrixlWJglQDRFcRo6JzbnYRxnDbvjMW4shMQ0TYTgKayMBDVz4u/ctMxRNoTHDActmR8YlVEkoh/R1hGmieAkVr1wBmc0OXq0UtiywFn7nrBsZ5bhEMIjKZT9fk/rMk1TsWwqSudIM5fBipMAvAc9YAqwRmNyQMWJ5BWehNaJGEvIUiaUTYHKkZzAunlTzJYpZOHSqsz+tENp8NFQNoqQBs6WF+R8ScqBuq64v3+g7zu6ruOv//W/xmYjaLGLi0sCI4fjHUrBOPWUZYV1hn440bYtzaKSJO3oORxOrJYbqmqFNnuaZs3zD16wXK+oqxXj+I5FKaEwddPgqpKLqyvWZ1uePn363evvL3Nx/1kfKScedjecuj1tzqiUGOc56RQmXFlQk3FFwdX1M8gFKY/0bcswTNT1mYy45pgoly1WS569JP9IQGf04/vEn5hF3z10HWVVYZ1jt9uhkVjqfupJObPaFox9T+gnbLWgVIb2eCDHTN1UuFKulzEnUvTiSkN+PXRHuSLH5QyMbKmqata1S8hG3w/kGZLR9/JDXyzXTLOhJyaxy66ahqpZIJFf4ov3UW5Mgjt2+ClIPLsSx93tbUeMEzF62vbA2J+whcHqAjuLflJK+MlTL/S88UhUvPfiGbTWCv03xhnh/Qucd55luKLgU+9NL1ornNHytaeAIpFjRKuMm91/2Si0pIiiMZLGNPYSMpoTVitSkOSfx5uWMRIVZp1lsWhIaSChaJqau/sDx35iDAJUXS7MLI2WVChjNSnKaBWlcM5CDKQwEWbscC40JEMKkg41tCPdeEQlw9XZE1whKU7aluyPR4rSsjvu2GyX9N2RspGg0dDXOOc4tQd++MMf8Ed//PvEGFkuJXD3o48/gqTYH/aUtQShvrt5Q1k2nJ9fiIM1B3b7Owrn5kAWw3q1xpqCaZwoy4aLS8tmvWSxXFAtGs7OLqiXC1Cavqg4Oz+j9wPrzVq0HN/x+NXYBFLk7v4d3anFjxOXFxesN2tevnzJ6CfpqgqCVJpKypGCpKq0XYs1De/evaGuNvNV9YJKFTgtPjfSjP8KkzDvYxAfeggU1mFQtIcj+9PIomlQCvqupW4WGAXt8YCyhVy/+5axPbFwDmcdYfJgBMKFMThnUFn03X7qKZwl5YD38vmnUZRyVVVROEd3OhGCRysY+x4fAs1iQXs8iFV4VpjF2RI8jR7tKsp6QUCRMihbkJTBx0nm9sFTVyWH/R5rFYumIoaRrt1T1AVOZ2xh55JI1HwpJjGrZHHaRaKEhyAde5jVf3MuXpix7I8BGSnNEUZZBEUYLeWClwWlcqQwhuQM2ovn36j8/kpODoQp058OMla1wiLwSahLWgkrUBSgktE3jIEpZApbYLRiGCKhn8COaFfitCJGSThSBnROxOCxxqBiIE4DBsAmsgGVC2FM+IG+y4zTkZAGFs2WZrWhcJb9/uF93HpVVxSFpSgL0bgcdnRdy49+9neIKXJ/f8s/+1/6L7DZbOYQ0oZ3795S1zUff/wJfd9hi5rt2Zbj6YRWGYh0/ZGcoSznuD0yxjQslmuiVyKd355htKDRj6cjU/IUhWW5WFDVlUBtQ+TQ7XGV5euXX3/n+vtTN4HvCB75nwL/eWACfgb88znn3Ywl/2Pgx/Nf/1s553/hT90EYmLyE8vlklD49yz7b755JXbawEyU6TntforGsDmriTHip4nDcc9XX31B1wVSyjTLkmqxFTpwnEhTT5p6gh/FgjmOWJ1FGlwUeO9p2x6t5Kqbk1hmnTVyTc1Qu4I09bSnI3GcsLUlToGH/R7jCqrFEqfAD73EcQX5HFZDnEainzBaWPBd2+MuL5mGgXHo3p+qMUTRFlgndB8lDjdbalmoWTH6SL1y1IsVIWb6qaUqK/wkEdYhRhkvrRYYa4lxYrmqCXkO/0gBUhBst5ZGZ9e2rH0Qnl7W+JBmX4QhhEgIEbLALKdpks8z9wNExl0xDe17pp+zFp0kSyAGL6VFThTWMCHjWqOYY93yvFFLxt/Qd1QzHsw4y5iSlE1JqFLWKLw1c0KPJSShL1unUNYxBNC9p6wjdeEIQShKMWXUvJl5MpNWuCIK7ThHklGQA9aUOKcprKEoGrQp2WwvWSwrUIp0SHRDx+XVGf14YrFczXZng/eRxWLFelPw+tUr3rx5xRdf/IzVaoH3I5vtmmEYGIZeDoHCcXd3y8X1louLC5bLFUpn3r57DRmapqYoFkxetAOH/UgMam46LiVkVEUedncYp2nqEmvgcDxQNzVTnLClIirPw+H2z78JIMEj/yvgf/9LH/s3gb+Rcw5Kqf8J8DeQzAGAn+Wc/9E/w+u+f/jgGfqRp1fPiFPg7eu3QglKmrpZoLKcWn5KfPP2FXVRUzdPgExZFVSVQ+nMfn/Py5dfcXV9zmJZoHUGLzkBaeqJs8pumkZcXc45dcLTV9qwqBvKQrTopXP0fcfYdzTLFVZnuuOeh8OJqm5QtWHsOw77HVWzpF4sIGe69iTACxKlE3ru1Hfk6HHGMA0D09BhFcQwzXmCkRhlAflxFNZHznN0uPQHlBI9QWZEKYuxmbYfOI5HjHG0XcdysSCmzOjH2eFnaYcTKRaUVSVBLDlCCqSA9EmiZxxk8ymdhGKG4LH2ETv0i4fIIBRm9r6P40RZiloz+lFwZ1rGfvqx+ThPG9QMQU3By41svooLwUkUjBnpt0CicCWucGijhKPoZyKSEvCLMXIbiBn600BROGwQLcXgI6OPWGOJSW5pIUpPQuVZRp4CVity0MSc0EYRQoVSDWVhWSxqmsWKGMXoFBkFZLuuqOpSyrfdwGq15XQ6Yo2cvqvlit/6rc+4vr7k+snFHLB7xhdffE5VX/DB82dY4xjHga5ref36NfvTPU+ePGW9Xs4/b+jajhAnlsvFHMRraI8dbTsSo+f+wbCOS4apZ5h6zuutoOsUfPXVDTmvKWrpa1gHq7O/AF7sTwoeyTn/33/pt38L+C//Gdb6dz5SysL7x/Kw26GVxY+Jzeqcqm4YfWB7fiFnhr9j2dQoJHZs0ZQoIut1zbLZ0LZHDrt74gdPoFBCqwmTnM7jSD/0lEVJ4Ur6rqesG1xRU9RBuG/aoFVm6Foe7m6pqoplsyB74e0RJggWUikLnkxdFlLTTqKmc1Z4+kbPqURhkrk5MHQt1shJE0JAk98bXyDT9x0uyYYQ4wwJVXr+vgu5JwMow+nUky3z+E1y9cqqpG2PHI9HjJbXadsTS2Mooif6kewMRDBzk6099QxtR1MuiFHSeh4NTCllwXxnGa/qWevetj1917NarbHWYWxBTP08DXFkm/CDf7+PpJTe3wqCl3SiEDwpR1RUpCjKvTybsgDKVAoT0WhI4noUGbg0ex/9/ZBFoGQzSkd89ILuMkIgHn1AOY1ByaaTE4FEqkqyE+SSxMmLvLlpahaLmqoSeAgqoPSINpb1VrQT+/2BoixFQRlgtbzkdDqSY8F2u6EsHd/77GO67kjdlGijiNGzWq05217w1Vdfc3Nzw2q9xsdhFoMNyARiRVE6Xr96RUye5WKJUopmIdSsySeO7QPaRrquJcRRbDFKpj8XZ0v2pz1DstR1hS02vPjoCd/1+IfRE/hvIpmEj49PlVJ/FzgA//2c8//rT/pLv5w7sFqVlK6i3bcM3UhTNmxW2/e18OFwS5gSm7Mzmo+WrJc1o98T40F030mxWTds11IzLRc1zmmUnt8wZIxWxPkUW202kCJFWbPebMXfPgw4V2CUZuhOHPZ7pnFgUZdoEmN7RKdEZTU6efw4kKNn0Qh1yE+jLFoNGivX3Nmqm9J8KoVInLMCyJGh74hzolHOMunQRU3KmTBN+HHAKi03hRC4f3igrGriLH89HA6cP9mI27ERkElVltJraDuMztSVo+9P2KHHzviyXJSPgUrSRY+R0/FAUy/BOIyW8Zr3UmJopeVKHRPOzZbnlMgwqwc12lhpkM6mJD0TiGKU6UNKSUw8MZBinGOxZJoiCzuDSuQsi0UHcfulGGAGo1prsUZ6BBIuyvxxub2ASJN9zvTDhNWaReUIMTHlSFmXOGPmRKJImITzoLQmkzDWUFQF9aLGzJ4IawxaZ4oSRt+jUkF3PHF7e0dTL3m42zNNkefPn2JUxXq9YZpGhqGjWZxRlgXkxGIhvg3rLMMoCdoXF+estktCmlgsFvS91P9F6aiqQhK5rSbnyOnUobWmqBRVI5mG6ES9qLCFfM/2+wFFYr1a8LC/ozudaDvFqT3w2Wff/84F/BfaBJRS/z0gAP+H+UOvgY9yzndKqb8G/J+VUr+Vc/6PQM9/OXfg448u83qxYr/bsV1tqMuGuqhJSZJ65QTSLKqa9WpFVWj6MdG2b1A5cH6+ZbNqIAXOzy65vrqgsIYcx/nNIUx54yzNYkHZNPTHE8vVhnq1wYc9qIBWWvoDp1ZO+XlSkfxEO98aVBT8dvKawmmUtaTo5dr5SAieNx2LiG+C9xDiLNyR03/oWvb39+8lwM46utORerUlTJqhayFGmmVDd+oFwHp3x7PnL8QTESLH45HLqxWkQOEcJAgps16uGIaOcRhYLrY4J9LloZNI8LxYCEdv3gS0UpyOR5rFkWqxQmk7L16JX3eF3DRSEit11w94H6ibRp6XEYqvKYghEVPAaKEED2NgnOT7E2OShixy05jfRGIMUlICxST0Xx2jZE0gZYAiY53BakNROIwdiUm8I0VZEIInhow1DgonQqgYsbahLAx56qUXoUXIZbQSd2MKKOR7UDc1q/Vqpl2PBN9RV4axH6lTjfc94zAw9J797oHkIcVM6SqMslSuYbM4Y5jumfzEz3/+Ff1w4ur6gmZRi5MvJ06nI+vNkqqqhVeR/ftma92UgDgg15vV+/5L3/e4wlCWDusKbNK0fUtV1WzPztHGEPxE156oSxlx7g5HoTntHjgc9v/wNwGl1H8DaRj+0zNhmJzzCIzzr/+OUupnwA+Bv/0Peq2yLLm+vOTV8BJnNOtlQ31xwf5wROvMclGDsZxfiFa+7/YUTnN+tiRnz/l2hcqe06Hn8uySRVNjlAhfQgyzOETSZaqqBjQhZlbrFVk7QgJjC3KKtMcjh/2e4+GIzpGpLDjtd7IZpYiPkapqMLp6f53NMaBmlLU10skW+KYR9VwITGli6EVS66eRw37H3d0NdVUzDT26htPxwPZKkoCnYaAwGqsVXdtSVzV+GslRmpZdGEXVNnT0Byfe+KQYe2k6xRBojwdSzEKZcZbJB8Z+JPo4u32lMVoWjnYYOJ2OoA2uFI5/mJ1+rijn0aAk5EzTRIoSIx9CxBQaTIEyGcxj2GcUaGeSTeIx9QctPweVHzcC9dgaFDVnzoiMR5DamSw/wyxMwTzX77IpQVJI/Hphyb2MVa1WRCOx8tZaVqsVoVeQvVB8QUbIWp5jjCDIy6pkuVqRyJzaFvKEwjCmDu4kfNT7iMbObk84225FbhwjVmuGrqNYFFRlwRdf/JS7+3cUxa9zfiFoudOplXyJQs0lIHPQTBAUmrEYq+aNQM9sDUGrLxY1TSM2dB88p7ZHa4dzlfRegmYaIjp6rBLCdIrQ1IIY+4e6CSil/nPAfxf4J3PO3S99/Aq4zzlHpdT3kGTiL/7U1wNWi4rry3PaQ4sfW0qr0AS26xqM4ez8guVqxdANHMaOmDvO1g1KZ5rKMvQncjBslgucUu+7yVpLLaiyhF5aJ6e9NhZbNfgpkNG40jEed4zjRNue2O/uqZxlWZccvNSopxBIOdOUBVpFTscdWUvWYFE1kBUqMwt/EjlmyR4Mv7D+QsJPowA1TkectVIjG83pdGDsO/TkGfsWW9eMw8Bht8OcKaw2dKeW1WYrNmBrGdsTpxlJltGMfU/pClQWfNmp7VmtFxTGoaJnHCamcRIxjk7zxljSDQPTOND1Jyqx58z5AyLXfWxU+jDNBhzJAhiniaYsRTAVEzqJ/TpOkcRMHy5KovcYN4m3P3r8TANGyYLn0eGnBK2mlBLtBTM5FxklJ52lJDAaY8EgFKBm0VB2LYMXZJqdpw/DMKA2Ehw7nnbiOCwsWs1RZUjfJCOj0Cl4ptPIqW1pKmksrlbbWanqCT5SOkNdNjhjWdQV0+CZxlbi6+JACGrOPdCkFDm1J84vzgjBM44DZVkxixkJ/Sh1DdID6vuesnIMQ0/fd/K1WEtRVNR1SVkWtN2A1o6rqw1XV0+x2rK7eyBHS2nFju1zoqk2vH37lu3HV5xt/wKKwe8IHvkbQAn8m7N67HEU+E8A/yOllPhW4V/IOd//aZ8jBI8mcrZdMhz3tKcHTod7yWc/PwNtWDQaZyNjHlnWjtFrRj9SFGIXzkHx5OqaJ1eX6CxX+EeMts4JooSDaiWhja4oQTuGoUMZkaX2KYk3YFbRKWfJKc4JSI5+EOmyVpkYxGde1QvKqnqf42e0JoZImCbGYWQcRnHgTQN+mjBzGq6fRhEJaTkLw+SJ3jN2LT5mpnGgsobDbqI9HigLJ5LTw46qket8WTjC0DEpRagamFOHY5CJg/gIujliq8BqS5g8QzeQZ+mucSLEsUZi3adxmE9tmQDIzUkWvOQUhnlaIbes2Efq1VpCTkxCm0g2UdJ+zAzw1JIiFYKnqhtyGPEpzSj0SMx5bnbORGCtZVFGCaARGCxzQ1AYBUb3on6cPQZVVdJUnmlqZ4SYhdlBeNjvub7YYo0hBjWj1cQB+cuPYRzZ7Q8oK0yCwilSNmzWl/icicHSticWdUPXdiy2C0pnCGPH8bgnliXl2Tn7/UjOiY8+/oj1ZsE09RyPUhE/PDyIH8RYjC0oywZXVCidObUiLTfhF3zJ1WqDuMoNXTfIzyIbnHOcnV1zfv6MafSURSQyop2DqSUlzbPrp+Ro2awuWTbbP/8m8B3BI//b73juvw7863/aa/5/P2IMlIVi93AihA5tHRow2nLY3xBz5ni6paobjCqY88IZ+xayRFo39RnPnl5TlcX7K7hySmS686I2xs4/eIUtJBR0nLzUzErNM3HpDZROwBWPnWg/Sh3sCokhH8eOtj3MufNSYxZlIfLP0TP0A33bM44D0Q/k5BmHgbquMVrhp2m+CMtNKKVAWRRM48DheCLnzGa5oG9PMxvAo3Km63revX3L9uJSFjwTOtfkFEkxYpQEn1or3xfvI23bC+hys4U54YgUgYSNwqOXTruZQSuejCUlKIvqfa1urSVEqdmFhx/F1JQho1HazB1+jXEFZdWgUiY7R/ATwUvEm+8LlBJ1pStBBbk9SFiJiIRCEik0VsxSSWVUzGQjqb1ZiRTZaDOLlixV4Wa/QUI7N98ooO1a0nYl74VgZriqnssAmQ5o2Y3pxxHJqJpVkcA0JYpqgdU9MfQSZosEuYxjyzS2pDQwTRNv3xzos2Oz3XB5ecHV1Rkvv/2acRxo25bdbi9BIkaayU2znhObJDEqzT2AcRQ9hrOOgPRT3rx+y3a7pVlu6IaR02lks04Er3BuwdQGgteslhesTMYta37j1y7IObO7+xWHiiil6YeRn/3sZ5Qz0rmuhCP/8HBPzBltrQSCJkPwgbpRKGVQyuJsyXazpSpLxqEnaMFgF64W0mySOlVJ7hSmKNHOiclIKaxzhHF4H6IZZhRVURbEGDBasGcZ0cT7ccCnxNC3syFpRKvVTAQ6Mg4TfhwZp5EQJikDcmSaRsqiIClN155EMzAOhDDNyUmWaRw4Hg4YrQl+oj3JDy9FAWaG4Lm5ectqs2YaB4zy0mFPCT8FtCnIOUmAZbN4r58fhxE2whKIMRCUzO+ztP/nRGBD9p7TqcVHi9GGZd0wOUuYmXzWCJxDxpZKLMlz+SCBHRqwuNJSOofVhjT3L2KcIaeuQNsR60qUdqAFMAuPbkVJUooRnK2kN5ATOmdCUKQs//aUE8YVAlpBUVb+F9OGlMjz1EJpiS9zxhDFNoHWBmUU2sz1t3lMNJZRa0ZjixLrCo6nDucVMcFysZaxIZovv/yS87M1WsnNwZiCt29fk4oFVe24vR1YLBe4ouD27p7b23eSRGysHDg+cjzKdGGxqIT/aOF0PNF3LUVRMAzCEYRMCInDsUOZWjIZkqbrBmEXaosPmZwUy/UZWQWO/Ym6qRj6nof7u+9cf78im4Dhjz9/w0Ob+fSjpxSrFcaJ+cXVkcqIJddPQRpGOoOpKVyNdQ7nVji3IsbEfn+HNY7tZgusyFQok1CFJcWJ4EdyVeHJnPoTdVOQo8Rfl84xWUvbdhhXkrSjbTvqwjGOIjcdYsvYtUxqZJgDSAdXcLY5Z2qP7O7uRXATPSlPjL5n7DsMoJIkH03jwHja4/ueqashJjHjjD0pZHRI+Ozxo1iMlVKibygd6hgZ+yNjd6AqFEMb6UfPIvOehmSToygM69WS4/EgJ1tKjENHdIayMigjHXWfAkVZSjkQPSZHxtOew1iwXa/oTwdMEJhJ7QrJO1Tg6oacPaWGFD02KbR2RFuBchirMWR8NujksRF5TszoekAFsKqA4NFqwGgRjWZFWAAAvMtJREFUEBHTbDiSsix7O8NQHhd3FgIT+X2WYFWVGJdYeYW2GoMhKEVW4kQNKhOUxWnD6DOLSth+2WayzmijhLWYNU3RgIG2n0A7grIMU4fvdhitubq65t2bI/008M23r8hKs1mueXdzoq4qji30xy9pu3dkDNuzKy6unnFoPYmG88unZFXg48D+sCNGT5wCYTRstmucMdzsdgxdix97TscdRVlS1zXb8zMmr1Gm4Gx7Tnvqebh7y9XVE6rGcXa1pu96cinuMDMNdP2RaRxYbv5yosn/oT1iglPn+e3f+Sucna1ZLWva9kiOnvOLK/zkaU8dp3Riu91QVIXEX48Tw5DwY0uOFoVl1SwxNWidxEOdFGiN0nMCD6CMpm9PTGPPpqlo24PIXGNkv9tR1TUpWtpOLKpGRSYvmC8/DgzbkWin947DHCNhHNjtj5wOB4qyEHKOhr4/iWtRGzTyOfzYk7xIiYfu9H60NvQ9pVPUZckYPH3XczqdcEUhuYuLhZhvspQjWmemkBimIBiymQyU51uHcwZrDFZrytkKnVVgnGamYEqkkDDz/N17jx9H4jTRtZ7KWU5xJFpNs1gQpxFtK5TWFM4Ro6bQcR6JSniqKzUWuYYrMm4u3QofKHxkHEds2VCETFaWNHSgvdB+ldCgBIsuDsfgg2gVtCIruRENcYAsp3eIMi0ojGO1cCxXC+73J1xTE7JiSsIayEYkxhgHylJUNV57spZSwGiZ5tRlTbmoQItxKw4R48DHkWM74EqHskJ9StpgXUXZrHBFx+HUsVxecrr/iq+/eUvdbHnYdxzbQEyGjz75PpBwRYXWMIwDOXjCOMqo02lCKKiKkuNxT9ueKEpH17d0nePqyWeElFk0DWXp2O8eqMqKcTwwecPmbMsQWu6ONyyXBQ+7t6QUZPS5Wn3n+vuV2AScNfzVv/LbXF9fUFcFfXdkvapZLi5w1kmDZLPkeGxYbTY0qzWvv33FNz//mnEYWC+WHI5HVIbiqWO1WqG1EViGGlFGkXOQUyeKmWU4HTAkDIk4Sjrw4/X78vycu9sbur6ldA4/TXgfCTnTnTp8iKAl7zCnSFUWDH3Pzbu3TJMn1DXDNFAUlqHvJAA86/n5j+IZQXudTieqekHMSpJ1cayXCyJwPJ44HVvKOqKVpigqaZwp9b5ulNeRuv8x4EMSeiUFSBuF0jJj11o60MMwvE+8fXx+VUlK0DRNM2h05HTYE02mN1BWlTyvURRVjXaWnAusiriiwboStMbMjb0YA8I0XEKcyGEix8DQHjHWUVUVOUsjUmmN1ZasFRFFehQUzX4K9X6MmAnzGFFEjY9KSin1ikJzfXHB/cP+vX15GHquzs8BxcNuz7qsiKj3gBY1Jy6puZRJCaqqpokNu8MtykSqpiBnTd9N/PzLr2maBSkk1qsNMWYeHnZcXV3hvWe9XIF7xfH4JVVZc+omPv/8Z5xfXvP06RO+/OJnGJOJfqDrOg7TJCxGCu4f9mhrWCwbYlL4AInIMPRMfk9RXYJyhDhS1+c8fXrJOE2MM5uznCynbsfxeKQfC969fYNSzKGow3euv1+JTaAoHL/7O7/FH/3x78Nmyd3dW56/eEZZG775+iucK7i+fkLVFGhbMYbA4D13ux1+GFmv1oBiGCe6bpCFY0sWuca5jLJKtOlhJMdJXIXTSF3XqCgfN7agMJr1oibHwNQPFEZEJIeTWJZTDByOLT4kUh7xc7OrdI6uPfFweytQkTSHQiaJ73JlPfvemcdRAssYhoGQMtoUkvs3TigmptIzTRPDMAjAZJDQi+3ZGdYKvy8EOVW1dsSQaU8dy9WS9MjZR6YuOQdSMqQYKIxj7AfaGEVlaMz7jSOnmcKrhCFQGM80nEjJE6xwBoZxxMbAsllgy4qUJpzNaF2gbTkvJtEC6JnA5KwlBU0ZGkKYcOUMa43ydWgjRiBNJgUZ6+WsZ1+HbFpaaYkNJ0uzkEfNAcBMHRKDNefbNeebDfthJERJW95st6zWGw53tyJaSjCGSNGUKONQ2pGUxsdM2/WE28ip2xFijy0y33z9lnESIMzQ96yWK6w2VFVDzvCw2xOmwHK5JOVMUTQ8ffqcqtngDj3923v8nG+YUuLh4YGhP3I8nSiMZbna8LDb8fbmlma5YNkuaXuPsRWH44FT280lZuL8bEF72vNwLylQ0+Q5tSeMNYyhpR0OYBJZKxaLZpaew/3dw3euv1+JTSDGwO3bb9ndvuPm7ddstktCGGhbz8Puls3mjNF3VE2DchWHuyOnvmecMVZTCBhtcCbTdi1hmiBmCtPglAa0OOdCmJFYgdoVNM7h+440iWnHaGjKip/97GekGLm4uGD38MDD3Q7vA+M4sN+fOLQD6JaxH+i7juA993d33N/eUTcN1ogKzpMY+47SFjDHVMk1WTj/0zgJ6TYmwoxFSynSdx3dOMwx545hHN8LaZyTcaYkz3oW1QZQDIPn8rIC4HQ6MQw9SjGbgR7xX3LFlumA3EiqqoKYSD7QLBoK6xi1YdkUnHYt09CCMbTHI9kr7PqS87LBlBUhWlypEQNDgSlLyBL4ZpwEfViTCDmIrBg9u//cDIKR09wYi8qJmD1CAp5DOo0h+tk/ATO1SPQfKUvHXNkCY6xszFFciddXF3TfvsEUDrTlcDjy/U8+obY/5Obl12StMK4GraUkUZaEYQqJ/anFjh0P+7f40LJeV9y8u+HYDhSuFHNY7imsIwbIIaGN5ptvX1K6grqpObWvuLx6grUlOY8smgXn5+d88803DMOAsyKEGoaeIUmg6263Y5wmqsWKUz+hdIkra6qo8dGyPdtyeX6Js4bd/sT93S2n7kTXtcQsoqqLqwtBrBmNK0sWC4m9X61W74Ns/qTHr8QmELzn519+Qekc7969BAIxjLjSslwv0U7zsy+/4Pr6Kc3qSnjyxnJ5/QSrNHXdkH3gcDyhsuJie4bWRubf+jGkUtJnc4agIotmidWw38kCVyHLbL098frlNzx/8SEpRO5uHzgeO8hwPPW03cTD7sRy4VEZ/OjpTi13N7fsHh7QWlEU9n2ybHc8sqgagYDMFKQ8a+nznLsdQiRlkbNqpZmmaU4jVnNabZxlpQFlZBGZOYuQecg4jhPOlXLt3z8wjgNVJQacGDN5jmIrC/derBSCoM6zjvT9hNaKqpJcg8opep0Y/cgwZt68eY1qBi4+/iFlvUCXFQSHrRwxJMCBdSKSmgNfReCfyPSkDKOfpItfSsdaz4tdRmOPWgHe5wIopWF+g/9CsCTqOdk4RSYs7MdIGCeO+xZnC5q6ItkCHzPvbt7x8ttX/NYPPsMpaPf3jDFQZotCY5TEzkc0/TBRoOm6jtvbb+lOFcZVXJ4/xRrDcrmkazt2u3uqqublt6+4vLxAG8OhPXE4HbEmcjz0ZDWx350YvLASP//8c0KY+PijD2Y14IFh8Nzc7WkWCyEOlYuZKGXZH3uMKbi4es56vWa3O9KfdujZT6CV4uXLr1mdbdAGjq0R5yWG+4d3pD5yf3/Ps2fPxAH6HY9fiU0gZwFDTNPA+dkFx+OenCLr7ZrNmUQ/7/dHtudX3L96xdubPdM4cnFxQQ6Rdn8gjh5i5GJzxuXlFecXF5JDaAvBccVAzpqsI1YZbAFhaAlJoUwhXXg/st/v0UrhtOXVy1e8e/OWySdQctomLLt9S1NbrHXkJOm4h92OoesYmwZnRXijNKTwGOCZ6LsWQyJM/v2JFoOc/NoWsyNNaDwpZVJMSFKQpBQPw4QtHFobtHaQ1Xvu4ORbYspUzlGWJcPUE1JAGcXgByY/Ql5QVzV920lktVLkZURZyVvsTie5YecMccJp6ZlMfuLduzesn1Si1agaVFlRZOkDqDwBBWBBxTkkZMaYzZ6KnMEHCWwxVkZ6xkrirvcTOc1KQWRer7XIrzNZCEEznkxeL8nIN4siE0QjkaZJJiwxUljNfduCceSs+NFPfozTio+eXGEU7Pf3qLlJiHagHcpWoDWHw4GURBIcQ4dWhqdXz0gZzrYbXvlX+ClQVTVVXcu/RSmGYWCz2bBdL8UvkTNNs6R72PHy5Su8H5mmAa0SZSHK1YvLK1AW7wO73Ymu9zQr0Q5IYIhmvdlwe3fAMnK+rjEp85Mf/wxbGPp+ZLnNDMOEv71jtVlSVBX9qWVdNXRDz/1uN2+2f/LjV2ITiDFBNOzuj7z4+BnXV0/IOnI47vnxjz6XmWpKjIPnyy++4eHQ8eL5c/q+59U3L2n3B5zS/OB7n/H8+XOxchaC/UI76QhjyFmLp1wVKBI+j1AscMrRngZC39H3PZvNlnEc+dnPvmC/O2Kc0Hz7KaBtwf7UczVW79OH2rana3v85Bn7AWfN+7lvYQyLpqYfRlnsj2/keUY/DSPjlKgaJdQg1fM4EJum6X0K0jRNjOMg/IpZryDec0ddLzgej+QsmQFVXRGSl0ZeXTCeBGSqzi8ktXc2DFVVhZ8mCe1ImX7sKJzDKE3wI0ZBWVjiOND1HRdFITHk2gLSUMs5EZKfQ1Ek0FQbQ06BGLMwFufGnZpFOTkjGHRjKArRduQkwFiVE0R5nTxzG1OW+TePtyfke6C1Ee2ED7NwSLNeLNgPwoi8v32g2ZxRVCU6J/7+3/89hu99wm//2g/ZnK25P+1kk8AQMcSkyBGKqmF71jAO93SnOzabS66vnvDmzVtubm55+fIlh8OJruuoqppu6FmtlixWC4qqoGt7Ntsz0TBUNbZs+PrlN1xcXAKRL7/4nKZyNM2KzeaMgKFte+7fvCMfe566hnHyLJZbDoeWn/z0K4x1/PXf+SHPnj5hmAZ8yHz51Rcklbm6/ICfffFTtM0cTy2nvmVZ11z/2q/zwQcfAPDm9bvvXH+/EptATnDz9p7Lqyf4Ud6MVVPSd4H72z2nTvzWd7d/h33bc3H9jN/8jd9gd3eP70dqW5BDZLPZUjcNoDDWgC2IyiFXVYXSpXStlUI5w+l+hyqWxNRx8pE8TuwOe55cPeWnP/mc07EVppxRZKPwIRHDhDu29INGK1Ec9p3EbfspMPSDqBYLJw7GPM+2+56YAsMwoBVEL5jvyXu0lVhy7wN1rTgejlRNQ9d1NIsF09DP7r2ePOvqq7p+3x+ATFVV3D88cHG5JaVEVVa03Ym6KUXxOE2y0BQzNizT9y3v3olyMKXE8Xicyb4WZ+R7lAuD22y4e/lWSE4xyelrFSkptNLYohIRzjy5eDQLiYLQSnNzGimKArtYkocOax2D92TynBwtI8UphveEIm0N1kmJEWOYoaWPbkPIOqOsxljH5AOT90JYSiLMX6+XDGGCERZNTblecf/wwKvXr/nt3/4N7KLi7c1OAGdKE5URXLmSxtp6cwZ5JOXE11//nHHyHI8HtIbNZsV+v5fb6/kZqERZFiyXC25e7XjYfY1xBR+8+FBky9fXPHv2DO8Hbt69RSE9p6pecr87MU2BJ8+e0bY9h2OLtQXHYw/a8MNf+02qqqYsDPd3J07tEescl5fPqBc1YYLvffprxOy5vb8FHJtVw+GwZ7lc4VzF6dR91/L71dgEpsljTMnZ5pKvvvmC+90tn37vY5p6RVUcuTx/yugDn3/xBXXT8OLpE968+pY0BZ49ucJ98IzCWDbLleDKrURtKWtRM51XLDZAiHPCTkAVK0qnwTrK1ZYcOzCWCDNU087wTE2M4EMmkvAhMU6BwmqmMTIM0yz2mRiHiaEfqapivinIzL7vW9kQciJF0eGTFCkmlMrkmHmMzp6mCVsUpPl0ezSRkDNd180TAsPFxQXWNeScqRc1RVlQNQ1106A0HNsDrihYzHHrXdtSGEsKQRqDwTPERHc8CdK7HzjudixXKxGbaJHXqtnRZmaUt0iDjfx71fy15HnyMSPHUxKUd5jmOO8QKZ0lBEn7iSmJVHgGjL13GeZZSK0eeznSL0gx8VjYSo+E+VYgUuM8W7ZjSvJ3kkiJVSENvxijwF+IvH33hvNXaz75tR9SNFu+fbcjROYysaKuFafTO1IZOb+45ObdLYfDXtiLwXNxecYnH3/E/cMdb968oSiFsxhTIKvM8+cfst8feXNzw9dff0M79Cxz5nDY0zQLPv3ke3z15RfkpLi7fUAXFWfnF+8Pk6KsaZoFD7sDu8OBs7MLLi4u+PlP/hidAuvNhn5/QJnMB8+vKUrLGEZWixU5Wy7Pn+IcTNMDX/z8K8hGRrjf8fiV2ASWyyWffvoZ6ITWTuTA2nK2EdzSD37wa7RdT/TSdb4829KeTkLtzYbCGrarFcVM+oWZMGsMypXzlU+T8yxi0Zq+3aPLJbbU+JSo1meo2FI0C6YQsUVJUdaMk/AOfQj4EMjGMIXI4dizWS2Ykse2AynBNMmG4FzPer3EGMsUZYJwOh5nqKUlh8cTTfz6OQRMeMwDgJwU3kdgvjqnjPeBqpYwj8KV5Ayr1ZqyWrLbPbBYLFgsFjjn0EZT1xWXV5d47ylcSZgiXdejUUzj+F6IA4FT27JcLMg5czqdWCwWpOxROcnGqRRlVVEUhchtlfql67pkJv7CijPbgecFG2aPREoBZzTTPM0J7/0CzDLgR7CITFZU1u+buzE9Eo1FlyGSYbEfy0akZ0l1JGfRFIQovoJ6uWaYRvkeG8WirJmGji+/+hJVOJ5/8n2urp/wcJoYpoBrtKDJiprd4Q1dG7BaUxTSdFuvzhmnieunl6w2C4yVcsVYTUiaP/7jP+Q3PvkNmmbBejVgy4qPP/se++OBv//3fp/vffYpn336Pdpjz2azpp88p37km69fcjq1bLdnPHv2jL4fybND8tSe8MFzfnmNyYmqLnn97i05J7bbI9uzDU2zYvfwwO3tPT5MZCbW28zD/QMpaVbLs+9cf9/dLfj/46OuaxaLJYfDnmfPnvJX/spf5cMXHzKNI8t6QeUKFlXNJx9+yKIsOdzecn12xvPrJzSlE/FPfyKEiaJ0xOwZp56YBWARkRoUbaSOtY4QwVU1WRuysRJgWlcszs6JaExRUS+WGGtkEcb0mK7NOE3s9kdigilEjqeOlBWg8V42gpQQFFiGaRrfh3hKglGcDfYKlTMxRKbJz7cCNXe/4zwnT+89/ORMWZacnZ1LHkJZcXl5gTGGzWbN9mzzHg6aMyyXa6bJM46B4AND13PY7+cE5ISfJkIIdKcT4zD8wjcR5Go9TJOo7ICmaXC2kJgwZebZvHqv9f8FC4BfGHTsY4hImoM9pE8hsJAZKxd/sQHkeTygtJlTgfScezDbmecJiagIE3F2fYYQRDCkDBkJGo0xgxYO4Xq1wphZhm4tq4VEyf3oR3/ET3/yU5xxbLcXNMsNi9WWarGmbFbErOknybTIecTZzPZsQVUb9rtbdrtbzi/OSDnwh3/4+6Qc+bXf/DWsk+TnmBKPROZpnPj+97/Ps6fPOT+74vrqGcHD+fkVVVVx2B9Yrzdst2eklLi6vuB73/uE8/MNOXumqeODF09JGv7gj/+Ydhg4v7wUtoVxPH3yAU25ZH934O//7d/n9Tev6fqWPK8vcSD+yY9fiZsACs6vznj38Iq7hxvqpsTHDu8H6qri3ZtvSSlzthHgZ4yey80G5xxjWcgbKCWMURgr3WUfAy5OmDinDWHFOGK0GJKMpjAF0+Sx1lDUFd2xYHNxye3wBu1KirqWhN5JFpU2mqQUUwi0/SS1ZMocj728Ca3cOLyP+ClSljBOssAXTSMnog9o48hh7oBrjfcSuhp8mCcCQg3WWr+P8ypLMVSVZcnFxQVn52cYa9lsl1SVY3u25uxsS1k6iqKQhagtOYmeoOsGTE4EL8rBnMUibYym6+SUrKqaGDzH4x5bi/MPL4IfkS4njHPwS53mDP+hW8B8qZ9bm2DKknyUzIGpHxj7nhjTHI8uTb2UHqPOkDJAScmg1BxoojRaQVLSg8hq/qxZTESkRM4a40qJIwsCZiFL07msK6zVTEPPOHTUy5qqamgnz+tX3xJ0w/Pv/xZl1dCPntWqYrWoWS0NftyBP7JeFWgFPrRcXK558/ZbvvjySz777DOaRclqu6RqSparJeEh4mNgv99juo6ykTShq6snrFYb/BQ4O7skRvjqy6/ZHQ8sFyv6ruNl1/LJJ5+yXn1AjJ7tZkUGNtslGNgdd0Q8f+Wv/i5lWfLq5Us+cM949/YdVxdXbJZbFtWCJ1dP0bqfLdENTbP4zuX3K7EJ5Bypl456UfB7f/AzUvKcn2/46MULysIw9EecK1ktVmyX1xROxmkmJzaLBcM4MI4j7UmirK6vr4QIq0AT0Vnm8MpElLb4tiMnTwSmoad0wqqLaKrlClPtcZWw/au6BjVhUsQmK2aUFPABQhTfw/HUiX9dCVdv8oFj22GdeMDdybLeCI14nPxcP0pJoJWBLKdvjKICDCGQFCIKmrvoZVlKc28WDG23WxkF+R5dCBTFlQUxJ+rlgrIsscbiqlvCsSXG+fofZ5Q5wGOnXUGYPNkV5Bg5HY4si817bULOE0rV+CBR62hFSpnHIkA6/7OcN2WhGec4k42ThHnEhB97/DR/fVk2Oh/EG6BmV+IjUOQxWeiXbxdaC84NICE8gJRmcAsSI6/0Y5q0gTkhqaxLqjkK3I89odTYpmS7XtJOitevX3EMluwacJanT8+5PK95/mzN5eWaUrWY3PHwcMup7TA20/UHmkXBqzcv2W63vHjxnKquadsTq3LDdnHG3e6Bh/2e+4cHUJrPP/+cjz76hO36jLP1Ob/1mx/w7es3lEXBxfk5P/7pj9ls1nzwwVOOxz13dzcslgtRjvoBu6q5fHoONqEtDGPHR598yNj1/N7f+w/43d/9XS7Pz/gn/5P/BLoIfPDDhq++/Jqvf/6GYYa3/kmPP7UcUEr975RS75RSf/BLH/sfKqW+VUr9vfm/f+aX/uxvKKU+V0r9WCn1n/2zbALjNPLv/Z2/xfH0QL0ouH5yzvWTc4zNWAtFYWgqiyIw9R1pmkjeU5cVdVnMoMwj+92O/e5BOvBGY8xsJCIiQLgg19KxnxNvRBGndWLyI2OMYC2mLKmXK1abNevtGYvlUjrbzkpzDAgxcWp7hinQj5Nc55PCh8g0Rdq2Y5w8IUQOhwPjOEKWLjcpv+9yP8pjH7MHpsnPC1b8BSFI80uhiQkWiyXOlZRVjXMFx8Neuv0qMYwD+8NBTtmcKcpKVGtIB/0xp2EaR9S8hMdxfN9089NEipFpHBkmT5g1+uPk50addOulDPiF+kQKA3k99b40SBLkMfQItFRm+CGIJHqaJibvJZB0XuTSf5CT/xdXA2YMmJqv/LL4Hz/no78ghZltoMwM9CxRRqS1j3yFwhmMUaicCDMifbVcUlUVX//853z++c8Yhombuwf+5r/zt/jDP/4xMSmqugIVQAU26wXBD1xenVFVBXVTcHa+5dkHz+RmOo18/fXP8X7i2bOnfPbZZ7x48YIf/vAHXF8/ZblYU7iKtu1ZLtZ8/3s/4MMXL1gsGp5/8AEvXrzgdDoyDD3Pnz9jvWpwTrHb3fLy1c/xaWB7vsSnkaKy/Pqv/4D1eom1mvZ44urykt/44Q8E7R4jm+2GzWbD5eXld66/P2/uAMD/Muf8P/vlDyilfhP454DfAj4A/h9KqR/mnL87AwkJwPgP/u7f5td+7ftsz1acn63ZrJcYBcumgZQx2hJDYr97oFUHnj1/zqIueXi0XXpxzZVlwfF0oDpUXLiNbAA5CENjvl6aGVfdjZNgplWNHwf6aSKGiLGOxWpJgcH3EkJ66jp08OgsnWvvvQR/FrUIe+a6Vmb96T0UwljLONNlK+fmhplgvOfv2i9O1F/aDNKcYMzkhT5sIkVR8OzZB5ydneGsvOGGsaeuK7Eb9x3H44miKLi7u2e1EqOQMZacFZOfsEqabHVdo5TieDzSNA3ljM9OKWGMoet76lIyFkMMOK0lQm2Gb+SZMvy+FMjp/dem57o+5cDUt5IZ4Uf8NEm/YZLegPd+nirI94EZAvLLpYEx4oSckABUNesG5PmzijAnYk4Er8lqBnFGMHNYawxyK1EKiqIQUKnR2HkDXi2XLCbLsl7z/MWH3D28ZZoCr1+/4adLww8/OeNsYbi8OGO13vD67TuefvCUq+trXFHS9QMpBYrC4sPEdrthsWhoVguKqiaQWSxXbDbnBJ9RyfDm1TsOh5YPP/qYxb7mcDjgCkeIHu9Hrp9c8+LD54ToOT8KmsxUln4cyDHTVBVNWfPu5i2LZc0PfvB9aldxttlwd3dDDJ7D4cD9/Y71ZsNm9RfYBP6k3IF/wOO/CPyrM3D0S6XU58A/Bvw7/6C/pK3m6oNLkoZPvvd9NqvVLF01vH71cz568YRhOHLY33IcHwgeVlNBeOi4effA8dAzdJHV8hybzzndRjb1Bv10SdQirTWFI6WJnA3GgNEF+12kblaMPhM82CmSfKIoayYsExY3RVQMmNORKozovgMfSOGM/lQTa41XQn+ZFAwxURqLj4bC1SzrwP3DwNhm9KLCas9isaLvB0bvCdkQsiIpjZ37G1qBMxaVYZpkbu6KiovrK4q6YHO5YYwDu9MDy0ZsqUZPvH79LYVzTEnT7u+Z2j1THymzJeBQriFHT/ADpilwpiJHjdj4NSEGlDHErNHHCeMVi7JgGCcm54UuPMM/lJGZenq8roPo1lVGW8jTCL6nMIqhG7EonLH4riWcdjD2mCRKzJQgAjkr4SnMkfEpyG3IFA4bSsYQmKaABMbIhmmNTA9iiCSViToixUGYScKZMIFKUfIQlMEnjY0GFcHmQGEz19sti7Nn+NOO/e1P+cf+2qeUruDdqyOHm8xHH15zfb1huWlYLycKo/ng4+9R1Ft+8sXXfP32NS8+fcGVMXzv2RVaFZwOI6vlFUaV3N+cmE4lOsto2IWaw5sDq23kapl5dnbBw87hUybEyHqzplSRulSsmzVtexAKdCmK0UUj5qC7uzsur65YLqQJvT5fcjfcsL7esCgNd/7E/thyvP/LySL8l5RS/zWEJPzfyTk/AM+RMJLHx8v5Y/+Rxy/nDpydNWzPzjnbnuOKmo8++h7t8cDQHXj95h2FA+9PfPvqa5HuZsvv/eHvUZULmmqNH8GYmmnM3L3bs2y2NG6NLQqiDqic8H5AKycAT1MgYRORarVmv3/geGxJ40TyEVsU+KwxaHLXo+sKV5XkqURNA0obulgyDoqgAslEYhY02RhlzjdF6VZXZUGK0LcT0WdyVO9vDfJ9kMmFuAnnJJ4819tJhEYoRbNc8PTpsxmdbtifHiirgto1TFPH/cMNu/t3bDZLnC2IvidOkf4UKNwCZwppbAZNd+rou5EwRYx2aGWFCpSl1s4oCJkwRZLJkAT3pWfX3/vjWylZjHO2AwB5Nv2mAHPoS3c6zcg3YS8SJkxKWCWfL86koPgIZlXpl8aTwgu0RYEZPT5I63FmnaKUIUVPComklZCNlYY54zBniF4yD62TkWvMkLKwCAgTNnoMNb4bOI73NBW8eL5l1ZzR3r3i3/13f8rf+vd/yiefrPlP/Se+z7Nri/cDZTHwk5/+iDEpPv3eD6m3FSMjXvVcnq/IKrNYlMTB4Kzm6mLL/n7H/f0tz59folTg9ubnXFwWnJ09pyrO6YaJh91B4tKzkdDU6Em+h2TQusCYjMmRuipQl2csmpL7sePi6opTeySowGKzpnYVl1vHcLzhtP/unsCfdxP4XwP/srwb+JeB/zkSQvJnfvxy7sBHH17k02Hge59csX944NtXb0l+IqcJYyp+//d/QggjD7s73r17wBoxxiyaJc8/+JCn1885Oz8jRwnqWG4cdZNQKhDDhMqath1YLbaMfY9daIL30jNQmbY90LYn9CjJNWWzJGWom1qwW87JhuIcxjm0jWADPsV5XOXlFIriUQ8p4WMgEihKi7aKcZrwQa72fS8KwLIsZh1CIs4z7kdBUZYVKT8kaykKR7OoOT/fMgyS2Pzpp59w9/o1MSbevn2HDwGlzOwtMPgxcDyd2K4rtJZ0X1tYQaqfTkQv7su5qY9WmjF4SBKD/pgRaZylKEphArriF0YeIyEgItqRmDFSIqUgY89h4ng40HUd0zgIsvvxJvFL2oI86wDSXI5oJZkI75uBSkmT04kPwPs4j1AlHNX7QIiRx7NOGekrKCVuPVQCM5Og536MTEiE56fnIJSu76gXNZ/88Le4vr7m+uIZ714GDqe/x5u7e949ZLQ78p/5J3+Xp1cb7nYt/9bf/LdZbbZcPdxw8WzD9nxJaFsGs2NVLTjuXtEeBgpXU9cLQurpphPFpiT4Eb878vqN59QOGFfT9iOHY8tqs+ZMb2gWMhUqXI01DucKdvs9Dy8fqJuaPAu0pMcy8Xu/93vs9jt+8P1fp9RnXF9ccLb6Pne3J74L//nn2gRyzm8ff62U+t8A/9f5t98CH/7SU1/MH/sHPpx1kB13Nztubm7oTh1GK9rTgeViw27/DW3bslxe0Z4OkqmnQKNJ0VEUNWUpwaJlZVhtDKYYGYc9x74HDHHKxLImpQk/iebcWYWfeokGs5qgNG3XwRzWYYuSqmmwBwFhYCxZG5LWKJfx2ZOjB50RabtCaSvNtRjop5F6W+FK0ZOPk8cZmcXXdU1ZFIx+QM5DiMHP9fFj2y3JqaxgGHpiDCwWC07dkbouJaTyeJTx1+QpipLNZsM4TJSViGLGcRTF3azQKyuJ0WqPrSwUrfEhShS5Vu//X7tHyEhmsVpR1Q1FWT0W8O/bghqBN5F+ye4bIuM4cDoe6Id+lvkmxmlkHEdhDc5f5XthUZIxb4qJqCSD+tFf8ZgZWRTlzDGcCD7+h/UJ8+b5qL3QzL6FWSFqjEXnPPMmH9OPHsePirJwRK+pypLrqysUipubG25u3hJSwjZL2nDi3/v7P+LJC0GGvbl/zan3tO0bjM2slzCakW3d8NWPPpd4NlPw8cef8PNvviazog8tl89LTt0b7u9v2Ww3MFV0w8Dx9oG2G1gsN4QQOBwOrFZPaRYNKUU5qIwCEvvDPbuD/PwmPzFOnruHe3LOlEUpobpWo7Ti2eUzjDl+5/r78+YOPMs5v55/+88Cj5ODfwP4Pyql/hdIY/AHwL/3p73eMEw83J1oqiNlWXM4nNjvHpimkX/kd36X7//wd/mDP/gjQijYrF6wfzhwcXnOixfPuDg/w1gh5RelRquAj3se9gHflrSDBzSb5Rn7XcIoERdJQy4w9B1+7KhKR1dIhFUEirIgKyVhpMUDyjlh69sCXEBirCZCnMT3jiZ6MChMVvST59C3nF/WFHXFfn+i6zs2q4XUzlphnRWYBxlnDcMYyRpQCaXsfO1NhBiY/CQ4rTDNZGDNH//RH/H08gkxjpRlyWJR4VzJw8NhHkOm9yfpMA4weorCkMlyAs7z/piiPNfYudWv3nfeJx9YGSukXm0IMeKUfu8UhCSEYyWOSZ0lMWjoO9q2xWpBcY29nFg+BEIUoY8scnh/FUF+HaMoPvN7q7T8mdYSh+ZcZtQT4yj8AWPMvBGIOEfnhNFgtSLN/zcKFLLJpGCIIWKdAFSM0ZjCYaolPg28/PolbWd4++od//a/9RPuHwKDKymqgm/f3fD//Jt/D20XjKcTZxfPOTy846/9o3+N4/5bxt0B70ve/fyW/XHPi49e8B/7q3+dL7/8QkaR7ZGr62sSGVs04k/wmXLyFFUNdw+cn29YrdeEFLFObl8xJl69fsVy2bBYLGkWDW03o8y1ousG3r274YMXL1guVzzs7ji/XHHq7tE2Ycr6O9ffnzd34J9SSv2j80/nK+C/DZBz/kOl1L8G/BEST/Yv/mmTAZCx2B/9wY+5vLjkw4+eMvmGYexZb7c0yy0vPvw1vvnmwN/+9/8uddFQVhdcXrygqdfEZLi93XH/cMPl5ZLlsuBud+JunxhTSUgWZwsM0HdvWS83NPWSGCOFsaLr7w6C+46JpDTGOarFgn4YMc4Kh0BrsjZgCrSL5NwRg3jaNZqUDVMEnRSFNkwp0Q4TSRvKuiYrzakbWCxqnFbz/F/qa60U2lr6fgKtyJr3cdqSISCBm8tlg3OOpm74+c+/YOg7VqsVr14f0Nqy2Z7T9SNt2+NMyThN79/kw9Azji3Ozhl/WcJR5twIUk5oLRmCEgoi8/8wTcSUmbykL8UEzEahx0Vn5gZhjpGsxOY7zSPRqioZ4ySbzDydebwBpCQtvFl3OLcZZPyYY5q9E3PS0HyTSVlCUY21hG5AKzPzBwxqphIrMk4r6sJh5yQirXnPeAx+IkxzOag1KQV0DjR1Ba5kGG75+us77m/uOR53jJPhNEbWtkbZFT/98gYf/l2eX1/z2z/8jB8++YCL86e8ffkF3//0A959e6DUS2qX8V3ix3/4ORfbp/zopz9F6xVff7GnqGo++OBjUsrc339NinD95CnWCatxuaplWhA8bSc3p6JwwoVQeWZtllIGBbkVlWXJzbsbrq+vedjf8nAwxGgZhp7Li4///JvA/y+5A/Pz/xXgX/nTXveXH1VZ8ekn32O9XLHb3XN5tUVpsM4Sc6aqV3zw4nu4v/+l5P2ZkrbzVLVnvV1RlQ1td8v+cKQol6joGYaWQ5uxxZL1as1hf8fu4UAYB7piT86wWa3ouo6H+xs26y1t3+NjBKOp6pppDgJFa0KGKWc8iqiNJPsGA1GBKcnRkqWfhs5JeIQ+McSMLipsUTFME6P3FItKVIta46zFWrHd5hxRyr0fkaV5/KURGW5VlZydbXk43nNzc8MPfvAZx+OJm3e3GFvQ1Ave7A/krGi7nmGYKMp6ViV6hnEUJ+IszkkpCRNwjmfPSZpwWX6OGDMvwCzy6JRFCv0LjeB8+sLc0JRrdkqBlCJlIQGg3Vzry8jPoY0l5UAmSOOTua8wNwPlaxdZsexRjxuA/LuU1vPN5PHfLWPVR31EnlkNhdVYo94bmmbXETFKE9clZlCLhzCSYmBztmWxqfjq61vGcWC1Knl92zL1E4MDUxTYouTbNy1Df4fKFf/cP/vP8O5di9YrQqho+1ua5Qqs4fLqknfv7vn13/wdtosDrlji+wfGfSZuNxyGG27ub+jaHmWzgGW0ZrdPNIsFDw87iqqirErOL86QbMZMWZWcXVxydfWUtu14+/aG1WrDm9dvOduc8/bdKz7/6Vecnz3l+fMPONv+xXQCf+mPuq74T/9T/wTLZcnf/rv/b0Z/4IuvfsLzDz7m+slzxmlisz3nH/vH/+O8ffUtYWypFzXnFxesN1uur1bEdEHb3qPNKM0s65jGA9aWAi9tD0xDS+csdzdvWTYLmtLxcH/L7uGW5UKMQyFnElBUJdv/D3N/FmPblqXnYd+cc7V77X7v6CNOe89tM7Mys6pYrCpVFVvLFmlIlgXZfDBMwy8C7AcDfrBg+MlPerEBPRkQYAM2YIi0LYmkDDZiUaJJFqvPyva2p42+2X2z+jmnH+aKc5NUXbLMKhp3AxfnnrhxIiJP7jnWHGP8//d7HrPpFD8KEZ5HbQWFtdQWaiRaeA5/ToDBf/vGLU2FEIpNVjNf54TCQ3i+iwVLc5JWTFVrB8cMA1ReUZYaa4zDaCPe7r4NoJR8CwDVRvPJJ5/Qbie0khYvPz8jzwtOHhxjjAOfSOlTFrlLEU4CDA2Iw9omS+/edNNM/KVwPbOUeEK5nbs2BL7/dhUolEcUtfCj2K31rHFvHutk3LZ2aC9hrWsLhHAKRu2YCPcaCCGkC3uxAOWX/bypm8PrbgMIdz+4FxPBvYxYuW5eKnzPb4aCzkOgQtXMFdyfubcey8Z3YBuHIbYpsAbqqiaIXOR3rZ2QqqtCdnf2mNzesd2uydYr6lyQURMkLZTsgFLcTgvm08/Z3zng4V6fYTvhd7/3iijOENJSU+K3fHwCkiTm6PAIWwf0WvuslxWtIGFZTFCBwqslfiiJQzfEzfMtNzdzVusN/cEQz++9NVxVdU25XjcsiTZxnLBabSmKiuPjE46Ojvni+XO++OwVg77kl37hf8jh/r+mVOI/sZeAxfyOIBzQbofkxZZBv00UewSBIi+2ZHnaSG9HoBPGOyPaSYw2Na9PzxCUDEcJWZ5SFBW7e2OsjRHSGUZubm8RQmN0SZ6tMXXJoN9jvVrgKXcgkBLZqAKjVoQUivV6jRd4KD8gLyvyytlY81pQatmo8WKscc41a3mL/drkhskiZdTuUCMptaWoDFlRvX3CB1GEl5aYbYmQ0r05cY7a+/EZuIisqq65u71jPp3xjW99xNnpGfPlguF4zGAw4ub6EmOaBKTC6efbSYc0LTBYlOc56a9Ub5+4Urn2xw8CVOCjrcFqQLvrehhF1NrSjhN29w9QfuDUgxZoEoOtrqHRN7jA1RJfKXwFWbF1sNAmwsxgfwof5n6K+6g3jG3CY7/MKNSNOcjzHPTVNopiISXS89BF1SQZi7ejhftWwynEXNSfuR8iQnN7EGAcx1hKkNKC1W4YWpScHD/A6Iq7yzWbJbRWBdsid3qFTQF+iB+EbPKav/Prv8n7j494dLiLEprDRyn9bhdVw916yuHeHqiK0aiFrXzKDGRVMhxEhLqP1zlC64qnjx+6lkRrFssFz1+8wtgS5UGtS6q6fiv/TbcZV5d3HB4e0el0MdrS7/Xx/YBO0ubk+BHXtxtur9f4XgddCb7q9bUoAgKoqoxax4x3+qzWll7/oRuU2ZIsW6FkTZrOGAwj8iynlUj8UOB5EWm2ZL1Z4wfO0CNVizDs0e0GjaouJwyDRlpaMRj0mE3nLgC0yBmNHCPOSk3civADv+Hqa+cgExY/8MjKgqKqAUGpFZUNnKLNb1HXGiF1kxTkoa1mm9WkuWHUj9DCQwsPGURsy5qsyOl3u4RRQBCVSJW7AZoxaEGjIpTOFK0NVV0zmUzJsi29bp/lfMlnX3xBr7vD4eEx0+mUm5sJSdKirlPKssb3A6I4YrFYOS09HmVVYZVz2N33/0I4CEsUO0qOqSs85QaLYRhSa0PS6dIfjR3q24JAYq1T8Ln1oH3LKNB16UhDunRMxcYpWVXVW4uRW4c6mXGta6qiaJyHzVtSuISpWpeuQAjVYNoadWYzSagaNLkbWroiLHBGI6GaHgeaAnLveHSdga5pOhqNNS69qBW36LRjev2IRzwk/1M1H75TMrtd8oMfP+dynlFoiTWCrHLJzEaGPD+dcHO75uhwn3BP09/vUqRrlPBQUczV7TWh9IhUSJ5lWOOsxJ5s45sT6jIjzTK2myX9Tpt2EuN5AiktVVUwXy1RKmK7yd7i5qyFXnfQ/B1Dvz9gtVxz+uaMvZ0dfu7nv80nH3/B5eUL1qv5V56/r0URAMPxyT47u33mC4tSBq0r6mrNejlDdhWjQZe6bLGz2yXNBFEUIIVLh213HpOlzhRkbUUYKupKMJnOMbpiMpnw5MkTB6OsHDJbSMF8saAsK+LYmXKqeuve9HVFWeXNm8W9U4RypJu8LvFVSFl71Nr1qdqA5wd0eh6L6QptLQrJJiup8TAyQMsALTy8OKHWBbqsGfkBnvKRno8KfKQ2iKYHd4ARC/fW2NpwdnrOaNynqgpubq9ZLVe8++43qWvNx598ShxFeHlFq9V22PUgYrtNqaqSMApZr3JKp7Zpemg3YHNtuSAIQrKycJAOC1YIirKi3e+QtDus1lsGgxoZONefEs2+/X5wp2usrlFKosuSzXrdYNElm8222VYojDSN0MgxFRz5uEaKxhzUrAaFchCToijBSrzEd4NC/VOz5uZAC2EQnmqWCY1Y6R5uiPs5EU6F6SLUnFOxKisoIfAqlBLEUcze7h7p9orNxh3q3fdHiMry8PiA//ev/xYvL+YopSjqEusrah0QxgmzTcn29S1rW3B9Y9kZ9XhwNGSzDYl9S42GoED5hv7QpzYzSgMaH4RhPr+j2C4o08zxCSonY7+4vOLq5o6dnWN63SFVlQOWbqfzZQsoFOdnFyRxi8lkwsPHD3j4cEQYGs4ufkyvO/zK0/f1KAICPF+SpluEVHTaHbSu8WTA1eUtQlsePuzR73l0uh4y8F3sdFYwn8/p9waMRuPG6lkQRz7rzZIoSthuVyxXa25ublENuff07IztJsOTaXNj9NDWo6pKkiT+0sGVJA2q24VYdHtdlquN60GtR1ZohKiRasvuzh4Kn+V8hhW1a0Rx+XVZZRCBmzxb5WN0hZUK2fjzhZSu96ZRvDV/KY0Ex03AheL2boKxGs+TbDYbDg+OCIKI169PAZeyNJnN2RmNEdJx/ZerVSPndRqAutZ4Ur2d8DfAY2hs0bV2B1WYxpxjDF4Q4och2zRnIJX7Xm8dD27H73byFVo7qW5Vlq4wa40uK7ZbVwSEkI0VuckU1AZj9FvvgeMEaKfme8tWcF6MIKjxPP/+uzY4Mwlo7nkC3BsQGy8CzeAQKfE831GdEW+VksbkLkuyLpvo8AKsYLPeMptO6SY9hCg4Pt4hij6i0Ibf+v1P+fTVJdrWYBWL2RLdHSNFBNrni88KZjdrjg5jqrRGVIL9cYI0KXIQ0m1Bupmxur5Gxz1u1in9TkxdeggRUeQG5Vl0DYvFhhevr8iKksPD9wijDq1Wn+PjY6aTKUVeM5+twFrWqxXBvu+KQ7PKPTgY8erla9L0q8PBvxZF4H5ffXl5Ta1LwkARh05nXWQZqmuosiW2WlMUhtqWbDc5V5dT1sucw/1jpFCMRyPyoiAvUsJAMRzvkOYpQRCRFxW+Z1ivViznK4IgxliQyr0x0iynqkqiKODmZs5kIpBiB10XbufsKY5PjonjNh9/8jnKiwhDS61TVusFJyd7mLpE2y1xyycMJZ5K3HAMiZY+WvhURlBWNbbUDl7iRyA9hPQQykNXGiNEk7brLNCh70Qyi/kSrGU46qO1RQrFfL5guVrT7fa5u7tzQaYNqkwbQ5qmBEFAlqUUZXF/elANqKOua6SnqI0mLwvXIiiFrJ12/75lcIXPGZ+EdIxC2yDE3FBPUzcyYaxuVoTgex6rxZy6GdJB4/7DHU6tteMtNusJ3fgFdKMhcHBSR+PN8owoBBc4It8WT3EPJRHSzQyaxaMrOO4WID0P6YWuXlmBEkHT0rhtQVUV5HnGYj7n7i5xw1SjyfOU+bJiZyekJuPx412CKCArVlSiJu62eXV6wWx6gZAhQdgl8U9IlyGv1huuXr3ki59MOdzt0usqfu67z6jGIaenS8pqw2C/zc1djjQRoUroJAm62jTR9oIis0gRI6WP57c4Pb1hvVpxcvKEo+NHhL5iu01ZLRZUZUWeZvR6PdqtDuvtmizNmVyvieP6K8/f16II5HnBzfUd1hpacRvfUwTKZ7acM+oNCT3J7O6Cqq4gKDG+YrNdkxeZiwOXcH5xznR2x2o5o6pyHj46wQ88VqstSbvHk6ePXWDDZsv+4SECD125vrCT9NDWcnv6hijyyfMtwiYUWUpdFpRlwWDYZzpZkrQTgiBkMStpdxL8IGSxKBiOYkytue1JlBDs7Y3odmKENFzd3bJebpGmZNcMQXrU1qXgtJXX6OdBej7aFo1X3gFFjHUqxKoyZFkGuIKU5VvM6TkH1oVrSCnZrLccHBxwdXVD4Htv9QBCuFTlWmtnTGpswdgv9/V5XhBEEe12m6IsMaZCC+3YDdaS5aUbCr59/jfrwQb5pXVNWRYOJ9b4KILACbOyLMNTihqDsdrBPKE5gLVjA2oXjuJWgc5KqK1BWDfQLKuKvCiR0nNGIOseHKLBkTlu4b168cu5gJTKJRwpD1TgOJHaoK3AF196OI11EvMsTbm6vGa8p4iiCF9KZAiZWbOtNuTlmk5H8Ys//4TesEuQRPz+Dz3+4IdnrLcleblitbqmihKisEWGx3pecHE2o9+PuLrbsrsXI72MIATRjhgMRmBqpFdT5imX51OMzpnMlmxWNUcHDyk0bNYVlxd3LFcrfvSjT/nud75NVlUMhzvk24zhcIhSHvP5nNvbO0ptydKCq7Mp77zz7leev69FEdBakxcVB3sHdDsd3rx6zW16RysO6LRb3FyfARVh5FPJilxItuvCeeuVY78b37BazinrHERNWaUsVwWbbcZgOGQwHAHgeQFFXnJ1cY2nYrabHKM9olYMuGjxMPTpdNoO0y0cYXg0HPLyxRvqCo5PTnj+5mPixGd32OHgIOLps0OEtQS+yw5wvn/J7e0NF1fXZGlFp+WzXG+JlJO3rtOcbtdQVDV5WTrACPeadtO0tAKtrYsWt4K60lxf3+L7Et8PuL6+YbFYYK0lCmNW6zVXV1ccHR660aJSFEVGUeS4K7ij7XiNv8DSKAa1k9eGQeTwZEK/XclpbdDGUYH1vST3/vre7N6rsmiu/zVKuEJltGa7StF17W4Upn67KjTNgE5rTdmEszrIihsa3k/7jbHNUFM0hcaJfMCFr9xzGrUxVLgtgbAW1QwWlXLaBCEVVkgMjjvgvpeHVAqUaWqbG1IulkuiVgDCoJSPihR3mztqCZ1hxPR6xoOjLqNxm7zO+OjZkFYMt9OK09M1dVawmM0oshZSdshtQF4lpFXFm+tr4o5k/7hPknjcLl5wvHtAIEuOdtsIs+X6ckk78UFHWFMShwMeHByT5gWt1oqDg2Nub6b89m//Lu+/+w6z6QStHWF6vV4iBLx+/YrDowM26ynGZETh1zx3wKGm3Bs/S0vmsxWmLqnynNfzG9LNnKOjMbq2UCkKIyjLEhlGhEmI73uoQFJXEaGBLF+z3S7ZpjVJu4OuDZeX1wTNE2GznXN5fc1773xEkWs22y3S8xgNB5wcHTnve5GDdbvyqirBGj766ANWyy1lLdnfn+AFljj2effdEzrdkDJL2Tvo04o6LBcrEJZWEtIb9PD8GklFVpRYpTGNoUY3/MKyqomioPEeuCHbPXm3KCrKqsaTbjW2TbcMwh6eH3B+fo7neeR5gRwozs/OATfwcrGAgvlySVXVTvyjHZDTVz5+4L09jGEQIVCkaUatmygx5YqEMS7xyfMDpybk/gnqZMNYN8HO09R9z8AnikJWi5TpdAqCLxOUmmGg/albSF3XWG0QQXMDaNKF7gEjSkqkFE2eX+HmMNJ7O+OotTMU1VK5VgrjNgPy3kzl5hBCSTwrMULhcd8OuJxKXZVYUWBlRWQVeV7QbnvErYib6QWb8o7Ia/H+o2+ijObm4oz5dMb17TkVklYgeXIycnFnkeWLz87Ic5+bm5Kz8ym6qlnnUAuDqmBebp3PxZvyRXTKoONzOI7ptAyhJ0jiiLgVc9zepTvYod8/QG6W7Owe8PTxI8oy5+L89G0gahwHRGFMp9NmvVkzHPYYjAWffv6KbreHtl/zmYCpNdlmxVm6djMjWSFCQ17XXEyuAdB3a4QU7B5G1MJdNzflnGG/i+/VCARxLLDGR9cCU1fUpSZJ2iwXKen2jH6/Q1Fsmc8mtGLFoB/jS8XN9ZzNcoUf1rRCifZ8FospgefTG+8Q+pYsnfP+ex+SpQW///0f8nO/8ISr63N6owAjS9JyS1mmtDoBStQYu6Hf6RL6HVA+i3XJejFH+AItHNdvVdaUyqOSilKot3FYZZpjaAI6PA9tBbVQVKZCaesizbRhk2asyw2RiMjKjHadkKYp3aTjDDbaUNcVWVoihDvwVV03rQZuDmGcGsEPYmoDZVaCEngYlOcGgLq2xGFM4Pl40jp3pqlAlwhj3UAw2yDqAi8Mmog0y3K9oag0vhdgbe4GdNagq5y6zqlqBzqttW3ChQTSCIdfb1DsQlokDgduhXVFs66wwrkutZWNH8FSy8q1B4rGeGVASfd7YVFS0IoCAm0xWmBN4R4+mRN9CZ1h5QZbQV0qPL8DniNOx2EbXdYuzyZUVLjAnCAc0ks6qEXBNrMc7o/54KOAb304pqoCPv/8mn/wD36PrFiw3NZMlilVrdBGUeYBufWYW0Ucenz+whKFHseH+3S7gjiJ2TvYw3RG6E1M5AkenEQYKxmOhlh8skKiZYuwPaC/t09epqymG1Lr8/Lylje3E94bjBBJ6yvP39eiCFhryTdLFusprXaIlTWr7Yajk4eodkRVK+5SQTvpUtoIYVM8pQhCjygUCFG4SruaoISkFQZIIZmXK1b1mqp0MtZ8myJVhacM7zw5xFMlcSiJg4CiyGm3FEpWWFOTbResK0MrDtjbHbBerynLOZPJDXU94cHDY5LugJ29HV68/IIwPiBoheyNh5RpSroReCoHv2b/oI3XzgmikjIvMZWFOCSXgtRYlmVFIcBTAoRPUWSNN96SxC6FuRYSIXxyI8AqVmlJrjU29JBhAEXJauOcgVpbiqxonpQ11jo1YFGlVNqRhu7VjaYBAllcKq8VTh/gCdOAUD2E8InDFr5SCGowBVYXSF0htMGWBTbfupjyBp662abkRU2728PWJfl2g68UpqowOqMqUsoyIysKitqtRsvS4CuLMAJlBMY2NmBr8aSEhsZUmoqirlxhQzXDVgPSoDwBzabF6YFcIKqwCk9ZfAnSQGELamGxtcDYEKl8lNKIeovOLVXRIU0N+Jb9vQdglsymd2ht6I9H+GGLF89PwYvY239Cv685Oz1n3BsyvfkxRVHR7Qx5dCz4xnsB3f4hN5MNL95oXp9NsUKhK0laOKn4OgvwvTZ66XOzmSPVljCO6HTXJJ1T9vfHHA89hi0HXe30uyxWCxDQSjpsiohZ6vH6zZQwGrGultQFVPKA11eWoP2vByryJ/ZqtZxZYrtN8UOPq9srkJIHDwM8FRIGLSBgZ2efOIKyKCnzjFYvoSot68WCoijYLNfUdUU7adHtdJFKuH7TuBXfcJgQxwFVtcGimczvKFKBVBF+5CNCgfElZ+dXzNcr2u0On7z4gqOjI+JOh1Jr8rpkd38P6QsePzoi6bRZLnr0ewnb7Zo0XdFvtzk63qfYbplOpuD7YEqydI3RhkGnhy+tOxzFhijykIM2Rhu8QDk2gbGYhu+flSVSSMIgJIxj8s0aYy1pWtPZH9NuJWRrQ55XhF5AXmjKckNd1y6gVEq0KSlqp1SsrEZjqYxu1mdNnHrtnpwGi5YGhRu8dbodByRp4KjmPl1ZCkcMrnUjxHErRdHIf9vtBF8JtssFSgm80Kc0pbMfawdXdWae8u36Vtgv03OttRi+RJEjJcI46Wxd66aPVxjj2inlKxQNn7ARJBljqHGkpvtWo6q1SysyNSBRDXAFaGjPAt8P3soMup0em9UGXVs+/fhT9vcOGQ33iKOIu5sLjDbs7e5ydnqGlII3L6+5urrmF37hlzElpJuUKMp4+uQB3f6QVucNKmihgogsLZhNVkwmK7TeUuSaRTrH4rYyl8KAqAmjgEEQ0o3clT+IAoQUeIFHnMTErZjeoMt8MafT7bikJAXZssN2rpnfvvzK8/e1KAIWgxWGVtJiudoQhgkWSZEZBwLJ3ZR8dzwGUTDNNyStHkmrR7otubuZEgQBWguq0rKotmRZjVWSOGlR5BVeEHPy8IiyXvP5Z6/Yph5S+AjbYtDtIT2frVjz4+efs1ltyYuc3BomtxO0kjx99IRtmRN1EshzlGfpJAFpuuDZOw8ZDHucn9Vs10ukrVBYtKmodckmnZFEHdotn7qsOTnZ4/TVK7ZVTl6sePz0iGyb8+LlC9r9IXK6osgd0rzSmtoIBxOVIVGrx2q5JVCKotIcJHtuVSdbCKUxCNKixFlyK3wcsaisCypd4UlBIAQi8LBKOiu0VBRVgdVO72+wLoHIc6Gr3U7XtQXaSXhpkD/WWHRVUWuDUB5CWYRUDTAlpJ20EFZTec7RJ42AKsPWNWXmfpWN3FgK0fgQau4zDYwxzbrvXhHcqP6EQ4rVunYafeOyKoVtAlHu/zxOwqCEamzSoiEcO12BVAKBwvNUk+34ZUakQBL4Eem24OY6pdsO8KTPi5evOD+74Vf/jV/j5MEDsrQkabd471vf5LPPPsP3JKP+MWWqiPwO5XaDEh5np68ZjsaMhhEPqyHDvT1anQ51rpnfLbm7XeD5CVUlmU5XrDcp682ass4pypI0X7AtehSrkusrFzAilUT6TYCrcgXB832M0RRFRaBit/6sqy+d2n/I62tRBPI8Iyu3dPtdFqdrHjx8RK8/wmiBDjy2qxlSSbbrFcaW5HnFzniH3d09Jnd3LJZbOokgDCKkEqTbDbPZCq+leDx4jMEQ+hGamtvJDet0TaU9xsM9TC0o6gxra2Y6Zb7d0u4kFGXG5PaGLM1orVc8EAZtNdeTO/Z2dlA2I/Jhudywu9tHVznWVlhbs1zNyTZb8q27nlsNgbR88O5TrDF0O12qfIc8b2NNjRcKWiqg3YsZD/aZrzNmL8+orSASsM1y2kkHjaTVGVBf3oKWlManlYxZr5YYG1GZAitAeJHb2QvT9Ns1ZeVaosD3CIVC+IFDrltDKL3mqSgIVeiGfhKCMKIVt2l3Onieu1FUZYHnuwyHqtaYe92+F+BZiecHhJEb1ko0Vb51wNIkdi2AhLrIyTebJrHZxcd7nkQpJwy6jynXpm5aFfFWmVm/lQy7HEcrNLW2jd+ARifgoEzSgrUSqXyk524C92nQvu9jPQewRXhvAadOsyLJ0gyjIK9L0k1FLxmRtHqcHD/i9nbCzfUt77//Ee+9+x7DwYiiSAl8ie8JHhy/RzveYXq7YrGY88F7HzKZz6irFKwgDEvaSU0cF+BBP+yy2w+J4i7tZMh6nbFNHT0qzbak+ZqizAnUPlUZcHNz40JjspSiKthmW7bbNVUJYZgABikqilQ3hfRLy/gf9vpaFAHlKcKWIk0Lh4aSPt/6xnf48Y8+RlISKI8wDlgv524g1IAl20kHJRQX55eURel4bsbieSFClBRVgbbu2qit5fzigizfOpWeJ0m6bW6vlpxfTKmQqJ2E4e4uy8WCvNa0BwP6wxFPnjwhiGPubm9J2h2kJzFZga5y+p0EXebcTiZcXV4QRxEY21iU51itSVp9VrMprSggSRyyejDoIkSPxWLBJl2xs7PDwyfHdJJdok6PEsP1zRQ/9vFCj02WNvFoEXG7S5EVhEno4trnS9apSxGurcb3FaXFhWxql8JUaWeasVJQo6hw0t5aG+IgwGJQShKEYePnL7BCEEQhXuA34A5Nmqa0o7CJCL/39gcI7aOEeosgw9MIE5LVGSIK8EzAqtw4dl7TMkggUArjeQgpUKpZjzYsAW00xjTXeqCsNGWlqY1FWyi1droDKxBKglTOdGRoskdcbiJNMpFFIjyJ3wTU1JRulWlcUXHSYqiriiwtSMsSLd1Qcnq3Yrtd0056PH0yZLlYsl5tsNaSZSk/+Yf/NUIYqjqn3FToSrKcpxgjODrcx/MltS2pbMWoH9JuWaJYY4ShncSMuhFpWhLIDcOOot+OEbLFfGFod0eMRgNq26cofVarFd1el812Q1mVvHr9krOzU3b39xmPRwShz8sXr7l4fUmeZYx3xoDl4m9P/tDz90eBivxfgL8M3Fprv9F87K8D7zWf0gcW1tpvN1TiT4DPmv/2W9ba/+BfXgYMldlisLSSiOl0ymq1ZrPeoLWmnUSEoYcVmuV6xWhnzHg0cuKUuubo6Iib6xuyLEVKSX84wAt8VvmMsqrxvZCqNlxcXVIUK6p6Q7ebUFuaLLiMSii6oosKQk7PL2i3Wox2dtiu1nh+wOXVFXc3N/yZX/01Pv/kY7LZDZ4U9Pp9ri+vmS3mZFmOEh7pNiXPSjw/Ii22VGWNMJp0tUEJSVG4Ch9FLtBDSsHxwxMmkwmIkIOHD7AeTKYrtmnJg8eP+YPv/Yh0m1FjGO/vcXd7x87uLnEnojQFq3TJznDIarVBlK7PN9ZSGu0Ohgaaw9S2itIoJ3u2AhnEGF0hlcAPInTDRyzqurl6a7TRpNsNUeSwYxLHJPSUj/FqhA5Q1jR5DwqMwfckxlMIT6IFWF2hqwJpIfR9N7REYAPfmX+wWElj9HLaBG1w8xErnJ4iL9EIKmPefo5F4HkBSNncGizaSDyaYaL0AOVMU1I4CAwOqqKtW4feQ02MdnL0VhDihS7kJssqLi5XpJsV3W7JRx9+iBIBr169oSpLdsZj6qpkb38PYwyFrrBW0W73iGOfKIwR1uIrSRzHRO0utdAEypDVGeu0II7aWF0jjGTQH6ONZbNZURcL4iCi05LM0zWVsSRdSdLR9IdddnZ3ePCox2RyQrfXZTqb4Xkevc4j9oZwe3fLYBARxxF/42//4afvXyl3wFr7P/qpIvF/AJY/9fkvrLXf/iN83S9LgNVokxK3OqxXGcvlnO9//3t40kfXJXHkIUSNH3jM6owo8olij8n0hjTNCIMQLxBkRYHQwjH4Wi1MBudnVygVoLX7S50vbjA2RxvBZLridrpgvS2pkGQ3d28HQM+ePiNdb7i5uqEdRiync44ODlAIOkmb9BZmdwuWiy3TxZy8qkjTElvnlEWNFBE7oy5ZK8PkNVVREgUxdVFzdXlDWZcEQaOKHI1p9/pM5guW6zmL7RoZ1PzZv/jLfPH5a7qdEd1elxfPX7F3uEMnafP8ech7773LOp0SxBYtMkSoyaoVdd1g07WlrG3jyzeOrGMFtQ2ocWhyJQRCRVjjpungY7QhStrYyjQBHq4332zWtHd37728Tt8hnMlJ+iECgxKu13bJQBVKWGgOv9UVReago0pIKtNAU4TE4th/96Yi07AUaiOaQFJLUdVkZY0VgqpJHbpPkBbiS8uw4wyCFQohfaQKXEK1kM3coYkva/IC79sfRyVyAzlrLL4KMNRM7u6wZounBGFkiaIOngrJi0vKKuP1m1PeffaMvCiI4xikZDQekiQRUhpaLZ+dnR2m8xvGgyH9nR4vT1+SliVWS9aLFVlQ4vsx1qR0uzW+7xG3QkbDPsJqri7OuFtpVqmh2+vy+vULojjmPf0ug8GAk5MjttstL18857PPPufZOw85Pu4zGjlF7cmD4688f3+s3AHhGo1/H/hz/78c+n/+paSglQS0kzZWK9qxReuCXjshLzRa5xhbkrR77Oz2CULBdOb6dZezZymrDG3c3nk+X7Af7KOUz2q9pcgzBIp33/uA5WrMD374u+S5ZbOu+OTjNyRJFy0k3ZbHMq/oJV32xnv8wetzVvMlwRMfXVa0/JAXn31BN0kI/Dabder6tarkbjZDKEWSjBBCsllv8GRNtikxRYWvfOpKMF8uuDi/Yfdgj8Fol8U6RfkR26xE+SHr9IpOt4sXGobjNuUna44efMDh0SF7B2MEikcPH7B7MOL4+JB//Jt/hw++9YC8mpKlGWEiqNYVaWHISmd0kjL8cgdvDVmhnWHKKMq6otACaxVJ3KasQcqQIPbA185XULtcvcOdXUxjRnL5CC4yzAANDYXAb/T4dYGuCqoiQ+uSIk9ZLRdMJ3dM76aNJ0BSlrWbLVhLbe3bpztN36+toNKWqtYUlaFobgl1Mx+4N1zVxqAapaE1rhC5VsBzMeo4BJm1Fl1pN1S8J/kIdxu4x6g7paJLgfI8pzvZbkukMIzGB9S1ZLFKiaMOrbjNb/zGP+bZO++y3rjUKXTNcOeYm+uc2WTKcPSQYBtijORg/4RNvkHYEKs1223Gzv4DkqTjYu3zilIbsiJDmxqhfKbTBZPpHZMl4LXpdXewuuD89JbryxlPnzzhvfffp91uo2hxd7Xk0bFAtgMeHu9TlAWh969PJ/ArwI219ouf+thjIcQfACvgf2et/cf/si/iBz4PH5xgTcCof4DVPtPbJaEfMBgmnJ+/wvMsiIper4U2BXd3M2qtiaLYhYqGHm0vZrVaEychta5oxW2S1pDLyztAMejt8c1vfovT01M8FbFYZixXW8pK8cGHH/HwvWNm0ymb1Yrnnz5nPV8x6g6osxJT1Nxd31IWBU8ePWKzrkgzzfXtlKKuWW3cXreXaqrCsl6XLGZXhJ5PKBXJIEGIkLwwtJI+R4ePePLsKYPRHvPlnMl0SRwn9AYd3nv/HcIgJggsUtZEISStDj/zM+9xc33HeK/L/uEIY2uSvuDJ42PaHcmrl6/pD1q8fHFGUQoCE5AVkOUuysxN+hM63QEqiED6ZGXGNiuxumQ82kGbmrjVot0NKbKMsqooF3P2Oz2qsqAs8sbi7Aw60pNIVbpeH4vyFOgKW9eNYCunyFIWizmTyYTNdguq4f836sWqrpsDz9uhoHNQflkEikpT1oaqNlTaoc7uY9IQ7sktagdwVVKhhAThWgGLwjRRbuDQb2V9n23QmKmEQal7iIttNh81nvSII7dym6/mhBd3RNHnTO5uePDghNGgx/7hCX7UYjQOmM0m3Fyfo0JFVeSo0Od2OuPlyzds0g2rVcnV7YzA79GKA67On3NytMN4Z5fbuwm3kwusyOn1euTphul8Q5YW+GGH4bCN8rt4ssVw0EJXiqvrK87e3BL7PZ6+85R+e5dvf+PnGfV22MzuiKTACxJef/GHzwPgj18E/grwn/7U76+AB9baqRDiZ4G/IYT4yFq7+uf/4E+Hj+zuJHRaXRcjZXykCGmdJO5K5nukWZ80XWOxaFtzN5lydXVNEIScnDxwQxftWICtdgvpKcqyYtg7pJOMWc5L1qstn3/2EmM0g/4e7W7Ex59+iucFtOIOO+M9OlFC76jN717+Dl9MPyOQil67TeSFHOztc3t9TbedcHdzx/XNgrwomUw3oBR+0AEUd7dr+r0e/d4us8mUbmdIttlyc7cgiNv0+jsMx3s8fPIMP4zo9UdM5gtm8yV92yeOY4JQMhx0MbXk0YMjtus56WbD/t4hus5JEkUUBbw5u6I/bBEnisfvHLLN5ihpmc2nCBEhRcJyXXNzu6TIa+IoYHd3l17fIa2tAOl7VNYQBgFWSvK8Igg03X6fhdak2xyrXTzYarmgvbOLNRqpvAZI4jYCtW4AI8agq4K6KijylCzbslwumM2mpFmKsRYvCJ0foaowzdVeN4nHUimHHdc12kClJWVtmiLghoKujaDp58VbTHtR1Sgp8ANJgMQKhUVhhcLwZe6ANs4RKRxO0hUSbZBSo6RCYBtPgjNIxa0EZMA2rfjixSnz5Zr1asl0vmJ3d8Sg3+X16RVxHHJ5ecf55Ski9Kjzkp3RmM+ev+Ds9WtGozGLRcHN7Yb5fM6jJ0+I4hGbTCBmGdvM4IcdvKDNzu4Rq+WcvNDEcZeqLhmEfVqdIZO7CUEQcnCwQ9IK0LXBU4LtesXezoijg30uzy+YbizP5+f4vs9m8986gn/8IiCE8IB/F/jZ+4818WNF8++/L4R4AbyLSyn6Z14/HT7y5GHflpkz92w2GXHsNcGMW+bzmZvmKw8vCEFYfF8Rx1ETkOlRa8P1zTVaW8bjXWpjsFKwu7vHel7gCUUcJizmK/7e3/11Hjza52D/iB/96GPG413GowNurm+x5PzMN7+FrQxFWnL08IRuu00QhpwcH1LmOcIarm9vWW1y1mmGlSFSeQSRu25ZpNM5aE3gb6kq4z7PWmbLDYNhn6TbQ0iPq+tbDIbeYIhFMJnPGY0Dbq9vubue8sG732BvPOazT74gjhK6SQtParBFYzLa8uDhA1aLBYHvMxr1CT2fPM0w1kfIhPWqQiDYrjN6na5bZ5qau+kdRZkRBj6lLuj1B6RlSpptaSctwjhC+T61yegkPW5ub9gJItd7S+fQoxHxCOmGcMI6s1WVZ1R5Spm7FOKiyNFaE8UxRVE7l6bnoYTCExLPgCkdl1E07EN3Q7CUlaAoa4qqoqwN2tL804BHcVxDF2BSY1AocLcDIZ1D82170WQbAEiBFB5Kyaa9cVwDYbRz9BnNPWzB90P8MAEVslwuKOspuq5YrF5wdXNHr9OmrHJacUSW5eR6ys7BMTeXV6y2Bek6Y+/wEfv7h2QF+H6X5fqW+aKk29vjbrphttjS63Z4+s4HGFOzWG7Is4r9vSOEsFxcnLPcTIg6EuGtiRPBcNCmlVTkeYkVS6SK8HzF1dUpWb6l1YrJphukhMPDwz/5IgD8BeBTa+35TxWGHWBmrdVCiCe43IGvlio1Lyl8pE4oCk3otdgd74E1zGYzrm9uuLq6Zm9/z4mBdM5ouEO3M2C1Xjeac4eTq2tLUWr8IOJgZ5dOu83ZiwuSVgtMgZSCskh5cHJCu52wszMG6zMcDlkul6zma4y27O8f8urFc5J2l+OTI85O39BqxYStmM3akYPy2lJbiReGICCIW2hdE/g+WZ6xmM7IthuqqKA3HNDudBESsqpEZikX11eEkc+DRw+Ik5hPP/sEX3pI4fHqxSllVvFg7zHb1ZYyzXl4+JBssyYIQ3SVMd3MMbok8Ed4qqAqSg72D8k7GcLAYr6llQxZLQuksGTbkjiK6PdDbu+WpNkMbTT7hw/odzsEkXLEGqnpDtqoIHBiGxzu/Ob2lgfP3iMInPNONsOze92xFA475tKHCqyp3Hwg8F0mQqdL1GqBDDAECFyCs01TpBcSVBVVWTUH1q39XCEwlLVxyPPaiYK0cVN9wz04yEWfuUAaJwCyDZGpNiBqi1W20Q04YpKUPlK6WDUlHWLM6ZVMQyd2DkUwCBHQ6Q0JWzPsassmdU5Gow2LVcbF5Q1g8JQijAL8xPLi9Jp0vUXbkMPdA77xwTfIsoyLi0ta7YSf+c4vsFiuWK0z1psNnXbSWKQdRerm+gYlYTQcEMchSgouZ68IWxnjwKPb9hj0FZYayNH1Fs9vM5vO+fjj32M03KfVGjOSLU5OThgO/xhkoT8sd8Ba+3/GpQ//p//cp/8q8L8XQlS4tu4/sPZfYF9qXoEf4Zkuq9WEKA7xZMjNzYTPP39OVhS4ZOyY6XSGNim+7yOFTxC0KEsDeBwdPUJbWG+29Ps7vPP0PdaTBVoX+CpkvZrTHw757nd+hk4vRAnLs3eeoLXAGIUQXVarBc9fvOTho8fMZzNmiwX9QY/FesVqu2qkx336Yczt4g5TabR0qb0tp3ukKHOqMqMoUrCand0Rj56+Q6k1SknSbIvyJIUuONo/4PHTR1zfXHF5fc63v/1t8vWCF5+fsjMYc3c1Y73ccLBzwMPjB5ydn+ErxXwy4er2yglggh7HR+/x5uUXBF7EfDsnzzI8aRh0Q+IwYDTsUhY1ZVZS1hWXVytqvcILfA6PhxweHDC9mxD7HTZLiwo0QjlmX1lV3N7doryQXq9LGAb/DBHYzQYENCEqSgp8T+IFPoYAU/vErRa9ft+tFr2IsNWjbhyB+s4NNP1GtGOKEiFMY6t2UBXXFljK+j77wDTKwSYB0TgrsC9lk5p8L2G+j3bXzhVpG7yqkCAsWgtUs1ZwNxzciqFRDmpdO+yZxcl9212svGO7TQkDt/pzfgufwHeCoyyvkEmbV6dTQt/H9zVj7XM321AUGaWGWPgcHJ1wM/0BEsHjBw8Jo4BWHLJZLVESklboBpbabRCSOML3Yb64cQ+zfE2nHXBwOOby7II4bGNMznRyyWjYQVBS6zVSWRAFy/Xtv3oR+IrcAay1f/UP+dh/xlcFnv0LXlL6KNtG2pTTV+dstylWwXS6wA8iHjx8h7qyzKZb2h2PsjTcXF/j+QEGQRDGjMY7ZHnJfJGC8EEEbDcbMFUjRfY4OtzDDyWb9YLeMKHTjmi3B2zTAs/LyPKMH//kY/7cn/+zHD96yOXFGaeXF6zzDF2XaKOJ6xbdzoD2oMcqy8mrkjTbUpmKMPTxJQy7Hdr7Y/LNhsGgQxD7KBm4+Ua1oT/s4gWKdjfh+vaS3/nd3+Ly4oyf+ZmPsEYSB13GgwOKtMYjIPJi8k1OXdQsqwU//vSHzJZzxnu77Bz9HEm8x3b9GbEvmNzOuLu+YWdnhCc1e+Mu+/tH1LXl5vqWi8sr4sjieRV+IBGqxAs0XqAZ7XSxNme1nVLrh2hryPOczWLL02fv0u12iYIQYW2zVnPkHtFQewUGJQXS9zHWp6wysC4Uo93pYIzBjxLCTFM3seSeH3L65pQyr5yrUWhAIqxCNKGhlXHDwLIZIupmRYl0SUiuINEc5vutgkVrN/mXooGNNEtFKZw/QGtLLZrQE5yS8D5s/f5GoHUN1uCHMd3eED+4BJEilI9FUVUV7d6Q0PcB69R8a0ebzrIKQYaSt7x+fcFw0CMMfWpryT/7lOcvX3C0t0fvyRPHSLQ1vrIoBcmgg65LPKGpsjWTyYSbu1uMpxn0+8ymC6S45tHJQ3zVop30KfKK9apgd+eEPC8wVpDnOa9ev6As/+QDSf9EX7rSREEPjw1ffP6K12eC0d6QIIhoJV18P0IpRZppwsgDq3j58g1CeiTtDkcnDxF4pOkGaxRGC1aLLVKA1iWBL9nb3WM06rNazyjKjKr2WC5ntJI2nU6LzSajlSS8ePWSsqpptd2VOCtLirqm02ljMdzOpmRljREhcRKz2lQusDMKkAo8TzIY9mkFPptAUdcFi9Wc3aNDlqsl55fnZPmGJ+88ppVEPH/xORdX5ySdFucXp3SDAd/48NuEXoAnQ5QvuL68Jt9m+KHH9eSa3/7N3yPpxXhBQL6VvH5xzeefvmG12wGrGI928CSUxZZBv0vS8gj8kMXsBt/XPHq0D0ENStJq++Tlmv4gwQsgjD3WqxXbdAtSUlbOnlvVNVHkWIPyLSfM/cs9DORL8nDzMSkJggDPu9e0W1oogrQiywuSuMVwtIOxgjcvX1NlRbM1aDBj2lKUFWVZOcjK/UBQGzTGZQxK1ZiZGgOQtU5p2ESdKXs/D/jSi2CFQQoPIdzKsK412BpjBB4K4QnEfSvA/b4AhqMRnW6PoiEjV1UJCKrKEPjSZS1WGgoI/YCyKthsK4rsBqsL1pstxpT0+m0QsFjMyNdrdJbx4Qfv0h906XZaGF26jMxsgzAF08mE169fMS1S9h8+YNg7Id8ors6XpMs37Ix2qaIQW3tI2yHdCNrtIZUpiFttsjTj7PzsK8/f16II1Lrm8uaCL158zsvXF+wfjcmqO3b39zg4OKI2Gj8UJF3JbHJBv2WxVUVaZfhRhPJ8hOcRRzHDrmXU6uCVNaNen+14QF4XyCBla6/QUUmdp2wL2GYpEoMyNYu7K+LWDk+PH+LXFk/6xAg2WY7JS9o7O5SmZjKdMZtfEgc7+GHIXmsXz4N+N8HUhRM3hYqqKEjaCdbCdrsgzVqstwuW6YqbxQTViQm6bRZpTtzu0mm1MDUU5Dx9/x0wlsV0wWwy583pKbu7eyTtFi9PL1llGi9RVLUiT6+Zb5dsNze83pzx7jsP2B3sMZ9NoK4pthtuL84x2nLx+jVKSj569oh3Hh+8XQfe3N4hdE25rrFlji0r0ukcUVUEgaTIa0QgMUo4JiIhWJdmbKkd5KMJ+tS1dmlKKKwXoWLl8gn9shH1KKRfg5eR9Aa02j0qFTNZlmwvr7FCUeqczMCmrkhrzaYsqHSNpQl+bQ60lNahxBoDkpGS0gU2II1G6gqpFJ6VSKMcZNg4lJumRuGUiG9dkUJSW4kwAt86D4FDlAm0qeh024zHY5bLJUWRU9capQSrzZpNtsZaDcrgK4muS8qyIFeStK5pxSG3izVGV2xrZ35SSrJcTsAa3nn/HUTgYZQE5bPdrpit17C0XF5ccH51hwx80kXKZpASR23m8w15ZZitNuSloS4rkIrLq2uOjx6wTjVxK+Dhww+xJMBv/KHn72tRBMq64Mdf/IBXr9/w5vyWVrdPIgLm8y2zxZLd/QFZuebgKGGiAzxjeP/ZU9ZlRS0UMlAIKdjZGZJ6isSz9AKB9EM63YRssWCZLyiDiEIbKiFY3q1YztdEfogylk7okwQhyXgMWcp7zx6j1zPOLwsCGyLqijLPqMua1WpNFYb0uj3ef/ddTJ0zn1zjSYijEGEdzKPdGyKEZL65Ybud0mq32Dnc5WoyZ7pJuf29PyAJQ6QX0+0O2B2OWK1mhG1FXVmul3f85LPPmE2XbAE1VSyWG3aPHuD7Pp3eHjsjw+lmwgfv73N9cU7oG0JfsDMY4CmPqqxYTmZsNinZKmM8HtKPXEuTpSuWqwViPScvwQvamFxTZa4IBFFIGAgmNqO720cr6VotGzgJsikwtkTYEluXzUrNYppkJi3Bep4DkKAQ1jjJsBYEocFrt4nHuxyohMPLBfNFQVUtqYXPVhvWWrPWFZuqKSAWrHHrRCcmEC4GruEcGeUIVUI1RaCuUJ6k0qJpCzyEFejaJUBXMnfJwZ7C9wKkHzYGpAZPjtMbKCWp0HhewO7umPPzc5bLFUoJXBS9JstWhKGHF3p4VFS6dlsSX2GMJdfGqVujCFMK8lyjpKTlhSSDEbX02JY1y2wLGLI8526dsd1sWW5LCq/FKImp84LzN6d0Bj1anRZFXZKvp9zObpEWIi/ADySrzYbLq5IsvSbwe3znu7/MV6UHfi2KQFGWnJ6fU5Q1SMX55TW7ZoeiKknLjGfVI7r9GM+L2Ds4QRlJJ2hRb7eIIGBnZ4TWJcpTBLFlsrxE2zV1oZgtlpS1Rnk+ddkw44KQeTpF14LVMmV+d8toMCJuRbx8cUavJ+kPQkbjFqgReIJtUWBWBdHaEsdQZmv6vX329oZcnp+yXC6wdc3OaIwnPJT03f7WV0RhhCcVnXabXidnm1WYuuby4pJRr083Dgn9EF95WAPz+ZKbmwl5XrJeb7m5uWWzyXjw4CHWgud5jEYjhsMB8/nc5RI+eYTQ2gV11gbP8/GUT7otCAKfpN0jyyHLa66uJmy3Eel2xWRyh1ABrVZEUYFQAuX7lGWJwbDZbgiCgDAMabXbjlHgNS4/7Xp7YZyXwAKyyQgsjXZBrliUCLDS9deOEWIQQrmf0VcMRwPe++BdJpeXfD6bUBtDXpakWUGWVVS167GtthitG+6AazekuqcHiEYyTBNvbqiFdjHoypGLHYrAWQ21dWIgbSyRdOtm2Uikgbf4M2vdCtL3HJh1PB6xu7vDdDpz60UFSilHrcaZn1TTzgjhBEpaV6TblKKqsAik8vH9EF1XtNs9ikJzenrFbB6TZY51oeuK+WLBfL7AaIPvB7hYzID1OqVC8uDRQ374kx8TBiGtuEW720UZODl6wKuX50g0p29eEfiWv/yX/uJXnr+vRRHwfd853iwcHO0zmc54c3bJ7t6YpNfle9//MY+fnuB5UG23RF7g1k1Ccnh4wNHhHs+ff871+pZOO6IwK744fYNfjalKyHRO2BUIz6PbGeJ7CbLucFXecH054ez1Ke89e8Le0YBWG4pqzun5Z6T5lDDSjHd3yHRFsgpYrG9ZrzVxz+fR4wPKckOebTk8OOT2+paiaIY70me7yVCqoDOI0UXN9HbC5OaWPM2xBnaHIzqtFrYsKdKcu/wWbTWff/aC6+s7Dg5cxFS/nzsIaOj20K1WxGDQByw/+v4PSdMNe6MRnU4XYTXb9Qbf8yjyDWVekrQUnheSFTWb9Ybl0oOLGqMrtKkZjXsYoSjqAiME3UEfnRsuz07ZlCmtXu/t90/TtNHEe18eEuHe2EboZjBXoa1oGP8S5UnAc6s5I6jrohnoC3Rd4QeK4wcHfPTN9zl9/dKFfZYV621GVUmMdZkIutZN+KibP2jHOXJtiTVgVAMyFdQalHDDQW1EE7AiXeugBOrt9N+xLSttELp2swVrqU2NrErKykPUFZ51JqBuO+Fgb4+z03OKxsBWG4sQzd+HgU26dZizn9JS1HXdFASnaVCeojY1upacn99xcXHrouUDSRj6aF2Tpimr1RohFXEUU6U5SiqQAq8WDEf7PHpccnV1SZJ0OTp8wN3NDZW2LBZLTCXRVUpdppy9ef6V5+9rUQSSJOHX/uyf5Td/83eZrzK2RYlFkJUaIzwuriYsNhs6nRbDXkToSb71+IS9/X0skK1X2Lpku1kS+JqkF7MtlqxnKfnaMN9OaA9DbGBZLSqOT94hifoIs6AVD/jlX3pMEBjiWPLg0SGfffoTynLF3t6YNM3IshUVFl2mFNs1voTjkzHWpCzmW3Z3x6zma6TwKAtDNEgYDgak2y3WanYGXW5n18yXC8ptRidOkMpjb28XhaDcbhHacvbmDe1+n1dnZ0zuZmzWBUmrw3e+c8LBwSGvX79GSsn+/i6r1ZIiTylS9/UCP2A0HDK5u8UKiZQB2lQYoZgu1oSBRqMotUDnhsV8QV0WaFNT6MCl8XYH1NawzTKMrqjqstHUQ7fbYTa5ozYxYdDFD1Rz+BUSF2qKdk/7WoPBCXWMFXjKd7t4IZBaYG3mhvueoqpLjK0JY4/3PnqX1y9fcHp+xnqbgfKhvh/2uZDU+9WkweL2CADWwUmsm+5bA0bYtwlRLpDYDQel56G8JofAvzdHWcqqAiHw/CYFqclAlEog8gDrB/hBiC4rBr0+O+MxF5eXFEWBMRbPD5ufyGJ0ThCGBEHoWiTtICV+k/ZU14ayTCnLituycng2DHGc4wf30XNO5HVPfC6LCmrNxfWMqq45PPZYrQuePH4PrUUjglJ4QczF9R2L1ZrTL86R1Pge/JN/9N985fn7WhQBIQV7xwdsy4z5eokfRbSSDp3+AINiZ/eQi6szJrMZR7/8DXpdj0KvyPKI7WqL0ZZu3KLlexR1wWQ65/JmxeZ0hSg9tDIkUY9KVCynWzqtFEnNd7/z8+yPdzjYHbPeTDByRlFU+H6Lly9Pnf8eTZ5rvDDAWsHBeJ8HRyFJf8iLl5/x6MEznjx+xD98+RskSZd+d4CSUJYFnifYGe3QH7TYbJbULUsUJRghuZ3MuD2/wmrNqNenFUasZksurie8Ob8AocizN/zMt75DkVdYA4cHR/iBwpiaV69e4inBQb+LAOKwxWAwZDab4wcxy/UG3wsZ9XcocteT5/UK4eVOeKM9sqJim5b0d3yCuIsMfIQpWE1XjFtdHj19wmKzIKs1/X6Pm+trOt1D93QTwFvmv3Q7e+lShI0VSBU44Y/AUYekdIQfIVBeTCgNfhBghCbLt2jl0ekn/NKv/CI3kymvLq/Js5xtWjkQSJPWrLXbEghpkcblEtwv+YV1Q0Nh3D7ACNvgy5tEMgRSevi+i38PbIQ1hqoqMVY3aUZOPeiUiA5sousSkecEyifbbEniiJPDI6YTJ4UOgoiwFZGXGVVV0ooT4lYLz/fZbFyP73IeGuKRtuR5ied7VJUl8GKs1biEOPHWT6GUT6/XRQpFnpdkpUFuNGVdYS6nbP7rf8Ljp08wuuLq8oKb2ymHe/ustyl7B/ucfXFBECjmszuK/F+DbPhP8lVWJRc35yy3SzrDLse9EYP+kNF4xGq94Nn77xO0AiazK+KOz3A3oNtRzGcXTC5njPt7ZJVhvcnYlgUvTl8zXWTEaZtu6NFudzjaOwIfXp6dYUrnImuFLYq8xPcDlBdS5D5aB3Tbe5zVN7x6dcM7T58Qh21WqzVWKHZ6J7R7XbZ6RbrZEPoBWZozHI4QeC7x12pqXVBXKauNxtoEX3qcHB6B5/Pq9JQqy9nMl2Ase/0hw+6Q0WDE8uIKKQK6vT5Jq8NwMOb7P/g+f+O/+C/5s3/u13j0+AHGVPR7PcbjEX6Zcnd3y6tXb4iiGM8LG9PNBmM0XRXSH/TZpjmaFSoMWazmrNKCbFsSBAmD8R6VgU8//gn9UZu446K5pQC1UKzSDGM0fuC9NQm5fte93CFzrj0hfYQvUMrFvEurUF6IELirrAbPi5G+0xoYU1GUW7QFvwUnTx7w5//Nv8Dry2v+1t/5J2y3OdJzUFkrJJWum9WfQGgDtcZIgd+YAASuQIBFGRphkOduJlZghXLZj0rhu2hl/LrC6BqpAFybgLBN4XE+AmksnnDJ05Hvc3x40PAfoLIGL/Sp6ppKGOdd8F3EHDbF90IsgrzIUcrDUx5x7GZEVVFR5SVVZRDS4gUBcRITaMda8IOEqnE9ZnmNEBJjFZPZltOLW16fXtHrd5jPJigpePjghDzd8uG77/Lu+4/JsxTlQRQFX3n+vhZFwGIpdMrxw30EAe3OkPF4D7C8OX/F3uGY8c6I8V6Hwchns72g7SuUEbSDgEQFfPLpC16fXRG0u4Rej2998D4D20NnBYv0jrp0FTpQPtO7Cb3ODrc31yRJRFnu8fyL5xijGY1G5Pk5H37483hS8OTRI0eP2b4kilqcHD8gjEIuFi/46L2PGA/HhEHEkyfvcH19x3x6x8OHh/S7IavFDXm+YTHPCIOEUX+AlYpPNimmKPCFpN1JSMKYfJMhEDx59A5+1AEr6XS6YAVxmCBFxHw6I0u3dLsJSTsm8H1aqs0yWHJ2ek5Va7r9LvPlkizN8b0QP9ywM2oRRjFhKwJP4fsRR8cPUfgEYcDewZi0XDLORxw/2qfVDqjmFWVeUFvN4yePsVikbPJ6lGqgn+6qarBOqmsdKFUaTVXmSOXheyHS89z6zFqQliBKIHAxX9o2aUdaU+YSaSVP33nC/+Df+3eZrFJ+63d+yHaTNU93l02IG8qjrUtBVtYZi0IRuGyGZoXo6oJoQlbEW8+BReAmeu4mE/g+WI0Q1tl3dflltJp2UetRYDBlhR+EoDWdVsQHH76HEZbLmxuKvKSuDL4KkdpQNRH2ZVHi+Q5OYg1o60Jxle9jDCjPJzM5QRQSxyGeLwkClxKltQWpyIqULC+II4/VtnIeB+VuVItlyjYtkNKyXi9YzNdYodluMnp+QLfbptVp0e21v/L8fU2KgGG2vOXgZIdW3KfINf1BQhRFPHvvCaOdAe1eQL8XE8drpjczJtd3PNh5xDe/8202s5zFcIs0MUYF7D94yP7JCaOoS75e8eL0M1pdn3a/wzP/CWleM59tkcIgpeHN6XNubi7BBoxHe+yMj2m3Yx48OGZ/d4+XL15wdPCYJOkgheL6ckpmKqKwy2Zd0G4rzq/OMVrz3ofPCAOLtSmdfsBh+4DlXYkpfQIvYLFaM7m+odimHB2d8OjkAbvjXTbrNboyBLHTtFelexO9fv0aEHzrW+/zK7/yb/Bbv/1PWa83HB8fUlUVs8UKowV1Ddc3t2zyjMVqhRSKvd0uaV5wN5nRHwwYjges1huCwDDq7aKkTxiFRLFPV0WMD9vEHY/nrz7j9mxKkrSo6or9gz2SpMV0tqTdOXjLq3N5gAolQTbUXyEAU2PKEqSP8v2G2GNBOOhp1AqwvqUQWwQCT7oQ9roq2ORzgmTIz//8zyFURJb+J/z+935AVd8nFEmCwHeHyRhHLNZNcKqvENYVKk+5zYRuzGQItxGojKEytlEGuhZCKeluKRhHWlKga0FVWWqtoSgIvBysIJGK0hj8KOb44ID5YsFsPme1XqOrmlarhScMVVVRliV5ViCKCovF9wNHOa7cFifPckbDIa0kwvMkSbvl2hwhqKryy5yFyq0cRQFJK6Y/aLPZrqnqgrI01LUhDH3KyuU6JO2Es6tbrrRhNBoiFTx8+MeAivz/41UUOVoUfOub3+H55294/eYVOzsjzi+uCHwIA+f42tsdYyuF7D1kls7wbRupfSgLfvab32Gx3fLF6zc8Ot6lN+pgC8Pe0ZjDJ0O0yLm8vmS0MwQRcH014+bmluPjAUni4/mam+s7FotdfuVXf4nv/cHvkRUp13fX/ODHP2BnuEsYRfzkxz9ACEF3r4U1HqtVRm2vSTqJs2sqjfAtWbom3UyYr2paYpdhd4dWnLDNMp4+fkKaZagmcTfLMtbLNVi4vLyiyJ1Kbm9vn+024+bmnDzPMUbT63Z5+s5j0u2G29srBn6EH4TErTbD3SGL9YL+YMRmkxLGLVpRm/Viw2q9YvdoRLsTcbuZsFrP6HX6LBYrRn4fa0s+e/kxo70ucRzQ7rYRQhCFivOLc1rJkLjVZdjvI4VEGEfmkUKi63vSkGiMN673Fp57ZFe6RNc1nvJQnocwEu3V+F6A0c1Tr6owZYUnDMVmjagsH7z3lP/On/8zvHp1xmR67eYNtcFvBVBbB0+1xoFPtMZ6wv0sVuA3Ia5V5cRL0ncMwtpYKq3xcSpD01iGhVIubNUKFBLP95q4+hI0FFmGFJIiz/CjECEMZZFyfLBPWZZUdcViuXKrSwTSOhBqFEdUZUXUarHZbKm1JgwVvucRBiF1XaI8QRAppDSUVe7wc2mKlIpW3EZ5As93K1Xlh2yzgna7RxB4DiXfAEjSNEUoQZqVBF4ESjBZZnR7CZd3y688f1+LIhCEAcNBm73dEZfnlwwGHbbbBYEvaEUtQh88JdFVwTDZpyMGTF59n80ClkHJ6Ztz9nZ3GOwM6PcknXZNt11gwpiqKlivlmzzBfPFhLTY8vrVJXmuee+990gSjzCCb37rGcPhhLvZa1bbI548O+D7P/geB/v7fPdPfchPfvQxKqgIIo2Qina7Tz6bE0WSo+NDWp2Ei8tTZATT5S3XFy+II8HB7hhfB7w+PeOzF8+JopDxeEiRl6xWa9I0ZdMM8dI0B2Pd/1YpKHP3+4cnD7i6uuL73/s+abbh2TvvMJ1MOT+75N1f+GVUoDh+9JDJak4+m7ItVvhewHqzYdQf4w0Uz198jhdbHj15iK0HBF5Enm65u7vhdnKK35KEgUe326GsM7yWT57mDLt9NtsNy9WCd548ctfxRrsvGjxXVTtNgBBuYiiVe8Naq5DCUtcNChw3G/NCH+m5LARdubwDaQW2rihqjaGm2qRYFfIrv/RzbNYz/tpf+8+5vLmlHXuYunJDQG1QviIMAvKyoipLvMYZWNc1UjqqcKVr5zFQksALUb6HFu49pYRESpBK3puNnUzYOOqxafJPjLBNcGqFZzyqMsegaSctTo4P2KYbTs8u2G4zqGuqqsJXnms3q9pJplsxxliiOHbEIq2pqwILqNo4/YWtkdISBArPC5p1oQuC0TVuJlK7hOFWa0g7apHlIWHoEQQBk9mEqi6pDGjpue3Pcst0nX7l+ftaFAGsZbmY88UXn2Kqig/efUaWloz3dgijkL29HZaLCTfXN/T8IYHs0u8+JFSSF68v+OzzT5lvp3xn8CFP3t+nqGe8fPkFRyc/S9Tt8+Pf/4LTi5cMBn02Z2ecn93Q7Y4Q1JyevmB3b8C77z5ld7fH3/v7L/j409/i8OSQTXHLupCouCLqaHaP2rR7xwwGY96czimKnMfvPiWIPN5cvGSbrfCiDmESYpWgqg1+1GY7Kbm6vmabbukP+nR7HQSCvf09ZtMFp1dn7O7uo4KIk90dLq6vGI1HvPvsPZbLNe2kwy/8wp/iJz/5MbUuub6+ZrFYsr+/z+fPX7F/tM9eOyErStpdl0yT5inddpe7u2t8PHqdNkWW8frFS4cvLysuzi+alZjBaoWvAmZ3M6Qn8AOfwaBP4HlcXt4QhiHdbpdmKc/9VNDYe/GPaGTE7jqr1P0cwCXf3GcJWMAq6/px64Q7Cg+FQ7pbrbG4FKhcb4iCHv/OX/qLCF3w//zP/i6n59fU1FgJYSBRgY+ULkXKlE7846PQpqKqLbXxKevCRZHVlkAEaGEwukTiICaOmipppgVIXMR544CA2nEIbBPGKtCYusDYCqUk/U7C+8+eIrB8/JNPUEoSRYFDmdUVSgqCICQIfNabDQKIopAwDKh1QVFusNZFqWldu9Yg6eEHEUbDdpu6jwcxnhcQhSF1VTKdOK5B0o4JgwFhEIGVhGHL6TaMQKmAokjJt1/zIlBVFevFCnUkefb0HeKow9XFDS++eMHhwR67wz5ojYdgvVwhqop2b8BidsfF7BbZDlnrDZPtLR8+e4/1OmeTZaTlmk1pSPodummPoi75+NOPCf2EskxRCgJPspjecn2liOKAR492CJOQTt/j0Tv7LJdLjIgYH/WZrK65vb4jTHzmyzsG4y55uaJYr7GqoCIlraDbTmj3xsxvb1kuKjbLGuF59AYD+sM+e7s71FqTphnb3IWS5kVFmLTQpuKdZ495+vQZ//Q3fpvLy2t+/ud/gYODfT799BM8z+Py8pL1esV49BRPS6ra8ONPPmW2nHL44IDeYMDt1RXKcxjvMk0p0pR1tmK5WXKwt8fTx0/otSMWyyUWQ609hIR+3Kfb7TJdLCiqguls0jjyapbLBf1uB1rOqmu1BgtKuZgzY5rhn3WKO6wDhALNE9cdNEP9dg3neQHKjzBFjdQSX4AvBcoXKC0o0iVBEPLf/Qu/wqDX4m/8l3+bz56fU1FjUeRVTVqWhJEPzUqvqlyKkRAuuLYsnZrUYsmLHNEQkZUN3c/fgEmlcDHlWIFogCNKehipsbV2eQtSNrMPQxQnjAZd+sMR2kISh4i65vd+/3uEUYzvBxRFQRTFBFGMtZYwChENwXmzWSOlcfJja6hLQ1UVzW0LN2w1omE3Bk5XURX4cYSPR124KPgql2SblLKukELRTjpU2lAUhlYcunWpMfyzPOAvX1+LIlBXNZPrGS/813zwl75N5MdMr+c8Pn7ENz58H0vN65vPWS0XvJq8YrtxeOy4FfLomw+BQyaTCy4Wt2R/sAGTc3t9Tm5zdg+fsLvbZ//oiND32a4zQr9Fvztkf3eXberz8SffJ82XPHny0F3VdcnN7R2v3rzm4vKaX/nVHRCSz148ZzVf4fkR3f4I6SvObl8zPhjRGye0x7uslhtmyzWd7i5V5rPeeNzeTrHacjjeR/keRV1R1RWn52eURc1gPHLMxNqgAsuxOuL29pLJ9Jpf+sU/Tafb49d//dcpioKT4xMuLy9YW4GQij/1p36ZTz7/Caeff0xlS/xpgOdLbu9uSFdLjnf2ib2QMi/ACHwRoBpNfOC7a60Q9ynGHmoQUmeSuNXi8uqMu9tbjvZPuLm5QdgO0bMdIgBrKesK4dFkErwNNWxuCa5wWKOd9VjIBkwKRjiEuQU8FWBlTaUzlHXjOmM0SPBDSTeKWa+2xJ7iz/3KL9JpJ/ytv/13+YMffsJqW9BJ2rQ7ivU6ddiwuiEJK4cSs6XFqzxUrfDwkGWOH3goP3aUawvWNMnHSjUhy6bZ6TeA0maZ4Hnu1qSNIfA9jg93ef/DD9nZ22OzzhgP+pwcHlCUFXd3d2RFjm8drmw2mzTbHkut3czHGIsfuJmKaWjHZVnhFIZgtPv+urZ4niKOQ/zAmdN0VdGOW3gSoiiCJjWp1UpQymOzXaO1RYkI3/NohdFXnr8/ClTkBIcbdzs7+E+stf+xEGII/HXgEfAa+PettfOGQPwfA/8WkAJ/1Vr7vX/R91DSY9zfY3d0wORqynKxYW9nj8ALaMddXr36gnSVEfo+YavE6xQUtSYZBOy9O2Cz3nKXeVRasMg1kQjZpgGT7YKdE8V8vcZULm7ruz/78/gyoshKnj9/wd7egPFozKtXXxBHbTr9HutsQ64LNlvNJtX8jb/13/DNjz4kiPq8/9EzyrxgtrwhK1MKSipCLu/u2Nnb4+L6nOU057sf/gLHxwdMrmaU9YTtes5gOGC1WbJYLeh02lR1jVCK3f19bqYLoqRFtxtyc33B1cUF40GfTjvmN/7RPyLPC5TyydsJ7VZC1SmZ3Nzx/MUrPv38OcbCwfERwjNsNkuCMKAsMqq6YNRp4wuBKAztVo8kCtksVqxXC0adLl4QgAqwwueTH72g0+3z0S8/pTfs4CnpAlVwPXHd+OtdxHmF73nOR1DfP33v7TyWuqodzFM0jYC1IF07YLTT82MkQisCFaECAXVOZWqEcJpAhaKXBGzSnCzNefLokP/+v/VvEoYR//R3foSxLlZdh5rcVhhTYmwTg95wT4qiQEhBFEUI6VKMQ+Hcq0I1DALZqB+VcnoDY5sU5Xsrsmt1kIJWEjIc9nj69CFHh7skSZt2FNNpt+j3OvyVv/I/5u/9V3+Pj3/yMdYafN8jn27xPOVs0M3tKAwjJFCW2mkSjHYZFUK6GDvPoISHr0I85eOSt0OqIqOuCogi0nTrQKnNzxeGEaWuKYvSBcDWJbauUF8dQPRHugnUwP/aWvs9IUQH+H0hxN8H/irwD6y1/5EQ4j8E/kPgfwP893BYsWfALwD/p+bXf0ERUHgy5me//XOcvrzkB7//I/7yX37E3niHKi/INhm2tkSRz8FJQvewjcZjkxtW+pag1+EX/9yvoWqP1c2csy/eEMeGceJRact6tmJ33OPl6zc8PnmCNIbr82uW8yndTsTuzh5Xl5fYOmJ6m1JaDWGE1TFl4fE7v/Oa0eCIdqtLXngIJIORpK9iTq9f8/L1x6zzlDcXr7m7WSPqNnkp6HbHDAYdyoOCz9cz8rKg1YoYDQccHR4AksVyTbffYzzeYXf/gCKb4XkGJT329g750Y9/SFkWjMdjrq9vuLm54fj4iDAMubi44PTigqrWxK22e9Komvn8DoQlCAPiMGCzXjG7W4JsuY3DKiWQgmGvT1WXbLOcpN2i1R5wWk0IvQ5CKrq9DuO9XaqsJo4j4jh2B6aJAtfGILUj/DpDjzv89wlRuq7wlOMROp+B25VbaUC4zAGMxZMeftBCWYkxtdMXBC5NuC5z6kqTxCG11igsH33wPv3BDrv7x/zm7/w+nz9/g/Jdjy2lQL+1BoPnK4zVbNONYwlgUL7ECz2UNMgm/szyZQaCkPJLRgJgpUSjMRiiOOb4+IjHT07Y2RnjKUlVZEjp0UlaVHnBw0cnfPe73wYLn33+OWEUsjMeU9YuTwAhyfKcbbp2ce+eoxLVxq37jNbUlUVrSRwpR97yQtJsjShrtK4IfEWZp0hryNPUZQ4ajReFFHXl4t2EpcxS8iJzWZb/qkXAWnuFowhjrV0LIT4BjoB/G/gzzaf9X4F/2BSBfxv4v1nHpf4tIURfCHHQfJ0/vAh4AWUFz59/wbN3ntIfJMwXU3Rl2d/dZ2/nAZO7O9J0xmJzxqTcELb6eGGfJEjwg4jRcJ/l7ZZXp5/z6Sdv3NVbSc7Orlkubhn0vsXB/jHGwPnpOQqPR4+fcns7pbrckLS6TG+2/ODHP2B8OObBs8dUpaAqJMeHQ5Rs89nnZ8ymGx4/Pma/k1DqzLn6lMezd55xO1nQii391iG2BqziYO8IXS45PY1pJTF7ewPiyEd6FYdHA+KWTxi6a2ae5sShR29/yN1kzmjY5ezsDf1hTG03fPCNJ3Q7fccZrOHg6JBAxczXO7x484L5fEnU8jG1AOMOsQXyIqfXa6ONT1WlSDR1XRDFHTbTDcvliizXHMc9Hj44IQhazO7mvHr5nCj08aXPcrWg086w1Ji6oCwrhxtHgLbOeNMQOIwxVFXtTELqXorbzAesQYhmV+AW/O4qLl1KkVYS4UeokGZSDp5vMQhaiUR6Adui5GBvwL/37/xl9vd2+Ov/r/+C09MpKEUYhmjrQkql8gjCGG0MRZ4jVem2GiLF9wNCKdB+gFUeRhtqNFbYBk7uVoVvvQoNkizptHn48ISnTx5hTU3gS4RQ1JVuhqk9ZlnBt3/mG3TbCev1guurG7AGT0rSPEd5bpKvvEZhauxbjLtEgrTN6rWmyF0RNGaLCi0qcLMOIQXbbeqArxhXDIrcCafcRYO6rt3/F2WBMX+MIvDTryaE5DvAbwN7P3Wwr3HtArgC8dMYk/PmY19ZBIz2GIwe8bs//seo7i1Cw+15zdOTX2JcdVmvVtRaEiUB6/MCWfqEoWQyuebkqMv7v/oeKk9YTqa8fHnFutKE4wEqU5y+ucMP2rRbI8bDHovZjN5gTLEtKA0EcZvrs2seHD3i4c47nL+54sP33qW32+OL588Zdnfo9x4w7B/yj/7R79BqGxZb2Mke0+2P2R/XyFDR6rS5u/iYR/sHJGGf+eSWSES09zzG4yHvvfcBsGV3NyYMCiaTTwj8kIP9LrouSSKPdL2lHWiqdEGoasryjlanojOWrLYrHn3wmKqsyDY1eSFphR1MOqXtSQJguUgptgGdeI91NSdJdqh13qTvaITZIpSPkD6lsaRljhYSL4qorWabz0HAdH5FtfFIszUPHx+hVc3pxQsQHodHxyRxxHaVMhgMCEQLrSuM57wVQigagjdSBk7mampsAwGRFvzKd4GpWe6wXoGk9CqKQKOD0HkYPHcwQwnKWnSV44mC2K9RoaAoKrwy55e+84y2+kv8zb/56/zo9Q3K90AosqxG+AoRtrBVhfQt2gjKvMSzgirMnU7Bd/24i2MFq1w6gdcIyYw11NZQSYiCgO6gx+HhPqHX0Ipp8hZ83yUDeyEHOx3m04oPnz0k/ZVf5O/93b/PcjohiBNMVWGBOA4pCqf+01WN1m6LokIfT7gEp9pUzQylxgpDWRgCG7gZjBD4SctF7wlAaDdXaJyQeZGjjSaKInzPWcD/2EVACNHG8QP/V9ba1U+nnFprrXBL4j/y66dzB3rdDrsHAz55ccVk1uX64oaWf4QfSWTgYsUm0ylmPiGO+xQUXF7MWC5ykmjFZrkhXVfM51NOTvY4ebCH7wvydUm30+OdZyccHR6z3SwAgVLKEV67CaPRAKFqDnePqdeKg+NDHj5+TH+nyzcWSz57fs5wdEC322Vn3GU6m/DRNx7R7iTUVYUfeLz/0QdMZjNOjo/RlWR2t2Q1S0nCDod7h+zs7FGVGa9e/5BaG6SuaSURURija2i3W/zsz3+byV2KsjO+/71P+Ma3vsNmk7Gzu0dabri6u2Kz3XB5PmW90OwMHyF0QT65ozMY0k4SLm7uUEFEGEUEQQhWUBQlVCX4ormKKspKk+UblNcC4WGtcwTe3N4RRRFlXbJau7+fszfnVLqgKCqmyR2r5RxFRFlq4sjH84XT8vuO8FsVlVv7SdGYZtzGwLUEtQsXEYa6KJ2N2Jf3vFJoWgfleS5DEAPaPfk8DMpzcWVk7uspKZFIvv2tb7BZbVj+V7/FZLUhK5se2GqHiZfuNiJw+/88z5FriZAeURRT+xqEdkwB6QxtUgqEsEgp8ZWHFNDpxAyGgwZ37yjFbgOinY7ZaIR0eot20iLwQr7x0UfcXN+y3GypaoP0hszWa8qyxBrjNARNAXC3AQiCgDgKGjuy+0cpxbr5c1Hkvr+QgiAMqaqKvCioqspZvj3XGmmtXcEwpole/8Nff6QiIITwmwLwf7fW/ufNh2/ur/lCiAPgHmd6AZz81B8/bj72z7x+Onfg0aN968cFj945IopDLq4v2evH5NWCVldy9HCH29kO3//Rp+ye9MirEiliDg726HQGzBcLp1+h5uhoTFnlvHr9HE/t0O11EQI22zWXF+csZnMCP2ZnZ0y/16MVh+RJi1evX1JvEparJT/8wff57p/+NuPxiD/44ad892dPuL274913nyBUxsnRIb1em+l0Sqcd43uS1WpOkadsNyW+F3ByfMjeaEwrDtmmBdb67O4e43kKzwMhaqqqpChL4sQwGnfIy5rFXcV494Ref5f56TmekCzmGXUluL2eYbSg2x4QBy1Wiw11nuPnhSMNtdu0Oj3CKCT05NuhmM5TTKAItI/WhYO3oPCDkiBUGOvEPfPlGoMrCEEQUpY1p2entNstknabwA9YzGeYShIELbI0AqvRWALT7OhV6IAhUqKkoK6ME+MISV3bpmNwmgAn5vFQkmaSLZrQUHcNN0IiPfe1hAJrKsBtFrDFW7xXFPgcHezys9/5Bp+9POWLl69JQp8gabHarGglbWQTe4YxVLp2ik0vImoVIH18YxvgiEV4DVGoMT1Z4Q5cr9tl0B808BGLBKoGTCKEoxl7nsCTina7Q0rGzs6YDz/4gIvLaz757AuUUiStFqWpETg9htWq8Qq4YuA17QI4cIu1rhglSQI4/obWmrquUcrJtcuyITs1n+v7PkkrcQGpRY62X/2M/qNsBwSOS/SJtfb/+FP/6W8B/1PgP2p+/Zs/9fH/pRDir+EGgst/0TzAfRPNfH3G4fEu22xNEPqkxZrTy88Y9Efsjvboj/ps0prW1rAtarSWjMdDut0BWZqRZylR7Ha7Ol+TxJbdwT6LWcFsdk3cgixLieOY4WCHYpszmU5YryEvV/zk449JFx57BwMm8wk//vjH1AiMrdndG3J3e8U7Tx5R6xW6qijylPliQquT8PrVC5aLGZv1AklIv99l1N3DVIbPv/iMwBuglKKV9DE2I8sr6rqgLFKCoEVtCkqTgaqYzpeMxkfkpeD16Q130wlJL+LBo0e0opjdUY/lpGQ5WyEJ8H2f2WxGaeFgf5/x3h7r7YaqyCiyFcZo4lZMt92mrmE5WZCmJVHUYrkqaCU+Uexsr54fY1AYXbPNU+I4JA4TDvcPnTVWKmaTO0xl2d05wNQF203BJt3iRSFSKAaDMZ4XIoUHeAhb4ykFzUVRNIh2bSr3JMVS1XVDH26Iv03AqBAS5QdOeFSDrh0eyFF5NFK67cSm3GCt5uHxPr1+nyjwef76DbWunOZAuBWbkzU7taMxlizL2Gy2WCGJjG2QaQaMh7QeXqAQOIS573n0uj36/T5KKScYMs4jUNfGORCF0xUIcPkTMgNrGI4GvPvsHU7PL7iZTGn3emzzjEproibmXTT5jj917t4e9PtiEEURvu8ckGmauq1HcyN3OgA3B6iqilYc00najSpRUfPHawd+GfifAD8SQny/+dj/tjn8/w8hxP8ceIMLJgX427j14HPcivB/9i/7BtqUtDqCVtzhzZuJ28Ebn4o13/vR/4ef/fafZrldELU6bLY1eW0Igog46tDt9MFq5vMprVzR6QQIm/Po4R6Pjt7lk49Pubp+SRxFDPsP6Hf7VKXl848/c2EbjfSt2+uQRAknjw/ZVgvysiDpdHnv3afEkYenDEkckaYSWxvW6zVFUWCF4frmyg3AjGEw7BEHIe12Qr7R/OjlJ4yGj9jd20cbQRDFbLeQFRVJq0UUJcwWU+bLnPkio6g181XKm4spn79wROXecMDhwUOMrlHWZza55u5qzfHBQ9J0w910hgpjBmOf+XxGXuR0uwmtqM+43yIOPTzpMZunKL+i1W7R7w8wBqJWh929PYoyd4M5U1GUGfWqxvgBrSih3erSbidYI1jMJvhSUfe7pKlEa8PNzQ2tTot+f0i6UsRxgu9HVLVL+L0PAnXUIUtZ5tS6QihBXWnKMkNXFUIKF28uGkOP8LCiaQ2Mj216Y6UgjGJMbahFRTtp0e0kTNcFTx4c0ElismzLF6/PaScdJxQSsgkydWgyi6AsKzabjZM8oxqeAAhr8ATQuPkQgigM6Pd7dLs9Zz0XNRjHOjDWcC+MdlBSgdY1Sri2qJ3EPHr4gHeePma+Wr5VEIbGoKRy4anw9oCXZUlVVW9vAW574IqEUooguI9Mz9xN7x651rQNVVVRlKXzEjQbDk/98bYD/+TtSflvv/78H/L5Fvhf/Mu+7j/zEhY/VFzfzLi9XYMVnBztELfh8vwNi+0jrKr58BvfZFtuKHRG0krY3zuk1++zXs6Zz6YsRUXwYJdeL2R3NMDzBVWdEwQeVV3SCxOGwyHptuTk5AFFkWL0ltqk7B/u4skhJ48OyfWA86tTWknMwdExxuQkLY9uO6Lb3iEKFdsyZTQckWYpd7dTR79VPv2OJi1TLvMrQq/Nzs4OrSSh3W2hJISRJQhH1HcbOr0uVW05vzhF147Vr/zQBYUWa6K4w9N3nlLVKbqGKIxIVxnTuzsuz+7otXoIZanrGi1K8jzDCBiNhhwc7PL/be/NYiRLs/u+3/fd/d64sUdGZlZulVW9L9McNmeG5GhkSiTHImGNaMCGXmwRMOAXG7Af/EBDL3q1AfvBgGHAhgXIhiS+WIJpW7IpS0MS0JAzPUtPT3VX175kZuUSGXvE3e/9/PBF5vSMps2haSqrMfUHAhkZkag+0Te+c8/yP//j2hJZZVRFShylOLGk3TFASIIgJElTOr11+hvrDEcDhAVJGiFTyNMacRxrufGkZJbPsR2bPEvJ8xrTyYDxeIRlOYyGZxiihVGvE81HlFmC5WhtA9fzodItwgsiUFFmq5Xmuldf5FpzUVOPBVLkGNJEGialFFTSQEkTDEt/qVWJa1moomJZFNTrIZ1Om+FkQVHENEOPN1+9SRTFTJcJjm3qZaarxSVCFwcwMEjTFLmM9PCQXO0vEorS1Lk06Faj57qEYYjj6LVzciVTJo0UqcrLdWYXj6IssUyTsBboMems4I03XuNsOOR4cI5A4bsuaZZpfpUQlxLtZVmuFIv0zIAmKf1oamDbNrZts1wuL9OIizTAcRyKvCBeRlqHcZVefRqeC8Zglubcu/+Y48Mxy3lCp92kv7GJEAk7+2sIO6G/2abmbyAtWCR6Xt4QBqZpUKvVaIQNsnSup8CEhcRgNp3guhb7+3vMZufUQ/8yzFrfWOd8cEISZzimx3RmgRS4gYustNxUXuTYjoGqMrZ3NjBQqConS1Ncp0aS5CwXKYOTEVme0+v1Wc5jovmM6fgp1zauc/PGqyAtwoaNaYJtpzQbfdJ0SlUJFvMFs1lCnufcfOkNJCbD8zHttS7LLGPvxnWG58c6zSkKouV8tU23IvAd2mtNvFqNwXCsWWWuzfXre+zuXCNaTDg/e8Z0NiOJMwzLx3IU48mEeRTT7Xa1Ym2ekOQxWZlSklFSUJQFjuPiWA5lrphNpnieje1IBAXjyRlRXLCxsY0hFfPJkKFlIqXeDSmkTX99E8c2KcuL++SKRcjF6nCoikrvNjSNlTpJRZEVoFIMW1K5AsuwLnfN6f0GeujHsh0sM0GVJUHg02rWODw6ZjqP6XdavPX6K3z3gw+RhiDNSqqqWHEC9H/KsiwqpUiSGGkYl6xG0/hhKF6YBpZh4boOvudhGKtWqJBIy1jNSOg7sOZLV0jAMgxKwLH1xqaqqti/vsvpYMBoOmM6HOmuTJ5rmjCs2qg6v7+4618c4Mt2X55fRlUXkYOmMa80FFZt0ixNKbPiMj34f8Nz4QTyvODB3SN8r88br73DzvY69bokTo/Z2rlGtEjIM4v+RhfHtxnPDMajCdPRjKIw8T2PnZ1dlrMxqkwpMkmeCgxL0WyFSOGzjEYItLLxyfExnu2xWCxwHEmlFJ7ngOkym0+ZLier72NFHM/xHBPXkQxOTqmKkjhKyIXBZDplNJ6jCgNLmqyvbRH4dSZnz7AMh1ajQ6+zRpQvkVaKH5i4joPtCJqtDqcnpyzmOaYRUJaS7WsvMRqfMZzc00tNpV6UsbbeYzafUmYxVIpWO8RQFvW6j+04vP76G4ymUwzLwnYd6mGNsixI04TJZMLp6SlSWggTTs+HPHigtQqb7TppHhOlU4bTAZYtKaqEghQ/8Njsb1HkFYup1k9UpRbxyNIFaZaTZJAXa+R5ysHhE4bnAxrNFnGcIU2LWs2n1WpSZKyKa4YWxFgtNBUr/UJpGHrEeKW/V5aVzmGlrrYLw9QqQsLUkmZKAiWm5WDaDsv5DMM0CDwbqpz5dEQQtlnrNGk36kyWCWWWocoSaZmrBSaag6CUDrM1q1DqQqNjaQdQFFSViRQWjmXheS6mZelBo9VGZmmarASKtFRZlmLYmqp7QaDyXYeiXqOXtrl58wYPnxxwOjgnjRM9Q7G6MV08gB+5e18IugKXsxBS6vkHz/NWcxvVj2wZUgp83yeOY2az2f8/LcK/SJimRZm5XLv+Mr/2V/8t1noht+78AdPplHbX4+nhMeenJXs7r7HMcpbJEChZLCYYQufoaZJjSAfbdrGkxWJWYroR0rBJ05hGI8S0DMbjMfPFgnkxxzQFQeAwGE5wPQfbrzGfz/G8gCDQW2Jms4m++8cJ5ycDWvU2w8GYcVSCgG57E9/VK7Ze2X8Ny3SoUotWs8/+9Veo11uwTFmm52S5dlhVpQtIs0nEfJaxsX4dx24S+F2++/638Gs2rucihhVn58es9Zrce3CH6zubhGGNqSNpND3ieEpSWfQ3N7nRvQFSENRqDEcD7t55CqthmsVySS2sU5UZ8+WUKFniOBbz5YSnhwmNVkhexNiugyEVrmnSbHVZ6/aZDGfkcY7vOlRVRKkWRNGEOM1BekxmU+7evcPg4Am99XV6cQTCwHY8omhGlkVYykVIE2lqGq4OURWGAKVMKhTGSp67UqVWKpIGQmoFo4vtwghN6xVVgVACJTRzMSv1QXJdR2sh+K6OFBB0Wg0m84iq1Jz8siioVnfNNEsxTAvLshBCkecpSWJQBPoalVVJWejowbZNbFuz+1SVc8EotmwbKYSeOagUVVlgKmClxMRqk5HrmIQ1n2sbfV595SUePTng9HyE7Tq6awGX0Ydt25d3+4tagJTyMlK4uPNfRAumaZKvNkV9sqBouJ7+t1DkxadHA8+HEzAsblx/g1duvEmz3sNx9PTUYrHk4OCQpwdH5EnAwyd3GY0H1EIDy/CYzydQSDw7IE0K6rWQjbU14mjB6ekxhYhp9zokyQLT1P8T9RjuBFEJtrc2sGxJmsb4NQ9pSCzbYnd3D8s2MS04GxxyeDggcF2EVPh+QFGMSWNNM/3iu69T5AlJEtNrr2EaNuWOge81dUtHgFIJy2jAcpFD2abX6TIZz4ijnPk0YWdrk153hzDokucZL79+Hc+rkyQJSZpSVjotWet3GZ8NGQxPEYWleW3K5e7du+zs7uAFAb21DpOJYLmYk2cJliXJ8owkTfBqLs1WjetiE9f3qFTG4ydH7IgtyiojGU/xfL0zMclj5tGcOEkBA9OQ+HUTYVicDJ6SV4ogdCmritF4jBKCosiZzqaEYR2lCmbTMefnp7TbPVwv0FPFSmmm4YrWKwwTUek7a1UqFBLDMjEtG2FaerV4WVKh/9YwJQITlSekeUlalChhIEwLvxbQ769hmi5ZIUlzWOsuOT2fMJkvdFiuFMIwsE0bVRYoJaiUgVQ6rM+yhOViTs21AC3yIajhOg6mYawyAUlZFchKYRgWEkGZF0C1aoUW6PNfUpQ5F1qovufQ7bR4+aWXuPXRXU5OB1yk6noIa1XfWRX5Lh5SyssI4aIucJEG/LBDUfwrnYIKhWGZOKuZifHoOZ4ilMLk5Zuvs97f4PHjRwQ1GI3GCGEyHs/wXJf13gZnZ8ecnBywt7vOsoiwDIM4ivHtOv21DRzLwTR9lEpJkgpllozH59i2JMsKsiTj7PSc8WiOiclbb74GIqJWC3Bdm7wsdJ5YKtrtLkLmnA2eMp2P2br2Ggsj1kKTTkA9NDAti1a9S6UysjRCrvTw67WALM+YToa4QUBRRuT5kmgxx1AFgetRFQrfDXFtKFLFdLKg17vO3t4OQuYYpqLTaxEtY5qtBusb65RlQZwmLJZzAruBF3gkBRweHWLaFjs7W+RZiuvYtJoNBucJy+VyFcJCpUo8zyYI1mi2WkxnU+RYMZ1NQBS4nl72Op1FBHaGZbrM5xnRLCH0ba7v74Gx4O7D72O6HiUVeVWytr4JyVJrEwiFH/ggYDA40avNTXMVvlogTQxp68IaimqVJggpUaWWArMcF8tyqIQJhtRTfau0QUgDQYWSluY0mDa2C2VpIClZW1/HCxpEcUlWCJZZxelwyvFgSF5Vuv9uGpiGiVlJ0jwjTRWGaWIoEyiZTDNaDQ/bCaiKnDiJNCtwlbsbhqlTlrJcpSc6FaDS25zzLNNOoCxWRVDtOGzLpFEP2dne4u233uT+w8cs4gglSqSUOI4D6JD/k7m+4zg/5E6suAAX7184DqUUtm1fjk+XVUmhdI3ADXxs1+G5HiU2TZN+r8PJyVOqqmAwiCjyhF6nz3Q+wDYltqHnojfXN3BMh5PBkM31PQzlUg+bGMJiOpkxOBuCKpGGhRdazKMJQVBnOh1hG47OoTo2VVbiOA5nZ8+o10M9K16a5AVEy4Qg9BlNpviBj+c52I5JlmcsZiP8oEXY6bCzvUMQhKTJDN9pMBoOiKIIDK2wW1aSqpKgCvI0hUpRZAVlXuE5PoFXsr/bw7JdJqMxZyfHmKbg7p2PaLc3yHOdo3a7PdJkycnpMc1mk53dPUKvRbPe5enxGbbjEAQ+nU6HNI1J0xjTlNTrIUfzqd4KXK+D1KyzRqOJ6zhkrsvNGzeZLiakWclar89ocs5oOKa23cSwJKPJGKO0sKwaeZ7jOoLuWhtpuSR5yXQ+xw/rVIbEdvSK7iCskSQJ48kYISWNegMpBLWgienoGUN919K7CTFACANh6iKhYTkI29VM+tWqcEMaK9ahzuMRJoblYjk+2DZKFSSqIBACYbiYdo4SNou4ZHe25OHBEcVijuO7ZKoizzICv0acxihRaXnxrEBgk6c5k+mYfq+FaUkW8ymL+ezSGSndC9R3+qqEQo9My5UakVCmvnOvxEsFSuf7qkIIk1oQ8M47n+POvUd845t/TKEqTNPEcRyCILhsP+e5bkEWRYHnebiue/m7Uuqy4HfRUbhwABfMwVJVCCUxLQu5cjA/8fz9aznlfwqEANtSDM4e8tbbr3H47HRF0QxYzBfkecHZ+ZC93Ze4tr7OwcNHWNKjUWtj4JLnFeeTc2bTKdPpGNMQ9HodHNeiwMEPPJyZRT2s0270WUwTKHW7ajqbsrunC2+G4VMLAhzboyoAJdjd3aWs9AGcLzNcs43t+kjTot9fI4mmZGmC4RrE0ZTjk2cso4hGu0ur08NwFGmakCU5gRsQeCGWtInLjDzN2d5eoygs8rTi8cM7jJMTikJLZQVBhzjJ8dyAvb190iim2Wrjv9LEMWpIYTGNchzHYWdnh3pYI00TBJAkMZZpYFyMztoOrh+wXMRYppasCoKQjc0NzgYnDIZnqEoQLRI67TWanSbKgPlyTq+xgecHPDl4ytZujV6/S1bC7DRGUrCYxlTRHN/3cF2LJE+xLJN8teFoeD6gLEqMdRNPSkRuYxpyNXSkWYECiRQrp2BaKGO1NRh9iAxj5QSKaiVboDUDlTS1fLjlQ5WBlFhVillIDMNd0b179Nd6JGWhRUfLiqLIqFYjy+bqDpokKZYpsEzJZDLi9Mzn2voaSlVMp9NV3i1X4bbAMCzyNKbKc0yhi4BVWWIath6HlgLTtFGqIskz0iSmEjamYbG7vcMv/9Iv8uDxQ54eHV6G87WaVgUuy/Kya3BRpLw46Eqpy8Nu2za+718Km+o1caYWLzG0QpK0dOTzaXgunECeZ8znJwS1EsNcMJsdkZwnSLkLlYMlbJbzKVVZYQobSpuX91+i1ewxnSw5OjxgMpkCFaUq8F0P2zNJ0ojlck6rGbJ/Y58sLlgulvhBSCtsMR6f4rkutVrAYjGlLAo2+5ucnk+YT+fUWyGNhku9EfLxs0M8q0lYb0Klo4L5fMpkdIIhckbnE9JkDiphPj8nLSIW8ZhCbVGWJY2gw1q3i2c7VAWoSoeRi9mEsjJxXZs79x5gBkuu7+4hpM/+9VdZLDOKEgK/zvbOPlVW4EgD2wgoMsXa2hpVVdBsNqkqPbte5BnLxfySViqEZLGMsJ2AMLxYrW3TarfxHJ/N9S18X2skokxeffl1Gms+g5M5hao4Hw7pd1sEQcBkMmaRj8grSZIq5ssB5ycTqniJbZt4rkVV5WxurFMLfJRSHB48ZT5b4NgubWFQSQtlWQjLxBACaep0RUj03gLD0gQhaSBWk4SGKRHKQEmhiT9lTlFBUYFhm1iuh6hCLWxaSIgKrTXgOFimRbfTZTifMVrMQAq9OFYobMdCmgZlqfcVmJZBsxESL+acnDyj12kQBiFJEmtVIql3EgohdTGxKCjSdDX0JFGFulRVklKstg6VGKUmDiEEruUgrYDPf/7zHBw/4//4P/8pBwcHP0L/dRwH13VXHAN9uOM4/pECoWVZeJ53mQpYqx2ShqFT1WrVzrzYFPVpeC6cgJQwmT4jyyd8fPec89Ez0rSgHjbY23uZOMo5OR6TRCl5UnGtv0fg1LGkhyTjfDBiNp8Rhh69fhu/5oBRMhyec3r6jGYjpB7WGI2GHD454603fw4pJfP5jDAMGZwPNF/brNHr9nj46Ig0S0k8xfGzEaBoNBqUmcloPGGt08SxPc6HZxiU+L7D2ckIIXI8z2Rrew0Mg8l8weD8GbYV0O9t0G2vYxsGk+EIVSlMw+Dk+AiERbuzRpEvIM8whIFh2IRhk0bd5nRwxv37jwl8F9+tMZrNCNqafBSqkka9RlWVxFlOFC04PjlmNBoznU7wAx/btqlKPTjjOVqpx3UEYdBgNl0gDYN+b5PlMsIwLDw3pN1tMJukWLbJ8HBIUPsctcDh/uOPsYKK8+kC0+nz8NFjWuEalUgYT2ZUjQDTAITCtEzSNOHxk8f0OhHtZhs/aGB4JYY0KKSuAchV5Uyxqg+YJsIw9RwBK50CqSvwF605pRQIA9OysRwbYZiYjotdKZxcwWRJUVS4rkcQajJRazJktJhiGAb1MMQSEjvPidMM0PLcYRhQDwIcS5LGS/I8xfd7pEnCbDqjqirMlVRZWaTkeUGeF9iGQJgO0jDIMj0XYBjykrFnGAau66Kkg7A8Kkx63S6/+qu/ymg6YT6fs1gsVnJkLr7vY1kWcRxfRgAXd3l9ZuRlDeAiXbAs3emAi/0KK6UnNCnr0/CcOAFBEqWYwsJ3FI2aQ9Bfp9ddY72/weB0QLvRQBYScq1Me+uD97m+/zJgYBhg2YJmJ2Rrd53jswOyJEaJkqAWkBc57337PRxTC0KkacLZ2QlxtKQWBNx/cJ/eWo/d3esoVWJbimarxeHxYwbDQ3avb7LVv87TxyccHz7jWm+PKs15+uwZu3vrOG6NIKyhqhSBg+O5mK5HOJ8zXUTE84gxA1q1ENd2GE9mTEZT0qwkLRRFGeNmC7r9OousYDSZ0Ww4zKZzpAqpMod4UZCnc5y1gFkU0+mX+HUbM28AGYVIGY6GnDw7Q6yEMoeTIWfDc7q9DXZ2tgg8H8s0yfICz/Op1XyiGBDgOjbXNjcpyi6VKhkNxlquKvBw9mpIx+bbP3ifvIp4a/8mZ9N7+DWPMDT5hXff4un9Ax4/fohpOTQaAY4fUKiCKi+ZRROCWkCURqRZgpcXqNU8gZJQoleAVUIfemFauu0mJdLUX2qFsVo5hk4hVqQY0wRztb+vwkCaDq4v8PyYJM6p1QKumxbLJOZsMuLw9IQccD2PZq1GFC0R8xm2Y9PrdXFsC0NAza8zm5SUVYphKJbLmPH5GVm0xApreodhqYeOlBKUSujURBqQp5oZWehCpjQMDFliGRUlBVkyJc0ERSXZ3Vzj13/lK2TLGQ8fPaIoK+IkRSkoKkWa5eRFheu5eK6HlII816pJUhokSYpArzZTmquEaZoURU5aaNKRZVmXWo8/Cc+FEyiKkuFRyka7h+MkOGXJeqeNH9T5+PZHnB4/pOGGeNgU8wXDySGnJ4eEocX65jadNR8vFZh+SW4sKM0lUTZmd+cmVb7GZDQkTpdsbW6x0VtDFQbnJ2fYpsA2od2soSo9f55Vc0xzSafhc/hwwcn9E7Yb11EYeGUDqxyynJ6y5m8xzxXHT4+oyMgqCLwaVZYRzQrsXFHz+pyPn7GYnGEXCaOBTVYoTk4nmGYdywoJOzWieIpwJbGa0NvcIkoXxOWS2WzJ/Fxw8PgZnbU6i8UZEy9lY3+PYXGICgRZ1EKpKa5TEk9GZGaGLH06/XVymXF0dIoTdAjr27RbLlUZgwTXs1EqodGwaXc7nJyc4LiCutfk8OiQ8ckIzw0J6wE3X3mD23fucfvolGvba6SyS72TkKZzru/5FPkxZyfHpHFBq+3T6m6wzBfkyZxOK6SxHoBdMI0nnJwdI5WJ024hXV3RF5lWKaqEAW5Na3yjkDJASU8XAis9jSiUrsJDhWXpQ1ZUBYiKQhkIQ+LXPHobBtF8SRjUWUOAhKOzU+4+eES2jKgKQa3eQFFg2zW2Nrs4tqDIEz2yrLRoRykcbKeiMCuy5Yx8OUMGHoAeRbZ9iqyiAAppo4TAdJfMJgvyDHyvhuc5qKpElRFJNGc0nDAazjDNgHZ/g9e3e2z9e/8Og/GEj27f5ff/+R9yMhgiLBffr2O5Fa7v41sWRZaB0jUXPciUkecZcZzieb4e1670zsay0MQsQ14sV/nJeC6cQFVW7O7ssd5ZJ88PkNYGVAZJlHNwcMhyMeLazQ0CUaesSpI0Ya3fI6j5ZHlOLahRb9WZLkdMxhPa7SbC0KGxLWygYvvaNRbzGQ9m91lrb7KMIna3+zSbDvPlgFLBaDQhzxKm0ylJtMBzXHa395iMtLhop93l8OgJvu/y+uuv4tc8Hp484u7dj7FdSbtZxygVNb9Gs9lmkRckSYpje5p8hMB2bEzToNGoI4RHt7/Gk4M5B08Pkaak2eowO5zS6bb53ve+Q7f2Nrdvf0R4XNDo2RiuT63Z5N79e2wWHsdPn9BsCtbWAtqdNmQF0VTRaTfY2Gpz86UIoZoI5eiJNQWGKajXa8RxhELpTUN5RpQmNE2T6XRKvIxwHV9TkK9vc3I+QBiS9Y0NHMfj9PSMdsul2Qo5OHhCnheMJ3Nefu1lrm1tce/BDzBskzRP6XQ7TM7nHJ+eEC9KZGES+i4SkyqPkVLnuIbtYRs20XKJUQk827tk9EkEK/UMtHyRFuaUUmJiUFYKaZiYq/TC93wsaUKlcByP3d0d3nrrLZ4cHfPhnXuAYDIZo8qc/lqX63u7LBcjFouSdrvJfLGk0agT1HyUquivddne2cR1ba0zsJoYVLat9QUMoWcdSj3e7NdCFjO9ezDLCqoyI88T4iQmihacnw+IoxPG8wX7L7/KjZs32K5gsUwRoFu7jpasdxwXISRJkupBJbFaWpprYtFisVjVB3QRMcsykixBmQrfCyiLEst6zp2AlIKbL7/EztYN4lmN0dzh2WBKWUiajR5rvYA4TkiSEXmeUSiohQ1qYQPT8pnOI8ajCaPpENsTZLlLkibEWYrKFc1mHVUWHDw9oN/dZjQacfzshMC3mC9KojghCOscPzvjSRoReJIP3r/N9uYua2t94mVJUKvT6TXZ3d1ld3cbQamXZ5QZ9XpIqRKEUDiurtYapoEsKzqdLo3NTcwyR4lS75a3BFm+xDAFg8ExqioZjkas9et6MaWQ9Hpd/q9H32AWetiOgeOD61k8evSQyWyJaXl88P7HDI4VYVjw1pv7BFtN1tc3qO12ePjgLgeP7tGoN+n3r2GKGrWazWIRo1RJHEe6ILtIAEjSFEOaTKdzDGnx0ssvo5RgbeMaZ6cnmKbk9Tde4Rd+4edxHcEPfvBNXM8jL2PqYcibv/ZzHB6d0Gy1cRybTqdLRcTZ2RF7O7s0r/do1HpkUcV0PuPg8JC97Q1MqUd6lVJYSHIikiLGrYPd6CKU3iy80uBmVelCKU38MQyBNCxUrnRrWOrtSLZtI5RiMpogsxLb9vjc229zPBhyeHKmNzqVCWGg9f981yNaCDzXpd/v47qzlTyZi22b7O5usbe7jetaCFmtmHk6ArEdC8PQY+x6VNnEC13iqMJxPdI0pkJQKEWptLMyTJOiylkuI8ajEUEtpNbQRKIvfemLnI+nDKdzlLBQWYHp2IiypCpyLYqyIhBdsAtBa0fEcaznC4RaqQppNuF8Pv/U8/dcOAHXc7l9+yM+/v5dOm3FPHnGIlGUQmDYku5Gn4MHD9nr7/L48RMG0yFKuNQWMbXQwzBsjg6POTh6hONLvECiZMlmp49n2biWQVIW9Htdtq9dQwoXz/Gp1T2Onj0gzxer1k2bg9MnvLS/w1tvvkO0iDl4esTO9j6WbTEcji4VW46On3J6dsRwdMbOjW2CepNoNmMyGRMvE+RognR9rXlXCJazhDRfUMmSMKzRW1snywUffnSPRrPB5uYGYUMX7YQwuf3RbVzXwnUlouUjzQlpHq00Ek/4zd/6S3zj20NaL3WYzZ+SxBknJ2eEjonXbzI4P+Po8AC5LXCdIfWalqNOU4llBRRFznKZ0Wq1KMqSRqOFYVrcunUL07Z56803+O733ieKFjx8qNWMoVrtBKz43Dufw5A53//etygTA8OEv/SVX+L4ZMB4NsZ2HM6H51i2Q1XpzTkoST2sUyYlURyRFzm1uk9VZaRpRpzEpMuURQZtw6V9wSdYpbMCdNJbVXpQZ8WoQ2gCD5VadZAAFPEyYjqdUVbg+CGbO9f5+Z9/l1sf3+G973yHwLeo+Q4nxyeEgUWz2WQ0PGU8GlEp6HW0+KvMFb5nE8czjCnUVEPrYhYlCnk5fAQS07CpciiLijQvkJaNKAsMUWJUNpblEDZabCiLsFEgTRvTtJjN5mSFwjJN3nnnHc7HM/7ln3yLrFQIw2SxXFIPfMKgjpSalZgkCWVZ4vu6QxCtBEcNw8BxbQzToMhyTCGZR8mnnr/nwgnkec7jg6d4qkZVWmRlwpvv/CLfv3VIXmYso5TReIRvnJJXJa4fUlSCo2enCDmm3e7SaXV5/Pghw7Mh3bWQs7MxZix449UbzMdjXMdl69omDx7cZ2P9Os1mh9H4nMdPj/ACSVYZ2IaPY4c4dkCrEZAnp6yv17m+d4OTszNG8wGng2NG4zO6bpu8KghDjyhZ4NVMTk6PmQxGCGXg1RrgeCyzgnKhIC/YuNZi41qbNI9YRGOqyiKOp0hT0V3r0Om0GY1j1ta7/Mtv/DMc12I6H/Lo4RO++ptf5L3vfoNXXv0S3e4e6/1dWvUNLLNJPYQiHyOUII1SDp4+xTINGo0aYd2n3a6zt3udmie4e+d7RHFCvdEgSROy84zB+YhaLcRxHYq8Yv/6DsvlgulszHg2YbmY8dqbb5LlGePxgKOjR2xdWyNKYurNFuOzObP5mMVixmgyJIpjlMyxLYf9/R0mowmT8ZTjwymO4XNzd59Ws6s1DJSDYzskaUqcJMS5oEDrFVIopCtR8kKyXNcDyjLT5BxDt+mqFYNOKEWWpAjToCrz1eBMTqUEURRhmAY3b+zzy7/8ZZ48fcrZ6RGSCqoU21J84d23MeUajx8/Ik5TamGDJMsJbZdkMmZwfszG5hZb27sEQYhleWBaCHSofUFtLjOYTmckWablxYWhmZK2gyNKLeTiNgijAoXEcl2E1GPNSV5Sr9f54he/SJIVfPeDW4wnc0zLYrlcUBUZtVqwGjPmso2oBUYUYRhgmiaWbRGnMVmqW4Y1P/jU8/dcOIGyKqmFAQ2zQ57NyAuDRriOZMj33v8Btz4c4JkKMp8bL71OWuU8eXSEZXiUZcTR0Rmua9Fudag3Ntm81ub+g49ZnA/4zjcH/NKXf4lOt814PKPf6+L7HmG9zny5oL9+jf5mC5AcP8l5/bXPUWZzvvvdD9jd2qXb7vMnf/JNFtGCV157CcMqsWzB6OCUKE9orXdwQ1+LmgQezWCX6XiOND2Ozofcf3rIq3ufoyxgMdcTXYt4xCKaUSmTRsMnK0tu377N/s2bHB6M6HQ1XfXatU3u3R6xvdfny1/5Eg+efoQQ8NqrbzEeTqh5bUwzoLVVJ0vO8cwaNi0m50tUVXFj/zr7N2/QbGzjuxaL+YDz4Sknp2e8+ebbdHtdDo+OtW5+WOf05FS31IIap2cntFpNnp1o4tbGepc4i/kH//Dv47om/X6T8/NzOr0+k/MlSbLkm9/6BkpY7L/0Eq12iDRKhsNTvvHH38ISLq1wjWQxJE9yVJUh6SJUjm1ClhcUZYVhuHi1JmFYX6n0SM0fqC72FGhJblWVl3r7oFWITNOilDlqJbiRZxmO7VAhqITFdDKl1u7yV37lVzg9O+V/+71/DCq/5N/fvv0xtcCBFVXX932iZYTMC8bzqRaBcSxqoY8QCt9XlJmJYTl4nrWaQ1AgDWbzBa7XQAmJYVqoosCxfGzXxJI2QmaUKtNphGmS5gWGKWgGIR3bZef6TfZvvkznf/8n/Is/+CMWywjTMrFtkzAMLxmGSZJcjhyD5kXo2QWBJQ0qIcnSlLDW+NTz91w4Adu2aHc6tN0Nau46jx9/zO/+g3+E5a+TJhVlGrOsIgbHC3obO1SGwfl4wt52F9ep8Ud/+IfE0ZLPvf0K77z1OqZVUvNshkcHbKx12d/f4/bHd0hziOIxTw8HeN4hpVJkZcrtu/fxnBq98GUCv0FU5Wysb9HpdFkuYh7cf4QwBOubPSazMf31NqXKefbsKZNkwrXrW6RZAqXi6PAp7VaPbrdDa2OD/vYOTtXFlVrPbjR9SEFCvowJak1uvPQyy7jg1ke3UYbJcqaoqlQTlzbXuXv7mLDR4c6DD/nCl97l/ASeHZ1QiZx3f/6XOXp2ijQmlJnFgwcPMas6oaejCiXnnJ+fMJvkUNlYRkq306DVbtJfX6PeaHN0fApKMp9p6fFWs8VsOqXVdRAGpHmMYTncv3+Ho+ND3n3353Aci2fPjrj98S0MQ+IZPp0gZzQ6p93bYLFYMJmOSLIF49EZICmKiryoGE+mJFGEIMcQBYu5hWkIfN+nEiaW5+C5Hp5fQ9jOZf6vywGrhaGqolIlotL9d0PqcXBWKj+L2Zz5dIqQEt8NUEKSVYInT5+wbdr0N67x1a9+lWeHT7n94fephzWm4ynPDh/z0s09ajWfslJMJhOmsyXtmkfNFjTqPaSsOD97RhxFeH6dJK+ohW22dms4lkuaZiRZzmwRsd7XXSU9LaWdQVmWKCGRhonj6nqGEJoqbbkelu0yi2JmiwVra2v89m//Nr/xG7/JnXv3+frX/wW3P/qQKIoIw5Bms8lsNrtMAS7oxEmS4Nj2qisgsaTx5xca/YtGnusuwNP5kLff2Off/q1/l+98/yH/5Pf/GCFMpLSZL87Z6PTob1wjqxSvv27TrHfJk4JWs0OVl2yub6FK+MFHt6hUwrVWSM3z+OC732MynxInFW7QIV5m3Lt3gO3a9LfWKKsKy/V5773v8dGHt/jaX/81siTm++/fYntrly984QuUqiRKZ4zHYxotn+VyhqIkiubcufsxzWaT6WgMOTSbLfI8Z5kkzGYzQqvJG2+9xv2H75FmOeubbT68fYvJdEZ/fQfPD7l+cx/XrfHg41tsrr/CbF4ym03JihRpwXvvfZO/+ut/BUqLOM4wTIVj1PC9BXE8oR422FgHWQRQmngeLOIp9+7dwTI6tBprBG7FbDYhrLe4f/8eYaPN9s4Os9mSj29/jOM4tJpNTfE1DWQBrVaTg6NDkizlF7/0CxRVQavTQt6q6PTafP0Pvs7GVpvRaECSxfT7fRqtNoXKaTR3ePLYJs9KRoM5+/s3qXvnDE6OkYahF6cuKyxL0Gg0Nee/NHHCRHcBkVx2t8WP1AVXwzN6QAbDgiLXQjB5znQ6YT6aUK+FWoHH9ZjGGc8Oj/DDJn6tzs61Lf7yV77C0dNHHB48pVl36ffW8Fb8fNO0saXF/v7LiDwmW4zp97tAyeHTx0jDJmx0iNOK3vo2/Wt7Wu0oy7XzmM5wXJc4yhDodegUijTNkKIEtVI6LirmiwWGaePXm/j1Jo4fYtkzJvOIWr3B22+/xTvvvMOv/Bt/mVu3fsDBwQEfffQRJycnlzMEFynBRb0g8H3KIkciaNQbmo79KRCfFDe8KgghBsASOL9qW/4c6PLZth8++5/hs24//MV+hl2lVO/HX3wunACAEOLbSql3r9qO/6/4rNsPn/3P8Fm3H67mM3x6ovACL/ACPxN44QRe4AV+xvE8OYH//qoN+HPis24/fPY/w2fdfriCz/Dc1ARe4AVe4GrwPEUCL/ACL3AFuHInIIT4N4UQd4QQ94UQv3PV9vy0EEI8FkL8QAjxvhDi26vX2kKIfyaEuLf62bpqOz8JIcTfFUKcCSFufeK1n2iz0PhvVtflAyHE56/O8ktbf5L9f0cIcbS6Du8LIX7jE+/95yv77wghvno1Vv8QQohtIcTXhRAfCSE+FEL8J6vXr/YaXCw2uIoHYAAPgH3ABr4PvH6VNv0ZbH8MdH/stf8S+J3V898B/ourtvPH7PsK8Hng1p9mM3qf5D9Fc3S+BHzzObX/7wD/2U/429dX3ycHuL76nhlXbP8G8PnV8xC4u7LzSq/BVUcCXwDuK6UeKqUy4HeBr12xTX8efA34e6vnfw/4G1dnyr8KpdQfAaMfe/nTbP4a8D8pjT8BmqsV9FeGT7H/0/A14HeVUqlS6hF6Qe4X/sKM+ymglDpWSn139XwO3AauccXX4KqdwDXg4BO/H65e+yxAAb8vhPiOEOI/XL3WVz9cw34C9K/GtD8TPs3mz9K1+Y9X4fLf/UQK9lzbL4TYA34O+CZXfA2u2gl8lvFlpdTngb8G/EdCiK988k2l47nPVOvls2gz8N8BN4B3gGPgv7pSa34KCCFqwP8C/KdKqdkn37uKa3DVTuAI2P7E71ur1557KKWOVj/PgH+MDjVPL8K11c+zq7Pwp8an2fyZuDZKqVOlVKmUqoD/gR+G/M+l/UIIC+0A/r5S6h+tXr7Sa3DVTuA94CUhxHUhhA38TeD3rtimPxVCiEAIEV48B34duIW2/W+t/uxvAf/r1Vj4Z8Kn2fx7wL+/qlB/CZh+ImR9bvBjOfJvoa8DaPv/phDCEUJcB14CvvWv275PQui1S/8jcFsp9V9/4q2rvQZXWS39RAX0Lrp6+7ev2p6f0uZ9dOX5+8CHF3YDHeCfA/eA/xtoX7WtP2b3P0SHzDk6v/wPPs1mdEX6v11dlx8A7z6n9v/PK/s+WB2ajU/8/d9e2X8H+GvPgf1fRof6HwDvrx6/cdXX4AVj8AVe4GccV50OvMALvMAV44UTeIEX+BnHCyfwAi/wM44XTuAFXuBnHC+cwAu8wM84XjiBF3iBn3G8cAIv8AI/43jhBF7gBX7G8f8Ag1cGsYI8cBsAAAAASUVORK5CYII=\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD8CAYAAAB3lxGOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAEAAElEQVR4nOz9WYxtWZrfh/3WsMczxhx3zLmGrDG7qrrJbplkAyRNGjYEw4ApGbAeDFh+0YMBP5jQkwG9+MED/GSbhgDDpiHLgA3TosmmRLYotbrZVd1dVV2VWZmV053vjenEGfe4Jj+sfaOachVFil1gAp0LCOSNyIgz7LPXt77hP4gQAp+vz9fn68/ukv+qX8Dn6/P1+fpXuz4PAp+vz9ef8fV5EPh8fb7+jK/Pg8Dn6/P1Z3x9HgQ+X5+vP+Pr8yDw+fp8/Rlfv7QgIIT4a0KInwohPhZC/M1f1vN8vj5fn69/uSV+GTgBIYQCPgT+CvAU+APg3wwh/ORP/ck+X5+vz9e/1PplZQK/CnwcQvg0hNAD/3fgX/8lPdfn6/P1+fqXWPqX9Lh3gCd/4vunwK/9whehZdAKRmWJQGCsxXmPdQ6pFEIIgg+IAAiBDx7nLFIKhAQlJd57QACCEEAKifcBrRTee0IIeO9IkoQ0TbHW0vcdUim8d2RphkQSBHSmx4f4s+A94uZxA23XgxBkWYIUAu88goB3HqUUWmt6a3DOgRAIpSCAcx6tFEJACJ4sTTHGYKxFaY1Sir7vQQSyLKVrW7RW6ESSppKubzk8PEDLlIuLa0wHzkGaAMqQFwLnJW0NSTLC+x7T10gJwidIkRNEwPke7+N1lUqQpglCCJqmxXlPlmWMRmPq3Y6u7ZBS4kMABEKI4RMTKCWRQmKtRWuFsQYpBEWRIxAIqTC9wRgDAtJUgQg4Z/E+oJSOn5EL8b13PW74fJx38fMUkKYpQgicdTjrIATKskBKycHBQXxu03F1dYX3EucC1nmss4QQyPOc/YN9lstr2rZhOh2T5xnT6ZjO1iyuV9R1T1nkCCHpe0vwYE18vuADPgSElGRZRp6nOG+x1mKtwYeA9wEhBePxlKZpQARCCAQf4jVyDq00UsQ7SSrJ4eEBm/WaqqoBCYKfXV8hEAK8j1m6khIhBNYN1wWQUkCAsizoe0PXdvF3dUKRF3RtS6olSivatmd/f48nz86uQghH/3/777/2Nv+XXEKIfxv4twGSRPL2lw956+5rSCRt33N2vWCx3SLzjFxn7E9muLbH+kBrGozt2FVrxpOE/f05zjnW6x2CBCkypEjoq44izciyjLquqOsdt26f8s473+C9n7xL27ZcXl6ytzfj3u27CCPopWexvaYyNV98403q9Q7XGvrOYoLk6fklJIq7p/uYpqGrGgqdkkrFq6+8wmgy5ve++z0u1tdMDvdIixxjFM22ZX9vSpYqunrH7dunXFyec7VYsHd4iFSa88tLmn7Dt7/zOrazNE3D3XuHCLljPJX8jb/x3+PO6Rv81n/0Pf72//m36OuEu7cyiv1r5rccXZ+jxBd58NGGansG/gw6Q2JuM8lfxRc9Vb/gen1FMcoYTQoOjw+w3nF+eUmaZnzhS1/i3t37/Od//z/h4uKSICRKpSR5wWa75Vfe+TYEuHjxIm5K50lzjRMd1nbcv3uHu6d3cUby4JOn7KqK+f6E6X7C+eIBL86eE1zCbLoHTjIppuRJydnZOVVdcXh8iA2GXbPD+Rggx+WE2XhKrlNs0/Hnv/Md7pwec+v2EUfHB/znv/vbnF+84N33zrlcVBjnWW13bOuKk1un3L5zTFXP+erXvsB/86/+JfbmI14sHrD1T7m82mD6hO3G8v57D7m+rLg4W7M8X9JsHUVaoPOSq/WWv/7X/zL37p/w6OEHPH76gOvVgqazzA/22DYdt+/e5/LqmnysSXSCaQzbTU2iUn7zL/wmP/7hD1lfX/HX/tpfIUkS/v2/9bd55dUJzms8HkfAhYAQgiRJ0EqRKI0Wkq5paawlLUqc62mbHWVZ8tf+6l/h+3/0Qz756FMSXXByfIff+I2/wMcfvIdZn/Ptb3+Lf/Ld32cym/Hk2dmjn7cXf1lB4Blw7098f3f42c0KIfwt4G8B7B9Mw2/8+r+GtoHHD5/QNA3b7Tae1FKgEMzmE84ePccLwd7ePuUo5ZNPP8B7S9d17O3tEYJkvaoQMtycKt578jzHmJ48z9Fa8+DBQ6RQOOfI85x79+4RbMD0PctmS+d6qrphs92SCInSGlu1eBSj0QiZJYggcMaT6ZRUJ3hjaaqGqm7IsowvfenL1LajMQYhwDpDCAEBWBs3uDUOrRK0VGyrit4YkkQQfEOWFbx4vo0X7+4xtgt8+JMz/sP/63/M9//oQ3aVQjMiyyd0bUpf5ayrlm98/RbOaN770VNm49usqzVajtGpYtc3VHVF27ToRNA2grMXZ7zy2qscH8HZ+SWLqwVlMSZJUpx3SCVwePp6S5InHJ0ccnVxhfEOEQI+eDbVhje+fJe+azi/fEaWJOzNjijKFKUlOlFsthuauiXRKfloGk/b3hAKqOs6ZlJS0fcGh6XvepIsIUkTjDU0TcNkf8TJwQldb5ju7fOTDz6k+1HF4dE+u13FrvqEtq0RWqOSQF5q0hxsaPlv/Xf+Kl/8wn32D0co6chzwdnVFu8dk8keDz55wHvv/pTNyrO6ahHeM8pTbt++jUoKrjctT58+4/mzB7Tdhu22ouss870p9++9wicPH7HbVkzGU2yoCM6hlKJtW2ZHc9566y1+/MOf8M1f+TZf/fo3+Fv/x/8DxVjSGwcIrHdY73DBEwDvPbKIGY9znrpp6J0nzXP8kLHdvn0LKRXG9CitkFJgTE+aJYzGYx4/+4jnl+e0pufq0cNfuFl/WUHgD4C3hBCvETf/vwH8D37RL3vn+fCnHzHLR4QQUyhnLZPpBCsCiZYURUbAkqYFeZEjREBJjXGGtu0pipI8H7NePYgpqlIkiSZYj7V2SHtjWg5gjMUYy+HhIY8ePeHV+68wGiVcbK5BBpq65npxzaQYUagUayyti+mfEiBcQAVQQuCtw/Q95+fn9NYw29/j6PSUj588BALeOvquwzuHdzGFbtsW7xxJmpAkKdZuCM7jFewfltQ7gfPw/MmaPNnj9Tdu84//0Q8pi5JJOaerKhIFSa7Q8phf/dZf5Gpzwfd//IcsrhakhUTrGUUxQvkMmViCjyVQkiTMZjN8iAF0tV5xcXlJmuY0TcuTJ0/pnYmllwgURUZelqRpxvf+8LuMRiNu3TlFenj08CHT+YiDkxkXFzVXq0vSVJPnOS706ESz227Z1JfIJDCdzpAU1LuGPC9xxmM6R1EUjCcThJa0piVNc7I8ZVNvSVRCcAHbG+pNhe06kkTzwx/8Id/5zq/w1a99k+9+73s0piFIi9AK3xnKUcLewZjpLMeFik11yf6RpCg0R9mUnzyq+OTjp9S7B+y2kpPju7S7CwQtUii0SvDBgbd8652vUxQZjx9/RFEqRmVGbyreeuMtsmKEM57a1oymmjxPqesG72P637Ytn3zyCavNgmL0Fr//vT/g2fMz0rygbw1KxtNfaYUUGiEEUkoIEEIsaYsixzYdddMgROC1V+/z9a9/nfPzC4yxZFkGXmFsR13t6ExL5w2Pnz7CuJ5yXPzCzfpLaQyGECzw7wD/AHgf+H+EEN77Z/2NsZ4sK9hstjjnef31N9BK432s/5u2BgKj8QhrLI8ePaFpO6RQKKWw1iGlQusEKeUQQV2spaxhNBpRliVd13FwcMhsNqOu6hhIjGE8HlOOSqSUZGlGmmX4AF3bEUKs1bquo+9b+m4oyK2jq1v6piXVKVmaMh5PcN7z6cMHrDcbnHPUTY3zDiFjXyHWuBYfAlmaoqSEIHDOMx4Hvv2dbyJEYLep6VpBvRUk4ZCL5y3vfO03+Df+xr/Fwf4B682CIFt2254//IMPefz4jIurR8hsjc4bNtsKqUbUXUVtFhSFpiwKDg72uX3rNuVohJCSq4tLkiRjvrfParnmwYMHnJ2fo7QiEDg+PeJLX/4SAU9vO6azKdYbrlcL6rbi9v0TrKhp+g0yDbRmR9WsafuKXbWl72PAI4AUiqbpyLKcLMtp25aqqui6HmMsTdNirUPrDO9jf8g4g1ASISUeaHvDw4ePsNbjUXzve9/ngw8/pWladKoYjTOk8hjXYlzL1775RS6vn/GDH/0Tvv/Hv8fj5x/y/OwBVbXl8OAAYxyPHj3n0cMz1uuWJMnxHoRUSClp6pr7d+/yza9/nXe+8Q7Hh8e43nL75Dbf/Oo32J8fcDA/wPaWro59FCkkIsSeRlVV/MH3/wjnPT9+7z3+4W//Z+TjCZ31+BB7RDEIaNI0JU3iPWydxdgYjNM0BTx93+Fc3PTz+Zzz87Ob3kdRFLH0bXZst2tkIlGZJi0zVPaLz/tfWk8ghPD3gL/3z/XLQrC/d8jdO/dYXi3ZP9zj9qv3+fD/+3fIJiMmZYm1PVKCVoreBapdRV7kJKkgeFhcXSOEvmniGWMRNjAqCrI8Yzyd0DQNvTV0xnDvlVe4vLpitVlzfHLKaDTG1fGEScqcY3nMdDymWseUXGmNAALgrMV1BokgVZrgPQzNszzPebG4pLId6aggCInpOoosoyxynOlItCKEgJKSIi+GJpPFGcf+fESik9gos4GutohQYPsU02f8p7/9+3znO+8QvKLrLElhmKg5L86uyKZb3v76MSe3cn7ywwXvP14wTlKc3zBOEpJsRqI1idYoJXHWUe0qDo8O+cY77/Dpw0fsdhVN3VIkCqSImcd0wnq74sXFc77yla+yP9/no59+yNXFFc5YVtsl67Dl6NYebbdld71F54L9wzlnT68QAsq8RKeO+XwPO0nY3ztgebViu6xo25Y8y0nSlDTN0GmC0IK2a5FSUdUNZTainIyYjqe0bYOzCePpnB//+D2urhd4L+n6nnIyZjKdstyu2awXPH3+iB++WzCdKerqEuuvcWHNwcGMpq55/OiaD396xYtnPfVOYo3GG4+1kGY5ewd7TC3YvkdLwav37pMlgsvzFxzsHaJQJDLhYO+As7MFwXm2mw1SKISQSCEw3nJ5eUmRj7hcXGOMI5MpMsmQ0qF8QGgFOjYAAyCGBqBAEIKPt9fQkA0hcH5+zuPHT+i6Dq01AolEoZOEq6tLttUWlac4EfBSoLX6hdvvX1lj8E8ugcD0lqpqAMFicc3TyzO0Tjg+OmJU5OgQKEc5velo+ljnz2ZjpILNZsVqtUMgSZIcQsA5RyJjRzrNMtq2xRhDmqa0TYtzjldefZWHDx+SpjlKaVTmKcoClaWkZUGRZexWWwgxPdOJJhcC6z3eOnBQ5CWm62nbls5aUmupmxaVpxRZwa5p8N4zn48ZlSNWyxqIaV6WZUwmU65XK0xv8D5wfV3xD/7+f4YzKQf7B5y3FbY35FmJ6QKPHpxxfHjG5fmGk8PbJBnstlu2u5bZacrBUcJsP3BwkqLLhiIPHOwfM59JLs9a+q6n7RuW1zmm7ynynL29faxxtHXLZDzFO8F0UlDXNWVZstqsePz0KcW4YFutmUzG7B/sUVcVfWd49OQxxVHP/funnNzeZ1LkTGYjMjHh+nJFU7VkecJkOubOnXsIWUIQrK63CCEZjUYUeUFelOwfHtI7Q9VUZDns7e9R72rSNMEYw/VqSTCOUTmi7zqsNfS9YzrZY121eC/oh1LP+zjNaduGNM85u7jE+Q3zecqrr93hYP+AP/zep6yWHbYXVDtHqjOSJCdLM4QQ7KodRTbi8aOHrJcLxuMC61q0TKh3Nd//ox+CUiyvrmOgSFJ0KhAagvNYY8AHrLWxtECg0oS67XDOMSsLpI0lDEoMkxhQSYJSCjHcyyHEe94GR9+1LJdLPvroo2HSohAIpNAoKVgsLrHOkBaaTV1hraVMy1+4/z4TQaBtO+qmpeu62ARpK86uLxgfzlFa0tQVWIuQkt1ux67uSdM4yhqNC9q2xbkGrdKhGehQQZIohTEG5yxXV0u898xmM9brDSEErHW88spr7O3t0fcWaS3G9HTeIVON6XrqqmFeTAghkKYJeZ6xrWqEF3R9T5okOO9x3oOF0HWAYLfdEaSgGXoBo7Ikz9MYPAg3QWBUllxeLbDGQhCcPTMsLp/y9pfv0HU9+3sT0kzQ9juyNOPenXu0dWCz6ijzCU+fXpHqfdI8cPvOASE8YblcgQgcHKbcPTrki1+4R3ANFy9+grWWznRsNjvSLOHNN96kHE/46KOPqXaxsUkA6zxpnmGt4+zsBbt6w1/4zf8GH330ETpV7B/s8eDTBzGjCJ6ub3j67DG3T0752he/yuLFjnbbkGhN7QNSKPKsJFEpnQm8eP6czWZDkiSc3rnHvbt3OTg84u4rr/Di4pwXZy8oxyWzozGXlxdcPD9js1ihhWK73CBEHFnudjXzeZwOJWlJ21jqbsHyeotMdUyPq5bXXr/FF976i1ycfwpI1ust1oA1kCY5e3sFfddhWolSCXvzOc5seXF2xv78gFE+x/SW1fWSNFUcHxxTtRVPHj0iLUvOzi9IEkWWKO7eOY2Nyu0O0xuCDySpGk50H0vWQiGAvqlJRTzHCeC8v2lmJ1pj+h5rDFoqyjKnahvCkHUulyvKLI+jxRAQEooyZ71ek+UJInFYPCDZNtUv3H+fiSAgBJjesFwuKYqc1WZJksT59XazwVtDJhUJKs7bdcKbb77BV7/2Nlme8A9+67domx4pFUqlpKmndR3OO6wxrDdrnPf0pqceLmKaZWR5hpCK58/PmE3GTFLBbrejmE7p+562bgkh0PcW7wKj8ZiTu3d49/2f0DYdcpjrChnrOaQkzTKO51N2bUPV1jjn0DrejCHEOXlRlAQXm4XVrsL0Buc8wQe6VpIoyeq6JUsTyjxDyIa+3zCfj8mznE8+fkxwKX0nCS4nLQre+dZXmR0FHr14wv7REUo0INbMD0aMJhPOnm0JQcZmKfG58yLl8OCI69WKtu7QOsG7aqjdNUIJmram7Vq0Uvylv/QXWCyu+M53vsXf+4/+Pkorml1NOhaUZUnb9jjrkUJS7XasFy1pmjEejxHSs1lvabtHeFKMscznc+588Ta/8s13+PKXv8xs/4BiNKbuWjabNTrRlPOMtq05e/qMj977KRfPz1hdLxEePvjJ+0wnE4IHJRQEBQgWl1e0jSVBIULCelWxWu6QwnF5sUFLiTUPObu4Zn++By7Q1Dmr6wVt3ZEXOa+8+grnZ49YXm+xxnDr1VvcOj6mbSvG44zRuGBbb3ny7AkfP3iAEB4hBbduHfOFN97i2fPnmM4QnENISZpolIDGxGYpztM0LZkSGGvIEh3vD2tpux49lG3OudgkROC9J3mZIQxjRB9CzBYGHMU3vvE1fvjDH+JxEUugBKCGcvXnr89EEAAI+FjfSHkD5KHVeNOiAF0UJDLWWHmeIaXC+0C1a+haAyjEUIcpGdN3Z3qcczRtS1mWFGWJUgpjLJPpDKUUT548Zb1a8613vo51BmsNIXiUTuLr8rG5Y41FCslsNiVNE+ymRQuJCwEPBCnRaUJWFMwP90mqHd1FjwgtWkqSRNHUFcYaZnqM8R5jYi+AALZ3JEmKFJpUKe7eeo3t9hyE4eCwYDpL6NsM23e8eHEGQWJa2J++ScBy9/Y9anPJxVPPfHRCSst2fcn19QoZCp4/WeCcpMhzptMxNkQAy3q95vnz57RtS5aVEGJDq8zH7OotXdujdYJONKvlitu3TpmMxzR1TfAeZz1pkjMpx4yKnN2mjmPeuqfMx8i0QMuMpq3YVTvazoIqODo85Ctvv82vfOObvPWFL3Kwt49OUlCKPaU5dbcI3mNEg/dTDmd73Dk6ZbNY0lQVTx89YbNa8eUvv81qteXDjz5kMppifeDSXyPQCK9QImN9veWD9x5gjEXKFuED5y8WeBSpHrNZX7JeVfR9h04kUga22xVKCXSi6fqOPMvYPzigqVKKMmU8KSjGBUmmObt8wWoXMN5wfLJHolKuL5dsVhskguA9fdcgBaQKvO0JHrI0YZSmbNfLCDwS4EMgSeK2tC6WD0miEQGcADGk/+ElAE1rrOljc5nAdrthPClZrK6RWuNDIACoXzwD+EwEAaUUSgmM7cnLEZPxiMX2GmEMxngSKaEo4SWa0ASePn3GdrvFOcN6s8M7j8EhMISh4/qyEx8nDA6lNCA4OjpCIOi7nq7rqZuG7XaHLmPDrq4qyumUNM3oiOOquqppmprl8hoICKXwHmwI2BBwDOhEYLer6Lue4ALBBhKtGZUF1XaDVoo0SWibBq3iZCIEKPIcGyBTY6bTKXdv3+fd959RVSsOD6ccH8/pasvZ8zUhGCaTAiWhryaotOGPf/A+UlsunwlCu6PpGuqt48XzBaYes1kFbAQ7kqYp0hm89zx/9pzLiyvSrEBrj5TxNK3rlr6z5FlBkim8sPze7/4u89mMn7z7HtVuh7MRi2H7QLv1KGPYmYom6ZjkexwcHGI7RZ44lJQY05KXJftHt3jl/n2+8+1v8+W3v8y4LJEhELwltl4FUkiCEgQDpreoINibzRglmr6dcry3R5nn3L1zjxcvzsmyhMfPL3l+domWGiUUIigUCfvzIxLpCaIiTXLqrcWYjiQrMK5hcbmmaTRpmjOdTkiThMX1FUdHUwJT6k1sUAqpSJIUEPHAyhRKS3ywjGcFvTME0bPd7Hj27AW73WY4xQPe9nStJ0tzskRxsH/EbDbn+uqK3WY14CMCeZEzGo/p+z5OoULAMaT7iUKIMIwQI4q2qRvqpqYscgiOn/70fb705S9StTtWTR0zVCD4z3gmkCQaKUESSFPNZDphtBnjtaI33ZDJSLwXsYE1nPCr1ZKXBChjPM6Zm3Ge9x7vbIT2hthpTdOU3a5ib38/jqLcANZRKdvtjoPJHlmWsqp26KJglI9pk/ZmbltXOx49ehSxDInG9obe+wj0cA7lPb5tuVou40gnCEQQpEnKeDyib2uKIgKWnLEUkwIzlBqT8ZTNbkeiMk6Ob9N1ESDTNC3n52dU1Q7rLBcXL0iTwGgEXddwdV4z24c/+O4POD05oV3nfHq5pek2CJGDTxmPjuh2KevtmuBD7D8I6LuezW4bN/M4GcoACUh26wqpBXmW4qwhK1M+/fgTXrl/nw/f/5C6qsnSgiRL2DYb6pWllz04QyNa1H5JeWdMYx2dqxmPpty+c8rR6TH3XnuDWycnvPraq0wmY7zp8SFmrt4FjDEonSKkRCIRHpyx2C7OwOvtjtlozBffeiOOdvOMo8N9Hj2/4o9+8CPquuLy+prOGqRQHO2fMJ1lmH5D12/Q0iF1xnpds91VpElKno9RuiBJS6RU7LaGvYM55Uhz3l+h0wwQyEQjZKCqKzI0dbNDqMCrr92lanc03YZmI2mqBmcsKkvQiYLgybKEr7z9JaSIB8jR0SGL8/MbwI/DI5ViOp2yvL6mNSZmWxhwAV2mqFQjBCil0ToGK2t7Ei0Bx3bXs7c3Z7wYcbHZkKu4xZ39jAeBWFMLyrSg61uapqEoCjrhkc4AcQzoh3mqQGCtjThz54dN6m9O/ZeBobeWNFEopTk4OuTo8Jh33/sJZ2fnjEYjlEpiHZ9n1E2DUgfMplOWux3eB8pyRFe2ZGnGeDzhYrmgXi6ZH+5juljfCxmwHiye1vTooOlNj5QSJQSZ1qRaoVUc0UghMdYCERvedz193+M8JDrhYG+PNJVcXp4jhSDLMjarhmdPLhgV+3RtRV4EvFiDMti+wVuNDJLgJF0VaBtH7yAtU06Ob/HWW2/huo/YLGIdWRQ5PniaTYM1ljRNSVRCayzWxFqyby3lKEehadqaLFOs1itOj08iGEmnFGnOKJ9gjMU1lsm0QMie7WqFHwtsF/AuNt7u3b/N21//ErfunDI/OGQ0HkUOQCzowXuC8LiXtXIBaZqhpCJRCabv8cZie4PpWpZd7NdoCQTH8eE+hye3yfOUk9NDLq6uefDkMZeLK4ILJDKjswItMsZF5G0sFi3eSg4PD8mKKZ1xbDY10iuSFKSCsizJsjzW4VKCl7hgsd5RLdd4Ybn/6h2mB2POrzydaXj+rL7B+EMg1QpnPHvTGf/6f/uvc352zm/9/f8YYQ3L60XkwIh4j3dtN/At4mEWfKA3BgL4HjI94CWG7FYIgVKx5pdKkucJi+tL6qbCBxAiZrc+2F+4/z4TQUArTZomKC3ZrDYsrhd4LbEi4D3xNPfgvCBLc7yw7HY7xuNJHAM1zZCmxc3vfXzDSisQgvFkwv7BAWKYsV5cXjI3hul0jk5S0jSnqmqM6SlHJYlOCCFG2ywrKMoRSaJwD128YUWsz9Khx+DaFoKPUFovyYsSCfiXxBEp6bo21n3e4V1EyKVJQtN0KBk/qPnBHvfunnJ+8RwhW4q8RAjYrhuW1x13v3LEeDxitdnS9hvSDLSumEzmGJdjuwbTVQQvSDQkSjAeZUxmGpX0COFIk5TZdErV1Fhrh9FnSgixOWudRwWPRCOCAi/QUmN7S111BOfRUpHIFNtbRCaZFHM22y0ne/eQ9NitZW96RNdYymLG7Tt3+MrXvsT91+4wmpaoLEUpicATvEXgbpq4bWfYVi06zZCqRCBBOWwIeGcRwSPxERXX9lTbDVIm5HlBUk6YTQt+9TvfpLeBjz79lPd/+lOulwskglE+oiimSOHY2ooi36MoBSrLyIqSbV3z9NkTrPWURcF6fU2R5zCQdwIR0CVlIMszlptzklJw+94pdbeh6bfs6prFlSMEgdYJiZKkicJ4w+nxPvfv3qKrdnHacb2krRt0lscpC4F2GP8x3M1DRT+Q1SIy9SXRqaHl5OSEzWZF3zeoINBJzieffMRqt8OT4l0cR3v3Gc8ElJakaUyRvXf0fU8ICkMgEBBSxYYVEYeeCoHWMR0iCPrODAysgA+OEGIgzbOMEDxJkrLb7Viv1rRtOzQeY4YhEIiB6dV1XXwcEfH9xhikkHRth1Lx8ZMkpelarIfJfI4A6q7FhhA7wUimo4KDvT36tuPs7AwI7HZbnHOkaUpZ5KjRiDTN6I0jCM1oPGPv4AAhHLvdJVmmODreQ20TpJ9S7zymi4Gt6bbM9hWHxylm2zKeBtbbjs16RVFqxuM5nakRiWdXLXj67H1Wmyf40KPUCOc8bdsSfEQsIiLTUrysxUMgTVKc9dRVQ5IrvLNMxmPapkEEkEHQdD2NrCnyKVYLsJosTyjzKfPpAePRjDde/wJvvPkmr7x+j2KcojOJ0BKCJ3hPcBbbd1gT8RXrbU3TWSbzfYQUCC9QQqJkzKa0FDhrqHdbqqrCGMt0usdms6Lzz+iMZe/gmCQree3V+8zmU54+e0rfN4xHKVI6Ls+fkyYjpJ7T9i3bZocQgTRTBOEwtsU6Sd3scM7hQ6Dre7oBlzAaZ6RZEkFU1RX5VLGtVtTtlrPzDcaW5FlOliucbSB4ppMRb7x2j7NnTzh//hRnehrbc7h/SNU7jHekKh5a1tqbrDbuD4VEYIPDGIPWKQLQWnP79h36vqOpdygV+QPb3QZrPagc50LskXnxC/ffZyIIxM47JGkCQiCVRChNZw2JToabQOKtx/aRaDGbx+6+dyGezD6OUoQQSCVumidSSNq24cWLlrpu6PouEoKcQ4iA1prgXcQZytgZDz7Qty2m73Hecn5xzmhUkCQRh9D2FtCRhuwcfW+o6wYRAqrUSK354pe+TFPXvDg/xzvYrnakqWY8mTAeFWid0HUdPnisNRweHZGkGR998hOC6hE6YzqboVTByeGM1bJmu9silOPo1oSvfP2E01slH7/rcFRM9wsaU3HrcJ833/oCH378Iav1il215MWZYFcvEMojtWS727DbbkmSBKk03VB7ZmkCWHpjiKxdB85RjEY4AQcHM/q+IxDQSWyKGW9IvCXPC87OLzjYG1OWEybjCa/cu8vbX/4Ch8cn5EWOShOklghhYxBwsZwxTY1znmqzYXG1xAYdiTQhILwYGnIZNknj9XKG3rR0XUO9q6h3O4SQOK2xztP1PaCZ7R8wH4+YfvELOG/IMkVvWqaTCW1v+OTRCx4+fkBVV/QYgggURYJSkGiFlGD6DiE1LjjqrkYIDzKj7Rtme1POl49RlRsg4AVts0JqwWQ6phwlrBYd3luydESepTx7+pS6rrh9eoQUgte+8CX+4I9/Qtd7RqOS8XTKdlfFA4l4b2sVpwPxwAAVuBm9ZnmK8wYfYnaptUBrgfUChEIGInJRJb9w/30mgkDfdWRSM5nMeP7iHIsgSxK0jyevMxYlAnXf0FlHkAqVpFgX66Ku7xFCoEWs46QQIALexL4BzsUbnbgJnAsY29N1LbdPTrFtw7rdEVRKniq00mx3Fd4YwLNrtvS2i131oCiUZnwwo6mrCBn2Fte3MXUdyoKQKrQs6QSUIaGvHPuTfebTMQiHx3F5fcnZ5TnWgQuWy7NrNu2SkzszJIradtRdQzmaoLIGr9dkU8e3vvEr7B1p2q4lKXq21YZ8XrCfTLn/xjH7pwX9w5plvaGYTNBygpY7OrnBqzhPdsFBiNcpWIeQnmJU4ENL19d4GUFXzhlQGVJ4yjKlqnfko4Ss0NT9jqTI6OyGyXifbmvY9TW3Tw65ff+Et966zdFBRlF4lAaVlQgZCH0Tv0xN6HaEtsIbS7dasjq/ROVzgg84OwQBoZBK4wVUfUvjOpwMuGAwpqVabijyAjmeEaTG1g3eBTZdR16OmO7NGI1HJEUOespoNme5XvLg2VPqdkvb1+zMLiJB8UglCd5h+o5E6pgFKoMJDc5aqsslu+qKO/cOqPotdtsynk4Z1yDDApn2HJyMEMIj5RRrOpbbHT/+yQd842tf4/4rr7G/f0Jwlr29Oe/++Ad0sudgOmfvaMZuu8K7HjE0tKUI8YCzmhAUIcSDRsjAcnNF0+1AWdICVOJQ2qGtR3uB7OMYUYh/BdyBf5HlrUX4OPronSPIl8IgCukDRVEQgsOYlt56ZJJTVTuyrAAEzvsb8YsowCDxzsRmCQJEFG0IAtI8ZbvZkSYZm82aLNUcHx6wXl5Rdz0nJ6cUWcFyscRZE5uPIQYRIRR9azk8OuLeq6/w3nvvonVMleUguiGkoO07fvrxR7gAje0pVU6RlJyenGJcg/OGbbUaPsCKvYMjetdycXnG/umc8d6U3WrHarPFGc9yfYVzLcvNC5I8cOvebZ6+eMj3f/Axpd9HJoA1JKOUdJzw+PwRV+slq23HndMMZxOcTfCAlx6VKpTWVNsqCnsET5ol9H1N3+8wtiEvZhjjMI1BJQzXMACe+XxGQID2ONmTFjn5RJGUc7TSnN69xf1X7nDr9gFJYpDSoFOF1AneG3zfY+odod8R2i1dvaFtWqrlmmp1TTrRBGMjq9IGpAqRUQeY4GmNwXhHb3ps3yHwCO9wvcELh7AOLSX9rsU2Fc12xXh/j8nhIeV8jkoKgtxSjlKyQuNWsRQxzmOtv/ks+64lyUsQlrpbo7ZQ1TvqZoNQPXde22f/aI73AdOD6QTBSvJCcHxrj+ViwSuv3eX6asGzp5d8/OkT3nrjTWbzff613/iLCODdH/8hiewoEk+RCVItsKan71tARhCUAKUFwugBEBUFc+qmout3uNCT5Yqs0OhUojqBUgJvLM4HZPADAennr89EEAhEhl7YbiMSSic4N7DtnGc2m9I19dBAc1hrb9h/LxVvQoj9g1j/RFWiTL5kngWCj8o/SZJSqZoQArvdjqqq2d/bw4fAbrvj/v0oQpIkMV2P8GJLOiC6IpgoQp2LoiBLU87cixi0ZKxdvXM8fPiQ3jqMNfjEsn8w4+TkkE8efoCnp+0qrO04OJpydLQHwnJ0PCMpJcurDZvljpPDU27dPmG3XnN1cYl1Lad3Zvzk3Z/y6OkjHj8859Ys4fBoD9tbdpsdz59ccrW4ZrfuyJICETQX55c0dY1QAuctUqUEYhmSJmmcXAi4ulpF+m+qBqUkzcHhmOPjE1bra5yNaM08K1httiQ6ZberuXU8R0tFlqS89uqr/Mo3vsbd27co8jyOfrVCJpqAx9keZzr6vsO3LbauqTZbttsd6/WOtmkJaYc1HQSPRyAGZZ8kScmSFBkEwQa89TgXSLIcpKbv49jXKU2RpngXpw2WQNO3dM4yDwGV56gguH16l9V6y3K9paq7m3sJIZEy9qh6F0vVF+fP2FYbvHe03Y5bdw44PT3lzp1jttuK3/md32dxuUP4yCfRQrHd7DjeP6JrDVpognN88skDZuM9vvn1dyjygrPzc3SiyMjojcU6x2Q2oen72BRXkRQkpSCO/FVko4o4NbBNG/UF8gStYkPb+Qj79sJgQ+QdvJxO/bz1mQgCANZZ+u02jveEwBpLANIkIQwXI4SYFtkQhsgmb2b43vuhq29puxbwFEkewRLBx/EOgBBMJlPqXRVhybsth/sHKK1ZXC/Y7WKDpSzLKPwxdNCjPFkUc1gsFjxfXHB0vI8xhizLaOs29iKIGIWm6bA+QmilAp0EdtWKpt3R9Ft6U4NwzOY5eSlR2nH77gEfPXjE4ydnSARH87uM8imXzxfUu55EC5TM+OiDT3DCkmrw1lEWYzb1huvLHc4qjPF4I9mbzkl1wWZ3jfcepaKAR6rtjTxbnmcopai6mt4E0jxhPp+wWNSMJ1OOjw6ZzfYiUce0GONp6pZqWwEC18c+TTHNOTk55ptf/Srf/OpXmJUZioiVV2mKGEA1zvYRz+Etpu+oq4pqV7Hd7thudzRtC3l3Qw6SUg2NMU2aJEyKEdusoBcbEhm1GLxzGGvp+i4iMLVGhxxc5HQYP5BwROwvFNMZSMG0nPLqvde5vt6w3bWsdpvYTcfh8Xjn6IxFSYXdrNizc2bTCVJ7skyzWi0ZjXJEUFydX7NetZRpgRSSq4srNqsNz56+YLPacHpyRL3b8PjRE8pszG//o3/MyfERQkiKyQRXt0QtONjb26PpDF1n4vRGJRFIpUAy9LqUIASB8wHrAyFI2t7hvaVqDKZ3SO1wysVGt/yMjwiliOOU3RDV3KClFoina9O2KBHYVQ06zwewhY/Y6EH7ryxLiiJjtV7inUcnEjPwsaWKvHAXAt450jShQUQu/dWCO7fuMJlMuLh6ymazQSlFnueRMzCQOYKPwSZJEparDdfbJeUoIwyPF0VLYqAxXY/pDUiBVprxqADhePb8MbtqTWt2JJlivj9Gac9kVmCtZ7m+4PzsktV1Q5lnrK93vJAXPH96FsEeXoGT5OmI2/eO2K4/4PDggMlowmq1QbqU3bUhy0tSMWacTRgXY9pRz3q9QGmJ89D13YDNUHR9x2QyplQ5b8znuNATpCPdetI0IyDoe8tkMmNx3bPdVEwne4zLKYvFNVlaIILi1ukpX/vK27z56n3GRYZWsckqk5SkLBBKRg0G3+P7Fryl7zp2u11s2HY9XRsxE4k1mL6jNy06TUhVDs4ivKFQmlle0kpNrxN8lrFarWnaFmsbQnB4Uiw2Vi8IgonlnPGegCAgyMsRtveM8wn3777K9WrNrm6wzseyz3RD03eA7qaB1myRjWM2LREyovOscYzyGV1twAayMuNwb45WioO9QxaXC5SQnJwec2YdXdPx6MFjzp5d8Y2vv82v/vlvko9GNNZTTKZ01lK3LUmaEm7kPgTGxoAWBAjUULIMmW6QOC+o66gf6azABYFMQOpY7vWh/4X77792EBBC3AP+L8AJMaP/WyGE/50Q4n8B/I+By+FX/91BW+AXBwGpyNKMaiABEeII0A+peOQUCKwNCOfQWUZvDFJExpW1loODA6QSdF2LUjFa2t5itSMZgkAIPxOAHE/GtFXDYnlN28fU/uVjFUXBZrOhLMuB9OMYj6ZkWcZ2uyXLUvbSecTbD4Kh3vt44xNLhe2uIgDHpyccHh1SjnOWq0t2uy06g/nelHJcstqsCcGyXK94/vyCtunJ0xyNpq16rlmihCYM9WGiMqbjObdP79B92fHKrS/GjnpvyVTJcrVD+pJxNmM+OWBvtkdbNVxetaSjhLIosZ1lXW+iSk2A0TjyKu69cpfl5pqLq0vK0RhrPUVRonVK3zU448ELDvePOD4+5bf/0X9KUYwZF2MO9vY52t9nOipJhcD2cfJR5gVCJxBcbHT5juB7ZPB4GwUyrXH4QSDUuXgCe2sHGHGcEti6ol4tMNsN2hgS70kIpEJgTBu7+BhkcGA9IUROhhAK23Z0DnzbE4KIUGiZ0LUdvTUcTA94/f4bbHcNV6trHH6Q+RLoTJKmisk0xTnDcn1NklqatmC32VFtO3K9RXhFniTcu3WXg1tTTG/QIuHqxRVHt46Zjmf4w8D14prVck3fLRiPco7vHrLc7PBCkqQZl9drrhZrhNSx70LAB27kx4QSKCEIQg6wGImQCu8FbRuBXlIWSN0jtUSqOOJu+vZPPwgAFvifhRC+L4SYAH8khPhPhv/3vw0h/K/+eR8opvRRIWg0GtHbQF6WjMdTVstr8ixhNplw5/Zd3v/4I3SacHp6irOBqop86TzPI4c6zfC+H2rXOK9FCPrKRKEO5ymKgsl4QqKS+KFs1qy3GyBKQY1GI4QQ7O3tsV6vWa/X7O8dcnh4yGq1QipFrjIAsjSlpR5EQhRKKgSRDWasZzaZcnR0RLtbUVUN1gVUUEBs8BjjODs/48HDa9q2pe8TEqUInoHtZphOxhBSppOcRGkcCYlO+OrbX2Bz1XN29hyBQAmN8BqcJMtzJuWEUZFDsIzKFJ3IG/ZZ11usdRR5hrGWfrfharFgW28x1mBMGNSHc6xxrFcbvBMUxRhnA1/6wtt8/3s/REpNluQUaca0LMi1RAaDdYbOWPJI5ABn4mnuehIJQYIUL9PbCKP1zkdk3MCfT5REix5b1azOX7B48ox6saDb7qi3W/quoesaZLBkaRTVcF0PzgJRycgHAdZgW4cRFlBsshIpU7xMMJ1Bpwl3Tu/Qdj3hk4+4WF6hpEYXySBIkzEaaZ4/e44LPdbWVPWGIi85nN2iqwPtNpLZDvcOuHv3lMePnrDdbOk7S7BwfnYFPlAWY0zv8G7L9fWSv/db/4CL9RWTyRj0jq636CSlabs4FZGxIRuEiNMtEVA6jsxfKlprpUGoQflZo7WGgX2YqITa1fTdLyETCCG8AF4M/94KId4nSo3/i69BXjmEwGQ8wQVJlpccHB5SVxVSCg4PD3nzzTf55PEjVJLyjW98g+AF6/Wa7XZL13V0fcvR0RGLxQWTyQScH5BeITKyBFEqWyryvODVV1+jbztOTm+RZZrZLGc2m5HnOWmaxmxkyA6MMTfBSitobMd4PKKuGmazKYmMqkZpmjKfzsiynLrtONjbY1SO2a1XeC9IkhznLM4KLi+XXC527B/OAEXbJjgrQThSpZnPpwTrKIuMzbrh5PgI8NR1xfJ6ifeGj95/hDWBcrRHte0Y5SXeOLx2bDYrDg7GKGV54417eA3L1Yq+Mdy6dYI3gc1qTW8Mxaig73u22y2rzYpqlzCfzfnkk0/x3jKdjMmyhODgg5/8lL/wG7/J3vyA7bZiVI7Z399nbz5lXOYoLLaLk5zI5QjDl0P6HiUCFo8kTnNeimk65+MIJ4TIuEsU2JpqueDy8Sc8++hTrp4+p68qlBRRaswZlJYkWYpE0ZiANwYhAR+bhzKAMx3GWbxXLILCOpge3EJ5iWl6iiLnlTv3ubi64unzZ/jgGY0mTGZTshzSLOA8ZJlm/2CKMx1eFty9ex/pco7md3j+9BnjcoRCsFou2ay3lOWIvrdsVkvGoxHee8ajCbPpjJNbR3z3h3+ISyyr7Y7OCWbzA4rRjGfPXgzgH0nAR8Ja8IhgUSqP0vXWR/n7vIiEOBuABILCW4mwKjY4u4Btf8mIQSHEq8A7wHeB3wD+HSHEvwX8ITFbWP5X/D15XnD14BGvv3XITCZ89NEnLK6vo1Zbqnn48BFPHj/BWkfdbfjxj3/MyfFp3Gx1zWazoW2jFLkQgjRJWG6WdG1HlmdRwz5JaNt+6CXA6ekpm82GW7dvcXl5xlu332J//4CnT58yHo8HAk/kMUgp2WwiK2w6GhF6SVVV4Byj0QFf/fJXWVwucNZxeus2B4cHXC+XXF1f8+Lsgq4xaJ0TpGA0KSjLESfze5ycdoynY+7eeYOmc1RbQ7ACbzy3T0/56IMPOTza40tfuk+gJ8gOv7I8GDjsQViyPCdNNLPJmNsne5y9uKSqdixCx8F+ThCG6d4B49mcNM05f3HOdDylb2G7a1Ey8iScczRNR9tavFPsdhXGdrz5xuu0TRXRnG0MFL/9D3+b1fWGruu4e/c+t2/fIc8yMi3p6w68i+o+DKexN9iuxjRbUt9FbIUxeOsQXsRywMQu9qgsGY0KlBZsLl5w/uQhV88es3j+mKcPHtDtqoi8nExI8gzvJELF6ZCSEp0kaCni67U9tvfIEMAF6t2OujV0vQORM5qM6dsGYQyjUc6rd+6zWF5zvV0DkrqqcS4Gstu35sz3cqrtNT5EnUjTOUZpyv7+AalOWa+XfPzdD9hstkzGE1575YTZZMZ7774fhW6NAyl451u/wmodRW0D4AO0reE0H+EGfExRRnHQpm3i5ElFRqLWir6LE5bZdE6e5SyuFvSdochHMQMygXpZ43OH2VmmowlL1j93//1LBwEhxBj4fwL/0xDCRgjxvwf+PWLo//eA/zXwP/o5f3fjOzAqM6qqou97tNKUZTrU5VsO9vYIPsTavW2QSYYeBBmbpmW5XPHixQu22w3WWkKwpJliV1V0pqfuWrIiJ00SrHGUZQEBri4v+aM/+iO22x3nl1dcnD0nU3cxxrJcLjHGUNdRkkzKSDLZbDbMZlN0mrG4XA0GEJLF4opMJ3gTx2oX5+fUdUXTdixXK1QAawzGGuq2Ic0L2sYzneeURc6onDCdJRRlQaILZuM5TdXw/PFTpPKc3jrizp1jLi6ekmQ5T88FvTF861vv0FQNP/rjnxD8liSZcOv0mLZuGJUps70ovyZVGOS9Jmyrhq43NFXPbLrP8fExu92WuulIMsne/j7TvTnBT/DWMptNePvtL/P+++/Rtw3lfkFwgo8+/ISDvQPu3LrLdDJDacV8PkNoi2l21LuK1kmEUtGAxTpEVyNdS9/V7DY7NqsldV0TfCxl8rxAIdnf36fIM1xbcfX0AdfPHrM8f8H6+oK22tLWDU1doxJNWmQIPLbrEWrAl0gxGNMolJSEYOOG7cE4j+sDni06u4oZhbN0TRSyvX/7Lr21fP/dP+ZytSBIgSRhcnzIycmE09MJf/C936cYFVTLjuX1Blem7M8PuHt3ynZ7zdOnjwHJdLJHlme88dYXODg85ez5Cz766Kec3DohyRJ+/N67BAmeaMKikwhk8y4wm8zwwWOcGUpdi/NRqr6pdwihojbEZEyWZKwWS6qqRVKzfzDHdAZhDKIPyB6C8v/lLfinEwSEEMkQAP5vIYT/F0AI4fxP/P//E/B3f97f/knfgaP9aWi7FqUUL168QKiUg8NDur6PmIA0jfLPrsYLG8UXfYin3SLW0lonvOSip1lCXVckacpE6+joIiUHB3OKouDZ0+fUTcP60SOElAPlFK6urri6WgwKxTY6AsHNaDBJkhu58O12x3hUEoRnt1rz6e5TZpMZWiVUVRWdk2SczRZZRt92pHkGKJrasFxteHFxRW96dJbQ94ZyVKJ1JBxpBDjQieQnH7zL1eKAXbXk9TdfoSwnPHm24OGjp/RtF3X49wtM63j08FOurhbcunXKrdunLJZnrDdrnj17znrXYQzk+YjWd0wnUySCi4sL6rZm73CPvf0DRpMRJ6dvcLC/z6gsaNuax49K9qZTfu3Xfo3f/73v8vTRc956463IpWgaJvMZ+cE+obrEmpbddgPlnCwr8NbS77bY9QLR71heX3H+4oLF5TVta1Eqw4uEJM1QUnN8fEyRZVTrNVdPnrBdXNJsN7RtDUqQjUqsC/Q+0LQ9Mk3QUiKFIiiJEgGlQKjoYtQZj3MdfefoncdL8KJmu1pQ5AlpnmPqhq0zTOUh909vs9vtaD9o2TUNRwenvPnq6zi/5umTp1ycVRzuK5zVnJ9f0o4EMiTsqsDV9Tku2Mh1kfDeT37CJx8/YjKeMpvP6axhNJkgE83T52ekM40NgUQllOWIpq6j9FuaDbB0DzL2SxKt6PBY0zMqx0gh2a7XbALUdYVpPPlcYZqWIsnoG4NUgkJlhM796QcBERk7/z7wfgjhf/Mnfn5r6BcA/HeBd/85HuxG9ODp06c4JLdv30VIiTGWZNgYLy3G2rbhyZMnKJUQQkQU7nY7mqZlb3/KrVvHeG9v8AQvnj7H9j1f/uKXIcDHH3481J+BoizRSUKW6mGDRyMT59wNVTNJEq6vr9nb2+PDDz+k7fuofDQZIbyIzbM2/qy1Dc56kjQ2bJI0QYhAEBHznmWRJ7/Z7ggC2q5DKEVV12RFRpIIhAhkSYbvHbdPTnj48EnEt/cVXgTyIkHKgh/98Yf0nWM2G7Fe7whOsrhYErxgca04OJ6zWK64vFry4jIq6cxmB+RpwWqxZHG5ZFSMyPKCJEtxLvD87BxxKfjhH38csx4pGI9K2rbBG8dv/8Pf5upyQVcbrq6u2Jvuc3pyi3uvvBLBSE1FtV2yWS/Zmx4ikwTTNLTbLduzF9hmxdnFC85eXLBdN4SgyYspQuc4FCqXTOdzlFZsNxvWV9d0u5rgAsgEpzOSskCjcCLQC00iErTKYtZBQAoXxfSDQ2nIModODaFuaPuOICGRiqbeUm1zErVHsH1UNxaKwzu3eOerX8c6x7vvf8DixZI/bt/j1p0RVbNiPk/4whfe5Ozxjlwf4nrJf/F7v0ugpigj5yDPC775zjf59NMnfPjBJ/S9Zble0/Udl4srdKawwaJDrNuTJCPVOV3T0ewaQgZJluC8offR7yL4ftCfiHDm4H0kwgWB8IH79484Pjzk/PwCrSQyLxiV5T91oP2pBgFi7f8/BH4shPjh8LN/F/g3hRDfJB7LD4H/yX/VAwkEdd1grYtEkc5weXUZacDT2Y1jjxBR7ikoGWfdGPK8YDabAVFkwznPer3GB4cLgdNbt6I01nVMPU3bR3ZhH0U7qroiL0t88LTdzxqAL3UJXuq5XV9fc+dO7Hv2fcQeJElK3e4AbrwPTNvfoAYJgSzL8b5nPI3qvSFInIvINE8kd2itKcoiRv5gMX1PWY6joUXbc3rrDsb0JOmI9brFBUVe7LGrHSk93mlWq4rjg2Om0whQUlpzfnHFbtfiQhy36iRDq5QsLSgKw3q14brdoLWiGBVxqmECwYUBRtyxbWpMH/UOrTVcnJ8jhWZ/b4/rqwX78wO+9PbbjPf28NUlm6tzHnz6CdebhsNXvkTwRFUnAl21ZXXxjMXlOdv1hqa2SJGjVYF3AhskeZKgBbRNxeLqktXlCms6jBcYkbKxFc50SJ1Fwkw2QWlN5yVJECgxKPF4i3Ce4AI6zShKj9oZur7BBk8hFVJ52irDljm+b+naqAOxvS7YPznhW199hywpePTsOVIFvvTWV6iafT765Mfcu32XWSnZrQTPHl+xq7dkhafuG4pxgXOB1WaFCw5HoOk7jLX4YDlfXLBpVoxmBSpPkcQsxhqLs57D/UMm4wlVU7Gro7S9ShR90xJcoKk8ImzJEk2ZF9SVQUvNK/fuIKXgm9/4Kp9+/AlGtRRFxnbTkSS/BBZhCOG/IDqA/pfXP5/XwD/9WAQf0DpBBEkS4kabTKYkSULwnqIcoZSktTbqCCp1Ax2+vLykbduYUQRPVdXM5mPOFwvSVc5kOmW9WvPhhx+iiEIdQmk609M08XSwfYfMNWpI4ZMkYTIZ0zQtu12Fkgm73Y7bt+/gnz+n2kbDUEKg73tSqQfN+HAjBum9j8YkBRyd7JEkJyyuVzx+8jTOcIUkEOmjSsdMJM1S2q5jb37Iwav7fPTBRzgTS6AsTxmLFERO2/aYXuOcJ00KZvMxQSQkaU6WpNx75T4PHj+maSyQoRNFkib0xqFFj04yppN5pGLb4YSRHmMdnoDWUaTy4PAA7wxNXTObTjHK0mxr9Eih05TvfOtbvP3lLxPw+OC4vrrg4aefIPMJk+kUZy279Zaw2yGtIRWBIksZFQWmq24MROO1i5Dduq7xl4GL8zO2mxbnDEalVE6y7ByrqqZ3O0ZFyeE8MB9PmYxG4AJBeqQaBDlEFCpBQJJm6CTFWEfTW1AKpRN86MGbGKTqit5YgogaC8d37vLOV7/JfHZIVqbMJwVp0nDr+JQ8y2k1PPj0Y8bjA9759q/w7MWHLJZPybNJ7Cv88PuU5Zz5/pym7qiaiulsTNO1GN8wnU9Y1zVCJWidxoaedfG5pjNGoxK9UixWUQ9hPBpFHcddFanFxhK8oywSBPD86WNGZcndW7eYT8fYLKGuKpp+R57nv3D/fSYQgyH4YXyn6Dsz+NJJynJ0oxO4v7/PannN5voaLyPgQ0pB25obffyyLPDe0PU1VSUxxrBarTiY76O15mqxokwSkrQbWIiOEALb3Q5vevbyGUIIjDHDRo61+stpwnK55PXXX+fy6gpCoBscil++xkhciDzvZJAib5qa8XzKvXunjCZTqh9t2VVbVJJErPgwuVEv1ZWrijwfMZ7Mme8dUjfv4Y1jOplSVR3G1sz2TxCiRCeGqlqiRUBOc9arNZLYVMqyAtM72tbR24DOJVk+SLIHwW5bYXtLopPBlKUnKzOyrERnUeQlkrQSRuWMF8+eslwuGZdj9vf3cH3PpJzw7W/9CulkjPMtrms5Oz+jqja8evdVpgeHeBvYbrc0Z+f4zZpMK27dOiHPdyi1QZBTjA8QqqBzgWwyiepLpsdZh1I5fe+pjGfZWBatY9laqqaHVcPZouJoPuf+rTvMC0meAZlGaRmFZ2XULhQykqaEkhjX0XUtee7w1uCdpchTUq1o2pZgLKbu2C43eCE4mB5QTsfUmzNEojg5ucWzp895/HBDVfd87Wvf4dnzBzx/fsb+YUkxGSNVx9MnF/igECJjvdugE002KjC2IfgI8c2LgjQdgwfpFcEbdtsdWmjG0zHRMbpBWcV0UjAaFZRZhtYJy+sl2+2ON994E7znhz94wHze8O6P/pg0Tbhz75R3330XE2BSfNa5AyJKfRsXT1GHoDc/k05q22jg8VI4RKsI4qir5mfgl8EiDBHhxF3XkqYpXddzeXlJ13UkOpYT1jqkjlp2SZpCiAKPDAjFzWYzAIbUjVeBNR5jDNPpbFCETWiaGuEjn8EYQxiQb3VdR9qxjDZeo1HGrTsnnF9c8fjJI6TyWBdTz0xr6qYl2Ej53Nvb5/j4lKdPn3F9tcL0DhEk69VuUEvOUConSwVa9aSpwVhHQFEUI9q6AiF49PAxVdVincB6STBExZxcE1zcmMHB3jynKEdol+JF1EYwzkbDjsEMNE5eDBBo25ZURFm3r3zlbV77yttgo+LSZrXixYvnKKU4vXuXtMjpW4eWimq3Y3d+xt5Ycev111EqI0lGzOanHB7dozWw3Fak4zF7+3t44ZnP52yKCW3vqOua5bZm1Rh2JlA78L2laSra2mKN4M5+wXycwSSHXJKIwTYeEXn4Wg8KVD1d39O2lrre0TQVB/vloCZ0xep6RTGa0TcdIdEUecl6taVzO2b7gjIf0eWBPOv5wluvobXi4wef4IXFeMhDVABSKlrdW9OTpPHQWSyuGI1zEJHN+Y13vsXquub5k2eYvicZtAMWiwXrzYqqrWnrlmKcs9tuaat41hweHjCZjFFSMRmVBB/Y20vIMx1l0aeH3L53mx/86IfoPKGclMDq526/z0QQ8MEjM0Vft0iR4LXGO9jUUf1mbzpF64y2XaBVTjmeRF01X5OnCSFEpZxExeah845gIE0iiaVvenASnaZIlbHZrggi5WB+RN00uM6TpRnrXYWUkr0sxweBkhG+XNc1INisVxAcEgemJysyrpZXJEqT5RkdhoCl6lsCGZNyTJHnvHrrNXyT8OG7D7m+qlBpQds0+GDRXpCrJGLBDRRoquWaerOl3mxJ0kiRLouCuq4IxtJ1G9IsQ0mL3V1SFDmhr2jqhlFZst3u2Gwrmqoh1Sm4mDY65fFJLCGUCkgt6Wx0Q9aJZjqdsq12sf8iNSIEGu8Y5Tm+TxAIdFFyvap45bXX+PW//JfJJmO6viO4msXV+2jVkxS3ufPGnyfoBO8vePbpe/zOP/4BT54/5zvvvMn+LdB6xP03XmF++xV0PoFdRV6mjEZjxrMpzvSUWcre8R6N7bDLLX1t0C7BtY7gU7I8Q0pJ1fd8enVNUhvk3phCnVJmY3rbIVxDrhVp8OTSkScJShU4AU56eg+dMTRNjZaePBX0/ZbV6gJVJpTzPbbVNaPJjLCboVzC8sUlm3XPq7e+xNViSb29xlQ1o3SfjJyjyTHUV7z69huY1vDpJw85PTxi7+iQH/zoR3Q7QzEpMUKxt3fCevMpvW+QuaRuarI0Z9Nsca0g0SVaz+lqCGXCarvBmBaZ9RRlQi88P330kIP9KUf39uMhtt5Qm44f/fGPCM6Dd6yXPx8jAJ+VIOAcQQ7sweAATZJluBDou47FteOia/HOkBVF5LkPsFetFFJBlqWkiR6syqPNGESZ8USnECLt2LrYX8iyaDratX20vq4bQugG/EEzjGT04GDkfoZnD548z6JLrDF89StfwRjDRx99FKcJIZCnOmoJWoPpwHSGxeWKF88vYopubKy7kySCXrLoqdDWDV3TDKIY7obr4H2cLCSpAjxdHxmLWgnyXNH3Lc470jzFhUBV1bRth3WOJI2SbEJAoqLrTfAOOcgseB/nzzJEvzqtFUpJsmIUNe6FxhqPd1EIlSApyjFf+erXeO3NNyI7UwlEcOQpzCcl09Et9g9vEYTnyeMP+Ye/9Xf5g9/9KS4b88rrd9jVUVq+nM7JZ3Oc1NTLJXXfsX90QJIq2mpQdgo29ixsBz6gpQQX8MaBzgevP0dvOjZdxVR72mrMLgkoDDq0CBsNOBIVgzqDX0RAEAb9wCjvTdSSdAHron1Z5saURU5wljTJ2G07ri53pGlOUYw5PcnY7rYorWlXPdZ4XrufIVHsT/d4fP2YUZ7zpS9+kawo+Z3f+QEHt0akacF0f59nT5+z3axARpei5bqmqmsEkiCjelWSJpjOMypnGOdouy7iPUyDs9E/c7VdURYF1lmquuXoSHG9XOIhCuzwGdcYdIOWmtYa46O8mFQpUijSkaata0zfRa31ENN4b22kkBqD9EO33Q+a9VLdsBG1Tm7osiGE2GDLykFWLNxAM6N9eXQK6k3sS4QsGzQLYgrWdR2r1QqtNNPJjOVqydHRcTTUzJ9ExR5jSHRKIiIPvGlazs4vCBeXbDYbhOCmzNGDyWmSaIo0w3Q9Td/H7vawXlKlm6ahLMvBpyAMWAZHmuUgXpY13DQXq7qJMm0SMp0hlUAl8oYQhfgTOgwhwqu7vv/ZWFRHGzYhJV0bZdfTLME5x+3bt/nWt77F0fEReIscdPWno5xJkTGZniKl4PryjE8+/ICHn37CZrVgelygpKBzgTzJUHkBw1Rlu9vSNDUyS2NmV+/o6orddhvl1vsecCgRYg8kOFxXgZFI15NrKHS06hIyxJNdGFTo6L1F6wxrMwge5yP12Pvo4ee9x1iDGhyuFX7o+UQH6nEx4nKxpKprLi7O2GxXfPvb34plX56xd3BAWY7w4ZLpbMazF2dsVisOv3PEhx99zMHREX/uz/05LhYLJrOM0XjEK6++SjYa8eDhRyAM49GIvfkek9GE87MLql0bIdcyoFNFCLGwURKyPAHvIUhunZ4iRODFi+cE6xmVI7TUZElGIyt0oqP03j9DT+CXYk3+L7pCCBBExJEP97+SUa+/riu6ponKwfGXSeRgwDi4EFkbxUmbpo2NRQ8gcYPb70sfwWQQZ3BD8Oj7boAs5zene1EUP9MnlJIw9CLKsiCEwIMHD6ib6L4TQuDRo0csFgtmszmTyYQ8LwiDJXWSphyfnBCAx0+e3EwB5vM5ycBLaLtusOO2MNiZ/UnGoxhEVSGO2rquYzQaMR6PabsW5wJZmqNkMgRByWw2I0sTsjxFSNBakqbJzSQjGrHEG97fMCs9XdfeBJm+72/YkdY58jy/IbPcuXuH1994gyzPsNZExfCuoUwViQzkWmCqDcvzc2zf8Pr9O5wczHF9EwPebJ/R3gHJaIoXkr7rcNaSaIHUEmvbCDFud7Fsco5EKzKtKBLJrEwYZxLlWoSpKVTgeFZyenLA0fEBo7JAK0GSDH5/pqNuKpq2xgdPYAiEARgEZ17iQl7K1Xsf6/qujSrR5aigrne8ePGC58+eU9cNSqes1mvG0wnT2YxXX3+dvYMDQDKazhhPpvgAk+ksZrbek5c5fQ+379xlu92yWW/om462brHGcuv0FkVRYKwhCI8PliAsQnuqKsqOTcqCNNEUWcbpySl5muNtYL2qKbIxs8mMpm4jFX+wdP9nBYHPRCYAMRu4IZwNXgOjckyiFflkRFNF8cWsKOKbGkRFIcQGXIi9gKhIFB/TDzezGPzdXzbw+sHyOppaRsUarQVSxpvee0+SDTc9kGUZ49GYtmk5O4t+8D1RbOTF8xc3tamAKIiqouqLRPD1r32d3lje//CnpFmKCbFE8cOJ7JzDWoOxFqUUqZK47mcb9WVAeOnN+PL1FEWBMRaNilzzEEjSSHXOfVSneYluRHo88XnsELxevjc/CK5IFRWXX4qoGGMQRHy8dw6VRmfn/f193nrrTQ4PDyAEvHeRQu0dSZqQKkHf1AjTMRnlvPXm6xyqknFywPm65otf+iJH915lNJuSTKYEIXHek6SaMk9JZMD3Da6vaXYbujb6II5HBZPeI5MoDpsoqGqLEjAe5Rzuz9jPPUWeEJSgSFNGZYLrJd3Ws6ta2q4BUpJUIZ0cSqOXaXJ8Ly+DAj4SnJzp6Zqag+MTdrst19d7GNuz3mzonWWxvEamKR8/eBA1LYCu71Eq4eNPH1C1LYv1mr/9H/wHXC02dMYwLqPFmBSS+WSKty27aoc6khwfHfLJp5/igyPVMROwvidISd81JCryYvzAvbi6uGS1XOGNp60sXWNwJnB5sUBIPxwm4ga9+vPWZyQIxAJVyPiCvA84Y8izlK999StcLxY8ffwYa4d01XMjxyxEVBQWSmJtrJ+FjDgCqeQN6EdKOQQLjxt8CZIkjamg6ZEyJQR3szFfbkAfAqPRKPYc0jSir4zBS8/tu3c5Pz+jaaLJqR4soossw3Y9venprWGz3UZ14xBl0pumJvj4Wl+y6MLgGSeG4KaUoigKlFLUdX1TFmit2Ww2g8NRIEmKqN4j1TANWbCrKiaTcbRL7zukEkPt7yJSEgbSir8xQpFB3gTAACjh0fpn71fKDtP3vPHGG3zlK19hOpsiRMTpe++QgzWb0joyP5VmNJmgOGRkNcKM6XXBm1/9ItPDA3RRIJMUY3oCgiLPYxAIlrZr6Oot9W4dJ0MukGpNmSWoBLJUk6pAX8bsYFRkcXwme6SIwhtZljMa5YQUEhxtF0VJjSOWRnqQEhsIji+zoZdZICFE3r611Lsto/GYw/053f17TKYTirKkalquVxvWux1VXdP2Pb/5zjvsNte89+57fPjxJ6R5gfGOTz56iEehMk1V76h2Fb/6q7/Kd3/vd9iuFgjg1ulpJHLZyIxMswQhJW1rhh6GiaAqE3sXwTkefvKMpmnJs4Q0EVxeXFHvOqTyyJQIroOhR/bz12ckCAyyyINKsJYh+q5L2JtPefr4EUIKppPJwP5i+MC4kRaTf0JqTA26bCHIKAIqJVqrQbNAo1S0JyvLkr7v2e0qwJOm3NTEYvhdgiDRKd6DkpogQclAUaZMJlMW19c0bYdUCuuiSCXZAESxjp9++BG7uo4SUL1BJRo7oAqNsQQRGWTGe5x1N3LpaZqS5/k/RWd+GQQuLi4oyzJiGaSmKHMQis5YkBJjzQ3hJM0TxqMRVVUTeksIg0VbCPHLR4ML4QJprtEpsc8yHJA+RD+CEAK3bt3inXfe4c033yDJM2DoTRgblZaHr13To/KSotQouUN3kuKLJySzA9K9MSLPEUmCD+B8QMhYbo0yjTAdptnR7Nao4AaacZT9zhOJRpKnUCQCyMlTRaYliZaUMo36e0qgk2GcnKgYwLOEQE3bNRirAR8vPGKQMY8iJFJqNArrY38H2ZFIwXazZDqbUxY5xk3YVTWWwJOnz6nbhslsjpSSN996i76t+d3f/UPOz5fs7U1xPjCdzVhuNhhjODo95Ytf/CLvvPMODz76gKP5lMPjQ+7eu8PzsxeD/0V05tKpQtgenEMIh+0NWnpGxZTgJcZEGzlV5KRZwmZd4YynnOQEbCyZ8bS/DD2BP8310v77JTeAEDVVqs2aJ48f0zTRw88Zi1axq25tH8dYQtD1/WDlJHHe4q3FhRDHeQMI6WWNLaW4+X40ihtpu90Opg76RjD0pQd8BA/ZwaMvNuNiOl6yXFzTd2YQpnzZjPQ0bXS+lR4uF9f01qK0xgeig4/S2OH0fmmEIlXEgnvcDSbiZQYzGo1u+gFa6wgeGXQQu6pFiMiahMiVcC5mH85ZJpMxs/kM6xxNGyXU3RAEGPoNznuEkJSjMSEENptNNKsIgSzNKJKMsij41rd+ha997SvsHx5AcENGFcuwRKUEkeCDpnMC0oIgIeiUtJyQ6oJs74AwykAp0FE0FhkDdqoKUunxdUU/fE3GI6azKcascAjKPMXFj4WQScCSaYmWIPEkitgFVwpJiBMVHd9jWZbotMa4GuuBIPDOD05RHiMsSkqUEiig6w1926C8BxFod5I8y5BAU9d89OAh49k8Ctb0hrppuXfvNl3X84ff+yM22xqlBb31mF10e/IB0ixjVI5YrVZ89NMPOTo4xHYNf/7X/jwez4/ffTf2Y6zFOkuqMnQqCdaBhuCiS7EclLnLIkNLOZDqQAQRLfakpHV2uIflTYP7563PRBCIMmKDeUiArmkoihHVbhNlv3VMM5d1PXxQUBZ5VBEOAecdSiusszeDEOcdYVCKfXmSvkQgvmwU+uAwJo4FhYivIU0iECkMIidK6cgKHJRcjDFDk0yw3e5izTw0z14+r28d3kZhECUjLFgoibOOYGx01hleZ/7S5spahJLIof5/iVp8qbc4Ho+BOEF52bTLsox6W9P2kVueFxlZkdN1AWOi23KaJTRdExtiIUTVIOsQMmYbIQSwljTPuX///o1Ii9IJQkBeFBRpyltvvMlbb73JyelJ5PDfNK4CUmms0eikpDaQT/fx6YjedwRdkI5Sgi4QaY5Ic7wIIAUixACN1mAMru9pthuWiwVd05Bqxagcs00b2r4nHWzl5EAOCs6TyJgxKkEkMCEi/fflNR5KqyzLyLIcKRucsxHJN7BRgx/ETIgZJoASkX/gjaCrHS5Y8jwjS3KctVwvrllvK05OTylGJe+//wF13fDhTz/ivfc+IM+z2MPKy8jwcy7KfyvFfD7n4uycjz/8iOkoZ28+GUpVOD+/oKoago9TMGN6pBLIIHAikCSKRKvBcDdqMIQQ9RikkBRF9E40pkMrjXdD2ak00P3c/feZCAIvazIlJM5Z8B4liIafeBAq1urWDLTiyNzr+p7eGJSKzDsRdUfjGjwAvI/69d1AVY6fcRRt6PueZkDGxR6jG6ycBT4wdNujZqAcmI5pmkVfAx+13EO1w/Q9etB1F8Pc2RoTyxuiLwLO32z2YEPMDFwgLTLUULcJJenbniyLH4vWmvV6jRCC8XgcNwwxqL3UQEQJbPBR1DRJUIkmIbrXVk1F03WsNhuCh4DADrZUSioSnUavOyFRSmOMpWk7vA8RpGQd3nn29/Y4OjpkNpsxnowH8cpormpNj1AZNigIml3rmB+f4FSOJSMtJmjhETqHvCBohRjkw0SIiD5JlHI3VU1b10MTuKet6sEPwEbEooBkMHiVSIKUCG9RBJSIsuRSSNygxhszPn8zOtZao5MEpEGEaG92ozsZXh4W0V9CD0EkOIexPdb1dGXJaJaSKs1sMuNytYpmrC5mYG3T8fHHn0SPS51ytbjm+PiEclQyn8/4J9/9LtZafuPXfwNnLX/n7/y/edjs+MbX3ua9995j72Cf6+sVIQiUjpD11DryURYBWcEzGk8o8pKmamnaOFl66T1Z5hlqGKNbH23j0ySN7Mh/xiDwsxEEBn+BJNe0TU2eZ3gXfQBj97yPIpTeQ3CkaWz6tW0UEdFJpBwzBIHoqyeR0RAo4gsINE0TnW6Vupkg3Ci2hICSacRq1y3JWNP3hq7rI0hGSoQIfyI1NyDCcBN41BDI0jSl7yzWB1zM/4mvJo4ijYkjS+883vlYXpQlfqivdeJvRoLxJDCkacp2u70ZYb1UPNrb2xsynzCMguINgIgeizpJEEPvQcqoq3BzkZDDKR6FP4y1vP/+B7FMAKyNAKldVUUSVwjcuXOH0WQc5b9UZEG+3EBCpphNQ9M7bo3neKFxMsEJjRUuGowmGWgQIRCsIQSL8B6cw3Yd2/WaZr3BWU/fGa6ulqzXLW3bDZ37aLKqhYpiIB4CPvoTuFjSBEXsDxFt4QkC0zusCzf8Aa3B+yFTGF5/vM/itVcqjuFFiA1r27f4LtAWJUpnKCE4OjxkXTdcXFxRVVXsF3nPcnGNDIqmaXnjjTf49V//dfb29xBS8vjpU5qm4a3X3+SnH7xPtd1R1xuePnuKUJLsaRTSuXV6m9ZEyrEg0oSDiwY7RVGQ5wXL6w3rVR2DuI/SbS4EgnfD/RV9MYs8J/ifNYB/3vpMBIFoChqtnwVi+IBj469r42nQGz9QbWM5IBUoFcikRiUJq+12aOy8NI8Q4GPqKGTEIMSJQrhh+PnBueglaShISVmWtG1706V/OV3QSg3KNXG0V7VVbMAZO4wb4yZ07mcnjxuAOEIqsiKnLErapom1v/dxO4rYXW+NIQCz8ZhExbr/5TLG0LZt5BpkUVnJDUKSLlhciPgK46KFaz+wI411ERVpLCK4iLS0wxiUKMcltbppEtZ1TZ7nAy3a3jgrd33P8fExd+/dIUk0wZlB/DIgxNBbUAlN1WCMYzKNmolBKBxE1JpSBKUg2BsTUi3jGC4Q6Jqas+cvuD4/o68qtuuKzaaharqh8Rtl+aUAQYi7lJcjMB/LvsHIU6jIqgsv90cQN5nQS5QgA0jIOx9LAhmGeyKOCZ0DJaNasTc91hna3RaVFAOrMwabumlxNrpPR/Kbpd60pFnK/XuvkCY5T588Y7leUWQ5qUr40Y9+xHd/7/fYXK+4de+Y49MTemv58N2f4HxgNtsjNz27qiYZmLVlltP2nqppaDtL3Xa4oXcWBsORIERUSfYWqSRaaUxv8NajxWd8RBiIqaWznixLsX2EQjoXbZS8j9bVOk1IEoXWgjTVCDmmNxbjHFIGhJAgI2mDALbrb+bhSRodWqQcPAjccMMMGYG1UYSkLEc3GoX7+/vREce/ZAnyM1ciY6ibGqSMGASt483cdjcMtjgWhDRPyPIcPwigeBsJMT9TK3JRG94YnCvRMtxMPeII09yAhtI0vRkd9n2PUAEhozJwLG3iWG+z3SGlYKIia9AYE9+jjXBr9bI5aAfkIFAURXw9LmrtCwGj0Yj5bM7R0RHO2RsnYZTABYtUEX4slYybMEA5GkfXICLoxgcXhT+Fx5qWdrPC9g1FkaIzjZIyOjg/f8GjTz/FNh3CQ9u5qOXnAsh4MvvBSOSl4vYNGFZEGTERXuJExIA7EbHpGlwElg2CplJER+vYzH0ZyLkJDN4FhBaARziHdBbfRyVj3xu2yw1920fthSSCpkzXxGDcW+bzParNjv/P3/m7XC8XHB4fMBmPuLy45B/8/d/i/NkLTOt4/bVXefvrX+F3f/e7XFxdURQll5cLfAiRVq8ybt86oap2nHXP2e4a+m6DFJrxZITpLaY3sc80lEFCx2uKg6qpML2JBLlfsD4bQSBEHnuqHXmWcb3dxJmwUHTWRWUhnTIaj5lN8ug3EAyjMkf1Pe16E/HxUmEGSKyxFkRE00WmYUqaZpg+puMvG5FxEhDLAobpQZT4Ukwm0xicgqXvLdZEKXOpYg2dpiku+JtpgrHxJM6yDGssTVdFI5NBMKTZVYyKMgaQuiZLs+H9EzUPNhvqpkaJEZPJBK012+2W6+vrIU2NUOPpdErTNOx2W7JMx1m9eQmMGsatUqCUHv4d1Y2EjH0JHwJayGGs6RCCgWeQ3vRSlNAoAVmes7e/x9XVJYvFgpPTk8HazeGcRaeRaCSDR4tAphVJniGcRUgbxT3wIBwEi2m2bK8vCc6S6TnkGqEVbV2zuLzi/OwS2xmKrIhBW0mMdcPJHzAmxGlAEj836+LpPQwcBtyFwgvFy0AGEufA2RADCvHaJEIgkFjropy3jH8fm4UBj4yqPc6igVRKMqURoaetG/q2xzhPkqWkOiHYOH0p8oK+Mzz45AGPXzxnPBkxHk0wfc9uu+X64gLhQcuU6WzOp48e8umjaFvnBzNV5wPOGWbTebRLW16zXW/Z7Xpa45iMSvK0oG16un4DBHobvTjyLIkBs7Xxv85j+1/idEAI8RDYAg6wIYRvCyH2gf8QeJWoLvTf/2cpDscKlnjji6gYmyYp5XjEcr0GYZFpzmQ6YTxKaesV3sSJgE4UOpEUKo/yUn0fTwAJ0meMRiVSSvK8QAhJr0ycAqTxRuj7Pjb5spREJWidUBQl3nmuF9fx1JAqAm8GTwLpY/BIk5Sm76ODbvA3/QOpNMrH5t+oHHF0dBwputbdlDxSRI+C3jpEEkU2d3WDaVt2ux3T6ZSiKG5MUV5CiLuuuznRje2RqRz0GGJ67kKk7uZFSaITjHH4EANDkqYkSRoDoFJonbyUQCD2HAVFWcb6uI0a9l3bcn5+zna94s/9+q8BIebJzg5gJ4UzLvoJiECZaWSqwHSIZGja4cB2OGdod2u6aoeSgLcwTHGqXcVus8V0BmfB6Ti9QIL2HryJEwGGaZLzUV7Lxoax1oFEiajtKFS0mI8pJj685JVIXprTKqlIlCRRCufiOE7KyJkQAB6CcHgb8NaQpYoiTZCAMxFnoqWmNS3WGIoiJ89y+rYmS1KcdXRdz/7ePLpWLZfszWfMpzPOdxX7szlKZUgpePD4IUIGhITeGJLBBVolBYf7RwQb6JoOJfWA7oz/DSL2fWIjO8QDRAmyXMcSCUmapZFJ+Iv5Q39qmcBvhhCu/sT3fxP4RyGE/6UQ4m8O3//Pf9EfJ0nC4eE+r7/+KrvVNa6vGY0K9g/20UnMBlA61j1DI3A2ncVZqo9KNV4IhNKxLveeVCT0IRqEJmmKUpq2bYcUMcRZ6zCnf+ljOJ/NB1BOPCEvLi4Yj8dYY6iqKjrB1k1k2iUaY1z0IUhSgnMooaPteR9xBaPRmKPDI/YPDjg7O0Moya6qsF0fBS5edi5FnD075wgI6jrOlXe7CqXkDa4BoGlqkiR+bFEpqBvSYzVAiyMMN80iLXez3t7oNMZsKKFt26HUCAOrTsSGU1lw995dltdLLl5cxN7H/4+5P4nRbd3T/KDf2632ayNi96e9bd7KzKrCxkYIBMYWCBDCYmLBgF7AhBkDmgEDPEEIhBggBEgIMwDJIFnYCBDYwkKis8slV1GuvHkz895zT7ebaL9mtW/H4L8i7q10nnSZrJTOkrb23nH2jrPji2+96988z++ZJ/7oF7/gB599Slk4cpJSPAb/VHWk5CHMKBJ1VaCtIoYJ7cBphUqRNHUMc6A7ydpXaU2YZiYyvjsz9iMZjbGPJh+F9wlXGgmUCYoYRtEIsJjOlsNNaYs2Vg7CzDJzQQayefkLi5ZD9B6/pTbVojT1fl7OBwmR0Uv+gcR6zeiqxhrNOPQcj2LLLauCcVm5+vkxX9EzDRO2cPg5sNquGOaR92/f8erFc64uL7l++57f+93flbZOG66urri/OzCOj+EhDq0069WK/X7P+/fXsjoceuZZKFMoQ0YtlabkEIxDTwppaSUWNsdCbRJc+Z99/WW1A/8k8I8tv/5ngH+ZP+cQKArDjz9/zqcf7XmrzmxXr0BF6trhbI2rSmKGu/sHUAG7Knj9as84jDw8HCmNI2b5pjulGH2SmzSt8aPIWYd5ZJj7RSWXiJTkpJlm0Vrv92vW6x13t7dM04w1Gh8jry4vuP5wQ0ZTVDVdN2CTptCZjJaSXmmUtehKJL7TMNM2DevVCmsc53NHPwyEFBhnzzxNVGUpPalOGKU4d4JMr1xNDqCU5XA8UxaO1Vo0AjEFQvTMQXp/lCLMSoamWrFer5Ypd6AsC4xWaLVMjmOisBprNWVZMM+BGCJVWZGAsqhpqoamqDgqRVKZ0Uus1/F0T9v8mO5wR/Iv0FkxDT2uqRB5TiCpQMaw2mxRaiIpJ5WZrglpAh/ox0Q/W0K0dFNg+nAmeM/D3R3fvuvoY4MqtigEd5aTxxhwVmThOWpQiYQMja2xWPsYVGtARXzS6JB4pFWhjBwOLJLynNExYuwCI13aFFk1ZHKIKGckAFdBIuEJJBvwauT21HF3uiNQgIWkMj5F8jQRw0wKgbKWVKeH48Ann79h8jV/+IcfeLi/59nlBcYathd7fuenP+YXX/0xaU7kAFVREWKmaetlNgW39/ccTg/000g/zEyDX1o88XugoF21oh0IXlbTy2Ys5UjX9+I+tH+5M4EM/F+UUhn4ny4o8Re/RRx+h+QV/j3Xb+cObNYVLy8VpTnw4pmladbM80RMHh/OrJpE3dQ8v1jT+RmfEus1XO5WXKyNRC11A8dzRzd4eq9xpWXWKx7CiFKJKXZUDXTTWZ4ABRhVsFptOB0GmnqJsk5JfOxGKgZlRHxiyoKmXtGPEeNKAYK4EmsL2SYohdWGQgvlaImM5ObmlvHdgE+BlCNaKXSp8DoQkpSgWllQkcJBWzfEKbPebCirinkWqtJuv+Hh4Ratk2jIyXR9h1EXGDR+nCl2FaPvCfNM6QzTeAZmCid9c84ZV2Q2puF06lm1K6qy4XQ6MZ570jzzR6cj3k8knclG0nyMAZUG+sMNOfQS/3Y8sCufQU4olUkWsq1ptjugJxUFQa0pzI6gR7LO5KJGhxXD6Wtu7u85nh64vrnj3dsPHO/vmc8zZa6odAF6IKYZnRcxstbowgIzMUdIkiblnEMvk++E2GZJAUMQHYISt2DMkdnPEAI2egoKtNWgPIqIVRmVFSomtJV2IWsICrwJ+MLT64EH3zPknpEErsBUDj8OWFdCUhjn8HnG1Y5tXfPJJ284Hg/8yR8pvvjVLyF5sk78nV/8XfYv9lx/uEFFzT/0e3+dfhwIKXFzd4eyhruHew7DkYT8O2JGqMsmL9J2udGNU7K9ygqdNCZajFUM+Ux0IjYz7V9uJfDvzTl/o5R6DvxflVI//+3/mHPOywHBn/r4U+7AJ28ucl1b9hcbPl69FEiDXjPNI8fTHcbCalWx3m64vn9A24L9ZkuhHfUbiYLu+5Gb21uOXU820i+9/+Y9thgoGstH1ZpkIsc+YJ1jt7tkVW/YtBd88+Vb7q47umPH0N9jrAxlZOAUeP36ghAGnM0Ym7i8XDGMR2KcF6GSyDyrssAYT1kBzHTnmQxYK6q0oigY+p7NZg1IsszFxU62IFHaom1T0p/uKFxgt11xf++ZppGhO9HWtXju0cQ5EUNC2562KZgMhHBGMWPN4+rSYbSnLN1TiZkjMlwzFc+u9pRlDXmQHtJIXLgxitoVBB8Zp5mqbRmnAVcY5nEgo2X19MhvULKitFbAlvI1O+acMdpSFhUxRZpyxRR7TqcT79695du3H/jm7Xs+vL9h6jpUmGksXK0bGqdF3r9ImmWVq8l5sY8vmwpJma5wrpB5jIcUw7L686SsUdpJib846XKSzYWIr9RvTGgLJzIs7YqyFlJEK4Mra2xRUtXQrhRzn1ivNlxctfz8538oA0lkSDr5yGa7fUq4evf+PZvtinEcePvuHdY5rq9v+Bf/xX8RYy2f/eBzfvrT36GqKzb7Hf/s//Z/x5/8+lcydF7eyxlwVtq5sMxirHX4JCvS8/nEOI04bVj6JLz3NE3N3d2Ji4tL/tLwYjnnb5afPyil/jngHwXeP+YPKKVeAR/+vM/hCstf/4f+CrvdjtlPbPuGzXbNhw/vCLFntW65uLgQSKbKuLJFA5fbPRrN6XDm9ZsXXF5dMHqPz4lz17HbRFJu2OxXrPYth/4Bbd/QrlYURU2YMnW55uULxfkA776yzPM9SsH9vXACnz9bs9nsmMYj3Xnk+bMVb95c8O7dif3FczabLe/ff2AcJ9br9ilq2y/46KauKNuakD1t23A6abyfWK1b6qri00+ecXi4Z5zGBXYZKMsJ72+Z55a2UazaFd4HVquWnCJGL16BnCHd42xDXWn8fId1hot1xel0QKmZ16+uhCyTIlVRkpJi6CesUcCZnDxF4cXK21Tk7BjHgckY+i6jG8dm2+DDyO3dDQ8PL9hsd5RlJU8/ZcnZExfZ8/l4JE0zdlVQmpKQFit30xC94/bmlp///Od8+dXXXN/cc/dwpO9HwjShoif5RF1oCluhEOUbLLZvvUzrs6g3fQqLK3Lp9+2iW8hS+aQ4k5Jk+aUgWgOjJMzT2MVluoiFUOpRYsSSnYZKCZUyRVXi6pbRZ7phQhlH3ViyUuz3Oy72e06HB9F2pIitSlwhLcGXf/RHEo7jNMY5DqcTP/nJj6mqii+//DWffPIJN9c3/OLnf8hnP/icoqw4PBw5n860qxaNJkRpbawWSlb20hIYY0k2Ef28zLvk6/fRk9BMY6CpJeW7qdvvvP/+oglELaCXQNIW+A8B/x3gnwf+s8B/d/n5f//nfZ6ydFw921I3FUOfePnyAqUVqJmqFmJsUze0qxXNekXXzzzcPbBaN0LMLTX7/UaMRTlz7nuGcUXxeUvdgLaAy4x+z2q7IgN9N/H+7S3n4y2F1fzuX/khn77eUNeZh4d7/ubfvOWHP/yIH//4Dc6VrFa/zzh4yrLm5uaGfjD8I//Iz3j+/AVffPEFXTdQliXv373nw4cb+n5gGEbqGnb7mn5MkAc2a8vtzYHL/RWr1Z7nzzcY1ZGSIaWAn05stpkQBvr+RFk0XFw8p20uyNlIT6sMQ9dTNzXby8Q4jFxdXYCSHr1tay4vLW3b8OzZFYfDAx8+vCdnjzEWrUeUGhnHAaUaynKmqmtWK7ERhzBQb9ZY7TEriyYSwsiXv/6Cj968Ybe/pK4LtBJoaZJIYXCWeZ6YTh2r54KEn6eOAtFX/PEvv+H/86/8Df7W3/pb3N7e00+e2UdQRnwT2uLDyLE/Y02iMhCTXqTNcmM+bkkee3Z5gj+WunITaKGwEb0cBpOfmL0mxYhG4bTBGiObhpiWfz+gf5OObZSwFEgJV1Rk5bg7nPnm/TXZtNh6w/X1e4Z+RpGZp0EoSwq0Nsyz5/r6hg8fPlBVFcMoZp7Vas1ut8MYw263Zxon3r//wN3dPXf3D8ze8+HdO5y2iykLITdFAZ/IZumR8yDZjWTRqBgrVcE0jxSuICfRrVSlozt333n//UUrgRfAP7fALizwv845/5+VUv8q8M8qpf6LwK+Bf+rP+ySimpvZbq/ouwMQxCNdWdarF4yTyGS9H9luN5RV5M3LFzjrONw/SHqtkkjnwjhinFitd1TFDmdhDhPn8cyq2tCYhuvbW/pO09rn2CJyfz7he2gq+N2ffc71dcv7t3/C7//uD/l3/SO/z+3tPavG4mzJD3/4E/7Vf/VfYdVOfP7ZC3JKXF3UvHi+JYbI2D9AWlNVz7m9u8Nax8efvuHm7pqvv/6azXbFdv2ai4uGnCOn4wfWrebFy5e8ffstQ+9pmx373QXznHi4P/Hq5Qs260vu78/s1iuaesXFdsPFxZ7NVU8MkbatF16Apygsz55fLUj2gd3W4uxE3w2igUBRWMdud8GbNx89zTTmeeL29obLy5LLj17SnwemrufDt++o6h0hzuQs0tqQxFugVJJJukLasBi5/fCB5tNAtCKYHrqeP/jX/w3+b/+Pv80f/OEvefjwFf0wETKSpKuMDCyNIqfIoR+wJuM2rRwAyxVzXNgJmsIWKIXcLLYUE5hRZJ1QtiCoQJhFRelDwnslVu3lJuJRKuy9zAEeLdVakqOUEiehQkxko890Y1iSfyMqJmII/OqXf0LbtIR5ZNXUWKuZs+F0OmKt4Opc4ciz6En2+x3eS/BNWRbc397hvWcaJ5xz3NzeMnQ9RV0t2ggwiM0+BLE7K8QV6PxMURYLjSsTougmjJkpS0e1xMoXzvH+3XcX43+hQyDn/Evgr/0ZH78F/om/38+TUuLu9pa2rQl+BkSmO0/jgpYqmNTE2A+EACjLar/jw/v3HI8n6rrm5uY97WpFVTdAorCObfuSr778lofDifNw5vmLKxqz492vvqWqVkKknUZsKDBpy4cPX8r6T3l+9IOPefFsy6qxvP3myNdf/gnPn72kLhTPLtZcXPyAwgYe7g989PqK9VripC/3LW9ePuP16zd8eH9NjIEpTNxcz9SF5uPXz/j44zfE6OmHM9988yWbiy2ff/IKw7z0uCU/+tGPaeo1X/76G55dvcLamvdvbwFLWTacPn7Jfr9Dlx8Wk0zCOYP3M11/5vNPX2KM5nB4IJNpa808eZq65f7+gRgSH330MT/4wQ95eDgQgudweOCLL8TZePXJ5wQfefv1t1RW8YOPP+f5i2eSlKQNMSRsVmhtyWkWWKmXf//XX3/F678WUFZhtOVXX37NP/8v/Av8v/7GHzJ7hWMkLGKcfvb4mCgLR6oK7CJCmlNEFxZjxaeQVV70EglrzVM0nbMOZwussURELut0gVGBsRMTmoiFhLisFIsZLOJjFrVgVou6UWYFeiFWkcWEo7Th4Txw7AbQFqUNKUWePbtiHEeMilSFpSyEVoRxHI49SmXRnziJmReremQcBzabDePYS/pUVXN/f8/x/oBVmtIVGKUF9PqIBlOK2QMuYczC3ljk0nMMiBxeYYyiKA1l5XC5kNVllorgu67vhWIQRMH3x3/0S8qqpKxqpmni/fv3aKV58+YNztWUpabvB06nB+5vrvn2m28pipJnz55xOBw5HB8Wdv8eZzUpGG4+dDw8dPRDT+0iD9df8s2v7tjuYN1YVDakyXG6H/HTQJgVNx/esdu0GJ3pu9NiWskUVvH226+Yx4711vHt22+4u5O1j8qBoTvx0etXfPzRx7x6+ZrT5x3jOPKHf/SHjMMzPvv4Na9evcCHifvzgdcvnlGYzKtXL/j49Uu64z2b9ZaiKFhVNXVV8uqFyE3rusXqhLMVxjjUmwvatuHt9S0hyF68qQuKbcvtbaA/H7m82LNZ1XjvWTc19cUlz66ecf/wAFmx3WxZ1QVxdjjX8vL5BZd72cxUF3uMdmQ/8uPPPuP5xQs0BRmY/EyIinoJXU0pkYJHzyMKuLuRSPmsND4kvn37Lb/4xS+4vr4BU7JywhEIaMZpZJg8KddYA9qJdyTkSFJIUpOW3X7+rbogLy5PHwI5q8Vg5tEWbOXEOqu0QEtgQW9njFZPKVPxSUIsO/dHKIpCiUjJGOqqwK1XHIaBYVpcq4svojCGZ1c7VIyEyjKNA5DJ2bFqazJZZiUkVq2E5Vxff6As7KJuVDR1CVloSTHM1E1NjB5lNW6xRKOWf13pKKzFR1HDOmexVok+oy7YrkqqoqCwFqMtw+CpVyWHw5HtesNbbv70jQd8Tw4BpTSvXn3ML/7wF8xTQuUHDscDMUb2+z3BQ86aoqqIc8fY9YyTsOeqqpShVyVWXj97op/pTgfm/kvmuSPGke2mwerE6XCkNECYqAp4/uw5P//5HzF0B/b7PdM08OL5c2Y/sWolJbZtaj755GP2u0u++NUvUUrRj3c83D+gUHTnjsPDgc16xZvXH+NcIUEl6xXrVUtT/1V+76/8lK47M449b999i1Wa7CN1UXKx3XF/c4dVBqcdV/vnpJR4uLvj3bt36NcilFFE6qpAqXn5f9wJY2+z4Xg8Mg0DVisu9xfiPYiJoRuFmjvMNEVDoR3PL64AuZHOh4MAK3SkbRvajz7icDgyWcVms8b9+IfsNxeo6MjRSOk6zWgrYS0ZeRrFJI5AoxRdd8YUJWEZ4A39SNf1i+GJJzFLiEHgHd5TRMGlx5TJMRJiXGTYohrUWmOCJgSezFMaMYkFIskHJj+JczQGSqMWJoIEfhqbQXnJuNAyZExZlIIiepKVICiMNcL4qwqqpsTWJa01tKsVPp+xzsoO/nRPUzfsLnbEMHN3myisYcrVk++jH3qK0kLyIkVvaxSBh/sbUvT4eaIsStqmIuWEMdDWpfAWtByIy9wSV4i2ZPKTMBlUpi0LCrPm5fM9bVVTlwVj35GjojvP4q5NAr7le30IoFnVWy52z4gxUtoGpycqZ6nciugVzpWo7ChdxcvnLxa3mF6SXcJCw4lQllgjarSg7ylrzxQ8q1VJzmeuLkvWqwusLbi8bLi8KPjVF6NYR2koi4q2abi9vcYow831Dbvtju1mi3MFCs1uu+Orb255dvWctllRFiXr1Ya2XeN94O7unpvrW9brDc+fP2O/3TGOPeO5w6B5/fwlzy8vOZ8OlBtHd+z4xS9+zk9/+ju09QarS2xh6U4DZEXhjCTROoWxnvP5CAOcTifqesN+e4FCy0Q8JaxxS4ISODPhmgKrSnabHU29JqVE153wIcjwC8l3KFzBdrslNpmqcOx3G3btFrKhP3uaeoWztYA+y2KBe6SFaCwu0Kosif7IPHn63FMXjk8++ZQf/ejHfPXub8mO38lBEFNc6L/gg2eaNIREoRM5l8QQRd9ARhmZ6MekZGPgWfIoxYEaY6YfPSl55smwW1cYV6KNA/TimwgLg9IIdTpl8U6gBD7ixImXtYhtmqambiuGHNhsVrx+85L87VsympTgYrfheDjgbGa9aiW4ZBwJWrB1Xd+zah273Y5pGtnvNux3rRwg3ZnCWurCsFnvnshR/dhjaiFHp/yY0ShzM6dljWxtYg4zsw9oAtt1w2azWijcmamXKsHtdozDwGrVEr7vZCFjDHXZ8vmnP1jw2xGyTJeneSKHSL2rqIqaXAWMTiijGOdp4QbYBdwJVhuC9zjrKCrNFDJttrStJcVIXRbUlXAK5ume+4eZ9UZyDkIwrDdbpnFgt9uLf7uqePXyFcdjzzjOvH71mv1+T9NYVqsWpTR3d3c0dSuruwWTHWOi73ti8KzahmHopMwuS5wTrNlDWZKzUIA/+ehzfucnP6Nd7TmeOuq6ZBwm9GuRDTvnqCpHWVkOR5mVKOXRGLSyrFvJUTwcD/g5UuxqMcOsNXVdYbWlsEJFHsceY0pyUoQs7MKiKJZQTI01BUVVC+Rl8epbW1DVDU29oSwLbOnQzghmzMvKMRvZsz/c3XH3/gNq95zSWF6//oh//B//x/nimyNffP2WxCQ9epaSOyaR+k7jiLJQN8WThj/lKE9rLVsCcfrJAfLoEclZE33k3M/MfqLQGWcN66YEJbkGIYoTNcOiFBTjWs7i+4BEMJmQIeRMVRas9htsoen9zGpVUdQlEJknsX2/fvOaX/7JL0lxRKFpm4IYxB05WVg1jrqp2O/3dN2Z58+vnuYab9+9JacsTENjQBna1mFPkio9+1mIQY++Ea1QyVMuDsG6KgnJEXMgTD1jB0MURWT0M9ZpCespJBw3Fu4777/vxSEActJtNjs2GzgeTzhXyYsx+2XoVWF0QVU0KOWFkxcmjBMQ6Gq1oXAOP45MQ09bN9i6ZfQJawq2m43Yeq3jeHjAFY4QA9fX79FasVo1oFp2uw3ffvsNV1eXwngrCoy1zH4CpXj2/BlN01DWhq6TtNf1ZsN2t1uchDP3D3coDU1boEwiRFEtrtdCLX7MOihLaRu0Vnzy8WdsNhtWqx3jOFMUJVVVs16vljI2UDcFVeXYrDdoI/kJlj1KWcpSUGbjKIaoVbNZtgXgbEXbtJImFAJVpXGu5Hw+4oPg0uq6wdmCYZgX2XFm6Caiz1hT07YrdrsLUlRLqYqEfcZAjhNET/IT8zRyd3vH3e09b55/AkqGcc451ts19v01KYwCXNFSySkjbAVLwmkoXUFVFLgnkpKg5FMWkvBjZoJYgRVhmhmHiXMv5e9AYNWKLkNZR/CROUbm4AkpLOIbublY2hNyxofIFDxFLmgrUQMqB01Z0a4qtvs9r19e8uH6mq7rWbcFP/jsNe/fv2e9rlEgN7JbCVD0+TNpsdoV0zQwzxMpywBXI98nlKHvRzGy6YyzGef0U/6FD/5pjakTtLUBbYlIfNqpm9Epk+YRP3usNtRFiXWWMabFMfvdQBH4nhwCKSXGcQDUkzmmaWoKV8ju2HtilKGOqxoUkZhlf1s1FVlBW7c4azmfjlhlWLdrgqm52G+wWqMzGKByjuwNTdsQUiDEd2irqcqGsl5JuovOrLcruv4MOXN3f83D4Y623bDdrTDG0axKSUrSmhADTdswjj27iy3T3DPPE7v9mrZtyDFCWujFyNPPGBhGT7tqhCxcVmRgGEXr3XUnpmlgtdqzv3jG/f0tWoFzNfvdM8pKwlaTLwHFer1ejCICXy2KSuTL2hFDZp4iduEKFq5g9gPOOXa7/TK1zguJWWNtQeEaQhTsttEFTbWiaVq6blxyH4AkZCdlMswJPw5LpWO5vb3lI6WlLC1rzl3P7e0d4zRRKDFKPd6ERhu0gkIbSqtwyuDQOG1wVqGM+i2LtEGrR7efJsyZ4GeGcWSYIjEpdIZ+Svio0LYkIZbfKXimKJgyltwBrRUpqgXIkZlCoEyJoq1YXWxxleaylrSk7a7l/sHjlOdq36JMZv/xC9rG8vLlq9/EmSVH3YgD1BrD/mJP15349u23aGVxhaNyz0k5kZKiH+fFoJbYbYXp8Ehz8gtPcpom1lXD88tnGOf45u1bvn7/jlVTPCHxCAGjFaWT90BZGoIPC/Tlew4VAXGlpTjzcD4yTvNT8Iiw86W819pisEQvfW/ZNBR1IYQfG8Xvr52U5sqSTMWm3VG6gqnrMVkAEbvtc4xThOh58eI1rrDMwVNUFejMdtuy3a5ABfp+QClYrSuUShSleYovu7i4QAClE85plC6wzrBafcrxdAASMY4YBXVbQVYyXbYG6xT7/UZciNZirOF0PKKsYrMVOGVRavEbaE1VNbLnzuXCmC+YhoOkLTsBjUiUWoHQZQNpYSLKJdRkpbRMlBUUhcM6caDNc1haApEYo2uGYUZbhzYV1paAoahqXFmgrFmUghM5hWVdpbBFSXfu+IN/4w/40T/072bV7mnWaz7/8Y959vwZX7+7lnI+Z0JI+JhRylBai7NWeARaY5WhtAXGRHHz5fwkGspqcQAqg9YCC/Fz4Nx7fMwUOtANnq6fl0m/IabEHIMMG7VCRYNhcSGiFuKaSHR9DFSrlo9/8CnttsY4/eS4nPojn37yiouLC7765luquuT1q59yefWclDNd3+Oyo2kb5nmi73uuLtfstxXWRBTSyoCEw9TtmqpZcXd/z/l0ktlWzqza1SJmipCEFbFvV1xdXmKLkqJQhCTqRVdUgKI79/jJY41lGKbFYRJpqoaiLL7z3vteHAI5J8apAyQauywKnFWczgfGYURrS+EkeLNQBpJCGYPVjhRgHKV/Cj5hlP1NVltRkBe5aV3XqBg53N3QtjXC5YPNtmX2I2Pf0W5bfPTsLtYYl6lqyzCKdfby2Z6hnyQNJsHYJ1zhWLUNxknmfVFaIRwZKMPSeKKwSlOXMqCqojzBxnFgu90zjWLJjWFe3G4eYy1161B6i9FuSQOyGOPQuiHPnhQcKZR4f8IY6fPFIpyX/y9IZSXf/MLJZDnMM3OKWCeZAuM4LfZVS0yB8TzStiu0Lok+UbYVxpRYU5Bixi6WXUF7RWKYCPOITgFjHefuzJe//ppx/QvOp45XLy4prOPjjz7iZz/7GX/y62/ojgPJCxPvaTfvHIV1lM5SOkPlHJV1QFwGiAgYBXhMjso5MQd5SvbDwOkc6MaZttRsN5F+nCksoLV4HVMiLf2/tAPyGskrJm1O1gqfI7YquHr5gssXW7rzgRQ8dVVxsfsRZVUxjAP399e4wvD5Z59QNQ1FVYm1egySbKQS9w8P7DbN4uqMWGvwwS9o+ETVrinqhv3FhtPxyP39PTkJy1K0C4nSSZ7Crm6oyxJXVlxcbvnBjz7n1I84W1JVLV03cHt9yzzN3N09cJpmjqcTq1X7BM39s67vxyGQEt6PdN2J9WpNXUsakPcjMYXFC51Ji4hkHmaMM1QpgdOczx1lWaC1EQOMdZDkTRPStAAmYZ4HhvGEcxHrNHMcqV2FLRKrTUm7qun6SF2v8GGmbkoSa8Zh4vJKbthpmogx0537hcU3kYnEDOeupywtKXtS9mgNVVVilWLoz+SM5AYaxzQPnM+GnBR1XQmbb73i1N9wOt8Ro6CxyrbCe8/5PFFVLQon5iBlKdya7U60+X3fEVNivV4TwhJkssBR59mL4cQsdOGc0MZSOLuQmBIpibV4nDxlWUEKhDnhNgXaFgt7YUlmnjNlVWAMArZMksYcg2c4nZinaQGYCKrMak1T1/zkJz/h9d/+u/yqv8GHhF1YCSCiH2tEzuu0EfLRIuxRC2bcaP00Xyit0H3HceR4PHI8HJkmx7kbsGqpXLQcyrBAUVm4iFpYg4/OSh4Vj8gPiSVQEjRTOKq6pC3XsHAoYkpMU+Z3fvIjjCu4vLrAFA5txT5t6rxUF4mqclRVzTyPXF7tqatSchqcbACKumVOCa0STV1Q11LVrZqWGMICxxVykc0ZnTJFJWa6V69f8827D4yT59Wrjyhtxf3dA2TFw/HIwzDy4cMHdjvhZPyP/2f/pz/z/vteHAKgaasthpEU4HToZCc7RaYxoJJDkzC6WIATiXGYOQ4dgczkZy4u9lRFyTiNdKcelTIualxTMZEZup7T4UDwE+mc5XCxin4OPH/+DB8mwqjoD4HVuiLOklWwqq9wecAP8mY+9mfp2UikOTCdE6vNCqWh8zNkxdh5+n6gdA4THdnAPMg03KkldSdphvPAarVFZS2WXWspTcu5OzFPsleuty0P5yOEJBWAn7m7fsc4TlxcXnD1YsPDw2Gh/FgJpzCOwpbM08y09JuhimzWm6VG0FhdkgtQs6XrOxSLWw9RA6ZykK2BjpTWYRMo5ZnShEfoN2bWuDGgh4nQn5iGCNWKj370Qz755CP26zUqO2Kqybbl2asrPv70Ge/frYjxjJ/TExDEFJbkFMmBqQ1FpXEm45ys8gAKZzEadJaEpMJZpmz54nbktnfMMaGcplqVFKVBG6n2HmPFSApNgaIkZzkAU1ILX9AsfoNAkcSjkZfKp90/oz+el3Wb8CPKsmW3vwSEd1CWVlqBxeVXuIJx9FT1irpqmSfFfnv5BKoty5JpGpY0qkipFVVbU1mB4EiAaIEr7BMOffYDRiHQ0Jjpu4nCaD7c31Pakh/+4IewW9PUNRcXK5KG7rMXkkwcvucxZMZYVs0OZyZSDkJ5yZqmWpHCgNEF62ZHVdV03QEfJhSG09CT4MnNVpYtcxEYZgkLSSERp5mzD5zPZ8ZxlDJMGc5DT1GUFFnR95njcUIFjx+gizPT7IUr0GiMtZzO58VKO1I5S+McU5rJPlBoA0bjbMU8ek6HgRgyq6JBpwLvJ1R2EikWJVsgJyFlWlPQdwOHw4kYM9YUhFFjVEVpC3JQEMAgII95Gjkfb/niiy/Y7ffM4VNOpzN11UiO/Ryf4KohpCdSsZ/9gtxaJuFzYPaTJBGHQFNXgABOFZpZDxSqQCdPlUFNnoBHmRltEwZFnhJ0kfBwJM8d4xw4+Uw3e1ZtTZomjC4JWaNcg6ss+4sVu/2eOEf6MEhJngXOEVUiGoVyYJ2SSiNL2yFZjUtqdfBM88TgHcfeczPAh9FQpJFNZWjaEmXESpwTkKN47VmwY1GTlBKbsdLCI0iKLHxVUogEHzBKo7PG6IJhiASvMVYSrNP5geAzRWGYxgFFhODJBG6P91hTcD6P1OWWVRsZB/E99L0YzZQyjENmGI5sVgYNhFmMWGGembzAQbURgdboZ1T2WBImGYyWIbLKUjmNXUd3PjH0HeNwIOfMetOw3wgnIzn1Xbff9+MQkIw9xzzPOOMWJHimqirZ/ZuC/cVeAkems6CltEw8bVlQtw3GWMqqILY1Tou2eoozYRFh5JyXF19RljIss9bSti3DMDL0A2phB87zjAK22y193yNWVi0EWoRXl2N+cq/NPhDnzDCMyw0nkVaCupKhGwoKUyyRVcMy6FwUePPMuTuTEYfgw8NBnGbW8fAgKCvh6kXGruPi8pKb21vmeearr74SpNgTM18m/fM8U5Y1j9mMIURurm8Qjbl5CtrwQZKM7AIxLQonGw8fUEkxMlOlEYJixuPLiFm5JezVM02DCF905Hw+8/7de96+e4sm8as//mMunr+RgaGz0hZUFbv1muFwJoyBFAVOSZIknUeM2uPXLGGxCEUppsUvrxknz/u7I9e3ExG1KA0DlSsorUblCEkxT+Hp+y9A2CWCXClIApZNSdgMRssNFVNkHAa895zOJ2Yv1OqiLBbGhH7CqmldMJ5PxCiJUX0/MI4dxsjhez4N3N3eE6OhLAUndnFxwbmblpxMwb5HL9yIEBNosQ+XVSXUYCXI/DAnAhHbFrSrLeOQOBw6Xrx4ASiOhwdSCsu6fbWg2mUIbJ6clv/m63txCEi8l0yuh6EXhLU2BL+UMCrT9yfO3Ynj8cD9/R3nocdVJa6qWG3WWCuDw0ykqCxTL9FbjyBGayXa+3Q6MU0zq9Watm1F2tkLvWcYzwvWSabxu90eAGMUGUeMnqpq0cqQVMA5B0ZIw+fzmX4YMFbShHOM9F2PtzMhCjsuuEwIonCxthARR0oLRdjQ9x0piXouLVPk0/lEVcka1BYOmyKb3ZZPP/sUHwL3h1tRMi7sdOfs8gaVQ1Qt/MAQAvd3t0sOgqw1UYqqqhfpdca6AusKfIgkLWTePoxYfcLgUKUmBI9TTnBfZHyUr88PHafTmQ8f3vP65Quc0fztf/1v8nt/7R/G1SVWK+qyYNXU1KVo3GdryUYUeh6ZK5DzUy6DJER7yko0ECprUlLCH8wSfHrzcMDHkqquaXzPqnZUVmPIkCJ939F1HfNCmX5kTOYs8mSd1SLKkaBbk4XXn5bvqT1oikLaK2sMx9NhSTUWDqEPMrS0tljox5mydBJyYlb0nWeapEKYJs/V1TNWm4Lz+YSxUunM88w8CyOgrksSoLxHISj8aZqWzAyFMY4YMw/3B66vr+m7iRcvXmGtoOvbVYtzlu12jfeyXnyUMH/X9b04BFJKdP1I01TEDH6U3AGFoixruv7MueuYJ8/xeODm5gMPxwOXz68whWO9aSnLUoAbKEprGacOV7c07Upy/haoZIyJYRjY7RxN03J9fb3AQh3OZNl7awtkzufT8nTV5PyI8RYzS1FJAEjICW2F8lJVUJaF2D1zJsx+SY6RN0UMCq0dq1WNtUuU+DwJ5tzJdL6qKzbbjQzVUkAZxRwmvJcBqbGK0/lIu26EIFwKHcfPctA4555SlqqqXoRWjrIsF1mxHKyPT8a6rinLUsAX1hJ8wDoL2aAwhJAY0kRdapqyZk6T5PRJCohwHzQ83AtU4+72ht//vb/K1J/56uuvebi95sXHb3BGs9ts2K5WVNZQW4d38oa2VcWUZqLyQvONsiWaVMYqGdxpZchZMc2RaQ4MHs5z4u440KVEs11xUTXsmoLaIX8vRvrzifO5x0dFTurpAFg2goBUd9IGyXuxbmo2283Te3OaJoGRGEVMgWEU2KxSinHqCTFSVkJpLjKE2ImDsCkXj0eW19JHqkZTlpmUDdM08nBzR+kcSzADrRXyNIuOQqqUgA8Bq2S1entzy93dkfu7E6vVhhiDaA/qAmNkiDqOw1N2xeNB8l3X9+IQyBmGcRYbsFKM0wQKqrrClY53H95DFqHIOI3EmGjbVgQ2KvNwfwcqUxUFTVWRoyErwWqnlJmmmZwzTdOw3e6YZ884TkzTzDCM7Pf7RQc/yRrMarSBrjuz2WwlnUiaSym9Y6JcVczBQ4qUdY0ylnk5dVWGyhVo1JLVV5CVllRhYygr2X4AfPj662W1qGjahs1my3othqB5nlmtW4ZhYO4n4hilRRk6Li4uSCmx2WzkKT+KZkA4DLI3ZolrFwVgKSGsSXIdH1uHsiwXBZ6VfbsPtKs1g09YXeBMhQkFxsq03wZJ/JUVq8YVcsD148jt7Q3T0PPq2RV//OuvCaPm7sN7Xrx5idWKzXrFbr2hcSVNURKKkq4bMMj3LmtDacU+G2LEe6hb4Tg+pkFNc6CfI/eniQ/3HafRM8VMUc/sryq2taUy4JBQ2HkUJHjKhpyWbMIcAYuzZskhWHInZKNLu1rx7NlztrsdVV2Qk4Bbc86LsCpxPg/LATEK1suVyyHq6UOPNQ5VCck5xhkwrDYr+v6BfDdijGYYBw7He+qyecoenOaZ/cUlSmuctVhXoJW4LU+He/rgxU9SFDx7/oyL/RX7iwumcTGKzSPr9ZqUxDiVF+rwb7dZf/r6//sQUEr9FMkWeLx+APy3gR3wXwKul4//t3LO/8c/73PJ03lC6wPD2DEMI26h1CgU0yQZgtYVNG1NVVWSiacSs5/op56+O7PZrDAqMY0J5yzTPHLuJ7quI8bIZrNhvV4TY+L29o559jKHcAVlWZGip2gqUlIL6ltuIjlFlej8YxCKTJSboKwrnCvxMZFnQYoVziHJN4ayqAlJE5JapsiSoqRU5tyd+PLLr9hfbLm6uqBpaqyzhBgkdsxIdl5ZVSitOZ1OWOfQ829mHSkK9jzGyGOc2uMcQL42t5TVQSLElpJYUnAFoe4XabaUsBatLaWpMMlS2JqqrCms6AMkEFigGlprCd6oSlJOfPGrX9HUJTnM3H14S7l6zv31O8LwY2wj5harRTdRaE3jCrpwYup63KrCOUPhNM4ur73WOFtitSVGz+Q9k0+MXvHu7sg3NydClkzCGDzrqqItFM6AToF5GiEJfz8uqcOCCpBAMqMlmEM8CUrWgW3FZrtlvdksceYa5yrWbcP9/T0pZ5ySmLhxHAHoup638f2y7swc52uuLq9o2y05S3s3Tz1tW9H3Z46ne8k4cA5XWMZ5wi3EZDFThUX9KdyEtITdhqrFz/MiO6+pyobVak1V1xyRNW/GojVYZzifOnKW2Vbb/iXgxXLOfwj8dXlhlQG+Af454D8P/A9zzv/9v+/PlRKHw4H7+xs2m5UIKhY4hrXy5Bz6gYeHA029Yre/pGmkTTDJU5el4J6jTLz9PElf5iwp2aeEoK7rnnLaH/v4x5SfnD3T1LHZbLi7O5LiiNaO0+nE6XRehpRx+eYPkDJVXaGsg3Gi64al7RCYZ4gRi9ycX379lq6fefb8is2mZZ6gXYsbr121NE1D3/dst2tijDwcbjmdTmy3W46nE2UpPe/sZag4e8/9w4MQZkG2HEVBURQ0TV7EPoK4eowfDyFQN9UyzBSIxjhO9ENPTpmyrJfDN9EPk6xoxyD9bila9IyXft17iBJvrq0FYxinmQ/X1+y2G26u32GJ+O7E4eY9w/lAYxq6h3vOh+OS3iNtRQ6RcejxaaZIJU6VYMuFIKyXFXFiGDr6eSTbgs4rvrk+cneS9inNM4WGykJhElYbYvDM47QcarIeZLmx0AarNUay0ckYtHU0qxWXL57x8tVLXFGIurMsKcqCqqlwvcPP84L6Fi+DtY7dfg8o7u/vWa1W+DmLwMqWKBTbjWKeg1Rc1gglOEWqqqKpVzzcnyjLGmvdEqNeLAPGUazDs5Tyu90OloNbsGoIkdsLSn69aYGWm5trrNuJVfx8pmkaSbD+jusfVDvwTwB/knP+9eOA6t/WpWCeZ6Z54MXLZ4Dl3J/ISdJ/jXEcjodlgAfrzY6+75+eeptmTeEM/XAm5bjw1mc5LetKBms5c39/z9u3b3n+/DmbzebJoTVNE9N0FrKQsUxj4HweqaqGaZQ/U1U14zhJZFWGjGacAome3Et46GNWYPaRGCKByNgPfPP1W27vDozjxG63pqwsL80zqsrx8UcfU5aWb99+xTzPGFssIApELNL3aKXk9Rllnff4Cu+2W+qqwi46+sf+f7PZUpYV4ziJFXU5BJpV+xQ64n0gjRMh5mVIumKab5jmwDjNNK3ocxNgnMMUC8duVszDiHU9RVHjY6AbR87njrZp2K7X3F+/py5q7g8nHt6/5eHtt1jznNPDA+fjkaHr8eNMaQt0zkTvGQ4jejKEsUKvWtxKzD/zecTqxPFwB9bw/KNP+dCduDtNjEFRaotWM+vKUtmM01loO34WSpV0cU8+BbcoTa0xWJVwxqCNo2hadpeXvProDS9evpR0aR+ockEInkRiuxeEXIwJ4xxlLRSm/f5C2itn2W63bOc17aolJyPDRl1wcbEFMoXrRJ7t5ClvjKMoJP9yGmf6vqdpWoqiWKApHj/LBkch1WVVVfR9L1sLIzZuY/XTDKHve9brDdvt9t9yKAj/4A6B/yTwv/mt3/9XlVL/GeBvAP+1Py+CDES/fXF1SUoeVzqmcZbeaL8l58w3X39DyokXL1+Ss/RHYoRxOCf9+2OGXAxRtPNh4nx7w3qdxU2mFIfDA7/+9a8ZR4mNds4xDD11XXM+n0kpcTwdqMoVt3e35HRPTpo3rz/CGklzEblnJGclh8LiSV9v1otsOaGyou8HjtOBeZxp65awUXSnM+NwZrNpaeuStKoY+x6jajarNRqYhpG2blAo+k7yCoMPUtKHRbizpBe55cmRYnp6Y9R18zQHkCj037QHwQdO57MMsIpqgWo+DhItFxeXklbkHP2poywaqqqmaCq0Etuu6uD+5pbudOLF608ZJ88wzaAUn376CZC5v70nxYxDc/3Nr/njv/v/par+KmpRFhojKjqbNRrxCUzBM3YzfhwIXU/qN8x1hSWQ/cDtzTUvPnrNq9cf8yfXPyeoAu0yKcO6Kdi1BZXNGISRIBj3hPdJBoLLfWCMBqMwKmOVBKmWTc16t+Pi6ordfsdqvaJcDtecM+M4Mw4jTdtQV83Ccaip64aqqnDO0TQNH3/8Md57Lq+e03VncQxaR9/d0zSOTMTaEgEoG4bBA56iLMUBuKRNpwVP771HK+n/y6J4Souepon7+ztSeqQLGeq6xTlH13Ws15vFN1LIOvJ85ng8fuf99w8ii7AA/uPAf3P50P8E+KcRBeY/DfwPgP/Cn/H3nsJHXr244NmzC1IK9P0JpRXzshZp2xWXl1eitnMFVb3m9uZ+cbs5zuczisw4DXLTVAXaKMqq4f3791hbst9vFpdW4Ec/+oH465c0HmO0rKFKx37/EfvdJb/85a+5u70lZ0NOmpcvX/Hhw3s22y0g09Z5cTY6pSRSTGnp/Y4nrJG4K2ccQXkudjtevfqIeR45HO5wRou01sI8TqQw0zQlVhvGOUCWfL/gI2S/mIQcbbPCGktZlVLeKfBe5hWP6cXTNC4JxHHR12f2+z0oRdfJuixnIelI7qJImZu6JbjfJCBX1lA4iSbTpSPHQBpHzscjN2/f46qS3cVLymrFerPj+csXUo11R4ieuRso7Zbj3Q1//Ad/l9cfv6S9aNmsNzx7/pJ1vebm23dSvVhLpUtJXiITfeZ46NA+s2lLVNJSOlc1/TBzf+iJ2aILTQyeolZsa4vKA/M0EiOQhW+YiU9gMgWy0WAJKc+LM9VairKkrEps4ZbNh1oO04gFzqee0+lMWVaEEJ4Gq0ophqF/irM7n88UxZp5yrJlwdDUa6xxeA9Gl/h55nQaF2oyxDRgF31M07R475ecCJlZsDzhjZHk5Jzj8pSPzH7k4UEyIJqmIadMVdY4W3A6nR7vtb+cweBvXf8R4G/mnN8DPP68/M//58D/4c/6S78dPvL7P/ssu0KGaYeTxzlNzJHD8chms2Wz2TKME0VRst5s6PuRru9JUWLMY5gxVlO4CoVis9mSNVT1gapawB3R0zQVRVFweXm5bAtq2lbimeZZk1Kg63q8j6xWa/puompWFEXB7e1bYpJvfl3XHI5HrBOia1nXT1P6eRgxKNkO5MzUj5y7CVdWhNJhdEbrLPJPYyisY54HdFORFhRU3wmAcr/dihqwLGmWwU7hCtYrSSwepuHppk2JJ6VgSonT6fj0sRDC00xBqhiZWeiFY6+0JP9Ok/T8Wht2V3ucK9FFs2ghZMAVY5S5QMikpGjWWwiZ8PoNhgRxwqqMH3ooCsYxcf3Wcn/zgd3VD9lsNnz2wx/hsuJfOf0/pboqS4IH5gnjLFZLe6OVw+gS7TKXF89wpuDbb9/z/uaOkMUdGKaetixZF5kUPdF7YlRkDCihBoOkJGkeb+5HrsgiV1Ri9skLskugaRL3js/CIkySYGS05FFkpI2s61rWrUpR183T5gCk+ri/f6Bta5wrOJ87pvG3NQENPsqW4eH8QFGUuCUzU8RpAnSZpongPfuLPatVKw9AZ6RNSXEJqPXk/Dgfaokxcj7fPN6HT7OwP+v6B3EI/Kf4rVbgMXRk+e1/Avg7/1afIMRAN5xkzddUGANXz55J+ZMyWSnmEJnmnlM3UlcNIXjapmHWmru7jlW5oi1KjLOs1nvmMHNxcYE2mZvba6qq4uWrFwzDwDB2hBCpm4p21S7kHsXNzTccHt7jXMGnn37KNEbW6y0XF5fEGDkchMq7WsvTb7Vac3F5wWotPSAZkg/kEHHaoFMmx8S5H5jnGT/PrFcV49gx+5GmrNiu1jwcZoauZ+h7EhbnKmwhNKCmaiic49mz59zd3VHYgsIUjMNAiqJySznjCvsE23ic/ltr0Fpzc3PDZruhqGqUMYspRzMvIRaME8PomafAaiE2l5sNVhlylrAUs9wwVVHw4vIZSVnpD1xFWa+om1aGZdHjNKg4E4aOpBLd8cCHt9/y8ecfsd1sUZ9f8mx3wbdffsPP/+7PBaSZBQ5TFiWlsbSmoG3WVE1J61bstxW3h3t+/eVXdP2EMi3JzzgFF21FxUwKXvp/BSFCzIq8RJRrwQgv+YgKjbQrWuvF/BSXmypLyIli2ZgIrLQwIhUPwVNUJQ8PD1IROMMw9MzzzI9+9CP2ly/pz4YYPTmLWElpOWSmaaRpGub5SPCJ4BL9OLK7aMldzzRNzMxM48Q4ThRLQvaj5mQaB1arGqUfK8DEfr9nHGdBqAdZ/WptGBZQyaNY6C9NJ7AEjvwHgf/Kb334v6eU+utIO/DFn/pvf+blg+fcSca6IJgCdf2CeZ45Hs+kCKt2Lb3ZJKrAspC1Xr30zForXFFQ1hUhZIwueP78Bcfjgb7vl4ogUVXF04T/9vaGcZQ+2lrNyxev6fuB+7ujPA13G5wraJqazz//jNvbW6ZpYrNds794SdOuqKoK4xYFXcqoAlRM5BBJw4gyju2+IvkJFrFO350Y5x7nDHG9IoeJ4+nIPI/EXLB/80JQZccz2+2W4D2ng8Ajh26iOw3kJFZmVWeGYZA9f0x0XYfWIkRarzdPA1HvvVhqU6IsqyefQYwJcqCqGlarimfPX7BqV7IKTBEfI4fzmVVdCx23qqmuntENgWkMrLBkJevIYYlvd9ZQFZZ5CvgwMZ0O3H34QPSe3W5Hudvy+vkrfvq7v88vfv4LvvjVr4SW3KwkPTkrSlfhTElVrtiuHVeXLe8+vOfu7kG4gcoS40RdFlysa2zsSYTFQaqIPhNiJmNBJZSWp7U2QvDVgFMatfw+I0/7RJbDYimhjdEYJYyGtl2hlMiuh354wr5Zu2L2E9oobm4/sFt/xDTPgGK1qpn9wFdfvUcpxYsXryiKAteUlFXF4XzL7D0XV5d0p06AsePE/d09bdMuSHWHXVucM4zTQNedGIYO6ywXlxcUhaPvZEPlXKZwivNZDibRjkhux3ddf9HcgQ64/FMf+0//2/08KcoKKATPZtMQlp1214nqSQZeK1arNVfPrjgeD+L80gqrLHUlYMfClnifiD5SNfWTBbVZ0nOPxwdevHhB2zZ89dVXfPjwjqurK+Z54nQ6sl2vePXqI07Hgb4fKFxNjPFpIGOtJkb5nBeXlzgJHRRnmA8kL5N7kxVhnBjPHSpnysKSU0BbScitqoKqtqTgUZsNhdUio51HdLFmu90JZDILM9GWhrvbO3a7HefhzOl8Yrfd4ayFJXUmhsg4joJp14Znz56xXst0uGkajBUOo7WWpmmpq4ZUSRthjGW7kfJ/tdmSY2K6eY+tK6It6PqOunQin21asAXTfGYOCYzEwocUcc7KbGW7hjlxe90zjDNKWbEvA+vNhlXzivVqy09/52d8+df/Hdzc3jLdy0GktcElReFqjClR2jKHzP39gdvbB6bRo23FOM14H6lbcXTmuUOVDokfUiQCMSliVqRFBfQba7J0APkxiXipcnLOpMVbEEIkk3BOUVSFiJqMZbVakXOS3j2MKC1eFDtrDod7/s7f+dv89MeJy6tLnNP4oBjnGW0Sxliur99RVStevXpF4SreXX/N7e0tn3/+AzJnCldQlTUpCqTliZfpRF/S92ceDvdSYSwq2O1mD2hhKE6eVbunKKQ1LopCFI/fe7KQgnkemeeJX3/5xYJHFinsRx99yjgEunOP1ha0TL5zSDRlJdbhriPGJIgoWKLIT9zfvUWpgHMF1i3ru8XJpbXsX7331HXN3e0df/Nv/Gv8/u//VVat9Pfn85nVasMvf/lLxnHgcHhgf7Fld3mx4LGU+OuVCJ5SyqiUJSdv8qQFzxW9Z/aDePajlJ2Pqx0UVGVN+Vz0+017wTSLbr4tWzncCmkBrLXkAE3ZsN1u0UYxF/LEcbZgniWTLsa0xI0JUHW324ksetEbVHWNURZtHNYUoAx1uxH0eM483N9z+uZrLl+9pL68kihwazCuYBko4KyDogJt8VNAG8PF5SVq7mEcMFHTHwMPxxMpeMZ+YOh71i8Lis0GPwdW6zWffPIZz5+/BK1FcwAUGCrlcK7EmILDwy1f3X3Fh+tbtC0Q/cCItVq4/XGmsIJHC1nckwm9fF8yKUqPb5bkIZUl2stoLa2BfsSXSf8sv+CJglxQotBPlVNZOpoGMiVdf6Lrztzf37LZbNjuNhxPD7x4eUkmkLIHEpeXe6qq4ttv35Nz4HB4ICYRnOmomecJHyN1U9PULfvtXh4sUVKVh2EgYRkneR9pLcEqXXdms97SNC3TmHi4P9I2e4qiZLvb0jQNDw8P33/ZsEJWaj7MnD6cFtljou/GBQ/unnTsd7f3zOPIN199zasXL9isBKIxzRPH7kyzXhFz5nA60Q93rFYV3gu4UynFr774AoUAQ/th4P5wz+QneVJG+NUff8HnP/gB4zDztntLVTb80R//CZeXV7Rty09+5/f4/Ce/T4yOsOCo9KPizGpUSsRZDEACv7B4PzJNnhi93Pgohm4gpbiUp5qmqRd8tqMo9CIUyaxXG3JOrJ+vefv2LUop1putbFBCYEweYwrqZo0rAttdj1aatm1pV+sn5Fhd16w2G1xVkROE2cshUNZI0kZBigNhmujPJx4OBzbPnrG2lqoopMy2lpwHwas5hykLmHqGTtqn1XrLfHpgaFp8M1NVGo2nO018/dWXvH9/w4sf/R5aW07nI+fjSQ6pzZbVekU/jnTnjgKN9ZlSGWpXcNcP/OrbazqfKNctYcpoldjWFReritpNtEVJtBbv80LREdtxTkm2A4+moSz6B200hStR1qGtXYAmkv5TFqU4TXUg5YCxhk21lpWsNrTtihgDh+M9Shnads3xdKRqaj797HP8BInENA6YRT8Qoud4krWtK0o+3HwghMRm19K0Ig0uC0keCt6z2+6Yp5nDwwNhDjw8PGB7gykUTbumXa3JOdG2DUVV0qxa+uHA3cMd2paSh7C/XB4ApRC8v+P6XhwCKWV+8YdfoJT0suNwpiprjGm4u+0EuOjFG75pdnz1xTVffPEtt+/PXD27YLdfs9o0qDny4fYdp/MBbTTrpiVOibZd01Zrgs9UruGbb7+lKDyb7UtQiZuba5p6yz/8e/8efBgoUFSrinVrGYaZH3z2Kbv9R/yVv/KP8vEnPyP5NedChj+F8rg0oZnIeiaoCdOUFEahBo+KGU9FsHuUFVS6mie0KUhpJkVxxb19P6G14eWbShRqZYVKkRxntII0zTRFgbWKOc4iE9aO2W+XoV/DeDyQVU/IiaK5wpYVZSMmobauRVMwebKXAdocZlm9lgV5GiDMqDDi6NBNQ7lak+bApqxwCXKEMSqyK9A24ownvv8FZu7Rdcs0ZFK9odpfMHQPlPbEtumIWXP99lv+4A++5Wf/zoZ2isSHEzdffs2vfv4LSm34D/z7/v38+stf86//a/8aRcys65JGG4opcThMfJlKhssdZtXS9W/ZlyM/cp4fWsWbukBFD9FQ5Cz5E0h2ITngtJbUnqxR2WIw6OyYR4MtM9rN6Dig44ROidKUlEXDEE6cxiN2grWuaIsd4ziQ7hTtaocyW5xq2OxbxqwIeRRMXtR88c1XyxO6oV6VnM8dp1OHUoaVK9hcrCUiLUwMpyM6qSUPMYHVEutmFVlncHDx4pLruzON3dFuVxSFxjpFyp4pB2oTmFTPmx8+l8/rItmseXvT052lGvyu63txCPjg8V5wYk3TsNvun6bcbdtye3vLMEzUdcPsPavNip/8zk85HO/o+o7L53vR1EcrsVLa4pzsfp21NE2DNoZp6MnZP4EaVpuWcRxkjx4z1jgaW1JVDnQk5Uy7Sny2vuSjj3/CxeUn2EJ08qWGFAMmByxRDEjGoRYKjLIGXZXoAHUJbiV3kU6RHD0pNIRpQjQsinqaMLagWTlimMnZUxSyIsspMo6yrqyMISCMf1vWTMGJI9EV7HZGKh4Qj0Up/804KwfKOBGmER8jpigoqkoqoBCIswwujdGs24bRZ1xVLwRiJyXygrZKM6R5lMzAWSo4a0pwlqosCYW0LlVdC+i0jyhtGBdf/+nhnrfffMNXX37FzYdryrrk6uoKW1hu3r/j9OEWE+V18VEgM8F70mxg1JTGsLY1m1XFbrvFGJk7xKQWLYAi+kAI8urGRRkKsIAFF04hT6YhAZdqqcaWP6a1oSxKckpc7PcoHPcPdyJAQtOsVriipd3UnLt7zsMk6cE+4ecZ1lvqqsaHCPTsdjvadi2T/JQpC4mWGzpHCOHJ9feo+fBelIJlWdK2LZtdJCkDRIpSUxSaaY50nef29oZV2wKKrhslcCSOXF+/pa4bSWD6jut7cQgoFC9evMB7z/NnL1ivxR55Wqal1ha8fLknxUyOcHV1Kbz8psCVVuAbhyPj1JOV5tnzF4vLsMD7eaHRziQydV1xcfUZ0yRBIGSRkpyPZ1b1mhg9D8cTwzhSVhVNs+bi6iVXL9+gdEFaJKRlVuQ8ouIEMZBVAqx4CbQCY1DOQMgYPEZHiDPZS2CKwkEolt5TU80e4xzWZaZTT0ozxq4wpiAF8atX1QpTt0Q02VhMVRNiWijLGm02bLdXQELl8PQmzynhhxEVZjlcF559HDJFU0ns17IG1DlTWMNud0FRN9I/WyECZzLaWlQWqzEhPTELjULoSkZjjRB8H8VKIQSCjpz7gRgz54c7bq5vuLtdPBL7Lc16jSkcz58/J5576CdImaRBOUVTOawrUNqAMdRasW4F4BnDuNibgaiBJWwk/ptBJfwpK7HAS3+jF3hEeeWUxRzV7FB5hJSJybNerZjnzN3drZi3rObU3XN9fU1RKS73l5weOi62O3a7C+Zp4nzqSN6z3q6I3uNcwWq9FkNXnCjslnGcmGcxuymlF0q0DB1lPX3EFg2rpmT2IzHM9N4zzQMf3r8np8Tz5y/YbHbUVUHfPbBaWZ4/3/Dhw3vevT995/33vTgEANpm9SR5HYZxsbnWYgAxBmsLtNPCzJ9mRj+w2W0oqxJ0ptUtw1Az+5mqrshK4cqGjDz1i0pTlOIlKIuC29trisKw2W6YphFjDLowzKPn7nDi7u6B3e6S3eXH7C5eoVxNmgXQaZ0GeogdeZ6IkxB8c1WhVxsoapQuycmSjIJ8QtETolBunU6CHDJCkdEaSm0kGCF2ZH8gzBOTCjhXk3NBxqFci6p28tRVFqzFFQGjJxlwGYFbEj1pzqTgySmQYhDjSwqYBCElIpmyrpYM+xEVPc5o+Tt+pm62KFOIC89Eye6LUZxqj9FgGlmxGdDLiu0xITmmSEwZyQFVkkGIJqEYhwFyXlSVhu1mQ2UNp8OAQhR8QUk6dTZgC8uqKQmVIy6z/E1ZsmlqUhRqsXYanYS9J7Fdwn1ISeLRtVaoxaOfUxTkmFoqByQVKMWI97JW04MgvwoDOSYeHh5EPKVLuvOZaQrcxYgrLdpm7u/u8EEUqxebC+Zx4rCsZq0tuNjv2ay3nM+dzCxiIubAOA54L8gxrS2gF+9AKSKhxavQNA1oRdcfGIYObcD7kdu7az5cv18qwQ23t2KTL4oKY2Q2ZGzgw4d333nvfS8OAdm3Fmw2G25v7+j7ge12J44sH9FKM43SB7dt4tyfmeYJly3deGa92XBxuadqGoZxYBhGYgjMPjIvhp+qXPTvOdIPHeM0cHsjq7XbuxusseSklkmyoWr3XL34hI8+/gmb3UtyNktgpAIVIJxhOpOHkTDOjFG2AkW1wlYl2dbEaNEoAah6US2qLKGYaRqZxx5FWgxASpJw5num8z3zFAjTiCs2uGJLwGCiwlCAXQPCStAWTLFMtKMnzSJDTiFAkh9KSfJSmGaOhxNzijTrlWgRzmemvqMpjUzeUySECa0FkU5eMGpaQ4pLbx1lGKrzUxKRIeFTeMogSFkAGGiDLRQqW1brtQhxcqaw4uev65LVgtd+uLvjdDxIYk7OKKMIJpJVojQak0UeXil4vlmzaWvm6YHKKTAWhQFEOWeMgEZT8Mv0Xy2R42lpCRbtgFYolXmMg5uniXme0ZNFqwjWolLgeDxQlTV9d0uIcHn5gpgS2ii8H8kxEmdPtoa6qrh/eFgOjoLd9mIxhEWqsuLu/oHz6cx6vQZk6Lff7ynLkt1u9+QDkZbFME0Ciy2bkq4/8/Bwx/5ii3WKeR7IOQixSmU+XL/l/ftr3rx5w+R7mnPNxdWemPvvvP++F4eAWCgt3gdCiEzTzOl0JifxC2QNIQjDbhg7EiKT9FEwWnPwT+EhAp7wWOcYp8DxeGKaRtqmpyzsgtaWWKZ+6DgcDlxf37C/2HP3cEThqJsdH3/2CT/9ye/x4s1naFeRU0QXCiwkP8D5gXQ64rsJ77PYX5cSWCkt+v+shVuT8pJAO6DCTMYzdUe644EwDTK/KArR0aczce7wPoFKxODQdktEkY0DU4KtQBlyCJBHuUFzIvgJPw6QAoYoz12tgMw0Dtx+eM/xdGS121E2LRkxNeU4Y6pWyuIcIcyitM8KlOTk5UWma62VTAed0DlJ9SJSSVQKaOTGYuH+pZzR1mER1aNBeATR+6WNMaiUOB+P3N/dcjoeCfOMTQntLLhENmLM0lEOxv2q4bJt0DnSDz3WWFQCkiItWQJyOqkFBiMdmoBj5LzUSkCmjwAZRSJGT4hyaBSFZC3WZUEIgeH8IGlAc+b5i9esN62YrQrL7C05XzHN9ZJpkVivWiFPT5NkU4bI2I9YW+CnmbIsWTUt2mS67kgIYQmRlQNrnucnMVKMkdPpxIYWYzJlZTBazEPPn19xPgtQ5nC4o64LmqbkdLrneJS1pbWJ9ab67vvvL/He/vu+HnkCh8OBqlrCOu/v6fuBVbsiJVitVmitxC6MX2S/4uI6njuGacQWBcZIgk7McD53PDzcMw5nyspRlZYUA+PQ09Qtzhbc3x8ga4LPHA93tKsLPvvhR/z4p3+VF68+w5br5T0V0coTfI8fzqSHe4aHB/zg0aak2taUdY12xVISi+c+R8hxIs8zcRpJU09Ms3yOsaM/HrBGodbCtSd2+GlgnBJJZUxVgB1x7QpTlEugSiIBUSV09iJyCTPz2JHChDNqcTMmSJ6hO3P94QO3N9cUZcFqs6FoWnwQzmBd1xRliQoT0U/4scOmR9ugzCzk3lLSk6Nh6bUTiRQ8wScZlCp5CgtRJ+GDtFCla3j15iPIMPU93enEPIyonDmfjvz6iy+4ublmmkaIAbfcwNkqkkqYLEYeoxXP1y21UQydQEAjggvPXvgHaE1eDD0sPb5Wi1wamVlYa55+CB/gN+KhwlrKoiAbcWA6a5iAaZ6o6zV1XdH3J1KCslxRFgWb9YqHh5l5munOHau2Zb3ePFWxc5rRaM6nDmsc26tnOGvohhOFKxb/gWaOM8ZYSZhajGOFK6ARkVpVW6pyu8w5Ivvthh//8IfCt9QaV5TcG4WfJ9abNVYrrj+85/Li8jvvv+/FITB7z+FwoG0FsJFS4ng4EkNcGIBCBbbWMgWJVipKS9vWlFVNP0+C7D6dqOuW/f6CcZy4vbllnkZ8GEnZM46R7nwixcjUzqxXO2KEomiZfaJdrfnBD3/E7/zu7/Pi9Q8wuiUltZzO8iYf+xP96UDses6njjgG2tbR2oKirFBWE3NYpMMiYldpQIWA9jNxGohhRPuZSoNyBmMUFUni1CZPfxoZ54QuHc5KUGZTlujCgo7E2BOyIpOxWRJ+wzQwzR1V5bAqQQqkONEfDxwf7pjGnu12w/bykvV2h0bjJ6Ho1k2LLSzZD4RxYOpO6MWDnpbg0LywwZNMzmSgmtJi2fXkyT8O3nnMDEQptDWY5Njutrx+/Vqeag8HTscj8zSRU+J4ODD/8k+4Py56fBDtfs74HJn9RAoThS5pS8vlqiGHmaE/oxVECRUiR0mmVoInlnmAWiCiRgjUeqFUay0EX2cN2hnSQgVKMYgCMwaGqWOMicLJatdqQ1FaTt2BaQlpub0bxTg2S0UXo7wuNzc3T0YhgegaonWCn0dxPp94eHjgeHqgqssnwrZbUGplWTEMoyQgZ0Q6bONT69J15ycY62azFbtxWTIMA2VZ8PrVK/b7SyFZn8+i2vyO63txCIDYXV++fIFzYs5Yb9YUhcRzF4WAHZyzPLu6JOaBzXZHVTfSOpwtSmXpL9cNOUfO3Yn7+3uauiLGSN+PGJs4HO5Yr9ZM88R8e49WJev1mrIq+ewnn/BXfvbXefbiE4wtSUmD0uIuQyi4OcnPXjlyUeOspl7vKdq1yIKjR+cZYoaQwEc0soO3KUKMEIJ46xXUTQ050Z8lh87PEqrpyhVlvaPZXtJcXNHuNminSHnAx0gEjDWS8BsnpvFMzl78BGEmpJF57DgcruUA2GzYXLzAVS1aO1KSexm0WGKXO3gaOsbziXLJwIsxoo1eqhuBcubHyT+Pk3e1RKU/4k4UxjmKqqZdReJU8OL1a168eM7hoef+9pbTwwPzMBC953w6cR47puCfnsYg4qvR94xTj4qBQpVsqorSCBtinnrq2jLNgQKDUTK3QQl8A6UlOTmEpzWgNUrWgDk9IbyMNWRjlrZzZp5H7GTpxiMmJnwhevyyKNFW0fUnQhCrcN8LmWkcez76+CPOpxNKweF4IMYo7EHjaBphOxhjKArH+XxiGCemaUBpSVWSSrgS5P3ChBiG4YkYrRScz0dubm44Hk9sNutlAPoIlClklVyVNM2KslhR1wZnBXX+Xdf34hBwzvHTn/54QXhlbm5K9vtXOFcwDGKr7LuedtWwv2zph3vatlogDIGmKXn18jntak1VN7Ju8RPrtmW7XfNwCPTDyKptmaeGsiqpqxW3NyfKouKTT3/Eqzev+fgHr7m8fIVxpayWQHLhlVoINSI1Lcsau7+iWe0otaMqakwh68McJnKOpBhQPmBSQpHQcaZYPO0iEpEeOvqZaZEkC0NwxcXVK5rNBapsqS+fU2y2mFUDOhFDT4iepDJEhUkjfu45HW/Zrlty6EgxEOaevjsQ44wrNPWqpmxbtKmIIOtWZTC2lBI/eVSMDOczQ39mk5J8/eS/h8zzmAOoWG4ypD3IC4Q1L3ZdYwvadk2Mltgp3rx5TVWVvD2/53w6cj4emaeR4GdiCqQgsV+i7F0CG5RhmEdiipTO0haWVVXi55HT+UjKHlNoTEg4Y0E/IrfECeicw8f0tAVQKj/pA0QZvHwcyavMS9UwTwN2tiiVqCuHtlF8/zni48zkR7yPdNcdpSuo64a1W2O0aFuGcWC1Wi0Pm3m5yS1Ky5AvLMCTqiopSkeMAWsdp9OZw0EAsykJD0Ki7OUmH8eBOUj0mkJjbcnF/oIQBHt3eDjhQ6QqG3JW9J1H68TQR+r6ex5IChCTnNZaKy4udjRNLfSZBe1sjPyAKHDGsYc8UjcNlxc7ERvFhCJyebmjaUpeXD4Tt52NzL7g+ctLmtYx+8R28wxj1+z3b/jZX/uHefH8BfXaYrQEdyqlFwr0kl2XRVfijME2Ldk2KJ8wEXRM5ODJKZOI5BzIYQQ/w1JOq5whBvzY4/sOnTykgJ9HUgxUZUVRltTtJUW1xZQ15XZPsdlBUYDVpDgT44wSpYA8yuPMNJ6YxhNu3xLmAZUi09gzjsNySArPMISEs3LzJhTGlZSPT00JP8T7WUr9ha68zBVlr27MsnXIi7DG4kyBV4Y5pqdVoLEFZVXTrtcoXXKee6xSTN2JoTsz9QNj35NClNARlYkBsBJH/lhPpJSYxgkFVEVB5Rx14fBhJCRPUomQhGsQo5LoMKUWiIiAPNUkAza9BNLID9BK5hopgk6PQaeQohdnaappVxWbTUU2AWM1wzASciQpWe/1i39hGAfW6zV939N1PVOYefnyJdvdjrffviOmhLKasihxRYH3EXxYtgNwe3tL08h6vO8HrHVM0yMWTAbN0+Tx6UxVay52F0yTZEUGn3l29ZL7BzEVla5aKp4CZ2sODwcOhyNx89333vfkEMg8PNwtp55it9twPsuTX+i8BdvtejH8nFGLOCVEDzlSVwUp9czzRDYyiGsqx7Ze0587wn7P7B3rtiZxgdIFq/aSF69WvHn9E958+mMKazFqImfR9mPyUpqmRUAi+3GNRhlLtlIeEwJ5uYFQkPMMOUCcUHGWJ2myYot+uKU/3GPw1IW89NoayroSpZ8xzNHSDSOFdTSV3PzKGaIficETc0BbeULnFEhhYu7O1JXDGUXwmRgC4zhJqo4pKMqGmDW+GzDFmmwtGYV2VsQ/OsgAcxmiFc6hVZYUn0VwpBMovSC6I6BBZYvTUu4OKRP9TIgZW5TUTUtdt2gKnBm5v/nA8e6GOIndeB7HpwNFYB7LU3l5arqkCDEwjCOFc6yKinp5cgY/Y+uSOXoSihQVac54LYKrx6+DxWUnDxGN0fppFqCVxK3JWaeegCMyD5DqabttKWtFNwt0VhdCi4o50Q09zUpi2vtuoM3iWZFPl+nHgXKq6MdevBrRkya42F+QEhwejozzRF3V4vI09okA/agSHMfxSTLvvWfyA9OcKIuKohDL/DxFxtFjdEFZZEKYmeaJqtRs1jtUNvT9xPn8vV8RWpqmRCmh2mhrBfjR1MJ7X/hwq1XDPEW6bpAUnRQZh46mXaFIlIXEkjsLMc7MkxwWF7sNd4eRmDx1XVFUK5pmy7Nnn/Py5Y8o6xVpjgiBRsl+Oway+k0gRU5edtdZoSJkG6Qfn2fUHNFKXIUxzygVUETUIgryMfJwuOfh4RZLpG4Fr52CxlbFAkJV9ONIPwqDwFaaECfy3GMKRxikBdBGvPBqKc+nacKPI1cvXqCVwRqY+4mhn9HKSclZrZlCIsyBdiHrqvxYFksmYopZHJFJuHhW3DdSMCu1RG2DRvPUtmcheBhlxfI7jwQfsY8JPNZAoSiLgtPhgeF0IKfAPI74aYIFhS4jgGX1mIWdqKOEyHbnjqqo2Kw3tIXB2IzGULQVadbi5IxLUaQCTmnZnKS0FGFCWzKLU/A3lYDs4PWSRWiNUJMfSx9rwFqF9wPn7oiNIrG2sv2lnwYu6hpbOtwyc0g5YwuHnxRfff2VQGG16D9iSlgn/zZjDcYZzueOwhU0TfN0w4N8T/XSzlhr8f43aPxzd8/Qe1btht32Eq0ttzcH6romBXkIKBxVWaPQrNdb1qcz7z98+O777+/nJlVK/S+A/xjwIef8e8vHLpDcgc8QeMg/lXO+VzId+h8B/1GgB/5zOee/+ed+fq1o2lqm3EHQzJeXe+CRntrRnTv2F3vWq4YUe4ZhIEYvWXVKJuUSihNkp82SL4elqSqGqZCQSmTg4sqSqxcvqVcbcjZPJXGKkZAC2IxywnQjpeWA0hA1cZ4YpzvG7gyjx2XZ8yssIc1kHbFL/59jpOsHzsMZ7TSbzZq6sMR5JC+GKayl63qO44BzhnZToRxMoafWK3IS6o+rhDbEMtRCKaZuJPpE3axhlqATHzJj77m4uMRojXEtOUyiTixKtCvRGVJOchPI8Yf3sjIsikKEVdEvwzUnVRGip09ZCdA1RaKXNGNblKg54OOM9xLPFWJEG0vTVJw8zNNAjhJhnnMii5yQlDNZS0S6UpCN9Mnd+UwIkf16Tds0WBUJcSSqjCodlbOEfsZPkdo4ks7MISxrTNlsaK2f/u0ZYS/kLAeDXjImH8NX5BCQV2OeR7oukc0ISta1KXrxP6REIjGHmSJ6XFkwec/Qn6RNcoJ+M85ycXVF3/UYZ6nbhmEaKVyBcZY5ePqhh5Tp+55Xr15hjOH+/h4hXFfs9ntSStzf3ZGyJuWZeZI0ZWtLSlfjp0xZ1JAFcSbMikYkyMhQsa6+Wyfw3SmFf+/1vwT+w3/qY/8N4F/KOf8Y+JeW34MwB3+8/PgvI+DRP/d6jGs2VnM8PTD7CWuFoTaOPX0vCr/D4YHDwwOHh3uhrcZIDJ7z6QAp4JwBEsfDA935SGEN0c8oEptVuwBAJenn4vKS9XpHygofMygHWZKMU4g8puHG5AlxJqWwlIswdgO377/hdP+B5EesRXb9v8WmCznjU6KfZsZ5oqgL9s8uaNYrsoGowFYlpq4YY+Q4DCRjqNY12EjIE6YAZRUhSN6AQCcVcY7kAOiC8dTLUz0vqbrZEJMmRmjqNcpUKF2icJiiEuiGMkhPsfzaWKSzCRhtKMpSpMbjSJjF9IOS6kMBOWWCj0QfZcCYJevRlSUpwzALLdqHQFUWtG1DUxVMfUeOUm1opZYQ0ix6/hRFSZniU3TWvFhqnz27WijPgcnPhBzBGJrNGm0sfvIQxQAUfBAl32OiEPKETik/MftSijK/WeaPPAaToJ5gnuPYczwdSFlmKplEPw48HB9AK66eXy0y3n5pPSQ0pGqkRbi4umR/ccFqtaJdryWlSmuGceTcd/gQyCRCDByPR87nMyCGudVqhbVWYKZWUqK22y3OFaxXGy72l1zsLtEYxnGmbdZYU2K1o3AlKYjHxs8Td3e3wh9Mf0Ercc75/66U+uxPffifBP6x5df/DPAvA//15eP/qyy1zf9bKbX7U9zBf/PnT3A6j7iioB9Ff308H7HGieJqnCmqin7oGIbI/d0dIQb2F1tyjmSVKXKkMCJQmYOXia9LJD8RvcKWFcl7snK8efMjXr78BGdKxs6TzQyFgtCTZ9HhG6yUyBlyEK03CtI8MY89eE1drES/4ERuqwgUOsiQMwb8PBHijK0cla2etPnjHCiKCls4MgrvE8En2tUKra2sQ2tLUVqm/sQce8KcyLEmeOkRy6pCJRi7Ufh+HonZ0gblGkzZoosGbSK4kqJYQBsxkuPyNMyi/X8U1MzzhNFgNUzHW4oUcfUKlZslwAVA5MA+BAqTyHjU8gR3Swsw9xOzj4SsicriyooiZbrjNTrV2HGkzgqvBJeelEIbi1ZGBpcGkk2s9jW7q4LCavxRAkoyCZLCKEPlamY30uWOoBPayoFgY3wKXSHJ10vOpKwJGXQ2RLVIonOWuULWaJTceFpeE43BqAqna9FHGQ0W1pUQgZPy4ltpRKSmcs16XaNNzbrZ0lQN4zCQY2T0ZxSIzDcpSThqV9SuJCyBJjFFSl2w3e3ouw5tNEPfLcnYEnbarnbkqDGmxE9yYysNp+M9IcycTqK6dcYx+Zn3H77BWQkz+QsdAt9xvfitG/sd8GL59Rvgq9/6c18vH/vuQyArplHx4fqOTEJbRcieh7szd7cPeB/Yuy0+BZyrwK0xLjBG0UtXpWOMM/3DJJPXLDHQ89BhciTMAR8NPtYUqy37y89pmueooClixDCippHYvSXMCV2tMFWNzsXyBll2/zmQ/IBVicvtJxSVRdtISB2JGZs9Jgrw0vuJ5APaWoq6gZgJPpBjImXJuU8hEbzHDwGXLZUqSD6hy5qyaAjTTNcdSbqgadcUNuJDIMRApUuin0hzor1YkaMmRA3aoqsVtt2RygoVA8rZBcYRfoPU8kGefCh0VqiYCNOE0ZkcR/oPX+JyFAR3mMm6FKuuUSRj8CrgTCD6M9aI2i5HT2ENU0zMAeZseRg9HjAuM5zfoXpN1XfklJi1Iy1yfkWB0Zay1JgioWrLs6sdznQc768Jflhcio65T5iocEGgI31lmMol/m02JDJaI0yHlBefvlQzSSmCEucpRqONIWqLUWZRREoWQV23bDZbGS4Gx/P9jt6cmcuBdVvx7duviFPPxcUFl/uWECJGWYyaaYorvPaooAlTAJ15++3XrDZbnC3wPrFRO5wtiClSVI7G1BirSQgCrlyi3WIMdMNZDtnSsd5e4WdFCgo/95JnkCf68R5nNOMwkqPh+sMHsvEMw4FRO5z9S5YN55yzUuq7Dct/xvXbuQNXF1t8iNwfbzFWnvAxRm5ubnh4OLHZbBnHgRCFs37/cBINdTCs1pJk3J+HBbLp2G4vUFh8CJgM3ge63lNsdvzgBz9jvb7EmhIVM9omCBO+65jP94RsKF0p7jmr0BiSySgiaepJKVA2Na7ZonIg5Q6LvIHy5AnDmbh49pW1GO2Is1QmOSbGYaIuSyEFIwi14/FEYZZE4CUGO8dEPwyMw0S1LlitW4zKzCkQgwcyyc80TbMkMidmHzBG4sWLukZbJ+tJLZPxx/xEEfdIbqI1BWZRAYq+XoJb+vOBarWlSYHsPVEpPEaAHNZgjMSbhxBkqEaWrIjF9TZ7T0hZXv8lAFVmGx5nE6MKQKKqHcTMTFx680xVGarWQR7x5wmVA+SAUm4Bqg5kpTBKCzquLFCFBLJY5zBG/As5LwpBZ2TmuKxGRfoldCHr5O+ljHg/tKZpJOOvbdbYqsBYy7rZkIO8RnVVs11vIScudnuUUlRlSXfuKeqGw0OHc5YUJ1KEqigJPnM+dbx8uaXvT5yOnbgSyVRlhVkYF94HycXM+UkvUBYFZVVSVBVKOVIMWFvSthZjEnrhHBqlqCrF0AXmyaMrz+XVBSlpEb59x/UXOQTeP5b5SqlXwOP48Rvg49/6cx8tH/t7rt/OHfjs4xf59uaWsgHvPXd3dzw8dE8QCmMMIUq+/M31LR8+3FHXjq1qaNuaafAcjmcUmtV2Q1Wu8HPk1A2EaWZKimqz56PnH/Hy5cdyKoa0aPtnwtDTH+7x0xllG0yYcWFGuWKRvkoIxDiNspJsanRhiZMk3ioCOczEYWDuexkYKkW1XqNSYhw7WenMnnkc2bYrwjwTk8SfT9NE0VpSihROCLMxZoL3GKVo6xpbVMRxJnu5KUmR5D2rVYu1hmU5JY7BolhItfKkYVH2ycz20TefpTdexDkqS6aBigUxytfl/cRSLMtMREu7JXODijAMIqpKEh1utGUez3Tn7in5SDwEkteQyITsaS9qZhUhKuqyRo+ePI8Y4zFq2ecnRYgDfh4XY5Z60iuEGBZpssKVBVXd4JzBoHAuYjRoY0nL12yMhKnmJW2AZZCMyigjDsmsWPrvmqZd0TQrmXPYErRkCUoRpRb69Za+77FW0oOqtUjMy7Ihlo66qZimAbJHq4Km3lBWFUZXaDWhlRNLeJrZbNZorRl6SRY2Wi/gm5kUI0VRUlc17WqND3Duhv8fc38Wa2ma7vlBv3f6pjXuMSIyIqfKqjqzj3vAlhhsgy9AlgAZIWRuAMMFluw7JKQWSCAsSyCmGyQuEFwgAZYlS2AhLjDiwjR22+52d5+xhqzMrMzImPa0pm98Jy6eb0eV8ck+h3P6oFpSKiN2rFg79t7rHZ7n+f9/fwyR9WrFclEyjnsWzYqcIqVbUthI09TcHl9SVTWLZsPu4S8ngejfAP7rwP94/v//5Zc+/i8ppf5V4B8H9v+gfgBIPTqMPdppFkvJZn88YapKE4LHOnkDn07tnMba0NSVjAy7I8djy2q1wehCPnbqCYcjh4cduIp/5JPf5ONPf42yWEBSci0OgTh19O0DXXuPzgGtvAh/op9VayJM92FknAYKlVFWiSkoTuTkSWki9D2+axn7lvZ4nBNtasYpEBNkYxm7VuSqZJTKdH3L6XiQN7DVFGVB4ax0n2PCaSMOQ+dIg8STh5iJkyf0PWGaZNSYAgY3C6vAKE1TNxLpnaIs/CybaI4B7SR+HcQzL5E9YvxJSs+GIWEipuhFL5BlcYYYUAYKV5J9QTSCfDdaU5cV76aJ/X6PUhpXlkxTIOSECgFbZDCwvloQbKJKGkxFOmVSrwipJ/hIe5hojxPWZPI0Se2emOXLsnkZKwvXOkvd1KQcMVmRrBUdiTZom/GTRI9lRCGplJbsAYXkVkotMluPRdWntaWpF2hrOeyPjN5T1ZIfqJVlmiJF0eBsRd97QvDUdUYrsTNvNmcsmppu6NjvQSnL5cUTmsWSlOH8rGCzPsPYgr59kLJMiQXZzCNNHgnWKVGVmnEYKcsFSjuaasE0RPpuJEWPNomqXsg0xUu/5+Lymtbf40NktVqx352+c/39WUeE/yekCXiplHoJ/A/mxf+vKaX+W8DPgf/K/PT/GzIe/BwZEf7zf9rra6N59uwpP//mJ6w3TynLknGMxPniNk0jSlse5/hPrp+y2iyoKs0wtvjgWSwgRei6gbLQTFOk60dOw8QHVy/45LPfYL15AtmiAW0SmR4/tYzDjmk8oLPCoCnmiYIyzC45CUuN2aMLQ0qeOA0olQRDPmaZeYdAPweNFK4gTp5x8hTVAnxk6gdWq6XcQEKadeFa9P5aUy8X5AzTOEkasDYi0BlGwugJs0Emh4hvW9HrWyumFOcwtsTHSB4nqXlRaG2lSa4UkVEWEBljpNQxOb33MeSsmHwg+IDSmRgnxu6E0xW5WJLnrn2KeSY6VWg9yOefJwXBB7nSzuiy4dSSFfhZ6pwdKJcoVprSlEwBbEjoEEhDT3c6EvwJayKF01glw1bBpYuGQ2lZ2CnLFMCW4rh7RHCnmKQEMm5ON1bvFzqzKMdYIw7sLO8yjCFrTVaayQeOp5bFakkMYggypqAqS1L2tKcDerVgtTpnHCQl+j7t8VOi7ycuNwWFK7HOSZDI2FMWNVXRgDKsVyVFIcAcpxPTKFkDdqYtT9M0U6nlBlU4xziNDEOPNklGjECKkWmcGKcDxiQOhyNhyoQJDseWru+YpsA4TlT1XzCVOOf8X/2OP/qn/4TnZuBf/LO87i/9JYzVuEKy3e4fHni4P3B58RTvw1zTRZSBq+srtptLYhoJMUpNXImgYvewn8dVcqVNGM4un/Ibv/1XePL8E4gGXTboGICJnERjH6YTYWqJE7iYqRZr8dUnT8qQkpfmoFOYwjCFkcl76sKglTTEHmfeYhopqKsKlTImZgqlGbqe5AOFLYghvnfQLZbSadYzCzCME3Ge1xuliT4yhBPGFihjST5Qu5I8SbS3cyV9P83kYnHB+flzmpgAhS4KFAZt+lkBKSXCLzP29ExwSoj3qagKlIb2dKRxC8pyKeNPHwhEmqpBKUvG4iMoH+YutqKqatQsTxYzz9yHUCK0aceWaBOurMhDQNlAyiMxjZyOD+TYsVnVpMnj1S8mGczTCWct2om6zoco9bwRO7HWWizQIIusKAnBz5J0wZprI6M3bbWExjxqhpVBGccUMvcPe4qqZr1aM3gxsfkgITfTGEgLJS7UAG07MQ6S8TgOgbfDa8GPb7Zst1umSaYrbS+EaVcIEakdRowS9F3X9cIw0IpxmohBbldVKbmTVVWTlaLtelIYKYslZelIKdOPwnZwzlEWBeOQ8GFisVhS18xS5F9xF2HKibc3b7l+co0rHW9u3kr9pg3KZkJKqJSoSsfHn3xIjgW3d0f82KN0KbBLP+EKS1UWPOKiVpstH334KZ/94DeoqiV+UpikyTFD9DNc02NMwNqEjhZjNM4ZtE7kaZCZs5WPZe2whaPzHcpmlE7EYWRsO0I/yNw8w6JpKIqK/tSikiL5wGG3lzFbjIT5RBV+XMIj2QfTOOIHYethrCy4mMjzZmCdoz91XFxfEcIkSTwaQhjJvcYkSNmSgGQiSUsdq4zF2Fkbn9NM/3lcWHoWQ2Wsc1hXklxBYwuU1XTdEdtsWRhDePyBZY3WVkRZs9bAaEfKCmMdRV2LXl5pVps14wzICElGapEAzoIFU1psZdBWNtvoJwqjKa0TQlKa1z4SN46WE/wxK2AaJ6Eru0LgKMbNTUExEVV1RdsldEyoOaZN4IIaZTXGivEILbkKMUOYYbWPO2TXduyPB7qu5dmzpxRFTVXWaCXSXD8JlnwaPdvNGc6UAJzavcTcWcU4ikfkcDoRY2KxWos2IAriPZPmgFyHs2buBTixQsfMYrHEp4mub1Eqo7XYibv+SFkp6kVJJjKNgTzfno0tUEnWT12571x/vxKbgHDxKxbLZg7pSHz6yfdoW09RFHTdCZMUYDAGju0JbTLLqsI6xf5w4ng8cn52SVUXxACXF2esl+d87we/zubsEq0MZdVAhBTSe82/MVAWFtUUqKLBlA3NqkFZSwgDMWWcqygLQ0pO3HsKqqbAxMgwDUxdR/bS+Q4xYYqSjKI7dVS2oEsdh8OR8/Nz2SiQhlVZVOz2O4wxVHXN4XggTxGjDDpm8RKME0obfOhQWnM8tpxvN6K7N4E4WoauJakJVyVM0YAVyW+OiWn0cjVGUpDSewSY0IJyDKRxkDBUpTG2QJsCU8qinvwos23ZWrDGzNdnN7cMFdo4XKUovceV3Wy+kpl71dRwOmGc5Xg8onIiGyPaCi0jvyYuOJ1aDnuFNY6q0OSkMLp4L+QhIUk+RSENvbmhGWMgpCQTEuRGoLIiEUX9WJb0w0B6NIUpM1ugE2BQxoiAyjjKeoFxJZMXutXD/Y7FshYNxThyOByEiuw0fS/Aj9VKEHibzYrT6cTl5SXEwO6w5/7+Hm2uKcqKGCe0kY2ubfcslzVlpfGDlDBaG4mEQ1yVj47HaeYBWFsQUoufTkwTgs+bPNZlzs7XLFcVOwt95zk7X7JYLjl1DxwOBxSZYfhLiiH7h/UIITCMPYdDJCQJEu2HgdOp5+zsgovLS1LqaRaON29foShARawr0CYjm3mm7Y5SY9ua7faCFx9+zNWTZ7iqmUnFI0Y7yDJH9jEIxkkpClfQ9QFbKUzhZmhFhy5KlK2lY58Uh9t3OKcEwtFPcnqPE4uy4HQ6ECO4qpHGlLa4ouJuf8CHyHp7RlEU7A8CUIlZXGIoTdv2TGPEKjULdzzX19e0xzeM00SzWHCYmXW+a9HWMPYtWSty8EzRkzHUtiR7T7YBn6A9HvFhwl2eEWIgxQmXpfmao2A7U5TEWuuclAVzEGaIsNE1wXvCOFCsV0xegKnEyDgFhkk882IE0tRNg9ISGV4VgsYqZ6aDtZbd/R1+mlhvHCmNrBY1H7y4ZhwCu7sddbPAad5nLeZH8IGFPHnqqiLMzc6ubUFpCuvou0EckVpIx4+hqzFFrHWgEtoarJPNLaQonf65b+LKCltWGOtIGfw8DbJOc3a+wfUWYzXWWlbLJa4QX8b9/T0hTNzdycL92c9+xrKRxKD9w55hPPHk+plsVEVNjhMeD0pMZNlrXnz4EXd3N3TdkfXmnGkaJENw6ORmUBTsdnuU7ciMHI5HjJLcyM1mibGBV6+/4vXrt5xtr9Da8vXXN/gw0iyWBJ9YLpffuf5+JTaBnLNAPnYtzknwY1GWbLYVMWXqwnJqJ/ox0vUD45BxhSYmifUexp6qLpkGz83NDU294unTD9ienVHWi9nnnslIt1WXhniamKaRRwXcNAVs0VAvGrL3jNNARs0ClAmMot3tGLqeYrsgK6m76rJkeXHF4UGyEZrljO7qR8pmiUlKTo2ixBjLODcL6wb6bqDrB8qyfE+giTlj5pHY6XSi76Weq8uSh+BnutHpfcTYEIOAUaulZAY2K5arFTFH4dq1R/rTgeXCCSs0CZlGZSQLcQZrKBwpSE1cVDUhJJS2OGtJSc36WqErWSNJyD4ErCuorZLmoDEUdUOzWpPblpRlHGeNnQk/iW6m5RjTY21FcVGz2mxZrtdCONKaPI1oImjYbhsWywUvX32LK53cvOZGsUWBVnN894AuC6qimOX/UnLlnKVJqsRQlHOmKCsKowWMqswv/tMWZYtZKh6xrsZqgXluzpZcPTnndOroh46r6494eHjgeDwSwoS1Al7ZH/a8DnfUdcPu4cCZuuSblyNlWbPZnjH5ga7r+dnU8fyD53ifGMaCxXJJs6wIcWQKE7vDjrbtubp8wursjNOh5f7+DcYG/Nhx7I+sNw3ea7746me8ffuK7dkZ/dDyzcs31LVEmG+2Z0xTz49//JPvXH+/IptAoqpLxsnLrWAYqcpI06z50Y9+zEcff8BqtZpxyx3b9TXH0463b+/ROnFqTxRFyaJZyzWrXKCVoSqbOYxD3Hxa6Vl4okhR5rH1Yk30mhwVhV1RrFaEydMPI8VihXGWGDwqKXb3D+QcMHpNCB6jxXjSd3tev3zFkD3nz5+BdQyhw5U1w/7EGCLnV5cUTcN4PJKVRlvHMHmUMiLwsZK/mKKEnpRFwel0wvuJZdOI9LRtCclTWf2e4zcoAZp+sNpye/uA0pazi0umUYCqYZLI82k4o6jNTEcSWpBSWUaWWrwHYT7NjasFgaakWaiS6PxJ4vqzzhFTwMdAXRVUVcmwfyBrTdk0VAsJick541yBVoopT0zDSN/26KwZuoGiNKisKV3FZnPGxeU1Oif293cYldE5sT5b8vGnH3HzcE8CukHKRenlWKxxZDRxDHgg1RWFcXMfIaJmf4LSCuEiznTrqkKnhLJy81GmQBmHdg4TEtaKhFkbRVUXtN2Bu3vRraQI3//B98kklsuG5fISV1hOpyOT7/n25S3aXHBz+5rDYU/MRoJuTyfKsubsbIMPnpQCwzjRtgPWKUKcmKZOyoacKMqCfhzZPTy8pwk3TcnzFx/QdZ7D4cDv/8Hfw4eOuilYb9Z0rWe323F+cSWbXVXTj56H/e4719+vxCaglOLDD19wPD1wc/OW/X5P8HB+bqjrmsNhT1YFxmaKoqQsa97dvOPdzR1lKWq1srDU1ZLt9or16pynTz+kaZaQIXgJ7ZQTL0sDSSdUXWASxFhg3ZJ6swVXEIdWBDHOoXOW9KJxJHovxg5liQjK248Tu5sHdvcHzp5fU6+2dMETjaUuNIfxAVOULDZbsrEkZdCuJCQYfaRqljjrxA4bRbATgwelJE4c6ZkcjjvGoaPvO2onzMUYE9FU7A5Hzs4uyN7z5uU3PHvyAVkb/DAK09APJN+TioIQPTYFrHGza1ba7jGmOcseyOBMScyzUSjLKU6MGCW4bu+lAWWLAlM4fAasQ1uxKmM0BlHXKdLMEOjJMeCKBcx5B217oh966sWCyyfXWKXo+o7SGqqqIOSBs6sr1tsNU/CUucT7wNDLpmJNSVk4dJ676jGhq7nzjzSUT6eTNBRn5FgIER0TdbMgawtFIRuBNhgreoDlqsIVFmsVTW1597Dj1atXrFZbrHF88/WX3N/fi9uvdrS7A5MfMFbz4oNnnJ9fEXzi7ZtbnCtE2ahlZGmMo6pqpnHi9u6O42ng2bMrQhy5f7ijbaX3cHX1lHGM7I4HUXYaQ3toMbahcCXtqccYx3pzhQ8D7ann449/AGqBD4nLqzXWWs7PL/id3/lHvnP9/UpsAimn951QGTNpiqJgGkcWiwXDeOR4HChKhVIN45BYLc4AxTgKQHOxWHJx/oQXzz/l/PwJZ9tLXFGTYyQFYbZhLBBRKqKdIkyJ9tQRRk9R1qiyIvW9aPPrRq6944RaLPD7A6UtUFoJoaeyTIcToRtwxnJ5dsnFtaDJumPPYrnG+oxyJauqxJQV3eQJgCsrhtETQmK5XOPHidOpYxgmlpdnhEnj/cQwTXjvGaaeruswRon0dhpompJh6NCFYex63nzzLeuzC3b7A3fvXrNebxm6jmk4olIk9CdcufwFLIRIjJ5HQFgKMudHKXyUn0fX9XR9QBcL0danPMuFBZElEWcanyJYg9FSk4unW6Pn7jwpMqeQUBiLm1WEU5g47B84tQeaZsn2Yiv8xixJvOfrFW/ePOBDYHO25ebuhrIq8MHT9wN931G4mkW5onIlp7GVkir9IlNAz+pGHx7dhcIa8D7hlAXtpBFqSrQR9WFVaaoClguHM4lTe6RpHE+fXZGzNCy/eflz6dhPI1134utvvmK9XrBer1jVH1JXC9arc75cfktRNMRkePLkKS9fvuKrr37MkyfXuEISlVxREMlszrZk7Rmmln7sGaeR1focZ2vGwXPz+p6bN98Qg6aoVhRFzaJes2hKqvpKSMuq4Mn1Cx52O4q5BA0+Udf1d66/X4lNYBxHfvz5TzieHtjvH/jww0+5fvaU3UNLiJ5Ipu9ODPcntosPKI1hvbpksVjRdgfquqYsKpp6y3Zzxfn2CUWxQCG3hJwjKQdU0OQcsAq5egXP6BOlW9AsRcp7f3ePKUrKWmrulDM2BGHFF6XYnic5SZTSs602sF5taRYrHrqePkSWVcUURoFtrlfYqhHxR9lQFyWnwxFb1KxXZxzTgXEYyEkSlhSZ9nRiGAb6thVFoRZ9+lSWkCN+GvHTiMNTWcvbV684O7/gfLNhaE+slgvC1JH8gDGK6AfIDdba2WMfmMYRZzTOSBxaIqO04NyCT0yDJ/qMq6x00Yki2/UenyK2sCgr8lZTVmQvzH4zC2UKY+bRnZd/v3PUZUkcPTGIZXjoTwz9ie3ZBlctmYYR5RT9NGLqkikn3j3cYQo3ZyhAWRY0dU0KHTpDYSyrZkEfR+lV+ImsZwCSgqauOfU9YRKOgZr9FNMQ0KXBFWK8ykrjioK6LkihpSgdOo/s9vcE7VksK+pqiVKGvpvYbLfsd3sWywpjwBUG6wQR9urbGzbrC5ypWC3O+Oqrb/CTYNCmITH0gWlKbC4uqZaGrjtwdrlhc3YGGk7tid3xQFaOGFv8lNC6pCyWvL69Zb1ecH15xRQG/OT57LNPMNZxe3ekKtc8uXqBjwfqeoGfeo7Hv6Bi8C/7kXPm/v6eaepm0UbmdDyRM1xdXfHzr3ccDgfGsWVVwuk4cti3aJMx1hCDxqM4Hkbub1sWdRQDjdbkGfGVcyTGCdLEkCJTP5ETVPWCwtZobdnv7tgfT6zWhil4TAatLd39AylEqqIUKkxpmeKJMA60pxPtseP88oqYNad+oFgI3dVPXsw8TYWpKpSXN3+9WRNiZOk9pqmpvKcsCvqum70KRjrgSuFjJMVEXVUMbZT8upxoT6f3yOymqjgcO25v3nF+/XRm9nmCH4hhFBNU9ECaT0dhAsToyQEKJzPxPPP5jS3ww4jSmrKyOOek0TaKki1GKQWMlSbcOAewhnHAuZKiLLCFoy4KCVWZMs7I6xTWMk3zNCNlgh9ouwM+XFA1DTFqqkVFP/UoK9TiV2/fYJWirGvag8SgV2VFmuR2EbynsAVlUcBsGMs6Sz6hkhBVa630PObZv1KPpiENyshEwAf6fqQsNSpH+r4lh56YPFMaMMngCsPQe4xVtO2Rl9/+nMPxjpvbt1gH49gxnloe7vY8NB1alXRt5t3bB/LbPZ98+ilPnz6Xen8QJqGrK7wPvHv3lpQmhrGfYaKecUwMXUBrx4urNc8/+JjSnbFcnvPRx59ineHd3WuOx4Ht+QLnFuRkubvf0w3vaI4tm82W9XbznevvV2ITKMuSy6tLprHj1B7Z7fbc3R74/me/IaqwyXN9dc39ww1KCUjhdDqyWtWCGE+a29sdRjdU1ZKqXKICJAJyHjDPXZN09aeBYRgoTYNzBXEKHLp2zpgbWaY0U3ZEl7B7eKCuKlxR4OoaKoufWm5vbjm9u6XRlsV2yzijslfbM5ntx0RVlKLFV8ImJEiJYFxJUTWA5CwWpcWae2JIaGeElluIH6CqxGU2DCMgHf6h7zg/27JoGnLbs16tePXyW9bbc8pmwdC19G0rNJ/sSNELaQeBZ8hD2AQplDIqzOIaEGFPwlmhEGENYRrxBOKcnMss2NGPISFkJu8pywLrLMZYqqoWpV0MwtBDYbVGFZZhGlFkQvR0/YlTeyBrAXPYwhJyIOSAKxwPDzsWTUVd1+x3O5wWA5BSmmEcMIcDdbWUZinMxig9p6hlhmEQEVOhmcIjwFNCXo2RXkBOMI3SbCuLzHKhORyOpNBiysjZ+ZbNdsvp1HNzc8fp2PHhRx+z2z1w8+4tw9Tx8Ucfslw33L56x3I5G9k83NzckSIsVitiBIX4D5QSwdU4jdze3/LN733JxeWasnR0fc9qsWa1ctSLhnEIfP7TL3jx5Am/8zu/S98Ffv7VS2ypcYVi8h7jOk7HAUXkYX/P6B/4/Kc/45NPv8dy+d2k0V+RTaBh7BqOx4mpLzAKqqqgfdhzf/NOhB25pq7O0A6KasTWiZwPTGmibx3Wrrm++h6XFx9R1VsxjZggoZjZo7PB5ozykLqB2B/JeU/UWkQ1w8ju5p4QAsVZQxU1qWs5TAGdMq5YYuoGVTYyWjtODIcTeVlh1xd0y5pu9NT1BUu1YeoGbHGGNxHrLBZLQmFLh46QsmZ1fo1PimBKumkiVgt6NNtmCXZPXWvubu/xMbOwFeMoYRN13XAcOrZmCVrT9h1Pnj7l62++5e3L1/zgh79Of+yJnZRAKWeMqlGpQbNAZS3GqOAJ/YG0sHJjKEQ8E1KiVY6irLHW4WMkDEeKsibHzNT24k5bL4hdj98fcFqzKgsRHoWAMwaMxlYVh/bEpBW5rpnajnEMqKqhnK/fQzdyOpwIU8I6ByFTu4bu2KOUYZoCTnv8EClsTX/qyFiUU8Q4cVQZrxLNwqGSJ8WJFBI+SmJS1ogxx9Uko/HKEExB4cQPYGwBWaOSBi/X9bSoiLZGlwUpHWnvDugxsT+0pH4iD4F+13O1/oCz80tCAqdqxqNC6ydkDK5cU9SAdpxfKU5tyxdf/QGr5ZqyrKnqkqfX55hCkcYlsX9Ke+w5vms5255RpjXrYgtkjqd7bu5vqBrLZvct5MQwvWE4ilR7c36GcQts43Glo7254dzV/N7nX5DajhcvXnzn+vuV2ATE799yf/dAYTWrRc16ueJ0OpBU4nDYceqOEiyxtiQ76+RzQKVAVS3Zrq9ZLtcCnIwRU1rRrs+wSZMNmjg76eL7TrdOif1hx/39Dh+1ZLdpiH5knAKHU0+1XFNVBQpRaqUoCUN1WeMKhVs0mLKkcZIRqJISA6I2knFfFTirGFMSr/0wMo0ToLHGMM7a8bJpMDpxOJ7mHsXw3tI6jmI9ljAUjfeJmBTaWuH7KcN2Tl5q25aUpRYeJ/EodG1HsQlgZENU2ZP8RPATOXq5JCslgM4kslZBjonL7lFl18+5emTpvOcwcwJTQCGk4+gldBWYGftxTpa2uKIQrsEcExZjJA8jx/0BP3mWKzmxjJYMAO+9jHZnK3FZlExmoh9HYtYY55hiImRJQtY5kdXjOFA4CTlEYp6k+WdL0HbOGWBmKIp11xqJH5Owm4BPnmVdoXxJGHoe7o6Mk+fq+gnnZ+AjbLYbqrrCuRo7lyOH/ZIQJrZnZ9zf33J1fcGpPXF3f8uPfvTH/PCHP2TRRG5u33HqHlhtGwpX84PPPsP7THvsubi4pFk07Pc7co5cX12yXFSUpeXdzTtIgYe7W96+fc1yucTVJe/evSVbx3g8sN8/UFaJ8+0579684x9E+/iV2ASUhmfPr1B4lk2F0Ug0V1HiSsux2zFMgzRzfKDvRrbbJcY6QogoY1hv1yw3S2xpSSS8n1A6zjprMFlBzHNu3gwnTZFu7Dkej0zTxPb8CU3TzKOyeQyWInUpja4YPWTZZMa2oyxLKu2wxmK0ZMpnFNFHjFZMc5dbZvoiXokxCmAkJ0Fnm4pxHOj7lvPzS4zOvH1zy9XFObu+p2oaEnBqW0IUeawxTkoVW0jqstKc+o7t2Rk3dw8cjkfKqmK9XrM/7emHgb4/sSWKeShDThK3lWIk+YhxIiMOs624aRaMw0AkYYwlA13XMYyjXKGVEh3CDBERWTGC8UoRRRYN/EwwftT7O2vQhUWRmfycK6A1XXsEMsvlksIZCifj3JREQ++cY6KX+b2z7I9HUlRoJ98/HxKhzBRWUVixDOcsakM9h40666XEEQ0T+jEsPQk0lRSJYcJPBm012iqs0WgqsAsgs97UnG3PsUXJqeuEBBwDWY1oK4zD58+fc3PzjpQSL1++5LPvfywKwv0OYzSH/R5nS66vrrClWOnrusJow3q14sMXH1KWIldu2wMoWK42lGWBc46hb3m4u2G3OzKNkfUHZ9T1gpfffIspa24fduxv95Tnms++/wO+/fYlV9dX37n+fiU2AWs110/OUExUznE6Hri7u+HFiw9QRrFYLsidMPpXqzW7+4MYV+A9kx2VcYWlbkrK2QGXckDn9J4YnPzE1Hf4aaIwkkq82+2IIbDdbFg0jZh2pgmtDT5GnC0oC0sYBgIaUyimsac7nXCmoKgalHVSZ+c8NyEDWsPkByEYhwIfJoa2xViLstIgs3aO/5pPqxwDrqwQTYMlSrAewzDfGlyJdRXKylhLGUs3egn+8F7CUJH6MCtwgyEGT9eeaLqWnCNaC8sAJZTZPJ/GrixlWJglQDRFcRo6JzbnYRxnDbvjMW4shMQ0TYTgKayMBDVz4u/ctMxRNoTHDActmR8YlVEkoh/R1hGmieAkVr1wBmc0OXq0UtiywFn7nrBsZ5bhEMIjKZT9fk/rMk1TsWwqSudIM5fBipMAvAc9YAqwRmNyQMWJ5BWehNaJGEvIUiaUTYHKkZzAunlTzJYpZOHSqsz+tENp8NFQNoqQBs6WF+R8ScqBuq64v3+g7zu6ruOv//W/xmYjaLGLi0sCI4fjHUrBOPWUZYV1hn440bYtzaKSJO3oORxOrJYbqmqFNnuaZs3zD16wXK+oqxXj+I5FKaEwddPgqpKLqyvWZ1uePn363evvL3Nx/1kfKScedjecuj1tzqiUGOc56RQmXFlQk3FFwdX1M8gFKY/0bcswTNT1mYy45pgoly1WS569JP9IQGf04/vEn5hF3z10HWVVYZ1jt9uhkVjqfupJObPaFox9T+gnbLWgVIb2eCDHTN1UuFKulzEnUvTiSkN+PXRHuSLH5QyMbKmqata1S8hG3w/kGZLR9/JDXyzXTLOhJyaxy66ahqpZIJFf4ov3UW5Mgjt2+ClIPLsSx93tbUeMEzF62vbA2J+whcHqAjuLflJK+MlTL/S88UhUvPfiGbTWCv03xhnh/Qucd55luKLgU+9NL1ornNHytaeAIpFjRKuMm91/2Si0pIiiMZLGNPYSMpoTVitSkOSfx5uWMRIVZp1lsWhIaSChaJqau/sDx35iDAJUXS7MLI2WVChjNSnKaBWlcM5CDKQwEWbscC40JEMKkg41tCPdeEQlw9XZE1whKU7aluyPR4rSsjvu2GyX9N2RspGg0dDXOOc4tQd++MMf8Ed//PvEGFkuJXD3o48/gqTYH/aUtQShvrt5Q1k2nJ9fiIM1B3b7Owrn5kAWw3q1xpqCaZwoy4aLS8tmvWSxXFAtGs7OLqiXC1Cavqg4Oz+j9wPrzVq0HN/x+NXYBFLk7v4d3anFjxOXFxesN2tevnzJ6CfpqgqCVJpKypGCpKq0XYs1De/evaGuNvNV9YJKFTgtPjfSjP8KkzDvYxAfeggU1mFQtIcj+9PIomlQCvqupW4WGAXt8YCyhVy/+5axPbFwDmcdYfJgBMKFMThnUFn03X7qKZwl5YD38vmnUZRyVVVROEd3OhGCRysY+x4fAs1iQXs8iFV4VpjF2RI8jR7tKsp6QUCRMihbkJTBx0nm9sFTVyWH/R5rFYumIoaRrt1T1AVOZ2xh55JI1HwpJjGrZHHaRaKEhyAde5jVf3MuXpix7I8BGSnNEUZZBEUYLeWClwWlcqQwhuQM2ovn36j8/kpODoQp058OMla1wiLwSahLWgkrUBSgktE3jIEpZApbYLRiGCKhn8COaFfitCJGSThSBnROxOCxxqBiIE4DBsAmsgGVC2FM+IG+y4zTkZAGFs2WZrWhcJb9/uF93HpVVxSFpSgL0bgcdnRdy49+9neIKXJ/f8s/+1/6L7DZbOYQ0oZ3795S1zUff/wJfd9hi5rt2Zbj6YRWGYh0/ZGcoSznuD0yxjQslmuiVyKd355htKDRj6cjU/IUhWW5WFDVlUBtQ+TQ7XGV5euXX3/n+vtTN4HvCB75nwL/eWACfgb88znn3Ywl/2Pgx/Nf/1s553/hT90EYmLyE8vlklD49yz7b755JXbawEyU6TntforGsDmriTHip4nDcc9XX31B1wVSyjTLkmqxFTpwnEhTT5p6gh/FgjmOWJ1FGlwUeO9p2x6t5Kqbk1hmnTVyTc1Qu4I09bSnI3GcsLUlToGH/R7jCqrFEqfAD73EcQX5HFZDnEainzBaWPBd2+MuL5mGgXHo3p+qMUTRFlgndB8lDjdbalmoWTH6SL1y1IsVIWb6qaUqK/wkEdYhRhkvrRYYa4lxYrmqCXkO/0gBUhBst5ZGZ9e2rH0Qnl7W+JBmX4QhhEgIEbLALKdpks8z9wNExl0xDe17pp+zFp0kSyAGL6VFThTWMCHjWqOYY93yvFFLxt/Qd1QzHsw4y5iSlE1JqFLWKLw1c0KPJSShL1unUNYxBNC9p6wjdeEIQShKMWXUvJl5MpNWuCIK7ThHklGQA9aUOKcprKEoGrQp2WwvWSwrUIp0SHRDx+XVGf14YrFczXZng/eRxWLFelPw+tUr3rx5xRdf/IzVaoH3I5vtmmEYGIZeDoHCcXd3y8X1louLC5bLFUpn3r57DRmapqYoFkxetAOH/UgMam46LiVkVEUedncYp2nqEmvgcDxQNzVTnLClIirPw+H2z78JIMEj/yvgf/9LH/s3gb+Rcw5Kqf8J8DeQzAGAn+Wc/9E/w+u+f/jgGfqRp1fPiFPg7eu3QglKmrpZoLKcWn5KfPP2FXVRUzdPgExZFVSVQ+nMfn/Py5dfcXV9zmJZoHUGLzkBaeqJs8pumkZcXc45dcLTV9qwqBvKQrTopXP0fcfYdzTLFVZnuuOeh8OJqm5QtWHsOw77HVWzpF4sIGe69iTACxKlE3ru1Hfk6HHGMA0D09BhFcQwzXmCkRhlAflxFNZHznN0uPQHlBI9QWZEKYuxmbYfOI5HjHG0XcdysSCmzOjH2eFnaYcTKRaUVSVBLDlCCqSA9EmiZxxk8ymdhGKG4LH2ETv0i4fIIBRm9r6P40RZiloz+lFwZ1rGfvqx+ThPG9QMQU3By41svooLwUkUjBnpt0CicCWucGijhKPoZyKSEvCLMXIbiBn600BROGwQLcXgI6OPWGOJSW5pIUpPQuVZRp4CVity0MSc0EYRQoVSDWVhWSxqmsWKGMXoFBkFZLuuqOpSyrfdwGq15XQ6Yo2cvqvlit/6rc+4vr7k+snFHLB7xhdffE5VX/DB82dY4xjHga5ref36NfvTPU+ePGW9Xs4/b+jajhAnlsvFHMRraI8dbTsSo+f+wbCOS4apZ5h6zuutoOsUfPXVDTmvKWrpa1gHq7O/AF7sTwoeyTn/33/pt38L+C//Gdb6dz5SysL7x/Kw26GVxY+Jzeqcqm4YfWB7fiFnhr9j2dQoJHZs0ZQoIut1zbLZ0LZHDrt74gdPoFBCqwmTnM7jSD/0lEVJ4Ur6rqesG1xRU9RBuG/aoFVm6Foe7m6pqoplsyB74e0RJggWUikLnkxdFlLTTqKmc1Z4+kbPqURhkrk5MHQt1shJE0JAk98bXyDT9x0uyYYQ4wwJVXr+vgu5JwMow+nUky3z+E1y9cqqpG2PHI9HjJbXadsTS2Mooif6kewMRDBzk6099QxtR1MuiFHSeh4NTCllwXxnGa/qWevetj1917NarbHWYWxBTP08DXFkm/CDf7+PpJTe3wqCl3SiEDwpR1RUpCjKvTybsgDKVAoT0WhI4noUGbg0ex/9/ZBFoGQzSkd89ILuMkIgHn1AOY1ByaaTE4FEqkqyE+SSxMmLvLlpahaLmqoSeAgqoPSINpb1VrQT+/2BoixFQRlgtbzkdDqSY8F2u6EsHd/77GO67kjdlGijiNGzWq05217w1Vdfc3Nzw2q9xsdhFoMNyARiRVE6Xr96RUye5WKJUopmIdSsySeO7QPaRrquJcRRbDFKpj8XZ0v2pz1DstR1hS02vPjoCd/1+IfRE/hvIpmEj49PlVJ/FzgA//2c8//rT/pLv5w7sFqVlK6i3bcM3UhTNmxW2/e18OFwS5gSm7Mzmo+WrJc1o98T40F030mxWTds11IzLRc1zmmUnt8wZIxWxPkUW202kCJFWbPebMXfPgw4V2CUZuhOHPZ7pnFgUZdoEmN7RKdEZTU6efw4kKNn0Qh1yE+jLFoNGivX3Nmqm9J8KoVInLMCyJGh74hzolHOMunQRU3KmTBN+HHAKi03hRC4f3igrGriLH89HA6cP9mI27ERkElVltJraDuMztSVo+9P2KHHzviyXJSPgUrSRY+R0/FAUy/BOIyW8Zr3UmJopeVKHRPOzZbnlMgwqwc12lhpkM6mJD0TiGKU6UNKSUw8MZBinGOxZJoiCzuDSuQsi0UHcfulGGAGo1prsUZ6BBIuyvxxub2ASJN9zvTDhNWaReUIMTHlSFmXOGPmRKJImITzoLQmkzDWUFQF9aLGzJ4IawxaZ4oSRt+jUkF3PHF7e0dTL3m42zNNkefPn2JUxXq9YZpGhqGjWZxRlgXkxGIhvg3rLMMoCdoXF+estktCmlgsFvS91P9F6aiqQhK5rSbnyOnUobWmqBRVI5mG6ES9qLCFfM/2+wFFYr1a8LC/ozudaDvFqT3w2Wff/84F/BfaBJRS/z0gAP+H+UOvgY9yzndKqb8G/J+VUr+Vc/6PQM9/OXfg448u83qxYr/bsV1tqMuGuqhJSZJ65QTSLKqa9WpFVWj6MdG2b1A5cH6+ZbNqIAXOzy65vrqgsIYcx/nNIUx54yzNYkHZNPTHE8vVhnq1wYc9qIBWWvoDp1ZO+XlSkfxEO98aVBT8dvKawmmUtaTo5dr5SAieNx2LiG+C9xDiLNyR03/oWvb39+8lwM46utORerUlTJqhayFGmmVDd+oFwHp3x7PnL8QTESLH45HLqxWkQOEcJAgps16uGIaOcRhYLrY4J9LloZNI8LxYCEdv3gS0UpyOR5rFkWqxQmk7L16JX3eF3DRSEit11w94H6ibRp6XEYqvKYghEVPAaKEED2NgnOT7E2OShixy05jfRGIMUlICxST0Xx2jZE0gZYAiY53BakNROIwdiUm8I0VZEIInhow1DgonQqgYsbahLAx56qUXoUXIZbQSd2MKKOR7UDc1q/Vqpl2PBN9RV4axH6lTjfc94zAw9J797oHkIcVM6SqMslSuYbM4Y5jumfzEz3/+Ff1w4ur6gmZRi5MvJ06nI+vNkqqqhVeR/ftma92UgDgg15vV+/5L3/e4wlCWDusKbNK0fUtV1WzPztHGEPxE156oSxlx7g5HoTntHjgc9v/wNwGl1H8DaRj+0zNhmJzzCIzzr/+OUupnwA+Bv/0Peq2yLLm+vOTV8BJnNOtlQ31xwf5wROvMclGDsZxfiFa+7/YUTnN+tiRnz/l2hcqe06Hn8uySRVNjlAhfQgyzOETSZaqqBjQhZlbrFVk7QgJjC3KKtMcjh/2e4+GIzpGpLDjtd7IZpYiPkapqMLp6f53NMaBmlLU10skW+KYR9VwITGli6EVS66eRw37H3d0NdVUzDT26htPxwPZKkoCnYaAwGqsVXdtSVzV+GslRmpZdGEXVNnT0Byfe+KQYe2k6xRBojwdSzEKZcZbJB8Z+JPo4u32lMVoWjnYYOJ2OoA2uFI5/mJ1+rijn0aAk5EzTRIoSIx9CxBQaTIEyGcxj2GcUaGeSTeIx9QctPweVHzcC9dgaFDVnzoiMR5DamSw/wyxMwTzX77IpQVJI/Hphyb2MVa1WRCOx8tZaVqsVoVeQvVB8QUbIWp5jjCDIy6pkuVqRyJzaFvKEwjCmDu4kfNT7iMbObk84225FbhwjVmuGrqNYFFRlwRdf/JS7+3cUxa9zfiFoudOplXyJQs0lIHPQTBAUmrEYq+aNQM9sDUGrLxY1TSM2dB88p7ZHa4dzlfRegmYaIjp6rBLCdIrQ1IIY+4e6CSil/nPAfxf4J3PO3S99/Aq4zzlHpdT3kGTiL/7U1wNWi4rry3PaQ4sfW0qr0AS26xqM4ez8guVqxdANHMaOmDvO1g1KZ5rKMvQncjBslgucUu+7yVpLLaiyhF5aJ6e9NhZbNfgpkNG40jEed4zjRNue2O/uqZxlWZccvNSopxBIOdOUBVpFTscdWUvWYFE1kBUqMwt/EjlmyR4Mv7D+QsJPowA1TkectVIjG83pdGDsO/TkGfsWW9eMw8Bht8OcKaw2dKeW1WYrNmBrGdsTpxlJltGMfU/pClQWfNmp7VmtFxTGoaJnHCamcRIxjk7zxljSDQPTOND1Jyqx58z5AyLXfWxU+jDNBhzJAhiniaYsRTAVEzqJ/TpOkcRMHy5KovcYN4m3P3r8TANGyYLn0eGnBK2mlBLtBTM5FxklJ52lJDAaY8EgFKBm0VB2LYMXZJqdpw/DMKA2Ehw7nnbiOCwsWs1RZUjfJCOj0Cl4ptPIqW1pKmksrlbbWanqCT5SOkNdNjhjWdQV0+CZxlbi6+JACGrOPdCkFDm1J84vzgjBM44DZVkxixkJ/Sh1DdID6vuesnIMQ0/fd/K1WEtRVNR1SVkWtN2A1o6rqw1XV0+x2rK7eyBHS2nFju1zoqk2vH37lu3HV5xt/wKKwe8IHvkbQAn8m7N67HEU+E8A/yOllPhW4V/IOd//aZ8jBI8mcrZdMhz3tKcHTod7yWc/PwNtWDQaZyNjHlnWjtFrRj9SFGIXzkHx5OqaJ1eX6CxX+EeMts4JooSDaiWhja4oQTuGoUMZkaX2KYk3YFbRKWfJKc4JSI5+EOmyVpkYxGde1QvKqnqf42e0JoZImCbGYWQcRnHgTQN+mjBzGq6fRhEJaTkLw+SJ3jN2LT5mpnGgsobDbqI9HigLJ5LTw46qket8WTjC0DEpRagamFOHY5CJg/gIujliq8BqS5g8QzeQZ+mucSLEsUZi3adxmE9tmQDIzUkWvOQUhnlaIbes2Efq1VpCTkxCm0g2UdJ+zAzw1JIiFYKnqhtyGPEpzSj0SMx5bnbORGCtZVFGCaARGCxzQ1AYBUb3on6cPQZVVdJUnmlqZ4SYhdlBeNjvub7YYo0hBjWj1cQB+cuPYRzZ7Q8oK0yCwilSNmzWl/icicHSticWdUPXdiy2C0pnCGPH8bgnliXl2Tn7/UjOiY8+/oj1ZsE09RyPUhE/PDyIH8RYjC0oywZXVCidObUiLTfhF3zJ1WqDuMoNXTfIzyIbnHOcnV1zfv6MafSURSQyop2DqSUlzbPrp+Ro2awuWTbbP/8m8B3BI//b73juvw7863/aa/5/P2IMlIVi93AihA5tHRow2nLY3xBz5ni6paobjCqY88IZ+xayRFo39RnPnl5TlcX7K7hySmS686I2xs4/eIUtJBR0nLzUzErNM3HpDZROwBWPnWg/Sh3sCokhH8eOtj3MufNSYxZlIfLP0TP0A33bM44D0Q/k5BmHgbquMVrhp2m+CMtNKKVAWRRM48DheCLnzGa5oG9PMxvAo3Km63revX3L9uJSFjwTOtfkFEkxYpQEn1or3xfvI23bC+hys4U54YgUgYSNwqOXTruZQSuejCUlKIvqfa1urSVEqdmFhx/F1JQho1HazB1+jXEFZdWgUiY7R/ATwUvEm+8LlBJ1pStBBbk9SFiJiIRCEik0VsxSSWVUzGQjqb1ZiRTZaDOLlixV4Wa/QUI7N98ooO1a0nYl74VgZriqnssAmQ5o2Y3pxxHJqJpVkcA0JYpqgdU9MfQSZosEuYxjyzS2pDQwTRNv3xzos2Oz3XB5ecHV1Rkvv/2acRxo25bdbi9BIkaayU2znhObJDEqzT2AcRQ9hrOOgPRT3rx+y3a7pVlu6IaR02lks04Er3BuwdQGgteslhesTMYta37j1y7IObO7+xWHiiil6YeRn/3sZ5Qz0rmuhCP/8HBPzBltrQSCJkPwgbpRKGVQyuJsyXazpSpLxqEnaMFgF64W0mySOlVJ7hSmKNHOiclIKaxzhHF4H6IZZhRVURbEGDBasGcZ0cT7ccCnxNC3syFpRKvVTAQ6Mg4TfhwZp5EQJikDcmSaRsqiIClN155EMzAOhDDNyUmWaRw4Hg4YrQl+oj3JDy9FAWaG4Lm5ectqs2YaB4zy0mFPCT8FtCnIOUmAZbN4r58fhxE2whKIMRCUzO+ztP/nRGBD9p7TqcVHi9GGZd0wOUuYmXzWCJxDxpZKLMlz+SCBHRqwuNJSOofVhjT3L2KcIaeuQNsR60qUdqAFMAuPbkVJUooRnK2kN5ATOmdCUKQs//aUE8YVAlpBUVb+F9OGlMjz1EJpiS9zxhDFNoHWBmUU2sz1t3lMNJZRa0ZjixLrCo6nDucVMcFysZaxIZovv/yS87M1WsnNwZiCt29fk4oFVe24vR1YLBe4ouD27p7b23eSRGysHDg+cjzKdGGxqIT/aOF0PNF3LUVRMAzCEYRMCInDsUOZWjIZkqbrBmEXaosPmZwUy/UZWQWO/Ym6qRj6nof7u+9cf78im4Dhjz9/w0Ob+fSjpxSrFcaJ+cXVkcqIJddPQRpGOoOpKVyNdQ7nVji3IsbEfn+HNY7tZgusyFQok1CFJcWJ4EdyVeHJnPoTdVOQo8Rfl84xWUvbdhhXkrSjbTvqwjGOIjcdYsvYtUxqZJgDSAdXcLY5Z2qP7O7uRXATPSlPjL5n7DsMoJIkH03jwHja4/ueqashJjHjjD0pZHRI+Ozxo1iMlVKibygd6hgZ+yNjd6AqFEMb6UfPIvOehmSToygM69WS4/EgJ1tKjENHdIayMigjHXWfAkVZSjkQPSZHxtOew1iwXa/oTwdMEJhJ7QrJO1Tg6oacPaWGFD02KbR2RFuBchirMWR8NujksRF5TszoekAFsKqA4NFqwGgRjWZFWAAAvMtJREFUEBHTbDiSsix7O8NQHhd3FgIT+X2WYFWVGJdYeYW2GoMhKEVW4kQNKhOUxWnD6DOLSth+2WayzmijhLWYNU3RgIG2n0A7grIMU4fvdhitubq65t2bI/008M23r8hKs1mueXdzoq4qji30xy9pu3dkDNuzKy6unnFoPYmG88unZFXg48D+sCNGT5wCYTRstmucMdzsdgxdix97TscdRVlS1zXb8zMmr1Gm4Gx7Tnvqebh7y9XVE6rGcXa1pu96cinuMDMNdP2RaRxYbv5yosn/oT1iglPn+e3f+Sucna1ZLWva9kiOnvOLK/zkaU8dp3Riu91QVIXEX48Tw5DwY0uOFoVl1SwxNWidxEOdFGiN0nMCD6CMpm9PTGPPpqlo24PIXGNkv9tR1TUpWtpOLKpGRSYvmC8/DgzbkWin947DHCNhHNjtj5wOB4qyEHKOhr4/iWtRGzTyOfzYk7xIiYfu9H60NvQ9pVPUZckYPH3XczqdcEUhuYuLhZhvspQjWmemkBimIBiymQyU51uHcwZrDFZrytkKnVVgnGamYEqkkDDz/N17jx9H4jTRtZ7KWU5xJFpNs1gQpxFtK5TWFM4Ro6bQcR6JSniqKzUWuYYrMm4u3QofKHxkHEds2VCETFaWNHSgvdB+ldCgBIsuDsfgg2gVtCIruRENcYAsp3eIMi0ojGO1cCxXC+73J1xTE7JiSsIayEYkxhgHylJUNV57spZSwGiZ5tRlTbmoQItxKw4R48DHkWM74EqHskJ9StpgXUXZrHBFx+HUsVxecrr/iq+/eUvdbHnYdxzbQEyGjz75PpBwRYXWMIwDOXjCOMqo02lCKKiKkuNxT9ueKEpH17d0nePqyWeElFk0DWXp2O8eqMqKcTwwecPmbMsQWu6ONyyXBQ+7t6QUZPS5Wn3n+vuV2AScNfzVv/LbXF9fUFcFfXdkvapZLi5w1kmDZLPkeGxYbTY0qzWvv33FNz//mnEYWC+WHI5HVIbiqWO1WqG1EViGGlFGkXOQUyeKmWU4HTAkDIk4Sjrw4/X78vycu9sbur6ldA4/TXgfCTnTnTp8iKAl7zCnSFUWDH3Pzbu3TJMn1DXDNFAUlqHvJAA86/n5j+IZQXudTieqekHMSpJ1cayXCyJwPJ44HVvKOqKVpigqaZwp9b5ulNeRuv8x4EMSeiUFSBuF0jJj11o60MMwvE+8fXx+VUlK0DRNM2h05HTYE02mN1BWlTyvURRVjXaWnAusiriiwboStMbMjb0YA8I0XEKcyGEix8DQHjHWUVUVOUsjUmmN1ZasFRFFehQUzX4K9X6MmAnzGFFEjY9KSin1ikJzfXHB/cP+vX15GHquzs8BxcNuz7qsiKj3gBY1Jy6puZRJCaqqpokNu8MtykSqpiBnTd9N/PzLr2maBSkk1qsNMWYeHnZcXV3hvWe9XIF7xfH4JVVZc+omPv/8Z5xfXvP06RO+/OJnGJOJfqDrOg7TJCxGCu4f9mhrWCwbYlL4AInIMPRMfk9RXYJyhDhS1+c8fXrJOE2MM5uznCynbsfxeKQfC969fYNSzKGow3euv1+JTaAoHL/7O7/FH/3x78Nmyd3dW56/eEZZG775+iucK7i+fkLVFGhbMYbA4D13ux1+GFmv1oBiGCe6bpCFY0sWuca5jLJKtOlhJMdJXIXTSF3XqCgfN7agMJr1oibHwNQPFEZEJIeTWJZTDByOLT4kUh7xc7OrdI6uPfFweytQkTSHQiaJ73JlPfvemcdRAssYhoGQMtoUkvs3TigmptIzTRPDMAjAZJDQi+3ZGdYKvy8EOVW1dsSQaU8dy9WS9MjZR6YuOQdSMqQYKIxj7AfaGEVlaMz7jSOnmcKrhCFQGM80nEjJE6xwBoZxxMbAsllgy4qUJpzNaF2gbTkvJtEC6JnA5KwlBU0ZGkKYcOUMa43ydWgjRiBNJgUZ6+WsZ1+HbFpaaYkNJ0uzkEfNAcBMHRKDNefbNeebDfthJERJW95st6zWGw53tyJaSjCGSNGUKONQ2pGUxsdM2/WE28ip2xFijy0y33z9lnESIMzQ96yWK6w2VFVDzvCw2xOmwHK5JOVMUTQ8ffqcqtngDj3923v8nG+YUuLh4YGhP3I8nSiMZbna8LDb8fbmlma5YNkuaXuPsRWH44FT280lZuL8bEF72vNwLylQ0+Q5tSeMNYyhpR0OYBJZKxaLZpaew/3dw3euv1+JTSDGwO3bb9ndvuPm7ddstktCGGhbz8Puls3mjNF3VE2DchWHuyOnvmecMVZTCBhtcCbTdi1hmiBmCtPglAa0OOdCmJFYgdoVNM7h+440iWnHaGjKip/97GekGLm4uGD38MDD3Q7vA+M4sN+fOLQD6JaxH+i7juA993d33N/eUTcN1ogKzpMY+47SFjDHVMk1WTj/0zgJ6TYmwoxFSynSdx3dOMwx545hHN8LaZyTcaYkz3oW1QZQDIPn8rIC4HQ6MQw9SjGbgR7xX3LFlumA3EiqqoKYSD7QLBoK6xi1YdkUnHYt09CCMbTHI9kr7PqS87LBlBUhWlypEQNDgSlLyBL4ZpwEfViTCDmIrBg9u//cDIKR09wYi8qJmD1CAp5DOo0h+tk/ATO1SPQfKUvHXNkCY6xszFFciddXF3TfvsEUDrTlcDjy/U8+obY/5Obl12StMK4GraUkUZaEYQqJ/anFjh0P+7f40LJeV9y8u+HYDhSuFHNY7imsIwbIIaGN5ptvX1K6grqpObWvuLx6grUlOY8smgXn5+d88803DMOAsyKEGoaeIUmg6263Y5wmqsWKUz+hdIkra6qo8dGyPdtyeX6Js4bd/sT93S2n7kTXtcQsoqqLqwtBrBmNK0sWC4m9X61W74Ns/qTHr8QmELzn519+Qekc7969BAIxjLjSslwv0U7zsy+/4Pr6Kc3qSnjyxnJ5/QSrNHXdkH3gcDyhsuJie4bWRubf+jGkUtJnc4agIotmidWw38kCVyHLbL098frlNzx/8SEpRO5uHzgeO8hwPPW03cTD7sRy4VEZ/OjpTi13N7fsHh7QWlEU9n2ybHc8sqgagYDMFKQ8a+nznLsdQiRlkbNqpZmmaU4jVnNabZxlpQFlZBGZOYuQecg4jhPOlXLt3z8wjgNVJQacGDN5jmIrC/derBSCoM6zjvT9hNaKqpJcg8opep0Y/cgwZt68eY1qBi4+/iFlvUCXFQSHrRwxJMCBdSKSmgNfReCfyPSkDKOfpItfSsdaz4tdRmOPWgHe5wIopWF+g/9CsCTqOdk4RSYs7MdIGCeO+xZnC5q6ItkCHzPvbt7x8ttX/NYPPsMpaPf3jDFQZotCY5TEzkc0/TBRoOm6jtvbb+lOFcZVXJ4/xRrDcrmkazt2u3uqqublt6+4vLxAG8OhPXE4HbEmcjz0ZDWx350YvLASP//8c0KY+PijD2Y14IFh8Nzc7WkWCyEOlYuZKGXZH3uMKbi4es56vWa3O9KfdujZT6CV4uXLr1mdbdAGjq0R5yWG+4d3pD5yf3/Ps2fPxAH6HY9fiU0gZwFDTNPA+dkFx+OenCLr7ZrNmUQ/7/dHtudX3L96xdubPdM4cnFxQQ6Rdn8gjh5i5GJzxuXlFecXF5JDaAvBccVAzpqsI1YZbAFhaAlJoUwhXXg/st/v0UrhtOXVy1e8e/OWySdQctomLLt9S1NbrHXkJOm4h92OoesYmwZnRXijNKTwGOCZ6LsWQyJM/v2JFoOc/NoWsyNNaDwpZVJMSFKQpBQPw4QtHFobtHaQ1Xvu4ORbYspUzlGWJcPUE1JAGcXgByY/Ql5QVzV920lktVLkZURZyVvsTie5YecMccJp6ZlMfuLduzesn1Si1agaVFlRZOkDqDwBBWBBxTkkZMaYzZ6KnMEHCWwxVkZ6xkrirvcTOc1KQWRer7XIrzNZCEEznkxeL8nIN4siE0QjkaZJJiwxUljNfduCceSs+NFPfozTio+eXGEU7Pf3qLlJiHagHcpWoDWHw4GURBIcQ4dWhqdXz0gZzrYbXvlX+ClQVTVVXcu/RSmGYWCz2bBdL8UvkTNNs6R72PHy5Su8H5mmAa0SZSHK1YvLK1AW7wO73Ymu9zQr0Q5IYIhmvdlwe3fAMnK+rjEp85Mf/wxbGPp+ZLnNDMOEv71jtVlSVBX9qWVdNXRDz/1uN2+2f/LjV2ITiDFBNOzuj7z4+BnXV0/IOnI47vnxjz6XmWpKjIPnyy++4eHQ8eL5c/q+59U3L2n3B5zS/OB7n/H8+XOxchaC/UI76QhjyFmLp1wVKBI+j1AscMrRngZC39H3PZvNlnEc+dnPvmC/O2Kc0Hz7KaBtwf7UczVW79OH2rana3v85Bn7AWfN+7lvYQyLpqYfRlnsj2/keUY/DSPjlKgaJdQg1fM4EJum6X0K0jRNjOMg/IpZryDec0ddLzgej+QsmQFVXRGSl0ZeXTCeBGSqzi8ktXc2DFVVhZ8mCe1ImX7sKJzDKE3wI0ZBWVjiOND1HRdFITHk2gLSUMs5EZKfQ1Ek0FQbQ06BGLMwFufGnZpFOTkjGHRjKArRduQkwFiVE0R5nTxzG1OW+TePtyfke6C1Ee2ED7NwSLNeLNgPwoi8v32g2ZxRVCU6J/7+3/89hu99wm//2g/ZnK25P+1kk8AQMcSkyBGKqmF71jAO93SnOzabS66vnvDmzVtubm55+fIlh8OJruuoqppu6FmtlixWC4qqoGt7Ntsz0TBUNbZs+PrlN1xcXAKRL7/4nKZyNM2KzeaMgKFte+7fvCMfe566hnHyLJZbDoeWn/z0K4x1/PXf+SHPnj5hmAZ8yHz51Rcklbm6/ICfffFTtM0cTy2nvmVZ11z/2q/zwQcfAPDm9bvvXH+/EptATnDz9p7Lqyf4Ud6MVVPSd4H72z2nTvzWd7d/h33bc3H9jN/8jd9gd3eP70dqW5BDZLPZUjcNoDDWgC2IyiFXVYXSpXStlUI5w+l+hyqWxNRx8pE8TuwOe55cPeWnP/mc07EVppxRZKPwIRHDhDu29INGK1Ec9p3EbfspMPSDqBYLJw7GPM+2+56YAsMwoBVEL5jvyXu0lVhy7wN1rTgejlRNQ9d1NIsF09DP7r2ePOvqq7p+3x+ATFVV3D88cHG5JaVEVVa03Ym6KUXxOE2y0BQzNizT9y3v3olyMKXE8Xicyb4WZ+R7lAuD22y4e/lWSE4xyelrFSkptNLYohIRzjy5eDQLiYLQSnNzGimKArtYkocOax2D92TynBwtI8UphveEIm0N1kmJEWOYoaWPbkPIOqOsxljH5AOT90JYSiLMX6+XDGGCERZNTblecf/wwKvXr/nt3/4N7KLi7c1OAGdKE5URXLmSxtp6cwZ5JOXE11//nHHyHI8HtIbNZsV+v5fb6/kZqERZFiyXC25e7XjYfY1xBR+8+FBky9fXPHv2DO8Hbt69RSE9p6pecr87MU2BJ8+e0bY9h2OLtQXHYw/a8MNf+02qqqYsDPd3J07tEescl5fPqBc1YYLvffprxOy5vb8FHJtVw+GwZ7lc4VzF6dR91/L71dgEpsljTMnZ5pKvvvmC+90tn37vY5p6RVUcuTx/yugDn3/xBXXT8OLpE968+pY0BZ49ucJ98IzCWDbLleDKrURtKWtRM51XLDZAiHPCTkAVK0qnwTrK1ZYcOzCWCDNU087wTE2M4EMmkvAhMU6BwmqmMTIM0yz2mRiHiaEfqapivinIzL7vW9kQciJF0eGTFCkmlMrkmHmMzp6mCVsUpPl0ezSRkDNd180TAsPFxQXWNeScqRc1RVlQNQ1106A0HNsDrihYzHHrXdtSGEsKQRqDwTPERHc8CdK7HzjudixXKxGbaJHXqtnRZmaUt0iDjfx71fy15HnyMSPHUxKUd5jmOO8QKZ0lBEn7iSmJVHgGjL13GeZZSK0eeznSL0gx8VjYSo+E+VYgUuM8W7ZjSvJ3kkiJVSENvxijwF+IvH33hvNXaz75tR9SNFu+fbcjROYysaKuFafTO1IZOb+45ObdLYfDXtiLwXNxecYnH3/E/cMdb968oSiFsxhTIKvM8+cfst8feXNzw9dff0M79Cxz5nDY0zQLPv3ke3z15RfkpLi7fUAXFWfnF+8Pk6KsaZoFD7sDu8OBs7MLLi4u+PlP/hidAuvNhn5/QJnMB8+vKUrLGEZWixU5Wy7Pn+IcTNMDX/z8K8hGRrjf8fiV2ASWyyWffvoZ6ITWTuTA2nK2EdzSD37wa7RdT/TSdb4829KeTkLtzYbCGrarFcVM+oWZMGsMypXzlU+T8yxi0Zq+3aPLJbbU+JSo1meo2FI0C6YQsUVJUdaMk/AOfQj4EMjGMIXI4dizWS2Ykse2AynBNMmG4FzPer3EGMsUZYJwOh5nqKUlh8cTTfz6OQRMeMwDgJwU3kdgvjqnjPeBqpYwj8KV5Ayr1ZqyWrLbPbBYLFgsFjjn0EZT1xWXV5d47ylcSZgiXdejUUzj+F6IA4FT27JcLMg5czqdWCwWpOxROcnGqRRlVVEUhchtlfql67pkJv7CijPbgecFG2aPREoBZzTTPM0J7/0CzDLgR7CITFZU1u+buzE9Eo1FlyGSYbEfy0akZ0l1JGfRFIQovoJ6uWaYRvkeG8WirJmGji+/+hJVOJ5/8n2urp/wcJoYpoBrtKDJiprd4Q1dG7BaUxTSdFuvzhmnieunl6w2C4yVcsVYTUiaP/7jP+Q3PvkNmmbBejVgy4qPP/se++OBv//3fp/vffYpn336Pdpjz2azpp88p37km69fcjq1bLdnPHv2jL4fybND8tSe8MFzfnmNyYmqLnn97i05J7bbI9uzDU2zYvfwwO3tPT5MZCbW28zD/QMpaVbLs+9cf9/dLfj/46OuaxaLJYfDnmfPnvJX/spf5cMXHzKNI8t6QeUKFlXNJx9+yKIsOdzecn12xvPrJzSlE/FPfyKEiaJ0xOwZp56YBWARkRoUbaSOtY4QwVU1WRuysRJgWlcszs6JaExRUS+WGGtkEcb0mK7NOE3s9kdigilEjqeOlBWg8V42gpQQFFiGaRrfh3hKglGcDfYKlTMxRKbJz7cCNXe/4zwnT+89/ORMWZacnZ1LHkJZcXl5gTGGzWbN9mzzHg6aMyyXa6bJM46B4AND13PY7+cE5ISfJkIIdKcT4zD8wjcR5Go9TJOo7ICmaXC2kJgwZebZvHqv9f8FC4BfGHTsY4hImoM9pE8hsJAZKxd/sQHkeTygtJlTgfScezDbmecJiagIE3F2fYYQRDCkDBkJGo0xgxYO4Xq1wphZhm4tq4VEyf3oR3/ET3/yU5xxbLcXNMsNi9WWarGmbFbErOknybTIecTZzPZsQVUb9rtbdrtbzi/OSDnwh3/4+6Qc+bXf/DWsk+TnmBKPROZpnPj+97/Ps6fPOT+74vrqGcHD+fkVVVVx2B9Yrzdst2eklLi6vuB73/uE8/MNOXumqeODF09JGv7gj/+Ydhg4v7wUtoVxPH3yAU25ZH934O//7d/n9Tev6fqWPK8vcSD+yY9fiZsACs6vznj38Iq7hxvqpsTHDu8H6qri3ZtvSSlzthHgZ4yey80G5xxjWcgbKCWMURgr3WUfAy5OmDinDWHFOGK0GJKMpjAF0+Sx1lDUFd2xYHNxye3wBu1KirqWhN5JFpU2mqQUUwi0/SS1ZMocj728Ca3cOLyP+ClSljBOssAXTSMnog9o48hh7oBrjfcSuhp8mCcCQg3WWr+P8ypLMVSVZcnFxQVn52cYa9lsl1SVY3u25uxsS1k6iqKQhagtOYmeoOsGTE4EL8rBnMUibYym6+SUrKqaGDzH4x5bi/MPL4IfkS4njHPwS53mDP+hW8B8qZ9bm2DKknyUzIGpHxj7nhjTHI8uTb2UHqPOkDJAScmg1BxoojRaQVLSg8hq/qxZTESkRM4a40qJIwsCZiFL07msK6zVTEPPOHTUy5qqamgnz+tX3xJ0w/Pv/xZl1dCPntWqYrWoWS0NftyBP7JeFWgFPrRcXK558/ZbvvjySz777DOaRclqu6RqSparJeEh4mNgv99juo6ykTShq6snrFYb/BQ4O7skRvjqy6/ZHQ8sFyv6ruNl1/LJJ5+yXn1AjJ7tZkUGNtslGNgdd0Q8f+Wv/i5lWfLq5Us+cM949/YdVxdXbJZbFtWCJ1dP0bqfLdENTbP4zuX3K7EJ5Bypl456UfB7f/AzUvKcn2/46MULysIw9EecK1ktVmyX1xROxmkmJzaLBcM4MI4j7UmirK6vr4QIq0AT0Vnm8MpElLb4tiMnTwSmoad0wqqLaKrlClPtcZWw/au6BjVhUsQmK2aUFPABQhTfw/HUiX9dCVdv8oFj22GdeMDdybLeCI14nPxcP0pJoJWBLKdvjKICDCGQFCIKmrvoZVlKc28WDG23WxkF+R5dCBTFlQUxJ+rlgrIsscbiqlvCsSXG+fofZ5Q5wGOnXUGYPNkV5Bg5HY4si817bULOE0rV+CBR62hFSpnHIkA6/7OcN2WhGec4k42ThHnEhB97/DR/fVk2Oh/EG6BmV+IjUOQxWeiXbxdaC84NICE8gJRmcAsSI6/0Y5q0gTkhqaxLqjkK3I89odTYpmS7XtJOitevX3EMluwacJanT8+5PK95/mzN5eWaUrWY3PHwcMup7TA20/UHmkXBqzcv2W63vHjxnKquadsTq3LDdnHG3e6Bh/2e+4cHUJrPP/+cjz76hO36jLP1Ob/1mx/w7es3lEXBxfk5P/7pj9ls1nzwwVOOxz13dzcslgtRjvoBu6q5fHoONqEtDGPHR598yNj1/N7f+w/43d/9XS7Pz/gn/5P/BLoIfPDDhq++/Jqvf/6GYYa3/kmPP7UcUEr975RS75RSf/BLH/sfKqW+VUr9vfm/f+aX/uxvKKU+V0r9WCn1n/2zbALjNPLv/Z2/xfH0QL0ouH5yzvWTc4zNWAtFYWgqiyIw9R1pmkjeU5cVdVnMoMwj+92O/e5BOvBGY8xsJCIiQLgg19KxnxNvRBGndWLyI2OMYC2mLKmXK1abNevtGYvlUjrbzkpzDAgxcWp7hinQj5Nc55PCh8g0Rdq2Y5w8IUQOhwPjOEKWLjcpv+9yP8pjH7MHpsnPC1b8BSFI80uhiQkWiyXOlZRVjXMFx8Neuv0qMYwD+8NBTtmcKcpKVGtIB/0xp2EaR9S8hMdxfN9089NEipFpHBkmT5g1+uPk50addOulDPiF+kQKA3k99b40SBLkMfQItFRm+CGIJHqaJibvJZB0XuTSf5CT/xdXA2YMmJqv/LL4Hz/no78ghZltoMwM9CxRRqS1j3yFwhmMUaicCDMifbVcUlUVX//853z++c8Yhombuwf+5r/zt/jDP/4xMSmqugIVQAU26wXBD1xenVFVBXVTcHa+5dkHz+RmOo18/fXP8X7i2bOnfPbZZ7x48YIf/vAHXF8/ZblYU7iKtu1ZLtZ8/3s/4MMXL1gsGp5/8AEvXrzgdDoyDD3Pnz9jvWpwTrHb3fLy1c/xaWB7vsSnkaKy/Pqv/4D1eom1mvZ44urykt/44Q8E7R4jm+2GzWbD5eXld66/P2/uAMD/Muf8P/vlDyilfhP454DfAj4A/h9KqR/mnL87AwkJwPgP/u7f5td+7ftsz1acn63ZrJcYBcumgZQx2hJDYr97oFUHnj1/zqIueXi0XXpxzZVlwfF0oDpUXLiNbAA5CENjvl6aGVfdjZNgplWNHwf6aSKGiLGOxWpJgcH3EkJ66jp08OgsnWvvvQR/FrUIe+a6Vmb96T0UwljLONNlK+fmhplgvOfv2i9O1F/aDNKcYMzkhT5sIkVR8OzZB5ydneGsvOGGsaeuK7Eb9x3H44miKLi7u2e1EqOQMZacFZOfsEqabHVdo5TieDzSNA3ljM9OKWGMoet76lIyFkMMOK0lQm2Gb+SZMvy+FMjp/dem57o+5cDUt5IZ4Uf8NEm/YZLegPd+nirI94EZAvLLpYEx4oSckABUNesG5PmzijAnYk4Er8lqBnFGMHNYawxyK1EKiqIQUKnR2HkDXi2XLCbLsl7z/MWH3D28ZZoCr1+/4adLww8/OeNsYbi8OGO13vD67TuefvCUq+trXFHS9QMpBYrC4sPEdrthsWhoVguKqiaQWSxXbDbnBJ9RyfDm1TsOh5YPP/qYxb7mcDjgCkeIHu9Hrp9c8+LD54ToOT8KmsxUln4cyDHTVBVNWfPu5i2LZc0PfvB9aldxttlwd3dDDJ7D4cD9/Y71ZsNm9RfYBP6k3IF/wOO/CPyrM3D0S6XU58A/Bvw7/6C/pK3m6oNLkoZPvvd9NqvVLF01vH71cz568YRhOHLY33IcHwgeVlNBeOi4effA8dAzdJHV8hybzzndRjb1Bv10SdQirTWFI6WJnA3GgNEF+12kblaMPhM82CmSfKIoayYsExY3RVQMmNORKozovgMfSOGM/lQTa41XQn+ZFAwxURqLj4bC1SzrwP3DwNhm9KLCas9isaLvB0bvCdkQsiIpjZ37G1qBMxaVYZpkbu6KiovrK4q6YHO5YYwDu9MDy0ZsqUZPvH79LYVzTEnT7u+Z2j1THymzJeBQriFHT/ADpilwpiJHjdj4NSEGlDHErNHHCeMVi7JgGCcm54UuPMM/lJGZenq8roPo1lVGW8jTCL6nMIqhG7EonLH4riWcdjD2mCRKzJQgAjkr4SnMkfEpyG3IFA4bSsYQmKaABMbIhmmNTA9iiCSViToixUGYScKZMIFKUfIQlMEnjY0GFcHmQGEz19sti7Nn+NOO/e1P+cf+2qeUruDdqyOHm8xHH15zfb1huWlYLycKo/ng4+9R1Ft+8sXXfP32NS8+fcGVMXzv2RVaFZwOI6vlFUaV3N+cmE4lOsto2IWaw5sDq23kapl5dnbBw87hUybEyHqzplSRulSsmzVtexAKdCmK0UUj5qC7uzsur65YLqQJvT5fcjfcsL7esCgNd/7E/thyvP/LySL8l5RS/zWEJPzfyTk/AM+RMJLHx8v5Y/+Rxy/nDpydNWzPzjnbnuOKmo8++h7t8cDQHXj95h2FA+9PfPvqa5HuZsvv/eHvUZULmmqNH8GYmmnM3L3bs2y2NG6NLQqiDqic8H5AKycAT1MgYRORarVmv3/geGxJ40TyEVsU+KwxaHLXo+sKV5XkqURNA0obulgyDoqgAslEYhY02RhlzjdF6VZXZUGK0LcT0WdyVO9vDfJ9kMmFuAnnJJ4819tJhEYoRbNc8PTpsxmdbtifHiirgto1TFPH/cMNu/t3bDZLnC2IvidOkf4UKNwCZwppbAZNd+rou5EwRYx2aGWFCpSl1s4oCJkwRZLJkAT3pWfX3/vjWylZjHO2AwB5Nv2mAHPoS3c6zcg3YS8SJkxKWCWfL86koPgIZlXpl8aTwgu0RYEZPT5I63FmnaKUIUVPComklZCNlYY54zBniF4yD62TkWvMkLKwCAgTNnoMNb4bOI73NBW8eL5l1ZzR3r3i3/13f8rf+vd/yiefrPlP/Se+z7Nri/cDZTHwk5/+iDEpPv3eD6m3FSMjXvVcnq/IKrNYlMTB4Kzm6mLL/n7H/f0tz59folTg9ubnXFwWnJ09pyrO6YaJh91B4tKzkdDU6Em+h2TQusCYjMmRuipQl2csmpL7sePi6opTeySowGKzpnYVl1vHcLzhtP/unsCfdxP4XwP/srwb+JeB/zkSQvJnfvxy7sBHH17k02Hge59csX944NtXb0l+IqcJYyp+//d/QggjD7s73r17wBoxxiyaJc8/+JCn1885Oz8jRwnqWG4cdZNQKhDDhMqath1YLbaMfY9daIL30jNQmbY90LYn9CjJNWWzJGWom1qwW87JhuIcxjm0jWADPsV5XOXlFIriUQ8p4WMgEihKi7aKcZrwQa72fS8KwLIsZh1CIs4z7kdBUZYVKT8kaykKR7OoOT/fMgyS2Pzpp59w9/o1MSbevn2HDwGlzOwtMPgxcDyd2K4rtJZ0X1tYQaqfTkQv7su5qY9WmjF4SBKD/pgRaZylKEphArriF0YeIyEgItqRmDFSIqUgY89h4ng40HUd0zgIsvvxJvFL2oI86wDSXI5oJZkI75uBSkmT04kPwPs4j1AlHNX7QIiRx7NOGekrKCVuPVQCM5Og536MTEiE56fnIJSu76gXNZ/88Le4vr7m+uIZ714GDqe/x5u7e949ZLQ78p/5J3+Xp1cb7nYt/9bf/LdZbbZcPdxw8WzD9nxJaFsGs2NVLTjuXtEeBgpXU9cLQurpphPFpiT4Eb878vqN59QOGFfT9iOHY8tqs+ZMb2gWMhUqXI01DucKdvs9Dy8fqJuaPAu0pMcy8Xu/93vs9jt+8P1fp9RnXF9ccLb6Pne3J74L//nn2gRyzm8ff62U+t8A/9f5t98CH/7SU1/MH/sHPpx1kB13Nztubm7oTh1GK9rTgeViw27/DW3bslxe0Z4OkqmnQKNJ0VEUNWUpwaJlZVhtDKYYGYc9x74HDHHKxLImpQk/iebcWYWfeokGs5qgNG3XwRzWYYuSqmmwBwFhYCxZG5LWKJfx2ZOjB50RabtCaSvNtRjop5F6W+FK0ZOPk8cZmcXXdU1ZFIx+QM5DiMHP9fFj2y3JqaxgGHpiDCwWC07dkbouJaTyeJTx1+QpipLNZsM4TJSViGLGcRTF3azQKyuJ0WqPrSwUrfEhShS5Vu//X7tHyEhmsVpR1Q1FWT0W8O/bghqBN5F+ye4bIuM4cDoe6Id+lvkmxmlkHEdhDc5f5XthUZIxb4qJqCSD+tFf8ZgZWRTlzDGcCD7+h/UJ8+b5qL3QzL6FWSFqjEXnPPMmH9OPHsePirJwRK+pypLrqysUipubG25u3hJSwjZL2nDi3/v7P+LJC0GGvbl/zan3tO0bjM2slzCakW3d8NWPPpd4NlPw8cef8PNvviazog8tl89LTt0b7u9v2Ww3MFV0w8Dx9oG2G1gsN4QQOBwOrFZPaRYNKUU5qIwCEvvDPbuD/PwmPzFOnruHe3LOlEUpobpWo7Ti2eUzjDl+5/r78+YOPMs5v55/+88Cj5ODfwP4Pyql/hdIY/AHwL/3p73eMEw83J1oqiNlWXM4nNjvHpimkX/kd36X7//wd/mDP/gjQijYrF6wfzhwcXnOixfPuDg/w1gh5RelRquAj3se9gHflrSDBzSb5Rn7XcIoERdJQy4w9B1+7KhKR1dIhFUEirIgKyVhpMUDyjlh69sCXEBirCZCnMT3jiZ6MChMVvST59C3nF/WFHXFfn+i6zs2q4XUzlphnRWYBxlnDcMYyRpQCaXsfO1NhBiY/CQ4rTDNZGDNH//RH/H08gkxjpRlyWJR4VzJw8NhHkOm9yfpMA4weorCkMlyAs7z/piiPNfYudWv3nfeJx9YGSukXm0IMeKUfu8UhCSEYyWOSZ0lMWjoO9q2xWpBcY29nFg+BEIUoY8scnh/FUF+HaMoPvN7q7T8mdYSh+ZcZtQT4yj8AWPMvBGIOEfnhNFgtSLN/zcKFLLJpGCIIWKdAFSM0ZjCYaolPg28/PolbWd4++od//a/9RPuHwKDKymqgm/f3fD//Jt/D20XjKcTZxfPOTy846/9o3+N4/5bxt0B70ve/fyW/XHPi49e8B/7q3+dL7/8QkaR7ZGr62sSGVs04k/wmXLyFFUNdw+cn29YrdeEFLFObl8xJl69fsVy2bBYLGkWDW03o8y1ousG3r274YMXL1guVzzs7ji/XHHq7tE2Ycr6O9ffnzd34J9SSv2j80/nK+C/DZBz/kOl1L8G/BEST/Yv/mmTAZCx2B/9wY+5vLjkw4+eMvmGYexZb7c0yy0vPvw1vvnmwN/+9/8uddFQVhdcXrygqdfEZLi93XH/cMPl5ZLlsuBud+JunxhTSUgWZwsM0HdvWS83NPWSGCOFsaLr7w6C+46JpDTGOarFgn4YMc4Kh0BrsjZgCrSL5NwRg3jaNZqUDVMEnRSFNkwp0Q4TSRvKuiYrzakbWCxqnFbz/F/qa60U2lr6fgKtyJr3cdqSISCBm8tlg3OOpm74+c+/YOg7VqsVr14f0Nqy2Z7T9SNt2+NMyThN79/kw9Azji3Ozhl/WcJR5twIUk5oLRmCEgoi8/8wTcSUmbykL8UEzEahx0Vn5gZhjpGsxOY7zSPRqioZ4ySbzDydebwBpCQtvFl3OLcZZPyYY5q9E3PS0HyTSVlCUY21hG5AKzPzBwxqphIrMk4r6sJh5yQirXnPeAx+IkxzOag1KQV0DjR1Ba5kGG75+us77m/uOR53jJPhNEbWtkbZFT/98gYf/l2eX1/z2z/8jB8++YCL86e8ffkF3//0A959e6DUS2qX8V3ix3/4ORfbp/zopz9F6xVff7GnqGo++OBjUsrc339NinD95CnWCatxuaplWhA8bSc3p6JwwoVQeWZtllIGBbkVlWXJzbsbrq+vedjf8nAwxGgZhp7Li4///JvA/y+5A/Pz/xXgX/nTXveXH1VZ8ekn32O9XLHb3XN5tUVpsM4Sc6aqV3zw4nu4v/+l5P2ZkrbzVLVnvV1RlQ1td8v+cKQol6joGYaWQ5uxxZL1as1hf8fu4UAYB7piT86wWa3ouo6H+xs26y1t3+NjBKOp6pppDgJFa0KGKWc8iqiNJPsGA1GBKcnRkqWfhs5JeIQ+McSMLipsUTFME6P3FItKVIta46zFWrHd5hxRyr0fkaV5/KURGW5VlZydbXk43nNzc8MPfvAZx+OJm3e3GFvQ1Ave7A/krGi7nmGYKMp6ViV6hnEUJ+IszkkpCRNwjmfPSZpwWX6OGDMvwCzy6JRFCv0LjeB8+sLc0JRrdkqBlCJlIQGg3Vzry8jPoY0l5UAmSOOTua8wNwPlaxdZsexRjxuA/LuU1vPN5PHfLWPVR31EnlkNhdVYo94bmmbXETFKE9clZlCLhzCSYmBztmWxqfjq61vGcWC1Knl92zL1E4MDUxTYouTbNy1Df4fKFf/cP/vP8O5di9YrQqho+1ua5Qqs4fLqknfv7vn13/wdtosDrlji+wfGfSZuNxyGG27ub+jaHmWzgGW0ZrdPNIsFDw87iqqirErOL86QbMZMWZWcXVxydfWUtu14+/aG1WrDm9dvOduc8/bdKz7/6Vecnz3l+fMPONv+xXQCf+mPuq74T/9T/wTLZcnf/rv/b0Z/4IuvfsLzDz7m+slzxmlisz3nH/vH/+O8ffUtYWypFzXnFxesN1uur1bEdEHb3qPNKM0s65jGA9aWAi9tD0xDS+csdzdvWTYLmtLxcH/L7uGW5UKMQyFnElBUJdv/D3N/FmPblqXnYd+cc7V77X7v6CNOe89tM7Mys6pYrCpVFVvLFmlIlgXZfDBMwy8C7AcDfrBg+MlPerEBPRkQYAM2YIi0LYmkDDZiUaJJFqvPyva2p42+2X2z+jmnH+aKc5NUXbLMKhp3AxfnnrhxIiJP7jnWHGP8//d7HrPpFD8KEZ5HbQWFtdQWaiRaeA5/ToDBf/vGLU2FEIpNVjNf54TCQ3i+iwVLc5JWTFVrB8cMA1ReUZYaa4zDaCPe7r4NoJR8CwDVRvPJJ5/Qbie0khYvPz8jzwtOHhxjjAOfSOlTFrlLEU4CDA2Iw9omS+/edNNM/KVwPbOUeEK5nbs2BL7/dhUolEcUtfCj2K31rHFvHutk3LZ2aC9hrWsLhHAKRu2YCPcaCCGkC3uxAOWX/bypm8PrbgMIdz+4FxPBvYxYuW5eKnzPb4aCzkOgQtXMFdyfubcey8Z3YBuHIbYpsAbqqiaIXOR3rZ2QqqtCdnf2mNzesd2uydYr6lyQURMkLZTsgFLcTgvm08/Z3zng4V6fYTvhd7/3iijOENJSU+K3fHwCkiTm6PAIWwf0WvuslxWtIGFZTFCBwqslfiiJQzfEzfMtNzdzVusN/cEQz++9NVxVdU25XjcsiTZxnLBabSmKiuPjE46Ojvni+XO++OwVg77kl37hf8jh/r+mVOI/sZeAxfyOIBzQbofkxZZBv00UewSBIi+2ZHnaSG9HoBPGOyPaSYw2Na9PzxCUDEcJWZ5SFBW7e2OsjRHSGUZubm8RQmN0SZ6tMXXJoN9jvVrgKXcgkBLZqAKjVoQUivV6jRd4KD8gLyvyytlY81pQatmo8WKscc41a3mL/drkhskiZdTuUCMptaWoDFlRvX3CB1GEl5aYbYmQ0r05cY7a+/EZuIisqq65u71jPp3xjW99xNnpGfPlguF4zGAw4ub6EmOaBKTC6efbSYc0LTBYlOc56a9Ub5+4Urn2xw8CVOCjrcFqQLvrehhF1NrSjhN29w9QfuDUgxZoEoOtrqHRN7jA1RJfKXwFWbF1sNAmwsxgfwof5n6K+6g3jG3CY7/MKNSNOcjzHPTVNopiISXS89BF1SQZi7ejhftWwynEXNSfuR8iQnN7EGAcx1hKkNKC1W4YWpScHD/A6Iq7yzWbJbRWBdsid3qFTQF+iB+EbPKav/Prv8n7j494dLiLEprDRyn9bhdVw916yuHeHqiK0aiFrXzKDGRVMhxEhLqP1zlC64qnjx+6lkRrFssFz1+8wtgS5UGtS6q6fiv/TbcZV5d3HB4e0el0MdrS7/Xx/YBO0ubk+BHXtxtur9f4XgddCb7q9bUoAgKoqoxax4x3+qzWll7/oRuU2ZIsW6FkTZrOGAwj8iynlUj8UOB5EWm2ZL1Z4wfO0CNVizDs0e0GjaouJwyDRlpaMRj0mE3nLgC0yBmNHCPOSk3civADv+Hqa+cgExY/8MjKgqKqAUGpFZUNnKLNb1HXGiF1kxTkoa1mm9WkuWHUj9DCQwsPGURsy5qsyOl3u4RRQBCVSJW7AZoxaEGjIpTOFK0NVV0zmUzJsi29bp/lfMlnX3xBr7vD4eEx0+mUm5sJSdKirlPKssb3A6I4YrFYOS09HmVVYZVz2N33/0I4CEsUO0qOqSs85QaLYRhSa0PS6dIfjR3q24JAYq1T8Ln1oH3LKNB16UhDunRMxcYpWVXVW4uRW4c6mXGta6qiaJyHzVtSuISpWpeuQAjVYNoadWYzSagaNLkbWroiLHBGI6GaHgeaAnLveHSdga5pOhqNNS69qBW36LRjev2IRzwk/1M1H75TMrtd8oMfP+dynlFoiTWCrHLJzEaGPD+dcHO75uhwn3BP09/vUqRrlPBQUczV7TWh9IhUSJ5lWOOsxJ5s45sT6jIjzTK2myX9Tpt2EuN5AiktVVUwXy1RKmK7yd7i5qyFXnfQ/B1Dvz9gtVxz+uaMvZ0dfu7nv80nH3/B5eUL1qv5V56/r0URAMPxyT47u33mC4tSBq0r6mrNejlDdhWjQZe6bLGz2yXNBFEUIIVLh213HpOlzhRkbUUYKupKMJnOMbpiMpnw5MkTB6OsHDJbSMF8saAsK+LYmXKqeuve9HVFWeXNm8W9U4RypJu8LvFVSFl71Nr1qdqA5wd0eh6L6QptLQrJJiup8TAyQMsALTy8OKHWBbqsGfkBnvKRno8KfKQ2iKYHd4ARC/fW2NpwdnrOaNynqgpubq9ZLVe8++43qWvNx598ShxFeHlFq9V22PUgYrtNqaqSMApZr3JKp7Zpemg3YHNtuSAIQrKycJAOC1YIirKi3e+QtDus1lsGgxoZONefEs2+/X5wp2usrlFKosuSzXrdYNElm8222VYojDSN0MgxFRz5uEaKxhzUrAaFchCToijBSrzEd4NC/VOz5uZAC2EQnmqWCY1Y6R5uiPs5EU6F6SLUnFOxKisoIfAqlBLEUcze7h7p9orNxh3q3fdHiMry8PiA//ev/xYvL+YopSjqEusrah0QxgmzTcn29S1rW3B9Y9kZ9XhwNGSzDYl9S42GoED5hv7QpzYzSgMaH4RhPr+j2C4o08zxCSonY7+4vOLq5o6dnWN63SFVlQOWbqfzZQsoFOdnFyRxi8lkwsPHD3j4cEQYGs4ufkyvO/zK0/f1KAICPF+SpluEVHTaHbSu8WTA1eUtQlsePuzR73l0uh4y8F3sdFYwn8/p9waMRuPG6lkQRz7rzZIoSthuVyxXa25ublENuff07IztJsOTaXNj9NDWo6pKkiT+0sGVJA2q24VYdHtdlquN60GtR1ZohKiRasvuzh4Kn+V8hhW1a0Rx+XVZZRCBmzxb5WN0hZUK2fjzhZSu96ZRvDV/KY0Ex03AheL2boKxGs+TbDYbDg+OCIKI169PAZeyNJnN2RmNEdJx/ZerVSPndRqAutZ4Ur2d8DfAY2hs0bV2B1WYxpxjDF4Q4och2zRnIJX7Xm8dD27H73byFVo7qW5Vlq4wa40uK7ZbVwSEkI0VuckU1AZj9FvvgeMEaKfme8tWcF6MIKjxPP/+uzY4Mwlo7nkC3BsQGy8CzeAQKfE831GdEW+VksbkLkuyLpvo8AKsYLPeMptO6SY9hCg4Pt4hij6i0Ibf+v1P+fTVJdrWYBWL2RLdHSNFBNrni88KZjdrjg5jqrRGVIL9cYI0KXIQ0m1Bupmxur5Gxz1u1in9TkxdeggRUeQG5Vl0DYvFhhevr8iKksPD9wijDq1Wn+PjY6aTKUVeM5+twFrWqxXBvu+KQ7PKPTgY8erla9L0q8PBvxZF4H5ffXl5Ta1LwkARh05nXWQZqmuosiW2WlMUhtqWbDc5V5dT1sucw/1jpFCMRyPyoiAvUsJAMRzvkOYpQRCRFxW+Z1ivViznK4IgxliQyr0x0iynqkqiKODmZs5kIpBiB10XbufsKY5PjonjNh9/8jnKiwhDS61TVusFJyd7mLpE2y1xyycMJZ5K3HAMiZY+WvhURlBWNbbUDl7iRyA9hPQQykNXGiNEk7brLNCh70Qyi/kSrGU46qO1RQrFfL5guVrT7fa5u7tzQaYNqkwbQ5qmBEFAlqUUZXF/elANqKOua6SnqI0mLwvXIiiFrJ12/75lcIXPGZ+EdIxC2yDE3FBPUzcyYaxuVoTgex6rxZy6GdJB4/7DHU6tteMtNusJ3fgFdKMhcHBSR+PN8owoBBc4It8WT3EPJRHSzQyaxaMrOO4WID0P6YWuXlmBEkHT0rhtQVUV5HnGYj7n7i5xw1SjyfOU+bJiZyekJuPx412CKCArVlSiJu62eXV6wWx6gZAhQdgl8U9IlyGv1huuXr3ki59MOdzt0usqfu67z6jGIaenS8pqw2C/zc1djjQRoUroJAm62jTR9oIis0gRI6WP57c4Pb1hvVpxcvKEo+NHhL5iu01ZLRZUZUWeZvR6PdqtDuvtmizNmVyvieP6K8/f16II5HnBzfUd1hpacRvfUwTKZ7acM+oNCT3J7O6Cqq4gKDG+YrNdkxeZiwOXcH5xznR2x2o5o6pyHj46wQ88VqstSbvHk6ePXWDDZsv+4SECD125vrCT9NDWcnv6hijyyfMtwiYUWUpdFpRlwWDYZzpZkrQTgiBkMStpdxL8IGSxKBiOYkytue1JlBDs7Y3odmKENFzd3bJebpGmZNcMQXrU1qXgtJXX6OdBej7aFo1X3gFFjHUqxKoyZFkGuIKU5VvM6TkH1oVrSCnZrLccHBxwdXVD4Htv9QBCuFTlWmtnTGpswdgv9/V5XhBEEe12m6IsMaZCC+3YDdaS5aUbCr59/jfrwQb5pXVNWRYOJ9b4KILACbOyLMNTihqDsdrBPKE5gLVjA2oXjuJWgc5KqK1BWDfQLKuKvCiR0nNGIOseHKLBkTlu4b168cu5gJTKJRwpD1TgOJHaoK3AF196OI11EvMsTbm6vGa8p4iiCF9KZAiZWbOtNuTlmk5H8Ys//4TesEuQRPz+Dz3+4IdnrLcleblitbqmihKisEWGx3pecHE2o9+PuLrbsrsXI72MIATRjhgMRmBqpFdT5imX51OMzpnMlmxWNUcHDyk0bNYVlxd3LFcrfvSjT/nud75NVlUMhzvk24zhcIhSHvP5nNvbO0ptydKCq7Mp77zz7leev69FEdBakxcVB3sHdDsd3rx6zW16RysO6LRb3FyfARVh5FPJilxItuvCeeuVY78b37BazinrHERNWaUsVwWbbcZgOGQwHAHgeQFFXnJ1cY2nYrabHKM9olYMuGjxMPTpdNoO0y0cYXg0HPLyxRvqCo5PTnj+5mPixGd32OHgIOLps0OEtQS+yw5wvn/J7e0NF1fXZGlFp+WzXG+JlJO3rtOcbtdQVDV5WTrACPeadtO0tAKtrYsWt4K60lxf3+L7Et8PuL6+YbFYYK0lCmNW6zVXV1ccHR660aJSFEVGUeS4K7ij7XiNv8DSKAa1k9eGQeTwZEK/XclpbdDGUYH1vST3/vre7N6rsmiu/zVKuEJltGa7StF17W4Upn67KjTNgE5rTdmEszrIihsa3k/7jbHNUFM0hcaJfMCFr9xzGrUxVLgtgbAW1QwWlXLaBCEVVkgMjjvgvpeHVAqUaWqbG1IulkuiVgDCoJSPihR3mztqCZ1hxPR6xoOjLqNxm7zO+OjZkFYMt9OK09M1dVawmM0oshZSdshtQF4lpFXFm+tr4o5k/7hPknjcLl5wvHtAIEuOdtsIs+X6ckk78UFHWFMShwMeHByT5gWt1oqDg2Nub6b89m//Lu+/+w6z6QStHWF6vV4iBLx+/YrDowM26ynGZETh1zx3wKGm3Bs/S0vmsxWmLqnynNfzG9LNnKOjMbq2UCkKIyjLEhlGhEmI73uoQFJXEaGBLF+z3S7ZpjVJu4OuDZeX1wTNE2GznXN5fc1773xEkWs22y3S8xgNB5wcHTnve5GDdbvyqirBGj766ANWyy1lLdnfn+AFljj2effdEzrdkDJL2Tvo04o6LBcrEJZWEtIb9PD8GklFVpRYpTGNoUY3/MKyqomioPEeuCHbPXm3KCrKqsaTbjW2TbcMwh6eH3B+fo7neeR5gRwozs/OATfwcrGAgvlySVXVTvyjHZDTVz5+4L09jGEQIVCkaUatmygx5YqEMS7xyfMDpybk/gnqZMNYN8HO09R9z8AnikJWi5TpdAqCLxOUmmGg/albSF3XWG0QQXMDaNKF7gEjSkqkFE2eX+HmMNJ7O+OotTMU1VK5VgrjNgPy3kzl5hBCSTwrMULhcd8OuJxKXZVYUWBlRWQVeV7QbnvErYib6QWb8o7Ia/H+o2+ijObm4oz5dMb17TkVklYgeXIycnFnkeWLz87Ic5+bm5Kz8ym6qlnnUAuDqmBebp3PxZvyRXTKoONzOI7ptAyhJ0jiiLgVc9zepTvYod8/QG6W7Owe8PTxI8oy5+L89G0gahwHRGFMp9NmvVkzHPYYjAWffv6KbreHtl/zmYCpNdlmxVm6djMjWSFCQ17XXEyuAdB3a4QU7B5G1MJdNzflnGG/i+/VCARxLLDGR9cCU1fUpSZJ2iwXKen2jH6/Q1Fsmc8mtGLFoB/jS8XN9ZzNcoUf1rRCifZ8FospgefTG+8Q+pYsnfP+ex+SpQW///0f8nO/8ISr63N6owAjS9JyS1mmtDoBStQYu6Hf6RL6HVA+i3XJejFH+AItHNdvVdaUyqOSilKot3FYZZpjaAI6PA9tBbVQVKZCaesizbRhk2asyw2RiMjKjHadkKYp3aTjDDbaUNcVWVoihDvwVV03rQZuDmGcGsEPYmoDZVaCEngYlOcGgLq2xGFM4Pl40jp3pqlAlwhj3UAw2yDqAi8Mmog0y3K9oag0vhdgbe4GdNagq5y6zqlqBzqttW3ChQTSCIdfb1DsQlokDgduhXVFs66wwrkutZWNH8FSy8q1B4rGeGVASfd7YVFS0IoCAm0xWmBN4R4+mRN9CZ1h5QZbQV0qPL8DniNOx2EbXdYuzyZUVLjAnCAc0ks6qEXBNrMc7o/54KOAb304pqoCPv/8mn/wD36PrFiw3NZMlilVrdBGUeYBufWYW0Ucenz+whKFHseH+3S7gjiJ2TvYw3RG6E1M5AkenEQYKxmOhlh8skKiZYuwPaC/t09epqymG1Lr8/Lylje3E94bjBBJ6yvP39eiCFhryTdLFusprXaIlTWr7Yajk4eodkRVK+5SQTvpUtoIYVM8pQhCjygUCFG4SruaoISkFQZIIZmXK1b1mqp0MtZ8myJVhacM7zw5xFMlcSiJg4CiyGm3FEpWWFOTbResK0MrDtjbHbBerynLOZPJDXU94cHDY5LugJ29HV68/IIwPiBoheyNh5RpSroReCoHv2b/oI3XzgmikjIvMZWFOCSXgtRYlmVFIcBTAoRPUWSNN96SxC6FuRYSIXxyI8AqVmlJrjU29JBhAEXJauOcgVpbiqxonpQ11jo1YFGlVNqRhu7VjaYBAllcKq8VTh/gCdOAUD2E8InDFr5SCGowBVYXSF0htMGWBTbfupjyBp662abkRU2728PWJfl2g68UpqowOqMqUsoyIysKitqtRsvS4CuLMAJlBMY2NmBr8aSEhsZUmoqirlxhQzXDVgPSoDwBzabF6YFcIKqwCk9ZfAnSQGELamGxtcDYEKl8lNKIeovOLVXRIU0N+Jb9vQdglsymd2ht6I9H+GGLF89PwYvY239Cv685Oz1n3BsyvfkxRVHR7Qx5dCz4xnsB3f4hN5MNL95oXp9NsUKhK0laOKn4OgvwvTZ66XOzmSPVljCO6HTXJJ1T9vfHHA89hi0HXe30uyxWCxDQSjpsiohZ6vH6zZQwGrGultQFVPKA11eWoP2vByryJ/ZqtZxZYrtN8UOPq9srkJIHDwM8FRIGLSBgZ2efOIKyKCnzjFYvoSot68WCoijYLNfUdUU7adHtdJFKuH7TuBXfcJgQxwFVtcGimczvKFKBVBF+5CNCgfElZ+dXzNcr2u0On7z4gqOjI+JOh1Jr8rpkd38P6QsePzoi6bRZLnr0ewnb7Zo0XdFvtzk63qfYbplOpuD7YEqydI3RhkGnhy+tOxzFhijykIM2Rhu8QDk2gbGYhu+flSVSSMIgJIxj8s0aYy1pWtPZH9NuJWRrQ55XhF5AXmjKckNd1y6gVEq0KSlqp1SsrEZjqYxu1mdNnHrtnpwGi5YGhRu8dbodByRp4KjmPl1ZCkcMrnUjxHErRdHIf9vtBF8JtssFSgm80Kc0pbMfawdXdWae8u36Vtgv03OttRi+RJEjJcI46Wxd66aPVxjj2inlKxQNn7ARJBljqHGkpvtWo6q1SysyNSBRDXAFaGjPAt8P3soMup0em9UGXVs+/fhT9vcOGQ33iKOIu5sLjDbs7e5ydnqGlII3L6+5urrmF37hlzElpJuUKMp4+uQB3f6QVucNKmihgogsLZhNVkwmK7TeUuSaRTrH4rYyl8KAqAmjgEEQ0o3clT+IAoQUeIFHnMTErZjeoMt8MafT7bikJAXZssN2rpnfvvzK8/e1KAIWgxWGVtJiudoQhgkWSZEZBwLJ3ZR8dzwGUTDNNyStHkmrR7otubuZEgQBWguq0rKotmRZjVWSOGlR5BVeEHPy8IiyXvP5Z6/Yph5S+AjbYtDtIT2frVjz4+efs1ltyYuc3BomtxO0kjx99IRtmRN1EshzlGfpJAFpuuDZOw8ZDHucn9Vs10ukrVBYtKmodckmnZFEHdotn7qsOTnZ4/TVK7ZVTl6sePz0iGyb8+LlC9r9IXK6osgd0rzSmtoIBxOVIVGrx2q5JVCKotIcJHtuVSdbCKUxCNKixFlyK3wcsaisCypd4UlBIAQi8LBKOiu0VBRVgdVO72+wLoHIc6Gr3U7XtQXaSXhpkD/WWHRVUWuDUB5CWYRUDTAlpJ20EFZTec7RJ42AKsPWNWXmfpWN3FgK0fgQau4zDYwxzbrvXhHcqP6EQ4rVunYafeOyKoVtAlHu/zxOwqCEamzSoiEcO12BVAKBwvNUk+34ZUakQBL4Eem24OY6pdsO8KTPi5evOD+74Vf/jV/j5MEDsrQkabd471vf5LPPPsP3JKP+MWWqiPwO5XaDEh5np68ZjsaMhhEPqyHDvT1anQ51rpnfLbm7XeD5CVUlmU5XrDcp682ass4pypI0X7AtehSrkusrFzAilUT6TYCrcgXB832M0RRFRaBit/6sqy+d2n/I62tRBPI8Iyu3dPtdFqdrHjx8RK8/wmiBDjy2qxlSSbbrFcaW5HnFzniH3d09Jnd3LJZbOokgDCKkEqTbDbPZCq+leDx4jMEQ+hGamtvJDet0TaU9xsM9TC0o6gxra2Y6Zb7d0u4kFGXG5PaGLM1orVc8EAZtNdeTO/Z2dlA2I/Jhudywu9tHVznWVlhbs1zNyTZb8q27nlsNgbR88O5TrDF0O12qfIc8b2NNjRcKWiqg3YsZD/aZrzNmL8+orSASsM1y2kkHjaTVGVBf3oKWlManlYxZr5YYG1GZAitAeJHb2QvT9Ns1ZeVaosD3CIVC+IFDrltDKL3mqSgIVeiGfhKCMKIVt2l3Onieu1FUZYHnuwyHqtaYe92+F+BZiecHhJEb1ko0Vb51wNIkdi2AhLrIyTebJrHZxcd7nkQpJwy6jynXpm5aFfFWmVm/lQy7HEcrNLW2jd+ARifgoEzSgrUSqXyk524C92nQvu9jPQewRXhvAadOsyLJ0gyjIK9L0k1FLxmRtHqcHD/i9nbCzfUt77//Ee+9+x7DwYiiSAl8ie8JHhy/RzveYXq7YrGY88F7HzKZz6irFKwgDEvaSU0cF+BBP+yy2w+J4i7tZMh6nbFNHT0qzbak+ZqizAnUPlUZcHNz40JjspSiKthmW7bbNVUJYZgABikqilQ3hfRLy/gf9vpaFAHlKcKWIk0Lh4aSPt/6xnf48Y8+RlISKI8wDlgv524g1IAl20kHJRQX55eURel4bsbieSFClBRVgbbu2qit5fzigizfOpWeJ0m6bW6vlpxfTKmQqJ2E4e4uy8WCvNa0BwP6wxFPnjwhiGPubm9J2h2kJzFZga5y+p0EXebcTiZcXV4QRxEY21iU51itSVp9VrMprSggSRyyejDoIkSPxWLBJl2xs7PDwyfHdJJdok6PEsP1zRQ/9vFCj02WNvFoEXG7S5EVhEno4trnS9apSxGurcb3FaXFhWxql8JUaWeasVJQo6hw0t5aG+IgwGJQShKEYePnL7BCEEQhXuA34A5Nmqa0o7CJCL/39gcI7aOEeosgw9MIE5LVGSIK8EzAqtw4dl7TMkggUArjeQgpUKpZjzYsAW00xjTXeqCsNGWlqY1FWyi1droDKxBKglTOdGRoskdcbiJNMpFFIjyJ3wTU1JRulWlcUXHSYqiriiwtSMsSLd1Qcnq3Yrtd0056PH0yZLlYsl5tsNaSZSk/+Yf/NUIYqjqn3FToSrKcpxgjODrcx/MltS2pbMWoH9JuWaJYY4ShncSMuhFpWhLIDcOOot+OEbLFfGFod0eMRgNq26cofVarFd1el812Q1mVvHr9krOzU3b39xmPRwShz8sXr7l4fUmeZYx3xoDl4m9P/tDz90eBivxfgL8M3Fprv9F87K8D7zWf0gcW1tpvN1TiT4DPmv/2W9ba/+BfXgYMldlisLSSiOl0ymq1ZrPeoLWmnUSEoYcVmuV6xWhnzHg0cuKUuubo6Iib6xuyLEVKSX84wAt8VvmMsqrxvZCqNlxcXVIUK6p6Q7ebUFuaLLiMSii6oosKQk7PL2i3Wox2dtiu1nh+wOXVFXc3N/yZX/01Pv/kY7LZDZ4U9Pp9ri+vmS3mZFmOEh7pNiXPSjw/Ii22VGWNMJp0tUEJSVG4Ch9FLtBDSsHxwxMmkwmIkIOHD7AeTKYrtmnJg8eP+YPv/Yh0m1FjGO/vcXd7x87uLnEnojQFq3TJznDIarVBlK7PN9ZSGu0Ohgaaw9S2itIoJ3u2AhnEGF0hlcAPInTDRyzqurl6a7TRpNsNUeSwYxLHJPSUj/FqhA5Q1jR5DwqMwfckxlMIT6IFWF2hqwJpIfR9N7REYAPfmX+wWElj9HLaBG1w8xErnJ4iL9EIKmPefo5F4HkBSNncGizaSDyaYaL0AOVMU1I4CAwOqqKtW4feQ02MdnL0VhDihS7kJssqLi5XpJsV3W7JRx9+iBIBr169oSpLdsZj6qpkb38PYwyFrrBW0W73iGOfKIwR1uIrSRzHRO0utdAEypDVGeu0II7aWF0jjGTQH6ONZbNZURcL4iCi05LM0zWVsSRdSdLR9IdddnZ3ePCox2RyQrfXZTqb4Xkevc4j9oZwe3fLYBARxxF/42//4afvXyl3wFr7P/qpIvF/AJY/9fkvrLXf/iN83S9LgNVokxK3OqxXGcvlnO9//3t40kfXJXHkIUSNH3jM6owo8olij8n0hjTNCIMQLxBkRYHQwjH4Wi1MBudnVygVoLX7S50vbjA2RxvBZLridrpgvS2pkGQ3d28HQM+ePiNdb7i5uqEdRiync44ODlAIOkmb9BZmdwuWiy3TxZy8qkjTElvnlEWNFBE7oy5ZK8PkNVVREgUxdVFzdXlDWZcEQaOKHI1p9/pM5guW6zmL7RoZ1PzZv/jLfPH5a7qdEd1elxfPX7F3uEMnafP8ech7773LOp0SxBYtMkSoyaoVdd1g07WlrG3jyzeOrGMFtQ2ocWhyJQRCRVjjpungY7QhStrYyjQBHq4332zWtHd37728Tt8hnMlJ+iECgxKu13bJQBVKWGgOv9UVReago0pIKtNAU4TE4th/96Yi07AUaiOaQFJLUdVkZY0VgqpJHbpPkBbiS8uw4wyCFQohfaQKXEK1kM3coYkva/IC79sfRyVyAzlrLL4KMNRM7u6wZounBGFkiaIOngrJi0vKKuP1m1PeffaMvCiI4xikZDQekiQRUhpaLZ+dnR2m8xvGgyH9nR4vT1+SliVWS9aLFVlQ4vsx1qR0uzW+7xG3QkbDPsJqri7OuFtpVqmh2+vy+vULojjmPf0ug8GAk5MjttstL18857PPPufZOw85Pu4zGjlF7cmD4688f3+s3AHhGo1/H/hz/78c+n/+paSglQS0kzZWK9qxReuCXjshLzRa5xhbkrR77Oz2CULBdOb6dZezZymrDG3c3nk+X7Af7KOUz2q9pcgzBIp33/uA5WrMD374u+S5ZbOu+OTjNyRJFy0k3ZbHMq/oJV32xnv8wetzVvMlwRMfXVa0/JAXn31BN0kI/Dabder6tarkbjZDKEWSjBBCsllv8GRNtikxRYWvfOpKMF8uuDi/Yfdgj8Fol8U6RfkR26xE+SHr9IpOt4sXGobjNuUna44efMDh0SF7B2MEikcPH7B7MOL4+JB//Jt/hw++9YC8mpKlGWEiqNYVaWHISmd0kjL8cgdvDVmhnWHKKMq6otACaxVJ3KasQcqQIPbA185XULtcvcOdXUxjRnL5CC4yzAANDYXAb/T4dYGuCqoiQ+uSIk9ZLRdMJ3dM76aNJ0BSlrWbLVhLbe3bpztN36+toNKWqtYUlaFobgl1Mx+4N1zVxqAapaE1rhC5VsBzMeo4BJm1Fl1pN1S8J/kIdxu4x6g7paJLgfI8pzvZbkukMIzGB9S1ZLFKiaMOrbjNb/zGP+bZO++y3rjUKXTNcOeYm+uc2WTKcPSQYBtijORg/4RNvkHYEKs1223Gzv4DkqTjYu3zilIbsiJDmxqhfKbTBZPpHZMl4LXpdXewuuD89JbryxlPnzzhvfffp91uo2hxd7Xk0bFAtgMeHu9TlAWh969PJ/ArwI219ouf+thjIcQfACvgf2et/cf/si/iBz4PH5xgTcCof4DVPtPbJaEfMBgmnJ+/wvMsiIper4U2BXd3M2qtiaLYhYqGHm0vZrVaEychta5oxW2S1pDLyztAMejt8c1vfovT01M8FbFYZixXW8pK8cGHH/HwvWNm0ymb1Yrnnz5nPV8x6g6osxJT1Nxd31IWBU8ePWKzrkgzzfXtlKKuWW3cXreXaqrCsl6XLGZXhJ5PKBXJIEGIkLwwtJI+R4ePePLsKYPRHvPlnMl0SRwn9AYd3nv/HcIgJggsUtZEISStDj/zM+9xc33HeK/L/uEIY2uSvuDJ42PaHcmrl6/pD1q8fHFGUQoCE5AVkOUuysxN+hM63QEqiED6ZGXGNiuxumQ82kGbmrjVot0NKbKMsqooF3P2Oz2qsqAs8sbi7Aw60pNIVbpeH4vyFOgKW9eNYCunyFIWizmTyYTNdguq4f836sWqrpsDz9uhoHNQflkEikpT1oaqNlTaoc7uY9IQ7sktagdwVVKhhAThWgGLwjRRbuDQb2V9n23QmKmEQal7iIttNh81nvSII7dym6/mhBd3RNHnTO5uePDghNGgx/7hCX7UYjQOmM0m3Fyfo0JFVeSo0Od2OuPlyzds0g2rVcnV7YzA79GKA67On3NytMN4Z5fbuwm3kwusyOn1euTphul8Q5YW+GGH4bCN8rt4ssVw0EJXiqvrK87e3BL7PZ6+85R+e5dvf+PnGfV22MzuiKTACxJef/GHzwPgj18E/grwn/7U76+AB9baqRDiZ4G/IYT4yFq7+uf/4E+Hj+zuJHRaXRcjZXykCGmdJO5K5nukWZ80XWOxaFtzN5lydXVNEIScnDxwQxftWICtdgvpKcqyYtg7pJOMWc5L1qstn3/2EmM0g/4e7W7Ex59+iucFtOIOO+M9OlFC76jN717+Dl9MPyOQil67TeSFHOztc3t9TbedcHdzx/XNgrwomUw3oBR+0AEUd7dr+r0e/d4us8mUbmdIttlyc7cgiNv0+jsMx3s8fPIMP4zo9UdM5gtm8yV92yeOY4JQMhx0MbXk0YMjtus56WbD/t4hus5JEkUUBbw5u6I/bBEnisfvHLLN5ihpmc2nCBEhRcJyXXNzu6TIa+IoYHd3l17fIa2tAOl7VNYQBgFWSvK8Igg03X6fhdak2xyrXTzYarmgvbOLNRqpvAZI4jYCtW4AI8agq4K6KijylCzbslwumM2mpFmKsRYvCJ0foaowzdVeN4nHUimHHdc12kClJWVtmiLghoKujaDp58VbTHtR1Sgp8ANJgMQKhUVhhcLwZe6ANs4RKRxO0hUSbZBSo6RCYBtPgjNIxa0EZMA2rfjixSnz5Zr1asl0vmJ3d8Sg3+X16RVxHHJ5ecf55Ski9Kjzkp3RmM+ev+Ds9WtGozGLRcHN7Yb5fM6jJ0+I4hGbTCBmGdvM4IcdvKDNzu4Rq+WcvNDEcZeqLhmEfVqdIZO7CUEQcnCwQ9IK0LXBU4LtesXezoijg30uzy+YbizP5+f4vs9m8986gn/8IiCE8IB/F/jZ+4818WNF8++/L4R4AbyLSyn6Z14/HT7y5GHflpkz92w2GXHsNcGMW+bzmZvmKw8vCEFYfF8Rx1ETkOlRa8P1zTVaW8bjXWpjsFKwu7vHel7gCUUcJizmK/7e3/11Hjza52D/iB/96GPG413GowNurm+x5PzMN7+FrQxFWnL08IRuu00QhpwcH1LmOcIarm9vWW1y1mmGlSFSeQSRu25ZpNM5aE3gb6kq4z7PWmbLDYNhn6TbQ0iPq+tbDIbeYIhFMJnPGY0Dbq9vubue8sG732BvPOazT74gjhK6SQtParBFYzLa8uDhA1aLBYHvMxr1CT2fPM0w1kfIhPWqQiDYrjN6na5bZ5qau+kdRZkRBj6lLuj1B6RlSpptaSctwjhC+T61yegkPW5ub9gJItd7S+fQoxHxCOmGcMI6s1WVZ1R5Spm7FOKiyNFaE8UxRVE7l6bnoYTCExLPgCkdl1E07EN3Q7CUlaAoa4qqoqwN2tL804BHcVxDF2BSY1AocLcDIZ1D82170WQbAEiBFB5Kyaa9cVwDYbRz9BnNPWzB90P8MAEVslwuKOspuq5YrF5wdXNHr9OmrHJacUSW5eR6ys7BMTeXV6y2Bek6Y+/wEfv7h2QF+H6X5fqW+aKk29vjbrphttjS63Z4+s4HGFOzWG7Is4r9vSOEsFxcnLPcTIg6EuGtiRPBcNCmlVTkeYkVS6SK8HzF1dUpWb6l1YrJphukhMPDwz/5IgD8BeBTa+35TxWGHWBmrdVCiCe43IGvlio1Lyl8pE4oCk3otdgd74E1zGYzrm9uuLq6Zm9/z4mBdM5ouEO3M2C1Xjeac4eTq2tLUWr8IOJgZ5dOu83ZiwuSVgtMgZSCskh5cHJCu52wszMG6zMcDlkul6zma4y27O8f8urFc5J2l+OTI85O39BqxYStmM3akYPy2lJbiReGICCIW2hdE/g+WZ6xmM7IthuqqKA3HNDudBESsqpEZikX11eEkc+DRw+Ik5hPP/sEX3pI4fHqxSllVvFg7zHb1ZYyzXl4+JBssyYIQ3SVMd3MMbok8Ed4qqAqSg72D8k7GcLAYr6llQxZLQuksGTbkjiK6PdDbu+WpNkMbTT7hw/odzsEkXLEGqnpDtqoIHBiGxzu/Ob2lgfP3iMInPNONsOze92xFA475tKHCqyp3Hwg8F0mQqdL1GqBDDAECFyCs01TpBcSVBVVWTUH1q39XCEwlLVxyPPaiYK0cVN9wz04yEWfuUAaJwCyDZGpNiBqi1W20Q04YpKUPlK6WDUlHWLM6ZVMQyd2DkUwCBHQ6Q0JWzPsassmdU5Gow2LVcbF5Q1g8JQijAL8xPLi9Jp0vUXbkMPdA77xwTfIsoyLi0ta7YSf+c4vsFiuWK0z1psNnXbSWKQdRerm+gYlYTQcEMchSgouZ68IWxnjwKPb9hj0FZYayNH1Fs9vM5vO+fjj32M03KfVGjOSLU5OThgO/xhkoT8sd8Ba+3/GpQ//p//cp/8q8L8XQlS4tu4/sPZfYF9qXoEf4Zkuq9WEKA7xZMjNzYTPP39OVhS4ZOyY6XSGNim+7yOFTxC0KEsDeBwdPUJbWG+29Ps7vPP0PdaTBVoX+CpkvZrTHw757nd+hk4vRAnLs3eeoLXAGIUQXVarBc9fvOTho8fMZzNmiwX9QY/FesVqu2qkx336Yczt4g5TabR0qb0tp3ukKHOqMqMoUrCand0Rj56+Q6k1SknSbIvyJIUuONo/4PHTR1zfXHF5fc63v/1t8vWCF5+fsjMYc3c1Y73ccLBzwMPjB5ydn+ErxXwy4er2yglggh7HR+/x5uUXBF7EfDsnzzI8aRh0Q+IwYDTsUhY1ZVZS1hWXVytqvcILfA6PhxweHDC9mxD7HTZLiwo0QjlmX1lV3N7doryQXq9LGAb/DBHYzQYENCEqSgp8T+IFPoYAU/vErRa9ft+tFr2IsNWjbhyB+s4NNP1GtGOKEiFMY6t2UBXXFljK+j77wDTKwSYB0TgrsC9lk5p8L2G+j3bXzhVpG7yqkCAsWgtUs1ZwNxzciqFRDmpdO+yZxcl9212svGO7TQkDt/pzfgufwHeCoyyvkEmbV6dTQt/H9zVj7XM321AUGaWGWPgcHJ1wM/0BEsHjBw8Jo4BWHLJZLVESklboBpbabRCSOML3Yb64cQ+zfE2nHXBwOOby7II4bGNMznRyyWjYQVBS6zVSWRAFy/Xtv3oR+IrcAay1f/UP+dh/xlcFnv0LXlL6KNtG2pTTV+dstylWwXS6wA8iHjx8h7qyzKZb2h2PsjTcXF/j+QEGQRDGjMY7ZHnJfJGC8EEEbDcbMFUjRfY4OtzDDyWb9YLeMKHTjmi3B2zTAs/LyPKMH//kY/7cn/+zHD96yOXFGaeXF6zzDF2XaKOJ6xbdzoD2oMcqy8mrkjTbUpmKMPTxJQy7Hdr7Y/LNhsGgQxD7KBm4+Ua1oT/s4gWKdjfh+vaS3/nd3+Ly4oyf+ZmPsEYSB13GgwOKtMYjIPJi8k1OXdQsqwU//vSHzJZzxnu77Bz9HEm8x3b9GbEvmNzOuLu+YWdnhCc1e+Mu+/tH1LXl5vqWi8sr4sjieRV+IBGqxAs0XqAZ7XSxNme1nVLrh2hryPOczWLL02fv0u12iYIQYW2zVnPkHtFQewUGJQXS9zHWp6wysC4Uo93pYIzBjxLCTFM3seSeH3L65pQyr5yrUWhAIqxCNKGhlXHDwLIZIupmRYl0SUiuINEc5vutgkVrN/mXooGNNEtFKZw/QGtLLZrQE5yS8D5s/f5GoHUN1uCHMd3eED+4BJEilI9FUVUV7d6Q0PcB69R8a0ebzrIKQYaSt7x+fcFw0CMMfWpryT/7lOcvX3C0t0fvyRPHSLQ1vrIoBcmgg65LPKGpsjWTyYSbu1uMpxn0+8ymC6S45tHJQ3zVop30KfKK9apgd+eEPC8wVpDnOa9ev6As/+QDSf9EX7rSREEPjw1ffP6K12eC0d6QIIhoJV18P0IpRZppwsgDq3j58g1CeiTtDkcnDxF4pOkGaxRGC1aLLVKA1iWBL9nb3WM06rNazyjKjKr2WC5ntJI2nU6LzSajlSS8ePWSsqpptd2VOCtLirqm02ljMdzOpmRljREhcRKz2lQusDMKkAo8TzIY9mkFPptAUdcFi9Wc3aNDlqsl55fnZPmGJ+88ppVEPH/xORdX5ySdFucXp3SDAd/48NuEXoAnQ5QvuL68Jt9m+KHH9eSa3/7N3yPpxXhBQL6VvH5xzeefvmG12wGrGI928CSUxZZBv0vS8gj8kMXsBt/XPHq0D0ENStJq++Tlmv4gwQsgjD3WqxXbdAtSUlbOnlvVNVHkWIPyLSfM/cs9DORL8nDzMSkJggDPu9e0W1oogrQiywuSuMVwtIOxgjcvX1NlRbM1aDBj2lKUFWVZOcjK/UBQGzTGZQxK1ZiZGgOQtU5p2ESdKXs/D/jSi2CFQQoPIdzKsK412BpjBB4K4QnEfSvA/b4AhqMRnW6PoiEjV1UJCKrKEPjSZS1WGgoI/YCyKthsK4rsBqsL1pstxpT0+m0QsFjMyNdrdJbx4Qfv0h906XZaGF26jMxsgzAF08mE169fMS1S9h8+YNg7Id8ors6XpMs37Ix2qaIQW3tI2yHdCNrtIZUpiFttsjTj7PzsK8/f16II1Lrm8uaCL158zsvXF+wfjcmqO3b39zg4OKI2Gj8UJF3JbHJBv2WxVUVaZfhRhPJ8hOcRRzHDrmXU6uCVNaNen+14QF4XyCBla6/QUUmdp2wL2GYpEoMyNYu7K+LWDk+PH+LXFk/6xAg2WY7JS9o7O5SmZjKdMZtfEgc7+GHIXmsXz4N+N8HUhRM3hYqqKEjaCdbCdrsgzVqstwuW6YqbxQTViQm6bRZpTtzu0mm1MDUU5Dx9/x0wlsV0wWwy583pKbu7eyTtFi9PL1llGi9RVLUiT6+Zb5dsNze83pzx7jsP2B3sMZ9NoK4pthtuL84x2nLx+jVKSj569oh3Hh+8XQfe3N4hdE25rrFlji0r0ukcUVUEgaTIa0QgMUo4JiIhWJdmbKkd5KMJ+tS1dmlKKKwXoWLl8gn9shH1KKRfg5eR9Aa02j0qFTNZlmwvr7FCUeqczMCmrkhrzaYsqHSNpQl+bQ60lNahxBoDkpGS0gU2II1G6gqpFJ6VSKMcZNg4lJumRuGUiG9dkUJSW4kwAt86D4FDlAm0qeh024zHY5bLJUWRU9capQSrzZpNtsZaDcrgK4muS8qyIFeStK5pxSG3izVGV2xrZ35SSrJcTsAa3nn/HUTgYZQE5bPdrpit17C0XF5ccH51hwx80kXKZpASR23m8w15ZZitNuSloS4rkIrLq2uOjx6wTjVxK+Dhww+xJMBv/KHn72tRBMq64Mdf/IBXr9/w5vyWVrdPIgLm8y2zxZLd/QFZuebgKGGiAzxjeP/ZU9ZlRS0UMlAIKdjZGZJ6isSz9AKB9EM63YRssWCZLyiDiEIbKiFY3q1YztdEfogylk7okwQhyXgMWcp7zx6j1zPOLwsCGyLqijLPqMua1WpNFYb0uj3ef/ddTJ0zn1zjSYijEGEdzKPdGyKEZL65Ybud0mq32Dnc5WoyZ7pJuf29PyAJQ6QX0+0O2B2OWK1mhG1FXVmul3f85LPPmE2XbAE1VSyWG3aPHuD7Pp3eHjsjw+lmwgfv73N9cU7oG0JfsDMY4CmPqqxYTmZsNinZKmM8HtKPXEuTpSuWqwViPScvwQvamFxTZa4IBFFIGAgmNqO720cr6VotGzgJsikwtkTYEluXzUrNYppkJi3Bep4DkKAQ1jjJsBYEocFrt4nHuxyohMPLBfNFQVUtqYXPVhvWWrPWFZuqKSAWrHHrRCcmEC4GruEcGeUIVUI1RaCuUJ6k0qJpCzyEFejaJUBXMnfJwZ7C9wKkHzYGpAZPjtMbKCWp0HhewO7umPPzc5bLFUoJXBS9JstWhKGHF3p4VFS6dlsSX2GMJdfGqVujCFMK8lyjpKTlhSSDEbX02JY1y2wLGLI8526dsd1sWW5LCq/FKImp84LzN6d0Bj1anRZFXZKvp9zObpEWIi/ADySrzYbLq5IsvSbwe3znu7/MV6UHfi2KQFGWnJ6fU5Q1SMX55TW7ZoeiKknLjGfVI7r9GM+L2Ds4QRlJJ2hRb7eIIGBnZ4TWJcpTBLFlsrxE2zV1oZgtlpS1Rnk+ddkw44KQeTpF14LVMmV+d8toMCJuRbx8cUavJ+kPQkbjFqgReIJtUWBWBdHaEsdQZmv6vX329oZcnp+yXC6wdc3OaIwnPJT03f7WV0RhhCcVnXabXidnm1WYuuby4pJRr083Dgn9EF95WAPz+ZKbmwl5XrJeb7m5uWWzyXjw4CHWgud5jEYjhsMB8/nc5RI+eYTQ2gV11gbP8/GUT7otCAKfpN0jyyHLa66uJmy3Eel2xWRyh1ABrVZEUYFQAuX7lGWJwbDZbgiCgDAMabXbjlHgNS4/7Xp7YZyXwAKyyQgsjXZBrliUCLDS9deOEWIQQrmf0VcMRwPe++BdJpeXfD6bUBtDXpakWUGWVVS167GtthitG+6AazekuqcHiEYyTBNvbqiFdjHoypGLHYrAWQ21dWIgbSyRdOtm2Uikgbf4M2vdCtL3HJh1PB6xu7vDdDpz60UFSilHrcaZn1TTzgjhBEpaV6TblKKqsAik8vH9EF1XtNs9ikJzenrFbB6TZY51oeuK+WLBfL7AaIPvB7hYzID1OqVC8uDRQ374kx8TBiGtuEW720UZODl6wKuX50g0p29eEfiWv/yX/uJXnr+vRRHwfd853iwcHO0zmc54c3bJ7t6YpNfle9//MY+fnuB5UG23RF7g1k1Ccnh4wNHhHs+ff871+pZOO6IwK744fYNfjalKyHRO2BUIz6PbGeJ7CbLucFXecH054ez1Ke89e8Le0YBWG4pqzun5Z6T5lDDSjHd3yHRFsgpYrG9ZrzVxz+fR4wPKckOebTk8OOT2+paiaIY70me7yVCqoDOI0UXN9HbC5OaWPM2xBnaHIzqtFrYsKdKcu/wWbTWff/aC6+s7Dg5cxFS/nzsIaOj20K1WxGDQByw/+v4PSdMNe6MRnU4XYTXb9Qbf8yjyDWVekrQUnheSFTWb9Ybl0oOLGqMrtKkZjXsYoSjqAiME3UEfnRsuz07ZlCmtXu/t90/TtNHEe18eEuHe2EboZjBXoa1oGP8S5UnAc6s5I6jrohnoC3Rd4QeK4wcHfPTN9zl9/dKFfZYV621GVUmMdZkIutZN+KibP2jHOXJtiTVgVAMyFdQalHDDQW1EE7AiXeugBOrt9N+xLSttELp2swVrqU2NrErKykPUFZ51JqBuO+Fgb4+z03OKxsBWG4sQzd+HgU26dZizn9JS1HXdFASnaVCeojY1upacn99xcXHrouUDSRj6aF2Tpimr1RohFXEUU6U5SiqQAq8WDEf7PHpccnV1SZJ0OTp8wN3NDZW2LBZLTCXRVUpdppy9ef6V5+9rUQSSJOHX/uyf5Td/83eZrzK2RYlFkJUaIzwuriYsNhs6nRbDXkToSb71+IS9/X0skK1X2Lpku1kS+JqkF7MtlqxnKfnaMN9OaA9DbGBZLSqOT94hifoIs6AVD/jlX3pMEBjiWPLg0SGfffoTynLF3t6YNM3IshUVFl2mFNs1voTjkzHWpCzmW3Z3x6zma6TwKAtDNEgYDgak2y3WanYGXW5n18yXC8ptRidOkMpjb28XhaDcbhHacvbmDe1+n1dnZ0zuZmzWBUmrw3e+c8LBwSGvX79GSsn+/i6r1ZIiTylS9/UCP2A0HDK5u8UKiZQB2lQYoZgu1oSBRqMotUDnhsV8QV0WaFNT6MCl8XYH1NawzTKMrqjqstHUQ7fbYTa5ozYxYdDFD1Rz+BUSF2qKdk/7WoPBCXWMFXjKd7t4IZBaYG3mhvueoqpLjK0JY4/3PnqX1y9fcHp+xnqbgfKhvh/2uZDU+9WkweL2CADWwUmsm+5bA0bYtwlRLpDYDQel56G8JofAvzdHWcqqAiHw/CYFqclAlEog8gDrB/hBiC4rBr0+O+MxF5eXFEWBMRbPD5ufyGJ0ThCGBEHoWiTtICV+k/ZU14ayTCnLituycng2DHGc4wf30XNO5HVPfC6LCmrNxfWMqq45PPZYrQuePH4PrUUjglJ4QczF9R2L1ZrTL86R1Pge/JN/9N985fn7WhQBIQV7xwdsy4z5eokfRbSSDp3+AINiZ/eQi6szJrMZR7/8DXpdj0KvyPKI7WqL0ZZu3KLlexR1wWQ65/JmxeZ0hSg9tDIkUY9KVCynWzqtFEnNd7/z8+yPdzjYHbPeTDByRlFU+H6Lly9Pnf8eTZ5rvDDAWsHBeJ8HRyFJf8iLl5/x6MEznjx+xD98+RskSZd+d4CSUJYFnifYGe3QH7TYbJbULUsUJRghuZ3MuD2/wmrNqNenFUasZksurie8Ob8AocizN/zMt75DkVdYA4cHR/iBwpiaV69e4inBQb+LAOKwxWAwZDab4wcxy/UG3wsZ9XcocteT5/UK4eVOeKM9sqJim5b0d3yCuIsMfIQpWE1XjFtdHj19wmKzIKs1/X6Pm+trOt1D93QTwFvmv3Q7e+lShI0VSBU44Y/AUYekdIQfIVBeTCgNfhBghCbLt2jl0ekn/NKv/CI3kymvLq/Js5xtWjkQSJPWrLXbEghpkcblEtwv+YV1Q0Nh3D7ACNvgy5tEMgRSevi+i38PbIQ1hqoqMVY3aUZOPeiUiA5sousSkecEyifbbEniiJPDI6YTJ4UOgoiwFZGXGVVV0ooT4lYLz/fZbFyP73IeGuKRtuR5ied7VJUl8GKs1biEOPHWT6GUT6/XRQpFnpdkpUFuNGVdYS6nbP7rf8Ljp08wuuLq8oKb2ymHe/ustyl7B/ucfXFBECjmszuK/F+DbPhP8lVWJRc35yy3SzrDLse9EYP+kNF4xGq94Nn77xO0AiazK+KOz3A3oNtRzGcXTC5njPt7ZJVhvcnYlgUvTl8zXWTEaZtu6NFudzjaOwIfXp6dYUrnImuFLYq8xPcDlBdS5D5aB3Tbe5zVN7x6dcM7T58Qh21WqzVWKHZ6J7R7XbZ6RbrZEPoBWZozHI4QeC7x12pqXVBXKauNxtoEX3qcHB6B5/Pq9JQqy9nMl2Ase/0hw+6Q0WDE8uIKKQK6vT5Jq8NwMOb7P/g+f+O/+C/5s3/u13j0+AHGVPR7PcbjEX6Zcnd3y6tXb4iiGM8LG9PNBmM0XRXSH/TZpjmaFSoMWazmrNKCbFsSBAmD8R6VgU8//gn9UZu446K5pQC1UKzSDGM0fuC9NQm5fte93CFzrj0hfYQvUMrFvEurUF6IELirrAbPi5G+0xoYU1GUW7QFvwUnTx7w5//Nv8Dry2v+1t/5J2y3OdJzUFkrJJWum9WfQGgDtcZIgd+YAASuQIBFGRphkOduJlZghXLZj0rhu2hl/LrC6BqpAFybgLBN4XE+AmksnnDJ05Hvc3x40PAfoLIGL/Sp6ppKGOdd8F3EHDbF90IsgrzIUcrDUx5x7GZEVVFR5SVVZRDS4gUBcRITaMda8IOEqnE9ZnmNEBJjFZPZltOLW16fXtHrd5jPJigpePjghDzd8uG77/Lu+4/JsxTlQRQFX3n+vhZFwGIpdMrxw30EAe3OkPF4D7C8OX/F3uGY8c6I8V6Hwchns72g7SuUEbSDgEQFfPLpC16fXRG0u4Rej2998D4D20NnBYv0jrp0FTpQPtO7Cb3ODrc31yRJRFnu8fyL5xijGY1G5Pk5H37483hS8OTRI0eP2b4kilqcHD8gjEIuFi/46L2PGA/HhEHEkyfvcH19x3x6x8OHh/S7IavFDXm+YTHPCIOEUX+AlYpPNimmKPCFpN1JSMKYfJMhEDx59A5+1AEr6XS6YAVxmCBFxHw6I0u3dLsJSTsm8H1aqs0yWHJ2ek5Va7r9LvPlkizN8b0QP9ywM2oRRjFhKwJP4fsRR8cPUfgEYcDewZi0XDLORxw/2qfVDqjmFWVeUFvN4yePsVikbPJ6lGqgn+6qarBOqmsdKFUaTVXmSOXheyHS89z6zFqQliBKIHAxX9o2aUdaU+YSaSVP33nC/+Df+3eZrFJ+63d+yHaTNU93l02IG8qjrUtBVtYZi0IRuGyGZoXo6oJoQlbEW8+BReAmeu4mE/g+WI0Q1tl3dflltJp2UetRYDBlhR+EoDWdVsQHH76HEZbLmxuKvKSuDL4KkdpQNRH2ZVHi+Q5OYg1o60Jxle9jDCjPJzM5QRQSxyGeLwkClxKltQWpyIqULC+II4/VtnIeB+VuVItlyjYtkNKyXi9YzNdYodluMnp+QLfbptVp0e21v/L8fU2KgGG2vOXgZIdW3KfINf1BQhRFPHvvCaOdAe1eQL8XE8drpjczJtd3PNh5xDe/8202s5zFcIs0MUYF7D94yP7JCaOoS75e8eL0M1pdn3a/wzP/CWleM59tkcIgpeHN6XNubi7BBoxHe+yMj2m3Yx48OGZ/d4+XL15wdPCYJOkgheL6ckpmKqKwy2Zd0G4rzq/OMVrz3ofPCAOLtSmdfsBh+4DlXYkpfQIvYLFaM7m+odimHB2d8OjkAbvjXTbrNboyBLHTtFelexO9fv0aEHzrW+/zK7/yb/Bbv/1PWa83HB8fUlUVs8UKowV1Ddc3t2zyjMVqhRSKvd0uaV5wN5nRHwwYjges1huCwDDq7aKkTxiFRLFPV0WMD9vEHY/nrz7j9mxKkrSo6or9gz2SpMV0tqTdOXjLq3N5gAolQTbUXyEAU2PKEqSP8v2G2GNBOOhp1AqwvqUQWwQCT7oQ9roq2ORzgmTIz//8zyFURJb+J/z+935AVd8nFEmCwHeHyRhHLNZNcKqvENYVKk+5zYRuzGQItxGojKEytlEGuhZCKeluKRhHWlKga0FVWWqtoSgIvBysIJGK0hj8KOb44ID5YsFsPme1XqOrmlarhScMVVVRliV5ViCKCovF9wNHOa7cFifPckbDIa0kwvMkSbvl2hwhqKryy5yFyq0cRQFJK6Y/aLPZrqnqgrI01LUhDH3KyuU6JO2Es6tbrrRhNBoiFTx8+MeAivz/41UUOVoUfOub3+H55294/eYVOzsjzi+uCHwIA+f42tsdYyuF7D1kls7wbRupfSgLfvab32Gx3fLF6zc8Ot6lN+pgC8Pe0ZjDJ0O0yLm8vmS0MwQRcH014+bmluPjAUni4/mam+s7FotdfuVXf4nv/cHvkRUp13fX/ODHP2BnuEsYRfzkxz9ACEF3r4U1HqtVRm2vSTqJs2sqjfAtWbom3UyYr2paYpdhd4dWnLDNMp4+fkKaZagmcTfLMtbLNVi4vLyiyJ1Kbm9vn+024+bmnDzPMUbT63Z5+s5j0u2G29srBn6EH4TErTbD3SGL9YL+YMRmkxLGLVpRm/Viw2q9YvdoRLsTcbuZsFrP6HX6LBYrRn4fa0s+e/kxo70ucRzQ7rYRQhCFivOLc1rJkLjVZdjvI4VEGEfmkUKi63vSkGiMN673Fp57ZFe6RNc1nvJQnocwEu3V+F6A0c1Tr6owZYUnDMVmjagsH7z3lP/On/8zvHp1xmR67eYNtcFvBVBbB0+1xoFPtMZ6wv0sVuA3Ia5V5cRL0ncMwtpYKq3xcSpD01iGhVIubNUKFBLP95q4+hI0FFmGFJIiz/CjECEMZZFyfLBPWZZUdcViuXKrSwTSOhBqFEdUZUXUarHZbKm1JgwVvucRBiF1XaI8QRAppDSUVe7wc2mKlIpW3EZ5As93K1Xlh2yzgna7RxB4DiXfAEjSNEUoQZqVBF4ESjBZZnR7CZd3y688f1+LIhCEAcNBm73dEZfnlwwGHbbbBYEvaEUtQh88JdFVwTDZpyMGTF59n80ClkHJ6Ztz9nZ3GOwM6PcknXZNt11gwpiqKlivlmzzBfPFhLTY8vrVJXmuee+990gSjzCCb37rGcPhhLvZa1bbI548O+D7P/geB/v7fPdPfchPfvQxKqgIIo2Qina7Tz6bE0WSo+NDWp2Ei8tTZATT5S3XFy+II8HB7hhfB7w+PeOzF8+JopDxeEiRl6xWa9I0ZdMM8dI0B2Pd/1YpKHP3+4cnD7i6uuL73/s+abbh2TvvMJ1MOT+75N1f+GVUoDh+9JDJak4+m7ItVvhewHqzYdQf4w0Uz198jhdbHj15iK0HBF5Enm65u7vhdnKK35KEgUe326GsM7yWT57mDLt9NtsNy9WCd548ctfxRrsvGjxXVTtNgBBuYiiVe8Naq5DCUtcNChw3G/NCH+m5LARdubwDaQW2rihqjaGm2qRYFfIrv/RzbNYz/tpf+8+5vLmlHXuYunJDQG1QviIMAvKyoipLvMYZWNc1UjqqcKVr5zFQksALUb6HFu49pYRESpBK3puNnUzYOOqxafJPjLBNcGqFZzyqMsegaSctTo4P2KYbTs8u2G4zqGuqqsJXnms3q9pJplsxxliiOHbEIq2pqwILqNo4/YWtkdISBArPC5p1oQuC0TVuJlK7hOFWa0g7apHlIWHoEQQBk9mEqi6pDGjpue3Pcst0nX7l+ftaFAGsZbmY88UXn2Kqig/efUaWloz3dgijkL29HZaLCTfXN/T8IYHs0u8+JFSSF68v+OzzT5lvp3xn8CFP3t+nqGe8fPkFRyc/S9Tt8+Pf/4LTi5cMBn02Z2ecn93Q7Y4Q1JyevmB3b8C77z5ld7fH3/v7L/j409/i8OSQTXHLupCouCLqaHaP2rR7xwwGY96czimKnMfvPiWIPN5cvGSbrfCiDmESYpWgqg1+1GY7Kbm6vmabbukP+nR7HQSCvf09ZtMFp1dn7O7uo4KIk90dLq6vGI1HvPvsPZbLNe2kwy/8wp/iJz/5MbUuub6+ZrFYsr+/z+fPX7F/tM9eOyErStpdl0yT5inddpe7u2t8PHqdNkWW8frFS4cvLysuzi+alZjBaoWvAmZ3M6Qn8AOfwaBP4HlcXt4QhiHdbpdmKc/9VNDYe/GPaGTE7jqr1P0cwCXf3GcJWMAq6/px64Q7Cg+FQ7pbrbG4FKhcb4iCHv/OX/qLCF3w//zP/i6n59fU1FgJYSBRgY+ULkXKlE7846PQpqKqLbXxKevCRZHVlkAEaGEwukTiICaOmipppgVIXMR544CA2nEIbBPGKtCYusDYCqUk/U7C+8+eIrB8/JNPUEoSRYFDmdUVSgqCICQIfNabDQKIopAwDKh1QVFusNZFqWldu9Yg6eEHEUbDdpu6jwcxnhcQhSF1VTKdOK5B0o4JgwFhEIGVhGHL6TaMQKmAokjJt1/zIlBVFevFCnUkefb0HeKow9XFDS++eMHhwR67wz5ojYdgvVwhqop2b8BidsfF7BbZDlnrDZPtLR8+e4/1OmeTZaTlmk1pSPodummPoi75+NOPCf2EskxRCgJPspjecn2liOKAR492CJOQTt/j0Tv7LJdLjIgYH/WZrK65vb4jTHzmyzsG4y55uaJYr7GqoCIlraDbTmj3xsxvb1kuKjbLGuF59AYD+sM+e7s71FqTphnb3IWS5kVFmLTQpuKdZ495+vQZ//Q3fpvLy2t+/ud/gYODfT799BM8z+Py8pL1esV49BRPS6ra8ONPPmW2nHL44IDeYMDt1RXKcxjvMk0p0pR1tmK5WXKwt8fTx0/otSMWyyUWQ609hIR+3Kfb7TJdLCiqguls0jjyapbLBf1uB1rOqmu1BgtKuZgzY5rhn3WKO6wDhALNE9cdNEP9dg3neQHKjzBFjdQSX4AvBcoXKC0o0iVBEPLf/Qu/wqDX4m/8l3+bz56fU1FjUeRVTVqWhJEPzUqvqlyKkRAuuLYsnZrUYsmLHNEQkZUN3c/fgEmlcDHlWIFogCNKehipsbV2eQtSNrMPQxQnjAZd+sMR2kISh4i65vd+/3uEUYzvBxRFQRTFBFGMtZYwChENwXmzWSOlcfJja6hLQ1UVzW0LN2w1omE3Bk5XURX4cYSPR124KPgql2SblLKukELRTjpU2lAUhlYcunWpMfyzPOAvX1+LIlBXNZPrGS/813zwl75N5MdMr+c8Pn7ENz58H0vN65vPWS0XvJq8YrtxeOy4FfLomw+BQyaTCy4Wt2R/sAGTc3t9Tm5zdg+fsLvbZ//oiND32a4zQr9Fvztkf3eXberz8SffJ82XPHny0F3VdcnN7R2v3rzm4vKaX/nVHRCSz148ZzVf4fkR3f4I6SvObl8zPhjRGye0x7uslhtmyzWd7i5V5rPeeNzeTrHacjjeR/keRV1R1RWn52eURc1gPHLMxNqgAsuxOuL29pLJ9Jpf+sU/Tafb49d//dcpioKT4xMuLy9YW4GQij/1p36ZTz7/Caeff0xlS/xpgOdLbu9uSFdLjnf2ib2QMi/ACHwRoBpNfOC7a60Q9ynGHmoQUmeSuNXi8uqMu9tbjvZPuLm5QdgO0bMdIgBrKesK4dFkErwNNWxuCa5wWKOd9VjIBkwKRjiEuQU8FWBlTaUzlHXjOmM0SPBDSTeKWa+2xJ7iz/3KL9JpJ/ytv/13+YMffsJqW9BJ2rQ7ivU6ddiwuiEJK4cSs6XFqzxUrfDwkGWOH3goP3aUawvWNMnHSjUhy6bZ6TeA0maZ4Hnu1qSNIfA9jg93ef/DD9nZ22OzzhgP+pwcHlCUFXd3d2RFjm8drmw2mzTbHkut3czHGIsfuJmKaWjHZVnhFIZgtPv+urZ4niKOQ/zAmdN0VdGOW3gSoiiCJjWp1UpQymOzXaO1RYkI3/NohdFXnr8/ClTkBIcbdzs7+E+stf+xEGII/HXgEfAa+PettfOGQPwfA/8WkAJ/1Vr7vX/R91DSY9zfY3d0wORqynKxYW9nj8ALaMddXr36gnSVEfo+YavE6xQUtSYZBOy9O2Cz3nKXeVRasMg1kQjZpgGT7YKdE8V8vcZULm7ruz/78/gyoshKnj9/wd7egPFozKtXXxBHbTr9HutsQ64LNlvNJtX8jb/13/DNjz4kiPq8/9EzyrxgtrwhK1MKSipCLu/u2Nnb4+L6nOU057sf/gLHxwdMrmaU9YTtes5gOGC1WbJYLeh02lR1jVCK3f19bqYLoqRFtxtyc33B1cUF40GfTjvmN/7RPyLPC5TyydsJ7VZC1SmZ3Nzx/MUrPv38OcbCwfERwjNsNkuCMKAsMqq6YNRp4wuBKAztVo8kCtksVqxXC0adLl4QgAqwwueTH72g0+3z0S8/pTfs4CnpAlVwPXHd+OtdxHmF73nOR1DfP33v7TyWuqodzFM0jYC1IF07YLTT82MkQisCFaECAXVOZWqEcJpAhaKXBGzSnCzNefLokP/+v/VvEoYR//R3foSxLlZdh5rcVhhTYmwTg95wT4qiQEhBFEUI6VKMQ+Hcq0I1DALZqB+VcnoDY5sU5Xsrsmt1kIJWEjIc9nj69CFHh7skSZt2FNNpt+j3OvyVv/I/5u/9V3+Pj3/yMdYafN8jn27xPOVs0M3tKAwjJFCW2mkSjHYZFUK6GDvPoISHr0I85eOSt0OqIqOuCogi0nTrQKnNzxeGEaWuKYvSBcDWJbauUF8dQPRHugnUwP/aWvs9IUQH+H0hxN8H/irwD6y1/5EQ4j8E/kPgfwP893BYsWfALwD/p+bXf0ERUHgy5me//XOcvrzkB7//I/7yX37E3niHKi/INhm2tkSRz8FJQvewjcZjkxtW+pag1+EX/9yvoWqP1c2csy/eEMeGceJRact6tmJ33OPl6zc8PnmCNIbr82uW8yndTsTuzh5Xl5fYOmJ6m1JaDWGE1TFl4fE7v/Oa0eCIdqtLXngIJIORpK9iTq9f8/L1x6zzlDcXr7m7WSPqNnkp6HbHDAYdyoOCz9cz8rKg1YoYDQccHR4AksVyTbffYzzeYXf/gCKb4XkGJT329g750Y9/SFkWjMdjrq9vuLm54fj4iDAMubi44PTigqrWxK22e9Komvn8DoQlCAPiMGCzXjG7W4JsuY3DKiWQgmGvT1WXbLOcpN2i1R5wWk0IvQ5CKrq9DuO9XaqsJo4j4jh2B6aJAtfGILUj/DpDjzv89wlRuq7wlOMROp+B25VbaUC4zAGMxZMeftBCWYkxtdMXBC5NuC5z6kqTxCG11igsH33wPv3BDrv7x/zm7/w+nz9/g/Jdjy2lQL+1BoPnK4zVbNONYwlgUL7ECz2UNMgm/szyZQaCkPJLRgJgpUSjMRiiOOb4+IjHT07Y2RnjKUlVZEjp0UlaVHnBw0cnfPe73wYLn33+OWEUsjMeU9YuTwAhyfKcbbp2ce+eoxLVxq37jNbUlUVrSRwpR97yQtJsjShrtK4IfEWZp0hryNPUZQ4ajReFFHXl4t2EpcxS8iJzWZb/qkXAWnuFowhjrV0LIT4BjoB/G/gzzaf9X4F/2BSBfxv4v1nHpf4tIURfCHHQfJ0/vAh4AWUFz59/wbN3ntIfJMwXU3Rl2d/dZ2/nAZO7O9J0xmJzxqTcELb6eGGfJEjwg4jRcJ/l7ZZXp5/z6Sdv3NVbSc7Orlkubhn0vsXB/jHGwPnpOQqPR4+fcns7pbrckLS6TG+2/ODHP2B8OObBs8dUpaAqJMeHQ5Rs89nnZ8ymGx4/Pma/k1DqzLn6lMezd55xO1nQii391iG2BqziYO8IXS45PY1pJTF7ewPiyEd6FYdHA+KWTxi6a2ae5sShR29/yN1kzmjY5ezsDf1hTG03fPCNJ3Q7fccZrOHg6JBAxczXO7x484L5fEnU8jG1AOMOsQXyIqfXa6ONT1WlSDR1XRDFHTbTDcvliizXHMc9Hj44IQhazO7mvHr5nCj08aXPcrWg086w1Ji6oCwrhxtHgLbOeNMQOIwxVFXtTELqXorbzAesQYhmV+AW/O4qLl1KkVYS4UeokGZSDp5vMQhaiUR6Adui5GBvwL/37/xl9vd2+Ov/r/+C09MpKEUYhmjrQkql8gjCGG0MRZ4jVem2GiLF9wNCKdB+gFUeRhtqNFbYBk7uVoVvvQoNkizptHn48ISnTx5hTU3gS4RQ1JVuhqk9ZlnBt3/mG3TbCev1guurG7AGT0rSPEd5bpKvvEZhauxbjLtEgrTN6rWmyF0RNGaLCi0qcLMOIQXbbeqArxhXDIrcCafcRYO6rt3/F2WBMX+MIvDTryaE5DvAbwN7P3Wwr3HtArgC8dMYk/PmY19ZBIz2GIwe8bs//seo7i1Cw+15zdOTX2JcdVmvVtRaEiUB6/MCWfqEoWQyuebkqMv7v/oeKk9YTqa8fHnFutKE4wEqU5y+ucMP2rRbI8bDHovZjN5gTLEtKA0EcZvrs2seHD3i4c47nL+54sP33qW32+OL588Zdnfo9x4w7B/yj/7R79BqGxZb2Mke0+2P2R/XyFDR6rS5u/iYR/sHJGGf+eSWSES09zzG4yHvvfcBsGV3NyYMCiaTTwj8kIP9LrouSSKPdL2lHWiqdEGoasryjlanojOWrLYrHn3wmKqsyDY1eSFphR1MOqXtSQJguUgptgGdeI91NSdJdqh13qTvaITZIpSPkD6lsaRljhYSL4qorWabz0HAdH5FtfFIszUPHx+hVc3pxQsQHodHxyRxxHaVMhgMCEQLrSuM57wVQigagjdSBk7mampsAwGRFvzKd4GpWe6wXoGk9CqKQKOD0HkYPHcwQwnKWnSV44mC2K9RoaAoKrwy55e+84y2+kv8zb/56/zo9Q3K90AosqxG+AoRtrBVhfQt2gjKvMSzgirMnU7Bd/24i2MFq1w6gdcIyYw11NZQSYiCgO6gx+HhPqHX0Ipp8hZ83yUDeyEHOx3m04oPnz0k/ZVf5O/93b/PcjohiBNMVWGBOA4pCqf+01WN1m6LokIfT7gEp9pUzQylxgpDWRgCG7gZjBD4SctF7wlAaDdXaJyQeZGjjSaKInzPWcD/2EVACNHG8QP/V9ba1U+nnFprrXBL4j/y66dzB3rdDrsHAz55ccVk1uX64oaWf4QfSWTgYsUm0ylmPiGO+xQUXF7MWC5ykmjFZrkhXVfM51NOTvY4ebCH7wvydUm30+OdZyccHR6z3SwAgVLKEV67CaPRAKFqDnePqdeKg+NDHj5+TH+nyzcWSz57fs5wdEC322Vn3GU6m/DRNx7R7iTUVYUfeLz/0QdMZjNOjo/RlWR2t2Q1S0nCDod7h+zs7FGVGa9e/5BaG6SuaSURURija2i3W/zsz3+byV2KsjO+/71P+Ma3vsNmk7Gzu0dabri6u2Kz3XB5PmW90OwMHyF0QT65ozMY0k4SLm7uUEFEGEUEQQhWUBQlVCX4ormKKspKk+UblNcC4WGtcwTe3N4RRRFlXbJau7+fszfnVLqgKCqmyR2r5RxFRFlq4sjH84XT8vuO8FsVlVv7SdGYZtzGwLUEtQsXEYa6KJ2N2Jf3vFJoWgfleS5DEAPaPfk8DMpzcWVk7uspKZFIvv2tb7BZbVj+V7/FZLUhK5se2GqHiZfuNiJw+/88z5FriZAeURRT+xqEdkwB6QxtUgqEsEgp8ZWHFNDpxAyGgwZ37yjFbgOinY7ZaIR0eot20iLwQr7x0UfcXN+y3GypaoP0hszWa8qyxBrjNARNAXC3AQiCgDgKGjuy+0cpxbr5c1Hkvr+QgiAMqaqKvCioqspZvj3XGmmtXcEwpole/8Nff6QiIITwmwLwf7fW/ufNh2/ur/lCiAPgHmd6AZz81B8/bj72z7x+Onfg0aN968cFj945IopDLq4v2evH5NWCVldy9HCH29kO3//Rp+ye9MirEiliDg726HQGzBcLp1+h5uhoTFnlvHr9HE/t0O11EQI22zWXF+csZnMCP2ZnZ0y/16MVh+RJi1evX1JvEparJT/8wff57p/+NuPxiD/44ad892dPuL274913nyBUxsnRIb1em+l0Sqcd43uS1WpOkadsNyW+F3ByfMjeaEwrDtmmBdb67O4e43kKzwMhaqqqpChL4sQwGnfIy5rFXcV494Ref5f56TmekCzmGXUluL2eYbSg2x4QBy1Wiw11nuPnhSMNtdu0Oj3CKCT05NuhmM5TTKAItI/WhYO3oPCDkiBUGOvEPfPlGoMrCEEQUpY1p2entNstknabwA9YzGeYShIELbI0AqvRWALT7OhV6IAhUqKkoK6ME+MISV3bpmNwmgAn5vFQkmaSLZrQUHcNN0IiPfe1hAJrKsBtFrDFW7xXFPgcHezys9/5Bp+9POWLl69JQp8gabHarGglbWQTe4YxVLp2ik0vImoVIH18YxvgiEV4DVGoMT1Z4Q5cr9tl0B808BGLBKoGTCKEoxl7nsCTina7Q0rGzs6YDz/4gIvLaz757AuUUiStFqWpETg9htWq8Qq4YuA17QI4cIu1rhglSQI4/obWmrquUcrJtcuyITs1n+v7PkkrcQGpRY62X/2M/qNsBwSOS/SJtfb/+FP/6W8B/1PgP2p+/Zs/9fH/pRDir+EGgst/0TzAfRPNfH3G4fEu22xNEPqkxZrTy88Y9Efsjvboj/ps0prW1rAtarSWjMdDut0BWZqRZylR7Ha7Ol+TxJbdwT6LWcFsdk3cgixLieOY4WCHYpszmU5YryEvV/zk449JFx57BwMm8wk//vjH1AiMrdndG3J3e8U7Tx5R6xW6qijylPliQquT8PrVC5aLGZv1AklIv99l1N3DVIbPv/iMwBuglKKV9DE2I8sr6rqgLFKCoEVtCkqTgaqYzpeMxkfkpeD16Q130wlJL+LBo0e0opjdUY/lpGQ5WyEJ8H2f2WxGaeFgf5/x3h7r7YaqyCiyFcZo4lZMt92mrmE5WZCmJVHUYrkqaCU+Uexsr54fY1AYXbPNU+I4JA4TDvcPnTVWKmaTO0xl2d05wNQF203BJt3iRSFSKAaDMZ4XIoUHeAhb4ykFzUVRNIh2bSr3JMVS1XVDH26Iv03AqBAS5QdOeFSDrh0eyFF5NFK67cSm3GCt5uHxPr1+nyjwef76DbWunOZAuBWbkzU7taMxlizL2Gy2WCGJjG2QaQaMh7QeXqAQOIS573n0uj36/T5KKScYMs4jUNfGORCF0xUIcPkTMgNrGI4GvPvsHU7PL7iZTGn3emzzjEproibmXTT5jj917t4e9PtiEEURvu8ckGmauq1HcyN3OgA3B6iqilYc00najSpRUfPHawd+GfifAD8SQny/+dj/tjn8/w8hxP8ceIMLJgX427j14HPcivB/9i/7BtqUtDqCVtzhzZuJ28Ebn4o13/vR/4ef/fafZrldELU6bLY1eW0Igog46tDt9MFq5vMprVzR6QQIm/Po4R6Pjt7lk49Pubp+SRxFDPsP6Hf7VKXl848/c2EbjfSt2+uQRAknjw/ZVgvysiDpdHnv3afEkYenDEkckaYSWxvW6zVFUWCF4frmyg3AjGEw7BEHIe12Qr7R/OjlJ4yGj9jd20cbQRDFbLeQFRVJq0UUJcwWU+bLnPkio6g181XKm4spn79wROXecMDhwUOMrlHWZza55u5qzfHBQ9J0w910hgpjBmOf+XxGXuR0uwmtqM+43yIOPTzpMZunKL+i1W7R7w8wBqJWh929PYoyd4M5U1GUGfWqxvgBrSih3erSbidYI1jMJvhSUfe7pKlEa8PNzQ2tTot+f0i6UsRxgu9HVLVL+L0PAnXUIUtZ5tS6QihBXWnKMkNXFUIKF28uGkOP8LCiaQ2Mj216Y6UgjGJMbahFRTtp0e0kTNcFTx4c0ElismzLF6/PaScdJxQSsgkydWgyi6AsKzabjZM8oxqeAAhr8ATQuPkQgigM6Pd7dLs9Zz0XNRjHOjDWcC+MdlBSgdY1Sri2qJ3EPHr4gHeePma+Wr5VEIbGoKRy4anw9oCXZUlVVW9vAW574IqEUooguI9Mz9xN7x651rQNVVVRlKXzEjQbDk/98bYD/+TtSflvv/78H/L5Fvhf/Mu+7j/zEhY/VFzfzLi9XYMVnBztELfh8vwNi+0jrKr58BvfZFtuKHRG0krY3zuk1++zXs6Zz6YsRUXwYJdeL2R3NMDzBVWdEwQeVV3SCxOGwyHptuTk5AFFkWL0ltqk7B/u4skhJ48OyfWA86tTWknMwdExxuQkLY9uO6Lb3iEKFdsyZTQckWYpd7dTR79VPv2OJi1TLvMrQq/Nzs4OrSSh3W2hJISRJQhH1HcbOr0uVW05vzhF147Vr/zQBYUWa6K4w9N3nlLVKbqGKIxIVxnTuzsuz+7otXoIZanrGi1K8jzDCBiNhhwc7PL/be/NYiRLs/u+3/fd/d64sUdGZlZulVW9L9McNmeG5GhkSiTHImGNaMCGXmwRMOAXG7Af/EBDL3q1AfvBgGHAhgXIhiS+WIJpW7IpS0MS0JAzPUtPT3VX175kZuUSGXvE3e/9/PBF5vSMps2haSqrMfUHAhkZkag+0Te+c8/yP//j2hJZZVRFShylOLGk3TFASIIgJElTOr11+hvrDEcDhAVJGiFTyNMacRxrufGkZJbPsR2bPEvJ8xrTyYDxeIRlOYyGZxiihVGvE81HlFmC5WhtA9fzodItwgsiUFFmq5Xmuldf5FpzUVOPBVLkGNJEGialFFTSQEkTDEt/qVWJa1moomJZFNTrIZ1Om+FkQVHENEOPN1+9SRTFTJcJjm3qZaarxSVCFwcwMEjTFLmM9PCQXO0vEorS1Lk06Faj57qEYYjj6LVzciVTJo0UqcrLdWYXj6IssUyTsBboMems4I03XuNsOOR4cI5A4bsuaZZpfpUQlxLtZVmuFIv0zIAmKf1oamDbNrZts1wuL9OIizTAcRyKvCBeRlqHcZVefRqeC8Zglubcu/+Y48Mxy3lCp92kv7GJEAk7+2sIO6G/2abmbyAtWCR6Xt4QBqZpUKvVaIQNsnSup8CEhcRgNp3guhb7+3vMZufUQ/8yzFrfWOd8cEISZzimx3RmgRS4gYustNxUXuTYjoGqMrZ3NjBQqConS1Ncp0aS5CwXKYOTEVme0+v1Wc5jovmM6fgp1zauc/PGqyAtwoaNaYJtpzQbfdJ0SlUJFvMFs1lCnufcfOkNJCbD8zHttS7LLGPvxnWG58c6zSkKouV8tU23IvAd2mtNvFqNwXCsWWWuzfXre+zuXCNaTDg/e8Z0NiOJMwzLx3IU48mEeRTT7Xa1Ym2ekOQxWZlSklFSUJQFjuPiWA5lrphNpnieje1IBAXjyRlRXLCxsY0hFfPJkKFlIqXeDSmkTX99E8c2KcuL++SKRcjF6nCoikrvNjSNlTpJRZEVoFIMW1K5AsuwLnfN6f0GeujHsh0sM0GVJUHg02rWODw6ZjqP6XdavPX6K3z3gw+RhiDNSqqqWHEC9H/KsiwqpUiSGGkYl6xG0/hhKF6YBpZh4boOvudhGKtWqJBIy1jNSOg7sOZLV0jAMgxKwLH1xqaqqti/vsvpYMBoOmM6HOmuTJ5rmjCs2qg6v7+4618c4Mt2X55fRlUXkYOmMa80FFZt0ixNKbPiMj34f8Nz4QTyvODB3SN8r88br73DzvY69bokTo/Z2rlGtEjIM4v+RhfHtxnPDMajCdPRjKIw8T2PnZ1dlrMxqkwpMkmeCgxL0WyFSOGzjEYItLLxyfExnu2xWCxwHEmlFJ7ngOkym0+ZLier72NFHM/xHBPXkQxOTqmKkjhKyIXBZDplNJ6jCgNLmqyvbRH4dSZnz7AMh1ajQ6+zRpQvkVaKH5i4joPtCJqtDqcnpyzmOaYRUJaS7WsvMRqfMZzc00tNpV6UsbbeYzafUmYxVIpWO8RQFvW6j+04vP76G4ymUwzLwnYd6mGNsixI04TJZMLp6SlSWggTTs+HPHigtQqb7TppHhOlU4bTAZYtKaqEghQ/8Njsb1HkFYup1k9UpRbxyNIFaZaTZJAXa+R5ysHhE4bnAxrNFnGcIU2LWs2n1WpSZKyKa4YWxFgtNBUr/UJpGHrEeKW/V5aVzmGlrrYLw9QqQsLUkmZKAiWm5WDaDsv5DMM0CDwbqpz5dEQQtlnrNGk36kyWCWWWocoSaZmrBSaag6CUDrM1q1DqQqNjaQdQFFSViRQWjmXheS6mZelBo9VGZmmarASKtFRZlmLYmqp7QaDyXYeiXqOXtrl58wYPnxxwOjgnjRM9Q7G6MV08gB+5e18IugKXsxBS6vkHz/NWcxvVj2wZUgp83yeOY2az2f8/LcK/SJimRZm5XLv+Mr/2V/8t1noht+78AdPplHbX4+nhMeenJXs7r7HMcpbJEChZLCYYQufoaZJjSAfbdrGkxWJWYroR0rBJ05hGI8S0DMbjMfPFgnkxxzQFQeAwGE5wPQfbrzGfz/G8gCDQW2Jms4m++8cJ5ycDWvU2w8GYcVSCgG57E9/VK7Ze2X8Ny3SoUotWs8/+9Veo11uwTFmm52S5dlhVpQtIs0nEfJaxsX4dx24S+F2++/638Gs2rucihhVn58es9Zrce3CH6zubhGGNqSNpND3ieEpSWfQ3N7nRvQFSENRqDEcD7t55CqthmsVySS2sU5UZ8+WUKFniOBbz5YSnhwmNVkhexNiugyEVrmnSbHVZ6/aZDGfkcY7vOlRVRKkWRNGEOM1BekxmU+7evcPg4Am99XV6cQTCwHY8omhGlkVYykVIE2lqGq4OURWGAKVMKhTGSp67UqVWKpIGQmoFo4vtwghN6xVVgVACJTRzMSv1QXJdR2sh+K6OFBB0Wg0m84iq1Jz8siioVnfNNEsxTAvLshBCkecpSWJQBPoalVVJWejowbZNbFuz+1SVc8EotmwbKYSeOagUVVlgKmClxMRqk5HrmIQ1n2sbfV595SUePTng9HyE7Tq6awGX0Ydt25d3+4tagJTyMlK4uPNfRAumaZKvNkV9sqBouJ7+t1DkxadHA8+HEzAsblx/g1duvEmz3sNx9PTUYrHk4OCQpwdH5EnAwyd3GY0H1EIDy/CYzydQSDw7IE0K6rWQjbU14mjB6ekxhYhp9zokyQLT1P8T9RjuBFEJtrc2sGxJmsb4NQ9pSCzbYnd3D8s2MS04GxxyeDggcF2EVPh+QFGMSWNNM/3iu69T5AlJEtNrr2EaNuWOge81dUtHgFIJy2jAcpFD2abX6TIZz4ijnPk0YWdrk153hzDokucZL79+Hc+rkyQJSZpSVjotWet3GZ8NGQxPEYWleW3K5e7du+zs7uAFAb21DpOJYLmYk2cJliXJ8owkTfBqLs1WjetiE9f3qFTG4ydH7IgtyiojGU/xfL0zMclj5tGcOEkBA9OQ+HUTYVicDJ6SV4ogdCmritF4jBKCosiZzqaEYR2lCmbTMefnp7TbPVwv0FPFSmmm4YrWKwwTUek7a1UqFBLDMjEtG2FaerV4WVKh/9YwJQITlSekeUlalChhIEwLvxbQ769hmi5ZIUlzWOsuOT2fMJkvdFiuFMIwsE0bVRYoJaiUgVQ6rM+yhOViTs21AC3yIajhOg6mYawyAUlZFchKYRgWEkGZF0C1aoUW6PNfUpQ5F1qovufQ7bR4+aWXuPXRXU5OB1yk6noIa1XfWRX5Lh5SyssI4aIucJEG/LBDUfwrnYIKhWGZOKuZifHoOZ4ilMLk5Zuvs97f4PHjRwQ1GI3GCGEyHs/wXJf13gZnZ8ecnBywt7vOsoiwDIM4ivHtOv21DRzLwTR9lEpJkgpllozH59i2JMsKsiTj7PSc8WiOiclbb74GIqJWC3Bdm7wsdJ5YKtrtLkLmnA2eMp2P2br2Ggsj1kKTTkA9NDAti1a9S6UysjRCrvTw67WALM+YToa4QUBRRuT5kmgxx1AFgetRFQrfDXFtKFLFdLKg17vO3t4OQuYYpqLTaxEtY5qtBusb65RlQZwmLJZzAruBF3gkBRweHWLaFjs7W+RZiuvYtJoNBucJy+VyFcJCpUo8zyYI1mi2WkxnU+RYMZ1NQBS4nl72Op1FBHaGZbrM5xnRLCH0ba7v74Gx4O7D72O6HiUVeVWytr4JyVJrEwiFH/ggYDA40avNTXMVvlogTQxp68IaimqVJggpUaWWArMcF8tyqIQJhtRTfau0QUgDQYWSluY0mDa2C2VpIClZW1/HCxpEcUlWCJZZxelwyvFgSF5Vuv9uGpiGiVlJ0jwjTRWGaWIoEyiZTDNaDQ/bCaiKnDiJNCtwlbsbhqlTlrJcpSc6FaDS25zzLNNOoCxWRVDtOGzLpFEP2dne4u233uT+w8cs4gglSqSUOI4D6JD/k7m+4zg/5E6suAAX7184DqUUtm1fjk+XVUmhdI3ADXxs1+G5HiU2TZN+r8PJyVOqqmAwiCjyhF6nz3Q+wDYltqHnojfXN3BMh5PBkM31PQzlUg+bGMJiOpkxOBuCKpGGhRdazKMJQVBnOh1hG47OoTo2VVbiOA5nZ8+o10M9K16a5AVEy4Qg9BlNpviBj+c52I5JlmcsZiP8oEXY6bCzvUMQhKTJDN9pMBoOiKIIDK2wW1aSqpKgCvI0hUpRZAVlXuE5PoFXsr/bw7JdJqMxZyfHmKbg7p2PaLc3yHOdo3a7PdJkycnpMc1mk53dPUKvRbPe5enxGbbjEAQ+nU6HNI1J0xjTlNTrIUfzqd4KXK+D1KyzRqOJ6zhkrsvNGzeZLiakWclar89ocs5oOKa23cSwJKPJGKO0sKwaeZ7jOoLuWhtpuSR5yXQ+xw/rVIbEdvSK7iCskSQJ48kYISWNegMpBLWgienoGUN919K7CTFACANh6iKhYTkI29VM+tWqcEMaK9ahzuMRJoblYjk+2DZKFSSqIBACYbiYdo4SNou4ZHe25OHBEcVijuO7ZKoizzICv0acxihRaXnxrEBgk6c5k+mYfq+FaUkW8ymL+ezSGSndC9R3+qqEQo9My5UakVCmvnOvxEsFSuf7qkIIk1oQ8M47n+POvUd845t/TKEqTNPEcRyCILhsP+e5bkEWRYHnebiue/m7Uuqy4HfRUbhwABfMwVJVCCUxLQu5cjA/8fz9aznlfwqEANtSDM4e8tbbr3H47HRF0QxYzBfkecHZ+ZC93Ze4tr7OwcNHWNKjUWtj4JLnFeeTc2bTKdPpGNMQ9HodHNeiwMEPPJyZRT2s0270WUwTKHW7ajqbsrunC2+G4VMLAhzboyoAJdjd3aWs9AGcLzNcs43t+kjTot9fI4mmZGmC4RrE0ZTjk2cso4hGu0ur08NwFGmakCU5gRsQeCGWtInLjDzN2d5eoygs8rTi8cM7jJMTikJLZQVBhzjJ8dyAvb190iim2Wrjv9LEMWpIYTGNchzHYWdnh3pYI00TBJAkMZZpYFyMztoOrh+wXMRYppasCoKQjc0NzgYnDIZnqEoQLRI67TWanSbKgPlyTq+xgecHPDl4ytZujV6/S1bC7DRGUrCYxlTRHN/3cF2LJE+xLJN8teFoeD6gLEqMdRNPSkRuYxpyNXSkWYECiRQrp2BaKGO1NRh9iAxj5QSKaiVboDUDlTS1fLjlQ5WBlFhVillIDMNd0b179Nd6JGWhRUfLiqLIqFYjy+bqDpokKZYpsEzJZDLi9Mzn2voaSlVMp9NV3i1X4bbAMCzyNKbKc0yhi4BVWWIath6HlgLTtFGqIskz0iSmEjamYbG7vcMv/9Iv8uDxQ54eHV6G87WaVgUuy/Kya3BRpLw46Eqpy8Nu2za+718Km+o1caYWLzG0QpK0dOTzaXgunECeZ8znJwS1EsNcMJsdkZwnSLkLlYMlbJbzKVVZYQobSpuX91+i1ewxnSw5OjxgMpkCFaUq8F0P2zNJ0ojlck6rGbJ/Y58sLlgulvhBSCtsMR6f4rkutVrAYjGlLAo2+5ucnk+YT+fUWyGNhku9EfLxs0M8q0lYb0Klo4L5fMpkdIIhckbnE9JkDiphPj8nLSIW8ZhCbVGWJY2gw1q3i2c7VAWoSoeRi9mEsjJxXZs79x5gBkuu7+4hpM/+9VdZLDOKEgK/zvbOPlVW4EgD2wgoMsXa2hpVVdBsNqkqPbte5BnLxfySViqEZLGMsJ2AMLxYrW3TarfxHJ/N9S18X2skokxeffl1Gms+g5M5hao4Hw7pd1sEQcBkMmaRj8grSZIq5ssB5ycTqniJbZt4rkVV5WxurFMLfJRSHB48ZT5b4NgubWFQSQtlWQjLxBACaep0RUj03gLD0gQhaSBWk4SGKRHKQEmhiT9lTlFBUYFhm1iuh6hCLWxaSIgKrTXgOFimRbfTZTifMVrMQAq9OFYobMdCmgZlqfcVmJZBsxESL+acnDyj12kQBiFJEmtVIql3EgohdTGxKCjSdDX0JFGFulRVklKstg6VGKUmDiEEruUgrYDPf/7zHBw/4//4P/8pBwcHP0L/dRwH13VXHAN9uOM4/pECoWVZeJ53mQpYqx2ShqFT1WrVzrzYFPVpeC6cgJQwmT4jyyd8fPec89Ez0rSgHjbY23uZOMo5OR6TRCl5UnGtv0fg1LGkhyTjfDBiNp8Rhh69fhu/5oBRMhyec3r6jGYjpB7WGI2GHD454603fw4pJfP5jDAMGZwPNF/brNHr9nj46Ig0S0k8xfGzEaBoNBqUmcloPGGt08SxPc6HZxiU+L7D2ckIIXI8z2Rrew0Mg8l8weD8GbYV0O9t0G2vYxsGk+EIVSlMw+Dk+AiERbuzRpEvIM8whIFh2IRhk0bd5nRwxv37jwl8F9+tMZrNCNqafBSqkka9RlWVxFlOFC04PjlmNBoznU7wAx/btqlKPTjjOVqpx3UEYdBgNl0gDYN+b5PlMsIwLDw3pN1tMJukWLbJ8HBIUPsctcDh/uOPsYKK8+kC0+nz8NFjWuEalUgYT2ZUjQDTAITCtEzSNOHxk8f0OhHtZhs/aGB4JYY0KKSuAchV5Uyxqg+YJsIw9RwBK50CqSvwF605pRQIA9OysRwbYZiYjotdKZxcwWRJUVS4rkcQajJRazJktJhiGAb1MMQSEjvPidMM0PLcYRhQDwIcS5LGS/I8xfd7pEnCbDqjqirMlVRZWaTkeUGeF9iGQJgO0jDIMj0XYBjykrFnGAau66Kkg7A8Kkx63S6/+qu/ymg6YT6fs1gsVnJkLr7vY1kWcRxfRgAXd3l9ZuRlDeAiXbAs3emAi/0KK6UnNCnr0/CcOAFBEqWYwsJ3FI2aQ9Bfp9ddY72/weB0QLvRQBYScq1Me+uD97m+/zJgYBhg2YJmJ2Rrd53jswOyJEaJkqAWkBc57337PRxTC0KkacLZ2QlxtKQWBNx/cJ/eWo/d3esoVWJbimarxeHxYwbDQ3avb7LVv87TxyccHz7jWm+PKs15+uwZu3vrOG6NIKyhqhSBg+O5mK5HOJ8zXUTE84gxA1q1ENd2GE9mTEZT0qwkLRRFGeNmC7r9OousYDSZ0Ww4zKZzpAqpMod4UZCnc5y1gFkU0+mX+HUbM28AGYVIGY6GnDw7Q6yEMoeTIWfDc7q9DXZ2tgg8H8s0yfICz/Op1XyiGBDgOjbXNjcpyi6VKhkNxlquKvBw9mpIx+bbP3ifvIp4a/8mZ9N7+DWPMDT5hXff4un9Ax4/fohpOTQaAY4fUKiCKi+ZRROCWkCURqRZgpcXqNU8gZJQoleAVUIfemFauu0mJdLUX2qFsVo5hk4hVqQY0wRztb+vwkCaDq4v8PyYJM6p1QKumxbLJOZsMuLw9IQccD2PZq1GFC0R8xm2Y9PrdXFsC0NAza8zm5SUVYphKJbLmPH5GVm0xApreodhqYeOlBKUSujURBqQp5oZWehCpjQMDFliGRUlBVkyJc0ERSXZ3Vzj13/lK2TLGQ8fPaIoK+IkRSkoKkWa5eRFheu5eK6HlII816pJUhokSYpArzZTmquEaZoURU5aaNKRZVmXWo8/Cc+FEyiKkuFRyka7h+MkOGXJeqeNH9T5+PZHnB4/pOGGeNgU8wXDySGnJ4eEocX65jadNR8vFZh+SW4sKM0lUTZmd+cmVb7GZDQkTpdsbW6x0VtDFQbnJ2fYpsA2od2soSo9f55Vc0xzSafhc/hwwcn9E7Yb11EYeGUDqxyynJ6y5m8xzxXHT4+oyMgqCLwaVZYRzQrsXFHz+pyPn7GYnGEXCaOBTVYoTk4nmGYdywoJOzWieIpwJbGa0NvcIkoXxOWS2WzJ/Fxw8PgZnbU6i8UZEy9lY3+PYXGICgRZ1EKpKa5TEk9GZGaGLH06/XVymXF0dIoTdAjr27RbLlUZgwTXs1EqodGwaXc7nJyc4LiCutfk8OiQ8ckIzw0J6wE3X3mD23fucfvolGvba6SyS72TkKZzru/5FPkxZyfHpHFBq+3T6m6wzBfkyZxOK6SxHoBdMI0nnJwdI5WJ024hXV3RF5lWKaqEAW5Na3yjkDJASU8XAis9jSiUrsJDhWXpQ1ZUBYiKQhkIQ+LXPHobBtF8SRjUWUOAhKOzU+4+eES2jKgKQa3eQFFg2zW2Nrs4tqDIEz2yrLRoRykcbKeiMCuy5Yx8OUMGHoAeRbZ9iqyiAAppo4TAdJfMJgvyDHyvhuc5qKpElRFJNGc0nDAazjDNgHZ/g9e3e2z9e/8Og/GEj27f5ff/+R9yMhgiLBffr2O5Fa7v41sWRZaB0jUXPciUkecZcZzieb4e1670zsay0MQsQ14sV/nJeC6cQFVW7O7ssd5ZJ88PkNYGVAZJlHNwcMhyMeLazQ0CUaesSpI0Ya3fI6j5ZHlOLahRb9WZLkdMxhPa7SbC0KGxLWygYvvaNRbzGQ9m91lrb7KMIna3+zSbDvPlgFLBaDQhzxKm0ylJtMBzXHa395iMtLhop93l8OgJvu/y+uuv4tc8Hp484u7dj7FdSbtZxygVNb9Gs9lmkRckSYpje5p8hMB2bEzToNGoI4RHt7/Gk4M5B08Pkaak2eowO5zS6bb53ve+Q7f2Nrdvf0R4XNDo2RiuT63Z5N79e2wWHsdPn9BsCtbWAtqdNmQF0VTRaTfY2Gpz86UIoZoI5eiJNQWGKajXa8RxhELpTUN5RpQmNE2T6XRKvIxwHV9TkK9vc3I+QBiS9Y0NHMfj9PSMdsul2Qo5OHhCnheMJ3Nefu1lrm1tce/BDzBskzRP6XQ7TM7nHJ+eEC9KZGES+i4SkyqPkVLnuIbtYRs20XKJUQk827tk9EkEK/UMtHyRFuaUUmJiUFYKaZiYq/TC93wsaUKlcByP3d0d3nrrLZ4cHfPhnXuAYDIZo8qc/lqX63u7LBcjFouSdrvJfLGk0agT1HyUquivddne2cR1ba0zsJoYVLat9QUMoWcdSj3e7NdCFjO9ezDLCqoyI88T4iQmihacnw+IoxPG8wX7L7/KjZs32K5gsUwRoFu7jpasdxwXISRJkupBJbFaWpprYtFisVjVB3QRMcsykixBmQrfCyiLEst6zp2AlIKbL7/EztYN4lmN0dzh2WBKWUiajR5rvYA4TkiSEXmeUSiohQ1qYQPT8pnOI8ajCaPpENsTZLlLkibEWYrKFc1mHVUWHDw9oN/dZjQacfzshMC3mC9KojghCOscPzvjSRoReJIP3r/N9uYua2t94mVJUKvT6TXZ3d1ld3cbQamXZ5QZ9XpIqRKEUDiurtYapoEsKzqdLo3NTcwyR4lS75a3BFm+xDAFg8ExqioZjkas9et6MaWQ9Hpd/q9H32AWetiOgeOD61k8evSQyWyJaXl88P7HDI4VYVjw1pv7BFtN1tc3qO12ePjgLgeP7tGoN+n3r2GKGrWazWIRo1RJHEe6ILtIAEjSFEOaTKdzDGnx0ssvo5RgbeMaZ6cnmKbk9Tde4Rd+4edxHcEPfvBNXM8jL2PqYcibv/ZzHB6d0Gy1cRybTqdLRcTZ2RF7O7s0r/do1HpkUcV0PuPg8JC97Q1MqUd6lVJYSHIikiLGrYPd6CKU3iy80uBmVelCKU38MQyBNCxUrnRrWOrtSLZtI5RiMpogsxLb9vjc229zPBhyeHKmNzqVCWGg9f981yNaCDzXpd/v47qzlTyZi22b7O5usbe7jetaCFmtmHk6ArEdC8PQY+x6VNnEC13iqMJxPdI0pkJQKEWptLMyTJOiylkuI8ajEUEtpNbQRKIvfemLnI+nDKdzlLBQWYHp2IiypCpyLYqyIhBdsAtBa0fEcaznC4RaqQppNuF8Pv/U8/dcOAHXc7l9+yM+/v5dOm3FPHnGIlGUQmDYku5Gn4MHD9nr7/L48RMG0yFKuNQWMbXQwzBsjg6POTh6hONLvECiZMlmp49n2biWQVIW9Htdtq9dQwoXz/Gp1T2Onj0gzxer1k2bg9MnvLS/w1tvvkO0iDl4esTO9j6WbTEcji4VW46On3J6dsRwdMbOjW2CepNoNmMyGRMvE+RognR9rXlXCJazhDRfUMmSMKzRW1snywUffnSPRrPB5uYGYUMX7YQwuf3RbVzXwnUlouUjzQlpHq00Ek/4zd/6S3zj20NaL3WYzZ+SxBknJ2eEjonXbzI4P+Po8AC5LXCdIfWalqNOU4llBRRFznKZ0Wq1KMqSRqOFYVrcunUL07Z56803+O733ieKFjx8qNWMoVrtBKz43Dufw5A53//etygTA8OEv/SVX+L4ZMB4NsZ2HM6H51i2Q1XpzTkoST2sUyYlURyRFzm1uk9VZaRpRpzEpMuURQZtw6V9wSdYpbMCdNJbVXpQZ8WoQ2gCD5VadZAAFPEyYjqdUVbg+CGbO9f5+Z9/l1sf3+G973yHwLeo+Q4nxyeEgUWz2WQ0PGU8GlEp6HW0+KvMFb5nE8czjCnUVEPrYhYlCnk5fAQS07CpciiLijQvkJaNKAsMUWJUNpblEDZabCiLsFEgTRvTtJjN5mSFwjJN3nnnHc7HM/7ln3yLrFQIw2SxXFIPfMKgjpSalZgkCWVZ4vu6QxCtBEcNw8BxbQzToMhyTCGZR8mnnr/nwgnkec7jg6d4qkZVWmRlwpvv/CLfv3VIXmYso5TReIRvnJJXJa4fUlSCo2enCDmm3e7SaXV5/Pghw7Mh3bWQs7MxZix449UbzMdjXMdl69omDx7cZ2P9Os1mh9H4nMdPj/ACSVYZ2IaPY4c4dkCrEZAnp6yv17m+d4OTszNG8wGng2NG4zO6bpu8KghDjyhZ4NVMTk6PmQxGCGXg1RrgeCyzgnKhIC/YuNZi41qbNI9YRGOqyiKOp0hT0V3r0Om0GY1j1ta7/Mtv/DMc12I6H/Lo4RO++ptf5L3vfoNXXv0S3e4e6/1dWvUNLLNJPYQiHyOUII1SDp4+xTINGo0aYd2n3a6zt3udmie4e+d7RHFCvdEgSROy84zB+YhaLcRxHYq8Yv/6DsvlgulszHg2YbmY8dqbb5LlGePxgKOjR2xdWyNKYurNFuOzObP5mMVixmgyJIpjlMyxLYf9/R0mowmT8ZTjwymO4XNzd59Ws6s1DJSDYzskaUqcJMS5oEDrFVIopCtR8kKyXNcDyjLT5BxDt+mqFYNOKEWWpAjToCrz1eBMTqUEURRhmAY3b+zzy7/8ZZ48fcrZ6RGSCqoU21J84d23MeUajx8/Ik5TamGDJMsJbZdkMmZwfszG5hZb27sEQYhleWBaCHSofUFtLjOYTmckWablxYWhmZK2gyNKLeTiNgijAoXEcl2E1GPNSV5Sr9f54he/SJIVfPeDW4wnc0zLYrlcUBUZtVqwGjPmso2oBUYUYRhgmiaWbRGnMVmqW4Y1P/jU8/dcOIGyKqmFAQ2zQ57NyAuDRriOZMj33v8Btz4c4JkKMp8bL71OWuU8eXSEZXiUZcTR0Rmua9Fudag3Ntm81ub+g49ZnA/4zjcH/NKXf4lOt814PKPf6+L7HmG9zny5oL9+jf5mC5AcP8l5/bXPUWZzvvvdD9jd2qXb7vMnf/JNFtGCV157CcMqsWzB6OCUKE9orXdwQ1+LmgQezWCX6XiOND2Ozofcf3rIq3ufoyxgMdcTXYt4xCKaUSmTRsMnK0tu377N/s2bHB6M6HQ1XfXatU3u3R6xvdfny1/5Eg+efoQQ8NqrbzEeTqh5bUwzoLVVJ0vO8cwaNi0m50tUVXFj/zr7N2/QbGzjuxaL+YDz4Sknp2e8+ebbdHtdDo+OtW5+WOf05FS31IIap2cntFpNnp1o4tbGepc4i/kH//Dv47om/X6T8/NzOr0+k/MlSbLkm9/6BkpY7L/0Eq12iDRKhsNTvvHH38ISLq1wjWQxJE9yVJUh6SJUjm1ClhcUZYVhuHi1JmFYX6n0SM0fqC72FGhJblWVl3r7oFWITNOilDlqJbiRZxmO7VAhqITFdDKl1u7yV37lVzg9O+V/+71/DCq/5N/fvv0xtcCBFVXX932iZYTMC8bzqRaBcSxqoY8QCt9XlJmJYTl4nrWaQ1AgDWbzBa7XQAmJYVqoosCxfGzXxJI2QmaUKtNphGmS5gWGKWgGIR3bZef6TfZvvkznf/8n/Is/+CMWywjTMrFtkzAMLxmGSZJcjhyD5kXo2QWBJQ0qIcnSlLDW+NTz91w4Adu2aHc6tN0Nau46jx9/zO/+g3+E5a+TJhVlGrOsIgbHC3obO1SGwfl4wt52F9ep8Ud/+IfE0ZLPvf0K77z1OqZVUvNshkcHbKx12d/f4/bHd0hziOIxTw8HeN4hpVJkZcrtu/fxnBq98GUCv0FU5Wysb9HpdFkuYh7cf4QwBOubPSazMf31NqXKefbsKZNkwrXrW6RZAqXi6PAp7VaPbrdDa2OD/vYOTtXFlVrPbjR9SEFCvowJak1uvPQyy7jg1ke3UYbJcqaoqlQTlzbXuXv7mLDR4c6DD/nCl97l/ASeHZ1QiZx3f/6XOXp2ijQmlJnFgwcPMas6oaejCiXnnJ+fMJvkUNlYRkq306DVbtJfX6PeaHN0fApKMp9p6fFWs8VsOqXVdRAGpHmMYTncv3+Ho+ND3n3353Aci2fPjrj98S0MQ+IZPp0gZzQ6p93bYLFYMJmOSLIF49EZICmKiryoGE+mJFGEIMcQBYu5hWkIfN+nEiaW5+C5Hp5fQ9jOZf6vywGrhaGqolIlotL9d0PqcXBWKj+L2Zz5dIqQEt8NUEKSVYInT5+wbdr0N67x1a9+lWeHT7n94fephzWm4ynPDh/z0s09ajWfslJMJhOmsyXtmkfNFjTqPaSsOD97RhxFeH6dJK+ohW22dms4lkuaZiRZzmwRsd7XXSU9LaWdQVmWKCGRhonj6nqGEJoqbbkelu0yi2JmiwVra2v89m//Nr/xG7/JnXv3+frX/wW3P/qQKIoIw5Bms8lsNrtMAS7oxEmS4Nj2qisgsaTx5xca/YtGnusuwNP5kLff2Off/q1/l+98/yH/5Pf/GCFMpLSZL87Z6PTob1wjqxSvv27TrHfJk4JWs0OVl2yub6FK+MFHt6hUwrVWSM3z+OC732MynxInFW7QIV5m3Lt3gO3a9LfWKKsKy/V5773v8dGHt/jaX/81siTm++/fYntrly984QuUqiRKZ4zHYxotn+VyhqIkiubcufsxzWaT6WgMOTSbLfI8Z5kkzGYzQqvJG2+9xv2H75FmOeubbT68fYvJdEZ/fQfPD7l+cx/XrfHg41tsrr/CbF4ym03JihRpwXvvfZO/+ut/BUqLOM4wTIVj1PC9BXE8oR422FgHWQRQmngeLOIp9+7dwTI6tBprBG7FbDYhrLe4f/8eYaPN9s4Os9mSj29/jOM4tJpNTfE1DWQBrVaTg6NDkizlF7/0CxRVQavTQt6q6PTafP0Pvs7GVpvRaECSxfT7fRqtNoXKaTR3ePLYJs9KRoM5+/s3qXvnDE6OkYahF6cuKyxL0Gg0Nee/NHHCRHcBkVx2t8WP1AVXwzN6QAbDgiLXQjB5znQ6YT6aUK+FWoHH9ZjGGc8Oj/DDJn6tzs61Lf7yV77C0dNHHB48pVl36ffW8Fb8fNO0saXF/v7LiDwmW4zp97tAyeHTx0jDJmx0iNOK3vo2/Wt7Wu0oy7XzmM5wXJc4yhDodegUijTNkKIEtVI6LirmiwWGaePXm/j1Jo4fYtkzJvOIWr3B22+/xTvvvMOv/Bt/mVu3fsDBwQEfffQRJycnlzMEFynBRb0g8H3KIkciaNQbmo79KRCfFDe8KgghBsASOL9qW/4c6PLZth8++5/hs24//MV+hl2lVO/HX3wunACAEOLbSql3r9qO/6/4rNsPn/3P8Fm3H67mM3x6ovACL/ACPxN44QRe4AV+xvE8OYH//qoN+HPis24/fPY/w2fdfriCz/Dc1ARe4AVe4GrwPEUCL/ACL3AFuHInIIT4N4UQd4QQ94UQv3PV9vy0EEI8FkL8QAjxvhDi26vX2kKIfyaEuLf62bpqOz8JIcTfFUKcCSFufeK1n2iz0PhvVtflAyHE56/O8ktbf5L9f0cIcbS6Du8LIX7jE+/95yv77wghvno1Vv8QQohtIcTXhRAfCSE+FEL8J6vXr/YaXCw2uIoHYAAPgH3ABr4PvH6VNv0ZbH8MdH/stf8S+J3V898B/ourtvPH7PsK8Hng1p9mM3qf5D9Fc3S+BHzzObX/7wD/2U/429dX3ycHuL76nhlXbP8G8PnV8xC4u7LzSq/BVUcCXwDuK6UeKqUy4HeBr12xTX8efA34e6vnfw/4G1dnyr8KpdQfAaMfe/nTbP4a8D8pjT8BmqsV9FeGT7H/0/A14HeVUqlS6hF6Qe4X/sKM+ymglDpWSn139XwO3AauccXX4KqdwDXg4BO/H65e+yxAAb8vhPiOEOI/XL3WVz9cw34C9K/GtD8TPs3mz9K1+Y9X4fLf/UQK9lzbL4TYA34O+CZXfA2u2gl8lvFlpdTngb8G/EdCiK988k2l47nPVOvls2gz8N8BN4B3gGPgv7pSa34KCCFqwP8C/KdKqdkn37uKa3DVTuAI2P7E71ur1557KKWOVj/PgH+MDjVPL8K11c+zq7Pwp8an2fyZuDZKqVOlVKmUqoD/gR+G/M+l/UIIC+0A/r5S6h+tXr7Sa3DVTuA94CUhxHUhhA38TeD3rtimPxVCiEAIEV48B34duIW2/W+t/uxvAf/r1Vj4Z8Kn2fx7wL+/qlB/CZh+ImR9bvBjOfJvoa8DaPv/phDCEUJcB14CvvWv275PQui1S/8jcFsp9V9/4q2rvQZXWS39RAX0Lrp6+7ev2p6f0uZ9dOX5+8CHF3YDHeCfA/eA/xtoX7WtP2b3P0SHzDk6v/wPPs1mdEX6v11dlx8A7z6n9v/PK/s+WB2ajU/8/d9e2X8H+GvPgf1fRof6HwDvrx6/cdXX4AVj8AVe4GccV50OvMALvMAV44UTeIEX+BnHCyfwAi/wM44XTuAFXuBnHC+cwAu8wM84XjiBF3iBn3G8cAIv8AI/43jhBF7gBX7G8f8Ag1cGsYI8cBsAAAAASUVORK5CYII=", "text/plain": [ "
" ] @@ -314,7 +314,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The ONNX format is a framework-agnostic way of describing and saving the structure and state of deep learning models. We can convert Tensorflow 2 Keras models to ONNX using the keras2onnx tool provided by the ONNX project. (You can find the ONNX project here: https://onnx.ai or on GitHub here: https://github.com/onnx/onnx)" + "The ONNX format is a framework-agnostic way of describing and saving the structure and state of deep learning models. We can convert Tensorflow 2 Keras models to ONNX using the tf2onnx tool provided by the ONNX project. (You can find the ONNX project here: https://onnx.ai or on GitHub here: https://github.com/onnx/onnx)" ] }, { @@ -325,7 +325,7 @@ }, "outputs": [], "source": [ - "import onnx, keras2onnx" + "import onnx" ] }, { @@ -548,10 +548,7 @@ "source": [ "model.save('my_model')\n", "!python -m tf2onnx.convert --saved-model my_model --output temp.onnx\n", - "onnx_model = onnx.load_model('temp.onnx')\n", - "\n", - "# This can also be done with keras2onnx:\n", - "# onnx_model = keras2onnx.convert_keras(model, model.name)" + "onnx_model = onnx.load_model('temp.onnx')" ] }, { diff --git a/quickstart/IntroNotebooks/Additional Examples/helper.py b/quickstart/IntroNotebooks/Additional Examples/helper.py index 6cbe417d..66c4e006 100644 --- a/quickstart/IntroNotebooks/Additional Examples/helper.py +++ b/quickstart/IntroNotebooks/Additional Examples/helper.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/quickstart/IntroNotebooks/helper.py b/quickstart/IntroNotebooks/helper.py index 6cbe417d..66c4e006 100644 --- a/quickstart/IntroNotebooks/helper.py +++ b/quickstart/IntroNotebooks/helper.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/quickstart/IntroNotebooks/onnx_helper.py b/quickstart/IntroNotebooks/onnx_helper.py index ccd88f7a..6bea97dd 100644 --- a/quickstart/IntroNotebooks/onnx_helper.py +++ b/quickstart/IntroNotebooks/onnx_helper.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/quickstart/Makefile b/quickstart/Makefile index cbdcced9..bf728ff4 100644 --- a/quickstart/Makefile +++ b/quickstart/Makefile @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/quickstart/Makefile.config b/quickstart/Makefile.config index 2c1234ba..d81f325d 100644 --- a/quickstart/Makefile.config +++ b/quickstart/Makefile.config @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/quickstart/SemanticSegmentation/Makefile b/quickstart/SemanticSegmentation/Makefile index e119309b..5c1bdea3 100644 --- a/quickstart/SemanticSegmentation/Makefile +++ b/quickstart/SemanticSegmentation/Makefile @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/quickstart/SemanticSegmentation/export.py b/quickstart/SemanticSegmentation/export.py index f106094a..e5168aaa 100644 --- a/quickstart/SemanticSegmentation/export.py +++ b/quickstart/SemanticSegmentation/export.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/quickstart/SemanticSegmentation/tutorial-runtime.cpp b/quickstart/SemanticSegmentation/tutorial-runtime.cpp index b6d31911..7f0854a3 100644 --- a/quickstart/SemanticSegmentation/tutorial-runtime.cpp +++ b/quickstart/SemanticSegmentation/tutorial-runtime.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/quickstart/common/logger.cpp b/quickstart/common/logger.cpp index d795754c..2eaccd54 100644 --- a/quickstart/common/logger.cpp +++ b/quickstart/common/logger.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/quickstart/common/logger.h b/quickstart/common/logger.h index c60c447e..513275c2 100644 --- a/quickstart/common/logger.h +++ b/quickstart/common/logger.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/quickstart/common/logging.h b/quickstart/common/logging.h index fcd8efb5..f323d22b 100644 --- a/quickstart/common/logging.h +++ b/quickstart/common/logging.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/quickstart/common/util.cpp b/quickstart/common/util.cpp index be301128..717b63aa 100644 --- a/quickstart/common/util.cpp +++ b/quickstart/common/util.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/quickstart/common/util.h b/quickstart/common/util.h index 571645f9..50455e97 100644 --- a/quickstart/common/util.h +++ b/quickstart/common/util.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/quickstart/deploy_to_triton/config.pbtxt b/quickstart/deploy_to_triton/config.pbtxt index 4ffbdc88..63046c8d 100644 --- a/quickstart/deploy_to_triton/config.pbtxt +++ b/quickstart/deploy_to_triton/config.pbtxt @@ -1,30 +1,36 @@ -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -# SPDX-License-Identifier: LicenseRef-NvidiaProprietary -# -# NVIDIA CORPORATION, its affiliates and licensors retain all intellectual -# property and proprietary rights in and to this material, related -# documentation and any modifications thereto. Any use, reproduction, -# disclosure or distribution of this material and related documentation -# without an express license agreement from NVIDIA CORPORATION or -# its affiliates is strictly prohibited. - - -name: "resnet50" -platform: "tensorrt_plan" -max_batch_size : 0 -input [ - { - name: "input" - data_type: TYPE_FP32 - dims: [ 3, 224, 224 ] - reshape { shape: [ 1, 3, 224, 224 ] } - } -] -output [ - { - name: "output" - data_type: TYPE_FP32 - dims: [ 1, 1000 ,1, 1] - reshape { shape: [ 1, 1000 ] } - } -] +# +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +name: "resnet50" +platform: "tensorrt_plan" +max_batch_size : 0 +input [ + { + name: "input" + data_type: TYPE_FP32 + dims: [ 3, 224, 224 ] + reshape { shape: [ 1, 3, 224, 224 ] } + } +] +output [ + { + name: "output" + data_type: TYPE_FP32 + dims: [ 1, 1000 ,1, 1] + reshape { shape: [ 1, 1000 ] } + } +] diff --git a/quickstart/deploy_to_triton/export_resnet_to_onnx.py b/quickstart/deploy_to_triton/export_resnet_to_onnx.py index f8357a75..fba1550a 100644 --- a/quickstart/deploy_to_triton/export_resnet_to_onnx.py +++ b/quickstart/deploy_to_triton/export_resnet_to_onnx.py @@ -1,27 +1,34 @@ -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -# SPDX-License-Identifier: LicenseRef-NvidiaProprietary -# -# NVIDIA CORPORATION, its affiliates and licensors retain all intellectual -# property and proprietary rights in and to this material, related -# documentation and any modifications thereto. Any use, reproduction, -# disclosure or distribution of this material and related documentation -# without an express license agreement from NVIDIA CORPORATION or -# its affiliates is strictly prohibited. - -import torch -import torchvision.models as models - -torch.hub._validate_not_a_forked_repo=lambda a,b,c: True - -# load model; We are going to use a pretrained resnet model -model = models.resnet50(pretrained=True).eval() -x = torch.randn(1, 3, 224, 224, requires_grad=True) - -# Export the model -torch.onnx.export(model, # model being run - x, # model input (or a tuple for multiple inputs) - "resnet50.onnx", # where to save the model (can be a file or file-like object) - export_params=True, # store the trained parameter weights inside the model file - input_names = ['input'], # the model's input names - output_names = ['output'], # the model's output names - ) +# +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import torch +import torchvision.models as models + +torch.hub._validate_not_a_forked_repo=lambda a,b,c: True + +# load model; We are going to use a pretrained resnet model +model = models.resnet50(pretrained=True).eval() +x = torch.randn(1, 3, 224, 224, requires_grad=True) + +# Export the model +torch.onnx.export(model, # model being run + x, # model input (or a tuple for multiple inputs) + "resnet50.onnx", # where to save the model (can be a file or file-like object) + export_params=True, # store the trained parameter weights inside the model file + input_names = ['input'], # the model's input names + output_names = ['output'], # the model's output names + ) diff --git a/quickstart/deploy_to_triton/triton_client.py b/quickstart/deploy_to_triton/triton_client.py index 8448efb3..a6e7553d 100644 --- a/quickstart/deploy_to_triton/triton_client.py +++ b/quickstart/deploy_to_triton/triton_client.py @@ -1,42 +1,49 @@ -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -# SPDX-License-Identifier: LicenseRef-NvidiaProprietary -# -# NVIDIA CORPORATION, its affiliates and licensors retain all intellectual -# property and proprietary rights in and to this material, related -# documentation and any modifications thereto. Any use, reproduction, -# disclosure or distribution of this material and related documentation -# without an express license agreement from NVIDIA CORPORATION or -# its affiliates is strictly prohibited. - -import numpy as np -from torchvision import transforms -from PIL import Image -import tritonclient.http as httpclient -from tritonclient.utils import triton_to_np_dtype - -def rn50_preprocess(img_path="img1.jpg"): - img = Image.open(img_path) - preprocess = transforms.Compose([ - transforms.Resize(256), - transforms.CenterCrop(224), - transforms.ToTensor(), - transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]), - ]) - return preprocess(img).numpy() - -transformed_img = rn50_preprocess() - -# Setup a connection with the Triton Inference Server. -triton_client = httpclient.InferenceServerClient(url="localhost:8000") - -# Specify the names of the input and output layer(s) of our model. -test_input = httpclient.InferInput("input", transformed_img.shape, datatype="FP32") -test_input.set_data_from_numpy(transformed_img, binary_data=True) - -test_output = httpclient.InferRequestedOutput("output", binary_data=True, class_count=1000) - -# Querying the server -results = triton_client.infer(model_name="resnet50", inputs=[test_input], outputs=[test_output]) -test_output_fin = results.as_numpy('output') - -print(test_output_fin[:5]) +# +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import numpy as np +from torchvision import transforms +from PIL import Image +import tritonclient.http as httpclient +from tritonclient.utils import triton_to_np_dtype + +def rn50_preprocess(img_path="img1.jpg"): + img = Image.open(img_path) + preprocess = transforms.Compose([ + transforms.Resize(256), + transforms.CenterCrop(224), + transforms.ToTensor(), + transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]), + ]) + return preprocess(img).numpy() + +transformed_img = rn50_preprocess() + +# Setup a connection with the Triton Inference Server. +triton_client = httpclient.InferenceServerClient(url="localhost:8000") + +# Specify the names of the input and output layer(s) of our model. +test_input = httpclient.InferInput("input", transformed_img.shape, datatype="FP32") +test_input.set_data_from_numpy(transformed_img, binary_data=True) + +test_output = httpclient.InferRequestedOutput("output", binary_data=True, class_count=1000) + +# Querying the server +results = triton_client.infer(model_name="resnet50", inputs=[test_input], outputs=[test_output]) +test_output_fin = results.as_numpy('output') + +print(test_output_fin[:5]) diff --git a/requirements.txt b/requirements.txt index cb4cf74b..f87a9b0c 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,5 +1,4 @@ -onnx==1.10.2; python_version<"3.10" -onnx==1.12.0; python_version=="3.10" +onnx tensorflow-gpu==2.9.1; (platform_machine=="x86_64" and sys.platform=="linux" and python_version>="3.7") onnxruntime==1.8.1; python_version<"3.10" onnxruntime==1.12.1; python_version=="3.10" diff --git a/samples/CMakeLists.txt b/samples/CMakeLists.txt index 73a0716a..ff543aaf 100644 --- a/samples/CMakeLists.txt +++ b/samples/CMakeLists.txt @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -25,6 +25,7 @@ set(OPENSOURCE_SAMPLES_LIST sampleOnnxMNIST sampleIOFormats sampleOnnxMnistCoordConvAC + sampleNamedDimensions trtexec) foreach(SAMPLE_ITER ${OPENSOURCE_SAMPLES_LIST}) diff --git a/samples/CMakeSamplesTemplate.txt b/samples/CMakeSamplesTemplate.txt index 47d2c240..b355fa1d 100644 --- a/samples/CMakeSamplesTemplate.txt +++ b/samples/CMakeSamplesTemplate.txt @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/common/BatchStream.h b/samples/common/BatchStream.h index 4990aade..c08c9e14 100644 --- a/samples/common/BatchStream.h +++ b/samples/common/BatchStream.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); @@ -120,7 +120,7 @@ class MNISTBatchStream : public IBatchStream file.read(reinterpret_cast(rawData.data()), numElements * sizeof(uint8_t)); mData.resize(numElements); std::transform( - rawData.begin(), rawData.end(), mData.begin(), [](uint8_t val) { return static_cast(val) / 255.f; }); + rawData.begin(), rawData.end(), mData.begin(), [](uint8_t val) { return static_cast(val) / 255.F; }); } void readLabelsFile(const std::string& labelsFilePath) @@ -177,7 +177,6 @@ class BatchStream : public IBatchStream mLabels.resize(mBatchSize, 0); mFileBatch.resize(mDims.d[0] * mImageSize, 0); mFileLabels.resize(mDims.d[0], 0); - reset(0); } BatchStream(int batchSize, int maxBatches, std::string const& prefix, std::vector const& directories) @@ -198,7 +197,6 @@ class BatchStream : public IBatchStream mLabels.resize(mBatchSize, 0); mFileBatch.resize(mDims.d[0] * mImageSize, 0); mFileLabels.resize(mDims.d[0], 0); - reset(0); } // Resets data members diff --git a/samples/common/EntropyCalibrator.h b/samples/common/EntropyCalibrator.h index 366b9beb..936d10e0 100644 --- a/samples/common/EntropyCalibrator.h +++ b/samples/common/EntropyCalibrator.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/common/ErrorRecorder.h b/samples/common/ErrorRecorder.h index f6b4c3ee..3cc8ef9e 100644 --- a/samples/common/ErrorRecorder.h +++ b/samples/common/ErrorRecorder.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/common/argsParser.h b/samples/common/argsParser.h index 52f42ab0..3b80797c 100644 --- a/samples/common/argsParser.h +++ b/samples/common/argsParser.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/common/buffers.h b/samples/common/buffers.h index 271dab52..6d87a11a 100644 --- a/samples/common/buffers.h +++ b/samples/common/buffers.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/common/common.h b/samples/common/common.h index 9437217b..b2a01c5e 100644 --- a/samples/common/common.h +++ b/samples/common/common.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); @@ -271,8 +271,8 @@ inline void readPGMFile(const std::string& fileName, uint8_t* buffer, int inH, i { std::ifstream infile(fileName, std::ifstream::binary); assert(infile.is_open() && "Attempting to read from a file that is not open."); - std::string magic, h, w, max; - infile >> magic >> h >> w >> max; + std::string magic, w, h, max; + infile >> magic >> w >> h >> max; infile.seekg(1, infile.cur); infile.read(reinterpret_cast(buffer), inH * inW); } @@ -352,7 +352,7 @@ inline void* safeCudaMalloc(size_t memSize) if (deviceMem == nullptr) { std::cerr << "Out of memory" << std::endl; - exit(1); + exit(EXIT_FAILURE); } return deviceMem; } @@ -535,7 +535,7 @@ inline int32_t calculateSoftmax(float* const prob, int32_t const numDigits) // // The default parameter values choosen arbitrarily. Range values should be choosen such that // we avoid underflow or overflow. Also range value should be non zero to avoid uniform zero scale tensor. -inline void setAllDynamicRanges(nvinfer1::INetworkDefinition* network, float inRange = 2.0f, float outRange = 4.0f) +inline void setAllDynamicRanges(nvinfer1::INetworkDefinition* network, float inRange = 2.0F, float outRange = 4.0F) { // Ensure that all layer inputs have a scale. for (int i = 0; i < network->getNbLayers(); i++) @@ -709,7 +709,7 @@ void writePPMFileWithBBox(const std::string& filename, PPM& ppm, const << ppm.w << " " << ppm.h << "\n" << ppm.max << "\n"; - auto round = [](float x) -> int { return int(std::floor(x + 0.5f)); }; + auto round = [](float x) -> int { return int(std::floor(x + 0.5F)); }; const int x1 = std::min(std::max(0, round(int(bbox.x1))), W - 1); const int x2 = std::min(std::max(0, round(int(bbox.x2))), W - 1); const int y1 = std::min(std::max(0, round(int(bbox.y1))), H - 1); @@ -750,7 +750,7 @@ inline void writePPMFileWithBBox(const std::string& filename, vPPM ppm, std::vec << "\n" << ppm.w << " " << ppm.h << "\n" << ppm.max << "\n"; - auto round = [](float x) -> int { return int(std::floor(x + 0.5f)); }; + auto round = [](float x) -> int { return int(std::floor(x + 0.5F)); }; for (auto bbox : dets) { @@ -789,7 +789,7 @@ class TimerBase virtual void stop() {} float microseconds() const noexcept { - return mMs * 1000.f; + return mMs * 1000.F; } float milliseconds() const noexcept { @@ -797,15 +797,15 @@ class TimerBase } float seconds() const noexcept { - return mMs / 1000.f; + return mMs / 1000.F; } void reset() noexcept { - mMs = 0.f; + mMs = 0.F; } protected: - float mMs{0.0f}; + float mMs{0.0F}; }; class GpuTimer : public TimerBase @@ -829,7 +829,7 @@ class GpuTimer : public TimerBase void stop() override { CHECK(cudaEventRecord(mStop, mStream)); - float ms{0.0f}; + float ms{0.0F}; CHECK(cudaEventSynchronize(mStop)); CHECK(cudaEventElapsedTime(&ms, mStart, mStop)); mMs += ms; diff --git a/samples/common/dumpTFWts.py b/samples/common/dumpTFWts.py index 4afa36f5..0b7a0123 100644 --- a/samples/common/dumpTFWts.py +++ b/samples/common/dumpTFWts.py @@ -1,6 +1,6 @@ #!/usr/bin/python # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/common/getOptions.cpp b/samples/common/getOptions.cpp index f55caa51..8bcf7958 100644 --- a/samples/common/getOptions.cpp +++ b/samples/common/getOptions.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/common/getOptions.h b/samples/common/getOptions.h index e4d4276a..e8460513 100644 --- a/samples/common/getOptions.h +++ b/samples/common/getOptions.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/common/getoptWin.h b/samples/common/getoptWin.h index 5f7f9fb0..7e1cf1ba 100644 --- a/samples/common/getoptWin.h +++ b/samples/common/getoptWin.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/common/half.h b/samples/common/half.h index 23b282e8..c5ebdb1a 100644 --- a/samples/common/half.h +++ b/samples/common/half.h @@ -16,7 +16,7 @@ // OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); @@ -1523,14 +1523,14 @@ class half /// \return incremented half value half& operator++() { - return *this += 1.0f; + return *this += 1.0F; } /// Prefix decrement. /// \return decremented half value half& operator--() { - return *this -= 1.0f; + return *this -= 1.0F; } /// Postfix increment. diff --git a/samples/common/logger.cpp b/samples/common/logger.cpp index 1f303cb0..0592db2c 100644 --- a/samples/common/logger.cpp +++ b/samples/common/logger.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/common/logger.h b/samples/common/logger.h index 94a95b7b..ff59bfa9 100644 --- a/samples/common/logger.h +++ b/samples/common/logger.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/common/logging.h b/samples/common/logging.h index 17aa58e6..38cbbd01 100644 --- a/samples/common/logging.h +++ b/samples/common/logging.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/common/parserOnnxConfig.h b/samples/common/parserOnnxConfig.h index 20ca20ac..b1c4e434 100644 --- a/samples/common/parserOnnxConfig.h +++ b/samples/common/parserOnnxConfig.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/common/safeCommon.h b/samples/common/safeCommon.h index 9cf8e4ec..326257ab 100644 --- a/samples/common/safeCommon.h +++ b/samples/common/safeCommon.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); @@ -154,11 +154,7 @@ class TrtCudaGraphSafe void endCapture(cudaStream_t& stream) { CHECK(cudaStreamEndCapture(stream, &mGraph)); -#if CUDART_VERSION >= 12000 - CHECK(cudaGraphInstantiate(&mGraphExec, mGraph, 0)); -#else CHECK(cudaGraphInstantiate(&mGraphExec, mGraph, nullptr, nullptr, 0)); -#endif CHECK(cudaGraphDestroy(mGraph)); } diff --git a/samples/common/sampleConfig.h b/samples/common/sampleConfig.h index bb2eb30b..6402b448 100644 --- a/samples/common/sampleConfig.h +++ b/samples/common/sampleConfig.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); @@ -56,9 +56,9 @@ class SampleConfig : public nvonnxparser::IOnnxConfig bool mDebugBuilder{false}; InputDataFormat mInputDataFormat{InputDataFormat::kASCII}; uint64_t mTopK{0}; - float mFailurePercentage{-1.0f}; - float mTolerance{0.0f}; - float mAbsTolerance{1e-5f}; + float mFailurePercentage{-1.0F}; + float mTolerance{0.0F}; + float mAbsTolerance{1e-5F}; public: SampleConfig() @@ -313,7 +313,7 @@ class SampleConfig : public nvonnxparser::IOnnxConfig { return mTimingCacheFilename.c_str(); } - + void setTimingCacheFileName(const char* timingCacheFilename) noexcept { mTimingCacheFilename = std::string(timingCacheFilename); diff --git a/samples/common/sampleDevice.h b/samples/common/sampleDevice.h index f73889b8..83cd53c3 100644 --- a/samples/common/sampleDevice.h +++ b/samples/common/sampleDevice.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/common/sampleEngines.cpp b/samples/common/sampleEngines.cpp index b9b838d9..8a442083 100644 --- a/samples/common/sampleEngines.cpp +++ b/samples/common/sampleEngines.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); @@ -236,8 +236,8 @@ void setTensorScalesFromCalibration(nvinfer1::INetworkDefinition& network, std:: //! //! \see Parser::operator bool() //! -Parser modelToNetwork(const ModelOptions& model, nvinfer1::INetworkDefinition& network, std::ostream& err, - std::vector* vcPluginLibrariesUsed) +Parser modelToNetwork(ModelOptions const& model, BuildOptions const& build, nvinfer1::INetworkDefinition& network, + std::ostream& err, std::vector* vcPluginLibrariesUsed) { sample::gLogInfo << "Start parsing network model." << std::endl; auto const tBegin = std::chrono::high_resolution_clock::now(); @@ -310,6 +310,15 @@ Parser modelToNetwork(const ModelOptions& model, nvinfer1::INetworkDefinition& n { using namespace nvonnxparser; parser.onnxParser.reset(createONNXParser(network)); + ASSERT(parser.onnxParser != nullptr); + // For version or hardware compatible engines, we must use TensorRT's native InstanceNorm implementation for + // compatibility. + if (build.versionCompatible + || (build.hardwareCompatibilityLevel != nvinfer1::HardwareCompatibilityLevel::kNONE)) + { + auto parserflags = 1U << static_cast(OnnxParserFlag::kNATIVE_INSTANCENORM); + parser.onnxParser->setFlags(parserflags); + } if (!parser.onnxParser->parseFromFile( model.baseModel.model.c_str(), static_cast(sample::gLogger.getReportableSeverity()))) { @@ -648,7 +657,20 @@ void setMemoryPoolLimits(IBuilderConfig& config, BuildOptions const& build) } if (build.dlaSRAM >= 0) { - config.setMemoryPoolLimit(MemoryPoolType::kDLA_MANAGED_SRAM, roundToBytes(build.dlaSRAM)); + size_t const sizeInBytes = roundToBytes(build.dlaSRAM); + size_t sizeInPowerOf2{1}; + // Using 2^30 bytes as a loose upper bound to prevent the possibility of overflows and infinite loops. + while (sizeInPowerOf2 < 31 && (static_cast(1) << sizeInPowerOf2) <= sizeInBytes) + { + ++sizeInPowerOf2; + } + --sizeInPowerOf2; + if (sizeInPowerOf2 == 30) + { + sample::gLogWarning << "User-specified DLA managed SRAM size is too large and has been clipped to 2^30 bytes. " + << "Please make sure that this is the intended managed SRAM size." << std::endl; + } + config.setMemoryPoolLimit(MemoryPoolType::kDLA_MANAGED_SRAM, static_cast(1) << sizeInPowerOf2); } if (build.dlaLocalDRAM >= 0) { @@ -886,7 +908,10 @@ bool setupNetworkAndConfig(BuildOptions const& build, SystemOptions const& sys, config.setFlag(BuilderFlag::kENABLE_TACTIC_HEURISTIC); } - config.setBuilderOptimizationLevel(build.builderOptimizationLevel); + if (build.builderOptimizationLevel != defaultBuilderOptimizationLevel) + { + config.setBuilderOptimizationLevel(build.builderOptimizationLevel); + } if (build.timingCacheMode == TimingCacheMode::kDISABLE) { @@ -1087,9 +1112,9 @@ bool setupNetworkAndConfig(BuildOptions const& build, SystemOptions const& sys, setLayerDeviceTypes(network, config, build.layerDeviceTypes); } - if (build.safe) + if (build.safe && sys.DLACore == -1) { - config.setEngineCapability(sys.DLACore != -1 ? EngineCapability::kDLA_STANDALONE : EngineCapability::kSAFETY); + config.setEngineCapability(EngineCapability::kSAFETY); } if (build.restricted) @@ -1104,8 +1129,11 @@ bool setupNetworkAndConfig(BuildOptions const& build, SystemOptions const& sys, config.setDefaultDeviceType(DeviceType::kDLA); config.setDLACore(sys.DLACore); config.setFlag(BuilderFlag::kPREFER_PRECISION_CONSTRAINTS); - - if (sys.fallback) + if (build.buildDLAStandalone) + { + config.setEngineCapability(EngineCapability::kDLA_STANDALONE); + } + if (build.allowGPUFallback) { config.setFlag(BuilderFlag::kGPU_FALLBACK); } @@ -1214,7 +1242,8 @@ bool modelToBuildEnv( std::vector vcPluginLibrariesUsed; SMP_RETVAL_IF_FALSE(env.network != nullptr, "Network creation failed", false, err); - env.parser = modelToNetwork(model, *env.network, err, build.versionCompatible ? &vcPluginLibrariesUsed : nullptr); + env.parser + = modelToNetwork(model, build, *env.network, err, build.versionCompatible ? &vcPluginLibrariesUsed : nullptr); SMP_RETVAL_IF_FALSE(env.parser.operator bool(), "Parsing model failed", false, err); if (build.versionCompatible && !sys.ignoreParsedPluginLibs && !vcPluginLibrariesUsed.empty()) diff --git a/samples/common/sampleEngines.h b/samples/common/sampleEngines.h index b53e1303..6c0a88b6 100644 --- a/samples/common/sampleEngines.h +++ b/samples/common/sampleEngines.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/common/sampleInference.cpp b/samples/common/sampleInference.cpp index e3fb116c..84850d93 100644 --- a/samples/common/sampleInference.cpp +++ b/samples/common/sampleInference.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); @@ -93,7 +93,7 @@ class FillBindingClosure void fillOneBinding(TensorInfo const& tensorInfo) { auto const name = tensorInfo.name; - auto const* bindingInOutStr = tensorInfo.isInput ? "input" : "output"; + auto const* bindingInOutStr = tensorInfo.isInput ? "Input" : "Output"; for (auto& binding : bindings) { auto const input = inputs.find(name); @@ -104,11 +104,23 @@ class FillBindingClosure } else { - sample::gLogInfo << "Using random values for " << bindingInOutStr << " " << name << std::endl; + if (tensorInfo.isInput) + { + sample::gLogInfo << "Using random values for input " << name << std::endl; + } binding->addBinding(tensorInfo); } - sample::gLogInfo << "Created " << bindingInOutStr << " binding for " << name << " with dimensions " - << tensorInfo.dims << std::endl; + if (tensorInfo.isDynamic) + { + sample::gLogInfo << bindingInOutStr << " binding for " << name + << " is dynamic and will be created during execution using OutputAllocator." + << std::endl; + } + else + { + sample::gLogInfo << bindingInOutStr << " binding for " << name << " with dimensions " << tensorInfo.dims + << " is created." << std::endl; + } } } @@ -505,14 +517,19 @@ class EnqueueImplicit : private Enqueue bool operator()(TrtCudaStream& stream) const { - if (mContext.enqueue(mBatch, mBuffers, stream.get(), nullptr)) + try { + bool const result = mContext.enqueue(mBatch, mBuffers, stream.get(), nullptr); // Collecting layer timing info from current profile index of execution context if (mContext.getProfiler() && !mContext.getEnqueueEmitsProfile() && !mContext.reportToProfiler()) { gLogWarning << "Failed to collect layer timing info from previous enqueue()" << std::endl; } - return true; + return result; + } + catch (const std::exception&) + { + return false; } return false; } @@ -539,14 +556,19 @@ class EnqueueExplicit : private Enqueue bool operator()(TrtCudaStream& stream) const { - if (mContext.enqueueV3(stream.get())) + try { + bool const result = mContext.enqueueV3(stream.get()); // Collecting layer timing info from current profile index of execution context if (mContext.getProfiler() && !mContext.getEnqueueEmitsProfile() && !mContext.reportToProfiler()) { gLogWarning << "Failed to collect layer timing info from previous enqueueV3()" << std::endl; } - return true; + return result; + } + catch (const std::exception&) + { + return false; } return false; } @@ -624,9 +646,13 @@ class EnqueueSafe bool operator()(TrtCudaStream& stream) const { - if (mContext.enqueueV3(stream.get())) + try { - return true; + return mContext.enqueueV3(stream.get()); + } + catch (const std::exception&) + { + return false; } return false; } @@ -854,7 +880,10 @@ class Iteration // Avoid capturing initialization calls by executing the enqueue function at least // once before starting CUDA graph capture. auto const ret = mEnqueue(stream); - assert(ret); + if (!ret) + { + throw std::runtime_error("Inference enqueue failed."); + } stream.synchronize(); mGraph.beginCapture(stream); @@ -922,6 +951,26 @@ bool inferenceLoop(std::vector>>& iStream float durationMs = 0; int32_t skip = 0; + if (maxDurationMs == -1.F) + { + sample::gLogWarning << "--duration=-1 is specified, inference will run in an endless loop until" + << " aborted with CTRL-C (SIGINT)" << std::endl; + while (true) + { + for (auto& s : iStreams) + { + if (!s->query(skipTransfers)) + { + return false; + } + } + for (auto& s : iStreams) + { + s->sync(cpuStart, gpuStart, trace, skipTransfers); + } + } + } + for (int32_t i = 0; i < iterations + skip || durationMs < maxDurationMs; ++i) { for (auto& s : iStreams) @@ -957,50 +1006,65 @@ bool inferenceLoop(std::vector>>& iStream template void inferenceExecution(InferenceOptions const& inference, InferenceEnvironment& iEnv, SyncStruct& sync, - int32_t const threadIdx, int32_t const streamsPerThread, int32_t device, std::vector& trace) + int32_t const threadIdx, int32_t const streamsPerThread, int32_t device, std::vector& trace) noexcept { - float warmupMs = inference.warmup; - float durationMs = inference.duration * 1000.F + warmupMs; + try + { + float warmupMs = inference.warmup; + float durationMs = -1.F; + if (inference.duration != -1.F) + { + durationMs = inference.duration * 1000.F + warmupMs; + } - cudaCheck(cudaSetDevice(device)); + cudaCheck(cudaSetDevice(device)); - std::vector>> iStreams; + std::vector>> iStreams; - for (int32_t s = 0; s < streamsPerThread; ++s) - { - int32_t const streamId{threadIdx * streamsPerThread + s}; - auto* iteration = new Iteration( - streamId, inference, *iEnv.template getContext(streamId), *iEnv.bindings[streamId]); - if (inference.skipTransfers) + for (int32_t s = 0; s < streamsPerThread; ++s) { - iteration->setInputData(true); + int32_t const streamId{threadIdx * streamsPerThread + s}; + auto* iteration = new Iteration( + streamId, inference, *iEnv.template getContext(streamId), *iEnv.bindings[streamId]); + if (inference.skipTransfers) + { + iteration->setInputData(true); + } + iStreams.emplace_back(iteration); } - iStreams.emplace_back(iteration); - } - for (auto& s : iStreams) - { - s->wait(sync.gpuStart); - } + for (auto& s : iStreams) + { + s->wait(sync.gpuStart); + } - std::vector localTrace; - if (!inferenceLoop(iStreams, sync.cpuStart, sync.gpuStart, inference.iterations, durationMs, warmupMs, localTrace, - inference.skipTransfers, inference.idle)) - { - iEnv.error = true; - } + std::vector localTrace; + if (!inferenceLoop(iStreams, sync.cpuStart, sync.gpuStart, inference.iterations, durationMs, warmupMs, localTrace, + inference.skipTransfers, inference.idle)) + { + sync.mutex.lock(); + iEnv.error = true; + sync.mutex.unlock(); + } - if (inference.skipTransfers) - { - for (auto& s : iStreams) + if (inference.skipTransfers) { - s->fetchOutputData(true); + for (auto& s : iStreams) + { + s->fetchOutputData(true); + } } - } - sync.mutex.lock(); - trace.insert(trace.end(), localTrace.begin(), localTrace.end()); - sync.mutex.unlock(); + sync.mutex.lock(); + trace.insert(trace.end(), localTrace.begin(), localTrace.end()); + sync.mutex.unlock(); + } + catch(...) + { + sync.mutex.lock(); + iEnv.error = true; + sync.mutex.unlock(); + } } inline std::thread makeThread(InferenceOptions const& inference, InferenceEnvironment& iEnv, SyncStruct& sync, diff --git a/samples/common/sampleInference.h b/samples/common/sampleInference.h index e906552b..909a71b8 100644 --- a/samples/common/sampleInference.h +++ b/samples/common/sampleInference.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/common/sampleOptions.cpp b/samples/common/sampleOptions.cpp index 16913c3e..41a13972 100644 --- a/samples/common/sampleOptions.cpp +++ b/samples/common/sampleOptions.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); @@ -957,6 +957,8 @@ void BuildOptions::parse(Arguments& arguments) throw std::invalid_argument("Invalid usage, fp8 and int8 aren't allowed to be enabled together."); } getAndDelOption(arguments, "--safe", safe); + getAndDelOption(arguments, "--buildDLAStandalone", buildDLAStandalone); + getAndDelOption(arguments, "--allowGPUFallback", allowGPUFallback); getAndDelOption(arguments, "--consistency", consistency); getAndDelOption(arguments, "--restricted", restricted); if (getAndDelOption(arguments, "--buildOnly", skipInference)) @@ -1145,7 +1147,12 @@ void BuildOptions::parse(Arguments& arguments) { timingCacheMode = TimingCacheMode::kLOCAL; } - getAndDelOption(arguments, "--heuristic", heuristic); + if (getAndDelOption(arguments, "--heuristic", heuristic)) + { + sample::gLogWarning << "--heuristic flag has been deprecated, use --builderOptimizationLevel= flag instead " + "(N <= 2 enables heuristic)." + << std::endl; + } getAndDelOption(arguments, "--builderOptimizationLevel", builderOptimizationLevel); std::string hardwareCompatibleArgs; @@ -1245,7 +1252,6 @@ void SystemOptions::parse(Arguments& arguments) { getAndDelOption(arguments, "--device", device); getAndDelOption(arguments, "--useDLACore", DLACore); - getAndDelOption(arguments, "--allowGPUFallback", fallback); std::string pluginName; while (getAndDelOption(arguments, "--plugins", pluginName)) { @@ -1431,28 +1437,37 @@ void AllOptions::parse(Arguments& arguments) } if (build.safe && system.DLACore >= 0) { - auto checkSafeDLAFormats = [](std::vector const& fmt) { - return fmt.empty() ? false : std::all_of(fmt.begin(), fmt.end(), [](IOFormat const& pair) { + build.buildDLAStandalone = true; + } + if (build.buildDLAStandalone) + { + build.skipInference = true; + auto checkSafeDLAFormats = [](std::vector const& fmt, bool isInput) { + return fmt.empty() ? false : std::all_of(fmt.begin(), fmt.end(), [&](IOFormat const& pair) { bool supported{false}; bool const isDLA_LINEAR{ pair.second == 1U << static_cast(nvinfer1::TensorFormat::kDLA_LINEAR)}; - bool const isCHW4{pair.second == 1U << static_cast(nvinfer1::TensorFormat::kCHW4)}; + bool const isHWC4{pair.second == 1U << static_cast(nvinfer1::TensorFormat::kCHW4) + || pair.second == 1U << static_cast(nvinfer1::TensorFormat::kDLA_HWC4)}; bool const isCHW32{pair.second == 1U << static_cast(nvinfer1::TensorFormat::kCHW32)}; bool const isCHW16{pair.second == 1U << static_cast(nvinfer1::TensorFormat::kCHW16)}; - supported |= pair.first == nvinfer1::DataType::kINT8 && (isDLA_LINEAR || isCHW4 || isCHW32); - supported |= pair.first == nvinfer1::DataType::kHALF && (isDLA_LINEAR || isCHW4 || isCHW16); + supported |= pair.first == nvinfer1::DataType::kINT8 + && (isDLA_LINEAR || (isInput ? isHWC4 : false) || isCHW32); + supported |= pair.first == nvinfer1::DataType::kHALF + && (isDLA_LINEAR || (isInput ? isHWC4 : false) || isCHW16); return supported; }); }; - if (!checkSafeDLAFormats(build.inputFormats) || !checkSafeDLAFormats(build.outputFormats)) + if (!checkSafeDLAFormats(build.inputFormats, true) || !checkSafeDLAFormats(build.outputFormats, false)) { throw std::invalid_argument( - "I/O formats for safe DLA capability are restricted to fp16/int8:dla_linear, fp16:chw16 or " + "I/O formats for safe DLA capability are restricted to fp16/int8:dla_linear, fp16/int8:hwc4, " + "fp16:chw16 or " "int8:chw32"); } - if (system.fallback) + if (build.allowGPUFallback) { - throw std::invalid_argument("GPU fallback (--allowGPUFallback) not allowed for safe DLA capability"); + throw std::invalid_argument("GPU fallback (--allowGPUFallback) not allowed for DLA standalone mode"); } } } @@ -1777,7 +1792,6 @@ std::ostream& operator<<(std::ostream& os, const BuildOptions& options) { // clang-format off os << "=== Build Options ===" << std::endl << - "Max batch: "; printBatch(os, options.maxBatch) << std::endl << "Memory Pools: "; printMemoryPools(os, options) << std::endl << "minTiming: " << options.minTiming << std::endl << @@ -1788,12 +1802,14 @@ std::ostream& operator<<(std::ostream& os, const BuildOptions& options) "Calibration: " << (options.int8 && options.calibration.empty() ? "Dynamic" : options.calibration.c_str()) << std::endl << "Refit: " << boolToEnabled(options.refittable) << std::endl << "Version Compatible: " << boolToEnabled(options.versionCompatible) << std::endl << - "TensorRT runtime: " << options.useRuntime << std::endl << - "Lean DLL Path: " << options.leanDLLPath << std::endl << + "TensorRT runtime: " << options.useRuntime << std::endl << + "Lean DLL Path: " << options.leanDLLPath << std::endl << "Tempfile Controls: "; printTempfileControls(os, options.tempfileControls) << std::endl << "Exclude Lean Runtime: " << boolToEnabled(options.excludeLeanRuntime) << std::endl << "Sparsity: "; printSparsity(os, options) << std::endl << "Safe mode: " << boolToEnabled(options.safe) << std::endl << + "Build DLA standalone loadable: " << boolToEnabled(options.buildDLAStandalone) << std::endl << + "Allow GPU fallback for DLA: " << boolToEnabled(options.allowGPUFallback) << std::endl << "DirectIO mode: " << boolToEnabled(options.directIO) << std::endl << "Restricted mode: " << boolToEnabled(options.restricted) << std::endl << "Skip inference: " << boolToEnabled(options.skipInference) << std::endl << @@ -1837,8 +1853,7 @@ std::ostream& operator<<(std::ostream& os, const SystemOptions& options) os << "=== System Options ===" << std::endl << "Device: " << options.device << std::endl << - "DLACore: " << (options.DLACore != -1 ? std::to_string(options.DLACore) : "") << - (options.DLACore != -1 && options.fallback ? "(With GPU fallback)" : "") << std::endl; + "DLACore: " << (options.DLACore != -1 ? std::to_string(options.DLACore) : "") << std::endl; os << "Plugins:"; for (const auto& p : options.plugins) @@ -2053,6 +2068,7 @@ void BuildOptions::help(std::ostream& os) " --workspace=N Set workspace size in MiB." "\n" " --memPoolSize=poolspec Specify the size constraints of the designated memory pool(s) in MiB." "\n" " Note: Also accepts decimal sizes, e.g. 0.25MiB. Will be rounded down to the nearest integer bytes." "\n" + " In particular, for dlaSRAM the bytes will be rounded down to the nearest power of 2." "\n" R"( Pool constraint: poolspec ::= poolfmt[","poolspec])" "\n" " poolfmt ::= pool:sizeInMiB" "\n" R"( pool ::= "workspace"|"dlaSRAM"|"dlaLocalDRAM"|"dlaGlobalDRAM")" "\n" @@ -2116,7 +2132,13 @@ void BuildOptions::help(std::ostream& os) R"( layerDeviceTypePair ::= layerName":"deviceType)" "\n" R"( deviceType ::= "GPU"|"DLA")" "\n" " --calib= Read INT8 calibration cache file" "\n" - " --safe Enable build safety certified engine" "\n" + " --safe Enable build safety certified engine, if DLA is enable, --buildDLAStandalone will be specified" "\n" + " automatically (default = disabled)" "\n" + " --buildDLAStandalone Enable build DLA standalone loadable which can be loaded by cuDLA, when this option is enabled, " "\n" + " --allowGPUFallback is disallowed and --skipInference is enabled by default. Additionally, " "\n" + " specifying --inputIOFormats and --outputIOFormats restricts I/O data type and memory layout" "\n" + " (default = disabled)" "\n" + " --allowGPUFallback When DLA is enabled, allow GPU fallback for unsupported layers (default = disabled)" "\n" " --consistency Perform consistency checking on safety certified engine" "\n" " --restricted Enable safety scope checking with kSAFETY_SCOPE build flag" "\n" " --saveEngine= Save the serialized engine" "\n" @@ -2139,10 +2161,9 @@ void BuildOptions::help(std::ostream& os) R"( flag ::= "fasterDynamicShapes0805")" "\n" R"( |"disableExternalTacticSourcesForCore0805")" "\n" R"( |"profileSharing0806")" "\n" - " --builderOptimizationLevel Set the builder optimization level. (default is 3" "\n" + " --builderOptimizationLevel Set the builder optimization level. (default is 3)" "\n" " Higher level allows TensorRT to spend more building time for more optimization options." "\n" - " The default level is 3. Valid values include integers from 0 to the maximum optimization level," "\n" - " which is currently 5." "\n" + " Valid values include integers from 0 to the maximum optimization level, which is currently 5." "\n" " --hardwareCompatibilityLevel=mode Make the engine file compatible with other GPU architectures. (default = none)" "\n" R"( Hardware Compatibility Level: mode ::= "none" | "ampere+")" "\n" " none = no compatibility" "\n" @@ -2171,8 +2192,6 @@ void SystemOptions::help(std::ostream& os) os << "=== System Options ===" << std::endl << " --device=N Select cuda device N (default = " << defaultDevice << ")" << std::endl << " --useDLACore=N Select DLA core N for layers that support DLA (default = none)" << std::endl << - " --allowGPUFallback When DLA is enabled, allow GPU fallback for unsupported layers " - "(default = disabled)" << std::endl << " --staticPlugins Plugin library (.so) to load statically (can be specified multiple times)" << std::endl << " --dynamicPlugins Plugin library (.so) to load dynamically and may be serialized with the engine if they are included in --setPluginsToSerialize (can be specified multiple times)" << std::endl << " --setPluginsToSerialize Plugin library (.so) to be serialized with the engine (can be specified multiple times)" << std::endl << @@ -2205,6 +2224,7 @@ void InferenceOptions::help(std::ostream& os) << defaultWarmUp << ")" << std::endl << " --duration=N Run performance measurements for at least N seconds wallclock time (default = " << defaultDuration << ")" << std::endl << + " If -1 is specified, inference will keep running unless stopped manually" << std::endl << " --sleepTime=N Delay inference start with a gap of N milliseconds between launch and compute " "(default = " << defaultSleep << ")" << std::endl << " --idleTime=N Sleep N milliseconds between two continuous iterations" diff --git a/samples/common/sampleOptions.h b/samples/common/sampleOptions.h index 25a5c790..36fb6839 100644 --- a/samples/common/sampleOptions.h +++ b/samples/common/sampleOptions.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); @@ -37,7 +37,7 @@ constexpr int32_t maxBatchNotProvided{0}; constexpr int32_t defaultMinTiming{1}; constexpr int32_t defaultAvgTiming{8}; constexpr int32_t defaultMaxAuxStreams{-1}; -constexpr int32_t defaultBuilderOptimizationLevel{3}; +constexpr int32_t defaultBuilderOptimizationLevel{-1}; // System default params constexpr int32_t defaultDevice{0}; @@ -206,6 +206,8 @@ class BuildOptions : public Options LayerOutputTypes layerOutputTypes; LayerDeviceTypes layerDeviceTypes; bool safe{false}; + bool buildDLAStandalone{false}; + bool allowGPUFallback{false}; bool consistency{false}; bool restricted{false}; bool skipInference{false}; @@ -249,7 +251,6 @@ class SystemOptions : public Options public: int32_t device{defaultDevice}; int32_t DLACore{-1}; - bool fallback{false}; bool ignoreParsedPluginLibs{false}; std::vector plugins; std::vector setPluginsToSerialize; diff --git a/samples/common/sampleReporting.cpp b/samples/common/sampleReporting.cpp index 0b3894c1..e7fc17f1 100644 --- a/samples/common/sampleReporting.cpp +++ b/samples/common/sampleReporting.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); @@ -181,7 +181,7 @@ PerformanceResult getPerformanceResult(std::vector const& timings PerformanceResult result; result.min = metricGetter(newTimings.front()); result.max = metricGetter(newTimings.back()); - result.mean = std::accumulate(newTimings.begin(), newTimings.end(), 0.0f, metricAccumulator) / newTimings.size(); + result.mean = std::accumulate(newTimings.begin(), newTimings.end(), 0.0F, metricAccumulator) / newTimings.size(); result.median = findMedian(newTimings, metricGetter); for (auto percentile : percentiles) { @@ -364,17 +364,14 @@ void Profiler::reportLayerTime(char const* layerName, float timeMs) noexcept void Profiler::print(std::ostream& os) const noexcept { - std::string const nameHdr("Layer"); - std::string const timeHdr(" Time (ms)"); - std::string const avgHdr(" Avg. Time (ms)"); - std::string const medHdr(" Median Time (ms)"); - std::string const percentageHdr(" Time %"); + std::string const nameHdr(" Layer"); + std::string const timeHdr(" Time(ms)"); + std::string const avgHdr(" Avg.(ms)"); + std::string const medHdr(" Median(ms)"); + std::string const percentageHdr(" Time(%)"); float const totalTimeMs = getTotalTime(); - auto const cmpLayer = [](LayerProfile const& a, LayerProfile const& b) { return a.name.size() < b.name.size(); }; - auto const longestName = std::max_element(mLayers.begin(), mLayers.end(), cmpLayer); - auto const nameLength = std::max(longestName->name.size() + 1, nameHdr.size()); auto const timeLength = timeHdr.size(); auto const avgLength = avgHdr.size(); auto const medLength = medHdr.size(); @@ -382,7 +379,7 @@ void Profiler::print(std::ostream& os) const noexcept os << std::endl << "=== Profile (" << mUpdatesCount << " iterations ) ===" << std::endl - << std::setw(nameLength) << nameHdr << timeHdr << avgHdr << medHdr << percentageHdr << std::endl; + << timeHdr << avgHdr << medHdr << percentageHdr << nameHdr << std::endl; for (auto const& p : mLayers) { @@ -392,17 +389,18 @@ void Profiler::print(std::ostream& os) const noexcept continue; } // clang-format off - os << std::setw(nameLength) << p.name << std::setw(timeLength) << std::fixed << std::setprecision(2) << getTotalTime(p) + os << std::setw(timeLength) << std::fixed << std::setprecision(2) << getTotalTime(p) << std::setw(avgLength) << std::fixed << std::setprecision(4) << getAvgTime(p) << std::setw(medLength) << std::fixed << std::setprecision(4) << getMedianTime(p) << std::setw(percentageLength) << std::fixed << std::setprecision(1) << getTotalTime(p) / totalTimeMs * 100 - << std::endl; + << " " << p.name << std::endl; } { - os << std::setw(nameLength) << "Total" << std::setw(timeLength) << std::fixed << std::setprecision(2) + os << std::setw(timeLength) << std::fixed << std::setprecision(2) << totalTimeMs << std::setw(avgLength) << std::fixed << std::setprecision(4) << totalTimeMs / mUpdatesCount << std::setw(medLength) << std::fixed << std::setprecision(4) << getMedianTime() - << std::setw(percentageLength) << std::fixed << std::setprecision(1) << 100.0 << std::endl; + << std::setw(percentageLength) << std::fixed << std::setprecision(1) << 100.0 + << " Total" << std::endl; // clang-format on } os << std::endl; diff --git a/samples/common/sampleReporting.h b/samples/common/sampleReporting.h index 186f8f06..fa0d706d 100644 --- a/samples/common/sampleReporting.h +++ b/samples/common/sampleReporting.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/common/sampleUtils.cpp b/samples/common/sampleUtils.cpp index 97646c24..93aeb69d 100644 --- a/samples/common/sampleUtils.cpp +++ b/samples/common/sampleUtils.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); @@ -383,6 +383,11 @@ void print(std::ostream& os, int8_t v) os << static_cast(v); } +void print(std::ostream& os, __half v) +{ + os << static_cast(v); +} + template void dumpBuffer(void const* buffer, std::string const& separator, std::ostream& os, Dims const& dims, Dims const& strides, int32_t vectorDim, int32_t spv) diff --git a/samples/common/sampleUtils.h b/samples/common/sampleUtils.h index 67efa86a..618c2782 100644 --- a/samples/common/sampleUtils.h +++ b/samples/common/sampleUtils.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/common.py b/samples/python/common.py index 36c78eee..76e2b17b 100644 --- a/samples/python/common.py +++ b/samples/python/common.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/detectron2/build_engine.py b/samples/python/detectron2/build_engine.py index 6b970c4d..42de4df0 100644 --- a/samples/python/detectron2/build_engine.py +++ b/samples/python/detectron2/build_engine.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/detectron2/create_onnx.py b/samples/python/detectron2/create_onnx.py index 5b25b2f6..38538464 100644 --- a/samples/python/detectron2/create_onnx.py +++ b/samples/python/detectron2/create_onnx.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/detectron2/eval_coco.py b/samples/python/detectron2/eval_coco.py index 6b890632..828413d4 100644 --- a/samples/python/detectron2/eval_coco.py +++ b/samples/python/detectron2/eval_coco.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -75,7 +75,7 @@ def main(args): pred_boxes = [] scores = [] pred_classes = [] - # Number of detections. + # Number of detections. num_instances = len(detections[i]) # Reserve numpy array to hold all mask predictions per image. pred_masks = np.empty((num_instances, 28, 28), dtype=np.float32) @@ -84,7 +84,7 @@ def main(args): # Loop over every single detection. for n in range(num_instances): det = detections[i][n] - # Append box coordinates data. + # Append box coordinates data. pred_boxes.append([det['ymin'], det['xmin'], det['ymax'], det['xmax']]) # Append score. scores.append(det['score']) @@ -105,21 +105,21 @@ def main(args): image_dict = [{'instances': instances}] input_dict = [{'image_id': source_id}] evaluator.process(input_dict, image_dict) - + # Final evaluations, generation of mAP accuracy performance. evaluator.evaluate() - + if __name__ == "__main__": parser = argparse.ArgumentParser() parser.add_argument("-e", "--engine", help="The TensorRT engine to infer with.") parser.add_argument("-i", "--input", - help="The input to infer, either a single image path, or a directory of images.") + help="The input to infer, either a single image path, or a directory of images.") parser.add_argument("-c", "--det2_config", help="The Detectron 2 config file (.yaml) for the model", type=str) - parser.add_argument("-w", "--det2_weights", help="The Detectron 2 model weights (.pkl)", type=str) - parser.add_argument("-t", "--nms_threshold", type=float, + parser.add_argument("-w", "--det2_weights", help="The Detectron 2 model weights (.pkl)", type=str) + parser.add_argument("-t", "--nms_threshold", type=float, help="Override the score threshold for the NMS operation, if higher than the threshold in the engine.") - parser.add_argument("--iou_threshold", default=0.5, type=float, + parser.add_argument("--iou_threshold", default=0.5, type=float, help="Select the IoU threshold for the mask segmentation. Range is 0 to 1. Pixel values more than threshold will become 1, less 0.") args = parser.parse_args() if not all([args.engine, args.input, args.det2_config, args.det2_weights]): diff --git a/samples/python/detectron2/image_batcher.py b/samples/python/detectron2/image_batcher.py index 7aa9af6d..35d8c8e1 100644 --- a/samples/python/detectron2/image_batcher.py +++ b/samples/python/detectron2/image_batcher.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -144,7 +144,7 @@ def resize_pad(image, pad_color=(0, 0, 0)): # Get characteristics. width, height = image.size - + # Replicates behavior of ResizeShortestEdge augmentation. size = self.min_size_test * 1.0 pre_scale = size / min(height, width) @@ -163,7 +163,7 @@ def resize_pad(image, pad_color=(0, 0, 0)): neww = int(neww + 0.5) newh = int(newh + 0.5) - # Scaling factor for normalized box coordinates scaling in post-processing. + # Scaling factor for normalized box coordinates scaling in post-processing. scaling = max(newh/height, neww/width) # Padding. @@ -177,7 +177,7 @@ def resize_pad(image, pad_color=(0, 0, 0)): image = Image.open(image_path) image = image.convert(mode='RGB') # Pad with mean values of COCO dataset, since padding is applied before actual model's - # preprocessor steps (Sub, Div ops), we need to pad with mean values in order to reverse + # preprocessor steps (Sub, Div ops), we need to pad with mean values in order to reverse # the effects of Sub and Div, so that padding after model's preprocessor will be with actual 0s. image, scale = resize_pad(image, (124, 116, 104)) image = np.asarray(image, dtype=np.float32) diff --git a/samples/python/detectron2/infer.py b/samples/python/detectron2/infer.py index a91d512d..aae41435 100644 --- a/samples/python/detectron2/infer.py +++ b/samples/python/detectron2/infer.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/detectron2/onnx_utils.py b/samples/python/detectron2/onnx_utils.py index ae4c3b02..56d280fa 100644 --- a/samples/python/detectron2/onnx_utils.py +++ b/samples/python/detectron2/onnx_utils.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -119,8 +119,8 @@ def gather(self, name, data, indices, axes=0): :param self: The gs.Graph object being extended. :param name: The name to use for the node. :param data: Data from which to gather specific tensors. - :param indices: Indices by which to gather data tensors. - :param axes: A list of axes on which to perform gather operation + :param indices: Indices by which to gather data tensors. + :param axes: A list of axes on which to perform gather operation """ data_tensor = data if type(data) is gs.Variable else data[0] indices_tensor = indices if type(indices) is gs.Variable else indices[0] @@ -217,7 +217,7 @@ def find_node_by_op_input_output_name(self, op, input_name, output_name, input_p @gs.Graph.register() def find_descendant_by_op(self, node, op, depth=10): """ - Starting from the given node, finds a node lower in the graph matching the given operation name. + Starting from the given node, finds a node lower in the graph matching the given operation name. This is not an exhaustive graph search. In order to graph search bfs is used, so runtime complexity is O(V+E). :param self: The gs.Graph object being extended. @@ -242,7 +242,7 @@ def find_ancestor_by_op(self, node, op, depth=10): """ Starting from the given node, finds a node higher in the graph matching the given operation name. This is not an exhaustive graph search. - In order to graph search bfs is used, so runtime complexity is O(V+E). + In order to graph search bfs is used, so runtime complexity is O(V+E). :param self: The gs.Graph object being extended. :param node: The node to start searching from. :param op: The operation name to search for. @@ -258,4 +258,4 @@ def find_ancestor_by_op(self, node, op, depth=10): return node for child in node.inputs[-1].inputs: queue.append(child) - return None \ No newline at end of file + return None diff --git a/samples/python/detectron2/requirements.txt b/samples/python/detectron2/requirements.txt index 7dca0535..d7361093 100644 --- a/samples/python/detectron2/requirements.txt +++ b/samples/python/detectron2/requirements.txt @@ -1,5 +1,7 @@ -onnx==1.8.1 -onnxruntime==1.8.0 +onnx==1.9.0; python_version<"3.8" +onnx==1.12.0; python_version>="3.8" +onnxruntime==1.8.1; python_version<"3.8" +onnxruntime==1.12.1; python_version>="3.8" Pillow git+https://github.com/facebookresearch/detectron2.git git+https://github.com/NVIDIA/TensorRT#subdirectory=tools/onnx-graphsurgeon diff --git a/samples/python/detectron2/visualize.py b/samples/python/detectron2/visualize.py index a7a0e780..546d7669 100644 --- a/samples/python/detectron2/visualize.py +++ b/samples/python/detectron2/visualize.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -69,7 +69,7 @@ def visualize_detections(image_path, output_path, detections, labels=[], iou_thr det_width = round(d['xmax'] - d['xmin']) det_height = round(d['ymax'] - d['ymin']) # Slight scaling, to get binary masks after float32 -> uint8 - # conversion, if not scaled all pixels are zero. + # conversion, if not scaled all pixels are zero. mask = d['mask'] > iou_threshold # Convert float32 -> uint8. mask = mask.astype(np.uint8) diff --git a/samples/python/downloader.py b/samples/python/downloader.py index 80e007b3..b4a436e2 100755 --- a/samples/python/downloader.py +++ b/samples/python/downloader.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/efficientdet/build_engine.py b/samples/python/efficientdet/build_engine.py index 3cf76c6e..638c604f 100644 --- a/samples/python/efficientdet/build_engine.py +++ b/samples/python/efficientdet/build_engine.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/efficientdet/compare_tf.py b/samples/python/efficientdet/compare_tf.py index 21413a5c..54c356cd 100644 --- a/samples/python/efficientdet/compare_tf.py +++ b/samples/python/efficientdet/compare_tf.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/efficientdet/create_onnx.py b/samples/python/efficientdet/create_onnx.py index 0c66620c..17fee5f6 100644 --- a/samples/python/efficientdet/create_onnx.py +++ b/samples/python/efficientdet/create_onnx.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/efficientdet/eval_coco.py b/samples/python/efficientdet/eval_coco.py index ad15918d..966f49be 100644 --- a/samples/python/efficientdet/eval_coco.py +++ b/samples/python/efficientdet/eval_coco.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/efficientdet/image_batcher.py b/samples/python/efficientdet/image_batcher.py index 82756447..e519a5db 100644 --- a/samples/python/efficientdet/image_batcher.py +++ b/samples/python/efficientdet/image_batcher.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/efficientdet/infer.py b/samples/python/efficientdet/infer.py index 8abe69ff..409738ac 100644 --- a/samples/python/efficientdet/infer.py +++ b/samples/python/efficientdet/infer.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/efficientdet/infer_tf.py b/samples/python/efficientdet/infer_tf.py index 7f9c91b0..a02f87ee 100644 --- a/samples/python/efficientdet/infer_tf.py +++ b/samples/python/efficientdet/infer_tf.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/efficientdet/onnx_utils.py b/samples/python/efficientdet/onnx_utils.py index 7155b171..e55f7e11 100644 --- a/samples/python/efficientdet/onnx_utils.py +++ b/samples/python/efficientdet/onnx_utils.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/efficientdet/visualize.py b/samples/python/efficientdet/visualize.py index c09b0bc6..742ecbce 100644 --- a/samples/python/efficientdet/visualize.py +++ b/samples/python/efficientdet/visualize.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/efficientnet/build_engine.py b/samples/python/efficientnet/build_engine.py index e71222e9..2391aca0 100644 --- a/samples/python/efficientnet/build_engine.py +++ b/samples/python/efficientnet/build_engine.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/efficientnet/compare_tf.py b/samples/python/efficientnet/compare_tf.py index c204ccfe..6d9ad88f 100644 --- a/samples/python/efficientnet/compare_tf.py +++ b/samples/python/efficientnet/compare_tf.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/efficientnet/create_onnx.py b/samples/python/efficientnet/create_onnx.py index 71dd6595..b98fd137 100644 --- a/samples/python/efficientnet/create_onnx.py +++ b/samples/python/efficientnet/create_onnx.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/efficientnet/eval_gt.py b/samples/python/efficientnet/eval_gt.py index f2b67627..88198b89 100644 --- a/samples/python/efficientnet/eval_gt.py +++ b/samples/python/efficientnet/eval_gt.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/efficientnet/image_batcher.py b/samples/python/efficientnet/image_batcher.py index a23ff136..996a72a3 100644 --- a/samples/python/efficientnet/image_batcher.py +++ b/samples/python/efficientnet/image_batcher.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/efficientnet/infer.py b/samples/python/efficientnet/infer.py index 8543a87c..46c11253 100644 --- a/samples/python/efficientnet/infer.py +++ b/samples/python/efficientnet/infer.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/engine_refit_onnx_bidaf/build_and_refit_engine.py b/samples/python/engine_refit_onnx_bidaf/build_and_refit_engine.py index f397a2c6..268a5cf5 100644 --- a/samples/python/engine_refit_onnx_bidaf/build_and_refit_engine.py +++ b/samples/python/engine_refit_onnx_bidaf/build_and_refit_engine.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/engine_refit_onnx_bidaf/data_processing.py b/samples/python/engine_refit_onnx_bidaf/data_processing.py index bdde98c2..6eb90fa0 100644 --- a/samples/python/engine_refit_onnx_bidaf/data_processing.py +++ b/samples/python/engine_refit_onnx_bidaf/data_processing.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/engine_refit_onnx_bidaf/prepare_model.py b/samples/python/engine_refit_onnx_bidaf/prepare_model.py index 11b1cf2e..cbeb6a92 100644 --- a/samples/python/engine_refit_onnx_bidaf/prepare_model.py +++ b/samples/python/engine_refit_onnx_bidaf/prepare_model.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/introductory_parser_samples/onnx_resnet50.py b/samples/python/introductory_parser_samples/onnx_resnet50.py index 8b980ec0..e7e845b6 100644 --- a/samples/python/introductory_parser_samples/onnx_resnet50.py +++ b/samples/python/introductory_parser_samples/onnx_resnet50.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/network_api_pytorch_mnist/model.py b/samples/python/network_api_pytorch_mnist/model.py index 9be0d11e..654532bf 100644 --- a/samples/python/network_api_pytorch_mnist/model.py +++ b/samples/python/network_api_pytorch_mnist/model.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/network_api_pytorch_mnist/sample.py b/samples/python/network_api_pytorch_mnist/sample.py index f8126a40..7c0a6417 100644 --- a/samples/python/network_api_pytorch_mnist/sample.py +++ b/samples/python/network_api_pytorch_mnist/sample.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/onnx_custom_plugin/CMakeLists.txt b/samples/python/onnx_custom_plugin/CMakeLists.txt index 09363eaf..a7ad8933 100644 --- a/samples/python/onnx_custom_plugin/CMakeLists.txt +++ b/samples/python/onnx_custom_plugin/CMakeLists.txt @@ -1,3 +1,20 @@ +# +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + # We need cmake >= 3.8, since 3.8 introduced CUDA as a first class language cmake_minimum_required(VERSION 3.8 FATAL_ERROR) project(CustomHardMax LANGUAGES CXX CUDA) diff --git a/samples/python/onnx_custom_plugin/load_plugin_lib.py b/samples/python/onnx_custom_plugin/load_plugin_lib.py index 4e2c3946..0a85f18e 100644 --- a/samples/python/onnx_custom_plugin/load_plugin_lib.py +++ b/samples/python/onnx_custom_plugin/load_plugin_lib.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/onnx_custom_plugin/model.py b/samples/python/onnx_custom_plugin/model.py index 3d29b74a..cde029c5 100644 --- a/samples/python/onnx_custom_plugin/model.py +++ b/samples/python/onnx_custom_plugin/model.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -52,7 +52,7 @@ def _do_graph_surgery(raw_model_path, trt_model_path): compress_node = node_by_name['Compress_31'] einsum_node = gs.Node( - 'Einsum', + 'Einsum', 'Dot_of_Hardmax_and_Transpose', attrs={'equation': 'ij,ij->i'}, # "Dot product" of 2d tensors inputs=[hardmax_node.outputs[0], transpose_node.outputs[0]], diff --git a/samples/python/onnx_custom_plugin/plugin/customHardmaxPlugin.cpp b/samples/python/onnx_custom_plugin/plugin/customHardmaxPlugin.cpp index 125d9577..4900292b 100644 --- a/samples/python/onnx_custom_plugin/plugin/customHardmaxPlugin.cpp +++ b/samples/python/onnx_custom_plugin/plugin/customHardmaxPlugin.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/onnx_custom_plugin/plugin/customHardmaxPlugin.h b/samples/python/onnx_custom_plugin/plugin/customHardmaxPlugin.h index c98e70f9..250291d5 100644 --- a/samples/python/onnx_custom_plugin/plugin/customHardmaxPlugin.h +++ b/samples/python/onnx_custom_plugin/plugin/customHardmaxPlugin.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/onnx_custom_plugin/sample.py b/samples/python/onnx_custom_plugin/sample.py index c2a3579a..65b1991f 100644 --- a/samples/python/onnx_custom_plugin/sample.py +++ b/samples/python/onnx_custom_plugin/sample.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/onnx_custom_plugin/test_custom_hardmax_plugin.py b/samples/python/onnx_custom_plugin/test_custom_hardmax_plugin.py index 5fac3d60..dc919b72 100644 --- a/samples/python/onnx_custom_plugin/test_custom_hardmax_plugin.py +++ b/samples/python/onnx_custom_plugin/test_custom_hardmax_plugin.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/onnx_packnet/convert_to_onnx.py b/samples/python/onnx_packnet/convert_to_onnx.py index be07ae64..df604f96 100644 --- a/samples/python/onnx_packnet/convert_to_onnx.py +++ b/samples/python/onnx_packnet/convert_to_onnx.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/onnx_packnet/post_processing.py b/samples/python/onnx_packnet/post_processing.py index 5dad482f..fd101b45 100644 --- a/samples/python/onnx_packnet/post_processing.py +++ b/samples/python/onnx_packnet/post_processing.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/scripts/download_mnist_data.sh b/samples/python/scripts/download_mnist_data.sh index 9572a805..809bcbc9 100755 --- a/samples/python/scripts/download_mnist_data.sh +++ b/samples/python/scripts/download_mnist_data.sh @@ -1,6 +1,6 @@ #!/bin/bash # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/scripts/download_mnist_pgms.py b/samples/python/scripts/download_mnist_pgms.py index 19bb6ca2..a1ee0cba 100644 --- a/samples/python/scripts/download_mnist_pgms.py +++ b/samples/python/scripts/download_mnist_pgms.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -66,4 +66,3 @@ def main(): if __name__ == '__main__': main() - diff --git a/samples/python/tensorflow_object_detection_api/build_engine.py b/samples/python/tensorflow_object_detection_api/build_engine.py index a7945a52..2efd6599 100644 --- a/samples/python/tensorflow_object_detection_api/build_engine.py +++ b/samples/python/tensorflow_object_detection_api/build_engine.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/tensorflow_object_detection_api/compare_tf.py b/samples/python/tensorflow_object_detection_api/compare_tf.py index 8d805f3a..409aec6b 100644 --- a/samples/python/tensorflow_object_detection_api/compare_tf.py +++ b/samples/python/tensorflow_object_detection_api/compare_tf.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -109,13 +109,13 @@ def infer(self, batch, scales=None, nms_threshold=None): # Depending on detection type you need slightly different data. if self.detection_type == 'bbox': mask = None - # Segmentation is only supported with Mask R-CNN, which has + # Segmentation is only supported with Mask R-CNN, which has # fixed_shape_resizer as image_resizer (lookup pipeline.config) elif self.detection_type == 'segmentation': # Select a mask mask = masks[0][n] # Slight scaling, to get binary masks after float32 -> uint8 - # conversion, if not scaled all pixels are zero. + # conversion, if not scaled all pixels are zero. mask = mask > self.iou_threshold # Convert float32 -> uint8. mask = mask.astype(np.uint8) @@ -131,7 +131,7 @@ def infer(self, batch, scales=None, nms_threshold=None): scale_x = scale if nms_threshold and scores[0][n] < nms_threshold: continue - # Append to detections + # Append to detections detections[0].append({ 'ymin': boxes[0][n][0] * scale_y, 'xmin': boxes[0][n][1] * scale_x, @@ -169,7 +169,7 @@ def parse_annotations(annotations_path, detection_type): # Depending on detection type you need slightly different data. if detection_type == 'bbox': mask = None - # Segmentation is only supported with Mask R-CNN, which has + # Segmentation is only supported with Mask R-CNN, which has # fixed_shape_resizer as image_resizer (lookup pipeline.config) elif detection_type == 'segmentation': # Get np.array segmentation mask from annotation @@ -270,4 +270,4 @@ def main(args): if not all([args.engine, args.saved_model, args.input, args.output, args.preprocessor]): parser.print_help() sys.exit(1) - main(args) \ No newline at end of file + main(args) diff --git a/samples/python/tensorflow_object_detection_api/create_onnx.py b/samples/python/tensorflow_object_detection_api/create_onnx.py index 3ecb1b93..919cc8e6 100644 --- a/samples/python/tensorflow_object_detection_api/create_onnx.py +++ b/samples/python/tensorflow_object_detection_api/create_onnx.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -69,12 +69,12 @@ def __init__(self, saved_model_path, pipeline_config_path): # Fold constants via ONNX-GS that TF2ONNX may have missed. self.graph.fold_constants() - + # Pipeline config parsing. pipeline_config = config_util.get_configs_from_pipeline_file(pipeline_config_path) # Get input resolution. self.height, self.width = config_util.get_spatial_image_size(config_util.get_image_resizer_config(pipeline_config["model"])) - + # If your model is SSD, get characteristics accordingly from pipeline.config file. if pipeline_config["model"].HasField("ssd"): # Getting model characteristics. @@ -83,11 +83,11 @@ def __init__(self, saved_model_path, pipeline_config_path): self.first_stage_nms_iou_threshold = float(pipeline_config["model"].ssd.post_processing.batch_non_max_suppression.iou_threshold) self.first_stage_max_proposals = int(pipeline_config["model"].ssd.post_processing.batch_non_max_suppression.max_detections_per_class) # If your model is Faster R-CNN get it's characteristics from pipeline.config file. - elif pipeline_config["model"].HasField("faster_rcnn"): + elif pipeline_config["model"].HasField("faster_rcnn"): # Getting model characteristics. - self.model = str(pipeline_config["model"].faster_rcnn.feature_extractor.type) + self.model = str(pipeline_config["model"].faster_rcnn.feature_extractor.type) self.num_classes = pipeline_config["model"].faster_rcnn.num_classes - self.first_stage_nms_score_threshold = float(pipeline_config["model"].faster_rcnn.first_stage_nms_score_threshold) + self.first_stage_nms_score_threshold = float(pipeline_config["model"].faster_rcnn.first_stage_nms_score_threshold) self.first_stage_nms_iou_threshold = float(pipeline_config["model"].faster_rcnn.first_stage_nms_iou_threshold) self.first_stage_max_proposals = int(pipeline_config["model"].faster_rcnn.first_stage_max_proposals) self.first_stage_crop_size = int(pipeline_config["model"].faster_rcnn.initial_crop_size) @@ -101,15 +101,15 @@ def __init__(self, saved_model_path, pipeline_config_path): self.matmul_crop_and_resize = pipeline_config["model"].faster_rcnn.use_matmul_crop_and_resize # If model is Mask R-CNN, get final instance segmentation masks resolution. if pipeline_config["model"].faster_rcnn.second_stage_box_predictor.mask_rcnn_box_predictor.HasField("mask_height") and pipeline_config["model"].faster_rcnn.second_stage_box_predictor.mask_rcnn_box_predictor.HasField("mask_width"): - self.mask_height = int(pipeline_config["model"].faster_rcnn.second_stage_box_predictor.mask_rcnn_box_predictor.mask_height) + self.mask_height = int(pipeline_config["model"].faster_rcnn.second_stage_box_predictor.mask_rcnn_box_predictor.mask_height) self.mask_width = int(pipeline_config["model"].faster_rcnn.second_stage_box_predictor.mask_rcnn_box_predictor.mask_width) - else: + else: log.info("Given Model type is not supported") sys.exit(1) # List of supported models. - supported_models = ["ssd_mobilenet_v2_keras", "ssd_mobilenet_v1_fpn_keras", "ssd_mobilenet_v2_fpn_keras", "ssd_resnet50_v1_fpn_keras", - "ssd_resnet101_v1_fpn_keras", "ssd_resnet152_v1_fpn_keras", "faster_rcnn_resnet50_keras", "faster_rcnn_resnet101_keras", + supported_models = ["ssd_mobilenet_v2_keras", "ssd_mobilenet_v1_fpn_keras", "ssd_mobilenet_v2_fpn_keras", "ssd_resnet50_v1_fpn_keras", + "ssd_resnet101_v1_fpn_keras", "ssd_resnet152_v1_fpn_keras", "faster_rcnn_resnet50_keras", "faster_rcnn_resnet101_keras", "faster_rcnn_resnet152_keras", "faster_rcnn_inception_resnet_v2_keras"] assert self.model in supported_models @@ -129,7 +129,7 @@ def __init__(self, saved_model_path, pipeline_config_path): if not (self.mask_height is None and self.mask_width is None): log.info("Mask height is {}".format(self.mask_height)) log.info("Mask width is {}".format(self.mask_width)) - + self.batch_size = None def sanitize(self): @@ -182,7 +182,7 @@ def save(self, output_path): def add_debug_output(self, debug): """ - Add a debug output to a given node. + Add a debug output to a given node. :param debug: Name of the output you would like to debug. """ tensors = self.graph.tensors() @@ -239,7 +239,7 @@ def update_preprocessor(self, batch_size, input_format): elif 'resnet' in self.model: sub_const = np.expand_dims(np.asarray([255 * 0.485, 255 * 0.456, 255 * 0.406], dtype=np.float32), axis=(0, 2, 3)) sub_out = self.graph.op_with_const("Sub", "preprocessor/mean", input_tensor, sub_const) - + # Backbone is not supported. else: log.info("Given model's backbone is not supported, pre-processor algorithm can't be generated") @@ -266,7 +266,7 @@ def update_preprocessor(self, batch_size, input_format): self.sanitize() def find_head_end(self, head_name, descendant, end_op): - # This helper function finds ends of Class Net and Box Net, based on a model type. + # This helper function finds ends of Class Net and Box Net, based on a model type. # :param head_name: This is a common name that nodes in either Class or Box Nets start with. # :param descendant: Descendant of head_name, identified by operation (Transpose, MatMul, etc.). # :param end_op: Operation of a node you would like to get in the end of each Net. @@ -297,8 +297,8 @@ def get_anchor(output_idx, op, depth=5): node = self.graph.find_descendant_by_op(split.o(0, output_idx), op) for i in range(depth): if node.op == op: - # Input of size 1 is not anchor data - if (node.inputs[1].values).size == 1: + # Input of size 1 is not anchor data + if (node.inputs[1].values).size == 1: node = node.o() # Find the node that with anchor data, multielement input elif (node.inputs[1].values).size > 1: @@ -308,7 +308,7 @@ def get_anchor(output_idx, op, depth=5): else: node = node.o() return None - + anchors_y = get_anchor(0, "Add") anchors_x = get_anchor(1, "Add") anchors_h = get_anchor(2, "Mul") @@ -320,19 +320,19 @@ def get_anchor(output_idx, op, depth=5): # Trim total number of anchors in order to not have copies introduced by growing number of batch_size. anchors = batched_anchors[0:num_anchors,0:num_anchors] return gs.Constant(name="nms/anchors:0", values=anchors) - + def NMS(self, box_net_tensor, class_net_tensor, anchors_tensor, background_class, score_activation, iou_threshold, nms_score_threshold, user_threshold, nms_name=None): - # Helper function to create the NMS Plugin node with the selected inputs. + # Helper function to create the NMS Plugin node with the selected inputs. # EfficientNMS_TRT TensorRT Plugin is suitable for our use case. - # :param box_net_tensor: The box predictions from the Box Net. + # :param box_net_tensor: The box predictions from the Box Net. # :param class_net_tensor: The class predictions from the Class Net. # :param anchors_tensor: The default anchor coordinates (from the extracted anchor constants) # :param background_class: The label ID for the background class. - # :param score_activation: If set to True - apply sigmoid activation to the confidence scores during NMS operation, + # :param score_activation: If set to True - apply sigmoid activation to the confidence scores during NMS operation, # if false - no activation, pass one from the graph. # :param iou_threshold: NMS intersection over union threshold, given by pipeline.config. # :param nms_score_threshold: NMS score threshold, given by pipeline.config. - # :param user_threshold: User's given threshold to overwrite default NMS score threshold. + # :param user_threshold: User's given threshold to overwrite default NMS score threshold. # :param nms_name: Name of NMS node in a graph, renames NMS elements accordingly in order to eliminate cycles. if nms_name is None: @@ -369,24 +369,24 @@ def NMS(self, box_net_tensor, class_net_tensor, anchors_tensor, background_class 'score_activation': score_activation, 'class_agnostic': False, 'box_coding': 1, - } + } ) log.info("Created 'nms/non_maximum_suppression{}' NMS plugin".format(nms_name)) return nms_outputs def CropAndResize(self, unsqeeze_input, relu_node_outputs, cnr_num): - # Helper function to create the NMS Plugin node with the selected inputs. + # Helper function to create the NMS Plugin node with the selected inputs. # CropAndResize TensorRT Plugin is suitable for our use case. - # :param unsqeeze_input: NMS's bonding boxes output, clipped and normalized if this is first CropAndResize, this is a souce of rois for CropAndResize. + # :param unsqeeze_input: NMS's bonding boxes output, clipped and normalized if this is first CropAndResize, this is a souce of rois for CropAndResize. # :param relu_node_outputs: 1st backbone's last Relu node, this is a souce of feature_maps for CropAndResize - # :param cnr_num: Positional number of CropAndResize node in a graph, renames CropAndResize elements accordingly in order to eliminate cycles. - + # :param cnr_num: Positional number of CropAndResize node in a graph, renames CropAndResize elements accordingly in order to eliminate cycles. + # CropAndResizePlugin requires 4th dimension of 1: [N, B, 4, 1], so - # we need to add unsqeeze node to make tensor 4 dimensional. + # we need to add unsqeeze node to make tensor 4 dimensional. unsqueeze_node = self.graph.unsqueeze("CNR/detection_boxes_unsqueeze_"+cnr_num, unsqeeze_input) - # CropAndResizePlugin's inputs + # CropAndResizePlugin's inputs feature_maps = relu_node_outputs rois = unsqueeze_node[0] @@ -394,7 +394,7 @@ def CropAndResize(self, unsqeeze_input, relu_node_outputs, cnr_num): cnr_pfmap = gs.Variable(name="cnr/pfmap_"+cnr_num, dtype=np.float32, shape=[self.batch_size, self.first_stage_max_proposals, feature_maps.shape[1], self.first_stage_crop_size, self.first_stage_crop_size]) - # Create the CropandResize Plugin node with the selected inputs. + # Create the CropandResize Plugin node with the selected inputs. # Two inputs are given to the CropAndResize TensorRT node: # - The feature_maps (from the relu_node_outputs): [batch_size, channel_num, height, width] # - The rois (clipped and normalized detection boxes resulting from NMS): [batch_size, featuremap, 4, 1] @@ -415,7 +415,7 @@ def CropAndResize(self, unsqeeze_input, relu_node_outputs, cnr_num): reshape_node = self.graph.op_with_const("Reshape", "cnr/reshape_"+cnr_num, cnr_pfmap, reshape_shape) return reshape_node[0] - + def process_graph(self, first_nms_threshold=None, second_nms_threshold=None): """ Processes the graph to replace the NMS operations by EfficientNMS_TRT TensorRT plugin nodes and @@ -434,7 +434,7 @@ def first_nms(background_class, score_activation, first_nms_threshold, nms_name= # Supported models ssd_models = ['ssd_mobilenet_v1_fpn_keras', 'ssd_mobilenet_v2_fpn_keras', 'ssd_resnet50_v1_fpn_keras', 'ssd_resnet101_v1_fpn_keras', 'ssd_resnet152_v1_fpn_keras'] frcnn_models = ['faster_rcnn_resnet50_keras', 'faster_rcnn_resnet101_keras', 'faster_rcnn_resnet152_keras', 'faster_rcnn_inception_resnet_v2_keras'] - + # Getting SSD's Class and Box Nets final tensors. if "ssd" in self.model: # Find the concat node at the end of the class net (multi-scale class predictor). @@ -456,7 +456,7 @@ def first_nms(background_class, score_activation, first_nms_threshold, nms_name= variance_adj = np.expand_dims(np.asarray([0.1, 0.1, 0.2, 0.2], dtype=np.float32), axis=(0, 1)) # Final Box Net tensor. box_net_tensor = self.graph.op_with_const("Mul", box_net_head_name+"/scale", box_net_output, variance_adj)[0] - + # Getting Faster R-CNN's 1st Class and Box Nets tensors. elif "faster_rcnn" in self.model: # Identify Class Net and Box Net head names @@ -465,7 +465,7 @@ def first_nms(background_class, score_activation, first_nms_threshold, nms_name= # Find the softmax node at the end of the class net (multi-scale class predictor). class_net = self.find_head_end(head_names[0], "Transpose", "Softmax") # Final Class Net tensor - class_net_tensor = class_net.outputs[0] + class_net_tensor = class_net.outputs[0] # Find the reshape node at the end of the box net (multi-scale localization predictor). box_net = self.find_head_end(head_names[1], "Transpose", "Reshape") @@ -495,7 +495,7 @@ def first_nms(background_class, score_activation, first_nms_threshold, nms_name= def first_cnr(input): """ Updates the graph to replace the 1st cropAndResize op by CropAndResize TensorRT plugin node. - :param input: Input tensor is the output from previous first_nms() step. + :param input: Input tensor is the output from previous first_nms() step. """ # Locate the last Relu node of the first backbone (pre 1st NMS). Relu node contains feature maps @@ -511,7 +511,7 @@ def first_cnr(input): div_out = self.graph.op_with_const("Div", "FirstNMS/detection_boxes_normalizer", clip_out[0], div_const) # Linear transformation to convert box coordinates from (TopLeft, BottomRight) Corner encoding - # to CenterSize encoding. 1st NMS boxes are multiplied by transformation matrix in order to + # to CenterSize encoding. 1st NMS boxes are multiplied by transformation matrix in order to # encode it into CenterSize format. matmul_const = np.matrix('0.5 0 -1 0; 0 0.5 0 -1; 0.5 0 1 0; 0 0.5 0 1', dtype=np.float32) matmul_out = self.graph.matmul("FirstNMS/detection_boxes_conversion", div_out[0], matmul_const) @@ -523,7 +523,7 @@ def first_cnr(input): maxpool_node = [node for node in self.graph.nodes if node.op == "MaxPool" and "MaxPool2D/MaxPool" in node.name][0] maxpool_node.inputs[0] = cnr_output - # Return linear transformation node, it will be located between 1st and 2nd NMS, + # Return linear transformation node, it will be located between 1st and 2nd NMS, # so we need to pass and connect it to 2nd NMS. # In case you are converting Mask R-CNN, feature maps are required for 2nd CropAndResize. return matmul_out[0], relu_node.outputs[0] @@ -531,9 +531,9 @@ def first_cnr(input): def second_nms(background_class, score_activation, encoded_boxes, second_nms_threshold, nms_name=None): """ Updates the graph to replace the 2nd (or final) NMS op by EfficientNMS_TRT TensorRT plugin node. - :param background_class: Set EfficientNMS_TRT's background_class atribute. - :param score_activation: Set EfficientNMS_TRT's score_activation atribute. - :param encoded_boxes: The boxes to use as input. + :param background_class: Set EfficientNMS_TRT's background_class atribute. + :param score_activation: Set EfficientNMS_TRT's score_activation atribute. + :param encoded_boxes: The boxes to use as input. :param second_nms_threshold: Override the NMS score threshold. :param nms_name: Set the NMS node name. """ @@ -550,7 +550,7 @@ def second_nms(background_class, score_activation, encoded_boxes, second_nms_thr # Final Class Net tensor. second_class_net_tensor = slice_out[0] - + # Find the add node at the end of the box net (multi-scale localization predictor). second_box_net = self.find_head_end(second_head_names[1], "MatMul", "Add") # Final Box Net tensor. @@ -560,7 +560,7 @@ def second_nms(background_class, score_activation, encoded_boxes, second_nms_thr # Based on type of Crop and Resize operation, second_box_net_output can be of two types, example: # If use_matmul_crop_and_resize in pipeline.config is set to True, expect: [batch_size, first_stage_max_proposals, 4]. # Else use_matmul_crop_and_resize is either False or absent, expect: [batch_size, first_stage_max_proposals, num_classes, 4] - if self.matmul_crop_and_resize: + if self.matmul_crop_and_resize: reshape_shape_second = np.asarray([self.batch_size, self.first_stage_max_proposals, second_box_net.outputs[0].shape[1]], dtype=np.int64) else: reshape_shape_second = np.asarray([self.batch_size, self.first_stage_max_proposals, self.num_classes, second_box_net.outputs[0].shape[1]/self.num_classes], dtype=np.int64) @@ -574,13 +574,13 @@ def second_nms(background_class, score_activation, encoded_boxes, second_nms_thr # Create NMS node. nms_outputs = self.NMS(second_box_net_tensor, second_class_net_tensor, encoded_boxes, background_class, score_activation, self.second_stage_iou_threshold, self.second_stage_nms_score_threshold, second_nms_threshold, nms_name) - + return nms_outputs def second_cnr(feature_maps, second_nms_outputs): """ Updates the graph to replace the 2nd cropAndResize op by CropAndResize TensorRT plugin node. - :param input: Input tensor is the output from previous first_nms() step. + :param input: Input tensor is the output from previous first_nms() step. """ # Before passing 2nd NMS's detection boxes (rois) to second CropAndResize, we need to clip them. @@ -590,7 +590,7 @@ def second_cnr(feature_maps, second_nms_outputs): # Create Crop and Resize node. cnr_output = self.CropAndResize(clip_out, feature_maps, "second") - # Find MaxPool node that summarizes CropAndResize structure + # Find MaxPool node that summarizes CropAndResize structure maxpool_node = [node for node in self.graph.nodes if node.op == "MaxPool" and "MaxPool2D/MaxPool_1" in node.name][0] maxpool_node.inputs[0] = cnr_output @@ -616,7 +616,7 @@ def second_cnr(feature_maps, second_nms_outputs): final_reshape_node[0].name = "detection_masks" return final_reshape_node[0] - + # If you model is SSD, you need only one NMS and nothin else. if "ssd" in self.model: # Set graph outputs. @@ -662,7 +662,7 @@ def main(args): parser.add_argument("-t1", "--first_nms_threshold", help="Override the score threshold for the 1st NMS operation", type=float) parser.add_argument("-t2", "--second_nms_threshold", help="Override the score threshold for the 2nd NMS operation", type=float) parser.add_argument("-d", "--debug", action='append', help="Add an extra output to debug a particular node") - parser.add_argument("-f", "--input_format", default="NHWC", choices=["NHWC", "NCHW"], + parser.add_argument("-f", "--input_format", default="NHWC", choices=["NHWC", "NCHW"], help="Set the input shape of the graph, as comma-separated dimensions in NCHW or NHWC format, default: NHWC") parser.add_argument("--tf2onnx", help="The path where to save the intermediate ONNX graph generated by tf2onnx, " "useful for debugging purposes, default: not saved", type=str) @@ -672,4 +672,3 @@ def main(args): print("\nThese arguments are required: --pipeline_config, --saved_model and --onnx") sys.exit(1) main(args) - diff --git a/samples/python/tensorflow_object_detection_api/eval_coco.py b/samples/python/tensorflow_object_detection_api/eval_coco.py index 5641ceb2..5086c660 100644 --- a/samples/python/tensorflow_object_detection_api/eval_coco.py +++ b/samples/python/tensorflow_object_detection_api/eval_coco.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -37,7 +37,7 @@ def main(args): # Read annotations json as dictionary. with open(args.annotations) as f: data = json.load(f) - groundtruth = coco_tools.COCOWrapper(data, detection_type=args.detection_type) + groundtruth = coco_tools.COCOWrapper(data, detection_type=args.detection_type) detections_list = [] for batch, images, scales in batcher.get_batch(): print("Processing Image {} / {}".format(batcher.image_index, batcher.num_images), end="\r") @@ -59,7 +59,7 @@ def main(args): detections_list.append(coco_det) elif args.detection_type == 'segmentation': # Get detection bbox resolution. - det_width = round(det['xmax'] - det['xmin']) + det_width = round(det['xmax'] - det['xmin']) det_height = round(det['ymax'] - det['ymin']) # Create an image out of predicted mask array. small_mask = Image.fromarray(det['mask']) @@ -94,13 +94,13 @@ def main(args): parser = argparse.ArgumentParser() parser.add_argument("-e", "--engine", help="The TensorRT engine to infer with.") parser.add_argument("-i", "--input", - help="The input to infer, either a single image path, or a directory of images.") + help="The input to infer, either a single image path, or a directory of images.") parser.add_argument("-d", "--detection_type", default="bbox", choices=["bbox", "segmentation"], help="Detection type for COCO, either bbox or if you are using Mask R-CNN's instance segmentation - segmentation.") parser.add_argument("-a", "--annotations", help="Set the json file to use for COCO instance annotations.") - parser.add_argument("-t", "--nms_threshold", type=float, + parser.add_argument("-t", "--nms_threshold", type=float, help="Override the score threshold for the NMS operation, if higher than the threshold in the engine.") - parser.add_argument("--iou_threshold", default=0.5, type=float, + parser.add_argument("--iou_threshold", default=0.5, type=float, help="Select the IoU threshold for the mask segmentation. Range is 0 to 1. Pixel values more than threshold will become 1, less 0.") parser.add_argument("--preprocessor", default="fixed_shape_resizer", choices=["fixed_shape_resizer", "keep_aspect_ratio_resizer"], help="Select the image preprocessor to use based on your pipeline.config, either 'fixed_shape_resizer' or 'keep_aspect_ratio_resizer', default: fixed_shape_resizer.") diff --git a/samples/python/tensorflow_object_detection_api/image_batcher.py b/samples/python/tensorflow_object_detection_api/image_batcher.py index 47a3c2e0..c40e86c8 100644 --- a/samples/python/tensorflow_object_detection_api/image_batcher.py +++ b/samples/python/tensorflow_object_detection_api/image_batcher.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/tensorflow_object_detection_api/infer.py b/samples/python/tensorflow_object_detection_api/infer.py index ef0d9f8a..cdb1f2a9 100644 --- a/samples/python/tensorflow_object_detection_api/infer.py +++ b/samples/python/tensorflow_object_detection_api/infer.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/tensorflow_object_detection_api/onnx_utils.py b/samples/python/tensorflow_object_detection_api/onnx_utils.py index a70ecb48..b539197a 100644 --- a/samples/python/tensorflow_object_detection_api/onnx_utils.py +++ b/samples/python/tensorflow_object_detection_api/onnx_utils.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -172,7 +172,7 @@ def find_node_by_op(self, op): @gs.Graph.register() def find_descendant_by_op(self, node, op, depth=10): """ - Starting from the given node, finds a node lower in the graph matching the given operation name. + Starting from the given node, finds a node lower in the graph matching the given operation name. This is not an exhaustive graph search. In order to graph search bfs is used, so runtime complexity is O(V+E). :param self: The gs.Graph object being extended. @@ -198,7 +198,7 @@ def find_ancestor_by_op(self, node, op, depth=10): """ Starting from the given node, finds a node higher in the graph matching the given operation name. This is not an exhaustive graph search. - In order to graph search bfs is used, so runtime complexity is O(V+E). + In order to graph search bfs is used, so runtime complexity is O(V+E). :param self: The gs.Graph object being extended. :param node: The node to start searching from. :param op: The operation name to search for. diff --git a/samples/python/tensorflow_object_detection_api/visualize.py b/samples/python/tensorflow_object_detection_api/visualize.py index 283f06e1..ca992d0f 100644 --- a/samples/python/tensorflow_object_detection_api/visualize.py +++ b/samples/python/tensorflow_object_detection_api/visualize.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -142,4 +142,4 @@ def draw_text(draw, font, text, width, bar_height, offset, color): if output_path is None: return concat - concat.save(output_path) \ No newline at end of file + concat.save(output_path) diff --git a/samples/python/yolov3_onnx/data_processing.py b/samples/python/yolov3_onnx/data_processing.py index 06a3bc26..8a68145f 100644 --- a/samples/python/yolov3_onnx/data_processing.py +++ b/samples/python/yolov3_onnx/data_processing.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/yolov3_onnx/onnx_to_tensorrt.py b/samples/python/yolov3_onnx/onnx_to_tensorrt.py index 7ce53098..7572f76a 100644 --- a/samples/python/yolov3_onnx/onnx_to_tensorrt.py +++ b/samples/python/yolov3_onnx/onnx_to_tensorrt.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/python/yolov3_onnx/yolov3_to_onnx.py b/samples/python/yolov3_onnx/yolov3_to_onnx.py index 4259d55f..59f8b3a6 100644 --- a/samples/python/yolov3_onnx/yolov3_to_onnx.py +++ b/samples/python/yolov3_onnx/yolov3_to_onnx.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/sampleAlgorithmSelector/CMakeLists.txt b/samples/sampleAlgorithmSelector/CMakeLists.txt index 4df908f2..8d31262a 100644 --- a/samples/sampleAlgorithmSelector/CMakeLists.txt +++ b/samples/sampleAlgorithmSelector/CMakeLists.txt @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/sampleAlgorithmSelector/sampleAlgorithmSelector.cpp b/samples/sampleAlgorithmSelector/sampleAlgorithmSelector.cpp index 14d8b12f..927d5ed3 100644 --- a/samples/sampleAlgorithmSelector/sampleAlgorithmSelector.cpp +++ b/samples/sampleAlgorithmSelector/sampleAlgorithmSelector.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/sampleCharRNN/CMakeLists.txt b/samples/sampleCharRNN/CMakeLists.txt index 656b9519..9c9b9f6c 100644 --- a/samples/sampleCharRNN/CMakeLists.txt +++ b/samples/sampleCharRNN/CMakeLists.txt @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/sampleCharRNN/sampleCharRNN.cpp b/samples/sampleCharRNN/sampleCharRNN.cpp index 677c812e..919f0929 100644 --- a/samples/sampleCharRNN/sampleCharRNN.cpp +++ b/samples/sampleCharRNN/sampleCharRNN.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); @@ -207,7 +207,7 @@ class SampleCharRNNBase //! //! \brief Looks up the embedding tensor for a given char and copies it to input buffer //! - void copyEmbeddingToInput(samplesCommon::BufferManager& buffers, const char& c); + void copyEmbeddingToInput(samplesCommon::BufferManager& buffers, char const& c); //! //! \brief Perform one time step of inference with the TensorRT execution context @@ -917,14 +917,14 @@ bool SampleCharRNNBase::infer() //! //! \brief Looks up the embedding tensor for a given char and copies it to input buffer //! -void SampleCharRNNBase::copyEmbeddingToInput(samplesCommon::BufferManager& buffers, const char& c) +void SampleCharRNNBase::copyEmbeddingToInput(samplesCommon::BufferManager& buffers, char const& c) { auto embed = mWeightMap[mParams.weightNames.EMBED_NAME]; float* inputBuffer = static_cast(buffers.getHostBuffer(mParams.bindingNames.INPUT_BLOB_NAME)); auto index = mParams.charMaps.charToID.at(c); + auto bufSize = buffers.size(mParams.bindingNames.INPUT_BLOB_NAME); - std::memcpy(inputBuffer, static_cast(embed.values) + index * mParams.dataSize, - buffers.size(mParams.bindingNames.INPUT_BLOB_NAME)); + std::memcpy(inputBuffer, static_cast(embed.values) + index * mParams.dataSize, bufSize); } //! diff --git a/samples/sampleDynamicReshape/CMakeLists.txt b/samples/sampleDynamicReshape/CMakeLists.txt index 0fa80a84..374b5566 100644 --- a/samples/sampleDynamicReshape/CMakeLists.txt +++ b/samples/sampleDynamicReshape/CMakeLists.txt @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/sampleDynamicReshape/sampleDynamicReshape.cpp b/samples/sampleDynamicReshape/sampleDynamicReshape.cpp index 8ba21f8d..003bfb55 100644 --- a/samples/sampleDynamicReshape/sampleDynamicReshape.cpp +++ b/samples/sampleDynamicReshape/sampleDynamicReshape.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/sampleINT8API/CMakeLists.txt b/samples/sampleINT8API/CMakeLists.txt index 138b3484..e8eed5c3 100644 --- a/samples/sampleINT8API/CMakeLists.txt +++ b/samples/sampleINT8API/CMakeLists.txt @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/sampleINT8API/sampleINT8API.cpp b/samples/sampleINT8API/sampleINT8API.cpp index e4eeaefb..9c421b6a 100644 --- a/samples/sampleINT8API/sampleINT8API.cpp +++ b/samples/sampleINT8API/sampleINT8API.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/sampleIOFormats/CMakeLists.txt b/samples/sampleIOFormats/CMakeLists.txt index 98c8f0e5..154b599e 100755 --- a/samples/sampleIOFormats/CMakeLists.txt +++ b/samples/sampleIOFormats/CMakeLists.txt @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/sampleIOFormats/sampleIOFormats.cpp b/samples/sampleIOFormats/sampleIOFormats.cpp index d7d364da..a8f94717 100644 --- a/samples/sampleIOFormats/sampleIOFormats.cpp +++ b/samples/sampleIOFormats/sampleIOFormats.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/sampleNamedDimensions/CMakeLists.txt b/samples/sampleNamedDimensions/CMakeLists.txt index f5d22c05..f03d19b1 100644 --- a/samples/sampleNamedDimensions/CMakeLists.txt +++ b/samples/sampleNamedDimensions/CMakeLists.txt @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/sampleNamedDimensions/create_model.py b/samples/sampleNamedDimensions/create_model.py index 610d9b9b..e4146aa5 100644 --- a/samples/sampleNamedDimensions/create_model.py +++ b/samples/sampleNamedDimensions/create_model.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -29,7 +29,7 @@ def main(): graph = gs.Graph(nodes=[node], inputs=[input0, input1], outputs=[output]) model = gs.export_onnx(graph) - onnx.save(model, "concat_layer.onnx") + onnx.save(model, "concat_layer.onnx") if __name__ == '__main__': main() diff --git a/samples/sampleNamedDimensions/sampleNamedDimensions.cpp b/samples/sampleNamedDimensions/sampleNamedDimensions.cpp index 1d09094e..c38d2bfb 100644 --- a/samples/sampleNamedDimensions/sampleNamedDimensions.cpp +++ b/samples/sampleNamedDimensions/sampleNamedDimensions.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/sampleOnnxMNIST/CMakeLists.txt b/samples/sampleOnnxMNIST/CMakeLists.txt index b25a83a8..23bd886b 100644 --- a/samples/sampleOnnxMNIST/CMakeLists.txt +++ b/samples/sampleOnnxMNIST/CMakeLists.txt @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/sampleOnnxMNIST/README.md b/samples/sampleOnnxMNIST/README.md index ec4fb594..05abeafd 100644 --- a/samples/sampleOnnxMNIST/README.md +++ b/samples/sampleOnnxMNIST/README.md @@ -56,7 +56,7 @@ To build the engine, create the builder and pass a logger created for TensorRT w `IBuilder* builder = createInferBuilder(sample::gLogger);` To build the engine from the generated TensorRT network, issue the following call: -`nvinfer1::ICudaEngine* engine = builder->buildCudaEngine(*network);` +`SampleUniquePtr plan{builder->buildSerializedNetwork(*network, *config)};` After you build the engine, verify that the engine is running properly by confirming the output is what you expected. The output format of this sample should be the same as the output of sampleMNIST. @@ -91,6 +91,9 @@ The Scale layer implements a per-tensor, per-channel, or per-element affine tran [Shuffle layer](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#shuffle-layer) The Shuffle layer implements a reshape and transpose operator for tensors. +## Preparing sample data + +Download the sample data from the [TensorRT release tarball](https://developer.nvidia.com/nvidia-tensorrt-download#). ## Running the sample diff --git a/samples/sampleOnnxMNIST/sampleOnnxMNIST.cpp b/samples/sampleOnnxMNIST/sampleOnnxMNIST.cpp index 2aea25c2..9fb5b678 100644 --- a/samples/sampleOnnxMNIST/sampleOnnxMNIST.cpp +++ b/samples/sampleOnnxMNIST/sampleOnnxMNIST.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/sampleOnnxMnistCoordConvAC/CMakeLists.txt b/samples/sampleOnnxMnistCoordConvAC/CMakeLists.txt index e89d5282..b094cf08 100644 --- a/samples/sampleOnnxMnistCoordConvAC/CMakeLists.txt +++ b/samples/sampleOnnxMnistCoordConvAC/CMakeLists.txt @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/sampleOnnxMnistCoordConvAC/coord_conv.py b/samples/sampleOnnxMnistCoordConvAC/coord_conv.py index b5749b25..b2572ad5 100644 --- a/samples/sampleOnnxMnistCoordConvAC/coord_conv.py +++ b/samples/sampleOnnxMnistCoordConvAC/coord_conv.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/sampleOnnxMnistCoordConvAC/mnist_coord_conv_train.py b/samples/sampleOnnxMnistCoordConvAC/mnist_coord_conv_train.py index 29ce9c17..8d0a9623 100644 --- a/samples/sampleOnnxMnistCoordConvAC/mnist_coord_conv_train.py +++ b/samples/sampleOnnxMnistCoordConvAC/mnist_coord_conv_train.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -28,14 +28,14 @@ class Net(nn.Module): - """ - Original implementation of Convnet from + """ + Original implementation of Convnet from PyTorch repo https://github.com/pytorch/examples/tree/master/mnist but with CoordConv2d layers instead of Conv layers """ def __init__(self): super(Net, self).__init__() - # Regular Conv layer replaced with CoordConv2d layer + # Regular Conv layer replaced with CoordConv2d layer self.conv1 = CoordConv2d(1, 32, 3, 1) # Regular Conv layer replaced with CoordConv2d layer self.conv2 = CoordConv2d(32, 64, 3, 1) @@ -138,8 +138,8 @@ def main(): transforms.Normalize((0.1307,), (0.3081,)) ])), batch_size=args.test_batch_size, shuffle=True, **kwargs) - - + + model = Net().to(device) print(model) optimizer = optim.Adadelta(model.parameters(), lr=args.lr) diff --git a/samples/sampleOnnxMnistCoordConvAC/modify_onnx_ac.py b/samples/sampleOnnxMnistCoordConvAC/modify_onnx_ac.py index ba0b1e9e..5462d6e0 100644 --- a/samples/sampleOnnxMnistCoordConvAC/modify_onnx_ac.py +++ b/samples/sampleOnnxMnistCoordConvAC/modify_onnx_ac.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -50,7 +50,7 @@ def main(): parser.add_argument('--output', default="mnist_with_coordconv.onnx", help='input batch size for testing (default: output.onnx)') args = parser.parse_args() - + # Load ONNX file graph = gs.import_onnx(onnx.load(args.onnx)) diff --git a/samples/sampleOnnxMnistCoordConvAC/sampleOnnxMnistCoordConvAC.cpp b/samples/sampleOnnxMnistCoordConvAC/sampleOnnxMnistCoordConvAC.cpp index 0beeb2e7..e161e4f8 100644 --- a/samples/sampleOnnxMnistCoordConvAC/sampleOnnxMnistCoordConvAC.cpp +++ b/samples/sampleOnnxMnistCoordConvAC/sampleOnnxMnistCoordConvAC.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/trtexec/CMakeLists.txt b/samples/trtexec/CMakeLists.txt index 15eadf26..35ad26de 100644 --- a/samples/trtexec/CMakeLists.txt +++ b/samples/trtexec/CMakeLists.txt @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/trtexec/prn_utils.py b/samples/trtexec/prn_utils.py index d71e0cf7..6d759238 100755 --- a/samples/trtexec/prn_utils.py +++ b/samples/trtexec/prn_utils.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/trtexec/profiler.py b/samples/trtexec/profiler.py index 66e9ebc5..e251254b 100755 --- a/samples/trtexec/profiler.py +++ b/samples/trtexec/profiler.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/trtexec/tracer.py b/samples/trtexec/tracer.py index 815862bc..8a9b7a62 100755 --- a/samples/trtexec/tracer.py +++ b/samples/trtexec/tracer.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/samples/trtexec/trtexec.cpp b/samples/trtexec/trtexec.cpp index 0c54aff6..3113ff81 100644 --- a/samples/trtexec/trtexec.cpp +++ b/samples/trtexec/trtexec.cpp @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); @@ -116,6 +116,12 @@ bool initLibrary(LibraryPtr& libPtr, std::string const& libName, FetchPtrs fetch libPtr.reset(new DynamicLibrary{libName}); fetchFunc(libPtr.get()); } + catch (std::exception const& e) + { + libPtr.reset(); + sample::gLogError << "Could not load library " << libName << ": " << e.what() << std::endl; + return false; + } catch (...) { libPtr.reset(); diff --git a/scripts/copyright-scan.py b/scripts/copyright-scan.py index 8a52ff76..cc51c8e1 100644 --- a/scripts/copyright-scan.py +++ b/scripts/copyright-scan.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/scripts/stubify.sh b/scripts/stubify.sh index ba6c300c..a6839b78 100755 --- a/scripts/stubify.sh +++ b/scripts/stubify.sh @@ -1,12 +1,13 @@ #!/bin/bash # -# Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 + +# http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -14,7 +15,6 @@ # See the License for the specific language governing permissions and # limitations under the License. # - # This short shell script will extract all the strong "text" symbols from the # shared library and create a new "stub" shared library with the same symbols. # The body of these functions will be empty and therefore have no dependencies. @@ -40,15 +40,22 @@ fi SONAME=$(readelf -d "${IN_LIBFILE}" | grep '(SONAME)' | cut -d [ -f 2 | cut -d ] -f 1) +OS=$(lsb_release -si)-$(lsb_release -sr | cut -d '.' -f 1-2) + +if [ "$OS" = "Ubuntu-22.04" ] ; then + EXTRA_NM_FLAG="--without-symbol-versions" +fi + # make stub library if [ -z "${CC_ARGS}" ] ; then nm -D "${IN_LIBFILE}" ${EXTRA_NM_FLAG} | \ awk '{if ($2 == "T") { print "void",$3,"() {}" }}' | \ - "${CC}" -c -x c -Og -fPIC -shared -Wl,-soname=${SONAME} -Wl,--strip-all -o "${OUT_LIBFILE}" - + "${CC}" -x c -Og -fPIC -shared -Wl,-soname=${SONAME} -Wl,--strip-all -o "${OUT_LIBFILE}" - else nm -D "${IN_LIBFILE}" ${EXTRA_NM_FLAG} | \ awk '{if ($2 == "T") { print "void",$3,"() {}" }}' | \ - "${CC}" -c -x c -Og -fPIC -shared -Wl,-soname=${SONAME} -Wl,--strip-all -o "${OUT_LIBFILE}" "${CC_ARGS}" - + "${CC}" -x c -Og -fPIC -shared -Wl,-soname=${SONAME} -Wl,--strip-all -o "${OUT_LIBFILE}" "${CC_ARGS}" - fi exit $? + diff --git a/third_party/ieee/half.h b/third_party/ieee/half.h index 017834e9..c1f20f16 100644 --- a/third_party/ieee/half.h +++ b/third_party/ieee/half.h @@ -1,5 +1,5 @@ /* - * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: Apache-2.0 * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/Makefile b/tools/onnx-graphsurgeon/Makefile index 38d847fb..5b0cf62d 100644 --- a/tools/onnx-graphsurgeon/Makefile +++ b/tools/onnx-graphsurgeon/Makefile @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/docs/conf.py b/tools/onnx-graphsurgeon/docs/conf.py index 575e2e3c..34433481 100644 --- a/tools/onnx-graphsurgeon/docs/conf.py +++ b/tools/onnx-graphsurgeon/docs/conf.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/examples/01_creating_a_model/example.py b/tools/onnx-graphsurgeon/examples/01_creating_a_model/example.py index 5fe4cb58..f4e0d80e 100644 --- a/tools/onnx-graphsurgeon/examples/01_creating_a_model/example.py +++ b/tools/onnx-graphsurgeon/examples/01_creating_a_model/example.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/examples/02_creating_a_model_with_initializer/example.py b/tools/onnx-graphsurgeon/examples/02_creating_a_model_with_initializer/example.py index 0ab2dfed..c0268236 100644 --- a/tools/onnx-graphsurgeon/examples/02_creating_a_model_with_initializer/example.py +++ b/tools/onnx-graphsurgeon/examples/02_creating_a_model_with_initializer/example.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/examples/03_isolating_a_subgraph/generate.py b/tools/onnx-graphsurgeon/examples/03_isolating_a_subgraph/generate.py index 82ae2ab3..e71a783b 100644 --- a/tools/onnx-graphsurgeon/examples/03_isolating_a_subgraph/generate.py +++ b/tools/onnx-graphsurgeon/examples/03_isolating_a_subgraph/generate.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/examples/03_isolating_a_subgraph/isolate.py b/tools/onnx-graphsurgeon/examples/03_isolating_a_subgraph/isolate.py index 15a9be95..2d4bfdf5 100644 --- a/tools/onnx-graphsurgeon/examples/03_isolating_a_subgraph/isolate.py +++ b/tools/onnx-graphsurgeon/examples/03_isolating_a_subgraph/isolate.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/examples/04_modifying_a_model/generate.py b/tools/onnx-graphsurgeon/examples/04_modifying_a_model/generate.py index 82ae2ab3..e71a783b 100644 --- a/tools/onnx-graphsurgeon/examples/04_modifying_a_model/generate.py +++ b/tools/onnx-graphsurgeon/examples/04_modifying_a_model/generate.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/examples/04_modifying_a_model/modify.py b/tools/onnx-graphsurgeon/examples/04_modifying_a_model/modify.py index 5dba5271..55d42234 100644 --- a/tools/onnx-graphsurgeon/examples/04_modifying_a_model/modify.py +++ b/tools/onnx-graphsurgeon/examples/04_modifying_a_model/modify.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/examples/05_folding_constants/README.md b/tools/onnx-graphsurgeon/examples/05_folding_constants/README.md index 905b0e51..7b860f68 100644 --- a/tools/onnx-graphsurgeon/examples/05_folding_constants/README.md +++ b/tools/onnx-graphsurgeon/examples/05_folding_constants/README.md @@ -2,7 +2,7 @@ ## Introduction -This example first generates a a model with several operations that can be evaluated +This example first generates a model with several operations that can be evaluated prior to inference time, then folds these operations and exports a new model. Constant folding involves pre-computing expressions that do not depend on runtime diff --git a/tools/onnx-graphsurgeon/examples/05_folding_constants/fold.py b/tools/onnx-graphsurgeon/examples/05_folding_constants/fold.py index e2378876..4514f808 100644 --- a/tools/onnx-graphsurgeon/examples/05_folding_constants/fold.py +++ b/tools/onnx-graphsurgeon/examples/05_folding_constants/fold.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/examples/05_folding_constants/generate.py b/tools/onnx-graphsurgeon/examples/05_folding_constants/generate.py index 12c61080..98c6bc43 100644 --- a/tools/onnx-graphsurgeon/examples/05_folding_constants/generate.py +++ b/tools/onnx-graphsurgeon/examples/05_folding_constants/generate.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/examples/06_removing_nodes/generate.py b/tools/onnx-graphsurgeon/examples/06_removing_nodes/generate.py index c1bd733a..e786f894 100644 --- a/tools/onnx-graphsurgeon/examples/06_removing_nodes/generate.py +++ b/tools/onnx-graphsurgeon/examples/06_removing_nodes/generate.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/examples/06_removing_nodes/remove.py b/tools/onnx-graphsurgeon/examples/06_removing_nodes/remove.py index c4823fe0..01a6d683 100644 --- a/tools/onnx-graphsurgeon/examples/06_removing_nodes/remove.py +++ b/tools/onnx-graphsurgeon/examples/06_removing_nodes/remove.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/examples/07_creating_a_model_with_the_layer_api/generate.py b/tools/onnx-graphsurgeon/examples/07_creating_a_model_with_the_layer_api/generate.py index 627d6682..516a9e33 100644 --- a/tools/onnx-graphsurgeon/examples/07_creating_a_model_with_the_layer_api/generate.py +++ b/tools/onnx-graphsurgeon/examples/07_creating_a_model_with_the_layer_api/generate.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/examples/08_replacing_a_subgraph/generate.py b/tools/onnx-graphsurgeon/examples/08_replacing_a_subgraph/generate.py index 1f02b0e0..680ad607 100644 --- a/tools/onnx-graphsurgeon/examples/08_replacing_a_subgraph/generate.py +++ b/tools/onnx-graphsurgeon/examples/08_replacing_a_subgraph/generate.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/examples/08_replacing_a_subgraph/replace.py b/tools/onnx-graphsurgeon/examples/08_replacing_a_subgraph/replace.py index 93006cb7..88652309 100644 --- a/tools/onnx-graphsurgeon/examples/08_replacing_a_subgraph/replace.py +++ b/tools/onnx-graphsurgeon/examples/08_replacing_a_subgraph/replace.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/examples/09_shape_operations_with_the_layer_api/generate.py b/tools/onnx-graphsurgeon/examples/09_shape_operations_with_the_layer_api/generate.py index 33cbba4c..29403b08 100644 --- a/tools/onnx-graphsurgeon/examples/09_shape_operations_with_the_layer_api/generate.py +++ b/tools/onnx-graphsurgeon/examples/09_shape_operations_with_the_layer_api/generate.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/examples/10_dynamic_batch_size/generate.py b/tools/onnx-graphsurgeon/examples/10_dynamic_batch_size/generate.py index 679af4a2..19435b9c 100644 --- a/tools/onnx-graphsurgeon/examples/10_dynamic_batch_size/generate.py +++ b/tools/onnx-graphsurgeon/examples/10_dynamic_batch_size/generate.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/examples/10_dynamic_batch_size/modify.py b/tools/onnx-graphsurgeon/examples/10_dynamic_batch_size/modify.py index b6bb64b5..959a36b2 100644 --- a/tools/onnx-graphsurgeon/examples/10_dynamic_batch_size/modify.py +++ b/tools/onnx-graphsurgeon/examples/10_dynamic_batch_size/modify.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/onnx_graphsurgeon/exporters/base_exporter.py b/tools/onnx-graphsurgeon/onnx_graphsurgeon/exporters/base_exporter.py index 0673a1d3..b596e5d1 100644 --- a/tools/onnx-graphsurgeon/onnx_graphsurgeon/exporters/base_exporter.py +++ b/tools/onnx-graphsurgeon/onnx_graphsurgeon/exporters/base_exporter.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/onnx_graphsurgeon/exporters/onnx_exporter.py b/tools/onnx-graphsurgeon/onnx_graphsurgeon/exporters/onnx_exporter.py index 057cc5de..4d4b2429 100644 --- a/tools/onnx-graphsurgeon/onnx_graphsurgeon/exporters/onnx_exporter.py +++ b/tools/onnx-graphsurgeon/onnx_graphsurgeon/exporters/onnx_exporter.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/onnx_graphsurgeon/importers/base_importer.py b/tools/onnx-graphsurgeon/onnx_graphsurgeon/importers/base_importer.py index 6d853951..153d076e 100644 --- a/tools/onnx-graphsurgeon/onnx_graphsurgeon/importers/base_importer.py +++ b/tools/onnx-graphsurgeon/onnx_graphsurgeon/importers/base_importer.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/onnx_graphsurgeon/importers/onnx_importer.py b/tools/onnx-graphsurgeon/onnx_graphsurgeon/importers/onnx_importer.py index 23d429c2..de5cf73f 100644 --- a/tools/onnx-graphsurgeon/onnx_graphsurgeon/importers/onnx_importer.py +++ b/tools/onnx-graphsurgeon/onnx_graphsurgeon/importers/onnx_importer.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/onnx_graphsurgeon/ir/graph.py b/tools/onnx-graphsurgeon/onnx_graphsurgeon/ir/graph.py index f785a553..ce71d82b 100644 --- a/tools/onnx-graphsurgeon/onnx_graphsurgeon/ir/graph.py +++ b/tools/onnx-graphsurgeon/onnx_graphsurgeon/ir/graph.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/onnx_graphsurgeon/ir/node.py b/tools/onnx-graphsurgeon/onnx_graphsurgeon/ir/node.py index f4d652f9..bf10f3f6 100644 --- a/tools/onnx-graphsurgeon/onnx_graphsurgeon/ir/node.py +++ b/tools/onnx-graphsurgeon/onnx_graphsurgeon/ir/node.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/onnx_graphsurgeon/ir/tensor.py b/tools/onnx-graphsurgeon/onnx_graphsurgeon/ir/tensor.py index 7286979e..eec11405 100644 --- a/tools/onnx-graphsurgeon/onnx_graphsurgeon/ir/tensor.py +++ b/tools/onnx-graphsurgeon/onnx_graphsurgeon/ir/tensor.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/onnx_graphsurgeon/logger/logger.py b/tools/onnx-graphsurgeon/onnx_graphsurgeon/logger/logger.py index 248d9188..7aac34d0 100644 --- a/tools/onnx-graphsurgeon/onnx_graphsurgeon/logger/logger.py +++ b/tools/onnx-graphsurgeon/onnx_graphsurgeon/logger/logger.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/onnx_graphsurgeon/util/exception.py b/tools/onnx-graphsurgeon/onnx_graphsurgeon/util/exception.py index ce27f3cb..79843c9c 100644 --- a/tools/onnx-graphsurgeon/onnx_graphsurgeon/util/exception.py +++ b/tools/onnx-graphsurgeon/onnx_graphsurgeon/util/exception.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/onnx_graphsurgeon/util/misc.py b/tools/onnx-graphsurgeon/onnx_graphsurgeon/util/misc.py index d66675cc..cc46c459 100644 --- a/tools/onnx-graphsurgeon/onnx_graphsurgeon/util/misc.py +++ b/tools/onnx-graphsurgeon/onnx_graphsurgeon/util/misc.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/setup.py b/tools/onnx-graphsurgeon/setup.py index 984fb55e..6982e8e0 100644 --- a/tools/onnx-graphsurgeon/setup.py +++ b/tools/onnx-graphsurgeon/setup.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/tests/ir/test_graph.py b/tools/onnx-graphsurgeon/tests/ir/test_graph.py index 7fa51ef6..5a082d98 100644 --- a/tools/onnx-graphsurgeon/tests/ir/test_graph.py +++ b/tools/onnx-graphsurgeon/tests/ir/test_graph.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/tests/onnx_models.py b/tools/onnx-graphsurgeon/tests/onnx_models.py index 5a5ec875..fe3b6a93 100644 --- a/tools/onnx-graphsurgeon/tests/onnx_models.py +++ b/tools/onnx-graphsurgeon/tests/onnx_models.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/tests/test_api.py b/tools/onnx-graphsurgeon/tests/test_api.py index adceb603..2e3ac861 100644 --- a/tools/onnx-graphsurgeon/tests/test_api.py +++ b/tools/onnx-graphsurgeon/tests/test_api.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/tests/test_examples.py b/tools/onnx-graphsurgeon/tests/test_examples.py index a89266a8..a7f29ef8 100644 --- a/tools/onnx-graphsurgeon/tests/test_examples.py +++ b/tools/onnx-graphsurgeon/tests/test_examples.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/tests/test_exporters.py b/tools/onnx-graphsurgeon/tests/test_exporters.py index 8f5a607d..7d069fe9 100644 --- a/tools/onnx-graphsurgeon/tests/test_exporters.py +++ b/tools/onnx-graphsurgeon/tests/test_exporters.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/tests/test_importers.py b/tools/onnx-graphsurgeon/tests/test_importers.py index 604d65b6..30570cb6 100644 --- a/tools/onnx-graphsurgeon/tests/test_importers.py +++ b/tools/onnx-graphsurgeon/tests/test_importers.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/tests/test_ir.py b/tools/onnx-graphsurgeon/tests/test_ir.py index d2b90a4f..6f7be80e 100644 --- a/tools/onnx-graphsurgeon/tests/test_ir.py +++ b/tools/onnx-graphsurgeon/tests/test_ir.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/tools/onnx-graphsurgeon/tests/test_util.py b/tools/onnx-graphsurgeon/tests/test_util.py index d1c7fa53..72023153 100644 --- a/tools/onnx-graphsurgeon/tests/test_util.py +++ b/tools/onnx-graphsurgeon/tests/test_util.py @@ -1,5 +1,5 @@ # -# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 # # Licensed under the Apache License, Version 2.0 (the "License");