diff --git a/README.md b/README.md index 6ca7185523d5..6bd29cf71533 100644 --- a/README.md +++ b/README.md @@ -207,7 +207,6 @@ up to 10x. Here is a list of the algorithms we support, and the platforms they c | [HRNet](/serverless/pytorch/saic-vul/hrnet/nuclio) | interactor | PyTorch | | ✔️ | | [Inside-Outside Guidance](/serverless/pytorch/shiyinzhang/iog/nuclio) | interactor | PyTorch | ✔️ | | | [Faster RCNN](/serverless/tensorflow/faster_rcnn_inception_v2_coco/nuclio) | detector | TensorFlow | ✔️ | ✔️ | -| [Mask RCNN](/serverless/tensorflow/matterport/mask_rcnn/nuclio) | detector | TensorFlow | ✔️ | ✔️ | | [RetinaNet](serverless/pytorch/facebookresearch/detectron2/retinanet_r101/nuclio) | detector | PyTorch | ✔️ | ✔️ | | [Face Detection](/serverless/openvino/omz/intel/face-detection-0205/nuclio) | detector | OpenVINO | ✔️ | | diff --git a/site/content/en/docs/getting_started/overview.md b/site/content/en/docs/getting_started/overview.md index 45ce1f08740c..258eb4890fbb 100644 --- a/site/content/en/docs/getting_started/overview.md +++ b/site/content/en/docs/getting_started/overview.md @@ -120,7 +120,6 @@ Below is a detailed table of the supported algorithms and the platforms they ope | [HRNet](https://github.com/cvat-ai/cvat/tree/develop/serverless/pytorch/saic-vul/hrnet/nuclio) | Interactor | PyTorch | | ✔️ | | [Inside-Outside Guidance](https://github.com/cvat-ai/cvat/tree/develop/serverless/pytorch/shiyinzhang/iog/nuclio) | Interactor | PyTorch | ✔️ | | | [Faster RCNN](https://github.com/cvat-ai/cvat/tree/develop/serverless/tensorflow/faster_rcnn_inception_v2_coco/nuclio) | Detector | TensorFlow | ✔️ | ✔️ | -| [Mask RCNN](https://github.com/cvat-ai/cvat/tree/develop/serverless/tensorflow/matterport/mask_rcnn/nuclio) | Detector | TensorFlow | ✔️ | ✔️ | | [RetinaNet](https://github.com/cvat-ai/cvat/tree/develop/serverless/pytorch/facebookresearch/detectron2/retinanet_r101/nuclio) | Detector | PyTorch | ✔️ | ✔️ | | [Face Detection](https://github.com/cvat-ai/cvat/tree/develop/serverless/openvino/omz/intel/face-detection-0205/nuclio) | Detector | OpenVINO | ✔️ | | diff --git a/site/content/en/docs/manual/advanced/ai-tools.md b/site/content/en/docs/manual/advanced/ai-tools.md index 14b628cc882d..d8c235242df4 100644 --- a/site/content/en/docs/manual/advanced/ai-tools.md +++ b/site/content/en/docs/manual/advanced/ai-tools.md @@ -205,7 +205,6 @@ see {{< ilink "/docs/manual/advanced/automatic-annotation" "Automatic annotation | Faster RCNN | The model generates bounding boxes for each instance of an object in the image.
In this model, RPN and Fast R-CNN are combined into a single network.

For more information, see:
  • [GitHub: Faster RCNN](https://github.com/ShaoqingRen/faster_rcnn)
  • [Paper: Faster RCNN](https://arxiv.org/pdf/1506.01497.pdf) | | YOLO v3 | YOLO v3 is a family of object detection architectures and models pre-trained on the COCO dataset.

    For more information, see:
  • [GitHub: YOLO v3](https://github.com/ultralytics/yolov3)
  • [Site: YOLO v3](https://docs.ultralytics.com/#yolov3)
  • [Paper: YOLO v3](https://arxiv.org/pdf/1804.02767v1.pdf) | | Semantic segmentation for ADAS | This is a segmentation network to classify each pixel into 20 classes.

    For more information, see:
  • [Site: ADAS](https://docs.openvino.ai/2019_R1/_semantic_segmentation_adas_0001_description_semantic_segmentation_adas_0001.html) | -| Mask RCNN with Tensorflow | Mask RCNN version with Tensorflow. The model generates polygons for each instance of an object in the image.

    For more information, see:
  • [GitHub: Mask RCNN](https://github.com/matterport/Mask_RCNN)
  • [Paper: Mask RCNN](https://arxiv.org/pdf/1703.06870.pdf) | | Faster RCNN with Tensorflow | Faster RCNN version with Tensorflow. The model generates bounding boxes for each instance of an object in the image.
    In this model, RPN and Fast R-CNN are combined into a single network.

    For more information, see:
  • [Site: Faster RCNN with Tensorflow](https://docs.openvino.ai/2021.4/omz_models_model_faster_rcnn_inception_v2_coco.html)
  • [Paper: Faster RCNN](https://arxiv.org/pdf/1506.01497.pdf) | | RetinaNet | Pytorch implementation of RetinaNet object detection.


    For more information, see:
  • [Specification: RetinaNet](https://paperswithcode.com/lib/detectron2/retinanet)
  • [Paper: RetinaNet](https://arxiv.org/pdf/1708.02002.pdf)
  • [Documentation: RetinaNet](https://detectron2.readthedocs.io/en/latest/tutorials/training.html) | | Face Detection | Face detector based on MobileNetV2 as a backbone for indoor and outdoor scenes shot by a front-facing camera.


    For more information, see:
  • [Site: Face Detection 0205](https://docs.openvino.ai/latest/omz_models_model_face_detection_0205.html) | diff --git a/site/content/en/docs/manual/advanced/serverless-tutorial.md b/site/content/en/docs/manual/advanced/serverless-tutorial.md index 5211886208e8..1b3c77c3e843 100644 --- a/site/content/en/docs/manual/advanced/serverless-tutorial.md +++ b/site/content/en/docs/manual/advanced/serverless-tutorial.md @@ -24,8 +24,7 @@ that can _perfectly_ annotate 50% of your data equates to reducing manual annota Since we know DL models can help us to annotate faster, how then do we use them? In CVAT all such DL models are implemented as serverless functions using the [Nuclio][nuclio-homepage] serverless platform. There are multiple implemented functions that can be -found in the [serverless][cvat-builtin-serverless] directory such as _Mask RCNN, -Faster RCNN, SiamMask, Inside Outside Guidance, Deep Extreme Cut_, etc. +found in the [serverless][cvat-builtin-serverless] directory such as _Mask RCNN, Faster RCNN, SiamMask, Inside Outside Guidance, Deep Extreme Cut_, etc. Follow [the installation guide][cvat-auto-annotation-guide] to build and deploy these serverless functions. See [the user guide][cvat-ai-tools-user-guide] to understand how to use these functions in the UI to automatically annotate data. @@ -161,7 +160,8 @@ Finally you will get bounding boxes. ![SiamMask results](/images/siammask_results.gif) `SiamMask` model is more optimized to work on Nvidia GPUs. -For more information about deploying the model for the GPU, [read on](#objects-segmentation-using-mask-rcnn). + +- For more information about deploying the model for the GPU, [read on](#objects-segmentation-using-mask-rcnn). ### Object detection using YOLO-v3 @@ -215,61 +215,6 @@ CVAT will run the serverless function on every frame of the task and submit results directly into database. For more details please read [the guide][cvat-auto-annotation-user-guide]. -### Objects segmentation using Mask-RCNN - -If you have a detector, which returns polygons, you can segment objects. One -of such detectors is `Mask-RCNN`. There are several implementations of the -detector available out of the box: - -- `serverless/openvino/omz/public/mask_rcnn_inception_resnet_v2_atrous_coco` is - optimized using [Intel OpenVINO framework][intel-openvino-url] and works well - if it is run on an Intel CPU. -- `serverless/tensorflow/matterport/mask_rcnn/` is optimized for GPU. - -The deployment process for a serverless function optimized for GPU is similar. -Just need to run `serverless/deploy_gpu.sh` script. It runs mostly the same -commands but utilize `function-gpu.yaml` configuration file instead of -`function.yaml` internally. See next sections if you want to understand the -difference. - -_Note: Please do not run several GPU functions at the same time. In many cases it -will not work out of the box. For now you should manually schedule different -functions on different GPUs and it requires source code modification. Nuclio -autoscaler does not support the local platform (docker)._ - -
    - - -```bash -serverless/deploy_gpu.sh serverless/tensorflow/matterport/mask_rcnn -``` - - - -``` -Deploying serverless/tensorflow/matterport/mask_rcnn function... -21.07.12 16:48:48.995 nuctl (I) Deploying function {"name": ""} -21.07.12 16:48:48.995 nuctl (I) Building {"versionInfo": "Label: 1.5.16, Git commit: ae43a6a560c2bec42d7ccfdf6e8e11a1e3cc3774, OS: linux, Arch: amd64, Go version: go1.14.3", "name": ""} -21.07.12 16:48:49.356 nuctl (I) Cleaning up before deployment {"functionName": "tf-matterport-mask-rcnn"} -21.07.12 16:48:49.470 nuctl (I) Function already exists, deleting function containers {"functionName": "tf-matterport-mask-rcnn"} -21.07.12 16:48:50.247 nuctl (I) Staging files and preparing base images -21.07.12 16:48:50.248 nuctl (I) Building processor image {"imageName": "cvat/tf.matterport.mask_rcnn:latest"} -21.07.12 16:48:50.249 nuctl.platform.docker (I) Pulling image {"imageName": "quay.io/nuclio/handler-builder-python-onbuild:1.5.16-amd64"} -21.07.12 16:48:53.674 nuctl.platform.docker (I) Pulling image {"imageName": "quay.io/nuclio/uhttpc:0.0.1-amd64"} -21.07.12 16:48:57.424 nuctl.platform (I) Building docker image {"image": "cvat/tf.matterport.mask_rcnn:latest"} -21.07.12 16:48:57.763 nuctl.platform (I) Pushing docker image into registry {"image": "cvat/tf.matterport.mask_rcnn:latest", "registry": ""} -21.07.12 16:48:57.764 nuctl.platform (I) Docker image was successfully built and pushed into docker registry {"image": "cvat/tf.matterport.mask_rcnn:latest"} -21.07.12 16:48:57.764 nuctl (I) Build complete {"result": {"Image":"cvat/tf.matterport.mask_rcnn:latest","UpdatedFunctionConfig":{"metadata":{"name":"tf-matterport-mask-rcnn","namespace":"nuclio","labels":{"nuclio.io/project-name":"cvat"},"annotations":{"framework":"tensorflow","name":"Mask RCNN via Tensorflow","spec":"[\n { \"id\": 0, \"name\": \"BG\" },\n { \"id\": 1, \"name\": \"person\" },\n { \"id\": 2, \"name\": \"bicycle\" },\n { \"id\": 3, \"name\": \"car\" },\n { \"id\": 4, \"name\": \"motorcycle\" },\n { \"id\": 5, \"name\": \"airplane\" },\n { \"id\": 6, \"name\": \"bus\" },\n { \"id\": 7, \"name\": \"train\" },\n { \"id\": 8, \"name\": \"truck\" },\n { \"id\": 9, \"name\": \"boat\" },\n { \"id\": 10, \"name\": \"traffic_light\" },\n { \"id\": 11, \"name\": \"fire_hydrant\" },\n { \"id\": 12, \"name\": \"stop_sign\" },\n { \"id\": 13, \"name\": \"parking_meter\" },\n { \"id\": 14, \"name\": \"bench\" },\n { \"id\": 15, \"name\": \"bird\" },\n { \"id\": 16, \"name\": \"cat\" },\n { \"id\": 17, \"name\": \"dog\" },\n { \"id\": 18, \"name\": \"horse\" },\n { \"id\": 19, \"name\": \"sheep\" },\n { \"id\": 20, \"name\": \"cow\" },\n { \"id\": 21, \"name\": \"elephant\" },\n { \"id\": 22, \"name\": \"bear\" },\n { \"id\": 23, \"name\": \"zebra\" },\n { \"id\": 24, \"name\": \"giraffe\" },\n { \"id\": 25, \"name\": \"backpack\" },\n { \"id\": 26, \"name\": \"umbrella\" },\n { \"id\": 27, \"name\": \"handbag\" },\n { \"id\": 28, \"name\": \"tie\" },\n { \"id\": 29, \"name\": \"suitcase\" },\n { \"id\": 30, \"name\": \"frisbee\" },\n { \"id\": 31, \"name\": \"skis\" },\n { \"id\": 32, \"name\": \"snowboard\" },\n { \"id\": 33, \"name\": \"sports_ball\" },\n { \"id\": 34, \"name\": \"kite\" },\n { \"id\": 35, \"name\": \"baseball_bat\" },\n { \"id\": 36, \"name\": \"baseball_glove\" },\n { \"id\": 37, \"name\": \"skateboard\" },\n { \"id\": 38, \"name\": \"surfboard\" },\n { \"id\": 39, \"name\": \"tennis_racket\" },\n { \"id\": 40, \"name\": \"bottle\" },\n { \"id\": 41, \"name\": \"wine_glass\" },\n { \"id\": 42, \"name\": \"cup\" },\n { \"id\": 43, \"name\": \"fork\" },\n { \"id\": 44, \"name\": \"knife\" },\n { \"id\": 45, \"name\": \"spoon\" },\n { \"id\": 46, \"name\": \"bowl\" },\n { \"id\": 47, \"name\": \"banana\" },\n { \"id\": 48, \"name\": \"apple\" },\n { \"id\": 49, \"name\": \"sandwich\" },\n { \"id\": 50, \"name\": \"orange\" },\n { \"id\": 51, \"name\": \"broccoli\" },\n { \"id\": 52, \"name\": \"carrot\" },\n { \"id\": 53, \"name\": \"hot_dog\" },\n { \"id\": 54, \"name\": \"pizza\" },\n { \"id\": 55, \"name\": \"donut\" },\n { \"id\": 56, \"name\": \"cake\" },\n { \"id\": 57, \"name\": \"chair\" },\n { \"id\": 58, \"name\": \"couch\" },\n { \"id\": 59, \"name\": \"potted_plant\" },\n { \"id\": 60, \"name\": \"bed\" },\n { \"id\": 61, \"name\": \"dining_table\" },\n { \"id\": 62, \"name\": \"toilet\" },\n { \"id\": 63, \"name\": \"tv\" },\n { \"id\": 64, \"name\": \"laptop\" },\n { \"id\": 65, \"name\": \"mouse\" },\n { \"id\": 66, \"name\": \"remote\" },\n { \"id\": 67, \"name\": \"keyboard\" },\n { \"id\": 68, \"name\": \"cell_phone\" },\n { \"id\": 69, \"name\": \"microwave\" },\n { \"id\": 70, \"name\": \"oven\" },\n { \"id\": 71, \"name\": \"toaster\" },\n { \"id\": 72, \"name\": \"sink\" },\n { \"id\": 73, \"name\": \"refrigerator\" },\n { \"id\": 74, \"name\": \"book\" },\n { \"id\": 75, \"name\": \"clock\" },\n { \"id\": 76, \"name\": \"vase\" },\n { \"id\": 77, \"name\": \"scissors\" },\n { \"id\": 78, \"name\": \"teddy_bear\" },\n { \"id\": 79, \"name\": \"hair_drier\" },\n { \"id\": 80, \"name\": \"toothbrush\" }\n]\n","type":"detector"}},"spec":{"description":"Mask RCNN optimized for GPU","handler":"main:handler","runtime":"python:3.6","env":[{"name":"MASK_RCNN_DIR","value":"/opt/nuclio/Mask_RCNN"}],"resources":{"limits":{"nvidia.com/gpu":"1"}},"image":"cvat/tf.matterport.mask_rcnn:latest","targetCPU":75,"triggers":{"myHttpTrigger":{"class":"","kind":"http","name":"myHttpTrigger","maxWorkers":1,"workerAvailabilityTimeoutMilliseconds":10000,"attributes":{"maxRequestBodySize":33554432}}},"volumes":[{"volume":{"name":"volume-1","hostPath":{"path":"/home/nmanovic/Workspace/cvat/serverless/common"}},"volumeMount":{"name":"volume-1","mountPath":"/opt/nuclio/common"}}],"build":{"functionConfigPath":"serverless/tensorflow/matterport/mask_rcnn/nuclio/function-gpu.yaml","image":"cvat/tf.matterport.mask_rcnn","baseImage":"tensorflow/tensorflow:1.15.5-gpu-py3","directives":{"postCopy":[{"kind":"WORKDIR","value":"/opt/nuclio"},{"kind":"RUN","value":"apt update \u0026\u0026 apt install --no-install-recommends -y git curl"},{"kind":"RUN","value":"git clone --depth 1 https://github.com/matterport/Mask_RCNN.git"},{"kind":"RUN","value":"curl -L https://github.com/matterport/Mask_RCNN/releases/download/v2.0/mask_rcnn_coco.h5 -o Mask_RCNN/mask_rcnn_coco.h5"},{"kind":"RUN","value":"pip3 install numpy cython pyyaml keras==2.1.0 scikit-image Pillow"}]},"codeEntryType":"image"},"platform":{"attributes":{"mountMode":"volume","restartPolicy":{"maximumRetryCount":3,"name":"always"}}},"readinessTimeoutSeconds":60,"securityContext":{},"eventTimeout":"30s"}}}} -21.07.12 16:48:59.071 nuctl.platform (I) Waiting for function to be ready {"timeout": 60} -21.07.12 16:49:00.437 nuctl (I) Function deploy complete {"functionName": "tf-matterport-mask-rcnn", "httpPort": 49155} -``` - -
    - -Now you should be able to annotate objects using segmentation masks. - -![Mask RCNN results](/images/mask_rcnn_results.jpg) - ## Adding your own DL models ### Choose a DL model diff --git a/site/content/en/images/mask_rcnn_results.jpg b/site/content/en/images/mask_rcnn_results.jpg deleted file mode 100644 index 7fd5770a9b6f..000000000000 Binary files a/site/content/en/images/mask_rcnn_results.jpg and /dev/null differ