Skip to content

Latest commit

 

History

History
317 lines (239 loc) · 14.7 KB

README.md

File metadata and controls

317 lines (239 loc) · 14.7 KB

COSMOduleS: Classification, Object detection, Segmentation MOduleS



Github: source

Introduction

This repo provides comprehensive preprocessing and post-processing tools for common Computer Vision tasks.

Tasks Subtasks Defined
Format		 Visualization Format
Conversion		 Output
Analysis		 Label
Merging		 Active
Learning
Classification binary1
binary-bg2
multi-class1
multiclass-bg2
multi-binary3
 single_label1
single_label_bg2
multi_label3		 - - metrics
plotting
export		 ALL Entropy
Detection - coco
voc
yolo
GENERAL
 ALL between ANY
two types		 metrics
plotting
export		 V horizontal
flip
Segmentation instance1
semantic2
 coco1+2
GENERAL1+2		 ALL coco2general metrics
plotting
export		 - instance
semantic
  • "bg" means background. If there is background class, it must be class 0 in this repo.
  • Adding prediction results after the defined format can use the visualization and output analysis. All the formats with predictions are in example/*/prediction, e.g. here.

Motivation

  • [Classification] Complicated tasks
task label idx min compute class-0 metrics threshold optimization data format
binary classification 0 V V single_label
binary classification (cls-0 background) 1 V single_label_background
multi-class classification 0 V single_label
multi-class classification (cls-0 background) 1 V single_label_background
multi-label classification (cls-0 background) 0 V multi_label

  • [Classification] threshold optimization

    • multi-class classification (cls-0 background) checks whether prob-cls-0 < threshold, if yes, the pd-cls is pd[1:].argmax()
    • multi-class classification (cls-0 background) and multi-label classification (cls-0 background) take the mean of all optimized threshold for each foreground class

  • [Object Detection] Develop a GENERAL format to be the most convenient.

The formats can be summarized as following:

format extension files type box disadvantage
coco .json 1 int (xmin, ymin, w, h) get label of an image
yolo .txt len(imgs) float (cx, cy, w/2, h/2) visualization, compute metrics, etc.
voc .xml len(imgs) int (xmin, ymin, xmax, ymax) get class list
general .json 1 int (xmin, ymin, xmax, ymax) NO
  • [Segmentation] Develop a GENERAL format to be the most convenient.
Includes Content Advantage
general.json Includes every imgs: path, contour, filled and boxes with class Searching
gt_contour_*.npy (H, W) with {0, 1, ..., num_classes} int Plotting
gt_filled_*.npy (num_classes, H, W) with 0 or 1 int values Compute IOU for Metrics
*.jpg Raw data -
  • Segmentation prediction format: (num_classes, H, W) with 0~1 float values (probability). e.g. here

Installation

pip install cosmodules

or

git clone https://github.com/bnbsking/COSMOduleS.git
pip install -e .

Quick Start - Classification

from cosmodules.classification import ClassificationAnalysis

ClassificationAnalysis(
    ant_path = "example/classification/data/single_label.json",
    save_folder = "example/classification/output/single_label",
)
  • Label Merging:
from cosmodules.classification import ClassificationLabelMerging

ClassificationLabelMerging(
    cfg_path_list = [
        "example/classification/data/single_label.json",
        "example/classification/data_another_labeler/single_label.json",
    ],
    save_path = f"example/classification/output/label_merging/single_label.json"
)
  • Active Learning (see more in the example):
from cosmodules.classification import ClassificationActiveLearning

ClassificationActiveLearning(
    pred_path = "example/classification/prediction/single_label.json",
    save_path = "example/classification/output/active_learning/single_label.json",
    loss_name = "entropy"
)

Quick Start - Object detection

  • Format Conversion (see more in the example)
from cosmodules.detection import coco2any

coco2any(
    tgt_foramt = "voc",
    img_folder = "example/detection/data/coco",
    ant_path = "example/detection/data/coco/coco.json",
    save_folder = "example/detection/output/visualization_gt_conversion/coco2voc"
)

or

from cosmodules.detection import coco2general

coco2general(
    img_folder = "example/detection/data/coco",
    ant_path = "example/detection/data/coco/coco.json",
    save_path = "example/detection/output/visualization_gt_conversion/coco2general/general.json"
)
  • Visualization (see more in the example)
from cosmodules.detection import show_coco

show_coco(
    img_name = "pic0.jpg",
    img_folder = "example/detection/data/coco",
    ant_path = "example/detection/data/coco/coco.json"
)

or

from cosmodules.detection import show_general

show_general(
    img_name = "pic0.jpg",
    ant_path = "example/detection/data/general.json",
)  # when the anntotation includes predictions it will be shown!
  • Output Analysis
    • Please use the above Format conversion to change data format as general
    • The analysis pipeline is at here
from cosmodules.detection import DetectionAnalysis

DetectionAnalysis(
    ant_path = "example/detection/data/general.json",
    save_folder = "example/detection/output/metrics"
)
  • Label Merging:
from cosmodules.detection import DetectionLabelMerging

DetectionLabelMerging(
    cfg_path_list = [
        "example/detection/data/general.json",
        "example/detection/data_another_labeler/general.json",
    ],
    save_path = "example/detection/output/label_merging/general.json",
    ties_handling = "union"
)
  • Active Learning:
from cosmodules.detection import DetectionActiveLearningByHFlip

DetectionActiveLearningByHFlip(
    pred_path_1 = f"{ROOT}/example/detection/prediction/general.json",
    pred_path_2 = f"{ROOT}/example/detection/prediction/general_horizontal_flip.json",
    save_path = f"{ROOT}/example/detection/output/active_learning/general.json"
)

Quick Start - Segmentation

  • Format Conversion (see more in the example)
from cosmodules.segmentation import coco2general

coco2general(
    img_folder = "example/segmentation/data/coco",
    ant_path = "example/segmentation/data/coco/coco.json",
    save_folder = f"example/segmentation/output/visualization_gt_conversion/coco2general"
)
  • Visualization (see more in the example)
from cosmodules.segmentation import show_coco

show_coco(
    img_name = "img1.jpg",
    img_folder = "example/segmentation/data/coco",
    ant_path = "example/segmentation/data/coco/coco.json"
)   # when the anntotation includes predictions it will be shown!

or

from cosmodules.segmentation import show_general

show_general(
    img_name = "img1.jpg",
    ant_path = "example/segmentation/data/general/general.json"
)
  • Output Analysis
    • Please use the above Format conversion to change data format as general
    • The analysis pipeline is at here
from cosmodules.segmentation import SegmentationAnalysis

SegmentationAnalysis(
    ant_path = "example/segmentation/prediction/instance/general.json",
    save_folder = "example/segmentation/output/metrics/instance",
    task = "instance",
)

or

from cosmodules.segmentation import SegmentationAnalysis

SegmentationAnalysis(
    ant_path = "example/segmentation/prediction/semantic/general.json",
    save_folder = "example/segmentation/output/metrics/semantic",
    task = "semantic"
)
  • Active Learning:
from cosmodules.segmentation import (
    InstanceSegmentationActiveLearningByHFlip,
    SemanticSegmentationActiveLearning
)

InstanceSegmentationActiveLearningByHFlip(
    pred_path_1 = "example/segmentation/prediction/instance/general.json",
    pred_path_2 = "example/segmentation/prediction/instance_horizontal_flip/general.json",
    save_path = "example/segmentation/output/active_learning/instance.json"
)

or

SemanticSegmentationActiveLearning(
    pred_path = "example/segmentation/prediction/semantic/general.json",
    save_path = "example/segmentation/output/active_learning/semantic.json",
    loss_name = "entropy"
)

Examples

  • [detection]: format conversion workflow .

  • detection visualization .

  • confusion .

  • prf curves .

More

  • Feel free to ask if you have any question.
  • Notice not supported
    • segmentation general2coco
    • segmentation label merging

Acknowledgement

  • Confusion Matrix reference here