Awesome-Panoptic-Segmentation

This repo is a collection of the challenging panoptic segmentation, including papers, codes, and benchmark results, etc.

Panoptic Segmentation

Summarize in one sentence : Panoptic Segmentation proposes to solve the semantic segmentation(*Stuff*) and instance segmentation(*Thing*) in a unified and general manner.

Datasets

Generally, the datasets which contains both semantic and instance annotations can be used to solve the challenging panoptic task.

• Cityscapes
• Mapillary Vistas
• ADE20K
• COCO-Panoptic
• BDD100K (the instance annotations are temporaily not released)

Benchmark Results

• PQ are the standard metrics described in Panoptic Segmentation.

• PC are the standard metrics described in DeeperLab.

• COCO Benchmark

Method	Backbone	PQ	PQ-Thing	PQ-Stuff	SQ	RQ	mIoU	AP-Mask	PC	e2e
AUNet	ResNet-101	45.2	54.4	31.3	80.6	54.7	-	-	-	✔️
UPSNet	ResNet-101	42.5	48.6	33.4	-	-	54.3	34.3	-	✔️
OANet	ResNet-101	41.3	50.4	27.7	-	-	-	-	-	✔️
Panoptic FPN	ResNet-101	40.9	48.3	29.7	-	-	-	-	-	✔️
DeeperLab	Xception-71	34.3	37.5	29.6	77.1	43.1	-	-	56.8	✔️

• Cityscapes Benchmark

Method	Backbone	PQ	PQ-Thing	PQ-Stuff	SQ	RQ	mIoU	AP-Mask	PC	e2e
Panoptic(Merge)	-	61.2	66.4	54.0	80.9	74.4	-	-	-	❌
UPSNet	ResNet-50	59.3	54.6	62.7	79.7	73.0	75.2	33.3	-	✔️
TASCNet	ResNet-101	59.2	56	61.5	-	-	77.8	37.6	-	✔️
Panoptic FPN	ResNet-101	58.1	52.0	62.5	-	-	75.7	33.0	-	✔️
DeeperLab	Xception-71	56.5	-	-	-	-	-	-	75.6	✔️
AUNet	ResNet-101	59.0	54.8	62.1	-	-	75.6	34.4	-	✔️

• Mapillary Benchmark

Method	Backbone	PQ	PQ-Thing	PQ-Stuff	SQ	RQ	mIoU	AP-Mask	PC	e2e
Panoptic(Merge)	-	38.3	41.8	35.7	73.6	47.7	-	-	-	❌
TASCNet	ResNet-101	32.6	31.3	34.4	-	-	35.0	18.5	-	✔️
DeeperLab	Xception-71	31.6	25.0	40.3	75.5	40.1	-	-	55.3	✔️

Papers

CVPR2019

• Panoptic Segmentation: Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, Piotr Dollár.
  "Panoptic Segmentation." CVPR (2019). [paper]

• Panoptic FPN: Alexander Kirillov, Ross Girshick, Kaiming He, Piotr Dollár.
  "Panoptic Feature Pyramid Networks." CVPR (2019 oral). [paper]

• AUNet: Yanwei Li, Xinze Chen, Zheng Zhu, Lingxi Xie, Guan Huang, Dalong Du, Xingang Wang.
  "Attention-guided Unified Network for Panoptic Segmentation." CVPR (2019). [paper]

• UPSNet: Yuwen Xiong, Renjie Liao, Hengshuang Zhao, Rui Hu, Min Bai, Ersin Yumer, Raquel Urtasun.
  "UPSNet: A Unified Panoptic Segmentation Network." CVPR (2019). [paper] [code]

• DeeperLab: Tien-Ju Yang, Maxwell D. Collins, Yukun Zhu, Jyh-Jing Hwang, Ting Liu, Xiao Zhang, Vivienne Sze, George Papandreou, Liang-Chieh Chen.
  "DeeperLab: Single-Shot Image Parser." CVPR (2019). [paper] [project] [code]

• TASCNet: Jie Li, Allan Raventos, Arjun Bhargava, Takaaki Tagawa, Adrien Gaidon.
  "Learning to Fuse Things and Stuff." CVPR (2019). [paper]

• OANet: Huanyu Liu, Chao Peng, Changqian Yu, Jingbo Wang, Xu Liu, Gang Yu, Wei Jiang.
  "An End-to-End Network for Panoptic Segmentation." CVPR (2019). [paper]

• Eirikur Agustsson, Jasper R. R. Uijlings, Vittorio Ferrari .
  "Interactive Full Image Segmentation by Considering All Regions Jointly." CVPR (2019). [paper]

ECCV2018

• Qizhu Li, Anurag Arnab, Philip H.S. Torr.
  "Weakly- and Semi-Supervised Panoptic Segmentation." ECCV (2018). [paper] [code]

ArXiv

• Daan de Geus, Panagiotis Meletis, Gijs Dubbelman.
  "Panoptic Segmentation with a Joint Semantic and Instance Segmentation Network." arXiv (2018). [paper]

• Daan de Geus, Panagiotis Meletis, Gijs Dubbelman.
  "Single Network Panoptic Segmentation for Street Scene Understanding." arXiv (2019). [paper]

• David Owen, Ping-Lin Chang.
  "Detecting Reflections by Combining Semantic and Instance Segmentation." arXiv (2019). [paper]

• Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji.
  "PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things." arXiv (2019). [paper]

Blogs

• Face++ Detection Team: https://zhuanlan.zhihu.com/p/59141570