A collection of resources on neural rendering.
If you think I have missed out on something (or) have any suggestions (papers, implementations and other resources), feel free to pull a request
Feedback and contributions are welcome!
markdown format:
**Here is the Paper Name.**<br>
*[Author 1](homepage), Author 2, and Author 3.*<br>
Conference or Journal Year. [[PDF](link)] [[Project](link)] [[Github](link)] [[Video](link)] [[Data](link)]- Intruduction of Neural Rendering
- Related Surveys and Course Notes
- Inverse Rendering
- Neural Rerendering
- Differentiable Rendering
- Volumetric Performance Capture (Free Viewpoint Video)
- Semantic Photo Synthesis and Manipulation
- Texture and Surface Embedding or Mapping
- Neural Scene Representation and Rendering
- Novel-View Synthesis for Objects and Scenes
- Light, Reflectance, Illuminance and Shade
- Dubbing and Talking-Head Animation
- Motion Transfer, Retargeting, and Reenactment
Neural Rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e.g., by the integration of differentiable rendering into network training.
Ayush Tewari et. al. define Neural Rendering as
Deep image or video generation approaches that enable explicit or implicit control of scene properties such as illumination, camera parameters, pose, geometry, appearance, and semantic structure.
A typical neural rendering approach takes as input images corresponding to certain scene conditions (for example, viewpoint, lighting, layout, etc.), builds a “neural” scene representation from them, and “renders” this representation under novel scene properties to synthesize novel images.
CVPR 2020 tutorial define Neural Rendering as
Neural rendering is a new class of deep image and video generation approaches that enable explicit or implicit control of scene properties such as illumination, camera parameters, pose, geometry, appearance, and semantic structure. It combines generative machine learning techniques with physical knowledge from computer graphics to obtain controllable and photo-realistic outputs.
Given high-quality scene specifications, Classic Rendering Methods can render photorealistic images for a variety of complex real-world phenomena. Moreover, rendering gives us explicit editing control over all the elements of the scene-camera viewpoint, lighting, geometry and materials. However, building high-quality scene models, especially directly from images, requires significant manual effort, and automated scene modeling from images is an open research problem. On the other hand, Deep Generative Networks are now starting to produce visually compelling images and videos either from random noise, or conditioned on certain user specifications like scene segmentation and layout. However, they do not yet allow for fine-grained control over scene appearance and cannot always handle the complex, non-local, 3D interactions between scene properties. In contrast, neural rendering methods hold the promise of combining these approaches to enable controllable, high-quality synthesis of novel images from input images/videos.
State of the Art on Neural Rendering.
Ayush Tewari, Ohad Fried, Justus Thies, Vincent Sitzmann, Stephen Lombardi, Kalyan Sunkavalli, Ricardo Martin-Brualla, Tomas Simon, Jason Saragih, Matthias Nießner, Rohit Pandey, Sean Fanello, Gordon Wetzstein, Jun-Yan Zhu, Christian Theobalt, Maneesh Agrawala, Eli Shechtman, Dan B Goldman, Michael Zollhöfer.
Eurographics 2020.
CVPR 2020 tutorial on Neural Rendering.
Ayush Tewari, Ohad Fried, Justus Thies, Vincent Sitzmann, Stephen Lombardi, Kalyan Sunkavalli, Ricardo Martin-Brualla, Tomas Simon, Jason Saragih, Matthias Nießner, Rohit K. Pandey, Sean Fanello, Gordon Wetzstein, Jun-Yan Zhu, Christian Theobalt, Maneesh Agrawala, Eli Shechtman, Dan B. Goldman, Michael Zollhöfer.
Differentiable Rendering: A Survey.
Hiroharu Kato, Deniz Beker, Mihai Morariu, Takahiro Ando, Toru Matsuoka, Wadim Kehl, Adrien Gaidon.
PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting.
Kai Zhang, Fujun Luan, Qianqian Wang, Kavita Bala, Noah Snavely.
CVPR 2021. [PDF] [Project]
Invertible Neural BRDF for Object Inverse Rendering.
Zhe Chen, Shohei Nobuhara, Ko Nishino.
ECCV 2020. [PDF]
Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF From a Single Image.
Zhengqin Li, Mohammad Shafiei, Ravi Ramamoorthi, Kalyan Sunkavalli, Manmohan Chandraker.
CVPR 2020.[PDF] [Project] [Github]
Polarimetric Multi-View Inverse Rendering.
Jinyu Zhao, Yusuke Monno, Masatoshi Okutomi.
ECCV 2020. [PDF]
NiLBS: Neural Inverse Linear Blend Skinning.
Timothy Jeruzalski, David I.W. Levin, Alec Jacobson, Paul Lalonde, Mohammad Norouzi, Andrea Tagliasacchi.
arxiv 2020. [PDF]
Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer.
Wenzheng Chen, Jun Gao, Huan Ling, Edward J. Smith, Jaakko Lehtinen, Alec Jacobson, Sanja Fidler.
NeurIPS 2019. [PDF]
DRWR: A Differentiable Renderer without Rendering for Unsupervised 3D Structure Learning from Silhouette Images.
Zhizhong Han, Chao Chen, Yu-Shen Liu, Matthias Zwicker.
ICML 2020. [PDF]
InverseRenderNet: Learning Single Image Inverse Rendering.
Ye Yu, William A. P. Smith.
CVPR 2019. [PDF] [Github] [IIW Dataset]
Learning Inverse Rendering of Faces from Real-world Videos.
Yuda Qiu, Zhangyang Xiong, Kai Han, Zhongyuan Wang, Zixiang Xiong, Xiaoguang Han.
arxiv, 2020. [PDF] [Github]
Neural Lumigraph Rendering.
Petr Kellnhofer, Lars Jebe, Andrew Jones, Ryan Spicer, Kari Pulli, Gordon Wetzstein.
CVPR 2021. [PDF] [Project] [Data]
NeRViS: Neural Re-rendering for Full-frame Video Stabilization.
Yu-Lun Liu, Wei-Sheng Lai, Ming-Hsuan Yang, Yung-Yu Chuang, Jia-Bin Huang.
arxiv 2021. [PDF] [Github]
Neural Re-Rendering of Humans from a Single Image.
Kripasindhu Sarkar, Dushyant Mehta, Weipeng Xu, Vladislav Golyanik, Christian Theobalt.
ECCV 2020. [PDF]
StyleUV: Diverse and High-fidelity UV Map Generative Model.
Myunggi Lee, Wonwoong Cho, Moonheum Kim, David Inouye, Nojun Kwak.
arxiv 2020. [PDF]
Neural Rerendering in the Wild.
Moustafa Meshry, Dan B Goldman, Sameh Khamis, Hugues Hoppe, Rohit Pandey, Noah Snavely, Ricardo Martin-Brualla.
CVPR 2019. [PDF]
Revealing Scenes by Inverting Structure from Motion Reconstructions.
Francesco Pittaluga, Sanjeev J. Koppal, Sing Bing Kang, Sudipta N. Sinha.
CVPR 2019. [PDF]
Neural Re-Rendering of Humans from a Single Image.
Kripasindhu Sarkar, Dushyant Mehta, Weipeng Xu, Vladislav Golyanik, Christian Theobalt.
ECCV 2020. [PDF]
Modular Primitives for High-Performance Differentiable Rendering.
Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, Timo Aila.
arxiv 2020. [PDF] [Github]
Monocular Differentiable Rendering for Self-Supervised 3D Object Detection.
Deniz Beker, Hiroharu Kato, Mihai Adrian Morariu, Takahiro Ando, Toru Matsuoka, Wadim Kehl, Adrien Gaidon.
ECCV 2020. [PDF]
ST-NeRF: Editable Free-viewpoint Video Using a Layered Neural Representation.
Jiakai Zhang, Xinhang Liu, Xinyi Ye, Fuqiang Zhao, Yanshun Zhang, Minye Wu, Yingliang Zhang, Lan Xu, Jingyi Yu.
arxiv 2021. [PDF]
POSEFusion: Pose-guided Selective Fusion for Single-view Human Volumetric Capture.
Zhe Li, Tao Yu, Zerong Zheng, Kaiwen Guo, Yebin Liu.
CVPR 2021 (oral). [PDF] [Github]
Monocular Real-Time Volumetric Performance Capture.
Ruilong Li, Yuliang Xiu, Shunsuke Saito, Zeng Huang, Kyle Olszewski, Hao Li.
arxiv 2020. [PDF]
Neural Sparse Voxel Fields.
Lingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, Christian Theobalt.
arxiv 2020. [PDF]
Volumetric Capture of Humans with a Single RGBD Camera via Semi-Parametric Learning.
Rohit Pandey, Anastasia Tkach, Shuoran Yang, Pavel Pidlypenskyi, Jonathan Taylor, Ricardo Martin-Brualla, Andrea Tagliasacchi, George Papandreou, Philip Davidson, Cem Keskin, Shahram Izadi, Sean Fanello.
CVPR 2019. [PDF]
Neural Volumes: Learning Dynamic Renderable Volumes from Images.
Stephen Lombardi, Tomas Simon, Jason Saragih, Gabriel Schwartz, Andreas Lehrmann, Yaser Sheikh.
SIGGRAPH 2019. [PDF]
StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows.
Rameen Abdal, Peihao Zhu, Niloy Mitra, Peter Wonka.
Siggraph Asia 2020. [PDF] [Github]
Layered Neural Rendering for Retiming People in Video.
Erika Lu, Forrester Cole, Tali Dekel, Weidi Xie, Andrew Zisserman, David Salesin, William T. Freeman, Michael Rubinstein.
SIGGRAPH Asia 2020. [PDF] [Project]
Neural Hair Rendering.
Menglei Chai, Jian Ren, Sergey Tulyakov.
arxiv 2020. [PDF]
MichiGAN: Multi-Input-Conditioned Hair Image Generation for Portrait Editing.
Zhentao Tan, Menglei Chai, Dongdong Chen, Jing Liao, Qi Chu, Lu Yuan, Sergey Tulyakov, Nenghai Yu.
SIGGRAPH 2020. [PDF]
pix2pixHD: High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs.
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, Bryan Catanzaro.
CVPR 2018. [PDF] [Github]
SPADE: Semantic Image Synthesis with Spatially-Adaptive Normalization.
Taesung Park, Ming-Yu Liu, Ting-Chun Wang, Jun-Yan Zhu.
CVPR 2019. [PDF] [Github]
Semantic Bottleneck Scene Generation.
Samaneh Azadi, Michael Tschannen, Eric Tzeng, Sylvain Gelly, Trevor Darrell, Mario Lucic.
arxiv, 2019. [PDF]
Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generation.
Hao Tang, Dan Xu, Yan Yan, Philip H. S. Torr, Nicu Sebe.
CVPR 2020. [PDF] [Github]
SelectionGAN: Multi-Channel Attention Selection GAN with Cascaded Semantic Guidance for Cross-View Image Translation.
Hao Tang, Dan Xu, Nicu Sebe, Yanzhi Wang, Jason J. Corso, Yan Yan.
VPR 2019. [PDF] [Github]
Neural Surface Maps.
Luca Morreale, Noam Aigerman, Vladimir Kim, Niloy J. Mitra.
arxiv 2021. [PDF] [Github]
NeuTex: Neural Texture Mapping for Volumetric Neural Rendering.
Fanbo Xiang, Zexiang Xu, Miloš Hašan, Yannick Hold-Geoffroy, Kalyan Sunkavalli, Hao Su.
arxiv 2021. [PDF]
Better Patch Stitching for Parametric Surface Reconstruction.
Zhantao Deng, Jan Bednařík, Mathieu Salzmann, Pascal Fua.
3DV 2020. [PDF]
Continuous Surface Embeddings.
Natalia Neverova, David Novotny, Marc Szafraniec, Vasil Khalidov, Patrick Labatut, Andrea Vedaldi.
NerIPS 2020. [PDF]
Transposer: Universal Texture Synthesis Using Feature Maps as Transposed Convolution Filter.
Guilin Liu, Rohan Taori, Ting-Chun Wang, Zhiding Yu, Shiqiu Liu, Fitsum A. Reda, Karan Sapra, Andrew Tao, Bryan Catanzaro.
arxiv 2020. [PDF]
Wasserstein Generative Models for Patch-based Texture Synthesis.
Antoine Houdard, Arthur Leclaire, Nicolas Papadakis, Julien Rabin.
arxiv 2020. [PDF]
Deep Geometric Texture Synthesis.
Amir Hertz, Rana Hanocka, Raja Giryes, Daniel Cohen-Or.
SIGGRAPH 2020. [PDF]
GramGAN: Deep 3D Texture Synthesis From 2D Exemplars.
Tiziano Portenier, Siavash Bigdeli, Orçun Göksel.
arxiv 2020. [PDF]
GPU-Accelerated Mobile Multi-view Style Transfer.
Puneet Kohli, Saravana Gunaseelan, Jason Orozco, Yiwen Hua, Edward Li, Nicolas Dahlquist.
arxiv 2020. [PDF]
Leveraging 2D Data to Learn Textured 3D Mesh Generation.
Paul Henderson, Vagia Tsiminaki, Christoph H. Lampert.
CVPR 2020. [PDF]
Articulation-aware Canonical Surface Mapping.
Nilesh Kulkarni, Abhinav Gupta, David F. Fouhey, Shubham Tulsiani.
CVPR 2020. [PDF] [Github] [Project]
UnrealText: Synthesizing Realistic Scene Text Images from the Unreal World.
Shangbang Long, Cong Yao.
CVPR 2020. [PDF] [Github]
Adversarial Texture Optimization from RGB-D Scans.
Jingwei Huang, Justus Thies, Angela Dai, Abhijit Kundu, Chiyu Jiang, Leonidas Guibas, Matthias Nießner, Thomas Funkhouser.
CVPR 2020. [PDF] [Project] [Github] [pyRender]
CSM: Canonical Surface Mapping via Geometric Cycle Consistency.
Nilesh Kulkarni, Abhinav Gupta, Shubham Tulsiani.
ICCV 2019. [PDF] [Github] [Project]
Texture Mapping for 3D Reconstruction with RGB-D Sensor.
Yanping Fu, Qingan Yan, Long Yang, Jie Liao, Chunxia Xiao.
CVPR 2018. [PDF] [thecvf] [Code on Github]
Let There Be Color! - Large-Scale Texturing of 3D Reconstructions.
Waechter, Michael and Moehrle, Nils and Goesele, Michael.
ECCV 2018. [PDF] [Project] [Github] [rayint] [Eigen] [Multi-View Environment] [mapMAP]
Learning Category-Specific Mesh Reconstruction from Image Collections.
Angjoo Kanazawa, Shubham Tulsiani Alexei A. Efros, Jitendra Malik.
ECCV 2018. [Github] [Project]
Texture Fields: Learning Texture Representations in Function Space.
Michael Oechsle, Lars Mescheder, Michael Niemeyer, Thilo Strauss, Andreas Geiger.
ICCV 2019. [PDF]
AtlasNet: A Papier-Mache Approach to Learning 3D Surface Generation.
Thibault Groueix, Matthew Fisher, Vladimir Kim, Bryan Russell, Mathieu Aubry.
CVPR 2018. [PDF] [Project] [Github]
Learning Elementary Structures For 3D Shape Generation And Matching.
Theo Deprelle, Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan C. Russell, Mathieu Aubry.
arxiv, 2019. [PDF]
[Project]
[Github]
Learning to Generate Textures on Meshes.
Amit Raj, Cusuh Ham, Connelly Barnes, Vladimir Kim, Jingwan Lu, James Hays.
CVPR Deep Generative Models for 3D Understanding 2019 (Best Paper). [PDF]
Unsupervised Texture Transfer from Images to Model Collections.
Tuanfeng Yand Wang, Hao Su, Qixing Huang, Jingwei Huang, Leonidas J. Guibas, Niloy J. Mitra.
SIGGRAPH Asia 2016. [PDF]
[Project]
[Data]
[Awesome Neural Radiance Fields]
Radiance Fields from a Single Image.
Konstantinos Rematas, Ricardo Martin-Brualla, Vittorio Ferrari.
ICML 2021. [PDF]
Deep Medial Fields.
Daniel Rebain, Ke Li, Vincent Sitzmann, Soroosh Yazdani, Kwang Moo Yi, Andrea Tagliasacchi.
arxiv 2021. [PDF]
DoubleField: Bridging the Neural Surface and Radiance Fields for High-fidelity Human Rendering.
Ruizhi Shao, Hongwen Zhang, He Zhang, Yanpei Cao, Tao Yu, Yebin Liu.
arxiv 2021. [PDF]
SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decomposition.
Rishabh Kabra, Daniel Zoran, Goker Erdogan, Loic Matthey, Antonia Creswell, Matthew Botvinick, Alexander Lerc.
arxiv 2021. [PDF]
Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering.
Vincent Sitzmann, Semon Rezchikov, William T. Freeman, Joshua B. Tenenbaum, Fredo Durand.
arxiv 2021. [PDF] [Project]
NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination.
Xiuming Zhang, Pratul P. Srinivasan, Boyang Deng, Paul Debevec, William T. Freeman, Jonathan T. Barron.
arxiv 2021. [PDF] [Project] [Github]
Recursive-NeRF: An Efficient and Dynamically Growing NeRF.
Guo-Wei Yang, Wen-Yang Zhou, Hao-Yang Peng, Dun Liang, Tai-Jiang Mu, Shi-Min Hu.
arxiv 2021. [PDF]
Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields.
Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, Pratul P. Srinivasan.
arxiv 2021. [PDF] [Project]
UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction.
Michael Oechsle, Songyou Peng, Andreas Geiger.
arxiv 2021. [PDF]
BARF: Bundle-Adjusting Neural Radiance Fields.
Chen-Hsuan Lin, Wei-Chiu Ma, Antonio Torralba, Simon Lucey.
arxiv 2021. [PDF]
SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes.
Xu Chen, Yufeng Zheng, Michael J. Black, Otmar Hilliges, Andreas Geiger.
CVPR 2021. [PDF]
Convolutional Neural Opacity Radiance Fields.
Haimin Luo, Anpei Chen, Qixuan Zhang, Bai Pang, Minye Wu, Lan Xu, Jingyi Yu.
arxiv 2021. [PDF]
KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs.
Christian Reiser, Songyou Peng, Yiyi Liao, Andreas Geiger.
arxiv 2021. [PDF]
MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo.
Anpei Chen, Zexiang Xu, Fuqiang Zhao, Xiaoshuai Zhang, Fanbo Xiang, Jingyi Yu, Hao Su.
arxiv 2021. [PDF]
PlenOctrees for Real-time Rendering of Neural Radiance Fields.
Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng, Angjoo Kanazawa.
arxiv 2021. [PDF]
FastNeRF: High-Fidelity Neural Rendering at 200FPS.
Stephan J. Garbin, Marek Kowalski, Matthew Johnson, Jamie Shotton, Julien Valentin.
arxiv 2021. [PDF]
DONeRF: Towards Real-Time Rendering of Neural Radiance Fields using Depth Oracle Networks.
Thomas Neff, Pascal Stadlbauer, Mathias Parger, Andreas Kurz, Chakravarty R. Alla Chaitanya, Anton Kaplanyan, Markus Steinberger.
arxiv 2021. [PDF] [Project]
Mixture of Volumetric Primitives for Efficient Neural Rendering.
Stephen Lombardi, Tomas Simon, Gabriel Schwartz, Michael Zollhoefer, Yaser Sheikh, Jason Saragih.
arxiv 2021. [PDF]
ShaRF: Shape-conditioned Radiance Fields from a Single View.
Konstantinos Rematas, Ricardo Martin-Brualla, Vittorio Ferrari.
arxiv 2021. [PDF] [Project]
NeRF--: Neural Radiance Fields Without Known Camera Parameters.
Zirui Wang, Shangzhe Wu, Weidi Xie, Min Chen, Victor Adrian Prisacariu.
arxiv 2021. [PDF] [Project]
A-NeRF: Surface-free Human 3D Pose Refinement via Neural Rendering.
Shih-Yang Su, Frank Yu, Michael Zollhoefer, Helge Rhodin.
arxiv 2021. [PDF] [Project]
Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes.
Towaki Takikawa, Joey Litalien, Kangxue Yin, Karsten Kreis, Charles Loop, Derek Nowrouzezahrai, Alec Jacobson, Morgan McGuire, Sanja Fidler.
arxiv 2021. [PDF]
Neural Volume Rendering: NeRF And Beyond.
Frank Dellaert, Lin Yen-Chen.
arxiv 2021. [PDF]
Non-line-of-Sight Imaging via Neural Transient Fields.
Siyuan Shen, Zi Wang, Ping Liu, Zhengqing Pan, Ruiqian Li, Tian Gao, Shiying Li, Jingyi Yu.
arxiv 2021. [PDF]
DeepSurfels: Learning Online Appearance Fusion.
Marko Mihajlovic, Silvan Weder, Marc Pollefeys, Martin R. Oswald.
arxiv 2021. [PDF]
Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video.
Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollhöfer, Christoph Lassner, Christian Theobalt.
arxiv 2020. [PDF] [Project] [Github]
Learned Initializations for Optimizing Coordinate-Based Neural Representations.
Matthew Tancik, [Ben Mildenhall, Terrance Wang, Divi Schmidt, Pratul P. Srinivasan, Jonathan T. Barron, Ren Ng.
arxiv 2020. [PDF] [Project]
Learning Compositional Radiance Fields of Dynamic Human Heads.
Ziyan Wang, Timur Bagautdinov, Stephen Lombardi, Tomas Simon, Jason Saragih, Jessica Hodgins, Michael Zollhöfer.
arxiv 2020. [PDF]
OSF: Object-Centric Neural Scene Rendering.
Michelle Guo, Alireza Fathi, Jiajun Wu, Thomas Funkhouser.
arxiv 2020. [PDF] [Project]
Deformed Implicit Field: Modeling 3D Shapes with Learned Dense Correspondence.
Yu Deng, Jiaolong Yang, Xin Tong.
arxiv 2020. [PDF]
Iso-Points: Optimizing Neural Implicit Surfaces with Hybrid Representations.
Wang Yifan, Shihao Wu, Cengiz Oztireli, Olga Sorkine-Hornung.
arxiv 2020. [PDF]
Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction.
Guy Gafni, Justus Thies, Michael Zollhöfer, Matthias Nießner.
arxiv 2020. [PDF] [Project] [Video]
Portrait Neural Radiance Fields from a Single Image.
Chen Gao, Yichang Shih, Wei-Sheng Lai, Chia-Kai Liang, Jia-Bin Huang.
arxiv 2020. [PDF] [Project]
iNeRF: Inverting Neural Radiance Fields for Pose Estimation.
Lin Yen-Chen, Pete Florence, Jonathan T. Barron, Alberto Rodriguez, Phillip Isola, Tsung-Yi Lin.
arxiv 2020. [PDF] [Project]
NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis.
Pratul P. Srinivasan, Boyang Deng, Xiuming Zhang, Matthew Tancik, Ben Mildenhall, Jonathan T. Barron.
CVPR 2021. [PDF] [Project]
pixelNeRF: Neural Radiance Fields from One or Few Images.
Alex Yu, Vickie Ye, Matthew Tancik, Angjoo Kanazawa.
CVPR 2021. [PDF] [Project]
AutoInt: Automatic Integration for Fast Neural Volume Rendering.
David B. Lindell, Julien N. P. Martel, Gordon Wetzstein.
CVPR 2021 (oral). [PDF] [Project]
DeRF: Decomposed Radiance Fields.
Daniel Rebain, Wei Jiang, Soroosh Yazdani, Ke Li, Kwang Moo Yi, Andrea Tagliasacchi.
CVPR 2021. [PDF] [Project]
D-NERF: Deformable Neural Radiance Fields.
Keunhong Park, Utkarsh Sinha, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Steven M. Seitz, Ricardo-Martin Brualla.
CVPR 2021. [PDF] [Project]
Space-time Neural Irradiance Fields for Free-Viewpoint Video.
Wenqi Xian, Jia-Bin Huang, Johannes Kopf, Changil Kim.
arxiv 2020. [PDF] [Project]
GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields.
Michael Niemeyer, Andreas Geiger.
arxiv 2020. [PDF]
Neural Unsigned Distance Fields for Implicit Function Learning.
Julian Chibane, Aymen Mir, Gerard Pons-Moll.
NeurIPS 2020. [PDF] [Github]
SIREN: Implicit Neural Representations with Periodic Activation Functions.
Vincent Sitzmann, Julien N. P. Martel, Alexander W. Bergman, David B. Lindell, Gordon Wetzstein.
NeurIPS 2020 (Oral). [PDF] [Project] [Github] [Data]
GRF: Learning a General Radiance Field for 3D Scene Representation and Rendering.
Alex Trevithick, Bo Yang.
arxiv 2020. [PDF] [Github]
3D Scene Generation.
Angel X. Chang, Daniel Ritchie, Qixing Huang, Manolis Savva.
CVPR 2019 Workshop.
Local Deep Implicit Functions for 3D Shape.
Kyle Genova, Forrester Cole, Avneesh Sud, Aaron Sarna, Thomas Funkhouser.
CVPR 2020. [PDF]
PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations.
Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollhöfer, Carsten Stoll, Christian Theobalt.
arxiv 2020. [PDF]
Unsupervised 3D Learning for Shape Analysis via Multiresolution Instance Discrimination.
Peng-Shuai Wang, Yu-Qi Yang, Qian-Fang Zou, Zhirong Wu, Yang Liu, Xin Tong.
arxiv 2020. [PDF]
Learning Implicit Fields for Generative Shape Modeling.
Zhiqin Chen, Hao Zhang.
CVPR 2019. [PDF] [Project] [Github]
Geometric Correspondence Fields: Learned Differentiable Rendering for 3D Pose Refinement in the Wild.
Alexander Grabner, Yaming Wang, Peizhao Zhang, Peihong Guo, Tong Xiao, Peter Vajda, Peter M. Roth, Vincent Lepetit.
ECCV 2020. [PDF]
Equivariant Neural Rendering.
Emilien Dupont, Miguel Angel Bautista, Alex Colburn, Aditya Sankar, Carlos Guestrin, Josh Susskind, Qi Shan.
ICML 2020. [PDF] [Github]
CoReNet: Coherent 3D Scene Reconstruction From a Single RGB Image.
Stefan Popov, Pablo Bauszat, Vittorio Ferrari.
arxiv 2020. [PDF]
Single-View View Synthesis with Multiplane Images.
Richard Tucker and Noah Snavely.
CVPR 2020. [PDF] [Project]
LIMP: Learning Latent Shape Representations with Metric Preservation Priors.
Luca Cosmo, Antonio Norelli, Oshri Halimi, Ron Kimmel, Emanuele Rodolà.
arxiv 2020. [PDF]
Learning 3D Part Assembly from a Single Image.
Yichen Li, Kaichun Mo, Lin Shao, Minhyuk Sung, Leonidas Guibas.
arxiv 2020. [PDF]
Curriculum DeepSDF.
Yueqi Duan, Haidong Zhu, He Wang, Li Yi, Ram Nevatia, Leonidas J. Guibas.
arxiv, 2020. [PDF] [Github]
PolyGen: An Autoregressive Generative Model of 3D Meshes.
Charlie Nash, Yaroslav Ganin, S. M. Ali Eslami, Peter W. Battaglia.
arxiv, 2020. [PDF]
Self-supervised Learning of 3D Objects from Natural Images.
Hiroharu Kato, Tatsuya Harada.
arxiv, 2019. [PDF] [Project]
BlockGAN: Learning 3D Object-Aware Scene Representations from Unlabelled Images.
Thu Nguyen-Phuoc, Christian Richardt, Long Mai, Yong-Liang Yang, Niloy Mitra.
arxiv, 2020. [PDF] [Project]
DualSDF: Semantic Shape Manipulation using a Two-Level Representation.
Zekun Hao, Hadar Averbuch-Elor, Noah Snavely, Serge Belongie.
CVPR 2020. [PDF]
Learning a Neural 3D Texture Space from 2D Exemplars.
Philipp Henzler, Niloy J. Mitra, Tobias Ritschel.
CVPR 2020. [PDF] [Project]
Neural Contours: Learning to Draw Lines from 3D Shapes.
Difan Liu, Mohamed Nabail, Aaron Hertzmann, Evangelos Kalogerakis.
CVPR 2020. [PDF] [Github]
Pix2Shape: Towards Unsupervised Learning of 3D Scenes from Images using a View-based Representation.
Sai Rajeswar, Fahim Mannan, Florian Golemo, Jérôme Parent-Lévesque, David Vazquez, Derek Nowrouzezahrai, Aaron Courville.
IJCV 2020. [PDF]
VCN: Volumetric Correspondence Networks for Optical Flow.
Gengshan Yang, Deva Ramanan.
NeurIPS 2019. [PDF] [GitHub] [Project]
Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations.
Vincent Sitzmann, Michael Zollhöfer, Gordon Wetzstein.
NeurIPS 2019 (Oral, Honorable Mention "Outstanding New Directions").
[PDF] [Project] [Github] [Dataset]
Transformable Bottleneck Networks.
Kyle Olszewski, Sergey Tulyakov, Oliver Woodford, Hao Li, Linjie Luo.
ICCV 2019. [PDF]
Equivariant Multi-View Networks.
Carlos Esteves, Yinshuang Xu, Christine Allen-Blanchette, Kostas Daniilidis.
ICCV 2019. [PDF]
DeepVoxels: Learning Persistent 3D Feature Embeddings.
Vincent Sitzmann, Justus Thies, Felix Heide, Matthias Nießner, Gordon Wetzstein, Michael Zollhöfer.
CVPR 2019 (Oral). [Project] [PDF] [Code]
Occupancy Networks: Learning 3D Reconstruction in Function Space.
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, Andreas Geiger.
CVPR 2019. [PDF] [Github]
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation.
eong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, Steven Lovegrove.
CVPR 2019. [PDF] [Github]
DeepSDF x Sim(3): Extending DeepSDF for automatic 3D shape retrieval and similarity transform estimation.
Oladapo Afolabi, Allen Yang, Shankar S. Sastry.
arxiv 2020. [PDF]
Learning View Priors for Single-view 3D Reconstruction.
Hiroharu Kato, Tatsuya Harada.
CVPR 2019. [PDF] [Project] [Github]
HoloGAN: Unsupervised Learning of 3D Representations from Natural Images.
Nguyen-Phuoc, Chuan Li, Lucas Theis, Christian Richardt Yong-liang Yang.
ICCV 2019. [PDF] [GitHub]
C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion.
David Novotny, Nikhila Ravi, Benjamin Graham, Natalia Neverova, Andrea Vedaldi.
ICCV 2019. [PDF] [Github] [Project]
CSM: Canonical Surface Mapping via Geometric Cycle Consistency.
Nilesh Kulkarni, Abhinav Gupta, Shubham Tulsiani.
ICCV 2019. [PDF] [Github] [Project]
Neural Scene Graphs for Dynamic Scenes.
Julian Ost, Fahim Mannan, Nils Thuerey, Julian Knodt, Felix Heide.
CVPR 2021. [PDF] [Project]
NeLF: Practical Novel View Synthesis with Neural Light Field.
Celong Liu, Zhong Li, Junsong Yuan, Yi Xu.
arxiv 2021. [PDF]
NeMI: Unifying Neural Radiance Fields with Multiplane Images for Novel View Synthesis.
Jiaxin Li, Zijian Feng, Qi She, Henghui Ding, Changhu Wang, Gim Hee Lee.
arxiv 2021. [PDF] [Project]
Stereo Radiance Fields (SRF): Learning View Synthesis for Sparse Views of Novel Scenes.
Julian Chibane, Aayush Bansal, Verica Lazova, Gerard Pons-Moll.
CVPR 2021. [PDF]
Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis.
Ajay Jain, Matthew Tancik, Pieter Abbeel.
arxiv 2021. [PDF] [Project]
Baking Neural Radiance Fields for Real-Time View Synthesis.
Peter Hedman, Pratul P. Srinivasan, Ben Mildenhall, Jonathan T. Barron, Paul Debevec.
arxiv 2021. [PDF] [Project]
ID-Unet: Iterative Soft and Hard Deformation for View Synthesis.
Mingyu Yin, Li Sun, Qingli Li.
CVPR 2021. [PDF]
NSFF: Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes.
Zhengqi Li, Simon Niklaus, Noah Snavely, Oliver Wang.
CVPR 2021. [PDF] [Project]
DyNeRF: Neural 3D Video Synthesis.
Tianye Li, Mira Slavcheva, Michael Zollhoefer, Simon Green, Christoph Lassner, Changil Kim, Tanner Schmidt, Steven Lovegrove, Michael Goesele, Zhaoyang Lv.
arxiv 2021. [PDF] [Project]
Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans.
Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, Xiaowei Zhou.
arxiv 2021. [PDF] [Project]
Neural Radiance Flow for 4D View Synthesis and Video Processing.
Yilun Du, Yinan Zhang, Hong-Xing Yu, Joshua B. Tenenbaum, Jiajun Wu.
arxiv 2020. [PDF] [Project]
NeRF++: Analyzing and Improving Neural Radiance Fields.
Kai Zhang, Gernot Riegler, Noah Snavely, Vladlen Koltun.
arxiv 2020. [PDF] [Github]
X-Fields: Implicit Neural View-, Light- and Time-Image Interpolation.
MOJTABA BEMANA, KAROL MYSZKOWSKI, HANS-PETER SEIDEL, TOBIAS RITSCHEL.
Siggraph Asia 2020. [PDF]
NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections.
Ricardo Martin-Brualla, Noha Radwan, Mehdi S. M. Sajjadi, Jonathan T. Barron, Alexey Dosovitskiy, Daniel Duckworth.
CVPR 2021. [PDF] [Github]
pixelNeRF: Neural Radiance Fields from One or Few Images.
Alex Yu, Vickie Ye, Matthew Tancik, Angjoo Kanazawa.
arxiv 2020. [PDF] [Project] [Github]
GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis.
Katja Schwarz, Yiyi Liao, Michael Niemeyer, Andreas Geiger.
arxiv 2020. [PDF]
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis.
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, Ren Ng.
ECCV 2020. [PDF] [Project] [Gtihub-Tensorflow] [krrish94-PyTorch] [yenchenlin-PyTorch]
Layout-Guided Novel View Synthesis from a Single Indoor Panorama.
Jiale Xu, Jia Zheng, Yanyu Xu, Rui Tang, Shenghua Gao.
CVPR 2021. [PDF]
pixelNeRF: Neural Radiance Fields from One or Few Images.
Alex Yu, Vickie Ye, Matthew Tancik, Angjoo Kanazawa.
CVPR 2021. [PDF] [Project]
Generative View Synthesis: From Single-view Semantics to Novel-view Images.
Tewodros Habtegebrial, Varun Jampani, Orazio Gallo, Didier Stricker.
arxiv 2020. [PDF] [Project]
ShaRF: Shape-conditioned Radiance Fields from a Single View.
Konstantinos Rematas, Ricardo Martin-Brualla, Vittorio Ferrari.
arxiv 2021. [PDF] [Project]
Worldsheet: Wrapping the World in a 3D Sheet for View Synthesis from a Single Image.
Ronghang Hu, Deepak Pathak.
arxiv 2020. [PDF] [Project]
Unsupervised Novel View Synthesis from a Single Image.
Pierluigi Zama Ramirez, Alessio Tonioni, Federico Tombari.
arxiv 2021. [PDF]
Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Image.
Andrew Liu, Richard Tucker, Varun Jampani, Ameesh Makadia, Noah Snavely, Angjoo Kanazawa.
arxiv 2020. [PDF]
Geometry-Free View Synthesis: Transformers and no 3D Priors.
Robin Rombach, Patrick Esser, Björn Ommer.
arxiv 2021. [PDF]
Self-Supervised Visibility Learning for Novel View Synthesis.
Yujiao Shi, Hongdong Li, Xin Yu.
CVPR 2021. [PDF]
Nex: Real-time View Synthesis with Neural Basis Expansion.
S. Wizadwongsa, P. Phongthawee, J. Yenphraphai, S. Suwajanakorn.
CVPR 2021 (oral). [PDF] [Project]
Bowtie Networks: Generative Modeling for Joint Few-Shot Recognition and Novel-View Synthesis.
Zhipeng Bao, Yu-Xiong Wang, Martial Hebert.
ICLR 2021. [PDF]
IBRNet: Learning Multi-View Image-Based Rendering.
Qianqian Wang, Zhicheng Wang, Kyle Genova, Pratul Srinivasan, Howard Zhou, Jonathan T. Barron, Ricardo Martin-Brualla, Noah Snavely, Thomas Funkhouser.
CVPR 2021. [PDF] [Project]
SVS: Stable View Synthesis.
Gernot Riegler, Vladlen Koltun.
arxiv 2020. [PDF] [Github] [Video]
FVS: Free View Synthesis.
Gernot Riegler, Vladlen Koltun.
ECCV 2020. [PDF] [Code&Data]
Deep View Synthesis via Self-Consistent Generative Network.
Zhuoman Liu, Wei Jia, Ming Yang, Peiyao Luo, Yong Guo, Mingkui Tan.
arxiv 2021. [PDF]
Novel View Synthesis via Depth-guided Skip Connections.
Yuxin Hou, Arno Solin, Juho Kannala.
arxiv 2021. [PDF]
Vid2Actor: Free-viewpoint Animatable Person Synthesis from Video in the Wild.
Chung-Yi Weng, Brian Curless, Ira Kemelmacher-Shlizerman.
arxiv 2020. [PDF] [Project] [Video]
Object-Centric Neural Scene Rendering.
Michelle Guo, Alireza Fathi, Jiajun Wu, Thomas Funkhouser.
arxiv 2020. [PDF] [Project]
RGBD-Net: Predicting Color and Depth Images for Novel Views Synthesis.
Phong Nguyen, Animesh Karnewar, Lam Huynh, Esa Rahtu, Jiri Matas, Janne Heikkila.
arxiv 2020. [PDF]
Rotationally-Temporally Consistent Novel-View Synthesis of Human Performance Video.
Youngjoong Kwon, Stefano Petrangeli, Dahun Kim, Haoliang Wang, Eunbyung Park, Viswanathan Swaminathan, Henry Fuchs.
ECCV 2020. [pdf] [Supplement]
A Recurrent Transformer Network for Novel View Action Synthesis.
Kara Marie Schatz, Erik Quintanilla, Shruti Vyas, Yogesh S Rawat.
ECCV 2020. [pdf] [Supplement]
MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images.
Benjamin Attal, Selena Ling, Aaron Gokaslan, Christian Richardt, James Tompkin.
ECCV 2020. [pdf] [Supplement]
Novel View Synthesis on Unpaired Data by Conditional Deformable Variational Auto-Encoder.
Mingyu Yin, Li Sun, Qingli Li.
ECCV 2020. [pdf] [Supplement]
Deep Multi Depth Panoramas for View Synthesis.
Kai-En Lin, Zexiang Xu, Ben Mildenhall, Pratul P. Srinivasan, Yannick Hold-Geoffroy, Stephen DiVerdi, Qi Sun, Kalyan Sunkavalli, Ravi Ramamoorthi.
ECCV 2020. [pdf] [Supplement]
Semantic View Synthesis.
Hsin-Ping Huang, Hung-Yu Tseng, Hsin-Ying Lee, Jia-Bin Huang.
ECCV 2020. [PDF] [Project]
Deep Multi Depth Panoramas for View Synthesis.
Kai-En Lin, Zexiang Xu, Ben Mildenhall, Pratul P. Srinivasan, Yannick Hold-Geoffroy, Stephen DiVerdi, Qi Sun, Kalyan Sunkavalli, Ravi Ramamoorthi.
ECCV 2020. [PDF]
Unsupervised Continuous Object Representation Networks for Novel View Synthesis.
Nicolai Häni, Selim Engin, Jun-Jee Chao, Volkan Isler.
arxiv 2020. [PDF]
AUTO3D: Novel View Synthesis Through Unsupervisely Learned Variational Viewpoint and Global 3D Representation.
Xiaofeng Liu, Tong Che, Yiqun Lu, Chao Yang, Site Li, Jane You.
ECCV 2020. [PDF]
A Free Viewpoint Portrait Generator with Dynamic Styling.
Anpei Chen, Ruiyang Liu, Ling Xie, Jingyi Yu.
arxiv 2020. [PDF]
Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Camera.
Jae Shin Yoon, Kihwan Kim, Orazio Gallo, Hyun Soo Park, Jan Kautz.
CVPR 2020. [PDF]
Neural Point-Based Graphics.
Kara-Ali Aliev, Artem Sevastopolsky, Maria Kolos, Dmitry Ulyanov, Victor Lempitsky.
arxiv 2020. [PDF] [Project]
LLFF: Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines.
Ben Mildenhall, Pratul Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng, Abhishek Kar.
SIGGRAPH 2019. [PDF] [Project] [Github]
Neural Volumes: Learning Dynamic Renderable Volumes from Images.
Stephen Lombardi, Tomas Simon, Jason Saragih, Gabriel Schwartz, Andreas Lehrmann, Yaser Sheikh.
SIGGRAPH 2019. [PDF] [Github]
Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis.
Jogendra Nath Kundu, Siddharth Seth, Varun Jampani, Mugalodi Rakesh, R. Venkatesh Babu, Anirban Chakraborty.
CVPR 2020. [PDF]
IGNOR: Image-guided Neural Object Rendering.
Justus Thies, Michael Zollhöfer, Christian Theobalt, Marc Stamminger, Matthias Nießner.
ICLR 2020. arxiv, 26 Nov 2018 (15 Jan 2020). [PDF] [Project]
Monocular Neural Image Based Rendering with Continuous View Control.
Xu Chen, Jie Song, Otmar Hilliges.
ICCV 2019. [PDF]
Extreme View Synthesis.
Inchang Choi, Orazio Gallo, Alejandro Troccoli, Min H. Kim, Jan Kautz.
ICCV 2019. [PDF]
Transformable Bottleneck Networks.
Kyle Olszewski, Sergey Tulyakov, Oliver Woodford, Hao Li, Linjie Luo.
ICCV 2019. [PDF]
View Independent Generative Adversarial Network for Novel View Synthesis.
Xiaogang Xu, Ying-Cong Chen, Jiaya Jia.
ICCV 2019. [PDF]
Total Relighting: Learning to Relight Portraits for Background Replacement.
Rohit Pandey, Sergio Orts-Escolano, Chloe LeGendre, Christian Haene, Sofien Bouaziz, Christoph Rhemann, Paul Debevec, and Sean Fanello.
SIGGRAPH 2021. [PDF] [Project]
DSRN: an Efficient Deep Network for Image Relighting.
Sourya Dipta Das, Nisarg A. Shah, Saikat Dutta, Himanshu Kumar.
arxiv 2021. [PDF]
Relightable 3D Head Portraits from a Smartphone Video.
Artem Sevastopolsky, Savva Ignatiev, Gonzalo Ferrer, Evgeny Burnaev, Victor Lempitsky.
arxiv 2020. [PDF] [Project] [Video]
NeRD: Neural Reflectance Decomposition from Image Collections.
Mark Boss, Raphael Braun, Varun Jampani, Jonathan T. Barron, Ce Liu, Hendrik P. A. Lensch.
arxiv 2020. [PDF] [Project] [Github]
Deferred Neural Lighting: Free-viewpoint Relighting from Unstructured Photographs.
Duan Gao, Guojun Chen, Yue Dong, Pieter Peers, Kun Xu, Xin Tong.
ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2020)
Cross-Camera Convolutional Color Constancy.
Mahmoud Afifi, Jonathan T. Barron, Chloe LeGendre, Yun-Ta Tsai, Francois Bleibel.
arxiv 2020. [PDF]
Towards Geometry Guided Neural Relighting with Flash Photography.
Di Qiu, Jin Zeng, Zhanghan Ke, Wenxiu Sun, Chengxi Yang.
arxiv 2020. [PDF]
Light Stage Super-Resolution: Continuous High-Frequency Relighting.
Tiancheng Sun, Zexiang Xu, Xiuming Zhang, Sean Fanello, Christoph Rhemann, Paul Debevec, Yun-Ta Tsai, Jonathan T. Barron, Ravi Ramamoorthi.
Siggraph Asia 2020. [PDF]
Neural Light Transport for Relighting and View Synthesis.
Xiuming Zhang, Sean Fanello, Yun-Ta Tsai, Tiancheng Sun, Tianfan Xue, Rohit Pandey, Sergio Orts-Escolano, Philip Davidson, Christoph Rhemann, Paul Debevec, Jonathan T. Barron, Ravi Ramamoorthi, William T. Freeman.
arxiv 2020. [PDF] [Project]
NuI-Go: Recursive Non-Local Encoder-Decoder Network for Retinal Image Non-Uniform Illumination Removal.
Chongyi Li, Huazhu Fu, Runmin Cong, Zechao Li, Qianqian Xu.
ACM MM 2020. [PDF]
Learning to Factorize and Relight a City.
Andrew Liu, Shiry Ginosar, Tinghui Zhou, Alexei A. Efros, Noah Snavely.
ECCV 2020. [PDF] [Project]
Object-based Illumination Estimation with Rendering-aware Neural Networks.
Xin Wei, Guojun Chen, Yue Dong, Stephen Lin, Xin Tong.
ECCV 2200. [PDF]
Learning Illumination from Diverse Portraits.
Chloe LeGendre, Wan-Chun Ma, Rohit Pandey, Sean Fanello, Christoph Rhemann, Jason Dourgarian, Jay Busch, Paul Debevec.
arxiv 2020. [PDF]
Enlighten Me: Importance of Brightness and Shadow for Character Emotion and Appeal.
Pisut Wisessing, Katja Zibrek, Douglas W. Cunningham, John Dingliana, Rachel McDonnell.
TOG 2020. [PDF]
Portrait Shadow Manipulation.
Xuaner Cecilia Zhang, J onathan T. Barron, Yun-Ta Tsai, Rohit Pandey, Xiuming Zhang, Ren Ng, David E. Jacobs.
SIGGRAPH 2020. [PDF] [Project]
Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illumination.
Pratul P. Srinivasan, Ben Mildenhall, Matthew Tancik, Jonathan T. Barron, Richard Tucker, Noah Snavely.
CVPR 2020. [PDF] GitHub] [Project]
Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Images.
Sai Bi, Zexiang Xu, Kalyan Sunkavalli, David Kriegman, Ravi Ramamoorthi.
CVPR 2020. [PDF]
Neural Illumination: Lighting Prediction for Indoor Environments.
Shuran Song and Thomas Funkhouser.
CVPR 2019. [PDF] [Project]
Learning to Shade Hand-drawn Sketches.
Qingyuan Zheng, Zhuoru Li, Adam Bargteil.
CVPR 2020. [PDF]
Generating Digital Painting Lighting Effects via RGB-space Geometry.
Lvmin Zhang, Edgar Simo-Serra, Yi Ji, and Chunping Liu.
SIGGRAPH 2020 (TOG 2020).
[Priject]
[Github]
Deep Single-Image Portrait Relighting.
Hao Zhou, Sunil Hadap, Kalyan Sunkavalli, David W. Jacobs.
ICCV 2019. [PDF] [Github] [Project] [DPR Dataset]
Single Image Portrait Relighting.
Tiancheng Sun, Jonathan T. Barron, Yun-Ta Tsai, Zexiang Xu, Xueming Yu, Graham Fyffe, Christoph Rhemann, Jay Busch, Paul Debevec, Ravi Ramamoorthi.
SIGGRAPH 2019. [PDF]
Multi-view Relighting using a Geometry-Aware Network.
Julien Philip, Michael Gharbi, Tinghui Zhou, Alexei (Alyosha) Efros, George Drettakis.
SIGGRAPH 2019. [PDF]
Illumination Decomposition for Photograph with Multiple Light Sources.
Ling Zhang, Qingan Yan, Zheng Liu, Hua Zou, Chunxia Xiao.
TIP 2017. [PDF] [Github]
Learning to Predict Indoor Illumination from a Single Image.
Marc-André Gardner, Kalyan Sunkavalli, Ersin Yumer, Xiaohui Shen, Emiliano Gambaretto, Christian Gagné, and Jean-François Lalonde.
ACM Transactions on Graphics (SIGGRAPH Asia), 2017. [PDF] [Dataset] [Homepage]
Deep Parametric Indoor Lighting Estimation.
Marc-André Gardner, Yannick Hold-Geoffroy, Kalyan Sunkavalli, Christian Gagné, and Jean-François Lalonde.
ICCV 2019. [PDF] [Supplementary material] [Laval Indoor HDR Database and Depth] [Project]
Fast Spatially-Varying Indoor Lighting Estimation.
Mathieu Garon, Kalyan Sunkavalli, Sunil Hadap, Nathan Carr, Jean-François Lalonde.
CVPR 2019. [PDF] [Supplementary material] [Project] [Lavel Indoor Spatially Varying HDR Dataset / 79 HDR Light Probes]
GLoSH: Global-Local Spherical Harmonics for Intrinsic Image Decomposition.
Hao Zhou, Xiang Yu, David W Jacobs.
ICCV 2019. [PDF] [Supplement] [Poster] [Spherical Harmonic Tools]
SfSNet: Learning Shape, Reflectance and llluminance of Faces in the Wild.
Soumyadip Sengupta, Angjoo Kanazawa, Carlos D. Castillo, David W. Jacobs.
CVPR 2018. [Project] [PDF] [Github]
Occlusion-aware 3D Morphable Models and an Illumination Prior for Face Image Analysis.
Bernhard Egger, Sandro Schoenborn, Andreas Schneider, Adam Kortylewski, Andreas Morel-Forster, Clemens Blumer and Thomas Vetter.
IJCV 2018. [BIP Dataset] [PDF]
DNR: A Neural Rendering Framework for Free-Viewpoint Relighting.
Zhang Chen, Anpei Chen, Guli Zhang, Chengyuan Wang, Yu Ji, Kiriakos N. Kutulakos, Jingyi Yu.
arxiv 2019. [PDF]
Deep Illumination: Approximating Dynamic Global Illumination with Generative Adversarial Network.
Manu Mathew Thomas, Angus G. Forbes.
arxiv 2017. [PDF]
Deep Shading: Convolutional Neural Networks for Screen-Space Shading.
Oliver Nalbach, Elena Arabadzhiyska, Dushyant Mehta, Hans-Peter Seidel, Tobias Ritschel.
arixv 2016. [PDF]
One Shot Radiance: Global Illumination Using Convolutional Autoencoders.
Giulio Jiang, Bernhard Kainz.
arxiv 2019. [PDF]
MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement.
Alexander Richard, Michael Zollhoefer, Yandong Wen, Fernando de la Torre, Yaser Sheikh.
arxiv 2021. [PDF] [Video]
AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis.
Yudong Guo, Keyu Chen, Sen Liang, Yongjin Liu, Hujun Bao, Juyong Zhang.
arxiv 2021. [PDF] [Project] [Video]
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation.
Hang Zhou, Yasheng Sun, Wayne Wu, Chen Change Loy, Xiaogang Wang, Ziwei Liu.
CVPR 2021. [PDF] [Project]
Everything's Talkin': Pareidolia Face Reenactment.
Linsen Song, Wayne Wu, Chaoyou Fu, Chen Qian, Chen Change Loy, Ran He.
CVPR 2021. [PDF] [Project] [Github]
One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing.
Ting-Chun Wang, Arun Mallya, Ming-Yu Liu.
CVPR 2021 (oral). [PDF] [Project]
MEAD: A Large-scale Audio-visual Dataset for Emotional Talking Face Generation.
Kaisiyuan Wang, Qianyi Wu, Linsen Song, Zhuoqian Yang, Wayne Wu, Chen Qian, Ran He, Yu Qiao, Chen Change Loy.
ECCV 2020. [PDF] [Project] [Github]
HeadGAN: Video-and-Audio-Driven Talking Head Synthesis.
Michail Christos Doukas, Stefanos Zafeiriou, Viktoriia Sharmanska.
arxiv 2020. [PDF]
DECA: Learning an Animatable Detailed 3D Face Model from In-The-Wild Images.
Yao Feng, Haiwen Feng, Michael J. Black, Timo Bolkart.
arxiv 2020. [PDF] [Github]
Large-Scale Multilingual Audio Visual Dubbing.
Yi Yang, Brendan Shillingford, Yannis Assael, Miaosen Wang, Wendi Liu, Yutian Chen, Yu Zhang, Eren Sezener, Luis C. Cobo, Misha Denil, Yusuf Aytar, Nando de Freitas.
arxiv 2020. [PDF] [Video]
EBT: Everybody's Talkin': Let Me Talk as You Want.
Linsen Song, Wayne Wu, Chen Qian, Ran He, Chen Change Loy.
arxiv 2020. [PDF] [Project]
Audio-driven Talking Face Video Generation with Learning-based Personalized Head Pose.
Ran Yi, Zipeng Ye, Juyong Zhang, Hujun Bao, Yong-Jin Liu.
arxiv 2020. [PDF] [Github]
Speech Driven Talking Face Generation from a Single Image and an Emotion Condition.
Sefik Emre Eskimez, You Zhang, Zhiyao Duan.
arxiv 2020. [PDF]
MakeItTalk: Speaker-Aware Talking-Head Animation.
Yang Zhou, Xintong Han, Eli Shechtman, Jose Echevarria, Evangelos Kalogerakis, Dingzeyu Li.
SIGGRAPH Asia 2020. [PDF] [Github]
FLNet: Landmark Driven Fetching and Learning Network for Faithful Talking Facial Animation Synthesis.
Kuangxiao Gu, Yuqian Zhou, Thomas Huang.
AAAI 2020. [PDF] [GitHub]
Text-based Editing of Talking-head Video.
Ohad Fried, Ayush Tewari, Michael Zollhöfer, Adam Finkelstein, Eli Shechtman, Dan B Goldman Kyle Genova, Zeyu Jin, Christian Theobalt, Maneesh Agrawala.
SIGGRAPH 2019. [PDF] [Project]
ATVGnet: Hierarchical Cross-Modal Talking Face Generationwith Dynamic Pixel-Wise Loss.
Lele Chen, Ross K. Maddox, Zhiyao Duan, Chenliang Xu.
CVPR 2019. [PDF] [Github]
Talking Face Generation by Adversarially Disentangled Audio-Visual Representation.
Hang Zhou, Yu Liu, Ziwei Liu, Ping Luo, Xiaogang Wang.
AAAI 2019. [PDF] [Github] [Project]
SimPoE: Simulated Character Control for 3D Human Pose Estimation.
Ye Yuan, Shih-En Wei, Tomas Simon, Kris Kitani, Jason Saragih.
CVPR 2021. [PDF] [Project]
Few-Shot Human Motion Transfer by Personalized Geometry and Texture Modeling.
Zhichao Huang, Xintong Han, Jia Xu, Tong Zhang.
CVPR 2021. [PDF]
Single-Shot Freestyle Dance Reenactment.
Oren Nuriel, Oron Ashual, Lior Wolf.
CVPR 2021. [PDF]
Local Motion Phases for Learning Multi-Contact Character Movements.
Sebastian Starke, Yiwei Zhao, Taku Komura, Kazi Zaman.
Siggraph 2020. [PDF] [AI4Animation]
Neural State Machine for Character-Scene Interactions.
Sebastian Starke, He Zhang, Taku Komura, Jun Saito.
Siggraph Asia 2019. [PDF] [Code & Data]
High-Fidelity Neural Human Motion Transfer from Monocular Video.
Moritz Kappel, Vladislav Golyanik, Mohamed Elgharib, Jann-Ole Henningson, Hans-Peter Seidel, Susana Castillo, Christian Theobalt, Marcus Magnor.
arxiv 2020. [PDF] [Project]
Image Animation with Perturbed Masks.
Yoav Shalev, Lior Wolf.
arxiv 2020. [PDF] [Github]
Photorealistic Audio-driven Video Portraits.
Xin Wen, Miao Wang, Christian Richardt, Ze-Yin Chen, Shi-Min Hu.
TVCG (ISMAR 2020). [PDF] [Github] [Project]
FACEGAN: Facial Attribute Controllable rEenactment GAN.
Soumya Tripathy, Juho Kannala, Esa Rahtu.
WACV 2021. [PDF]
Generative Adversarial Networks in Human Emotion Synthesis:A Review.
Noushin Hajarolasvadi, Miguel Arjona Ramírez, Hasan Demirel.
arxiv 2020. [PDF]
Mesh Guided One-shot Face Reenactment using Graph Convolutional Networks.
Guangming Yao, Yi Yuan, Tianjia Shao, Kun Zhou.
ACM MM 2020. [PDF]
Learning to Generate Diverse Dance Motions with Transformer.
Jiaman Li, Yihang Yin, Hang Chu, Yi Zhou, Tingwu Wang, Sanja Fidler, Hao Li.
arxiv 2020. [PDF]
Action2Motion: Conditioned Generation of 3D Human Motions.
Chuan Guo, Xinxin Zuo, Sen Wang, Shihao Zou, Qingyao Sun, Annan Deng, Minglun Gong, Li Cheng.
ACM MultiMedia 2020. [PDF]
ReenactNet: Real-time Full Head Reenactment.
Mohammad Rami Koujan, Michail Christos Doukas, Anastasios Roussos, Stefanos Zafeiriou.
FG 2020. [PDF]
FaR-GAN for One-Shot Face Reenactment.
Hanxiang Hao, Sriram Baireddy, Amy R. Reibman, Edward J. Delp.
AI for content creation workshop at CVPR 2020. [PDF]
Skeleton-Aware Networks for Deep Motion Retargeting.
Kfir Aberman, Peizhuo Li, Dani Lischinski, Olga Sorkine-Hornung, Daniel Cohen-Or, Baoquan Chen.
SIGGRAPH 2020. [Github] [Project]
Unpaired Motion Style Transfer from Video to Animation.
Kfir Aberman, Yijia Weng, Dani Lischinski, Daniel Cohen-Or, Baoquan Chen.
SIGGRAPH 2020. [Github] [Project]
LipGAN: Towards Automatic Face-to-Face Translation.
Prajwal K R, Rudrabha Mukhopadhyay, Jerin Philip, Abhishek Jha, Vinay Namboodiri, C.V. Jawahar.
ACM MM 2019. [PDF] [Github]
One-Shot Identity-Preserving Portrait Reenactment.
Sitao Xiang, Yuming Gu, Pengda Xiang, Mingming He, Koki Nagano, Haiwei Chen, Hao Li.
arriv, 2020. [PDF]
Neural Head Reenactment with Latent Pose Descriptors.
Egor Burkov, Igor Pasechnik, Artur Grigorev, Victor Lempitsky.
CVPR 2020. [PDF]
Neural Human Video Rendering by Learning Dynamic Textures and Rendering-to-Video Translation.
Lingjie Liu, Weipeng Xu, Marc Habermann, Michael Zollhoefer, Florian Bernard, Hyeongwoo Kim, Wenping Wang, Christian Theobalt.
arriv, 2020. [PDF]
StyleRig: Rigging StyleGAN for 3D Control over Portrait Images.
A. Tewari, M. Elgharib, G. Bharaj, F. Bernard, H-P. Seidel, P. Perez, M. Zollhöfer, C.Theobalt.
CVPR 2020. [PDF] [Project]
TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting.
Zhuoqian Yang, Wentao Zhu, Wayne Wu, Chen Qian, Qiang Zhou, Bolei Zhou, Chen Change Loy.
CVPR 2020. [PDF] [Github] [Project]
Human Motion Transfer from Poses in the Wild.
Jian Ren, Menglei Chai, Sergey Tulyakov, Chen Fang, Xiaohui Shen, Jianchao Yang.
arxiv 2020. [PDF]
First Order Motion Model for Image Animation.
Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, Nicu Sebe.
NeurIPS 2019. [PDF] [Project] [Github]
Neural Human Video Rendering: Joint Learning of Dynamic Textures and Rendering-to-Video Translation.
Lingjie Liu, Weipeng Xu, Marc Habermann, Michael Zollhoefer, Florian Bernard, Hyeongwoo Kim, Wenping Wang, Christian Theobalt.
arxiv 2020. [PDF]
Deferred Neural Rendering: Image Synthesis using Neural.
Justus Thies, Michael Zollhöfer, Matthias Nießner.
SIGGRAPH 2019. [PDF]
LOGAN: Unpaired Shape Transform in Latent Overcomplete Space.
Kangxue Yin, Zhiqin Chen, Hui Huang, Daniel Cohen-Or, Hao Zhang.
SIGGRAPH Asia, 2019. [PDF]
Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis.
Wen Liu, Zhixin Piao, Jie Min, Wenhan Luo, Lin Ma, Shenghua Gao.
ICCV 2019. [PDF] [HomePage] [Github].
Learning Character-Agnostic Motion for Motion Retargeting in 2D.
Kfir Aberman, Rundi Wu, Dani Lischinski, Baoquan Chen, Daniel Cohen-Or.
SIGGRAPH 2019. [PDF] [Github] [Project]
Progressive Pose Attention Transfer for Person Image Generation.
Zhen Zhu, Tengteng Huang, Baoguang Shi, Miao Yu, Bofei Wang, Xiang Bai.
CVPR 2019. [Project] [PDF]
Textured Neural Avatars.
Aliaksandra Shysheya, Egor Zakharov, Kara-Ali Aliev, Renat Bashirov, Egor Burkov, Karim Iskakov, Aleksei Ivakhnenko, Yury Malkov, Igor Pasechnik, Dmitry Ulyanov, Alexander Vakhitov, Victor Lempitsky.
CVPR 2019 (oral). [PDF] [Project]
Appearance Composing GAN: A General Method for Appearance-Controllable Human Video Motion Transfer.
Dongxu Wei, Haibin Shen, Kejie Huang.
arxiv 2019. [PDF]
This work is licensed under a Creative Commons Attribution 4.0 International License.
