SOL-NeRF: Sunlight Modeling for Outdoor Scene Decomposition and Relighting

Jia Mu Sun, Tong Wu, Yong Liang Yang, Yu Kun Lai, Lin Gao

Research output: Chapter or section in a book/report/conference proceedingChapter in a published conference proceeding

1 Citation (SciVal)
266 Downloads (Pure)

Abstract

Outdoor scenes often involve large-scale geometry and complex unknown lighting conditions, making it difficult to decompose them into geometry, reflectance and illumination. Recently researchers made attempts to decompose outdoor scenes using Neural Radiance Fields (NeRF) and learning-based lighting and shadow representations. However, diverse lighting conditions and shadows in outdoor scenes are challenging for learning-based models. Moreover, existing methods may produce rough geometry and normal reconstruction and introduce notable shading artifacts when the scene is rendered under a novel illumination. To solve the above problems, we propose SOL-NeRF to decompose outdoor scenes with the help of a hybrid lighting representation and a signed distance field geometry reconstruction. We use a single Spherical Gaussian (SG) lobe to approximate the sun lighting, and a first-order Spherical Harmonic (SH) mixture to resemble the sky lighting. This hybrid representation is specifically designed for outdoor settings, and compactly models the outdoor lighting, ensuring robustness and efficiency. The shadow of the direct sun lighting can be obtained by casting the ray against the mesh extracted from the signed distance field, and the remaining shadow can be approximated by Ambient Occlusion (AO). Additionally, sun lighting color prior and a relaxed Manhattan-world assumption can be further applied to boost decomposition and relighting performance. When changing the lighting condition, our method can produce consistent relighting results with correct shadow effects. Experiments conducted on our hybrid lighting scheme and the entire decomposition pipeline show that our method achieves better reconstruction, decomposition, and relighting performance compared to previous methods both quantitatively and qualitatively.

Original languageEnglish
Title of host publicationProceedings - SIGGRAPH Asia 2023 Conference Papers, SA 2023
EditorsStephen N. Spencer
Place of PublicationNew York, U. S. A.
PublisherAssociation for Computing Machinery
Pages1-11
ISBN (Electronic)9798400703157
DOIs
Publication statusPublished - 11 Dec 2023
Event2023 SIGGRAPH Asia 2023 Conference Papers, SA 2023 - Sydney, Australia
Duration: 12 Dec 202315 Dec 2023

Publication series

NameProceedings - SIGGRAPH Asia 2023 Conference Papers, SA 2023

Conference

Conference2023 SIGGRAPH Asia 2023 Conference Papers, SA 2023
Country/TerritoryAustralia
CitySydney
Period12/12/2315/12/23

Funding

This work was supported by grants from the National Natural Science Foundation of China (No. 62061136007), the Beijing Municipal Natural Science Foundation for Distinguished Young Scholars (No. JQ21013), the Royal Society Newton Advanced Fellowship (No. NAF\R2\192151), RCUK grant CAMERA (EP/M023281/1, EP/T022523/1), and a gift from Adobe.

FundersFunder number
Beijing Municipal Natural Science Foundation for Distinguished Young ScholarsJQ21013
Royal Society NewtonNAF\R2\192151
Parliamentary Office of Science and Technology EP/T022523/1, EP/M023281/1
National Natural Science Foundation of China62061136007

    Keywords

    • inverse rendering
    • neural radiance fields
    • outdoor scene reconstruction

    ASJC Scopus subject areas

    • Computer Graphics and Computer-Aided Design
    • Software
    • Computer Vision and Pattern Recognition

    Fingerprint

    Dive into the research topics of 'SOL-NeRF: Sunlight Modeling for Outdoor Scene Decomposition and Relighting'. Together they form a unique fingerprint.

    Cite this