Projects per year
Abstract
Outdoor scenes often involve large-scale geometry and complex unknown lighting conditions, making it difficult to decompose them into geometry, reflectance and illumination. Recently researchers made attempts to decompose outdoor scenes using Neural Radiance Fields (NeRF) and learning-based lighting and shadow representations. However, diverse lighting conditions and shadows in outdoor scenes are challenging for learning-based models. Moreover, existing methods may produce rough geometry and normal reconstruction and introduce notable shading artifacts when the scene is rendered under a novel illumination. To solve the above problems, we propose SOL-NeRF to decompose outdoor scenes with the help of a hybrid lighting representation and a signed distance field geometry reconstruction. We use a single Spherical Gaussian (SG) lobe to approximate the sun lighting, and a first-order Spherical Harmonic (SH) mixture to resemble the sky lighting. This hybrid representation is specifically designed for outdoor settings, and compactly models the outdoor lighting, ensuring robustness and efficiency. The shadow of the direct sun lighting can be obtained by casting the ray against the mesh extracted from the signed distance field, and the remaining shadow can be approximated by Ambient Occlusion (AO). Additionally, sun lighting color prior and a relaxed Manhattan-world assumption can be further applied to boost decomposition and relighting performance. When changing the lighting condition, our method can produce consistent relighting results with correct shadow effects. Experiments conducted on our hybrid lighting scheme and the entire decomposition pipeline show that our method achieves better reconstruction, decomposition, and relighting performance compared to previous methods both quantitatively and qualitatively.
Original language | English |
---|---|
Title of host publication | Proceedings - SIGGRAPH Asia 2023 Conference Papers, SA 2023 |
Editors | Stephen N. Spencer |
Place of Publication | New York, U. S. A. |
Publisher | Association for Computing Machinery |
Pages | 1-11 |
ISBN (Electronic) | 9798400703157 |
DOIs | |
Publication status | Published - 11 Dec 2023 |
Event | 2023 SIGGRAPH Asia 2023 Conference Papers, SA 2023 - Sydney, Australia Duration: 12 Dec 2023 → 15 Dec 2023 |
Publication series
Name | Proceedings - SIGGRAPH Asia 2023 Conference Papers, SA 2023 |
---|
Conference
Conference | 2023 SIGGRAPH Asia 2023 Conference Papers, SA 2023 |
---|---|
Country/Territory | Australia |
City | Sydney |
Period | 12/12/23 → 15/12/23 |
Funding
This work was supported by grants from the National Natural Science Foundation of China (No. 62061136007), the Beijing Municipal Natural Science Foundation for Distinguished Young Scholars (No. JQ21013), the Royal Society Newton Advanced Fellowship (No. NAF\R2\192151), RCUK grant CAMERA (EP/M023281/1, EP/T022523/1), and a gift from Adobe.
Funders | Funder number |
---|---|
Beijing Municipal Natural Science Foundation for Distinguished Young Scholars | JQ21013 |
Royal Society Newton | NAF\R2\192151 |
Parliamentary Office of Science and Technology | EP/T022523/1, EP/M023281/1 |
National Natural Science Foundation of China | 62061136007 |
Keywords
- inverse rendering
- neural radiance fields
- outdoor scene reconstruction
ASJC Scopus subject areas
- Computer Graphics and Computer-Aided Design
- Software
- Computer Vision and Pattern Recognition
Fingerprint
Dive into the research topics of 'SOL-NeRF: Sunlight Modeling for Outdoor Scene Decomposition and Relighting'. Together they form a unique fingerprint.Projects
- 1 Active
-
Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA) - 2.0
Campbell, N. (PI), Cosker, D. (PI), Bilzon, J. (CoI), Campbell, N. (CoI), Cazzola, D. (CoI), Colyer, S. (CoI), Cosker, D. (CoI), Lutteroth, C. (CoI), McGuigan, P. (CoI), O'Neill, E. (CoI), Petrini, K. (CoI), Proulx, M. (CoI) & Yang, Y. (CoI)
Engineering and Physical Sciences Research Council
1/11/20 → 31/10/25
Project: Research council