we present neural radiance fields or nerf a new method for representing complex scenes that achieves state-of-the-art results review synthesis given a set of input images of a scene which in this visualization is a Lego bulldozer we optimize a volumetric representation of the scene as a vector-valued function which is defined for any continuous 5d coordinate consisting of a location and view direction we parameterize this scene representation as a fully connected deep network that takes each single 5d coordinate and outputs the corresponding volume density and view dependent emitted RGB radiance at that location we can then use
techniques from volume rendering to composite these values along a camera array to render any pixel this rendering is fully differentiable so we are able to optimize the scene representation by minimizing the error of rendering all camera rates from a collection of standard RGB images we first show comparisons to top performing methods on synthetically rendered data scene representation networks implicitly represent a scene using a fully connected neural network as some runs have issues of multi-view consistency and are unable to represent high frequency details local light field fusion uses a pre trained network to promote each input
view to a high resolution 3d volume inconsistencies between adjacent volumes causes flickering artifacts neural volumes in coda seen as 128 cubed voxel grid and use a work field to better allocate these limited samples however they're still unable to represent high resolution content such as the fine details in the ship's texture and rigging we see the same trend in all of our synthetic test scenes our method both qualitatively and quantitatively outperforms the other methods here we show the results of our method on the six other objects in a realistically rendered dataset with complex geometry and non
Lambert materials next we will show comparisons on real scenes captured with a forward-facing phone camera SSRN uses a recurrent network to rain march through encoded scenes resulting in inconsistent appearance between rendered views lff blends between multiple renderings with different limited fields of view resulting in flickering artifacts along image borders here only our method can represent fine details with complex occlusion effects such as the ribs in this fossilized t-rex skeleton now let's take a look at some additional real results we can visualize the view dependent effects captured by our representation such as the semi-transparent appearance the
olive oil bottle and speckle areas on the stove reflections on this room's television and specularity x' on the conference table and sharp reflections and specularity x' on this red car's body and windshield our representation captures high quality scene geometry such as the details on this red flower and complex occlusions in the Christmas tree leaves and ornaments this geometry is precise enough to be used for additional graphics applications such as virtual object insertion with convincing occlusion effects here we visualize additional results across a wide variety of real-world scenes each captured with only 20 to 50 input
images our neural radiance field representations are able to render photo realistic novel views of entire scenes with fine geometry details and realistic view dependent reflectance effects we're able to capture all of this visual complexity in just the weights of a simple fully connected Network we also show preliminary results for 360 degree capture of real scenes