in this video we present Smurf a streamable memory efficient Radiance field representation for real-time exploration of large baces neural Radiance Fields have demonstrated near photorealistic reconstruction quality on large unbounded 3D scenes nevertheless High Fidelity methods such as zip Nerf are still too compute intensive for real-time rendering our method Smurf produces renders that are nearly IND distinguishable from zip Nerf while capturing detail at the centimeter level at the same time our method's compute and memory requirements are on par with existing real-time methods such as murf as a result our method is readily Deployable to low resource
devices including consumer grade smartphones and laptops via the web browser while rendering at 60 frames per second we achieve this by streaming in the appropriate part of the scene representation based on camera position our method Builds on three contributions the first is a realtime representation for smaller unbounded 3D scenes our representation Builds on murf while significantly improving visual Fidelity and geometric reconstruction quality our second contribution is a novel distillation training regime for radiance Fields rather than learning from posed photos our method distills geometric and photometric information directly from a slower High Fidelity teacher our method
inherits its teachers inductive biases and surpasses the quality of other real-time view synthesis methods such as 3D gausian splatting which exhibits artifacts in under observed parts of the scene these artifacts are Amplified in larger scenes where a thorough capture is prohibitively time intensive and representation size scale sces with scene size our method sidesteps these issues by partitioning the scene into sub regions based on camera position each sub region is assigned a trif submodel all of which are supervised by a common teacher at any point in time a single submodel suffices to render a Target view
strongly bounding compute and memory requirements without reducing visual Fidelity compared to state-of-the-art synthesis methods like 3D gou and splatting our method demonstrates fewer floaters and better representation of view dependent effects such as specular highlights and Reflections our method reliably reconstructs scenes up to 300 m s at a spatial resolution as small as 3.5 mm cubed unlike 3D gaou and splatting our method is compatible with natural lens models without undistortion enabling the use of fishey cameras during training our method exhibits fewer artifacts more convincing textures and geometry and captures nearby and Far Away content with equal
fidelity most importantly our method is readily Deployable to Consumer devices such as smartphones and laptops via the web browser we maintain modest memory requirements by streaming in the most appropriate submodel parameters based on the user's current position in the scene at the same time the user can continue to navigate the scene without interruption using the currently loaded submodel renders and interactive demos of these and more scenes are waiting for you at our project website thank you for listening