<story><title>SMERF: Streamable Memory Efficient Radiance Fields</title><url>https://smerf-3d.github.io/</url><text>We built SMERF, a new way for exploring NeRFs in real-time in your web browser. Try it out yourself!&lt;p&gt;Over the last few months, my collaborators and I have put together a new, real-time method that makes NeRF models accessible from smartphones, laptops, and low-power desktops, and we think we’ve done a pretty stellar job! SMERF, as we like to call it, distills a large, high quality NeRF into a real-time, streaming-ready representation that’s easily deployed to devices as small as a smartphone via the web browser.&lt;p&gt;On top of that, our models look great! Compared to other real-time methods, SMERF has higher accuracy than ever before. On large multi-room scenes, SMERF renders are nearly indistinguishable from state-of-the-art offline models like Zip-NeRF and a solid leap ahead of other approaches.&lt;p&gt;The best part: you can try it out yourself! Check out our project website for demos and more.&lt;p&gt;If you have any questions or feedback, don’t hesitate to reach out by email (smerf@google.com) or Twitter (@duck).</text></story><parent_chain><item><author>nightpool</author><text>The refigerator in the NYC scene has a very slick specular lighting effect based on the angle you&amp;#x27;re viewing it from, and if you go &amp;quot;into&amp;quot; the fridge you can see it&amp;#x27;s actually generating a whole 3d scene with blurry grey and white colors that turn out to precisely mimic the effects of the light from the windows bouncing off the metal, and you can look &amp;quot;out&amp;quot; from the fridge into the rest of the room. Same as the full-length mirror in the bedroom in the same scene—there&amp;#x27;s a whole virtual &amp;quot;mirror room&amp;quot; that&amp;#x27;s been built out behind the mirror to give the illusion of depth as you look through it. Very cool and unique consequence of the technology</text></item><item><author>barrkel</author><text>The mirror on the wall of the bathroom in the Berlin location looks through to the kitchen in the next room. I guess the depth gauging algorithm uses parallax, and mirrors confuse it, seeming like windows. The kitchen has a blob of blurriness as the rear of the mirror intrudes into kitchen, but you can see through the blurriness to either room.&lt;p&gt;The effect is a bit spooky. I felt like a ghost going through walls.</text></item></parent_chain><comment><author>pavlov</author><text>Wow, thanks for the tip. Fridge reflection world is so cool. Feels like something David Lynch might dream up.&lt;p&gt;A girl is eating her morning cereal. Suddenly she looks apprehensively at the fridge. Camera dollies towards the appliance and seamlessly penetrates the reflective surface, revealing a deep hidden space that exactly matches the reflection. At the dark end of the tunnel, something stirs... A wildly grinning man takes a step forward and screams.</text></comment>
<story><title>SMERF: Streamable Memory Efficient Radiance Fields</title><url>https://smerf-3d.github.io/</url><text>We built SMERF, a new way for exploring NeRFs in real-time in your web browser. Try it out yourself!&lt;p&gt;Over the last few months, my collaborators and I have put together a new, real-time method that makes NeRF models accessible from smartphones, laptops, and low-power desktops, and we think we’ve done a pretty stellar job! SMERF, as we like to call it, distills a large, high quality NeRF into a real-time, streaming-ready representation that’s easily deployed to devices as small as a smartphone via the web browser.&lt;p&gt;On top of that, our models look great! Compared to other real-time methods, SMERF has higher accuracy than ever before. On large multi-room scenes, SMERF renders are nearly indistinguishable from state-of-the-art offline models like Zip-NeRF and a solid leap ahead of other approaches.&lt;p&gt;The best part: you can try it out yourself! Check out our project website for demos and more.&lt;p&gt;If you have any questions or feedback, don’t hesitate to reach out by email (smerf@google.com) or Twitter (@duck).</text></story><parent_chain><item><author>nightpool</author><text>The refigerator in the NYC scene has a very slick specular lighting effect based on the angle you&amp;#x27;re viewing it from, and if you go &amp;quot;into&amp;quot; the fridge you can see it&amp;#x27;s actually generating a whole 3d scene with blurry grey and white colors that turn out to precisely mimic the effects of the light from the windows bouncing off the metal, and you can look &amp;quot;out&amp;quot; from the fridge into the rest of the room. Same as the full-length mirror in the bedroom in the same scene—there&amp;#x27;s a whole virtual &amp;quot;mirror room&amp;quot; that&amp;#x27;s been built out behind the mirror to give the illusion of depth as you look through it. Very cool and unique consequence of the technology</text></item><item><author>barrkel</author><text>The mirror on the wall of the bathroom in the Berlin location looks through to the kitchen in the next room. I guess the depth gauging algorithm uses parallax, and mirrors confuse it, seeming like windows. The kitchen has a blob of blurriness as the rear of the mirror intrudes into kitchen, but you can see through the blurriness to either room.&lt;p&gt;The effect is a bit spooky. I felt like a ghost going through walls.</text></item></parent_chain><comment><author>TaylorAlexander</author><text>Oh wow yeah. It&amp;#x27;s interesting because when I look at the fridge my eye maps that to &amp;quot;this is a reflective surface&amp;quot;, which makes sense because that&amp;#x27;s true in the source images, but then it&amp;#x27;s actually rendered as a cavity with appropriate features rendered in 3D space. What&amp;#x27;s a strange feeling is to enter the fridge and then turn around! I just watched Hbomberguy&amp;#x27;s Patreon-only video on the video game Myst, and in Myst the characters are trapped in books. If you choose the wrong path at the end of the game you get trapped in a book, and the view you get trapped in a book looks very similar to the view from inside the NYC fridge!</text></comment>