<story><title>Unbounded High Dynamic Range Photography Using a Modulo Camera</title><url>http://www.media.mit.edu/research/highlights/unbounded-high-dynamic-range-photography-using-modulo-camera</url></story><parent_chain><item><author>Tloewald</author><text>&amp;gt; This is a &amp;quot;computer scientists&amp;quot; understanding of photography, and this phrase alone can even be seen as &amp;quot;dangerous&amp;quot; by photographers.&lt;p&gt;The actual headline of the article is actually &lt;i&gt;more&lt;/i&gt; informative and less misleading than the subhead:&lt;p&gt;&amp;quot;Unbounded High Dynamic Range Photography Using a Modulo Camera&amp;quot;&lt;p&gt;In photographer&amp;#x27;s terms, they&amp;#x27;re referring to ending &lt;i&gt;blown highlights&lt;/i&gt; (caused by filling the wells in a sensel) not &lt;i&gt;saturated&lt;/i&gt; images (caused by exaggerating color data).&lt;p&gt;There&amp;#x27;s a school of photography that basically lives and dies by over-saturating images (e.g. &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;kenrockwell.com&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;kenrockwell.com&lt;/a&gt; ) and they&amp;#x27;d likely have a cow over the headline, but understand and be totally onboard for the actual goal.&lt;p&gt;Depending on how quickly the well can reset and how many times resets can be counted this will either be better or worse than simply improving existing approaches. The state of the art in full frame sensors is just under 15-bits of dynamic range. So to &lt;i&gt;equal&lt;/i&gt; it an 8-bit sensor would need to be able to count resets at least 128 times AND be able to do this in 1&amp;#x2F;8000 of a second, at the same sensel density and efficiency! Don&amp;#x27;t hold your breath.&lt;p&gt;&lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;www.dxomark.com&amp;#x2F;Cameras&amp;#x2F;Nikon&amp;#x2F;D810&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;www.dxomark.com&amp;#x2F;Cameras&amp;#x2F;Nikon&amp;#x2F;D810&lt;/a&gt;&lt;p&gt;(I&amp;#x27;d also suggest that trends towards using multiple sensors to assemble high resolution images will easily blow this away since they can use high- and low- sensitivity sensors to synthesize more dynamic range and resolution. Also see recent Olympus patents to capture polarization data at the same time.)&lt;p&gt;BTW: assuming they can do this stuff, they presumably can read the sensor pretty darn fast — so they should be able to eliminate image &amp;quot;tearing&amp;quot; in digital video and the need for mechanical shutters altogether. That&amp;#x27;s probably a bigger issue than dynamic range.&lt;p&gt;I suspect that clearing a well in &amp;lt; 1&amp;#x2F;10,000,000 of a second (which would only lose 10% of the photon detection time during a 1&amp;#x2F;8000s exposure) is going to be tough. Assuming 25% of the sensel real estate needs to be sacrificed to the modulo circuitry, that&amp;#x27;s a loss of 32.5% of photon data which is about the same loss as for pellicle mirrors (as seen in Sony&amp;#x27;s pretty unsuccessful SLT cameras) for a feature that won&amp;#x27;t be much use in many situations.</text></item><item><author>dr_zoidberg</author><text>&amp;gt; No more will photographers or even ordinary people have to fumble with aperture size and exposure length.&lt;p&gt;This is a &amp;quot;computer scientists&amp;quot; understanding of photography, and this phrase alone can even be seen as &amp;quot;dangerous&amp;quot; by photographers. There&amp;#x27;s more to aperture size and exposure length than &amp;quot;how much light reaches the sensor&amp;quot;, like focus, depth of field, motion blur and bokeh, to name a few, that is not aknowledged by that understanding. The technology is good, but it won&amp;#x27;t change the way photographers deal with photography, at best, it will give them a better tool to work with.&lt;p&gt;I also wonder how much impact this could have. Foveon Inc. had developed a great sensor that was going to be revolutionary, about 20 years ago. Today only Sigma uses it, and the UX is so bad in those cameras that the (very good) technology they had their hands on never got a chance to shine.&lt;p&gt;Also, is this implementation very different from having a 16-bit ADC in normal sensors? The end results would be the same: higher dynamic range. And the increase in bits seems a lot closer (because it would mean little changes to the current manufacture processes) than a whole new type of sensor being use in cameras.</text></item></parent_chain><comment><author>saidajigumi</author><text>&amp;gt; There&amp;#x27;s a school of photography that basically lives and dies by over-saturating images (e.g. &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;kenrockwell.com&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;kenrockwell.com&lt;/a&gt; ) and they&amp;#x27;d likely have a cow over the headline, but understand and be totally onboard for the actual goal.&lt;p&gt;Actually, I suspect that even that crowd, after the initial shock would be &lt;i&gt;absolutely&lt;/i&gt; on board.  This would mean that no longer would the sensor control how the high-end clipping happens, but now that could be tailored specifically via a post-processing filter. E.g. to exactly match a long-gone film emulsion&amp;#x27;s behavior, &amp;quot;enhance&amp;quot; a classic, or to simply to create new clipping functions.&lt;p&gt;This is a bit analogous to the flexibility that black and white shooters gain by working with modern color sensors.  When shooting B&amp;amp;W film, the tonal result was controlled via a combination of the film&amp;#x27;s spectral response and any colored lens filters applied, the latter used to manipulate tonal contrast between different colored subject matter (e.g. sky, plants, skin tones, etc.).  Now a skilled photographer can shoot color RAW w&amp;#x2F; B&amp;amp;W preview (for a preview of the image luminosity) then adjust in post to nail the image effect without having to mess even thinking about what colored lens filter(s) to have used.</text></comment>
<story><title>Unbounded High Dynamic Range Photography Using a Modulo Camera</title><url>http://www.media.mit.edu/research/highlights/unbounded-high-dynamic-range-photography-using-modulo-camera</url></story><parent_chain><item><author>Tloewald</author><text>&amp;gt; This is a &amp;quot;computer scientists&amp;quot; understanding of photography, and this phrase alone can even be seen as &amp;quot;dangerous&amp;quot; by photographers.&lt;p&gt;The actual headline of the article is actually &lt;i&gt;more&lt;/i&gt; informative and less misleading than the subhead:&lt;p&gt;&amp;quot;Unbounded High Dynamic Range Photography Using a Modulo Camera&amp;quot;&lt;p&gt;In photographer&amp;#x27;s terms, they&amp;#x27;re referring to ending &lt;i&gt;blown highlights&lt;/i&gt; (caused by filling the wells in a sensel) not &lt;i&gt;saturated&lt;/i&gt; images (caused by exaggerating color data).&lt;p&gt;There&amp;#x27;s a school of photography that basically lives and dies by over-saturating images (e.g. &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;kenrockwell.com&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;kenrockwell.com&lt;/a&gt; ) and they&amp;#x27;d likely have a cow over the headline, but understand and be totally onboard for the actual goal.&lt;p&gt;Depending on how quickly the well can reset and how many times resets can be counted this will either be better or worse than simply improving existing approaches. The state of the art in full frame sensors is just under 15-bits of dynamic range. So to &lt;i&gt;equal&lt;/i&gt; it an 8-bit sensor would need to be able to count resets at least 128 times AND be able to do this in 1&amp;#x2F;8000 of a second, at the same sensel density and efficiency! Don&amp;#x27;t hold your breath.&lt;p&gt;&lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;www.dxomark.com&amp;#x2F;Cameras&amp;#x2F;Nikon&amp;#x2F;D810&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;www.dxomark.com&amp;#x2F;Cameras&amp;#x2F;Nikon&amp;#x2F;D810&lt;/a&gt;&lt;p&gt;(I&amp;#x27;d also suggest that trends towards using multiple sensors to assemble high resolution images will easily blow this away since they can use high- and low- sensitivity sensors to synthesize more dynamic range and resolution. Also see recent Olympus patents to capture polarization data at the same time.)&lt;p&gt;BTW: assuming they can do this stuff, they presumably can read the sensor pretty darn fast — so they should be able to eliminate image &amp;quot;tearing&amp;quot; in digital video and the need for mechanical shutters altogether. That&amp;#x27;s probably a bigger issue than dynamic range.&lt;p&gt;I suspect that clearing a well in &amp;lt; 1&amp;#x2F;10,000,000 of a second (which would only lose 10% of the photon detection time during a 1&amp;#x2F;8000s exposure) is going to be tough. Assuming 25% of the sensel real estate needs to be sacrificed to the modulo circuitry, that&amp;#x27;s a loss of 32.5% of photon data which is about the same loss as for pellicle mirrors (as seen in Sony&amp;#x27;s pretty unsuccessful SLT cameras) for a feature that won&amp;#x27;t be much use in many situations.</text></item><item><author>dr_zoidberg</author><text>&amp;gt; No more will photographers or even ordinary people have to fumble with aperture size and exposure length.&lt;p&gt;This is a &amp;quot;computer scientists&amp;quot; understanding of photography, and this phrase alone can even be seen as &amp;quot;dangerous&amp;quot; by photographers. There&amp;#x27;s more to aperture size and exposure length than &amp;quot;how much light reaches the sensor&amp;quot;, like focus, depth of field, motion blur and bokeh, to name a few, that is not aknowledged by that understanding. The technology is good, but it won&amp;#x27;t change the way photographers deal with photography, at best, it will give them a better tool to work with.&lt;p&gt;I also wonder how much impact this could have. Foveon Inc. had developed a great sensor that was going to be revolutionary, about 20 years ago. Today only Sigma uses it, and the UX is so bad in those cameras that the (very good) technology they had their hands on never got a chance to shine.&lt;p&gt;Also, is this implementation very different from having a 16-bit ADC in normal sensors? The end results would be the same: higher dynamic range. And the increase in bits seems a lot closer (because it would mean little changes to the current manufacture processes) than a whole new type of sensor being use in cameras.</text></item></parent_chain><comment><author>paulmd</author><text>I identified the same issue with cycle time, but to me the biggest problem is the assumption of an accurate, sensitive, non-destructive-read sensor.  They&amp;#x27;re assuming that they can charge a capacitor from a photodiode accurately, and also hook a voltage comparator (which is effectively the same as an ADC) to it constantly to get an accurate pulse to count.&lt;p&gt;Assuming you have an accurate sensor that can be read non-destructively, then none of this modulus stuff is necessary anyway.  Read once to set the proper gain on the ADC, then read again to get the fine bits.  That way you still get the &amp;quot;unbounded HDR&amp;quot;, but you don&amp;#x27;t need an ADC for every single receptor site, and you don&amp;#x27;t waste space on the counter or comparator.  Instead, all of that lives on the image processor where it&amp;#x27;s not going to cause heat noise.&lt;p&gt;That&amp;#x27;s such an incredibly enormous hand-wave to me that I feel I&amp;#x27;ve got to be missing something.  Why make it so complex with all the modulus stuff if we have a magic sensor?</text></comment>