<story><title>Diffusion Is Spectral Autoregression</title><url>https://sander.ai/2024/09/02/spectral-autoregression.html</url></story><parent_chain></parent_chain><comment><author>HarHarVeryFunny</author><text>The high and low frequency components of speech are produced and perceived in different ways.&lt;p&gt;The lower frequencies (roughly below 4KHz) are created by the vocal chords opening and closing at the fundamental frequency, and harmonics of this fundamental frequency (e.g. 100Hz + 2&amp;#x2F;3&amp;#x2F;400Hz etc harmonics), with this frequency spectrum then being modulated by the resonances of the vocal tract which change during pronunciation. What we perceive as speech is primarily the &lt;i&gt;changes&lt;/i&gt; to these resonances (aka formants) due to articulation&amp;#x2F;pronunciation.&lt;p&gt;The higher frequencies present in speech mostly comes from &amp;quot;white noise&amp;quot; created by the turbulence of forcing air out through closed teeth&amp;#x2F;etc (e.g. &amp;quot;S&amp;quot; sound), and our perception of these &amp;quot;fricative&amp;quot; speech sounds is based on onset&amp;#x2F;offset of energy in these higher 4-8KHz frequencies. Frequencies above 8KHz are not very perceptually relevant, and may be filtered out (e.g. not present in analog telephone speech).</text></comment>
<story><title>Diffusion Is Spectral Autoregression</title><url>https://sander.ai/2024/09/02/spectral-autoregression.html</url></story><parent_chain></parent_chain><comment><author>thho23i4234343</author><text>I don&amp;#x27;t mean to mean but: what is surprising about any of this ?&lt;p&gt;Joseph Fourier&amp;#x27;s solution to the heat-equation (linear diffusion) was in fact the origin of the FT. The high-freq coefficients decay (as -t^2 IIRC) in there; the reverse is also known to be &amp;quot;unstable&amp;quot; (numerically, and is singular from the equillibrium).&lt;p&gt;More over, the reformulation doesn&amp;#x27;t immediately reveal some computational speedup, or a better alternative formulation (which is usually a measure of how valuable it is epistemically).&lt;p&gt;(Edit: note that Heat-equation is more akin to the Fokker-Planck eqn, not actual Diffusion as an SDE as is used in Diffusion models).</text></comment>