<story><title>My First Impressions of Nix</title><url>https://mtlynch.io/notes/nix-first-impressions/</url></story><parent_chain><item><author>henrydark</author><text>I agree with everything but the last statement. This all comes down to: do you consider memoization to be state.&lt;p&gt;I predict people&amp;#x27;s answers to this question will come from experience with memoization. Here&amp;#x27;s mine: I kept trying to get nix to build tensorflow locally, so that I would get the avx512 benefits of the big, but gpu-less machine I had. I hadn&amp;#x27;t realized some other derivation had already downloaded tensorflow from online cache, so didn&amp;#x27;t have avx512 enabled. I kept making shells, trying tensorflow, seeing it doesn&amp;#x27;t have support. The solution was to tell nix to disregard the nix store, in order to force the local build. This experience has left me with the concrete feeling that the nix store is full-on state, and I the user must be aware of it.</text></item><item><author>danieldk</author><text>Just to elaborate a bit for those not familiar to Nix (slightly simplified to exclude recent support for content addressing). Nix work with derivations, a derivation is basically a data structure that specifies how a package is built. Derivations are normally not created by hand but using a function (eg. &lt;i&gt;stdenv.mkDerivation&lt;/i&gt;).&lt;p&gt;When you ask Nix to build a package, it hashes a normalized form of derivation data structure. This hash is useful in various ways, but one way it is used [1] is to look up whether the derivation is already in the Nix Store. Because if it is, there is no need to build it. So Nix looks up whether&lt;p&gt;&lt;pre&gt;&lt;code&gt;    &amp;#x2F;nix&amp;#x2F;&amp;lt;the_derivation_hash&amp;gt;
&lt;/code&gt;&lt;/pre&gt;
exists. If it exists, the build is done. If it doesn&amp;#x27;t exist and you have a binary cache configured (which by default is the binary cache provided by the NixOS project), Nix will look up the derivation hash in the binary cache. If it exists in the binary cache, Nix will download the path to the local Nix store. After that&lt;p&gt;&lt;pre&gt;&lt;code&gt;    &amp;#x2F;nix&amp;#x2F;&amp;lt;the_derivation_hash&amp;gt;
&lt;/code&gt;&lt;/pre&gt;
exists in the store and the build is done (without building anything). Only if that fails, Nix will actually build the derivation.&lt;p&gt;Now, one of the cool things about Nix is that it is derivations all the way down. So, it&amp;#x27;s not that just what we traditionally think of as packages is a derivation, but people wrap up all kinds of things as derivations, including configuration, etc. Since derivations are usually generated by functions, there are all kinds of useful functions that make derivations for eg.: single configuration files, scripts, etc.&lt;p&gt;In the end, building a NixOS system generation is just building a derivation. nixos-rebuild switches to a different generation by just setting a bunch of symlinks to an output path in the store containing that system generation (&lt;i&gt;&amp;#x2F;nix&amp;#x2F;&amp;lt;system_config_derivation_hash&lt;/i&gt;).&lt;p&gt;At any rate, when &lt;i&gt;you make a one-line change to a 200-line Nix configuration&lt;/i&gt;, Nix does have state to keep track of what it needs to rebuild or not. Nix will just try to build the derivation (and its dependencies), but it hashes the derivations, finds that their output paths are already in the store.&lt;p&gt;Some might argue that then the store is state. But it&amp;#x27;s not, at build time you are evaluating a pure function with memoization (the Nix Store).&lt;p&gt;[1] There is also a package name and version in the store path, but lets keep it simple.</text></item><item><author>hamandcheese</author><text>&amp;gt; Nix, on the other hand, does have a concept of state. If you make a one-line change to a 200-line Nix configuration, it doesn’t have to re-do all the work from the other 199 lines. It can evaluate the state of the system against the configuration file and recognize that it just has to apply the one-line change. And that change usually happens in a few seconds.&lt;p&gt;The author seems to have some misguided ideas about Nix. Nix is not fast because it is stateful. It is fast because it is functional and reproducible, which allows for caching without compromising correctness. I don&amp;#x27;t want to split hairs, but referentially transparent caching like this is not quite what I&amp;#x27;d call state.&lt;p&gt;Yes, there is some statefulness in system activation, but this is not what makes Nix Nix -- quite the opposite.</text></item></parent_chain><comment><author>dezgeg</author><text>&amp;gt; The solution was to tell nix to disregard the nix store, in order to force the local build.&lt;p&gt;If this actually led to avx512 being enabled in the package, then that&amp;#x27;s a bug. Nix builds should not be dependant on the machine doing the compilation, all such autodetection should be disabled via configure flag or patched out.&lt;p&gt;Then, the right way to enable avx512 would be to pass some &amp;#x27;enable avx512 please&amp;#x27; flag to the package&amp;#x27;s configure flags. Which would then trigger recompilation, without any &amp;#x27;disregard the nix store, in order to force the local build&amp;#x27; options.</text></comment>
<story><title>My First Impressions of Nix</title><url>https://mtlynch.io/notes/nix-first-impressions/</url></story><parent_chain><item><author>henrydark</author><text>I agree with everything but the last statement. This all comes down to: do you consider memoization to be state.&lt;p&gt;I predict people&amp;#x27;s answers to this question will come from experience with memoization. Here&amp;#x27;s mine: I kept trying to get nix to build tensorflow locally, so that I would get the avx512 benefits of the big, but gpu-less machine I had. I hadn&amp;#x27;t realized some other derivation had already downloaded tensorflow from online cache, so didn&amp;#x27;t have avx512 enabled. I kept making shells, trying tensorflow, seeing it doesn&amp;#x27;t have support. The solution was to tell nix to disregard the nix store, in order to force the local build. This experience has left me with the concrete feeling that the nix store is full-on state, and I the user must be aware of it.</text></item><item><author>danieldk</author><text>Just to elaborate a bit for those not familiar to Nix (slightly simplified to exclude recent support for content addressing). Nix work with derivations, a derivation is basically a data structure that specifies how a package is built. Derivations are normally not created by hand but using a function (eg. &lt;i&gt;stdenv.mkDerivation&lt;/i&gt;).&lt;p&gt;When you ask Nix to build a package, it hashes a normalized form of derivation data structure. This hash is useful in various ways, but one way it is used [1] is to look up whether the derivation is already in the Nix Store. Because if it is, there is no need to build it. So Nix looks up whether&lt;p&gt;&lt;pre&gt;&lt;code&gt;    &amp;#x2F;nix&amp;#x2F;&amp;lt;the_derivation_hash&amp;gt;
&lt;/code&gt;&lt;/pre&gt;
exists. If it exists, the build is done. If it doesn&amp;#x27;t exist and you have a binary cache configured (which by default is the binary cache provided by the NixOS project), Nix will look up the derivation hash in the binary cache. If it exists in the binary cache, Nix will download the path to the local Nix store. After that&lt;p&gt;&lt;pre&gt;&lt;code&gt;    &amp;#x2F;nix&amp;#x2F;&amp;lt;the_derivation_hash&amp;gt;
&lt;/code&gt;&lt;/pre&gt;
exists in the store and the build is done (without building anything). Only if that fails, Nix will actually build the derivation.&lt;p&gt;Now, one of the cool things about Nix is that it is derivations all the way down. So, it&amp;#x27;s not that just what we traditionally think of as packages is a derivation, but people wrap up all kinds of things as derivations, including configuration, etc. Since derivations are usually generated by functions, there are all kinds of useful functions that make derivations for eg.: single configuration files, scripts, etc.&lt;p&gt;In the end, building a NixOS system generation is just building a derivation. nixos-rebuild switches to a different generation by just setting a bunch of symlinks to an output path in the store containing that system generation (&lt;i&gt;&amp;#x2F;nix&amp;#x2F;&amp;lt;system_config_derivation_hash&lt;/i&gt;).&lt;p&gt;At any rate, when &lt;i&gt;you make a one-line change to a 200-line Nix configuration&lt;/i&gt;, Nix does have state to keep track of what it needs to rebuild or not. Nix will just try to build the derivation (and its dependencies), but it hashes the derivations, finds that their output paths are already in the store.&lt;p&gt;Some might argue that then the store is state. But it&amp;#x27;s not, at build time you are evaluating a pure function with memoization (the Nix Store).&lt;p&gt;[1] There is also a package name and version in the store path, but lets keep it simple.</text></item><item><author>hamandcheese</author><text>&amp;gt; Nix, on the other hand, does have a concept of state. If you make a one-line change to a 200-line Nix configuration, it doesn’t have to re-do all the work from the other 199 lines. It can evaluate the state of the system against the configuration file and recognize that it just has to apply the one-line change. And that change usually happens in a few seconds.&lt;p&gt;The author seems to have some misguided ideas about Nix. Nix is not fast because it is stateful. It is fast because it is functional and reproducible, which allows for caching without compromising correctness. I don&amp;#x27;t want to split hairs, but referentially transparent caching like this is not quite what I&amp;#x27;d call state.&lt;p&gt;Yes, there is some statefulness in system activation, but this is not what makes Nix Nix -- quite the opposite.</text></item></parent_chain><comment><author>hamandcheese</author><text>Yes, you have unfortunately discovered that sometimes the hardware itself is an input that isn&amp;#x27;t always captured explicitly, but also isn&amp;#x27;t controlled for with sandboxing. Ideally enabling avx512 would be an explicit input to the tensorflow package, but based on your experience it sounds like this feature is detected during the build automatically.&lt;p&gt;I hope that issues like this get better over time thanks to projects like Trustix, which would make non-reproducibility like this more apparent.</text></comment>