<story><title>Smol Developer</title><url>https://github.com/smol-ai/developer</url></story><parent_chain><item><author>wokwokwok</author><text>Seems like the wrong approach in the long term.&lt;p&gt;Realistically, a large code base LLM generation tool is going to look something like old-school C code.&lt;p&gt;An initial pass will generate an architecture and a series of independent code unit definitions (.h files) and then a &amp;#x27;detail pass&amp;#x27; will generate the code for (.c files) for each header file.&lt;p&gt;The header files will be &amp;#x27;relatively independent&amp;#x27; and small, so they fit inside the context for the LLM, and because the function definition and comments &amp;#x27;define&amp;#x27; what a function is, the LLM will generate consistent multi-file code.&lt;p&gt;The anti-patterns we see at the moment in this type of project are:&lt;p&gt;1) The entire code is passed to the LLM as context using a huge number of tokens meaninglessly. (you only need the function signature)&lt;p&gt;2) 1-page spaghetti definition files are stupid and unmaintainable (just read prompt.md if you don&amp;#x27;t believe me).&lt;p&gt;3) No way of verifying the &amp;#x27;plan&amp;#x27; before you go and generate the code that implements it (expensive and a waste of time; you should generate function signatures &lt;i&gt;first&lt;/i&gt; and then verify they are correct before generating the code for them).&lt;p&gt;4) Generating unit tests for full functions instead of just the function signatures (leaks implementation details).&lt;p&gt;It&amp;#x27;s interesting to me that modern language try to move all of these domains (header, implementation, tests) into a single place (or even a single &lt;i&gt;file&lt;/i&gt;, look at rust), but passing &lt;i&gt;all&lt;/i&gt; of that to an LLM is wrong.&lt;p&gt;It&amp;#x27;s &lt;i&gt;architecturally&lt;/i&gt; wrong in a way that won&amp;#x27;t scale.&lt;p&gt;Even an LLM with 1000k tokens that &lt;i&gt;could&lt;/i&gt; process a code base like this will be prohibitively slow and expensive to do so.&lt;p&gt;I suspect we&amp;#x27;ll see a class &amp;#x27;generative languages&amp;#x27; emerge in the future that walk the other direction so they are easier to use with LLM and code-gen.</text></item></parent_chain><comment><author>swyx</author><text>(author here) i suspect this maybe one of those “feels like a bad idea until you try it” things. Multiple times when i wanted to add a feature or be more specific, throwing it into prompt.md knowing the LLM would put it in the right file, or coordinate it across files, for me, was the most delightful experience of the “engineering with prompts” workflow.&lt;p&gt;that said if you see my future directions notes i do think theres room for file specific .md instructions.&lt;p&gt;the shared dependencies file is essentially a plan. i didnt realize it at the moment but now looking at it with fresh eyes i can do a no-op `smol plan` command pretty trivially.</text></comment>
<story><title>Smol Developer</title><url>https://github.com/smol-ai/developer</url></story><parent_chain><item><author>wokwokwok</author><text>Seems like the wrong approach in the long term.&lt;p&gt;Realistically, a large code base LLM generation tool is going to look something like old-school C code.&lt;p&gt;An initial pass will generate an architecture and a series of independent code unit definitions (.h files) and then a &amp;#x27;detail pass&amp;#x27; will generate the code for (.c files) for each header file.&lt;p&gt;The header files will be &amp;#x27;relatively independent&amp;#x27; and small, so they fit inside the context for the LLM, and because the function definition and comments &amp;#x27;define&amp;#x27; what a function is, the LLM will generate consistent multi-file code.&lt;p&gt;The anti-patterns we see at the moment in this type of project are:&lt;p&gt;1) The entire code is passed to the LLM as context using a huge number of tokens meaninglessly. (you only need the function signature)&lt;p&gt;2) 1-page spaghetti definition files are stupid and unmaintainable (just read prompt.md if you don&amp;#x27;t believe me).&lt;p&gt;3) No way of verifying the &amp;#x27;plan&amp;#x27; before you go and generate the code that implements it (expensive and a waste of time; you should generate function signatures &lt;i&gt;first&lt;/i&gt; and then verify they are correct before generating the code for them).&lt;p&gt;4) Generating unit tests for full functions instead of just the function signatures (leaks implementation details).&lt;p&gt;It&amp;#x27;s interesting to me that modern language try to move all of these domains (header, implementation, tests) into a single place (or even a single &lt;i&gt;file&lt;/i&gt;, look at rust), but passing &lt;i&gt;all&lt;/i&gt; of that to an LLM is wrong.&lt;p&gt;It&amp;#x27;s &lt;i&gt;architecturally&lt;/i&gt; wrong in a way that won&amp;#x27;t scale.&lt;p&gt;Even an LLM with 1000k tokens that &lt;i&gt;could&lt;/i&gt; process a code base like this will be prohibitively slow and expensive to do so.&lt;p&gt;I suspect we&amp;#x27;ll see a class &amp;#x27;generative languages&amp;#x27; emerge in the future that walk the other direction so they are easier to use with LLM and code-gen.</text></item></parent_chain><comment><author>verdverm</author><text>You might like what we are working on, generate an intermediate representation with LLMs, then use deterministic code gen to write the real code. This setup also allows for modifications and updates too, so you can use this the whole time rather than a one time bootstrapping.&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;twitter.com&amp;#x2F;verdverm&amp;#x2F;status&amp;#x2F;1655481985685389313?t=d7u2z6cPoE0f0M40nYqJfQ&amp;amp;s=19&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;twitter.com&amp;#x2F;verdverm&amp;#x2F;status&amp;#x2F;1655481985685389313?t=d7...&lt;/a&gt;</text></comment>