<story><title>DeepSeek Coder: Let the Code Write Itself</title><url>https://deepseekcoder.github.io/</url></story><parent_chain><item><author>rickstanley</author><text>Hello, I would like to take this opportunity and ask for help here, about using A.I. with my own codebase.&lt;p&gt;Context: 
I missed [almost] the entire A.I. wave, but I knew that one day I would have to learn something about and&amp;#x2F;or use it. That day has come. I&amp;#x27;m allocated in one team, that is migrating to another engine, let&amp;#x27;s say &amp;quot;engine A → engine B&amp;quot;. We are looking from the perspective of A, to map the entries for B (inbound), and after the request to B is returned, we map back to A&amp;#x27;s model (outbound). This is a chore, and much of the work is repetitive, but it comes with its edge cases that we need to look out for and unfortunately there isn&amp;#x27;t a solid foundation of patterns apart from the Domain-driven design (DDD) thing. It seemed like a good use case for an A.I.&lt;p&gt;Attempts: I began by asking to ChatGPT and Bard, with questions similar to: &amp;quot;how to train LLM on own codebase&amp;quot; and &amp;quot;how to get started with prompt engineering using own codebase&amp;quot;.&lt;p&gt;I concluded that, fine-tuning is expensive, for large models, unrealistic for my RTX 3060 with 6Gb VRAM, no surprise there; so, I searched here, in Hacker News, for keywords like &amp;quot;llama&amp;quot;, &amp;quot;fine-tuning&amp;quot;, &amp;quot;local machine&amp;quot;, etc, and I found out about ollama and DeepSeek.&lt;p&gt;I tried both ollama and DeepSeek, the former was slow but not as slow as the latter, which was &lt;i&gt;dead slow&lt;/i&gt;, using a 13B model. I tried the 6&amp;#x2F;7B model (I think it was codellama) and I got reasonable results and speed. After feeding it some data, I was on my way to try and train on the codebase when a friend of mine came and suggested that I use Retrieval-Augmented Generation (RAG), I have yet to try it, with a setup Langchain + Ollama.&lt;p&gt;Any thoughts, suggestions or experiences to share?&lt;p&gt;I&amp;#x27;d appreciate it.</text></item></parent_chain><comment><author>WeMoveOn</author><text>&amp;gt; much of the work is repetitive, but it comes with its edge cases&lt;p&gt;for the repetitive stuff, just use copilot embedded in whatever editor you use.&lt;p&gt;the edge cases are tricky, to actually avoid these the model would need an understanding of both the use case (which is easy to describe to the model) and the code base itself (which is difficult, since description&amp;#x2F;docstring is not enough to capture the complex behaviors that can arise from interactions between parts of your codebase).&lt;p&gt;idk how you would train&amp;#x2F;finetune a model to somehow have this understanding of your code base, I doubt just doing next token prediction would help, you&amp;#x27;d likely have to create chat data discussing the intricacies of your code base and do DPO&amp;#x2F;RLFH to bake it into your model.&lt;p&gt;look into techniques like qlora that&amp;#x27;ll reduce the needed memory during tuning. look into platforms like vast ai to rent GPUs for cheap.&lt;p&gt;RAG&amp;#x2F;Agents could be useful but probably not. could store info about functions in your codebase such as the signature, the function it calls, its docstring, and known edge cases associated with it. if you don&amp;#x27;t have docstrings using a LLM to generate them is feasible.</text></comment>
<story><title>DeepSeek Coder: Let the Code Write Itself</title><url>https://deepseekcoder.github.io/</url></story><parent_chain><item><author>rickstanley</author><text>Hello, I would like to take this opportunity and ask for help here, about using A.I. with my own codebase.&lt;p&gt;Context: 
I missed [almost] the entire A.I. wave, but I knew that one day I would have to learn something about and&amp;#x2F;or use it. That day has come. I&amp;#x27;m allocated in one team, that is migrating to another engine, let&amp;#x27;s say &amp;quot;engine A → engine B&amp;quot;. We are looking from the perspective of A, to map the entries for B (inbound), and after the request to B is returned, we map back to A&amp;#x27;s model (outbound). This is a chore, and much of the work is repetitive, but it comes with its edge cases that we need to look out for and unfortunately there isn&amp;#x27;t a solid foundation of patterns apart from the Domain-driven design (DDD) thing. It seemed like a good use case for an A.I.&lt;p&gt;Attempts: I began by asking to ChatGPT and Bard, with questions similar to: &amp;quot;how to train LLM on own codebase&amp;quot; and &amp;quot;how to get started with prompt engineering using own codebase&amp;quot;.&lt;p&gt;I concluded that, fine-tuning is expensive, for large models, unrealistic for my RTX 3060 with 6Gb VRAM, no surprise there; so, I searched here, in Hacker News, for keywords like &amp;quot;llama&amp;quot;, &amp;quot;fine-tuning&amp;quot;, &amp;quot;local machine&amp;quot;, etc, and I found out about ollama and DeepSeek.&lt;p&gt;I tried both ollama and DeepSeek, the former was slow but not as slow as the latter, which was &lt;i&gt;dead slow&lt;/i&gt;, using a 13B model. I tried the 6&amp;#x2F;7B model (I think it was codellama) and I got reasonable results and speed. After feeding it some data, I was on my way to try and train on the codebase when a friend of mine came and suggested that I use Retrieval-Augmented Generation (RAG), I have yet to try it, with a setup Langchain + Ollama.&lt;p&gt;Any thoughts, suggestions or experiences to share?&lt;p&gt;I&amp;#x27;d appreciate it.</text></item></parent_chain><comment><author>anotherpaulg</author><text>You could try using aider [0], which is my open source ai coding tool. You need an OpenAI API key, and aider supports any of the GPT 3.5 or 4 models. Aider has features that make it work well with existing code bases, like git integration and a &amp;quot;repository map&amp;quot;. The repo map is used to send GPT-4 a distilled map of your code base, focused specifically on the coding task at hand [1]. This provides useful context so that GPT can understand the relevant parts of the larger code base when making a specific change.&lt;p&gt;[0] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;paul-gauthier&amp;#x2F;aider&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;paul-gauthier&amp;#x2F;aider&lt;/a&gt;&lt;p&gt;[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;aider.chat&amp;#x2F;docs&amp;#x2F;repomap.html&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;aider.chat&amp;#x2F;docs&amp;#x2F;repomap.html&lt;/a&gt;</text></comment>