<story><title>Show HN: GPT Repo Loader – load entire code repos into GPT prompts</title><url>https://github.com/mpoon/gpt-repository-loader</url><text>I was getting tired of copy&amp;#x2F;pasting reams of code into GPT-4 to give it context before I asked it to help me, so I started this small tool. In a nutshell, gpt-repository-loader will spit out file paths and file contents in a prompt-friendly format. You can also use .gptignore to ignore files&amp;#x2F;folders that are irrelevant to your prompt.&lt;p&gt;gpt-repository-loader as-is works pretty well in helping me achieve better responses. Eventually, I thought it would be cute to load itself into GPT-4 and have GPT-4 improve it. I was honestly surprised by PR#17. GPT-4 was able to write a valid an example repo and an expected output and throw in a small curveball by adjusting .gptignore. I did tell GPT the output file format in two places: 1.) in the preamble when I prompted it to make a PR for issue #16 and 2.) as a string in gpt_repository_loader.py, both of which are indirect ways to infer how to build a functional test. However, I don&amp;#x27;t think I explained to GPT in English anywhere on how .gptignore works at all!&lt;p&gt;I wonder how far GPT-4 can take this repo. Here is the process I&amp;#x27;m following for developing:&lt;p&gt;- Open an issue describing the improvement to make&lt;p&gt;- Construct a prompt - start with using gpt_repository_loader.py on this repo to generate the repository context, then append the text of the opened issue after the --END-- line.&lt;p&gt;- Try not to edit any code GPT-4 generates. If there is something wrong, continue to prompt GPT to fix whatever it is.&lt;p&gt;- Create a feature branch on the issue and create a pull request based on GPT&amp;#x27;s response.&lt;p&gt;- Have a maintainer review, approve, and merge.&lt;p&gt;I am going to try to automate the steps above as much as possible. Really curious how tight the feedback loop will eventually get before something breaks!</text></story><parent_chain><item><author>ftufek</author><text>This is awesome, can&amp;#x27;t wait to get api access to the 32k token model. Rather than this approach of just converting the whole repo to a text file, what I&amp;#x27;m thinking is, you can let the model decide the most relevant files.&lt;p&gt;The initial prompt would be, &amp;quot;person wants to do x, here are the file list of this repo: ...., give me a list of files that you&amp;#x27;d want to edit, create or delete&amp;quot; -&amp;gt; take the list, try to fit the contents of them into 32k tokens and re-prompt with &amp;quot;user is trying to achieve x, here&amp;#x27;s the most relevant files with their contents:..., give me a git commit in the style of git patch&amp;#x2F;diff output&amp;quot;. From playing around with it today, I think this approach would work rather well and can be like a huge step up from AI line autocompletion.</text></item></parent_chain><comment><author>wantsanagent</author><text>Please see the following repos for tools in this area:&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;jerryjliu&amp;#x2F;llama_index&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;jerryjliu&amp;#x2F;llama_index&lt;/a&gt;&lt;p&gt;and&amp;#x2F;or&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;hwchase17&amp;#x2F;langchain&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;hwchase17&amp;#x2F;langchain&lt;/a&gt;</text></comment>
<story><title>Show HN: GPT Repo Loader – load entire code repos into GPT prompts</title><url>https://github.com/mpoon/gpt-repository-loader</url><text>I was getting tired of copy&amp;#x2F;pasting reams of code into GPT-4 to give it context before I asked it to help me, so I started this small tool. In a nutshell, gpt-repository-loader will spit out file paths and file contents in a prompt-friendly format. You can also use .gptignore to ignore files&amp;#x2F;folders that are irrelevant to your prompt.&lt;p&gt;gpt-repository-loader as-is works pretty well in helping me achieve better responses. Eventually, I thought it would be cute to load itself into GPT-4 and have GPT-4 improve it. I was honestly surprised by PR#17. GPT-4 was able to write a valid an example repo and an expected output and throw in a small curveball by adjusting .gptignore. I did tell GPT the output file format in two places: 1.) in the preamble when I prompted it to make a PR for issue #16 and 2.) as a string in gpt_repository_loader.py, both of which are indirect ways to infer how to build a functional test. However, I don&amp;#x27;t think I explained to GPT in English anywhere on how .gptignore works at all!&lt;p&gt;I wonder how far GPT-4 can take this repo. Here is the process I&amp;#x27;m following for developing:&lt;p&gt;- Open an issue describing the improvement to make&lt;p&gt;- Construct a prompt - start with using gpt_repository_loader.py on this repo to generate the repository context, then append the text of the opened issue after the --END-- line.&lt;p&gt;- Try not to edit any code GPT-4 generates. If there is something wrong, continue to prompt GPT to fix whatever it is.&lt;p&gt;- Create a feature branch on the issue and create a pull request based on GPT&amp;#x27;s response.&lt;p&gt;- Have a maintainer review, approve, and merge.&lt;p&gt;I am going to try to automate the steps above as much as possible. Really curious how tight the feedback loop will eventually get before something breaks!</text></story><parent_chain><item><author>ftufek</author><text>This is awesome, can&amp;#x27;t wait to get api access to the 32k token model. Rather than this approach of just converting the whole repo to a text file, what I&amp;#x27;m thinking is, you can let the model decide the most relevant files.&lt;p&gt;The initial prompt would be, &amp;quot;person wants to do x, here are the file list of this repo: ...., give me a list of files that you&amp;#x27;d want to edit, create or delete&amp;quot; -&amp;gt; take the list, try to fit the contents of them into 32k tokens and re-prompt with &amp;quot;user is trying to achieve x, here&amp;#x27;s the most relevant files with their contents:..., give me a git commit in the style of git patch&amp;#x2F;diff output&amp;quot;. From playing around with it today, I think this approach would work rather well and can be like a huge step up from AI line autocompletion.</text></item></parent_chain><comment><author>groestl</author><text>Maybe someone can correct me, but my understanding is that you would calculate the embeddings of code chunks, and the embedding of the prompt, and take those chunks that are most similar to the embedding of the prompt as context.&lt;p&gt;Edit: This, btw, is also the reason why I think that this here popped up on the hackernews frontpage a short while ago: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;pgvector&amp;#x2F;pgvector&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;pgvector&amp;#x2F;pgvector&lt;/a&gt;</text></comment>