| -------------------------------------------------------------------------------- | |
| -------------------------------------------------------------------------------- | |
| Spaghetti AI for Windows using NVIDIA GPU | |
| I wanted to come up with a unique name for the app, so I chose something | |
| silly. | |
| -------------------------------------------------------------------------------- | |
| -------------------------------------------------------------------------------- | |
| -------------------------------------------------------------------------------- | |
| Important Information | |
| -------------------------------------------------------------------------------- | |
| -------------------------------------------------------------------------------- | |
| -------------------------------------------------------------------------------- | |
| Storage Requirements | |
| You should make sure that you have at least 10 gigabytes of space, | |
| perhaps more, before installing this initially. However, you should make | |
| sure you have at least 50 gigabytes more of space before downloading | |
| model data. | |
| For perspective, on my computer the ".venv" folder has just over 50,000 | |
| files, and nearly 6,000 folders, and is nearly 6 gigabytes. The model | |
| data folder ("model_data") has the following folders and these are | |
| their sizes: | |
| 5.10 GB models--circulus--canvers-realistic-v3.6 | |
| 5.10 GB models--circulus--canvers-real-v3.7.5 | |
| 3.97 GB models--circulus--canvers-real-v3.8 | |
| 3.97 GB models--circulus--canvers-real-v3.8.1 | |
| 1.98 GB models--runwayml--stable-diffusion-v1-5 | |
| 2.15 GB models--stabilityai--sd-x2-latent-upscaler | |
| 12.50 GB models--stabilityai--sdxl-turbo | |
| 6.62 GB models--stabilityai--stable-diffusion-xl-base-1.0 | |
| 5.81 GB models--stabilityai--stable-diffusion-xl-refiner-1.0 | |
| General Requirements | |
| You must have a NVIDIA graphics card with Cuda installed. I already had | |
| Cuda installed. If you need Cuda, it's in a step below. This script is | |
| written to be used on my Windows computer using the GPU. This script | |
| will not work using a CPU. While there is code in the script for CPU | |
| usage, it doesn't work on a Windows computer. It might be related to the | |
| Triton module not being available in Windows. There seem to be something | |
| that might work, but it would have meant installing something from | |
| elsewhere that I was not willing to try. | |
| The version of CUDA might matter. If you have one that doesn't work, you | |
| might have to install a different version of CUDA. | |
| If you have installed Automatic1111, you don't need to do step 1 and 2 | |
| as that program required Python and Git. You don't need Automatic1111 | |
| installed to use this app. | |
| About Automatic1111: | |
| https://github.com/AUTOMATIC1111/stable-diffusion-webui#installation-and-running | |
| Antivirus/Firewall Issues | |
| Your Antivirus/Firewall software will need to be set to allow the | |
| command prompt to download and install Python components, as well as run | |
| the app. What you allow or don't allow is up to you. You may need to | |
| reinstall some of it if you don't allow it through the first time and it | |
| doesn't work for you. I don't know what the minimum you need to allow | |
| would be, or if some or all of it can operate in a sandboxed | |
| environment. It is important to remember that security settings by your | |
| firewall and antivirus program may prevent the app from installing | |
| and/or functioning properly. I use Comodo and I had to allow everything, | |
| and install it more than once, to get it to install properly. In | |
| Advanced View, I had to set files to trusted in Contained Apps and | |
| unblock files in Blocked Applications. When popup alerts came up, I had | |
| to allow the files, have it remember my answer and have it install as an | |
| installer or updater. I also had to go to | |
| "Tasks > Containment Tasks > Reset the Container" in order to have the | |
| files not install in a sandboxed environment. Then I had to install it | |
| again. | |
| It is also important to note that you should not have any other programs | |
| opened while installing this app. Until I had fully allowed the app | |
| through my firewall and antivirus program, at times my computer | |
| completely froze up. You don't want to lose any work you are working on. | |
| -------------------------------------------------------------------------------- | |
| -------------------------------------------------------------------------------- | |
| -------------------------------------------------------------------------------- | |
| Instructions | |
| -------------------------------------------------------------------------------- | |
| -------------------------------------------------------------------------------- | |
| -------------------------------------------------------------------------------- | |
| Step 1 (if you haven't done already): | |
| Install Python 3.10.6: | |
| https://www.python.org/ftp/python/3.10.6/python-3.10.6-amd64.exe | |
| Make sure to check "Add Python to PATH" | |
| -------------------- | |
| Step 2 (if you haven't done already): | |
| I'm not certain if you need this. You might not. I already had it | |
| installed on my system. | |
| Install git | |
| https://git-scm.com/download/win | |
| -------------------- | |
| Step 3: | |
| Create directory manually: | |
| C:/Spaghetti_AI | |
| This can be anywhere, and the directory can be named anything, but if | |
| you change it here you need to change it in all the other places in | |
| these instructions and in some of the files you download. | |
| -------------------- | |
| Step 4: | |
| You need to open a regular Command Prompt. You can press Win key + R to | |
| get "Run" window. In that, enter: | |
| cmd | |
| In Command Prompt, copy this line and press Enter: | |
| cd C:/Spaghetti_AI && py -m venv .venv && .venv\Scripts\activate.bat | |
| This will change the directory of the command prompt to the directory | |
| you created. Next, it creates a virtual environment for Python as | |
| described here: | |
| https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/ | |
| https://huggingface.co/docs/diffusers/installation | |
| Finally, it activates that virtual environment as also described there. | |
| The command prompt will now look like this: | |
| (.venv) C:/Spaghetti_AI> | |
| You don't need to reopen the Command Prompt each time. Wait until a task | |
| finishes and then you will be able to enter the next command. It's best | |
| you do not close it. | |
| If you do close it, you will need to go to the directory again each time | |
| and activate the virtual environment like this, by copying this and | |
| pressing Enter: | |
| cd C:/Spaghetti_AI && .venv\Scripts\activate.bat | |
| You do not want to install Python packages outside of the virtual | |
| environment as it would replace the packages in your main installation. | |
| That could cause conflicts in other programs. | |
| Always make sure you have: | |
| (.venv) C:/Spaghetti_AI> | |
| In future steps. (though I have added the code to activate it every time | |
| as a precaution) | |
| -------------------- | |
| Step 5 (optional): | |
| pip is the package installer for Python. | |
| If you want to make sure pip is up-to-date, in Command Prompt (that is | |
| doing things in your virtual environment), copy this line and press | |
| Enter: | |
| cd C:/Spaghetti_AI && .venv\Scripts\activate.bat && py -m pip install --upgrade pip | |
| Since you are using a virtual environment, packages that are installed | |
| will be installed into this folder rather than into Python's main | |
| program: | |
| C:/Spaghetti_AI/.venv/Lib/site-packages | |
| That way you don't cause issues updating things you may not want to have | |
| updated. | |
| -------------------- | |
| Step 6 (if needed; unsure of this step, I didn't do it): | |
| If you don't have Cuda and need to get it, it's here: | |
| https://developer.nvidia.com/cuda-toolkit-archive | |
| I have 12.2 installed. Some things mention needing 12.1. In Step 8, as | |
| of writing this, it specifically mentions Cuda 12.1. You may want to | |
| check the PyTorch website first in Step 8 if you need to install Cuda to | |
| see what version it is compatible with or you will not get past that | |
| step. | |
| That link above has a list of prior versions. As packages update, you | |
| might need something even newer. | |
| To see what version of Cuda you have, in Command Prompt, copy this line | |
| and press Enter: | |
| nvidia-smi | |
| In the top right corner of the text output that is displayed, your Cuda | |
| version will be displayed. | |
| An older page on this command is here: | |
| https://developer.nvidia.com/nvidia-system-management-interface | |
| The text output now looks different. | |
| -------------------- | |
| Step 7: | |
| You need to install PyTorch. | |
| You should visit their website and see what version they recommend based | |
| on the version of Cuda you have. | |
| https://pytorch.org/ | |
| As of writing this, in the "INSTALL PYTORCH" section I selected the | |
| Stable PyTorch build, Windows, Pip, Python and Cuda 12.1. | |
| That generated in the "Run this Command" section the command you see | |
| below. (with the added code to do this in the virtual environment) | |
| To install PyTorch in Command Prompt (that is doing things in your | |
| virtual environment), copy this line and press Enter: | |
| cd C:/Spaghetti_AI && .venv\Scripts\activate.bat && pip3 install -U xformers --index-url https://download.pytorch.org/whl/cu121 | |
| A lot of things will download and install, including any packages needed | |
| to run this. When complete, move on to the next step. (when the command | |
| prompt returns to where you can paste into it again) | |
| The torch file alone is over 2 gigabytes. | |
| -------------------- | |
| Step 8: | |
| This step installs the rest of what you need. | |
| You can read about them if you want: | |
| https://huggingface.co/docs/diffusers/index | |
| https://huggingface.co/docs/transformers/index | |
| https://huggingface.co/docs/accelerate/index | |
| https://pypi.org/project/ftfy/ | |
| https://github.com/modin-project/modin | |
| https://pypi.org/project/invisible-watermark/ | |
| https://pypi.org/project/torchsde/ | |
| https://github.com/gradio-app/gradio | |
| To install these, in Command Prompt (that is doing things in your | |
| virtual environment), all you need to do is copy this line and press | |
| Enter: | |
| cd C:/Spaghetti_AI && .venv\Scripts\activate.bat && pip install diffusers transformers accelerate ftfy modin[all] invisible_watermark torchsde | |
| A whole bunch of things, what might be 100 packages or more, will | |
| download and install, including any packages needed to run these. | |
| As of writing this, the script needs version 4.11.0 of Gradio. In some | |
| versions before that there was a bug that made dropdown menus not work | |
| properly. Another bug was that an image couldn't be downloaded using the | |
| download button when viewing a gallery. Even older versions simply won't | |
| work at all with this app. This worked for me: | |
| cd C:/Spaghetti_AI && .venv\Scripts\activate.bat && pip install https://gradio-builds.s3.amazonaws.com/d4068557953746662235d595ec435c42ceb24414/gradio-4.11.0-py3-none-any.whl | |
| That was found here: | |
| https://www.gradio.app/docs/blocks | |
| After selecting "main" from the version number dropdown in the left | |
| column. The link changes when there is a new version. A version history | |
| of Gradio is here: | |
| https://www.gradio.app/changelog | |
| When complete, move on to the next step. | |
| -------------------- | |
| Step 9: | |
| Add these files: | |
| spaghetti_ai_install.bat | |
| spaghetti_ai_launcher.bat | |
| spaghetti_ai_script.py | |
| To this folder: | |
| C:/Spaghetti_AI/.venv/app_files/ | |
| So that a file is here for example: | |
| C:/Spaghetti_AI/.venv/app_files/spaghetti_ai_script.py | |
| That's really the most important one that needs to be there as those two | |
| .bat files call it. | |
| There are some things you will need to change in that file, most notably | |
| the directory specified in "main_dir". | |
| -------------------- | |
| Step 10 (Important): | |
| I feel this is a very important step. When you try to create model data | |
| for each base model, as well as use the refiner and upscaler, model data | |
| will try to be downloaded. There is also an option to download all, or | |
| just the default data, at one time. That we will cover in a later step. | |
| But before you can download model data, you need to make sure two things | |
| are set to "0". At least one will not be when you get this script so | |
| that you read this section. | |
| All the model data could be 40 gigabytes or more of data. I strongly | |
| recommend that you then disable the script from downloading updates to | |
| the model data once you have downloaded what you want. HuggingFace | |
| doesn't just download a model once and never try again. If something is | |
| updated, it will download it again. It will not delete the older | |
| version. This could eventually allow it to use all the space on your | |
| hard drive as that could potentially add 5 to 15 gigabytes of data each | |
| time a model updates. There are two variables that you need to make sure | |
| are "0". If you do not use "spaghetti_ai_launcher.bat", then there is | |
| only one. | |
| In "spaghetti_ai_launcher.bat", if you use it, set this to 0: | |
| HF_HUB_OFFLINE | |
| In "spaghetti_ai_script.py", set this to 0: | |
| only_use_local_files | |
| To have model data download, both variables must be set to 0 or model | |
| data will not download. Eventually I may only use one of those. You | |
| can't set, that I can see, "HF_HUB_OFFLINE" in the python script. It | |
| should now block all requests online, including to download model data, | |
| whereas the other only prevents model data from being downloaded. Some | |
| people might not use "spaghetti_ai_launcher.bat", so it's hard to just | |
| pick one to use. | |
| If you use "spaghetti_ai_launcher.bat" to launch the script, you can set | |
| "HF_HUB_OFFLINE" in that file to 1 to not download data again and not | |
| set "only_use_local_files". That variable will automatically be changed | |
| to "1" if "HF_HUB_OFFLINE" is set to "1" | |
| "HF_HUB_OFFLINE" can be set permanently in Windows, until you change it, | |
| but this is the easiest way that doesn't impact other installations. If | |
| you wanted to permanently store that environment variable in Windows, | |
| which I don't recommend, it would impact other installations that use | |
| Hugging Face. This only has it apply when using the script. | |
| You can read about environment variables here at Hugging Face: | |
| https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables#hfhuboffline | |
| In regard to Hugging Face caching things, you can learn more on this | |
| page: | |
| https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations | |
| The first time you use the script you will need to set HF_HUB_OFFLINE to | |
| 0 like this there: | |
| set HF_HUB_OFFLINE=0 | |
| Do not include any spaces before or after the equals sign. | |
| Or access the app from the command line when you want to download data | |
| if "only_use_local_files" is set to "0". | |
| The script will not work otherwise. Because of the importance of this | |
| variable, I prefer to not have the script work with the .bat file until | |
| someone changes that value themselves. | |
| -------------------- | |
| Step 11: | |
| You're ready to run the script that runs the web interface. | |
| The first time the models download it will take time as it's gigabytes | |
| of data that you need to download. By default, an individual model file | |
| isn't downloaded until you actually try creating an image for that | |
| model. You can change that however which we will get to in a moment. | |
| In Command Prompt (that is doing things in your virtual environment), | |
| copy this line and press Enter: | |
| cd C:/Spaghetti_AI && .venv\Scripts\activate.bat && py .venv\app_files\spaghetti_ai_script.py | |
| That will launch the web interface. | |
| When done, you will get a message like this: | |
| Running on local URL: http://127.0.0.1:7860 | |
| If your browser did not open a web browser, visit the link that was in | |
| the command prompt. If it was not the one above, note it. However, if | |
| you have something like Automatic1111 open already, it will use a | |
| different port, like "7861". | |
| It might be this: | |
| http://127.0.0.1:7860 | |
| To download a lot of data at once rather than waiting until you use each | |
| model, you can add this at the end of the URL that opens after your | |
| browser opens: | |
| ?download_data=1 | |
| It will download dozens of gigabytes of data, so you may not want to do | |
| that. A "1" will download only the default model configuration for each | |
| main model, as well as refiner and upscaler data. A "2" will download | |
| all model data needed in "model_configuration_links_object", including | |
| online configurations you may not want. | |
| Or you can just download it as you use each model. | |
| Downloading all the model data will mean downloading dozens of gigabytes | |
| of data. | |
| Model files are downloaded to: | |
| C:/Spaghetti_AI/model_data | |
| You should know that this folder can grow considerably. You need to | |
| manually clean it out when you need to get rid of old models. | |
| Occasionally, new versions will be downloaded when you create an image. | |
| O(unless you turned off the ability to download model data) When that | |
| Ohappens, you may want to remove the old version if the new version | |
| Oworks. Otherwise, you could have 5 to 15 gigabytes of space added each | |
| Otime you use a model that updates. | |
| You should also know that temporary images created are likely stored | |
| Ohere: | |
| %USERPROFILE%/AppData/Local/Temp/gradio | |
| Those however are likely cleaned out automatically by your computer. | |
| A package called Triton apparently can't be installed on Windows so you | |
| will see an error in the Command Prompt go by about that every time you | |
| launch the web interface. It doesn't seem to impact anything when using | |
| a GPU on Windows. | |
| https://github.com/openai/triton/issues/1057 | |
| https://github.com/openai/triton/issues/1640 | |
| Someone offers something there that might work, but I didn't want to | |
| install something from someone randomly. Again, it seems to work without | |
| it when using a GPU. If trying to use a CPU using Windows, that might be | |
| what prevents the app from being able work. | |
| -------------------- | |
| Step 12: | |
| When you open the link, you'll see the display to create the imagery. | |
| You must leave the command prompt open. When you process an image, the | |
| command prompt will tell you the progress of what you are creating and | |
| approximately how long it will take for the step it is on. It will take | |
| longer as there will be things that happen for which there is no | |
| progress indicator, either in the command prompt or the progress shown | |
| in the browser. | |
| -------------------- | |
| Step 13: | |
| To create a shortcut to the command that needs to be launched every | |
| time, use: | |
| spaghetti_ai_launcher.bat | |
| You can then create a shortcut to that, which you could add to the | |
| desktop. | |
| You can follow the instructions below or use the shortcut that comes | |
| with this app: | |
| Spaghetti AI.lnk | |
| Right click on your desktop and then click on "New > Shortcut". | |
| For "Type the location of the item", enter: | |
| cmd | |
| Then click "Next". You can name the shortcut whatever you want, like | |
| "Spaghetti AI". | |
| Then click "Finish". | |
| Then right click on the shortcut you created and click "Properties". | |
| In "Target", replace what is there with the following: | |
| C:\Windows\System32\cmd.exe /k C:/Spaghetti_AI/.venv/app_files/spaghetti_ai_launcher.bat | |
| That will execute that file each time you click on it. | |
| If for some reason that doesn't work sometimes, and rather than get the | |
| program to launch it simply opens the command prompt, then you must | |
| enter this each time in the command prompt: | |
| cd C:/Spaghetti_AI && .venv\Scripts\activate.bat && py .venv\app_files\spaghetti_ai_script.py | |
| You might also not be able to use a shortcut. If so, just copy | |
| "spaghetti_ai_launcher.bat" and put it on your desktop and name it what | |
| you want. The downside, if you do that, would be reconfiguring your | |
| firewall to use it from that location too. It's best to create a | |
| shortcut. | |
| -------------------- | |
| File List | |
| You can add all of these files to: | |
| C:\Spaghetti_AI\.venv\app_files | |
| Although some of these files are not needed. | |
| - Instructions.txt | |
| This file. Instructions on how to install the app. | |
| - README.md | |
| - requirements.txt | |
| These files are for use on HuggingFace. They are not otherwise | |
| needed. | |
| - spaghetti_ai_install.txt | |
| You should avoid using this. I wanted to create an easier way to | |
| install it, but it's best to read the instructions. | |
| "C:/Spaghetti_AI" needs to be set in "main_dir" in this file if | |
| you change the path and use this file. | |
| - spaghetti_ai_launcher.bat | |
| This is the file that launches "spaghetti_ai_script.py". This sets | |
| some variables that need to be set there, like disabling the | |
| automatic downloading of model data and turning off telemetry. The | |
| shortcuts in this section should point to this. | |
| "C:/Spaghetti_AI" needs to be set in "main_dir" in this file if | |
| you change the path. | |
| - spaghetti_ai_script.py | |
| The main script. | |
| "C:/Spaghetti_AI" needs to be set in "main_dir" in this file if | |
| you change the path. If "only_use_local_files" is set to "1", you | |
| will need to change it to download model data. | |
| - Spaghetti AI (no icon).lnk | |
| - Spaghetti AI.lnk | |
| - spaghetti_ai_icon_for_shortcut.ico | |
| These files are simply to provide a shortcut to the app. I created | |
| a .ico file to have a nice icon that you can use with the | |
| shortcut. Using a custom icon in a shortcut might not work, unless | |
| you do it yourself, so I also included one that does not have a | |
| custom icon. In Windows, you can right click on the shortcut and | |
| then click "Properties". On the "Shortcut" tab you can then click | |
| "Change Icon". You can then browse to the location of the icon | |
| provided or another. If you change the path from | |
| "C:/Spaghetti_AI", the shortcut will have to be edited in the | |
| "Target" field of the "Shortcut" tab. | |
| -------------------- | |
| Various Information | |
| You get this error: | |
| "CUDA out of memory. Tried to allocate 82.00 MiB. GPU 0 has a total | |
| capacty of 12.00 GiB of which 0 bytes is free. Of the allocated | |
| memory 16.97 GiB is allocated by PyTorch, and 439.95 MiB is reserved | |
| by PyTorch but unallocated. If reserved but unallocated memory is | |
| large try setting max_split_size_mb to avoid fragmentation. See | |
| documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF" | |
| Try setting these to 1 in spaghetti_ai_script.py: | |
| use_sequential_cpu_offload_for_base_model = 1 | |
| use_sequential_cpu_offload_for_refiner = 1 | |
| use_sequential_cpu_offload_for_upscaler = 1 | |