Instructions.txt

--------------------------------------------------------------------------------
--------------------------------------------------------------------------------


    Spaghetti AI for Windows using NVIDIA GPU


        I wanted to come up with a unique name for the app, so I chose something
        silly.


--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------

    Important Information

--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------


    Storage Requirements


        You should make sure that you have at least 10 gigabytes of space,
        perhaps more, before installing this initially. However, you should make
        sure you have at least 50 gigabytes more of space before downloading
        model data.

        For perspective, on my computer the ".venv" folder has just over 50,000
        files, and nearly 6,000 folders, and is nearly 6 gigabytes. The model
        data folder ("model_data") has the following folders and these are
        their sizes:

             5.10 GB    models--circulus--canvers-realistic-v3.6
             5.10 GB    models--circulus--canvers-real-v3.7.5
             3.97 GB    models--circulus--canvers-real-v3.8
             3.97 GB    models--circulus--canvers-real-v3.8.1
             1.98 GB    models--runwayml--stable-diffusion-v1-5
             2.15 GB    models--stabilityai--sd-x2-latent-upscaler
            12.50 GB    models--stabilityai--sdxl-turbo
             6.62 GB    models--stabilityai--stable-diffusion-xl-base-1.0
             5.81 GB    models--stabilityai--stable-diffusion-xl-refiner-1.0


    General Requirements


        You must have a NVIDIA graphics card with Cuda installed. I already had
        Cuda installed. If you need Cuda, it's in a step below. This script is
        written to be used on my Windows computer using the GPU. This script
        will not work using a CPU. While there is code in the script for CPU
        usage, it doesn't work on a Windows computer. It might be related to the
        Triton module not being available in Windows. There seem to be something
        that might work, but it would have meant installing something from
        elsewhere that I was not willing to try.

        The version of CUDA might matter. If you have one that doesn't work, you
        might have to install a different version of CUDA.

        If you have installed Automatic1111, you don't need to do step 1 and 2
        as that program required Python and Git. You don't need Automatic1111
        installed to use this app.

        About Automatic1111:
            https://github.com/AUTOMATIC1111/stable-diffusion-webui#installation-and-running


    Antivirus/Firewall Issues


        Your Antivirus/Firewall software will need to be set to allow the
        command prompt to download and install Python components, as well as run
        the app. What you allow or don't allow is up to you. You may need to
        reinstall some of it if you don't allow it through the first time and it
        doesn't work for you. I don't know what the minimum you need to allow
        would be, or if some or all of it can operate in a sandboxed
        environment. It is important to remember that security settings by your
        firewall and antivirus program may prevent the app from installing
        and/or functioning properly. I use Comodo and I had to allow everything,
        and install it more than once, to get it to install properly. In
        Advanced View, I had to set files to trusted in Contained Apps and
        unblock files in Blocked Applications. When popup alerts came up, I had
        to allow the files, have it remember my answer and have it install as an
        installer or updater. I also had to go to
        "Tasks > Containment Tasks > Reset the Container" in order to have the
        files not install in a sandboxed environment. Then I had to install it
        again.

        It is also important to note that you should not have any other programs
        opened while installing this app. Until I had fully allowed the app
        through my firewall and antivirus program, at times my computer
        completely froze up. You don't want to lose any work you are working on.


--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------


    Instructions


--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------


    Step 1 (if you haven't done already):

        Install Python 3.10.6:

            https://www.python.org/ftp/python/3.10.6/python-3.10.6-amd64.exe

        Make sure to check "Add Python to PATH"


--------------------


    Step 2 (if you haven't done already):

        I'm not certain if you need this. You might not. I already had it
        installed on my system.

        Install git

            https://git-scm.com/download/win


--------------------


    Step 3:

        Create directory manually:

            C:/Spaghetti_AI

        This can be anywhere, and the directory can be named anything, but if
        you change it here you need to change it in all the other places in
        these instructions and in some of the files you download.


--------------------


    Step 4:

        You need to open a regular Command Prompt. You can press Win key + R to
        get "Run" window. In that, enter:

cmd

        In Command Prompt, copy this line and press Enter:


cd C:/Spaghetti_AI && py -m venv .venv && .venv\Scripts\activate.bat


        This will change the directory of the command prompt to the directory
        you created. Next, it creates a virtual environment for Python as
        described here:

            https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/
            https://huggingface.co/docs/diffusers/installation

        Finally, it activates that virtual environment as also described there.

        The command prompt will now look like this:

            (.venv) C:/Spaghetti_AI>

        You don't need to reopen the Command Prompt each time. Wait until a task
        finishes and then you will be able to enter the next command. It's best
        you do not close it.

        If you do close it, you will need to go to the directory again each time
        and activate the virtual environment like this, by copying this and
        pressing Enter:


cd C:/Spaghetti_AI && .venv\Scripts\activate.bat


        You do not want to install Python packages outside of the virtual
        environment as it would replace the packages in your main installation.
        That could cause conflicts in other programs.

        Always make sure you have:

            (.venv) C:/Spaghetti_AI>

        In future steps. (though I have added the code to activate it every time
        as a precaution)


--------------------


    Step 5 (optional):

        pip is the package installer for Python.

        If you want to make sure pip is up-to-date, in Command Prompt (that is
        doing things in your virtual environment), copy this line and press
        Enter:


cd C:/Spaghetti_AI && .venv\Scripts\activate.bat && py -m pip install --upgrade pip


        Since you are using a virtual environment, packages that are installed
        will be installed into this folder rather than into Python's main
        program:

            C:/Spaghetti_AI/.venv/Lib/site-packages

        That way you don't cause issues updating things you may not want to have
        updated.


--------------------


    Step 6 (if needed; unsure of this step, I didn't do it):

        If you don't have Cuda and need to get it, it's here:

            https://developer.nvidia.com/cuda-toolkit-archive

        I have 12.2 installed. Some things mention needing 12.1. In Step 8, as
        of writing this, it specifically mentions Cuda 12.1. You may want to
        check the PyTorch website first in Step 8 if you need to install Cuda to
        see what version it is compatible with or you will not get past that
        step.

        That link above has a list of prior versions. As packages update, you
        might need something even newer.

        To see what version of Cuda you have, in Command Prompt, copy this line
        and press Enter:

nvidia-smi

        In the top right corner of the text output that is displayed, your Cuda
        version will be displayed.

        An older page on this command is here:

            https://developer.nvidia.com/nvidia-system-management-interface

        The text output now looks different.


--------------------


    Step 7:

        You need to install PyTorch.

        You should visit their website and see what version they recommend based
        on the version of Cuda you have.

            https://pytorch.org/

        As of writing this, in the "INSTALL PYTORCH" section I selected the
        Stable PyTorch build, Windows, Pip, Python and Cuda 12.1.

        That generated in the "Run this Command" section the command you see
        below. (with the added code to do this in the virtual environment)

        To install PyTorch in Command Prompt (that is doing things in your
        virtual environment), copy this line and press Enter:


cd C:/Spaghetti_AI && .venv\Scripts\activate.bat && pip3 install -U xformers --index-url https://download.pytorch.org/whl/cu121


        A lot of things will download and install, including any packages needed
        to run this. When complete, move on to the next step. (when the command
        prompt returns to where you can paste into it again)

        The torch file alone is over 2 gigabytes.


--------------------


    Step 8:

        This step installs the rest of what you need.

        You can read about them if you want:

            https://huggingface.co/docs/diffusers/index
            https://huggingface.co/docs/transformers/index
            https://huggingface.co/docs/accelerate/index
            https://pypi.org/project/ftfy/
            https://github.com/modin-project/modin
            https://pypi.org/project/invisible-watermark/
            https://pypi.org/project/torchsde/
            https://github.com/gradio-app/gradio

        To install these, in Command Prompt (that is doing things in your
        virtual environment), all you need to do is copy this line and press
        Enter:


cd C:/Spaghetti_AI && .venv\Scripts\activate.bat && pip install diffusers transformers accelerate ftfy modin[all] invisible_watermark torchsde


        A whole bunch of things, what might be 100 packages or more, will
        download and install, including any packages needed to run these.

        As of writing this, the script needs version 4.11.0 of Gradio. In some
        versions before that there was a bug that made dropdown menus not work
        properly. Another bug was that an image couldn't be downloaded using the
        download button when viewing a gallery. Even older versions simply won't
        work at all with this app. This worked for me:


cd C:/Spaghetti_AI && .venv\Scripts\activate.bat && pip install https://gradio-builds.s3.amazonaws.com/d4068557953746662235d595ec435c42ceb24414/gradio-4.11.0-py3-none-any.whl


        That was found here:

            https://www.gradio.app/docs/blocks

        After selecting "main" from the version number dropdown in the left
        column. The link changes when there is a new version. A version history
        of Gradio is here:

            https://www.gradio.app/changelog

        When complete, move on to the next step.


--------------------


    Step 9:

        Add these files:

            spaghetti_ai_install.bat
            spaghetti_ai_launcher.bat
            spaghetti_ai_script.py

        To this folder:

            C:/Spaghetti_AI/.venv/app_files/

        So that a file is here for example:

            C:/Spaghetti_AI/.venv/app_files/spaghetti_ai_script.py

        That's really the most important one that needs to be there as those two
        .bat files call it.

        There are some things you will need to change in that file, most notably
        the directory specified in "main_dir".


--------------------


    Step 10 (Important):

        I feel this is a very important step. When you try to create model data
        for each base model, as well as use the refiner and upscaler, model data
        will try to be downloaded. There is also an option to download all, or
        just the default data, at one time. That we will cover in a later step.
        But before you can download model data, you need to make sure two things
        are set to "0". At least one will not be when you get this script so
        that you read this section.

        All the model data could be 40 gigabytes or more of data. I strongly
        recommend that you then disable the script from downloading updates to
        the model data once you have downloaded what you want. HuggingFace
        doesn't just download a model once and never try again. If something is
        updated, it will download it again. It will not delete the older
        version. This could eventually allow it to use all the space on your
        hard drive as that could potentially add 5 to 15 gigabytes of data each
        time a model updates. There are two variables that you need to make sure
        are "0". If you do not use "spaghetti_ai_launcher.bat", then there is
        only one.

        In "spaghetti_ai_launcher.bat", if you use it, set this to 0:

            HF_HUB_OFFLINE

        In "spaghetti_ai_script.py", set this to 0:

            only_use_local_files

        To have model data download, both variables must be set to 0 or model
        data will not download. Eventually I may only use one of those. You
        can't set, that I can see, "HF_HUB_OFFLINE" in the python script. It
        should now block all requests online, including to download model data,
        whereas the other only prevents model data from being downloaded. Some
        people might not use "spaghetti_ai_launcher.bat", so it's hard to just
        pick one to use.

        If you use "spaghetti_ai_launcher.bat" to launch the script, you can set
        "HF_HUB_OFFLINE" in that file to 1 to not download data again and not
        set "only_use_local_files". That variable will automatically be changed
        to "1" if "HF_HUB_OFFLINE" is set to "1"

        "HF_HUB_OFFLINE" can be set permanently in Windows, until you change it,
        but this is the easiest way that doesn't impact other installations. If
        you wanted to permanently store that environment variable in Windows,
        which I don't recommend, it would impact other installations that use
        Hugging Face. This only has it apply when using the script.

        You can read about environment variables here at Hugging Face:

            https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables#hfhuboffline

        In regard to Hugging Face caching things, you can learn more on this
        page:

            https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations

        The first time you use the script you will need to set HF_HUB_OFFLINE to
        0 like this there:

set HF_HUB_OFFLINE=0

        Do not include any spaces before or after the equals sign.

        Or access the app from the command line when you want to download data
        if "only_use_local_files" is set to "0".

        The script will not work otherwise. Because of the importance of this
        variable, I prefer to not have the script work with the .bat file until
        someone changes that value themselves.


--------------------


    Step 11:

        You're ready to run the script that runs the web interface.

        The first time the models download it will take time as it's gigabytes
        of data that you need to download. By default, an individual model file
        isn't downloaded until you actually try creating an image for that
        model. You can change that however which we will get to in a moment.

        In Command Prompt (that is doing things in your virtual environment),
        copy this line and press Enter:


cd C:/Spaghetti_AI && .venv\Scripts\activate.bat && py .venv\app_files\spaghetti_ai_script.py


        That will launch the web interface.

        When done, you will get a message like this:

            Running on local URL:  http://127.0.0.1:7860

        If your browser did not open a web browser, visit the link that was in
        the command prompt. If it was not the one above, note it. However, if
        you have something like Automatic1111 open already, it will use a
        different port, like "7861".

        It might be this:

            http://127.0.0.1:7860

        To download a lot of data at once rather than waiting until you use each
        model, you can add this at the end of the URL that opens after your
        browser opens:

            ?download_data=1

        It will download dozens of gigabytes of data, so you may not want to do
        that. A "1" will download only the default model configuration for each
        main model, as well as refiner and upscaler data. A "2" will download
        all model data needed in "model_configuration_links_object", including
        online configurations you may not want.

        Or you can just download it as you use each model.

        Downloading all the model data will mean downloading dozens of gigabytes
        of data.

        Model files are downloaded to:

            C:/Spaghetti_AI/model_data

        You should know that this folder can grow considerably. You need to
        manually clean it out when you need to get rid of old models.
        Occasionally, new versions will be downloaded when you create an image.
        O(unless you turned off the ability to download model data) When that
        Ohappens, you may want to remove the old version if the new version
        Oworks. Otherwise, you could have 5 to 15 gigabytes of space added each
        Otime you use a model that updates.

        You should also know that temporary images created are likely stored
        Ohere:

            %USERPROFILE%/AppData/Local/Temp/gradio

        Those however are likely cleaned out automatically by your computer.

        A package called Triton apparently can't be installed on Windows so you
        will see an error in the Command Prompt go by about that every time you
        launch the web interface. It doesn't seem to impact anything when using
        a GPU on Windows.

            https://github.com/openai/triton/issues/1057
            https://github.com/openai/triton/issues/1640

        Someone offers something there that might work, but I didn't want to
        install something from someone randomly. Again, it seems to work without
        it when using a GPU. If trying to use a CPU using Windows, that might be
        what prevents the app from being able work.


--------------------


    Step 12:

        When you open the link, you'll see the display to create the imagery.
        You must leave the command prompt open. When you process an image, the
        command prompt will tell you the progress of what you are creating and
        approximately how long it will take for the step it is on. It will take
        longer as there will be things that happen for which there is no
        progress indicator, either in the command prompt or the progress shown
        in the browser.


--------------------


    Step 13:

        To create a shortcut to the command that needs to be launched every
        time, use:

            spaghetti_ai_launcher.bat

        You can then create a shortcut to that, which you could add to the
        desktop.

        You can follow the instructions below or use the shortcut that comes
        with this app:

            Spaghetti AI.lnk

        Right click on your desktop and then click on "New > Shortcut".

        For "Type the location of the item", enter:

cmd

        Then click "Next". You can name the shortcut whatever you want, like
        "Spaghetti AI".

        Then click "Finish".

        Then right click on the shortcut you created and click "Properties".

        In "Target", replace what is there with the following:

C:\Windows\System32\cmd.exe /k C:/Spaghetti_AI/.venv/app_files/spaghetti_ai_launcher.bat

        That will execute that file each time you click on it.

        If for some reason that doesn't work sometimes, and rather than get the
        program to launch it simply opens the command prompt, then you must
        enter this each time in the command prompt:


cd C:/Spaghetti_AI && .venv\Scripts\activate.bat && py .venv\app_files\spaghetti_ai_script.py


        You might also not be able to use a shortcut. If so, just copy
        "spaghetti_ai_launcher.bat" and put it on your desktop and name it what
        you want. The downside, if you do that, would be reconfiguring your
        firewall to use it from that location too. It's best to create a
        shortcut.


--------------------


    File List

        You can add all of these files to:
            C:\Spaghetti_AI\.venv\app_files
        Although some of these files are not needed.


        - Instructions.txt

              This file. Instructions on how to install the app.

        - README.md
        - requirements.txt

              These files are for use on HuggingFace. They are not otherwise
              needed.

        - spaghetti_ai_install.txt

              You should avoid using this. I wanted to create an easier way to
              install it, but it's best to read the instructions.

              "C:/Spaghetti_AI" needs to be set in "main_dir" in this file if
              you change the path and use this file.

        - spaghetti_ai_launcher.bat

              This is the file that launches "spaghetti_ai_script.py". This sets
              some variables that need to be set there, like disabling the
              automatic downloading of model data and turning off telemetry. The
              shortcuts in this section should point to this.

              "C:/Spaghetti_AI" needs to be set in "main_dir" in this file if
              you change the path.

        - spaghetti_ai_script.py

              The main script.

              "C:/Spaghetti_AI" needs to be set in "main_dir" in this file if
              you change the path. If "only_use_local_files" is set to "1", you
              will need to change it to download model data.

        - Spaghetti AI (no icon).lnk
        - Spaghetti AI.lnk
        - spaghetti_ai_icon_for_shortcut.ico

              These files are simply to provide a shortcut to the app. I created
              a .ico file to have a nice icon that you can use with the
              shortcut. Using a custom icon in a shortcut might not work, unless
              you do it yourself, so I also included one that does not have a
              custom icon. In Windows, you can right click on the shortcut and
              then click "Properties". On the "Shortcut" tab you can then click
              "Change Icon". You can then browse to the location of the icon
              provided or another. If you change the path from
              "C:/Spaghetti_AI", the shortcut will have to be edited in the
              "Target" field of the "Shortcut" tab.


--------------------


    Various Information

        You get this error:

            "CUDA out of memory. Tried to allocate 82.00 MiB. GPU 0 has a total
            capacty of 12.00 GiB of which 0 bytes is free. Of the allocated
            memory 16.97 GiB is allocated by PyTorch, and 439.95 MiB is reserved
            by PyTorch but unallocated. If reserved but unallocated memory is
            large try setting max_split_size_mb to avoid fragmentation. See
            documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF"

        Try setting these to 1 in spaghetti_ai_script.py:

use_sequential_cpu_offload_for_base_model = 1
use_sequential_cpu_offload_for_refiner = 1
use_sequential_cpu_offload_for_upscaler = 1