<jupyter_start><jupyter_text>*** Python Lesson ILM & Escape Studios [<NAME>](https://www.pearsoncollegelondon.ac.uk/escape-studios/meet-your-tutors/allar-kaasik.html) - Senior VFX Tutor. Based on code samples by <NAME> *** Compositing exerise  Getting Started - Click or double click on a block of code (or text or image, any block) to activate it for editing - Then press the "Run" ▶ button above to "run" that block - For example edit the phrase `print("Hello World")` in the next block to say `print("Hello Blairgowrie")` - When you run ▶ the block it will output `Hello Blairgowrie` below the block<jupyter_code>print("Hello World")<jupyter_output>Hello World<jupyter_text>*** Let's read in the required libraries first<jupyter_code>from PIL import Image                     # Lets us import images
from matplotlib.pyplot import imshow      # Lets us display images
import numpy as np                        # Lets us operate on images<jupyter_output><empty_output><jupyter_text>*** A quick reminder of the Over functionWe'll leave defining the function for just a bit later to go through the steps of the Over operation first1. **Multiply the matte and FG** together to remove undesired parts of the FG image2. **Invert your matte**3. Use the **inverted matte and multiply it with the BG** to make a hole in it4. **Add** the "**premultiplied FG**" together with the "**premultiplied BG**" *** Let's define the functions we need Inverting images (per pixel and per channel): `1 - img`<jupyter_code>def imgInvert(input_image,x,y):
    out = Image.new('RGB', (x, y))
    for xPos in range(x):
        for yPos in range(y):
            fPxlr, fPxlg, fPxlb = input_image.getpixel((xPos,yPos))
            rOutF = 1 - (float(fPxlr)/256)
            gOutF = 1 - (float(fPxlg)/256)
            bOutF = 1 - (float(fPxlb)/256)
            rOutI = int(rOutF*256)
            gOutI = int(gOutF*256)
            bOutI = int(bOutF*256)
            out.putpixel((xPos, yPos), (rOutI, gOutI, bOutI))
    return out<jupyter_output><empty_output><jupyter_text>Adding images (per pixel and per channel): `img_1 + img_2`<jupyter_code>def compAdd(front,back,x,y):
    out = Image.new('RGB', (x, y))
    for xPos in range(x):
        for yPos in range(y):
            fPxlr, fPxlg, fPxlb = front.getpixel((xPos,yPos))
            bPxlr, bPxlg, bPxlb = back.getpixel((xPos,yPos))
            rOutF = (float(fPxlr)/256) + (float(bPxlr)/256)
            gOutF = (float(fPxlg)/256) + (float(bPxlg)/256)
            bOutF = (float(fPxlb)/256) + (float(bPxlb)/256)
            rOutI = int(rOutF*256)
            gOutI = int(gOutF*256)
            bOutI = int(bOutF*256)
            out.putpixel((xPos, yPos), (rOutI, gOutI, bOutI))
    return out  ### Credit for these comp functions goes to <NAME><jupyter_output><empty_output><jupyter_text>Multiplying images (per pixel and per channel): `img_1 * img_2`<jupyter_code>def compMult(front,back,x,y):
    out = Image.new('RGB', (x, y))
    for xPos in range(x):
        for yPos in range(y):
            fPxlr, fPxlg, fPxlb = front.getpixel((xPos,yPos))
            bPxlr, bPxlg, bPxlb = back.getpixel((xPos,yPos))
            rOutF = (float(fPxlr)/256) * (float(bPxlr)/256)
            gOutF = (float(fPxlg)/256) * (float(bPxlg)/256)
            bOutF = (float(fPxlb)/256) * (float(bPxlb)/256)
            rOutI = int(rOutF*256)
            gOutI = int(gOutF*256)
            bOutI = int(bOutF*256)
            out.putpixel((xPos, yPos), (rOutI, gOutI, bOutI))
    return out<jupyter_output><empty_output><jupyter_text>Compositing images (per pixel and per channel): `img_1 * matte + (1 - matte) * img_2`<jupyter_code>def compOver(front,back,matte,x,y):
    out = Image.new('RGB', (x, y))
    premult_FG = compMult(front, matte, x, y)
    inverted_matte = imgInvert(matte,x,y)
    premult_BG = compMult(back, inverted_matte, x, y)
    out = compAdd(premult_FG, premult_BG, x, y)
    return out
def compOverFaster(front,back,matte,x,y):
    out = back #out will be our output image. We set it to be the BG for now. Every pixel will be overwritten though, as the function runs
    for xPos in range(x):
        for yPos in range(y):#These two lines will iterate over every pixel in the image. The compositing algorithm will be run for every pixel to compute the final result

            fPxlr, fPxlg, fPxlb = front.getpixel((xPos,yPos))#get the R, G, and B values for the FG image. These will be 8-bit integer values.
            bPxlr, bPxlg, bPxlb = back.getpixel((xPos,yPos))#get the R, G, and B values for the BG image. These will be 8-bit integer values.
            mPxlr, mPxlg, mPxlb = matte.getpixel((xPos,yPos))#get the R, G, and B values for the matte image. In truth since the matte is monochrome we only need one channel.  These will be 8-bit integer values.

            #Next we compute (FG * MATTE) + (BG * inverse MATTE) which is Porter and Duff's over compositing algorithm. We need to convert the 8-bit values into normalised floating point values so the pixel values range from 0.0-1.0, rather than 0-255 as they do in an 8-bit image. The compositing maths only works if we do this conversion. We do this calulation for each channel: R, G, and B

            rOutF = (float(fPxlr)/256) * (float(mPxlr)/256) + (float(bPxlr)/256) * (1-(float(mPxlr)/256))
            gOutF = (float(fPxlg)/256) * (float(mPxlg)/256) + (float(bPxlg)/256) * (1-(float(mPxlg)/256))
            bOutF = (float(fPxlb)/256) * (float(mPxlb)/256) + (float(bPxlb)/256) * (1-(float(mPxlb)/256))

            #Because Pillow is expecting an 8-bit image output, we now convert our normalised floating point composite results back to 8-bit integer values 

            rOutI = int(rOutF*256)
            gOutI = int(gOutF*256)
            bOutI = int(bOutF*256)

            #We now over-write the current pixel being worked on in our output image with our computed values

            out.putpixel((xPos, yPos), (rOutI, gOutI, bOutI))

    return out<jupyter_output><empty_output><jupyter_text>*** Read in the imagesMake sure to upload the file path from "provided_images/..." to "uploaded_images/..." to access your uploaded imagesBG = Image.open('provided_images/seekerBG.png')FG = Image.open('provided_images/seekerRobot.png')matte_image = Image.open('provided_images/seekerMatte.png')<jupyter_code>BG = Image.open('provided_images/seekerBG.png')
FG = Image.open('provided_images/seekerRobot.png')
matte_image = Image.open('provided_images/seekerMatte.png')<jupyter_output><empty_output><jupyter_text>(Optional:) You can then display them to veryify that you've got the right images (don't forget to uncomment the correct lines)<jupyter_code>%matplotlib inline
#imshow(np.asarray(BG))
%matplotlib inline
#imshow(np.asarray(FG))
%matplotlib inline
#imshow(np.asarray(matte_image))<jupyter_output><empty_output><jupyter_text>*** Set the image bounds and sizeAll the images need to be the same size for our operations to work. We're resizing to HD 1920x1080 for simplicity using the `PIL.Image.resize((size))` function. Note that if your original image had a different aspect ratio, then it will squeeze the result. Also note that there are two set's of brackets in `.resize((size))` because `size` is a tuple of two values.<jupyter_code>width = 1920
height = 1080
BG_resized = BG.resize((width,height))
FG_resized = FG.resize((width,height))
matte_resized = matte_image.resize((width,height))<jupyter_output><empty_output><jupyter_text>(Optional:) You can then display them to veryify that you've got the right images (don't forget to uncomment the correct lines)<jupyter_code>%matplotlib inline
#imshow(np.asarray(BG_resized))
%matplotlib inline
#imshow(np.asarray(FG_resized))
%matplotlib inline
#imshow(np.asarray(matte_resized))<jupyter_output><empty_output><jupyter_text>*** The Over operation, step-by-step  Multiply the matte and the FG<jupyter_code>premult_FG = compMult(FG_resized, matte_resized, width, height)
%matplotlib inline
#imshow(np.asarray(premult_FG))<jupyter_output><empty_output><jupyter_text>In order to save your image you can use the `Image.save("filepath/filename")` function from the PIL library. To keep things tidy, output them to the "output" folder<jupyter_code>#premult_FG.save('output/premult_FG_out.png')<jupyter_output><empty_output><jupyter_text>Invert the matte<jupyter_code>inverted_matte = imgInvert(matte_resized,width,height)
%matplotlib inline
#imshow(np.asarray(inverted_matte))
#inverted_matte.save('output/inverted_matte_out.png')<jupyter_output><empty_output><jupyter_text>Multiply the inverted matte and the BG<jupyter_code>premult_BG = compMult(BG_resized, inverted_matte, width, height)
%matplotlib inline
#imshow(np.asarray(premult_BG))
#premult_BG.save('output/premult_BG_out.png')<jupyter_output><empty_output><jupyter_text>Add the "premultiplied FG" and "premultiplied BG"<jupyter_code>out = compAdd(premult_FG, premult_BG, width, height)
%matplotlib inline
#imshow(np.asarray(out))
#out.save('output/comp_out.png')<jupyter_output><empty_output><jupyter_text>*** Over operation - without separating the stepsIf you don't need to save the intermediate steps, but just want to comp two images using matte you can use the `compOverFaster()` function. (The PIL library actually comes with it's own PIL.Image.composite() function as well, but we wanted to keep the code completely visible to you)<jupyter_code>BG = Image.open('provided_images/seekerBG.png')
FG = Image.open('provided_images/seekerRobot.png')
matte_image = Image.open('provided_images/seekerMatte.png')
width = 1920
height = 1080
BG_resized = BG.resize((width,height))
FG_resized = FG.resize((width,height))
matte_resized = matte_image.resize((width,height))
straight_out = compOverFaster(FG, BG, matte_image ,  width, height)
%matplotlib inline
#imshow(np.asarray(straight_out))
#out.save('output/straight_comp_out.png')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11580119445287984, 'fraction_numerical': 0.017613118736714242, 'mean_word_length': 4.2497343251859725, 'pattern_counts': {'":': 0, '<': 67, '<?xml version=': 0, '>': 67, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '601280', 'license': 'MIT', 'n_tokens_mistral': 3628, 'n_tokens_neox': 3405, 'n_words': 958, 'path': 'Exercise_Session.ipynb', 'repo_name': 'allarkaasik/PythonLessonILM', 'size': '9907'}