<jupyter_start><jupyter_text>EIEFollowing are some represenations of my understanding of the EIE fully-connected layer acceleratator.First, include some libraries<jupyter_code># Run boilerplate code to set up environment

%run ../prelude.py<jupyter_output><empty_output><jupyter_text>Read matricesSince EIE works on fully-connected layers, the input (for a single input channel) is a flattened dimension CHW. The weights (CRS) are the same size as the inputs so they have shape [M, CHW].<jupyter_code>i = Tensor.fromUncompressed(["CHW"], 
                            [ 1, 2, 0, 3, 4, 0, 0, 7, 8])
i.setColor("blue").setName("I")

f = Tensor.fromUncompressed(["M", "CHW"],
                            [[ 1, 2, 0, 3, 4, 0, 0, 7, 8],
                            [ 0, 1, 0, 0, 8, 0, 0, 7, 0],
                            [ 0, 0, 0, 0, 0, 0, 0, 0, 0],
                            [ 1, 0, 0, 3, 0, 0, 0, 2, 0],
                            [ 0, 0, 0, 2, 0, 0, 0, 4, 0],
                            [ 0, 0, 0, 0, 0, 0, 0, 0, 0],
                            [ 2, 4, 0, 6, 0, 0, 0, 14, 16],
                            [ 2, 0, 0, 2, 0, 0, 3, 0, 8]
                            ])
f.setColor("green").setName("F")

o_verify = Tensor.fromUncompressed(["M"],
                                   [143, 83, 0, 24, 34, 0, 254, 72])
o_verify.setName("O-Verify")


displayTensor(i)
displayTensor(f)
displayTensor(o_verify)<jupyter_output><empty_output><jupyter_text>Fully connected - NaiveAn input stationary dataflow for a fully-connected computation, which is the base dataflow of the EIE design<jupyter_code>swapped_f = f.swapRanks()

o = Tensor(rank_ids=["M"])

canvas = createCanvas(i, swapped_f, o)

i_chw = i.getRoot()
f_chw = swapped_f.getRoot()
o_m = o.getRoot()

for chw, (i_val, f_m) in i_chw & f_chw:    
    for m, (o_ref, f_val) in  o_m << f_m:
        o_ref += i_val * f_val
        canvas.addFrame((chw,), (chw, m), (m,))

displayTensor(o)
displayCanvas(canvas)<jupyter_output><empty_output><jupyter_text>Check result<jupyter_code>o_verify == o<jupyter_output><empty_output><jupyter_text>EIE DataflowThe EIE design splits the filter weights between multiple PEs, which result in a filter weight tensor with rank ids of M1, CHW and M0, where the M1 rank is the paritioning of output channel filter weights between PEs. Following is the transform of the original weights into the form used by a 2 PE instantiation of EIE.<jupyter_code>PEs = 2
split_size = (len(f.getRoot())+1)//PEs

eie_f = f.splitEqual(split_size).swapRanks(depth=1)
displayTensor(eie_f)

eie_o_verify = o_verify.splitEqual(split_size)
eie_o_verify
displayTensor(eie_o_verify)<jupyter_output><empty_output><jupyter_text>EIE DataflowFollowing is an EIE-like dataflow, but with the parallel units running serially. So with two PEs first one runs then the other.<jupyter_code>o = Tensor(rank_ids=["M1", "M0"]).setName("O")

canvas = createCanvas(i, eie_f, o)

i_chw = i.getRoot()
f_m1 = eie_f.getRoot()
o_m1 = o.getRoot()

for chw, i_val in i_chw:
    for m1, (o_m0, f_chw) in o_m1 << f_m1:   # parallel
        f_m = f_chw.getPayload(chw)
        for m, (o_ref, f_val) in o_m0 << f_m:
            o_ref += i_val * f_val
            canvas.addFrame((chw,), (m1, chw, m), (m1, m))

displayTensor(o)
displayCanvas(canvas)<jupyter_output><empty_output><jupyter_text>Check result<jupyter_code>displayTensor(eie_o_verify)
displayTensor(o)
eie_o_verify.getRoot() == o.getRoot()<jupyter_output><empty_output><jupyter_text>EIE parallel dataflowFollowing is a representation of the EIE-like dataflow with parallel workers. The current cycle in each PE is tracked with the CycleManager class.<jupyter_code>o = Tensor(rank_ids=["M1", "M0"]).setName("O")

canvas = createCanvas(i, eie_f, o)

i_chw = i.getRoot()
f_m1 = eie_f.getRoot()
o_m1 = o.getRoot()

cycle = CycleManager()

for chw, i_val in i_chw:

    cycle.startParallel()
    
    for pe, (m1, (o_m0, f_chw)) in enumerate(o_m1 << f_m1):   # parallel
        cycle.startWorker()
        
        f_m = f_chw.getPayload(chw)
        for m, (o_ref, f_val) in o_m0 << f_m:
            o_ref += i_val * f_val
            canvas.addActivity(
                        (chw,), (m1, chw, m), (m1, m),
                        worker=f"PE{pe}",
                        skew=cycle())

        cycle.finishWorker()

    cycle.finishParallel()
    
displayTensor(o)
displayCanvas(canvas)<jupyter_output><empty_output><jupyter_text>Check result<jupyter_code>displayTensor(eie_o_verify)
displayTensor(o)
eie_o_verify.getRoot() == o.getRoot()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1267543859649123, 'fraction_numerical': 0.028070175438596492, 'mean_word_length': 3.0650623885918002, 'pattern_counts': {'":': 0, '<': 47, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '457335', 'license': 'MIT', 'n_tokens_mistral': 1814, 'n_tokens_neox': 1688, 'n_words': 438, 'path': 'notebooks/sparse-dnn/eie.ipynb', 'repo_name': 'FPSG-UIUC/fibertree-notebooks', 'size': '4560'}