<jupyter_start><jupyter_text>Determine the latencies and bandwidths quantitatively<jupyter_code>import numpy as np
from collections import OrderedDict as odict
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.cm as cm<jupyter_output><empty_output><jupyter_text>This notebook computes the latencies and bandwidths of the three primitive function types. Again, start with a list of files and architectures that we want to investigate.<jupyter_code>#(hardware name, number of nodes)
files = odict({})
files['i5'] = ('i5',1)
files['gtx1060'] = ('gtx1060',1)
files['skl_mpi1'] = ('skl',1)
files['skl_mpi2'] = ('skl',2)
files['skl_mpi4'] = ('skl',4)
files['knl_mpi1'] = ('knl',1)
files['knl_mpi2'] = ('knl',2)
files['knl_mpi4'] = ('knl',4)
files['p100nv_mpi1'] = ('p100',1)
files['p100nv_mpi2'] = ('p100',2)
files['p100nv_mpi4'] = ('p100',4)
files['titanxp'] = ('titanXp',1)
files['v100nv_mpi1'] = ('v100',1)
files['v100nv_mpi2'] = ('v100',2)
files['v100nv_mpi4'] = ('v100',4)

pd.set_option('precision',2)<jupyter_output><empty_output><jupyter_text>Assumptions- there are three basic functions: trivially parallel(axpby), nearest neighbor (dxdy), global reduction (dot)- each can be represented by the single node bandwidth, the single node latency and the multinode latency But- does not capture cache effect e.g. in SKl First, we read in files and extract runtimes and compute bandwidth for every measurement. We also compute the average runtime and bandwidth for every combination of n, Nx and Ny. Axpby and Dot BandwidthsThe bandwidths are determined by taking the average bandwidth of the 30 bandwidths corresponding to the 3 largest sizes. The error is the standard deviation on these bandwidths. Axpby and Dot LatenciesWe try three different methods to compute the latency. First, we try the mean average runtime for the 3 smallest sizes. Second, we take the minimum of the average runtime among all combinations of n, Nx, and Ny. The error is the standard deviation corresponding to this average runtime. Finally, we use a correction factor from the more accurate bandwidths. Dx-Dy BandwidthsSince the efficiency of the matrix-vector multiplications depends on the polynomial coefficient we should compute these bandwidths separately for each coefficient and use sizes between 10MB and 1000MB Dx-Dy LatenciesAs in Axpby<jupyter_code>names={'axpby':3,'dot':2,'dx':3, 'dy':3}
#ns=[3,4]
values = []
for f, v in files.items() :
    runtimes=pd.read_csv('benchmark_'+f+'.csv', delimiter=' ')
    #add size and bandwidth columns
    runtimes.insert(0,'size', 8*runtimes['n']*runtimes['n']
                    *runtimes['Nx']*runtimes['Ny']/1e6/v[1]) #inplace transformation
    for name,memops in names.items() :
        runtimes.insert(0,name+'_bw',runtimes['size']/1000*memops/runtimes[name])
    runtimes = runtimes.assign( dxdy=(runtimes['dx']+runtimes['dy'])/2)
    runtimes = runtimes.assign( dxdy_bw=2.0*runtimes['dx_bw']*runtimes['dy_bw']
                               /(runtimes['dx_bw']+runtimes['dy_bw']))
    #compute one version with aggregated grouped sizes and one without
    avgruntimes=runtimes.groupby(['n', 'Nx','Ny','size']).agg(['mean', 'std'])
    avgruntimes=avgruntimes.reset_index(level=['n','Nx','Ny','size'])
    avgruntimes.sort_values(by='size',inplace=True) #sort by size
    runtimes.sort_values(by='size',inplace=True)
    ##first compute axpby and dot latencies and bandwidths 
    nmax = 3
    s =30

    line = []
    l=len(runtimes)
    line.append(v[0]) #0
    line.append(v[1]) #1
    for q in ['axpby','dot']:
        bandwidth       = runtimes[l-s:l][q+'_bw'].mean()
        err_bandwidth   = runtimes[l-s:l][q+'_bw'].std()
        mean_latency    = avgruntimes[0:nmax][(q,'mean')].mean()/1e-6
        min_latency     = avgruntimes[(q,'mean')].min()/1e-6
        idx_min_latency = avgruntimes[(q,'mean')].idxmin()
        corr_min_latency= min_latency - avgruntimes['size'].loc[idx_min_latency]*names[q]/bandwidth/1e-3
        err_latency     = avgruntimes[(q,'std')].loc[idx_min_latency]/1e-6
        if corr_min_latency <0 : corr_min_latency = 0 
        line.append(bandwidth) #2 bandwidth
        line.append(err_bandwidth)  #3 err_bandwidth
        line.append(mean_latency) #4 latency mean
        line.append(min_latency)  #5 latency min
        line.append(corr_min_latency ) #6 corrected lat 
        line.append(err_latency ) #6 error lat 
    ##now compute latency and bandwidths of dx and y
    for n in [2,3,4,5]:
        #take n
        dxdy=runtimes[runtimes['n']==n]
        
        avgdxdy = avgruntimes[avgruntimes['n']==n]
        dxdy=dxdy.sort_values(by='size')
        avgdxdy=avgdxdy.sort_values(by='size') #sort by size
        bandwidth       = dxdy[(dxdy['size']>10)&(dxdy['size']<1000)]['dxdy_bw'].mean()
        err_bandwidth   = dxdy[(dxdy['size']>10)&(dxdy['size']<1000)]['dxdy_bw'].std()
        mean_latency    = avgdxdy[0:nmax][('dxdy','mean')].mean()/1e-6
        min_latency     = avgdxdy[('dxdy','mean')].min()/1e-6
        idx_min_latency = avgdxdy[('dxdy','mean')].idxmin()
        corr_min_latency= min_latency - avgdxdy['size'].loc[idx_min_latency]*names['dx']/bandwidth/1e-3
        err_latency     = avgdxdy[('dxdy','std')].loc[idx_min_latency]/1e-6
        if corr_min_latency <0 : 
            corr_min_latency = 0 
        line.append(bandwidth) #2 bandwidth
        line.append(err_bandwidth)  #3 err_bandwidth
        line.append(mean_latency) #4 latency mean
        line.append(min_latency)  #5 latency min
        line.append(corr_min_latency ) #6 corrected lat  
        line.append(err_latency ) #6 error lat     
    values.append(line)<jupyter_output><empty_output><jupyter_text>In the following cells we just display the results in a more accessible way or extract values to show in publications.<jupyter_code>#now construct new more ordered table with values from previous cell      
tuples=[('arch','',''),('nodes','','')]        
for q in ['axpby','dot','dxdy2','dxdy3','dxdy4','dxdy5']:
    tuples.append((q,'bw','avg'))
    tuples.append((q,'bw','std'))
    tuples.append((q,'lat','avg'))
    tuples.append((q,'lat','min'))
    tuples.append((q,'lat','corr'))
    tuples.append((q,'lat','std'))
    
cols=pd.MultiIndex.from_tuples(tuples)
arr = pd.DataFrame(values,index=files.keys(), columns=cols)
arr.sort_values(by='arch',inplace=True)
arr.set_index(['arch','nodes'],inplace=True)
#arr.loc[:,[('dot','bw','avg'),('dot','lat','avg')]]
arr
#arr=arr.reset_index()
#define conversion function for writing nice output tables
def toString(x): 
    if pd.isnull(x) : return 'n/a'
    #string = '%.1f'% x
    string = '%d' %np.ceil(x)
    #if np.ceil(x)<100 : string = '0'+string
    if np.ceil(x)<10 : string = '0'+string
    return string
addto = []
for n in ['axpby','dot','dxdy2','dxdy3','dxdy4','dxdy5']:
    arr.loc[:,(n,'bw','string')]= arr[n]['bw']['avg'].apply(toString) +" ± "+arr[n]['bw']['std'].apply(toString)
    arr.loc[:,(n,'lat','string')]= arr[n]['lat']['corr'].apply(toString) +" ± "+arr[n]['lat']['std'].apply(toString)
    addto.append((n,'lat','string'))
    addto.append((n,'bw','string'))

#make a table for display
nicetable=arr[addto]
drop = nicetable.columns.droplevel(2)
nicetable.columns=drop
#nicetable.reset_index(inplace=True)
#nicetable.set_index('arch')
newindex=[('i5',1)]
newindex.append(('gtx1060',1))
newindex.append(('titanXp',1))
for n in ['skl','knl']:
    for m in [1,2,4]:
        newindex.append((n,m))
for n in ['p100','v100']:
    for m in [1,2,4]:
        newindex.append((n,m))
    
nicetable=nicetable.reindex(newindex)

nicetable
index = ['i5','gtx1060','skl','knl','p100','titanXp','v100']  
lines = []
for arch in  index: 
    line = []
    line.append(arch)
    #first the bandwidths
    for n in ['axpby','dot','dxdy2','dxdy3','dxdy4','dxdy5']:
        line.append( arr.loc[(arch,1),(n,'bw','avg')] )
        line.append( arr.loc[(arch,1),(n,'bw','std')])
    for n in ['axpby','dot','dxdy2'] :
        line.append( arr.loc[(arch,1),(n,'lat','corr')] )
        line.append( arr.loc[(arch,1),(n,'lat','std')])
        if arch == 'i5' or arch == 'gtx1060' or arch == 'titanXp':
            line.append(None)
            line.append(None)
        else:
            line.append( arr.loc[(arch,4),(n,'lat','corr')] )
            line.append( arr.loc[(arch,4),(n,'lat','std')] )
    lines.append(line)
    
tuples=['arch']     

for n in ['axpby','dot','dxdy2','dxdy3','dxdy4','dxdy5']:
    tuples.append(n+'_bw')
    tuples.append(n+'_bw_err')
for n in ['axpby','dot','dxdy']:
    tuples.append(n+'_lat_shared')
    tuples.append(n+'_lat_shared_err')
    tuples.append(n+'_lat_dist')
    tuples.append(n+'_lat_dist_err')
cols=tuples
toDisk = pd.DataFrame(lines, columns=cols)
toDisk.to_csv('performance.csv',sep=' ',index=False)
pd.set_option('display.float_format', lambda x: '%.2f' % x)
test = pd.read_csv('performance.csv',delimiter=' ')
test<jupyter_output><empty_output><jupyter_text>Observations- note the high latency in the knl MPI implementation of dxdy. It seems to suffer from the same problem as the GPUs. (Is this the speed of PCIe we see?)<jupyter_code>#index = ['i5','gtx1060','titanXp', 'skl','knl','p100','v100']  

lines = []
for arch in  index: 
    line = []        
    for n in ['axpby','dot','dxdy2','dxdy3','dxdy4','dxdy5']:
        bw  = arr.loc[(arch,1),(n,'bw','avg')]
        bw_err = arr.loc[(arch,1),(n,'bw','std')]
        lat1 = arr.loc[(arch,1),(n,'lat','corr')]
        lat1_err = arr.loc[(arch,1),(n,'lat','std')]
        line.append( toString( bw)+" $\pm$ "+toString(bw_err))
        line.append( toString( lat1)+" $\pm$ "+toString(lat1_err) )
        if arch == 'i5' or arch == 'gtx1060' or arch == 'titanXp':
            line.append(toString(None))
        else:
            if (n == 'dot') or (n == 'axpby'):
                lat4 = arr.loc[(arch,4),(n,'lat','corr')]
                lat4_err = arr.loc[(arch,4),(n,'lat','std')]
            else:
                lat4 = arr.loc[(arch,4),('dxdy2','lat','corr')]
                lat4_err = arr.loc[(arch,4),('dxdy2','lat','std')]  
            line.append( toString( lat4)+" $\pm$ "+toString(lat4_err))

                
    lines.append(line)
    
tuples=[]  


for p in ['axpby','dot','B(P=2)','B(P=3)','B(P=4)','B(P=5)']:
    #for q in ['efficiency [\% bw]','lat s [us]','lat d [us]']:
    for q in ['bandwidth [GB/s]','$T_{lat}(1)$ [$\mu$s]','$T_{lat}(4)$ [$\mu$s]']:
        tuples.append((p,q))
tuples[0] = ('axpby','bandwidth [GB/s]')
    

cols=pd.MultiIndex.from_tuples(tuples)

toDisk = pd.DataFrame(lines, index=index, columns=cols)
theo = [34,192,256,'>400',732,547,898]
toDisk.insert(0,('peak','bandwidth [GB/s]'),theo)
pd.set_option('display.float_format', lambda x: '%.0f' % x)
filename='axpby-dot.tex'
with open(filename, 'wb') as f:
    f.write(bytes(toDisk.iloc[:,0:7].to_latex(
        escape=False,
        column_format='@{}lp{1.5cm}p{1.5cm}p{1.2cm}p{1.2cm}p{1.5cm}p{1.2cm}p{1.4cm}@{}',
        bold_rows=True),'UTF-8'))
toDisk.iloc[:,0:7]
dxdy = toDisk.loc[:,[('B(P=2)','bandwidth [GB/s]'),
                     ('B(P=3)','bandwidth [GB/s]'),
                     ('B(P=4)','bandwidth [GB/s]'),
                     ('B(P=5)','bandwidth [GB/s]'),
                     ('B(P=2)','$T_{lat}(1)$ [$\mu$s]'),
                     ('B(P=2)','$T_{lat}(4)$ [$\mu$s]'),
                    ]]
dxdy.columns = ['B(P=2) [GB/s]','B(P=3) [GB/s]','B(P=4) [GB/s]','B(P=5) [GB/s]',
                         '$T_{lat}(1)$ [$\mu$s]','$T_{lat}(4)$ [$\mu$s]']

filename='dxdy.tex'
with open(filename, 'wb') as f:
    f.write(bytes(dxdy.to_latex(
        escape=False,
        column_format='lp{1.5cm}p{1.5cm}p{1.5cm}p{1.5cm}p{1.2cm}p{1.2cm}',
        bold_rows=True),'UTF-8'))
dxdy<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.19450820368953498, 'fraction_numerical': 0.029244240414860154, 'mean_word_length': 3.8813278008298755, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '384280', 'license': 'MIT', 'n_tokens_mistral': 4568, 'n_tokens_neox': 4357, 'n_words': 845, 'path': '4-latencies-and-bandwidths.ipynb', 'repo_name': 'mwiesenberger/performance', 'size': '11763'}