First things first: no knowledge of either slurm or Infiniband is required - this is a purely text processing problem.Second - I'm aware of ib2slurm - the code is somehow broken and quite possibly outdated - it core dumps each time it runs regardless of the existence or format of a map file.I can reduce the output of ibnetdiscover to 37 line blocks each of the form:Switch  36 S-0002c90200423e70     # MF0;ibsw20:SX6036/U1 enhanced port 0 lid 3 lmc 0[1] H-0002c903000c26f2[1](2c903000c26f3)      # compute061 HCA-1 lid 49 4xQDR[2] H-0002c903000bf36e[1](2c903000bf36f)      # compute060 HCA-1 lid 1 4xQDR[3] H-0002c903000bf35a[1](2c903000bf35b)      # compute063 HCA-1 lid 28 4xQDR[4] H-0002c903000c2646[1](2c903000c2647)      # compute062 HCA-1 lid 25 4xQDR[5] H-0002c903000bf35e[1](2c903000bf35f)      # compute064 HCA-1 lid 31 4xQDR[6] H-0002c903000c26de[1](2c903000c26df)      # compute065 HCA-1 lid 47 4xQDR[7] S-0002c90200423e80[31]        # Infiniscale-IV Mellanox Technologies lid 6 4xQDR[8] S-0002c90200423e80[32]        # Infiniscale-IV Mellanox Technologies lid 6 4xQDR[9] S-0002c90200423e80[33]        # Infiniscale-IV Mellanox Technologies lid 6 4xQDR[10]    S-0002c90200423e80[34]        # Infiniscale-IV Mellanox Technologies lid 6 4xQDR[11]    S-0002c90200423e80[35]        # Infiniscale-IV Mellanox Technologies lid 6 4xQDR[12]    S-0002c90200423e80[36]        # Infiniscale-IV Mellanox Technologies lid 6 4xQDR[13]    S-0002c90200423eb8[35]        # Infiniscale-IV Mellanox Technologies lid 11 4xQDR[14]    S-0002c90200423eb8[36]        # Infiniscale-IV Mellanox Technologies lid 11 4xQDR[15]    S-0002c90200423eb8[33]        # Infiniscale-IV Mellanox Technologies lid 11 4xQDR[16]    S-0002c90200423eb8[34]        # Infiniscale-IV Mellanox Technologies lid 11 4xQDR[17]    S-0002c90200423eb8[31]        # Infiniscale-IV Mellanox Technologies lid 11 4xQDR[18]    S-0002c90200423eb8[32]        # Infiniscale-IV Mellanox Technologies lid 11 4xQDR[19]    S-0002c90200423ee0[31]        # Infiniscale-IV Mellanox Technologies lid 15 4xQDR[20]    S-0002c90200423ee0[32]        # Infiniscale-IV Mellanox Technologies lid 15 4xQDR[21]    S-0002c90200423ee0[33]        # Infiniscale-IV Mellanox Technologies lid 15 4xQDR[22]    S-0002c90200423ee0[34]        # Infiniscale-IV Mellanox Technologies lid 15 4xQDR[23]    S-0002c90200423ee0[35]        # Infiniscale-IV Mellanox Technologies lid 15 4xQDR[24]    S-0002c90200423ee0[36]        # Infiniscale-IV Mellanox Technologies lid 15 4xQDR[25]    H-0002c903000c26fa[1](2c903000c26fb)      # compute046 HCA-1 lid 112 4xQDR[26]    H-0002c903000c26e2[1](2c903000c26e3)      # compute047 HCA-1 lid 63 4xQDR[27]    H-0002c903000c263a[1](2c903000c263b)      # compute048 HCA-1 lid 59 4xQDR[28]    H-0002c903000c27c2[1](2c903000c27c3)      # compute049 HCA-1 lid 117 4xQDR[29]    H-0002c903000c27a6[1](2c903000c27a7)      # compute051 HCA-1 lid 34 4xQDR[30]    H-0002c903000c2732[1](2c903000c2733)      # compute050 HCA-1 lid 22 4xQDR[31]    H-0002c903000c265e[1](2c903000c265f)      # compute052 HCA-1 lid 29 4xQDR[32]    H-0002c903000c266a[1](2c903000c266b)      # compute055 HCA-1 lid 32 4xQDR[33]    H-0002c903000c264e[1](2c903000c264f)      # compute054 HCA-1 lid 26 4xQDR[34]    H-0002c903000c26ee[1](2c903000c26ef)      # compute056 HCA-1 lid 48 4xQDR[35]    H-0002c903000bf246[1](2c903000bf247)      # compute057 HCA-1 lid 33 4xQDR[36]    H-0002c903000c27ca[1](2c903000c27cb)      # compute053 HCA-1 lid 44 4xQDRand can extract the node name, e.g. compute061 using awk or sed.  I would like to get a single row for each block starting with switch name followed by node names, i.e:ibsw20 compute061 compute060 compute063 compute062 compute064 compute065 compute046 compute047 compute048 compute049 compute051 compute050 compute052 compute055 compute054 compute056 compute057 compute053I plan to use slurm's scontrol show hostlist <nodename> <nodename> ... to compress several nodes into a single entity to push into slurm's topology.conf file which must have the form:SwitchName=ibsw20 Nodes=compute[046-057,060-061]Any ideas?I should mention after all the switch mappings, the ibnetdiscover file continues with the reverse - a node-by-node mapping to switches, in the form:vendid=0x2c9devid=0x673csysimgguid=0x2c903000bf371caguid=0x2c903000bf36eCa  1 H-0002c903000bf36e      # compute060 HCA-1[1](2c903000bf36f)  S-0002c90200423e70[2]     # lid 1 lmc 0 MF0;ibsw20:SX6036/U1 lid 3 4xQDREach block separated by empty lines.A reduced question that can get me started - how do I parse several lines of text into a single row, extracting different parts of each row (treating header and body rows differently) and discarding rows which do not contain relevant data?EDIT:The blocks might not be full - if nothing is connected to some of the ports in some of the switches, then the output will skip those line, and can result in something like:Switch  36 S-0002c90200423e70     # MF0;ibsw20:SX6036/U1 enhanced port 0 lid 3 lmc 0[2] H-0002c903000bf36e[1](2c903000bf36f)      # compute060 HCA-1 lid 1 4xQDR[3] H-0002c903000bf35a[1](2c903000bf35b)      # compute063 HCA-1 lid 28 4xQDR[4] H-0002c903000c2646[1](2c903000c2647)      # compute062 HCA-1 lid 25 4xQDR[15]    S-0002c90200423eb8[33]        # Infiniscale-IV Mellanox Technologies lid 11 4xQDR[33]    H-0002c903000c264e[1](2c903000c264f)      # compute074 HCA-1 lid 26 4xQDR[34]    H-0002c903000c26ee[1](2c903000c26ef)      # compute076 HCA-1 lid 48 4xQDR So I can't simply rely on there being 36 lines following each switch line or that [36] will always be the last line in a switch block. 
Text processing - Building a slurm topology.conf file from ibnetdiscover output
bash;text processing;slurm;infiniband
Q1This awk command extract a sorted list of unique computer names from the file, assuming:The source file is much longer, having a block of lines for each switch.An script to get a whole switch block (assuming the switch line is always the first line of a continuous set of lines for each switch) sorted and removing repeated nodes is:awk -v FS='[#]' '    BEGIN{c=0}    $1~/Switch/     {c++; j=0; split($5,arr,[;:] ); sw[c,0]=arr[2] }    $1~/\[[0-9]+\]/ {     j++; split($5,arr,     ); sw[c,j]=arr[1] }    END {            print(final count of switches= c)            for (i=1; i<=c; i++) {                print( switch= i, sw[i,0] )     # show switch number.                split(, out , : )             # delete array out.                split(, indices , : )         # delete array indices.                j=0                while (sw[i,++j]) {               # for all array elements.                    if (out[sw[i,j]]++ < 1) {     # Is it a new value?                        indices[sw[i,j]]=j        # add to array indices.                    }                }                n=asorti(indices)                 # sort the keys of indices                printf( %s , sw[i,0] )                for (k=1; k<=n; k++) {            # all values for a switch.                    printf( %s , indices[k] )                }                printf( %s\n,  )            }    }    ' infileResults: final count of switches=3switch=1 ibsw20ibsw20 Infiniscale-IV compute060 compute061 compute062 compute063compute064 compute065 compute066 compute067 compute068 compute069compute070 compute071 compute072 compute073 compute074 compute075compute076 compute077 switch=2 ibsw21ibsw21 Infiniscale-IV compute060 compute061 compute062 compute063compute064 compute065 compute066 compute067 compute068 compute069compute070 compute071 compute072 compute073 compute074 compute075compute076 compute077 switch=3 ibsw22ibsw22 Infiniscale-IV compute060 compute062 compute063 compute074 compute076I am not sure if Infiniscale-IV should be removed, and if you are also asking for the additional processing needed to get: SwitchName=ibsw20 Nodes=compute[060-077] Q2From the man awk:If RS is set to the null string, then records are separated by blank lines.That is the record separator (RS) set to null:awk -v RS='' 'script to process lines' file