Q:

awk or Perl to modify data

using the below awk and Perl solutions: I need to understand how to add additional field that don't repeat their output much like "name" in these examples.
ie devtwr1 only occurs once in the sample data, If i need to add additional fields that also only occur once, how is this done?
Sorry Tried extensively but can't work it out... 
here is the source:
have a file: Ntab.txt which is a sample file of two hosts, there is numerous hosts in the real file one after each other.
Each host has multiple "displayName"s (addresses) with corresponding figures for each 'displayName'.
>cat Ntab.txt
name    devtwr1
displayName     00:67:BB
capacityInKB    104,857,600
consumedCapacityInKB    4,042,752
dpPoolID        20
displayName     00:7B:FD
capacityInKB    52,428,800
consumedCapacityInKB    14,880,768
dpPoolID        10
displayName     00:7C:28
capacityInKB    34,179,712
consumedCapacityInKB    29,804,544
dpPoolID        20
displayName     00:7C:29
capacityInKB    34,179,712
consumedCapacityInKB    5,462,016
dpPoolID        20
name    devtwr2
displayName     00:67:BB
capacityInKB    104,857,600
consumedCapacityInKB    4,042,752
dpPoolID        20
displayName     00:7B:FD
capacityInKB    52,428,800
consumedCapacityInKB    14,880,768
dpPoolID        10
displayName     00:7C:28
capacityInKB    34,179,712
consumedCapacityInKB    29,804,544
dpPoolID        20
displayName     00:7C:29
capacityInKB    34,179,712
consumedCapacityInKB    5,462,016
dpPoolID        20

I need to be able to produce the $2 data from the rows into columns after each 'name' (host) and in csv type format, headings optional. I cant use (,:) as separators as the data includes them so either (tab or ;).
Like:
name;displayName;capacityInKB;consumedCapacityInKB;dpPoolID        
devtwr1;00:67:BB;104,857,600;4,042,752;20
devtwr1;00:7B:FD;52,428,800;14,880,768;10
devtwr1;00:7C:28;34,179,712;29,804,544;20
devtwr1;00:7C:29;34,179,712;5,462,016;20
devtwr2;00:67:BB;104,857,600;4,042,752;20
devtwr1;00:7B:FD;52,428,800;14,880,768;10
devtwr1;00:7C:28;34,179,712;29,804,544;20
devtwr1;00:7C:29;34,179,712;5,462,016;20

Here is a Awk solution but cant work out how to add additional field like "name" that dosn't have data in all fields.
$ awk '$1=="name"{name=$2}
    $1 ~/^(displayName|capacityInKB|consumedCapacityInKB)$/{out=out";"$2} 
    $1=="dpPoolID"{print name out";"$2; out=""}' tmp2.txt
devtwr1;00:67:BB;104,857,600;4,042,752;20
devtwr1;00:7B:FD;52,428,800;14,880,768;10
devtwr1;00:7C:28;34,179,712;29,804,544;20
devtwr1;00:7C:29;34,179,712;5,462,016;20
devtwr2;00:67:BB;104,857,600;4,042,752;20
devtwr2;00:7B:FD;52,428,800;14,880,768;10
devtwr2;00:7C:28;34,179,712;29,804,544;20
devtwr2;00:7C:29;34,179,712;5,462,016;20

And here is a Perl solution but once again couldn't work out how to add additional fields like the "name" field that dosn't always have data.
perl -lane '
BEGIN {
  @fields = qw(name displayName capacityInKB consumedCapacityInKB dpPoolID);
  print join ";", @fields;
}
if (/^(name|displayName)/ && $data{displayName}) {
  print join ";", @data{@fields};
  %data = ( name => $data{name} );
}
$data{$F[0]} = $F[1];
END {
  print join ";",@data{@fields};
}' tmp2.txt > Report.csv

Hope that provides enough info to get a firm understanding.
thanks in advance.

A:

Save this to the ntab.pl file, do chmod +x ntab.pl and run it: cat ntab.txt | ./ntab.pl
#!/usr/bin/perl
use strict;
use constant CSV_SEP=>';';
my ($name,%flds);
my @fl=qw(displayName capacityInKB consumedCapacityInKB dpPoolID);
sub csv4name {
 my ($name,$vals)=@_;
 return 0 unless $name and %$vals;
 print join("\n",map { my $i=$_; join(CSV_SEP,$name,map { $vals->{$_}[$i]||='' } @fl) } 0..$#{$vals->{'displayName'}})."\n";
}
while (my ($attr,$val)=split /\s+/,<>) {
 if ($attr eq 'name') {
  csv4name($name,\%flds);
  $name=$val;
  %flds=();
  next;
 }
 push $flds{$attr}||=[],$val;
}
csv4name($name,\%flds);

Thare are no any performance-optimizations, but with not very large files  (<3G) this scripts will work smoothly.

