This task has been flagged for clarification. Code on this page in its current state may be flagged incorrect once this task has been clarified. See this page's Talk page for discussion.
Often data is produced by one program, in the wrong format for later use by another program or person. In these situations another program can be written to parse and transform the original data into a format useful to the other. The term "Data Munging" is often used in programming circles for this task.

A request on the comp.lang.awk newsgroup led to a typical data munging task:

I have to analyse data files that have the following format:
Each row corresponds to 1 day and the field logic is: $1 is the date,
followed by 24 value/flag pairs, representing measurements at 01:00,
02:00 ... 24:00 of the respective day. In short:

<date> <val1> <flag1> <val2> <flag2> ...  <val24> <flag24>

Some test data is available at: 
... (nolonger available at original location)

I have to sum up the values (per day and only valid data, i.e. with
flag>0) in order to calculate the mean. That's not too difficult.
However, I also need to know what the "maximum data gap" is, i.e. the
longest period with successive invalid measurements (i.e values with
flag<=0)
The data is free to download and use and is of this format:

Data is no longer available at that link. Zipped mirror available here (offsite mirror).

1991-03-30	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1
1991-03-31	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	10.000	1	20.000	1	20.000	1	20.000	1	35.000	1	50.000	1	60.000	1	40.000	1	30.000	1	30.000	1	30.000	1	25.000	1	20.000	1	20.000	1	20.000	1	20.000	1	20.000	1	35.000	1
1991-03-31	40.000	1	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2	0.000	-2
1991-04-01	0.000	-2	13.000	1	16.000	1	21.000	1	24.000	1	22.000	1	20.000	1	18.000	1	29.000	1	44.000	1	50.000	1	43.000	1	38.000	1	27.000	1	27.000	1	24.000	1	23.000	1	18.000	1	12.000	1	13.000	1	14.000	1	15.000	1	13.000	1	10.000	1
1991-04-02	8.000	1	9.000	1	11.000	1	12.000	1	12.000	1	12.000	1	27.000	1	26.000	1	27.000	1	33.000	1	32.000	1	31.000	1	29.000	1	31.000	1	25.000	1	25.000	1	24.000	1	21.000	1	17.000	1	14.000	1	15.000	1	12.000	1	12.000	1	10.000	1
1991-04-03	10.000	1	9.000	1	10.000	1	10.000	1	9.000	1	10.000	1	15.000	1	24.000	1	28.000	1	24.000	1	18.000	1	14.000	1	12.000	1	13.000	1	14.000	1	15.000	1	14.000	1	15.000	1	13.000	1	13.000	1	13.000	1	12.000	1	10.000	1	10.000	1

Only a sample of the data showing its format is given above. The full example file may be downloaded here.

Structure your program to show statistics for each line of the file, (similar to the original Python, Perl, and AWK examples below), followed by summary statistics for the file. When showing example output just show a few line statistics and the full end summary.

       IDENTIFICATION DIVISION.
       PROGRAM-ID. data-munging.
 
       ENVIRONMENT DIVISION.
       INPUT-OUTPUT SECTION.
       FILE-CONTROL.
           SELECT input-file ASSIGN TO INPUT-FILE-PATH
               ORGANIZATION LINE SEQUENTIAL
               FILE STATUS file-status.
 
       DATA DIVISION.
       FILE SECTION.
       FD  input-file.
       01  input-record.
           03  date-stamp          PIC X(10).
           03  FILLER              PIC X.
           *> Curse whoever decided to use tabs and variable length
           *> data in the file!
           03  input-data-pairs    PIC X(300).
 
       WORKING-STORAGE SECTION.
       78  INPUT-FILE-PATH         VALUE "readings.txt".
 
       01  file-status             PIC 99.
           88 file-is-ok           VALUE 0.
           88 end-of-file          VALUE 10.
 
       01  data-pair.
           03  val                 PIC 9(3)V9(3).
           03  flag                PIC S9.
               88  invalid-flag    VALUE -9 THRU 0.
 
       01  val-length              PIC 9.
       01  flag-length             PIC 9.
       01  offset                  PIC 99.
 
       01  day-total               PIC 9(5)V9(3).
       01  grand-total             PIC 9(8)V9(3).
       01  mean-val                PIC 9(8)V9(3).
 
       01  day-rejected            PIC 9(5).
       01  day-accepted            PIC 9(5).
 
       01  total-rejected          PIC 9(8).
       01  total-accepted          PIC 9(8).
 
       01  current-data-gap        PIC 9(8).
       01  max-data-gap            PIC 9(8).
       01  max-data-gap-end        PIC X(10).
 
       PROCEDURE DIVISION.
       DECLARATIVES.
       *> Terminate the program if an error occurs on input-file.
       input-file-error SECTION.
           USE AFTER STANDARD ERROR ON input-file.
 
           DISPLAY
               "An error occurred while reading input.txt. "
               "File error: " file-status
               ". The program will terminate."
           END-DISPLAY
 
           GOBACK
           .
 
       END DECLARATIVES.
 
       main-line.
           *> Terminate the program if the file cannot be opened.
           OPEN INPUT input-file
           IF NOT file-is-ok
               DISPLAY "File could not be opened. The program will "
                   "terminate."
               GOBACK
           END-IF
 
           *> Process the data in the file.
           PERFORM FOREVER
               *> Stop processing if at the end of the file.
               READ input-file
                   AT END
                       EXIT PERFORM
               END-READ
 
               *> Split the data up and process the value-flag pairs.
               PERFORM UNTIL input-data-pairs = SPACES
                   *> Split off the value-flag pair at the front of the
                   *> record.
                   UNSTRING input-data-pairs DELIMITED BY X"09"
                       INTO val COUNT val-length, flag COUNT flag-length
 
                   COMPUTE offset = val-length + flag-length + 3
                   MOVE input-data-pairs (offset:) TO input-data-pairs
 
                   *> Process according to flag.
                   IF NOT invalid-flag
                       ADD val TO day-total, grand-total
 
                       ADD 1 TO day-accepted, total-accepted
 
                       IF max-data-gap < current-data-gap
                           MOVE current-data-gap TO max-data-gap
                           MOVE date-stamp TO max-data-gap-end
                       END-IF
 
                       MOVE ZERO TO current-data-gap
                   ELSE
                       ADD 1 TO current-data-gap, day-rejected,
                           total-rejected
                   END-IF
               END-PERFORM
 
               *> Display day stats.
               DIVIDE day-total BY day-accepted GIVING mean-val
               DISPLAY
                   date-stamp 
                   " Reject: " day-rejected
                   " Accept: " day-accepted
                   " Average: " mean-val
               END-DISPLAY
 
               INITIALIZE day-rejected, day-accepted, mean-val,
                   day-total
           END-PERFORM
 
           CLOSE input-file
 
           *> Display overall stats.
           DISPLAY SPACE
           DISPLAY "File:         " INPUT-FILE-PATH
           DISPLAY "Total:        " grand-total
           DISPLAY "Readings:     " total-accepted
 
           DIVIDE grand-total BY total-accepted GIVING mean-val
           DISPLAY "Average:      " mean-val
 
           DISPLAY SPACE
           DISPLAY "Bad readings: " total-rejected
           DISPLAY "Maximum number of consecutive bad readings is "
               max-data-gap
           DISPLAY "Ends on date " max-data-gap-end
 
           GOBACK
           .