<commit_before>Byfl: Compiler-based Application Analysis
=========================================

Description
-----------

Byfl helps application developers understand code performance in a _hardware-independent_ way.  The idea is that it instruments your code at compile time then gathers and reports data at run time.  For example, suppose you wanted to know how many bytes are accessed by the following C code:

    double array[100000][100];
    volatile double sum = 0.0;

    for (int row=0; row<100000; row++)
      sum += array[row][0];

Reading the hardware performance counters (e.g., using [PAPI](http://icl.cs.utk.edu/papi/)) can be misleading.  The performance counters on most processors tally not the number of bytes but rather the number of cache-line accesses.  Because the array is stored in row-major order, each access to `array` will presumably reference a different cache line while each access to `sum` will presumably reference the same cache line.

Byfl does the equivalent of transforming the code into the following:

    unsigned long int bytes accessed = 0;
    double array[100000][100];
    volatile double sum = 0.0;

    for (int row=0; row<100000; row++) {
      sum += array[row][0];
      bytes_accessed += 3*sizeof(double);
    }

In the above, one can consider the `bytes_accessed` variable as a "software performance counter," as it is maintained entirely by software.

In practice, however, Byfl doesn't do source-to-source transformations (unlike, for example, [ROSE](http://www.rosecompiler.org/)) as implied by the preceding code sample.  Instead, it integrates into the [LLVM compiler infrastructure](http://www.llvm.org/) as an LLVM compiler pass.  If your application compiles with [LLVM's Clang](http://clang.llvm.org/) or any of the [GNU compilers](http://gcc.gnu.org/), you can instrument it with Byfl.

Because Byfl instruments code in LLVM's intermediate representation (IR), not native machine code, it outputs the same counter values regardless of target architecture.  In contrast, binary-instrumentation tools such as [Pin](http://www.pintool.org/) may tally operations differently on different platforms.

The name "Byfl" comes from "bytes/flops".  The very first version of the code counted only bytes and floating-point operations (flops).


Installation
------------

If you have reasonably recent versions of [GCC](http://gcc.gnu.org/), [LLVM](http://www.llvm.org/), [Clang](http://clang.llvm.org/), and [DragonEgg](http://dragonegg.llvm.org/), you should be able to perform the usual

    ./configure
    make
    make install

procedure.  See [INSTALL.md](https://github.com/losalamos/Byfl/blob/master/INSTALL.md) for a more complete explanation.

Usage
-----

### Basic usage

Byfl comes with a set of wrapper scripts that simplify instrumentation.  `bf-gcc`, `bf-g++`, `bf-gfortran`, and `bf-gccgo` wrap, respectively, the GNU C, C++, Fortran, and Go compilers.  `bf-mpicc`, `bf-mpicxx`, `bf-mpif90`, and `bf-mpif77` further wrap the similarly named [Open MPI](http://www.open-mpi.org/) and [MPICH](http://www.mpich.org/) wrapper scripts to use the Byfl compiler scripts instead of the default C, C++, and Fortran compilers.  Use any of these scripts as you would the underlying compiler.  When you run your code, Byfl will output a sequence of `BYFL`-prefixed lines to the standard output device and a superset of the data to a binary file (called *filename*`.byfl` by default):

    BYFL_SUMMARY: -----------------------------------------------------------------
    BYFL_SUMMARY:                     1,280 bytes (512 loaded + 768 stored)
    BYFL_SUMMARY:                        96 flops
    BYFL_SUMMARY:                       549 integer ops
    BYFL_SUMMARY:                        67 memory ops (2 loads + 65 stores)
    BYFL_SUMMARY:                        35 branch ops (1 unconditional and direct + 32 conditional or indirect + 1 function calls + 1 function returns + 0 other)
    BYFL_SUMMARY:                       747 TOTAL OPS
    BYFL_SUMMARY: -----------------------------------------------------------------
    BYFL_SUMMARY:                    10,240 bits (4,096 loaded + 6,144 stored)
    BYFL_SUMMARY:                    12,288 flop bits
    BYFL_SUMMARY:                    94,880 op bits (excluding memory ops)
    BYFL_SUMMARY: -----------------------------------------------------------------
    BYFL_SUMMARY:                    0.6667 bytes loaded per byte stored
    BYFL_SUMMARY:                  373.0000 ops per load instruction
    BYFL_SUMMARY:                  152.8358 bits loaded/stored per memory op
    BYFL_SUMMARY:                    3.0000 flops per conditional/indirect branch
    BYFL_SUMMARY:                   23.3125 ops per conditional/indirect branch
    BYFL_SUMMARY: -----------------------------------------------------------------
    BYFL_SUMMARY:                   13.3333 bytes per flop
    BYFL_SUMMARY:                    0.8333 bits per flop bit
    BYFL_SUMMARY:                    1.7158 bytes per op
    BYFL_SUMMARY:                    0.1079 bits per (non-memory) op bit
    BYFL_SUMMARY: -----------------------------------------------------------------

"Bits" are simply bytes*8.  "Flop bits" are the total number of bits in all inputs and outputs to each floating-point function.  As motivation, consider the operation `A = B + C`, where `A`, `B`, and `C` reside in memory.  This operation consumes 12 bytes per flop if the arguments are all single-precision but 24 bytes per flop if the arguments are all double-precision.  Similarly, `A = -B` consumes either 8 or 16 bytes per flop based on the argument type.  However, all of these examples consume one bit per flop bit regardless of numerical precision: every bit loaded or stored either enters or exits the floating-point unit.  Bit:flop-bit ratios above 1.0 imply that more memory is moved than fed into the floating-point unit; Bit:flop-bit ratios below 1.0 imply register reuse.

The Byfl wrapper scripts accept a number of options to provide more information about your program at a cost of increased execution times.  These can be specified either on the command line or within the `BF_OPTS` environment variable.  (The former takes precedence.)  See the `bf-gcc`, `bf-g++`, `bf-gfortran`, or `bf-gccgo` man page for a description of all of the information Byfl can report.

The following represents some sample output from a code instrumented with Byfl and most of the available options:

    BYFL_INFO: Byfl command line: -bf-unique-bytes -bf-vectors -bf-every-bb -bf-mem-footprint -bf-types -bf-inst-mix -bf-by-func -bf-call-stack
    BYFL_FUNC_HEADER:             LD_bytes             ST_bytes               LD_ops               ST_ops                Flops              FP_bits              Int_ops          Int_op_bits           Uniq_bytes             Cond_brs          Invocations Function
    BYFL_FUNC:                         512                  768                    2                   65                   96                12288                  549                94880                  768                   32                    1 main
    BYFL_CALLEE_HEADER:   Invocations Byfl Function
    BYFL_CALLEE:                    1 No   __printf_chk
    BYFL_VECTOR_HEADER:             Elements             Elt_bits Type                Tally Function
    BYFL_VECTOR:                          32                   64 FP                      1 main
    BYFL_SUMMARY: -----------------------------------------------------------------
    BYFL_SUMMARY:                     1,280 bytes (512 loaded + 768 stored)
    BYFL_SUMMARY:                       768 unique bytes
    BYFL_SUMMARY:                       512 addresses cover 50% of all dynamic loads and stores
    BYFL_SUMMARY:                        96 flops
    BYFL_SUMMARY:                       549 integer ops
    BYFL_SUMMARY:                        67 memory ops (2 loads + 65 stores)
    BYFL_SUMMARY:                        35 branch ops (1 unconditional and direct + 32 conditional or indirect + 1 function calls + 1 function returns + 0 other)
    BYFL_SUMMARY:                       747 TOTAL OPS
    BYFL_SUMMARY: -----------------------------------------------------------------
    BYFL_SUMMARY:                    10,240 bits (4,096 loaded + 6,144 stored)
    BYFL_SUMMARY:                     6,144 unique bits
    BYFL_SUMMARY:                    12,288 flop bits
    BYFL_SUMMARY:                    94,880 op bits (excluding memory ops)
    BYFL_SUMMARY: -----------------------------------------------------------------
    BYFL_SUMMARY:                         1 vector operations (FP & int)
    BYFL_SUMMARY:                   32.0000 elements per vector
    BYFL_SUMMARY:                   64.0000 bits per element
    BYFL_SUMMARY: -----------------------------------------------------------------
    BYFL_SUMMARY:                         2 loads of vectors of 64-bit floating-point values
    BYFL_SUMMARY:                        64 stores of 64-bit floating-point values
    BYFL_SUMMARY:                         1 stores of vectors of 64-bit floating-point values
    BYFL_SUMMARY: -----------------------------------------------------------------
    BYFL_SUMMARY:                        96 Add            instructions executed
    BYFL_SUMMARY:                        67 BitCast        instructions executed
    BYFL_SUMMARY:                        65 Store          instructions executed<commit_msg>Updated the README.md file to reflect Byfl's latest textual output<commit_after>Byfl: Compiler-based Application Analysis
=========================================

Description
-----------

Byfl helps application developers understand code performance in a _hardware-independent_ way.  The idea is that it instruments your code at compile time then gathers and reports data at run time.  For example, suppose you wanted to know how many bytes are accessed by the following C code:

    double array[100000][100];
    volatile double sum = 0.0;

    for (int row=0; row<100000; row++)
      sum += array[row][0];

Reading the hardware performance counters (e.g., using [PAPI](http://icl.cs.utk.edu/papi/)) can be misleading.  The performance counters on most processors tally not the number of bytes but rather the number of cache-line accesses.  Because the array is stored in row-major order, each access to `array` will presumably reference a different cache line while each access to `sum` will presumably reference the same cache line.

Byfl does the equivalent of transforming the code into the following:

    unsigned long int bytes accessed = 0;
    double array[100000][100];
    volatile double sum = 0.0;

    for (int row=0; row<100000; row++) {
      sum += array[row][0];
      bytes_accessed += 3*sizeof(double);
    }

In the above, one can consider the `bytes_accessed` variable as a "software performance counter," as it is maintained entirely by software.

In practice, however, Byfl doesn't do source-to-source transformations (unlike, for example, [ROSE](http://www.rosecompiler.org/)) as implied by the preceding code sample.  Instead, it integrates into the [LLVM compiler infrastructure](http://www.llvm.org/) as an LLVM compiler pass.  If your application compiles with [LLVM's Clang](http://clang.llvm.org/) or any of the [GNU compilers](http://gcc.gnu.org/), you can instrument it with Byfl.

Because Byfl instruments code in LLVM's intermediate representation (IR), not native machine code, it outputs the same counter values regardless of target architecture.  In contrast, binary-instrumentation tools such as [Pin](http://www.pintool.org/) may tally operations differently on different platforms.

The name "Byfl" comes from "bytes/flops".  The very first version of the code counted only bytes and floating-point operations (flops).


Installation
------------

If you have reasonably recent versions of [GCC](http://gcc.gnu.org/), [LLVM](http://www.llvm.org/), [Clang](http://clang.llvm.org/), and [DragonEgg](http://dragonegg.llvm.org/), you should be able to perform the usual

    ./configure
    make
    make install

procedure.  See [INSTALL.md](https://github.com/losalamos/Byfl/blob/master/INSTALL.md) for a more complete explanation.

Usage
-----

### Basic usage

Byfl comes with a set of wrapper scripts that simplify instrumentation.  `bf-gcc`, `bf-g++`, `bf-gfortran`, and `bf-gccgo` wrap, respectively, the GNU C, C++, Fortran, and Go compilers.  `bf-mpicc`, `bf-mpicxx`, `bf-mpif90`, and `bf-mpif77` further wrap the similarly named [Open MPI](http://www.open-mpi.org/) and [MPICH](http://www.mpich.org/) wrapper scripts to use the Byfl compiler scripts instead of the default C, C++, and Fortran compilers.  Use any of these scripts as you would the underlying compiler.  When you run your code, Byfl will output a sequence of `BYFL`-prefixed lines to the standard output device and a superset of the data to a binary file (called *filename*`.byfl` by default):

    BYFL_INFO: Byfl command line:
    BYFL_SUMMARY: -----------------------------------------------------------------
    BYFL_SUMMARY:                     1,280 bytes (512 loaded + 768 stored)
    BYFL_SUMMARY:                        96 flops
    BYFL_SUMMARY:                       549 integer ops
    BYFL_SUMMARY:                        67 memory ops (2 loads + 65 stores)
    BYFL_SUMMARY:                        35 branch ops (1 unconditional and direct + 32 conditional or indirect + 2 function calls or returns + 0 other)
    BYFL_SUMMARY:                       747 TOTAL OPS
    BYFL_SUMMARY: -----------------------------------------------------------------
    BYFL_SUMMARY:                    10,240 bits (4,096 loaded + 6,144 stored)
    BYFL_SUMMARY:                    12,288 flop bits
    BYFL_SUMMARY:                    94,880 op bits (excluding memory ops)
    BYFL_SUMMARY: -----------------------------------------------------------------
    BYFL_SUMMARY:                    0.6667 bytes loaded per byte stored
    BYFL_SUMMARY:                  373.0000 ops per load instruction
    BYFL_SUMMARY:                  152.8358 bits loaded/stored per memory op
    BYFL_SUMMARY:                    3.0000 flops per conditional/indirect branch
    BYFL_SUMMARY:                   23.3125 ops per conditional/indirect branch
    BYFL_SUMMARY: -----------------------------------------------------------------
    BYFL_SUMMARY:                   13.3333 bytes per flop
    BYFL_SUMMARY:                    0.8333 bits per flop bit
    BYFL_SUMMARY:                    1.7158 bytes per op
    BYFL_SUMMARY:                    0.1079 bits per (non-memory) op bit
    BYFL_SUMMARY: -----------------------------------------------------------------

"Bits" are simply bytes*8.  "Flop bits" are the total number of bits in all inputs and outputs to each floating-point function.  As motivation, consider the operation `A = B + C`, where `A`, `B`, and `C` reside in memory.  This operation consumes 12 bytes per flop if the arguments are all single-precision but 24 bytes per flop if the arguments are all double-precision.  Similarly, `A = -B` consumes either 8 or 16 bytes per flop based on the argument type.  However, all of these examples consume one bit per flop bit regardless of numerical precision: every bit loaded or stored either enters or exits the floating-point unit.  Bit:flop-bit ratios above 1.0 imply that more memory is moved than fed into the floating-point unit; Bit:flop-bit ratios below 1.0 imply register reuse.

The Byfl wrapper scripts accept a number of options to provide more information about your program at a cost of increased execution times.  These can be specified either on the command line or within the `BF_OPTS` environment variable.  (The former takes precedence.)  See the `bf-gcc`, `bf-g++`, `bf-gfortran`, or `bf-gccgo` man page for a description of all of the information Byfl can report.

The following represents some sample output from a code instrumented with Byfl and most of the available options:
    BYFL_INFO: Byfl command line: -bf-unique-bytes -bf-vectors -bf-every-bb -bf-mem-footprint -bf-types -bf-inst-mix -bf-by-func -bf-call-stack
    BYFL_FUNC_HEADER:             LD_bytes             ST_bytes               LD_ops               ST_ops                Flops              FP_bits              Int_ops          Int_op_bits           Uniq_bytes             Cond_brs          Invocations Function
    BYFL_FUNC:                         512                  768                    2                   65                   96                12288                  549                94880                  768                   32                    1 main
    BYFL_CALLEE_HEADER:   Invocations Byfl Function
    BYFL_CALLEE:                    1 No   __printf_chk
    BYFL_VECTOR_HEADER:             Elements             Elt_bits Type                Tally Function
    BYFL_VECTOR:                          32                   64 FP                      1 main
    BYFL_SUMMARY: -----------------------------------------------------------------
    BYFL_SUMMARY:                     1,280 bytes (512 loaded + 768 stored)
    BYFL_SUMMARY:                       768 unique bytes
    BYFL_SUMMARY:                       512 addresses cover 50% of all dynamic loads and stores
    BYFL_SUMMARY:                        96 flops
    BYFL_SUMMARY:                       549 integer ops
    BYFL_SUMMARY:                        67 memory ops (2 loads + 65 stores)
    BYFL_SUMMARY:                        35 branch ops (1 unconditional and direct + 32 conditional or indirect + 2 function calls or returns + 0 other)
    BYFL_SUMMARY:                       747 TOTAL OPS
    BYFL_SUMMARY: -----------------------------------------------------------------
    BYFL_SUMMARY:                    10,240 bits (4,096 loaded + 6,144 stored)
    BYFL_SUMMARY:                     6,144 unique bits
    BYFL_SUMMARY:                    12,288 flop bits
    BYFL_SUMMARY:                    94,880 op bits (excluding memory ops)
    BYFL_SUMMARY: -----------------------------------------------------------------
    BYFL_SUMMARY:                         1 vector operations (FP & int)
    BYFL_SUMMARY:                   32.0000 elements per vector
    BYFL_SUMMARY:                   64.0000 bits per element
    BYFL_SUMMARY: -----------------------------------------------------------------
    BYFL_SUMMARY:                         2 loads of vectors of 64-bit floating-point values
    BYFL_SUMMARY:                        64 stores of 64-bit floating-point values
    BYFL_SUMMARY:                         1 stores of vectors of 64-bit floating-point values
    BYFL_SUMMARY: -----------------------------------------------------------------
    BYFL_SUMMARY:                        96 Add            instructions executed
    BYFL_SUMMARY:                        67 BitCast        instructions executed
    BYFL_SUMMARY:                        65 Store          instructions executed
{'fraction_non_alphanumeric': 0.13392857142857142, 'fraction_numerical': 0.0305672268907563, 'mean_word_length': 1.7271555428244056, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 12, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 26, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '278122', 'n_tokens_mistral': 5334, 'n_tokens_neox': 4548, 'n_words': 1922}