<commit_before>FrameReceiver
=============

FrameReceiver is an application to enable reception of frames of detector data transmitted over a network connection 
as a UDP stream. FrameReceiver constructs data frames in shared memory buffers and communicates with external 
applications to allow processing and storage.

[![Build Status](https://travis-ci.org/percival-detector/framereceiver.svg)](https://travis-ci.org/percival-detector/framereceiver)
[![Stories in Ready](https://badge.waffle.io/percival-detector/framereceiver.png?label=ready&title=Ready)](https://waffle.io/percival-detector/framereceiver)
[![Stories in In Progress](https://badge.waffle.io/percival-detector/framereceiver.png?label=In%20Progress&title=In%20Progress)](https://waffle.io/percival-detector/framereceiver)

External Software Dependencies
==============================

The following libraries and packages are required:

* [CMake](http://www.cmake.org) : build management system (version >= 2.8)
* [Boost](http://www.boost.org) : portable C++ utility libraries. The following components are used - program_options, unit_test_framework, date_time, interprocess, bimap (version >= 1.41)
* [ZeroMQ](http://zeromq.org) : high-performance asynchronous messaging library (version >= 3.2.4)
* [Log4CXX](http://logging.apache.org/log4cxx/): Configurable message logger (version >= 0.10.0)
* [HDF5](https://www.hdfgroup.org/HDF5): __Optional:__ if found, the filewriter application will be built (version >= 1.8.14) 

Installing dependencies
-----------------------

The supported development platform is Linux RedHat 6 (RHEL6) however distributions like CentOS and Scientific 
Linux are essentially clones of RHEL and can be used as free alternatives to RHEL.

Most of the dependencies are available from the RHEL (or CentOS) repositories, but a few (log4cxx and zeromq) 
must be installed from the EPEL repository (or built from source).

The following is an exmaple of installing dependencies on CentOS 6:

    # Install the development environment: gcc, make, etc.
    yum groupinstall "Development tools"
    
    # Install the cmake and boost libraries
    yum install cmake boost-devel
    
    # Enable the EPEL repositories - the exact method may be dependent
    # on the distribution. This works for CentOS 6:
    yum install epel-release
    
    # Install the zeromq and log4cxx libraries from the EPEL repo
    yum --enablerepo=epel install zeromq3-devel log4cxx-devel


Building
========

FrameReceiver is built using CMake and make. FrameReceiver is configured and built with the following commands:

    mkdir build && cd build
    cmake -DBoost_NO_BOOST_CMAKE=ON ..
    make

The Boost_NO_BOOST_CMAKE=ON flag is required on systems with Boost installed on system paths (e.g. from a package 
repository) where there is a bug in the installed CMake package discovery utilities. Non-default installations of 
the above libraries and packages can be used by setting the following flags:

* BOOST_ROOT
* ZEROMQ_ROOTDIR
* LOG4CXX_ROOT_DIR
* HDF5_ROOT

Applications are compiled into the `bin` directory within `build` and the additional python etc tools installed into 
tools. Sample configuration files for testing the system are installed into `test_config`.  See the sections below 
further information on usage. 


Usage
=====

In addition to the core FrameReceiver application, there are a number of associated tools that can be used in 
conjunction to operate a system. These are described below.

Running the FrameReceiver
-------------------------

The frameReceiver is a C++ compiled application and is configured by a combination of command-line options
and equivalent parameters stored in an INI-formatted configuration file. The command-line options take
precedence over the configuration file if specified. The options and their default values can be viewed 
by invoking the`--help` option:

	$ bin/frameReceiver --help
	usage: frameReceiver [options]
	
	
	Generic options:
	  -h [ --help ]         Print this help message
	  -v [ --version ]      Print program version string
	  -c [ --config ] arg   Specify program configuration file
	
	Configuration options:
	  -d [ --debug ] arg (=0)                Set the debug level
	  -n [ --node ] arg (=1)                 Set the frame receiver node ID
	  -l [ --logconfig ] arg                 Set the log4cxx logging configuration 
	                                         file
	  -m [ --maxmem ] arg (=1048576)         Set the maximum amount of shared 
	                                         memory to allocate for frame buffers
	  -s [ --sensortype ] arg (=unknown)     Set the sensor type to receive frame 
	                                         data from
	  -p [ --port ] arg (=8989,8990)         Set the port to receive frame data on
	  -i [ --ipaddress ] arg (=0.0.0.0)      Set the IP address of the interface to
	                                         receive frame data on
	  --sharedbuf arg (=FrameReceiverBuffer) Set the name of the shared memory 
	                                         frame buffer
	  --frametimeout arg (=1000)             Set the incomplete frame timeout in ms
	  -f [ --frames ] arg (=0)               Set the number of frames to receive 
	                                         before terminating
	  --packetlog arg (=0)                   Enable logging of packet diagnostics 
	                                         to file
	  --rxbuffer arg (=30000000)             Set UDP receive buffer size

The meaning of the configuration options are as follows:

* `-h` or `--help`

   Print the help message shown above.

* `-v` or `--version`
   
   Print the program version string (to be implemented).

* `-c` or `--config` 

   Specify the program configuration file to be loaded.
   
 * `-d` or `--debug`
 
   Specify the debug level. Increasing the value increases the verbosity of the debug output.
   
 * `-n` or `--node`

   Set the frame receiver node ID. Identifies the node in a multi-receiver system.
   
 * `-l` or `--logconfig`

   Set the log4cxx logging configuration file to use, which configures the format and destination
   of logging output from the application. See the README.md file in the `config` directory for<commit_msg>Minor formatting tweaks to README.md<commit_after>FrameReceiver
=============

FrameReceiver is an application to enable reception of frames of detector data transmitted over a network connection 
as a UDP stream. FrameReceiver constructs data frames in shared memory buffers and communicates with external 
applications to allow processing and storage.

[![Build Status](https://travis-ci.org/percival-detector/framereceiver.svg)](https://travis-ci.org/percival-detector/framereceiver)
[![Stories in Ready](https://badge.waffle.io/percival-detector/framereceiver.png?label=ready&title=Ready)](https://waffle.io/percival-detector/framereceiver)
[![Stories in In Progress](https://badge.waffle.io/percival-detector/framereceiver.png?label=In%20Progress&title=In%20Progress)](https://waffle.io/percival-detector/framereceiver)

External Software Dependencies
==============================

The following libraries and packages are required:

* [CMake](http://www.cmake.org) : build management system (version >= 2.8)
* [Boost](http://www.boost.org) : portable C++ utility libraries. The following components are used - program_options, unit_test_framework, date_time, interprocess, bimap (version >= 1.41)
* [ZeroMQ](http://zeromq.org) : high-performance asynchronous messaging library (version >= 3.2.4)
* [Log4CXX](http://logging.apache.org/log4cxx/): Configurable message logger (version >= 0.10.0)
* [HDF5](https://www.hdfgroup.org/HDF5): __Optional:__ if found, the filewriter application will be built (version >= 1.8.14) 

Installing dependencies
-----------------------

The supported development platform is Linux RedHat 6 (RHEL6) however distributions like CentOS and Scientific 
Linux are essentially clones of RHEL and can be used as free alternatives to RHEL.

Most of the dependencies are available from the RHEL (or CentOS) repositories, but a few (log4cxx and zeromq) 
must be installed from the EPEL repository (or built from source).

The following is an exmaple of installing dependencies on CentOS 6:

    # Install the development environment: gcc, make, etc.
    yum groupinstall "Development tools"
    
    # Install the cmake and boost libraries
    yum install cmake boost-devel
    
    # Enable the EPEL repositories - the exact method may be dependent
    # on the distribution. This works for CentOS 6:
    yum install epel-release
    
    # Install the zeromq and log4cxx libraries from the EPEL repo
    yum --enablerepo=epel install zeromq3-devel log4cxx-devel


Building
========

FrameReceiver is built using CMake and make. FrameReceiver is configured and built with the following commands:

    mkdir build && cd build
    cmake -DBoost_NO_BOOST_CMAKE=ON ..
    make

The Boost_NO_BOOST_CMAKE=ON flag is required on systems with Boost installed on system paths (e.g. from a package 
repository) where there is a bug in the installed CMake package discovery utilities. Non-default installations of 
the above libraries and packages can be used by setting the following flags:

* BOOST_ROOT
* ZEROMQ_ROOTDIR
* LOG4CXX_ROOT_DIR
* HDF5_ROOT

Applications are compiled into the `bin` directory within `build` and the additional python etc tools installed into 
tools. Sample configuration files for testing the system are installed into `test_config`.  See the sections below 
further information on usage. 


Usage
=====

In addition to the core FrameReceiver application, there are a number of associated tools that can be used in 
conjunction to operate a system. These are described below.

Running the FrameReceiver
-------------------------

The frameReceiver is a C++ compiled application and is configured by a combination of command-line options
and equivalent parameters stored in an INI-formatted configuration file. The command-line options take
precedence over the configuration file if specified. The options and their default values can be viewed 
by invoking the`--help` option:

	$ bin/frameReceiver --help
	usage: frameReceiver [options]
	
	
	Generic options:
	  -h [ --help ]         Print this help message
	  -v [ --version ]      Print program version string
	  -c [ --config ] arg   Specify program configuration file
	
	Configuration options:
	  -d [ --debug ] arg (=0)                Set the debug level
	  -n [ --node ] arg (=1)                 Set the frame receiver node ID
	  -l [ --logconfig ] arg                 Set the log4cxx logging configuration 
	                                         file
	  -m [ --maxmem ] arg (=1048576)         Set the maximum amount of shared 
	                                         memory to allocate for frame buffers
	  -s [ --sensortype ] arg (=unknown)     Set the sensor type to receive frame 
	                                         data from
	  -p [ --port ] arg (=8989,8990)         Set the port to receive frame data on
	  -i [ --ipaddress ] arg (=0.0.0.0)      Set the IP address of the interface to
	                                         receive frame data on
	  --sharedbuf arg (=FrameReceiverBuffer) Set the name of the shared memory 
	                                         frame buffer
	  --frametimeout arg (=1000)             Set the incomplete frame timeout in ms
	  -f [ --frames ] arg (=0)               Set the number of frames to receive 
	                                         before terminating
	  --packetlog arg (=0)                   Enable logging of packet diagnostics 
	                                         to file
	  --rxbuffer arg (=30000000)             Set UDP receive buffer size

The meaning of the configuration options are as follows:

* `-h` or `--help`

   Print the help message shown above.

* `-v` or `--version`
   
   Print the program version string (to be implemented).

* `-c` or `--config` 

   Specify the program configuration file to be loaded.
   
* `-d` or `--debug`
 
   Specify the debug level. Increasing the value increases the verbosity of the debug output.
   
* `-n` or `--node`

   Set the frame receiver node ID. Identifies the node in a multi-receiver system.
   
* `-l` or `--logconfig`

   Set the log4cxx logging configuration file to use, which configures the format and destination
   of logging output from the application. See the README.md file in the `config` directory for
{'fraction_non_alphanumeric': 0.08682778489116517, 'fraction_numerical': 0.011363636363636364, 'mean_word_length': 2.811222933821287, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 13, 'https://': 14, 'lorem ipsum': 0, 'www.': 6, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 30, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2571587', 'n_tokens_mistral': 3477, 'n_tokens_neox': 3161, 'n_words': 1399}