<commit_before>### Purpose
For several years now utilities have been using "smart meters" to optimize their residential meter reading infrastructure. Smart meters continuously transmit consumption information in the 900MHz ISM band allowing utilities to simply send readers driving through neighborhoods to collect commodity consumption information. The protocol used to transmit these messages is fairly straight forward, however I have yet to find any reasonably priced product for receiving these messages.

This project is a proof of concept software defined radio receiver for these messages. We make use of an inexpensive rtl-sdr dongle to allow users to non-invasively record and analyze the commodity consumption of their household.

Currently the only known supported and tested meter is the Itron C1SR. However, the protocol is designed to be useful for several different commodities and should be capable of receiving messages from any ERT capable smart meter.

For more info check out the project page: [http://bemasher.github.io/rtlamr/](http://bemasher.github.io/rtlamr/)

[![Build Status](https://travis-ci.org/bemasher/rtlamr.svg?branch=master)](https://travis-ci.org/bemasher/rtlamr)

### Requirements
 * GoLang >=1.2 (Go build environment setup guide: http://golang.org/doc/code.html)
 * GCC (on windows TDM-GCCx64 works nicely)
 * libfftw >=3.3
   * Windows: [pre-built binaries](http://www.fftw.org/install/windows.html)
   * Linux (debian): [package](https://packages.debian.org/stable/libfftw3-dev)
 * rtl-sdr
   * Windows: [pre-built binaries](http://sdr.osmocom.org/trac/attachment/wiki/rtl-sdr/RelWithDebInfo.zip)
   * Linux: [source and build instructions](http://sdr.osmocom.org/trac/wiki/rtl-sdr)

### Building
This project requires two other packages I've written for SDR related things in Go. The package [`github.com/bemasher/rtltcp`](http://godoc.org/github.com/bemasher/rtltcp) provides a means of controlling and sampling from rtl-sdr dongles. This package will be automatically downloaded and installed when getting rtlamr.

The second package needed is [`github.com/bemasher/fftw`](http://godoc.org/github.com/bemasher/fftw), which may require more effort to build. Assuming for linux you already have the necessary library, no extra work should need to be done. For windows a library file will need to be generated from the dll and def files for gcc. The FFTW defs and dlls can be found here: http://www.fftw.org/install/windows.html

#### On Windows

	go get -d github.com/bemasher/fftw
	dlltool -d libfftw3-3.def -D libfftw3-3.dll -l $GOPATH/src/github.com/bemasher/fftw/libfftw3.a
	go get github.com/bemasher/rtlamr

#### On Linux (Debian/Ubuntu)
	
	sudo apt-get install libfftw3-dev
	go get github.com/bemasher/rtlamr

This will produce the binary `$GOPATH/bin/rtlamr`. For convenience it's common to add `$GOPATH/bin` to the path.

#### With Docker
This project can be built and executed using docker:

```
docker pull bemasher/rtlamr
docker run --name rtlamr bemasher/rtlamr
```

This can also be run using the `bemasher/rtl-sdr` container:

```
docker pull bemasher/rtl-sdr
docker run -d --privileged -v /dev/bus/usb:/dev/bus/usb --name rtl_tcp bemasher/rtl-sdr rtl_tcp -a 0.0.0.0
docker run --name rtlamr --link rtl_tcp:rtl_tcp bemasher/rtlamr -server=rtl_tcp:1234
```

### Usage
Available command-line flags are as follows:

```
Usage of rtlamr:
  -centerfreq=920299072: center frequency to receive on
  -duration=0: time to run for, 0 for infinite
  -filterid=0: display only messages matching given id
  -format="plain": format to write log messages in: plain, json, xml or gob
  -gobunsafe=false: allow gob output to stdout
  -help=false: print long help
  -logfile="/dev/stdout": log statement dump file
  -quiet=false: suppress printing state information at startup
  -samplefile="NUL": raw signal dump file
  -server="127.0.0.1:1234": address or hostname of rtl_tcp instance
  -single=false: one shot execution
  -symbollength=73: symbol length in samples, see -help for valid lengths
```

Long Help via `-help`:
```
Usage of rtlamr:
  -centerfreq=920299072: Sets the center frequency of the rtl_tcp server. Defaults to 920.29MHz.

  -duration=0: Sets time to receive for, 0 for infinite. Defaults to infinite.
	If the time limit expires during processing of a block (which is quite
	likely) it will exit on the next pass through the receive loop. Exiting
	after an expired duration will print the total runtime to the log file.

  -filterid=0: Sets a meter id to filter by, 0 for no filtering. Defaults to no filtering.
	Any received messages not matching the given id will be silently ignored.
	Defaults to no filtering.

  -format="plain": Sets the log output format. Defaults to plain.
	Plain text is formatted using the following format string:

		{Time:%s Offset:%d Length:%d SCM:{ID:%8d Type:%2d Tamper:%+v Consumption:%8d Checksum:0x%04X}}

	No fields are omitted for json, xml or gob output. Plain text conditionally
	omits offset and length fields if not dumping samples to file via -samplefile.

	For json and xml output each line is an element, there is no root node.

  -gobunsafe=false: Must be true to allow writing gob encoded output to stdout. Defaults to false.
        Doing so would normally break a terminal, so we disable it unless
	explicitly enabled.

  -help=false: Print this help.

  -logfile="/dev/stdout": Sets file to dump log messages to. Defaults to os.DevNull and prints to stderr.

	Log messages have the following structure:

		type Message struct {
			Time   time.Time
			Offset int64
			Length int
			SCM    SCM
		}

		type SCM struct {
			ID          uint32<commit_msg>Minor formatting fix.<commit_after>### Purpose
For several years now utilities have been using "smart meters" to optimize their residential meter reading infrastructure. Smart meters continuously transmit consumption information in the 900MHz ISM band allowing utilities to simply send readers driving through neighborhoods to collect commodity consumption information. The protocol used to transmit these messages is fairly straight forward, however I have yet to find any reasonably priced product for receiving these messages.

This project is a proof of concept software defined radio receiver for these messages. We make use of an inexpensive rtl-sdr dongle to allow users to non-invasively record and analyze the commodity consumption of their household.

Currently the only known supported and tested meter is the Itron C1SR. However, the protocol is designed to be useful for several different commodities and should be capable of receiving messages from any ERT capable smart meter.

For more info check out the project page: [http://bemasher.github.io/rtlamr/](http://bemasher.github.io/rtlamr/)

[![Build Status](https://travis-ci.org/bemasher/rtlamr.svg?branch=master)](https://travis-ci.org/bemasher/rtlamr)

### Requirements
 * GoLang >=1.2 (Go build environment setup guide: http://golang.org/doc/code.html)
 * GCC (on windows TDM-GCCx64 works nicely)
 * libfftw >=3.3
   * Windows: [pre-built binaries](http://www.fftw.org/install/windows.html)
   * Linux (debian): [package](https://packages.debian.org/stable/libfftw3-dev)
 * rtl-sdr
   * Windows: [pre-built binaries](http://sdr.osmocom.org/trac/attachment/wiki/rtl-sdr/RelWithDebInfo.zip)
   * Linux: [source and build instructions](http://sdr.osmocom.org/trac/wiki/rtl-sdr)

### Building
This project requires two other packages I've written for SDR related things in Go. The package [`github.com/bemasher/rtltcp`](http://godoc.org/github.com/bemasher/rtltcp) provides a means of controlling and sampling from rtl-sdr dongles. This package will be automatically downloaded and installed when getting rtlamr.

The second package needed is [`github.com/bemasher/fftw`](http://godoc.org/github.com/bemasher/fftw), which may require more effort to build. Assuming for linux you already have the necessary library, no extra work should need to be done. For windows a library file will need to be generated from the dll and def files for gcc. The FFTW defs and dlls can be found here: http://www.fftw.org/install/windows.html

#### On Windows

	go get -d github.com/bemasher/fftw
	dlltool -d libfftw3-3.def -D libfftw3-3.dll -l $GOPATH/src/github.com/bemasher/fftw/libfftw3.a
	go get github.com/bemasher/rtlamr

#### On Linux (Debian/Ubuntu)
	
	sudo apt-get install libfftw3-dev
	go get github.com/bemasher/rtlamr

This will produce the binary `$GOPATH/bin/rtlamr`. For convenience it's common to add `$GOPATH/bin` to the path.

#### With Docker
This project can be built and executed using docker:

```
docker pull bemasher/rtlamr
docker run --name rtlamr bemasher/rtlamr
```

This can also be run using the `bemasher/rtl-sdr` container:

```
docker pull bemasher/rtl-sdr
docker run -d --privileged -v /dev/bus/usb:/dev/bus/usb --name rtl_tcp bemasher/rtl-sdr rtl_tcp -a 0.0.0.0
docker run --name rtlamr --link rtl_tcp:rtl_tcp bemasher/rtlamr -server=rtl_tcp:1234
```

### Usage
Available command-line flags are as follows:

```
Usage of rtlamr:
  -centerfreq=920299072: center frequency to receive on
  -duration=0: time to run for, 0 for infinite
  -filterid=0: display only messages matching given id
  -format="plain": format to write log messages in: plain, json, xml or gob
  -gobunsafe=false: allow gob output to stdout
  -help=false: print long help
  -logfile="/dev/stdout": log statement dump file
  -quiet=false: suppress printing state information at startup
  -samplefile="NUL": raw signal dump file
  -server="127.0.0.1:1234": address or hostname of rtl_tcp instance
  -single=false: one shot execution
  -symbollength=73: symbol length in samples, see -help for valid lengths
```

Long Help via `-help`:
```
Usage of rtlamr:
  -centerfreq=920299072: Sets the center frequency of the rtl_tcp server. Defaults to 920.29MHz.

  -duration=0: Sets time to receive for, 0 for infinite. Defaults to infinite.
	If the time limit expires during processing of a block (which is quite
	likely) it will exit on the next pass through the receive loop. Exiting
	after an expired duration will print the total runtime to the log file.

  -filterid=0: Sets a meter id to filter by, 0 for no filtering. Defaults to no filtering.
	Any received messages not matching the given id will be silently ignored.
	Defaults to no filtering.

  -format="plain": Sets the log output format. Defaults to plain.
	Plain text is formatted using the following format string:

		{Time:%s Offset:%d Length:%d SCM:{ID:%8d Type:%2d Tamper:%+v Consumption:%8d Checksum:0x%04X}}

	No fields are omitted for json, xml or gob output. Plain text conditionally
	omits offset and length fields if not dumping samples to file via -samplefile.

	For json and xml output each line is an element, there is no root node.

  -gobunsafe=false: Must be true to allow writing gob encoded output to stdout. Defaults to false.
    Doing so would normally break a terminal, so we disable it unless
	explicitly enabled.

  -help=false: Print this help.

  -logfile="/dev/stdout": Sets file to dump log messages to. Defaults to os.DevNull and prints to stderr.

	Log messages have the following structure:

		type Message struct {
			Time   time.Time
			Offset int64
			Length int
			SCM    SCM
		}

		type SCM struct {
			ID          uint32
{'fraction_non_alphanumeric': 0.08487899664370252, 'fraction_numerical': 0.013601837131248896, 'mean_word_length': 4.118896925858952, 'pattern_counts': {'":': 12, '<': 3, '<?xml version=': 0, '>': 7, 'https://': 6, 'lorem ipsum': 0, 'www.': 4, 'xml': 6}, 'pii_count': 4, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7261631', 'n_tokens_mistral': 3615, 'n_tokens_neox': 3305, 'n_words': 1411}