<commit_before># Round 2 Binary

Some concerns were raised about the impact JSON encoding and parsing may have on the benchmark reliability. To test this possibility the Go (golang.org/x/net/websocket), C++ (uWebsockets), and C++ (websocketpp) servers have been updated to support a binary mode.

In addition, it was suggested that using a Go program and using a complete websocket client (even when the benchmark tool is parallelized on multiple machines)  would impact the results. So as a further test, a C++ benchmark tool that directly works with libuv instead of any websocket client library has been added that supports the binary broadcast subset of the benchmark protocol. The C++ benchmark is not parallelized.

Finally, the test now starts at 10,000 clients instead of 0. This allows the tests to run a little faster. In all other ways, this round of testing should be the same as the original Round 2.

## Environments

Go

```
dev@earth:~/hashrocket/websocket-shootout(master*)% go version
go version go1.7.3 linux/amd64
dev@earth:~/hashrocket/websocket-shootout(master*)% bin/go-websocket-server -address 0.0.0.0 -port 5000
2016/12/02 13:45:33 Starting to listen on: 0.0.0.0:5000
```

C++

```
dev@earth:~/hashrocket/websocket-shootout/cpp/uwebsockets(master*)% gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/5/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 5.4.0-6ubuntu1~16.04.2' --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-5 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.2)
```

## Raw Go bench / JSON Results

### C++ / uWebSockets / RapidJSON -- 1 thread

Server Command

```
dev@earth:~/hashrocket/websocket-shootout/cpp/uwebsockets(master*)% bin/cpp-uwebsockets-server --port 5000 --data-type json
```

Benchmark Output
<commit_msg>Remove copy-and-paste error<commit_after># Round 2 Binary

Some concerns were raised about the impact JSON encoding and parsing may have on the benchmark reliability. To test this possibility the Go (golang.org/x/net/websocket), C++ (uWebsockets), and C++ (websocketpp) servers have been updated to support a binary mode.

In addition, it was suggested that using a Go program and using a complete websocket client (even when the benchmark tool is parallelized on multiple machines)  would impact the results. So as a further test, a C++ benchmark tool that directly works with libuv instead of any websocket client library has been added that supports the binary broadcast subset of the benchmark protocol. The C++ benchmark is not parallelized.

Finally, the test now starts at 10,000 clients instead of 0. This allows the tests to run a little faster. In all other ways, this round of testing should be the same as the original Round 2.

## Environments

Go

```
dev@earth:~/hashrocket/websocket-shootout(master*)% go version
go version go1.7.3 linux/amd64
```

C++

```
dev@earth:~/hashrocket/websocket-shootout/cpp/uwebsockets(master*)% gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/5/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 5.4.0-6ubuntu1~16.04.2' --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-5 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.2)
```

## Raw Go bench / JSON Results

### C++ / uWebSockets / RapidJSON -- 1 thread

Server Command

```
dev@earth:~/hashrocket/websocket-shootout/cpp/uwebsockets(master*)% bin/cpp-uwebsockets-server --port 5000 --data-type json
```

Benchmark Output

{'fraction_non_alphanumeric': 0.14739583333333334, 'fraction_numerical': 0.041666666666666664, 'mean_word_length': 4.053508771929825, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '90616', 'n_tokens_mistral': 2215, 'n_tokens_neox': 1956, 'n_words': 505}