<commit_before># Muxque (WIP)

[![Build Status](https://travis-ci.org/chzyer/muxque.svg?branch=master)](https://travis-ci.org/chzyer/muxque)
[![Software License](https://img.shields.io/badge/license-MIT-brightgreen.svg)](LICENSE.md)

muxque is design to use in IM, follows pub/sub model, to achieve serve millions users by multiplex sockets.

### goal
* possible to subscribe multiple topics/channels in one client.
* always persist messages to disk to keep them safe
* zero-extra-cost(disk) for multicasted messages/channels
* distributed and decentralized topologies without single points of failure
* the number of topic can be unlimit(depends on hardware)
* provide a horizontally scaling solution

### benchmark

* http test
```
$ go run bench/httpserver/server.go
$ make bench-http
go test -v -benchtime 10s -benchmem -bench=Http -run=Nothing github.com/chzyer/muxque/bench
testing: warning: no tests to run
PASS
BenchmarkHttpPut	 3000000	      4808 ns/op (aka 207,986 rps)	      16 B/op	       0 allocs/op
ok  	github.com/chzyer/muxque/bench	18.811s
```

* api sync put (single client wait until server reply)
```
$ go run muxque.go
$ make bench-sync-put
go test -v -benchtime 10s -benchmem -bench=SyncPut -run=Nothing github.com/chzyer/muxque/bench
testing: warning: no tests to run
PASS
BenchmarkApiSyncPut	 5000000	      3445 ns/op (aka 290,275 rps)	       2 B/op	       0 allocs/op
ok  	github.com/chzyer/muxque/bench	20.579s
```

* internal test (without network)
```
$ make bench-topic
go test -v -benchtime 10s -benchmem -bench=. -run=Nothing github.com/chzyer/muxque/topic
PASS
BenchmarkTopicGet	 5000000	      2805 ns/op	     432 B/op	       8 allocs/op
BenchmarkTopicPut	 5000000	      2622 ns/op	     119 B/op	       3 allocs/op
ok  	github.com/chzyer/muxque/topic	46.333s
```

nsq
```
# using --mem-queue-size=1000000 --data-path= --size=200 --batch-size=200
# compiling/running nsqd
# creating topic/channel
# compiling bench_reader/bench_writer
PUB: [bench_writer] 2015/07/19 20:07:15 duration: 10.003577746s - 29.725mb/s - 155844.243ops/s - 6.417us/op
SUB: [bench_reader] 2015/07/19 20:07:25 duration: 10.014623761s - 17.258mb/s - 90481.282ops/s - 11.052us/op
```<commit_msg>update benchmark<commit_after># Muxque (WIP)

[![Build Status](https://travis-ci.org/chzyer/muxque.svg?branch=master)](https://travis-ci.org/chzyer/muxque)
[![Software License](https://img.shields.io/badge/license-MIT-brightgreen.svg)](LICENSE.md)

muxque is design to use in IM, follows pub/sub model, to achieve serve millions users by multiplex sockets.

### goal
* possible to subscribe multiple topics/channels in one client.
* always persist messages to disk to keep them safe
* zero-extra-cost(disk) for multicasted messages/channels
* distributed and decentralized topologies without single points of failure
* the number of topic can be unlimit(depends on hardware)
* provide a horizontally scaling solution

### benchmark

* http test
```
$ go run bench/httpserver/server.go
$ make bench-http
go test -v -benchtime 10s -benchmem -bench=Http -run=Nothing github.com/chzyer/muxque/bench
testing: warning: no tests to run
PASS
BenchmarkHttpPut	 3000000	      4808 ns/op (aka 207,986 rps)	      16 B/op	       0 allocs/op
ok  	github.com/chzyer/muxque/bench	18.811s
```

* api sync put (single client wait until server reply)
```
$ go run muxque.go
$ make bench-sync-put
go test -v -benchtime 10s -benchmem -bench=SyncPut -run=Nothing github.com/chzyer/muxque/bench
testing: warning: no tests to run
PASS
BenchmarkApiSyncPut	 5000000	      3445 ns/op (aka 290,275 rps)	       2 B/op	       0 allocs/op
ok  	github.com/chzyer/muxque/bench	20.579s
```

* internal test (without network)
```
$ make bench-topic
go test -v -benchtime 10s -benchmem -bench=. -run=Nothing github.com/chzyer/muxque/topic
PASS
BenchmarkTopicGet	 5000000	      2805 ns/op (381,388 rps)	     432 B/op	       8 allocs/op
BenchmarkTopicPut	 5000000	      2622 ns/op (356,506 rps)	     119 B/op	       3 allocs/op
ok  	github.com/chzyer/muxque/topic	46.333s
```

nsq
```
# using --mem-queue-size=1000000 --data-path= --size=200 --batch-size=200
# compiling/running nsqd
# creating topic/channel
# compiling bench_reader/bench_writer
PUB: [bench_writer] 2015/07/19 20:07:15 duration: 10.003577746s - 29.725mb/s - 155844.243ops/s - 6.417us/op
SUB: [bench_reader] 2015/07/19 20:07:25 duration: 10.014623761s - 17.258mb/s - 90481.282ops/s - 11.052us/op
```
{'fraction_non_alphanumeric': 0.11480225988700565, 'fraction_numerical': 0.08813559322033898, 'mean_word_length': 3.6297071129707112, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1934293', 'n_tokens_mistral': 1867, 'n_tokens_neox': 1545, 'n_words': 439}