<commit_before>Conclusion
==========

Benchmarking vs non-progressive REST
------------------------------------

I feel it is important to experimentally answer the question, *is this
actually any faster?*. To do this I have created a small benchmarking
suite that runs under Node.js. I chose Node because it at its code is a
very basic platform which I feel it gives a more repeatable environment
than modern browsers which at during the tests could be performing any
number of background tasks. These tests may be seen in the `benchmark`
folder of the project. Node also has the advantage in measuring the
memory of a running process is not swamped by the memory taken up by the
browser itself.

One of the proposed advantages of progressive REST is an improved user
experience because of earlier, more progressive interface rendering and
a perceptual improvement in speed. I am not focusing on this area for
benchmarking because it would be much more difficult to measure,
involving human participants. While I can't provide numbers on the
perceptual improvements, I have created sites using Oboe and the
improvement in responsiveness over slower networks is large enough to be
obvious.

The benchmark mimics a relational database-backed REST service.
Relational databases serve data to a cursor one tuple at a time. The
simulated service writes out twenty tuples as JSON objects, one every
two milliseconds. To simulate network slowness, Apple's *Network Line
Conditioner* was used. I chose the named presets "3G, Average Case" and
"Cable modem" to represent poor and good networks respectively [^1].
Each test involves two node processes, one acting as the client and one
as the server, with data transfer between them via normal http.

Memory was measured using Node's built in memory reporting tool at
various points and the maximum figure returned on each run was taken:
[^2]

### Aggregating services

Each object in the returned JSON contains a URL to a further resource.
Each further resource is fetched and parsed. The aggregation is complete
when we have them all.

  Strategy         Network conditions   Total time   Max. Memory
  ---------------- -------------------- ------------ -------------
  Oboe.js          Poor                 ?            ?
  Oboe.js          Good                 ?            ?
  JSON.parse       Poor                 ?            ?
  JSON.parse       Good                 ?            ?
  Clarinet (SAX)   Poor                 ?            ?
  Clarinet (SAX)   Good                 ?            ?

### Simple download

This is a much simpler test which involved downloading just one
resource. To reduce the size of the data only good network conditions
are tested.

  Strategy         Network conditions   Total time   Max. Memory
  ---------------- -------------------- ------------ -------------
  Oboe.js          Good                 ?            ?
  JSON.parse       Good                 ?            ?
  Clarinet (SAX)   Good                 ?            ?

### Commentary on the results

Although Clarinet is known to be slower than JSON.parse, in practice
this didn't show in the results.

Doing things faster vs doing things earlier. "Hurry up and wait"
approach to optimisation. Already know Clarinet is slower than browser's
inbuilt parsing mechanism although not by a significant amount [^3]

Comparative Programmer Ergonomics
---------------------------------

  Strategy         Lines of Code Required
  ---------------- ------------------------
  Oboe.js          not many
  JSON.parse       bit more
  Clarinet (SAX)   lots

### vs non-progressive REST

Consider difficulty in upgrades.

### vs Clarinet

In terms of syntax: compare to SAX (clarinet) for getting the same job
done. Draw examples from github project README. Or from reimplementing
Clarinet's examples.

Consider:

-   Difficulty to program
-   Ease of reading the program / clarity of code
-   Ease of use vs SAX

Status as a micro-lib
---------------------

![A pie chart showing the sizes of the various parts of the
codebase](images/placeholder.png)

Comment on the size of the libraray

Most of the size comes from Clarinet.js inclusion. Comment on making a
smaller version. c'net is \~30% pre-minification, \~60% post.

Performance of code styles under various engines
------------------------------------------------

Complex JSONPath tested against JSON with approx 2,000 nodes, finding
100 matches. Real http, full stack Oboe.

  Platform                                      Time
  ----------------------------------------- --------
  Firefox 24.0.0 (Mac OS X 10.7)               547ms
  Chrome 30.0.1599 (Mac OS X 10.7.5)           237ms
  Chrome Mobile iOS 30.0.1599 (iOS 7.0.2)      431ms
  Safari 6.0.5 (Mac OS X 10.7.5)               231ms
  IE 8.0.0 (Windows XP)                       3048ms

For example with only mono-morphic callsites and without a functional
style. Once either of those programming techniques is taken up
performance drops rapidly
[http://rfrn.org/\~shu/2013/03/20/two-reasons-functional-style-is-slow-in-spidermonkey.html]
9571 ms vs 504 ms. When used in a functional style, not 'near-native' in
the sense that not close to the performance gained by compiling a well
designed functional language to natively executable code. Depends on
style coded in, comparison to native somewhat takes C as the description
of the operation of an idealised CPU rather than an abstract machine
capable of executing on an actual CPU.

potential future work
---------------------

implementation keeps 'unreachable' listeners difficult
decidability/proof type problem to get completely right but could cover
most of the easy cases

Parse time for large files spread out over a long time. Reaction to
parsed content spread out over a long time, for example de-marshalling
to domain objects. For UX may be preferable to have many small delays
rather than one large one.

Doesn't support all of jsonpath. Not a strict subset of the language.

### Mutability

Rest client as a library is passing mutable objects to the caller. too
inefficient to re-create a new map/array every time an item is not as
efficient in immutability as list head-tail type storage

An immutability wrapper might be possible with defineProperty. Can't
casually overwrite via assignment but still possible to do
defineProperty again.

Would benefit from a stateless language where everything is stateless at
all times to avoid having to program defensively.<commit_msg>write about micro-lib<commit_after>Conclusion
==========

Benchmarking vs non-progressive REST
------------------------------------

I feel it is important to experimentally answer the question, *is this
actually any faster?*. To do this I have created a small benchmarking
suite that runs under Node.js. I chose Node because it at its code is a
very basic platform which I feel it gives a more repeatable environment
than modern browsers which at during the tests could be performing any
number of background tasks. These tests may be seen in the `benchmark`
folder of the project. Node also has the advantage in measuring the
memory of a running process is not swamped by the memory taken up by the
browser itself.

One of the proposed advantages of progressive REST is an improved user
experience because of earlier, more progressive interface rendering and
a perceptual improvement in speed. I am not focusing on this area for
benchmarking because it would be much more difficult to measure,
involving human participants. While I can't provide numbers on the
perceptual improvements, I have created sites using Oboe and the
improvement in responsiveness over slower networks is large enough to be
obvious.

The benchmark mimics a relational database-backed REST service.
Relational databases serve data to a cursor one tuple at a time. The
simulated service writes out twenty tuples as JSON objects, one every
two milliseconds. To simulate network slowness, Apple's *Network Line
Conditioner* was used. I chose the named presets "3G, Average Case" and
"Cable modem" to represent poor and good networks respectively [^1].
Each test involves two node processes, one acting as the client and one
as the server, with data transfer between them via normal http.

Memory was measured using Node's built in memory reporting tool at
various points and the maximum figure returned on each run was taken:
[^2]

### Aggregating services

Each object in the returned JSON contains a URL to a further resource.
Each further resource is fetched and parsed. The aggregation is complete
when we have them all.

  Strategy         Network conditions   Total time   Max. Memory
  ---------------- -------------------- ------------ -------------
  Oboe.js          Poor                 ?            ?
  Oboe.js          Good                 ?            ?
  JSON.parse       Poor                 ?            ?
  JSON.parse       Good                 ?            ?
  Clarinet (SAX)   Poor                 ?            ?
  Clarinet (SAX)   Good                 ?            ?

### Simple download

This is a much simpler test which involved downloading just one
resource. To reduce the size of the data only good network conditions
are tested.

  Strategy         Network conditions   Total time   Max. Memory
  ---------------- -------------------- ------------ -------------
  Oboe.js          Good                 ?            ?
  JSON.parse       Good                 ?            ?
  Clarinet (SAX)   Good                 ?            ?

### Commentary on the results

Although Clarinet is known to be slower than JSON.parse, in practice
this didn't show in the results.

Doing things faster vs doing things earlier. "Hurry up and wait"
approach to optimisation. Already know Clarinet is slower than browser's
inbuilt parsing mechanism although not by a significant amount [^3]

Comparative Programmer Ergonomics
---------------------------------

  Strategy         Lines of Code Required
  ---------------- ------------------------
  Oboe.js          not many
  JSON.parse       bit more
  Clarinet (SAX)   lots

### vs non-progressive REST

Consider difficulty in upgrades.

### vs Clarinet

In terms of syntax: compare to SAX (clarinet) for getting the same job
done. Draw examples from github project README. Or from reimplementing
Clarinet's examples.

Consider:

-   Difficulty to program
-   Ease of reading the program / clarity of code
-   Ease of use vs SAX

Performance of code styles under various engines
------------------------------------------------

Complex JSONPath tested against JSON with approx 2,000 nodes, finding
100 matches. Real http, full stack Oboe.

  Platform                                      Time
  ----------------------------------------- --------
  Firefox 24.0.0 (Mac OS X 10.7)               547ms
  Chrome 30.0.1599 (Mac OS X 10.7.5)           237ms
  Chrome Mobile iOS 30.0.1599 (iOS 7.0.2)      431ms
  Safari 6.0.5 (Mac OS X 10.7.5)               231ms
  IE 8.0.0 (Windows XP)                       3048ms

For example with only mono-morphic callsites and without a functional
style. Once either of those programming techniques is taken up
performance drops rapidly
[http://rfrn.org/\~shu/2013/03/20/two-reasons-functional-style-is-slow-in-spidermonkey.html]
9571 ms vs 504 ms. When used in a functional style, not 'near-native' in
the sense that not close to the performance gained by compiling a well
designed functional language to natively executable code. Depends on
style coded in, comparison to native somewhat takes C as the description
of the operation of an idealised CPU rather than an abstract machine
capable of executing on an actual CPU.

Status as a micro-library
-------------------------

Built versions of Oboe as delivered reside in the project's `dist`
folder. The file `oboe-browser.min.js` is the minified version which
should be sent to browsers gzipped. After gzip is applied this file
comes to 4966 bytes; close to but comfortably under the 5120 limit. At
roughly the size as a very small image, the size of Oboe should not
discourage adoption.

potential future work
---------------------

implementation keeps 'unreachable' listeners difficult
decidability/proof type problem to get completely right but could cover
most of the easy cases

Parse time for large files spread out over a long time. Reaction to
parsed content spread out over a long time, for example de-marshalling
to domain objects. For UX may be preferable to have many small delays
rather than one large one.

Doesn't support all of jsonpath. Not a strict subset of the language.

### Mutability

Rest client as a library is passing mutable objects to the caller. too
inefficient to re-create a new map/array every time an item is not as
efficient in immutability as list head-tail type storage

An immutability wrapper might be possible with defineProperty. Can't
casually overwrite via assignment but still possible to do
defineProperty again.

Would benefit from a stateless language where everything is stateless at
all times to avoid having to program defensively.
{'fraction_non_alphanumeric': 0.09438288116163546, 'fraction_numerical': 0.013144822315628583, 'mean_word_length': 2.387522650789542, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1527639', 'n_tokens_mistral': 3309, 'n_tokens_neox': 3063, 'n_words': 1730}