<commit_before>---
layout:     post
title:      Node.js
summary:    Timothy explores the world of Node.js
categories: node web programming
---

## Introduction

Some time last semester, I first heard about [Node.js][1] from
[a friend of mine][2] who really likes doing web stuff. Our conversation were
something like:

<dl>
<dt>TG:</dt>
<dd>Hey Ethan I am trying to rewrite the <a href="http://fate.ffmpeg.org/">
FFmpeg FATE website</a> in something faster than
<a href="https://github.com/ffmpeg/fateserver">Perl CGI</a>.</dd>
<dt>EL:</dt>
<dd>You should <em>really</em> look into Node.js.</dd>
<dt>TG:</dt>
<dd>I've heard of it, but what the heck is that?</dd>
<dt>EL:</dt>
<dd>It's basically server-side JavaScript.</dd>
</dl>

There, I started on my journey learning JavaScript and Node.js.

[1]: https://nodejs.org/
[2]: http://ethanl.ee/
[3]: http://fate.ffmpeg.org/
[4]: https://github.com/ffmpeg/fateserver

## Learning

So I just Googled something like "learnign node.js" (with spelling errors),
and [NodeSchool][5] pops up. Nice yellow-themed website. And there there were,
some "workshoppers" that are "self-guided ... lesson modules". I already knew
JavaScript quite a bit, so I directly picked up on "learnyounode".

After wrestling with npm and Ubuntu `nodejs` vs `node` mess for an hour, there
it was, a fine workshopper with atomic units that are, frankly, extremely
beginner-friendly. Node.js beginner-friendly, not JS beginner-friendly. Now I
think of it, I literally knew nothing about JS except `var` works for all
types and `function abc(d, e, f)` is how you right a function.

So I thought, *nope, this doesn't work at all*, and went back to NodeSchool
for the "javascripting" workshopper. And I have to say, I *really* enjoyed
learning JS while my teacher lectures in my math class.

After finishing the two workshoppers I just mentioned, I started doing the
"expressworks" workshopper. And it was equally amazing.

After about three day, I thought I was proficient enough to at least *port*
the FATE CGI over. And to be honest, I almost was, but not proficient enough
to do all the caching stuff, etc.

**tl;dr:** To learning JavaScript, 3+ things should be enough: some NodeSchool
workshoppers, proficient Googling experiences, and a learning spirit.

[5]: http://nodeschool.io/index.html

## FATE

Not much to say here, except when I tried to add some caching some race
conditions occurred. So I used a rwlock. Yes, rwlock, in JavaScript. I don't
even have any experiences for mutexes in C (and that shows my level of C...)
and I do now in JS.

I also chose to use [EJS][6] because:

1. it is what's used in the first blog post regarding Express.js on Google
2. it's *fast*

But nevertheless, my friend told me about [Jade][7]. So I looked that up.
*Wow,* said me in the past thought, *it's so cool ... and clean ...*

Until I **ported my entire project over** and benchmarked it. Without caching
it was something like 7x slower than EJS and slower than the Perl CGI version.
Even with caching it had only ⅔ of the performance of EJS. So, I reverted
my 500-line change and cried for 10 minutes, and moved on.

[6]: https://github.com/tj/ejs
[7]: http://jade-lang.com/

## Jade

### Gaining maintainer status
Interestingly, I started sending PRs fixing broken links on the website of
Jade, even when I am not using Jade. After my
[second very trivial PR to Jade][8], the wonderful @ForbesLindesay added me
to the Jade maintainers team. I was genuinely surprised, as I had literally
less than a week of real JS experiences, and now I am on the maintainer team
of a module that is [installed over 70,000 times per day][9].

[8]: https://github.com/jadejs/jade/pull/1709#issuecomment-61475378
[9]: https://www.npmjs.com/package/jade

### All the easy stuff

Back to the story. I continued contributing to Jade in some documentation
stuff and some trivial deprecation stuff. But gradually, I became more
familiar with the code base, and more comfortable with working with JS, which
frankly is a bastard language, but a bastardly (assuming that's a word) easy-
to-write language.

### Harder stuff

One of my most recent commits to Jade was [adding tests for caching][10] and
[the CLI][11], including CLI watching mode where the output is recompiled
whenever you modify the source file. The async nature of caching and process
spawning were extremely challenging even now as I recall it. I am still fairly
proud of myself for being able to figure out a way to do that.

[10]: https://github.com/jadejs/jade/pull/1808
[11]: https://github.com/jadejs/jade/pull/1805

### Now and beyond

Now I am the person with the [third most commits][12] in Jade, plus I gained a
whole lot more experiences with working with a mature Node.js project and
@ForbesLindesay. Huge shoutout to everybody who helped me in the process!

I still haven't gotten to study the Jade parser and lexer, the two *beasts* TJ
wrote a long time ago and are starting to show some age. Forbes said he is
going to split them to new repos in the Jade v2 rewrite, so for now let's wait
until a roadmap to v2 is established.

[12]: https://github.com/jadejs/jade/graphs/contributors

## EJS

### Discovery of EJS v2

A few weeks ago, when tracking dependencies of fateserver on [David][13], I
saw EJS 2.0.2 was just published. As any good project maintainer would do, I
went to the tj/ejs GitHub repo, and saw a huge deprecation notice there,
saying that EJS v2 is now maintained in mde/ejs. After tinkering around a
little bit, I found out that mde/ejs was basically a reimplementation of TJ's
EJS.

[13]: https://david-dm.org/TimothyGu/fateserver-node

### Contributing

While trying to port my app over, I saw that quite a bit was missing from the
new implementation. One of those was the `_with` option I depend upon for
enhanced performance. So I opened a [PR][14] for that.

Similar to my experiences in Jade, @mde quickly gave me the maintainer
permission to EJS as well. And then I started [fixing a lot of things][15] in
EJS v2, making it almost completely compatible to v1.

[14]: https://github.com/mde/ejs/pull/6
[15]: https://github.com/mde/ejs/commits/master?author=TimothyGu

### Performance issues

When benchmarking my app, I notice that the performance of it has
**significantly** declined from EJS v1 (by ~70%). So I started optimizing it.
In EJS v2.2.1, two changes of mine made the template functions almost 4x
faster in some conditions. But still, it was slower than EJS v1 in more
complex apps, both in compilation and compiled functions (though not as much
as compilation).

### Forking

At last I pinned the slower performance in compilation to the overuse of
regexes and OOP in the new implementation, two issues to which there are no
good fixes. Without a second choice, I forked tj/ejs's repo and started
backporting features in EJS v2 to that.

I have to say that my backport worked fairly well. For compilation it is 1.1x
faster than EJS v1 and 3.52x than EJS v2. For template functions, it is almost
1.36x faster than EJS v2 in regular mode and 1.26x faster in `!_with` mode,
albeit 32% slower than EJS v1 in extreme cases in regular mode.

So I was hoping I could merge my branch to EJS v2, but @mde said (in private):

> I'm really not a fan of the dense, acrobatic code in TJ's repo. If it's
> hard for me to figure out, it would also make it hard for people in the
> community.

So no, we ain't gonna merge it. Do note that what he said is 100% true. TJ's
code is, for a lack of a better word, godly, both godly fast and godly
spaghetti. But for me, at least, @mde's implementation basically splits *one*
function in TJ's implementation to something like 7 plus an enum, most of as
simple as heck, but the 3 most important functions no less spaghetti IMO as
TJ's. True, TJ's `parse()` is *long*, but that doesn't mean it's harder to
read. Rather, it is easier for me because you don't need to jump back and
forth to look at the three functions doing some very procedural things.

As a result, I officially forked the package, to a very unique name
[ejs-tj][16]. I will keep maintaining both forks, so fear not if you don't
want to use my fork but want to use my code. For my fork, I intend to keep
compatibility with EJS v2 forever, so I even used the same version number as
EJS v2's.

[16]: https://www.npmjs.com/package/ejs-tj

## Conclusion

Node.js is really a world of endless wonders, friendly developers, and
unlimited job opportunities. I shall continue in my journey to the mastery of
the JavaScript language and the environment of Node.js.

**<NAME>**

EDIT: why did I put the last sentence here?
<commit_msg>Add missing period<commit_after>---
layout:     post
title:      Node.js
summary:    Timothy explores the world of Node.js.
categories: node web programming
---

## Introduction

Some time last semester, I first heard about [Node.js][1] from
[a friend of mine][2] who really likes doing web stuff. Our conversation were
something like:

<dl>
<dt>TG:</dt>
<dd>Hey Ethan I am trying to rewrite the <a href="http://fate.ffmpeg.org/">
FFmpeg FATE website</a> in something faster than
<a href="https://github.com/ffmpeg/fateserver">Perl CGI</a>.</dd>
<dt>EL:</dt>
<dd>You should <em>really</em> look into Node.js.</dd>
<dt>TG:</dt>
<dd>I've heard of it, but what the heck is that?</dd>
<dt>EL:</dt>
<dd>It's basically server-side JavaScript.</dd>
</dl>

There, I started on my journey learning JavaScript and Node.js.

[1]: https://nodejs.org/
[2]: http://ethanl.ee/
[3]: http://fate.ffmpeg.org/
[4]: https://github.com/ffmpeg/fateserver

## Learning

So I just Googled something like "learnign node.js" (with spelling errors),
and [NodeSchool][5] pops up. Nice yellow-themed website. And there there were,
some "workshoppers" that are "self-guided ... lesson modules". I already knew
JavaScript quite a bit, so I directly picked up on "learnyounode".

After wrestling with npm and Ubuntu `nodejs` vs `node` mess for an hour, there
it was, a fine workshopper with atomic units that are, frankly, extremely
beginner-friendly. Node.js beginner-friendly, not JS beginner-friendly. Now I
think of it, I literally knew nothing about JS except `var` works for all
types and `function abc(d, e, f)` is how you right a function.

So I thought, *nope, this doesn't work at all*, and went back to NodeSchool
for the "javascripting" workshopper. And I have to say, I *really* enjoyed
learning JS while my teacher lectures in my math class.

After finishing the two workshoppers I just mentioned, I started doing the
"expressworks" workshopper. And it was equally amazing.

After about three day, I thought I was proficient enough to at least *port*
the FATE CGI over. And to be honest, I almost was, but not proficient enough
to do all the caching stuff, etc.

**tl;dr:** To learning JavaScript, 3+ things should be enough: some NodeSchool
workshoppers, proficient Googling experiences, and a learning spirit.

[5]: http://nodeschool.io/index.html

## FATE

Not much to say here, except when I tried to add some caching some race
conditions occurred. So I used a rwlock. Yes, rwlock, in JavaScript. I don't
even have any experiences for mutexes in C (and that shows my level of C...)
and I do now in JS.

I also chose to use [EJS][6] because:

1. it is what's used in the first blog post regarding Express.js on Google
2. it's *fast*

But nevertheless, my friend told me about [Jade][7]. So I looked that up.
*Wow,* said me in the past thought, *it's so cool ... and clean ...*

Until I **ported my entire project over** and benchmarked it. Without caching
it was something like 7x slower than EJS and slower than the Perl CGI version.
Even with caching it had only ⅔ of the performance of EJS. So, I reverted
my 500-line change and cried for 10 minutes, and moved on.

[6]: https://github.com/tj/ejs
[7]: http://jade-lang.com/

## Jade

### Gaining maintainer status
Interestingly, I started sending PRs fixing broken links on the website of
Jade, even when I am not using Jade. After my
[second very trivial PR to Jade][8], the wonderful @ForbesLindesay added me
to the Jade maintainers team. I was genuinely surprised, as I had literally
less than a week of real JS experiences, and now I am on the maintainer team
of a module that is [installed over 70,000 times per day][9].

[8]: https://github.com/jadejs/jade/pull/1709#issuecomment-61475378
[9]: https://www.npmjs.com/package/jade

### All the easy stuff

Back to the story. I continued contributing to Jade in some documentation
stuff and some trivial deprecation stuff. But gradually, I became more
familiar with the code base, and more comfortable with working with JS, which
frankly is a bastard language, but a bastardly (assuming that's a word) easy-
to-write language.

### Harder stuff

One of my most recent commits to Jade was [adding tests for caching][10] and
[the CLI][11], including CLI watching mode where the output is recompiled
whenever you modify the source file. The async nature of caching and process
spawning were extremely challenging even now as I recall it. I am still fairly
proud of myself for being able to figure out a way to do that.

[10]: https://github.com/jadejs/jade/pull/1808
[11]: https://github.com/jadejs/jade/pull/1805

### Now and beyond

Now I am the person with the [third most commits][12] in Jade, plus I gained a
whole lot more experiences with working with a mature Node.js project and
@ForbesLindesay. Huge shoutout to everybody who helped me in the process!

I still haven't gotten to study the Jade parser and lexer, the two *beasts* TJ
wrote a long time ago and are starting to show some age. Forbes said he is
going to split them to new repos in the Jade v2 rewrite, so for now let's wait
until a roadmap to v2 is established.

[12]: https://github.com/jadejs/jade/graphs/contributors

## EJS

### Discovery of EJS v2

A few weeks ago, when tracking dependencies of fateserver on [David][13], I
saw EJS 2.0.2 was just published. As any good project maintainer would do, I
went to the tj/ejs GitHub repo, and saw a huge deprecation notice there,
saying that EJS v2 is now maintained in mde/ejs. After tinkering around a
little bit, I found out that mde/ejs was basically a reimplementation of TJ's
EJS.

[13]: https://david-dm.org/TimothyGu/fateserver-node

### Contributing

While trying to port my app over, I saw that quite a bit was missing from the
new implementation. One of those was the `_with` option I depend upon for
enhanced performance. So I opened a [PR][14] for that.

Similar to my experiences in Jade, @mde quickly gave me the maintainer
permission to EJS as well. And then I started [fixing a lot of things][15] in
EJS v2, making it almost completely compatible to v1.

[14]: https://github.com/mde/ejs/pull/6
[15]: https://github.com/mde/ejs/commits/master?author=TimothyGu

### Performance issues

When benchmarking my app, I notice that the performance of it has
**significantly** declined from EJS v1 (by ~70%). So I started optimizing it.
In EJS v2.2.1, two changes of mine made the template functions almost 4x
faster in some conditions. But still, it was slower than EJS v1 in more
complex apps, both in compilation and compiled functions (though not as much
as compilation).

### Forking

At last I pinned the slower performance in compilation to the overuse of
regexes and OOP in the new implementation, two issues to which there are no
good fixes. Without a second choice, I forked tj/ejs's repo and started
backporting features in EJS v2 to that.

I have to say that my backport worked fairly well. For compilation it is 1.1x
faster than EJS v1 and 3.52x than EJS v2. For template functions, it is almost
1.36x faster than EJS v2 in regular mode and 1.26x faster in `!_with` mode,
albeit 32% slower than EJS v1 in extreme cases in regular mode.

So I was hoping I could merge my branch to EJS v2, but @mde said (in private):

> I'm really not a fan of the dense, acrobatic code in TJ's repo. If it's
> hard for me to figure out, it would also make it hard for people in the
> community.

So no, we ain't gonna merge it. Do note that what he said is 100% true. TJ's
code is, for a lack of a better word, godly, both godly fast and godly
spaghetti. But for me, at least, @mde's implementation basically splits *one*
function in TJ's implementation to something like 7 plus an enum, most of as
simple as heck, but the 3 most important functions no less spaghetti IMO as
TJ's. True, TJ's `parse()` is *long*, but that doesn't mean it's harder to
read. Rather, it is easier for me because you don't need to jump back and
forth to look at the three functions doing some very procedural things.

As a result, I officially forked the package, to a very unique name
[ejs-tj][16]. I will keep maintaining both forks, so fear not if you don't
want to use my fork but want to use my code. For my fork, I intend to keep
compatibility with EJS v2 forever, so I even used the same version number as
EJS v2's.

[16]: https://www.npmjs.com/package/ejs-tj

## Conclusion

Node.js is really a world of endless wonders, friendly developers, and
unlimited job opportunities. I shall continue in my journey to the mastery of
the JavaScript language and the environment of Node.js.

**<NAME>**

EDIT: why did I put the last sentence here?

{'fraction_non_alphanumeric': 0.07902964335340436, 'fraction_numerical': 0.01412691060676239, 'mean_word_length': 4.068368544600939, 'pattern_counts': {'":': 0, '<': 53, '<?xml version=': 0, '>': 59, 'https://': 26, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6511521', 'n_tokens_mistral': 5398, 'n_tokens_neox': 5059, 'n_words': 2750}