<commit_before># server setup

## Ubuntu setup / installing
For a remote server, SSH in to login with `ssh [user]@[ip address or domain of server]` - i.e. `ssh root@192.168.127.12` then type the password for that user.

1. add users
	- `adduser lmadera` (then follow prompts)
	- give sudo privilges `adduser lmadera sudo`
	- optional but recommended - add group and add user to group
		- `groupadd developers`
		- add user to group: `usermod -a -G developers lmadera`
2. log out of root then re-login as the user you just created
3. `sudo apt-get update`
4. Install git: `sudo apt-get install git-core`
5. Install node.js
	- https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager
		- scroll down to the 'Ubuntu' section and follow the instructions / commands there
		- instructions say to symlink at the end but this didn't work / it wasn't necessary?
			- `sudo ln -s /usr/bin/nodejs /usr/bin/node`
			- http://stackoverflow.com/questions/18130164/nodejs-vs-node-on-ubuntu-12-04
			- UPDATE: got an error: `failed to create symbolic link `/usr/bin/node': File exists` - so maybe it's already done / don't need to do this?
	- optional but recommended
		1. run on port 80 (assuming node runs on port 3000 - change according to config) `sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 3000`. This is only if using one server/domain on this host and want to be able to access it without typing in a port number at the end of the url - i.e. 'http://mydomain.com' instead of 'http://mydomain.com:3000'
			- see more details: http://stackoverflow.com/questions/7929563/node-js-express-app-wont-start-listening-on-port-80
		2. create 'node' user for using with things like 'forever' (since it's user specific so whatever user started it is the only one who can stop it..)
			- use `su node` to switch to node users
6. Install MongoDB - NOTE: maybe want to use hosted service (i.e. MongoLab or MongoHQ) to save on diskspace - see below
	- http://docs.mongodb.org/manual/tutorial/install-mongodb-on-debian-or-ubuntu-linux/ (have to go through some setup steps before can run the standard apt-get install command..)
		- Use "touch" to create a new file
		- NOTE: use a 64 bit build since the 32 bit builds of mongo/ubuntu have a 2GB memory limit! http://www.mongodb.org/downloads#32-bit-limit
7. Install PhantomJS
	1. `sudo apt-get install phantomjs`

	
## Note on file sizes / server space
May want to use MongoLab or MongoHQ or another hosted/cloud MongoDB solution since MongoDB can easily take up 6GB - journal of 3ish GB (apparently it doesn't go past this though?) plus the data itself plus log files plus some other stuff.
MongoDB and node forever took up space; here's one of my servers that maxed out with 20GB - forever though was mostly log files and while `forever cleanlogs` didn't seem to work, I was able to just delete the file(s) that were large:

- / 18G
	- /var 11G
		- /lib 6.4G
			- /mongodb 6.1G
				- /journal 3.1G<commit_msg>docs:update<commit_after># server setup

## Ubuntu setup / installing
For a remote server, SSH in to login with `ssh [user]@[ip address or domain of server]` - i.e. `ssh root@192.168.127.12` then type the password for that user.

1. add users
	- `adduser lmadera` (then follow prompts)
	- give sudo privilges `adduser lmadera sudo`
	- optional but recommended - add group and add user to group
		- `groupadd developers`
		- add user to group: `usermod -a -G developers lmadera`
2. log out of root then re-login as the user you just created
3. `sudo apt-get update`
4. Install git: `sudo apt-get install git-core`
5. Install node.js
	- https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager
		- scroll down to the 'Ubuntu' section and follow the instructions / commands there
		- instructions say to symlink at the end but this didn't work / it wasn't necessary?
			- `sudo ln -s /usr/bin/nodejs /usr/bin/node`
			- http://stackoverflow.com/questions/18130164/nodejs-vs-node-on-ubuntu-12-04
			- UPDATE: got an error: `failed to create symbolic link `/usr/bin/node': File exists` - so maybe it's already done / don't need to do this?
	- optional but recommended
		1. run on port 80 (assuming node runs on port 3000 - change according to config) `sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 3000`. This is only if using one server/domain on this host and want to be able to access it without typing in a port number at the end of the url - i.e. 'http://mydomain.com' instead of 'http://mydomain.com:3000'
			- see more details: http://stackoverflow.com/questions/7929563/node-js-express-app-wont-start-listening-on-port-80
		2. create 'node' user for using with things like 'forever' (since it's user specific so whatever user started it is the only one who can stop it..)
			- use `su node` to switch to node users
6. Install MongoDB - NOTE: maybe want to use hosted service (i.e. MongoLab or MongoHQ) to save on diskspace - see below
	- http://docs.mongodb.org/manual/tutorial/install-mongodb-on-debian-or-ubuntu-linux/ (have to go through some setup steps before can run the standard apt-get install command..)
		- Use "touch" to create a new file
		- NOTE: use a 64 bit build since the 32 bit builds of mongo/ubuntu have a 2GB memory limit! http://www.mongodb.org/downloads#32-bit-limit
7. Install PhantomJS
	1. `sudo apt-get install phantomjs`

	
## Note on file sizes / server space
May want to use MongoLab or MongoHQ or another hosted/cloud MongoDB solution since MongoDB can easily take up 6GB - journal of 3ish GB (apparently it doesn't go past this though?) plus the data itself plus log files plus some other stuff.

MongoDB and node forever took up space; here's one of my servers that maxed out with 20GB - forever though was mostly log files and while `forever cleanlogs` didn't seem to work, I was able to just delete the file(s) that were large:

- / 18G
	- /var 11G
		- /lib 6.4G
			- /mongodb 6.1G
				- /journal 3.1G
{'fraction_non_alphanumeric': 0.09057425079524527, 'fraction_numerical': 0.02645236899380546, 'mean_word_length': 3.3733528550512446, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3507826', 'n_tokens_mistral': 2076, 'n_tokens_neox': 1801, 'n_words': 845}