<commit_before>
Server Upgrade
  o) copy katas/ on current server to new server
     Current disk is SSD 200GB and is 75% inode full.
     Current spec is 2CPU, 7.5GB.

     Copying files from old-server to new-server using rsync
     http://www.jveweb.net/en/archives/2011/01/running-rsync-as-a-daemon.html

     On the server I'm copying FROM I created /etc/rsyncd.conf
     and put the INTERNAL ip address of the machine I'm copying TO in the hosts-allow

     lock file = /var/run/rsync.lock
     log file = /var/log/rsyncd.log
     pid file = /var/run/rsyncd.pid
     [katas]
         path = /var/www/cyber-dojo/katas
         comment = cyber-dojo katas
         read only = yes
         list = yes
         auth users = jj
         secrets file = /etc/rsyncd.secrets
         hosts allow = 10.240.0.5/255.255.255.0

     Then I created an /etc/rsyncd.secrets file
     jj:password-goes-here

     Then I started the server
     $ sudo rsync --daemon

     Then on the server I'm copying to, cd to the folder where I want all the katas (00,01,etc)
     $ rsync -rtv jj@172.16.31.10::katas .
     $ enter password

     Note jj is not a real user.
     Note :: has two colons
     Note katas is the [katas] from the rsyncd.conf file


  o) Make sure katas folder had user-rights matching user inside the web container.
     On Ubuntu 16.04 ($ lsb_release -a)
     $ userdel cyber-dojo
     $ sudo adduser --disabled-password --disabled-login --uid 19661 cyber-dojo
     $ cd /var/www/cyber-dojo
     $ sudo chown -R cyber-dojo:cyber-dojo katas

  o) use modified docker-compose.yml

     Need to remove this line from docker-compose.yml
       volumes_from:
         - ${CYBER_DOJO_KATAS_DATA_CONTAINER}:rw
     Need to add this
       volumes:
         - /var/www/cyber-dojo/katas:${CYBER_DOJO_ROOT}/katas:rw

     $ cd ~
     $ sudo ./cyber-dojo up
     $ sudo ./cyber-dojo start-point languages pull

     Need to comment out this line in cyber-dojo.sh
       # one_time_creation_of_katas_data_container

  o) This should hopefully fix the 100% cpu bug I've been occasionally hitting
     See https://github.com/docker/docker/issues/18180
     One GCE creating a bare server using Ubuntu 14.04 shows
     $ uname -r
     3.19.0-64-generic
     One GCE creating a bare server using Ubuntu 15.10 shows<commit_msg>[ci skip] notes update<commit_after>
Server Upgrade
  o) install docker on new server
     $ curl -O https://raw.githubusercontent.com/cyber-dojo/web/master/cli/install-docker.sh
     $ chmod +x install-docker.sh
     $ sudo ./install-docker.sh

  o) copy katas/ on current server to new server
     Current disk is SSD 200GB and is 75% inode full.
     Current spec is 2CPU, 7.5GB.

     Copying files from old-server to new-server using rsync
     http://www.jveweb.net/en/archives/2011/01/running-rsync-as-a-daemon.html

     On the server I'm copying FROM I created /etc/rsyncd.conf
     and put the INTERNAL ip address of the machine I'm copying TO in the hosts-allow

     lock file = /var/run/rsync.lock
     log file = /var/log/rsyncd.log
     pid file = /var/run/rsyncd.pid
     [katas]
         path = /var/www/cyber-dojo/katas
         comment = cyber-dojo katas
         read only = yes
         list = yes
         auth users = jj
         secrets file = /etc/rsyncd.secrets
         hosts allow = 10.240.0.5/25172.16.17.32

     Then I created an /etc/rsyncd.secrets file
     jj:password-goes-here

     Then I started the server
     $ sudo rsync --daemon

     Then on the server I'm copying to, cd to the folder where I want all the katas (00,01,etc)
     $ rsync -rtv jj@172.16.31.10::katas .
     $ enter password

     Note jj is not a real user.
     Note :: has two colons
     Note katas is the [katas] from the rsyncd.conf file


  o) Make sure katas folder had user-rights matching user inside the web container.
     On Ubuntu 16.04 ($ lsb_release -a)
     $ userdel cyber-dojo
     $ sudo adduser --disabled-password --disabled-login --uid 19661 cyber-dojo
     $ cd /var/www/cyber-dojo
     $ sudo chown -R cyber-dojo:cyber-dojo katas
       This will take a while...

  o) install cyber-dojo
     $ curl -O https://raw.githubusercontent.com/cyber-dojo/web/master/cli/cyber-dojo
     $ chmod +x cyber-dojo

  o) use modified docker-compose.yml

     Need to remove this line from docker-compose.yml
       volumes_from:
         - ${CYBER_DOJO_KATAS_DATA_CONTAINER}:rw
     Need to add this
       volumes:
         - /PATH/TO/katas:${CYBER_DOJO_ROOT}/katas:rw

     $ cd ~
     $ sudo ./cyber-dojo up
     $ sudo ./cyber-dojo start-point languages pull

     Need to comment out this line in cyber-dojo.sh
       # one_time_creation_of_katas_data_container

  o) This should hopefully fix the 100% cpu bug I've been occasionally hitting
     See https://github.com/docker/docker/issues/18180
     One GCE creating a bare server using Ubuntu 14.04 shows
     $ uname -r
     3.19.0-64-generic
     One GCE creating a bare server using Ubuntu 15.10 shows
{'fraction_non_alphanumeric': 0.0858, 'fraction_numerical': 0.0302, 'mean_word_length': 2.275049115913556, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 7, 'xml': 0}, 'pii_count': 6, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '598527', 'n_tokens_mistral': 1825, 'n_tokens_neox': 1626, 'n_words': 542}