<commit_before># Development

## Ruby

1.  Follow the [Ruby on Rails Coding Conventions](http://guides.rubyonrails.org/contributing_to_ruby_on_rails.html#follow-the-coding-conventions).

    > * Two spaces, no tabs (for indentation).
    > * No trailing whitespace. Blank lines should not have any spaces.
    > * Indent after private/protected.
    > * Use Ruby >= 1.9 syntax for hashes. Prefer { a: :b } over { :a => :b }.
    > * Prefer &&/|| over and/or.
    > * Prefer class << self over self.method for class methods.
    > * Use my_method(my_arg) not my_method( my_arg ) or my_method my_arg.
    > * Use a = b and not a=b.
    > * Use assert_not methods instead of refute.
    > * Prefer method { do_stuff } instead of method{do_stuff} for single-line blocks.
    > * Follow the conventions in the source you see used already.

1.  Pass keyword arguments to methods. This is more verbose and lengthens
    refactoring, but it makes function calls self-descriptive. Ideally, there would 
    be a way to combine
    [arbitrary keyword arguments](https://github.com/ruby/ruby/blob/trunk/doc/syntax/methods.rdoc#keyword-arguments)
    with default values in the function definition, thus allowing transparent
    passing of hashes across functions and avoiding needing to go into the definition
    of the function to find the default; however, Ruby doesn't support this.
    Instead, follow a hybrid approach:
    1.  If a method does not expect to pass its options as a hash
        to another method, then define explicit keyword arguments (with
        required and optional arguments as needed).
        This should be the most common case. Example:
            
            def foo(required_param1:, optional_param1: "default")
              # ... definition ...
            end
            
            foo(required_param1: 1, optional_param1: "test")
            
    1.  If a method expects to pass its options as a hash to another
        method (or might do so in the future), then define an
        arbitrary keyword arguments options hash and set default values
        at the top of the function (if needed). If a parameter is required,
        throw a MissingArgumentError using the check_hash method. Example:
            
            def foo(**options)
              options[:optional_param1] ||= "foo default"
              bar(options)
            end
            
            def bar(**options)
              MissingArgumentError.check_hash(name: :required_param1, hash: options)
              options[:optional_param1] ||= "bar default"
              # ... definition ...
            end
            
            foo(required_param1: 1, optional_param1: "test")
1.  Split long function calls into multiple lines. Example:
        
        foo(
          required_param1: 1,
          required_param2: 2,
          required_param3: 3,
          optional_param1: "test"
        )

## General

* Site tests:
  * https://www.google.com/webmasters/tools/mobile-friendly/

## Architecture

                                               web1
                                            +--------+            +---------+
                                            + nginx  +            +         |
                                         XXXX  RoR   XXXXXXXXXXXXXX   db1   |
                       frontend2       XXX  +        +        XX  +         |
    +--------+       +-----------+   XXX    +--------+       XX   +----+----+
    |        |       |           + XXX                     XXX         | Streaming
    |  user  +------->  haproxy  XXXX                     XX           | Replication
    |        |       |           +  XX         web2      XX            |
    +--------+       +-----------+   XXX    +--------+  XX        +----v----+
                                       XXX  + nginx  +  X         |         |
                                         XXXX  RoR   XXXX         |   db2   |
                                            +        +            |         |
                                            +--------+            +---------+
    
    
                                     admin             sendgrid
                                  +----------+        +---------+
                                  |          |        |         |
                                  |   chef   |        |  email  |
                                  |          |        |         |
                                  +----------+        +---------+

* Made with ASCIIFlow Infinity: http://asciiflow.com/

### High Availability

* The database is backed up asynchronously using PostgreSQL streaming
  replication; however, it is not currently a hot-standy.
* Some user-uploaded files are stored on the filesystem instead of in the
  database. These are stored on the primary database server and the web servers
  read/write over NFS. This NFS share is rsync'ed nightly to the backup
  database server.
* If the frontend server is unavailable, create another frontend server
  and then change the DigitalOcean Floating IP address to point to it.

### Administration

* Frontend HTTP requests:
  ssh <EMAIL> "date; tail -f /var/log/haproxy.log" | grep -v -e STATS

* Rails response times:

        ssh <EMAIL> "tail -f /var/log/messages" | grep "response time in millis"

* HAProxy statistics (admin/{passwords/haproxy/stats}): https://myplaceonline.com:9443/
  * /var/log/haproxy.log
  * "8.2.3. HTTP log format" in http://www.haproxy.org/download/1.7/doc/configuration.txt
* Chef administration: https://admin.myplaceonline.com/
* ElasticSearch:
  * curl http://db2-internal.myplaceonline.com:9200/_stats?pretty=1
* Grafana: https://admin.myplaceonline.com:3000/<commit_msg>Update submodules<commit_after># Development

## Ruby

1.  Follow the [Ruby on Rails Coding Conventions](http://guides.rubyonrails.org/contributing_to_ruby_on_rails.html#follow-the-coding-conventions).

    > * Two spaces, no tabs (for indentation).
    > * No trailing whitespace. Blank lines should not have any spaces.
    > * Indent after private/protected.
    > * Use Ruby >= 1.9 syntax for hashes. Prefer { a: :b } over { :a => :b }.
    > * Prefer &&/|| over and/or.
    > * Prefer class << self over self.method for class methods.
    > * Use my_method(my_arg) not my_method( my_arg ) or my_method my_arg.
    > * Use a = b and not a=b.
    > * Use assert_not methods instead of refute.
    > * Prefer method { do_stuff } instead of method{do_stuff} for single-line blocks.
    > * Follow the conventions in the source you see used already.

1.  Pass keyword arguments to methods. This is more verbose and lengthens
    refactoring, but it makes function calls self-descriptive. Ideally, there would 
    be a way to combine
    [arbitrary keyword arguments](https://github.com/ruby/ruby/blob/trunk/doc/syntax/methods.rdoc#keyword-arguments)
    with default values in the function definition, thus allowing transparent
    passing of hashes across functions and avoiding needing to go into the definition
    of the function to find the default; however, Ruby doesn't support this.
    Instead, follow a hybrid approach:
    1.  If a method does not expect to pass its options as a hash
        to another method, then define explicit keyword arguments (with
        required and optional arguments as needed).
        This should be the most common case. Example:
            
            def foo(required_param1:, optional_param1: "default")
              # ... definition ...
            end
            
            foo(required_param1: 1, optional_param1: "test")
            
    1.  If a method expects to pass its options as a hash to another
        method (or might do so in the future), then define an
        arbitrary keyword arguments options hash and set default values
        at the top of the function (if needed). If a parameter is required,
        throw a MissingArgumentError using the check_hash method. Example:
            
            def foo(**options)
              options[:optional_param1] ||= "foo default"
              bar(options)
            end
            
            def bar(**options)
              MissingArgumentError.check_hash(name: :required_param1, hash: options)
              options[:optional_param1] ||= "bar default"
              # ... definition ...
            end
            
            foo(required_param1: 1, optional_param1: "test")
1.  Split long function calls into multiple lines. Example:
        
        foo(
          required_param1: 1,
          required_param2: 2,
          required_param3: 3,
          optional_param1: "test"
        )

## General

* Site tests:
  * https://www.google.com/webmasters/tools/mobile-friendly/

## Architecture

                                               web1
                                            +--------+            +---------+
                                            + nginx  +            +         |
                                         XXXX  RoR   XXXXXXXXXXXXXX   db1   |
                       frontend2       XXX  +        +        XX  +         |
    +--------+       +-----------+   XXX    +--------+       XX   +----+----+
    |        |       |           + XXX                     XXX         | Streaming
    |  user  +------->  haproxy  XXXX                     XX           | Replication
    |        |       |           +  XX         web2      XX            |
    +--------+       +-----------+   XXX    +--------+  XX        +----v----+
                                       XXX  + nginx  +  X         |         |
                                         XXXX  RoR   XXXX         |   db2   |
                                            +        +            |         |
                                            +--------+            +---------+
    
    
                                     admin             sendgrid
                                  +----------+        +---------+
                                  |          |        |         |
                                  |   chef   |        |  email  |
                                  |          |        |         |
                                  +----------+        +---------+

* Made with ASCIIFlow Infinity: http://asciiflow.com/

### High Availability

* The database is backed up asynchronously using PostgreSQL streaming
  replication; however, it is not currently a hot-standy.
* Some user-uploaded files are stored on the filesystem instead of in the
  database. These are stored on the primary database server and the web servers
  read/write over NFS. This NFS share is rsync'ed nightly to the backup
  database server.
* If the frontend server is unavailable, create another frontend server
  and then change the DigitalOcean Floating IP address to point to it.

### Administration

* Frontend HTTP requests:
  ssh root@frontend2.myplaceonline.com "date; tail -f /var/log/haproxy.log" | grep -v -e STATS

* Rails response times:
        ssh root@db2.myplaceonline.com "tail -f /var/log/messages" | grep "response time in millis"

* HAProxy statistics (admin/{passwords/haproxy/stats}): https://myplaceonline.com:9443/
  * /var/log/haproxy.log
  * "8.2.3. HTTP log format" in http://www.haproxy.org/download/1.7/doc/configuration.txt
* Chef administration: https://admin.myplaceonline.com/
* ElasticSearch:
  * curl http://db2-internal.myplaceonline.com:9200/_stats?pretty=1
* Grafana: https://admin.myplaceonline.com:3000/
{'fraction_non_alphanumeric': 0.10342722595273393, 'fraction_numerical': 0.008720676724513822, 'mean_word_length': 1.388669027285982, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 33, 'https://': 10, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 70, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2084144', 'n_tokens_mistral': 3029, 'n_tokens_neox': 2885, 'n_words': 952}