<commit_before><div class="post-header">
  <h1>How We Became the Most Up-To-Date Monitor of Open-Source Github Projects on the Web</h1>
</div>

<div class="post-content" itemprop="articleBody">

  <p>This is the story of how two developers set out to discover the top contributors to open-source. We do this by tracking the top 100,000 most starred repositories on Github. Some stats about the project:</p>
  <ul>
    <li>1,200,000+ Requests Per Day (RPD)</li>
    <li>20,000,000+ Records (and counting)</li>
  </ul>

  <p>We are going to walk you through the major struggles we faced and lessons learned along the way.</p>

  <p>Wait, who are <em>we</em> you ask?</p>

  <% require 'digest' %>

  <% lukas = { name: '<NAME>', url: 'http://numbluk.com/background', email: '<EMAIL>'} %>
  <% michael = { name: '<NAME>', url: 'http://getmichaelonboard.com', email: '<EMAIL>' } %>
  <% info = [ lukas, michael ].shuffle %>

  <div>
    <a href="<%= info[0][:url] %>">
      <img class="gravitar" src="https://secure.gravatar.com/avatar/<%= Digest::MD5.hexdigest(info[0][:email]) %>" />
    </a>

    <a href="<%= info[1][:url] %>">
      <img class="gravitar" src="https://secure.gravatar.com/avatar/<%= Digest::MD5.hexdigest(info[1][:email]) %>" />
    </a>
  </div>

  <p>We are two full stack developers… and <b>we are looking for work!</b> Click our beautiful faces to check out us out!</p>

  <h2 id="table-of-contents">Table of Contents</h2>
  <ol>
    <li><strong>How it All Began</strong></li>
    <li>
      <a href="#getting-started">
        <strong>Getting Started: Developing the Data Acquisition Process</strong>
      </a><commit_msg>Adde full path for Lukas' resume<commit_after><div class="post-header">
  <h1>How We Became the Most Up-To-Date Monitor of Open-Source Github Projects on the Web</h1>
</div>

<div class="post-content" itemprop="articleBody">

  <p>This is the story of how two developers set out to discover the top contributors to open-source. We do this by tracking the top 100,000 most starred repositories on Github. Some stats about the project:</p>
  <ul>
    <li>1,200,000+ Requests Per Day (RPD)</li>
    <li>20,000,000+ Records (and counting)</li>
  </ul>

  <p>We are going to walk you through the major struggles we faced and lessons learned along the way.</p>

  <p>Wait, who are <em>we</em> you ask?</p>

  <% require 'digest' %>

  <% lukas = { name: '<NAME>', url: 'http://www.numbluk.com/background', email: '<EMAIL>'} %>
  <% michael = { name: '<NAME>', url: 'http://getmichaelonboard.com', email: '<EMAIL>' } %>
  <% info = [ lukas, michael ].shuffle %>

  <div>
    <a href="<%= info[0][:url] %>">
      <img class="gravitar" src="https://secure.gravatar.com/avatar/<%= Digest::MD5.hexdigest(info[0][:email]) %>" />
    </a>

    <a href="<%= info[1][:url] %>">
      <img class="gravitar" src="https://secure.gravatar.com/avatar/<%= Digest::MD5.hexdigest(info[1][:email]) %>" />
    </a>
  </div>

  <p>We are two full stack developers… and <b>we are looking for work!</b> Click our beautiful faces to check out us out!</p>

  <h2 id="table-of-contents">Table of Contents</h2>
  <ol>
    <li><strong>How it All Began</strong></li>
    <li>
      <a href="#getting-started">
        <strong>Getting Started: Developing the Data Acquisition Process</strong>
      </a>
{'fraction_non_alphanumeric': 0.1852187028657617, 'fraction_numerical': 0.01870286576168929, 'mean_word_length': 3.716927453769559, 'pattern_counts': {'":': 0, '<': 113, '<?xml version=': 0, '>': 113, 'https://': 4, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7570362', 'n_tokens_mistral': 1245, 'n_tokens_neox': 1147, 'n_words': 342}