<commit_before><% title 'Open Data - Libraries.io' %>

<div class="row">
  <div class="col-md-12">
    <h1>
      <%= fa_icon 'database' %>
      Libraries.io Open Data
    </h1>

    <p>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
    <hr>
    <h4>Get notified of new releases</h4>
    <p>We'll be releasing more open data over the next few months, join the mailing list to be notified via email whenever we publish a new data set.</p>
    <form action="//libraries.us10.list-manage.com/subscribe/post?u=15c11768c94674bc6f18d079f&amp;id=08d43abc2f" method="post" id="mc-embedded-subscribe-form" name="mc-embedded-subscribe-form" class="form-inline" novalidate>
      <input type="email" value="" name="EMAIL" size='20' placeholder='Your email...' class="form-control" id="mce-EMAIL">
      <div style="position: absolute; left: -5000px;"><input type="text" name="b_15c11768c94674bc6f18d079f_08d43abc2f" tabindex="-1" value=""></div>
      <input type="submit" value="Subscribe" name="subscribe" id="mc-embedded-subscribe" class="btn btn-primary">
    </form>
    <hr>
    <h2>License</h2>
    <p>
      Data from Libraries.io is available under the following license:
      <br><a href='https://creativecommons.org/licenses/by-sa/4.0/'>Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0)</a>
    </p>

    <p>The license grants you the following freedoms with this data:</p>

    <ul>
      <li><strong>The freedom to use and perform the work:</strong> The licensee must be allowed to make any use, private or public, of the work. For kinds of works where it is relevant, this freedom should include all derived uses ("related rights") such as performing or interpreting the work. There must be no exception regarding, for example, political or religious considerations.</li>
      <li><strong>The freedom to study the work and apply the information:</strong> The licensee must be allowed to examine the work and to use the knowledge gained from the work in any way. The license may not, for example, restrict "reverse engineering".</li>
      <li><strong>The freedom to redistribute copies:</strong> Copies may be sold, swapped or given away for free, as part of a larger work, a collection, or independently. There must be no limit on the amount of information that can be copied. There must also not be any limit on who can copy the information or on where the information can be copied.</li>
      <li><strong>The freedom to distribute derivative works:</strong> In order to give everyone the ability to improve upon a work, the license must not limit the freedom to distribute a modified version (or, for physical works, a work somehow derived from the original), regardless of the intent and purpose of such modifications. However, some restrictions may be applied to protect these essential freedoms or the attribution of authors.</li>
    </ul>

    <p>The license places three key restrictions on those freedoms:</p>

    <ul>
      <li>You must not <strong>restrict access</strong> to the work using technical measures, or otherwise attempt to impose limitations on the freedoms above.</li>
      <li>You must <strong>give proper attribution to the author and retain the license notice.</strong></li>
      <li>You must <strong>release derivative works under an identical or similar license.</strong></li>
    </ul>

    <p>Full legal text of the license:</p>

    <pre class='well creative_commons'><%= render 'creative_commons' %></pre>

    <hr>
    <h2>Download</h2>

    <p>Latest release: <strong><%= latest_open_data_release_number %> - <%= latest_open_data_release_date %></strong></p>

    <p>Download individual csv files or an archive of all of them, all downloads are gzipped:</p>

    <table class='table table-bordered table-hover'>
      <thead>
        <tr>
          <th>
            File name and digest
          </th>
          <th>
            Archive Size
          </th>
          <th>
            Number of rows
          </th>
          <th>Download link</th>
        </tr>
      </thead>
      <% open_data_release(latest_open_data_release_number)[:files].each do |file| %>
        <tr>
          <td>
            <strong>
              <%= file[:filename] %>
            </strong>
            <br><small class='text-muted'><%= file[:sha256] %></small>
          </td>
          <td>
            <%= file[:size] %>
          </td>
          <td><%= file[:rows] %></td>
          <td>
            <a href="<%= file[:href] %>" class='btn btn-primary'>Download</a>
          </td>
        </tr>
      <% end %>
    </table>

    <h3>Mirrors</h3>

    <p>If you're providing a mirror of this dataset and would like to feature it here, email <EMAIL> or send a pull request!</p>

    <hr>
    <h2>Documentation</h2>

    <%= render 'open_data_documentation' %>
  </div>
</div><commit_msg>Hide the open data download section for now<commit_after><% title 'Open Data - Libraries.io' %>

<div class="row">
  <div class="col-md-12">
    <h1>
      <%= fa_icon 'database' %>
      Libraries.io Open Data
    </h1>

    <p>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
    <hr>
    <h4>Get notified of new releases</h4>
    <p>We'll be releasing more open data over the next few months, join the mailing list to be notified via email whenever we publish a new data set.</p>
    <form action="//libraries.us10.list-manage.com/subscribe/post?u=15c11768c94674bc6f18d079f&amp;id=08d43abc2f" method="post" id="mc-embedded-subscribe-form" name="mc-embedded-subscribe-form" class="form-inline" novalidate>
      <input type="email" value="" name="EMAIL" size='20' placeholder='Your email...' class="form-control" id="mce-EMAIL">
      <div style="position: absolute; left: -5000px;"><input type="text" name="b_15c11768c94674bc6f18d079f_08d43abc2f" tabindex="-1" value=""></div>
      <input type="submit" value="Subscribe" name="subscribe" id="mc-embedded-subscribe" class="btn btn-primary">
    </form>
    <hr>
    <h2>License</h2>
    <p>
      Data from Libraries.io is available under the following license:
      <br><a href='https://creativecommons.org/licenses/by-sa/4.0/'>Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0)</a>
    </p>

    <p>The license grants you the following freedoms with this data:</p>

    <ul>
      <li><strong>The freedom to use and perform the work:</strong> The licensee must be allowed to make any use, private or public, of the work. For kinds of works where it is relevant, this freedom should include all derived uses ("related rights") such as performing or interpreting the work. There must be no exception regarding, for example, political or religious considerations.</li>
      <li><strong>The freedom to study the work and apply the information:</strong> The licensee must be allowed to examine the work and to use the knowledge gained from the work in any way. The license may not, for example, restrict "reverse engineering".</li>
      <li><strong>The freedom to redistribute copies:</strong> Copies may be sold, swapped or given away for free, as part of a larger work, a collection, or independently. There must be no limit on the amount of information that can be copied. There must also not be any limit on who can copy the information or on where the information can be copied.</li>
      <li><strong>The freedom to distribute derivative works:</strong> In order to give everyone the ability to improve upon a work, the license must not limit the freedom to distribute a modified version (or, for physical works, a work somehow derived from the original), regardless of the intent and purpose of such modifications. However, some restrictions may be applied to protect these essential freedoms or the attribution of authors.</li>
    </ul>

    <p>The license places three key restrictions on those freedoms:</p>

    <ul>
      <li>You must not <strong>restrict access</strong> to the work using technical measures, or otherwise attempt to impose limitations on the freedoms above.</li>
      <li>You must <strong>give proper attribution to the author and retain the license notice.</strong></li>
      <li>You must <strong>release derivative works under an identical or similar license.</strong></li>
    </ul>

    <p>Full legal text of the license:</p>

    <pre class='well creative_commons'><%= render 'creative_commons' %></pre>

    <hr>
    <h2>Download</h2>

    <p>We've not released the data yet, come back in a few days!</p>
    <!-- <%= render 'open_data_downloads' %> -->

    <h3>Mirrors</h3>

    <p>If you're providing a mirror of this dataset and would like to feature it here, email <EMAIL> or send a pull request!</p>

    <hr>
    <h2>Documentation</h2>

    <%= render 'open_data_documentation' %>
  </div>
</div>
{'fraction_non_alphanumeric': 0.1079219288174512, 'fraction_numerical': 0.015969105521344326, 'mean_word_length': 3.5240793201133145, 'pattern_counts': {'":': 0, '<': 219, '<?xml version=': 0, '>': 219, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 22, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '227638', 'n_tokens_mistral': 2924, 'n_tokens_neox': 2725, 'n_words': 1115}