<commit_before>{{#contentFor 'head'}}
    <link rel=stylesheet href=/css/about.css>
{{/contentFor}}

<main id=about>

  <section>

    <h2>Introduction</h2>

    <p>
      <dfn>Digital Linguistics</dfn> (<abbr title='Digital Linguistics'>DLx</abbr>) is the science of digital data management for linguistics, including the digital storage, representation, manipulation, and dissemination of linguistic data. It concerns itself with how to represent linguistic data in digital form, as well as best practices for working with that data, using all the benefits of the modern <a href=https://en.wikipedia.org/wiki/Open_Web_Platform rel=noopener>Open Web Platform</a> (<abbr title='Open Web Platform'>OWP</abbr>).
    </p>

    <p>The Digital Linguistics project at the University of California, Santa Barbara has three primary aims:</p>

    <ul class=bulleted>

      <li>
        <p>
          Define a standard data format for storing linguistic data in computer-readable form. This format should be platform- and software-independent (i.e. not restricted to use with Windows or specific software like ELAN), and encode the linguistic concepts that linguists are already familiar with, while still maintaining flexibility and human readability. <a href=https://spec.digitallinguistics.io>You can read more about this format here.</a>
        </p>
      </li>

      <li>
        <p>
          Create a variety of web-based tools that allow linguists to more easily enter, search, and manage their data. All of the DLx tools are open source, and contributions from the community are very welcome. Check out the <a href=https://app.digitallinguistics.io>Lotus web app for managing linguistic data</a> (including texts, lexicons, corpus tagging, and concordance searches), as well as <a href=https://tools.digitallinguistics.io>Tulip (Tools for Linguistic Productivity)</a>—a set of standalone tools for converting and manipulating linguistic data.
        </p>
      </li>

      <li>
        <p>
          Educate linguists about best practices in linguistic data management, and how they can apply the concepts of Digital Linguistics to their own work. <a href=http://docling.net/>Click here to begin learning about the use of Digital Linguistics principles in documentary linguistics.</a>
        </p>
      </li>

    </ul>

  </section>

  <section>

    <h2>Austin Principles of Data Citation</h2>

    <p>
      The <a href=http://site.uit.no/linguisticsdatacitation/>Austin Principles for Data Citation in Linguistics</a> are <q>a set of guidelines that enable linguists to make informed decisions regarding the accessibility and transparency of their research data.</q> (<a href=http://site.uit.no/linguisticsdatacitation/>source</a>) The Data Format for Digital Linguistics both aligns with and helps foster the Austin Principles:
    </p>

    <ol class=numbered>

      <li>

        <h3>Importance</h3>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Data should be considered legitimate, citable products of research.
        </blockquote>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          the data on which linguistic analyses are based are of fundamental importance to the field and should be treated as such […] Linguistic data should be citable and cited
        </blockquote>

        <p>
          The Digital Linguistics enterprise is premised on the idea that primary linguistic data is of fundamental importance to the field of linguistics, and that the management of such data merits careful attention.
        </p>

      </li>

      <li>

        <h3>Credit &amp; Attribution</h3>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Data citations should facilitate giving scholarly credit and normative and legal attribution to all contributors to the data
        </blockquote>

        <p>
          The Data Format for Digital Linguistics includes recommendations regarding how to store information about the people who contributed to the production and management of the data, and their role(s) in the process.
        </p>

      </li>

      <li>

        <h3>Evidence</h3>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Linguists should cite the data upon which scholarly claims are based. In order for data to be citable, it should be stored in an accessible location, preferably a data archive or other trusted repository.
        </blockquote>

        <p>
          The DLx format, being web-compatible, makes it possible to publish data sets on the web at stable URLs for consumption and citation. Since the JSON format used by DLx is a simple text format, it can easily be stored in any archive or online database.
        </p>

      </li>

      <li>

        <h3>Unique Identification</h3>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          A data citation should include a persistent method for identification that is machine actionable, globally unique, and widely used by a community.
        </blockquote>

        <p>
          Each of the DLx linguistic schemas allows for both URL and ID fields which are unique identifiers, so that each linguistic object can be permanently associated with a unique URL, and/or unique ID in a database.
        </p>

      </li>

      <li>

        <h3>Access</h3>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Data citations should facilitate access to the data themselves and to such associated metadata […] as are necessary for both humans and machines to make informed use of the referenced data.
        </blockquote>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Linguistic data should be as open as possible, in order to facilitate reproducibility; and as closed as necessary, to honor relevant ethical, legal, and speaker community constraints.
        </blockquote>

        <p>
          The use of JSON format for Digital Linguistics means that data is both human- and machine-readable. The Data Format for Digital Linguistics also makes recommendations regarding how to indicate access rights to various kinds of linguistic data, at every level of granularity.
        </p>

      </li>

      <li>

        <h3>Persistence</h3>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Unique identifiers, and metadata describing the data, and its disposition, should persist — even beyond the lifespan of the data they describe.
        </blockquote>

        <p>
          The cheap cost of cloud storage today makes it possible to continue storing metadata about the objects of language documentation even when that data no longer exists, is no longer public, or becomes too large for its repository.
        </p>

      </li>

      <li>

        <h3>Specificity &amp; Verifiability</h3>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Data citations should make it easy for a curious reader to find the specific datum or subset of data within the larger dataset that support a claim.
        </blockquote>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Citations should specify which version of the data is being referenced.
        </blockquote>

        <p>
          The Data Format for Digital Linguistics provides a convention for assigning human-readable keys to each piece of data, and every level of granularity, allowing interested parties to easily reference or look up data at the level of the text, utterance, word, morpheme, and phoneme.
        </p>

      </li>

      <li>

        <h3>Interoperability &amp; Flexibility</h3>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Data citation methods should be sufficiently flexibile to accommodate the variant practices among communities, but should not differ so much that they compromise interoperability
        </blockquote>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Citation standards developed for linguistics need to meet the needs of the research community, while also meeting the principles described above.
        </blockquote>

        <p>
          The Data Format for Digital Linguistics is highly interoperable because it is based on JSON, which has become the standard format for interchanging data on the web. Using web-based tools for Digital Linguistics also means that user interfaces can be designed to allow users to interact with the data in the ways that are most comfortable to them. Using open source tools and software also facilitates the creation of new tools to meet the needs of users.
        </p>

      </li>

    </ol>

  </section>

</main>
<commit_msg>CONTENT: spec.digitallinguistics.io > format.digitallinguistics.io (#156)<commit_after>{{#contentFor 'head'}}
    <link rel=stylesheet href=/css/about.css>
{{/contentFor}}

<main id=about>

  <section>

    <h2>Introduction</h2>

    <p>
      <dfn>Digital Linguistics</dfn> (<abbr title='Digital Linguistics'>DLx</abbr>) is the science of digital data management for linguistics, including the digital storage, representation, manipulation, and dissemination of linguistic data. It concerns itself with how to represent linguistic data in digital form, as well as best practices for working with that data, using all the benefits of the modern <a href=https://en.wikipedia.org/wiki/Open_Web_Platform rel=noopener>Open Web Platform</a> (<abbr title='Open Web Platform'>OWP</abbr>).
    </p>

    <p>The Digital Linguistics project at the University of California, Santa Barbara has three primary aims:</p>

    <ul class=bulleted>

      <li>
        <p>
          Define a standard data format for storing linguistic data in computer-readable form. This format should be platform- and software-independent (i.e. not restricted to use with Windows or specific software like ELAN), and encode the linguistic concepts that linguists are already familiar with, while still maintaining flexibility and human readability. <a href=https://format.digitallinguistics.io>You can read more about this format here.</a>
        </p>
      </li>

      <li>
        <p>
          Create a variety of web-based tools that allow linguists to more easily enter, search, and manage their data. All of the DLx tools are open source, and contributions from the community are very welcome. Check out the <a href=https://app.digitallinguistics.io>Lotus web app for managing linguistic data</a> (including texts, lexicons, corpus tagging, and concordance searches), as well as <a href=https://tools.digitallinguistics.io>Tulip (Tools for Linguistic Productivity)</a>—a set of standalone tools for converting and manipulating linguistic data.
        </p>
      </li>

      <li>
        <p>
          Educate linguists about best practices in linguistic data management, and how they can apply the concepts of Digital Linguistics to their own work. <a href=http://docling.net/>Click here to begin learning about the use of Digital Linguistics principles in documentary linguistics.</a>
        </p>
      </li>

    </ul>

  </section>

  <section>

    <h2>Austin Principles of Data Citation</h2>

    <p>
      The <a href=http://site.uit.no/linguisticsdatacitation/>Austin Principles for Data Citation in Linguistics</a> are <q>a set of guidelines that enable linguists to make informed decisions regarding the accessibility and transparency of their research data.</q> (<a href=http://site.uit.no/linguisticsdatacitation/>source</a>) The Data Format for Digital Linguistics both aligns with and helps foster the Austin Principles:
    </p>

    <ol class=numbered>

      <li>

        <h3>Importance</h3>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Data should be considered legitimate, citable products of research.
        </blockquote>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          the data on which linguistic analyses are based are of fundamental importance to the field and should be treated as such […] Linguistic data should be citable and cited
        </blockquote>

        <p>
          The Digital Linguistics enterprise is premised on the idea that primary linguistic data is of fundamental importance to the field of linguistics, and that the management of such data merits careful attention.
        </p>

      </li>

      <li>

        <h3>Credit &amp; Attribution</h3>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Data citations should facilitate giving scholarly credit and normative and legal attribution to all contributors to the data
        </blockquote>

        <p>
          The Data Format for Digital Linguistics includes recommendations regarding how to store information about the people who contributed to the production and management of the data, and their role(s) in the process.
        </p>

      </li>

      <li>

        <h3>Evidence</h3>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Linguists should cite the data upon which scholarly claims are based. In order for data to be citable, it should be stored in an accessible location, preferably a data archive or other trusted repository.
        </blockquote>

        <p>
          The DLx format, being web-compatible, makes it possible to publish data sets on the web at stable URLs for consumption and citation. Since the JSON format used by DLx is a simple text format, it can easily be stored in any archive or online database.
        </p>

      </li>

      <li>

        <h3>Unique Identification</h3>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          A data citation should include a persistent method for identification that is machine actionable, globally unique, and widely used by a community.
        </blockquote>

        <p>
          Each of the DLx linguistic schemas allows for both URL and ID fields which are unique identifiers, so that each linguistic object can be permanently associated with a unique URL, and/or unique ID in a database.
        </p>

      </li>

      <li>

        <h3>Access</h3>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Data citations should facilitate access to the data themselves and to such associated metadata […] as are necessary for both humans and machines to make informed use of the referenced data.
        </blockquote>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Linguistic data should be as open as possible, in order to facilitate reproducibility; and as closed as necessary, to honor relevant ethical, legal, and speaker community constraints.
        </blockquote>

        <p>
          The use of JSON format for Digital Linguistics means that data is both human- and machine-readable. The Data Format for Digital Linguistics also makes recommendations regarding how to indicate access rights to various kinds of linguistic data, at every level of granularity.
        </p>

      </li>

      <li>

        <h3>Persistence</h3>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Unique identifiers, and metadata describing the data, and its disposition, should persist — even beyond the lifespan of the data they describe.
        </blockquote>

        <p>
          The cheap cost of cloud storage today makes it possible to continue storing metadata about the objects of language documentation even when that data no longer exists, is no longer public, or becomes too large for its repository.
        </p>

      </li>

      <li>

        <h3>Specificity &amp; Verifiability</h3>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Data citations should make it easy for a curious reader to find the specific datum or subset of data within the larger dataset that support a claim.
        </blockquote>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Citations should specify which version of the data is being referenced.
        </blockquote>

        <p>
          The Data Format for Digital Linguistics provides a convention for assigning human-readable keys to each piece of data, and every level of granularity, allowing interested parties to easily reference or look up data at the level of the text, utterance, word, morpheme, and phoneme.
        </p>

      </li>

      <li>

        <h3>Interoperability &amp; Flexibility</h3>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Data citation methods should be sufficiently flexibile to accommodate the variant practices among communities, but should not differ so much that they compromise interoperability
        </blockquote>

        <blockquote class=blockquote cite=http://site.uit.no/linguisticsdatacitation/austinprinciples/>
          Citation standards developed for linguistics need to meet the needs of the research community, while also meeting the principles described above.
        </blockquote>

        <p>
          The Data Format for Digital Linguistics is highly interoperable because it is based on JSON, which has become the standard format for interchanging data on the web. Using web-based tools for Digital Linguistics also means that user interfaces can be designed to allow users to interact with the data in the ways that are most comfortable to them. Using open source tools and software also facilitates the creation of new tools to meet the needs of users.
        </p>

      </li>

    </ol>

  </section>

</main>

{'fraction_non_alphanumeric': 0.06933520094817369, 'fraction_numerical': 0.0023165607154401464, 'mean_word_length': 3.200724145734329, 'pattern_counts': {'":': 0, '<': 257, '<?xml version=': 0, '>': 258, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1434878', 'n_tokens_mistral': 4900, 'n_tokens_neox': 4537, 'n_words': 2105}