<commit_before>---
layout: documentation
title: Concepts and terminology
category: getting-started
---
<h1>Concepts and Terminology</h1>
<p>
  This page provides an overview of the source tree layout and the
  terminology used in Bazel.
</p>
<h2 id="intro">Introduction</h2>

<p>Bazel builds software from source code organized in a directory called
   a workspace. Source files in the workspace are organized in a nested
   hierarchy of packages, where each package is a directory that contains a set
   of related source files and one BUILD file. The BUILD file specifies what
   software outputs can be built from the source.
</p>
<h2 id="packages_targets">Workspace, packages, and targets</h2>
<h3 id="workspace">Workspace</h3>

<p>A <em>workspace</em> is a directory on your filesystem that contains the
   source files for the software you want to build, as well as symbolic links
   to directories that contain the build outputs. Each workspace directory has
   a text file named <code>WORKSPACE</code> which may be empty, or may contain
   references to <a href="external.html">external dependencies</a>
   required to build the outputs.</p>

<p>Directories containing a file called
   <code>WORKSPACE</code> are considered the root of a workspace.
   Therefore, Bazel ignores any directory trees in a workspace rooted
   at a subdirectory containing a <code>WORKSPACE</code> file (as they form
   another workspace).</p>

<p>Bazel also supports <code>WORKSPACE.bazel</code> file as an alias of <code>WORKSPACE</code> file.
   If both files exist, <code>WORKSPACE.bazel</code> will take the priority.</p>

<h3 id="repositories">Repositories</h3>
<p>Code is organized in <em>repositories</em>. The directory containing
   the <code>WORKSPACE</code> file is the root of the main repository, also
   called <code>@</code>. Other, (external) repositories
   are defined in the <code>WORKSPACE</code> file using workspace rules.

<p>The workspace rules bundled with Bazel are documented in the
   <a href="be/workspace.html">Workspace Rules</a> section in the
   <a href="be/overview.html">Build Encyclopedia</a> and the documentation on
   <a href="repo/index.html">embedded Starlark repository rules</a>.</p>

<p>As external repositories are repositories themselves, they often contain
   a <code>WORKSPACE</code> file as well. However, these additional
   <code>WORKSPACE</code> files are ignored by Bazel. In particular,
   repositories depended upon transitively are not added automatically.</p>

<h3 id="packages">Packages</h3>
<p>
  The primary unit of code organization in a repository is
  the <i>package</i>. A package is a collection of related files and a
  specification of the dependencies among them.
</p>

<p>
  A package is defined as a directory containing a file
  named <code>BUILD</code> or <code>BUILD.bazel</code>,
  residing beneath the top-level directory in the
  workspace. A package includes all files in its directory, plus all
  subdirectories beneath it, except those which themselves contain a BUILD
  file.
</p>

<p>
  For example, in the following directory tree
  there are two packages, <code>my/app</code>,
  and the subpackage <code>my/app/tests</code>.
  Note that <code>my/app/data</code> is not a package, but a directory
  belonging to package <code>my/app</code>.
</p>

<pre>
src/my/app/BUILD
src/my/app/app.cc
src/my/app/data/input.txt
src/my/app/tests/BUILD
src/my/app/tests/test.cc
</pre>
<h3 id="targets">Targets</h3>

<p>
  A package is a container. The elements of a package are called
  <i>targets</i>. Most targets are one of two principal kinds, <i>files</i>
  and <i>rules</i>. Additionally, there is another kind of target,
  <a href="be/functions.html#package_group">package groups</a>,
  but they are far less numerous.
</p>

<p>
  Files are further divided into two kinds.
  <i>Source files</i> are usually written by the efforts of people,
  and checked in to the repository.
  <i>Generated files</i>, sometimes called derived files,
  are not checked in, but are generated by the build tool from source
  files according to specific rules.
</p>

<p>
  The second kind of target is the <i>rule</i>. A rule specifies the
  relationship between a set of input and a set of output files,
  including the necessary steps to derive the outputs from the inputs.
  The outputs of a rule are always generated files. The inputs to a
  rule may be source files, but they may be generated files also;
  consequently, outputs of one rule may be the inputs to another,
  allowing long chains of rules to be constructed.
</p>

<p>
  Whether the input to a rule is a source file or a generated file is
  in most cases immaterial; what matters is only the contents of that
  file. This fact makes it easy to replace a complex source file with
  a generated file produced by a rule, such as happens when the burden
  of manually maintaining a highly structured file becomes too
  tiresome, and someone writes a program to derive it. No change is
  required to the consumers of that file. Conversely, a generated
  file may easily be replaced by a source file with only local
  changes.
</p>

<p>
  The inputs to a rule may also include <i>other rules</i>. The
  precise meaning of such relationships is often quite complex and
  language- or rule-dependent, but intuitively it is simple: a C++
  library rule A might have another C++ library rule B for an input.
  The effect of this dependency is that B's header files are
  available to A during compilation, B's symbols are available to A
  during linking, and B's runtime data is available to A during
  execution.
</p>

<p>
  An invariant of all rules is that the files generated by a rule
  always belong to the same package as the rule itself; it is not
  possible to generate files into another package. It is not uncommon
  for a rule's inputs to come from another package, though.
</p>

<p>
  Package groups are sets of packages whose purpose is to limit accessibility
  of certain rules. Package groups are defined by the
  <code>package_group</code> function. They have two properties: the list of
  packages they contain and their name. The only allowed ways to refer to them
  are from the <code>visibility</code> attribute of rules or from the
  <code>default_visibility</code> attribute of the <code>package</code>
  function; they do not generate or consume files. For more information, refer
  to the <a
  href='be/functions.html#package_group'><code>package_group</code>
  documentation</a>.
</p>


<h3 id="labels">Labels</h3>

<p>
  All targets belong to exactly one package. The name of a target is
  called its <em>label</em>.  Every label uniquely identifies a target. A
  typical label in canonical form looks like:
</p>


<pre>
@myrepo//my/app/main:app_binary
</pre>

<p>
  The first part of the label is the repository name, <code>@myrepo//</code>.
  In the typical case that a label refers to the same repository it is used
  in, the repository identifier may be abbreviated as <code>//</code>. So, inside
  <code>@myrepo</code> this label is usually written as
</p>

<pre>
//my/app/main:app_binary
</pre>

<p>

  The second part of the label is the un-qualified package name
  <code>my/app/main</code>, the path to the package
  relative to the repository root.  When the label refers to the same
  package it is used in, the package name (and optionally, the colon)
  may be omitted.  So, inside <code>@myrepo//my/app/main</code>,
  this label may be written either of the following ways:

</p>

<pre>
app_binary
:app_binary
</pre>

<p>
  It is a matter of convention that the colon is omitted for files,
  but retained for rules, but it is not otherwise significant.
</p>

<p>
  The part of the label after
  the colon, <code>app_binary</code> is the un-qualified target name.
  When it matches the last component of the package path, it, and
  the colon, may be omitted.  So, these two labels are
  equivalent:
</p>

<pre>
//my/app/lib
//my/app/lib:lib
</pre>

<p>
  The name of a file target in a subdirectory of the
  package is the file's path relative to the package
  root (the directory containing the <code>BUILD</code> file).
  So, this file is in the <code>my/app/main/testdata</code>
  subdirectory of the repository:
</p>

<pre>
//my/app/main:testdata/input.txt
</pre>

<p>
  Don't confuse labels like <code>//my/app</code> with package names. Labels
  always start with a repository identifier (often abbreviated <code>//</code>)
  but package names never do. Thus, <code>my/app</code> is the package
  containing <code>//my/app/lib</code> (a.k.a.
  <code>//my/app/lib:lib</code>).

  (A common misconception is that <code>//my/app</code> refers
  to a package, or to <em>all</em> the targets in a package; neither
  is true.  Remember, it is equivalent to `//my/app:app`, so it names
  the `app` target in the `my/app` package of the current repository).
</p>

<p>
  Relative labels cannot be used to refer to targets in other
  packages; the repository identifier and package name must
  always be specified in this case. For example, if the source
  tree contains both the package
  <code>my/app</code> and the package
  <code>my/app/testdata</code> (i.e., each of these two
  packages has its own BUILD file). The latter package contains a
  file named <code>testdepot.zip</code>. Here are two ways (one
  wrong, one correct) to refer to this file within
  <code>//my/app:BUILD</code>:
</p>

<pre>
<span class="discouraged">testdata/testdepot.zip</span>  # Wrong: testdata is a different package.
//my/app/testdata:testdepot.zip   # Right.
</pre>

<p>
  If, by mistake, you refer to <code>testdepot.zip</code> by the wrong
  label, such as <code>//my/app:testdata/testdepot.zip</code>
  or <code>//my:app/testdata/testdepot.zip</code>, you will get an
  error from the build tool saying that the label "crosses a package
  boundary". You should correct the label by putting the colon after
  the directory containing the innermost enclosing BUILD file, i.e.,
  <code>//my/app/testdata:testdepot.zip</code>.
</p>

<p>
  Labels starting with <code>@//</code> are references to the main
  repository, which will still work even from external repositories.
  Therefore <code>@//a/b/c</code> is different from
  <code>//a/b/c</code> when referenced from an external repository.
  The former refers back to the main repository, while the latter
  looks for <code>//a/b/c</code> in the external repository itself.
  This is especially relevant when writing rules in the main
  repository that refer to targets in the main repository, and will be
  used from external repositories.
</p>

<h3 id="lexi">Lexical specification of a label</h3>

<p>
  The syntax of labels is intentionally strict, so as to
  forbid metacharacters that have special meaning to the shell. This
  helps to avoid inadvertent quoting problems, and makes it easier to
  construct tools and scripts that manipulate labels, such as the

  <a href='query.html'>Bazel Query Language</a>.

  The precise details of allowed target names are below.
</p>

<h4 id="name">Target names, <code>//...:<b>target-name</b></code></h4>

<p><code>target-name</code> is the name of the target within the package.
  The name of a rule is the value of the <code>name</code>
  attribute in the rule's declaration in a BUILD file; the name
  of a file is its pathname relative to the directory containing
  the BUILD file.
  Target names must be composed entirely of
  characters drawn from the set <code>a</code>–<code>z</code>,
  <code>A</code>–<code>Z</code>, <code>0</code>–<code>9</code>,
  and the punctuation symbols <code>!%-@^_` "#$&'()*-+,;<=>?[]{|}~/.</code>.
  Do not use <code>..</code> to refer to files in other packages; use
  <code>//<var>packagename</var>:<var>filename</var></code> instead.
  Filenames must be relative pathnames in normal form, which means
  they must neither start nor end with a slash
  (e.g. <code>/foo</code> and <code>foo/</code> are forbidden) nor
  contain multiple consecutive slashes as path separators
  (e.g. <code>foo//bar</code>). Similarly, up-level references
  (<code>..</code>) and current-directory references
  (<code>./</code>) are forbidden. The sole exception to this
  rule is that a target name may consist of exactly
  '<code>.</code>'.
</p>

<p>While it is common to use <code>/</code> in the name of a file
  target, we recommend that you avoid the use of <code>/</code> in the
  names of rules. Especially when the shorthand form of a label is
  used, it may confuse the reader. The
  label <code>//foo/bar/wiz</code> is always a shorthand
  for <code>//foo/bar/wiz:wiz</code>, even if there is no such package
  <code>foo/bar/wiz</code>; it never refers to <code>//foo:bar/wiz</code>,
  even if that target exists.</p>

<p>However, there are some situations where use of a slash is
  convenient, or sometimes even necessary. For example, the name of
  certain rules must match their principal source file, which may
  reside in a subdirectory of the package.</p>

<h4>Package names, <code>//<b>package-name</b>:...</code></h4>
<p>
  The name of a package is the name of the directory containing its

  BUILD file, relative to the top-level directory of the containing
  repository. For example: <code>my/app</code>.

  Package names must be composed entirely of characters drawn from
  the set <code>A</code>-<code>Z</code>, <code>a</code>–<code>z</code>,
  <code>0</code>–<code>9</code>, '<code>/</code>', '<code>-</code>',
  '<code>.</code>', and '<code>_</code>', and cannot start with
  a slash.
<p>
  For a language with a directory structure that is significant
  to its module system (e.g. Java), it is important to choose directory names
  that are valid identifiers in the language.
</p>

<p>
  Although Bazel allows a package at the build root (e.g. <code>//:foo</code>), this
  is not advised and projects should attempt to use more descriptively named
  packages.
</p>
<p><commit_msg>Clarify that a label names a target.<commit_after>---
layout: documentation
title: Concepts and terminology
category: getting-started
---
<h1>Concepts and Terminology</h1>
<p>
  This page provides an overview of the source tree layout and the
  terminology used in Bazel.
</p>
<h2 id="intro">Introduction</h2>

<p>Bazel builds software from source code organized in a directory called
   a workspace. Source files in the workspace are organized in a nested
   hierarchy of packages, where each package is a directory that contains a set
   of related source files and one BUILD file. The BUILD file specifies what
   software outputs can be built from the source.
</p>
<h2 id="packages_targets">Workspace, packages, and targets</h2>
<h3 id="workspace">Workspace</h3>

<p>A <em>workspace</em> is a directory on your filesystem that contains the
   source files for the software you want to build, as well as symbolic links
   to directories that contain the build outputs. Each workspace directory has
   a text file named <code>WORKSPACE</code> which may be empty, or may contain
   references to <a href="external.html">external dependencies</a>
   required to build the outputs.</p>

<p>Directories containing a file called
   <code>WORKSPACE</code> are considered the root of a workspace.
   Therefore, Bazel ignores any directory trees in a workspace rooted
   at a subdirectory containing a <code>WORKSPACE</code> file (as they form
   another workspace).</p>

<p>Bazel also supports <code>WORKSPACE.bazel</code> file as an alias of <code>WORKSPACE</code> file.
   If both files exist, <code>WORKSPACE.bazel</code> will take the priority.</p>

<h3 id="repositories">Repositories</h3>
<p>Code is organized in <em>repositories</em>. The directory containing
   the <code>WORKSPACE</code> file is the root of the main repository, also
   called <code>@</code>. Other, (external) repositories
   are defined in the <code>WORKSPACE</code> file using workspace rules.

<p>The workspace rules bundled with Bazel are documented in the
   <a href="be/workspace.html">Workspace Rules</a> section in the
   <a href="be/overview.html">Build Encyclopedia</a> and the documentation on
   <a href="repo/index.html">embedded Starlark repository rules</a>.</p>

<p>As external repositories are repositories themselves, they often contain
   a <code>WORKSPACE</code> file as well. However, these additional
   <code>WORKSPACE</code> files are ignored by Bazel. In particular,
   repositories depended upon transitively are not added automatically.</p>

<h3 id="packages">Packages</h3>
<p>
  The primary unit of code organization in a repository is
  the <i>package</i>. A package is a collection of related files and a
  specification of the dependencies among them.
</p>

<p>
  A package is defined as a directory containing a file
  named <code>BUILD</code> or <code>BUILD.bazel</code>,
  residing beneath the top-level directory in the
  workspace. A package includes all files in its directory, plus all
  subdirectories beneath it, except those which themselves contain a BUILD
  file.
</p>

<p>
  For example, in the following directory tree
  there are two packages, <code>my/app</code>,
  and the subpackage <code>my/app/tests</code>.
  Note that <code>my/app/data</code> is not a package, but a directory
  belonging to package <code>my/app</code>.
</p>

<pre>
src/my/app/BUILD
src/my/app/app.cc
src/my/app/data/input.txt
src/my/app/tests/BUILD
src/my/app/tests/test.cc
</pre>
<h3 id="targets">Targets</h3>

<p>
  A package is a container. The elements of a package are called
  <i>targets</i>. Most targets are one of two principal kinds, <i>files</i>
  and <i>rules</i>. Additionally, there is another kind of target,
  <a href="be/functions.html#package_group">package groups</a>,
  but they are far less numerous.
</p>

<p>
  Files are further divided into two kinds.
  <i>Source files</i> are usually written by the efforts of people,
  and checked in to the repository.
  <i>Generated files</i>, sometimes called derived files,
  are not checked in, but are generated by the build tool from source
  files according to specific rules.
</p>

<p>
  The second kind of target is the <i>rule</i>. A rule specifies the
  relationship between a set of input and a set of output files,
  including the necessary steps to derive the outputs from the inputs.
  The outputs of a rule are always generated files. The inputs to a
  rule may be source files, but they may be generated files also;
  consequently, outputs of one rule may be the inputs to another,
  allowing long chains of rules to be constructed.
</p>

<p>
  Whether the input to a rule is a source file or a generated file is
  in most cases immaterial; what matters is only the contents of that
  file. This fact makes it easy to replace a complex source file with
  a generated file produced by a rule, such as happens when the burden
  of manually maintaining a highly structured file becomes too
  tiresome, and someone writes a program to derive it. No change is
  required to the consumers of that file. Conversely, a generated
  file may easily be replaced by a source file with only local
  changes.
</p>

<p>
  The inputs to a rule may also include <i>other rules</i>. The
  precise meaning of such relationships is often quite complex and
  language- or rule-dependent, but intuitively it is simple: a C++
  library rule A might have another C++ library rule B for an input.
  The effect of this dependency is that B's header files are
  available to A during compilation, B's symbols are available to A
  during linking, and B's runtime data is available to A during
  execution.
</p>

<p>
  An invariant of all rules is that the files generated by a rule
  always belong to the same package as the rule itself; it is not
  possible to generate files into another package. It is not uncommon
  for a rule's inputs to come from another package, though.
</p>

<p>
  Package groups are sets of packages whose purpose is to limit accessibility
  of certain rules. Package groups are defined by the
  <code>package_group</code> function. They have two properties: the list of
  packages they contain and their name. The only allowed ways to refer to them
  are from the <code>visibility</code> attribute of rules or from the
  <code>default_visibility</code> attribute of the <code>package</code>
  function; they do not generate or consume files. For more information, refer
  to the <a
  href='be/functions.html#package_group'><code>package_group</code>
  documentation</a>.
</p>


<h3 id="labels">Labels</h3>

<p>
  All targets belong to exactly one package. The name of a target is
  called its <em>label</em>.  Every label uniquely identifies a target. A
  typical label in canonical form looks like:
</p>


<pre>
@myrepo//my/app/main:app_binary
</pre>

<p>
  The first part of the label is the repository name, <code>@myrepo//</code>.
  In the typical case that a label refers to the same repository it is used
  in, the repository identifier may be abbreviated as <code>//</code>. So, inside
  <code>@myrepo</code> this label is usually written as
</p>

<pre>
//my/app/main:app_binary
</pre>

<p>

  The second part of the label is the un-qualified package name
  <code>my/app/main</code>, the path to the package
  relative to the repository root.  When the label refers to the same
  package it is used in, the package name (and optionally, the colon)
  may be omitted.  So, inside <code>@myrepo//my/app/main</code>,
  this label may be written either of the following ways:

</p>

<pre>
app_binary
:app_binary
</pre>

<p>
  It is a matter of convention that the colon is omitted for files,
  but retained for rules, but it is not otherwise significant.
</p>

<p>
  The part of the label after
  the colon, <code>app_binary</code> is the un-qualified target name.
  When it matches the last component of the package path, it, and
  the colon, may be omitted.  So, these two labels are
  equivalent:
</p>

<pre>
//my/app/lib
//my/app/lib:lib
</pre>

<p>
  The name of a file target in a subdirectory of the
  package is the file's path relative to the package
  root (the directory containing the <code>BUILD</code> file).
  So, this file is in the <code>my/app/main/testdata</code>
  subdirectory of the repository:
</p>

<pre>
//my/app/main:testdata/input.txt
</pre>

<p>
  Don't confuse labels like <code>//my/app</code> with package names. Labels
  always start with a repository identifier (often abbreviated <code>//</code>)
  but package names never do. Thus, <code>my/app</code> is the package
  containing <code>//my/app/lib</code> (a.k.a.
  <code>//my/app/lib:lib</code>).

  (A common misconception is that <code>//my/app</code> refers
  to a package, or to <em>all</em> the targets in a package; neither
  is true.  Remember, it is equivalent to `//my/app:app`, so it names
  the `app` target in the `my/app` package of the current repository).
</p>

<p>
  Relative labels cannot be used to refer to targets in other
  packages; the repository identifier and package name must
  always be specified in this case. For example, if the source
  tree contains both the package
  <code>my/app</code> and the package
  <code>my/app/testdata</code> (i.e., each of these two
  packages has its own BUILD file). The latter package contains a
  file named <code>testdepot.zip</code>. Here are two ways (one
  wrong, one correct) to refer to this file within
  <code>//my/app:BUILD</code>:
</p>

<pre>
<span class="discouraged">testdata/testdepot.zip</span>  # Wrong: testdata is a different package.
//my/app/testdata:testdepot.zip   # Right.
</pre>

<p>
  If, by mistake, you refer to <code>testdepot.zip</code> by the wrong
  label, such as <code>//my/app:testdata/testdepot.zip</code>
  or <code>//my:app/testdata/testdepot.zip</code>, you will get an
  error from the build tool saying that the label "crosses a package
  boundary". You should correct the label by putting the colon after
  the directory containing the innermost enclosing BUILD file, i.e.,
  <code>//my/app/testdata:testdepot.zip</code>.
</p>

<p>
  Labels starting with <code>@//</code> are references to the main
  repository, which will still work even from external repositories.
  Therefore <code>@//a/b/c</code> is different from
  <code>//a/b/c</code> when referenced from an external repository.
  The former refers back to the main repository, while the latter
  looks for <code>//a/b/c</code> in the external repository itself.
  This is especially relevant when writing rules in the main
  repository that refer to targets in the main repository, and will be
  used from external repositories.
</p>

<h3 id="lexi">Lexical specification of a label</h3>

<p>
  The syntax of labels is intentionally strict, so as to
  forbid metacharacters that have special meaning to the shell. This
  helps to avoid inadvertent quoting problems, and makes it easier to
  construct tools and scripts that manipulate labels, such as the

  <a href='query.html'>Bazel Query Language</a>.

  The precise details of allowed target names are below.
</p>

<h4 id="name">Target names, <code>//...:<b>target-name</b></code></h4>

<p><code>target-name</code> is the name of the target within the package.
  The name of a rule is the value of the <code>name</code>
  attribute in the rule's declaration in a BUILD file; the name
  of a file is its pathname relative to the directory containing
  the BUILD file.
  Target names must be composed entirely of
  characters drawn from the set <code>a</code>–<code>z</code>,
  <code>A</code>–<code>Z</code>, <code>0</code>–<code>9</code>,
  and the punctuation symbols <code>!%-@^_` "#$&'()*-+,;<=>?[]{|}~/.</code>.
  Do not use <code>..</code> to refer to files in other packages; use
  <code>//<var>packagename</var>:<var>filename</var></code> instead.
  Filenames must be relative pathnames in normal form, which means
  they must neither start nor end with a slash
  (e.g. <code>/foo</code> and <code>foo/</code> are forbidden) nor
  contain multiple consecutive slashes as path separators
  (e.g. <code>foo//bar</code>). Similarly, up-level references
  (<code>..</code>) and current-directory references
  (<code>./</code>) are forbidden. The sole exception to this
  rule is that a target name may consist of exactly
  '<code>.</code>'.
</p>

<p>While it is common to use <code>/</code> in the name of a file
  target, we recommend that you avoid the use of <code>/</code> in the
  names of rules. Especially when the shorthand form of a label is
  used, it may confuse the reader. The
  label <code>//foo/bar/wiz</code> is always a shorthand
  for <code>//foo/bar/wiz:wiz</code>, even if there is no such package
  <code>foo/bar/wiz</code>; it never refers to <code>//foo:bar/wiz</code>,
  even if that target exists.</p>

<p>However, there are some situations where use of a slash is
  convenient, or sometimes even necessary. For example, the name of
  certain rules must match their principal source file, which may
  reside in a subdirectory of the package.</p>

<h4>Package names, <code>//<b>package-name</b>:...</code></h4>
<p>
  The name of a package is the name of the directory containing its

  BUILD file, relative to the top-level directory of the containing
  repository. For example: <code>my/app</code>.

  Package names must be composed entirely of characters drawn from
  the set <code>A</code>-<code>Z</code>, <code>a</code>–<code>z</code>,
  <code>0</code>–<code>9</code>, '<code>/</code>', '<code>-</code>',
  '<code>.</code>', and '<code>_</code>', and cannot start with
  a slash.
<p>
  For a language with a directory structure that is significant
  to its module system (e.g. Java), it is important to choose directory names
  that are valid identifiers in the language.
</p>

<p>
  Although Bazel supports targets in the workspace's root package
  (e.g. <code>//:foo</code>), it's best to leave that package empty
  so all meaningful packages have descriptive names.
</p>
<p>
{'fraction_non_alphanumeric': 0.10166153405428632, 'fraction_numerical': 0.0018620640263553678, 'mean_word_length': 3.8943217665615144, 'pattern_counts': {'":': 0, '<': 651, '<?xml version=': 0, '>': 651, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3794720', 'n_tokens_mistral': 8432, 'n_tokens_neox': 8286, 'n_words': 3791}