<commit_before>============
Coding style
============

.. contents::

Naming conventions
==================

When reading libvirt code, a number of different naming
conventions will be evident due to various changes in thinking
over the course of the project's lifetime. The conventions
documented below should be followed when creating any entirely new
files in libvirt. When working on existing files, while it is
desirable to apply these conventions, keeping a consistent style
with existing code in that particular file is generally more
important. The overall guiding principal is that every file, enum,
struct, function, macro and typedef name must have a 'vir' or
'VIR' prefix. All local scope variable names are exempt, and
global variables are exempt, unless exported in a header file.

File names
   File naming varies depending on the subdirectory. The preferred
   style is to have a 'vir' prefix, followed by a name which
   matches the name of the functions / objects inside the file.
   For example, a file containing an object 'virHashtable' is
   stored in files 'virhashtable.c' and 'virhashtable.h'.
   Sometimes, methods which would otherwise be declared 'static'
   need to be exported for use by a test suite. For this purpose a
   second header file should be added with a suffix of 'priv',
   e.g. 'virhashtablepriv.h'. Use of underscores in file names is
   discouraged when using the 'vir' prefix style. The 'vir' prefix
   naming applies to src/util, src/rpc and tests/ directories.
   Most other directories do not follow this convention.

Enum type & field names
   All enums should have a 'vir' prefix in their typedef name, and
   each following word should have its first letter in uppercase.
   The enum name should match the typedef name with a leading
   underscore. The enum member names should be in all uppercase,
   and use an underscore to separate each word. The enum member
   name prefix should match the enum typedef name.

   ::

     typedef enum _virSocketType virSocketType;
     enum _virSocketType {
         VIR_SOCKET_TYPE_IPV4,
         VIR_SOCKET_TYPE_IPV6,
     };

Struct type names
   All structs should have a 'vir' prefix in their typedef name,
   and each following word should have its first letter in
   uppercase. The struct name should be the same as the typedef
   name with a leading underscore. For types that are part of the
   public API, a second typedef should be given for a pointer to
   the struct with a 'Ptr' suffix. Do not introduce new such
   typedefs for internal types.

   ::

     typedef struct _virSomeType virSomeType;
     typedef virSomeType *virSomeTypePtr;
     struct _virSomeType {
         ...
     };

Function names
   All functions should have a 'vir' prefix in their name,
   followed by one or more words with first letter of each word
   capitalized. Underscores should not be used in function names.
   If the function is operating on an object, then the function
   name prefix should match the object typedef name, otherwise it
   should match the filename. Following this comes the verb /
   action name, and finally an optional subject name. For example,
   given an object 'virSomeType', all functions should have a
   name 'virSomeType$VERB' or 'virSomeType$VERB$SUBJECT", e.g.
   'virHashTableLookup' or 'virHashTableGetValue'.

Macro names
   All macros should have a "VIR" prefix in their name, followed
   by one or more uppercase words separated by underscores. The
   macro argument names should be in lowercase. Aside from having
   a "VIR" prefix there are no common practices for the rest of
   the macro name.

Code indentation
================

Libvirt's C source code generally adheres to some basic
code-formatting conventions. The existing code base is not totally
consistent on this front, but we do prefer that contributed code
be formatted similarly. In short, use spaces-not-TABs for
indentation, use 4 spaces for each indentation level, and other
than that, follow the K&R style.

If you use Emacs, the project includes a file .dir-locals.el that
sets up the preferred indentation. If you use vim, append the
following to your ~/.vimrc file:

::

  set nocompatible
  filetype on
  set autoindent
  set smartindent
  set cindent
  set tabstop=8
  set shiftwidth=4
  set expandtab
  set cinoptions=(0,:0,l1,t0,L3
  filetype plugin indent on
  au FileType make setlocal noexpandtab
  au BufRead,BufNewFile *.am setlocal noexpandtab
  match ErrorMsg /\s\+$\| \+\ze\t/

Or if you don't want to mess your ~/.vimrc up, you can save the
above into a file called .lvimrc (not .vimrc) located at the root
of libvirt source, then install a vim script from
https://www.vim.org/scripts/script.php?script_id=1408, which will
load the .lvimrc only when you edit libvirt code.

Code formatting (especially for new code)
=========================================

With new code, we can be even more strict. Please apply the
following function (using GNU indent) to any new code. Note that
this also gives you an idea of the type of spacing we prefer
around operators and keywords:

::

  indent-libvirt()
  {
    indent -bad -bap -bbb -bli4 -br -ce -brs -cs -i4 -l100 -lc100 \
           -sbi4 -psl -saf -sai -saw -sbi4 -ss -sc -cdw -cli4 -npcs -nbc \
           --no-tabs "$@"
  }

Note that sometimes you'll have to post-process that output
further, by piping it through ``expand -i``, since some leading
TABs can get through. Usually they're in macro definitions or
strings, and should be converted anyhow.

The maximum permitted line length is 100 characters, but lines
should aim to be approximately 80 characters.

Libvirt requires a C99 compiler for various reasons. However, most
of the code base prefers to stick to C89 syntax unless there is a
compelling reason otherwise. For example, it is preferable to use
``/* */`` comments rather than ``//``. Also, when declaring local
variables, the prevailing style has been to declare them at the
beginning of a scope, rather than immediately before use.

Bracket spacing
---------------

The keywords ``if``, ``for``, ``while``, and ``switch`` must have
a single space following them before the opening bracket. E.g.

::

  if(foo)   // Bad
  if (foo)  // Good

Function implementations must **not** have any whitespace between
the function name and the opening bracket. E.g.

::

  int foo (int wizz)  // Bad
  int foo(int wizz)   // Good

Function calls must **not** have any whitespace between the
function name and the opening bracket. E.g.

::

  bar = foo (wizz);  // Bad
  bar = foo(wizz);   // Good

Function typedefs must **not** have any whitespace between the
closing bracket of the function name and opening bracket of the
arg list. E.g.

::

  typedef int (*foo) (int wizz);  // Bad
  typedef int (*foo)(int wizz);   // Good

There must not be any whitespace immediately following any opening
bracket, or immediately prior to any closing bracket. E.g.

::

  int foo( int wizz );  // Bad
  int foo(int wizz);    // Good

Commas
------

Commas should always be followed by a space or end of line, and
never have leading space; this is enforced during 'make
syntax-check'.

::

  call(a,b ,c);// Bad
  call(a, b, c); // Good

When declaring an enum or using a struct initializer that occupies
more than one line, use a trailing comma. That way, future edits
to extend the list only have to add a line, rather than modify an
existing line to add the intermediate comma. Any sentinel
enumerator value with a name ending in \_LAST is exempt, since you
would extend such an enum before the \_LAST element. Another
reason to favor trailing commas is that it requires less effort to
produce via code generators. Note that the syntax checker is
unable to enforce a style of trailing commas, so there are
counterexamples in existing code which do not use it; also, while
C99 allows trailing commas, remember that JSON and XDR do not.

::

  enum {
      VALUE_ONE,
      VALUE_TWO // Bad
  };
  enum {
      VALUE_THREE,
      VALUE_FOUR, // Good
  };

Semicolons
----------

Semicolons should never have a space beforehand. Inside the
condition of a ``for`` loop, there should always be a space or
line break after each semicolon, except for the special case of an
infinite loop (although more infinite loops use ``while``). While
not enforced, loop counters generally use post-increment.

::

  for (i = 0 ;i < limit ; ++i) { // Bad
  for (i = 0; i < limit; i++) { // Good
  for (;;) { // ok
  while (1) { // Better

Empty loop bodies are better represented with curly braces and a
comment, although use of a semicolon is not currently rejected.

::

  while ((rc = waitpid(pid, &st, 0) == -1) &&
         errno == EINTR); // ok
  while ((rc = waitpid(pid, &st, 0) == -1) &&
         errno == EINTR) { // Better
      /* nothing */
  }

Curly braces
------------

Curly braces around an ``if``, ``while``, ``for`` etc. can be omitted if the
body and the condition itself occupy only a single line.
In every other case we require the braces. This
ensures that it is trivially easy to identify a
single-\ *statement* loop: each has only one *line* in its body.

::

  while (expr)             // single line body; {} is optional
      single_line_stmt();

::

  while (expr(arg1,
              arg2))      // indentation makes it obvious it is single line,
      single_line_stmt(); // {} is optional (not enforced either way)

::

  while (expr1 &&
         expr2) {         // multi-line, at same indentation, {} required
      single_line_stmt();
  }

However, the moment your loop/if/else body extends on to a second
line, for whatever reason (even if it's just an added comment),
then you should add braces. Otherwise, it would be too easy to
insert a statement just before that comment (without adding
braces), thinking it is already a multi-statement loop:

::

  while (true) // BAD! multi-line body with no braces
      /* comment... */
      single_line_stmt();

Do this instead:

::

  while (true) { // Always put braces around a multi-line body.
      /* comment... */
      single_line_stmt();
  }

There is one exception: when the second body line is not at the
same indentation level as the first body line:

::

  if (expr)
      die("a diagnostic that would make this line"
          " extend past the 80-column limit"));

It is safe to omit the braces in the code above, since the
further-indented second body line makes it obvious that this is
still a single-statement body.

To reiterate, don't do this:

::

  if (expr)            // BAD: no braces around...
      while (expr_2) { // ... a multi-line body
          ...
      }

Do this, instead:

::

  if (expr) {
      while (expr_2) {
          ...
      }
  }

However, there is one exception in the other direction, when even
a one-line block should have braces. That occurs when that
one-line, brace-less block is an ``if`` or ``else`` block, and the
counterpart block **does** use braces. In that case, put braces
around both blocks. Also, if the ``else`` block is much shorter
than the ``if`` block, consider negating the ``if``-condition and
swapping the bodies, putting the short block first and making the
longer, multi-line block be the ``else`` block.

::

  if (expr) {
      ...
      ...
  }
  else
      x = y;    // BAD: braceless "else" with braced "then",
                // and short block last

  if (expr)
      x = y;    // BAD: braceless "if" with braced "else"
  else {
      ...
      ...
  }

Keeping braces consistent and putting the short block first is
preferred, especially when the multi-line body is more than a few
lines long, because it is easier to read and grasp the semantics
of an if-then-else block when the simpler block occurs first,
rather than after the more involved block:

::

  if (!expr) {
    x = y; // putting the smaller block first is more readable
  } else {
      ...
      ...
  }

But if negating a complex condition is too ugly, then at least add
braces:

::

  if (complex expr not worth negating) {
      ...
      ...
  } else {
      x = y;
  }

Use hanging braces for compound statements: the opening brace of a
compound statement should be on the same line as the condition
being tested. Only top-level function bodies, nested scopes, and
compound structure declarations should ever have { on a line by
itself.

::

  void
  foo(int a, int b)
  {                          // correct - function body
      int 2d[][] = {
        {                    // correct - complex initialization
          1, 2,
        },
      };
      if (a)
      {                      // BAD: compound brace on its own line
          do_stuff();
      }
      {                      // correct - nested scope
          int tmp;
          if (a < b) {       // correct - hanging brace
              tmp = b;
              b = a;
              a = tmp;
          }
      }
  }

Conditional expressions
-----------------------

For readability reasons new code should avoid shortening
comparisons to 0 for numeric types:

::

  size nfoos = 0;

  GOOD:
    if (nfoos != 0)
    if (nfoos == 0)

  BAD:
    if (nfoos)
    if (!nfoos)

Prefer the shortened version for boolean values. Boolean values
should never be compared against the literal ``true``, as a
logical non-false value need not be ``1``.

::

  bool hasFoos = false;

  GOOD:
    if (hasFoos)
    if (!hasFoos)

  BAD:
    if (hasFoos == true)
    if (hasFoos != false)
    if (hasFoos == false)
    if (hasFoos != true)

Pointer comparisons may be shortened. All long forms are okay.

::

  virFoo *foo = NULL;

  GOOD:
    if (foo)                 # or: if (foo != NULL)
    if (!foo)                # or: if (foo == NULL)

New code should avoid the ternary operator as much as possible.
Specifically it must never span more than one line or nest:

::

  BAD:
    char *foo = baz ?
                virDoSomethingReallyComplex(driver, vm, something, baz->foo) :
                NULL;

    char *foo = bar ? bar->baz ? bar->baz->foo : "nobaz" : "nobar";

Preprocessor
------------

Macros defined with an ALL_CAPS name should generally be assumed
to be unsafe with regards to arguments with side-effects (that is,<commit_msg>coding-style: Allow some use of ternary operators<commit_after>============
Coding style
============

.. contents::

Naming conventions
==================

When reading libvirt code, a number of different naming
conventions will be evident due to various changes in thinking
over the course of the project's lifetime. The conventions
documented below should be followed when creating any entirely new
files in libvirt. When working on existing files, while it is
desirable to apply these conventions, keeping a consistent style
with existing code in that particular file is generally more
important. The overall guiding principal is that every file, enum,
struct, function, macro and typedef name must have a 'vir' or
'VIR' prefix. All local scope variable names are exempt, and
global variables are exempt, unless exported in a header file.

File names
   File naming varies depending on the subdirectory. The preferred
   style is to have a 'vir' prefix, followed by a name which
   matches the name of the functions / objects inside the file.
   For example, a file containing an object 'virHashtable' is
   stored in files 'virhashtable.c' and 'virhashtable.h'.
   Sometimes, methods which would otherwise be declared 'static'
   need to be exported for use by a test suite. For this purpose a
   second header file should be added with a suffix of 'priv',
   e.g. 'virhashtablepriv.h'. Use of underscores in file names is
   discouraged when using the 'vir' prefix style. The 'vir' prefix
   naming applies to src/util, src/rpc and tests/ directories.
   Most other directories do not follow this convention.

Enum type & field names
   All enums should have a 'vir' prefix in their typedef name, and
   each following word should have its first letter in uppercase.
   The enum name should match the typedef name with a leading
   underscore. The enum member names should be in all uppercase,
   and use an underscore to separate each word. The enum member
   name prefix should match the enum typedef name.

   ::

     typedef enum _virSocketType virSocketType;
     enum _virSocketType {
         VIR_SOCKET_TYPE_IPV4,
         VIR_SOCKET_TYPE_IPV6,
     };

Struct type names
   All structs should have a 'vir' prefix in their typedef name,
   and each following word should have its first letter in
   uppercase. The struct name should be the same as the typedef
   name with a leading underscore. For types that are part of the
   public API, a second typedef should be given for a pointer to
   the struct with a 'Ptr' suffix. Do not introduce new such
   typedefs for internal types.

   ::

     typedef struct _virSomeType virSomeType;
     typedef virSomeType *virSomeTypePtr;
     struct _virSomeType {
         ...
     };

Function names
   All functions should have a 'vir' prefix in their name,
   followed by one or more words with first letter of each word
   capitalized. Underscores should not be used in function names.
   If the function is operating on an object, then the function
   name prefix should match the object typedef name, otherwise it
   should match the filename. Following this comes the verb /
   action name, and finally an optional subject name. For example,
   given an object 'virSomeType', all functions should have a
   name 'virSomeType$VERB' or 'virSomeType$VERB$SUBJECT", e.g.
   'virHashTableLookup' or 'virHashTableGetValue'.

Macro names
   All macros should have a "VIR" prefix in their name, followed
   by one or more uppercase words separated by underscores. The
   macro argument names should be in lowercase. Aside from having
   a "VIR" prefix there are no common practices for the rest of
   the macro name.

Code indentation
================

Libvirt's C source code generally adheres to some basic
code-formatting conventions. The existing code base is not totally
consistent on this front, but we do prefer that contributed code
be formatted similarly. In short, use spaces-not-TABs for
indentation, use 4 spaces for each indentation level, and other
than that, follow the K&R style.

If you use Emacs, the project includes a file .dir-locals.el that
sets up the preferred indentation. If you use vim, append the
following to your ~/.vimrc file:

::

  set nocompatible
  filetype on
  set autoindent
  set smartindent
  set cindent
  set tabstop=8
  set shiftwidth=4
  set expandtab
  set cinoptions=(0,:0,l1,t0,L3
  filetype plugin indent on
  au FileType make setlocal noexpandtab
  au BufRead,BufNewFile *.am setlocal noexpandtab
  match ErrorMsg /\s\+$\| \+\ze\t/

Or if you don't want to mess your ~/.vimrc up, you can save the
above into a file called .lvimrc (not .vimrc) located at the root
of libvirt source, then install a vim script from
https://www.vim.org/scripts/script.php?script_id=1408, which will
load the .lvimrc only when you edit libvirt code.

Code formatting (especially for new code)
=========================================

With new code, we can be even more strict. Please apply the
following function (using GNU indent) to any new code. Note that
this also gives you an idea of the type of spacing we prefer
around operators and keywords:

::

  indent-libvirt()
  {
    indent -bad -bap -bbb -bli4 -br -ce -brs -cs -i4 -l100 -lc100 \
           -sbi4 -psl -saf -sai -saw -sbi4 -ss -sc -cdw -cli4 -npcs -nbc \
           --no-tabs "$@"
  }

Note that sometimes you'll have to post-process that output
further, by piping it through ``expand -i``, since some leading
TABs can get through. Usually they're in macro definitions or
strings, and should be converted anyhow.

The maximum permitted line length is 100 characters, but lines
should aim to be approximately 80 characters.

Libvirt requires a C99 compiler for various reasons. However, most
of the code base prefers to stick to C89 syntax unless there is a
compelling reason otherwise. For example, it is preferable to use
``/* */`` comments rather than ``//``. Also, when declaring local
variables, the prevailing style has been to declare them at the
beginning of a scope, rather than immediately before use.

Bracket spacing
---------------

The keywords ``if``, ``for``, ``while``, and ``switch`` must have
a single space following them before the opening bracket. E.g.

::

  if(foo)   // Bad
  if (foo)  // Good

Function implementations must **not** have any whitespace between
the function name and the opening bracket. E.g.

::

  int foo (int wizz)  // Bad
  int foo(int wizz)   // Good

Function calls must **not** have any whitespace between the
function name and the opening bracket. E.g.

::

  bar = foo (wizz);  // Bad
  bar = foo(wizz);   // Good

Function typedefs must **not** have any whitespace between the
closing bracket of the function name and opening bracket of the
arg list. E.g.

::

  typedef int (*foo) (int wizz);  // Bad
  typedef int (*foo)(int wizz);   // Good

There must not be any whitespace immediately following any opening
bracket, or immediately prior to any closing bracket. E.g.

::

  int foo( int wizz );  // Bad
  int foo(int wizz);    // Good

Commas
------

Commas should always be followed by a space or end of line, and
never have leading space; this is enforced during 'make
syntax-check'.

::

  call(a,b ,c);// Bad
  call(a, b, c); // Good

When declaring an enum or using a struct initializer that occupies
more than one line, use a trailing comma. That way, future edits
to extend the list only have to add a line, rather than modify an
existing line to add the intermediate comma. Any sentinel
enumerator value with a name ending in \_LAST is exempt, since you
would extend such an enum before the \_LAST element. Another
reason to favor trailing commas is that it requires less effort to
produce via code generators. Note that the syntax checker is
unable to enforce a style of trailing commas, so there are
counterexamples in existing code which do not use it; also, while
C99 allows trailing commas, remember that JSON and XDR do not.

::

  enum {
      VALUE_ONE,
      VALUE_TWO // Bad
  };
  enum {
      VALUE_THREE,
      VALUE_FOUR, // Good
  };

Semicolons
----------

Semicolons should never have a space beforehand. Inside the
condition of a ``for`` loop, there should always be a space or
line break after each semicolon, except for the special case of an
infinite loop (although more infinite loops use ``while``). While
not enforced, loop counters generally use post-increment.

::

  for (i = 0 ;i < limit ; ++i) { // Bad
  for (i = 0; i < limit; i++) { // Good
  for (;;) { // ok
  while (1) { // Better

Empty loop bodies are better represented with curly braces and a
comment, although use of a semicolon is not currently rejected.

::

  while ((rc = waitpid(pid, &st, 0) == -1) &&
         errno == EINTR); // ok
  while ((rc = waitpid(pid, &st, 0) == -1) &&
         errno == EINTR) { // Better
      /* nothing */
  }

Curly braces
------------

Curly braces around an ``if``, ``while``, ``for`` etc. can be omitted if the
body and the condition itself occupy only a single line.
In every other case we require the braces. This
ensures that it is trivially easy to identify a
single-\ *statement* loop: each has only one *line* in its body.

::

  while (expr)             // single line body; {} is optional
      single_line_stmt();

::

  while (expr(arg1,
              arg2))      // indentation makes it obvious it is single line,
      single_line_stmt(); // {} is optional (not enforced either way)

::

  while (expr1 &&
         expr2) {         // multi-line, at same indentation, {} required
      single_line_stmt();
  }

However, the moment your loop/if/else body extends on to a second
line, for whatever reason (even if it's just an added comment),
then you should add braces. Otherwise, it would be too easy to
insert a statement just before that comment (without adding
braces), thinking it is already a multi-statement loop:

::

  while (true) // BAD! multi-line body with no braces
      /* comment... */
      single_line_stmt();

Do this instead:

::

  while (true) { // Always put braces around a multi-line body.
      /* comment... */
      single_line_stmt();
  }

There is one exception: when the second body line is not at the
same indentation level as the first body line:

::

  if (expr)
      die("a diagnostic that would make this line"
          " extend past the 80-column limit"));

It is safe to omit the braces in the code above, since the
further-indented second body line makes it obvious that this is
still a single-statement body.

To reiterate, don't do this:

::

  if (expr)            // BAD: no braces around...
      while (expr_2) { // ... a multi-line body
          ...
      }

Do this, instead:

::

  if (expr) {
      while (expr_2) {
          ...
      }
  }

However, there is one exception in the other direction, when even
a one-line block should have braces. That occurs when that
one-line, brace-less block is an ``if`` or ``else`` block, and the
counterpart block **does** use braces. In that case, put braces
around both blocks. Also, if the ``else`` block is much shorter
than the ``if`` block, consider negating the ``if``-condition and
swapping the bodies, putting the short block first and making the
longer, multi-line block be the ``else`` block.

::

  if (expr) {
      ...
      ...
  }
  else
      x = y;    // BAD: braceless "else" with braced "then",
                // and short block last

  if (expr)
      x = y;    // BAD: braceless "if" with braced "else"
  else {
      ...
      ...
  }

Keeping braces consistent and putting the short block first is
preferred, especially when the multi-line body is more than a few
lines long, because it is easier to read and grasp the semantics
of an if-then-else block when the simpler block occurs first,
rather than after the more involved block:

::

  if (!expr) {
    x = y; // putting the smaller block first is more readable
  } else {
      ...
      ...
  }

But if negating a complex condition is too ugly, then at least add
braces:

::

  if (complex expr not worth negating) {
      ...
      ...
  } else {
      x = y;
  }

Use hanging braces for compound statements: the opening brace of a
compound statement should be on the same line as the condition
being tested. Only top-level function bodies, nested scopes, and
compound structure declarations should ever have { on a line by
itself.

::

  void
  foo(int a, int b)
  {                          // correct - function body
      int 2d[][] = {
        {                    // correct - complex initialization
          1, 2,
        },
      };
      if (a)
      {                      // BAD: compound brace on its own line
          do_stuff();
      }
      {                      // correct - nested scope
          int tmp;
          if (a < b) {       // correct - hanging brace
              tmp = b;
              b = a;
              a = tmp;
          }
      }
  }

Conditional expressions
-----------------------

For readability reasons new code should avoid shortening
comparisons to 0 for numeric types:

::

  size nfoos = 0;

  GOOD:
    if (nfoos != 0)
    if (nfoos == 0)

  BAD:
    if (nfoos)
    if (!nfoos)

Prefer the shortened version for boolean values. Boolean values
should never be compared against the literal ``true``, as a
logical non-false value need not be ``1``.

::

  bool hasFoos = false;

  GOOD:
    if (hasFoos)
    if (!hasFoos)

  BAD:
    if (hasFoos == true)
    if (hasFoos != false)
    if (hasFoos == false)
    if (hasFoos != true)

Pointer comparisons may be shortened. All long forms are okay.

::

  virFoo *foo = NULL;

  GOOD:
    if (foo)                 # or: if (foo != NULL)
    if (!foo)                # or: if (foo == NULL)

New code should avoid the ternary operator as much as possible.
Its usage in basic cases is warranted (e.g. when deciding between
two constant strings), however, it must never span more than one
line or nest.

::

  BAD:
    char *foo = baz ?
                virDoSomethingReallyComplex(driver, vm, something, baz->foo) :
                NULL;

    char *foo = bar ? bar->baz ? bar->baz->foo : "nobaz" : "nobar";

  GOOD:
    virBufferAsprintf(buf, "<element>%s</element>\n", boolVar ? "yes" : "no");

Preprocessor
------------

Macros defined with an ALL_CAPS name should generally be assumed
to be unsafe with regards to arguments with side-effects (that is,
{'fraction_non_alphanumeric': 0.08832288126686053, 'fraction_numerical': 0.0041341134428756615, 'mean_word_length': 2.794735442701409, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 13, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5177632', 'n_tokens_mistral': 8297, 'n_tokens_neox': 7918, 'n_words': 3969}